Stack Overflow has made the decision to temporarily stop answers being posted on its platform that are created using the ChatGPT AI-powered natural language processor (NLP).
The moderators of the software engineering knowledge exchange platform said that it’s a temporary policy aiming to slow down the rise in answers created with the AI tool.
The decision was made since the average rate of getting a ChatGPT answer that is correct is 'quite low', said the moderators. Due to this, accepting ChatGPT answers to the site is “substantially harmful” not only to Stack Overflow but also to users looking for solutions to their problems.
In the post, moderators said that the main issue is that the answers have a high rate of being incorrect but appear to look authentic, as well as being very easy to create.
“There are also many people trying out ChatGPT to create answers, without the expertise or willingness to verify that the answer is correct prior to posting,” said the moderators.
“Because such answers are so easy to produce, a large number of people are posting a lot of answers. The volume of these answers (thousands) and the fact that the answers often require a detailed read by someone with at least some subject matter expertise in order to determine that the answer is actually bad has effectively swamped our volunteer-based quality curation infrastructure.”
This has led the moderators to ban the use of ChatGPT to create posts on the platform. If a moderator believes that a user has broken this policy, sanctions are set to be imposed to stop users from continuing this behaviour, even if normally the post would be acceptable. The moderators didn’t clarify what these sanctions would be.
The platform is still developing a final policy on the use of ChatGPT and similar AI tools which it said will need to be discussed with staff as well as users on Meta Stack Overflow.
ChatGPT is an AI-driven NLP created by OpenAI. It acts as an interface with which users can send prompts and it will return answers. It can simply respond to questions using natural language, as well as suggest additions or completions in code, and in some cases, spot security vulnerabilities in a codebase.
It uses a dialogue format where the tool can answer follow-up questions or reject inappropriate requests. However, OpenAI stated on its website that ChatGPT has several limitations, including that it sometimes writes “plausible-sounding but incorrect or nonsensical answers.”
Its creators said this is hard to fix, as during reinforcement learning there’s no source of truth and training it to be more cautious makes it decline questions that it’s able to answer correctly.
When IT Pro asked the AI tool what the problems are with ChatGPT, it produced this result:
Since the launch of ChatGPT on 30 November, tech enthusiasts have experimented with the API. People have found wide uses for it from the fun, such as text-based adventure games, to the professional, such as potentially innovative ways to educate students on various topics.
the advent of the technology has impressed individuals across the sector and some have claimed it's one of the biggest technological advancements of the year.
However, this isn’t the first AI tool from OpenAI to have problems. In November 2021, OpenAI brought to market its GPT-3 natural language processing API.
It opened up the technology to help developers make applications faster. However, it had been described in the past as “too dangerous to release to the general public” since there were fears over its ability to produce fake news that appeared to be real.
