{"id":7186,"date":"2023-10-10T14:16:16","date_gmt":"2023-10-10T13:16:16","guid":{"rendered":"https:\/\/seecheck.org\/?p=7186"},"modified":"2024-05-03T12:35:01","modified_gmt":"2024-05-03T11:35:01","slug":"black-boxes-and-palindromes-chatgpt-in-the-serbian-language-class","status":"publish","type":"post","link":"https:\/\/seecheck.org\/index.php\/2023\/10\/10\/black-boxes-and-palindromes-chatgpt-in-the-serbian-language-class\/","title":{"rendered":"Black boxes and palindromes: ChatGPT in the Serbian language class"},"content":{"rendered":"\n
\n

Original article<\/a> (in Serbian) was published on 25\/09\/2023; Author: Marija Zemunovi\u0107<\/em><\/p>\n<\/blockquote>\n\n\n\n

A reader asked us to check the statements made by marketing expert Petar Vasic about ChatGPT in a short video<\/a> posted on Instagram. The video was taken from Milan Strongman’s podcast<\/a>, where Vasic, as a guest, said the following, among other things:<\/p>\n\n\n\n

\n

\u201cWhen OpenAI was training ChatGPT, it was actually all in English. They do not know, as they have publicly admitted, how it learned Serbian and other languages. It did it by itself. So it looks like it has loosened up a bit. Who knows what is really happening. That Black Box that they call, the neuro network. In fact, we don’t know what’s going on inside. We have input that we give, it goes through the Black Box. And then the output. But we don’t know what it developed there, and we see that it wants to learn some forbidden things. Especially things related to chemistry. Since the fear is for bombs, for things like that. We see that it already knows chemistry, probably at the level of someone who has finished college. Which is very dangerous\u201d.<\/em><\/p>\n<\/blockquote>\n\n\n\n

ChatGPT<\/a> is a chatbot based on artificial intelligence. It was trained based on a large and varied corpus of texts from the Internet (hundreds of billions of units) and thus \u201clearned\u201d to statistically predict the next word in a sentence. In this way, it more or less successfully responds to user inquiries. We checked the claims made by Vasic about this chatbot, and in the rest of the text, we will try to briefly present what is true and what is not.<\/p>\n\n\n\n

ChatGPT and English language<\/strong><\/p>\n\n\n\n

Vasic’s statement regarding \u201cwhen OpenAI trained ChatGPT, everything was actually in English\u201d can be assessed as incorrect. Although it is not known exactly which units were used to train this system, since there are too many of them, the framework corps<\/a> are known. The largest among them – Common Crawl – provided 60% of the \u201ctokens\u201d, while, for the sake of comparison, the contents pulled from Wikipedia make up a 12 times smaller part of the corpus. Common Crawl<\/a> has been collecting material from the Internet for 16 years and is an open repository with 240 billion pages.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Common Crawl stores texts in different languages, which we could divide into several categories<\/a> according to the level of participation: languages with high, medium, low and extremely low participation. In this regard, English is a \u201ccategory by itself\u201d, because it occupies 45-46% of the space. In this sense, we can really say that ChatGPT is biased towards English (like the Internet as a whole). However, other languages<\/a> are also included in the training material, primarily German, Russian, Chinese, Japanese, French and Spanish – each of them has more than 4% participation.<\/p>\n\n\n\n

Serbian is at the bottom of the language category of medium participation (between 0.1% and 1%), after Turkish, Swedish, Arabic, Persian, Korean, Greek, Hungarian and Bulgarian, and ahead of Hindi, Lithuanian and Slovenian. For example, Albanian, Malay, Tamil and Georgian belong to the lowest category, and Scottish Gaelic, Tibetan, Yiddish and Kyrgyz to the lowest. Although the \u201csmall\u201d languages occupy a very small part of the corpus, it is still a matter of extensive material that has been incorporated.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

To the question \u201cAre you trained exclusively on the corpus of content in English?\u201d, ChatGPT answers no, stating that the corpus also includes content in other languages (Spanish, French, Italian…). The dominant position of the English language is unquestionable. The mentioned bias towards English is noticeable on different levels, from grammatical to cultural. If we ask ChatGPT to write us a rhyming poem in Serbian, to give us an example of a word game or to write some palindrome, we will see that it does not do well, i.e. that – to put it simply – it answers in Serbian, but still \u201cthinks\u201d in English. However, it is also unquestionable that a significant part of the \u201cknowledge\u201d of this chatbot is built based on content from other languages, including some very small ones.<\/p>\n\n\n\n

ChatGPT as a black box<\/strong><\/p>\n\n\n\n

When it comes to Vasic’s thesis about the \u201cblack box\u201d, that is, the claim that we do not know how this system processes and connects data – it is largely correct. The way this model processes a huge amount of information is not completely known even to its creators<\/a>. Why? Because the system is set up in such a way that it learns itself how to connect data at one point. Therefore, this process has become too complex to be comprehensively analyzed and deconstructed.<\/p>\n\n\n\n

An additional note of mystification is introduced by the fact that the companies that develop AI models are quite secretive<\/a> when it comes to the mechanisms they implement and test. However, there are researchers<\/a> who conduct research and ask questions<\/a> about this system, its functioning and its future.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

ChatGPT and bombshell questions<\/strong><\/p>\n\n\n\n

ChatGPT is indeed able to provide answers<\/a> to questions that it should not be able to answer. Although there is an ethical filter that blocks giving immoral or \u201cdangerous\u201d answers, users have found and are finding various \u201choles\u201d in the system. One of such \u201choles\u201d refers precisely to the case when this chatbot was indirectly prompted to provide instructions for making a bomb<\/a>.<\/p>\n\n\n\n

When it comes to the assessment that this system knows chemistry at the level of a university graduate, we can draw attention to two analyzes in parallel. OpenAI<\/a> wrote about the first one, presenting GPT4. The chatbot managed to solve the chemistry test (AP Chemistry<\/a>) better than 70% of the real test takers. On the other hand, the research called \u201cChatGPT also needs a chemistry teacher\u201d is also interesting. It proved that by changing the context it is easily possible to induce ChatGPT to give the wrong answer to a question from an entrance exam for chemistry studies.<\/p>\n","protected":false},"excerpt":{"rendered":"

Original article (in Serbian) was published on 25\/09\/2023; Author: Marija Zemunovi\u0107 A reader asked us to check the statements made by marketing expert Petar Vasic about ChatGPT in a short video posted on Instagram. The video was taken from Milan Strongman’s podcast, where Vasic, as a guest, said the following, among other things: \u201cWhen OpenAI was […]<\/p>\n","protected":false},"author":5,"featured_media":7187,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_uf_show_specific_survey":0,"_uf_disable_surveys":false,"footnotes":""},"categories":[316],"tags":[231,230,357,28],"class_list":["post-7186","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-fact-checks","tag-ai","tag-artificial-intelligence","tag-chatgpt","tag-serbia"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/seecheck.org\/index.php\/wp-json\/wp\/v2\/posts\/7186","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/seecheck.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/seecheck.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/seecheck.org\/index.php\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/seecheck.org\/index.php\/wp-json\/wp\/v2\/comments?post=7186"}],"version-history":[{"count":3,"href":"https:\/\/seecheck.org\/index.php\/wp-json\/wp\/v2\/posts\/7186\/revisions"}],"predecessor-version":[{"id":8772,"href":"https:\/\/seecheck.org\/index.php\/wp-json\/wp\/v2\/posts\/7186\/revisions\/8772"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/seecheck.org\/index.php\/wp-json\/wp\/v2\/media\/7187"}],"wp:attachment":[{"href":"https:\/\/seecheck.org\/index.php\/wp-json\/wp\/v2\/media?parent=7186"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/seecheck.org\/index.php\/wp-json\/wp\/v2\/categories?post=7186"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/seecheck.org\/index.php\/wp-json\/wp\/v2\/tags?post=7186"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}