Feeds:
Posts
Comments

Posts Tagged ‘chatbot’

Translation Isn’t Just Words

Posted in Uncategorized, tagged , , , , , on May 26, 2026| 5 Comments »

Photo: Patrick Hendry/Unsplash.
AI can translate the individual words, and perhaps common phrases, but without nuance, all you get is fog.

I worry a lot about AI in education, especially as, in my volunteer capacity teaching English as a second language, I see repeatedly how it interferes with learning. How to Copy and Paste — that is what students are learning, I fear.

At the Conversation, Gareth Barkin, professor of Anthropology and Asian Studies at the University of Puget Sound, explains why students may not even get an accurate translation of individual words when they use AI.

“A friend in Indonesia recently told me about a conversation he had with ChatGPT,” he writes. “He had typed a question in Indonesian – Bahasa Indonesia – about how to handle a difficult family dispute. The chatbot responded fluently, in perfect Indonesian, with advice about communication strategies and conflict resolution. The grammar was flawless. The tone was appropriate. And yet something felt off.

“What the AI offered was advice rooted in American cultural assumptions: prioritize your own preferences, communicate directly, and if family members don’t respect your boundaries, consider cutting them off.

“The response was in Indonesian but shaped by values that centered individual autonomy over the consensus-building, social harmony and collective family dynamics that tend to matter more in Indonesian social life.

“My friend was skeptical enough to notice the mismatch. … Many users might not. That is what prompted my research, published in the International Review of Modern Sociology, into a pattern I found across major AI systems: Even when they were fluent in several languages, the language models retained their Western worldview. I call this ‘epistemological persistence.’ …

“Large language models – LLMs – like ChatGPT, Claude and Gemini can now speak dozens of languages with remarkable fluency. That fluency creates the impression that AI understands local cultures.

“Producing grammatically correct Indonesian, Arabic, Swahili or Hindi, however, does not change the underlying worldview through which these systems reason. It does not alter how they think about people, relationships, responsibility or what counts as a good outcome.

“Those assumptions are shaped by training data drawn predominantly from English-language sources based in the United States. Meta’s open-weight model LLaMA 2 was trained on approximately 89.7% English-language text; LLaMA 3 includes only about 5% non-English data. Major commercial models don’t publish equivalent breakdowns but draw heavily on the same sources. Arabic, the fifth-most-spoken language globally, accounts for under 1% of content in large training datasets. Languages with tens of millions of speakers, including Bengali and Hausa, barely appear.

“Beneath the surface of these multilingual conversations, English functions as a hidden intermediary. A study by researchers at the University of Oxford found that LLMs routinely conduct their core reasoning in English, even when prompted in other languages. They translate the output at the final stage. A user receives flawless text in their preferred language, but the underlying logic originates elsewhere.

“To examine how this plays out in practice, I ran experiments with ChatGPT, Claude and Gemini. I asked questions in both English and Indonesian about concepts such as education, responsibility, well-being and several Indonesian terms that resist direct translation into English. These included terms such as ‘gotong royong,’ which describes a tradition of communal mutual assistance.

“Then I asked questions about education in both languages, using the word ‘pendidikan’ in Indonesian. The answers were consistently centered on individual development, personal autonomy, critical thinking and preparation for the labor market.

“What largely disappeared were the dimensions of pendidikan that Indonesian educational traditions have historically emphasized. In Indonesia, education has long been focused on ethical discipline. …

“The Indonesian concept of ‘malu’ offers a starker example. Often translated as ‘shame’ or ’embarrassment,’ malu has been analyzed by anthropologists Clifford Geertz and Tom Boellstorff as something closer to a shared social awareness.

“A person might feel malu when speaking out of turn in front of elders, or when a family member’s behavior reflects poorly on the household. It regulates conduct and signals awareness of one’s position within a web of relationships. It is cultivated, not merely felt. It is a form of relational awareness rather than a private psychological event.

“When asked directly to define malu, the models acknowledged its social dimensions. In scenario-based questions that simply used the word without asking for a definition, however, all three fell back on the English translation of shame, consistently framing it as an individual emotional experience.

“One representative response framed malu as a normal emotional reaction to be managed through self-reflection and confidence-building – a personal psychological problem rather than a social one. The relational dimensions of the concept disappeared entirely, replaced by the language of individual emotional regulation. A distinctly American worldview travels inside the translation, largely unannounced. …

“As media scholar Safiya Umoja Noble argues about algorithmic systems more broadly, what looks like a technical outcome is actually a structural one, shaped by who has the wealth and infrastructure to build these systems. …

“The main exceptions are Chinese models such as DeepSeek and Alibaba’s Qwen. They represent a genuine alternative to the U.S.-dominated pipeline, though research shows they operate through a distinctly Chinese cultural lens. Asked about a workplace disagreement, for instance, they tend to advise silence or indirect phrasing to preserve harmony rather than the direct, private correction that Western models recommend.”

More at the Conversation, here. I wonder if any of you who speak different languages have run into any of these translation issues.

Read Full Post »

Learning to Spot AI Text

Posted in Uncategorized, tagged , , , , , on September 22, 2025| 7 Comments »

Today is my second online ESL (English as a Second Language) class for the ’25-’26 school year. I assist a more experienced teacher once a week — have been doing so for nearly ten years. One task she likes me to do is to go over the writing homework that students put on an edublog.

Lately, it feels like these otherwise highly motivated adults may not be learning much about writing English. Often they seem to have copied from Google Translate or another AI program. What I want to see is a few mistakes in their answers. At the same time, I am wary of accusing anyone of not doing their own work.

Today’s article didn’t give me a clear answer to my ESL situation, but I was intrigued to learn about programs that help identify who the real writer of a book was or whether AI was used in a journal article.

Roger J. Kreuz, associate dean and professor of psychology, University of Memphis, writes at the Conversation that although it’s common to use chatbots “to write computer code, summarize articles and books, or solicit advice … chatbots are also employed to quickly generate text from scratch, with some users passing off the words as their own.

“This has, not surprisingly, created headaches for teachers tasked with evaluating their students’ written work. It’s also created issues for people seeking advice on forums like Reddit, or consulting product reviews before making a purchase.

“Over the past few years, researchers have been exploring whether it’s even possible to distinguish human writing from artificial intelligence-generated text. … Research participants recruited for a 2021 online study, for example, were unable to distinguish between human- and ChatGPT-generated stories, news articles and recipes.

“Language experts fare no better. In a 2023 study, editorial board members for top linguistics journals were unable to determine which article abstracts had been written by humans and which were generated by ChatGPT. And a 2024 study found that 94% of undergraduate exams written by ChatGPT went undetected by graders at a British university. …

“A commonly held belief is that rare or unusual words can serve as ‘tells’ regarding authorship, just as a poker player might somehow give away that they hold a winning hand.

“Researchers have, in fact, documented a dramatic increase in relatively uncommon words, such as ‘delves’ or ‘crucial,’ in articles published in scientific journals over the past couple of years. This suggests that unusual terms could serve as tells that generative AI has been used. It also implies that some researchers are actively using bots to write or edit parts of their submissions to academic journals. …

“In another study, researchers asked people about characteristics they associate with chatbot-generated text. Many participants pointed to the excessive use of em dashes – an elongated dash used to set off text or serve as a break in thought – as one marker of computer-generated output. But even in this study, the participants’ rate of AI detection was only marginally better than chance.

“Given such poor performance, why do so many people believe that em dashes are a clear tell for chatbots? Perhaps it’s because this form of punctuation is primarily employed by experienced writers. In other words, people may believe that writing that is ‘too good’ must be artificially generated.

“But if people can’t intuitively tell the difference, perhaps there are other methods for determining human versus artificial authorship.

“Some answers may be found in the field of stylometry, in which researchers employ statistical methods to detect variations in the writing styles of authors.

“I’m a cognitive scientist who authored a book on the history of stylometric techniques. In it, I document how researchers developed methods to establish authorship in contested cases, or to determine who may have written anonymous texts.

“One tool for determining authorship was proposed by the Australian scholar John Burrows. He developed Burrows’ Delta, a computerized technique that examines the relative frequency of common words, as opposed to rare ones, that appear in different texts.

“It may seem counterintuitive to think that someone’s use of words like ‘the,’ ‘and’ or ‘to’ can determine authorship, but the technique has been impressively effective.

“Burrows’ Delta, for example, was used to establish that Ruth Plumly Thompson, L. Frank Baum’s successor, was the author of a disputed book in the Wizard of Oz series. It was also used to determine that love letters attributed to Confederate Gen. George Pickett were actually the inventions of his widow, LaSalle Corbell Pickett.

“A major drawback of Burrows’ Delta and similar techniques is that they require a fairly large amount of text to reliably distinguish between authors. A 2016 study found that at least 1,000 words from each author may be required. A relatively short student essay, therefore, wouldn’t provide enough input for a statistical technique to work its attribution magic.

“More recent work has made use of what are known as BERT language models, which are trained on large amounts of human- and chatbot-generated text. The models learn the patterns that are common in each type of writing, and they can be much more discriminating than people: The best ones are between 80% and 98% accurate.

“However, these machine-learning models are ‘black boxes’ – that is, we don’t really know which features of texts are responsible for their impressive abilities. Researchers are actively trying to find ways to make sense of them, but for now, it isn’t clear whether the models are detecting specific, reliable signals that humans can look for on their own.

“Another challenge for identifying bot-generated text is that the models themselves are constantly changing – sometimes in major ways.

“Early in 2025, for example, users began to express concerns that ChatGPT had become overly obsequious, with mundane queries deemed ‘amazing’ or ‘fantastic.’ OpenAI addressed the issue by rolling back some changes it had made.

“Of course, the writing style of a human author may change over time as well, but it typically does so more gradually.

“At some point, I wondered what the bots had to say for themselves. I asked ChatGPT-4o: ‘How can I tell if some prose was generated by ChatGPT? Does it have any “tells,” such as characteristic word choice or punctuation?’

“[It provided] me with a 10-item list, replete with examples. These included the use of hedges – words like ‘often’ and ‘generally’ – as well as redundancy, an overreliance on lists and a ‘polished, neutral tone.’ It did mention ‘predictable vocabulary,’ which included certain adjectives such as ‘significant’ and ‘notable,’ along with academic terms like ‘implication’ and ‘complexity.’ However, though it noted that these features of chatbot-generated text are common, it concluded that ‘none are definitive on their own.’ ” More at the Conversation, here.

If I were in the room with students, I could more or less stand over them and see how they go about writing. But these are adults, after all, and they want to learn, so the goal is to persuade them how learning is more likely to happen. Let me know if you have ideas that could help me.

Read Full Post »