The Illusion of Intelligence: Generative AI and the Literacy Gap

Aviation took almost a generation to reach 100 million users. Netflix took roughly a decade to reach the same scale on streaming. ChatGPT did it in two months. No technology in modern history has been adopted as quickly, and almost every public conversation about generative AI has unfolded after the technology was already in our hands.

That speed is the problem. Not the technology. The speed.

In 1950, Alan Turing replaced "Can machines think?" with an operational test about conversational behaviour: could a machine's responses be reliably distinguished from a human's? The behaviour is now built. The deeper question he set aside, what these systems actually understand, has not been answered.

The mechanism

Beneath the fluency, a large language model is a system that predicts the most statistically likely next token in a sequence. It is trained on data, predominantly text scraped from the open internet, supplemented by books, code and licensed datasets. The internet is not a neutral substrate. It is overwhelmingly Western, overwhelmingly English-language, and shaped by whoever happened to publish online over the last three decades. The model learns the statistical regularities of that data. It does not know the world. It learns what the world has written down.

When the model produces an answer, there is no internal representation of truth, no consultation of a verified database, no reasoning step in the human sense before output. It speaks with the same authority whether it is right or wrong. It will invent a citation, a case, a statistic with the same composure it uses to deliver a correct one. Fluent does not mean accurate. The illusion is not in what the machine does. It is in the confidence with which it does it.

Confident, fluent, wrong

In February 2023, during a live demonstration of Google's Bard chatbot, the system was asked a simple question about the James Webb Space Telescope. It confidently answered that JWST took the first images of an exoplanet, wrong by nearly two decades. Google's stock dropped roughly $100 billion in a single day.

Later that year, two New York lawyers used ChatGPT to research case law for a federal court submission. The model produced six citations to support their argument. None of the cases existed. The system had invented them, with fictitious case names, fictitious judges, and fictitious rulings, in the same fluent register it uses for genuine citations.

This is what the field calls hallucination. It is not a defect to be patched. It is how the system works. The same property that makes generative AI extraordinary is the property that produces confident error.

Bias at machine speed

A 2024 University of Washington study tested AI résumé screening tools by submitting identical résumés, same qualifications, same experience, with only the names changed. The model overwhelmingly preferred white-associated names. It rarely preferred Black-associated names. In some configurations, Black male candidates were disadvantaged in every test.

The model was not malicious. It had learned the patterns embedded in historical hiring decisions and reproduced them at scale, with the same fluency it uses for correct outputs. To the procurement officer signing the contract, the system performs as advertised.

A single biased decision can be appealed. A model answering hundreds of thousands of screening queries does not give the appeal loop time to form. Bias replicates faster than any institution can respond. That is the structural difference between an old failure and a new one.

What people share without knowing

Most people would not paste their bank details into a random website, upload a confidential work document to an unknown server, or enter a child's full name and school into a public form. Yet many of those same people will type all of this into a chatbot, because the interface looks like a conversation rather than an upload. Public AI services log inputs. Many retain them. Some use them to refine future models. The channel feels casual. The data is not.

Teaching via AI, not about AI

The early waves of computerisation in the 1990s and 2000s produced certificates at scale but rarely produced comprehension. Computer education centres taught typing, file management, the operation of the machine. They very rarely taught what was happening inside it. The curriculum chased the tools, not the underlying logic.

The same shape is now repeating with generative AI, on a steeper curve, everywhere. Schools are being asked to teach with AI tools their own staff are still interpreting. Government departments are deploying AI assistants without the workforce trained to evaluate the output. This is the difference between teaching via AI (using a chatbot to summarise, draft, plan) and teaching about AI (building the mental model that lets a person judge whether the summary, draft or plan is reliable). The first is happening at speed. The second, where it is happening at all, is happening much more slowly.

Children should learn AI as a subject, not as a tool. The same is true for adults, and just as true for the policy adviser, the headteacher, and the chief executive.

What foundational literacy actually means

Not just prompt engineering. Not "AI tools for educators." Not vendor certifications. The literacy I am arguing for is a mental model with a small number of moving parts:

• How does Generative AI function

• Generative AI is trained on data and predicts patterns from that data

• Fluency is not truth

• Confidence is not verification

• Bias is inherited from the training data, not chosen by the model

• The user is responsible for the output, regardless of how it was produced

That last point is the professional standard: I am responsible for everything I submit, regardless of how it was produced. If AI helped me write it, I own the facts. If AI generated the code, I own the bugs. Accountability does not transfer to the tool. Verification of the output is NOT optional.

The safety net

When adoption moves at this speed, misinformation produced by a fluent machine compounds faster than any institution can correct it. Regulation will help, in time. Verification tools will help, in time. The only intervention that travels at the same speed as the technology is an AI-literate population.

I am an enthusiast for what AI can do. AlphaFold made dramatic progress on a fifty-year problem in biology. Properly used, generative AI is doing serious work across research, medicine, education and industry. The argument is not against the tool. It is for the literacy that lets the tool be used well, that teaches people to recognise when a system is fluent and wrong.

We will either teach about AI, or be taught by it. On present evidence, being taught by it is not harmless.

Anubhav Shrivastava

Anubhav Shrivastava is a London-based AI practitioner advancing foundational AI literacy through precise, ethics-led system design and education.

A former enterprise architect for global financial institutions, he now builds scalable learning frameworks for underserved communities.

He is the architect of a globally adaptable AI curriculum and the “Lab on Wheels” initiative for low-resource environments.

Through Shiksha Setu, he works to close the widening AI literacy divide in rural India.