Unmasking the Bullshit: What ChatGPT and Political Speeches Have in Common

When it comes to giving us human-like conversations, AI chatbots like ChatGPT are pretty impressive. But are they just serving us smooth talk without substance? That’s the juicy question researchers Alessandro Trevisan, Harry Giddens, Sarah Dillon, and Alan F. Blackwell from the University of Cambridge set out to answer. They dove into the world of AI chatter and human political rhetoric to find out if the language used is fact or fiction—or in their own words, “bullshit.”

What’s the Big Deal?

Chatbots might seem modern and savvy, but their knack for stringing together words has more in common with political speeches than you might expect. That’s right. Much like a crafty politician, AI can sometimes sound meaningful but lack genuine substance. This isn’t just a juicy tidbit for tech geeks; it impacts anyone who uses these tools for information, teaching, or dealing with mountains of daily tasks.

Let’s dive into the findings and explore what makes AI chatter tick and how politics plays a part in this fascinating dance of words.

The Language Game: Breaking Down the Bullshit

Stepping Up with a Wittgensteinian Twist

Ludwig Wittgenstein, a philosopher famous for seeing language as a series of social games, has ideas that hit right at the heart of this study. The researchers created a “Wittgensteinian Language Game Detector” (WLGD)—think of it as a BS-meter. They tested AI-generated text alongside political verbiage to identify if ChatGPT is merely acting in one big theatrical “language game.”

Data vs. Drivel: The Experiment

The team compared 1,000 scientific articles—a gold standard of clarity and factual accuracy—with texts generated by ChatGPT on similar topics. By setting the BS-detector loose, they aimed to see which one reads more like “real talk” and which veers into bullshit territory.

The Political Connection: Orwell’s Influence

George Orwell, in his legendary essay “Politics and the English Language,” criticized the slippery nature of political speech. It’s often used to manipulate rather than inform, creating a fog that obscures the truth. In similar fashion, the AI’s language generation technique replicates this dance, appearing to provide insight while sometimes just giving wordy but vague responses that sidestep genuine answers.

ChatGPT’s Role Play

So, why is it that ChatGPT seems fluent in this particular brand of BS? Trevisan and his team propose a neat analogy: like playing a role in a scripted play, the AI doesn’t really “know” what it’s saying. The Language Model (LLM) cranks out responses based on probabilities from its data bank—which is essentially the whole internet, truthful or not.

From the Office to the Screen: Bullshit Jobs Unwrapped

Enter David Graeber’s concept of “bullshit jobs”—roles that might look busy on the outside but are actually full of fluff and very little substance. Interestingly, the similarities between bullshit jobs and ChatGPT’s style highlight a broader social critique. It’s not just about tech; it’s about workplaces that generate things that sound impressive but actually deliver squat!

Experimenting with Jobs and Language

When they tossed text samples from regular jobs and “bullshit jobs” into their BS-meter, the results were clear: bullshit job writings had way more in common with ChatGPT’s style than practical, everyday tasks. Whether it’s white-collar jargons or AI-produced blather, it turns out that some jobs might just be noisier than others.

Why It All Matters: Practical Implications

Where politics, tech, and sociology intersect, the implications are vast:

• For Users: Understanding how AI processes language helps users critically evaluate the information they receive.
• In Education: Teachers can guide students to discern valuable content from AI-generated noise.
• In Innovation: Developers might tweak AI designs for richer, more pointed interactivity by recognizing patterns that lead to BS.

Key Takeaways

1. AI and Politics Share the Stage: Both can be powerful yet slippery when balancing a message of truth with engaging delivery.

2. Role-Playing Like a Pro: Just because AI sounds like it’s dropping knowledge bombs, doesn’t always mean there’s wisdom there.

3. BS Jobs and AI: Two Sides of a Coin: If you’ve felt your work experience sometimes mirrors ChatGPT’s responses, you’re onto something.

4. Be the AI Whisperer: Recognizing these patterns can help sharpen your interaction skills and even inform your AI-driven projects or studies.

Navigating the world of AI and its conversational pitfalls isn’t just necessary for data scientists and tech junkies. It’s crucial for anyone engaging with automated systems as these rapidly populate diverse digital platforms. So the next time you ask ChatGPT something profound, remember: it’s all about playing the language game—are you ready to tell fake pearls from real wisdom?

And if ever you want to test your BS meter—just engage a bot, or your local politician, for a lively chat.

If you are looking to improve your prompting skills and haven’t already, check out our free Advanced Prompt Engineering course.

This blog post is based on the research article “Measuring Bullshit in the Language Games played by ChatGPT” by Authors: Alessandro Trevisan, Harry Giddens, Sarah Dillon, Alan F. Blackwell. You can find the original article here.