Can AI Ace an Engineering Degree? Examining College Exams’ Exposure in the Age of AI Assistants

If you’ve ever wondered if AI tools like ChatGPT could casually breeze through your college exams, you’re not alone. A team of curious researchers decided to roll up their sleeves and dig into this exact question: Could AI, specifically those powered by large language models like GPT-3.5 and GPT-4, earn an engineering degree? And more importantly, how vulnerable are current higher education systems to these clever digital companions?

A New Player in the Classroom

ChatGPT, an AI assistant built on a large language model foundation, has been a hot topic since its launch in November 2022. With millions of enthusiastic users eager to explore its capabilities, it has sparked discussions about the seismic shifts AI might trigger, particularly in educational settings. Many educators and policymakers are raising eyebrows at the prospect of students using AI to find shortcuts in coursework without truly learning the material. But how real is this concern?

Testing AI’s Aptitude

Imagine you’re at the forefront of trying to determine if AI could tackle university-level assessments. You gather a hefty dataset of 5,579 exam questions from 50 STEM courses at a university, known as EPFL, to see just how AI stacks up against the rigors of higher education. This collection spans multiple disciplines, from mathematics to biology, posed as both multiple-choice and open-ended questions.

Armed with GPT-3.5 and GPT-4, the researchers set out to test the models’ capabilities. Remarkably, GPT-4 managed to correctly answer about 66% of these questions using a straightforward strategy: a majority vote on answers from eight different prompting methods. With correct answers emerging from at least one strategy for a whopping 85.1% of questions, it’s clear that AI’s got some academic game.

Making Sense of It All: How GPT-4’s Abilities Span Institutions

Despite the impressive performance, the AI assistants didn’t always hit it out of the park. They had their moments of struggle, particularly with complex questions that require deep reasoning—where students traditionally find the going tough as well. Yet surprisingly, for many of the standard multiple-choice and some open-ended questions, GPT-4 often scored passing grades across various scientific disciplines and university courses.

Academic Challenges Ahead

Sure, the models proved insightful on simpler inputs, and their performance remained relatively stable across several scientific fields. But what exactly does this mean for schools and students? Well, for starters, these findings underscore a vulnerability in many educational assessments that heavily lean on factual recall. If AI can ace your average midterm without breaking a sweat, it begs the question: Are we testing student knowledge or the AI’s prowess?

Keeping It Fair in the Exam Room

So, what’s the way forward? If AI has the potential to sidestep traditional learning routes, then universities must rethink how they assess students. Moving forward, assessment designs could emphasize real-world complexities and creative problem-solving. Instead of sticking to rote memorization tasks, professors might inspire students with projects that encourage critical thinking and collaboration—skills that AI can’t replicate so easily.

Real-World Repercussions

The research isn’t just an academic curiosity. It throws open the door to intriguing possibilities and challenges within the education sector. As AI tools become more ubiquitous, they could become unprecedented allies in enhancing learning. However, without suitable safeguards, they also pose substantial risks, threatening the authenticity of academic credentials and the very process of education itself. Schools might need proactive strategies like tougher questions, AI-resistant assessments, and guidance on ethical AI usage.

Key Takeaways

• AI In Academia: GPT-4 successfully answers around 66% of university exam questions, highlighting potential vulnerabilities in traditional assessment methods.
• Assessment Vulnerability: Existing academic evaluations need updates to cope with AI’s increasing capabilities, focusing on critical thinking and hands-on problem solving.
• Educational Evolution: Embracing AI in education can enhance learning but must be carefully managed to prevent misuse.
• Future Prospects: As AI continues to evolve, educational frameworks must adapt, combining technological understanding with ethical considerations.

Talking about AI might seem futuristic, but its implications are immediate and profound. The task isn’t just to come up with tougher exams but to rethink what education means in an AI-empowered world. Let’s acknowledge the AI’s rising star, aiming to craft an educational experience well-rounded enough to prepare students for whatever the future holds.

If you are looking to improve your prompting skills and haven’t already, check out our free Advanced Prompt Engineering course.

This blog post is based on the research article “Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistants” by Authors: Beatriz Borges, Negar Foroutan, Deniz Bayazit, Anna Sotnikova, Syrielle Montariol, Tanya Nazaretzky, Mohammadreza Banaei, Alireza Sakhaeirad, Philippe Servant, Seyed Parsa Neshaei, Jibril Frej, Angelika Romanou, Gail Weiss, Sepideh Mamooler, Zeming Chen, Simin Fan, Silin Gao, Mete Ismayilzada, Debjit Paul, Alexandre Schöpfer, Andrej Janchevski, Anja Tiede, Clarence Linden, Emanuele Troiani, Francesco Salvi , et al. (65 additional authors not shown). You can find the original article here.