Can AI Become the Valedictorian? How Generative AI is Shaking Up University Exams

Artificial Intelligence (AI) has been making waves across various sectors, and recently, it’s got academia buzzed. With AI models like ChatGPT, capable of answering exam questions impressively well, higher education must reconsider how it assesses learning. This transformative research led by a large group of researchers evaluated whether AI assistants like GPT-3.5 and GPT-4 could actually earn an engineering degree, probing the potential vulnerability of academic programs to these tools. Let’s dive into what they discovered and what it means for the future of education.

The Background Buzz: AI in Classrooms

AI isn’t just about self-driving cars or chatbots anymore. In November 2022, ChatGPT’s release sparked substantial excitement, amassing 100 million users in just one month. AI’s potential to revolutionize education has been front and center as classrooms become more tech-savvy. But with great power comes great responsibility—and a fair dose of apprehension. Educational institutions are worried that students might leverage AI not just as a learning partner but as a crutch, bypassing the deep learning these courses aim to instill.

Until now, the debate had lots of smoke but little fire; concrete data on AI’s actual impact on education was sparse. This research fills that gap with data-driven insights, examining over 5,000 exam and assignment questions from 50 courses at EPFL (École Polytechnique Fédérale de Lausanne) to gauge AI’s prowess in acing university assessments.

Crunching the Numbers: How Did AI Perform?

Armed with their dataset, the researchers put AI models GPT-3.5 and GPT-4 to the test. The goal? See how accurately these models could answer questions from diverse STEM disciplines using various prompting techniques.

• Performance Scorecard: GPT-4 answered about 65.8% of questions correctly on average, and even managed to get at least one correct answer through different prompting techniques for 85.1% of questions. That’s an impressive showing against university-level assessments!

• Course Coverage: The research spanned courses in fields like Computer Science, Mathematics, Biology, and more, offering a comprehensive picture of AI’s capabilities across different disciplines. Notably, the AI models performed consistently well in problem-solving areas like coding but struggled with more complex questions like mathematical derivations.

What Does this Mean for Higher Education?

This research uncovered startling implications for universities. If AI can tackle such a high percentage of exam questions correctly, the integrity of traditional assessment methods is at risk. These findings suggest that students could use AI to pass course assessments without truly learning the material, threatening the value and quality of degrees.

Vulnerability to AI: A Double-Edged Sword

• Risk: There’s a clear short-term risk: students might use AI to cheat themselves out of learning.
• Reward: On the flip side, AI could be an ally in learning, augmenting students’ understanding when used responsibly.

Shifting Academic Landscapes

In light of these findings, academia is at a crossroads. Traditional assessments may need an overhaul to minimize AI’s cheating potential. This involves creating AI-resistant exam questions, such as those that require deeper comprehension and unique insights.

• More Complex Assessments: By increasing question difficulty and focusing on analytical skills, universities could outsmart AI’s current capabilities, ensuring students engage with course content more deeply.
• Project-Based Evaluations: Shifting towards projects and real-world problem-solving could provide a more hands-on learning experience for students, lessening AI’s impact.

A Shift in Educational Philosophy

Beyond tweaking assessments, this research underlines a broader need to rethink how education integrates AI. It’s not just about adjusting exams but reimagining education to carve out meaningful roles for AI. Reluctance to adapt could inhibit growth, while embracing AI might enrich learning experiences.

• Teaching AI Literacy: Embedding AI into the curriculum, not just as a topic of study but as a tool for inquiry and exploration, could prepare students for the AI-augmented workspace they will soon enter.
• Ethics and Critical Thinking: Educators should guide students in understanding the ethical use of AI, preparing them not just to use these tools, but to scrutinize and challenge them.

Key Takeaways

• AI Prowess: AI like GPT-4 can answer up to 85% of exam questions correctly across a broad range of subjects, signaling a need for updated assessments.
• Education Risk and Reward: While AI could augment learning, there’s a looming risk of it enabling ways to bypass true learning.
• Assessment Evolution: Universities may need to pivot toward more complex, applied, and project-based evaluations that require deeper critical thinking.
• AI-Empowered Education: Incorporating AI into classroom learning and teaching students AI literacy can better prepare them for future workplaces.

This study marks a pivotal moment for education systems worldwide, as they navigate the delicate balance of harnessing AI’s potential while safeguarding the substance of learning. Adapting to these AI-driven changes is not just a choice but a necessity to ensure that the value of education endures.

In the end, the real question isn’t just whether ChatGPT can get an engineering degree, but how education can evolve so that students, alongside AI, are well-equipped for the challenges and opportunities of tomorrow.

If you are looking to improve your prompting skills and haven’t already, check out our free Advanced Prompt Engineering course.

This blog post is based on the research article “Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistants” by Authors: Beatriz Borges, Negar Foroutan, Deniz Bayazit, Anna Sotnikova, Syrielle Montariol, Tanya Nazaretzky, Mohammadreza Banaei, Alireza Sakhaeirad, Philippe Servant, Seyed Parsa Neshaei, Jibril Frej, Angelika Romanou, Gail Weiss, Sepideh Mamooler, Zeming Chen, Simin Fan, Silin Gao, Mete Ismayilzada, Debjit Paul, Alexandre Schöpfer, Andrej Janchevski, Anja Tiede, Clarence Linden, Emanuele Troiani, Francesco Salvi , et al. (65 additional authors not shown). You can find the original article here.