When Professors Meet ChatGPT: How AI Is Reshaping Programming Problem Design

The AI Teaching Assistant You Didn’t Know You Needed

If you’ve ever taught a coding class—or taken one—you know the challenge of getting just the right problem set to make concepts click. Designing coding exercises that strike the right balance between difficulty, relevance, and engagement is more art than science—and it takes time, lots of time. But what if professors had a sidekick to help with this task? Not a TA, but an AI.

In a recent study, researchers explored how college instructors can team up with Large Language Models (LLMs)—like ChatGPT—to co-create programming problems, model student mistakes, and even write feedback. The idea was simple, but powerful: blend the human judgment of seasoned instructors with the speed and breadth of AI to improve how we teach coding.

What they found is fascinating—and it points to a future where AI isn’t replacing educators, but becoming their creative partner in shaping better learning experiences.

Welcome to the world where professors and ChatGPT build better coding classrooms—together.

Why This Research Matters

Introductory programming courses (think: CS101) are the on-ramp to computer science. But creating meaningful, varied, and pedagogically sound exercises is a heavy lift for instructors, especially when trying to meet the needs of many different students. Add in the pressure to provide personalized feedback, and you get a high-effort, high-stakes situation.

So far, most conversations around ChatGPT in education have focused on how students are using it—sometimes causing concern. But what about instructors? Could AI help educators be more efficient, creative, and effective?

This study flips the script and puts instructors in the spotlight, examining how they can use LLMs to design better programming content—and explores what works and what doesn’t.

Meet INSIGHT: AI’s Classroom Assistant for Instructors

To test how effective AI really is in supporting educators, the researchers built a special tool called INSIGHT. INSIGHT is part teacher’s assistant, part brainstorming buddy. It has two key components:

1. Instructor Authoring Tool – Helps instructors craft programming problems, sample solutions (correct or intentionally wrong), and feedback. The AI can chip in to co-generate problems or point out patterns in student mistakes.

2. Student App – Where learners interact with these problems and receive tailored, real-time feedback based on common misconceptions.

Basically, instructors can input what they want to teach, and the AI helps fill in the blanks—offering suggestions for exercises, detecting common pitfalls, and generating feedback.

This tool doesn’t just let instructors use AI—it helps them work with it.

The Experiment: Going Guided vs. Going Free-Form

To see how helpful this collaboration could be, the study worked with three instructors who teach introductory programming. Each was given a chance to design programming problems using the INSIGHT tool in two different modes:

• Unguided Mode: ChatGPT is all yours. No instructions, no examples—see what you can come up with.

• Guided Mode: Structured prompts, examples, and best-practice advice from the research team to help instructors get the most out of ChatGPT.

Each instructor completed both rounds and was then interviewed to get their thoughts on what worked best.

Spoiler: both approaches had their pros and cons… but the real magic came in the mix.

What the Instructors Learned (and What We Can Learn Too)

1. AI is great for idea generation—but humans still lead

Instructors found ChatGPT incredibly helpful for brainstorming new programming questions or generating variations. One said, “It saved me from having to start from scratch.” But none of them felt comfortable using AI-generated content without reviewing it first. Trust was a big issue.

There’s still a gap between what AI can suggest and what an instructor is willing to greenlight.

Lesson for us all? LLMs can be powerful co-creators—but you’re still the editor-in-chief.

2. Guidance makes AI much more useful

When instructors were given structured prompts, like “Ask for a problem that targets loops and conditionals for novice learners,” the AI’s output got better—faster.

One instructor mentioned, “The guided way was more efficient—it helped me cover something I missed.”

But not everyone loved the structure. Some felt it pushed them toward too-standard problems. Creativity, it turns out, sometimes prefers chaos. One summed it up with: “Unguided was spaghetti-at-the-wall—but some of the spaghetti was good.”

Bottom line: Prompt templates and guidance can improve quality and speed—but don’t box yourself in. Use them as rails, not walls.

3. ChatGPT isn’t ready to give feedback solo

While the AI did generate detailed feedback for common student mistakes, instructors often found it too detailed—even giving away the solution. That defeats the goal of helping students think through the problem.

One instructor put it bluntly: “It’s not what I want to give to the student.”

So when it comes to feedback, the AI is like an overenthusiastic tutor—helpful, but a bit of a blabbermouth. The professors had to step in to filter and reframe the feedback into more helpful (and less hand-holding) hints.

Takeaway: Think of ChatGPT’s feedback as a draft—not the final word.

4. Experience matters

The more experienced an instructor was with ChatGPT, the better they navigated the unguided mode. Newer instructors found the structured prompts helpful, while seasoned ones were better at freelancing.

The study suggests training teachers on how to talk to AI (aka prompt engineering) could be just as important as giving them access.

Real-World Implications: A Smarter Way to Teach Code

So, what does all this mean beyond one study and three instructors?

This research presents a new model for educator-AI collaboration—one where LLMs boost teaching effectiveness rather than just automate tasks.

• Personalized Practice: With the AI helping generate multiple variations of a problem, students can get fresh challenges aligned with their needs.

• Faster Curriculum Design: Want five new loop problems by tomorrow? You don’t have to spend all night.

• Better Error Handling: If instructors can tag common student mistakes once, AI can help detect them and offer feedback again and again.

But most critically, it shows that AI tools like ChatGPT are not replacements for educators. They’re partners—sparring partners, sounding boards, and co-authors.

Just like calculators didn’t make math teachers obsolete, ChatGPT won’t replace coding instructors. Instead, it might just make them even better.

Key Takeaways

• LLMs like ChatGPT can help instructors brainstorm and create programming problems, identify common student errors, and generate feedback—but they work best as collaborators, not solo operators.

• Structured prompts and guidance increase the usefulness of AI outputs, especially for instructors less familiar with prompt-writing. But unguided use still has value for creative ideation.

• AI-generated feedback often needs refining, as it can be too detailed or even give away answers. Human oversight is essential before sharing with students.

• Instructor experience with prompt engineering matters—those comfortable with ChatGPT can leverage it more effectively, with or without structured help.

• A hybrid model appears most promising: AI handles the grunt work and generation, while instructors shape and validate the results, ensuring alignment with learning goals.

Want to improve your own AI-assisted problem design? Start by thinking like a teacher. Be specific in your prompts, provide context (difficulty level, topic, common errors you want to address), and don’t be afraid to edit what the AI gives you. Your job isn’t just to accept AI output—it’s to curate it.

This groundbreaking research points toward a classroom future where educators are empowered—not replaced—by AI partners. And if the early signs are correct, tools like INSIGHT could be just the beginning of a smarter, more collaborative era in education.

Time to sharpen your prompts and fire up that AI teaching assistant. Class is in session.

If you are looking to improve your prompting skills and haven’t already, check out our free Advanced Prompt Engineering course.

This blog post is based on the research article “Facilitating Instructors-LLM Collaboration for Problem Design in Introductory Programming Classrooms” by Authors: Muntasir Hoq, Jessica Vandenberg, Shuyin Jiao, Seung Lee, Bradford Mott, Narges Norouzi, James Lester, Bita Akram. You can find the original article here.