--- title: "Learning to Teach AI Collaboration" author: "Geoffrey Challen" date: "2026-11-02" description: "A short talk to Illinois colleagues on what teaching AI collaboration in CS 124 and in Conversational Programming taught us this fall: what students' collaboration logs make possible, what providing the AI lets us shape, the principles we teach, and what students build." canonical: "https://www.geoffreychallen.com/talks/2026-11-02-learning-to-teach-ai-collaboration" audience: "Siebel School of Computing and Data Science faculty, Urbana, Illinois" slides: "https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html" film: "https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/film.mp4" --- # Learning to Teach AI Collaboration [View the slides](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html) [Watch the film](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/film.mp4) ## Abstract This fall we started teaching AI collaboration in two very different courses: [CS 124](https://www.cs124.org/ai), where about eleven hundred students each build a project with a coding agent while classical programming is still taught and assessed, and [Conversational Programming](https://www.conversationalprogramming.org/educators), a studio of nineteen students with no prerequisites who build websites, tools, and games by talking to an agent and never touch code. Four things we learned so far. First, AI collaboration is a skill, and it can be taught. A coding agent only feels like magic when what you ask of it isn’t hard enough; ask for something that matters and the work is a conversation. Nobody knew how to teach that in August, including me; we have [eight principles](https://www.conversationalprogramming.org/educators), the agent’s own answer when I finally asked it, as a starting point. Second, how we teach it. In Conversational Programming, a studio: three projects in a semester, and a second agent that watches each student’s conversation and steps in, in the builder’s own voice, at the moment a question will help. In CS 124, lessons that run inside each student’s own agent against their own prompts, and fourteen Tuesdays of activities around a project. Third, you can see everything. Because the agent does the implementing, students have to say what they want, and the saying can be recorded: every request in Conversational Programming through our proxy, every prompt in CS 124 through Claude Code’s hooks. The logs grade CS 124’s project by time on task and show us where students struggle. Finally, students create remarkable things and engage with them at the level of ideas rather than implementation, very early in the program: the CS 124 [Duckies](https://www.cs124.org/duckies), the [Using and Understanding AI](https://usingandunderstanding.ai) projects, and the conversational programming work. It is fun, and it puts students where the meaning is: the impact their work has on actual users. All of it is fifteen weeks; imagine four years. ## Slides ### A skill that can be taught [![Slide 1: Title](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/01-title.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/0) Thanks, everybody. What I want to do today is tell you a little about a couple of things we are doing to learn how to teach this new skill of AI collaboration. These are ongoing projects in my main course, CS 124, and in a new course I am teaching this semester on conversational programming. [![Slide 2: Not Magic](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/02-not-magic.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/1) A big starting point for me is that these tools only feel like magic when you use them to do our current assignments. Ask a coding agent for something small and it is a conjuring trick. Ask it for something that matters and it is work, and the work is a conversation. Some students are better at that conversation than others, and so are some faculty. It is a skill, not magic, and our job is to figure out how to teach it. As Eames says to Arthur in Inception: you mustn’t be afraid to dream a little bigger, darling. In both courses we have had to adjust our expectations for what students can build. [![Slide 3: A Year of My Own Practice](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/03-my-usage.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/2) I have been using these tools heavily for the past year: almost forty-seven thousand prompts to a coding agent, on three hundred and sixty-nine days. Right now that is the only way to learn to teach them. There is no pedagogy yet for this new skill. You have to learn by doing, and the doing is a lot more fun. This is what is active this fall: websites, projects and software, supporting my teaching, my research and other parts of my life, while also teaching me how to teach this to students. And this line is August 30, the day I finally asked. [![Slide 4: The day I asked](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/04-the-ask.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/3) Last spring in CS 124 we just let students practice. But as I was designing the new conversational programming course, I asked my agent: I want to teach students to communicate with you effectively. How do I do that? It answered with nine things, in its own words, and they matched my own experience, so I took them as a starting point. Here are the nine, without the glosses. I am not going to go through them in depth. And this was my reply. I was kind of embarrassed to say that I had never asked it that question. I had used it for over a year before we had that conversation. [![Slide 5: Down from the mountain](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/05-mount-agent.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/4) I am certainly not claiming to be coming down from the mountain with the stone tablets. These eight principles are just a starting point for helping students collaborate with AI in a way that lets them bring in what they know, while working alongside the agent and taking advantage of what it knows, which is quite a lot. One example: talk about ideas, not implementation. Treat the agent like an intern and it behaves like an intern. [![Slide 6: Never Accept Done Without Looking](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/06-hwyk-strategies.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/5) One principle is that you have to validate what the agent produces. Some people still cling to the idea that you do this by reading code. Reading code is a terrible way to determine whether code works. Humans have been proving that for decades: look at all the bugs being found in programs that people tested carefully for years. We are not good at this, and we should stop pretending that we are. The fun thing is that there are all these ways to bring agents into the validation. This list came from my conversational programming students in a brainstorming exercise: two hundred and sixty-one ideas from fourteen students, and only one said read the code. They have started implementing some of these in their own projects. The skill is asking for the evidence and looking at it, and it is one of the things we can be teaching. ### How we teach it [![Slide 7: Two Courses](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/07-two-courses.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/6) I am experimenting with this in parallel in two courses, and they differ. CS 124 is about eleven hundred students, each building a project with Claude Code since October 6, and we still teach and assess classical programming, a big part of each student’s grade. That creates some tension: students can see that we are teaching them one thing that used to be important, alongside a new thing that suggests the old thing may not be as important anymore. I think classical programming is a dead skill, but to articulate with the rest of our program we still teach it. To work with a different population, I am piloting Conversational Programming this semester as a section of CS 199: nineteen students, starting from zero, who never see or touch code. They use Codex, and every request goes through our proxy. We are pushing the agents as far as they can go. [![Slide 8: One Semester, Many Iterations](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/08-collapsed-loop.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/7) We have had courses before where students built real things, but building was so slow that over a semester the build dominated. Students had very little time to design a prototype, much less evaluate what they made. Agents shrink the building time, so we can iterate within the same semester. Students can do two or three or four or five projects in a semester, and the time they spend building shrinks relative to the other phases, which I would argue are the more meaningful ones: design and evaluation. A lot of our students graduate without ever building something that they or anyone else used to do something real. Not a linked list reversal: a program they installed on their own device and used themselves, or had a friend use. Now we can do that in the first semester. [![Slide 9: The course](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/09-the-course.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/8) Conversational Programming is the more experimental course. There are no prerequisites and absolutely no working with code. Students install Codex on their laptops and work through a series of projects. The first was a personal website, a nice chance to practice some design skills. Sixteen of the nineteen are live. The one they are working on now is something for another member of the class, built from the classmate’s own idea. Fourteen of nineteen are live so far. Next is one idea the entire room chooses, by ranked ballot on Friday, November 6. The things students are building are remarkable, and I will be able to share more as the semester goes on. [![Slide 10: How the proxy shapes it](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/10-shaping-it.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/9) Because the course is small, all of the AI the students use is provided through a proxy that Codex is configured to use, running on the course infrastructure. That gives us complete visibility into what students are doing, and it also lets us shape it. These are the levers. A course note rides at the end of the agent’s context on a class day, saying what today is and what the course would like help with: context, never orders. Then the Course TA, partner nudges in pair activities, a scope watcher, and a practice watcher whose ratings never reach the student. We are observing what students do and teaching them, through those principles, at the same time. [![Slide 11: One student, two agents, a room](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/11-agent-buildup.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/10) From the student’s side, there is one agent, building the thing they want to build. Behind it there is a second agent, watching the conversation, that can step in at times to redirect the student toward more effective requests. It answers in the builder’s voice, and it is configured with context, never instructions: this student is working on interface design this week, never don’t write the interface. Every table in the room is a three: a student, a builder and a watcher. These are studio courses. I don’t believe in lecture-based pedagogy, and neither of my courses uses it. Students work on things in class, interact with each other, give and get feedback. I am not standing at the front waving my arms, because that doesn’t work. And the whole room: there is always human interaction, which is the main reason to bring students together, but there are agents in the room too, observing, helping students engage with each other, and reporting back to me. Over six days the Course TA’s one hundred and forty-three step-ins reached all nineteen students. [![Slide 12: What did you see?](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/12-step-in.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/11) Here is one of those principles being applied, from a real interaction. The agent had said the bug was fixed, with three hundred and sixty-two tests passing, and the student asked it to switch to a different API. That is a bad prompt, because what is missing is intent. Why switch? What did you see that made you conclude that? So the pedagogical agent stepped in: before I switch, what did you see when you opened the app and tried it? And the student said what they saw: after they clicked that they had resumed, the app window disappeared. Now the agent understands what the student is trying to accomplish, and it fixes that first. Telling the agent what you saw gets a better result, because the agent can apply its own understanding of the problem. A TA beside every student, in real time, configured in paragraphs: sixteen hundred turns judged, most of them left alone. [![Slide 13: A lesson inside their own agent](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/13-agent-lessons.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/12) In CS 124, students have done independent projects with a coding agent since last semester, and we are building this instruction in too. One principle that comes up very early is to talk about ideas. Don’t tell the agent what to do; it knows a lot more about computing than you do. Bring in your knowledge, your interests and what you see in the world, in ways that complement what the agent can do. We do this with what I call agentic lessons. They are part of the course website alongside our other lessons, but they run inside the student’s own Claude Code. The student pastes one sentence, the agent fetches the lesson, and it walks them through it in their own project. Because it has their history, it can find their own prompts that worked at the level of ideas, and the ones where they told the agent what to do rather than what they were trying to accomplish. Each step ends with something the student types, and our server verifies it from the hook stream. Four of eight lessons are out, nine hundred and sixty-seven students have been credited on the first, and they seem to be working pretty well. [![Slide 14: Fourteen Tuesdays](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/14-tuesday-activities.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/13) Students also work with agents in our Tuesday discussion sections, in person: fourteen activities, one hour a week. This has been super fun. We lean into rapid prototyping, three rounds of it before the project launched, then pair prompting and receiving criticism from a classmate who tried your project. As far as we can tell, students really enjoy it. There are so many ideas opening up for getting students to talk about far more interesting aspects of their work than whether the code is properly formatted or the variable names could be better. ### You can see everything [![Slide 15: Intent above, implementation below](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/15-intent.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/14) Here is an exciting opportunity for us as educators. To use an agent, you have to put everything into words. When you watch a student work with an agent, you get a really rich view of exactly what they are trying to do, and that is powerful. Words can be kept. [![Slide 16: The Stack](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/16-the-stack.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/15) Every word of that saying runs through us. In Conversational Programming, that is the proxy: about forty-seven thousand requests from nineteen students, and about three thousand three hundred messages they typed themselves. Every conversation passes through one point where we can see it and react to it. [![Slide 17: Hooks, Not a Proxy](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/17-cs124-hooks.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/16) CS 124 is different, because we are not providing the AI: students buy it themselves, and their Claude Code talks straight to the model. So instead of a proxy, we configured Claude Code to collect all of their interactions, instrumented when a student starts a project in the repository. Fifteen hooks in the project template post every prompt, every tool call and every session event to our server as it happens, with server-side timestamps, so nothing can be backdated. That is about a hundred and thirty-seven thousand prompts from about eleven hundred students. [![Slide 18: You Can See Everything](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/18-see-everything.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/17) The title of this slide is an obvious joke, hopefully, but you really can see everything. This is one afternoon in Conversational Programming, October 21. Every student is a line, and every request their agent made is a cell, brighter for more. The lines carry no names; they are ordered by where students sat. Then every message they typed, sixty-three that afternoon, and every time the Course TA stepped in, thirteen, with the pairs forming and re-forming. We can see so much more about how students work, what they are trying to accomplish, and how they are learning to collaborate with AI along the way. Process evidence at a scale and resolution we have never had. [![Slide 19: Time on Task Grading](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/19-time-on-task.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/18) Another use of the data is evaluation. When CS 124 started independent projects, we had to decide how to grade them, and we use the logs to do what we call time-on-task grading. Each student is asked to do a certain amount of work with their agent over the semester. We collect their logs and estimate the time they spent: twenty-four hours for full credit, at most five a week. The estimate is a deterministic rule per prompt, not a language model, and attendance alone earns nothing. Students see it on a dashboard on the grades page: this is one student from last spring against that term’s target. It is much fairer, because everybody gets the same amount of practice with the tool. And the logs show where they struggle. The lesson step that asks what you did to look at it withholds credit most often, and the Course TA’s most common step-in is a how without a what. Validation is the hard part. ### Remarkable things, at the level of ideas [![Slide 20: The Duckies: CS 124, Spring 2026](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/20-showcase-duckies.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/19) What are students doing with this? Incredible things. Last spring was the first time CS 124 students did an independent project: two hundred and eighty-seven of them, three point two million messages, most built for someone. We held a little awards event, the Duckies, and the breadth of student interest and engagement with the world was awesome to see, down to the categories students came up with. This fall’s gallery walk is December 1. [![Slide 21: For real users](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/21-for-real-users.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/20) One whole category was apps students built on their own to help elderly relatives use their devices better, like this grandmother with the app her grandson built. Early in the program, students are engaging with the part that matters: the impact on actual users, not the implementation. And it is fun. [![Slide 22: Using and Understanding AI: Final Projects](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/22-showcase-uai.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/21) Last semester I piloted another course, Using and Understanding AI, and its students spent a couple of hours on final projects using a platform called Replit. These students did not sign up for a programming course and never read code. One built a tank planner for an aquascaping hobby; another, a tool to help a sorority manage its events, something it currently pays several services to do. We are already seeing a whole new kind of student participate in creating technology: students who never showed up for our courses, or did not survive deep into our degree program, turned off by how difficult or unpleasant it was. Now they can build things in natural language, and that is going to change how computing is expressed in the world. [![Slide 23: Four Years](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/23-four-years.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/22) I love to extrapolate. This is one student finishing one project, in fifteen weeks. Pull back, and that is one of three projects in one semester, from students who had never done this in August. The other seven semesters are empty. What could we do if, instead of one semester, we taught this over four years, and students iterated, experimented, learned more about the world, tried different projects, got feedback from actual users and gave feedback to their classmates? I was in the Bay Area last week talking with people about this, and we agreed it is almost unimaginable what kind of student we would graduate. Nobody on earth has had more than a year of practice with this, because these tools are about a year old. I would love to see us try. ### Questions [![Slide 24: Takeaways](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/24-takeaways.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/23) Four early takeaways from our work in this space. AI collaboration is a skill, and it can be learned and taught. We should teach it early and often. AI can help us teach it. And students really like learning it. [![Slide 25: The Race Is On](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/25-the-race.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/24) The race is on. The leaders are well down the track, each with an agent running beside them, and we are still at the line, looking around at the others who have not started either. We need to get started. There is no pedagogy to receive. The only way to learn to teach this is to do it, with frontier models pushed as hard as they go. [![Slide 26: Educators Who Build](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/26-educators-who-build.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/25) One way to get started: I run Educators Who Build, a group on campus for educators who want to use these tools to support teaching and learning. People from all over campus come to discover how to use them in their own courses, and you are welcome to join us. Everything is at educatorswhobuild.org. ### Questions? [![Slide 27: Questions](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/27-hand-off.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/26) The materials from both courses are open, at cs124.org/ai and conversationalprogramming.org/educators. I am happy to take questions, and thanks for listening. ### Backup slides [![Slide 28: Backup Slides](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/28-backup-slides.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/27) The slides that follow were held for questions: running class from a chat window and a critique film from Conversational Programming, how new this skill is, what the course costs, why CS 124 grades engagement rather than completion, and more of what students built. [![Slide 29: Running class from the chat](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/29-conversational-classroom.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/28) On October 23, I ran seventy-five minutes of Conversational Programming from a chat window: thirty-three messages and twenty-six room actions. Here is a status check, partway through the class. Then paging a student, from the same window. And moving the room on to the next section. CS 124 cannot do this yet: hooks see, but they do not speak. The lessons are the first lever there. [![Slide 30: Who was it for?](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/30-critique-film.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/29) This is a rehearsal, with invented personas, of a critique of the second project. The person the project was built for says what happened when they used it, and the room talks about that. The transcript and the user’s experience are the artifact, not the code. [![Slide 31: A New Skill](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/31-a-year-old.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/30) This is a new skill, barely a year old. On a log scale: people using a chatbot on the web, and people working with a frontier coding agent. First the web line, about nine hundred million people a week by February 2026. Then the agents line. The good coding agents date from Opus 4.5, about a year ago. On a linear scale, the agents line is squashed onto the axis. The gap between the two lines is our job, and it is urgent. [![Slide 32: What It Costs](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/32-costs.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/31) The agents for nineteen students cost about seven hundred and thirteen dollars through October 26, estimated at list prices: thirty-seven fifty-four a student. The university pays, students pay nothing, and nobody holds an account. The watcher cost thirty-five cents. By week, the climb is October, when students were building with the strongest model. [![Slide 33: Engagement, Not Completion](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/33-engagement-vs-completion.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/32) This is why CS 124 grades time on task rather than task completion. Grading completion rewards getting the agent to finish. Grading time rewards working with it. [![Slide 34: What They Built (CS 124)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/34-student-apps.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/33) Three of last spring’s CS 124 independent projects and their awards: a simplified launcher for seniors, with medication reminders; an assistant for multi-day materials-science experiments; and a quote app with emotion tagging. Most were built for someone, and cs124.org/duckies has all of them. [![Slide 35: What They Built (Using and Understanding AI)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/browse/35-uai-final-projects.jpg)](https://www.geoffreychallen.com/talks/2026-11-02-teaching-ai-collaboration/slides.html#/34) Two Using and Understanding AI final projects: an aquascaping tracker and a sorority platform, built through conversation by students who never read code.