Taking Flaky Farther
16 min read
A reading of this essay using a voice clone.
Progress toward getting computing educators to recognize what AI coding agents can do has been slow. That’s not surprising. The good agents are new. But their capabilities are genuinely impressive, and I think many computing educators don’t use them enough to really believe it.
So I was excited to see a new piece in Communications of the ACM by Kathi Fisler, Shriram Krishnamurthi, and Michael Littman on teaching novices in the agentic AI era. My colleague Cory Bart linked it on our computingeducators.org forum this week. There’s a difference between heading in the right direction and even being pointed in the right direction, and not many people are doing either yet. This piece is pointed in the right direction. Students working at the level of specifications and testing. Even the radical option of dropping most code writing from early courses.
It also acknowledges, although I wish more forcefully, that reading code is not a good way to validate a system. In their words: “Tracing does not scale.”
That matters, because a position I’m seeing some introductory courses begin to take is: we’ll use AI to produce the code, but students still have to learn to read it. I think that’s worse than having students read and write code. When you write code, it follows your own mental model of how to solve the problem, so reading it back, it’s much easier to see what it’s supposed to do and to spot what’s wrong. Reading somebody else’s code is really difficult. It’s so difficult that one of the things we’re discovering in this new era is how broken a lot of software is, including important code that dozens or hundreds of highly skilled engineers have read over and over again. Agents are finding bugs in code that has been deployed for decades, written and maintained by very intelligent people. One lesson of the AI era is that humans are bad at reading code. We could try to retrain everyone to be better at it. But I think it points to basic human limitations.
Two Problems With the MetaphorTwo Problems With the Metaphor
My worry is the metaphor they build the piece around: an AI agent as a flaky compiler. I see two problems with it.
The first is that it supports, maybe unintentionally, the view that we should teach students to generate code with AI and then read it to decide whether it’s correct.
It ignores all of the new ways agents can take part in validation, which can put humans in a position where they don’t have to read code and yet get stronger guarantees that it works.
The flaky compiler reinforces that middle ground, which I think is the worst of the options.
Reading and writing code is fun.
Reading code that someone else wrote, particularly a coding agent that may have its own ideas about what good code looks like,
The second problem is that the metaphor ignores nearly everything AI can do to support software development beyond compiling an English specification into code. When Cory shared the piece, I replied with a list, off the top of my head:
A compiler can’t read code and make architectural suggestions. A compiler can’t generate tests and use them to correct its own mistakes. A compiler can’t generate user personas and use them to validate and improve a computing artifact. A compiler can’t take a vague bug report, explore the history of the project to pinpoint the cause, apply and deploy a local fix, and then file a bug report, complete with a local demonstration. A compiler can’t take the instruction “something’s not working right now”, analyze logs from a production system, spot a performance problem, fix the code, and redeploy the affected component, all in the space of five minutes. A compiler can’t brainstorm new project ideas, coordinate a team of other compilers to solve a more complex problem.
I’ve been recording my work with AI for the past year, so I went back to find real examples. What follows are excerpts from those sessions: my prompts exactly as I typed them, and the agent’s replies trimmed. All times are Central.
- Chasing a Vague Bug Report Upstream
- Fixing Production in Nine Minutes
- Validating With Simulated Students
- Checking In on a Live Class
- Investigating a Storage Outage
- Brainstorming With Data
- A Research Assistant That Pushes Back
- When I Have to Push Back
Chasing a Vague Bug Report UpstreamChasing a Vague Bug Report Upstream
A few students reported on our course forum that code was vanishing from our in-browser editor, Ace, while they typed. It took me a while to even understand what they were describing: as they typed, the lines being pushed downward disappeared. My agent is connected to the forum.
Me 7:52 PM: Then we have a new bug report to chase. Check the forum.
Agent 7:55 PM: Production is running Ace 1.45.0, but
package.jsonpinsace-buildsto 1.44.0, so something pulled in a second copy. Tracing where it comes from and when it arrived:
Our first hypothesis, that two versions of the editor in the bundle were interfering with each other, turned out to be wrong. The real cause was a change in the editor library itself, released the day before. We pinned the previous version, and the fix was live in production 17 minutes after I asked. Then I wanted to know exactly what had broken.
Me 8:21 PM: I’m starting the dev server. Let’s try to repro this locally. Otherwise we’re just guessing about a fix.
Agent 8:31 PM: I reproduced the bug locally and confirmed the fix. […] I replayed [a student]’s recorded […] session […] (1,123 edits and cursor moves) through our real editor component on the dev site, with Ace 1.45.0 installed locally. At step 96, the Enter that takes the code from 3 lines to 4, the editor grows to 88 px but the inner box Ace draws the code into […] stays at 66 px.
That’s about 40 minutes, while I was doing other things. The agent built a minimal reproduction, I filed the bug report the next day, and the maintainers merged a fix two days later. A flaky compiler cannot do this.
Fixing Production in Nine MinutesFixing Production in Nine Minutes
On Tuesdays my course runs new kinds of activities, and early in the semester we play a get-to-know-each-other bingo game. Students open a board on their phones, walk around the room asking each other the questions on the squares, record the answer, take a selfie, and scan each other’s codes to fill the square. The projector shows their selfies live as they’re taken, along with the number of bingos and blackouts. We ran it last spring, so I didn’t think it needed many changes. This fall’s first section had more students than any we’d run it with before, and within about five minutes it was clear something was wrong.
Me 10:11 AM: It seems like the bingo endpoints are somewhat unstable at the moment.
Agent 10:13 AM: Yes, the readiness probe is the mechanism turning this into hard errors. […] Cause:
/staff-display. Every square completion callsnotifyBingoStaffUpdate()→ every open staff display refetches → the server doescollection.find(query).toArray()over every board with every base64 photo […]Me 10:14 AM: We need to fix this now.
Agent 10:20 AM: Fixed and deployed. […]
Me 10:21 AM: INCREDIBLE WORK! Thank you. That was fast.
Nine minutes from my first message to a fix in production, while the class kept playing. Then it wrote up the incident and posted it to our staff forum. Again, not something a flaky compiler can do. I have other examples like this, including agents taking memory traces of our running backend containers to find performance problems.
Validating With Simulated StudentsValidating With Simulated Students
Another thing a flaky compiler can’t do: validation that goes far beyond reading the code. In those same activities we’ve had to design around students who don’t read the directions, arrive late, or sit at the wrong table. So now, as part of developing every activity, we have agents create simulated student personas and use them to walk through it, looking for the corner cases. Some are designed to be students who don’t follow instructions carefully or won’t read a long block of text.
Me 10:18 AM: Let’s do adversarial review of tomorrow’s activity. Please design an approach and use Opus agents with different student personas. We’ve done this before so there may be notes in the repository. The goal is to complete the process of thoroughly testing the activity and instructions so things go smoothly tomorrow. Also take a look at the existing automated test suites and correct any gaps. Thanks!
Agent 10:38 AM: […] Early signal from the traces: 7 of 9 simulated students saw the red “Too Late for Attendance in This Section” banner at :11 and were confused by it (several decided it was “for other people”, one left early) […]
Fifty-three minutes later the banner was fixed, other agents had found six server bugs and fixed four of them, and the tests had gone from 46 passing to 111. Reading the code is a terrible way to find a problem like that banner. You can’t even get a sense from the code of what’s on the screen. You have to look at the interface and connect it back to the code, and agents can do all of that quickly.
Checking In on a Live ClassChecking In on a Live Class
I’m also teaching a new course this semester on conversational programming. I don’t lecture, because lectures don’t work. Students work together, and each meeting is designed differently. The course has about 20 students right now, but I’m imagining teaching it to many more, and wondering how I’d keep track of what’s happening. So I’ve been building myself a chat interface for the class: an agent that has data about the room and can help me decide that the 20 minutes we planned are done after 15, or that one student has hit a strange bug and needs me, or that a late arrival needs a group. Here’s part of one class, while it was happening:
Me 12:30 PM: Sweet. Hey, can we start the playlist?
Me 12:35 PM: Next!
Me 12:43 PM: OK. How’s it going? We can also work together to decide when to call these stages. We’re seven minutes in and students have already collected a lot of ideas, although a lot of them are falling into the “something else” category :-).
Me 12:44 PM: I guess that means students are thinking hard!
Agent 12:44 PM: The ideas are specific, which is a good sign. […] To be fair, though, most of the hard thinking so far is the agents’. Only 13 of the 105 ideas were written by students; the agents proposed the rest. Students are asking good questions and getting long lists back. That’s a natural place to start, and the go-around is a good moment to push on it: “Which one on your list would you have thought of yourself? Which one do you actually trust?”
This is still new. I built the system with coding agents, and used them to validate it. But you can’t talk to a flaky compiler about what’s going on in your class.
Investigating a Storage OutageInvestigating a Storage Outage
Agents are also good at much broader investigations. My course runs on virtual machines provided by the university, with Kubernetes on top to manage everything, and agents are great at Kubernetes. We’ve used this setup for a long time. In October the cluster suddenly became unstable: machines rebooting, database hosts going down, the whole system essentially unusable. Of course part of it happened during a quiz in our computer-based testing facility, so some quizzes were spoiled and students had to reschedule. It went on for a couple of weeks, coming and going, and we thought it had been fixed when it hadn’t.
The investigation itself was not fast. But the instrumentation needed to see the problem came together quickly. On the first afternoon a team of agents went through storage history, metrics, host logs, and our database machines in parallel, and I asked the obvious question:
Me 2:59 PM: So wait, sorry. This sounds like an underlying storage problem, correct? If it’s hitting nodes across the cluster.
Agent 2:59 PM: Yes. The evidence says the storage under the VMs degraded on Thursday afternoon, and everything since has been happening on top of that. […] db1, db2 and db3 run MongoDB on their own virtual disks, no Ceph, no Kubernetes. Their write latency jumped 10–30× on Thursday […]
Within a few hours of work with agents we had new metrics in the cluster, new reporting, and alerts that reach my phone through Healthchecks and Pushover:
Me 8:24 AM: No need to alert staff here; I’m the cluster operator and only maintainer. Let’s definitely set up email and the Healthchecks. I’ll install Pushover for the mission-critical stuff.
Two days later the problem came back.
Me 2:15 PM: OK. Overall, great work on this. The alerts fired right at the right time and before (I suspect) Tech Services even knew there was problem. Nice job.
The data the agents gathered helped campus IT figure out what was happening. They provide storage to my machines using a distributed storage system called Ceph, and they had made a change to it, not realizing that I also run Ceph inside my cluster. Turtles all the way down: Ceph running on Ceph produced terrible storage performance. The fix was to move my machines onto local disks, which is safe in my case because everything I need to keep is replicated by my own Ceph and backed up as well. A flaky compiler cannot investigate a strange storage problem on your cluster, gather the data, give it to the people you’re working with, and put monitoring in place so you see the next one first.
Brainstorming With DataBrainstorming With Data
You also don’t brainstorm with a flaky compiler. Last week we grouped students at tables by what they were working on: agents had read every student’s project summary, so we could seat students with others working on similar ideas. I was trying to decide how big the tables should be, since students don’t always show up.
Me 8:44 AM: I mean, don’t small groups work better here? We could also use a mix depending on project fit, yes? Can we incorporate the project summaries into the assignment process? What I mean is that perhaps students working on more common ideas should sit it larger groups whereas those working on more unique ideas in smaller. Does that make sense? It seems like it could complicate the assignment process.
Agent 8:44 AM: Your idea makes sense. Before I answer, I’m testing it against the stored embeddings, so the answer comes from data, not intuition.
Agent 8:46 AM: Smaller groups do work better here, but sizing groups by how common each project is didn’t help. I tested both on the stored project summaries, regrouping all 1,047 students who have one, within their hours, four ways. […]
That’s data-driven brainstorming, in 97 seconds.
A Research Assistant That Pushes BackA Research Assistant That Pushes Back
A flaky compiler isn’t a research assistant, either. While writing my last essay I remembered hearing that studies had found no relationship between research and teaching ability.
Me 4:21 PM: Trying to work logically here. Universities might say: Hey, we have the best researchers, therefore we have the best educators. But there’s evidence against this, yet?
Agent 4:22 PM: You’re right, and the evidence runs in one direction across forty years. Four results, in order of strength for your purposes: […] Feldman 1987. A review of 29 studies: the correlation between research productivity and teaching effectiveness is 0.12 […] Hattie and Marsh 1996. The meta-analysis of 58 studies: the overall correlation is about 0.06 […]
Those are Feldman’s 1987 review and Hattie and Marsh’s 1996 meta-analysis.
And compilers don’t push back. Later in the same work, the agent warned me that a paper I wanted to cite “does not literally say” what I wanted it to.
When I Have to Push BackWhen I Have to Push Back
It isn’t always right, and I have to push back too. In my conversational programming course we put limits on the AI we provide to students, and in the middle of one class we started hitting them.
Me 1:13 PM: Is the proxy up? Students are getting errors.
Agent 1:13 PM: The proxy is up and healthy. The errors come from Azure, which is refusing our requests with 429 (rate limited). […]
Me 1:17 PM: OK. This was a 492 from our ai.geoffreychallen.com proxy. […]
Agent 1:17 PM: That changes the picture: the 429 is your gateway’s own limit, not Azure’s. […]
The limit was one I had set myself, on a proxy the agent didn’t know about yet. I raised it, and the errors stopped six minutes after I first asked.
Beyond the Flaky CompilerBeyond the Flaky Compiler
That’s a laundry list, but that’s the point. If we want to help our colleagues understand how these tools are going to reshape computing, we need to go far beyond a metaphor like the flaky compiler, because these tools can do so much more than that.
I think some people are reluctant to acknowledge everything AI can contribute to software development because they’re worried about what that leaves for humans. But look at the technology around us. There are so many problems, so many places where technology doesn’t support human flourishing, so many problems it could help solve that we haven’t even started on. Sometimes we have solutions and they aren’t very good. Sometimes we haven’t started at all.
All of the ways these tools can help create technology are good news, because they mean humans get to contribute the human things. We bring our lived experience, the things we care about, and the things that concern us. We watch the results, see what happens in our communities, and use technology to make the world better. And by using these tools well, we’ll bring in a lot of people who could never create technology that serves their needs before. That’s what’s really exciting.