What is an LLM doing in my classroom?

Part 1: AI Fight Club

The essay below kicks off a series drawn from a talk I gave at the AAC&U’s Institute on AI, Pedagogy, and the Curriculum on January 10, 2025.

The name of the LLM (large language model) tool I used this past fall term is JeepyTA. I chuckled the first time I heard it but I find the “TA” part of the name problematic, even though it is necessary for the joke to work. I do not think of JeepyTA as a teaching assistant. Maybe a teaching aid is better. Like Prezi, the software I use for my talks, and the whiteboards with colored markers my students and I use in nearly every one of my classes, JeepyTA is an instructional tool.

This kind of semantic wrangling may not matter much. After all, assistant and aid are both job titles used for people who work in schools. But language shapes our understanding of new technologies and how humans relate to them. Until the 1960s, the term computer meant a person who calculates large numbers. By then, nearly all these jobs were performed by women who worked with electronic computing machines like ENIAC. Slowly, and then all at once, these machines could perform computations faster and with fewer errors than humans.¹

That example suggests something about the stakes when it comes to using LLMs in teaching. People have strong feelings about teaching machines because new technology has the potential to improve education, and they feel strongly that humans should be at the center of teaching and learning.

AI Fight Club

Those strong feelings inform much of what you read about AI and teaching on social media. Henry Farrell calls this AI Fight Club, “a no-holds-barred wrestling competition between two starkly opposed perspectives on the world, that keeps on going, and going, and going.” I’m a member of the club. I can tell you this because, as Farrell says, unlike the Fight Club of the Chuck Palahniuk novel, “The first rule of AI Fight Club is not that you don’t talk about it, but that anything that you do say will be sucked into AI Fight Club, so that it is interpreted as supporting one or the other position.”

I like to pick fights with people who anthropomorphize LLMs. Treating a transformer-based AI model like a person obscures how it actually works and confuses people who are trying to use these tools to teach and learn. Hence, my dislike for the term teaching assistant and my scorn for those hyping the technology as a labor-replacing technology, especially those who label balloons “ChatGPT but for teachers” and fill them from Silicon Valley helium tanks. Shooting my BB gun of a blog at All Day TA or Khanmigo feels satisfying, but it doesn’t answer important questions about what educational value, if any, LLMs offer.

The question on most people’s minds seems to be: Can an LLM serve as a substitute when human TAs are busy doing something else? Post-training a transformer-based foundation model on a college teacher’s syllabus and course materials is a popular answer to the “problem” that teachers are not always available. An LLM does not get tired, grumpy, or hit on an undergraduate who is just trying to ask a question. LLMs may have an unfortunate habit of confabulating answers, but human TAs are far from perfect in this regard. I’ve known more than a few humans to offer convincing-sounding answers to questions based on incomplete or erroneous information.

Having a specially configured LLM answer questions might be a useful addition to a teacher’s toolbox. Yet, as we consider that possibility, we should keep in mind that TAs do critical work that an LLM cannot. First-year undergraduates struggle in introductory classes for all sorts of reasons, not just because they cannot get their questions about course content answered on demand. Human TAs have been first-year undergrads themselves. They notice when a student seems distracted or unwell. They look skeptical or stern or pleased at an answer. They nod and smile. They chat about the weather. Or share gossip. They can offer a tissue and a sympathetic smile as a student cries over the grade on their first paper in Expository Writing. These are things LLMs cannot and will never be able to do.

I am struck by how many people who are worried about the future of fields like art history, English, and sociology are unable to connect the dot of automating the work of teaching assistants to the dot of fewer majors next year, and then to the dot of fewer graduate assistantships the year after, and then to the dot of the dean deciding not to replace the faculty line for your colleague who is retiring.

Passionate seniors and graduate students are better at promoting an academic discipline to first-year students than greybeards and greyhairs, no matter how distinguished and charismatic. Teaching assistants model what advanced students in a subject look and sound like in a way that encourages sophomores to imagine themselves looking and sounding that way. That’s not to say older teachers should get out of the game. I am pretty confident that I am a better teacher today than I was twenty years ago. But teaching excellence is not the same thing as effective recruiting.

Solving a small problem while making the big problem bigger

Farrell’s description of an LLM-based teaching assistant is “a course syllabus that speaks, combined with a self-summarizing reading packet.” Does such a tool solve the problem that students don’t read the syllabus carefully (or at all) and read course materials superficially (if at all)? A digital twin that cheerfully answers a student's question at 2am relieves teachers of the burden of waking up on Monday morning to desperate messages from students who just realized they don’t understand the material and are about to take a midterm. Or, it seems too.

I grant that having a machine summarize the assigned reading is more efficient than paying advanced students to lead small group discussions. Students don’t come to office hours, study sessions, or visit the tutoring center. They’re too busy, or it seems to them that’s why. Building an answering machine for students pulling an all-nighter or “an on-demand digital study buddy” addresses the long-standing problem of students not organizing their time well. What problem does it address for the teacher?

I don’t think the tenured tutorbot enthusiasts have thought enough about what this means for them or for the long-term prospects of their field of study. As John Warner points out, this solution continues the trend of devaluing academic labor. It does so when declining enrollments and program closures are much bigger threats to academic jobs, including tenure-stream positions, than the proliferation of extremely limited chatbot assistants being touted as educational solutions. That alone is a good enough reason to stop using the term teaching assistant to refer to LLM-based instructional aids, even for those who are jazzed about their potential.

Leaving aside fights over nomenclature and labor economics, there is the question of why automate the solution to the specific problem of reading what’s been assigned in a course. A student with a question, even a question that is answered in bold on the first page of a syllabus, is a profound opportunity. Answering such a question accurately while expressing interest in the student’s interest is the beginning of a conversation that could lead to a lifelong interest in a field of study, or at least help fill a seat in an upper-level course next term. Even a stern look and a “Do me the favor of reading the syllabus more closely, Mr. Nelson” imparts a lesson about the importance of reading a text with care.

I teach history to master’s and doctoral students studying higher education. Few of them walk in the first day thinking that history and historical thinking have much to do with their professional practice. My class is built around the idea that learning to read and write as historians do is of value to administrators in higher education. Any question they ask is an invitation to start a conversation about the value of what I teach. Why would I outsource my answer to a machine?²

Solving a bigger problem

Given the ongoing declines in budgets and enrollments, we should take care not to buy short-term technical solutions to small problems that exacerbate the larger problems. To the extent that faculty involve themselves in the allocation of resources, it seems pretty obvious to me that they should clamor for better pay and more opportunities for humans involved directly in instruction rather than for subscriptions to tools sold by AGI enthusiasts.

I like a good online brawl as much as anyone, but the real action here is not on social media. It happens in budget allocation processes in individual institutions, which, thanks to the algorithmic nature of bureaucratic decision-making, often seems like a process no longer accountable to human beings. They could be accountable, though. Faculty have more power than they realize, especially when it comes to what the bureaucrats call the “spend” on edtech. After all, the ostensible reason for buying students a software license or subscription to an AI tool is to help them in the classes you teach.

Rather than posting another jeremiad about AI and Silicon Valley, perhaps we should learn something about how decisions get made at our institution and try to influence how the money flows.³ If your institution is buying subscriptions from big AI companies or purchasing AI teaching assistants from a start-up promising to revolutionize education at scale, organize your colleagues to tell your CIO or provost to knock it off. Invest in experiments with small, open-source models managed by people who work on your campus. Better yet, tell them to hire more humans to help teach introductory courses and tutor students who need it. Maybe argue those humans should receive a living wage.

Now that I have unburdened myself on the subject of TAs, I would like to share some details about what an LLM was doing in my classroom last semester. How can I reconcile my commitment to putting humans at the center of instruction with using JeepyTA to teach historical research and writing?

I will be posting a series on that topic in the coming weeks. I invite you to bring some fight to the comments. As Farrell says, these “disagreements reflect actually important arguments that we need to have.”

---

1. Clive Thompson offers an overview of the history of women computers in Smithsonian Magazine (2019) that includes brief discussions of several books on the topic. Two of my favorites are Programmed Inequality by Mars Hicks and Hidden Figures by Margot Lee Shetterly, which was developed into a film released in 2016.

2. Let me acknowledge my privilege here. I teach one class per term of no more than 20 students. For me, teaching is a side project done for love. I realize that someone teaching a 4-4 load of large introductory courses is a different situation and that answering basic questions efficiently means something different in that context. There are solutions to this problem that do not involve AI. I do not understand why giving an AI company million-dollar contracts makes more sense than paying advanced students to help teach these courses.

3. Don’t get me wrong. I love a good jeremiad about AI and Silicon Valley. I just love faculty taking a role in making decisions about academic technology more.