Dinosaurs don't do digital. Plus, fun vs learning.

Logpodge #2

Photo by Alex Conradt on Unsplash

𝐀𝐈 𝐋𝐨𝐠 mostly publishes long-form essays and book reviews, but as I start more pieces than I have time to finish, I use Logpodge posts to clear the decks of drafts so I can craft those 7000-word essays on William James the Internet craves.

---

Dinosaurs don’t do digital. Why do teachers have to?

Not so long ago, every university had a story about some old dinosaur of a professor who refused to use a computer, and so had the department admin deliver printouts of emails to be marked up and responded to with handwritten notes. My guess is that those dinosaurs, if any still existed, were wiped out by the astroid-like impact of the shift to remote teaching in 2019. Everyone adapted. Or, like old generals in Douglas MacArthur’s imagination, faded away. These days, though, resistance to the digitalization of work is no longer led by fuddy-duddy professors.

Nilay Patel, editor-in-chief of The Verge and host of the podcast Decoder, recently declared that his goal in life is to “never use software at work.” He says that if he could, he would simply have his staff bring him printouts for him to circle things with Sharpie and send them away. He describes this as “the dream I work toward every day,” even as he admits it seems increasingly impossible. Patel is not alone in his dream. Many of us are working through our compromises with the move from paper to digital, and dreaming of a return to the world of print. If only the screens would release my eyeballs, we whisper, I would spend more time reading books.

As software has eaten the world and our eyeballs, it has increased the capacity of organizations to gather and organize information about what our eyeballs do all day. That capacity gets sold to individual knowledge workers in the form of personal productivity tools that present data back to each individual worker to optimize their effort, but digitalization has also increased the bureaucratic appetite for data about your work and your employer’s ability to compel you to provide it.

The theory of digital transformation—that’s fancy for moving work from paper to software—was that the change would save money through increased efficiency and productivity. In practice, any savings have gone to pay for the new tools, the licensing and implementation costs, of course, and for additional IT staff to maintain and integrate data across systems. The theory of AI in the workplace is that it, too, will increase efficiency, with every worker having their own personal assistant. We don’t know yet what will happen in practice, but cost savings or a spike in worker productivity do not appear supported by the data.

In the Decoder episode where Patel declared his desire to return to paper, he and David Peirce, editor-at-large at The Verge, talk about how AI is expected to drive the next wave of apps and new productivity features, but I was more interested in their analysis of productivity tools that we have adopted over the past ten years, the tools that brought us out of the paper-based world and saw us through the pandemic. Working through Covid-19 has left teachers, perhaps more than other knowledge workers, feeling at odds with digital technology. Shifting business meetings to Zoom was rough, but teaching third grade virtually meant spending your entire workday at the epicenter of a slow-motion educational disaster.

We’re back in the classroom. Yay! And, here is a new technology that upends expectations around homework. Boo! But, look, it will do half a lesson plan and sort of write a draft of that weekly letter home to parents. Ummm.. The puzzle of why digital tools tend to make teachers’ jobs worse, not better, is not that difficult. If you want help solving it, you should read

Mathworlds

by Dan Meyer.¹

Meyer is a leading skeptic of the movement to imagine a world of personalized tutors transforming education, and his critique is bigger than AI. Mathworlds chronicles the way ed-tech companies have always ignored teachers as they design, build, and market products that are meant to be used in the classroom. And, having failed to serve teachers, tech executives turn around and say, or at least imply, that this failure is the fault of teachers.

The advantages of a new teaching machine are always self-evident to those who build them, and in their minds, it is always the slow-witted, under-trained teachers who are the problem. This long-standing cycle is shaping explanations for the disconnect between the large numbers of students using AI for schoolwork while few teachers adopt these new tools. AI company executives and AI consultants say that teachers need to get with the AI program. The survey data showing teachers are slow to adopt AI products is evidence that they are failing to prepare students for the brave new world of AI.

Meyer will have none of this:

To the degree that tech and business leaders reckon with this data at all, they offer explanations that are, to my eyes, unconvincing. They say that teachers just need more training, that schools just need to offer more guidance. The most hopeless among them use the low usage numbers to indict schools and their goals, arguing we need to change the nature of schooling to fit the strengths of AI.

Much of Meyer’s writing, including this post, chronicles how little consideration there is for how actual teachers would use AI tools in actual classrooms.

This blame-the-teachers framework for new ed-tech has consistently failed as a program for educational reform, but it is a successful sales tactic. The key to understanding why is to understand that teachers are not the customer. Educational software is sold to educational bureaucrats and managers. This is true of most software designed to make knowledge workers more productive. It explains why, as Patel and Peirce say, we have a love-hate relationship with our digital tools. They don’t have to delight us. They just have to delight our bosses.

Of course, many of these tools provide some value to individual workers, but using them comes with tradeoffs. The blessing of Zoom is that it enables working remotely. The curse is that your days at home to work are no longer yours. “I know you're working at home that day, so I sent you a Zoom link,” your department chair or boss says helpfully. The new payroll or reimbursement software enables you to make a request or submit a form anytime, anywhere, but it also scatters distracting reminders all over your email and calendar. And, if you don’t have the information you need to complete that required field, good luck getting a person to help you find it.

Behind the freedoms and constraints of our new work-life balance is the reality that implementations of institution-wide tools are overseen by back-office managers, mostly for the benefit of the back-office managers. Meyer details the ways the tools themselves fail teachers because they do not take into account the context of the actual work of teachers, but this is not always a problem with the tool itself. Sometimes it is that organizations implement the tools to solve management’s problems on management’s terms. Thus, the needs of teachers are sacrificed to the twin gods of regulatory compliance and bureaucratic standardization.

The back office has priorities. They need to track the money, enforce regulations about data privacy, document that the district curriculum is being followed, limit the risk of nuisance lawsuits, and prevent people from doing things that generate parent complaints. For a manager, these priorities more than justify paying for new software. More information and better financial controls are worth a lot to a bureaucrat. To make sure the tools work properly means fitting those who teach to the digital machinery, not the other way around. Meyer highlights a teacher's comment on his recent post about the types of paperwork schools demand that teachers complete:

The task of collecting paperwork and money needs to be taken away from teachers now! There are plenty of tools to collect information and money. Things like lesson plans, worksheets, and creating other resources should all be part of a sound resource/curriculum. If we want education to improve, we need to give teachers more time to improve instruction by being creative, working together, honing their craft, and honestly...resting. 

Everyone wants education to improve, but if you are an educational bureaucrat, your idea of improvement means efficient and frictionless workflows dedicated to data collection and financial accounting. For management, teachers matter to the effective functioning of the digital machinery of schools primarily as the flexible interface for collecting data from students.

School reform will not be accomplished through introducing new technology to make it more efficient. Real reform, if it happens, will be accomplished by altering the machinery to give teachers greater autonomy and freedom—freedom from bureaucratic demands and the freedom to collaborate with colleagues on instructional methods and tools that they, not expert consultants and back office managers pushing “evidence-based” solutions, believe support learning.

Teachers need the freedom of the dinosaur to stick with paper.²

---

Note to readers: I suppose now would be a good time to let you know that at the end of October, I am leaving my full-time job as an educational bureaucrat to write and teach more. Financially, this makes no sense. However, I expect that the autonomy and freedom I gain will improve every other dimension of my life. So, here goes.

Putting up a paywall is not part of the plan, so if you like what you read here, recommending 𝐀𝐈 𝐋𝐨𝐠 to others remains the best way to support what I do.

---

Photo by Markus Spiske on Unsplash

Fun vs learning

I believe, as do many others, that the drive to push personalized AI tutors as the answer to our educational problems will end in tears, both for teachers forced to use new AI tools and for investors who convinced themselves that this time is different. The actual educational value of generative AI, at least in the short term, seems to be improving tools we already use. Grammarly is the example I know best because I’ve used the product for years to help correct my atrocious spelling and try to impose order on my consistently inconsistent decisions about where to put commas. I don’t have to chat with a bot to get editing advice, and if the advice distracts me, I just turn off the interface. The new AI features do not tempt me, mostly because, in my experience, LLMs do nothing to improve my prose. However, I see how they might provide value for those who do not have such a high opinion of their own writing.

This Decoder episode showed me another example of how AI is changing an existing educational product. AI has arrived as a feature in Duolingo, and listening to how computer scientist and CEO Luis von Ahn understands its educational value was eye-opening for me, especially in light of this essay from

Josh Brake

about the relationship between learning and fun. Josh is as skeptical of efficiency as Dan Meyer.

As we continue to encounter a flood of new technological tools offering to make our learning more "efficient" we had best be asking what exactly that means. As I've said before: if this technology helps us to more effectively engage with the frictionful state of authentic learning, then I'm all for it. But I am suspicious that this will rarely be the case.

The drive for efficiency is an important context for thinking about the tradeoffs between fun and learning, and Josh’s essay is well worth reading for its taxonomy of fun and how each category of fun relates to learning. In von Ahn’s view, fun is the most important element of Duolingo’s success. After all, you cannot teach someone who is not there, so Duolingo makes engagement, not efficient mastery of content, its first priority. This commitment to fun over mastery invites many questions, but no matter how you see that tradeoff, it seems to have worked out pretty well for the company and maybe for its customers. According to von Ahn:

It’s the most popular way to learn languages in the world. A fun fact: there are more people learning languages on Duolingo in the US than in all US high schools combined. This is true in most countries in the world. We teach languages to more people than the public school systems.³

My family is part of the platform’s growth. My daughter studies French in school, supplemented with a daily dose of Duolingo (or is it the other way around?). My son, who is really into his German heritage, is learning the language on his own using the platform, supplemented with conversations with his mother and grandmother. I have offered to pay for weekly lessons with a native speaker, but they both refuse. Thinking about why they prefer an app to a human, I was struck by von Ahn’s report that Duolingo’s internal research shows

somewhere between 80 and 90 percent of language learners don’t want to talk to another human. They may tell you they do, but they don’t. It’s because when you’re learning a language, you’re pretty shy about it, and only the extreme extroverts are okay talking to a stranger on video in a language that they’re not very good at. The majority of people won’t do it.

That rings true, and it relates to Grammarly’s success and history as well, which, in my experience, is used differently and often in ways I find more educationally appropriate by writers whose first language is not English. Students are pretty shy when it comes to using an unfamiliar language with strangers, so having an LLM as a practice field for writing or conversation makes sense.

That makes the case for incorporating LLMs into Duolingo pretty clear. The company has a cast of AI characters for learners to interact with and is using AI to create simulations, which I have long suspected is where the educational value of LLMs will be found. Here is von Ahn:

The way Duolingo works is that the homescreen is basically a path, and you’re just doing lessons. Some of the lessons are now going to be this thing we call an “adventure,” which is really just one of those video games where you move characters around. What’s cool about it is that you’re learning how to solve real-world situations on Duolingo.

For example, it’s like a little video game where you are one of the characters and you’re told, “Okay, go buy a pizza.” You move and have to ask around, and then you ask some people, and they tell you, “Oh, the pizza place is over there.” It’s super fun and it helps you learn to navigate the real world. So we’ve been working on that. What’s cool about that feature is that all the scenarios were mostly generated by AI. In the past, that feature would’ve taken a long time to scale, but we were able to scale it pretty quickly because of AI.

Playing a pizza-buying game or simulating a conversation with an emo teen AI character in order to learn a new language seems fine to me? My hackles don’t rise the way they do when I see a human instructor’s digital twin or a chatbot based on a historical figure I revere. That said, I don’t teach modern languages, so maybe my ignorance explains why I think it is okay for my kids to use AI features on Duolingo even as I resist the idea of writing a book report based on chatting with the reanimated digital corpse of W. E. B. Du Bois. To demand we use AI the same way across all academic disciplines and learning contexts seems like a foolish consistency.

If you think that sounds like a self-serving rationalization and that my fanboy admiration of Luis von Ahn, who coined the term human computation and doesn’t live in Silicon Valley, is just an effect of the brand of snake oil he sells, let me know in the comments.

---

---

𝑨𝑰 𝑳𝒐𝒈 © 2024 by Rob Nelson is licensed under CC BY-SA 4.0.

1. In response to a comment I made to one of his posts about the return of Audrey Watters to blogging about ed-tech, Meyer joked that he was honored to have served as a “discount Audrey Watters.” The truth is that Meyer, like Watters, is among the clearest and strongest voices shouting into the winds of techno-optimism that blow out of Silicon Valley across the educational landscape. Those voices are important given how well-funded the wind machines have become.

2. In case this reads as an argument that teachers should refuse to use AI on principle, let me be clear. I believe that teachers should be the ones to evaluate and determine the educational value of the tools used in their classrooms. Here is an essay by

Alicia Bankhofer

about the different roles teachers should play in evaluating and adopting AI tools. In my view, this teacher-centered, experimental approach should guide all educational technology decision-making.

3. Ben Riley of Cognitive Resonance asked in a comment if I knew of any evidence that people actually learn a language using Duolingo. Honestly, I don’t, and there is a case to be made that the app is used so superficially gamified that it can only scratch the surface of actual learning. That said, I find Duolingo refreshingly open about the limitations of their product and the value of experiences the app cannot provide:

Duolingo is a good tool for learning languages, but it won’t make you fluent by itself. It’s great for building basic skills in a fun way. To really master a language, you need to do more than just use the app. Try taking classes, talking with native speakers, and learning about the culture. Use Duolingo as part of a bigger plan to learn the language.

In the interview, von Ahn is happy to agree that a human tutor is better than the app at teaching and that there is uncertainty about the educational value of AI. That openness intrigues me. As does the fact that his company uses subscriptions paid by rich people in the Global North to provide an ad-supported product used by millions of people in the Global South for free.

I understand why he gets less attention than a master showman like Sam Altman, but von Ahn is running a publicly traded company worth over 12 billion dollars that sells educational products built on LLMs. And Duolingo is turning a profit.