How ChatGPT changed my plans for the fall
Until now, I’ve avoided getting myself worked up about ChatGPT.
Prompted by this article by a Columbia undergraduate this past spring, I thought that if a student knows enough about paper-writing to make ChatGPT work for them, in the way this student describes in his piece, without detection by a minimally alert instructor, that student has probably already mastered the skills of essay-writing far more than the author of this piece seems to realize. I at least could rest easy with the knowledge that if a student used ChatGPT to write a paper for me, and it was good, I wasn’t not teaching that student what they needed to learn how to do.
But this recent article, by a Harvard undergraduate, made me think again.
I decided to ask my daughter to run through ChatGPT a bunch of take-home essay questions that I had assigned to my students this past year. One course was on politics and literature, the other course was on American political thought. I’ll admit I take a certain foolish (and, I now realize, complacent) pride in asking somewhat out-of-the-way type questions of my students. So I wasn’t too worried.
The initial answers ChatGPT spat back reinforced my complacence. All of them were well written and structured, good on the surface. There were none of the usual flaws you find in student writing: each sentence logically followed the other, paragraphs had points, transitions were purposeful rather than obfuscatory, that is, trying to cover the tracks of an unjustified leap in logic or evidence.
But all of the answers lacked a clear or strong thesis, provided needless exposition, referred to texts not read for class, and made basic errors about the texts, mistaking their genre and so on. If not easy to prove as not the student’s own work, they’d still be easy to assign a grade of C or lower to, simply on the basis of my rubrics for papers.
I was feeling pretty good about things.
Then my daughter started refining her inputs, putting in more parameters and prompts. The essays got better, more specific, more pointed. Each of them now did what a good essay should do: they answered the question. It became clear that so long as a student has a minimal sense of what a paper is supposed to look like or do, or at least knows what a bad paper (by my lights) looks like, they could easily use ChatGPT to come up with excellent answers to even the most out-of-the-way questions.
Where I had initially thought that such a student would have to have mastered quite a few skills in order to do this—that is, would be able to write such a paper on their own—it’s clear to me now that that’s not necessarily the case. Students just have to be able to spot the difference between good work and not good work, which even the most struggling students can already do. It’s always been amazing to me that students who have a difficult time writing a thesis statement can spot it a mile away in another student’s essay. Likewise, a well structured paragraph or paper. That doesn’t mean they can do it themselves, though.
In all my nearly 30 years of teaching, I’ve never once assigned an in-class test. But it looks like until a better option comes along, I’m going to have to go with in-class midterms and finals. It makes me sad, but I’m not sure what else to do.
Update (4pm)
As much as I complain about grading papers, it makes me sad that I may not be able to do this kind of work with students anymore.
For me, grading was never about grades; it was about the intensive feedback, the ongoing revisions of drafts, the individual conversations with students, that went into doing good work.
Good work was never about writing good papers. It was about being able to order your world, to take the confusion that one is confronted with, and turn it into something meaningful and coherent. And to know that that doesn’t just happen spontaneously or instinctively; it’s a practice, requiring, well, work.
That’s not simply a skill for college classes. That’s a life-long practice, of being able to see a situation, pick out those elements that matter and lend it significance, and bring clarity out of chaos.
That’s critical to being a good friend, a good parent, a good citizen, a good neighbor, and having a good life. I really, firmly believe that. I wouldn’t spend as much time as I do on student papers if I didn’t.
But now it all kind of seems pointless. I’ll still do that work in class, obviously, but there is something about clear writing that is connected to clear thinking and acting in the world, that I don’t think can easily be replicated in other mediums.
The only thing, in my life, that has even come close to what writing forces me to do is psychoanalysis, not therapy, but five days on the couch, with your analyst behind you saying almost nothing. Only on the couch have I been led to externalize myself, to throw my thoughts and feelings onto a screen and to look at them, to see them as something other, coldly and from a distance, the way I do when I write.
34 Comments
-
Here’s a sincere question (it might come across as snarky, but it’s really not): why bother switching the in-class exams if your learning goals are better served by essays (as are mine)? You can say up-front in the syllabus that students are expected to do their own work without using AI to write essays, and that violating that rule is academic dishonesty. Then it’s up to students as to whether they want to learn what you’re teaching with your essay prompts, or want to cheat (as it has always been). Probably the honest students will be better served by writing essays & getting your feedback than taking in-class exams – why punish those students to try to minimize cheating? What’s the real harm for students who opt to cheat by using AI to write papers in passing the class? (I know if feels shitty to be deceived, but is that worth the trade-off to change pedagogy?)
After 23 years of teaching, I’ve come to realize that my job is neither to police students who don’t want to learn nor to rank students via grades, but to maximize learning for those who want to learn, and try to inspire the others to try to join in the learning.
-
RE: “why bother switching the in-class exams if your learning goals are better served by essays”?
The learning goal is
“being able to order your world, to take the confusion that one is confronted with, and turn it into something meaningful and coherent. And to know that that doesn’t just happen spontaneously or instinctively; it’s a practice, requiring, well, work… a life-long practice, of being able to see a situation, pick out those elements that matter and lend it significance, and bring clarity out of chaos.”
It should be clear that this is something that is associated not (just) with the ability to write an essay, but actually writing it. I recall that on his blog, Brad Delong has more than once said (and it is not original to him) that he writes so that he knows what it is that he thinks. Having Chat GPT (or “Tchatte, j’ai pété” as they say in French) construct the essay vitiates the exercise. This is obviously true for those who use it. The fact/likely possibility that others are using it and getting good grades demoralizes those students (“honest students”) inclined not to use it. As with any case of widespread cheating (or honor code violation), it increases cynicism and lowers the tone of the class, the morale if you will, effecting participation, the quality of discussion, etc. Time spent in class and on studying for class becomes widely seen as time-serving. This of course is true not only of the students but also the instructor…
-
“What’s the real harm for students who opt to cheat by using AI to write papers in passing the class?”
As a teacher, your first job is to accurately assess the students’ ability. Cheating prevents you from doing that.
Moreover, you are unfairly giving good grades to students who cheated and very likely giving worse grades to students who didn’t.
Teachers who ask questions like yours tend to see school as just a personal development system. Not sure why you were able to get this far in teaching without seeing the catastrophic impact that laissez faire attitudes towards cheating have on the entire system. Teachers like you kill the system entirely, creating ever more cheaters.
-
I’m torn about cheating in just the way Corey is. I don’t give take-home exams any more because the cheating annoys me so much. But nevertheless I don’t believe that my first job as a teacher is to assess students abilities; it is to help them learn difficult material. It would probably be possible to assess their abilities without teaching them anything (see for example, the SAT).
I’m wondering if a possibly approach to the “essay” issue would be to give them the question the day before the test, let them think about it (and discuss it with AI), but have them write the essay in class without notes. My guess is that if the AI is helping them clarify their thinking, you’ll get a good in-class essay, and if it isn’t, you won’t.
-
“As a teacher, your first job is to accurately assess the students’ ability.”
Well, that’s not even in the top ten of my jobs as a teacher! My first job is to facilitate learning – everything else stems from that. Assessing students’ abilities usually runs counter to facilitating learning more than helping it.
-
Jason –
As a long time teacher (sometimes teacher of writing), I wholly agree with your comments.
A story: one time I was teaching at a very prestigious university, where there was a rapidly increasing problem with students cheating. There was a big meeting held in my department with much discussion of ways to solve the problem. Almost all of the focus was on preventing cheating, enforcement of rules, and punishment.
It struck me that no one was discussing the underlying reasons why the institutiom had created an atmosphere where so many students thought it was in their best (educational) interests to cheat
I often find it surprisingly difficult as a teacher to keep my eyes on the prize, and to refocus myself on the goal of empowering students to be more actualized leaners with agency and executive control over their learning process.
-
-
-
Here’s one sincere answer to the question, which doesn’t strike me as snarky: the prospect of grading and commenting on work not actually written by the students is just too depressing to bear. There’s always been a risk here. Many of us put *roughly* the same effort into grading all student work, even though only some of them profit from our efforts, and there are always some who have cheated. So grading and commenting involves a leap of faith. But the prospect of grading heaps of symbolic output generated by LLMs rather than students makes makes it hard to find the strength needed to make that leap.
-
Jason Mittell,
I agree with you in many ways. The pandemic brought on new waves a cheating, and our best defensive strategies tended to harm non-cheaters more than they deterred cheaters. The same with this mess. Still, it is not all about virtue. Consider sexual consent. Condoning a culture of cheating — even a naïve one — violates our mission as educators.
Cheating is also a crime of opportunity. Easy assignments are low-lying fruit. I tend to assign topics that have not been answered: “Author’s response to the flying-monkey problem seems inadequate. How might you provide a better response, that takes into account the further objections raised in lecture and discussion?” GJ
-
-
Offhand, I might suggest that this situation is a deeper challenge for society. I suppose there is a short-term response, in the education system, as you’re planning to follow. Do the tests with in-class supervision to ensure honesty. But beyond that, this questions what is being taught, why, and to what purpose. If AI can write better than most humans, will the companies employing college grads really care as long as it serve’s company needs? In the end, romantic scholarly ideals are irrelevant to higher education within capitalism. Being nostalgic for literary talent might be akin to being nostalgic for cursive writing, abacus calculations, reading dead languages, equestrian skills, and such.
The employable skill set of the future could, instead, be knowing how to use such writing technologies and software to better effect than others; similar to how we’ve offloaded information retention to our devices, rather than relying on the mnemonic systems maintained by highly respected practitioners. Think of how many elite abilities of the past (e.g., medieval martial skills of knights) were eventually done more easily, cheaply, and effectively by the commoner (e.g., peasants killing knights with crossbows) or simply made entirely obsolete (e.g., riding horses into battle), such that they are no longer used as indicators of class status. Maybe that is where writing skills are heading. But no worries, smart clever elites will find ever new proxies and symbols to demonstrate class status and, by way of social Darwinism, to filter out the rabble.
That likely sounds dismissive, condescending, or cynical. But that isn’t my intention. It’s simply an observation where old values will likely be overturned and discarded, as has happened so many times before. I’m thinking of this more in terms of philology and media studies (Marshall McLuhan, Walter J. Ong, etc). Changing media technologies transform all of society and culture, not just our values but also our mentalities and identities, our behaviors and ways of relating. For example, some go so far as to argue that literacy created modern ego-consciousness (Bruno Snell, E. R. Dodds, Julian Jaynes, Joseph Henrich, etc). Human nature is not a singular, unchanging thing but is socially constructed; where new media technologies have unforeseen consequences that take generations or centuries to become apparent. We spent centuries, millennia really, building a literary culture. But what now?
-
I had a thought about the change of literacy over time. A fully literary culture is a rather recent invention. In the West, we’ve technically had a literate majority possibly for a few centuries. But even then most of the literate were probably barely functional literates, until mandatory public education.
In early civilization, writing and reading weren’t even seen as elite skills. Most leaders and aristocrats were illiterate. That changed at least to some degree with some later societies like the Alexandrian Empire and the Roman Empire. But by the time the Middle Ages came around, most of the elite were illiterate again.
For most of Western history, literacy was mostly a lowly skill of scribes and monks. Even into quite recent history, many elites would use lowly workers to transcribe their spoken word; and transcribing likely often involved editing and adding style. That might’ve been still true until not long ago.
My grandmother only had a high school degree, but that made her literate. She worked as a secretary at a state college where she made money on the side typing up the work for professors. She also would write my uncle’s essays when he was in college. My uncle, though getting a degree, never liked reading.
-
I believe he demonstrated that the purpose of the essays is not the output itself but the process and the kind of thinking it encourages the participant to engage with.
It could be the employable skill set does change and but you are still left with a pool that cannot think and reason in the fashion Corey hopes to install in them.
We may have google constantly at the tips of our fingers which makes memorizing so much data no longer useful but I would categorical reject the idea that knowing how (the process and methods) to retain information remains and invaluable skill.
-
I get it. I wasn’t personally criticizing Corey Robin for his own purposes. But my point is that the capitalists who rule our society might not particularly care what is valued by either leftist nobodies like me or leftist professors like him. And these days, universities increasingly get funding from private sources, including corporations and corporate-backed organizations. Like it or not, it’s the world we live in, even as there are those who nobly fight the good fight in trying to hold back the incoming tide.
To respond directly to your last point, I suspect, if not categorically reject, that the new technologies and software are inherently in conflict with any of the old ways of knowing (the processes and methods). And I doubt prospective employers are likely to see a conflict. Such things are just yet more tools that will be used for good and ill with consequences we won’t predict, hence likely transforming our society and maybe our very identities and psyches. I’m not necessarily advocating this change, just pointing it out. Don’t kill the messenger.
-
Why wouldn’t they be able to think or reason without writing? This attitude enshrines writing as the only mechanism for getting to clarity of thought, which seems strange to me.
-
What would you replace it with? Nothing? I mean, other than, I guess, declaiming. I’m a musician and I believe very much in ideas beyond, and not subservient to or strictly derivative of ideas in verbal language – I’m even am considering a politics about it. But I think verbal language has enshrined itself, and for good reason. It’s indispensable. The almost unbearable explicitness is verbal language’s great strength. There is nowhere to hide on a page. This is this and that is that. I had to laugh at Corey’s comparison of writing to psychoanalysis, wherein the doc sits behind you says almost nothing. Exactly!
-
@Bob – Thanks for adding in your view. That gets at the point I was trying to communicate. Every technological medium promotes a thinking style, while suppressing other thinking styles. This isn’t limited to a change from text to new media. Many scholars like Walter J. Ong observed the vast difference of mentality that was caused by the transition from written text to printed text.
Even within a single media technology changes happen. With the introduction of punctuation, it finally became increasingly common for people to read silently, that is to say voicing the words internally, which maybe is what made inner speech possible at all. Think about it. Not that long ago, in historical terms, most humans probably didn’t hear a voice inside their head, or not strongly and clearly, with a clear inner/outer demarcation..
Talk about a new thinking style. So, there is an example of media changes that built on what came before. But there was no inherent conflict, just a change. Even so, according Julian Jaynes, this creation of an internal mental made possible greater capacity for deception, in individuals being able to play out scenarios on an internal stage of imagination and narratize it. But at the same time, it allowed for ever more complex thought. Punctuation would’ve allowed more linguistic recursion, for example.
By the way, in my previous comment, I meant to write, “I suspect that it isn’t the case, if not categorically rejecting it, that the new technologies and software are inherently in conflict with any of the old ways of knowing (the processes and methods).” Apparently, I was confusing myself and have yet to master this whole writing thing. Maybe new media is altering me.
-
-
-
-
I’m retired now (and grateful for that) but this new challenge in the assessment and evaluation of student work doesn’t strike me as too terribly different from what has gone before — enterprising but dishonest pupils could always avail themselves of paid writing on their behalf. It should be possible to use a variety of in-class on-demand writing with other forms of assessment to meet your needs as an instructor, though I understand how making changes to long established practice is an unappetizing prospect.
One question — isn’t generative AI particularly weak on providing accurate and properly documented sources?
-
A possible solution might be to have proctored essay writing; the students could write their essays in a room (hopefully COVID free) with a proctor. The room could have internet access but no access to ChatGPT. There would be no time limit and they could use their books. It might be possible to do this using ZOOM. Admittedly this could be a bit Orwellian.
Another possibility might be to have ChatGPT watermark the essays they write.
-
How old is your daughter, if you don’t mind me asking? I am curious as to how young or less extensively educated a person is able to do this to a college level.
If a professor’s junior in HS daughter could do this, I wouldn’t be shocked. If she was 12, wow.
-
You have an interesting dilemma: does Chat GPT foster critical thinking and writing skills? Or, does it support increasing helplessness? Fortunately, it cannot fix plumbing, wire a wall plug, or change a light bulb! So, some skills will remain un-replaceable.
The real problem with moving your tests into supervised circumstances is that those students who will perform well are likely to be the ones who come from privilege and are adept at writing the sort of essays that you expect. Thus, for someone who is concerned with equity in social relations, you find yourself in the position of now reinventing anew and re-institutionalizing the sorts of class and social divisions that you have criticized for so much of your life.
-
I agree whole heartedly about the sadness and sense of loss. I head a Writing committee for all of UCLA and teach a required writing course yearly. I have found that that there’s a band-aid for this coming year. I posted it on X on 8/9/23 if it helps anyone not sure what to do in the short run. https://x.com/davidshorter/status/1689297822754172928?s=46&t=skAukdQ9HYk29Q6wnyt_-Q
-
I’m not so impressed to the students who can write a good paper, if they don’t have a good social sense to share positive thing. In my observation, students who are good in math are not necessarily successful in life.
-
An additional issue is What the Univ or College Administration wants. Profs may assess, or inspire learning, but their teaching and assessing is validated by an institution. (Unless you are teaching in a private arrangement.) And then we as profs, cannot ignore the issue of pay and contracts, because our effort in assessing or inspiring, has been given a hard currency value by these institutions (sure, the value can be disputed, or is incomplete). Most of us have finite hours, and limited emotional/cognitive energy, that we can apply to teaching. And to emphasize, no students will come to learn from us in class, if they do not get Credit on the Transcript, from the Institution, from that learning. The Institutions could have a role to Support us with the whole AI-composition issue, do you see that happening?
-
You raise valid concerns about the potential for AI-generated writing to compromise the educational process.
While I share your concerns about academic integrity, I also believe that AI offers an opportunity to rethink our educational paradigms. In my article, “Unleashing Human Potential: Rethinking Education to Harness the Power of AI”, I explore how AI can be a tool for personalized learning and skill development, rather than a threat to it.
I wrote my article to prompt dialog on both sides of this timely issue.I look forward to your reaction.