AI models are now ingesting scholarly content in an attempt to dispel their hallucinations. But another possibility looms: that AI will instead drag down scholarship into its muddy realm. A new essay on my newsletter: “AI Is Coming for Scholarship Next”
@ayjay Thanks for this thoughtful response! I do think that a possible outcome—for scientists at least—is that they simply expose their experimental data and write a much thinner layer over it. After all, most of these science articles aren’t read much by humans, but they are mined for info and for synthetic data purposes. (And, of course, for tenure/promotion/sigh.)
@dancohen Indeed — maybe I’m too literary in my thinking, but the way I’d put it is: AI calls for new genres of academic writing.
@ayjay I am, unsurprisingly, all in favor of that. But as a dormant historian of science, I’m a little depressed that we can see the end of the (rare but appreciated) well-written scientific article.
@ablerism Do you have any assignment ideas/approaches you'd be willing to share? I've been wondering a lot this summer about how to design assignments that students won't be tempted to just plug into ChatGPT.
@drewbelf Only the two I mentioned above, and I haven't scoped out how that would look in my domain of design yet: 1) Write a "critical edition" essay for a given text (so, from a distinct POV, can't do just summaries), and 2) Write an imagined dialogue between two (or more?) thinkers. Also thinking about just reviving blue books?
@ablerism Thanks! Sorry—I missed the earlier replies mentioning those two assignments. Some kind of "annotate a text" assignment (which sounds similar to the "critical edition" assignment, maybe?) is intriguing to me. I've had pretty good success assigning gobbets (I got the idea from a colleague who did her PhD in the UK), though those aren't quite AI-proof. I like the dialogue idea, too!
I remember no papers required of the students. A final exam was the sole written exercise. Deciding to prepare for it together, we students divided the huge reading list amongst ourselves, each contributing a brief précis of some portion of it. The exam, however, surprised us all. It consisted of a single challenging question, formulated roughly as follows: “Criticize the readings in the course, indicating what works you would omit, and what others you would include.” In this inventive bit of pedagogy, Barzun not only paid respect to his students, but also demonstrated, however radically, the idea of the teacher as learner. —Carl Schorske
I'm not a teacher, but I would imagine that requiring students to show some personal knowledge of the particular course (and of the teacher!) could deter the use of artificial intelligence.
@drewbelf Annotation is a great idea! Thanks for that. Also I had...never heard of gobbets, but yes, that's great too. I would imagine the discipline required to do a very specific and short response would be hard to AI out of. But as Alan says, it's really the chance to back up again and say: what am I trying to assess?
@ablerism yes, exactly, the gobbet is helpful on both deterring AI & encouraging reflection on pedagogical goals, especially in courses that center on interpreting texts. & students told me that they really liked the gobbet assignment (especially as compared to a more traditional research paper).
@LeoWong that's a great idea! I could see that yielding great results in an upper-level course for majors. I've assigned a similar prompt in my theology 101 course: why, based on what you learned in the course, does the university require theology 101 of all its students?
@drewbelf I love this too. (I was actually just telling some folks how glad I am, 30 years on, that I had to take two classes on the Bible in college. Encounter first-order questions and their meaningful address, grapple with ancient texts, hear the language of worship.)
@ablerism @drewbelf @LeoWong @dancohen @ayjay Love this conversation! I’m taking notes as I also look for new ways to assess. I had been assigning a poem project that included writing a poet bio and I now think that’s utterly useless - way too easy to get B-level results via AI. Short reflective answers to synthetic questions or imitations with tight prompts written in class seemed to work best last year.
@ayjay Been thinking about this lately and wonder what you have in mind? I have taken inspiration from some assgts you've set for your students (thank you!) and have been experimenting with epistolary form with mine, aiming for 'warmth' and connection. Audience matters to purpose and performance.
@towittowoo This was one big thing that came up when our faculty was discussing ChatGPT at the end of the year, is that it’s not good at writing for a specific audience. So including that in the writing prompt is a great idea, for now anyway.
@faithx5 This has been something I’ve been trying to take into account. If I can ask questions that require a specific contextualised process or answer, ChatGPT can’t answer well.