on writing, and an MIT study

Key Points

• De­spite wide­spread in­ter­pre­ta­tions of a re­cent MIT study as ev­i­dence that Chat­G­PT erodes crit­i­cal think­ing, the study it­self is pre­lim­i­nary, high­light­ing re­duced cog­ni­tive en­gage­ment and mem­o­ry when users de­pend on AI tools for short, timed writ­ing tasks, but stops short of de­clar­ing AI in­her­ent­ly harm­ful.
• EEG-mea­sured brain ac­tiv­i­ty dur­ing stan­dard­ized, time-con­strained es­says may not ac­cu­rate­ly cap­ture gen­uine crit­i­cal think­ing or mean­ing­ful cog­ni­tive en­gage­ment; deep­er thought and orig­i­nal ideas often emerge through it­er­a­tive, re­flec­tive writ­ing tasks.
• Human eval­u­a­tors in the study con­sis­tent­ly rated es­says writ­ten with­out AI as­sis­tance as more orig­i­nal and mean­ing­ful, yet prior re­search has shown human eval­u­a­tors strug­gle sig­nif­i­cant­ly at dis­tin­guish­ing AI-gen­er­at­ed writ­ing, often per­form­ing no bet­ter than chance.
• Cur­rent stan­dard­ized essay as­sess­ments re­ward su­per­fi­cial struc­ture and flu­en­cy rather than gen­uine learn­ing, idea own­er­ship, or in­ter­nal re­vi­sion, a flaw now am­pli­fied by gen­er­a­tive AI’s ca­pa­bil­i­ty to mimic sur­face-level pol­ish.
• More au­then­tic as­sess­ments (al­low­ing prepa­ra­tion with var­i­ous tools but re­quir­ing un­aid­ed writ­ing) would bet­ter mea­sure crit­i­cal think­ing and mem­o­ry re­ten­tion, chal­leng­ing ed­u­ca­tion­al sys­tems to raise stan­dards rather than la­bel­ing tech­nol­o­gy as harm­ful.

When a thought cross­es my mind, or I start con­sis­tent­ly com­ing across a topic that I may find value in writ­ing about later, I typ­i­cal­ly store these ideas into a lit­tle notepad to re­vis­it. The in­ter­me­di­ary time is usu­al­ly where most of my think­ing hap­pens, though it is large­ly un­struc­tured and in some vague, ethe­re­al form at best. Once I hit some sort of vague crit­i­cal mass of in­for­ma­tion or ideas on a topic, I will un­der­take the task of for­mal­iz­ing my thoughts in writ­ing, af­ter­ward in which I may share it pub­licly for oth­ers to read and share their own thoughts in re­turn. I typ­i­cal­ly give these ideas a title that may not be in­dica­tive of the view­point of the final piece, but is a good re­minder of where my head­space was when ini­tial­ly writ­ing it on my list.

The re­sult is that I end up writ­ing less and prob­a­bly fall amiss of con­tent gen­er­a­tion al­go­rithms, but I do hope it ends up in the cre­ation of pieces that are more thought­ful. I want my time and writ­ing to be more than what is trend­ing at the time. What is com­ing up is a bit of an ex­cep­tion to avoid­ing trends, at least some­what, but is at least a bit more sub­stan­tive than talk­ing about how you can cre­ate ac­tion fig­ure im­ages using Chat­G­PT.

This most re­cent gap since writ­ing my last piece, how­ev­er, was due to com­plet­ing my Mas­ter’s in Ed­u­ca­tion­al Tech­nol­o­gy and In­struc­tion­al De­sign be­fore sum­mer, which had me at the the edge of my men­tal band­width. Now that I have fin­ished my Mas­ter’s de­gree, culminating in a capstone project built for rubric requirements, but that will large­ly be tossed into a void, I can get back to this style of per­son­al writ­ing, which I do find much more en­joy­able and with a bit more value.

De­spite being busy, I still man­aged to amass a piece of writ­ing ap­prox­i­mate­ly six thou­sand words long ti­tled the essay is dead, but Deep Re­search didn’t kill it. I had been slow­ly piec­ing to­geth­er thoughts on es­says over some time with a rather strong the­sis on how es­says, at least as typ­i­cal­ly taught in sec­ondary in­sti­tu­tions, have ex­treme­ly lim­it­ed value. Essay as­sign­ments have be­come less of an ex­er­cise in gen­er­at­ing orig­i­nal thought and in­stead focus on fac­tu­al re­gur­gi­ta­tion and syn­tax, and gen­er­a­tive AI tools have done a great job of ex­pos­ing their flaws. While I may go into this idea with a bit more depth later (I feel like that piece of writ­ing is still half cooked), I have de­cid­ed to move to a topic that is some­what tan­gen­tial­ly re­lat­ed and cur­rent­ly mak­ing the rounds on so­cial media.

Re­cent­ly, a piece from An­drew R. Chow, ti­tled ChatGPT May Be Eroding Critical Thinking Skills, According to a New MIT Study, made some waves across so­cial media and teach­ing cir­cles. It was the first thing I saw when head­ing back to X after a break, and, the very next day, a cowork­er of mine also sent the piece my way. If you are in­ter­est­ed more in the text of the study it­self, you may get it from MIT’s own page, or from arXiv. If the conversations on social media sites and in teach­ing cir­cles about the study are to be cited, de­trac­tors from AI usage seem to have found their smok­ing gun to show that tools such as LLMs are mak­ing you dumb­er. If re­ac­tions on­line are to be be­lieved, AI tools’ wide avail­abil­i­ty and ex­is­tence is going to be the cause of cog­ni­tive at­ro­phy across so­ci­ety.

I think the study is being mis­in­ter­pret­ed, though the au­thors’ vocal anti-AI view­points have not helped (they even tried to in­clude a prompt in­jec­tion “at­tack” in the paper, which failed to work). The study method­ol­o­gy is also flawed for essay writ­ing tasks and for eval­u­at­ing the ac­cu­mu­la­tion of cog­ni­tive debt.

I ac­tu­al­ly ap­plaud the ap­proach the au­thors took here on get­ting their study out, pub­licly, be­fore peer re­view. Set­ting aside that peer re­views are far from in­fal­li­ble (you are ask­ing busy peo­ple to do large­ly un­com­pen­sat­ed work on con­sis­tent­ly tight dead­lines), it shows agency on tak­ing ac­tion against some­thing they view as a prob­lem (some pol­i­cy­mak­ers rolling out LLM tools at age lev­els way be­fore they are like­ly to be de­vel­op­men­tal­ly ap­pro­pri­ate). If there is any skill that is going to be most valu­able in the age of wide­spread AI tools, it’s agency, and putting this paper out pub­licly means it will at the very least be read, and that dis­course around the paper and AI tool usage can ac­tu­al­ly hap­pen. Putting their work out like this at least as­sures that it will meet a kinder fate than, for ex­am­ple, a third of World Bank reports that are never downloaded or the deluge of academic research that starts and ends its life in a void. At the very least, hope­ful­ly its data will get scraped for LLM train­ing.

The paper is over 150 pages long, but in short, they took three groups of peo­ple and had them write es­says on SAT-style prompts. One group used no tools at all (the “brain-only” group). The sec­ond group could use Google Search, and the third used only Chat­G­PT (a con­trolled ver­sion of GPT-4o). EEG head­sets mea­sured their brain ac­tiv­i­ty as they wrote. Most par­tic­i­pants com­plet­ed three ses­sions in their as­signed con­di­tions, but a small­er sub­set re­turned for a fourth ses­sion, where the brain-only and LLM users swapped roles. Each essay was writ­ten in 20 min­utes, mak­ing time a real con­straint across all con­di­tions.

From their re­sults, across ses­sions, the brain-only group showed the high­est lev­els of men­tal en­gage­ment. The LLM group con­sis­tent­ly showed the low­est, par­tic­u­lar­ly in brain re­gions as­so­ci­at­ed with mem­o­ry re­call and at­ten­tion­al con­trol. In ses­sion four, when LLM users were asked to write with­out as­sis­tance, they demon­strat­ed sig­nif­i­cant­ly lower brain ac­tiv­i­ty than the brain-only group did in their un­aid­ed ses­sions, sug­gest­ing that prior ex­po­sure to AI may re­sult in sus­tained re­duc­tions in cog­ni­tive ef­fort, even when the tool is no longer in use.

De­spite this re­duc­tion in neur­al ac­ti­va­tion, es­says pro­duced by the LLM group were fre­quent­ly rated high­er by an AI scor­ing sys­tem, like­ly due to sur­face-level flu­en­cy and struc­tur­al pol­ish. How­ev­er, the human eval­u­a­tors con­sis­tent­ly fa­vored es­says writ­ten by the brain-only group, scor­ing them high­er for orig­i­nal­i­ty, depth of ar­gu­ment, and ev­i­dence of in­de­pen­dent rea­son­ing. These es­says, al­though some­times less re­fined, showed more var­ied vo­cab­u­lary and thought pat­terns in the nat­ur­al lan­guage pro­cess­ing analy­sis.

In­ter­view data fur­ther re­vealed a marked con­trast in par­tic­i­pants’ sense of own­er­ship and mem­o­ry. Those in the LLM group were fre­quent­ly un­able to re­call or quote from their own es­says short­ly after writ­ing, and many de­scribed their work as feel­ing less per­son­al or mean­ing­ful. In con­trast, brain-only par­tic­i­pants ex­hib­it­ed strong mem­o­ry re­call and ex­pressed a clear sense of au­thor­ship. Al­though the paper does not call these es­says “soul­less,” it sug­gests that LLM-as­sist­ed writ­ing lacked the dis­tinc­tive­ness and per­son­al in­vest­ment that char­ac­ter­ize human-gen­er­at­ed work.

The au­thors them­selves are far more mea­sured in their con­clu­sions than much of the com­men­tary cir­cu­lat­ing on­line would sug­gest, and the ac­tu­al find­ings are much more re­strained. What the study shows is that using Chat­G­PT for short-form, time-con­strained writ­ing tasks ap­pears to re­duce cog­ni­tive en­gage­ment, lower short-term mem­o­ry re­call, and di­min­ish par­tic­i­pants’ sense of own­er­ship over their writ­ing. These ef­fects were most ev­i­dent when users tran­si­tioned away from AI and con­tin­ued to show di­min­ished neur­al ac­tiv­i­ty, which the au­thors in­ter­pret as po­ten­tial de­pen­den­cy on ex­ter­nal tools.

How­ev­er, the paper wise­ly stops well short of de­clar­ing that AI tools are in­her­ent­ly harm­ful or in­tel­lec­tu­al­ly cor­ro­sive. The au­thors frame their re­sults as pre­lim­i­nary and ex­plorato­ry — im­por­tant trends that war­rant fur­ther study, par­tic­u­lar­ly as LLMs get in­te­grat­ed into ed­u­ca­tion­al set­tings with lit­tle struc­tured guid­ance. They raise valid con­cerns about how fric­tion­less au­toma­tion may re­shape the writ­ing process, es­pe­cial­ly when paired with shal­low as­sess­ments, but they do not treat the ob­served changes as signs of ir­re­versible cog­ni­tive de­cline. If any­thing, the study makes a case that how we in­te­grate these tools into learn­ing en­vi­ron­ments mat­ters more than whether we do.

The paper de­serves cred­it for the trans­paren­cy of its re­lease and the tone of its con­clu­sions. The paper it­self avoids panic and opens space for pro­duc­tive de­bate. This is the kind of work we should want cir­cu­lat­ing pub­licly: data-rich, method­olog­i­cal­ly trans­par­ent, and cau­tious in in­ter­pre­ta­tion.

Sev­er­al things jumped out at me from this study as odd:

1. The re­liance on EEG data to infer cog­ni­tive en­gage­ment dur­ing writ­ing and whether this truly re­flects mean­ing­ful or pro­duc­tive thought.
2. The use of stan­dard­ized, time-con­strained essay prompts, which pri­or­i­tize ex­ter­nal form over in­ter­nal idea de­vel­op­ment.
3. The strength of the claims around mem­o­ry and au­thor­ship, par­tic­u­lar­ly given the im­per­son­al na­ture of the writ­ing task.
4. The lack of clar­i­ty around the human scor­ing process, es­pe­cial­ly in light of re­cent find­ings that sug­gest peo­ple often can­not dis­tin­guish be­tween human- and AI-gen­er­at­ed writ­ing.

Purpose vs. Real-World Relevance of EEG Use to Measure Brain Activity

Using EEG (elec­troen­cephalog­ra­phy) to mea­sure brain ac­tiv­i­ty dur­ing writ­ing tasks does not straight­for­ward­ly map high ac­ti­va­tion to bet­ter think­ing or bet­ter writ­ing.

Also, in every­day con­texts, most deep think­ing hap­pens be­fore and after writ­ing, not al­ways dur­ing (see the start of this piece). I know when­ev­er I hit stan­dard­ized essay as­sess­ments as a stu­dent I went into an au­topi­lot state of trance, and noth­ing I would have been in­volved in I would con­sid­er deep think­ing as I was most­ly fo­cused on ex­ter­nal re­vi­sion process­es. Do EEG read­ings dur­ing a 20-minute typ­ing task truly map to idea-level en­gage­ment or syn­the­sis, es­pe­cial­ly with writ­ing as mul­ti­fac­eted as essay com­po­si­tion?

While I have no ev­i­dence, I also could not shake the feel­ing that the au­thor’s want­ed to do a cool look­ing EEG study, and then fit a piece of re­search to that, rather than ac­tu­al­ly think­ing if EEG’s would be im­por­tant to the out­comes that are ac­tu­al­ly im­por­tant to essay writ­ing.

Task Suitability as Standardized Essays Emphasize External Revision

The SAT-style prompts in this study em­pha­sized con­strained, im­promp­tu writ­ing, a for­mat that trains stu­dents to per­form under pres­sure, not to de­vel­op rich, in­ter­nal­ly re­vised ideas over time.

Re­search (in­clud­ing my own ex­pe­ri­ence) sup­ports that:

• Short-timed es­says push stu­dents to­ward sur­face-level strate­gies.
• Deep cog­ni­tive pro­cess­ing and own­er­ship emerge more in self-se­lect­ed, it­er­a­tive projects or col­lab­o­ra­tive writ­ing.

Would the find­ings hold for more au­then­tic writ­ing tasks: long-form es­says, per­son­al projects, col­lab­o­ra­tive re­ports, or in­quiry-based writ­ing? The study does not an­swer this. It im­plic­it­ly treats SAT-style es­says as a proxy for aca­d­e­m­ic writ­ing and crit­i­cal think­ing, which is a flawed as­sump­tion in terms of au­then­tic­i­ty, au­ton­o­my, and mem­o­ry rel­e­vance.

Very sim­ply, our cog­ni­tive sys­tems aren’t built to re­tain un­mean­ing­ful con­tent, which is how I would cat­e­go­rize most of the writ­ing prompts used in the study.

Reported Ownership and Memory

The study does show that in in­ter­views, many LLM par­tic­i­pants couldn’t quote or sum­ma­rize their own es­says short­ly after writ­ing, while brain-only par­tic­i­pants did so more ef­fec­tive­ly. But we should in­ter­pret this cau­tious­ly:

• 20 min­utes is not long enough for durable mem­o­ry en­cod­ing, es­pe­cial­ly for a non-per­son­al task.
• Mem­o­ry dif­fer­ences may re­flect writ­ing method fa­mil­iar­i­ty, not AI dis­en­gage­ment. LLM users may have fo­cused on prompt en­gi­neer­ing or edit­ing rather than in­ter­nal­iz­ing con­tent.
• In­ter­view data is sub­jec­tive and po­ten­tial­ly in­flu­enced by par­tic­i­pant ex­pec­ta­tions or con­fir­ma­tion bias, es­pe­cial­ly if they sus­pect what the study is test­ing.

Also, the idea of “own­er­ship” is slip­pery. What does it mean to “own” an idea gen­er­at­ed under time pres­sure in a lab set­ting? Would those same par­tic­i­pants feel dif­fer­ent­ly about an essay, a poem, or a story co-writ­ten with AI over days or weeks? Quite pos­si­bly.

Human Consistency and Bias in Essay Scoring

The study says that human teach­ers “con­sis­tent­ly scored brain-only es­says high­er,” but it’s un­clear:

• Who the teach­ers were (e.g., were they trained scor­ers, writ­ing in­struc­tors, or gen­er­al ed­u­ca­tors?).
• What rubric they used (was it holis­tic? Trait-based? Aligned with SAT stan­dards?).
• Whether es­says were blind­ed or ran­dom­ized in terms of ori­gin.

In prior re­search from Clark et al. (2021), hu­mans have con­sis­tent­ly strug­gled to re­li­ably dis­tin­guish AI-gen­er­at­ed writ­ing from human-au­thored work across a wide range of for­mats. With­out spe­cif­ic train­ing or prompts, eval­u­a­tors per­form at chance lev­els, rough­ly as re­li­able as flip­ping a coin. Even when train­ing is in­tro­duced, ac­cu­ra­cy only im­proves mar­gin­al­ly, often hov­er­ing around 55%.

Other re­search has found sim­i­lar pat­terns. Un­less the writ­ing con­tains ob­vi­ous give­aways—com­mon AI phras­es or for­mu­la­ic sen­tence struc­tures—eval­u­a­tors fre­quent­ly fall vic­tim to an­chor­ing bias. Pol­ished writ­ing is often as­sumed to be AI-gen­er­at­ed (I hate how I now get called out for AI writ­ing due to know­ing what an em dash is, or for my gen­er­al writ­ing pat­terns I have had for a decade), while dis­flu­en­cy or in­for­mal­i­ty is mis­tak­en for human au­then­tic­i­ty. Iron­i­cal­ly, at­tempts to mimic nat­ur­al “er­rors” can make AI out­put more con­vinc­ing to read­ers who as­soc­iate im­per­fec­tion with sin­cer­i­ty.

Ex­pert re­view­ers, such as pro­fes­sors, do out­per­form gen­er­al au­di­ences but still com­mon­ly fall prey to false pos­i­tives. And both human and al­go­rith­mic de­tec­tors are eas­i­ly thrown off by para­phras­ing tools or minor styl­is­tic tweaks, which sug­gests that the bound­ary be­tween human and ma­chine writ­ing is porous. I find it in­cred­i­bly sus­pect that a blind as­ses­sor would be able to con­sis­tent­ly tell the dif­fer­ence be­tween AI and human writ­ing.

What I Would Like To See

I’m not going to argue that copy­ing and past­ing from an LLM like Chat­G­PT leads to mem­o­ry re­ten­tion. But I also don’t think this study eval­u­at­ed the right thing, be­cause copy­ing and past­ing from any source in tight time­line con­straints isn’t going to re­sult in mean­ing­ful mem­o­ry re­ten­tion. If you gave me twen­ty min­utes and ac­cess to any tool, I’d do ex­act­ly what most peo­ple would: copy, para­phrase, and move on, just to com­plete the task and score well. They, know­ing­ly or not, gave the dif­fer­ent groups dif­fer­ent tasks. The LLM group would like­ly view the task as “gen­er­ate an essay,” while the other groups would view it as “write an essay” due to the time con­straints in­volved.

The study’s method­ol­o­gy re­in­forced how flawed our cur­rent essay-based as­sess­ments are at mea­sur­ing ac­tu­al learn­ing or thought. The writ­ing task re­ward­ed speed and sur­face-level struc­ture, not deep en­gage­ment. If any­thing, the re­sults re­flect the task more than the tool.

What I would have liked to see is some­thing clos­er to how I’ve taught in hu­man­i­ties class­rooms, though I’ll admit it’s hard­er to con­trol ex­per­i­men­tal­ly. For ex­am­ple:

Give stu­dents five po­ten­tial essay prompts ahead of time (I typ­i­cal­ly got the stu­dents to gen­er­ate these ideas to start them think­ing on how we could tie over­ar­ch­ing themes to­geth­er in class). Allow one group to use AI dur­ing their prepa­ra­tion, an­oth­er group ac­cess to search en­gine tools, and the other only their writ­ten notes or hard copy sources. Then, under proc­tored con­di­tions, three of the five po­ten­tial essay prompts will ap­pear on the exam, and stu­dents choose one to write about. No tools are al­lowed dur­ing writ­ing. Then com­pare per­for­mance, orig­i­nal­i­ty, and re­call.

That kind of de­sign would speak more di­rect­ly to how peo­ple pre­pare, en­code, and trans­fer knowl­edge and not just how they com­plete a timed task.

So, does this study prove that Chat­G­PT at­ro­phies your brain or makes you less in­tel­li­gent? Not re­al­ly. It does pro­vide ev­i­dence that over-re­liance on AI tools can lead to mea­sur­able changes in cog­ni­tive en­gage­ment, mem­o­ry en­cod­ing, and the per­ceived au­then­tic­i­ty of one’s own writ­ing (but I would argue this could be as a re­sult of an over-re­liance on any tool).

I don’t think their re­sults are in­valid, but I am con­cerned about how it was over­gen­er­al­ized from this sam­ple to broad­er claims about the at­ro­phy of crit­i­cal think­ing. We have been saying for decades about how technology is going to make people dumber, when re­al­ly we need to raise the bar as the tech­nol­o­gy ad­vances. How to do so in such an in­dus­tri­al ma­chine as ed­u­ca­tion in the cur­rent era is a story for an­oth­er day.

Other Pieces of Interest

Here are some other links that peo­ple may find in­ter­est­ing, but didn’t find their way into the main body of this piece:

• An­oth­er paper came out of Mi­crosoft, which I found much more in­ter­est­ing on AI and Crit­i­cal Think­ing ti­tled, The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers.