I built an AI Literacy course

In my last piece on this site, I closed with a line about how my goal for 2026 was to build. At the time, I did not know ex­act­ly what that would look like. I had just fin­ished a stretch of writ­ing about LLMs, bench­marks, and the state of the field, and I was ready to stop talk­ing about AI tools and start mak­ing some­thing with them. That vague as­pi­ra­tion re­sult­ed in ex­pand­ing my mas­ter’s work into a four-mod­ule, re­search ground­ed AI lit­er­a­cy pro­gram for the cor­po­rate work­force, built on a cus­tom coded plat­form, live now at ai-literacy.ritchot.me.

I fin­ished my Mas­ter’s in Ed­u­ca­tion­al Tech­nol­o­gy and In­struc­tion­al De­sign just under a year ago, and the cap­stone project was the seed of what you see at that link. The cap­stone was a course teach­ing AI Lit­er­a­cy through the mech­a­nism of to­k­eniza­tion: how lan­guage mod­els ac­tu­al­ly break text apart, pre­dict the next token, and gen­er­ate lan­guage one prob­a­bil­i­ty at a time. I built it on Canva, de­liv­ered it through Google Class­room (I had an ex­pe­dit­ed time­line I had to meet be­fore I left my pre­vi­ous em­ploy­er), and it worked well enough to ful­fill aca­d­e­m­ic re­quire­ments. But the cap­stone was con­strained by the rubric and by the re­al­i­ty that aca­d­e­m­ic de­liv­er­ables are writ­ten for eval­u­a­tors, not for the peo­ple who would ac­tu­al­ly use them (I men­tioned this in my piece on the MIT study, where I called it a capstone built for rubric requirements that would largely be tossed into a void). The core idea, though, was sound. I want­ed to take it and re­build it as a real learn­ing prod­uct, some­thing an L&D di­rec­tor would en­vi­sion as a use­ful cor­po­rate AI Lit­er­a­cy course.

The gap the cap­stone had start­ed to ad­dress still large­ly ex­ists when I see peo­ple in­ter­act with “AI.” Peo­ple have ac­cess to AI tools and are al­ready using them, but they do not un­der­stand how these sys­tems ac­tu­al­ly gen­er­ate lan­guage. Peo­ple most­ly still treat them as magic in­for­ma­tion re­trieval ma­chines, and they lack the judg­ment frame­work to eval­u­ate whether what comes back is re­li­able enough to act on. The World Economic Forum’s 2025 Future of Jobs Report found that 63% of em­ploy­ers iden­ti­fy skill gaps as the pri­ma­ry bar­ri­er to AI-dri­ven trans­for­ma­tion. Tamkin and McCrory’s productivity research doc­u­ment­ed an 81% me­di­an task time re­duc­tion in­side the AI con­ver­sa­tion it­self (with RCTs mea­sur­ing the full work cycle show­ing be­tween 14-56%), and the dis­tance be­tween those num­bers is doing a lot of work. The pro­duc­tiv­i­ty gains are avail­able. The work­force does not yet have the skills to cap­ture them. And the train­ing that ex­ists tends to ad­dress tool fa­mil­iar­i­ty (how to prompt your LLM of choice for email draft­ing) rather than the un­der­ly­ing un­der­stand­ing that would let some­one eval­u­ate whether to del­e­gate cer­tain tasks and if the out­put is re­li­able.

I read An­throp­ic’s 4D competency framework a few months after the com­ple­tion of my cap­stone. The frame­work de­fines four ob­serv­able di­men­sions of AI flu­en­cy: Del­e­ga­tion (know­ing which tasks to as­sign to AI and which to re­tain), De­scrip­tion (com­mu­ni­cat­ing ef­fec­tive­ly with AI sys­tems), Dis­cern­ment (eval­u­at­ing the re­li­a­bil­i­ty of AI out­puts), and Dili­gence (main­tain­ing trans­paren­cy and ac­count­abil­i­ty in AI aug­ment­ed work). The 4D frame­work pro­vid­ed a clean vo­cab­u­lary be­hind ideas I sim­i­lar­ly in­tu­it­ed, and it pro­vid­ed the com­pe­ten­cy lan­guage that con­nects learn­ing ob­jec­tives to mea­sur­able work­place be­hav­iors. The re­search I had been col­lect­ing, from Handa et al.’s task level adoption data to cog­ni­tive sci­ence find­ings on over­con­fi­dence and unchecked AI ac­cep­tance, mapped clean­ly enough that the frame­work be­came the or­ga­niz­ing spine of the pro­gram.

The four mod­ules fol­low a spe­cif­ic se­quence: con­text, then ev­i­dence, then mech­a­nism, then ap­pli­ca­tion. Mod­ule 1 es­tab­lish­es why AI lit­er­a­cy is a busi­ness prob­lem using work­force data. Mod­ule 2 sur­faces what the work­force is ac­tu­al­ly doing with AI through in­ter­ac­tive data dash­boards built on task level adop­tion re­search and pro­duc­tiv­i­ty data. Mod­ule 3 teach­es how lan­guage mod­els ac­tu­al­ly gen­er­ate text (to­k­eniza­tion, next-token pre­dic­tion, at­ten­tion mech­a­nisms, con­text win­dows). This is the mod­ule that traces most di­rect­ly back to my orig­i­nal cap­stone: if you do not un­der­stand how these sys­tems pro­duce lan­guage, you can­not eval­u­ate what they pro­duce. Mod­ule 4 in­te­grates all four com­pe­ten­cy di­men­sions into ap­plied prac­tice: task de­com­po­si­tion, prompt re­for­mu­la­tion, out­put ver­i­fi­ca­tion, it­er­a­tive re­fine­ment, and a dili­gence state­ment ex­er­cise that asks learn­ers to ar­tic­u­late their own ac­count­abil­i­ty frame­work for AI as­sist­ed work.

Across those four mod­ules, there are 37 sec­tions, 12 in­ter­ac­tive prac­tice ac­tiv­i­ties (in­clud­ing fil­ter­able data dash­boards, a to­k­eniz­er play­ground, a next-token pre­dic­tion demon­stra­tion, and a multi-step AI in­ter­ac­tion sand­box), and 7 down­load­able ref­er­ence ma­te­ri­als de­signed as take-home tools for on-the-job ap­pli­ca­tion. Every mod­ule sec­tion traces back­ward to a doc­u­ment­ed gap in the re­search cor­pus and for­ward to a mea­sur­able as­sess­ment. If it could not be jus­ti­fied by ev­i­dence and mea­sured by an ob­serv­able be­hav­ior change, I cut it. The course today is what I con­sid­er the min­i­mal vi­able prod­uct. I in­tend to ex­pand it with fur­ther re­search pa­pers I have al­ready col­lect­ed and read, but I felt an ur­gency to get this course out now. It has been al­most a year since my orig­i­nal cap­stone, and I still feel that the L&D/AI En­able­ment pro­grams I have seen are teach­ing AI Lit­er­a­cy fun­da­men­tal­ly wrong.

The pro­gram scopes to the AI in­ter­ac­tion model most peo­ple use today: con­ver­sa­tion­al in­ter­faces, not agen­tic sys­tems. Tools like Claude Code, Cowork, Codex, and Antigravity op­er­ate on a slight­ly dif­fer­ent par­a­digm (au­tonomous multi-step ex­e­cu­tion rather than turn-by-turn di­a­logue) and would com­pli­cate the cur­ricu­lum sub­stan­tial­ly. I may build that as a sep­a­rate course. But the core com­pe­ten­cies the pro­gram de­vel­ops (know­ing what to del­e­gate, how to eval­u­ate out­puts, and when to in­ter­vene) trans­fer up­ward when the tools get more ca­pa­ble.

The course was built on a cus­tom coded plat­form rather than in an off the shelf au­thor­ing tool. The Canva and Google Class­room ver­sions had done what they could, but au­thor­ing tools im­pose their own con­straints on how con­tent flows and how in­ter­ac­tions be­have, and they limit what data you can cap­ture. I want­ed a learn­ing ex­pe­ri­ence where the in­struc­tion­al de­sign drove the ar­chi­tec­ture rather than the other way around. Build­ing the plat­form from scratch was also a bet on a ques­tion I think L&D will have to an­swer soon: how do you ar­chi­tect a learn­ing ex­pe­ri­ence as soft­ware? Or­ga­ni­za­tions build­ing in­ter­nal tools rather than pur­chas­ing them will need peo­ple who can bridge in­struc­tion­al de­sign and soft­ware en­gi­neer­ing, and the im­pli­ca­tions of that shift ex­tend well be­yond plat­form choice.

I built this pro­gram in ap­prox­i­mate­ly 150–160 hours of total de­vel­op­ment time, as a solo de­vel­op­er, while hold­ing a full-time teach­ing po­si­tion at an in­ter­na­tion­al school in Sin­ga­pore. Rough­ly 120 of those hours went to re­search, in­struc­tion­al de­sign, and plat­form build. The re­main­ing 30–40 hours went to it­er­a­tive re­view: ver­i­fy­ing con­tent ac­cu­ra­cy against source pa­pers, re­fin­ing in­struc­tion­al se­quenc­ing, and in­te­grat­ing ad­di­tion­al re­search. That work is easy to leave off a project time­line but rep­re­sents the dif­fer­ence be­tween a prod­uct that pass­es a sur­face-level re­view and one that holds up under scruti­ny. The early phas­es (re­search gath­er­ing, ev­i­dence com­pi­la­tion, in­struc­tion­al de­sign doc­u­men­ta­tion) con­sumed rough­ly four hours per week­end ses­sion (and at a rather ca­su­al pace). Once I had the spec­i­fi­ca­tions locked and the con­tent doc­u­ments writ­ten, the build phase picked up in pace, but even then, the total cal­en­dar span was ap­prox­i­mate­ly eight weeks, punc­tu­at­ed by a full week lost to ill­ness and re­duced ca­pac­i­ty from a sep­a­rate in­jury. To put that in con­text, the in­dus­try bench­mark for this level of in­ter­ac­tiv­i­ty es­ti­mates rough­ly 735 hours of de­vel­op­ment time.

That 735-hour fig­ure comes from Bryan Chap­man’s industry benchmark, a sur­vey of rough­ly 4,000 learn­ing pro­fes­sion­als. His Level 3 cat­e­go­ry (sim­u­la­tions, in­di­vid­u­al­ized in­ter­ac­tions, gam­i­fied el­e­ments) re­ports an av­er­age of about 490 de­vel­op­ment hours per fin­ished hour of con­tent; at the pro­gram’s rough­ly 1.5 fin­ished hours of Level 3 in­ter­ac­tiv­i­ty, that is a base­line near 735 hours for the con­tent alone. Chap­man’s num­bers as­sume off-the-shelf au­thor­ing tools and cover con­tent de­vel­op­ment only, so a cus­tom coded plat­form, an eval­u­a­tion frame­work with xAPI event tax­on­o­my, a re­search cor­pus built from pri­ma­ry sources, and project man­age­ment doc­u­men­ta­tion would push the real fig­ure high­er, though I will not put a spe­cif­ic mul­ti­pli­er on work the sur­vey was not de­signed to mea­sure.

Against that base­line, the com­pres­sion is rough­ly 4.6–4.9x, a 78–80% re­duc­tion across the full de­vel­op­ment cycle rather than the build phase alone, and it falls with­in the range Tamkin and McCrory doc­u­ment­ed for AI aug­ment­ed knowl­edge work. I achieved that as an in­de­pen­dent de­vel­op­er with no cor­po­rate legal re­view, no com­pli­ance re­quire­ments, and no learn­er data col­lec­tion to man­age. An en­ter­prise team would ab­sorb se­cu­ri­ty au­dits, ac­ces­si­bil­i­ty cer­ti­fi­ca­tion, data han­dling poli­cies, and cross func­tion­al co­or­di­na­tion that nar­row the ratio. The sav­ings would stay sig­nif­i­cant but less dra­mat­ic, and an­tic­i­pat­ing that gap is it­self one of the things the pro­gram teach­es learn­ers when scop­ing AI aug­ment­ed work.

The 150–160 hour fig­ure cov­ers this launch build; I am not count­ing the hours I will spend re­vis­ing and ex­pand­ing the course from here, and I could not find a re­li­able bench­mark that ac­counts for on­go­ing it­er­a­tion in any case. Sev­er­al re­search sources I have al­ready read still need to be worked more clean­ly into the con­tent, and the front end like­ly needs a re­design. Build­ing it taught me to rec­og­nize the tells of AI gen­er­at­ed front-end code, much as reg­u­lar users of AI tools de­vel­op an eye for the tells of AI writ­ing.

I was able to build this pro­gram in such a com­pressed time­line be­cause I had the do­main ex­per­tise to spec­i­fy what need­ed to be built and the in­struc­tion­al de­sign back­ground to know why. I also had enough hours with AI de­vel­op­ment tools to treat them as a work­ing part­ner. With­out those three things, the same tools pro­duce some­thing that looks plau­si­ble (the for­mat­ting is clean and the sec­tions are log­i­cal­ly or­dered), but the in­struc­tion­al se­quenc­ing is wrong, the as­sess­ment align­ment is off, and the prac­tice ac­tiv­i­ties test re­call rather than judg­ment. It pass­es a sur­face-level re­view. It does not change be­hav­ior. The tool did not re­place the ex­per­tise. The ex­per­tise is what made the tool pro­duc­tive.

The eco­nom­ics of learn­ing de­vel­op­ment have changed, but not in the di­rec­tion most peo­ple as­sume. AI does not make L&D cheap­er. It makes ex­pert L&D prac­ti­tion­ers sig­nif­i­cant­ly more pro­duc­tive and forces every mem­ber to more flu­id­ly work across the en­tire stack. A sin­gle prac­ti­tion­er with do­main ex­per­tise, in­struc­tion­al de­sign train­ing, and flu­en­cy in AI aug­ment­ed de­vel­op­ment work­flows can now pro­duce work that pre­vi­ous­ly re­quired an ex­pan­sive cross func­tion­al team. The value propo­si­tion of L&D teams has shift­ed. Vol­ume of out­put and mas­tery of a spe­cif­ic au­thor­ing tool are no longer valu­able met­rics. What sep­a­rates use­ful L&D teams from ob­so­lete ones is “taste”: the abil­i­ty to de­sign pro­grams ground­ed in ev­i­dence, spec­i­fy clear­ly what needs to be built (and what “done” looks like), more care­ful­ly eval­u­ate whether what was built ac­tu­al­ly changes be­hav­ior, it­er­ate quick­ly, and use AI tools as a de­vel­op­ment part­ner through­out.

The work­flow that pro­duced this com­pres­sion mir­rors what the pro­gram teach­es. Every build ses­sion fol­lowed a spec­i­fi­ca­tion-ver­i­fi­ca­tion loop: I would write a de­tailed spec­i­fi­ca­tion doc­u­ment (what to build, what con­tent to use, what ac­cep­tance cri­te­ria to meet, what de­ci­sions the AI should make silent­ly ver­sus what re­quired my ap­proval), hand it to Claude Code for im­ple­men­ta­tion, re­view the out­put against the spec­i­fi­ca­tion, and it­er­ate. The AI han­dled the vol­ume (gen­er­at­ing com­po­nent code, pop­u­lat­ing con­tent, wiring up state man­age­ment), and that freed me to work on other tasks in par­al­lel. I han­dled the judg­ment: ver­i­fy­ing ac­cu­ra­cy against source pa­pers, check­ing in­struc­tion­al se­quenc­ing, and catch­ing the mo­ments where a tech­ni­cal­ly cor­rect im­ple­men­ta­tion missed the ped­a­gog­i­cal in­tent. That loop, spec­i­fi­ca­tion then ver­i­fi­ca­tion, is the Del­e­ga­tion-Dis­cern­ment cycle the pro­gram teach­es. The build process be­came the case study.

The plat­form tracks learn­er progress (lo­cal­ly, I am not col­lect­ing any­thing on my end for this ver­sion), knowl­edge check re­spons­es, prac­tice ac­tiv­i­ty com­ple­tion, and time-on-task using an xAPI-aligned event tax­on­o­my. A built-in sam­ple admin dash­board (Cmd+Shift+A on Mac, Ctrl+Shift+A on Win­dows) sur­faces com­ple­tion pat­terns, knowl­edge check re­sponse dis­tri­b­u­tions, and event time­lines. The Kirk­patrick eval­u­a­tion frame­work (re­ac­tion, learn­ing, be­hav­ior, re­sults) is vis­i­ble in the pro­gram’s ar­chi­tec­ture di­a­grams by de­fault to sig­nal think­ing at the man­age­ment level rather than the course level. Be­cause eval­u­a­tion lives in the ar­chi­tec­ture, the pro­gram gen­er­ates the data an L&D di­rec­tor needs to jus­ti­fy con­tin­ued in­vest­ment and mea­sure be­hav­ior change: re­port­ing that usu­al­ly re­quires a sep­a­rate project. If the bet I de­scribed ear­li­er pays off (if or­ga­ni­za­tions do start ar­chi­tect­ing learn­ing ex­pe­ri­ences as soft­ware rather than pur­chas­ing them), L&D teams will need to work much more close­ly with tech­ni­cal teams. Some­one has to ver­i­fy that the AI gen­er­at­ed code is se­cure, ac­ces­si­ble, and main­tain­able. Some­one has to en­sure the plat­form ar­chi­tec­ture does not in­tro­duce com­pli­ance risks or data han­dling prob­lems. The era of “vibe coding” a learn­ing plat­form into ex­is­tence with­out tech­ni­cal over­sight is a li­a­bil­i­ty wait­ing to ma­te­ri­al­ize. This means or­ga­ni­za­tions will need to hire L&D prac­ti­tion­ers who are tech­ni­cal enough to col­lab­o­rate with en­gi­neers on shared prob­lems, or en­gi­neers who un­der­stand in­struc­tion­al de­sign well enough to eval­u­ate what the AI pro­duces against ped­a­gog­i­cal in­tent. I can do both for this project be­cause I am an in­de­pen­dent de­vel­op­er with none of the en­ter­prise con­straints I de­scribed ear­li­er. An en­ter­prise team would not have that lux­u­ry. But the peo­ple who can bridge that gap be­tween in­struc­tion­al de­sign and soft­ware en­gi­neer­ing are ex­act­ly the peo­ple or­ga­ni­za­tions will need to teach AI flu­en­cy to their work­force in the first place. For me, my de­sire is that this ends up with L&D teams being small, tech­ni­cal­ly mind­ed, and able to move and pivot quick­ly.

I keep think­ing about id Soft­ware¹¹ John Romero has described the early id team as a “hive mind”: no manager, each person owning a specific domain, everyone technically deep enough to self-direct. David Kushner’s Masters of Doom is the definitive account of those years and is worth reading for anyone interested in what small, technically obsessive teams can produce when the constraints are self-imposed rather than organizational. . Four peo­ple built Wolfenstein 3D in four months. Six built DOOM in thir­teen. Fewer than ten de­vel­op­ers shipped 28 games (if we in­clude their Soft­disk era) across the stu­dio’s first five and a half years. Their speed did not come from cut­ting cor­ners. It came from cu­mu­la­tive ex­per­tise, tight it­er­a­tion loops, and an ob­ses­sive in­vest­ment in build­ing their own tools rather than work­ing around some­one else’s con­straints. Deep tech­ni­cal own­er­ship per per­son rather than dis­trib­uted ac­count­abil­i­ty across a de­part­ment, and speed from do­main mas­tery and cus­tom tool­ing rather than head­count. John Car­ma­ck made the coun­ter­point him­self when he left Meta in 2022, describing an organization with a “ridicu­lous amount of peo­ple and re­sources” that con­stant­ly self-sab­o­taged and squan­dered ef­fort. The ver­sion of L&D I want to see looks more like id in 1993 than a cor­po­rate train­ing de­part­ment in 2024.

Or­ga­ni­za­tions that in­vest in build­ing this in­ter­nal ca­pa­bil­i­ty will have an ad­van­tage. The lever­age is in the spec­i­fi­ca­tion-ver­i­fi­ca­tion flu­en­cy that makes those tools pro­duc­tive: the abil­i­ty to write clear build in­struc­tions and eval­u­ate out­puts against ev­i­dence until the prod­uct meets a stan­dard. Teams that de­vel­op this flu­en­cy will pro­duce learn­ing pro­grams at a pace and cost their com­peti­tors can­not match. The 85% of employers who plan to pri­or­i­tize work­force up­skilling by 2030 will need to de­cide whether that up­skilling hap­pens through tra­di­tion­al de­vel­op­ment time­lines or through the kind of AI aug­ment­ed work­flow this project demon­strates. The pro­gram is live. You can ex­plore it at ai-literacy.ritchot.me. The full port­fo­lio doc­u­men­ta­tion be­hind it lives on the site: the needs analysis and research grounding, the evaluation framework, and the program design and project management records are avail­able there as in­ter­ac­tive walk-throughs with­in the course and as down­load­able ref­er­ences, for any­one who wants the full scope of what went into this. The ques­tion I keep sit­ting with is whether the field will de­vel­op its own flu­en­cy fast enough to shape what that change looks like, or whether it will be shaped by peo­ple who un­der­stand the tech­nol­o­gy but not the dis­ci­pline.

Footnotes

1. John Romero has described the early id team as a “hive mind”: no man­ag­er, each per­son own­ing a spe­cif­ic do­main, every­one tech­ni­cal­ly deep enough to self-di­rect. David Kush­n­er’s Masters of Doom is the de­fin­i­tive ac­count of those years and is worth read­ing for any­one in­ter­est­ed in what small, tech­ni­cal­ly ob­ses­sive teams can pro­duce when the con­straints are self-im­posed rather than or­ga­ni­za­tion­al. ↩︎