fun things with my smart mirror

When I moved into my new apart­ment, one of the things I was cu­ri­ous about was the Vercon Smart Intercom Mirror. It tout­ed a full smart home en­vi­ron­ment, ties to the in­ter­com, etc. I gen­er­al­ly never bought into smart home sys­tems (there is a joke here some­where about a soft­ware en­gi­neer and keep­ing an axe for his toast­er) but I de­cid­ed hey, I’m rent­ing the place, this could be cool to try out.

…only to find out that the apart­ment didn’t have any smart fea­tures to ac­tu­al­ly take ad­van­tage of any of the soft­ware on the mir­ror. Dis­ap­point­ing, though I was safe from a toaster uprising and saved money on hav­ing to buy an axe. It also had an in­cred­i­bly ugly, clunky, and awful to use, user in­ter­face! I hated it! Un­plugged you go! To the island of misfit toys!

I kept the smart mir­ror off per­ma­nent­ly for my first year as a ten­ant, but I hate hav­ing some­thing just sit­ting there—use­less. I start­ed to think of ways I could make this smart mir­ror use­ful. I have seen sev­er­al per­son­al projects of peo­ple cre­at­ing their own smart mir­rors over the years (usu­al­ly going all the way to the hard­ware side) and de­cid­ed to give it a run. LLMs have com­pressed time­lines on tasks that would have taken me a lot longer than they were worth in the past, and I had a week­end where I felt like being a her­mit. Why not throw Fable at it?

My needs are rel­a­tive­ly sim­ple. I want­ed to have my work and per­son­al cal­en­dars on the mir­ror so I could have an overview of my day, the abil­i­ty to look at the next two weeks, and to make the Smart Mir­ror ac­tu­al­ly func­tion as an in­ter­com sys­tem for call­ing peo­ple up so I didn’t have to have my phone so at­tached to me all day. I had a gen­er­al de­sign sys­tem in mind (grow­ing up with a heavy in­flu­ence from Japan­ese in­ter­net cul­ture and cy­ber­punk played a role in what I en­vi­sioned), so de­cid­ed to just throw it all to­geth­er from pro­to­type to end prod­uct, it­er­at­ing along the way.

The ex­pe­ri­ence made me re­al­ize that not only is Fable a great coder, it was fan­tas­tic at un­der­stand­ing my de­sign in­tent and mak­ing im­ple­men­ta­tion sug­ges­tions along the way!

The first thing I did was pull the mir­ror off the wall. I as­sumed I would need a cable into the thing to load my own soft­ware, so I flipped it around to reach the back and found a USB port, an Eth­er­net jack, and a power lead. Then I re­mem­bered that An­droid has had wireless debugging since ver­sion 11, and the de­fault OS was clear­ly based off of that. Seven taps on the build num­ber, a six-digit pair­ing code, and my lap­top was talk­ing to the mir­ror over the apart­ment Wi-Fi.

The fact that it was based on An­droid also made my life eas­i­er, as this ecosys­tem is rel­a­tive­ly easy to cus­tomize based on ages old ex­pe­ri­ence. Re-mount­ing the mir­ror on the wall, Fable and I began to in­ves­ti­gate what some best im­ple­men­ta­tions could be.

An hour of read-only pok­ing told us what we had: An­droid 11 on a Rockchip board, a 1920 by 1080 panel be­hind two-way glass, and no root. Two shell com­mands could swap the stock launch­er out, and two more could put it back if I moved out. Noth­ing need­ed flash­ing. Every­thing had to be re­versible, be­cause I rent the place.

Fable’s first plan was pret­ty dull. Its sug­ges­tion was serve a web page into a kiosk brows­er and call it a launch­er. I told it that was lame. I want­ed a na­tive app that owned the whole screen. Fable re­planned in about a minute and never ar­gued for the brows­er again. A good as­sis­tant will build the safe thing un­less you tell it what you want, which is where even hav­ing a touch of ex­pe­ri­ence can re­al­ly lead an LLM to a bet­ter prod­uct.

Three rounds of mock-ups later we had a left col­umn, which leaves the right half of the glass empty for the in­ter­com app when a call comes in. The type is IBM Plex Mono with M PLUS 1 Code for the Japan­ese, port­ed from my startpage. After 19:15 every­thing turns a deep red so the glass avoids being an eye-sore late at night. The greet­ing runs down the side in Japan­ese be­cause I sim­ply liked how it looked.

It also sug­gest­ed things I never thought of!

Pres­ence was the best of them. The mir­ror pings my iPhone over Wi-Fi once a minute, and once the phone has been silent for twen­ty-five min­utes the cal­en­dar folds away. The glass shows only the clock, the weath­er, and 留守 · USER AWAY. Re­turn, and it is back with­in a minute or two. A pri­va­cy fea­ture I re­al­ly may never use, but nice nonethe­less.

The guest lock and its un­lock were my ideas, and Fable or­ches­trat­ed a way to make it work. A long press hides the cal­en­dar on de­mand, and the only way out is a tap pat­tern. I’m not going to write it here, as this guest lock was made all be­cause my friends can not help but just tap every­where on this screen when they come over, leav­ing their grub­by lit­tle fin­ger­prints all over the place. My se­cret life and cal­en­dar are hid­den from my cu­ri­ous guests!

Two more came from Fable un­prompt­ed. When a day has more events than the col­umn can hold, the over­flow line reads +3 MORE, and a tap on it opens that day. After the first food de­liv­ery left the in­ter­com app’s call screen strand­ed over the mir­ror for an af­ter­noon, Fable pro­posed a watch­er. It wakes the dis­play when the lobby rings, and once no ring or call has been heard for a minute it brings the mir­ror back. I would have lived with the stuck screen for a week be­fore think­ing of that.

The build ran in phas­es over a week­end. Sat­ur­day morn­ing was re­con­nais­sance and the launch­er swap; by the af­ter­noon the glass showed the clock, the date, and the weath­er from Open-Meteo. Sun­day brought the cal­en­dars. A rule hides the work cal­en­dar from Fri­day af­ter­noon until Sun­day evening. Pres­ence and the four­teen-day view came after.

The lit­tle touch­es and fea­tures at the end were the fun part, each one a small piece that made the thing mine. They added sec­onds to the clock, a swipe through the cities I care about, a hol­i­day mode, and a dim step at thir­ty min­utes with a screen off at sixty. The strip of light be­hind the glass took longer. Fable de­com­piled the stock launch­er to find it, traced it to two PWM chan­nels, and now a triple tap on the right side of the glass turns it on. Vol­ume and the mi­cro­phone got their own rows on the con­fig panel once we con­firmed the mir­ror had both.

The first night the mir­ror slept prop­er­ly, but the app restart­ed it­self every fif­teen min­utes until morn­ing. An­droid’s Doze had cut its net­work, every cal­en­dar fetch failed, and a safe­guard we had writ­ten for a dif­fer­ent prob­lem kept re­launch­ing it. Fable read the 6 a.m. logs, ex­plained it, and fixed it in one pass. Light work, and work that I re­al­ly had no time for!

Bene­dict Evans wrote last week about AI, tools and transformation, and this project is re­al­ly the epit­o­me of his ar­gu­ment. Tools are cheap­er to build than they have ever been. But most peo­ple are not tool builders, and most peo­ple do not in­stinc­tive­ly look for a dif­fer­ent way to do any­thing. The hard part is “know­ing that you need a tool for this in the first place”, and then know­ing what it should do. I knew both. I had lived with the dead glass for a year and had the de­sign in my head. I also had enough An­droid in my his­to­ry to judge what Fable hand­ed back. Take any of the three away and the mir­ror is still un­plugged.

The mod­els are ex­tra­or­di­nary at the part I am worst at, the vol­ume, and no help at all with the part I care about, the want­i­ng. Evans’s open ques­tion is whether the trans­for­ma­tion comes from hand­ing every­one a model or from the new things a few peo­ple build with one. I do not know. For me the change is small­er and bet­ter: I can now take on a project like this when­ev­er the fancy hits me, and fin­ish it be­fore the fancy leaves.