A unique yet simple case for rogue AGI from Bengio. Claudestitution. Paper on eliciting truth without external ground truth from the residual streams. Really packed 80 page paper on conditioning models, I read it in one sitting it was awesome. Trick reflections on RLHF from Christiano. Preparing myself for success without dignity, non-zero chance that the solution to alignment is really good RLHF. Apollo Research (Conjecture fork) announcement with a research agenda I quite like. Languages Models Don’t Always Say What They Think, a good line of thinking to go down for alignment.

Tree of Thought paper, which was well done and nifty though I don’t think the premise will be fruitful. A pretty good shot at replacements for tokenizers with (mega)byte-level models. Thorough data paper with really awesome figures. Gwern comment on mode collapse with CAI

PaLM2’s tech report, which doesn’t contain the unfortunate leak that it was 3x undertrained. I’m kind of glad to see tokenizer work happening. Multilingual chain-of-thought. Evaluating bilingualism in PaLM. Loose thinking about data mixes. Thoughts of designing progress for AGI. DeepMind optimises their lil mov instructions. I reread the GLM paper in desperation of inspiration for multilingual. Another paper towards “data is all you need”. Inflection’s perfectly typeset tech “memo” on their LMs.

The Blackberry movie was the best thing I’ve ever seen, it so perfectly captured how startup founders go insane and start believing deranged things about their company. Also had an iconic line where when Jim asked Mike something about “i thought these were the best engineers in the world” and the quote from the movie was “I didn’t say they were the best engineers in the world, I said they were the best engineers in Canada.” Anyway, I combed through the Blackberry wikipedia page after that to keep my facts straight. Unrelated wikipedia pages: model organisms, the aquatic ape hypothesis and Copy Exactly! Design for more reasonable dataframes in python.

Investing in pandemic prevention. Learning about communal baths, from the archives of boot boyz. Reddit still comes through with this guy who commuted to UC Berkeley from LA. I attempted to read a few of Zizek’s Jokes, it was the worst fucking book. Short story about morality in the grand schemes: The ants and the grasshopper. Manifold Markets leaked the AI extinction letter, and purposely has it be this way and I like it? My leaving Cohere was also leaked by Manifold lol. Some fairly safe forecasts on what GPT 2030 will look like from Jacob Steindhart. Devon Zuegel on why all the good urbanism in the US is in the south. A really cute short story about dogs that made me cry. Agnes Callard’s profile in the New Yorker is so based, I love her. Do you want a living room that looks exactly like mine? Check out the guide to rationalist interior decorating.

Book: Rats

Upon moving to Sat Francisco, I read Cool Gray City of Love and immediately wished for an equivalent for New York (I lived there on and off). It would be hard to do as a book about New York, as the cuteness of CGCL came from the fact that most of the chapters weren’t about the Gold Rush or Summer of Love but about local history that could only be engaging to someone sitting on top of it. Maybe a book about a specific neighbourhood would cut it, but how could I choose a neighbourhood?

I found Rats (Robert Sullivan) at a Battery Park City housewarming in September where the book was not just recommended to me, but physically handed to me (which is the ultimate purpose of keeping one’s books collection). I lived in the Robert Moses part of town (Alphabet City) in May and begrudgingly thought about giving the Power Broker an attempt but then realised now was definitely the time to pick up Rats — only to realise I had left it in San Francisco. Thankfully I returned in June and finished the book in a week. It lived up to my CGCL standard.

Sullivan and I disagree, in that I rather like rats, but the book could've surely had me mistaken. He never quite explains how someone who wouldn’t even fancy a fancy rat would spend a year researching and watching them — even traveling to Chicago to meet an exterminator celebrity.

The book pops back in forth between the present of Sullivans rat adventures and history involving rats ranging from that Manhattan dentist who tried to shoot the rats on Rikers to the role of rats in plague over centuries. It covers the animal rights activists who tried to stop rat fights and the sanitation worker strike of NYC. Rats ends with a beautiful memo to the type it uses and has the greatest cover — look it up!