Machine Learning

1. Large Language Models are Zero-Shot Reasoners: "Simply adding “Let’s think step by step” before each answer increases the accuracy on MultiArith from 17.7% to 78.7% and GSM8K from 10.4% to 40.7% with GPT-3." Here's how different prompts compare:

2. DALL·E 2 is pretty crazy. Tons of good threads on twitter featuring its work, here's one of my favorites.

3. Gwern comments on GPT-3's 2nd Anniversary

A psychologist thrown back in time to 2012 is a one-eyed man in the kingdom of the blind, with no advantage, only cursed by the knowledge of the falsity of all the fads and fashions he is surrounded by; a DL researcher, on the other hand, is Prometheus bringing down fire.

4. Speaking of AI and Prometheus...

5. A model trained on /pol/ data successfully(?) sends out thousands of shitposts.

6. Yarvin contra AI x-risk fears. I am not convinced.


7. Very good and important: Beware boasting about non-existent forecasting track records.

8. Future Fed Chair Basil Halperin on prediction markets and monetary policy.

9. Nuño Sempere released 3 short and sweet papers on designing prediction scoring rules. Also subscribe to his excellent forecasting newsletter if you haven't already.


10. The New Science report on the NIH. Enormous but very much worth your time.

11. On that baby brainwave study and more general issues around that sort of research.

12. In the Guardian: The big idea: should we get rid of the scientific paper?

13. Ideological biases in research evaluations? The case of research on majority—minority relations

Within this field, social contact and conflict theories emphasize different aspects of majority—minority relations, where the former has a left-liberal leaning in its assumptions and implications. We randomized the conclusion of the research they evaluated so that the research supported one of the two perspectives. Although the research designs are the same, those receiving the social contact conclusion evaluate the quality and relevance of the design more favorably. We do not find similar differences in evaluations of a study on a nonpoliticized topic.

Note the effect is quite small though.

Economic History

14. On the role of millet, rice, and timing of agriculture in Chinese state formation.

Book Reviews

15. The SSC book review contest is pretty strong this year as well. My favorite thus far: The Dawn Of Everything

A “Gossip Trap” is when your whole world doesn’t exceed Dunbar’s number and to organize your society you are forced to discuss mostly people. It is Mean Girls (and mean boys), but forever. And yes, gossip can act as a leveling mechanism and social power has a bunch of positives—it’s the stuff of life, really. But it’s a terrible way to organize society. So perhaps we leveled ourselves into the ground for 90,000 years.

16. Judge Woolsey on Ulysses: ""[i]n respect of the recurrent emergence of the theme of sex in the minds of [Joyce's] characters, it must always be remembered that his locale was Celtic and his season Spring. [...] [W]hilst in many places the effect of Ulysses on the reader undoubtedly is somewhat emetic, nowhere does it tend to be an aphrodisiac."

17. SMTM on Disco Elysium.

18. Devis Kedrosky reviews Koyama & Rubin's How the World Became Rich.

The Rest

19. Latest news from the Good, Actually dpt: incarceration cuts mortality by half.

20. Matt Lakeman continues his travel blogs, this time he reports from Ukraine.

21. What I learned gathering thousands of nootropic ratings.

22. Daniël Lakens has released a free ebook on improving your statistical inferences.

23. New evidence on the genetic history of Ashkenazi Jews: "our results suggest that the AJ founder event and the acquisition of the main sources of ancestry pre-dated the 14th century and highlight late medieval genetic heterogeneity no longer present in modern AJ."

24. Mechanical Watch (lots of crazy shit on this blog)

25. On foreign aid and ethnic conflict.

26. Eigenrobot gives advice to academic refugees.

Academia is characterized by well-trodden problems, hashed over for decades, and negligible novel data for resolving them. Industry is by comparison a mass of green field areas of inquiry with large budgets, minimal bureaucracy, and ample data.


27. And here's Masayoshi Takanaka's The Rainbow Goblins.

What I've Been Reading

  • Democracy in America, by Alexis de Tocqueville. Lives up to its reputation. Fascinating observations on law, politics, psychology, sociology, America's Westward expansion, and more. Prefigures Timur Kuran in many ways. Incredibly prescient. Interesting both in terms of what has stayed the same since it was written, but also for a look at all that has changed. "The French lawyer is simply a man extensively acquainted with the statutes of his country; but the English or American lawyer resembles the hierophants of Egypt, for like them he is the sole interpreter of an occult science."

  • The Memoirs of the Baron de Tott, on the Turks and the Tartars, by the Baron de Tott. Found through Braudel. Written just a few decades before Democracy in America, the Baron de Tott went East instead of West. And instead of seeing the future, he saw the past. Roughly at the time the American revolution was happening, the same time when Johnson and Boswell were drinking too much claret at the Mitre, de Tott was joining the Crimean Tatars on a slave raid into Southern Russia. Most fascinating for its observations of Ottoman society, and the role de Tott played in the Russo-Turkish war of '68-'74. Somewhat niche and obviously nowhere near as insightful as Democracy in America, but definitely worth a read if this is your kind of thing. What causes the fall of empires? Culture, Tott says: all decay ultimately comes from within.

  • Collapse of Complex Societies, by Joseph Tainter. Tainter's theory mostly comes down to decreasing marginal returns to additional societal complexity, which eventually leads to collapse. Parts of it are highly reminiscent of Chaisson's Energy-Rate Density paper (which everyone should read), but much more limited in scope. He's too focused on explaining everything with a single theory, leaving little room for contingency in history. He ignores the aspect of time: just because a system works well for 10 years does not mean it can work for 1000. And he treats rulers as being virtually unconstrained in their policy choices.

    The examples he marshals in support of this theory are not particularly convincing, and (at least in some cases like the Western Roman Empire), the Mancur Olson view which focuses on public choice issues (which Tainter pretty much dismisses out of hand) seems like a vastly better fit to me. Especially when it comes to contemporary society, the examples Tainter brings up seem like a slam dunk in favor of Olson and against Tainter! Take education for example: is it really plausible that the ballooning costs and declining efficiency of educational spending over the past few decades is due to increased complexity? Of course not, it's clearly an issue of special interest groups with socially misaligned incentives. Tainter misses it because he never actually dives into the details of exactly how increased complexity is supposed to be working to produce all these effects.

  • The Machiavellians: Defenders of Freedom, by James Burnham. On Machiavelli and some of his successors: Mosca, Sorel, Michels, Pareto. Published in 1943 and it shows. Strong on the general ideas about the objective treatment of power and politics, divorced from sentimentality and moralizing. Pretty weak on the specifics. I was expecting something deeper based on its reputation. A bit dull overall.

  • Sartor Resartus, by Thomas Carlyle. Borges mentions that it inspired him to read German philosophy and that's how it ended up on my list. What can I even say about this crazy book? Carlyle invents a fictional German philosopher, who has written a treatise on clothing, and then also invents a fictional English editor who tries to explain the German philosopher's work, which turns out to be a philosophy of everything. Layer upon layer of irony and postmodern misdirection, and that outrageous Carlylean 19th century style to top it off. Heavily influenced by Tristram Shandy. Surprisingly influential (especially in America), though I'm really not sure how seriously one is meant to take the ideas presented within.

  • Liftoff: Elon Musk and the Desperate Early Days That Launched SpaceX, by Eric Berger. Fast-paced and exciting, mostly based on insider interviews, Liftoff gives a good idea of what the crazy early years were like at SpaceX. Once they start launching the Falcon 9 it skips over a decade in a few paragraphs, which kind of sucks. If you were wondering exactly what factors made SpaceX succeed where everyone else has failed, you will probably come away from the book disappointed. Still, recommended.

  • Apollo: The Race to the Moon, by Charles Murray (yes, that Charles Murray). One of the better Apollo books, this one is focused mostly on the bureaucratic aspects with a few glimpses into engineering as well. At 500 pages it still feels far too short, as some major events and personalities are given very little space. Overall very strong, and it's truly astonishing how there was almost nothing at all in terms of the space program in 1960, how young everyone was, how nobody really knew what they were doing, etc. For some reason it seems to be out of print.

  • The Book That Changed Europe: Picart & Bernard's Religious Ceremonies of the World, by Lynn Hunt. The story of the publication of the titular book, and a look at the religious environment of the 18th century. Freethinking Protestant refugees congregate in cosmopolitan Amsterdam and make waves through their printing presses. Fascinating subject, terrible execution. Unorganized, repetitive, badly written, and filled with pointless digressions. There's an irrelevant digression in the very first paragraph of the book! Maybe if a competent editor had gone to town on it...Also, I think the authors wildly overrate the book's ultimate importance.

  • A Distant Mirror: The Calamitous 14th Century, by Barbara Tuchman. Audiobook. A look at 14th century Europe, mostly as it was seen from the perspective of the French nobleman Enguerrand VII de Coucy. Mainly based on the Chronicles of Froissart. Plague, the 100 years' war, religious fanaticism, popes and antipopes, peasant revolts, crusades, etc. Very entertaining, but it sacrifices quite a lot of rigor to get there. Too many blatantly false statements from the 14th century are taken at face value. And there's more than a bit of Monty Python about this: at points, I thought I discerned the distant—but unmistakable—beat of coconuts in the background of the audiobook. This sentence gives you the vibe: "A decision was perforce taken to march straight through the dark, fobidding forest of the Ardennes, where, Froissart remarks with awed inaccuracy, "no traveler had ever before passed.""

  • Project Hail Mary, by Andy Weir. Audiobook. It's the same schtick as The Martian all over again, but with more plotholes and a more impressive setting. Pleasant scifi entertaintment for the gym.