Metasciencey Stuff

I tried to report scientific misconduct. How did it go? It's exactly as tragicomic as you expect. Hilariously blatant fraud, recalcitrant journal editors (though some did their job properly), and of course "A month after that, Southwest University cleared Dr. Zhang of all charges."

The clearest consequence of my actions has been that Zhang has gotten better at publishing. Every time I reported an irregularity with his data, his next article would not feature that irregularity.

Elisabeth Bik twitter thread on which journals are worst at responding to image duplication problems. "An old favorite, @Oncotarget. I flagged 161 of their papers (most reported a year ago), but only 2 retractions. They have been sending annoyed responses though, that I should look at other journals, and what my conflicts of interests are."

Gelman on me: My thoughts on “What’s Wrong with Social Science and How to Fix It: Reflections After Reading 2578 Papers”

I think that the vast majority of researchers think that they have strong theories, and they think their theories are “true” (whatever that means). It’s tricky, though: they think their theories are not only true but commonsensical (see item 2 of Rolf Zwaan’s satirical research guidelines), but at the same time they find themselves pleasantly stunned by the specifics of what they find. Yes, they do a lot of conscious p-hacking, but this does not seem like cheating to them. Rather, these researchers have the attitude that research methods and statistics are a bunch of hoops to jump through, some paperwork along the lines of the text formatting you need for a grant submission or the forms you need to submit to the IRB. From these researchers’ standpoint, they already know the truth; they just need to do the experiment to prove it. They’re acting in good faith on their terms but not on our terms.

On the Newton Hypothesis: is science done by a small elite? Plus good HN discussion.

Groundhog is a package that fixes one of the (many) great evils of R: the lack of reproducible scripts. It does so by getting old package versions from the time the script was written.

The influence of hidden researcher decisions in applied microeconomics: investigating the consequences of "researcher degrees of freedom" by doing seven independent replications of two studies. One of them ended up all over the place, the other relatively uniform. "The standard deviation of estimates across replications was 3-4 times the typical reported standard error." Twitter thread.

Ulrich Schimmack argues that psychology has improved in the past decade: Replicability Rankings 2010-2020. "...about 20% of [120 psychology journals] have responded to the crisis of confidence by publishing studies with higher power that are more likely to replicate." Plus some back and forth on the IAT.

Reactivation-Dependent Amnesia for Contextual Fear Memories: Evidence for Publication Bias. Look at these funnel plots:

 

Corona

Exploring the Supply Chain of the Pfizer/BioNTech and Moderna COVID-19 vaccines. An exhaustive look into how vaccines are produced, distributed, and used. "I’ll start with the bad news: Nobody will be making an mRNA vaccine in their garage any time soon."

Lemoine contra lockdowns. Argues that estimates of the effect of lockdowns are confounded by changes in behavior due to a rise in cases, and that the small effect of lockdowns does not pass a cost-benefit analysis. The studies are indeed weak but I'm not 100% convinced.

Youyang Gu of covid19-projections.com: One Year Later. "The most frequent question I receive is how I was able to do what I did with no background in infectious disease modeling. [...] You don’t need decades of experience to be able to think critically and adapt to new information."

 

Forecasting & Experts

There's been a lot of interesting writing on credentials, expertise, and social epistemology lately. Here's a couple of good pieces:

Scott Alexander: WebMD, And The Tragedy Of Legible Expertise

When the Director of the CDC asserts an opinion, she has to optimize for two things - being right, and keeping power. If she doesn't optimize for the second, she gets replaced as CDC Director by someone who does. That means she's trying to solve a harder problem than Zvi is, and it makes sense that sometimes, despite having more resources than Zvi, she does worse at it.

Then Zvi's response to Scott: Why I Am Not in Charge.

The idea of Zeroing Out is that you can do better by taking a market price or model output as a baseline, then taking into account a bias or missing consideration. Thus, you can be much worse at creating the right answer from scratch, and yet do better than the consensus answer. [...] The true beauty of Zeroing Out is that I can take all the analysis of the CDC as given, find one place where they are biased or making a mistake or missing a piece of information, and then reliably do better than the CDC by otherwise taking the CDC’s answers.

Being an expert in biology is one thing. Being an expert in leadership or politics, or the politics of biologists, is another.

When the CDC issues its guidelines it seems very clear that those guidelines are designed to let those involved tell stories about being safe and following proper procedure, and they have at best vague bearing on whether things are actually safe. They’re certainly not even attempts to create a correct physical model of anything. They’re giving people a chance to avoid blame and to show adherence to power, while themselves avoiding blame and showing their adherence to power.

Vitalik on betting in the election prediction markets, great stuff as usual. "The amazing promises of what prediction markets could bring if they work mathematically optimally will, of course, continue to collide with the limits of human reality, and hopefully, over time, we will get a much clearer view of exactly where this new social technology can provide the most value."

metaforecast.org allows you to search for forecasts from Metaculus, PolyMarket, and a bunch of other sources through a single convenient interface.

What makes good forecasters good? The Bias, Information, Noise model.

Predictable Financial Crises:

Using historical data on post-war financial crises around the world, we show that crises are substantially predictable. The combination of rapid credit and asset price growth over the prior three years, whether in the nonfinancial business or the household sector, is associated with about a 40% probability of entering a financial crisis within the next three years.

 

AI

An excellent new science fiction short from Sam Hughes: Lena. (Inspired by Age of Em?)

OpenAI's DALL·E creates images from text.

OpenAI: Multimodal Neurons in Artificial Neural Networks. Lots of cool stuff, including the Stroop effect in ML models.

The National Security Commission on Artificial Intelligence (chaired by Eric Schmidt) has produced its "Final Report". 756 pages long, do read the executive summary and skim though. Strikes me as rather fantastically idealistic.

 

The Rest

A whirlwind tour of Ethereum finance. "Why should I care about this? For one thing, it's the coolest, most cypherpunk thing going." Plus Wrecking sandwich traders for fun and profit: "Being a DeFi degenerate is a lot of fun, but be careful in your trading - because this game is highly adversarial and we play for keeps."

One man's experience with early retirement. Keeping up with the Joneses is an underrated aspect of psychology and economic decision-making. Selection happens on relative fitness—so much of human behavior is downstream from that.

A chart from The Recent Large Reduction in Space Launch Cost. Note the logarithmic y axis. Also check out The New Space Race: What Took So Long, for an explainer of why this wasn't possible a few decades ago.

Tariana is a language spoken by about 100 people in the Amazon rainforest, and it has evidentiality baked into it, with different suffixes depending on whether you saw something, inferred it, or learned it from someone else. Here's a paper that goes into more detail. And it's not the only language with evidentiality features!

Joe Carlsmith on objecting to moral claims because their implications are too demanding. "In general, I think that the abstract arguments for very “demanding” forms of moral behavior are very strong (even for non-consequentialists). But I think that the fuzzier, harder-to-articulate heuristics, evidence, and forms of wisdom encoded in our hesitations about such behavior are important sources of information as well." Plus Alienation and meta-ethics (or: is it possible you should maximize helium?).

Can You Ever Be Too Smart for Your Own Good? Comparing Linear and Nonlinear Effects of Cognitive Ability on Life Outcomes. "We found no support for any downside to higher ability and no evidence for a threshold beyond which greater scores cease to be beneficial. Thus, greater cognitive ability is generally advantageous—and virtually never detrimental."

bmgarfinkel reviews Renaissance polymath Gerolamo Cardano's memoir The Book of My Life (1575). "Cardano kicks off the chapter “Things of Worth Which I Have Achieved in Various Studies” by reminding the reader that “there is practically no new idea which one may bring forward.”"

A mouse embryo has been grown in an artificial womb—humans could be next. "Hanna says to make such experiments more acceptable, human embryos could be altered to limit their potential to develop fully. One possibility would be to install genetic mutations in a calcium channel so as to prevent the heart from ever beating." (What a sentence!)

A Group of Orca Outcasts Is Now Dominating an Entire Sea. "Granny and her kin are considered part of the same species as transient killer whales, Orcinus orca. But residents and transients have lived separate lives for at least a quarter-million years. They generally do their best to avoid each other, and they don’t even speak the same language—the patterns and sounds they use to communicate are completely different. Over time, each type has established cultural traditions that are passed from generation to generation."

On the SawStop and the table saw industry. "The CPSC says table saws result in about $4B in damage annually. The market for table saws is about $200-400M. This is a product that does almost 10x in damage as the market size."

There were a few hundred pieces written about the NYT/Scott thing, most of them incredibly tiresome. @yashkaf produced the best take by just writing about the features of the mainstream narrative and how one ought to interact with it. "There are three main goals of engaging in argument: for truth, for power, for lulz."

Apparently there are bacteria that do photosynthesis from the infrared glow of hydrothermal vents deep in the ocean.

A fan-made visual effects demo for the unfilmed Shane Carruth script A Topiary.

And here is Wall of Voodoo's cover of Ring of Fire.

What I've Been Reading

  • A Masterpiece to Those Who Contemplate the Wonders of Cities and the Marvels of Travelling, AKA The Travels of Ibn Battutah by Ibn Battutah. A few months back I read John Mandeville's fanciful travelogue; this is from the same time (mid 14th century), but completely different: grounded, realistic, and focused on the world of Islam. Ibn Battutah spent nearly 30 years traveling from Morocco to China and then back again, and he left us a wonderful book filled with adventure and ethnographic observations. Full review.
  • The Peregrine by J. A. Baker. 10 years of obsessive peregrine-watching distilled into 200 pure, intense, astonishing pages. An incredibly rich dish that you can only eat so much of before needing to take a break. Reflects and contains nature both in its form and content. I picked it up it because Werner Herzog recommended it and it is indeed very Herzogian. There's no green idealism here: the endless cycle of killing which sustains the peregrine is presented unapologetically. "Beauty is vapour from the pit of death."
  • The Persecution and Assassination of Jean-Paul Marat as Performed by the Inmates of the Asylum of Charenton Under the Direction of the Marquis de Sade by Peter Weiss. Based on historical fact: Sade really did direct plays while held at the asylum of Charenton, and they were quite popular. This is one of those plays that's probably better seen than read, as it's filled with song and music and elaborate stage instructions. The movie by Peter Brook is good. In the afterword Weiss claims he's trying to reclaim Marat from the "bourgeois historians" but in those terms it's a complete failure. His Sade on the other hand evokes a delightful Lovecraftian cosmic nihilism. I also enjoyed the double historical irony, as the play is set at the height of Napoleon's reign ("For today we live in far different times / We have no oppressors no violent crimes / and although we're at war anyone can see / it can only end in victory").
  • Hopscotch by Julio Cortazar. Re-read, not quite as good as I remembered it. An experimental novel about a group of bohemians in Paris. The first half is excellent, the gimmick of the "expendable chapters" works well, and some moments are absolute perfection (in one chapter the main character writes a letter between the lines of a book, and we get to read both the letter and the book on alternate lines; then there's that unforgettable scene in the apartment). But the novel kind of disintegrates in the second half, when Oliveira gets back to Argentina.
  • Cognitive Capitalism: Human Capital and the Wellbeing of Nations by Heiner Rindermann. Skimmed. Mostly a retread of his papers. Final chapter with comparisons to Mokyr, Clark, Acemoglu, Diamond, Hanushek, etc. was interesting.
  • The Evolutionary Limits of Liberalism by Filipe Nobre Faria. A rather strange book, in which the focus is arguing against individualism in general and public choice theory in particular, based on their disregard for group selectionism. The idea is that groups of selfish rational agents lose to altruistic groups in the long run. A very abstract call for a morality that is pro-group cohesion, pro-altruism, anti-markets. Slightly longer review.
  • Molloy by Samuel Beckett. Beckett wrote it originally in French, "because in French it’s easier to write without style", then translated it into English. The resulting language fits perfectly into this captivating experimental enigma of a novel. Don't expect much in terms of plot, scenes, dialogue, or most of the usual trappings of the novel. Long stretches don't even bother with paragraphs. Decay, aging, disability, confusion, vagrancy, and all sorts of strange obsessions. Often funny, somehow. A double quest that may or may not be a parallel, a loop, or something completely different.
  • Malone Dies by Samuel Beckett. The second entry in the trilogy. Aging, misery, the process of dying. Molloy was already sparse, but Malone Dies goes even further, doing away with the humor and any semblance of linearity. Abstract and amorphous, like a novel dissolved in hydrochloric acid. Didn't capture me in the same way Molloy did.
  • Use of Weapons and Excession by Iain M. Banks. Both re-reads (first-time listens though). The audiobooks narrated by Peter Kenny are fantastic, he does different voices for every character and his style is perfect for space opera. Use of Weapons has two great characters at its core, but gets lost in a bazillion side-plots and flash-whatevers. Excession remains the best Culture book because of how ship-centric it is. I had completely forgotten the pointless subplot about the ditzy socialite, but luckily it doesn't ruin the rest of the book.
  • Revelation Space by Alastair Reynolds. Re-read. The audiobook is nowhere near as good as the Culture ones, but this is top-tier space opera. The adventures of an egomaniacal xenoarchaeologist and the insane posthuman crew of an ancient spaceship, as they get into galactic-scale trouble. Really nails the scale. Not 100% hard SF, but the fantastical elements are not too outlandish. There's a new novel in the Revelation Space series coming out soon.
  • Childhood's End by Arthur C. Clarke. An incredibly silly relic from the cold war. Silly aliens, silly utopias, and a very silly resolution that literally involves a séance and a ouija board! Very pessimistic about the future of humanity. Tries to go for that epic space-is-giant, humans-are-tiny-and-irrelevant feeling but it really didn't work for me.
  • Undaunted Courage: Meriwether Lewis, Thomas Jefferson, and the Opening of the American West by Stephen E. Ambrose. Ambrose has come under attack for his rather cavalier attitude toward plagiarism and telling the truth, but he's undeniably a great storyteller. And it's pop history after all, not an academic tome. Despite some distracting jingoism and fanboyism he does a great job of documenting Lewis & Clark's incredible adventure across the Rockies to the Pacific. It's almost unbelievable how peaceful their interactions with the Native Americans were, but you can also see the seeds of future conflict right away. Jefferson plays a big role and he was an incredibly cool figure. Not sure what to make of the pathetic and tragic ending to Lewis's life...