Five years of spaced repetition flashcards

Attention conservation notice: Excessive navel-gazing.

First, some shameless Anki nerd bragging:

Screen Shot 2016-02-05 at 12.16.42 PM

Screen Shot 2016-02-05 at 12.16.49 PM

Screen Shot 2016-02-05 at 12.16.59 PM

Screen Shot 2016-02-05 at 12.17.06 PM

Screen Shot 2016-02-05 at 12.17.22 PM

Although I’ve been doing spaced repetition via Anki for 5 years now, I actually wanted to start doing SR flashcards about 8 years ago — approximately speaking, since 5/6/08, when I read this Wired article about Piotr Wozniak. Wozniak truly devoted himself to spaced repetition. He just generally seemed like an interesting person, his technique seemed like a good idea, and I wanted to do it [1].

For the next 2.5 to 3 years I had a pretty constant low-level of guilt/anxiety about how I should be doing SR flashcards. This anxiety spiked whenever I forgot something I had previously learned or clicked on a purple Wikipedia link. Of course, after a year or so, I thought it was too late and that I was a lost cause with respect to spaced repetition.

Coincidentally, I now have a flashcard for how to solve this exact problem. It’s called “future weaponry” — instead of thinking about what you could have done with a particular tool/knowledge/ability in the past, choose to think about what you will be able to do with it in the future.

Spaced repetition flashcards weren’t really feasible, though, until I got a smartphone. Say what you want about smartphones with respect to productivity overall, but the ability to do SR flashcards on them while walking and waiting around is insanely crucial and underrated. And this basically requires easy syncing via an internet connection.

Looking back at my cards from when I started, they’re pretty terrible. For example, I used tons of cloze deletion cards, a lazy and less effective way of making flashcards, rather than thinking about what knowledge I really wanted to retain and framing it in question form. I was also obsessed with memorizing math equations even though these have been pretty much completely useless.

That said, by far the most important thing for me back then was to maintain motivation, and I’ve been able to do that so far. You can find some of my spaced repetition flashcards cards for statistics, R programming, and other topics here.

The spike in flashcards that you see the middle of this time period was due to studying during the first two years of med school, for which I made and did a lot of flashcards. I think studying this way actually made me less good at any individual exam, since other cramming methods are more effective, but my hope is that it will help me to retain the knowledge better in the long run.

During this time, my friends and I were also studying for a big med school exam, called Step 1, and which we referred to as “D-Day.” Watching the documentary Somm, that is definitely what our lives were like, especially the us-against-the-world feeling. Spaced repetition flashcards were a big part of it. On any given day we usually had 400+ flashcards to do, which we called “slaying the beast.” We all ultimately did passed and did pretty well on Step 1, although looking back it was more about the journey.

Since starting full-time on my PhD program a year and a half ago, I’ve been using SR flashcards for two purposes:

  1. Learning programming languages, both the syntax and the concepts. The syntax has probably been more useful, but learning vocabulary related to the concepts has also been especially high yield for knowing what to search for.
  2. Learning about my research topics, e.g., Alzheimer’s and genomics. The jury is still out on whether this is a good use of time, but in general, I would say not to sleep on the potential for it if you’re a researcher. A lot of people like to read, but there’s a lot of value to be gained from systematically reflecting upon what you’ve read, especially if you have an imperfect memory like me.

I owe a lot to Damien Elmes, who wrote the open-source, free Anki software. I also owe a lot to Gwern, who wrote about spaced repetition extensively, and who made thousands of his Mnemosyne cards available to anyone to download for free. I downloaded these one day on a whim, converted them to Anki, and that was what really made me think of having my own cards as being a realistic, practical option. Thanks, Damien and Gwern.


[1]: The other thing I remember from that article, 8 years later: I often think about how he tries to minimize how often he drove in cars.

Nine paradoxes with a statistical theme

Questions

  1. A drill sergeant always yells at one of his trainees when she messes up. The drill sergeant notices that after he yells at her, her performance improves. Later it turns out that the trainee is deaf, blind, and has no other way of actually noticing that drill sergeant is yelling at her. Ignoring the effect of practice, why might the trainee’s performance have improved anyway?
  2. You have 100 pounds of Martian potatoes, which are 99 percent water by weight. You let them dehydrate until they’re 98 percent water by weight. How much do they weigh now and why?
  3. Imagine that your parents had rolled a six-sided die to decide how many children to have. What did they most likely roll and why?
  4. You have access to planes that have returned from military missions and the distribution of the bullet “wounds” on the planes. Which areas should you recommend to have extra armor?
  5. Why would few people choose to play in a lottery with a small but actual probability of success with an infinite monetary expected value?
  6. Do most people have have the same, more, or fewer friends than their friends have on average and why? 
  7. Hypothetically, say that 80% of people dream in color, and 68% of sexual partners have the same (concordant) coloring of their dreams. If you dream in color, what’s the probability that your partner will too?
  8. Are we biased to think that cars in the lanes next to us are going faster or slower than they really are and why? 
  9. Why is the expression “the smallest positive integer not nameable in under eleven words” paradoxical?

Answers

  1. regression to the mean — the screwup is likely a random deviation below the trainee’s average, which will tend to improve on the subsequent iteration just due to random chance, regardless of any action by the drill sergeant (more here
  2. 50 pounds, since the percentage of non-water by weight has doubled, so the overall weight must have halved (more here
  3. they most likely rolled a six, because there’s a higher chance of you existing to observe the event in that case (more here
  4. the areas with no damage, because of selection effects — planes that fell likely suffered an attack in a place that was untouched on those that survived (more here
  5. because the marginal utility of money is diminishing (more here)
  6. fewer, because sampling bias suggests that people with greater numbers of friends have an increased likelihood of being observed among one’s own friends (more here
  7. 80%. Since basic probability theory tells us that 0.8 * 0.8 + 0.2 * 0.2 = 0.68, so we know that the probability of dreaming is color is independent of that of your sexual partner. Therefore, the probability that your partner dreams is color is independent of yours and is simply the base rate. Some people think 68%, perhaps because they are getting wrapped up in the causal story. (more here)
  8. we are biased to think they are going faster, likely because because more time is generally spent being overtaken by other vehicles than is spent in overtaking them (more here)
  9. there are finitely many words, so there are finitely many numbers that can be defined in under eleven words, so there must be such an integer, but since this expression itself is under eleven words, there cannot be any such integer (more here; resolved by assigning priority to the naming process either within or outside of the expression) 
Screen Shot 2015-12-29 at 7.50.09 PM

these are totally Martian potatoes; https://www.flickr.com/photos/53255320@N07/6762195603

Reasons to Avoid Spaced Repetition Like a Culture of Y. Pestis

Broadly, I’m an evangelist, but a) spaced repetition truly is not the ideal tool for all purposes and b) it is a useful exercise to consider some of the strongest counters to your positions. Below you will find examples of things that you might be willing to read about, but would not want to actually invest time in learning via spaced repetition.

1) Facts in your field change rapidly relative to your available study time. If this is the case, you might well experience interference between the data in your outdated knowledge base and the new facts you want to replace them with. It still might be worth it to pay this interference cost to gain the value of deep memorization for the lifespan of the fact, but it becomes much less likely as the turnover rate increases, especially given high start-up costs in making the flashcards. Real-world examples: syntax for Python packages, lists of political figures, stock prices, material to cram for a near-term exam, etc.

2) You are considering learning facts that you don’t assign a high probability of truth to. That is, things that you read or hear about and just don’t believe; this is related to the fear of interference above if you are correct to disbelieve and you expect to learn the truth later. Sometimes learning these anyway can be worth it as a way to master the arguments of your ideological adversaries, but in general it is not. Real-world examples: practical advice that you consider dubious, conclusions of scientific articles whose methods you distrust, etc.

3) You are considering learning facts that you would much rather learn through a discovery process. There are things that we do in life for the sake of experience rather than learning, and all of those can fall under this umbrella, but this category also includes things that you might want to learn about in a less formal manner. For example, consider reading an engaging narrative about cultural differences in approaches to airplane safety such as the one presented in Gladwell’s Outliers. You could consistently a) not want to learn the relevant facts via spaced repetition, but b) enjoying learning the facts via his engaging narrative. In general, the limiting factors here are start-up costs and lack of fun. Real-world examples: it will differ for everybody depending upon baseline knowledge and inclinations, but things I have heard of include food, TV shows (e.g., plots, key characters), and information about places (e.g., monuments & local histories).