Confessions of a researchaholic

Feeling down from some recent rejections? I hope this post will make you more positive. The gist: never, ever, quit.

Assume you are throwing N loaded dices, each with a probability p for coming up head.

Now, if p is greater than 0 and smaller than 1, there is always a chance that the N dices will come up with all heads or all tails. And the smaller the N value, the more likely for such extreme cases to happen.

This is all pure chance. But unfortunately, human brains have difficulty accepting randomness, and always want to impose determinism, e.g. patterns or rules or causalities.
For example, if you are a scholar submitting N papers to a conference, you will likely consider yourself to be very good/bad (or the paper committee has treated you very well/badly) if all N submissions are accepted/rejected.

This human fallacy is brilliantly illustrated by Nassim Nicholas Taleb’s book “Fooled by Randomness”.

However, even without reading that book, I can recommend a very simple remedy: law of large numbers. This is a well-known mathematical theorem, which says that the expected value of a random variable can be more accurately predicted by averaging a larger number of samples.

So, for example, to measure your intrinsic paper acceptance rate towards a specific conference, you can take the total number of acceptances divided by the total number of submissions. This will be a much more meaningful measure than your acceptance rate for a single year, especially if you have a sufficient number of submissions across multiple years.

For example, the plot below shows my cumulative acceptance rate for SIGGRAPH, the top venue for computer graphics and interactive techniques. As you can see, the rate seems to be gradually converging to a certain value, around 0.34. This is much more stable measure than my yearly rate, which can be anywhere between 0 and 1.

Now, if you are new to a field, your rate will have a higher variance, just like the initial portions of mine. I was lucky that I had a good start which boosted my confidence. (Initial condition is actually very important and has been found to greatly influence the performance of many careers, e.g. hockey players. Note to myself: dig out that book/article. I guess it should be Geoff Colvin’s “Talent is Overrated” or Malcolm Gladwell’s “Outliers”.) However, if you happen have an unlucky start, do not give up too early; hang on for a while, so that you can have a chance to see your *intrinsic performance*.
As you can see, my intrinsic performance did not really show up until about a decade doing SIGGRAPH.

(With all these rational arguments, I have to confess that it still hurts to get rejected!)

Some notes about the graph: (1) I plot SIGGRAPH at integer years and SIGGRAPH Asia at integer + 0.5 years, (2) missing data points are for years which I did not submit anything (2004 and 2005 while in NVIDIA and 2011.5 when I have nothing to submit for SIGGRAPH Asia 2011), (3) a more accurate measure would be “moving average” (with exponential decay of past values) but I probably need another 10 years to warrant this, (4) I really want to improve my intrinsic rate to at least 50 percent!, (5) I guess the ultimate test is to have multiple disjoint committees + reviewers, all with similar qualities, to evaluate the same batch of submissions, and see if they will accept similar sets of papers.