Confessions of a researchaholic

I use paper drafts to manage my research projects including everything: writing, experiments, results, and communications. Other than external storage (e.g. Dropbox or Google site) for data that does not need fine-grained revision control, I never need anything else, like meetings or emails. So far, this mechanism has worked superbly well, for my team projects as well as single authored papers.

This is one of those things that are better experienced than told, but here is a brief description of what it is like.

First of all, you need to have some idea what a research paper is like. All you see are finished and polished products. However, few, if any, research paper is written in one pass. They all went through some iterative evolution process. And here is mine.

I usually start drafting up the abstract the moment I have the idea. Do not worry if it is immature. It should be, as otherwise the idea cannot be new. Just write it down. Do not worry about grammar or anything cosmetic. Just make sure you can understand yourself later.

Gradually add more materials to that paper draft, including everything you see in published papers like introduction, literature survey, algorithm descriptions, math equations, experimental results, as well as those you might not see, such as thoughts, discussions, failed experiments, and ideas for future works. I usually keep a main part that eventually becomes published papers, as well as a blog part keeping daily progress and discussions.

I use svn/git for revision control, and Latex for document source. I structure the files so that I can simultaneously compile multiple versions from the same set of source (with different flags), such as final.pdf for final submission and draft.pdf that contains everything, including discussions among co-authors.

Because I put everything in writing, I never need to be reminded what has been done, what is yet to be done, and who is doing what. I can concurrently manage a double digit number of projects without losing my marbles. I also can collaborate with anyone anywhere without any other forms of meetings, because all discussions can be conducted through the paper draft. For example, if I see a mistake in an equation, I correct it, and put a note nearby explaining what I have done and why. The other guy, who wakes up in another time zone 8 hours later, can see exactly what has happened. Even for my UI papers, one guy can record a short video (or better, the editing history) for another guy to replay.

I have provided a didactic case, for file structure, research methodology, and the revision history.
One day I might be able to share a real project repository. Everything will naturally make sense when you start working with me (or one of my academic descendants) on real projects.

Because (borrowing a metaphor from finance) doing incremental research is like picking up nickels in front of a running steamroller.

The steamroller is driven by folks who have been publishing on a specific topic recently.

You see (incremental) opportunities to improve on their direction, like nickels on the floor.

And you feel like picking up those nickels. Why not? They are right over there.

However, the steamroller is also on its way to get to the nickels. (An obvious point, but people are surprisingly ignorant to it, especially when they are focusing on the nickels.)

So the questions you should ask are: are you fast enough to beat the steamroller to the nickels? (This depends on your competitive advantage.) And in case you just barely make it, are you tough enough to keep running in front of that steamroller at least for a while? (You have to do some tough comparisons with competing methods.)

Meanwhile, there are golden nuggets elsewhere. They are hidden and harder to find, but they are far from the steamroller and a whole bunch of other people trying to pick up nickels.

I will let you decide where you want to go. If you want to pick up nickels, fine, but I will just sit there watching you running in front of that steamroller. I prefer to be the first guy finding the golden nuggets, and when others notice me, I am already moving on to other places.

Students take courses in college, and do research in graduate school.

Taking courses and doing research share some common intellectual underpinnings. On the other hand, they are also very different. Different enough to make some college superstars completely fail in graduate schools.

When you take a course, everything is pre-designed and deterministic. You follow the instructions. And if you are smart and hardworking enough, you get good grades.

When you conduct a research project, very little is known a priori. The process is ambiguous and stochastic; your assumption may be wrong, your experiments may fail, sometimes you might not know what you are doing, and in the end your papers may get rejected. Being smart and hardworking can help. But if you have to follow clear instructions and you expect deterministic outcomes, you probably will not enjoy or be very good at research.

This is probably why, in my personal experience, the best research students might not have the best GPA, but they are usually creative, motivated, tough, and (most importantly) can get things done no matter what happens.
(Perfect GPA can be a reflection of conforming personality, which is not suitable for research. On the other hand, bad GPA is usually a reflection of intellectual or personality problems.)

The transition from undergrad to grad school is not always easy for everyone. My suggestion is to adopt the research mentality as early as possible. In particular, do not waste your time on courses; all you need is to pass the minimal requirements. If you have to, take courses that are research or project oriented. And start your research as early as possible, so that you will have more time to learn and adjust, and to accommodate all the potential failures and setbacks.

What

Stage 1: I will train your basic skills through a series of pet projects (e.g. building a mini software system or reproducing a paper). You are supposed to complete all these entirely on your own.

Stage 2: I will then directly take you to a real project aiming for the best venue of your field. I will work very closely with you on everything.

Why

I only have time for the best projects with the best students. The above 2-stage process is designed as a consequence.

I understand this is not an ordinary methodology. But experiences indicate that it is very effective in helping all students. In particular, the good ones can learn and publish as quickly as possible, while the not-so-good ones can realize as early as possible that they should explore other career options.

I disagree with the usual process of gradual bring-up by submitting to lower tier venues. This can spoil your mentality and motivation, as you will not work as hard as you should and grow as fast as you could. And even if you need non-top publications, it is faster to have those as failed attempts for top venues than direct submissions to lower venues.

For those of you who are not my internal or primary students, your primary adviser should have the final say about your study plan. Just come to me when you are ready for stage-2.

Unfortunately, your paper was rejected by your favorite venue.
Now what? The decision for the next step depends on review opinions and your preferences.

It is not hard science: reviews are stochastic and humans are irrational.
(See earlier posts about rejections, and how to deal with them.)

So, the first thing is to figure out how much you should trust the reviews and your gut feelings.
Sometimes the reviews are right; maybe your paper is really not suitable for that venue (e.g. off topic, not enough contribution) and it is better to choose another venue.
But sometimes they are not; I have at least one submission that was summarized as “not having enough contribution” and I just went ahead with resubmission and the paper got in.

It also depends on whether you will enjoy continuing the work, and whether you think you can make significant improvements. If both conditions hold, it is probably a good idea to ignore the reviews.

Finally, consider opportunity cost. Given the same amount of time and efforts, you can either (1) resubmit the paper to your most favorite venue or (2) submit it to a lower tier and start a new work. Which one (you think) will make you happier and more productive?

PS: see my collaboration policy.