Confessions of a researchaholic

September 29, 2012

How to get source code and data

Filed under: Real — liyiwei @ 11:00 pm
Tags: ,

In the ideal world, every CS research paper should be published with the relevant source code and data. This is good for both the original authors (you and your paper are more likely to become popular) and everyone else (easier to reproduce and compare against a previous method).

If you are a member of the latter, look for public domain code and data first.
You can download these without having the authors knowing.
(If you worry about IP tracking, use some VPN tools to hide your trail.)

If you cannot find any similar or related code, email the authors telling them how much you admire their paper and would like to try experiment with their code and/or data.
There is no need to mention anything about your research, or even the fact you are doing a research project.
When you submit your paper, it is usually not possible for the authors to figure out you are even using their code, especially for venues with double blind review processes.

If your paper is accepted for publication, make sure you acknowledge all authors who kindly share the code or data (either publicly or privately with you only), and cite their papers as a good gesture.

If none of the above route works, you might have to reproduce the code.
Do this only as a last resort, as it not only takes time and resources but also poses the question of whether your reproduction is correct.

September 21, 2012

Want to Publish More? Then Train Like an Athlete

Filed under: Real — liyiwei @ 11:49 pm
Tags: ,

I received this through tomorrow’s professor mailing list a while ago but strangely cannot find the article on the list server. So I reproduced it entirely here.

Overall, the article echos well with my habit of using svn paper drafts to manage projects.


The posting below has some great advice on making regular, sustained progress in research writing. It is by Brian Martin, a professor of social sciences at the University of Wollongong, Australia. He is the author of twelve books and hundreds of articles on nonviolent action, dissent, scientific controversies, democracy, education, and other topics. Email, web


Rick Reis
UP NEXT: Helping Faculty Members Sharpen Their Focus

Tomorrow’s Research

—-2,325 words —-

Want to Publish More? Then Train Like an Athlete.

For many years, I have observed new faculty members devote enormous time to their teaching, neglecting their research. When I recommend putting a greater priority on research, they listen appreciatively but postpone action until “things aren’t so busy” — a time that never comes.

Then, in early 2008, I came across a short, punchy book by Tara Gray titled Publish & Flourish (Gray, 2005). It spells out a 12-step plan to become a prolific academic author and cites research to back up the plan. Gray’s plan enabled me to support faculty and graduate students to become much more productive.

The foundation of Gray’s 12-step program is quite simple: write for 15 to 30 minutes every day. Yes, that’s it: the core requirement is daily writing, at least five days a week, preferably seven.

Gray draws heavily on the work of Robert Boice, who studied the habits of productive new academics (Boice 1990, 2000) and found that daily writing is the key to success. Should this be surprising? Coaches expect their athletes — swimmers, runners and so forth — to train daily. Junior athletes are expected to show up for training every day, at the same time. Swimmers put in the laps and runners put in the miles. This sort of training enables dedicated high school athletes to achieve times better than world champions a century ago.

So what were top athletes doing back then? Those were the days of amateurs, usually from the upper class with spare time and access to facilities, who trained when they felt like it, typically on weekends. Very gentlemanly. But their performances weren’t outstanding by today’s standards.

What about writing? Most academics seem to be operating like the gentleman athletes of the past. They wait until they feel like writing. That usually means when they have a big block of time, or are forced to meet a deadline.

Boice found that aiming to write in big blocks of time is not a good approach. The first problem is finding a big block. An earnest academic might say, “I’ll wait until the weekend … or until teaching is over … or until I’m on sabbatical.” Some never get started at all. Then, when the putative writing times arrive, it is all too hard to actually write.

The second problem is that a big block of time for writing makes the task seem onerous. Some writers are able to overcome their inertia — often when a deadline is looming — and push themselves into a marathon session of frenzied writing. This is exhausting. When finished, there’s little energy left for writing on following days. It takes a while to recover before mobilizing the mental strength for another lengthy session. Weeks can go by with only a few days of actual writing.

This pattern is analogous to a weekend athlete who is physically exhausted after a long workout and takes days to recover. Boice calls this pattern binge writing. It’s analogous to drinking or eating too much — you feel terrible afterwards.

Most academics learn binge-writing from doing assignments in high school or undergraduate years. Bingeing becomes increasingly dysfunctional as tasks become larger. Writing an essay overnight is possible, but completing a 300–page thesis requires planning and sustained work.

Boice’s alternative is simple: brief regular writing sessions. For academics, the easiest regular pattern is daily. A daily writing session might be for half an hour, or even less.

Many academics, as soon as this option is proposed, begin a series of objections. “It takes me quite a while to get started — to get myself immersed in the subject.” “I can’t just turn on inspiration at will.” True enough. If you write infrequently, it does take a while to get back into the topic. If you write in binges, you won’t feel like doing it again soon.

Regular sessions provide a solution to these obstacles. When you get used to writing every day, you don’t need as much start-up time because you were dealing with the topic yesterday. The result is greater efficiency, as memory is primed and maintained more easily.

As for inspiration, Boice (1984) found that waiting for good ideas simply doesn’t work very well. Writing is the crucible for sparking ideas, rather than ideas being the trigger for productive writing.

The core of Boice’s and Gray’s prescription for productivity is daily writing — but not too much. The idea is to make writing so inoffensive, over so quickly, that doing it doesn’t seem like such a big deal. When expectations aren’t so high, it’s easier to overcome your internal censor, that little voice that says to you, “What you’re writing is no good. In fact, it’s crap. Give up and wait for a better time.”

Perfectionism is a deadly enemy of good performance. It’s like being judged every time you write a sentence or paragraph. It’s far better to go ahead, make mistakes and learn from them. Rather than expecting great output from a burst of frenzied inspiration, the idea behind Boice’s brief regular sessions is to work with moderate daily expectations, knowing this will lead in time to better results.

Writing programs

My next step was to encourage others to adopt the Boice-Gray writing program. I started with my PhD students, most of whom were highly receptive. Six months into the program, one of them, Jody, wrote “It is just wonderful and I know if I keep it up I will get better and writing will become easier for me.”

I also set up programs with faculty and graduate students in the Arts Faculty. One of the participants, Nichole, wrote that the program has “provided me with a non-threatening way of untangling my messy thought process, thread by thread.” Running these programs enabled me to learn much more about obstacles to writing and what helps to overcome them.

Boice and Gray recommend that writers make themselves accountable to someone, as this will help sustain the habit of writing regularly. I asked my students to send weekly totals to me listing the numbers of minutes they had written each day and the number of new words produced. That way I could assess how they were doing and discuss, in our weekly phone calls, ways to fine-tune the program.

In helping others use the Boice-Gray writing program, I make some specific recommendations. I suggest making notes about the points to be covered in your new writing, doing this a day or week beforehand. I recommend that when you sit down to write, you close or remove all books, articles and other polished text. Why? Because reading the polished text switches your mind into its flaw-noticing mode, the enemy of creating your own new words. I also recommend not reading yesterday’s writing, but instead using just your notes to provide guidance to today’s writing.

I also recommend closing the door, turning off the telephone, closing email and web applications and generally removing all distractions. Producing new words, for many writers, is a delicate process. Interruptions are temptations to do something else.

Some academics say they are so busy they have no time to do 15 minutes of daily writing. What this usually means is that they have put writing too low on their priority list. These busy academics spend hours preparing lectures, marking essays, attending seminars and committee meetings — and checking emails, surfing the web, and gossiping with colleagues. Devoting 15 minutes to writing at the beginning of a nominal eight-hour working day can’t make much difference to getting other things done, can it?

The title of chapter 4 in Boice’s 2000 book Advice for New Faculty Members is a single word: “Stop.” If the first principle of productive writing is to start, the second is to stop — before doing too much. For regular writing, you need to feel fresh when you start. If you feel worn out from too much writing yesterday or the day before, then you may postpone your session until tomorrow, starting a cycle of boom and bust, namely binge writing. So, Boice says, stop sooner rather than later.

Gray in her 12-step program made the advice more specific: write for 15 to 30 minutes per day. This means stopping when you get to 30 minutes. That may not seem like much, but it’s only the writing part. There’s a lot of additional work required before this becomes publishable prose: studying key texts, obtaining data, running experiments, seeking comments on drafts, submitting articles, and perhaps revising and resubmitting. Writing is the core activity, something akin to the highest intensity part of an athletic training program, but it has to be supplemented by a lot of other work.

I added one tweak to the Boice-Gray program. I ask participants to begin each 15–30 minute session by writing new words, for 5 to 20 minutes, and only doing other writing activities, such as taking notes or editing previous text, after the new words have been produced. I request this because composing new text is, for most writers, the most difficult task they face and the one most commonly postponed.

One of the common laments of people using this program is “I don’t know what to write,” often accompanied by “I’m not ready. I need to do more reading, or thinking, or investigation.” This is an indirect expression of the familiar formula of researching first and then writing up the results. Boice and Gray want to turn this on its head. Their motto: “Write before you’re ready!”

This means starting writing even though you don’t know enough about the topic, you haven’t read all the background material and haven’t done the experiments or fieldwork or interviews. Indeed, you’re just starting work in a field that’s entirely new to you. How can you write about it?

One approach is to write about what you’re going to do. Describe the things you know and the things you need to find out. Tell about the experiments you’re planning and how you’ll set them up. Tell how you’ll analyze the data.

Another approach is pretty similar: start writing the paper that you’d normally write at the end of your research. When you come to any part that you don’t know or don’t understand, just do as well as you can and keep going.

This feels very strange at first. Here’s how it works. By writing, you stimulate your thinking. In order to make progress on your project, you need to think about it — and writing is an efficient way of making this happen. Even after you’ve finished writing for the day, your unconscious mind will be working away at the topic, trying to address the matters you expressed.

Of course it’s quite possible to think about your topic without writing about it. Writing is just a reliable way of sustaining and focusing the thinking process. How many people schedule 15 minutes per day of concentrated thinking about a topic? If you’ve tried it, you’ll know it’s not easy to sustain.

Unconscious mental processing — during the time you’re not writing — is one thing that makes daily writing more efficient than bingeing. When you do a long stint of writing, you’re attempting to do all the thinking in one burst. This intensive effort can be exciting, but despite appearances it’s not as productive as harnessing the mind over longer periods. The brain is like a muscle: it responds best to sustained, incremental training.

There’s another, more practical reason why writing first — before doing all the research — is more efficient than writing only at the end. Let’s say there are ten major books in the area you want to write about. The normal approach is to read them first, and probably you’ll want to read even more books and articles just to be sure you understand the topic.

When you write first, before doing all the reading, you find out exactly what you need to know. You find gaps in your argument, points where you need examples, and places where you need a reference. So when you turn to the ten books, you don’t need to read them in full. You know exactly what you’re looking for, so you can just check the relevant bits.

Does this mean you learn less? Not at all. When you read a book or article with a purpose, you’re much more likely to be able to remember crucial information because it fits within a framework you’ve developed.


Regular writing is a powerful tool, but for many it is extremely challenging. The temptations of procrastination are powerful. Therefore, rather than relying on willpower every day, the key is to establish conditions in your life that help develop and maintain a habit. These include finding a dedicated place and time for writing, keeping tallies of minutes spent writing, and reporting to a mentor. The task of undertaking writing sessions that are brief and regular helps reduce psychological resistance to starting, which is often the greatest barrier. Putting these steps into place can make it far easier to establish and maintain a habit that leads to high productivity.

However, only a few writers find themselves in the fortunate position of being encouraged and supported to make these sorts of arrangements. The wider social circumstances are not particularly supportive — indeed, they are at the foundation of bingeing behavior. Boice says that established writers and editors are actually unsympathetic, as they think people who aren’t publishing don’t have anything to say. He quotes one editor as saying, concerning a writing program, “Why bother? Too much is already being written and good writers don’t need help.” (Boice, 1990, p. 126). This sort of view, which Boice calls “elitist,” assumes that writers are born, not made.

The Boice-Gray program challenges this sort of elitist attitude. It is based on the assumption that with the right conditions, just about anyone who wants to become a much better writer can do so. The program is also a challenge to every academic — you can do better too.


Boice, R. (1984). Contingency management in writing and the appearance of creative ideas: implications for the treatment of writing blocks. Behaviour Research & Therapy, 21, 537–543.
Boice, R. (1990). Professors as writers: A self-help guide to productive writing. Stillwater, OK: New Forums Press.
Boice, R. (2000). Advice for new faculty members: Nihil nimus. Boston: Allyn & Bacon.
Gray, T. (2005). Publish & flourish: Become a prolific scholar. University Park, NM: Teaching Academy, New Mexico State University.

I thank Sharon Callaghan, Don Eldridge, Ian Miles and Kirsti Rawstron for helpful comments and Tara Gray for inspiration and encouragement as well as detailed advice. This is an edited extract from Brian Martin, Doing good things better (Ed, Sweden: Irene Publishing, 2011),

September 8, 2012

How to manage project progress

Filed under: Real — liyiwei @ 11:26 pm
Tags: ,

Managing project is a huge topic that involves multiple aspects. This post focuses on one specific aspect, on how to manage and ensure progress. To me this is the most important aspect, because a project that is not done is worth nothing.

My strategy is very simple: use schedules to manage progress.

Here is a quick example.

First milestone demo: October 17, 2012.
(A milestone is a demo that can show the project (1) is of SIGGRAPH potential and (2) is likely to be completed.)

Demo production (assuming we have 3, each with complete and concrete descriptions):
. Demo 1: November 1, 2012.
. Demo 2: November 16, 2012.
. Demo 3: December 1, 2012.

First complete paper draft: December 17, 2012.
(SIGGRAPH is not just about getting things done; it is a refinement progress towards perfection. My hope is to have a one month margin, even though few of my projects, mostly first authored by myself, have made this.)

SIGGRAPH 2013 deadline: January 17, 2013.
(Hard deadline; everything has to be done before this.)

The schedule should be drafted, usually by the first author student, in the beginning of the project, and constantly updated and maintained throughout the project progress. All team members will look at the schedule and agree with its content.

The most important part is to make sure you actually meet your schedule. Do not let anything slip. Otherwise, it will not work.

Of course, research, or any project, can and will bump into unexpected snags. It is also not possible to accurately predict the amount of time needed for every item. The key is to keep the schedule as accurate as possible. For example, if you originally think demo 2 above will take only 2 weeks but it ends up taking 3 weeks, you should question whether you are too optimistic about your schedule and allocate more time for other items.
(If you find yourself not being able to meet the deadline, it means you probably have to speed things up by either working harder or working more efficiently. But the good thing is that you realize this early rather than late.)
As you become more experienced, you will also become more accurate in managing and predicting schedules. Without this practice, you will less likely learn this very important life skill.

I guess it probably very hard to fully comprehend what I mean here. If you are my student, you will learn this through a lot of practice with me.

I learned this through my NVIDIA days. One thing I find particularly proud about NVIDIA is that to my memory *not a single product* has ever been delayed during my stay. This was in sharp contrast to ATI, the (then) fierce competitor to NVIDIA, which (for reasons I do not know) constantly bumped into product delays. The rest is history.

The way the NVIDIANS did it is like this. Everybody and their brothers used schedules to manage progress, and slipping a milestone is considered a more serious crime than shooting someone. The CEO dropped in group meetings constantly and unannounced, and called out those responsible for delays. (Trust me, you will prefer being shot rather than getting called out.)

I like to list one most important thing I learned with every one of my jobs. And for NVIDIA, it will be project management. I do not think I will have learned this if I started as a professor immediately after school.

Theme: Rubric. Get a free blog at