Confessions of a researchaholic

July 27, 2017


Filed under: Real — liyiwei @ 9:06 am
Tags: ,

Looking at the paper titles for HPG 2017, I wonder if it can be renamed to HPRT (similar to how CVPR can be renamed to CVML).

When we have real-time ray tracing on mobile devices (the day will come), all the legacy graphics algorithms (i.e. tricks) will become obsolete.
I believe real-time RT, instead of machine learning, will be the end of traditional rendering tricks.

April 10, 2017

Open source texture synthesis

Filed under: Real — liyiwei @ 6:00 pm
Tags: , , ,

Well I guess this is long overdue; knock yourself out here.

Open source facilitates collaboration and code reuse, and propels our progress forward. I shall participate more in the future.

April 5, 2015

Furious 7

Filed under: Real — liyiwei @ 10:40 am
Tags: ,

They actually parachuted the cars for real instead of CG 🙂

October 13, 2014

How to make a SIGGRAPH paper

Filed under: Real — liyiwei @ 11:08 pm
Tags: , ,

There is a course in the upcoming SIGGRAPH Asia 2014 conference on how to make a SIGGRAPH paper.

The content has not been completed decided. If there is anything particular you like to hear about, feel free to leave a comment below within the next 13 days. You can do so with anonymous or real identity.

Please also help spread and share this.

December 31, 2012

How to design demos

Filed under: Real — liyiwei @ 9:51 pm
Tags: , , , ,

In a nutshell, a demo should properly demonstrate technical aspects with sufficient artistic appeals.

The technical part is usually more important, and can suffice alone for many science and engineering disciplines. However, the artistic part is also very important for graphics and HCI, or any fields which involve direct human perception and consumption.

Demos usually take a lot of time and efforts, on top of the usual workload in ideation, writing, algorithm, implementation, and experimentation. And whether you like it or not, a solid and novel algorithm cannot be adequately assessed or appreciated by the readers if it is not demonstrated through proper demos.

Thus, designing demos is kind of an art. Below are recent suggestions from Sylvain Lefebvre which I have found to be excellent.

A guideline that worked fine for me is to consider whether 1) the result demonstrates the technique properly and 2) the result looks just good enough that it appears useful; in particular we want to avoid people think that the example is contrived to only show the advantage of our approach.

The problem is that 1) and 2) sometimes compete with each other (e.g. a fantastic rendering possibly making it hard to properly see the motion, etc.). Also we do not want to spend too much time on 2), only enough that people will think that it is convincing.

March 26, 2012

Another data point

Filed under: Real — liyiwei @ 1:49 am
Tags: , , ,

In responding to my earlier post, a very talented graphics researcher has shared with me his statistics, as shown below.

As you can see, his has much better success rate than me, especially considering that he has been doing rendering, a tough field for SIGGRAPH. 🙂

November 20, 2011

Ray tracing

Filed under: Real — liyiwei @ 6:43 pm
Tags: , ,

Many mentors have their own favorite entry level project for both training and sieving purposes. And mine is ray tracing.

Read + code

Read the following books and implement your own ray tracer from scratch.
Yes, I mean writing every single line of the code yourself, except for standard libraries like iostream and cmath. And yes, I recommend c++ as the programming language.

Ray tracing in one weekend, by Peter Shirley.
Path tracing with soft phenomena. Very short and cute introduction to ray tracing in particular and computer graphics in general. Full of hidden gems and wisdom, but beginners might not appreciate all the nuances.

An introduction to ray tracing, by Eric Haines, Pat Hanrahan, Rob Cook, Jim Arvo, David Kirk, Paul Heckbert, and (edited by) Andrew Glassner.
Whitted-style ray tracing with hard phenomena. More comprehensive than the book above.

You should read the books (especially the first one) on a computer and code along. Do not just read; it is less fun and less effective.
Software engineer your code from the very beginning; ray tracing is nicely object oriented with natural object-like components such as rays, shapes, materials, colors, motions, and cameras.


Produce your own ray traced images. Start with something simple, like mirrors and glasses for basic reflections and refractions. Ultimately, I want to see 2 things:

  • Produce ray traced animations from the BART benchmark. (If you can reach this stage, I consider your training complete. If you cannot even get to this point for whatever reason, you have no chance to survive in our future projects.)
  • Design your own favorite scene, and ray trace it. Show me your creativity and artistic + scientific blend. You can find examples from prior Stanford cs348b rendering competitions.
    (This part can be good for a group project, even though I expect everyone to complete everything else individually.)


I did ray tracing as part of Stanford CS348b image synthesis, and I would say this is perhaps the class that I have learned the most from.
First, ray tracing is a superb training for coding and software engineering; it is inherently modular and suitable for object oriented programming, and the amount of coding is non-trivial (actually, quite hefty for new-comers, especially if you code from scratch).
Second, ray tracing is a superb introduction to image formation, and you will learn the fundamentals of physics, computer graphics, computer vision, and image processing.
Third, and perhaps most importantly, it is heck of fun. It is beyond words for me to describe the satisfaction I (and many people I heard from) derived from producing my first ray traced image. I am not an artist, but I can paint with algorithms.


There is another excellent book for ray tracing: Physically based rendering, by Matt Pharr and Greg Humphreys, but I personal opinion is that you should read it as your second, not first, ray tracing book.

September 17, 2011

Producing research results

Filed under: Real — liyiwei @ 4:06 pm
Tags: , , ,

Sylvain Lefebvre has a nice post about how to structure code for producing research results; see here. (Full disclosure: Sylvain is my collaborator, but he is a superb researcher and coder, as evident from his resume.)

December 20, 2010

How to use the papers committee list

Filed under: Real — liyiwei @ 11:49 am
Tags: , , ,

SIGGRAPH now publishes the technical papers committee list a few weeks prior to the submission deadline. Here is my suggestion on how to take this information to your advantage.

First, let me tell you what NOT to do. It is very common for people to try to guess who is likely to review your papers, but this is entirely useless and counterproductive. In general, there is indeed a positive correlation between the committee member expertise and your paper subject, but it is not a very strong correlation and there is a non-trivial chance that a paper will be assigned to an unexpected committee member. (This is mainly due to how the paper sorting and assignment process goes but I am not going to elaborate on this here because I do not think it is necessary for authoring a good paper.)

So my personal strategy is to make sure the paper will survive *any* committee member. This is the safest bet and you cannot go wrong with that. (But do check the committee member list constantly though as sometimes it may change prior to the deadline.) Specifically, you should write your paper (especially the title and abstract) in a way so that it is as clear as possible on what it is all about to minimize the chance of misunderstanding. (If the paper sorters misunderstand your paper, they are more likely to assign your paper to the wrong committee member. And if the senior reviewers misunderstand your paper, they are more likely to assign your paper to the wrong tertiaries. And if reviewers misunderstand your paper they are more likely to score it lowly.)

It also helps to read and cite papers from the committee members for two reasons. The first reason is that the background and expertise of a committee member will obviously have a huge influence on how she is going to review a paper, so if you understand her, you stand a better chance of appreciating how she might approach a paper. The second reason is that psychological studies indicate that it is almost never too much to flatter, and people are usually happy to see their own papers cited and attributed in a positive manner.

If you want to push this further, my personal philosophy is to assume that my paper will be reviewed by the worst possible reviewers and prepare for that. Like Andy Grove said, only the paranoid survive.

November 6, 2010

Looking for intern: somatic computing in MSR SF

Filed under: Real — liyiwei @ 10:52 am
Tags: , , ,

Jaron Lanier and I recently started a new project termed “somatic computing” in Microsoft Research San Francisco. We are looking for an intern who would be interested in working with us. For more information, please take a look at our project page.

August 12, 2010

Reviewing computer science papers

Filed under: Imaginary,Real — liyiwei @ 5:06 pm
Tags: , ,

Computer theory: A hundred page proof that takes many very smart people weeks or even months to verify and (eventually) reject.

Computer graphics: A 4 to 8 page paper that takes one Joe Six Pack maybe just 3 seconds to reject because the teaser image does not look beautiful enough.

August 7, 2010

Social pass

Filed under: Real — liyiwei @ 10:15 am
Tags: , ,

It will be great if SIGGRAPH (and other conferences) could have a new registration category called “social pass”. This social pass will be (significantly) cheaper than full registration, and is designed primarily for people who go to conferences mainly to socialize with colleagues. When I was in grad school I used to listen attentively to all the technical sessions, but nowadays (for better or worse) I found myself spending so much time chatting with people to the point that I serious doubt the value I paid for a full registration. I still go to many technical sessions, but my main purpose now turn to meet specific people (e.g. paper authors or people who work on related fields) and pay enough attention to the presentations so that I could have enough information to chat up with the authors later on. (Man, I like your presentation, especially demo XXX. What a cool paper!)

On a further thought, I could be tricky to design such a social pass. If I just want to talk to people in the hallways, I actually do not have to pay a dime and just need to walk into the convention center. But then that might not be enough, as a lot of the conversations happen inside the sessions (especially at the beginnings and ends) and the parties (most of which, hosted by various companies and research groups, I could already go for free). So I still need the credential to attend a certain subset of full registration.

Maybe one possibility is to charge for individual sessions so that I could pick only these that I plan to attend (to socialize). For the rest, I could simply loiter in the hallways and the parties.

April 12, 2010

Accidental art

Filed under: Imaginary,Real — liyiwei @ 2:55 pm
Tags: , , ,

For some reason, the most beautiful images I have produced tend to be the buggy ones.

I guess this is a unique advantage of graphics research (compared to other CS fields): when we screw up, we might be able to claim the result as an art.

February 16, 2010

The Rowling criterion

Filed under: Imaginary,Real — liyiwei @ 3:41 pm
Tags: ,

I guess I am not the only one who ever heard people commenting that “computer graphics research is mostly done” or “all the big ideas in graphics have been discovered and the remaining research is mainly about incremental ideas” or something like that.

If you look at history, similar comments have been made about physics in the early twentieth century when people believed that Newtonian physics is near perfection and all the remaining problems are incremental. And then came quantum physics, relativity, and other ground breaking stuff.

I do not think computer graphics is done for a very simple reason. Every year I have been kept seeing interesting/exciting papers coming out of various graphics conferences. So these pessimistic comments may reflect more about the status of people who made them rather than the status of computer graphics research.

And I would not claim graphics is done until people can author images, movies, and animations as easily as writing articles. And these do not even have to be good; not everyone can be Shakespeare, but at least almost everyone (who is literate without severe handicaps like blindness) can easily write an article, a letter, or a diary entry via existing tools, ranging from simple pen and pencil to more advanced text editing software. But how many people today can easily make an image or a movie all alone? It is true that amazing effects have been achieved in many movies, but these almost always involve significant resources, not least a large team of talented professionals. (Take a look of the end credits of the kinds of movies like Harry Potter or Avatar.)

Talking about Harry Potter, this is actually a good example on the difference between authoring textual and non-textual artifacts: for the former, a good writer like J.K. Rowling could single-handedly author an entire book series, but for the latter, a large team of talented staff has to be devoted to realize the books into movies. And this a technology, not talent issue; no existing tools today would have allowed J.K. Rowling to turn her visions into movies, even though these imagery would likely have existed in her minds while writing the books. (I believe she can do that because that is exactly what happened in my minds when reading her books.)

Thus, allow me to propose a necessary condition to claim the completeness of graphics research, the “Rowling criterion”: it should be as easy for a single individual to author a movie/animation/image as to write an article. And when such technology is available, I would happily turn myself from a researcher into a content creator.

December 15, 2009

Toon shading

Filed under: Imaginary,Real — liyiwei @ 6:52 pm
Tags: ,


Appleseed: Ex Machina is the best toon shaded animation I have ever seen. It really beats 2D cel animation.
I am too busy right now to write down more details, but just checkout this movie if you like anime or computer graphics.

Also, don’t miss the bonus features. I find it particularly interesting that the American crew talked about the “amazing collaboration” between Chinese (John Woo, producer) and Japanese (Shinji Aramaki, director) as though these two countries ought to start the third world war instead of collaborating on animation projects.

September 29, 2009


Filed under: Imaginary — liyiwei @ 5:56 pm
Tags: ,

If I were a character in The Matrix, I want to be Anti-Neo who drags everyone from the real world back into the matrix. The synthetic world is so much easier to manipulate than the real one.

September 1, 2009

Waltz with Bashir

Filed under: Imaginary,Real — liyiwei @ 4:48 pm
Tags: , ,

The phrase “animated documentary” may sound like an oxymoron, but that is exactly what I would use to describe Waltz with Bashir. I was originally attracted to this movie due to its graphics effects, especially on the masterful use of large regions of monotonic colors. What I did not expect was thought provoking storytelling, and I was thrilled to find plenty in this movie.

I highly recommend this movie, as well as the behind the scene bonus features on how the movie is made.


August 16, 2009


Filed under: Imaginary — liyiwei @ 12:02 pm
Tags: , ,

Renaissance is a stylish binary animation. By binary, I mean it is rendered in only two colors, black and white, not even gray scales which the movie used only rarely for special effects like transparency.


I watched this movie because it was mentioned in a research paper titled Artistic Thresholding, and I am curious about the artistic and technical effects. In particular, even though binary rendering has been performed on static images (e.g. posters), these are usually abstract renditions. Thus, it was not clear to me how the effect would carry over for 3D animations, for which frame-to-frame coherence is required in addition to single frame stylization.

Overall, the binary rendering of the movie looks very good, in terms of both intra frame stylization and inter frame coherence. The binary rendering, combined with the nature of the story line, pushes the film noir genre to the extreme.

In the bonus materials, the creators explained that the movie was authored as a combination of motion capture and 3D polygonal rendering. This is quite comprehensible. Unfortunately, no explanation was given to the binary shading part. From what I could see, no published research algorithms have achieved binary animation anywhere near what is shown this movie. The usual suspect is heavy manual work, but even with that I still cannot figure out how things were really done.

Looks like this is still a open research problem, and probably a darn hard one.

July 15, 2009

First encounter with Ciao

Filed under: Real — liyiwei @ 4:40 pm
Tags: , , ,

A good friend of mine, Ciao, is relocating to another part of the world.

Ciao is a very nice dude, and we have had a lot of fun hanging out together. Perhaps the funniest ever experience is our first encounter, when Ciao went to interview with a company that I previously worked for.

I guess most companies have a certain interviewing style that reflects the company culture. And for that particular company, the custom seems to be grilling the interviewees with trick questions, mostly in low-level math and computer science. I could see the value of such a style in testing the interviewee’s character and intellect, but I simply couldn’t bring myself to do that to someone who has recently obtained his Ph.D. with a SIGGRAPH paper. So instead I talked with Ciao a bunch of high-level research questions. We had a great time.

The troubles came after I filled out the interview evaluation. The hiring manager informed me that my interview style is “not very proper” and sent me to a course teaching people how to do proper interviews. (I confess I never attended the course.) Furthermore, the interview with Ciao turned out to be not only the first but also the last one that I ever conducted for that company.

Fortunately, my “not very proper” interview with Ciao did not cost his job offer, and later he joined the company. Together we had some additional fun for a couple of more years. It soon became pretty apparent that we are both not very good fit for that company (my “not very proper” interview with him is probably a good early sign), so eventually we both left. But I would like to thank that company for enabling me to learn a few very important lessons that I wouldn’t have otherwise, and most importantly, the chance for getting to know Ciao.

July 10, 2009

DEADLINE post-it stop motion

Filed under: Imaginary,Real — liyiwei @ 2:25 pm
Tags: , ,

Idea is more important than graphics in producing a good video.


May 11, 2009

A Scanner Darkly

Filed under: Imaginary,Real — liyiwei @ 1:26 pm
Tags: ,

A Scanner Darkly is a movie that I have wanted to see for a long time but somehow did not manage to do so until recently. And it turned out to be so good that I regret haven’t watched it earlier.

The movie depicts the world and life seen from the view points of a group of drug addicts. To convey the distorted views from these druggies with impaired perceptions and brain functions, the movie deployed not only the traditional story line and dialogues, but also a cartoonish, non-photorealistic rendering achieved by rotoscoping.
The combined effect is so realistically surreal (oxymoron?) and disoriented that for the first time I kind of be able to feel what it is like to be a druggie. Rotoscoping not only preserves the original acting, but also facilitates hallucinatory or science-fiction effects such as bugs crawling all over one’s body and the “scramble suit” that disguises an undercover’s identity by constant shifting through the difference appearances of millions of people.

And yes, like all good movies, this one has a great story as well.

I believe this is one of these movies that are so unique that it is going to hang in my head for a long time.


April 21, 2009

Similar image search

Filed under: Real — liyiwei @ 11:19 am
Tags: , , , , ,

Google’s new toy similar image search looks intriguing, but still it does not allow query by images directly.

True content-based image search is definitely very hard, and so far I haven’t seen any system that (remotely) works.

I do not know exactly how Google’s similar image search is implemented, but it could entirely avoid content-based image search by providing similarity links between images that are already indexed in Google’s database. Thus, the similarity computation could not only be conducted offline, but also leverage contextual information such as texts surrounding images. The similarity links could even be edited by humans if necessary.

I still remember the first encounter that prompted me to feel the need for content-based image search. After submitting my camera-ready paper to SIGGRAPH 2000, Stephen Spencer asked me to secure proper copyrights for all images used in the paper. But there is this famous little green texture that I simply could not track down the original ownership (even until today).

If there is a system that would allow me to do so, I will consider it a real success. (I am looking forward to Google’s official deployment of their similar image search system to see if I could unveil the true origin of “my little green friend”).

Theme: Rubric. Get a free blog at