(I need to write this down while I still feel the slight amount of excitement.)
Recently I have been involved in two projects for automatic video captioning, one for a research paper and the other for a product feature.
The research paper will be presented at UIST 2023; see this page for more details.
The product feature can be accessed via this page; if you have any feedback feel free to let me know.
The two projects share some high-level ideas (such as maintaining temporal coherence for the captions while optimizing their spatial parameters with respect to the video content), but the specific methods and implementations are quite different.
Shipping a product involves a lot of testing and tuning to ensure robust experiences for a wide range of users and use cases (often beyond what the creators can initially anticipate), while publishing a research paper often requires a lot of work in writing and presentation that can be a dominating factor in deciding its acceptance and dissemination.
Leave a Reply