Demo Video

Download Full Video
Champion Demo at Student TechFest in Microsoft Research Asia in 2013.
Selected Demo from MSRA to Microsoft TechFest 2013.
Several other videos for Microsoft TechFest 2013 in WMV format could be downloaded here:
Reconstruction; Augmented Reality; Holography.

This project demonstrates an exploration from online 3D reconstruction to various applications including virtual reality, holography and augmented reality. Firstly, we presents how portable 3D reconstruction with iterative segmentation enables ordinary users to create and consume 3D models. Secondly, we demonstrate that users could interact with reconstructed models in the virtual world using proxies and produce awesome artworks with favorite reconstructed models. Moreover, we introduce "Holography" with reconstruction for both advertisement and comparison between virtual and real models. Eventually, we propose that reconstructed models could decorate your room, enrich magazines and innovate story-telling in augmented reality scenarios on the head-mounted or mobile hand-held devices. In summary, our video presents a promising trend from 3D reconstruction to virtual and augmented reality. Real-time rendered demo by myself:

Vision Video

Reviews

The following reviews from CHI experts are very useful for me to make high-quality videos in the future. It was not a successful submission since it was my first created video. Anyway, more than 1,000 people watched the video on YouTube, TechFest or even start-up meetings. We even have a patent thanks to Siyuan Fang and Jiawei Gu for designing new UI. So I believe this work has some sort of impact. As far as I know, this project has inspired two start-up and two research projects.
R1: This seems like a very interesting technology, but the video does not do it justice. It is difficult to understand what we're seeing, except very superficially (not helped by the 1-second in duration text slides (!)), and so we end up looking at the same idea over and over for four minutes. Either those four minutes need to communicate much more, and much more engagingly, or the video should be no more than 75 seconds. But as it is, the video is not a good fit for the Video Showcase.
Ruofei: Thank you for the review, this is actually my first demo video I have ever made. No post-processing and as it is.
R2: The video is long and the first few segments include large portions of text with very little time for the reader to read. Please do not include a whole page worth of text in a 2 second segment. It is impossible for someone to read (and pausing the video to read defeats the purpose of the video). I did not understand what the holograph display had to do with the project and I wasn't sure I understood what the 4 parts are. Overall, the authors have some really interesting project but it is poorly presented. The video is not of a great quality and not compelling enough for me to give it a high score.
Ruofei: The holography is a potential application field (advertisement via spatial AR) of the project.
R3: This video is interesting to watch and clearly involved a lot of effort, but it's also somewhat long and confusing. At the beginning, the video displays a large amount of text, then skips past it after 1-2 seconds, far too quickly to read. There are also more than a few spelling/grammar errors. The procedure for creating 3D models is not clear from the video. Is it necessary to (1) rotate a camera around the object, *and* (2) rotate the object in front of a stationary camera, or just one of these? The video demonstrates several ways to use the models, including holographic store advertisements and interactive storytelling, but the scenarios are rather simplistic and contrived. A more fleshed-out demo could be more compelling. At 4 minutes, the video is also quite long. The contribution to HCI is also somewhat unclear, partly due to the presentation issues described above. The procedure for creating models is hard to follow and seems laborious. For example, it appears necessary to manually mask/rotoscope each frame of video where the object appears, in order to remove the background--this could take hours, yet automated approaches exist. The proposed system also doesn't seem to have a way to handle seams in the mesh.
Ruofei: I am honestly showing what I have in the video and each reconstruction takes up to 3min including 1-4 segmentations. Thank you for the review.

3DVAR: From 3D Reconstruction to Virtual and Augmented Reality

Ruofei Du, Zhiwei Li, Rui Cai, Bao-Liang Lu, Lei Zhang

StereoScanner: Building Your Favorites in 3D

Ruofei Du, Zhiwei Li, Rui Cai, Lei Zhang

TechFest Gives Glimpse of Microsoft’s Future

The Seattle Times | Janet I. Tu
Mar. 5, 2013

The First Student TechFest in Microsoft Research Asia

Microsoft Research Asia Official Blog | Huan Huang
Jan. 17, 2013

BibTeX

@online{
    du2013d3var,
    title={3DVAR: From 3D Reconstruction to Virtual and Augmented Reality},
    author = {Ruofei Du, Zhiwei Li, Rui Cai, Siyuan Fang, Jiawei Gu, Lei Zhang, Bao-Liang Lu},
    url={http://duruofei.com/Research/3DVAR},
    year={2013},
    urldate={2013-11-17}
}