Ting-Chun Wang

Senior Research Scientist

NVIDIA

Santa Clara, CA

Email:

tingchunw at nvidia dot com

 

GitHub | Google Scholar

 

I'm a senior research scientist at NVIDIA, working on computer vision, machine learning and computer graphics.
I received my PhD from University of California, Berkeley in 2017, advised by Professor Ravi Ramamoorthi and Alexei A. Efros.
My recent research focus is on using generative adversarial models to synthesize realistic images and videos, with applications to rendering, visual manipulations and beyond.


News

Publications

facevid2vid

One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing

 

Ting-Chun Wang, Arun Mallya, Ming-Yu Liu

 

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021 (oral presentation)

 

project | paper | YouTube | Talk | Slides | Poster | bibtex

@inproceedings{wang2021facevid2vid,
   author    = {Ting-Chun Wang and Arun Mallya and Ming-Yu Liu},
   title     = {One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing},
   booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
   year      = {2021},
}
      
transposer_tog

Transposer: Universal Texture Synthesis Using Feature Maps as Transposed Convolution Filter

 

Guilin Liu, Rohan Taori, Ting-Chun Wang, Zhiding Yu, Shiqiu Liu, Fitsum A. Reda, Karan Sapra, Andrew Tao, Bryan Catanzaro

 

ACM Transactions on Graphics, 2021 (provisionally accepted)

 

paper | YouTube | bibtex

@article{liu2021transposer,
   author  = {Guilin Liu and Rohan Taori and Ting-Chun Wang and Zhiding Yu and Shiqiu Liu 
              and Fitsum Reda and Karan Sapra and Andrew Tao and Bryan Catanzaro},
   title   = {Transposer: Universal Texture Synthesis Using Feature Maps as Transposed Convolution Filter},
   journal = {ACM Transactions on Graphics},
   year      = {2021}
}
      
gan_survey

Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications

 

Ming-Yu Liu, Xun Huang, Jiahui Yu, Ting-Chun Wang, Arun Mallya

 

Proceedings of The IEEE, 2021

 

paper | bibtex

          
@article{liu2021generative,
   title={Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications},
   author={Ming-Yu Liu and Xun Huang and Jiahui Yu and Ting-Chun Wang and Arun Mallya},
   journal={Proceedings of The IEEE},
   year={2021}
}
  
wcvid2vid_eccv

World-Consistent Video-to-Video Synthesis

 

Arun Mallya*, Ting-Chun Wang*, Karan Sapra, Ming-Yu Liu (*equal contribution)

 

European Conference on Computer Vision (ECCV), 2020

 

project | paper | YouTube | bibtex

          
@inproceedings{mallya2020world,
   author    = {Arun Mallya and Ting-Chun Wang and Karan Sapra and Ming-Yu Liu},
   title     = {World-Consistent Video-to-Video Synthesis},  
   booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},   
   year      = {2020},
}
      
ds_tpami

Domain Stylization: A Fast Covariance Matching Framework towards Domain Adaptation

 

Aysegul Dundar, Ming-Yu Liu, Zhiding Yu, Ting-Chun Wang, John Zedlewski, Jan Kautz

 

Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020

 

paper | bibtex

@article{dundar2020domain,
  title={Domain Stylization: A Fast Covariance Matching Framework towards Domain Adaptation},
  author={Aysegul Dundar and Ming-Yu Liu and Zhiding Yu and Ting-Chun Wang and John Zedlewski and and Jan Kautz},
  booktitle={IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
  year={2020}
}
      
avid2vid_nips

Few-shot Video-to-Video Synthesis

 

Ting-Chun Wang, Ming-Yu Liu, Andrew Tao, Guilin Liu, Jan Kautz, Bryan Catanzaro

 

Conference on Neural Information Processing Systems (NeurIPS), 2019

 

project | paper | github | YouTube | bibtex

@inproceedings{wang2019fewshotvid2vid,
  title={Few-shot Video-to-Video Synthesis},
  author={Ting-Chun Wang and Ming-Yu Liu and Andrew Tao and Guilin Liu and Jan Kautz and Bryan Catanzaro},
  booktitle={Conference on Neural Information Processing Systems (NeurIPS)},
  year={2019}
}
      
d2m_nips

Dancing to Music

 

Hsin-Ying Lee, Xiaodong Yang, Ming-Yu Liu, Ting-Chun Wang, Yu-Ding Lu, Ming-Hsuan Yang, Jan Kautz

 

Conference on Neural Information Processing Systems (NeurIPS), 2019

 

paper | github | YouTube | bibtex

@inproceedings{lee2019dancing2music,
  title={Dancing to Music},
  author={Hsin-Ying Lee and Xiaodong Yang and Ming-Yu Liu and Ting-Chun Wang and Yu-Ding Lu and Ming-Hsuan Yang and Jan Kautz},
  booktitle={Conference on Neural Information Processing Systems (NeurIPS)},
  year={2019}
}
      
spade_cvpr

Semantic Image Synthesis with Spatially-Adaptive Normalization

 

Taesung Park, Ming-Yu Liu, Ting-Chun Wang, Jun-Yan Zhu

 

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019 (best paper finalist)

 

project | paper | github | YouTube | bibtex

@inproceedings{park2019SPADE,
  title={Semantic Image Synthesis with Spatially-Adaptive Normalization},
  author={Park, Taesung and Liu, Ming-Yu and Wang, Ting-Chun and Zhu, Jun-Yan},
  booktitle={Proceedings of the IEEE Conference on 
             Computer Vision and Pattern Recognition (CVPR)},
  year={2019}
}
      
vid2vid_nips

Video-to-Video Synthesis

 

Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Guilin Liu, Andrew Tao, Jan Kautz, Bryan Catanzaro

 

Conference on Neural Information Processing Systems (NeurIPS), 2018

 

project | paper | arXiv | github | YouTube | bibtex

@inproceedings{wang2018vid2vid,
   author    = {Ting-Chun Wang and Ming-Yu Liu and Jun-Yan Zhu and Guilin Liu
                and Andrew Tao and Jan Kautz and Bryan Catanzaro},
   title     = {Video-to-Video Synthesis},
   booktitle = {Conference on Neural Information Processing Systems (NeurIPS)},   
   year      = {2018},
}
      
partconv_arxiv

Partial Convolution based Padding

 

Guilin Liu, Kevin J. Shih, Ting-Chun Wang, Fitsum A. Reda, Karan Sapra, Zhiding Yu, Andrew Tao, Bryan Catanzaro

 

arXiv preprint arXiv:1811.11718

 

arXiv | bibtex

          
@article{liu2018partial,
   author    = {Liu, Guilin and Shih, Kevin J and Wang, Ting-Chun and Reda, Fitsum A 
                and Sapra, Karan and Yu, Zhiding and Tao, Andrew and Catanzaro, Bryan},
   title     = {Partial convolution based padding},
   journal   = {arXiv preprint arXiv:1811.11718},
   year      = {2018},
}
      
ds_arxiv

Domain Stylization: A Strong, Simple Baseline for Synthetic to Real Image Domain Adaptation

 

Aysegul Dundar, Ming-Yu Liu, Ting-Chun Wang, John Zedlewski, Jan Kautz

 

arXiv preprint arXiv:1807.09384

 

arXiv | bibtex

@article{dundar2018domain,
  title={Domain Stylization: A Strong, Simple Baseline for Synthetic to Real Image Domain Adaptation},
  author={Dundar, Aysegul and Liu, Ming-Yu and Wang, Ting-Chun and Zedlewski, John and Kautz, Jan},
  journal={arXiv preprint arXiv:1807.09384},
  year={2018}
}
      
parconv_eccv

Image Inpainting for Irregular Holes Using Partial Convolutions

 

Guilin Liu, Fitsum A. Reda, Kevin J. Shih, Ting-Chun Wang, Andrew Tao, Bryan Catanzaro

 

European Conference on Computer Vision (ECCV), 2018

 

project | arXiv | YouTube | bibtex

          
@inproceedings{liu2018image,
   author    = {Liu, Guilin and Reda, Fitsum A and Shih, Kevin J 
                and Wang, Ting-Chun and Tao, Andrew and Catanzaro, Bryan},
   title     = {Image inpainting for irregular holes using partial convolutions},
   booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},   
   year      = {2018},
}
      
pix2pixHD_cvpr

High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs

 

Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, Bryan Catanzaro

 

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018 (oral presentation)

 

project | paper | arXiv | github | YouTube | slides | bibtex

@inproceedings{wang2018pix2pixHD,
   author    = {Ting-Chun Wang and Ming-Yu Liu and Jun-Yan Zhu 
                and Andrew Tao and Jan Kautz and Bryan Catanzaro},
   title     = {High-Resolution Image Synthesis and Semantic Manipulation 
                with Conditional GANs},
   booktitle = {Proceedings of the IEEE Conference on 
                Computer Vision and Pattern Recognition (CVPR)},   
   year      = {2018},
}
      
thesis

Beyond Photo-Consistency: Shape, Reflectance, and Material Estimation Using
Light-Field Cameras

 

Ting-Chun Wang

 

PhD Thesis, 2017

 

lfv_sig

Light Field Video Capture Using a Learning-Based Hybrid Imaging System

 

Ting-Chun Wang, Jun-Yan Zhu, Nima Khademi Kalantari, Alexei Efros, Ravi Ramamoorthi

 

ACM Transactions on Graphics (SIGGRAPH), 2017

 

paper | low-res pdf | YouTube | bibtex | project page

@article{wang2017light,
   author  = {Ting-Chun Wang and Jun-Yan Zhu and Nima Khademi Kalantari 
              and Alexei A. Efros and Ravi Ramamoorthi},
   title   = {Light Field Video Capture Using a Learning-Based Hybrid 
              Imaging System},
   journal = {ACM Transactions on Graphics (Proceedings of SIGGRAPH)},
   volume  = {36},
   number  = {4},
   year    = {2017},
}
      
brdf_pami

SVBRDF-Invariant Shape and Reflectance Estimation from Light-Field Cameras

 

Ting-Chun Wang, Manmohan Chandraker, Alexei Efros, Ravi Ramamoorthi

 

Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2017

 

paper | bibtex

@article{wang2017svbrdf,
   title={{SVBRDF}-Invariant Shape and Reflectance 
   Estimation from Light-Field Cameras},
   author={Wang, Ting-Chun and Chandraker, Manmohan
   and Efros, Alexei and Ramamoorthi, Ravi},
   journal={IEEE Transactions on Pattern 
   Analysis and Machine Intelligence (TPAMI)},
   year={2017},
}
      
lfvs_sig

Learning-Based View Synthesis for Light Field Cameras

 

Nima Khademi Kalantari, Ting-Chun Wang, Ravi Ramamoorthi

 

ACM Transactions on Graphics (SIGGRAPH Asia), 2016

 

project page | paper | bibtex

@article{Kalantari16ViewSynthesis,
   author={Nima Khademi Kalantari and Ting-Chun Wang 
   and Ravi Ramamoorthi},
   title={Learning-Based View Synthesis for Light 
   Field Cameras},
   journal={ACM Transactions on Graphics (Proceedings 
   of SIGGRAPH Asia 2016)},
   year={2016},
}
      
lfmr_eccv

A 4D Light-Field Dataset and CNN Architectures for Material Recognition

 

Ting-Chun Wang, Jun-Yan Zhu, Ebi Hiroaki, Manmohan Chandraker, Alexei Efros, Ravi Ramamoorthi

 

European Conference on Computer Vision (ECCV), 2016

 

paper | HTML comparison | bibtex | dataset (2D thumbnail) | full dataset (15.9G)

@inproceedings{wang2016dataset,
   title={A {4D} light-field dataset and {CNN} 
   architectures for material recognition},
   author={Wang, Ting-Chun and Zhu, Jun-Yan 
   and Hiroaki, Ebi and Chandraker, Manmohan 
   and Efros, Alexei and Ramamoorthi, Ravi},
   booktitle={Proceedings of European Conference on 
   Computer Vision (ECCV)},
   year={2016}
}
      
brdf_cvpr

SVBRDF-Invariant Shape and Reflectance Estimation from Light-Field Cameras

 

Ting-Chun Wang, Manmohan Chandraker, Alexei Efros, Ravi Ramamoorthi

 

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016 (oral presentation)

 

paper | supplementary | HTML comparison | bibtex

@inproceedings{wang2016svbrdf,
   title={SVBRDF-invariant shape and reflectance 
   estimation from light-field cameras},
   author={Wang, Ting-Chun and Chandraker, Manmohan 
   and Efros, Alexei and Ramamoorthi, Ravi},
   booktitle={Proceedings of the IEEE Conference on 
   Computer Vision and Pattern Recognition (CVPR)},
   year={2016}
}
      
stereo_cvpr

Depth from Semi-Calibrated Stereo and Defocus

 

Ting-Chun Wang, Manohar Srikanth, Ravi Ramamoorthi

 

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016

 

paper | HTML comparison | bibtex

@inproceedings{wang2016semi,
   title={Depth from semi-calibrated stereo and defocus},
   author={Wang, Ting-Chun and Srikanth, Manohar
   and Ramamoorthi, Ravi},
   booktitle={Proceedings of the IEEE Conference on 
   Computer Vision and Pattern Recognition (CVPR)},
   year={2016}
}
      
occlusion_pami

Depth Estimation with Occlusion Modeling Using Light-field Cameras

 

Ting-Chun Wang, Alexei Efros, Ravi Ramamoorthi

 

Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2016

 

paper | bibtex

@article{wang2016depth,
   title={Depth estimation with occlusion modeling 
   using light-field cameras},
   author={Wang, Ting-Chun and Efros, Alexei and 
   Ramamoorthi, Ravi},
   journal={IEEE Transactions on Pattern 
   Analysis and Machine Intelligence (TPAMI)},
   volume={38},
   number={11},
   pages={2170--2181},
   year={2016},
}
      
occlusion

Occlusion-aware depth estimation using light-field cameras

 

Ting-Chun Wang, Alexei Efros, Ravi Ramamoorthi

 

International Conference on Computer Vision (ICCV), 2015

 

paper | supplementary | bibtex | code | dataset (3.3GB)

@inproceedings{wang2015occlusion,
  title={Occlusion-aware depth estimation 
  using light-field cameras.},
  author={Wang, Ting-Chun and 
  Efros, Alexei and Ramamoorthi, Ravi},
  booktitle={Proceedings of the IEEE International 
  Conference on Computer Vision (ICCV)},
  year={2015}
}
      
glossy_pami

Depth estimation and specular removal for glossy surfaces using
point and line consistency with light-field cameras

 

Michael Tao, Jong-Chyi Su, Ting-Chun Wang, Jitendra Malik, Ravi Ramamoorthi

 

Transactions on Pattern Analysis and Machine Intelligence (TPAMI)

 

paper | bibtex

@article{tao2015depth,
title={Depth Estimation and Specular Removal 
for Glossy Surfaces Using Point and 
Line Consistency with Light-Field Cameras},
author={Tao, Michael and Su, Jong-Chyi 
and Wang, Ting-Chun and Malik, Jitendra 
and Ramamoorthi, },
journal={IEEE Transactions on Pattern 
Analysis \& Machine Intelligence},
number={1},
pages={1--1},
year={2015},
publisher={IEEE}
}
      
glossy

Depth estimation for glossy surfaces with light-field cameras

 

Michael Tao, Ting-Chun Wang, Jitendra Malik, Ravi Ramamoorthi

 

ECCV workshop on Light Fields for Computer Vision (L4CV), 2014

 

paper | bibtex

@inproceedings{tao2014depth,
title={Depth estimation for glossy 
surfaces with light-field cameras},
author={Tao, Michael W and Wang, Ting-Chun 
and Malik, Jitendra and Ramamoorthi, Ravi},
booktitle={Computer Vision-ECCV 2014 Workshops},
pages={533--547},
year={2014},
organization={Springer}
}
      

Professional Activities

 

Tutorial/Workshop Co-organizer

Conference Area Chair: WACV

 

Conference Reviewer: CVPR, ICCV, ECCV, SIGGRAPH, SIGGRAPH Asia, Eurographics, NeurIPS, ICLR, ICRA

 

Journal Reviewer: TPAMI, IJCV, TOG, TIP, TMM, TVCG, TCSVT, CVIU



Awards


Software

Few-shot vid2vid: Few-shot video-to-video translation.

SPADE: Semantic image synthesis on diverse datasets including Flickr and coco.

vid2vid: High-resolution video-to-video translation.

pix2pixHD: High-resolution image-to-image translation.

Light Field Video: Light field video applications (e.g. video refocusing, changing aperture and view).



Talks

Few-Shot Video-to-Video Synthesis

ICCV Workshop on Advances in Image Manipulation (2019)

 

Mixed-Precision Training for pix2pixHD

ICCV Tutorial on Accelerating Computer Vision with Mixed Precision (2019)

 

Image-to-Image Translation

ICIP Tutorial on Image-to-Image Translation (2019)

 

Video-to-Video Synthesis

CVPR Workshop on Deep Learning for Content Creation (2019)

 

High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs (pix2pixHD)

CVPR (2018)

 

Beyond Photo-Consistency: Shape, Reflectance, and Material Estimation Using Light-Field Cameras

Dissertation talk (2017)

 

Light Field Video Capture Using a Learning-Based Hybrid Imaging System

Siggraph (2017)

 

SVBRDF-Invariant Shape and Reflectance Estimation from Light-Field Cameras

CVPR (2016)


web page statistics from GoStats