Sifei Liu

Sr. Research Scientist

A portrait of Sifei Liu

sifeiliu.net

Liusifei

SifeiL

zwrq0star@gmail.com

Education

University of California, Merced
Ph. D in EECS, advised by Ming-Hsuan Yang
Fall 2012 - Fall 2027
University of Science and Technology of China
M.S. in Electronic Engineering and Information Sciences, advised by Stan Z. Li.
Fall 2008 - Spring 2011

Employment

NVIDIA
Sr. Research Scientist
Nov 2017 - Current
Santa Clara, CA
NVIDIA
Research Intern with NVIDIA Research.
March 2017 - Aug 2017
Santa Clara, CA
Chinese University of HongKong
Visiting Schoolar at the MMLAB.
Jul 2016 - Dec 2016
HongKong
Baidu Inc.
Applied Scientist Intern on IDL. Worked on face parsing and beutification Apps.
May 2013 - Jan 2016
Beijing, China

Workshop and tutorial organization

4D Hand Object Interaction: Geometric Understanding and Applications in Dexterous Manipulation
CVPR 2023
Human-centric Trustworthy Computer Vision From Research to Applications
ICCV 2021
Sensing, Understanding and Synthesizing Humans
ECCV 2020
New Frontiers for Learning with Limited Labels or Data
ECCV 2020. (Co-orgainizor and Speaker)
Learning Representations via Graph-structured Networks Tutorial
CVPR 2019 and 2020. (Co-orgainizor and Speaker)

(Co)-Mentees at NVIDIA Research

Recent Publications

TUVF: Learning Generalizable Texture UV Radiance Fields
A. Cheng, X. Li, S. Liu, X. Wang
The paper introduces TUVF, a method for learning generalizable texture UV radiance fields.
arXiv, 2023
Affordance diffusion: Synthesizing hand-object interactions
Y. Ye, X. Li, A. Gupta, S. De Mello, S. Birchfield, J. Song, S. Tulsiani, S. Liu
The paper proposes a method for interaction synthesis that addresses issues using diffusion models. They build upon the classic idea of disentangling where to interact (layout) from how to interact (content).
CVPR, 2023
Scraping Textures from Natural Images for Synthesis and Editing
X. Li, S. Liu, X. Wang, M. Yang, A. Efros
ECCV, 2022
Open-vocabulary panoptic segmentation with text-to-image diffusion models
J. Xu, S. Liu, A. Vahdat, W. Byeon, X. Wang, S. De Mello
We present ODISE: Open-vocabulary DIffusion-based panoptic SEgmentation, which unifies pre-trained text-image diffusion and discriminative models to perform open-vocabulary panoptic segmentation.
CVPR, 2023
CoordGAN: Self-Supervised Dense Correspondences Emerge from GANs
J. Mu, S. De Mello, Z. Yu, N. Vasconcelos, X. Wang, J. Kautz, S. Liu
This work introduces Coordinate GAN (CoordGAN), a structure-texture disentangled GAN that learns a dense correspondence map for each generated image.
CVPR, 2022
GroupViT: Semantic Segmentation Emerges from Text Supervision
J. Xu, S. De Mello, S. Liu, W. Byeon, T. Breuel, J. Kautz, X. Wang
This paper proposes a hierarchical Grouping Vision Transformer (GroupViT), which learns to group image regions into progressively larger arbitrary-shaped segments.
CVPR, 2022
Learning continuous environment fields via implicit functions
X. Li, S. De Mello, X. Wang, M. Yang, J. Kautz, S. Liu
ICLR, 2022
Autoregressive 3D Shape Generation via Canonical Mapping
A. Cheng, X. Li, S. Liu, M. Sun, M. Yang
The paper demonstrates a solution for 3D point cloud generation using transformers. The key idea is to decompose a point cloud into a sequence of semantically meaningful shape compositions, which are further encoded by an autoregressive model for point cloud generation.
ECCV, 2022
Learning Continuous Image Representation with Local Implicit Image Function
Y. Chen, S. Liu, X. Wang
The paper presents a method for learning continuous image representation with local implicit image function.
CVPR, 2021
Coupled Segmentation and Edge Learning via Dynamic Graph Propagation
Z. Yu, R. Huang, W. Byeon, S. Liu, G. Liu, T. Breuel, A. Anandkumar, J. Kautz
NeurIPS, 2021
Video Autoencoder: self-supervised disentanglement of static 3D structure and motion
Z. Lai, S. Liu, A. Efros, X. Wang
This paper presents a video autoencoder for learning disentangled representations of 3D structure and camera pose from videos in a self-supervised manner.
ICCV, 2021
Learning 3D Dense Correspondence via Canonical Point Autoencoder
A. Cheng, X. Li, M. Sun, M. Yang, S. Liu
The paper presents a method for learning 3D dense correspondence using a canonical point autoencoder.
NeurIPS, 2021
Learning to track instances without video annotations
Y. Fu, S. Liu, U. Iqbal, S. De Mello, H. Shi, J. Kautz
CVPR, 2021
Contrastive syn-to-real generalization
W. Chen, Z. Yu, S. De Mello, S. Liu, J. Alvarez, Z. Wang, A. Anandkumar
ICLR, 2021
Self-supervised object detection via generative image synthesis
S. Mustikovela, S. De Mello, A. Prakash, U. Iqbal, S. Liu, T. Nguyen-Phuoc, C. Rother, J. Kautz
CVPR, 2021
Video matting via consistency-regularized graph neural networks
T. Wang, S. Liu, Y. Tian, K. Li, M. Yang
CVPR, 2021
Synthesizing long-term 3d human motion and interaction in 3d scenes
J. Wang, H. Xu, J. Xu, S. Liu, X. Wang
CVPR, 2021
Joint-task self-supervised learning for temporal correspondence
X. Li, S. Liu, S. De Mello, X. Wang, J. Kautz, M. Yang
This paper proposes to learn reliable dense correspondence from videos in a self-supervised manner.
NeurIPS, 2019
Self-supervised single-view 3D reconstruction via semantic consistency
X. Li, S. Liu, K. Kim, S. De Mello, V. Jampani, M. Yang, J. Kautz
ECCV, 2021
Hierarchical contrastive motion learning for video action recognition
X. Yang, X. Yang, S. Liu, D. Sun, L. Davis, J. Kautz
BMVC, 2021
Regularizing meta-learning via gradient dropout
H. Tseng, Y. Chen, Y. Tsai, S. Liu, Y. Lin, M. Yang
ACCV, 2020
Self-supervised viewpoint learning from image collections
S. Mustikovela, V. Jampani, S. De Mello, S. Liu, U. Iqbal, C. Rother, J. Kautz
CVPR, 2020
Online Adaptation for Consistent Mesh Reconstruction in the Wild
X. Li, S. Liu, S. De Mello, K. Kim, X. Wang, M. Yang, J. Kautz
NeurIPS, 2020
Learning Propagation for Arbitrarily-structured Data
S. Liu, X. Li, V. Jampani, S. De Mello, J. Kautz
ICCV, 2019
Learning Linear Transformations for Fast Image and Video Style Transfer
X. Li, S. Liu, J. Kautz, M. Yang
CVPR, 2019
Few-Shot Viewpoint Estimation
H. Tseng, S. De Mello, J. Tremblay, S. Liu, S. Birchfield, M. Yang, J. Kautz
BMVC, 2019
Putting Humans in a Scene: Learning Affordance in 3D Indoor Environments
X. Li, S. Liu, K. Kim, X. Wang, M. Yang, J. Kautz
CVPR, 2019
SCOPS: Self-Supervised Co-Part Segmentation
W. Hung, V. Jampani, S. Liu, P. Molchanov, M. Yang, J. Kautz
CVPR, 2019
Switchable temporal propagation network
S. Liu, G. Zhong, S. De Mello, J. Gu, V. Jampani, M. Yang, J. Kautz
ECCV, 2018
Rendering Portraitures from Monocular Camera and Beyond
X. Xu, D. Sun, S. Liu, W. Ren, Y. Zhang, M. Yang, J. Sun
ECCV, 2018
Context-Aware Synthesis and Placement of Object Instances
D. Lee, S. Liu, J. Gu, M. Liu, M. Yang, J. Kautz
NeurIPS, 2018
Learning Video-Story Composition via Recurrent Neural Network
G. Zhong, Y. Tsai, S. Liu, Z. Su, M. Yang
WACV, 2018
Learning Dual Convolutional Neural Networks for Low-Level Vision
J. Pan, S. Liu, D. Sun, J. Zhang, Y. Liu, J. Ren, Z. Li, J. Tang, H. Lu, Y. Tai, M. Yang
CVPR, 2018
Learning Affinity via Spatial Propagation Networks
S. Liu, S. De Mello, J. Gu, G. Zhong, M. Yang, J. Kautz
NeurIPS, 2017
Face Parsing via Recurrent Propagation
S. Liu, D. Yi, Z. Lei, S. Li
BMVC, 2017
Unsupervised Domain Adaption for Face Recognition in Unlabeled Videos
K. Sohn, S. Liu, G. Zhong, X. Yu, M. Yang, M. Chandraker
ICCV, 2017
Learning to Segment Instances in Videos with Spatial Propagation Network
J. Cheng, S. Liu, S. De Mello, J. Gu, J. Kautz, S. Wang, M. Yang
CVPR workshop, 2017
Generative Face Completion
Y. Li, S. Liu, J. Yang, M. Yang
CVPR, 2017
Learning Recursive Filters for Low-Level Vision via a Hybrid Neural Network
S. Liu, J. Pan, M. Yang
ECCV, 2016
Deep Cascaded Bi-Network for Face Hallucination
S. Zhu, S. Liu, C. Loy
ECCV, 2016
Multi-Objective Convolutional Learning for Face Labeling
S. Liu, J. Yang, C. Huang, M. Yang
CVPR, 2015
Compressed Face Hallucination
S. Liu, M. Yang
ICIP, 2014
Structured Face Hallucination
C. Yang, S. Liu, M. Yang
CVPR, 2013