Zhongdao Wang

I am currently a researcher at Noah's Ark Lab, Huawei. Before joining Huawei, I recieved my Ph.D. degree from the Department of Electronic Engineering, Tsinghua Univerisity, and my bechalor's degree from the Department of Physics, Tsinghua University. During my Ph.D. time, I have been fortunate to work closely with Prof. Liang Zheng, Dr. Yifan Sun, Dr. Luca Bertinetto, and Prof. Hengshuang Zhao

Email  /  CV  /  Google Scholar  /  Github

In the long term, my research interest lies in developing general embodied intelligent agents (an autonomous driving car can be a simplified example). I have a faith that the emergence of general machine intelligence requires observing, interacting with, and learning from the physical world. In the future, embodied agents may be able to train themselves on the edge, and possibly explore the world in clusters, learning together to form a foundational intelligent model (somewhat like the Sibyl system in the Sci-Fi anime Psycho-Pass, but an enhanced version that can interact with the world). Such a strong intelligent model may looks a little bit evil, but with efforts I believe we can make it used for good purposes :)

In the short term, my research interest lies in perception algorithms for automonous driving. Including but not limited to 2D/3D object detection, segmentation, tracking, 3D reconstruction, and offboard pereception (for auto-labeling). I have several openings for self-motivated reserach interns on these topics, please feel free to drop me an e-mail.

Previously, I did research on representation learning and object tracking. Representative works include JDE and UniTrack for object tracking, the CycAs series [v1,v2,v3] for person re-identification, and Circle Loss, D-Softmax Loss for metric learning.

Generative AI

PixArt-alpha is the first DiT-based T2I diffusion model whose image generation quality is competitive with SoTA image generators (e.g., Imagen, SDXL, and even Midjourney), reaching near-commercial application standards. With efficient designs, our training only takes 10.8% of Stable Diffusion v1.5's cost in terms of GPU days. Many recent Text-to-Video models such like Latte, Open-Sora, Open-Sora-Plan are inspired by or built upon PixArt-alpha.

An update version of PixArt-alpha that is capable of generating ultra-high resolution (up to 4K) images in super realistic quality.

Perception (object detection/segmentation/tracking)

By incorporating the appearance embedding model into the detector, we introduce JDE, the first open-source real-time multiple object trackor with a running speed of 22 ~ 38 FPS. This speed takes all the steps into account, including detection, appearance embedding extraction and association. Code is released! If you are looking for an easy-to-use and fast pedestrian detector/tracker, JDE is a good option!

Object tracking has been fragmented into multiple different experimental setups due to different use cases and benchmarks. We investigate if it is possible to address the various setups with a single, shared apperance model.

MetaBEV is a 3D Bird-Eye's View (BEV) perception model that is robust to sensor missing/failure, supporting both single modality mode (camera/lidar) and multi-modal fusion mode with strong performance.

Representation learning / re-identification

CycAs is the fisrt self-supervised method for learning re-identifiable features. We use videos as input, the supervision signal is cycle consistency emerging from instance association in a forward-backward cycle. In CycAs v1, we train and show promising results on (mostly small) canonical datasets; In CycAs v2, we collect a large amount of video data and train a much more generalizable model with improved building blocks. The zero-shot transfer ablility is suprisingly good, with a rank-1 accuracy of 82.2% on Market-1501.

A new self-supervised learning objective further improves upon CycAs v2, refreshing the zero-shot transfer performance on Market-1501 to 87.1% rank-1 accuracy, already on par with supervised models trained with in-domain data.

A unified perspective for proxy-based and pair-based metric learning.

Self-supervised learning via Maximum Entropy Coding improves generalization ability of the learned representation on a variaty of down-stream tasks.

(* Interns & Students, ⁺ Equal contribution)

Design and source code modified based on Jon Barron's website.