Hello, I’m

Yaofeng "Desmond" Zhong

Founding Research Scientist @ Breakpoint AI

About me

I'm Yaofeng "Desmond" Zhong, a founding research scientist at Breakpoint AI. Previous, I was a research scientist in the physics-informed AI research group at Siemens Technology. I'm broadly interested in computer vision and scientific machine learning. In particular, my research interest lies in the intersection of deep learning, dynamical systems and control. I've been working on incorporating physics domain knowledge into machine learning as well as differentiable simulation for optimal control. I also make relevant research code public since I believe the research community thrives through open-source. At Siemens, I've contributed to the research and development of SIMATIC Robot Pick AI, an AI-based computer vision software that enables robots for intelligent bin picking.

I received my Ph.D. from Princeton University, advised by Prof. Naomi Leonard. My dissertation investigates cascade dynamics, decision making and physics-constrained machine learning. While at Princeton, I co-initiated the Robotics Reading Group. Prior to Princeton, I received my B.Eng. from Tsinghua University. My undergraduate thesis is on adaptive remeshing in finite element method, supervised by Prof. Hongzhi Zhong. While at Tsinghua, I led a team to participate in COMAP's international Mathematical Contest in Modeling (MCM) and my team was designated as Outstanding Winner.

Production Projects

SIMATIC Robot Pick AI @ Siemens

Our AI-driven vision software enables robots to perform previously manual-only tasks such as picking unknown objects, empowering our industrial partners to unlock unprecedented levels of efficiency and productivity.

Product Page

Prototype Projects

Towards Industrial Metaverse @ Siemens

We developed an early prototype of a surrogate machine learning model for computational fluid dynamics (CFD) simulation. We designed and implemented surrogate models for predicting air flow around a race car (SimRod). The air flow computed from the surrogate model is visualized along with the air flow computed from Siemens Simcenter STAR-CCM+. The whole 3D scene is assembled in NVIDIA Omniverse.

Research Projects

Improving Gradient Computation

Differentiable simulation enables gradients to be back-propagated through physics simulations. Then we can learn the dynamics and properties of a physics system by gradient-based optimization. However, differentiable simulation at its current stage might provide wrong gradients that deteriorate its performance in learning tasks, especially when contact and collision exist. In this work, we propose to improve gradient computation by continuous collision detection.

Blog L4DC 2023 arXiv

Benchmark: Differentiable Simulators

Many open-source differentiable physics simulators appear in recent years: diffTaichi, Google Brax, Nvidia Warp, Stanford Nimble, DiffCoSim, etc. Do the calculated gradients agree with one another? We find that they don't agree, which affects their performance on optimiazation and learning tasks.

ICML 2022 (AI4Science)arXiv

EMVLight

Emergency vehicles (EMVs) play a crucial role in responding to time-critical events. To reduce the travel time of EMVs, techniques of route optimization and traffic signal control has been studied as two separate modules. However, the routing problem and the traffic signal control problem are coupled and traditional solution usually increases the travel time of other vehicles significantly. Thus, we propose EMVLight, a decentralized reinforcement learning framework, to tackle these two problems at the same time.

AAAI 2022 arXiv

Differentiable Contact Models

One of the limitation of Lagrangian/Hamiltonian neural networks is that they cannot handle dynamical systems with discontinuity. A common type of discontinuity in physical systems is collision and contact. In order to model collision and contact, we extend Lagrangian/Hamiltonian neural networks with a differentiable contact model. The proposed simulation pipeline is end-to-end differentiable, so that we can simutaneously learn unknown mass, energy, coefficient of friction and restitution. The learned dynamics can also be used as a differentiable physics simulator for downstream gradient-based optimization tasks.

NeurIPS 2021 arXiv

Multi-Robot Task Allocation

We design a decision-making algorithm that defines how the robots select tasks to perform and how they repeatedly revise their task selections in response to changes in the environment. Through experiments with a multi-robot trash collection application, we assess the algorithm’s responsiveness to changing environments and resilience to failure of individual robots.

ICRA 2021 arXiv

Learning Dynamics from Images

How to learn interpretable dynamics from image sequences? As an attempt to answer this question, we introduce an unsupervised neural network model that learns Lagrangian dynamics from images, with interpretability that benefits prediction and control. Our proposed model infers Lagrangian dynamics on generalized coordinates that are simultaneously learned with a coordinate-aware variational autoencoder (VAE).

NeurIPS 2020 arXiv

Symplectic ODE-Net

How to incorporate physics domain knowledge into deep learning so that the learned physics model obeys the physics law? In this work, we propose Symplectic ODE-Net to learn Hamiltonian dynamics with control by designing the computation graph. In this way, we can use less data to train a model that is interpretable and generalizes better.

ICLR 2020 arXiv