According to a study published this week in the IEEE Transactions on Pattern Analysis and Machine Intelligence, the method uses control theory to steer sampling trajectories without retraining pretrained models.
The Challenge of Hard Constraints in Generative AI
Pretrained generative AI models, including diffusion models like Stable Diffusion and flow-matching models like FLUX, transform random noise to create new data. While these models excel at producing answers that nearly satisfy queries, high-stakes environments demand absolute compliance with nonnegotiable limits. According to MIT researchers, a path that is nearly correct in robot path planning on a crowded factory floor can still result in a collision with a human co-worker.
To enforce rules, users frequently apply projection-based sampling. This approach repeatedly forces partial solutions—known as intermediate samples—to meet strict requirements during the generation process. However, according to lead author Zeyang Li, constraining the entire generation process prevents models from reaching better final solutions. Li notes that projection-based methods typically focus solely on hard constraints, missing opportunities to improve other qualities like reducing a robot’s trajectory length.
How HardFlow Uses Optimal Control Theory
To give models more freedom while guaranteeing compliance, the MIT team developed HardFlow. According to the research paper, HardFlow reformulates hard-constrained sampling as a trajectory-optimization problem by borrowing tools from optimal control. Navid Azizan, senior author of the study and Alfred H. and Jean M. Hayes Career Development Associate Professor, states that control theory provides a powerful framework for making optimal corrections.
Because solving trajectory-optimization problems around enormous neural networks with hundreds of interconnected layers presents significant computational hurdles, the researchers leveraged flow-matching model structures. They decomposed the problem into a sequence of smaller, single-step subproblems. By applying systematic transformations and approximations, they derived an efficient, scalable algorithm that operates directly at deployment time without retraining.
Experimental Results Across Robotics and Computer Vision
Across experiments spanning robotic manipulation, maze navigation, and text-guided image editing, HardFlow consistently satisfied required constraints while outperforming baseline methods. According to the study, the framework enabled a robotic manipulator to avoid obstacles while identifying the quickest path to a target object, whereas competing methods resulted in collisions or longer travel times.
In addition to superior constraint satisfaction and solution quality, HardFlow demonstrated computation times comparable to or lower than existing methods. Because the framework treats task execution as an optimization problem, it can simultaneously handle multiple goals, such as finding a collision-free path that also minimizes total distance.
Frequently Asked Questions
What is HardFlow?
HardFlow is an algorithm developed by MIT researchers that steers the sampling process of generative AI models. It enforces nonnegotiable safety and physical constraints on the final output rather than at every intermediate step.
Do pretrained models need retraining to use HardFlow?
No. HardFlow is a plug-and-play technique that operates at deployment time, meaning it can be applied to existing pretrained generative models without requiring retraining.
Who led the research?
The research was led by Zeyang Li, a graduate student in mechanical engineering and the Laboratory for Information and Decision Systems (LIDS), alongside graduate student Kaveh Alim and senior author Navid Azizan at MIT.
Keep reading