By combining machine learning and control theory, Poveda expands field of autonomy

Jorge Poveda has been working for years to improve the ways autonomous systems overcome problems they encounter on the job. It’s complex work that could impact our everyday life, like our daily commute in autonomous cars, to more imaginative goals like swarms of robots working in unison.

By combining concepts from artificial intelligence and machine learning with well-known control theories, he may have found a new approach that could prove key to moving forward on many fronts.

Poveda is an assistant professor in the Department of Electrical, Computer and Energy Engineering. His new approaches and concepts – developed in collaboration with researchers from Harvard, Georgia Tech, and several schools in the University of California system – were recently published in prestigious journals such as IEEE Transactions on Automatic Control and Automatica. 

Broadly, Poveda’s group designs algorithms to control machines through real-time data collected via feedback loops, as well as historic data. Picture an autonomous robot scanning a room for new information about its location while also checking those results against its previous experiences in the same or similar spaces, as well as considering its size and location, before any taking action. 

This kind of approach is beneficial for many reasons, but primarily because it uses algorithms that are both fast and resistant to measurement “noise” that can lead to operational problems as decisions are made on incorrect or incomplete information. 

“Adaptive systems rely on intrinsic balances between ‘exploration and exploitation,’ where exploration is usually related to the experience of the system and the knowledge gathered from past actions, and exploitation concerns how to use this knowledge to generate a new control action,” Poveda said. 

The team’s recent papers have shown that it is possible to accelerate the exploitation part, while minimizing the undesirable aspects that come with exploration, he added. “All without losing the critical stability and robustness properties needed for the safe operation of the systems.” 

Poveda said this approach is appealing for applications within a variety of cyber-physical systems, in which software and hardware are deeply intertwined. This includes things like power grids, transportation systems and robotics. He added that the result is a family of algorithms able to operate faster, significantly improving the learning and adaptability capabilities.

“It also allows for minimal human supervision and improves the efficiency overall,” Poveda said. “And it is different from more traditional ‘model-based’ approaches that have been used in the past. Those use detailed mathematical and physics-based models of both the actual system and environment it is operating in to ensure they don’t get stuck or confused about what we are asking them to do on their own.” 

Poveda said his research combines ideas from optimization, adaptive control and machine learning – a branch of artificial intelligence in which algorithms improve automatically through experience over time – with hybrid control theoretical tools. The machine learning aspect of the work pulls from efforts and algorithms pioneered in social media and search engines while monitoring web comments or improving search results.

“Until recently, those approaches have not been applied to physical systems because just a little noise in the data can make the algorithms unstable or unpredictable. Which is not as big a deal on social media as it is with a moving autonomous car where there are real safety and operational concerns around decision making,” he said.

Poveda and his team have begun applying these algorithms and techniques to engineering problems in a variety of contexts, including autonomous multi-vehicle systems, smart power grids, and traffic light systems able to autonomously adjust their behavior in real time to minimize congestion. 

They have also looked into possible applications in other energy and battery systems, all in collaboration with industry partners like Mitsubishi, Robert Bosch and with researchers through the Autonomous Systems Interdisciplinary Research Theme in the college. Much of the work has been funded by the National Science Foundation, and Poveda also recently received a seed grant through the Research & Innovation Office on campus to continue the effort in other directions, including game-theoretic settings.

“This work illustrates what I see as one of my main goals at CU Boulder: Teaching my students how to create innovative and rigorous theoretical frameworks for the design of practical control algorithms that can have an impact in modern engineering applications,” he said.