Researchers at CU Boulder are leading four new NSF-funded projects that are exploring the safety and security of autonomous systems, including those used in self-driving vehicles.

The work is part of an international effort to address the significant safety and security obstacles to widespread adoption of these systems in the very near future.

Assistant Professor Majid Zamani is leading several of these projects within the Department of Computer Science. He is also part of the Autonomous Systems Interdisciplinary Research Theme in the college. He said this cluster of projects all address safety and security but embrace and apply knowledge from different fields such as control theory, formal methods and machine learning.

“One of the projects looks at how to prevent outside intruders from gaining private information about autonomous systems through their sensor measurements,” Zamani said. “Another ensures the actual auto-pilot systems–the embedded control software–work safely as intended, both in calm and warm days in Arizona and in snowy weather in Michigan, by embracing ideas from transfer learning.”

Assistant Professor Ashutosh Trivedi is also heavily involved in the work, leading one of the projects that looks at machine learning techniques for creating foolproof safety systems. A member of the research theme as well, he said the answers that will come out of this kind of work over the next three years will have many applications for aerospace systems and more tangible aspects of everyday life.

“The safety and security of cyber-physical systems will eventually go well beyond the autonomous cars we are working with here,” Trivedi said. “These systems are the technological backbone of the increasingly interconnected and smart world where a design fault or security vulnerability can be catastrophic to the system, to the user or to those around them. This work has implications for wearable and implantable medical devices, smart infrastructure and connected communities, to name only a few areas.”

Here are the project details including staffing and funding totals: 

Project Title: CPS: Medium: Correct-by-Construction Controller Synthesis using Gaussian Process Transfer Learning
Principal Investigator: Majid Zamani, Department of Computer Science
Co-Principal Investigators: Morteza Lahijanian and Eric Frew, Department of Aerospace Engineering Sciences
Amount: $1,200,000
About: This project explores improvements to embedded control software for safety-critical cyber-physical systems in autonomous vehicles. Embedded control software forms the main core of autonomous systems wherein software components interact with physical systems such as traffic networks and power networks to name a few. These systems often have complex dynamics that are difficult to predict and ensure when it comes to safe operation. This project investigates a novel correct-by-construction controller synthesis scheme for these systems by embracing ideas from Gaussian processes. If successful, this could allow safety controllers developed for one type of autonomous vehicle to be transferred to another of a wholly new type – or for use in a new environment all together ¬– while still ensuring the original safety guarantee. This would save time on production and will be tested on underwater and aerial vehicles with an eye to future applications outside of self-driving cars.

Project Title: Secure-by-Construction Controller Synthesis for Cyber-Physical Systems
Principal Investigator: Majid Zamani, Department of Computer Science
Co-Principal Investigator: Ashutosh Trivedi, Department of Computer Science
Amount: $387,640
About: The security of autonomous vehicles from outside intruders is a new and growing area of research which has previously lagged behind more obvious safety concerns around the car’s actual operation. But because these vehicles collect and use a tremendous amount of personal data, they are appealing targets for hackers who can intrude through internet connected systems or other linked personal devices. From there they can deduce private internal information such as destination history or even potentially tamper with the vehicle. The proposed research aims to address this in parallel with the physical safety of the vehicle on the road. The ultimate goal is to develop algorithmic techniques and computational tools for constructing discrete controllers guaranteeing both safety and privacy properties, which are then automatically refined as hybrid controllers for the original systems. Doing would speed up overall development as security features would not have to be added on after the systems are already fully designed. This should also allow for more overlapping protection in both the physical and security arenas.

Project Title: SHF: Small: Omega-Regular Objectives for Model-Free Reinforcement Learning
Principal Investigator: Ashutosh Trivedi, Department of Computer Science
Co-Principal Investigator: Fabio Somenzi, Department of Electrical, Computer and Energy Engineering
Amount: $500,000
About: In reinforcement learning, software agents rely on and receive rewards that promote the achievement of given objectives or tasks. Scalar rewards can be used to reinforce the desired behavior, like keeping the car on the road and between the lines or withheld when drifting out of bounds. This machine learning technique has been demonstrated to be effective in many autonomous systems such as self-driving cars and manufacturing systems as well as other aspects of modern life such as social networks and internet connected devices. However, their integration into safety-critical settings for self-driving cars requires a new set of methods to ensure the decisions they ultimately make are the right ones. This project develops a rigorous approach to the design and verification of reinforcement learning-enabled systems that addresses issues of safety, efficiency, and scalability. This project also aims to develop an open source tool to create reinforcement learning algorithms to that end.

Project Title: An Entropy Approach to Invariance and Reachability of Uncertain Control Systems with Limited Information
Individual Principal Investigator: Majid Zamani, Department of Computer Science
Amount: $379,327
About: This project explores how autonomous vehicles can cope with limited bandwidth while interacting with cloud-based servers to share information and at the same time ensure their physical safety. Currently, communication systems, digital sensors, and microprocessors are being used by the car’s embedded control systems to respond to the needs and requests of external traffic network or the needs of the internal engine control for example. The interplay between those safety and reliability requirements – and the car’s ability to respect or enforce them – is key to keeping the vehicle safe on the road. While possible now, there is a finite amount of communication bandwidth available for maintaining that balance and the number of cars looking to use it is expected to go up over time. This research aims to establish the fundamental minimum data rates – or bandwidth – needed to make sure that the safety of the vehicles are not compromised. The results of this project will also enable the first step towards the efficient deployment of many innovative applications including underwater vehicles, sensor networks, and industrial control networks.