IEEE.org | IEEE Xplore Digital Library | IEEE Standards | IEEE Spectrum | More Sites
Call for Award Nominations
More Info
Since 1987 I have highlighted how attempts to deploy autonomous capabilities into complex, risky worlds of practice have been hampered by brittleness — descriptively, a sudden collapse in performance when events challenge system boundaries. This constraint has been downplayed on the grounds that the next advance in AI, algorithms, or control theory will lead to the deployment of systems that escape from brittle limits. However, the world keeps providing examples of brittle collapse such as the 2003 Columbia Space Shuttle accident or this years’ Texas energy collapse. Resilience Engineering, drawing on multiple sources including safety of complex systems, biological systems, & joint human-autonomy systems, discovered that (a) brittleness is a fundamental risk and (b) all adaptive systems develop means to mitigate that risk through sources for resilient performance.
The fundamental discovery, covering biological, cognitive, and human systems, is that all adaptive systems at all scales have to possess the capacity for graceful extensibility. Viability of a system, in the long run, requires the ability to gracefully extend or stretch at the boundaries as challenges occur. To put the constraint simply, viability requires extensibility, because all systems have limits and regularly experience surprise at those boundaries due to finite resources and continuous change (Woods, 2015; 2018; 2019).
The problem is that development of automata consistently ignores this constraint. As a result, we see repeated demonstrations of the empirical finding: systems-as-designed are more brittle than stakeholders realize, but fail less often as people in various roles adapt to fill shortfalls and stretch system performance in the face of smaller & larger surprises. (Some) people in some roles are the ad hoc source of the necessary graceful extensibility.
The promise comes from the science behind Resilience Engineering which highlights paths to build systems with graceful extensibility, especially systems that utilize new autonomous capabilities. Even better, designing systems with graceful extensibility draws on basic concepts in control engineering, though these are reframed substantially when combined with findings on adaptive systems from biology, cognitive work, organized complexity, and sociology.
In this talk, I will discuss the problem of interactive learning by discussing how we can actively learn objective functions from human feedback capturing their preferences. I will then talk about how the value alignment and reward design problem can have solutions beyond active preference-based learning by tapping into the rich context available from large language models. In the second section of the talk, I will more generally talk about the role of large pretrained models in today’s robotics and control systems. Specifically, I will present two viewpoints: 1) pretraining large models for downstream robotics tasks, and 2) finding creative ways of tapping into the rich context of large models to enable more aligned embodied AI agents. For pretraining, I will introduce Voltron, a language-informed visual representation learning approach that leverages language to ground pretrained visual representations for robotics. For leveraging large models, I will talk about a few vignettes about how we can leverage LLMs and VLMs to learn human preferences, allow for grounded social reasoning, or enable teaching humans using corrective feedback. Finally, I will conclude the talk by discussing some preliminary results on how large models can be effective pattern machines that can identify patterns in a token invariant fashion and enable pattern transformation, extrapolation, and even show some evidence of pattern optimization for solving control problems.
More information provided here.
