Large language models (LLMs) have rich physical knowledge about worldly objects, but cannot directly reason robot grasps for them. Paired with open-world localization and pose estimation (left), our method (middle), queries LLMs for the salient physical characteristics of mass, friction, and compliance as the basis for an adaptive grasp controller. DeliGrasp policies successfully grasp delicate and deformable objects

DeliGrasp: Inferring Object Mass, Friction, and Compliance with LLMs for Adaptive and Minimally Deforming Grasp Policies

March 12, 2024

Large language models (LLMs) can provide rich physical descriptions of most worldly objects, allowing robots to achieve more informed and capable grasping. We leverage LLMs’ common sense physical reasoning and code-writing abilities to infer an object’s physical characteristics—mass, friction coefficient, and spring constant —from a semantic description, and then translate...

A multifunctional soft robotic shape display with high-speed actuation, sensing, and control

A multifunctional soft robotic shape display with high-speed actuation, sensing, and control

July 31, 2023

Shape displays that actively manipulate surface geometry are an expanding robotics domain with applications to haptics, manufacturing, aerodynamics, and more. However, existing displays often lack high-fidelity shape morphing, high-speed deformation, and embedded state sensing, limiting their potential uses. Here, we demonstrate a multifunctional soft-shape display driven by a 10 × 10 array...

The Versand

A versatile robotic hand with 3D perception, force sensing for autonomous manipulation

July 10, 2023

We describe a force-controlled robotic gripper with built-in tactile and 3D perception. We also describe a complete autonomous manipulation pipeline consisting of object detection, segmentation, point cloud processing, force-controlled manipulation, and symbolic (re)-planning. The design emphasizes versatility in terms of applications, manufacturability, use of commercial off-the-shelf parts, and open-source software...

Transformer architecture

Early failure prediction during robotic assembly using Transformers

July 10, 2023

Peg-in-hole assembly of tightly fitting parts often requires multiple attempts. Parts need to be put together by performing a wiggling motion of undetermined length and can get stuck, requiring a restart. Recognizing unsuccessful insertion attempts early can help in reducing the makespan of the assembly. This can be achieved by...

Different grasps that require palmar sensing

Distributed Tactile Sensors for Palmar Surfaces of Prosthetic Hands

May 19, 2023

Sensory feedback provided by prosthetic hands shows promise in increasing functional abilities and promoting embodiment of the prosthetic device. However, sensory feedback is limited based on where sensors are placed on the prosthetic device and has mainly focused on sensorizing the fingertips. Here we describe distributed tactile sensors for the...

A complete peg-in-hole assembly sequence: A The bearing is presented in a 3Dprinted jig, B The bearing is picked up by the robot and transported to the assembly plate C. Force and torque measurements are used to D locate the hole E and complete insertion. Insertion failure due to misalignment F. Friction with the edge of the hole has caused the twisting action to pull the bearing further from the hole center.

Optimal decision making in robotic assembly and other trial-and-error tasks

Jan. 25, 2023

Uncertainty in perception, actuation, and the environment often require multiple attempts for a robotic task to be successful. We study a class of problems providing (1) low-entropy indicators of terminal success / failure, and (2) unreliable (high-entropy) data to predict the final outcome of an ongoing task. Examples include a...

Embedded Magnetic Sensing for Feedback Control of Soft HASEL Actuators

Embedded Magnetic Sensing for Feedback Control of Soft HASEL Actuators

Sept. 10, 2022

The need to create more viable soft sensors is increasing in tandem with the growing interest in soft robots. Several sensing methods, like capacitive stretch sensing and intrinsic capacitive self-sensing, have proven to be useful when controlling soft electro-hydraulic actuators, but are still problematic. This is due to challenges around...

Introduction to Autonomous Robots: Mechanisms, Sensors, Actuators, and Algorithms

Introduction to Autonomous Robots: Mechanisms, Sensors, Actuators, and Algorithms

June 2, 2022

Textbooks that provide a broad algorithmic perspective on the mechanics and dynamics of robots almost unfailingly serve students at the graduate level. Introduction to Autonomous Robots offers a much-needed resource for teaching third- and fourth-year undergraduates the computational fundamentals behind the design and control of autonomous robots. The authors use...

Atoms assembled into a chemical molecule using augmented reality to show bonds.

Augmented reality for human–swarm interaction in a swarm-robotic chemistry simulation

May 3, 2022

We present a novel augmented reality (AR) framework to show relevant information about swarm dynamics to a user in the absence of markers by using blinking frequency to distinguish between groups in the swarm. In order to distinguish between groups, clusters of the same group are identified by blinking at...

Motion sequence of the electrohydraulic rolling soft wheel around a pivot on a square platform

Electro-Hydraulic Rolling Soft Wheel: Design, Hybrid Dynamic Modeling, and Model Predictive Control

May 2, 2022

Locomotion through rolling is attractive compared to other forms of locomotion thanks to uniform designs, high degree of mobility, dynamic stability, and self-recovery from collision. Despite previous efforts to design rolling soft systems, pneumatic and other soft actuators are often limited in terms of high-speed dynamics, system integration, and/or functionalities...