Skip to main content

Grasping With Common Sense using VLMs and LLMs

Deligrasp overview

How to leverage large language models for robotic grasping and code generation

Grasping and manipulation remain a hard, unsolved problem in robotics. Grasping is not just about identifying points where to put your fingers on an object to create sufficient constraints. Grasping is also about applying just enough force to pick up the object with breaking it, while making sure it can be put to its intended use. At the same time, grasping provides critical sensor input to detect what an object is and what its properties are. With mobility essentially solved, grasping and manipulation remains the final frontier in unlocking truely autonomous labor replacements.

Continue reading on Toward Data Science....