in Cluttered Environment
Lei Zhang1,2, Kaixin Bai1,2, Qiang Li1†, Zhaopeng Chen2†, Jianwei Zhang1
1 TAMS (Technical Aspects of Multimodal Systems), Department of Informatics, University of Hamburg, Germany
2 Agile Robots AG, Munich, Germany
† corresponding authors
Fig. 1 Grasping cables from cluttered scenes exploiting the grasping samples approach. The grasp candidates are obtained considering the force closure principle. a) CG-CNN estimates grasp quality of grasp candidates. Positive grasp candidates are in green: quality > 0.5; Negative grasp candidates are in red : quality < 0.5. b) Optimal grasp candidate. c) Physics simulation for grasping.
Abstract
We introduce a Cable Grasping-Convolutional Neural Network (CG-CNN) designed to facilitate robust cable grasping in cluttered environments. Utilizing physics simulations, we generate an extensive dataset that mimics the intricacies of cable grasping, factoring in potential collisions between cables and robotic grippers. We employ the Approximate Convex Decomposition technique to dissect the non-convex cable model, with grasp quality autonomously labeled based on simulated grasping attempts. The CG-CNN is refined using this simulated dataset and enhanced through domain randomization techniques. Subsequently, the trained model predicts grasp quality, guiding the optimal grasp pose to the robot’s controller for execution. Grasping efficacy is assessed across both synthetic and real-world settings. Given our model’s implicit collision sensitivity, we achieved commendable success rates of 92.3% for known cables and 88.4% for unknown cables, surpassing contemporary state-of-the-art approaches.
Videos of Experiments
Architecture
Fig. 2 illustrates the architectural framework of the cable grasping method integrating CG-CNN. CG-CNN is designed to acquire the capability to assess grasp quality based on synthetic depth data generated within a simulated environment. The determination of grasp quality is accomplished through the execution of physics-based grasping simulations conducted within complex and cluttered settings. To assess the performance of CG-CNN during its training phase, a total of 100 grasping simulation experiments are conducted within the simulation environment every 10 epochs, and the success rate of grasping within this simulated context is chosen as the evaluation metric. Ultimately, the trained model is deployed for real-world cable grasping within cluttered environments.
Experiments Results
Synthetic Experiments: Successful Cable Grasping
Success Grasp 01
Success Grasp 02
Success Grasp 03
Examples of Failed Grasps
Failed Grasp 01
Failed Grasp 02
Failed Grasp 03
Examples of Cable Grasping in Real-World Scenes featuring various numbers of cables
Comparison Experiments with Dex-Net, GraspNet-1Billion, Contact-GraspNet
Failure: Multi-Cable Grasping (Dex-Net)
Failure: Multi-Cable Grasping (Dex-Net)
Failure: Multi-Cable Grasping (Contact-GraspNet)
Failure: Dropping (CG-CNN)
Failure: Dropping (Contact-GraspNet)
Failure: Non Force-Closure Grasp (GraspNet-1Billion)
Failure: Non Force-Closure Grasp (GraspNet-1Billion)
Failure: Multi-Cable Grasping (Dex-Net)
Failure: Multi-Cable Grasping (Dex-Net)
Failure: Multi-Cable Grasping (Dex-Net)
Failure: Multi-Cable Grasping (Contact-GraspNet)
Failure: Dropping (CG-CNN)
Failure: Dropping (Contact-GraspNet)
Failure: Non Force-Closure Grasp (GraspNet-1Billion)
Failure: Non Force-Closure Grasp (GraspNet-1Billion)
Failure: Multi-Cable Grasping (Dex-Net)
Unknown Cable Grasping using CG-CNN
Datasets
We provided examples of cable grasping dataset for the community for research usage. The example datasets contain of the cable grasping dataset collected from simulation environment featuring various numbers of cables, RGB, Depth, Segmentation of the entire scenarios.
Cable Grasping Datasets collected from Simulation:
Cable Num 1. (Google Drive, 237MB)
Cable Num 5. (Google Drive, 216MB)
Cable Num 10. (Google Drive, 262MB)
Cable Num 15. (Google Drive, 341MB)
RGB, Depth, Segmentation Data of Entire Scene:
Cable Num 1. (Google Drive, 435MB)
Cable Num 5. (Google Drive, 795MB)
Cable Num 10. (Google Drive, 1.28GB)
Cable Num 15. (Google Drive, 1.99GB)
Bibtex
title={A Collision-Aware Cable Grasping Method in Cluttered Environment},
author={Zhang, Lei and Bai, Kaixin and Li, Qiang and Chen, Zhaopeng and Zhang, Jianwei},
journal={arXiv preprint arXiv:2402.14498},
year={2024}
}
Contact
Feel free to directly contact Lei Zhang (lei.zhang-1@studium.uni-hamburg.de) about any questions regarding the example datasets.
Acknowledgements
Special thanks to Surya Prabhakaran for software setup, Guowen Huang, and Jiaxin Hu for valuable experimental supports, Huajian Zeng for providing helpful suggestions about writing. This research was supported by DFG, NSFC, DEXMAN project, and partially supported by European projects and ULTRACEPT.