A Collision-Aware Cable Grasping Method
in Cluttered Environment

Lei Zhang^1,2, Kaixin Bai^1,2, Qiang Li^1†, Zhaopeng Chen^2†, Jianwei Zhang¹

¹ TAMS (Technical Aspects of Multimodal Systems), Department of Informatics, University of Hamburg, Germany
² Agile Robots AG, Munich, Germany

^† corresponding authors

Github Webpage Code Videos Example Dataset

Fig. 1 Grasping cables from cluttered scenes exploiting the grasping samples approach. The grasp candidates are obtained considering the force closure principle. a) CG-CNN estimates grasp quality of grasp candidates. Positive grasp candidates are in green: quality > 0.5; Negative grasp candidates are in red : quality < 0.5. b) Optimal grasp candidate. c) Physics simulation for grasping.

Abstract

We introduce a Cable Grasping-Convolutional Neural Network (CG-CNN) designed to facilitate robust cable grasping in cluttered environments. Utilizing physics simulations, we generate an extensive dataset that mimics the intricacies of cable grasping, factoring in potential collisions between cables and robotic grippers. We employ the Approximate Convex Decomposition technique to dissect the non-convex cable model, with grasp quality autonomously labeled based on simulated grasping attempts. The CG-CNN is refined using this simulated dataset and enhanced through domain randomization techniques. Subsequently, the trained model predicts grasp quality, guiding the optimal grasp pose to the robot’s controller for execution. Grasping efficacy is assessed across both synthetic and real-world settings. Given our model’s implicit collision sensitivity, we achieved commendable success rates of 92.3% for known cables and 88.4% for unknown cables, surpassing contemporary state-of-the-art approaches.

Videos of Experiments

Architecture

Fig. 2 illustrates the architectural framework of the cable grasping method integrating CG-CNN. CG-CNN is designed to acquire the capability to assess grasp quality based on synthetic depth data generated within a simulated environment. The determination of grasp quality is accomplished through the execution of physics-based grasping simulations conducted within complex and cluttered settings. To assess the performance of CG-CNN during its training phase, a total of 100 grasping simulation experiments are conducted within the simulation environment every 10 epochs, and the success rate of grasping within this simulated context is chosen as the evaluation metric. Ultimately, the trained model is deployed for real-world cable grasping within cluttered environments.

Experiments Results

Synthetic Experiments: Successful Cable Grasping

Success Grasp 01

Success Grasp 02

Success Grasp 03

Examples of Failed Grasps

Failed Grasp 01

Failed Grasp 02

Failed Grasp 03

Examples of Cable Grasping in Real-World Scenes featuring various numbers of cables

Comparison Experiments with Dex-Net, GraspNet-1Billion, Contact-GraspNet

Failure: Multi-Cable Grasping (Dex-Net)

Failure: Multi-Cable Grasping (Contact-GraspNet)

Failure: Dropping (CG-CNN)

Failure: Dropping (Contact-GraspNet)

Failure: Non Force-Closure Grasp (GraspNet-1Billion)

Failure: Multi-Cable Grasping (Dex-Net)

Unknown Cable Grasping using CG-CNN

Code

Coming soon.

Installation

conda create -n cable-grasping python=3.10.9
conda activate cable-grasping
conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch
pip install tensorboardX
pip install matplotlib
pip install torchsummary
pip install pybullet
pip install gym
pip install cup
pip install opencv-python
pip install scikit-image
pip install ruamel.yaml
pip install open3d
pip install attrdict
pip install h5py

Datasets

We provided examples of cable grasping dataset for the community for research usage. The example datasets contain of the cable grasping dataset collected from simulation environment featuring various numbers of cables, RGB, Depth, Segmentation of the entire scenarios.

Cable Grasping Datasets collected from Simulation:
    Cable Num 1. (Google Drive, 237MB)
    Cable Num 5. (Google Drive, 216MB)
    Cable Num 10. (Google Drive, 262MB)
    Cable Num 15. (Google Drive, 341MB)
RGB, Depth, Segmentation Data of Entire Scene:
    Cable Num 1. (Google Drive, 435MB)
    Cable Num 5. (Google Drive, 795MB)
    Cable Num 10. (Google Drive, 1.28GB)
    Cable Num 15. (Google Drive, 1.99GB)

Bibtex

@article{zhang2024collision,
    title={A Collision-Aware Cable Grasping Method in Cluttered Environment},
    author={Zhang, Lei and Bai, Kaixin and Li, Qiang and Chen, Zhaopeng and Zhang, Jianwei},
    journal={arXiv preprint arXiv:2402.14498},
    year={2024}
}

Contact

Feel free to directly contact Lei Zhang (lei.zhang-1@studium.uni-hamburg.de) about any questions regarding the example datasets.

Acknowledgements

Special thanks to Surya Prabhakaran for software setup, Guowen Huang, and Jiaxin Hu for valuable experimental supports, Huajian Zeng for providing helpful suggestions about writing. This research was supported by DFG, NSFC, DEXMAN project, and partially supported by European projects and ULTRACEPT.