Semi-automated excavation system for partially buried objects using stereo vision-based three-dimensional localization

We present a semi-automated excavation system for partially buried objects in unstructured environments. This system is a prototype of an excavation system for abandoned chemical weapons. It integrates three-dimensional (3-D) sensing by a stereo camera system, 3-D object localization, verification by a human operator and robot manipulation. The 3-D object localization method used consists of three steps. (i) Candidate regions are extracted from a range image obtained by an area-based stereo-matching method. (ii) For each region, multiple hypotheses for the position and orientation are generated for each object model. (iii) Each hypothesis is verified and improved by an iterative method. The operator verifies the object localization results and then selects one of the objects as the best object that is suitable for grasping by the robot. The robot grasps objects based on the object localization result. Experiments demonstrate the effectiveness of the proposed excavation system in various unstructured environments, which include objects that are partially buried, inclined, overlapping each other and that have been sprinkled with water.