Nowadays, autonomous underwater vehicles (AUVs) are expected be one of the essential tools in applications such as inspection of underwater structures (e.g., dams and bridges) and underwater cable tracking . Even though many studies have been conducted and published worldwide, researchers face rapidly increasing demands to expand the roles of AUVs. In spite of developing technologies related to power storage devices, underwater vehicles are especially limited in operations that take longer than the duration supported by the vehicle’s power capacity. A recharging unit with an underwater docking function can enable AUVs to operate for extended periods in the sea independently of a surface vessel, making the docking operation important not only for battery recharging applications but also other applications such as sleeping under the mother ship or new mission downloading. Moreover, the docking capacity can be extended to provide navigation for other underwater vehicles on the way to their own missions. However, a number of challenging issues hinder these applications, which require high accuracy and robustness against disturbances that occur in the underwater environment. To achieve these tasks in underwater vehicles, we have developed a vision-based docking system using stereo vision.
Recently, due to the progress in computer vision, a vision-based system has been highlighted as a promising navigation system. As in land and space systems, numerous studies on underwater vehicles using visual servoing have recently been conducted worldwide. Each study has different merits and limitations depending on the intended application. Most research is based on monocular vision. In contract, we have developed a 3D pose tracking system for docking operation using stereo vision providing high homing accuracy. To the best of our knowledge, our proposed system is the world first initiated research using two cameras as stereo vision in underwater vehicle environment. We have developed an efficient and robust real-time autonomous docking system by means of visual servoing using stereo vision, named as Three-Dimensional Move on Sensing (3D-MoS). The system recognizes a relative pose between a ROV and a target object through a newly proposed 3D model-based recognition method using Real-time Multi-step GA in real-time.
We conducted many docking experiments in the pool and in real sea using an ROV and also an AUV. The robustness of visual servoing has been evaluated against different disturbances while the underwater vehicle is controlled by our stereo vision based visual servoing. We conducted docking experiment in the sea of Wakayama prefecture to evaluate how much our 3D-MoS system would be robust against natural sea environment. According to the experimental result, it was confirmed that docking performance in sea using proposed system was achieved successfully with centimeter level accuracy in recognition and visual servoing.
●Video1: Stereo Vision-based Docking Experiment of AUV (Tuna Sand2) for Sea Bottom Battery Recharging
(Conducted in Tokyo University on October 4th, 2016)
The AUV docking experiments using our proposed stereo vision based visual servoing for sea bottom battery recharging application was conducted on October 4th, 2016. Our real-time 3D pose tracking system was installed in an AUV “Tuna-Sand 2.” An underwater battery recharging unit with a unidirectional docking function was designed and fixed in a pool. Tuna-Sand 2 approached to the docking station using other sensors and final docking operation was performed by our newly proposed stereo vision based visual servoing system. The main task in this experiment is to insert the docking pole (that is attached in AUV) into the docking hole (that is fixed with a 3D marker in the docking station) automatically by visual servoing. There are two steps in this experiment: (1) Approaching to the station following preset waypoints using other sensors, and (2) Docking step by visual servoing. In the first step, AUV followed the preset way points to approach the station using other sensors until 3D marker was detected by two cameras. When AUV approached to the station and 3D marker was in the field of view of two cameras, AUV switched to docking step in which the vehicle is controlled by visual servoing to insert docking pole into the docking hole precisely. The experimental results showed the performance of the proposed system with accurate docking accuracy. This project was conducted in Tokyo University in cooperation with Project Assistant Prof. Yuya NISHIDA from Kyushu Institute of Technology and Assoc. Professor,Toshihiro MAKI from Tokyo University.
(The study of this research was reported in the following papers.)
 Xiang Li , Yuya Nishida , Myo Myint , Kenta Yonemori , Naoki Mukada , Khin Nwe Lwin , Matsuno Takayuki, and Mamoru Minami, Dual-eyes Vision-based Docking Experiment of AUV for Sea Bottom Battery Recharging, the International Conference OCEANS17 MTS/IEEE, Aberdeen, Scotland, June 19-22, 2017. PDF (will be published soon)
●Video2: Stereo Vision-based Docking Experiment in a real sea for Sea Bottom Battery Recharging Application
(Conducted in the sea near Wakayama city in Japan on December 16th, 2015)
The docking experiments using stereo vision-based docking approach for sea bottom battery recharging application was conducted on December 16th, 2015. A docking station was designed as a unidirectional one to which the AUV has to dock in a specific entry. Real-time pose tracking using stereo vision was developed using two cameras mounted on an underwater vehicle and a known 3D marker fixed at the docking station. Real-time relative pose (position and orientation) estimation was implemented utilizing 3D model-based matching method and Real-time Multi-step Genetic Algorithm. A remotely Operated Vehicle (ROV) was used as a test bed. To verify the proposed approach for underwater battery recharging, a docking pole attached to the vehicle and a docking hole with a diameter of 70 mm fixed with a 3D marker at the docking station were designed. The main task in this experiment is to insert the docking pole into the docking hole automatically by visual servoing. Firstly, the vehicle approaches the docking station manually until the 3D marker is in the field of view with 1 m distance. Then, final docking operation was performed by visual servoing. Totally four times success docking experiments using the proposed approach that simulate for underwater battery recharging were conducted in the real sea. The experimental results have confirmed the functionality and possibility of the proposed approach for the sea bottom docking application of AUVs, having proved the proposed homing approach to be practical under real-world sea conditions. This project was conducted in the sea near Wakayama City, Japan in cooperation with KOWA cooperation for development of the ROV.
(The study of this research was reported in the following papers.)
 Myo Myint, Kenta YONEMORI, Akira YANOU, Khin Nwe Lwin, Maoki Mukada and Mamoru MINAMI Dual eyes visual based sea docking for sea bottom battery recharging, Proceedings of the International Conference OCEANS16 MTS/IEEE, Monterey, USA, 2016. [PDF]
 Kenta YONEMORI , Akira YANOU , Myo MYINT , Khin Nwe LWIN and Mamoru MINAMI, Docking experiment of underwater vehicle by dual-eye visual servoing in sea, Transactions of the JSME (in Japanese), Vol.83, No.848, 2017. [PDF]
●Video3: Docking experiment from different arbitrary stating positions according to docking strategy that includes approaching step, visual servoing step, and docking step
This work is concentrated on the dual-eye visual servoing as a possible new docking strategy rather than conventional docking methods. The proposed docking strategy consists of three steps. First, the ROV has to approach the 3D target until the target is in its field of view. Second, detecting the object and regulating the vehicle to the defined relative pose of the target is performed in the visual servoing step. Third, the docking operation is completed. In this experiment, after approaching docking station with constant speed and a constant proceeding direction while trying to detect the 3D marker, the vehicle is stabilized in the visual servoing step and controlled to keep the ROV with a defined pose relative to the target. In the docking step, when the vehicle is stable within the tolerance of the position error for the defined time period, the forward thrust that enables the docking pole attached to the ROV to fit into the dock is generated by gradually decreasing the distance between the vehicle and the target object. A pool (2 m (L) × 3 m (W) × 0.75 m (H)) filled with tap water was used as an experimental tank for the underwater vehicle experiments. The ROV was tethered by a cable 200 m in length to receive image information and output control signals. Experiments were carried out with different start positions: (1) on the left side of the pool relative to the 3D marker, (2) in front of the 3D marker, and (3) on the right side of the pool relative to the 3D marker. The experimental results showed the proposed system could successfully carry out docking operations.
(The study of this research was reported in the following paper.)
 Akira YANOU , Shota OHNISHI , Shintaro ISHIYAMA and Mamoru MINAMI, Autonomous docking control of visual-servo type underwater vehicle system aiming at underwater automatic charging, Transactions of the JSME (in Japanese), Vol.81, No.832, 2015 [PDF]
●Video4: Docking performance against different disturbances (background, air bubbles, physical disturbance)
As in the space environment, the underwater world gives complexity to underwater vehicle operation due to disturbances. Because the proposed system is a vision-based system, not only the physical disturbances of ocean currents but also the noise in recognized images should be considered in the experiments. By completing the experimental tasks while including these considerations, the proposed docking system demonstrates its effectiveness against different disturbances. In this experiment, docking performance against different disturbances was verified. Three kinds of disturbances namely as background, air bubbles, and physical disturbance were given to the docking environment. The background sheet that has sea patterns was set behind the 3D marker to verify recognition of 3D marker against the real sea patterns. Air bubbles were generated in front of the 3D marker to provide random noise to the captured images. Additionally, physical disturbances simulating water currents by pushing the ROV using a rod in different directions during docking operation were given to the experimental conditions. The experimental result shows the successful performance of docking against different disturbances.
●Video5: Docking performance against different disturbances (background, air bubbles)
The robustness of 3D post estimation against air bubbles and background disturbances was verified. Air bubbles address not only noise to the captured images but also physical distance to the movement of the ROV. The background sheet that has sea patterns was set behind the 3D marker to verify recognition of 3D marker against the real sea patterns. In this experiment, the air bubbles were generated when the ROV approaches the station by visual servoing.
●Video6: Tracking a moving target by visual servoing against air bubbles disturbance
In this experiment, the tracking ability of the ROV using proposed stereo-vision based visual servoing was verified experimentally. The ROV can recognize the 3D pose of the 3D marker that is moving in a camera depth direction even though there are some air bubbles in front of the cameras, and track the moving 3D marker by visual servoing. Real time pose estimation through Real-time Multi-step GA was input directly in the feedback of the controller. 3D motion of the ROV is controlled by P controller. The experimental results show that the ROV can track the moving 3D marker by visual servoing.
●Video7:Tracking a moving target by visual servoing against air bubbles disturbance
In this experiment, the tracking ability of the ROV using proposed stereo-vision based visual servoing was verified experimentally. The ROV can recognize the 3D pose of the 3D marker for duty cycles of 20 s and an amplitude of 280 mm from the ROV that is moving in a camera depth direction even though there are some air bubbles in front of the cameras, and track the moving 3D marker by visual servoing. Real time pose estimation through Real-time Multi-step GA was input directly in the feedback of the controller. 3D motion of the ROV is controlled by P controller. The experimental results show that the ROV can track the moving 3D marker by visual servoing.
(The study of this research was reported in the following paper.)
 M. Mint，K. Yonemori，S. Ishiyama，A. Yanou，M. Minami，Real-time 3D Pose Estimation and Tracking 3D Marker using Dual-eye Cameras for Under Water Vehicle, Proceedings of the 24th Meeting of the Institute of Measurement and Automatic Control，pp.62-63，2015.11.28 [PDF]
●Video8: Evening news program broadcasted our research on August 27,2014. (By Sanyo Broadcasting)
This video is an evening news program broadcasted our research on August 27th, 2014 by Sanyo Broadcasting in Japan.
●Video9: Tracking a moving 3D marker by stereo-vision based visual servoing
In this experiment, The ROV can recognize the 3D pose of the 3D marker that is moving in the pool, and track the moving 3D marker by visual servoing in real-time. Real time pose estimation through Real-time Multi-step GA was input directly in the feedback of the controller. The experimental results show that the ROV can track the moving 3D marker by visual servoing.