“Tracking a Swinging Target with a Robot Manipulator using Visual Sensing”
Authors: Torstein A. Myhre and Olav Egeland,Affiliation: NTNU
Reference: 2016, Vol 37, No 1, pp. 53-62.
Keywords: Industrial Robots, Particle Filter, Computer Vision for Manufacturing
Abstract: In this paper we develop a method for loading parts onto a swinging target using an industrial robot. The orientation of the target is estimated by a particle filter using camera images as measurements. Robust and accurate tracking is achieved by using an accurate dynamic model of the target. The dynamical model is also used to compensate for the time delay between the acquisition of images and the motion response of the robot. The target dynamics is modeled as a spherical pendulum. To ensure robust visual tracking the position of the target mass center is estimated. The method is experimentally validated in a laboratory loading station with a swinging conveyor trolley as target, which is commonly used in industry.
PDF (1434 Kb)
DOI: 10.4173/mic.2016.1.5DOI forward links to this article:
| [1] Torstein A. Myhre and Olav Egeland (2016), doi:10.1109/IECON.2016.7793396 |
[1] Bradski, G. and Kaehler, A. (2008). Bradski, G, and Kaehler, A. Learning OpenCV: Computer vision with the OpenCV library. O'Reilly Media, Incorporated. .
[2] Cappé, O., Godsill, S.J., and Moulines, E. (2007). Cappé, O, , Godsill, S.J., and Moulines, E. An overview of existing methods and recent advances in sequential Monte Carlo. Proceedings of the IEEE. 95(5):899--924. doi:10.1109/jproc.2007.893250
[3] Chitchian, M., Simonetto, A., van Amesfoort, A.S., and Keviczky, T. (2013). Chitchian, M, , Simonetto, A., van Amesfoort, A.S., and Keviczky, T. Distributed Computation Particle Filters on GPU Architectures for Real-Time Control Applications. IEEE Trans. Contr. Sys. Techn.. 21(6):2224--2238. doi:10.1109/tcst.2012.2234749
[4] Choi, C. and Christensen, H. (2013). Choi, C, and Christensen, H. RGB-D object tracking: A particle filter approach on GPU. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems. pages 1084--1091. doi:10.1109/iros.2013.6696485
[5] Choi, C., Christensen, H., and others. (2011). Choi, C, , Christensen, H., and others. Robust 3D visual tracking using particle filtering on the SE (3) group. In IEEE Int. Conf. on Robotics and Automation. IEEE, pages 4384--4390. doi:10.1109/icra.2011.5980245
[6] Choi, C. and Christensen, H.I. (2012). Choi, C, and Christensen, H.I. Robust 3D visual tracking using particle filtering on the special Euclidean group: A combined approach of keypoint and edge features. The International Journal of Robotics Research. 31(4):498--519. doi:10.1177/0278364912437213
[7] Concha, D., Cabido, R., Pantrigo, J., and Montemayor, A. (2014). Concha, D, , Cabido, R., Pantrigo, J., and Montemayor, A. Performance evaluation of a 3D multi-view-based particle filter for visual object tracking using GPUs and multicore CPUs. Journal of Real-Time Image Processing. pages 1--19. doi:10.1007/s11554-014-0483-1
[8] Corke, P.I. and others. (1996). Corke, P, I. and others. Visual Control of Robots: high-performance visual servoing. Research Studies Press Baldock. .
[9] Doucet, A. and Johansen, A.M. (2011). Doucet, A, and Johansen, A.M. A tutorial on particle filtering and smoothing: Fifteen years later. In D.Crisan and B.Rozovskii, editors, The Oxford handbook of nonlinear filtering, chapter24, pages 656--704. Oxford University Press, 2011. .
[10] Hartley, R. and Zisserman, A. (2003). Hartley, R, and Zisserman, A. Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge.. .
[11] Ibarguren, A., Martínez-Otzeta, J.M., and Maurtua, I. (2014). Ibarguren, A, , Martínez-Otzeta, J.M., and Maurtua, I. Particle filtering for industrial 6dof visual servoing. Journal of Intelligent & Robotic Systems. 74(3-4):689--696. doi:10.1007/s10846-013-9854-2
[12] Isard, M. and Blake, A. (1998). Isard, M, and Blake, A. Condensation—conditional density propagation for visual tracking. International journal of computer vision. 29(1):5--28. doi:10.1023/a:1008078328650
[13] Kantas, N., Doucet, A., Singh, S.S., and Maciejowski, J.M. (2009). Kantas, N, , Doucet, A., Singh, S.S., and Maciejowski, J.M. An overview of sequential monte carlo methods for parameter estimation in general state-space models. In Proc. 15th IFAC Symposium on System Identification (SYSID)., volume15. pages 774--785. doi:10.3182/20090706-3-fr-2004.00129
[14] Kantas, N., Doucet, A., Singh, S.S., Maciejowski, J.M., and Chopin, N. (2014). Kantas, N, , Doucet, A., Singh, S.S., Maciejowski, J.M., and Chopin, N. On Particle Methods for Parameter Estimation in State-Space Models. arXiv:1412.8695, submitted to Statistical Science. doi:10.1214/14-sts511
[15] Kermorgant, O. and Chaumette, F. (2011). Kermorgant, O, and Chaumette, F. Multi-sensor data fusion in sensor-based control: Application to multi-camera visual servoing. In IEEE Int. Conf. on Robotics and Automation. pages 4518--4523. doi:10.1109/icra.2011.5979715
[16] Khalil, H. (2002). Khalil, H, Nonlinear systems, volume3. Prentice hall. .
[17] Kröger, T. (2011). Kröger, T, Opening the door to new sensor-based robot applications—The Reflexxes Motion Libraries. In Proc. IEEE Int. Conf. on Robotics and Automation. pages 1--4. doi:10.1109/icra.2011.5980578
[18] Kwon, J., Choi, M., Park, F.C., and Chun, C. (2007). Kwon, J, , Choi, M., Park, F.C., and Chun, C. Particle filtering on the Euclidean group: framework and applications. Robotica. 25(06):725--737. doi:10.1017/s0263574707003529
[19] Lin, C.-Y., Wang, C., and Tomizuka, M. (2013). Lin, C, -Y., Wang, C., and Tomizuka, M. Visual tracking with sensing dynamics compensation using the Expectation-Maximization algorithm. In Proc. American Control Conference. IEEE, pages 6281--6286, 2013. doi:10.1109/acc.2013.6580823
[20] Lippiello, V., Siciliano, B., and Villani, L. (2007). Lippiello, V, , Siciliano, B., and Villani, L. Position-based visual servoing in industrial multirobot cells using a hybrid camera configuration. IEEE Transactions on Robotics. 23(1):73--86. doi:10.1109/tro.2006.886832
[21] Murray, R.M., Li, Z., and Sastry, S.S. (1994). Murray, R, M., Li, Z., and Sastry, S.S. A mathematical introduction to robotic manipulation. CRC press. .
[22] Myhre, T.A. and Egeland, O. (2015). Myhre, T, A. and Egeland, O. Parameter Estimation for Visual Tracking of a Spherical Pendulum with Particle Filter. In Proc. IEEE Int. Conf. on Multisensor Fusion and Integration for Intelligent Systems. pages 116--121. doi:10.1109/mfi.2015.7295795
[23] Park, F.C. and Martin, B.J. (1994). Park, F, C. and Martin, B.J. Robot sensor calibration: solving AX= XB on the Euclidean group. IEEE Transactions on Robotics and Automation. 10(5):717--721. doi:10.1109/70.326576
[24] Pauwels, K., Rubio, L., Diaz, J., and Ros, E. (2013). Pauwels, K, , Rubio, L., Diaz, J., and Ros, E. Real-time model-based rigid object pose estimation and tracking combining dense and sparse visual cues. In IEEE Conf. on Computer Vision and Pattern Recognition. IEEE, pages 2347--2354. doi:10.1109/cvpr.2013.304
[25] Poyiadjis, G., Doucet, A., and Singh, S.S. (2011). Poyiadjis, G, , Doucet, A., and Singh, S.S. Particle approximations of the score and observed information matrix in state space models with application to parameter estimation. Biometrika. 98(1):65--80. doi:10.1093/biomet/asq062
[26] Schön, T.B., Wills, A., and Ninness, B. (2011). Schön, T, B., Wills, A., and Ninness, B. System identification of nonlinear state-space models. Automatica. 47(1):39--49. doi:10.1016/j.automatica.2010.10.013
[27] Wang, C., Lin, C.-Y., and Tomizuka, M. (2013). Wang, C, , Lin, C.-Y., and Tomizuka, M. Visual servoing for robot manipulators considering sensing and dynamics limitations. In Proc. ASME Dynamic Systems and Control Conference. American Society of Mechanical Engineers, pages 21--23. doi:10.1115/dscc2013-3833
[28] Wang, C., Lin, C.-Y., and Tomizuka, M. (2015). Wang, C, , Lin, C.-Y., and Tomizuka, M. Statistical learning algorithms to compensate slow visual feedback for industrial robots. Journal of Dynamic Systems, Measurement, and Control. 137(3). doi:10.1115/1.4027853
BibTeX:
@article{MIC-2016-1-5,
title={{Tracking a Swinging Target with a Robot Manipulator using Visual Sensing}},
author={Myhre, Torstein A. and Egeland, Olav},
journal={Modeling, Identification and Control},
volume={37},
number={1},
pages={53--62},
year={2016},
doi={10.4173/mic.2016.1.5},
publisher={Norwegian Society of Automatic Control}
};