留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

显微条纹投影小视场三维表面成像技术综述

王永红 张倩 胡寅 王欢庆

王永红, 张倩, 胡寅, 王欢庆. 显微条纹投影小视场三维表面成像技术综述[J]. 中国光学, 2021, 14(3): 447-457. doi: 10.37188/CO.2020-0199
引用本文: 王永红, 张倩, 胡寅, 王欢庆. 显微条纹投影小视场三维表面成像技术综述[J]. 中国光学, 2021, 14(3): 447-457. doi: 10.37188/CO.2020-0199
WANG Yong-hong, ZHANG Qian, HU Yin, WANG Huan-qing. 3D small-field surface imaging based on microscopic fringe projection profilometry:a review[J]. Chinese Optics, 2021, 14(3): 447-457. doi: 10.37188/CO.2020-0199
Citation: WANG Yong-hong, ZHANG Qian, HU Yin, WANG Huan-qing. 3D small-field surface imaging based on microscopic fringe projection profilometry:a review[J]. Chinese Optics, 2021, 14(3): 447-457. doi: 10.37188/CO.2020-0199

显微条纹投影小视场三维表面成像技术综述

doi: 10.37188/CO.2020-0199
基金项目: 国家重点研发计划(No. 2016YFF0101803);国家自然科学基金资助项目(No. 51805137)
详细信息
    作者简介:

    王永红(1972—),男,安徽合肥人,博士,教授,博士生导师,美国Oakland University博士后。主要从事光学精密测试、激光散斑干涉检测和机器视觉等方面的研究。 E-mail:yhwang@hfut.edu.cn

  • 中图分类号: TP391;TP274.5

3D small-field surface imaging based on microscopic fringe projection profilometry:a review

Funds: Supported?by National Key Research and Development Program of China (No. 2016YFF0101803); National Natural Science Foundation of China (No. 51805137)
More Information
  • 摘要: 智能制造不断向着精密化、微型化、集成化的方向发展,具有代表性的集成电路技术及其衍生出的MEMS等微型传感器技术等得以迅猛发展,快速精确地获取微型器件表面信息并进行缺陷检测对于集成电路和MEMS等产业发展具有重要意义。基于结构光的条纹投影技术具有非接触、高精度、高效率、全场测量等优点,在精密测量中发挥着重要的作用。近年来,显微条纹投影测量系统,包括其光学系统结构,系统标定,相位获取以及三维重建方法等各个方面取得了重大发展。本文回顾了显微条纹投影三维测量系统的结构原理,分析了不同于传统投射模型的小视场系统标定问题,介绍了显微投影系统结构发展过程,同时对由系统结构以及金属测量时造成的反光问题进行了分析,在此基础上,对显微条纹投影三维测量系统的发展前景进行了展望。
  • 图  1  光学三角法测量原理图

    Figure  1.  Principle diagram of optical triangulation projection measuring system

    图  2  (a)针孔成像模型及(b)双远心成像模型

    Figure  2.  (a) Pinhole imaging model and (b) dual-telecentric imaging model

    图  3  相机与投影仪标定流程

    Figure  3.  Flow chart of calibration of the camera and projector

    图  4  (a)一种常用的小型化和通用的DLP LightCrafter[18]和(b)其二元投影机制

    Figure  4.  (a) A commonly used miniaturized and versatile DLP LightCrafter [18] and (b) its binary projection mechanism

    图  5  基于立体显微镜的MFPP系统。 (a)系统测量方案原理图; (b)测量系统实物图

    Figure  5.  Real-time MFPP system using stereoscopic microscope. (a) Schematic diagram of the system measurement and (b) physical diagram of the measurement system

    图  6  不同曝光时间下的条纹图像

    Figure  6.  Measurement results of captured fringe images under different exposure times

    表  1  基于体视显微镜的MFPP系统的比较

    Table  1.   Comparison of MFPP systems based on off-the-shelf microscopes

    文章投影技术系统复杂度测量视场大小
    Leonhardt等[7]Ronchi光栅0.10 mm×0.10 mm~
    2.50 mm×2.50 mm
    Proll等[9]LCD芯片1.40 mm×1.00 mm~
    16.5 mm×12.0 mm
    Zhang等[12]DMD芯片1.20 mm×0.90 mm~
    7.60 mm×5.70 mm
    Proll等[9]LCOS芯片0.83 mm×0.62 mm~
    21.2 mm×15.7 mm
    Chen等[30]DLP投影仪未给出
    Li等[31]LCOS投影仪3.0 mm×3.0 mm(变倍可调)
    [28]LightCrafter20.0 mm×15.0 mm(变倍可调)
    Jeught等[29]LightCrafter10.7 mm×8.0 mm(变倍可调)
    Hu等[26]LightCrafter8.0 mm×6.0 mm(变倍可调)
    下载: 导出CSV

    表  2  基于LWD镜头的MFPP系统对比

    Table  2.   Comparison of MFPP systems based on an LWD lens

    文章投影技术长工作距离镜头类型测量视场大小
    Quan等[8]LCD投影针孔+针孔镜头1.2 mm×1.5 mm
    Quan等[38]精细的正弦光栅针孔+针孔镜头0.1 mm×0.1 mm
    Wang等[39]LCD投影针孔+针孔镜头768 pixel×576 pixel
    Yin等[34]DLP投影针孔+针孔镜头5.0 mm×4.0 mm
    Li等[20]LightCrafter针孔+远心镜头10.0 mm×8.0 mm
    Li等[32]DLP投影仪远心+远心镜头30.0 mm×20.0 mm
    Liu等[21]LCD投影仪远心+远心镜头34.6 mm×29.0 mm
    Peng等[33]DMD芯片远心+远心镜头1280 pixel×
    1024 pixel
    Wang等[35]DMD芯片远心+4个远心镜头1280 pixel×
    1024 pixel
    Hu等[36]LightCrafter远心+2个远心镜头10.0 mm×7.0 mm
    下载: 导出CSV

    表  3  两类MFPP系统对比

    Table  3.   Comparison of the two kinds of method for MFPP

    基于立体显微镜的MFPP基于LWD透镜的MFPP
    优点灵活调整放大率
    良好的景深
    仅单相机系统
    条纹对比度高
    良好的景深
    标定结构简单
    结构紧凑
    缺点系统体积大
    构造复杂
    标定费时
    放大倍数固定
    公共视野受限
    适用领域需要快速调整
    视场的被测物
    表面形貌复杂,小空间
    物体测量
    下载: 导出CSV

    表  4  HDR 技术中各类方法的优缺点对比

    Table  4.   Comparison of typical methods in HDR technology

    文章实现方法优点缺点适用范围
    Zhang等[47]相机多重曝光法测量精度和信噪比较高,不需要搭建额外的硬件系统大范围反射率变化表面需采集大量的条纹图像,测量效率降低,未知场景有一定的盲目性复杂纹理表面;多颜色的表面;反射率变化不大表面;静态物体
    Chen等[48]调整投影图案强度法高信噪比,不受环境
    约束
    对未知的场景有一定的盲目性,测量效率低,不能自动预测参数复杂纹理表面;多颜色的表面;反射率变化不大表面;静态物体
    Feng等[54]偏振滤光片法测量精度高信噪比低,空间分辨率降低,硬件系统相对复杂镜面物体测量;快速动态测量
    Benveniste R等[56]颜色不变量法无需前期参数设置容易受到表面颜色和复杂纹理的影响,精度低快速动态测量
    Meng等[58]光度立体技术测量精度高系统结构的限制,单次测量的表面范围很小小范围物体测量;静态物体
    下载: 导出CSV
    ag币游app_币游娱乐官网(官网推荐)
  • [1] WANG J H, YANG Y X, SHAO M W, et al. Three-dimensional measurement for rigid moving objects based on multi-fringe projection[J]. IEEE Photonics Journal, 2020, 12(4): 6802114.
    [2] XIA P, WANG Q H, RI SH E. Random phase-shifting digital holography based on a self-calibrated system[J]. Optics Express, 2020, 28(14): 19988-19996. doi: 10.1364/OE.395819
    [3] 屈铭, 郑俊杰, 李敏, 等. 基于扫描白光干涉法的LCOS芯片像素级相位分析[J]. 光子学报,2019,48(9):0911004. doi: 10.3788/gzxb20194809.0911004

    QU M, ZHENG J J, LI M, et al. Pixel-level observation of phase profile in liquid crystal on silicon device by white light scanning interferometry[J]. Acta Photonica Sinica, 2019, 48(9): 0911004. (in Chinese) doi: 10.3788/gzxb20194809.0911004
    [4] MURAKAMI H, KATSUKI A, SAJIMA T, et al. Investigation of factors affecting sensitivity enhancement of an optical fiber probe for microstructure measurement using oblique incident light[J]. Applied Sciences, 2020, 10(9): 3191. doi: 10.3390/app10093191
    [5] 李成辉, 田云飞, 闫曙光. 激光扫描共聚焦显微成像技术与应用[J]. 实验科学与技术,2020,18(4):33-38. doi: 10.12179/1672-4550.20190257

    LI CH H, TIAN Y F, YAN SH G. Laser scanning confocal microscopy and its application[J]. Experiment Science and Technology, 2020, 18(4): 33-38. (in Chinese) doi: 10.12179/1672-4550.20190257
    [6] HU Y, CHEN Q, FENG SH J, et al. Microscopic fringe projection profilometry: a review[J]. Optics and Lasers in Engineering, 2020, 135: 106192. doi: 10.1016/j.optlaseng.2020.106192
    [7] LEONHARDT K, DROSTE U, TIZIANI H J. Microshape and rough-surface analysis by fringe projection[J]. Applied Optics, 1994, 33(31): 7477-7488. doi: 10.1364/AO.33.007477
    [8] QUAN C, HE X Y, WANG C F, et al. Shape measurement of small objects using LCD fringe projection with phase shifting[J]. Optics Communications, 2001, 189(1-3): 21-29. doi: 10.1016/S0030-4018(01)01038-0
    [9] PROLL K P, NIVET J M, K?RNER K, et al. Microscopic three-dimensional topometry with ferroelectric liquid-crystal-on-silicon displays[J]. Applied Optics, 2003, 42(10): 1773-1778. doi: 10.1364/AO.42.001773
    [10] NOTNI G, RIEHEMANN S, KUEHMSTEDT P, et al. OLED microdisplays: a new key element for fringe projection setups[J]. Proceedings of SPIE, 2004, 5532: 170-177. doi: 10.1117/12.560433
    [11] ZHANG SH F, LI B, REN F J, et al. High-precision measurement of binocular telecentric vision system with novel calibration and matching methods[J]. IEEE Access, 2019, 7: 54682-54692. doi: 10.1109/ACCESS.2019.2913181
    [12] ZHANG CH P, HUANG P S, CHIANG F P. Microscopic phase-shifting profilometry based on digital micromirror device technology[J]. Applied Optics, 2002, 41(28): 5896-5904. doi: 10.1364/AO.41.005896
    [13] LIU Y H, ZHANG Q C, ZHANG H H, et al. Improve temporal Fourier transform profilometry for complex dynamic three-dimensional shape measurement[J]. Sensors, 2020, 20(7): 1808. doi: 10.3390/s20071808
    [14] ZHANG H H, ZHANG Q C, LI Y, et al. High speed 3D shape measurement with temporal Fourier transform profilometry[J]. Applied Sciences, 2019, 9(19): 4123. doi: 10.3390/app9194123
    [15] 史耀群, 邓林嘉, 王朝旭, 等. 一种基于结构光条纹投影的微小物体测量系统[J]. 应用光学,2019,40(6):1120-1125. doi: 10.5768/JAO201940.0603007

    SHI Y Q, DENG L J, WANG ZH X, et al. Micro-objects measurement system based on structured light fringe projection[J]. Journal of Applied Optics, 2019, 40(6): 1120-1125. (in Chinese) doi: 10.5768/JAO201940.0603007
    [16] ZHONG M, CUI J, HYUN J S, et al. Uniaxial three-dimensional phase-shifting profilometry using a dual-telecentric structured light system in micro-scale devices[J]. Measurement Science and Technology, 2020, 31(8): 085003. doi: 10.1088/1361-6501/ab63b2
    [17] 殷永凯, 张宗华, 刘晓利, 等. 条纹投影轮廓术系统模型与标定综述[J]. 红外与激光工程,2020,49(3):0303008. doi: 10.3788/IRLA202049.0303008

    YIN Y K, ZHANG Z H, LIU X L, et al. Review of the system model and calibration for fringe projection profilometry[J]. Infrared and Laser Engineering, 2020, 49(3): 0303008. (in Chinese) doi: 10.3788/IRLA202049.0303008
    [18] ZHANG ZH Y. A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330-1334. doi: 10.1109/34.888718
    [19] LI B W, KARPINSKY N, ZHANG S. Novel calibration method for structured-light system with an out-of-focus projector[J]. Applied Optics, 2014, 53(16): 3415-3426. doi: 10.1364/AO.53.003415
    [20] LI B W, ZHANG S. Flexible calibration method for microscopic structured light system using telecentric lens[J]. Optics Express, 2015, 23(20): 25795-25803. doi: 10.1364/OE.23.025795
    [21] LIU H B, LIN H J, YAO L SH. Calibration method for projector-camera-based telecentric fringe projection profilometry system[J]. Optics Express, 2017, 25(25): 31492-31508. doi: 10.1364/OE.25.031492
    [22] 安东, 陈李, 丁一飞, 等. 光栅投影相位法系统模型及标定方法[J]. 中国光学,2015,8(2):248-254. doi: 10.3788/co.20150802.0248

    AN D, CHEN L, DING Y F, et al. Optical system model and calibration of grating projection phase method[J]. Chinese Optics, 2015, 8(2): 248-254. (in Chinese) doi: 10.3788/co.20150802.0248
    [23] 丁一飞, 王永红, 胡悦, 等. 样本块匹配光栅投影阶梯标定方法[J]. 中国测试,2016,42(8):7-12. doi: 10.11857/j.issn.1674-5124.2016.08.002

    DING Y F, WANG Y H, HU Y, et al. Step calibration method of grating projection based on exemplar matching[J]. China Measurement &Test, 2016, 42(8): 7-12. (in Chinese) doi: 10.11857/j.issn.1674-5124.2016.08.002
    [24] LU P, SUN CH K, LIU B, et al. Accurate and robust calibration method based on pattern geometric constraints for fringe projection profilometry[J]. Applied Optics, 2017, 56(4): 784-794. doi: 10.1364/AO.56.000784
    [25] CHEN Z, LIAO H Y, ZHANG X M. Telecentric stereo micro-vision system: calibration method and experiments[J]. Optics and Lasers in Engineering, 2014, 57: 82-92. doi: 10.1016/j.optlaseng.2014.01.021
    [26] HU Y, CHEN Q, LI H Y, et al. Absolute three-dimensional micro surface profile measurement based on a Greenough-type stereomicroscope[J]. Measurement Science and Technology, 2017, 28(4): 045004. doi: 10.1088/1361-6501/aa5a2d
    [27] Overview: DLP products[EB/OL]. [2020-10-18]. http://www.ti.com/dlp-chip/overview.html.
    [28] 肖萍萍. 基于光栅投射的小尺寸物体三维形状测量系统研究[D]. 武汉: 华中科技大学, 2019.

    XIAO P P. Research on 3D shape measurement system of small scale object based on grating projection[D]. Wuhan: Huazhong University of Science and Technology, 2019. (in Chinese).
    [29] VAN DER JEUGHT S, SOONS J A M, DIRCKX J J J. Real-time microscopic phase-shifting profilometry[J]. Applied Optics, 2015, 54(15): 4953-4959.
    [30] CHEN L C, LIAO CH CH, LAI M J. Full-field micro surface profilometry using digital fringe projection with spatial encoding principle[J]. Journal of Physics:Conference Series, 2005, 13: 147-150. doi: 10.1088/1742-6596/13/1/034
    [31] LI A M, PENG X, YIN Y K, et al. Fringe projection based quantitative 3D microscopy[J]. Optik, 2013, 124(21): 5052-5056. doi: 10.1016/j.ijleo.2013.03.070
    [32] LI D, LIU CH Y, TIAN J D. Telecentric 3D profilometry based on phase-shifting fringe projection[J]. Optics Express, 2014, 22(26): 31826-31835. doi: 10.1364/OE.22.031826
    [33] PENG J ZH, WANG M, DENG N N, et al. Distortion correction for microscopic fringe projection system with Scheimpflug telecentric lens[J]. Applied Optics, 2015, 54(34): 10055-10062. doi: 10.1364/AO.54.010055
    [34] YIN Y K, WANG M, GAO B Z, et al. Fringe projection 3D microscopy with the general imaging model[J]. Optics Express, 2015, 23(5): 6846-6857. doi: 10.1364/OE.23.006846
    [35] WANG M, YIN Y K, DENG D N, et al. Improved performance of multi-view fringe projection 3D microscopy[J]. Optics Express, 2017, 25(16): 19408-19421. doi: 10.1364/OE.25.019408
    [36] HU Y, CHEN Q, FENG SH J, et al. A new microscopic telecentric stereo vision system-calibration, rectification, and three-dimensional reconstruction[J]. Optics and Lasers in Engineering, 2019, 113: 14-22. doi: 10.1016/j.optlaseng.2018.09.011
    [37] HU Y, LIANG Y CH, TAO T Y, et al. Dynamic 3D measurement of thermal deformation based on geometric-constrained stereo-matching with a stereo microscopic system[J]. Measurement Science and Technology, 2019, 30(12): 125007. doi: 10.1088/1361-6501/ab35a1
    [38] QUAN C, TAY C J, HE X Y, et al. Microscopic surface contouring by fringe projection method[J]. Optics &Laser Technology, 2002, 34(7): 547-552.
    [39] WANG W H, WONG Y S, HONG G S. 3D measurement of crater wear by phase shifting method[J]. Wear, 2006, 261(2): 164-171. doi: 10.1016/j.wear.2005.09.036
    [40] 张莲涛, 卢荣胜, 程子怡. 基于相移偏折法的高反射表面面形测量技术[J]. 光子学报,2020,49(1):0112002. doi: 10.3788/gzxb20204901.0112002

    ZHANG L T, LU R SH, CHENG Z Y. Measurement technique of high-reflected surface based on phase measuring deflectometry[J]. Acta Photonica Sinica, 2020, 49(1): 0112002. (in Chinese) doi: 10.3788/gzxb20204901.0112002
    [41] LIU X H, ZHANG Z H, GAO N, et al. 3D shape measurement of diffused/specular surface by combining fringe projection and direct phase measuring deflectometry[J]. Optics Express, 2020, 28(19): 27561-27574. doi: 10.1364/OE.402432
    [42] 陶迁, 周志峰, 吴明晖, 等. 基于相位测量偏折术的反射表面缺陷检测[J]. 液晶与显示,2020,35(12):1315-1322. doi: 10.37188/YJYXS20203512.1315

    TAO Q, ZHOU ZH F, WU M H, et al. Detection of reflective surface defects based on phase measuring deflectometry[J]. Chinese Journal of Liquid Crystals and Displays, 2020, 35(12): 1315-1322. (in Chinese) doi: 10.37188/YJYXS20203512.1315
    [43] 王月敏, 张宗华, 高楠. 基于全场条纹反射的镜面物体三维面形测量综述[J]. 光学 精密工程,2018,26(5):1014-1027. doi: 10.3788/OPE.20182605.1014

    WANG Y M, ZHANG Z H, GAO N. Review on three-dimensional surface measurements of specular objects based on full-field fringe reflection[J]. Optics and Precision Engineering, 2018, 26(5): 1014-1027. (in Chinese) doi: 10.3788/OPE.20182605.1014
    [44] ZHANG L, CHEN Q, ZUO CH, et al. High-speed high dynamic range 3D shape measurement based on deep learning[J]. Optics and Lasers in Engineering, 2020, 134: 106245. doi: 10.1016/j.optlaseng.2020.106245
    [45] JIANG H ZH, ZHAO H J, LI X D. High dynamic range fringe acquisition: a novel 3-D scanning technique for high-reflective surfaces[J]. Optics and Lasers in Engineering, 2012, 50(10): 1484-1493. doi: 10.1016/j.optlaseng.2011.11.021
    [46] RAO L, DA F P. High dynamic range 3D shape determination based on automatic exposure selection[J]. Journal of Visual Communication and Image Representation, 2018, 50: 217-226. doi: 10.1016/j.jvcir.2017.12.003
    [47] ZHANG S. Rapid and automatic optimal exposure control for digital fringe projection technique[J]. Optics and Lasers in Engineering, 2020, 128: 106029. doi: 10.1016/j.optlaseng.2020.106029
    [48] CHEN CH, GAO N, WANG X J, et al. Adaptive projection intensity adjustment for avoiding saturation in three-dimensional shape measurement[J]. Optics Communications, 2018, 410: 694-702. doi: 10.1016/j.optcom.2017.11.009
    [49] WANG J H, YANG Y X. High-speed three-dimensional measurement technique for object surface with a large range of reflectivity variations[J]. Applied Optics, 2018, 57(30): 9172-9182. doi: 10.1364/AO.57.009172
    [50] SONG ZH, JIANG H L, LIN H B, et al. A high dynamic range structured light means for the 3D measurement of specular surface[J]. Optics and Lasers in Engineering, 2017, 95: 8-16. doi: 10.1016/j.optlaseng.2017.03.008
    [51] LIU Y ZH, FU Y J, CAI X Q, et al. A novel high dynamic range 3D measurement method based on adaptive fringe projection technique[J]. Optics and Lasers in Engineering, 2020, 128: 106004. doi: 10.1016/j.optlaseng.2020.106004
    [52] 万钇良, 王建立, 张楠. 一种基于相位相关与子图像的偏振图像配准方法[J]. 液晶与显示,2019,34(5):530-536. doi: 10.3788/YJYXS20193405.0530

    WAN Y L, WANG J L, ZHANG N. Polarized image registration method based on phase correlation and sub-graph[J]. Chinese Journal of Liquid Crystals and Displays, 2019, 34(5): 530-536. (in Chinese) doi: 10.3788/YJYXS20193405.0530
    [53] RIVIERE J, RESHETOUSKI I, FILIPI L, et al. Polarization imaging reflectometry in the wild[J]. ACM Transactions on Graphics, 2017, 36(6): 206.
    [54] FENG SH J, ZHANG Y ZH, CHEN Q, et al. General solution for high dynamic range three-dimensional shape measurement using the fringe projection technique[J]. Optics and Lasers in Engineering, 2014, 59: 56-71. doi: 10.1016/j.optlaseng.2014.03.003
    [55] BENVENISTE R, üNSALAN C. Binary and ternary coded structured light 3D scanner for shiny objects[M]//GELENBE E, LENT R, SAKELLARI G, et al. . Computer and Information Sciences. Dordrecht: Springer, 2011: 241-244.
    [56] BENVENISTE R, üNSALAN C. A color invariant for line stripe-based range scanners[J]. The Computer Journal, 2011, 54(5): 738-753. doi: 10.1093/comjnl/bxq014
    [57] BENVENISTE R, üNSALAN C. Nary coded structured light-based range scanners using color invariants[J]. Journal of Real-Time Image Processing, 2014, 9(2): 359-377. doi: 10.1007/s11554-011-0235-4
    [58] MENG L F, LU L Y, BEDARD N, et al. . Single-shot specular surface reconstruction with gonio-plenoptic imaging[C]. Proceedings of 2015 IEEE International Conference on Computer Vision, IEEE, 2015.
    [59] ZHANG L, CHEN Q, ZUO CH, et al. High dynamic range and real-time 3D measurement based on a multi-view system[J]. Proceedings of SPIE, 2019, 11427: 1142715.
    [60] HU Y, CHEN Q, LIANG Y CH, et al. Microscopic 3D measurement of shiny surfaces based on a multi-frequency phase-shifting scheme[J]. Optics and Lasers in Engineering, 2019, 122: 1-7. doi: 10.1016/j.optlaseng.2019.05.019
  • 加载中
图(6) / 表(4)
计量
  • 文章访问数:  493
  • HTML全文浏览量:  147
  • PDF下载量:  162
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-11-10
  • 修回日期:  2021-01-07
  • 网络出版日期:  2021-03-27
  • 刊出日期:  2021-05-14

目录

    /

    返回文章
    返回