研究生: |
劉貞佑 Chen-Yu Liu |
---|---|
論文名稱: |
由RGB-D影像資訊產生多視角立體電影用之影像 Multiview Stereo Images Generation from RGB-D Images |
指導教授: |
陳世旺
Chen, Sei-Wang |
學位類別: |
碩士 Master |
系所名稱: |
資訊工程學系 Department of Computer Science and Information Engineering |
論文出版年: | 2014 |
畢業學年度: | 102 |
語文別: | 中文 |
論文頁數: | 46 |
中文關鍵詞: | 2D轉3D 、矩陣完整化 、影像分割 、3D 、影像修補 、深度影像繪圖法 |
英文關鍵詞: | 2D to 3D, matrix completion, image segmentation, 3D, image inpainting, DIBR |
論文種類: | 學術論文 |
相關次數: | 點閱:266 下載:35 |
分享至: |
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
目前3D顯示技術已是很成熟的技術,然而目前可用的3D內容來源有限,造成推廣難度較高,即便用戶有經濟能力購買高檔3D影音設備也受限于可用的數位內容不夠多,讓3D功能等同虛設。本研究提出的將RGB+D 影像轉換成3D立體內容影像即為了舒緩部分立體內容不足的問題。
近年來有許多關於如何產生3D立體影像的研究,而從事這類研究,無可避免常會面臨到深度資訊估測以及產生新視角影像後修補的問題。深度估測的方式很多,有人工判斷、有根據深度線索判斷或是使用深度攝影機取得,人工判斷及根據深度線索判斷都相較耗時,尤其根據深度線索做判斷誤差的機率也較高。而深度攝影機則避免了深度資訊取得的困難,其誤差也較根據深度線索判斷來的低。然而使用深度攝影機取得的影像會有一個很大的問題,依照拍攝的情景不同,其影像可能會有不同程度的深度影像破洞,而深度資訊的完整及精準度直接影響了立體內容的好壞,因此需要對深度影像做合理的填補。而影像填補在過去亦有許多研究,主要考慮了顏色、紋理結構等因素,本研究使用兩種方式作處理,第一種是利用矩陣完整化技術的修補方式;第二種是使用以影像分割為基礎的修補方式,其中矩陣完整化是依據影像本身低rank的性質對影像做合理的填補,而基於影像分割的修補方式則是考慮顏色跟空間上位置的關係對影像作分割後再做填補。
實驗結果顯示我們的立體內容深度層次感相較於2D轉3D的立體內容明顯更佳,且相較于2D轉3D的技術,我們的研究使用的是深度攝影機所取得的深度資訊,因此深度資訊具有較低的誤差,產生出的立體影像能給予觀看者更好的立體感受。
Nowadays, 3D display technology has been well developed and gradually became a matured technology. However, limited 3D contain resources obstruct this technology to be popularized to the market. Even if the customers can afford expensive media equipment, there is still lack of useable resources to function 3D display technology. This research provides the solution of converting RGB+D image to 3D image to partially improve the shortage of 3D resources.
In recent decades, many researches are already working on how to create 3D images, which always involved depth measurement and generating image with another perspective. Depth measurement can be done by implementing the solutions such as manual judgments, depths cues, or using depth cameras. The former two solutions are relatively time consuming than the latter one. Especially the depths cues usually cause inaccuracy. Moreover, using depth cameras simplifies the difficulties of getting the depth data and decreases the inaccuracy as well. But there is a problem when using the cameras to collect the depth data, the images may have holes occurs which depends on shooting scenarios. The depth data need to be repaired under a reasonable condition because these two factors impact the 3D images’ qualities. In the past, solution to image inpainting has been proposed from many researches. The main considerations are about the colors and the texture. This research implements two methods to process the missing value of depth images. One is based on images’ low rank feature to use matrix completion technique; the other is based on image segmentation technique to do the depth image repairing.
The results of experiment show that our 3D depth quality is obviously higher than the traditional 2D convert to 3D method. Furthermore, depth camera collects the depth data with higher accuracy so we can provide viewers a better experience in 3D display technology.
[Ide08] I. Ideses, L. Yaroslavsky, and B. Fishbain, “Depth Map Manipulation for 3D Visualization,” 3DTV Conference: The True Vision - Capture, Transmission
and Display of 3D Video, pp.337-340, 2008.
[Bat04] S. Battiato, S. Curti, M. La Cascia, E. Scordato, and M. Tortora, “Depth Map
Generation By Image Classification,” Proc. of SPIE IS&T/SPIE's 16th Annual
Symposium on Electronic Imaging, pp. 95-104, 2004.
[Com97] D. Comaniciu, and P. Meer, “Robust Analysis of Feature Spaces: Color Image Segmentation,” Proc. of IEEE Conference on Computer Vision and Pattern
Recognition, pp. 750-755, June 1997.
[Ang10] L. J. Angot, W. J. Huang and K. C. Liu, “A 2D to 3D video and image conversion technique based on a bilateral filter,” Proc. of SPIE-IS&T Electronic Imaging, Vol. 7526, 2010.
[Che10] C. C. Cheng, C. T. Li, and L. G. Chen, “ A 2D-to-3D conversion system using edge information,” Int’l Conf. on Consumer Electronics (ICCE), pp.377-378, 2010.
[Li06] P. Li, and R. K. Gunnewiek, “On Creating Depth Maps from Monoscopic Video using Structure from Motion,” Proc. of IEEE Workshop on Content Generation and Coding for 3D-television, pp.508-515, 2006.
[Har02] P. Harman, J. Flack, S. Fox and M. Dowley, “Rapid 2D to 3D Conversion,” Proc. SPIE, Vol. 4660, pp.78-86, 2002.
[Rob88] L. S. Robert, (1994), “Cognition and the Visual Arts, ” Cambridge, MA, The MIT Pres.
[Cam12] M. Camplani and L. Salgado, “Efficient spatio-temporal hole filling strategy for Kinect depth maps,” Proc. SPIE, Vol. 8290, 2012.
[Xu12] K. Xu, J. Zhou and Z. Wang, “A method of hole-filling for the depth map generated by Kinect with moving objects detection,” IEEE on Broadband Multimedia Systems and Broadcasting (BMSB), pp.1-5, 2012.
[Cri04] A. Criminisi, P. Perez, and K. Toyama,“Region filling and object removal by exemplar-based image inpainting,”IEEE Transactions on Image Processing, pp. 1200 - 1212, 2004 .
[Feh04] C. Fehn, “Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV,” Proc. SPIE, Stereoscopic Displays and Virtual Reality Systems, 2004.
[Zin10] S. Zinger, L. Do and P. H. N. de With, “Free-viewpoint depth image based rendering,” Journal of Visual Communication and Image Representation, Vol. 21, pp.533-541, 2010
[Lai13] Y. K. Lai, Y. F. Lai, and Y. C. Chen “An Effective Hybrid Depth-Generation Algorithm For 2d-To-3d Conversion In 3d Displays,” Journal of Display Technology, vol.9, pp. 154-161, 2013.
[Ach12] R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua and S. Süsstrunk, S. “
SLIC Superpixels Compared to State-of-the-Art Superpixel Methods,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 34, Issue 11, pp.2274-2282, 2012.
[Kom06] N. Komodakis and G. Tziritas, “Image Completion Using Global Optimization,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[Ras05] C. Rasmussen and T. Korah, “Spatiotemporal Inpainting for Recovering Texture Maps of Partially Occluded Building Facades,” Proc. IEEE Int’l Conf. Image Processing, 2005.
[Ji10] H. Ji, C. Liu, Z. Shen, and Y. Xu, “Robust Video Denoising Using Low Rank Matrix Completion,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[Lee10] J. A. Lee and M. Verleysen “Unsupervised dimensionality reduction: Overview and recent advances,” The 2010 International Joint Conference on Neural Networks (IJCNN), pp. 1-8, 2010.
[Can08] E. J. Candes and B. Recht, “Exact low-rank matrix completion via convex optimization,” Annual Allerton Conference on Communication, Control, and Computing, pp. 806-812, 2008
[Zha05] L. Zhang and W. J. Tam, “Stereoscopic Image Generation Based on Depth Images for 3DTV,” IEEE Trans. Broadcast, vol. 51, pp. 191–199, 2005.
[Tom98] C. Tomasi and R. Manduchi, “Bilateral Filtering for Gray and Color Images,” Proc. IEEE Int’l Conf. on Computer Vision, vol. 51, pp. 191–199, 2005
[Cai10] J. F. Cai, E. J. Cande`s, and Z. Shen, “ (ASi),” SIAM J. Optimization, vol. 20, pp. 1956-1982, 2010.
[Hu13] Y. Hu, D. Zhang, J. Ye, X. Li, and X. He, “Fast and Accurate Matrix Completion via Truncated Nuclear Norm Regularization,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 35, No. 9, Sept. 2013.
[Rec10] B. Recht, M. Fazel, and P.A. Parrilo, “Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization,” SIAM Rev., Vol. 52, No. 3, pp. 471-501, 2010.
[Toh10] K. C. Toh and S. Yun, “An Accelerated Proximal GradientAlgorithm for Nuclear Norm Regularized Least Squares Problems,” Pacific J. Optimization, pp. 615-640, 2010.
[Wri09] J. Wright, A. Ganesh, S. Rao, Y. Peng, and Y. Ma, “RobustPrincipal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization,” Proc. Advances in Neural Information Processing Systems, 2009.
[Tel04] A. Telea, “An Image Inpainting Technique Based on the Fast Marching Method,” Graphics, GPU, & Game Tools, 2004.
[Ber01] B. Bertlmio, A. L. Bertozzim, and G. Sapiro, “Navier-stokes, fluid dynamics,and image and video inpainting,” Computer Vision and Pattern Recognition, Vol. 1, pp.355-362, 2001
[劉 10] 劉楷哲、吳其霖、黃偉豪、陳信榮、李錕、羅豐祥,“基於3D顯示器格式之即時3D內容合成技術”,影像與識別期刊(IPPR) 2010年,Vol. 16,No. 2。
[林12] 林晃巖,“試看紅塵的深度 –– 漫談 3D 立體顯示器的發展” http://www.ee.ntu.edu.tw/hischool/doc/2012.04.pdf。
[賴 99] 賴文能,陳韋志,“淺談 2D 至 3D 視訊轉換技術”,影像與識別期刊(IPPR) 2010年,Vol. 16,No. 2。