Structure-Aware Completion of Photogrammetric Meshes in Urban Road Environment

Abstract

Photogrammetric mesh models obtained from aerial oblique images have been widely used for urban reconstruction. However, photogrammetric meshes suffer from severe texture problems, particularly in typical road areas, owing to occlusion. This paper proposes a structure-aware completion approach to improve mesh quality by seamlessly removing undesired vehicles. Specifically, a discontinuous texture atlas is first integrated into a continuous screen space by rendering trough a graphics pipeline. The rendering also records the necessary mapping for deintegration to the original texture atlas after editing. Vehicle regions are masked by a standard object detection approach, namely, Faster RCNN. Subsequently, the masked regions are completed, guided by the linear structures and regularities in the road region; this is implemented based on PatchMatch. Finally, the completed rendered image is deintegrated to the original texture atlas, and the triangles for the vehicles are also flattened so that improved meshes can be obtained. Experimental evaluation and analysis are conducted on three datasets, which were captured with different sensors and ground sample distances. The results demonstrate that the proposed method can produce quite realistic meshes after removing the vehicles. The structure-aware completion approach for road regions outperforms popular image completion methods, and an ablation study further confirms the effectiveness of the linear guidance. It should be noted that the proposed method can also handle tiled mesh models for large-scale scenes. Code and datasets are available at the project website https://vrlab.org.cn/~hanhu/~hanhu/projects/mesh.