Structure-Aware Completion of Photogrammetric Meshes in Urban Road Environment

ZHU Qinga
SHANG Qisena
HU Hana,*
YU Haojiaa
Ruofei Zhongb
aFaculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu, China

bBeijing Advanced Innovation Center for Imaging Technology, Capital Normal University, Beijing, China

Abstract

Photogrammetric mesh models obtained from aerial oblique images have been widely used for urban reconstruction. However, the photogrammetric meshes also suffer from severe texture problems, especially on the road areas due to occlusion. This paper proposes a structure-aware completion approach to improve the quality of meshes by removing undesired vehicles on the road seamlessly. Specifically, the discontinuous texture atlas is first integrated to a continuous screen space through rendering by the graphics pipeline; the rendering also records necessary mapping for deintegration to the original texture atlas after editing. Vehicle regions are masked by a standard object detection approach, e.g. Faster RCNN. Then, the masked regions are completed guided by the linear structures and regularities in the road region, which is implemented based on Patch Match. Finally, the completed rendered image is deintegrated to the original texture atlas and the triangles for the vehicles are also flattened for improved meshes. Experimental evaluations and analyses are conducted against three datasets, which are captured with different sensors and ground sample distances. The results reveal that the proposed method can quite realistic meshes after removing the vehicles. The structure-aware completion approach for road regions outperforms popular image completion methods and ablation study further confirms the effectiveness of the linear guidance. It should be noted that the proposed method is also capable to handle tiled mesh models for large-scale scenes.

Texture Integration

SWJTU
SWJTU
SWJTU

Mesh Completion

Xipu Shenzhen Dortmund

The video demonstrations are also available in Bilibili.

Code

We release a demo for the completion of the integrated texture after vehicle detection. The code is released to github.

Datasets

We release the mesh models before & after completion.

Acknowledgements

This work was supported in part by the National Key Research and Development Program of China (Project No. 2018YFC0825803) and by the National Natural Science Foundation of China (Project No. 41631174, 42071355, 41871291). In addition, the authors gratefully acknowledge the provision of the datasets by ISPRS and EuroSDR, which were released in conjunction with the ISPRS Scientific Initiatives 2014 and 2015, led by ISPRS ICWG I/Vb.