Semi-Supervised Adversarial Recognition of Refined Window Structures for Inverse Procedural Facade Modeling

Han Hua
Xinrong Lianga
Yulin Dinga,*
Xuekun Yuana
Qisen Shanga
Bo Xua
Xuming Gea
Min Chena
Ruofei Zhongb
Qing Zhua
aFaculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu, China

bBeijing Advanced Innovation Center for Imaging Technology, Capital Normal University, Beijing, China

Abstract

Deep learning methods are notoriously data hungry, which requires a large amount of labeled samples. Unfortunately, the large amount of interactive sample labeling efffforts has greatly hindered the application of deep learning methods, especially for 3D modeling tasks, which requires heterogeneous samples. To alleviate the work of data annotation for learned 3D modeling of facade, this paper proposed a semi-supervised adversarial recognition methods for the inverse procedural modeling. Beginning with textured LOD-2 (level of details) models, we use the classical consti tutional neural networks to recognize the types and estimate the parameters of windows, from the image patches. The window types and parameters are assembled into a procedural grammar. A simple procedural engine is built inside an existing 3D modeling software, which produces fifine-grained window geometries. In order to obtain a useful model from few labeled samples, we leverage the generate adversarial network to train the feature extractor semi-supervisedly. The adversarial training strategy can also exploit the large amount of unlabeled data to make the training phase more stable. Experiments using publicly available facade image dataset reveals that the proposed training strategy can obtain about 10% improvement in classifification accuracy and 50% improvement in parameter estimation under the same network structure. In addition, performance gains are more pronounced when testing against unseen data featuring difffferent facade styles.

Pipeline

pipeline

Results

Reconstruction of Single Buildings

without group processing

with group processing

without group processing

with group processing

without group processing

with group processing

without group processing

with group processing

without group processing

with group processing

without group processing

with group processing
Reconstruction of Blocks

without group processing

with group processing

without group processing

with group processing
Rendering Results

Modeling Processes

Batch Editing

The video demonstrations are also available in Bilibili.

LOD Creation

The video demonstrations are also available in Bilibili.

Window Generation

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China(Project No. 42071355, 41871291 and 41971411), in part by the National Key Research and Development Program of China (Project No. 2018YFC0825803), and in part by the Sichuan Science and Technology Program (Project No. 2020JDTD0003)