Synthetic Data-Driven Augmentation for Precise 6-DoF Pose Estimation of Building Components in Automated Facility Inspections

This paper tackles the challenge of automating
facility inspections by detecting building components, estimating
their six-degree-of-freedom (6-DoF) poses (position and orienta
tion), and comparing these estimations to Building Information
Modeling (BIM) ground truth data. Vision based Deep learn
ing methods offer promising results in pose estimation. They
rely heavily on large annotated image datasets for training,
which are often lacking or too specific for building settings.
Manually annotating large datasets is both labor-intensive and
time-consuming, particularly when they contain hundreds of
images. This challenge further underscores the need for efficient
and automated methods for dataset annotation, particularly in
building environments where dataset size and complexity are
significant. We present a study examining the point where the
performance of a 6-DoF object detection algorithm is efficient.
In particular, we analyze the proportion of real data required for
detection and pose estimation models to perform well on double
leaf doors. Synthetic data are generated using the Digital Twin
(DT), which is intended to serve as the future reference for BIM
in facility inspections. Adding synthetic data reduces rotation
error by up to 1.7°, starting from 1,000 images. We analyze
the effectiveness of different proportions of real and synthetic
data to provide insight into optimizing dataset composition. The
accuracy of the 3D center point and the accuracy at 25 cm/25
degrees are better than 90% accuracy when more than 50 real
images combined with synthetic data.