I. Introduction

China boasts an extensive collection of stone cultural relics. As indicated by the findings of the third national cultural relics census, the country is home to a remarkable total of 24,422 grotto temples and stone carvings. Among these treasures, notable examples include the renowned Dunhuang Caves, Yungang Grottoes, and Dazu Rock Carvings, all of which stand as exceptional artistic masterpieces. Differing from other cultural artifacts, most of these relics cannot be housed within museums due to their significant size. Stone structures, in particular, having weathered the elements over prolonged periods, experience unavoidable deterioration caused by various factors. After enduring centuries of erosion, a significant number of these relics can no longer retain their original state.

To better preserve these stone cultural relics, experts have employed various methods in studying and restoring them to ensure their conservation. Nevertheless, as time progresses, stone structures that have been exposed to the natural environment for extended durations remain susceptible to inevitable deterioration due to natural forces.

Therefore, the precise documentation of the current state of cultural relics has become a vital component of cultural heritage conservation. An authentic recording of the digital information associated with cultural relics can contribute to their perpetual existence in digital format and provide crucial support for their conservation through measures such as deterioration monitoring and virtual restoration. Moreover, this digital record can serve as a valuable reference for future restoration endeavors. In contrast to conventional mapping, photographs, and videos, three-dimensional (3D) digitization methods can record information about cultural relics in a more extensive, comprehensive, and precise manner. This not only facilitates the digital archiving of cultural relics but also provides more convenient and diverse tools for conserving, researching, and exhibiting these artifacts.

II. Background

In December 2020, the project “China-Greece Joint Laboratory Construction and Collaborative Research on Cultural Heritage Conservation Technology”, led by the Palace Museum in application, received approval as part of the Strategic Scientific and Technological Innovation Cooperation Key Project under the National Key R&D Program. Centering on the application of advanced optical and digital technologies such as laser cleaning, large-format hyperspectral scanning, laser holographic speckle interference, and ultra-high-definition (UHD) 3D digitization to fulfill the needs for conserving cultural relics in both China and Greece, the two countries have established the China-Greece Belt and Road Joint Laboratory on Cultural Heritage Conservation Technology, a platform dedicated to equipment research and development, innovative applications, and training and exchange in the field of cultural relics conservation technology.

Within this context, the Subtopic IV titled “Research and Demonstration of Ultra-high-definition Three-dimensional Digital Acquisition and Processing of Stone Cultural Relics” focuses on the 3D digitization of stone cultural relics in both China and Greece. It encompasses the exploration of a thorough collection of basic spatial information and textural details for cultural relics, hardware configurations, and operational procedure standardization.

With the stone architectural components of the Palace Museum’s Lingzhao Pavilion as its target, this sub-topic aims to delve into comprehensive cultural relic data classification and establish a hierarchical deep-processing system with specific criteria. By examining the particularities and disparities in UHD 3D data acquisition and processing for cultural relics, this initiative seeks to present a process specification and technical solution that yield consistent outcomes with minimal manual intervention, quantifiable process and outcome evaluation, and easy engineering implementation. Ultimately, the goal is to establish a system for digitizing cultural relics and rules for acquisition and processing. This, grounded in the value of cultural relics’ data, aims to meet the requirements for heritage conservation and forward-looking exhibition and utilization.

III. 3D Digital Conservation of the Lingzhao Pavilion

1. Introduction to the Lingzhao Pavilion and its Conservation

The Lingzhao Pavilion, located within the Palace of Prolonging Happiness, or Yanxi Palace, at the Palace Museum, is a structure crafted from metal, glass, and masonry. Dating back to its construction in 1909, it is also known as the “Crystal Palace”. This unique structure serves as a remarkable representation of Western-style architecture within the Forbidden City, with only a limited number of similar buildings in existence. Due to fire safety concerns, the pavilion did not adopt the conventional Chinese wooden structure. Instead, it embraced a fusion of stone and metal components, resulting in an aesthetically pleasing and uniquely designed edifice. Embodying a characteristic Western architectural style, it stands as an early example in China of a steel-masonry hybrid structure that holds significant historical, artistic, and scientific significance. Based on the site investigation, the masonry structure of the pavilion retains a relatively intact appearance. Nonetheless, certain walls and upper sections of window and door arches display visible cracks (Fig. 1). As for the metal structure, while the overall appearance of the steel beams remains relatively complete, various degrees of corrosion have occurred.

Fig. 1 Cracks in the upper sections of the door and window arches

2. 3D Digitalization Solution

The approaches to 3D digitization for cultural relics are not standardized; they require a comprehensive evaluation that takes into account factors like the size, structural characteristics, and condition of deterioration of the historical artifact. This assessment helps in selecting the appropriate method for acquiring comprehensive information. Common techniques include 3D laser scanning, structured-light scanning, and multi-image photogrammetric modeling, among others. In the case of stone cultural relics, to attain a better acquisition outcome, an independent UHD image capturing of the relic surfaces can be employed. Additionally, there is a method to conduct UHD close-up image acquisition, which particularly focuses on instances of deterioration. These collaborative efforts will result in the capture of comprehensive data including UHD, high-precision spatial information, as well as detailed textural attributes of the cultural relics.

Building upon the acquisition of comprehensive data from cultural relics, and guided by the principle of digitally preserving stone cultural artifacts, this approach involves analyzing the demands for digital preservation and exhibition and the professional needs for preventive protection of these relics. Drawing from the professional requirements of both aspects, a range of techniques, including point cloud-based reverse modeling, photogrammetric modeling, virtual entity modeling, digital image processing, and two-dimensional vector graphic extraction, are employed to yield data outcomes spanning multiple categories and levels for cultural relics. In this manner, a comprehensive digital conservation system tailored to stone cultural artifacts is established.

The technology roadmap of the UHD 3D digitization of stone cultural relics is illustrated in Fig. 2, and the operational roadmap is delineated in Fig. 3.

Fig. 2 The technology roadmap of the UHD 3D digitization of stone cultural relics

Fig. 3 The operational roadmap of the UHD 3D digitization of stone cultural relics

3. Research on 3D Digitization Technology Methods

3.1 Data Acquisition Methods

(1) Research on Site-Wide Digital High-Precision Control

Establishing a control network for site-wide digital precision control should align with the guidelines outlined in GB50026-2020 “Standard for Engineering Surveying”. This typically involves employing a control network adopting second-level horizontal control surveys and fourth-level vertical control surveys to ensure that the precision of the ground control network is sufficient to meet the requirement for digitization. However, under restricted conditions, the process of measuring from the points within the ground control network to the 3D scanning and photogrammetric control points may not fully satisfy the requirements for high-precision digitization. In the case of demonstrating the digitization of the Lingzhao Pavilion, challenges arose when some spatial limitations and obstructions hindered a direct line of sight for vertical surveys. Additionally, situations occurred where a direct line of sight for vertical surveys was possible, yet specification requirements were not met due to exceeding limits in side length or angles. After a thorough analysis, the single-span method using a suspended steel tape (Fig. 4) was adopted due to the relatively low height of about twenty meters. This approach yielded the required level of precision.

Fig. 4 Suspended steel tape transmission

(2) Research on Individual Component Digital High-Precision Control

Concerning the stone components disassembled or removed from their original location, the conventional site-wide 3D digitization control method is not appropriate in terms of both data accuracy and operational efficiency. Through our research, we have found that employing an integrated control approach that combines 3D scanning and photogrammetry with the corresponding specialized device (as shown in Fig. 5) not only guarantees precise control accuracy but also significantly improves operational efficiency.

Fig. 5 The integrated 3D scanning and photogrammetry control device

When employed as a tool for both 3D scanning and photogrammetric 3D reconstruction, the device enables highly precise alignment between the scanning and photogrammetric mesh models, or between the scanning point clouds and photogrammetric mesh models. By employing these alignment methods, the achieved precision significantly surpasses what can be accomplished through manual alignment processes. This approach plays a pivotal role in achieving an exceptionally accurate reproduction in the digitalization of cultural relics. Alignment tasks during data acquisition and processing can be carried out automatically, leading to a significant reduction in workload. This not only enhances efficiency but also ensures that alignment outcomes remain largely unaffected by manual intervention. When employed exclusively for photogrammetric 3D reconstruction, the integrated device utilizes its two-dimensional codes to enable automatic image control, eliminating the need for manually marked points. This not only significantly enhances precision control compared to marked point-based manual image control, but also eliminates the labor-intensive process associated with marking points. Moreover, this approach eliminates discrepancies in control precision arising from variations in marked points due to different operators. When utilized solely for 3D scanning, the integrated device employs target balls to provide control during scanning and streamline image stitching and concatenation processes. In cases where marked points need to be applied during scanning, they can be affixed to the frame of the device. This approach effectively addresses the challenges arising from restrictions on attaching marked points to cultural relics and the need to apply such points to complete data acquisition during scanning.

The procedure for achieving integrated control of 3D scanning and photogrammetry is outlined below (Fig. 6).

Fig. 6 The flowchart of the integrated control of 3D scanning and photogrammetry

(3) Research on 3D Scanning Data Acquisition

A combination of stationary scanning and hand-held scanning techniques was employed in acquiring the UHD 3D scanning data of the stone structures of the Lingzhao Pavilion.

In terms of stationary scanning (also known as scene scanning) for data acquisition, it was essential to determine the appropriate methods and parameters based on the specific conditions of the site. This involved a consideration of various aspects. The deployment of scanning stations entailed factors such as station distribution, scanning distances, regional coverage, indoor and outdoor spaces, depth and openings, single-story and multi-story structures, complex structures, and dead-end sections. The aspect concerning target deployment needed to be addressed as well. This included target type, target distribution, the number of targets per station, the number of shared targets among neighboring stations, alignment of targets and stations, and line of sight for control points. Additionally, scanning operations required attention to point cloud overlapping between adjacent scanning sites, accuracy and completeness of scanned data, and operational sketches.

As for hand-held scanning (also known as close-range scanning) for data acquisition, it was essential to determine the appropriate methods and parameters based on the characteristics of the cultural relics. This primarily involved factors such as target deployment, scanning direction, scanning distance, block scanning, and the correlation between point cloud density and the shape of the cultural relics, among others.

(4) Research on Photogrammetric Data Acquisition

Close-range photogrammetry was employed in acquiring the UHD 3D data of the stone structures of the Lingzhao Pavilion. The acquisition process required careful consideration of the specific conditions to determine suitable methods and parameters. This included factors like photo resolution, control point deployment, lighting conditions during photography, longitudinal overlap, lateral overlap, the number of neighboring photographs, and color reproduction management, among others.

3.2 Data Processing Methods

(1) Research on 3D Scanning Data Processing

In order to archive and process the point cloud data acquired through the UHD 3D scanning of the Lingzhao Pavilion, it was crucial to select the appropriate methods and parameters guided by the characteristics of the data. This process primarily encompassed point cloud stitching and error correction, coordinate system conversion, exclusion of non-target point clouds, elimination of abnormal point clouds, noise reduction within the point clouds, thinning of the point clouds, creation of mesh models, and elimination of overlapping and intersecting surfaces.

Fig. 7 The scene scanning result

(2) Research on Photogrammetric Data Processing

In order to archive and process the data acquired through 3D reconstruction using the UHD 3D photogrammetric images of the Lingzhao Pavilion, it was crucial to identify the appropriate methods and parameters in accordance with the data. This process encompassed various steps, such as excluding substandard photographs, restoring photo colors, conducting aerotriangulation, generating texture models, eliminating overlapping and intersecting surfaces and seams in mesh models, rectifying blurring, stretching, seams, and highlights in texture mapping, achieving color balancing and dodging in texture mapping, eliminating fragmented UV maps, and optimizing both the mesh models and the quantity of mapped data.

(3) Research on Integrated 3D Scanning and Photogrammetric 3D Reconstruction Data Processing

In order to archive and process the data acquired by combining the point clouds acquired during high-definition 3D scanning and 3D reconstruction using the photogrammetric images of the Lingzhao Pavilion, it was essential to determine the appropriate methods and parameters that aligned with the specific data. This process entailed various steps, encompassing distinct procedures for both 3D scanning data processing and photogrammetric data processing. Additionally, it involved the segmentation and unwrapping of the 3D scanning mesh model for UV mapping, evaluating discrepancies between the photogrammetric and 3D scanning mesh models, and baking texture mapping from the photogrammetric texture model to the 3D scanning mesh model.

4. High-Precision 3D Digitization Outcomes of the Lingzhao Pavilion

Following the criteria of UHD 3D digitization for stone cultural relics in control surveying, and considering the specific attributes of the pavilion, precise control measurements were executed. This encompassed the establishment of a ground control network and the positioning of scene-specific control points for both 3D scanning and photogrammetry. The data gathered from each measurement demonstrated conformity with the prescribed criteria.

The integration of scene-based 3D scanning with hand-held 3D scanning yielded the following data (Table 1):

Table 1 Statistics for 3D point cloud data

Close-range photogrammetry yielded the following data (Table 2):

Table 2 Statistics for photogrammetric data

Experimental processing was carried out on stone components using UHD resolutions of 4K, 8K, 12K, and 16K, resulting in the following model:

The data processing sample of the stone component of the Lingzhao Pavilion