I. Introduction China boasts anextensive collection of stone cultural relics. As indicated by the findings ofthe third national cultural relics census, the country is home to a remarkabletotal of 24,422 grotto temples and stone carvings. Among these treasures,notable examples include the renowned Dunhuang Caves, Yungang Grottoes, andDazu Rock Carvings, all of which stand as exceptional artistic masterpieces.Differing from other cultural artifacts, most of these relics cannot be housedwithin museums due to their significant size. Stone structures, in particular,having weathered the elements over prolonged periods, experience unavoidabledeterioration caused by various factors. After enduring centuries of erosion, asignificant number of these relics can no longer retain their original state. To better preservethese stone cultural relics, experts have employed various methods in studyingand restoring them to ensure their conservation. Nevertheless, as timeprogresses, stone structures that have been exposed to the natural environmentfor extended durations remain susceptible to inevitable deterioration due tonatural forces. Therefore, theprecise documentation of the current state of cultural relics has become avital component of cultural heritage conservation. An authentic recording ofthe digital information associated with cultural relics can contribute to theirperpetual existence in digital format and provide crucial support for theirconservation through measures such as deterioration monitoring and virtualrestoration. Moreover, this digital record can serve as a valuable referencefor future restoration endeavors. In contrast to conventional mapping,photographs, and videos, three-dimensional (3D) digitization methods can recordinformation about cultural relics in a more extensive, comprehensive, andprecise manner. This not only facilitates the digital archiving of culturalrelics but also provides more convenient and diverse tools for conserving,researching, and exhibiting these artifacts. II. Background In December 2020,the project “China-Greece Joint Laboratory Construction and CollaborativeResearch on Cultural Heritage Conservation Technology”, led by the PalaceMuseum in application, received approval as part of the Strategic Scientificand Technological Innovation Cooperation Key Project under the National KeyR&D Program. Centering on the application of advanced optical and digitaltechnologies such as laser cleaning, large-format hyperspectral scanning, laserholographic speckle interference, and ultra-high-definition (UHD) 3Ddigitization to fulfill the needs for conserving cultural relics in both Chinaand Greece, the two countries have established the China-Greece Belt and RoadJoint Laboratory on Cultural Heritage Conservation Technology, a platformdedicated to equipment research and development, innovative applications, andtraining and exchange in the field of cultural relics conservation technology. Within thiscontext, the Subtopic IV titled “Research and Demonstration ofUltra-high-definition Three-dimensional Digital Acquisition and Processing ofStone Cultural Relics” focuses on the 3D digitization of stone cultural relicsin both China and Greece. It encompasses the exploration of a thoroughcollection of basic spatial information and textural details for culturalrelics, hardware configurations, and operational procedure standardization. With the stonearchitectural components of the Palace Museum’s Lingzhao Pavilion as itstarget, this sub-topic aims to delve into comprehensive cultural relic dataclassification and establish a hierarchical deep-processing system withspecific criteria. By examining the particularities and disparities in UHD 3Ddata acquisition and processing for cultural relics, this initiative seeks topresent a process specification and technical solution that yield consistentoutcomes with minimal manual intervention, quantifiable process and outcomeevaluation, and easy engineering implementation. Ultimately, the goal is toestablish a system for digitizing cultural relics and rules for acquisition andprocessing. This, grounded in the value of cultural relics’ data, aims to meetthe requirements for heritage conservation and forward-looking exhibition andutilization. III. 3D DigitalConservation of the Lingzhao Pavilion 1. Introduction tothe Lingzhao Pavilion and its Conservation The LingzhaoPavilion, located within the Palace of Prolonging Happiness, or Yanxi Palace,at the Palace Museum, is a structure crafted from metal, glass, and masonry.Dating back to its construction in 1909, it is also known as the “CrystalPalace”. This unique structure serves as a remarkable representation ofWestern-style architecture within the Forbidden City, with only a limited numberof similar buildings in existence. Due to fire safety concerns, the paviliondid not adopt the conventional Chinese wooden structure. Instead, it embraced afusion of stone and metal components, resulting in an aesthetically pleasingand uniquely designed edifice. Embodying a characteristic Western architecturalstyle, it stands as an early example in China of a steel-masonry hybridstructure that holds significant historical, artistic, and scientificsignificance. Based on the site investigation, the masonry structure of thepavilion retains a relatively intact appearance. Nonetheless, certain walls andupper sections of window and door arches display visible cracks (Fig. 1). Asfor the metal structure, while the overall appearance of the steel beams remainsrelatively complete, various degrees of corrosion have occurred. Fig. 1 Cracks inthe upper sections of the door and window arches 2. 3DDigitalization Solution The approaches to3D digitization for cultural relics are not standardized; they require acomprehensive evaluation that takes into account factors like the size,structural characteristics, and condition of deterioration of the historicalartifact. This assessment helps in selecting the appropriate method foracquiring comprehensive information. Common techniques include 3D laserscanning, structured-light scanning, and multi-image photogrammetric modeling,among others. In the case of stone cultural relics, to attain a betteracquisition outcome, an independent UHD image capturing of the relic surfacescan be employed. Additionally, there is a method to conduct UHD close-up imageacquisition, which particularly focuses on instances of deterioration. Thesecollaborative efforts will result in the capture of comprehensive dataincluding UHD, high-precision spatial information, as well as detailed texturalattributes of the cultural relics. Building upon theacquisition of comprehensive data from cultural relics, and guided by theprinciple of digitally preserving stone cultural artifacts, this approachinvolves analyzing the demands for digital preservation and exhibition and theprofessional needs for preventive protection of these relics. Drawing from theprofessional requirements of both aspects, a range of techniques, includingpoint cloud-based reverse modeling, photogrammetric modeling, virtual entitymodeling, digital image processing, and two-dimensional vector graphicextraction, are employed to yield data outcomes spanning multiple categoriesand levels for cultural relics. In this manner, a comprehensive digitalconservation system tailored to stone cultural artifacts is established. The technology roadmapof the UHD 3D digitization of stone cultural relics is illustrated in Fig. 2,and the operational roadmap is delineated in Fig. 3. Fig. 2 Thetechnology roadmap of the UHD 3D digitization of stone cultural relics Fig. 3 Theoperational roadmap of the UHD 3D digitization of stone cultural relics 3. Research on 3DDigitization Technology Methods 3.1 DataAcquisition Methods (1) Research onSite-Wide Digital High-Precision Control Establishing acontrol network for site-wide digital precision control should align with theguidelines outlined in GB50026-2020 “Standard for Engineering Surveying”. Thistypically involves employing a control network adopting second-level horizontalcontrol surveys and fourth-level vertical control surveys to ensure that theprecision of the ground control network is sufficient to meet the requirementfor digitization. However, under restricted conditions, the process of measuringfrom the points within the ground control network to the 3D scanning andphotogrammetric control points may not fully satisfy the requirements forhigh-precision digitization. In the case of demonstrating the digitization ofthe Lingzhao Pavilion, challenges arose when some spatial limitations andobstructions hindered a direct line of sight for vertical surveys.Additionally, situations occurred where a direct line of sight for verticalsurveys was possible, yet specification requirements were not met due toexceeding limits in side length or angles. After a thorough analysis, thesingle-span method using a suspended steel tape (Fig. 4) was adopted due to therelatively low height of about twenty meters. This approach yielded therequired level of precision. Fig. 4 Suspendedsteel tape transmission (2) Research onIndividual Component Digital High-Precision Control Concerning thestone components disassembled or removed from their original location, theconventional site-wide 3D digitization control method is not appropriate interms of both data accuracy and operational efficiency. Through our research,we have found that employing an integrated control approach that combines 3Dscanning and photogrammetry with the corresponding specialized device (as shownin Fig. 5) not only guarantees precise control accuracy but also significantlyimproves operational efficiency. Fig. 5 Theintegrated 3D scanning and photogrammetry control device When employed as atool for both 3D scanning and photogrammetric 3D reconstruction, the deviceenables highly precise alignment between the scanning and photogrammetric meshmodels, or between the scanning point clouds and photogrammetric mesh models.By employing these alignment methods, the achieved precision significantlysurpasses what can be accomplished through manual alignment processes. Thisapproach plays a pivotal role in achieving an exceptionally accuratereproduction in the digitalization of cultural relics. Alignment tasks duringdata acquisition and processing can be carried out automatically, leading to asignificant reduction in workload. This not only enhances efficiency but alsoensures that alignment outcomes remain largely unaffected by manualintervention. When employed exclusively for photogrammetric 3D reconstruction,the integrated device utilizes its two-dimensional codes to enable automaticimage control, eliminating the need for manually marked points. This not onlysignificantly enhances precision control compared to marked point-based manualimage control, but also eliminates the labor-intensive process associated withmarking points. Moreover, this approach eliminates discrepancies in controlprecision arising from variations in marked points due to different operators.When utilized solely for 3D scanning, the integrated device employs targetballs to provide control during scanning and streamline image stitching andconcatenation processes. In cases where marked points need to be applied duringscanning, they can be affixed to the frame of the device. This approacheffectively addresses the challenges arising from restrictions on attachingmarked points to cultural relics and the need to apply such points to completedata acquisition during scanning. The procedure forachieving integrated control of 3D scanning and photogrammetry is outlinedbelow (Fig. 6). Fig. 6 Theflowchart of the integrated control of 3D scanning and photogrammetry (3) Research on 3DScanning Data Acquisition A combination ofstationary scanning and hand-held scanning techniques was employed in acquiringthe UHD 3D scanning data of the stone structures of the Lingzhao Pavilion. In terms ofstationary scanning (also known as scene scanning) for data acquisition, it wasessential to determine the appropriate methods and parameters based on thespecific conditions of the site. This involved a consideration of variousaspects. The deployment of scanning stations entailed factors such as stationdistribution, scanning distances, regional coverage, indoor and outdoor spaces,depth and openings, single-story and multi-story structures, complexstructures, and dead-end sections. The aspect concerning target deploymentneeded to be addressed as well. This included target type, target distribution,the number of targets per station, the number of shared targets amongneighboring stations, alignment of targets and stations, and line of sight forcontrol points. Additionally, scanning operations required attention to pointcloud overlapping between adjacent scanning sites, accuracy and completeness ofscanned data, and operational sketches. As for hand-heldscanning (also known as close-range scanning) for data acquisition, it wasessential to determine the appropriate methods and parameters based on thecharacteristics of the cultural relics. This primarily involved factors such astarget deployment, scanning direction, scanning distance, block scanning, andthe correlation between point cloud density and the shape of the culturalrelics, among others. (4) Research onPhotogrammetric Data Acquisition Close-rangephotogrammetry was employed in acquiring the UHD 3D data of the stonestructures of the Lingzhao Pavilion. The acquisition process required carefulconsideration of the specific conditions to determine suitable methods andparameters. This included factors like photo resolution, control pointdeployment, lighting conditions during photography, longitudinal overlap,lateral overlap, the number of neighboring photographs, and color reproductionmanagement, among others. 3.2 Data ProcessingMethods (1) Research on 3DScanning Data Processing In order toarchive and process the point cloud data acquired through the UHD 3D scanningof the Lingzhao Pavilion, it was crucial to select the appropriate methods andparameters guided by the characteristics of the data. This process primarilyencompassed point cloud stitching and error correction, coordinate systemconversion, exclusion of non-target point clouds, elimination of abnormal pointclouds, noise reduction within the point clouds, thinning of the point clouds,creation of mesh models, and elimination of overlapping and intersectingsurfaces. Fig. 7 The scenescanning result (2) Research onPhotogrammetric Data Processing In order to archiveand process the data acquired through 3D reconstruction using the UHD 3Dphotogrammetric images of the Lingzhao Pavilion, it was crucial to identify theappropriate methods and parameters in accordance with the data. This processencompassed various steps, such as excluding substandard photographs, restoringphoto colors, conducting aerotriangulation, generating texture models,eliminating overlapping and intersecting surfaces and seams in mesh models,rectifying blurring, stretching, seams, and highlights in texture mapping,achieving color balancing and dodging in texture mapping, eliminatingfragmented UV maps, and optimizing both the mesh models and the quantity ofmapped data. (3) Research onIntegrated 3D Scanning and Photogrammetric 3D Reconstruction Data Processing In order toarchive and process the data acquired by combining the point clouds acquiredduring high-definition 3D scanning and 3D reconstruction using thephotogrammetric images of the Lingzhao Pavilion, it was essential to determinethe appropriate methods and parameters that aligned with the specific data.This process entailed various steps, encompassing distinct procedures for both3D scanning data processing and photogrammetric data processing. Additionally,it involved the segmentation and unwrapping of the 3D scanning mesh model forUV mapping, evaluating discrepancies between the photogrammetric and 3Dscanning mesh models, and baking texture mapping from the photogrammetrictexture model to the 3D scanning mesh model. 4. High-Precision3D Digitization Outcomes of the Lingzhao Pavilion Following thecriteria of UHD 3D digitization for stone cultural relics in control surveying,and considering the specific attributes of the pavilion, precise controlmeasurements were executed. This encompassed the establishment of a groundcontrol network and the positioning of scene-specific control points for both3D scanning and photogrammetry. The data gathered from each measurementdemonstrated conformity with the prescribed criteria. The integration ofscene-based 3D scanning with hand-held 3D scanning yielded the following data(Table 1): Table 1 Statisticsfor 3D point cloud data 3D point cloud Number of points 394,639,253 Original single-station spacing 0.6 mm The overall error between point clouds 2.78 mm Point cloud data size after stitching 22.1 G Close-rangephotogrammetry yielded the following data (Table 2): Table 2 Statisticsfor photogrammetric data Photogrammetry Number of photographs 122,008 photos Photographic data size 8.77 T Experimentalprocessing was carried out on stone components using UHD resolutions of 4K, 8K,12K, and 16K, resulting in the following model: The dataprocessing sample of the stone component of the Lingzhao Pavilion

Foreword Oil painting protection is an important part of the protection of culturalheritage across the world. Oil painting usually consists of a canvas fixed to awooden or metal frame, oil paints and a layer of clear protective varnish. Thevarnish plays an important role in protecting the oil painting. Small cracksinevitably appear in the varnish over time due to the impact of temperature,humidity, and transportation, among other conditions. Once the expansion anddeterioration of cracks in the varnish speed up, the aging of the whole oilpainting accelerates accordingly. Therefore, research on cracks in oilpaintings has always been one of the most-talked-about topics in oil paintingprotection. Studies show that under certain lighting conditions or in laser cleaningantique oil paintings, small damage may be caused to the surface of oilpainting, or the expansion of existing cracks may accelerate. Therefore,Shanghai University, the Palace Museum, the Institute of Electronic Structureand Laser of the Foundation for Research and Technology-Hellas (IESL-FORTH) ofGreece, as well as other organizations, collaboratively leveraging the analysistechniques and scientific research collaboration platform of the JointLaboratory, adopted digital holography for the experimental research on thegrowth of cracks in materials by focusing on the Dammer varnish that serves asthe protection layer of oil paintings. The stimulation of crack growth ispowered by laser irradiation, so as to simulate the growth of leaf-tip-shapedcracks in varnish under laser irradiation. Figure 1. Distribution of common small cracks in oil paintings Basic principles of digital holography Digital holography (DH) is an interferometric imaging technique that usesdigital image acquisition devices to directly record the diffraction lightfield of the measured object and performs numerical reconstruction of thediffraction light field through digital processing technology, obtainingrelevant parameters of the measured object. The diffraction light fieldcontains the intensity and phase information of the measured object, making theavailability of three-dimensional information a unique advantage of digitalholography. In addition, digital holography is featured by being microscopicand non-destructive with real-time imaging. Design of an experimental system and sample making The integrated optical path system of laser irradiation and digitalhologram recording is shown in Figure 2. SC-Pro multi-spectral pulsed laser isused as the system’s light source, with the output wavelength ranging from 430nm to 2,400 nm and a full-band operating power of 8W. The image acquisitiondevice is a CCD (DHC-MER-500-7UM) camera, with the size of a single pixel being2.2μm×2.2μm and the number of pixels in a pixel array being 1,544 (H) × 1,944(V). In the system, the reflector M1 can be translated along the directionperpendicular to the optical axis: When the reflector M1 moves to the positionof the solid line, the optical path LASER-M1-M4-L3 irradiates the sample, whilethe focal length of the convex lens L3 is 150mm, enabling concentratedirradiation from the light source. The Mach-Zehnder off-axis holographicinterference optical path is formed when the reflector M1 moves to the positionof the dotted line. After the laser beam is expanded and collimated by thespatial filter SF, it is divided into two beams by the prism BS1. One beam isirradiated through the reflector M2 to form the object beam of sample cracks,and the other beam passes through the reflector M3 to serve as the referencebeam. The object beam and reference beam are expanded and amplified afterpassing through the convex lenses L1 and L2 respectively and merged through theprism BS2 and interfere on the CCD surface to form a hologram. In the system, “p” is the focal length of the convex lenses L1 and L2,which is 120mm, while “q” is the distance from the rear focal plane of the convexlenses L1 and L2 to the photosensitive surface of the CCD. According to theprinciple of lens imaging, the L1-enabled magnification of the sample is q/p,and the beam expansion magnification enabled by the convex lenses L1 and L2 is1.976. In other words, the measured object will be magnified by 1.976 times inthe hologram recording process. Figure 2. Schematic diagram of laser irradiation & digital holographicinterferometric light path system (LASER-M4-L3 is the irradiation light path) The experiment consists of two steps. Step 1: Use different wave bands,such as near-ultraviolet, visible light and near-infrared bands, to irradiatethe cracked sample with laser for the photosensitivity analysis experiment ofthe material; Step 2: conduct a crack growth analysis experiment by irradiatingthe cracked sample with laser in a sensitive band range based on the results ofStep 1. The sample preparation method in the two steps of the experiment is thesame, and the photo of the sample is shown in Figure 3. Samples in Figure 3(a)and (b) are cracked samples with Dammar varnish which were used in the aboveexperiment respectively. Meanwhile, for better comparative analysis, acrack-free sample coated with Dammar varnish as shown in Figure 2(c) was madeas well for the crack growth analysis experiment. In the experiment, therecorded wavelength of the hologram was 633nm, and the frequency of irradiationlaser was 5Mhz. The irradiation each time lasted 5 minutes. Figure 3. Samples for the experiment: (a) Cracked sample 1; (b) Crackedsample 2, (c) Crack-free sample Experimental analysis of crack growth under laser irradiation In the first step of the experiment, the irradiation bands used werenear-infrared laser, visible light laser, and near-ultraviolet laser. To ensureenough irradiation power in place, two wavelengths were selected for each bandto form a mixed light source. The process of the experiment was as follows: Forthe cracked sample shown in Figure 3(a), the hologram in its original state wasfirstly recorded. Then a mixed light source with wavelengths of 740 nm and 750nm were used to irradiate the sample, and a hologram was recorded. Then thesample continued to be irradiated by a mixed light source with wavelengths of430 nm and 440 nm and a hologram was recorded as well. Finally, the sample wasirradiated by a mixed light source with wavelengths of 500 nm and 510 nm, and ahologram was recorded accordingly. The above-mentioned holograms werenumerically reconstructed to obtain wrapped phases of the cracks in each stateas shown in Figure 4(a)-(d) in sequence. By observing the phases in red dottedline boxes in these figures, we found that the phase changes of the toppositions of cracks were most obvious after the irradiation by the mixed lightsource with wavelengths of 740 nm and 750 nm. Therefore, compared with lasers invisible light band and near ultraviolet band, the laser in near infrared bandhas a greater impact on the crack growth. Figure 4. Reconstructed phases in holograms of the crack growth under theirradiation of light sources in different wavelength bands: (a) initial state;(b) 740 nm + 750 nm laser irradiation; (c) 430 nm + 440 nm laser irradiation;(d) 500 nm + 510 nm laser irradiation. In the second step of the experiment, four mixed light sources in thenear-infrared band with wavelengths of 740nm + 750 nm, 840nm + 850 nm, 940nm +950 nm and 1,040nm + 1,050 nm were used to irradiate the cracked samples shownin Figure 2(b) with laser. The irradiation each time lasted 5 minutes, and theabove process of hologram recording, and numerical reconstruction was repeatedevery time. The phase changes of crack growth after irradiation with lightsources with different wavelengths were shown in Figure 5(a)-(d) in turn. Tovisualize the crack growth after irradiation, the No. 457, No. 1625, and No.2525 pixels on the X-axis coordinates were selected, and then the interceptsalong the Y-axis were extracted respectively. The intercepts at the same pixelposition were reflected in the same picture as shown in Figure 6(a), Figure6(c) and Figure 6(e), through which we could see that those cracks grew tovarying degrees after the laser irradiation. The experiment process for thesample without cracks as shown in Figure 2(c) was repeated, and theexperimental results were shown in Figure 6(b), Figure 6(d) and Figure 6(f).The result has shown that a crack-free sample will not be destroyed under thesame laser irradiation for cracked samples. Based on the pixel sizes and the system’s calibrated magnification,quantitative calculations were performed to quantify the crack growth for eachgroup in Figure 6(a), Figure 6(c) and Figure 6(e). See results in Table 1. Fromthe calculation data in Table 1, we can see that when the head of theleaf-tip-shaped crack was irradiated, the growth distance of the middle andtail parts of the cavity in the tested material also changed with thewavelength. It is fair to conclude that the crack’s head part is more sensitiveto the change of the irradiation wavelength than the middle and the tail parts;when irradiated by lasers in the same band, the growth distance of the middleand tail parts of the crack is greater than that of the crack’s head, but thegrowth direction of the crack’s head is clearer. Figure 5. Phase changes of crack growth under irradiation of light sourcesin the near-infrared band: (a) After irradiation at 740 nm + 750 nm; (b) afterirradiation at 840 nm + 850 nm; (c) after irradiation at 940 nm + 950 nm; (d)after irradiation at 1,040 nm + 1,050 nm. Figure 6. Sectional diagrams of phase changes of crack growth at differentpositions under irradiation of light sources in the near-infrared band (upper:samples with cracks; lower: crack-free samples): (a)-(b) X = 457；(c)-(d) X = 1625；(e)-(f) X = 2525 Table 1. Crack extension distance of experimental groups and control groupsunder irradiation with different wavelengths (unit: µm) Summary Through an experimental study on laser irradiation of Dammar varnish cracksbased on digital holography, we have found that if tiny cracks exist in theprotective coating of oil paintings, crack growth is likely to occur at the tipof the crack under infrared radiation, leading to further deterioration of thecoating which in turn causes more damage to the oil painting. Therefore, tobetter protect the oil painting, infrared light should be avoided as much aspossible in the environment where oil paintings are preserved. When cleaningantique or fragile oil paintings with lasers, it is critical to use infraredlasers with caution to avoid irreversible damage to artworks. The digitalholography system mentioned in this study is small in size, easy to integrateand carry, making it suitable to be applied to the on-site protection ofcultural relics and research in the future. In addition, this study has provedthat digital holography has unique advantages in the detection of defects ofcultural relics. In the future, we will try to use digital holography for theresearch on tiny defects on non-transparent objects. This research has recently been published online in Applied Sciences,a SCI journal from MDPI. Link to the published research: https://www.mdpi.com/2076-3417/12/15/7799

3D display of bronze wares         3D display of stone cultural relics