A team of researchers just scanned two famous artworks by the High Renaissance artist Raphael with X-rays, and then used artificial intelligence to discern the chemical and color makeup of the artworks.
The artworks in question are God the Father and Virgin Mary, two panels of the now-destroyed Baronci Altarpiece. The altarpiece was Raphael’s first recorded commission, completed in 1501, that depicted the Devil, the Virgin Mary, and the coronation of Saint Nicholas of Tolentino by God the Father (hence the panel’s titles—duh). How the artist was able to get this work done while keeping his side-hustle as a sewer-dwelling turtle is hard to say. That joke was obligatory.
In its recent study—published today in Science Advances—the research team took macro X-ray fluorescence (MA-XRF) data of the two panels and fed them into a neural network taught on a synthetic dataset representing more than half a million spectra from 57 pigments and compounds. In other words, they taught an artificial intelligence model to sift through the range of visible colors and the chemical compounds that make them with a fine-toothed comb.
The model was a neural network, so named for its mimicry of the human brain’s ability to take in and interpret information, and then make decisions based on that information. Applied to the X-ray data of the panels, the neural network correctly identified the chemical elements applied by Raphael over 500 years ago. The white in the panel’s preparatory layers was correctly identified as lead-based, while figures’ skin tones contained red vermilion, a mercury-based pigment.
You may guess that the green drapes surrounding God the Father were copper-based. But the drapes were also chemically associated with potassium, indicating that the paint composing them was made from a mineral like azurite or a copper resinate mixed with a yellow lake pigment, according to an AAAS release.
But that’s not all. “The MA-XRF scans also revealed the gilded motifs of the two panel paintings, partially obscured in the current visible pictorial composition, and detected restorative work [that] occurred over time involving anachronistic pigments,” the team wrote in the paper. In other words, the neural network was able to spot work done to fix up the Raphaels later in time, as well as original motifs that didn’t make it into the final cut.
The altarpiece was in a church in Umbria for three centuries before it was severely damaged in 1789 during an earthquake, and the surviving fragments of the masterpiece were split up. The undamaged panels then went on a tumultuous provenance escapade, being picked up by Pope Pius VI before being seized by Napoleon and put in the Musée Napoléon (now the Louvre). God the Father and Virgin Mary were brought to Naples where they remain. You can read the full history of the altarpiece panels on the Frick Collection website.
Because the team trained the model on synthetic data, it effectively had an answer sheet against which to test the model’s performance. The model performed better in areas with deconvolution issues—a five-syllable way of saying sifting out actual information from noise—in parts of the panels where the High Renaissance painter used multiple pigments. In those areas, the muddled use of different pigments and compounds makes it difficult for traditional deconvolutional algorithms to properly analyze the element mades made by MA-XRF.
The authors stated that the model “effectively overcomes the limitations and artifacts commonly associated with traditional deconvolution analysis methods.” In the future, such work could inform conservation strategies of priceless artworks—as well as determine hidden details that are hard to spot otherwise.
It’s been a big week for artificial intelligence systems and image analysis—earlier this week, a different team of researchers trained a neural network on images of geoglyphs—earthen artworks—and ended up more than doubling the total number of Nazca lines, a group of monumental geoglyphs in Peru.
The real utility in many of these cases is that artificial intelligence systems can do similar work to human experts, but at a much faster pace. In the Nazca line case, the researchers had archaeologists review the images the network said contained a possible geoglyph—ensuring that while moving at a faster pace, there was still a human at the wheel.
