What a confidence score can’t tell you: AI and the authorship of paintings
Imagine that a painting of uncertain origin has been submitted to an AI attribution service, and a report has come back: the work is 85.7 percent likely to be authentic. Versions of this scene are now familiar, heralded in headlines announcing that artificial intelligence has settled long-disputed attributions across the field of art history. The number looks like science. It is precise, reproducible, and untouched by the rivalries and liabilities that complicate human connoisseurship. But what does it actually communicate?
What follows aims to give art historians and art professionals, and collectors an understanding of these technologies that is sufficient to evaluate such claims independently. And it situates these tools within our actual work as art historians, returning throughout to the first axiom Sonja Drimmer and Christopher Nygren proposed for the discipline's encounter with AI: "the history of art is not a problem to be solved." Everything contained in that proposition is key to understanding what computational tools can offer and where they are categorically unhelpful in our work as art historians.
Authentication, attribution, and inclusion: three terms, one standard of evidence
The question of who made a work of art travels under different names depending on who is asking. The computer science literature most often uses authentication: a work is verified against a target artist, and the system reports a result. Art historians largely speak of attribution: a scholarly judgment about authorship, argued from close looking together with documentary evidence, and held open to revision whenever new evidence appears. The catalogue raisonné field speaks of inclusion: a work either enters the established corpus of an artist or it does not, on the strength of the case that can be assembled for it. The vocabularies differ because the stakes and settings differ.
But beneath the three vocabularies lies a single standard of evidence, and it is expansive. Any serious judgment of authorship rests on the record of where an object has been and through whose hands it passed, its exhibition history, the account books and correspondence that mention it, and the material analysis that establishes when and where the object was physically made. Visual examination by expert connoisseurs forms one kind of evidence within that body — indispensable, but one kind among several. The College Art Association's guidelines on authentication and attribution make the point explicitly: documentation, connoisseurship, and technical analysis are three necessary aspects of best practice, and it is their convergence that carries a judgment, in whichever vocabulary it is finally expressed. That standard supplies the question that should be asked of every AI tool described below. Does it add to that body of historical evidence, or does it deliver something else under the name of authentication?
How artificial intelligence technology in attribution work
The AI systems discussed in the news belong to a family of programs called image classifiers. To build an image classifier, engineers begin with a collection of digital photographs that experts have sorted into labeled groups. One labeled group holds works accepted as genuine, and another holds works by imitators or by other artists. This labeled collection is the training data. A program is then adjusted automatically, over many rounds, until it can sort the images it has already seen into their assigned groups. The adjusted program is the model. When the finished model receives a new image, it reports which group that image most resembles, along with a number expressing how strongly the new image matches the visual patterns of the group. That number is the confidence score, and it is what surfaces in press coverage as a "likelihood," as in the claim that a painting is 85.7 percent likely to be authentic.
Every recent announcement rests on this mechanism, whatever the surrounding language of genuine, hands, or artistic fingerprints suggests. The mechanism measures resemblance between images. Whether resemblance can answer a historical question is the problem that occupies the rest of this article.
What AI researchers have actually done
It helps to skip the headlines and flashy startups to look at the peer-reviewed publications in which academic researchers have applied these classifiers to authorship problems, because the scholars building the systems tend to be far more careful about the limits of AI than the coverage and startups that follow them. Three notable cases examine very different bodies of work: Jackson Pollock, Raphael, and ancient Chinese painting.
A team led by the physicist Richard Taylor at the University of Oregon, with image contributions from The Pollock-Krasner Foundation, The Pollock-Krasner Study Center, The International Foundation for Art Research, and Francis O’Connor, worked with a corpus of nearly six hundred digitized images to train a classifier to distinguish the poured compositions of Jackson Pollock from those of his imitators. It expanded the small corpus by breaking multiple images of each painting into thousands of smaller tiles at varying scales. When tested on images withheld from training, the system separated Pollock's paintings from imitations with an accuracy approaching ninety-nine percent.
A group led by the computer scientist Hassan Ugail at the University of Bradford, together with the imaging scientist David Stork and colleagues, adapted a model that had already learned general visual features from millions of ordinary photographs and refined it on a small set of authenticated Raphael paintings, adding measurements of the fine edges left by the brush. When the group applied the system to the Madonna della Rosa, a painting whose attribution scholars have long debated, the model found most of the composition consistent with Raphael while returning a markedly lower score for the face of Joseph, an output that aligned with existing scholarly suspicion of workshop participation in that passage.
A third team, computer scientists based at Zhejiang University built a system for ancient Chinese painting in which large language models coordinate image matching, seal recognition, and the retrieval of historical documentation to assist connoisseurs through the stages of their existing workflow. In evaluations with practicing experts, the system shortened the labor of assembling comparative evidence, and its authors report that its usefulness rises and falls with the coverage of its underlying archive; where the documentation is thin, the system has little to offer.
These are serious research projects, published in peer-review journals, and in each case the authors are explicit that their tool addresses one component of a larger authentication process. The Raphael team states plainly that a full authentication protocol depends on provenance, history, material studies, iconography, and the study of a work's condition, and that their contribution is only a portion of that whole. The three systems differ in ambition, design, and focus, yet each performs the operation described above: it reports how closely a new image resembles the patterns it was trained to recognize. But the helpfulness of confidence scores, training data, and pattern recognition in the attribution process must be evaluated.
What the confidence score can and cannot say
The most consequential misunderstanding concerns the confidence score itself. When a system reports a confidence score, the figure is a statement about the model rather than about the painting. It describes the degree to which the image resembles the examples the system was trained to treat as authentic. A confidence score is not an accuracy score, and it only rates the probability that an example work has been correctly assigned to the authentic or inauthentic categories according to its training. It is not, and cannot be, the certainty that a particular artist stood before the canvas, because the model has no access to that historical fact. It only has access to digitized images of the painting and to the labels those images were assigned.
The AI systems described match patterns, such as regularities of brushwork, texture, and structure that are difficult for the eye to quantify across many works. But pattern matching is the very work of successful forgers who, by definition, create patterns convincing enough to deceive. The entire history of forgery is the history of the skilled reproduction of an artist's surface manner. While the team at the University of Oregon developed a system to spot Pollock forgeries, the authors themselves do not suggest its system would work for other artists, nor do they preclude the development of better AI-robotic forgeries. But this is largely beside the point. Pattern-matching cannot serve as the foundation of authentication, because it evaluates the one dimension that a competent forgery is built to pass. This is not a deficiency that a more powerful model will remedy. A more powerful pattern-matcher is a more powerful evaluator of the thing that was never sufficient on its own.
The question the machine answers is how closely a new work resembles the ones it was told were genuine, and that question, however precisely its answer is expressed, differs from the question of who made the work. These are qualitatively different questions: the first concerns visual resemblance, which a machine can measure in some ways, and the second concerns history, which no measurement of the object alone can resolve. The precision of the reported figure, moreover, lends it an authority it has not earned, since the same system trained on a different set of examples would return a different number.
Training data and the persistence of prior judgment
That last observation points toward the deeper difficulty, which concerns the training data. Deep learning ordinarily depends on enormous quantities of examples, and an individual artist's surviving corpus is nothing of the kind. The Pollock study assembled roughly six hundred images only by subdividing each painting into tiles, and the Raphael study trained on fewer than fifty authenticated works. Scarcity of this order produces two problems. The first is technical: with so few examples, a model may learn features that have nothing to do with the artist — the lighting of a particular photographic campaign, the varnish of a particular era of restoration — and sort images confidently on grounds that are art-historically meaningless.
The second problem is more fundamental. These methods tend to re-inscribe existing knowledge rather than to create new knowledge. The set of works labeled authentic is itself the accumulated product of prior human attribution, some of it contested and some of it simply mistaken, and a model trained on those prior judgments cannot independently verify them. It can only absorb and reproduce them, lending the appearance of fresh, objective computation to conclusions the discipline had already drawn. The aspiration to remove human bias from the process founders on this point. One may remove the human from the moment of visual judgment, but the human judgments that built the training set remain, encoded in the very labels that define what the machine treats as genuine.
What the tools can honestly offer
None of this renders the technology worthless. Responsibly built with art historians in the room, these tools can flag a work as anomalous and deserving of a scholar's interest, can organize and search image corpora at a scale no individual could manage, and can surface comparisons a researcher might not have thought to make. The Chinese painting system points toward the soundest role: an expert-led model in which the machine assembles the materials of historical argument — seals, inscriptions, provenance documentation — and leaves the argument itself to the historian. Deployed as one instrument among many, in the hands of someone who understands what its numbers mean, computational analysis can support the patient labor of attribution.
This is not a condemnation of all AI technologies. Developed in response to real needs defined by art historians, tools could serve the discipline on a few plain principles. Their methods should be open to scrutiny, so that a result can be questioned the way any scholarly claim is questioned. And they are at their best expanding access to evidence rather than pronouncing on it. Making provenance records searchable across collections, linking documents to the objects they mention, assembling comparative images that once required months of travel to see are all possible applications. Tools built this way lengthen the chain of evidence on which every judgment of authorship depends, and lengthen it for every scholar, not only the one holding the report.
What AI technology cannot be is an arbiter. When experts disagree about a work, the disagreement almost always arises because the documentary evidence is thin or contradictory, because the historical chain has a missing link. It offers instead another judgment of the same limited kind the experts were already exchanging: an opinion about appearance, where the dispute turned on evidence about history. Presenting such a measure as the deciding vote does not resolve the deadlock but conceals it. That an AI attribution of a Raphael was publicly contested by other scholars almost as soon as it was announced is a reminder that the machine's verdict enters the same contested field as every human one and settles nothing on its own.
The history of art is not a problem to be solved
Authorship is a historical claim that rests on a chain of evidence reaching beyond the object's surface. The AI tools in the news are classifiers that measure resemblance between surfaces. At every step, the machine answers a question about appearance while the discipline asks a question about history. Beneath the particular failures catalogued here lies the deeper mistake: a misconception of what the past is and how it is known. The histories we write are arguments built from evidence, woven into and against the work of the scholars who came before us, and revised by those who come after as new evidence appears. When the work has been done well, the argument holds as evidence accumulates, and the picture grows more exact without ever becoming final. This is not a limitation awaiting a technical remedy. It is the discipline's method, and its strength. A confidence score offers to finish what cannot be finished, and in the offer is the revelation that it never understood the work in the first place.
