AI Can Find Personal Data. But Can It Reliably Protect It?
Imagine a government agency preparing hundreds of documents for public release. An AI system scans the files and identifies names, addresses, and identification numbers for redaction. The results look impressive. But what if the system misses just one identifying detail?
Researchers at the Technical University of Applied Sciences Würzburg-Schweinfurt (THWS) have found that this is a serious challenge. In a new evaluation, even the strongest tested AI configuration missed at least one critical personal identifier on 96.8% of applicable document pages. The finding highlights a gap between conventional AI detection scores and the level of protection needed before sensitive documents can be shared.
When Almost Right Is Not Enough
An AI tool may recognize nearly every sensitive detail on a page and still leave someone identifiable. Real documents are particularly difficult: scanned correspondence, tables, complex layouts, and image noise can interfere with text recognition and the precise positioning of redaction boxes.
To investigate these failures, THWS researcher Vishnu Prasad Vijaya Kumar developed LeakageBench as part of research supervised by Prof. Dr. Ivan P. Yamshchikov at the Center for Artificial Intelligence (CAIRO), in collaboration with co-author Santhosh Venkatesh of DataX.
LeakageBench contains 500 document-page images with 11,954 annotated instances of potentially identifying information. Beyond obvious names and contact details, it includes linkage identifiers — such as reference numbers that may reveal identities when combined with other records.
A Different Measure of Privacy Risk
The team evaluated conventional optical character recognition pipelines, specialized detectors, and advanced vision-language models that analyze document images directly.
The systems failed for different reasons. Some did not support all relevant identifier categories. Others missed text or located it imprecisely. Giving GPT-5.5 additional tools substantially improved its localization performance, but at least one critical identifier still went undetected on 96.8% of applicable pages under the benchmark’s criteria.
Unlike tests that focus on the number of individual identifiers detected correctly, LeakageBench asks whether a page still contains critical information a system has overlooked. This makes the results directly relevant to decisions about whether a document is ready for disclosure.
Why Governments and Businesses Should Care
Public authorities process citizen records and disclosure requests; companies handle contracts, customer data, and employee files. Automated redaction can reduce manual work, but organizations need evidence about what it misses before relying on it in sensitive workflows.
“An organization might believe its documents are protected because an AI system successfully identified thousands of sensitive details. But privacy is ultimately compromised by the information the system misses, not the information it finds,” says Vijaya Kumar.
The benchmark gives developers a way to diagnose weaknesses and helps organizations ask more useful questions of their AI providers: Which identifiers are covered? What errors remain? Where is human review necessary?
A Connection to Europe’s AI Sovereignty Debate
As Europe seeks greater control over AI infrastructure, data, and technology choices, independent evaluation is another part of responsible adoption. Governments and businesses need to assess the systems they deploy on the basis of evidence, not vendor claims alone.
LeakageBench does not measure whether AI is European-owned, locally hosted, or legally compliant. Its contribution is narrower and concrete: a way to examine residual privacy risks in automated document redaction. The study therefore speaks to one practical dimension of AI sovereignty — the ability to understand and govern AI systems used in sensitive settings.
The researchers caution that the 500-page collection is a deliberately challenging test set, not a representative sample of every document type or language. The reported failure rate should not be generalized to all redaction deployments.
The paper has been accepted to Findings of EMNLP 2026, an international venue for research in natural language processing.
About the Research
The study, “LeakageBench: Document-Level Leakage Risk for Redacting Personally Identifiable Information in Document Images,” was conducted by Vishnu Prasad Vijaya Kumar (THWS and DataX), Santhosh Venkatesh (DataX), and Prof. Dr. Ivan P. Yamshchikov (THWS). The paper has been accepted to Findings of EMNLP 2026.
Read the paper: https://aclanthology.org/2026.findings-emnlp.2350.pdf
Research Contact
Vishnu Prasad Vijaya Kumar
Center for Artificial Intelligence (CAIRO)
Technische Hochschule Würzburg-Schweinfurt
vishnuprasad.vijayakumar[at]thws.de