Why Digital Government Records Are So Hard To Preserve

In May, a national judge ordered White House unit to comply pinch nan Presidential Records Act, nan 1978 rule that makes a president’s charismatic records nationalist property and governs their preservation and eventual release.

A period earlier, nan Justice Department had argued nan rule exceeds Congress’s law authority. The American Historical Association and nan watchdog group American Oversight sued, informing that nan sentiment could fto nan White House wantonness policies meant to restrict officials from conducting authorities business done individual email aliases encrypted messages. The risk, they argued, was a existent nonaccomplishment of accountability and a imperishable spread successful nan humanities record.

Judge John D. Bates has truthful acold recovered nan rule “likely constitutional.” But nan tribunal conflict is conscionable 1 portion of a overmuch broader challenge. The records that uncover really governments and nationalist figures make decisions are now calved successful email, chat apps and unreality documents, often wrong proprietary systems whose lifespans are measured successful merchandise cycles. Preserving them agelong capable for nan nationalist to spot them has go a method problem successful its ain right, 1 that grows harder arsenic nan measurement climbs. The National Archives added 463 terabytes of physics records to its imperishable postulation successful 2024 alone.

On supporting subject journalism

If you're enjoying this article, see supporting our award-winning publicity by subscribing. By purchasing a subscription you are helping to guarantee nan early of impactful stories astir nan discoveries and ideas shaping our world today.

“The world is creating integer records astatine a gait nary statement anticipated,” says Mike Quinn, CEO of integer preservation institution Preservica.

Before archivists tin sphere a record, nan grounds must past agelong capable to make it into their hands. Public-records laws tin require preservation, and nan exertion exists to seizure and shop messages moreover from immoderate encrypted platforms erstwhile accounts aliases devices are configured to clasp them. The integer preservation institution Smarsh, for instance, advertises it tin seizure information from much than 100 communications channels. But caller incidents propose really easy important records tin still vanish, from U.S. Cabinet officials discussing subject plans via nan encrypted app Signal to UK Prime Minister Keir Starmer’s reported use of disappearing WhatsApp messages.

The aforesaid fragility follows backstage archives too. Even erstwhile individuals specified arsenic politicians aliases artists—or their estates—donate beingness papers to a assemblage library, nan integer worldly that erstwhile sat alongside them tin beryllium overlooked and lost, says Thorsten Ries, an adjunct professor astatine nan University of Texas astatine Austin who applies digital-forensics techniques to archival work.

Pulling nan information disconnected a difficult thrust aliases USB thrust without altering files aliases metadata for illustration timestamps besides takes skill, Ries says. Different package versions, and moreover different retention media, tin sphere different record fragments and automatic backups. Those connection valuable clues to really a archive was drafted and really its creators thought, but recovering and interpreting them is painstaking, specialized work. “This benignant of knowledge and expertise is really still very sparse,” he says.

Cloud-based systems specified arsenic Google Docs tin clasp nan astir elaborate record histories of all, but extracting files from them without nan original passwords and two-factor authentication is its ain challenge, he adds.

Survival is conscionable nan first step; nan worldly besides must stay readable arsenic package changes. “All these types of integer contented don't property for illustration paper,” Quinn says. “They go unreadable erstwhile formats go obsolete.”

That often requires regularly migrating worldly for illustration connection processing documents, spreadsheets and computer-aided creation files to existent record formats while keeping a observant log of precisely what’s been done. If handled carelessly, those conversions tin misrepresent nan original, says Christopher J. Prom of nan University of Illinois Urbana-Champaign library. That appears to beryllium what happened when nan Justice Department released emails tied to nan precocious financier and activity offender Jeffrey Epstein that were marred by rendering errors.

A preserved record tin still beryllium difficult to use. Digital archives tin incorporate copyrighted worldly alongside delicate correspondence, including individual messages and aesculapian bills, sitting successful nan aforesaid inboxes and folders arsenic nan files a interrogator wants. That makes institutions cautious astir opening collections broadly. And though a integer record could successful mentation beryllium opened from anyplace pinch an net connection, archives still routinely require an onsite visit, if they assistance entree astatine all, says Lise Jaillant, professor of integer taste practice astatine Loughborough University. Researchers must schedule and salary for travel, past comb done tremendous collections connected perchance unfamiliar systems successful immoderate clip they have.

The “staggering volumes” of integer worldly produced by U.S. authorities agencies person likewise slowed nan handling of Freedom of Information Act requests, says Jason R. Baron, a professor astatine nan University of Maryland’s College of Information and erstwhile head of litigation astatine nan National Archives and Records Administration. Agencies must first effort to find perchance applicable files, often by keyword search, past region aliases redact thing classified, sensitive, aliases different exempt from disclosure.

“It is not different for a requester to hold years aliases moreover successful immoderate cases complete a decade to person complete responses,” Baron says.

Automation whitethorn help, pinch important quality oversight. In a 2025 paper, Baron explored utilizing artificial intelligence and machine-learning techniques to emblem paragraphs apt to beryllium exempt nether nan FOIA proviso that shields an agency’s “deliberative process.” Software tin besides thief spot delicate accusation for illustration Social Security numbers and extract matter from scanned documents aliases archived video done optical characteristic nickname and automated transcription.

AI tin besides aboveground files applicable to a peculiar mobility successful a sprawling archive, including documents a elemental keyword hunt would miss. As Baron points out, nan aforesaid techniques are already utilized successful litigation for physics discovery, erstwhile immense sets of firm files, emails, and different records often must beryllium searched for worldly base connected a lawsuit.

Still, challenges remain, says Jaillant, who is leading an world task on AI’s applications to authorities records. One is simply a shortage of publically disposable email information to train AI to grip messages of various types and origins. Partly because of privateness concerns, researchers still often thin connected a now-decades-old group of messages that authorities investigators obtained from Enron, Jaillant says.

And moreover arsenic AI gets amended astatine parsing archival material, it is improbable to relieve quality researchers of nan request to publication nan applicable documents themselves. “It's still important for a quality personification to spell backmost to nan documents and beryllium capable to publication individual emails conscionable to understand nan context,” she says.

All of that assumes nan records past agelong capable to beryllium read—which is precisely what nan conflict successful Washington has put successful doubt. Archivists, and nan package they dangle on, are moving to make judge they do, earlier nan records of today’s decisions go trapped successful dormant formats aliases erased from connection threads without nan nationalist ever getting nan chance to spot them.