Are we at risk of losing our digital history?

As the world becomes more digitalized, we are at of risk losing our history. We are already seeing this happen in ways that we could have predicted -- as storage media deteriorates or is lost to natural disasters and other events. But media deterioration is not the most serious of our digital preservation challenges. We are saving files everywhere, often with little idea of how future generations will find them, or whether they will even have an appropriate application to open them. Most digital preservation strategies feel like the digital equivalent of the massive warehouse in Raiders of the Lost Ark -- petabytes and petabytes of information that we'll "someday" go back and sort through.

What is the cost of negligent digital stewardship? If the past two decades are any indication, it will be high. Not just in dollars and cents, but also in terms of what we lose about our history. What will future generations know about our culture, values, and accomplishments if they can't access our records? And what do we need to do NOW so that they have a chance? In this blog post, I'll explore these important questions - and offer recommendations for what you can do now.

I have spent an inordinate amount of time over the past five years working to understand the story of my pair of Italian immigrant grandparents (https://www.searchformygrandparents.com/). They simply disappeared in the 1930s, never to be seen, mentioned, or heard from again, a disappearing act that lasted for 122 years. At the core my search has been records -- some surfacing now only because of the application of sophisticated OCR and AI technologies to provide context and findability to records gathered a century ago. After having spent 25 years working in a professional capacity with records and governance people, I have new appreciation for the smart decisions that were made a century ago that allowed this information to be preserved and findable in 2021. Preservation decisions made a century ago preserved information in a stable media -- paper -- and scanning, OCR, and AI innovations in the past decade have made this information available and useful to a degree never before possible.

So why is thinking about preservation important right now? Isn't this all a lost easier when so much information is born-digital? Think again.

Communication has moved from an oral practice meant to survive generations, to a written one meant to survive centuries, to a digital one that too often survives mere weeks.

Publish or Vanish, 2020

According to research I conducted for AIIM, The volume of information grows by a factor of four every two years. Over 60% of this information is unstructured "stuff" (application files, jpegs, Office documents, audio, video, and in countless other varieties), which is all stored on hard drives, file shares, memory sticks, on laptops, in the cloud, and [insert your own favorite place that you store information!]

How resilient will our digital history be based on these haphazard practices? What decisions do we need to make NOW so that future generations can access this information and find the answers they're looking for - not just about us, but also about themselves? In both our business and professional lives, what are some things we can do to sort through all of this chaos and make sure that we preserve what is truly important?

Some steps you can take today:

Understand the problem. Invest time to understand why the digital preservation challenge is at such a profound tipping point. Vendors like Preservica have a variety of white papers, case studies, and videos outlining the nature of the problem, and illustrate how true preservation is different from traditional records management. Analysts like Cheryl McKinnon from Forrester have worked for years to help organizations understand the risks of digital amnesia.

Elevate metadata to center stage. At a personal level, label everything with metadata that will help you search for it later. Regardless of the format, always create a title for the document, add keywords when appropriate and keep track of what year and month it was created in. Add tags to your files as well. At an organizational level, make sure you are investing in systems to automate all of this, because if we have discovered one thing in the last quarter century, it's that expecting end users to care about records and governance is a fool's errand.

Review current records and governance policies and procedures. When were your records, governance, and preservation policies last reviewed, and by whom? In technology terms, the world has shifted on its axis in the past decade; a policy written before the cloud revolution is likely mostly useless. What are your criteria for what needs to be retained, and for how long? How do you know when it's time to dispose of something instead? And who is accountable for making sure that the policies are followed?

Create a storage plan that will scale as the volume of records continues to grow exponentially. Define archiving and purging criteria that can be quickly implemented at all levels, from individual users to corporate managers. Truly commit to automated disposition; a strategy that relies on "somebody" at some future date to manually review thousands of "old" documents and be held accountable for their destruction is a dream that will never be fulfilled.

Adopt a "forward-leaning" bias. Every organization that has ever undertaken a content or records project struggles with the balance between focusing day-forward or managing all of the information that has already accumulated. There is no perfect answer. But given the rising scale of the chaos problem as the volume and variety of information rises around us, my bias is to lean forward.

The digital age is here, and we need to start thinking now about how we can preserve it. Protect the digital records of today so that future generations will have access to them tomorrow. The good news we have a long history of saving paper records in archives - which has been crucial for preserving the past, so it can be understood by subsequent generations. We know how to do it and we've learned a lot along the way.

Digital preservation is not a new problem, but it's one that has become significantly more urgent and complicated in the last few years. There are principles from the world of paper archives that can be applied to digital preservation, but It takes investment and innovation to solve this growing challenge. The truth is that we're not doing a very good job. We are quickly running out of time, and it's no longer enough to simply back up our data in the hope that maybe someday someone will figure out how to find it again. We need to start thinking proactively about what work needs to be done now.

The question now is how do our old approaches need to change to guarantee digital preservation, and who in our organizations will step forward to lead this charge?

Have you ordered your copy of IMMIGRANT SECRETS? Get it today.