Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
It is interesting to consider DevOps from the storage perspective. DevOps in general—and specifically newer development and deployment models—tried to pretend that long-term storage was not a necessary component of software delivery. In short, it appears that because there wasn’t a good answer to long-term state in stateless development models, the storage and databases that used state were hand-waved away. Which, of course, failed. Container deployment models did the same thing and failed just as miserably. Oh, containers didn’t fail, but the attempt to ignore persistent storage did; from the very beginning, it was clear that was going to happen. We were doing early container implementations and working through how to ensure that containers ABC had access to the same shared storage. That need never went away and was never going to. Apps can run without persistence, but business doesn’t run without persistent data.
Checking out the wave of cool new products like, for example, Ionir (disclosure—I know more about them simply because I’m friends with their CMO. Other vendors likely have good solutions, too) makes me realize the answers that we needed were right in front of us. And not just the answer to persistent storage, but to storage portability, which has been a problem from the beginning of the cloud. Data gravity is the idea that data holds apps in place. Tools like Ionir reduce the gravity of data. A lot. They can’t get rid of data gravity, because some platforms charge for data storage, data ingress and egress, etc. but that becomes a TCO operations problem rather than a near-insurmountable tech problem.
But just as interesting is this: What if the slow but steady increase in core Internet speeds will allow us to access data from anywhere? We could do that now, but performant applications struggle if you force them to leave cloud A and go to your data center or to cloud B to get a dataset. Yet we use APIs that do just that on a regular basis. What are Google Maps and Google Places if not simply big old honking databases? And yet we call out to them from just about every platform.
So, it’s worth trying out and seeing if the idea of data gravity is maybe an obsolete idea for you. For some applications that really do need super-fast response times, the concept is still relevant. But for many applications? Where your app sits in relation to data may not be as important as you think. Write or grab an API wrapper for a critical dataset and throw a simple test app up on a different platform. See what you get. If it performs well, consider putting apps where it makes sense and using a wrapper API with secrets to grant access to data from any validated app.
Some of you are doing this already. Not many, but it’s out there. Excellent job; keep rolling!
Everyone else is kicking rear and taking names, with the world still online every day because of your collective work. So keep rocking it, but consider if you really need the application close to the data these days and, if you do, how data mobility tools might facilitate moving data to the app—something most of us would not have considered for large datasets before the last year or so.
