Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
The technology sector has always been about problem-solving. When the value of big data was finally embraced, thanks to new analysis capabilities developed in the late nineties and early aughts, the industry adapted its mindset toward storage by investing in on-premises data centers to help store the data that would drive better business decisions. When this rise of data accelerated software development and deployment, creating organizational silos between developers and operations, the industry responded by creating DevOps to improve collaboration.
Now DevOps has revolutionized the way companies do work by eliminating silos, increasing agility and creating greater visibility, resulting in faster deployments and better service overall. These new work styles have only increased the amount of data produced and the necessity to access it anytime, anywhere has made migrating to the cloud the only real viable solution to any successful DevOps team. Unfortunately, this has spawned a new challenge for the technology industry to solve: the cloud.
Data and the Cloud Predicament
Early DevOps adopters quickly leveraged the benefits of the cloud in increasing collaboration brought on by improved data accessibility. However, these early adopters typically entered agreements with the big three cloud vendors that offered increased storage capacity and a seemingly more flexible and accessible format. In most cases, these agreements provide instant gratification, but as DevOps expands over time new pain points are discovered as service and egress fees pile up and teams are forced to either increase budgets to accommodate storage needs or risk meeting limits and data loss.
This situation created a cloud storage predicament: Companies either pay for more cloud storage than they’ll ever need or have to choose what data was kept and what was deleted. These unsustainable vendor agreements introduce financial pain points for scaling up or scaling down storage space, outweighing the benefits of keeping all of the valuable data being produced and collected day-to-day. As a result, most are stuck thinking about how much and what to store instead of using all of its data to drive the business forward.
This form of vendor lock-in is a detriment to DevOps teams that rely on seemingly endless amounts of data to experiment, conduct maintenance and develop and deploy new applications.
Accelerating DevOps With a Bottomless Mindset
An IDC report recently indicated that the amount of data stored is now expected to hit 59 zettabytes this year alone, with the next three years of creation and consumption almost eclipsing that of the previous 30 years combined. And while DevOps has certainly contributed to this spike, the reality of our digital worlds is that data is king, and a company’s ability to seamlessly store, access and leverage it is what will set it apart from the competition.
The problem with this growth is finding ways to sustainably store this data in the cloud, particularly when teams are handcuffed to existing storage agreements that are draining budgets. This starts with DevOps teams rethinking their approach. Instead of thinking of cloud storage and accessibility as a recurring cycle of bills and service limit notifications, cloud strategies need to focus on the ability to only pay for the space needed at any given time without the worry of additional fees for accessing archived data. Need to scale up storage capacity? Great! Add as much as you’d like. Need to scale down for a particular reason? Don’t stress over unused real estate just sitting in your vendor’s data center. Need to access archived data at a moment’s notice? Don’t hesitate out of fear of a knee-buckling egress fee.
With this shift, DevOps teams can move away from conserving, archiving and destroying data toward gathering and utilizing all of it to drive the data-driven insights required in today’s digital economy. This change in mindset toward a “bottomless” cloud removes constraints and enables the ability to innovate at a faster and more sustainable rate.
Unlocking DevOps Innovation
Too often people think of DevOps and data management as being separate entities, but as software and application development evolves to include capabilities such as predictive analytics, it’s clear that a marriage between the two entities is needed. By rethinking its data strategy and embracing a bottomless approach, DevOps teams can unlock even greater efficiencies, leading to the next revolution in technology innovation.
