An enterprise with many databases and various data requirements for DevOps continuous delivery presents a problem. For example, test data provisioning is a challenge today, particularly in pre-release testing of database updates. Standard practices involve shared database environments that need to be maintained from one test to the next, slowing test cycles and generally not fitting into an automated CI pipeline. As I explained in A New DevOps Database Platform is Needed, at best, it makes things chaotic; at worst, problems with database integrations with DevOps is a gross impediment to overall organization agility—which undermines the very purpose that DevOps is meant to serve.
A recent whitepaper, A New Kind of DevOps Database Platform, indicates that there are multiple reasons why a new kind of DevOps database platform is needed, which are:
- Compatibility requirements
- Integration with current CI/CD pipelines
- Integration with portfolios of value stream management
- Infrastructure cost efficiency
- Productivity for developers, testers and operations
- Security and governance
- DevOps performance
The paper explained that a DevOps database platform requires the following capabilities to satisfy these requirements.
- Automated database provisioning: the ability to deploy a database using automated procedures.
- Database cloning: writable database environments are delivered on demand, based on a built versioned image.
- Distributed repository: database images can reside on Linux and Windows servers, as well as storage systems.
- Simple declarative builds: a simple, predefined method to make the creation of versioned data images simple, while enabling user/group access controls, data masking and other data engineering functions.
- Scalable data cloning: database clones are served on demand, with many simultaneous clones served from a single image.
- Comprehensive database support: the repo supports all common relational, NoSQL, machine learning and file-based data stores.
- Versioned databases: each image becomes a versioned, immutable DevOps artifact.
- Deploys equally to all cloud environments: DevOps databases can be deployed and operated on any on-premises infrastructure or cloud.
- Cross-platform: DevOps databases can be deployed and operated on common operating system environments such as Windows and Linux.
- Security features including access management, data masking and encryption: database images easily incorporate access control, data masking and synthetic test data via SQL scripts, batch files or command line tools.
- Compatible with CI/CD platforms: DevOps databases can be deployed and operated equally well in multiple CI/CD platform tools such as Azure DevOps, Jenkins and CloudBees.
- Compatible with containers and container orchestrators: DevOps databases can be deployed and operated equally well in multiple container environments such as Docker, Kubernetes and Red Hat OpenShift.
- Deliver writeable data at the speed of DevOps: Deliver writable databases in seconds.
- API for easy integration with DevOps tools: An API is available to enable integration of the DevOps database control plane with other tools.
- Audit logging: The DevOps database includes a complete audit log of operations.
Transforming to the New Database DevOps Platform
A DevOps database platform in an enterprise requires more than simply buying and installing software and telling everyone to use it. With multiple entrenched database platforms already in place, there are practical and political considerations that must be overcome. Introducing any change to a large group of applications in an enterprise can be chaotic, and will often be more costly and take too much time if the changes are not compelling.
To be sufficiently compelling, the database DevOps platform needs to be affordable, easy to understand, easy to set up and use with a wide variety of CI/CD pipelines. The following will be sufficiently compelling benefits to drive adoption:
- Versioned, secure database environments or images managed as a DevOps artifact.
- Images can be built and managed by the DevOps team, without requiring new infrastructure.
- Multi-DB and multi-TB data environments in seconds.
- Database environments can be served on demand for a CI/CD pipeline.
- Lower-level database environment infrastructure is streamlined, with a single VM capable of supporting many simultaneous database environments which drives an immediate cost savings and positive ROI.
- Data security and governance benefits with versioned database environments, with built-in and auditable security policies.
What This Means
Digital transformation is imperative for modern enterprises and organizations to remain competitive. However, the transformations of many enterprises and organizations are seriously impeded by the challenges of integrating databases into their DevOps application pipelines and value streams. A new DevOps database platform can allow the databases to be treated like any other DevOps artifacts that are orchestrated and automated within the DevOps toolchains.
Suggested resources for automating database environment delivery include Windocks, Cohesity and other storage appliances that provide virtualized (cloned) database environments that can be adapted for DevOps purposes. I want to particularly acknowledge input from Windocks, who has introduced a DevOps data repo that features database virtualization on standard Linux and Windows servers, for versioned data environments and CI pipelines and cloud-native support for data delivery for containers, Kubernetes and conventional instances.
