As preamble for this post, I wrote about the Unicorn Installer challenge. The challenge is that user needs are too deeply heterogeneous for good reasons to create a vertically integrated way (aka an “easy button”) that works for all users. The problem is not just current diversity, it’s also about how we cope with needed changes in the future that would force users to straddle configurations.
Don’t despair! There is a practical solution to shared ops on project installers: composability.
If projects focused on maintaining install modules for their specific components, then the community could collaborate on well-bounded problems that should be infrastructure-agnostic. Even building in multiple scripting DSLs (Ansible, Chef, Puppet, Salt or Bash) is a much smaller-scope problem. Further, all of these DSLs have modularity concepts so it’s feasible to pull in community roles/modules/gains/plays into larger automation script. This approach allows us to decouple snowflaked infrastructure and prerequisite configuration from the project configuration.
At that point, the proverbial “easy button” becomes an opinionated set of project, prerequisite setup and infrastructure-specific steps. These then become vendor-specific buttons: “Press here for the fast AWS install with Dejour SDN and L8istSh9y O/S” as was showcased to great effect at Dockercon.
With that said, let’s look at how trying to create an “easy button” installer inside the project backfires.
Cloud Foundry chose to have an in-project installer, BOSH, that has an opinionated installation approach based on generated images. To work, the installer has to maintain an increasing number of BOSH-specific integrations and image generators. These efforts are redundant with other tools non-core to installing Cloud Foundry. Further complicating the story is Pivotal, a primary Cloud Foundry maintainer, which considered BOSH a product differentiator. This creates ecosystem conflict with other Cloud Foundry vendors. While BOSH helps create robust installations, it also drags along a lot of baggage that keeps those improvements from being more generally accessible. For all of the expected ease of installation for BOSH, Cloud Foundry is still considered difficult to install.
OpenStack installers are a very confusing story, with multiple competing in-project installers. Starting from the single-node DevStack, which provides all of the continuous integration/continuous delivery (CI/CD) testing to the confusing over-cloud/under-cloud OpenStack-On-OpenStack (aka Triple-O) concept. Neither of these initial in-project attempts created practical, universal installers. In fact, they’ve spawned waves of other installers and configuration scripts. Recently, OpenStack decided to allow many of these attempts into the project (aka “big tent”) without coordinated governance making it impossible for operators to determine which efforts they should be tracking. Having some efforts stamped with “community approved” only makes it more difficult to solve cross-platform issues. The effect is further balkanization, as vendors and advanced operators attempt to build working automation silos against competing interests.
As a positive example, Linux pre-seed and kickstart options are distribution-specific and support a healthy ecosystem. It would be easy to think of consolidating these installs into a single system; however, the likely result would be to lock together rivals and slow down innovation.
What we want is a thriving ecosystem around a project without endorsement as one being THE install process.
Why is lack of endorsement critical? It allows the project to advance without being slowed down by installation entanglements created when a single way is anointed. Worse, it discourages ecosystem activity around the project.
The first successful installers were targeted at a single vertical stack, such as Kubernetes on CoreOS or a version of Salt Kube-up on Google. These monolithic install approaches are helpful jump-starts but limit community diversity. Yet, if they are presented as “the right way,” then other participants will be discouraged from innovating. It only takes one variation in a vertically integrated process for the whole process to become exception hell.
Our early experience with Crowbar was that our initial installation ideas were not going to remain useful as the project and targets expanded. It was important that we could maintain Crowbar AND replace it with something better. The fundamental lesson is that operations around projects is an evolving practice.
Hindering the evolution by selecting winners early actually slows the development process.
This may be counter intuitive; most projects want to embrace “easy button” installers to reduce start-up costs. These critical builders of initial traction may actually undermine projects as they enter a operational maturity phase by creating a fidelity gap between initial and sustaining use cases.
