DevOps’ ultimate aim is to create efficiencies in the software delivery process. But it doesn’t always work out that way. Sometimes DevOps initiatives lead to mistakes—something we lovingly call a “DevOoops.�? CloudBees recently polled some colleagues and came up with five examples of a DevOoops in action. Following is a discussion submitted by solution architect Will Refvem about siloed departments.
Scenario:
One of the biggest challenges in DevOps is SCM. Early in my career, being given access to Git repositories was a bit like handing a belt-fed machine gun to a drunken toddler. My biggest DevOoops moment came when I was trying to master the basics of Git in a project that used Git submodules—don’t, just don’t—and had a series of bash scripts that implemented a primitive CI/CD workflow by running in the environments we were deploying to (seriously). I can’t tell you how many merge conflicts and detached HEADs I encountered, or how I got out of them. Fortunately, we were using the fork-PR model, so I did no real damage.
Software configuration management (SCM) is key not only for versioning, but also for providing a shared view of code and other assets, from development through to production. Giving proper consideration to your SCM structure, branching and merging model, access controls, etc., will support collaboration, reliability and traceability—all key for DevOps success.
In the 2019 DevOps and Jenkins Community Report, 66 percent of organizations respondents reported the use of modern solutions SCMs, specifically GIT-based solutions GitHub or BitBucket. But the older SCM solutions aren’t going away completely—45 percent of all organizations still use pre-GIT, pre-DVCS (distributed version control systems) such as Perforce, Subversion and CVS. While it’s possible to apply DevOps practices using these legacy solutions, it is important (and more challenging) to design a branching/merging model and surrounding workflow, which supports the key aspects of DevOps such as peer review, collaboration and short-lived branches.
Modern SCMs such as GitHub have features that inherently support collaboration, including the game-changing Pull Request, making it much easier to implement processes that best support CD and DevOps.
Poorly designed CD workflows or inadequate CD systems create friction multiple ways. If the interactions are too cumbersome, users will not commit code to the pipeline frequently enough. If it’s difficult to access the pipeline and create visibility into the process, it can lead to unresolved conflicts.
You need to pick the right tool. If you don’t have the option to get the modern tool, you need to take the time to design a legacy tool.
It’s also important to consider two other issues in relation to this DevOoops example: branching and acquiring the appropriate level of DevOps skills.
- Branching: A proper branching and merging model is required to ensure you are feeding the appropriate inputs into your DevOps pipeline, such that you are able to continuously integrate, test and ultimately deploy changes. A simple guide here is encapsulated in the KISS (keep it simple, stupid) principle. If you need a full-wall whiteboard or a spreadsheet fit for an accountant to list your active branches, you need to rethink things. Get as close to the ideal of mainline trunk development as possible, with short-lived feature branches, minimal application versions, etc.
- Skills: As organizations seek to implement DevOps, most likely new tools and practices will be implemented simultaneously. It’s important to understand and accommodate the fact that there will be bumps and bruises as you learn to use these new tools and practices. For instance, shifting from centralized version control such as Apache Subversion (SVN) to distributed version control such as Git is a paradigm shift that can be difficult to fully grasp and result in destructive mistakes, such as deletion of nodes from the change history, overwriting changes or misconfiguration access control. Consider hiring for these new skills and/or account for time and training.
Takeaways
- A proper branching and merging model are required to ensure that you are feeding the appropriate inputs into your DevOps pipeline to let you continuously integrate, test and ultimately deploy changes.
- If you are using a pre-Git legacy SCM solution, develop a branching and merging strategy within the legacy solution to achieve appropriate velocity in your DevOps model.
- Consider that you will need to develop new skills to adopt new SCM tools and practice and account for this in your plan.
