Support Through Freedom
As we’ll see, supporting the work of the build-release engineer enhances DevOps by adding consistency, easing minds, and freeing DevOps teams to do more. To support the work of the release engineer, the enterprise must set them free as well, by letting them automate everything, treat manual interventions as process failures, and configure the target deployment machine and its monitoring context.
Enable the Release Engineer to Automate Everything & Treat Manual Interventions as Process Failures
One of the best ways to support the work of the build-release engineer is to encourage them to become actively hostile to manual interventions, says Robert Fischer, VP of Engineering, Webonise Lab www.webonise.com. Release engineers should shoulder the responsibility for putting manual interventions out of reach.
An enterprise’s internal DevOps community should empower the release engineer to automate as many steps as possible, which may require the enterprise to give them programmatic access to environment credentials, says Fischer. “For the internal DevOps community, the reward for permitting this risk is that they will have a cleaner environment that everyone understands better. This will make it easier for them to respond correctly in stressful situations,” says Fischer.
“So, let’s say you have some Linux VMs spinning up as part of the release. The release engineers could be tempted to use some kind of global SSH key for accessing these VMs, which they would install nicely on their laptops or workstations,” starts Fischer. Unfortunately, this will make it easy to add undesirable and unnecessary manual steps such as “‘Log in and set this IPTables value’ or ‘Log in and manually determine this hostname’”, says Fischer.
And while it would be easier to enable and apply these changes in the short run, it would leave a monkey wrench under the hood of automation and discombobulate the efforts of DevOps true believers in the long run. And it would all start simply by enabling the global SSH key and making these manual steps possible.
These kinds of manual processes also crop up in branches in the release logic: ‘Login to the server and see if packages X, Y, or Z are up-to-date in the image. IF they aren’t, THEN…’ All of that needs to be automated. “If the release engineers make it hard to implement manual steps, then they are more likely to A) notice and B) avoid these manual dependencies,” clarifies Fischer.
Instead of making manual steps possible, the release engineer should create active blocks for manual intervention, such as putting virtual machines within a virtual private cloud or making them only accessible using the automation tools’ SSH keys, says Fischer. This will actively impede manual interventions like those first cited herein and force the release engineer to think. “The result is that you are more likely to build the appropriate automation steps and error handling instead of creating a dependency on manual intervention,” says Fischer.
Empower the Release Engineer to Configure the Target Deployment Machine and It’s Monitoring Context
Another way to undergird the work of the release engineer is to open up the target deployment machine and monitoring context configurations to them. To enable the release engineer to configure the target deployment machine, make tools like Docker www.docker.com available. To enable them to configure the monitoring context, automate the configuration of the virtual machine platform (e.g.: AWS http://aws.amazon.com security groups, load balancers, etc.). “This allows for tuning to be a part of the release planning,” says Fischer.
The enterprise DevOps folks can still make manual adjustments if they must, but they should resist the impulse to use that option, except when handling anomalies. “Leaders in enterprise DevOps approaches can audit these deployment instructions for sanity and compliance, of course,” says Fischer.
As is the case with automating otherwise manual processes, the enterprise DevOps team should have a lot less to worry about once they set the release engineer free to configure these things. “This should give them more time to tackle the hard and interesting problems instead of performing repetitious and error-prone manual interventions by letting the computer handle the brain-dead computational and rote part of DevOps, which is what computers are good at,” says Fischer.
