Automation and DevOps undeniably improve the agility and responsiveness of IT. Most of us have taken up at least a bit of the wave of automation that DevOps and Agile have brought to our door, and many are far along on their DevOps journey.
But every once in a while, we need to look back and lend a hand to those still coming. The fact of the matter is that sometimes we forget to warn those who are new to massive automation about some serious issues.
It is intuitively obvious, if you think about security at all, that putting together a command and control server—we’ll use Ansible in this blog, but any tool that can roll out applications to a collection of servers—creates a big screaming target for any hacker that gets into your infrastructure. You cannot assume that will never happen, so organizations need to think about protective systems/architectures early, or spend more time patching things up. Remember, the more you build, the more weight there is to making major changes.
For starters, severely limit access to the command and control machine. It belongs on a management subnet/VLAN with only a few access points. The servers it is managing absolutely should not have access back to the command and control server. If they do, any one of them getting controlled will provide a pathway to all of them getting controlled.
Next up, early on there were quite a few debates/discussions about which user accounts should be able to run command and control scripts (Playbooks in Ansible). Eventually, the bulk of the community settled on each individual running them, not a single generic user with a name like “ansibledeploy.” Some still use a central user, but the majority wants the ability to track down the leak that individuals running commands allow—searching for strange/uncommon command execution in logs will cough up which account got compromised and allow for rapid response—which outweighs centralization.
Third, learn and use the security features available in your application. Ansible offers Ansible Vault to allow encryption of playbooks. Since many playbooks require root access, encryption allows those playbooks to hide root access. Use of Ansible Vault has grown over time, and there are excellent tutorials out there to help use it to lock down command and control. Your product will have a similar tool, because putting root passwords in plain text, all over the disk, is just not acceptable.
In the Ansible Galaxy/Github repositories, you can find excellent roles to help in your quest to lock things down. Geerlingguy is one of the most popular, and his Ansible Role – Security is excellent if you follow the caveats he provides in the readme.
This is simply an advisory, not a step-by-step solution set. Overall, just remember that putting together a command and control server makes a tempting target. And like Geerlingguy points out in his readme, your security is your concern.
Look at where the server can be accessed from, who can access it, where sensitive data bits such as passwords are stored—hint: plain text passwords in files went out of fashion when mainframes were hooked up to LANs—and how to protect all of the above. It’s been done, there are plenty of resources to help in your given environment. Take the time to look them up and implement relevant suggestions.
After all, I won’t get a call in the dead of night if your servers are all taken over, you will. Do everything you can to avoid getting that call. And keep rocking. Everyone remote working has made many organizations more susceptible to all forms of attack, don’t let your guard down.
