This is the first in a new series called DevOps in Action. A look at organizations who are putting DevOps into practice. Does your organization have experience putting DevOps into action that you think our readers might find useful or interesting? If so contact us at [email protected] and perhaps we can feature your story here.
Australia-based MYOB is a long-established developer of accounting systems and related software. During the last several years its original focus on desktop software has changed, and it now offers cloud-enabled and pure SaaS products. This change had two significant impacts on the company.
Firstly, it meant that instead of simply delivering packaged software to customers, MYOB now has to manage the infrastructure that its SaaS products run on.
Secondly, the traditional annual update cycle primarily driven by tax and compliance issues goes out of the window in a SaaS world, where the norm is continuous improvement based on customer feedback.
“Traditional operations management just doesn’t keep pace with that,” said MYOB business division product delivery manager John Sullivan. “The pace of change has changed dramatically.”
Not only are there more ideas for improvements and new features, but the changes are increasingly “externally fuelled,” he said. This makes it difficult to plan infrastructure requirements very far ahead.
MYOB joined the DevOps movement as it was seen as a way of releasing better software more quickly, and of ensuring that the hardware could be scaled to meet the demands of new software releases.
At some companies, DevOps is about building infrastructure for developers to use, or having operations build infrastructure in parallel with software development, he said.
But MYOB now has operations staff that literally work alongside developers to translate features requirements into infrastructure requirements, and then use repeatable methods to build that infrastructure.
“We scale our infrastructure at the time features are being built,” said Sullivan.
The company uses multiple cloud providers, reflecting the varying needs of its 40-plus products. AccountRight Live (a desktop program that stores its data in the cloud) was written using .NET, so it made sense to use Azure for the cloud element.
And it has adopted a microservices architecture so a single implementation of a function can be used by multiple products (read more about this in ‘MYOB moving ahead with microservices’ http://www.itwire.com/business-it-news/accounting-software/65878-myob-moving-ahead-with-microservices). These microservices mostly run on Amazon Web Services.
Everything else runs on virtual cloud infrastructure provided by Macquarie Telecom.
Using these larger providers means MYOB can be confident that the physical resources are available whenever they are needed, without having to invest large amounts of money in private infrastructure. Having the infrastructure “come alive” at the moment any particular piece of software goes live provides significant savings.
MYOB had an important advantage when starting its DevOps journey: the need for operations staff was new, as it was going from being a traditional software vendor to also offering SaaS products. Sullivan said that made it easier to put development and operations staff together than it would be for an organisation that has a history of managing its own infrastructure, especially with a steady release cadence as opposed to the rapid tempo associated with cloud services.
“We were very lucky that we needed to manage infrastructure for the first time,” he said.
But there were some cultural issues, particularly around making the delivery teams understand they had to take infrastructure into account when building features, as it is important that the cost of delivering the service is kept within reasonable limits. So developers need to tap into operational expertise to ensure that the patterns they use will scale adequately, and that the hooks for monitoring and managing the software are included.
MYOB started with an independent DevOps team with an oversight role, but this was less than optimal, Sullivan said. “If you say you have a ‘DevOps team,’ you probably haven’t” because the closer that development and operations staff are, the better the systems become – getting the right design means looking at application features from several different angles.
Those on the operations side need to be able to advise developers about the software patterns and tools that will be able to handle increasing load. Not only does that potentially avoid the need to build complex failover arrangements, it also saves you running into a situation where you have to re-architect the whole system when demand reaches a certain level. “That’s the advantage of having operations and development people close together.”
MYOB uses the Agile methodology, and infrastructure concerns and requirements are included on feature cards so the software and infrastructure can be built together. As the company gets better at DevOps, developers become increasingly aware of infrastructure requirements and so they build as they go – “we’re just about there with one of our teams.”
“You need operations people with the mindset of a developer … or you’re going to struggle,” he said. For example, you want systems that can automatically scale on demand, and that requires scriptable deployment processes: “a difficult thing to do at times.” Fortunately for MYOB, the company was able to find the right people. It can be hard for development staff to adapt to DevOps, Sullivan suggested, but it’s even harder for traditional operations staff.
One of the ‘tricks’ used by MYOB to build the DevOps culture was to rotate developers into operations roles after the software they had been working on entered production in order to learn about operational requirements. “Then they actually feel the pain of phone calls in the middle of the night if something goes wrong.”
Another is that the company favours ‘self-reflective teams’ (regardless of function) that review what has or has not worked for them during the last two weeks, and adapt their practices accordingly. DevOps is just one area where this practice is used. Sullivan notes that because the environment is so dynamic, ideas are not rejected permanently. So teams can decide that just because a certain technique or idea did not work on a previous project, it might be successful this time because the situation is different in a relevant way.
Early in the transition MYOB filled some key roles with DevOps practitioners that had Azure and AWS experience. But Sullivan warned “there are very few conferences or training courses on DevOps … so most companies don’t do DevOps correctly.” Consequently, special care is needed when hiring people that claim experience in the field.
“The term ‘DevOps’ is overused and sometimes misunderstood.” So some people say they have DevOps experience when all they really know about is doing scripted builds, and do not have the skills and practices to be embedded in a DevOps team.
“There are few people that have the right mindset,” said Sullivan, so it is very important to check a candidate’s skill set against your requirements – and on the flip side, to take the steps necessary to retain your good DevOps staff.
So did MYOB get what it wanted from DevOps?
One initial goal was to go from the traditional software-only annual release to a six-week ‘software and infrastructure’ cycle. “We met that pretty quickly,” said Sullivan, and the company is now targeting ‘any time’ releases for online services with minimal or no interruption. This will allow the company to pick and choose which new features are to be released and when according to marketing priorities, rather than being hamstrung by arbitrary release cycles.
The other was to achieve “one click deployment,” that is to be able to release code and infrastructure to customers with no manual tasks beyond initiating the process. The automated process should either run to completion, or back itself out if something goes wrong. This has been achieved for most products, the main exception being those for mobile devices because the requirements of the various app stores get in the way.
“Other companies may have different metrics, but ours were about release capability and infrastructure efficiency,” said Sullivan.
Asked if MYOB had any regrets about going down the DevOps route, the answer was succinct: “None whatsoever.”
And his advice for others considering following this path sounds sensible for any part of working life. “Embrace the idea that you have to grow and learn, and keep making the best decisions you can at every step.”
About the Author:
Stephen Withers is one of Australia¹s most experienced IT journalists, covering everything from gadgets to enterprise systems. In previous lives he has been an academic, a systems programmer, an IT support manager, and an online services manager. Stephen holds an honours degree in Management Sciences, a PhD in Industrial and Business Studies, and is a senior member of the Australian Computer Society. You can see his Linkedin profile here: http://au.linkedin.com/pub/stephen-withers/0/222/687/
