Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
One of the highlights of AWS re:Invent, judging by the twitter stream and the points emphasized in Amazon CTO Werner Vogel’s Day 2 keynote and blog post, was the launch of Lambda.
In his blog announcing Lambda, Vogel calls the solution “the easiest way to compute in the cloud.” He goes on to describe it as both a way to run arbitrary code on AWS with complete abstraction away from the underlying infrastructure including EC2 instances, AND as a fundamentally better way to compose applications through the use of Lambda as an event-driven approach to intra-application behavior.
What struck me as interesting was Vogel calling Lambda “an event-driven computing service for dynamic applications.”
In his twitter feed, he has this to say about it:
Most heard comment at #reinvent “dude, lambda is the coolest shit, ever” http://t.co/b1DBbV8NMZ
— Werner Vogels (@Werner) November 15, 2014
So Lambda is cool, and is getting a massive amount of interest. And it has a lot to do with being event-driven.
A natural question that has arisen is could AWS Lambda be used for the automation of operations? And how does Lambda compare to the home-built scripts that are used by many operators to tie events to actions or to broader automation projects like StackStorm?
So, what is similar and what is different?
Deployment model:
The easy and obvious comparison is that scripts and projects like StackStorm or Chef for example or others are open source software used on premise or by users on their public clouds, whereas Lambda is of course an AWS service.
What events:
AWS Lambda comes already able to listen to a few different publishers of events from within AWS – “either a particular Amazon S3 bucket, Amazon DynamoDB table, or Amazon Kinesis stream.” By comparison, your own scripts or other automation software are of course completely agnostic: you can integrate with anything you want, you just need to write that software.
Scheduling:
AWS Lambda does not currently enable you to run events on a particular schedule, whereas some solutions like StackStorm come with such an ability built-in. Or you can leverage chron itself or, if you are Mesos aficionado, you might select chronos as did AirBnB for example.
What code:
Lambda today needs JavaScript / node.js; this is a real restriction however AWS says they will likely expand the range of supported software.
By comparison, Python is extremely common in home-made automation systems. Also many solutions including Ansible and StackStorm support the use of YAML to define workflows and other automation.
Method of execution: Workflow included or abstract?
Speaking of workflow, some automation solutions include workflow to tie together underlying atomic events. Like the use of YAML, workflow can simplify the process of authoring and then maintaining automation.
You could do the same with Lambda; however, it would require you to write your own workflow within JavaScript. Corner cases in workflows are unfortunately often exciting resulting in a lot of important code to write and maintain so the lack of workflow would seem to be an important difference; presumably AWS will eventually address this by integrating their own simple workflow into Lambda.
Abstraction of underlying resources:
Lambda enables you to auto scale and to not worry about what the underlying resources are when you run it: “AWS Lambda seamlessly deploys your code, does all the administration, maintenance, and security patches, and provides built-in logging and monitoring through Amazon CloudWatch.”
By comparison, scripts and other automation projects are just code. You can run them in an autoscaling group, but then you have to manage that. Lambda is also free up to a certain volume as AWS is betting on it further increasing the stickiness of your usage of AWS.
Community:
It is pretty easy to argue that open source projects have larger communities than AWS. And also easy to argue that by being dependent on AWS and Lambda you are locking yourself in ever tighter. On the other hand, Amazon continues to cut prices and their development pace is incredibly fast. I tend to believe in the power of open innovation however especially as Amazon regularly strip mines open source projects – exploiting those advances and adding their own – it could well be that Amazon is going to be able to progress at least at the pace of open source projects.
Summary:
On the whole, AWS Lambda is pretty clearly intended to be the internal wiring for AWS based software by tying together many loosely coupled services into an overall application. By comparison, automation software including your own scripts are used in operations to improve efficiency and frankly to eliminate some of the annoying manual work that otherwise would need to be done; also, home-built automations and automation projects are increasingly built to be easy to author and maintain whereas you have to be a java script coder to use Lambda; last but not least, enterprises are extremely heterogeneous so they must have a broad range of integrations available or easy to add including multiple public cloud providers and on premise technologies as well.
That’s not to say that AWS Lambda and open source automation projects will not converge. It seems likely that there is enough demand on the part of users to support multiple approaches to automation as automation is increasingly seen as one of the fundamental pillars of DevOps, along with all important cultural and skills improvements.
