AWS Official Blog

As people across the Northeastern United States are stocking up their pantries and preparing their disaster supplies kits, AWS is also preparing for winter snow storms and the subsequent hurricane season. After fielding several customer requests for information about our preparation regime, my colleagues Brian Beach and Ilya Epshteyn wrote the following guest post in order to share some additional information.

AWS takes extensive precautions to help ensure that we will remain fully operational, with no loss of service for our hosted applications even during a major weather event or natural disaster.  How reliable is an application hosted by AWS? In 2014, Nucleus Research surveyed 198 AWS customers that reported moving existing workloads from on-premises to AWS and found that they were able to reduce unplanned downtime by 32% (see Availability and Reliability in the Cloud: Amazon Web Services for more info).

AWS replicates critical system components across multiple Availability Zones to ensure high availability both under normal circumstances and during disasters such as fires, tornadoes, or floods.  Our services are available to customers from 12 regions in the United States, Brazil, Europe, Japan, Singapore, Australia, Korea, and China with 32 Availability Zones.  Each Availability Zone runs on its own independent infrastructure, engineered to be highly reliable so that even extreme disasters or weather events should only affect a single Availability Zone. The datacenters’ electrical power systems are designed to be fully redundant and maintainable without impact to operations. Common points of failure, such as generators, UPS units, and air conditioning, are not shared across Availability Zones.

At AWS, we plan for failure by maintaining contingency plans and regularly rehearsing our responses. In the words of Werner Vogels, Amazon’s CTO: “Everything fails, all the time.”  We regularly perform preventative maintenance on our generators and UPS units to ensure that the equipment is ready when needed.  We also maintain a series of incident response plans covering both common and uncommon events and update them regularly to incorporate lessons learned and prepare for emerging threats.  In the days leading up to a known event such as a hurricane, we make preparations such as increasing fuel supplies, updating staffing plans, and adding provisions like food and water to ensure the safety of the support teams.  Once it is clear that a storm will impact a specific region, the response plan is executed and we post updates to the Service Health Dashboard throughout the event.

During Hurricane Sandy—the most destructive hurricane of the 2012 Atlantic hurricane season, and the second-costliest hurricane in United States history— AWS remained online throughout the entire storm. An extensive Hurricane Sandy Response Plan, including 24/7 staffing by all service teams, escalation plans and continuous status updates, assured normal operations and service quality for our customers.

In fact, AWS’s highly reliable platform also played a key role in enabling a more effective storm response.  A&T Systems (ATS.com), an AWS Advanced Consulting partner, used AWS in support of a statewide emergency management agency as Hurricane Sandy struck.  Another AWS customer, MapBox, provided maps for several storm-related services to help predict and track Sandy’s progression, communicate evacuation plans, and track surges.

In the aftermath of the storm, some companies established operations in the AWS Cloud to replace datacenters lost to flooding and power outages. One such example is NYU’s Langone Medical Center. As noted in the article Still Recovering from Sandy, “…NYU researchers [were] able to push forward with their sequencing experiments. They were able to salvage 200 terabytes of backup sequencing data, and have set up temporary data storage in a New Jersey facility, using computing power from the NYU Center for Genomics and Systems Biology and the Amazon cloud.”

What’s even more interesting is that AWS provided a unique capability for our customers to prepare for worst case scenarios by copying and replicating their data to other AWS regions proactively.  Although ultimately this was not necessary, since US East (Northern Virginia) stayed up without any issues, our customers had peace of mind that they would be able to continue their business as usual even if it did fail.  One example is the Obama 2012 Campaign: in a nine hour period, they proactively replicated their entire environment from the US East (Northern Virginia) to the US West (Northern California) region, providing cross-continent fault tolerance on demand.  The Obama campaign was able to copy over 27 terabytes of data from East to West in less than four hours (watch the re:Invent video, Continuous Integration and Deployment Best Practices on AWS, to learn more).  Leo Zhadanovsky, a DevOps engineer for the Obama Campaign & Democratic National Committee, who now works for AWS commented that “AWS’s scalable, on-demand capacity allowed Obama for America to quickly spin up a disaster-recovery copy of their infrastructure in another region in a matter of hours — something that would normally take weeks, or months in on premise environment.”

While AWS goes to great lengths to provide availability of the cloud, our customers share responsibility for ensuring availability within the cloud.  These customers and others like them have succeeded because they designed for failure and have adopted best practices for high availability, such as taking advantage of multiple Availability Zones and configuring Auto Scaling groups to replace unhealthy instances. The Building Fault-Tolerant Applications on AWS whitepaper is a great introduction to achieving high availability in the cloud. In addition, the AWS Well-Architected Framework codifies the experiences of thousands of customers, helping customers assess and improve their cloud-based architectures and mitigate disruptions.

As winter storms threaten the East Coast, AWS customers can rest assured that our Services and Availability Zones provide the most solid foundation upon which to build a reliable application. Together, we can build a highly available and resilient application in the cloud, ready to weather the storm.

— Brian Beach (Cloud Architect) and Ilya Epshteyn (Solutions Architect)

I am fascinated by things that are simple on the surface and complex underneath! For example, consider the popular padlock icon that is used to signify that traffic to and from a web site is encrypted:

How does the browser know that it should display the green padlock? Well, that’s quite the story! It all starts with a digital file known as an SSL/TLS certificate.  This is an electronic document that is used to establish identity and trust between two parties. In this case, the two parties are the web site and the web browser.

SSL/TLS is a must-have whenever sensitive data is moved back and forth. For example, sites that need to meet compliance requirements such as PCI-DSS, FISMA, and HIPAA make extensive use of SSL/TLS.

Certificates are issued to specific domains by Certificate Authorities, also known as CAs. When you want to obtain a certificate for your site, the CA will confirm that you are responsible for the domain. Then it will issue a certificate that is valid for a specific amount of time, and only for the given domain (subdomains are allowed). Traditionally, you were also responsible for installing the certificate on your system, tracking expiration dates, and getting fresh certificates from time to time (typically, certificates are valid for a period of 12 months).

Each certificate is digitally signed; this allows the browser to verify that it was issued by a legitimate CA. To be a bit more specific, browsers start out with a small, predefined list of root certificates and use them to verify that the other certificates can be traced back to the root. You can access this information from your browser:

As you can probably see from what I have outlined above (even though I have hand-waved past a lot of interesting details), provisioning and managing SSL/TLS certificates can entail a lot of work, far too much of it manual and not easily automated. In many cases you also need to pay an annual fee for each certificate.

Time to change that!

New AWS Certificate Manager
The new AWS Certificate Manager (ACM) is designed to simplify and automate many of the tasks traditionally associated with management of SSL/TLS certificates. ACM takes care of the complexity surrounding the provisioning, deployment, and renewal of digital certificates! Certificates provided by ACM are verified by Amazon’s certificate authority (CA), Amazon Trust Services (ATS).

Even better, you can do all of this at no extra cost. SSL/TLS certificates provisioned through AWS Certificate Manager are free!

ACM will allow you to start using SSL in a matter of minutes. After your request a certificate, you can deploy it to your Elastic Load Balancers and your Amazon CloudFront distributions with a couple of clicks. After that, ACM can take care of the periodic renewals without any action on your part.

Provisioning and Deploying a Certificate
Let’s step through the process of provisioning and deploying a digital certificate using the console (APIs are also available). I’ll use one of my own domains (jeff-barr.com) for this exercise. I start by opening the AWS Certificate Manager Console and clicking on Get started.

Then I enter the domain name of the site that I want to secure. In this case I want to secure the “naked” domain and all of the first-level sub-domains within it:

Then I review my request and confirm my intent:

I flip over to my inbox, find the email or emails (one per domain) from Amazon (certificates.amazon.com), and click on Amazon Certificate Approvals:

I visit the site and click on I Approve:

And that’s all it takes! The certificate is now visible in the console:

Deploying the Certificate
After the certificate is issued, I can deploy it to my Elastic Load Balancers and/or CloudFront distributions.

Because ELB supports SSL offload, deploying a certificate to a load balancer (rather than to the EC2 instances behind it) will reduce the amount of encryption and decryption work that the instances need to handle.

And for a CloudFront distribution:

Available Now
AWS Certificate Manager (ACM) is available now in the US East (Northern Virginia) region, with additional regions in the works. You can provision, deploy, and renew certificates at no charge.

We plan to add support for other AWS services and for other types of domain validation. As always, your suggestions and feedback are more than welcome and will help us to prioritize our work.

If you are using AWS Elastic Beanstalk, take a look at Enabling SSL/TLS (for free) via AWS Certificate Manager.

It is big. It is cold. It is secure. And now it is empty, because you have gone all-in to the AWS Cloud. So, what do you do with your data center? Once the pride and joy of your IT staff, it is now a stark, expensive reminder that the world has changed.

Many AWS customers are migrating from their existing on-premises data centers to AWS. Here are just a few of their stories (links go to case studies, blog posts, and videos from re:Invent):

• Delaware North – This peanut and popcorn vendor reduced their server footprint by more than 90% and expects to save over $3.5 million in IT acquisition and maintenance costs over five years by using AWS.
• Seaco – This global sea-container leasing company implemented the SAP Business Suite on AWS and reduced latency by more than 90%.
• Kaplan – This education and test prep company moved a set of development, test, staging, and production environments that once spanned 12 separate data centers to AWS, eliminating 8 of the 12 in the process.
• Talen Energy – As part of a divestiture, this nuclear power company decided to move to AWS and found that they were able to focus more of their energy on their core business.
• Condé Nast – This well-known publisher migrated over 500 servers and 1 petabyte of storage to AWS and went all-in.
• Hearst Corporation – This diversified communications company migrated 10 of their 29 data centers to AWS.
• University of Notre Dame – This university has already migrated its web site to AWS and plans to move 80% of the remaining workloads in the next three years.
• Capital One – This finance company has made AWS a central part of its technology strategy.
• General Electric – This diversified company is migrating more than 9,000 workloads to AWS, while closing 30 out of 34 data centers.

If you are curious, here’s what happens when a data center closes down:

Now What?
You may be wondering what you are supposed to do with all of that cold, empty space after your migration!

With generous contributions from my colleagues (they did 85% of the work), I have compiled a list of 120 possible uses for your data center. For your reading pleasure, I have arranged them by category. As you can see, my colleagues are incredibly imaginative! There is some overlap here, but I didn’t want to play favorites. So, here you go…

Sports and Recreation

1. Ice hockey rink.
2. Whirlyball arena.
3. Go-kart track.
4. Snowshoe practice area.
5. Laser tag.
6. Paintball arena.
7. Sweat lodge.
8. Hot yoga studio.
9. Immersive VR gaming arena.
10. Largest paper football game. Ever.
11. Portal arena.
12. Paint walls black. Turn off lights. Dress in black. Water balloon fight with fluorescent paint.
13. Extreme weather survival training.
14. Giant domino rally.
15. Venue for world’s longest paper airplane flight.
16. twitch.tv live video game championship arenas.
17. Bubble football arena.
18. World’s largest ball pit.
19. Ultimate LAN party room.
20. Indoor lazy river.
21. Indoor ski resort.
22. Shooting range.
23. All weather theme park – “Datacenter Land.”
24. Segway racing track.
25. Indoor surf park.
26. Massive spinning studio.
27. Create the World Trampoline Wallyball League (WTWL).
28. Ultimate Boda Borg quest challenge.
29. Indoor hang gliding wind tunnel.
30. 48 state of the art gyms.
31. Indoor dog park.
32. Zamboni driver training facility.

Food and Beverages

33. A large, cold area is perfect for growing and preserving food.
34. Meat locker / meat packing facility.
35. Popsicle factory.
36. Wine cellar.
37. Penguin sanctuary.
38. Mushroom farm.
39. Cheese grotto.
40. Practice area for growing potatoes on Mars.
41. World’s largest Easy Bake Oven.

Eminently Practical

42. Classroom.
43. Electric car charging station.
44. Storm shelter.
45. Drone zone.
46. Giant pencil case.
47. Bomb shelter.
48. Car wash.
49. Community theater.
50. Art gallery.
51. Secure storage for all the money saved by using AWS.
52. Cloud University.
53. Solar power generation plant.
54. 48 really large Airbnb opportunities.
55. Maker‘s den.

Space, the Final Frontier

56. Blimp Hanger with “Moffett Field, You Got Nothing on Me!” painted on the side.
57. UFO storage.
58. Fill with water and use as a NASA zero-gravity training facility.
59. Time portal for the Restaurant at the end of the Internet.
60. Blue Origin space terminal with interplanetary duty free zone.
61. Use racks as studio apartments in San Francisco. Rent at $5000/month.

Dead or Alive

62. Homeless shelter.
63. Orphanage / charity home.
64. Morgue.
65. Cryogenic human storage.
66. Zombie apocalypse refuge.
67. Cold therapy spa.
68. Rehabilitation center.
69. Snowman preservation facility.

Just Plain Weird

70. Unicorn farm.
71. Mattress testing facility.
72. Biohazard isolation area.
73. Stress-relief shattering emporium.
74. Grow operation (where permitted by law).
75. Military parade ground.
76. Venue for all 2016 US presidential debates.
77. Super ball testing facility.
78. Corporate meditation center.
79. Duck echo testing facility.
80. Automated paper making factory for paper towels and toilet paper.
81. Tour facility for worlds largest ball of Ethernet or fiber optic cable.
82. Robot cat toy factory for Ethernet / fiber cable yarn balls.
83. Storage for recently unearthed E.T. The Extra-Terrestrial game cartridges.
84. Giant sensory deprivation tank.
85. Biosphere 3.
86. “Can you hear me now?” test facility.

TV and Movies

87. Derek Zoolander Center for Kids Who Can’t Read Good.
88. Battle of Hoth reenactment.
89. Trash compactor where we can put people who reveal Star Wars spoilers.
90. American Gladiator or American Ninja Warrior arena.
91. Hangar for Death Star.
92. Fill it with water. Re-enact Finding Nemo.
93. Take old printers and re-enact scene from Office Space. Call it a Silent Meditation Retreat.
94. Erect transparent aluminum walls, fill with water, store whales for when aliens come to contact them.
95. Raiders of the Lost Ark warehouse.
96. Training center for the Knights of Ren.
97. Mythbusters science lab.
98. Top Gear secret race track.
99. Mad Max: Server Room Rampage.
100. High security storage facility for broken down Daleks.
101. Tardis repair center for all things wibley wobley or timey wimey.

Uniquely Amazonian

102. Venue for next re:Play party.
103. Amazon fulfillment center.
104. Alternate venue for re:Invent 2017.
105. AWS Import/Export Snowball processing facility.
106. Amazon Fresh greeenhouse.
107. Brewery for the Amazon Simple Beer Service.
108. Amazon Locker site.
109. Actual cloud storage (leave the A/C on and pipe in some steam).

It’s Dead, Jim

110. Electronics shredding center.
111. Warehouse used to refurbish decommissioned corporate computers/servers to deploy to underprivileged schools world wide for education.
112. Venue to host auction for empty data centers.
113. Server to host the auction website for empty data centers.
114. Museum of data center history.
115. “Co-location Data Center” – National Trust for Historic Preservation.
116. Outreach centers for all those IT Admins that claimed they would never go all-in in the cloud.
117. Retro-style storage for paper files.
118. Data center resort with gardens grown on servers.
119. Buggy whip factory.
120. Museum of technology history.

PS – Please feel free to leave suggestions for additional uses in the comments.

Ever since we launched AWS, customers have been curious about how they can use it to build and run applications that must meet many different types of regulatory requirements. For example, potential AWS users in the pharmaceutical, biotech, and medical device industries are subject to a set of guidelines and practices that are commonly known as GxP. In those industries, the x can represent Laboratory (GLP), Clinical (GCP), or Manufacturing (GMP).

These practices are intended to ensure that a product is safe and that it works as intended. Many of the practices are focused on traceability (the ability to reconstruct the development history of a drug or medical device) and accountability (the ability to learn who has contributed what to the development, and when they did it). For IT pros in regulated industries, GxP is important because it has requirements on how electronic records are stored, as well as how the systems that store these records are tested and maintained.

Because the practices became prominent at a time when static, on-premises infrastructure was the norm, companies have developed practices that made sense in this environment but not in the cloud. For example, many organizations perform point-in-time testing of their on-premises infrastructure and are not taking advantage of the all that the cloud has to offer. With the cloud, practices such as dynamic verification of configuration changes, compliance-as-code, and the use of template-driven infrastructure are easy to implement and can have important compliance benefits.

New Resource
Customers are already running GxP-workloads on AWS! In order to help speed the adoption for other pharma and medical device manufacturers, we are publishing our new GxP compliance resource today.

The GxP position paper (Considerations for Using AWS Products in GxP Systems) provides interested parties with a brief overview of AWS and of the principal services, and then focuses on a discussion of how they can be used in a GxP system. The recommendations within the paper fit in to three categories:

Quality Systems – This section addresses management, personnel, audits, purchasing controls, product assessment, supplier evaluation, supplier agreement, and records & logs.

System Development Life Cycle – This section addresses system development, validation, and operation. As I read this section of the document, it was interesting to learn how the software-defined infrastructure-as-code AWS model allows for better version control and is a great fit for GxP. The ability to use a common set of templates for development, test, and production environments that are all configured in the same way simplifies and streamlines several aspects of GxP compliance.

Regulatory Affairs – This section addresses regulatory submissions, inspections by health authorities, and personal data privacy controls.

We hired Lachman Consultants (an internationally renowned compliance consulting firm), and had them contribute to and review an earlier draft of the position paper. The version that we are publishing today reflects their feedback.

Join our Webinar
If you are interested in building cloud-based systems that must adhere to GxP, please join our upcoming GxP Webinar. Scheduled for February 23, this webinar will give you an overview of the new GxP compliance resource and will show you how AWS can facilitate GxP compliance within your organization. You’ll learn about rules-based consistency, compliance-as-code, repeatable software-based testing, and much more.

PS – The AWS Life Sciences page is another great resource!

Did you resolve to learn something new in 2016? If so, you should attend an AWS webinar!

Each month, we run a series of webinars that are designed to bring you up to speed on the latest AWS services & features, and to make sure that you are aware of the best ways to put them to use. The webinars are conducted by senior AWS Product Managers and Solution Architects and often include a guest speaker from our customer base.

The webinars are free but “seating” is limited and you should definitely sign up ahead of time if you want to attend (all times are Pacific). Here’s what we have in store this month:

Tuesday, January  26
There are several different ways to deploy your applications on AWS including AWS Elastic Beanstalk, AWS CodeDeploy, and Amazon EC2 Container Service. This webinar will help you to understand the strengths of each service and provide you with a framework to help you to decide which one to use.

• Webinar: Introduction to Deploying Applications on AWS (9 – 10 AM).

AWS IoT is a managed cloud platform that lets connected devices interact with cloud applications and other devices. This webinar will show you how constrained devices can send data to the cloud and receive commands back to the device.

• Webinar: Getting Started with AWS IoT (10:30 – 11:30 AM).
• Blog Post: AWS IoT – Cloud Services for Connected Devices.

AWS provides many options and tools that are a great fit for your big data needs. This webinar will provide you with an overview of the options including Hadoop, Spark, and NoSQL databases.

• Webinar: Getting Started with Big Data on AWS (Noon – 1 PM).
• Blog Post: Amazon EMR Update – Apache Spark 1.5.2, Ganglia, Presto, Zeppelin, and Oozie.

Wednesday, January 27
Amazon Aurora is a fast and cost-effective relational database designed to be compatible with MySQL.

• Webinar: Amazon Aurora for Enterprise Database Applications (9 – 10 AM).
• Blog Post: Amazon Aurora – New Cost-Effective MySQL-Compatible Database Engine for Amazon RDS.

Do you have existing data that you want to bring to the cloud? Our data migration webinar will review the options and help you to choose the one that fits your use case:

• Webinar: Cloud Data Migration: 6 Strategies for Getting Data into AWS (10:30 – 11:30 AM).
• Blog Post: AWS Import/Export Snowball – Transfer 1 Petabyte Per Week Using Amazon-Owned Storage Appliances.

Many developers are using containers to simplify the packaging, deployment, and operation of their applications. This webinar will show you how to use Amazon EC2 Container Service to simplify the use of Docker in production.

• Webinar: Introduction to Docker on AWS (Noon – 1:00 PM).
• Blog Post: Amazon EC2 Container Service (ECS) – Container Management for the AWS Cloud.

Thursday, January 28
Machine Learning helps you to extract value from data. This webinar will show you how to build machine learning models and use them to make predictions.

• Webinar: Building Smart Applications with Amazon Machine Learning (9 – 10 AM).
• Blog Post: Amazon Machine Learning – Make Data-Driven Decisions at Scale.

I am always happy to see ways that two or more AWS services can be combined to create something cool. This webinar will show you how to use AWS Lambda and the new AWS IoT service to do just that:

• Webinar: Build IoT Backends with AWS IoT & AWS Lambda (10:30 – 11:30 AM).
• Blog Posts: AWS Lambda – Run Code in the Cloud and AWS IoT – Cloud Services for Connected Devices.

The phrase “infrastructure as code” has become increasingly popular over the last couple of years. This webinar will show you how do implement this new practice using AWS:

• Webinar:  Managing your Infrastructure as Code (Noon – 1 PM).
• Blog Post: New AWS Tools for Code Management and Deployment.

— Jeff;

Let’s take a quick look at what happened in AWS-land last week:

New & Notable Open Source

• Sparta helps you to build & deploy Go functions in AWS Lambda.
• Apex is a new serverless architecture for AWS Lambda.
• AWSResco is a standalone SPA (Single Page Application) to get instance and reservation cost info for running infrastructure.
• Bragg is a web framework for Lambda.
• Kinesis-common is a common Java package for interacting with Kinesis.
• aws-cloud-examples is a collection of files (CloudFormation templates) for testing and developing on AWS.
• mock-aws-s3 is a simple interface that mocks the AWS SDK for Node.js.
• Vominator is a CLI tool for managing AWS resources at scale.
• get-aws-pricing queries AWS pricing information using the Price List API.
• s3-utilities is a tool for undeleting files, folders, and entire buckets from S3 versioned buckets.

New SlideShare Presentations

• AWS APAC Webinar Week:
  • 2015 an Amazing Year in AWS.
  • Introduction to Cloud Computing with Amazon Web Services.
  • Getting the Most from EC2.
  • AWS MySQL Relational Database Services Best Practices.
  • Understanding AWS Storage Options.
  • Securing Your Business on AWS.
  • Real Time Data Processing with Kinesis.

New Customer Success Stories

• Brooks Brothers – Launch and test new SAP-based retail projects.
• flydubai – Host online checkin platform on AWS.
• Gilt – Move from on-premises to AWS.
• Linden Lab – Host a new platform for virtual experiences.
• Novi Digital – Reduce cost and time to market for hotstar.com.

New YouTube Videos

• Amazon WorkMail – Fully Managed Business Email and Calendar Service.
• AWSome Day Tel Aviv:
  • Introduction to AWS.
  • AWS Compute Services and Networking Part 1.
  • AWS Compute Services and Networking Part 2.
  • AWS Deployment and Management.
  • AWS Managed Services and Databases.
  • AWS Storage and Content Delivery Services.
• AWS Partner Success:
  • Looker.
  • Chef.

Upcoming Events

• January 20 – Online (Webinar) – AWS Cost Optimization (Spotinst).
• January 23 – Live (Bangalore) – Everything You Need to Know About DNS.
• January 28 – Online (Webinar) – Mastering the Fundamentals of AWS Cost Management.

Help Wanted

• AWS Careers.

Stay tuned for next week! In the meantime, follow me on Twitter and subscribe to the RSS feed.

— Jeff;

Back in November 2014 AWS made a long-term commitment to achieve 100% renewable energy usage for our global infrastructure footprint, and we continue to make progress towards this goal. Today’s news on this topic is particularly exciting for us – our Amazon Wind Farm Fowler Ridge, located in Benton County, Indiana, is now live and producing electricity! This marks the first of our four announced renewable energy projects to move into full operation.

We first announced that we teamed with Pattern Energy Group LP (Pattern Development) on the construction and operation of the 150 megawatt (MW) Amazon Wind Farm Fowler Ridge in January of last year. Over the course of the summer and fall of 2015, Pattern erected the wind farm’s 65 utility-scale turbines. Each of those turbines is tall, so tall in fact that the rotor axis, called a nacelle, is about the height of a 26-story building with blades long enough to sweep an area about the size of a football field.

On January 1, 2016, its first day of full operation, Amazon Wind Farm Fowler Ridge generated more than 1.1 million kilowatt-hours of renewable electricity, enough to power over 100 US homes for an entire year! Each year the project is expected to generate enough renewable electricity to power the equivalent of approximately 46,000 homes. With this wind farm, we’re able to increase the amount of renewable energy produced in the grid that powers AWS’s US East Region in Northern Virginia and upcoming US Region in Ohio. Over time we’ll continue to add more wind and solar power delivered into the grids, reducing the amount of coal and other fossil fuels needed to power those grids.

Today’s news definitely helps us progress towards our goal of 40% renewable energy for our global infrastructure by the end of 2016 and marks more progress in AWS’s march towards our long-term 100% renewable goal, with much more soon to come. Since the Fowler Ridge project, we have announced three other agreements for new wind and solar projects that will be constructed over the coming months and start generating renewable power in late 2016 and early 2017. Stay tuned for more exciting announcements to come in the future as well.

Getting Ready to present (photo by Danilo Poccia)

Late last year I traveled to Barcelona and delivered the keynote address at one of the final AWS Summit events of 2015.

My re:Invent recap was well received; the AWS users and partners in the area were excited by our newest services and left the session eager to learn more and to put them to use. As part of the keynote session, attendees learned about AWS security from my colleague Bill Murray and also had the opportunity to hear how several of our customers had used AWS.

After the keynote, attendees spent the afternoon attending deep-dive technical sessions, exploring the partner area, and meeting with each other. We wrapped up the day with snacks, drinks, and a Battle of the Bands. It was wonderful to meet so many AWS users and to learn more about their businesses and their applications.

Summits for 2016
We are now making plans for our 2016 Global Summit series. As in past years, we will focus on education—we want to make sure that you know as much as possible about whatever aspects of AWS are of interest to you, and we want you to leave the summit charged up and ready to build something new, cool, and powerful!

To see the list of cities and dates, check out the AWS Summits page. Click on the Want More Information button to express your interest in a local Summit and to receive registration information as it becomes available.

Follow Along
In addition to the page above, you can also subscribe to the AWS Events RSS feed, follow @AWSSummits on Twitter, and find us on Facebook.

PS – Members of the AWS Partner Network (APN) should take a look at the 2016 Global Sponsorship Opportunities.

When you pull the curtain back on an AWS-powered application, you’ll find that a lot is happening behind the scenes. EC2 instances are launched and terminated by Auto Scaling policies in response to changes in system load, Amazon DynamoDB tables, Amazon SNS topics and Amazon SQS queues are created and deleted, and attributes of existing resources are changed from the AWS Management Console, the AWS APIs, or the AWS Command Line Interface (CLI).

Many of our customers build their own high-level tools to track, monitor, and control the overall state of their AWS environments. Up until now, these tools have worked in a polling fashion. In other words, they periodically call AWS functions such as DescribeInstances, DescribeVolumes, and ListQueues to list the AWS resources of various types (EC2 instances, EBS volumes, and SQS queues here) and to track their state. Once they have these lists, they need to call other APIs to get additional state information for each resources, compare it against historical data to detect changes, and then take action as they see fit. As their systems grow larger and more complex, all of this polling and state tracking can become onerous.

New CloudWatch Events
In order to allow you to track changes to your AWS resources with less overhead and greater efficiency, we are introducing CloudWatch Events today.

CloudWatch Events delivers a near real-time stream of system events that describe changes in AWS resources. Using simple rules that you can set up in a couple of minutes, you can easily route each type of event to one or more targets: AWS Lambda functions, Amazon Kinesis streams, Amazon SNS topics, and built-in targets.

You can think of CloudWatch Events as the central nervous system for your AWS environment. It is wired in to every nook and cranny of the supported services, and becomes aware of operational changes as they happen. Then, driven by your rules,  it activates functions and sends messages (activating muscles, if you will) to respond to the environment, making changes, capturing state information, or taking corrective action.

We are launching CloudWatch Events with an initial set of AWS services and events today, and plan to support many more over the next year or so.

Diving in to CloudWatch Events
The three main components that you need to know about are events, rules, and targets.

Events (represented as small blobs of JSON) are generated in four ways. First, they arise from within AWS when resources change state. For example, an event is generated when the state of an EC2 instance changes from pending to running or when Auto Scaling launches an instance. Second, events are generated by API calls and console sign-ins that are delivered to Amazon CloudWatch Events via CloudTrail. Third, your own code can generate application-level events and publish them to Amazon CloudWatch Events for processing. Fourth, they can be issued on a scheduled basis, with options for periodic or Cron-style scheduling.

Rules match incoming events and route them to one or more targets for processing. Rules are not processed in any particular order; all of the rules that match an event will be processed (this allows disparate parts of a single organization to independently look for and process events that are of interest).

Targets process events and are specified within rules. There are four initial target types: built-in, Lambda functions, Kinesis streams, and SNS topics, with more types on the drawing board. A single rule can specify multiple targets. Each event is passed to each target in JSON form. Each rule has the opportunity to customize the JSON that flows to the target. They can elect to pass the event as-is, pass only certain keys (and the associated values) to the target, or to pass a constant (literal) string.

CloudWatch Events in Action
Let’s go ahead and set up a rule or two! I’ll use a simple Lambda function called SomethingHappened. It will simply log the contents of the event:

Next, I switch to the new CloudWatch Events Console, click on Create rule and choose an event source (here’s the menu with all of the choices):

Just a quick note before going forward. Some of the AWS services fire events directly. Others are fired based on the events logged to CloudTrail; you’ll need to enable CloudTrail for the desired service(s) in order to receive them.

I want to keep tabs on my EC2 instances, so I choose EC2 from the menu. I can choose to create a rule that fires on any state transition, or on a transition to one or more states that are of interest:

I want to know about newly launched instances, so I’ll choose Running. I can make the rule respond to any of my instances in the region, or to specific instances. I’ll go with the first option; here’s my pattern:

Now I need to make something happen. I do this by picking a target. Again, here are my choices:

I simply choose Lambda and pick my function:

I’m almost there! I just need to name and describe my rule, and then click on Create rule:

I click on Create Rule and the rule is all set to go:

Now I can test it by launching an EC2 instance. In fact, I’ll launch 5 of them just to exercise my code! After waiting a minute or so for the instances to launch and to initialize, I can check my Lambda metrics to verify that my function was invoked:

This looks good (the earlier invocations were for testing). Then I can visit the CloudWatch logs to view the output from my function:

As you can see, the event contains essential information about the newly launched instance. Your code can call AWS functions in order to learn more about what’s going on. For example, you could call DescribeInstances to access more information about newly launched instances.

Clearly, a “real” function would do something a lot more interesting. It could add some mandatory tags to the instance, update a dynamic visualization, or send me a text message via SNS. If you want to do any (or all of these things), you would need to have a more permissive IAM role for the function, of course. I could make the rule more general (or create another one) if  I wanted to capture some of the other state transitions.

Scheduled Execution of Rules
I can also set up a rule that fires periodically or according to a pattern described in a Cron expression. Here’s how I would do that:

You might find it interesting to know that this is the underlying mechanism used to set up scheduled Lambda jobs, as announced at AWS re:Invent.

API Access
Like most AWS services, you can access CloudWatch Events through an API. Here are some of the principal functions:

• PutRule to create a new rule.
• PutTargets and RemoveTargets to connect targets to rules, and to disconnect them.
• ListRules, ListTargetsByRule, and DescribeRule to find out more about existing rules.
• PutEvents to submit a set of events to CloudWatch events. You can use this function (or the CLI equivalent) to submit application-level events.

Metrics for Events
CloudWatch Events reports a number of metrics to CloudWatch, all within the AWS/Events namespace. You can use these metrics to verify that your rules are firing as expected, and to track the overall activity level of your rule collection.

The following metrics are reported for the service as a whole:

• Invocations – The number of times that target have been invoked.
• FailedInvocations – The number of times that an invocation of a target failed.
• MatchedEvents – The number of events that matched one or more rules.
• TriggeredRules – The number of rules that have been triggered.

The following metrics are reported for each rule:

• Invocations – The number of times that the rule’s targets have been invoked.
• TriggeredRules – The number of times that the rule has been triggered.

In the Works
Like many emerging AWS services, we are launching CloudWatch Events with an initial set of features (and a lot of infrastructure behind the scenes) and some really big plans, including AWS CloudFormation support. We’ll adjust our plans based on your feedback, but you can expect coverage of many more AWS services and access to additional targets over time. I’ll do my best to keep you informed.

Getting Started
We are launching CloudWatch Events in the US East (Northern Virginia), US West (Oregon), Europe (Ireland), and Asia Pacific (Tokyo) regions. It is available now and you can start using it today!

— Jeff;

Last November I gave you a heads-up that we planned to increase the length of the resource IDs for EC2 instances, reservations, volumes, and snapshots in early 2016. We are now entering a transition period that will last until early December (2016). During this period, you can opt in to the new format (a resource identifier followed by a 17-character string) on a region-by-region, user-by-user, type-by-type basis.  Once you opt in for a given region and IAM user (or for the root account), newly created resources will receive the longer IDs.

Effective today, you can opt in to the new format for EC2 instances and EC2 reservations. Each reservation represents the results of a single instance launch request, and can be associated with multiple instances.

If you build libraries, tools, or applications that make direct calls to the AWS API, now is the time to opt in and to start your testing process! If you store the IDs in memory or in a database, take a close look at fixed-length fields, data structures, schema elements, string operations, and regular expressions. Resources that were created before you opt in will retain their existing short identifiers; be sure that your revised code can still handle them!

You can opt in to the new format using the AWS Management Console, the AWS Command Line Interface (CLI), the AWS Tools for Windows PowerShell, or though an API function.You can also opt out if your testing uncovers issues that you need to address.

Opting In – Console
To opt in via the Console, simply log in, choose EC2, and click on Resource ID length management:

Then click on Use Longer IDs for the desired resource types:

Opting In – AWS CLI
To opt in using the AWS CLI, use the modify-id-format command and specify the desired resource type:

$ modify-id-format --resource instance --use-long-ids

You can opt out like this:

$ modify-id-format --resource instance --no-use-long-ids

Opting In – AWS Tools for Windows PowerShell
To opt in using the AWS Tools for Windows PowerShell, use the Edit-EC2IdFormat cmdlet and specify the desired resource type:

PS C:\> Edit-EC2IdFormat -Resource instance -UseLongId True

You can opt out like this:

PS C:\> Edit-EC2IdFormat -Resource instance -UseLongId False

Opting In – AWS SDKs
To opt in from your own tools or application code, call the ModifyIdFormat function, specify the resource type (instance or reservation), and set the UseLongIds parameter to True.

Things to Know
You can opt in at the IAM user or account level. After you have opted in, newly created resources of the specified types will receive the new, longer identifiers as soon as the setting takes effect (typically a few minutes). The identifiers will be visible in API results, the command line, and the console.

If a particular IAM user opts in and then creates some resources, the longer identifiers will be visible to other users, even if they have not opted in. You may want to create a separate AWS account for testing purposes in order to avoid confusion. You can also choose to do your testing in an AWS region that you don’t use on a regular basis (see the AWS Global Infrastructure page for a list of current and planned regions).

This blog post applies to all AWS regions and accounts. AWS accounts created on or after March 7, 2016 will default to longer instance and reservation IDs, and can be opted out if necessary.

After you have performed your testing and are satisfied that your tools, code, and applications work to your satisfaction, we recommend that you opt in for all of your AWS accounts before December 2016. This will give you confidence that no surprises will arise when the new identifiers become mandatory.

To learn more about this important update to AWS, please read the Longer EC2 and EBS Resource IDs FAQ.

PS – We’ll be adding support for EBS volumes and snapshots in April; stay tuned for more information.