AWS Official Blog

  • Now Available – Amazon Linux AMI 2015.09

    by Jeff Barr | on | in Amazon EC2, Amazon Linux AMI | | Comments

    My colleague Max Spevack runs the team that produces the Amazon Linux AMI. He wrote the guest post below to announce the newest release!

    — Jeff;

    The Amazon Linux AMI is a supported and maintained Linux image for use on Amazon EC2.

    We offer new major versions of the Amazon Linux AMI after a public testing phase that includes one or more Release Candidates. The Release Candidates are announced in the EC2 forum and we welcome feedback on them.

    Launching 2015.09 Today
    Today we announce the 2015.09 Amazon Linux AMI, which is supported in all regions and on all current-generation EC2 instance types.  The Amazon Linux AMI supports both PV and HVM mode, as well as both EBS-backed and Instance Store-backed AMIs.

    You can launch this new version of the AMI in the usual ways. You can also upgrade an existing EC2 instance by running the following commands:

    $ sudo yum clean all
$ sudo yum update

    And then rebooting the instance.

    New Kernel
    A major new feature in this release is the 4.1.7 kernel, which is the most recent long-term stable release kernel. Of particular interest to many customers is the support for OverlayFS in the 4.x kernel series.

    New Features
    The roadmap for the Amazon Linux AMI is driven in large part by customer requests. During this release cycle, we have added a number of features as a result of these requests; here’s a sampling:

    • Based on numerous customer requests and in order to support joining Amazon Linux AMI instances to an AWS Directory Service directory, we have added Samba 4.1 to the Amazon Linux AMI repositories, available via sudo yum install samba.
    • Numerous customers have asked for PostgreSQL 9.4 and it is now available in our Amazon Linux AMI repositories as a separate package from PostgreSQL 9.2 and 9.3. PostgreSQL 9.4 is available via sudo yum install postgresql94 and the 2015.09 Amazon Linux AMI repositories include PostgreSQL 9.4.4.
    • A frequent customer request has been MySQL 5.6, and we are pleased to offer it in the 2015.09 repositories as a separate package from MySQL 5.1 and 5.5. MySQL 5.6 is available via sudo yum install mysql56 and the 2015.09 Amazon Linux AMI repositories include MySQL 5.6.26.
    • We introduced support for Docker and Go in our 2014.03 AMI, and we continue to follow upstream developments in each. The lead-up to the 2015.09 release included an update to Go 1.4 and to Docker 1.7.1.
    • We already provide Python 2.6, 2.7 (default), and 3.4 in the Amazon Linux AMI, but several customers have also asked for the PyPy implementation of Python. We’re pleased to include PyPy 2.4 in our preview repository. PyPy 2.4 is compatible with Python 2.7.8 and is installable via sudo yum --enablerepo=amzn-preview install pypy.
    • In our 2015.03 release we added an initial preview of the Rust programming language. Upstream development has continued on this language, and we have updated from Rust 1.0 to Rust 1.2 for the 2015.09 release. You can install the Rust compiler by running sudo yum --enablerepo=amzn-preview install rust.

    The release notes contain a longer discussion of the new features and updated packages, including an updated version of Emacs prepared specially for Jeff in order to ensure timely publication of this blog post!

    — Max Spevack, Development Manager, Amazon Linux AMI.

    PS – If you enjoy the Amazon Linux AMI offering and would like to work on future versions, let us know!

  • AWS Week in Review – September 14, 2015

    by Jeff Barr | on | in Week in Review | | Comments

    Let’s take a quick look at what happened in AWS-land last week:

    Monday

    September 14
    Tuesday

    September 15
    Wednesday

    September  16
    Thursday

    September 17
    Friday

    September 18

    New & Notable Open Source

    New Customer Success Stories

    • The Democratic National Committee (DNC) – Gather, store, and deliver voter data to other political organizations.
    • Domain Group – Provide real estate information to Australians via online and print platforms.
    • Infraware -Deliver software to mobile users; run customer analytics platform.
    • Ividata – Big data for retailers.
    • Nexon – Build and distribute games to an international audience.
    • SM Entertainment – Host websites, deliver mobile apps, run internal ERP and Groupware systems.

    New SlideShare Content

    New YouTube Videos

    Upcoming Events

    Upcoming Events at the AWS Loft (San Francisco)

    Upcoming Events at the AWS Loft (New York)

    Help Wanted

    Stay tuned for next week! In the meantime, follow me on Twitter and subscribe to the RSS feed.

    — Jeff;

  • New AWS Public Data Set – 3000 Rice Genome

    by Jeff Barr | on |

    My colleague Angel Pizzaro wrote the guest post below to tell you about an amazing new AWS Public Data Set!

    — Jeff;

    You can now access the genome sequence data of 3,024 rice varieties that have been aligned and analyzed against five different reference genomes as an AWS Public Data Set.  The data contains over 30 million genetic variations that span across all known and predicted rice genes, as well as potential regulatory regions surrounding these genes. Through analysis of this data, researchers can potentially identify genes associated with important agronomic traits such as crop yield, climate stress tolerance, and disease resistance. Together, they represent an unprecedented resource for advancing rice science and breeding technology.

    Rice is a staple food source for half the world’s population, and accounts for over 20% of all calories per capita. In order to keep up with global population increases, we must find some way to increase rice crop yields by 25% by 2030. The current rate of increasing rice yield by traditional breeding is insufficient, especially when taking into account observed trends in climate change and pollution. In order to meet the world’s projected demand for a stable food supply, modern methods of breeding that take into account the underlying genetic information must be adopted by the community at large.

    The 3,000 Rice Genome sequencing project is an international effort to sequence the genomes of 3,024 rice varieties from 89 countries. The collaborating centers involved are the Chinese Academy of Agricultural Sciences, BGI Shenzhen, and the International Rice Research Institute (IRRI). The consortium partnered with DNAnexus to analyze the sequence data of the 3,024 different rice varieties against five published draft genome builds of the rice genome. Partnering with DNAnexus allowed them to take advantage of the scalable computing capability at AWS to process all of the source genomic data across 37,000 compute cores working together in just two days — more than 200 times faster than would have been possible on local computing infrastructure. In addition, the data are accessible via DNAnexus for further analysis. For more details on accessing the data within DNAnexus, refer to the project documentation.

    More in-depth analyses of this dataset could lead to inferences about higher yield and stress tolerance to pests, diseases, and climate change. You can learn more about the data and how to access it on the 3000 Rice Genome Public Data Set page.

    Working with the Genomic Data Set on AWS
    Because the data are hosted on S3 and accessible over common HTTP protocols, researchers have already done some amazing integrations within pre-existing tools. I’ve included some initial examples here and we’ll work with IRRI to share more examples as they emerge.

    Visualizing the data using SNP-Seek
    The International Rice Informatics Consortium (IRIC) has made the data available for querying and visualization through their SNP-Seek portal.  User are now able to query across all of the strains and narrow down regions of interest that show diversity across multiple genome references, integrated with the rice research community’s genomic annotation data:

    Open Source Tools
    In addition to the rich set of AWS partner offerings for life sciences, the full genomics open source ecosystem is available for use with the data. From command line applications such as samtools to rich user interfaces such as Galaxy or iobio, researchers can get started right away to analyze the data.

    What’s Next?
    The challenge for the research community is now to comprehensively and systematically mine this dataset to link genotypic variation to functional variation with the ultimate goal of creating new and sustainable rice varieties. Combining these efforts with other studies such as careful trait phenotyping in controlled and wild environments, as well as environmental studies based on satellite imagery like the Landsat data you can already access on AWS, can help is to keep up with the demands of the future world’s population growth.

    Visit the 3000 Rice Genome Public Data Set page to access the data and sign up for project updates.

    — Angel Pizzaro, Technical Business Development Manager, AWS Scientific Computing

  • Announcing the AWS Pop-up Loft in Berlin

    by Jeff Barr | on | in AWS Loft | | Comments

    The AWS Pop-up Lofts in San Francisco and New York have become hubs and working spaces for developers, entrepreneurs, students, and others who are interested in working with and learning more about AWS. They come to learn, code, meet, collaborate, ask questions, and to hang out with other like-minded folks. I expect the newly opened London Loft to serve as the same type of resource for the UK.

    I’m happy to be able to announce that we will be popping up a fourth loft, this one in Berlin. Once again, we have created a unique space and assembled a full calendar of events, with the continued help from our friends at Intel. We look forward to using the Loft to meet and to connect with our customers, and expect that it will be a place that they visit on a regular basis.

    Startups and established businesses have been making great use of our new Europe (Frankfurt) region; in fact, it is currently growing even faster than all of our other international regions! While this growth has been driven by many factors, we do know that startups in Berlin have been early adopters of the AWS cloud, with some going all the way back to 2006. Since then some of the most well-known startups in Germany, and across Europe, have adopted AWS including SoundCloud, Foodpanda, and Zalando.

    With a high concentration of talented, ambitious entrepreneurs, Berlin is a great location for the newest Pop-up Loft. Startups and other AWS customers in the area have asked for access to more local technical resources and expertise in order to help them to continue to grow and to succeed with AWS.

    Near Berlin Stadtmitte Station
    This loft is located on the 5 floor of Krausenstrasse 38 in Berlin, close to Stadtmitte Station and convenient to Spittelmarkt. The opening party will take place on October 14th and the Loft will open for business on the morning of October 15th. After that it will be open from 10 AM to 6 PM Monday through Friday, with special events in the evening.

    During the day, you will have access to the Ask an Architect Bar, daily education sessions, Wi-Fi, a co-working space, and snacks, all at no charge. There will also be resources to help you to create, run, and grow your startup including educational sessions from local AWS partners, accelerators, and incubators including Axel Springer’s Plug & Play and Deutsche Telecom’s Hub:Raum.

    Ask an Architect
    Step up to the Ask an Architect Bar with your code, architecture diagrams, and your AWS questions at the ready! Simply walk in. You will have access to deep technical expertise and will be able to get guidance on AWS architecture, usage of specific AWS services and features, cost optimization, and more.

    Echo Hackathon
    My colleague David Isbitski will be running an Alexa Hackathon at the Loft. After providing an introduction to the Amazon Echo, David will show you how to build your first Alexa Skill using either AWS Lambda or AWS Elastic Beanstalk (your choice). He will show you how to monitor it using Amazon CloudWatch and will walk you through the process of certifying the Skill as a prerequisite to making it available to customers later this year. The event will conclude with an open hackathon.

    AWS Education Sessions
    During the day, AWS Solution Architects, Product Managers, and Evangelists will be leading 60-minute educational sessions designed to help you to learn more about specific AWS services and use cases. You can attend these sessions to learn about Mobile & Gaming, Databases, Big Data, Compute & Networking, Architecture, Operations, Security, Machine Learning, and more, all at no charge. Hot startups such as EyeEm, Zalando, and Stormforger will talk about how they use AWS.

    Startup Education Sessions
    AWS startup community representatives, Berlin-based incubators & accelerators, startup scene influencers & hot startup customers of AWS will share best-practices, entrepreneurial know-how and lessons learned. Pop in to learn the art of pitching, customer validation & profiling, PR for startups & corporations. Get to know Axel Springer’s accelerator Plug & Play and the Hub:Raum incubator of the Deutsche Telekom.

    The Intel Perspective
    AWS and Intel share a passion for innovation and a history of helping startups to succeed. Intel will bring their newest technologies to Berlin, with talks and training that focus on the Internet of Things and the latest Intel Xeon processors.

    On the Calendar
    Here are some of the events that we have scheduled for October and November.

    Tech Sessions:

    • October 15 – Processing streams of data with Amazon Kinesis and Other Tools (10 AM – 11 AM).
    • October 15 – STUPS – A Cloud Infrastructure for Autonomous Teams (Zalando) (5 PM – 6 PM).
    • October 19 – Building a global real-time discovery platform on AWS (Rocket Internet) (6 PM – 7 PM).
    • October 23 – Amazon Echo hackathon (10 AM – 4 PM).
    • October 27 – DevOps at Amazon: A Look at Our Tools and Processes (9 – 10 AM).
    • October 27 – Automating Software Deployments with AWS CodeDeploy (10 AM – 11 AM).
    • October 30 – Redshift Deep Dive (5 PM – 6 PM).
    • November 3 – Cost Optimization Workshop (5 PM – 6 PM).
    • November 3 – Amazon Aurora (6 PM – 7 PM).
    • November 6 – Introduction to Amazon Machine Learning (9 AM – 10 AM).
    • November 10 – Security Master Class (6 PM – 7 PM).

    Business Sessions:

    • October 23 – Lessons Learned from 7 Accelerator Programs (6 PM – 7 PM).
    • October 26 – Funding cycles and Term Sheets (5 PM – 6 PM).
    • November 9 – Things to consider when PR-ing your startup (6 PM – 7 PM).

    Get-Togethers & Networking:

    • October 22 – Berlin’s Godfather of Tech (6 PM – 7 PM).
    • November 11 – Watch out: The Bavarians are in town! (6 8 PM).

    If you would like to learn more about a topic that’s not on this list, please let us know (you can stop by the Loft in person or you can leave a comment on this post).

    Come in and Say Hello
    Please feel free to stop in and say hello to my colleagues at the Berlin Loft if you happen to find yourself in Berlin!

    — Jeff;

  • AWS Storage Update – New Lower Cost S3 Storage Option & Glacier Price Reduction

    by Jeff Barr | on | in Amazon Glacier, Amazon S3, Price Reduction | | Comments

    Like all AWS services, the Amazon S3 team is always listening to customers in order to better understand their needs. After studying a lot of feedback and doing some analysis on access patterns over time, the team saw an opportunity to provide a new storage option that would be well-suited to data that is accessed infrequently.

    The team found that many AWS customers store backups or log files that are almost never read. Others upload shared documents or raw data for immediate analysis. These files generally see frequent activity right after upload, with a significant drop-off as they age. In most cases, this data is still very important, so durability is a requirement. Although this storage model is characterized by infrequent access, customers still need quick access to their files, so retrieval performance remains as critical as ever.

    New Infrequent Access Storage Option
    In order to meet the needs of this group of customers, we are adding a new storage class for data that is accessed infrequently. The new S3 Standard – Infrequent Access (Standard – IA) storage class offers the same high durability, low latency, and high throughput of S3 Standard. You now have the choice of three S3 storage classes (Standard, Standard – IA, and Glacier) that are designed to offer 99.999999999% (eleven nines) of durability.‎  Standard – IA has an availability SLA of 99%.

    This new storage class inherits all of the existing S3 features that you know (and hopefully love) including security and access management, data lifecycle policies, cross-region replication, and event notifications.

    Prices for Standard – IA start at $0.0125 / gigabyte / month (one and one-quarter US pennies), with a 30 day minimum storage duration for billing, and a $0.01 / gigabyte charge for retrieval (in addition to the usual data transfer and request charges). Further, for billing purposes, objects that are smaller than 128 kilobytes are charged for 128 kilobytes of storage. We believe that this pricing model will make this new storage class very economical for long-term storage, backups, and disaster recovery, while still allowing you to quickly retrieve older data if necessary.

    You can define data lifecycle policies that move data between Amazon S3 storage classes over time. For example, you could store freshly uploaded data using the Standard storage class, move it to Standard – IA 30 days after it has been uploaded, and then to Amazon Glacier after another 60 days have gone by.

    The new Standard – IA storage class is simply one of several attributes associated with each S3 object. Because the objects stay in the same S3 bucket and are accessed from the same URLs when they transition to Standard – IA, you can start using Standard – IA immediately through lifecycle policies without changing your application code. This means that you can add a policy and reduce your S3 costs immediately, without having to make any changes to your application or affecting its performance.

    You can choose this new storage class (which is available today in all AWS regions) when you upload new objects via the AWS Management Console:

    You can set up lifecycle rules for each of your S3 buckets. Here’s how you would establish the policies that I described above:

    These functions are also available through the AWS Command Line Interface (CLI), the AWS Tools for Windows PowerShell, the AWS SDKs, and the S3 API.

    Here’s what some of our early users have to say about S3 Standard – Infrequent Access:

    “For more than 13 years, SmugMug has provided unlimited storage for our customer’s priceless photos. With many petabytes of them stored on Amazon S3, it’s vital that customers have immediate, instant access to any of them at a moment’s notice – even if they haven’t been viewed in years. Amazon S3 Standard – IA offers the same high durability and performance as Amazon S3 Standard so we can continue to deliver the same amazing experience for our customers even as their cameras continue to shoot bigger, higher-quality photos and videos.”

    Don MacAskill, CEO & Chief Geek

    SmugMug

    “We store a ton of video, and in many cases an object in Amazon S3 is the only copy of a user’s video. This means durability is absolutely critical, and so we are thrilled that Amazon S3 Standard – IA lets us significantly reduce storage costs on our older video objects without sacrificing durability. We also really appreciate how easy it is to start using Amazon S3 Standard – IA. With a few clicks we set up lifecycle policies that will transition older objects to Amazon S3 Standard – IA at regular intervals –we don’t have to worry about migrating them to new buckets, or impacting the user experience in any way.”

    Brian Kaiser, CTO

    Hudl

    See the S3 Pricing page for complete pricing information on this new storage class.

    Reduced Price for Glacier Storage
    Effective September 1, 2015, we are reducing the price for data stored in Amazon Glacier from $0.01 / gigabyte / month to $0.007 / gigabyte / month. As usual, this price reduction will take effect automatically and you need not do anything in order to benefit from it. This price is for the US East (Northern Virginia), US West (Oregon), and Europe (Ireland) regions; take a look at the Glacier Pricing page for full information on pricing in other regions.

    — Jeff;

  • Docker Trusted Registry – Now in the AWS Marketplace

    by Jeff Barr | on | in AWS Marketplace, EC2 Container Service | | Comments

    During my trip to the AWS Loft earlier this month, I spoke to 8 startups for the AWS Podcast.  Almost all of them told me that they are making use of Docker on AWS, either directly or via Amazon EC2 Container Service. They love the flexibility that it gives them, and appreciate the ease with which they can move from development (often on a laptop) to test, and then on to production while remaining highly confident that their code and their configurations will work as expected in each environment.

    In order to enable a very wide variety of use cases, we are making the Docker Trusted Registry (DTR) available in the AWS Marketplace. You can launch it on an EC2 instance in order to create a private registry.

    This new offering supports the popular laptop-to-cloud workflow by giving you a central, highly accessible location to store and manage your Docker images for deployment in your chosen on-premises or cloud environment. You can create custom access control levels and use them to regulate access to the images in your registry. You can require the use of SSL certificates or LDAP entries, and you can take advantage of all of the network access controls that are part of the Virtual Private Cloud (VPC).

    To learn more about the configuration options that are available to you, read the post New AWS Support for Commercially-Supported Docker Applications: Docker Trusted Registry and Docker Engine on the AWS Partner Network Blog.

    — Jeff;

  • Alert Logic Cloud Insight – Product Tour

    by Jeff Barr | on | in Guest Post, Security | | Comments

    I love to see all of the cool products and services that the Members of the AWS Partner Network (APN) build and bring to market. In the guest post below, my colleague Shawn Anderson takes you on a tour of Alert Logic’s new Cloud Insight product.

    — Jeff;

    In August, Alert Logic introduced Alert Logic Cloud Insight, which identifies vulnerabilities in operating systems and applications running on EC2 instances and configuration issues with AWS accounts and services. This product discovers and evaluates an AWS environment using data provided by EC2, Virtual Private Cloud, Auto-Scaling, Elastic Load Balancing, IAM, and RDS APIs. Currently, Alert Logic is offering a 30-day free trial of Cloud Insight.

    To begin using Cloud Insight you first login to the Cloud Insight web portal and give Cloud Insight access to your AWS environment via an IAM role. There are step-by-step instructions provided in the product describing how you set up this access:

    Cloud Insight will automatically discover all of the hosts and services associated with your AWS environment. Cloud Insight then automatically creates a dedicated security subnet in your VPC and launches a virtual Alert Logic appliance in the subnet. Within a few minutes you will see the results of the discovery process in the topology view:

    The topology view shows the relationship between your  AWS assets, The relationships (lines between assets) are updated dynamically as your AWS environment changes. To complete your setup, you select the assets you want to be part of Cloud Insight’s continuous assessments. You can choose to protect an entire region, VPC, or subnet. You can make adjustments to this scope at any time.

    Once you finish this step, Cloud Insight is up and running. It will continuously scan your assets and audit your environment configuration, and identify vulnerabilities and configuration issues it encounters. In the topology view you can see where the issues were discovered, color-coded for severity:

    By accessing the Remediation page, you can see a list of prioritized remediation actions that will address the identified vulnerabilities and configuration issues. The prioritization of actions is based on contextual analysis using a proprietary methodology:

    By taking these steps you can see that, for example, an upgrade to one Apache HTTP_server image addresses several vulnerabilities discovered in the environment:

    When a remediation action is completed, mark it complete and Cloud Insight will rescan the impacted hosts to verify that the vulnerability has been eliminated.

    Cloud Insight is well suited for a security analyst who wants to identify critical exposures in their environment.  Additionally Cloud Insight is accessible via APIs meaning that you could incorporate it into a continuous deployment program.  For more information on Cloud Insight you can visit Alert Logic’s website where you can access a few short videos, product documentation, and request your free trial.

    — Shawn Anderson, Global Ecosystem Alliance Lead, AWS Partner Network

  • New Spot Fleet Option – Distribute Your Fleet Across Multiple Capacity Pools

    by Jeff Barr | on | in Amazon EC2 | | Comments

    Last week I spoke to a technically-oriented audience at the Pacific Northwest PHP Conference. As part of my talk, I described cloud computing as a mix of technology and business, and made my point by talking about Spot instances. The audience looked somewhat puzzled at first, but as I explained further I could see their eyes light up as they started to think about the ways that they could save money for their companies by way of creative coding!

    Earlier this year I wrote about the Spot Fleet API, and showed you how to use it to manage thousands of Spot instances with a single call to the RequestSpotFleet function. Today we are introducing a new “allocation strategy” option for that API. This option will allow you to create a Spot fleet that contains instances drawn from multiple capacity pools (a set of instances of a given type, within a particular region and Availability Zone).

    As part of your call to RequestSpotFleet, you can include up to 20 launch specifications. If you make an untargeted request (by not specifying an Availability Zone or a subnet), you can target multiple capacity pools within an AWS region. This gives you access to a lot of EC2 capacity, and allows you to set up fleets that are a good match for your application.

    You can set the allocation strategy to either of the following values:

    • lowestPrice – This is the default strategy. It will result in a Spot fleet that contains instances drawn from the lowest priced pool(s) specified in your request.
    • diversified – This is the new strategy, and it must be specified as part of your request. It will result in a Spot fleet that contains instances drawn from all of the pools specified in your request, with the exception of those where the current Spot price is above the On-Demand price.

    This option allows you to choose the strategy that most closely matches your goals for each Spot fleet. The following table can be used as a guide:

    lowestPrice diversified
    Fleet Size Fine for modest-sized fleets. However, a request for a large fleet can affect pricing in the pool with the lowest price. Works well with larger fleets.
    Total Fleet Operating Cost Can be unexpectedly high if pricing in the pool spikes. Should average 70%-80% off of On-Demand over time.
    Consequence of Capacity Fluctuation in a Pool Entire fleet subject to possible interruption and subsequent replenishment. Fraction of fleet (1/Nth of total capacity) subject to possible interruption and subsequent replenishment.
    Application Characteristics Short-running.
    Not time sensitive.    		 Long-running.
    Time sensitive.
    Typical Applications Scientific simulations, research computations. Transcoding, customer-facing web servers, HPC, CI/CD.

    If you create a fleet using the diversified strategy and use it to host your web servers, it is a good idea to select multiple pools and to have a fallback option in case all of them become unavailable.

    Diversified allocation works really well in conjunction with the new resource-oriented bidding feature that we launched last month. When you use resource-oriented bidding and specify diversified allocation, each of the capacity pools in your launch specification will include the same number of capacity units.

    To make use of this new strategy, simply include it in your CLI or API-driven request. If you are using the CLI, simply add the following entry to your configuration file:

    "AllocationStrategy": "diversified"

    If you are using the API, specify the same value in your SpotFleetRequestConfigData.

    This option is available now and you can start using it today.

    — Jeff;

  • Elastic Load Balancing Update – More Ports & Additional Fields in Access Logs

    by Jeff Barr | on | in Amazon EC2, Amazon Elastic Load Balancer | | Comments

    Many AWS applications use Elastic Load Balancing to distribute traffic to a farm of EC2 instances. An architecture of this type is highly scalable since instances can be added, removed, or replaced in a non-disruptive way. Using a load balancer also gives the application the ability to keep on running if an instance encounters an application or system problem of some sort.

    Today we are making Elastic Load Balancing even more useful with the addition of two new features: support for all ports and additional fields in access logs.

    Support for All Ports
    When you create a new load balancer, you need to configure one or more listeners for it. Each listener accepts connection requests on a specific port. Until now, you had the ability to configure listeners for a small set of low-numbered, well-known ports (25, 80, 443, 465, and 587) and to a much larger set of ephemeral ports (1024-65535).

    Effective today, load balancers that run within a Virtual Private Cloud (VPC) can have listeners for any port (1-65535). This will give you the flexibility to create load balancers in front of services that must run on a specific, low-numbered port.

    You can set this up in all of the usual ways: the ELB API, AWS Command Line Interface (CLI) / AWS Tools for Windows PowerShell, a CloudFormation template, or the AWS Management Console. Here’s how you would define a load balancer for port 143 (the IMAP protocol):

    To learn more, read about Listeners for Your Load Balancer in the Elastic Load Balancing Documentation.

    Additional Fields in Access Logs
    You already have the ability to log the traffic flowing through your load balancers to a location in S3:

    In order to allow you to know more about this traffic, and to give you some information that will be helpful as you contemplate possible configuration changes, the access logs now include some additional information that is specific to a particular protocol. Here’s the scoop:

    • User Agent – This value is logged for TCP requests that arrive via the HTTP and HTTPS ports.
    • SSL Cipher and Protocol – These values are logged for TCP requests that arrive via the HTTPS and SSL ports.

    You can use this information to make informed decisions when you think about adding or removing support for particular web browsers, ciphers, or SSL protocols. Here’s a sample log entry:

    2015-05-13T23:39:43.945958Z my-loadbalancer 192.168.131.39:2817 10.0.0.1:80 0.000086 0.001048 0.001337 200 200 0 57 "GET https://www.example.com:443/ HTTP/1.1" "curl/7.38.0" DHE-RSA-AES128-SHA TLSv1.2

    You can also use tools from AWS Partners to view and analyze this information. For example, Splunk shows it like this:

    And Sumo Logic shows it like this:

     

    To learn more about access logging, read Monitor Your Load Balancer Using Elastic Load Balancing Access Logs.

    Both of these features are available now and you can start using them today!

    — Jeff;

  • Route 53 Improvements – Calculated Health Checks and Latency Checks

    by Jeff Barr | on | in Route 53 | | Comments

    Amazon Route 53 is a highly available and scalable Domain Name System (DNS) web service. Route 53 connects user requests to infrastructure running in AWS (EC2 instances, load balancers, and S3 buckets), and can also be used to route users to infrastructure outside of AWS. You can configure Route 53 to perform periodic health checks, and to fail over to alternate endpoints should a check fail. You can also create a monitoring & alerting system for your websites and applications using the health checks.

    Today we are adding two new types of health checks: calculated and latency measurement.

    Calculated Health Checks
    You can now combine the results of multiple Route 53 health checks into a single value using Boolean operations (AND, OR, and NOT). This allows you to create a single health check that combines the results of other checks in a useful way. For example, I have three web sites on the same EC2 instance. I’ll start by creating health checks for each one:

    Next I’ll create a calculated health check that represents the overall health of the instance:

    As you can see from the screen shot, I could also choose to create a health check that would report as healthy as long as a certain number (perhaps 2 out of 3) of the other health checks were also healthy. This would avoid false alarms and would allow me to take one site down for maintenance without causing this health check to fail.

    While my example checked three different domains that happened to be running on the same instance, this is not a constraint. You could check multiple website components that happened to be on the same domain but managed by different administrative groups within your organization. Or, you could check multiple domains that supply web services to your app, and then roll up the results into a single check that represents the state of your dependencies.

    Latency Measurement Health Checks
    You can also configure Route 53 to measure and report on metrics that affect latency: TCP connection time, time to first byte, and (for SSL connections) the time to complete the SSL handshake. The first one indicates how long it takes Route 53 to establish a connection to the endpoint; the second one indicates the overall time until the first byte of data is returned. The third measures the time it takes to set up an SSL connection, an operation which involves 2 round-trips.

    The latency measurement is performed as a part of the health check, and (if you check on Latency graphs) reported to CloudWatch.

    Here’s how you configure latency measurement when you are setting up a health check:

    The results are visible in the Console:

    You can view the results with respect a single AWS region by selecting it from menu (a blended result is shown by default). You can also see the individual metrics (currently 32 per endpoint) in the CloudWatch console:

    Available Now
    The new health checks are available now and you can start using them today. Take a look at the Route 53 Pricing page to learn more about pricing for health checks.

    — Jeff;