AWS Partner Network (APN) Blog

Editor’s Note: Each month, we plan on doing an Amazon Aurora digest to highlight pieces from our APN Partners that profile Amazon Aurora.

Have you used Amazon Aurora? Have you helped your customers migrate to Aurora?

More and more, our APN Partners are telling us about the work that they’re doing with Aurora. We want to share with you a note from our Amazon Aurora team, along with links to a number of pieces of informational content developed by a few APN Partners highlighting Amazon Aurora best practices.

Amazon Aurora is a MySQL-compatible relational database engine that combines the speed, availability and security of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. Our customers run mission-critical database workloads across multiple industries and our partners have been an important driver of customer success. We value the expertise that our partners bring to the table and support them through tools and training as customers continue to migrate increasingly complex workloads to Aurora. As we continue to help organizations move to Aurora and AWS, we look forward to strengthening and expanding our relationship with APN Partners.

– The Amazon Aurora team

Case Studies

CorpInfo, a Premier APN Consulting Partner: AWS Case Study – GoGuardian

AppsAssociates, a Premier APN Consulting Partner: PetTrax – Migrating to Amazon Aurora

Webinars & Presentations

AppsAssociates recorded joint webinar: Deploying High Performance Databases in the Cloud

Alfresco, an Advanced APN Technology Partner: Scaling Massive Content Stores with Amazon Aurora

Whitepapers/eBooks

AppsAssociates ebook: Three Reasons to Migrate your Database to the Cloud

AppsAssociates whitepaper: Amazon Aurora, A Fast, Affordable and Powerful RDBMS

Blog Posts

Alfresco: How Alfresco powered a 1.2 Billion document deployment on Amazon Web Services

BluePi, a Standard APN Consulting Partner: Amazon Aurora – Superior Cloud Database

Do you have links to content you’d like to share with us detailing how your firm has used Amazon Aurora to provide additional value to customers? Let us know! Email us: apn-blog@amazon.com

Are you a telecom provider looking to transform your business on the cloud? If yes, then we have an exciting announcement to share with you. Today, APN Consulting Partner Ericsson announced plans to form a global business, technology, and services alliance with AWS to accelerate cloud transformation for telecom service providers. Ericsson is creating a global team of experts focused on AWS, and is opening cloud innovation centers with customers, with AWS support.

Why Ericsson and AWS?

As the Ericsson team explained today in their press release, “New technologies such as mobile broadband and the cloud have transformed the competitive battleground for service providers. Today’s leading telecoms companies seek to capitalize on new opportunities such as Internet of Things (IoT) and big data analytics and at the same time, leverage the latest technologies to improve productivity and efficiency, increase agility, reduce complexity and risk, as well as create unique value for their customers.

Ericsson will leverage AWS’ services and expertise to work closely with service providers to implement a cloud adoption framework to fit their unique business objectives. The Ericsson teams will consist of program directors, solutions architects and system engineers who are trained on AWS and Ericsson technologies and bring together deep service provider industry and cloud domain expertise. AWS is supporting Ericsson in this effort with a broad range of resources that may include solutions architects, professional services and training. The companies’ efforts are designed to accelerate cloud adoption through cloud transformation programs, data center consolidation and application migration.”

To learn more about this alliance and about Telstra, an Australian operator and the first Ericsson customer who is hosting a joint cloud innovation center with Ericsson, read the company’s press release.

We have some exciting news for US veterans working at AWS Partner Network (APN) firms.

To help veterans succeed, we’ve worked with the Department of Veterans Affairs to make AWS Certification exams eligible for VA reimbursement.

Qualifying US veterans covered under a GI Bill can now submit a reimbursement request to the Department of Veterans Affairs for exams taken after December 10, 2015 and purchased from Webassessor. The VA will cover exam fees up to $2000. (Costs connected with preparing for a certification such as training courses or practice exams are not reimbursable.)

Learn more on the AWS Blog, and on our website.

Want to hear about the value of AWS Training & Certification from other APN Partners? Click here.

We recently announced that we’ll be opening an AWS Region in Canada in the coming year. Today, we’d like to share the story of The Globe and Mail, a Canadian customer already taking advantage of the benefits of the AWS platform, with the help of Premier APN Consulting Partner ClearScale. The company is also an AWS Big Data, DevOps, Marketing & Commerce, and Mobile Competency holder. ClearScale helped The Globe build an article recommendation engine on AWS.

“ClearScale was introduced to The Globe and Mail by one of the Amazon Web Services (AWS) representatives. We were more than impressed by a publication with a 170-year history looking to develop a cutting-edge recommendation engine for their mobile application,” explains Pavel Pragin, CEO, ClearScale. “The Globe & Mail was interested in a reliable cloud services partner that could deliver an innovative and reliable solution. ClearScale’s solution helped The Globe & Mail expand their readership and gain additional advertising opportunities while meeting their tight project deadlines.”

Who is The Globe and Mail? 

The Globe and Mail is Canada’s most read newspaper with a national weekly digital readership of 4.7 million. In print for 170 years, the newspaper delivers coverage of national, international, business, technology, arts, entertainment, and lifestyle news.

Why did The Globe and Mail engage AWS? 

The Globe and Mail was planning to launch a new application that enables its growing online readership to access stories and breaking news from mobile devices. And to increase reader engagement, it wanted to serve up targeted articles based on each reader’s individual interests. The Globe team considered building a custom system on premises, but concluded that hosting its article recommendation engine in the cloud would be faster and would provide greater flexibility for testing different algorithms. The Globe team had already been prototyping a number of recommendation algortithms on AWS. Having had a great experience, the team decided to use AWS for its official platform. The Globe uses a number of AWS services, including:

• Amazon Kinesis to collect massive amounts of information (including user clicks, article engagement, and article metadata)
• Amazon DynamoDB for a fast and flexible database to store data
• Amazon Elastic MapReduce (Amazon EMR) for data processing
• Amazon EC2 for compute capacity
• Amazon S3 for secure, durable, highly scalable object storage
• Amazon Simple Queue Service (Amazon SQS) for queuing messages
• Amazon CloudWatch to monitor the performance of the system
• AWS CloudFormation to give developers and systems administrators an easy way to create and manage a collection of AWS resources
• AWS OpsWorks to help users configure and operate applications using Chef automation

What are the benefits for The Globe and Mail? 

By building its personalized recommendation system on AWS, The Globe has experienced a number of benefits:

• The team was able to get the service to market in just three months, less than half the time it would have taken had the newspaper built an on-premises solution
• The Globe has obtained a flexible platform for testing, allowing the company to improve its mobile app over time
• The Globe is dramatically increasing reader engagement; initial results show that parts of the mobile app that promote a personalized selection of articles based on the recommendation engine are seeing a 25% greater click-through than a selection of the current most popular articles that would otherwise be the top performers

Want to learn more? Read the full case study here.

To learn more about ClearScale, visit the company’s website, or check out the company’s AWS Partner Directory page.

As a Partner Solutions Architect, I look at a lot of AWS Partner Network (APN) Partner software and services. I like trying new things and experiencing the exciting solutions that our APN Partners are building. Security is job zero at AWS, so when I work with our APN Partners, there’s one thing I look for above all others, and that’s to understand if the APN Partner is following best practices to protect customer data and any customer AWS account they may access. If it seems like the APN Partner’s product will need to access a customer account, I’ll check to see how the APN Partner is getting credentials from the customer. If a partner is asking customers for AWS Identity and Access Management (IAM) access keys and secret keys, I halt my investigation and focus on helping the partner fix this approach.

It’s not that I have a problem with partners accessing customer accounts—APN Partners can add incredible functionality and value to the resources in an AWS account. For example, they can analyze AWS CloudTrail logs, or help optimize costs by monitoring a customer’s Amazon Elastic Compute Cloud (Amazon EC2) usage. The problem here is how the APN Partner is accessing the AWS account. IAM access keys and secret keys could be used anywhere, by anyone who has them. If a customer gives these keys to an APN Partner, they need to be able to trust that the APN Partner is adhering to best practices to protect those keys. This should really resonate with APN Partners, who need to store and protect their customers’ keys, but lack control over how customers manage those keys. Using IAM access keys and secret keys for cross-account access is not ideal for anyone. Fortunately, there is a better way.

Cross-account IAM roles allow customers to securely grant access to AWS resources in their account to a third party, like an APN Partner, while retaining the ability to control and audit who is accessing their AWS account. Cross-account roles reduce the amount of sensitive information APN Partners need to store for their customers, so that they can focus on their product instead of managing keys. In this blog post, I explain some of the risks of sharing IAM keys, how you can implement cross-account IAM roles, and how cross-account IAM roles mitigate risks for customers and for APN Partners, particularly those who are software as a service (SaaS) providers.

The problem(s) with sharing IAM keys

On AWS, access and secret keys are credentials that allow access to AWS APIs in an account. These can be associated with an IAM user in an account or with the root user of an account. Sharing these keys with external parties can create a lot of headache for everyone involved. The root of the problem with sharing IAM keys is that they can be used until explicitly revoked by a customer, and that the keys can be used from any computer that has Internet access (this includes servers, laptops, mobile phones, etc.). If an APN Partner wants to use IAM access and secret keys in a customer’s product, here are some important questions that both the APN Partner and the customer should be able to confidently answer:

• Are keys being managed securely? Are they encrypted when they are transmitted and stored? Who has access to the keys? What processes protect the keys from being exfiltrated from the partner’s systems?
• Are keys being rotated frequently? If you are a customer, will the APN Partner tell you when to rotate your keys? As an APN Partner, how can you make sure all your customers frequently rotate their keys? How do you coordinate key rotation with your customers to minimize downtime?
• Can you control who has access to the customer’s AWS account? For both customers and APN Partners, how will you know who uses a key, and from where?
• Is the access policy associated with the key too permissive? For APN Partners that have a database of keys, how many of those keys provide too much access, or are root account keys?

APN Partners could build solutions that address these considerations, and customers could take on more work to ensure that their keys are being handled in a secure way. However, this involves a lot of undifferentiated heavy lifting that cross-account IAM roles can handle for both parties.

How cross-account IAM roles work

An IAM role is an AWS identity with an access policy that determines what the role can and can’t do in AWS. They are designed to be assumable by another AWS identity that is already authenticated to AWS. When an identity assumes a role, it receives temporary credentials and the same access policy as the role. You may be familiar with how roles work if you have used EC2 instance profiles, or have set up an AWS Lambda function.

A cross-account IAM role is an IAM role that includes a trust policy that allows AWS identities in another AWS account to assume the role.  Put simply, I can create a role in one AWS account that delegates specific permissions to another AWS account. Let’s take a look at the overall process as it applies to APN Partner software that needs to access a customer account:

1. A customer creates an IAM role in their account with an access policy for accessing the resources that the APN partner requires. They specify that the role can be assumed by the partner’s AWS account by providing the APN Partner’s AWS account ID in the trust policy for that role.
2. The customer gives the Amazon Resource Name (ARN) of the role to the APN partner. The ARN is the fully qualified name of the role.
3. When the APN Partner’s software needs to access the customer’s account, the software calls the AssumeRole API in the AWS Security Token Service (STS) with the ARN of the role in the customer’s account. STS returns a temporary AWS credential that allows the software to do its work.

Customers can include conditional checks on the trust policy associated with an IAM role to limit how third parties can assume the role. An example of this would be the external ID check. The external ID is a string defined in the trust policy that the partner must include when assuming a role.  External IDs are a good way to improve the security of cross-account role handling in a SaaS solution, and should be used by APN Partners who are implementing a SaaS product that uses cross-account roles.

How cross-account roles mitigate risks

Using cross-account roles addresses and mitigates a number of risks, so it’s worth taking a closer look at how cross-account roles help address the security questions we listed earlier.

• Are keys being managed securely? The Role allows the partner to get temporary credentials when they need to use them. Unlike Access and Secret Keys, these don’t need to be stored, so partner doesn’t need to be concerned with managing keys.
• Are keys being rotated frequently? Credentials generated by STS expire after an hour. Many of our software development kits (SDKs) have credential providers that handle this automatically, so neither the APN Partner nor the customer needs to manage credential rotation manually.
• Can you control who has access to the customer’s AWS account? The role in the customer’s account can be assumed only by an authenticated AWS identity in the partner’s account. The customer knows that only the APN Partner is accessing their resources, and the APN Partner can focus solely on managing and protecting the IAM roles and users in their own account.
• Is the access policy associated with the key too permissive? A role can’t have root key permissions, and since the cross-account role’s trust policy specifies the partner’s account, it is more likely that the permissions in the role’s access policy will reflect the partner’s requirements. APN Partners can encode cross-account IAM roles in AWS CloudFormation templates to make sure that customers are giving them exactly the permissions they need.

Using AWS CloudFormation to help customers create roles

Providing documentation to customers about how to create roles is straightforward, but if an APN Partner wants to make this simpler for their customers, they can package the role in an AWS CloudFormation template. This approach lets customers deploy a role into their account quickly without having to copy and paste trust and access policy documents.

I’ve provided an example AWS CloudFormation template that creates a cross-account role with an external ID for accessing Amazon S3. If you’re an APN Partner who needs access customer accounts, you can create templates like this with the access and trust policies you need, and instruct your customers to instantiate the template in their account. This template has parameters for the account number to make testing easy. In practice, partners will want to hard code their account number in the trust policy in the template. This template outputs the ARN that the customer can give to you to allow access to their account.

Where to go from here

In this blog post, I’ve explained why using IAM keys to provide AWS account access to third parties is not ideal, and talked about how APN Partners can implement cross-account IAM roles in their products. To learn more, take a look at AWS documentation on IAM roles, cross-account IAM roles, and external IDs. If you’re an APN Partner who wants to discuss this practice in more detail, please feel free to email me: rocamora@amazon.com.

Looking for implementation details? Take a look at our SDK documentation that explains how to use AWS SDKs to build this into your product. Here are links to documentation about assuming roles for the Java, Ruby, Golang, .NET, and Python SDKs. Most partners won’t need this, but if you want to see the low-level details, take a look at the STS API documentation. I also recommend that you audit your own AWS accounts using a credential report to see if you are providing cross-account access with an IAM user.

If you’re a consulting partner or an MSP, you probably find yourself needing to access your customers’ AWS accounts through the AWS Management Console. You can use cross-account IAM roles for this as well by using the switch roles feature of the AWS Management Console. This gives you access to your customers’ accounts without having to manage users, passwords, or keys.

Finally, if you’re an AWS customer and work with an APN Partner who is requiring keys, ask them how you can use cross-account roles with their products, and don’t hesitate to share this post with them.

Are you looking to gain deep, hands-on technical training for AWS NoSQL and IoT services?

We’re excited to announce a series of NoSQL and IoT technical workshops for APN Partners we’ll be delivering in Seattle on Feb 23rd – 24th, and Boston on March 29th – 30th. Intended for Advanced and Premier APN Partners who hold at least one AWS Competency, these workshops will be led by our AWS Solution Architecture (SA), AWS Business Development (BD), and AWS Partner Network (APN) teams.

What Will the Workshops Cover?

The NoSQL workshop will cover data lifecycle management and an overview of how customers are using NoSQL to meet business challenges.  Participants will learn how to create low-latency NoSQL databases through a deep dive into DynamoDB schema creation, tuning, and performance management, including caching with Elasticache.  In addition to DynamoDB best practices, we will also cover Cassandra and MongoDB on Amazon EC2 as well as tips to migrate from RDBMS to NoSQL databases.

The IoT workshop will cover an introduction to the IoT ecosystem including core IoT services including AWS IoT, AWS Lambda, API Gateway, Amazon SNS, and Amazon SQS.  We will also include a deep dive and walkthrough of AWS IoT integration with Amazon DynamoDB, Amazon Kinesis, and AWS Lambda.  You will see demos of AWS IoT services with AWS Lambda + Amazon DynamoDB capture and AWS IoT Device Shadows, as well as participate in hands-on labs where you will set up your IoT device to capture data into Amazon DynamoDB using AWS IoT and Lambda.

Attendance at both workshops is not required but highly encouraged.

Who Should Attend?

We will be prioritizing seats for Advanced/Premier APN Partners who hold at least one AWS Competency. As both workshops will be technical, 400+ level, and hands-on, we encourage senior engineers to attend. There is no cost to attend these workshops. However, travel and related expenses must be covered by your firm.

What’s the Schedule?

Our first workshops will be hosted in Seattle and Boston, with more to come around the globe throughout 2016.

Seattle Workshops:

• February 23rd: AWS NoSQL
• February 24th: AWS IoT

Boston Workshops:

• March 29th: AWS NoSQL
• March 30th: AWS IoT

NoSQL Agenda:

IoT Agenda:

*please note that these agendas may be subject to change.

Interested in Attending?

If you’d like to learn more about the workshops, or would like to reserve a spot, please reach out to your Partner Development Manager.

The APN Premier tier is currently comprised of 46 APN Consulting Partners globally.

Qualifying for the Premier tier, our top tier for APN Consulting Partners, is no small task. These APN Partners have invested significantly in their AWS practice, have extensive experience in deploying customer solutions on AWS, have a strong bench of trained and certified technical consultants, hold at least one APN Competency, are AWS MSP-certified and/or hold the AWS DevOps Competency, have expertise in project management, and have a healthy revenue-generating consulting business on AWS.

In addition to all of the hard work they do for customers on AWS, our Premier Partners develop some fantastic thought leadership pieces for general consumption. This year, we’re going to be highlighting some of the content we see coming from our Premier APN Partners’ blogs. We’re starting with posts from January 29th – February 9th.

As you’ll find in this first roundup, while the posts cover a number of different topics, there are some common themes. Many of our Premier Partners are talking about the Internet of Things (IoT), DevOps, and the impact of digital transformation on company culture.

Check out some of the most recent posts we’ve seen from a number of our Premier Consulting Partners:

• 2nd Watch
  • Cloud Cost Optimization with AWS
• Accenture 
  • Free Your Mind: With a Public Cloud-First Attitude, You’re Not Trading Down on Security 
• Bulletproof
  • Is Your IT Department Ready for the Internet of Things? 
• Capgemini
  • The Security Paradox: Greater Privacy = Greater Freedom
• Cloudreach
  • Keeping it Cloudy: How to Build Company Culture 
  • Cloudreach Partners with CloudEndure to Add Leading Disaster Recovery & Migration Solution to its Cloud Platform Offering 
• Classmethod
  • ゲーム開発のためのサービス Amazon Lumberyard & Amazon GameLift が公開
• Cloud Technology Partners
  • Why Will the Internet of Things Drive Data Integration in 2016? 
• Cognizant
  • How Data Fuels Banking’s Digital Transformation
• CorpInfo
  • Managing Cloud Services
• CSC
  • IDSM: Inside the IT Hub
  • Are You an Ecosystem Driver? Business Models for the Digital Age 
• Datapipe
  • How to Develop a Well-Architected Framework by Using AWS’s Four Pillars for Best Practices
• DLT
  • How to Understand and Utilize DevOps for Your Agency
• Logicworks
  • Company Culture is Key to Cloud Success
  • Incident Response on AWS Cloud: The Case for Outsourcing
• NORDCLOUD
  • Welcome to the AWSome Day Nordic Roadshow 2016
  • Learn to Develop Secure and Scalable Cloud Applications on the New Developing on AWS Course
• Serverworks
  • NAT ゲートウェイを簡単セットアップ
• Slalom
  • The Case for Delivery Centers, Part 1, Part 2, and Part 3 
• TriNimbus
  • Overcoming the Challenges to DevOps
  • No More Excuses, Canada – The AWS Cloud is Here to Stay!
• Wipro
  • Beyond Traditional Business Intelligence 

Stay tuned for continued highlights of content produced by our Premier Partners throughout 2016.

This is a guest post co-authored by Josh Campbell and Brandon Chavis, two of our Partner Solutions Architect (SA) team members. 

What is Terraform?

Terraform by HashiCorp, an APN Technology Partner and AWS DevOps Competency Partner, is an “infrastructure as code” tool similar to AWS CloudFormation that allows you to create, update, and version your AWS infrastructure. Terraform has a great set of features that make it worth adding to your toolbelt, including:

• Friendly custom syntax, but also has support for JSON.
• Visibility into changes before they actually happen.
• Built-in graphing feature to visualize the infrastructure.
• Understands resource relationships. One example is failures are isolated to dependent resources while non-dependent resources still get created, updated, or destroyed.
• Open source project with a community of hundreds of contributors who add features and updates.
• The ability to break down the configuration into smaller chunks for better organization, re-use, and maintainability. The last part of this article goes into this feature in detail.

New to Terraform?

This article assumes you have some familiarity with Terraform already.

We recommend that you review the HashiCorp documentation for getting started to understand the basics of Terraform.  Conveniently, their documentation uses AWS as the example cloud infrastructure of choice!

Keeping Secrets

You can provide Terraform with an AWS access key directly through the provider, but we recommend that you use a credential profile already configured by one of the AWS SDKs.  This prevents you from having to maintain secrets in multiple locations or accidentally committing these secrets to version control.  In either scenario, you’ll want to be sure to read our best practices for maintaining good security habits.  Alternatively, you can run Terraform from one or more control servers that use an IAM instance profile.

Each instance profile should include a policy that provides the appropriate level of permissions for each role and use case.  For example, a development group may get a control server with an attached profile that enables them to run Terraform plans to create needed resources like Elastic Load Balancers and Auto Scaling Groups, but not resources outside the group’s scope like Redshift clusters or additional IAM roles.  You’ll need to plan your control instances carefully based on your needs.

To use an instance or credential profile with Terraform, inside your AWS provider block simply remove the access_key and secret_key declarations and any other variables that reference access and secret keys.  Terraform will automatically know to use the instance or credential profile for all actions.

If you plan to share your Terraform files publicly, you’ll want to use a terraform.tfvars file to store sensitive data or other data you don’t want to make public.  Make sure this file is excluded from version control (for example, by using .gitignore).  The file can be in the root directory and might look something like this:

region = “us-west-2”
keypair_name = “your_keypair_name”
corp_ip_range = “192.168.1.0/24”
some_secret = “your_secret”

Building Blocks

An advantage of using an infrastructure of code tool is that your configurations also become your documentation.  Breaking down your infrastructure into components makes it easier to read and update your infrastructure as you grow. This, in turn, helps makes knowledge sharing and bringing new team members up to speed easier.

Because Terraform allows you to segment chunks of infrastructure code into multiple files (more on this below), it’s up to you to decide on a logical structure for your plans.  With this in mind, one best practice could be to break up Terraform files by microservice, application, security boundary, or AWS service component. For example, you might have one group of Terraform files that build out an ECS cluster for your inventory API and another group that builds out the Elastic Beanstalk environment for your production front-end web application.

Additionally, Terraform supports powerful constructs called modules that allow you to re-use infrastructure code.  This enables you to provide infrastructure as building blocks that other teams can leverage.  For example, you might create a module for creating EC2 instances that uses only the instance types your company has standardized on.  A service team can then include your module and automatically be in compliance.  This approach creates enablement and promotes self-service.

Organizing Complex Services with Modules

Modules are logical groupings of Terraform configuration files.  Modules are intended to be shared and re-used across projects, but can also be used within a project to help better structure a complex service that includes many infrastructure components.  Unfortunately, Terraform does not recursively load files in subdirectories, so you have to use modules to add structure to your project instead of using one large file or multiple smaller files in the same directory.  You can then execute these modules from a single configuration file (we’ll use main.tf for this example) in the parent directory where your sub-directories (modules) are located.  Let’s examine this concept a bit closer.

Modules run sequentially, so you must understand your order of dependencies.  For example, a module to create a launch configuration must run before a module that creates an Auto Scaling group, if the Auto Scaling group depends on the newly created launch configuration.  Each module should be ordered top (first to run) to bottom (last to run) in your main.tf file (more on this later) appropriately.

Terraform allows you to reference output variables from one module for use in different modules. The benefit is that you can create multiple, smaller Terraform files grouped by function or service as opposed to one large file with potentially hundreds or thousands of lines of code.  To use Terraform modules effectively, it is important to understand the interrelationship between output variables and input variables.   At a high level, these are the steps you would take to make an object in one module available to another module:

1. Define an output variable inside a resource configuration (module_A).  The scope of resource configuration details are local to a module until declared as an output.
2. Declare the use of module_A’s output variable in the configuration of another module, module_B. Create a new key name in module_B and set the value equal to the output variable from module_A.
3. Finally, create a variables.tf file for module_B. In this file, create an input variable with the same name as the key you defined in module_B in step 2. This variable is what allows dynamic configuration of resource(s) in a module. Because this variable is limited to module_B in scope, you need to repeat this process for any other module that needs to reference module_A’s output.

As an example, let’s say we’ve created a module called load_balancers that defines an Elastic Load Balancer. After declaring the resource, we add an output variable for the ELB’s name:

output "elb_name" {
value = "${aws_elb.elb.name}"
}

You can then reference this ELB name from another module using ${module.load_balancers.elb_name}. Each module (remember that a module is just a set of configuration files in their own directory) that wants to use this variable must have its own variables.tf file with an input variable of elb_name defined.

Sample Directory Layout Using Local Modules for Organization

There are a few things to note about this layout:

• main.tf is the file that will invoke each module. It provides no configuration itself aside from declaring the AWS provider.
• Each subdirectory is a module. Each module contains its own variables.tf file.
• We’re using version control to store our infrastructure configuration. Because this is a public repository, we’ve asked Git to not store our .tfstate files since they contain sensitive information.  In production, you’ll want to store these files in private version control, such as AWS CodeCommit or Amazon S3, where you can control access.
• The site module contains security groups and a VPC. These are resources that have no other dependencies, and most other resources depend on these.  You could create separate security_groups and vpcs modules as well. This module is the first to be called in main.tf.

Following Along with an Example

In this section, we’ll walk you through an example project that creates an infrastructure with several components, including an Elastic Load Balancer and an Auto Scaling group, which will be our focus.

Main.tf

Looking at main.tf you will see that there are several modules defined. Let’s focus on the autoscaling_groups module first:

module "autoscaling_groups" {
source = "./autoscaling_groups"
public_subnet_id = "${module.site.public_subnet_id}"
webapp_lc_id = "${module.launch_configurations.webapp_lc_id}"
webapp_lc_name =
"${module.launch_configurations.webapp_lc_name}"
webapp_elb_name = "${module.load_balancers.webapp_elb_name}"
}

The first thing to notice is the line source = "./autoscaling_groups".  This simply tells Terraform that the source files for this module are in the autoscaling_groups subdirectory.  Modules can be local folders as they are above, or they can come from other sources like an S3 bucket or different Git repository. This example assumes you will run all Terraform commands from the parent directory where main.tf exists.

autoscaling_groups Modules

If you then examine the autoscaling_groups directory you’ll notice that  it includes two files:  variables.tf and webapp-asg.tf.  Terraform will run any .tf files it finds in the Module directory, so you can name these files whatever you want.  Now look at line 20 of autoscaling_groups/webapp-asg.tf:

load_balancers = ["${var.webapp_elb_name}"]

Here we’re setting the load_balancers parameter to an array that contains a reference to the variable webapp_elb_name.  If you look back at main.tf, you’ll notice that this name is also part of the configuration of the autoscaling_groups module.  Looking in autoscaling_groups/variables.tf, you’ll see this variable declared with empty curly braces ({}).  This is the magic behind using outputs from other modules as input variables.

load_balancers Modules

To bring it together, examine load_balancers/webapp-elb.tf and find this section:

output "webapp_elb_name" {
value = "${aws_elb.webapp_elb.name}"
}

Here we’re telling Terraform to output a variable named webapp_elb_name, whose value is equal to our ELB name as determined by Terraform after the ELB is created for us.

Summary

In this example:

1. We created an output variable for the load_balancers module named webapp_elb_name in load_balancers/webapp-elb.tf.
2. In main.tf under the autoscaling_groups module configuration, we set the webapp_elb_name key to the output variable of the same name from the load_balancers module as described above. This is how we reference output variables between modules with Terraform.
3. Next, we defined this input variable in autoscaling_groups/variables.tf by simply declaring variable webapp_elb_name {}. Terraform automatically knows to set the value of webapp_elb_name to the output variable from the load_balancers module, because we declared it in the configuration of our autoscaling_groups module in step 2.
4. Finally, we’re able to use the webapp_elb_name variable within autoscaling_groups/webapp-asg.tf.

Collaborating with Teams

Chances are, if you’re using Terraform to build production infrastructure, you’re not working alone. If you need to collaborate on your Terraform templates, the best way to sync is by using Atlas by HashiCorp. Atlas allows your infrastructure templates to be version controlled, audited, and automatically deployed based on workflows you configure. There’s a lot to talk about when it comes to Atlas, so we’ll save the deep dive for our next blog post.

Wrapping up

We hope we’ve given you a good idea of how you can leverage the flexibility of Terraform to make managing your infrastructure less difficult. By using modules that logically correlate to your actual application or infrastructure configuration, you can improve agility and increase confidence in making changes to your infrastructure. Take a look at Terraform by HashiCorp today: https://www.terraform.io/.

We work across internal teams, including our AWS Professional Services team, APN Partner Programs teams, and the AWS Quick Start creation team, to create and publish a wide range of documents on the APN Portal. For example, AWS Quick Start guides are often published on the Portal prior to being generally available. And our AWS Professional Services colleagues are very proactive in producing content on topics of importance to Partners.

A top priority for our team is to publish more and more business and technical content for your consumption on the Portal in 2016. Today, I want to highlight some of the new and updated documents available to you on the APN Portal.

Best Practices for Tagging AWS Resources

Written by two of our AWS Professional Services colleagues, this document provides a detailed overview of AWS tagging best practices, and provides numerous tagging examples.

In AWS, tagging enables customers to track their AWS resources by assigning key-value pairs to an AWS resource so that they can differentiate the function, environment, owner, operating system, line of business (LOB), or any other delineating factors of that resource, in order to automate, scale, secure, govern, and manage their AWS infrastructure. We recommend that AWS customers and APN Partners use tagging for AWS services, whenever supported, and establish a tagging standard for their environment. This document focuses on the AWS services that support tagging, and provides tagging examples.

Download here.

SaaS Solutions on AWS: Tenant Isolation Architectures (Updated)

Are you thinking about, or in the process of, developing software as a service (SaaS) on AWS? Written by a senior AWS Partner Solutions Architect (SA), this document covers the different SaaS deployment models and the combination of AWS services and AWS Partner Network (APN) partner solutions that can be used to achieve a scalable, available, secure, performant, and cost-effective SaaS offering.

First released on the Portal in October 2015, it has been updated to reflect new AWS services and our newest AWS Competency for DevOps solutions.

Download here.

Trend Micro Deep Security on the AWS Cloud Quick Start (Updated)

Trend Micro Deep Security is a host-based security product that provides Anti-Malware, Host Firewall, Intrusion Prevention, File Integrity Monitoring, Log Inspection, Web Application Firewalling, and Content Filtering modules in a single agent running in the guest operating system. This Quick Start reference deployment guide describes how to deploy Trend Micro Deep Security version 9.6 on the Amazon Web Services (AWS) cloud. It contains links to AWS CloudFormation templates that automate this deployment as well as additional supporting information.

The guide was recently updated to reflect simplified templates, and an update Quick Start for deployment on existing Amazon VPC architectures. Download here.

Standardized Architecture for NIST 800 – 53 on the AWS Cloud Quick Start

This Quick Start reference deployment guide discusses architectural considerations and steps for deploying security-focused baseline environments on the Amazon Web Services (AWS) cloud. Specifically, this Quick Start deploys a standardized environment that helps support National Institute of Standards and Technology (NIST) 800-53 / Risk Management Framework (RMF) certifications, accreditations, and compliance processes. The deployment guide includes links for viewing and launching AWS CloudFormation templates that automate the deployment.

This Quick Start is first in a set of AWS Enterprise Accelerator – Compliance offerings, which provide security-focused, standardized architecture solutions to help Managed Service Organizations (MSOs), cloud provisioning teams, developers, integrators, and information system security officers (ISSOs) adhere to strict security, compliance, and risk management controls.

Download here.

The AWS Managed Service Partner Program Validation Checklist, Version 3.0

The AWS Managed Service Program Partner Validation Checklist is intended for APN Partners who are interested in applying for the Managed Service Program. This checklist provides the criteria necessary to achieve the Managed Service Program Partner designation and represents AWS’ view of the capabilities that a “Next Generation Managed Service Provider” should have to support customers through all phases of the customer engagement lifecycle: plan, design, migrate or build, run, and optimize. Partners should fill out this checklist based on their own assessment of capabilities, and the assessment will serve as the basis for discussion during the Partner capabilities audit.

The first AWS Validation Checklist (version 1.0) was released in 2014, and the team has made a number of version changes in the past year. In the newest checklist, Version 3.0, the following changes have been made:

1. Added Business Health Section and Controls (Section 1)
2. Added Billing and Cost Management Section and Controls (Section 5)
3. Added new standards specific to Next-Generation MSP differentiation (Sections 3, 7, and 9)
4. Removed “process and documentation” as allowable evidence in some sections; these are now only met with technology demonstrations
5. Removed optional Business and Technical Capability Identifiers; we encourage MSP Partners to achieve AWS Competencies to differentiate their business
6. Moved all White Label Audit Requirements and Policies to Appendix B

We wrote all about the 3.0 checklist in a previous blog post we encourage you to check out.

Download the updated checklist here.

APN Partners, what do you want to learn about?

What topics are important to you as an APN Partner? We want to hear from you!

Email us at apn-blog@amazon.com, and tell us what topics would be beneficial for us to focus on for content development.

Tod Golding is an AWS Partner Solutions Architect (SA). He works closely with our SaaS Partner ecosystem. 

If you’re designing a true multi-tenant software as a service (SaaS) solution, you’re likely to devote a significant amount of time to selecting a strategy for effectively partitioning your system’s tenant data. On Amazon Web Services (AWS), your partitioning options mirror much of what you see in the wild. However, if you’re looking at using Amazon DynamoDB, you’ll find that the global, managed nature of this NoSQL database presents you with some new twists that will likely influence your approach.

Before we dig into the specifics of the DynamoDB options, let’s look at the traditional models that are generally applied to achieve tenant data partitioning. The list of partitioning solutions typically includes the following variations:

• Separate database – each tenant has a fully isolated database with its own representation of the data
• Shared database, separate schema – tenants all reside in the same database, but each tenant can have its own representation of the data
• Shared everything – tenants all reside in the same database and all leverage a universal representation of the data

These options all have their strengths and weaknesses. If, for example, you’d like to support the ability for tenants to have their own data customizations, you might want to lean toward a model that supports separate schemas. If that’s not the case, you’ll likely prefer a more unified schema. Security and isolation requirements are also key factors that could shape your strategy. Ultimately, the specific needs of your solutions will steer you toward one or more of these approaches. In some cases, where a system is decomposed into more granular services, you may see situations where multiple strategies are applied. The requirements of each service may dictate which flavor of partitioning best suits that service.

With this as a backdrop, let’s look at how these partitioning models map to the different partitioning approaches that are available with DynamoDB.

Linked Account Partitioning (Separate Database)

This model is by far the most extreme of the available options. Its focus is on providing each tenant with its own table namespace and footprint with DynamoDB. While this seems like a fairly basic goal, it is not easily achieved. DynamoDB does not have the notion of an instance or some distinct, named construct that can be used to partition a collection of tables. In fact, all the tables that are created by DynamoDB are global to a given region.

Given these scoping characteristics, the best option for achieving this level of isolation is to introduce separate linked AWS accounts for each tenant. To leverage this approach, you need to start by enabling the AWS Consolidated Billing feature. This option allows you to have a parent payer account that is then linked to any number of child accounts.

Once the linked account mechanism is established, you can then provision a separate linked account for each new tenant (shown in the following diagram). These tenants would then have distinct AWS account IDs and, in turn, have a scoped view of DynamoDB tables that are owned by that account.

While this model has its advantages, it is often cumbersome to manage. It introduces a layer of complexity and automation to the tenant provisioning lifecycle. It also seems impractical and unwieldy for environments where there might be a large collection of tenants. Caveats aside, there are some nice benefits that are natural byproducts of this model. Having this hard line between accounts makes it a bit simpler to manage the scope and schema of each tenant’s data. It also provides a rather natural model for evaluating and metering a tenant’s usage of AWS resources.

Tenant Table Name Partitioning (Shared Database, Separate Schema)

The linked account model represents a more concrete separation of tenant data. A less invasive approach would be to introduce a table naming schema that adds a unique tenant context to each DynamoDB table. The following diagram represents a simplified version of this approach, prepending a tenant ID (T1, T2, and T3) to each table name to identify the tenant’s ownership of the table.

This model embraces all the freedoms that come with an isolated tenant scheme, allowing each tenant to have its own unique data representation. With this level of granularity, you’ll also find that this aligns your tenants with other AWS constructs. These include:

• The ability to apply AWS Identity and Access Management (IAM) roles at the table level allows you to constrain table access to a given tenant role.
• Amazon CloudWatch metrics can be captured at the table level, simplifying the aggregation of tenant metrics for storage activity.
• IOPS is applied at the table level, allowing you to create distinct scaling policies for each tenant.

Provisioning also can be somewhat simpler under this model since each tenant’s tables can be created and managed independently.

The downside of this model tends to be more on the operational and management side. Clearly, with this approach, your operational views of a tenant will require some awareness of the tenant table naming scheme in order to filter and present information in a tenant-centric context. The approach also adds a layer of indirection to any code you might have that is metering tenant consumption of DynamoDB resources.

Tenant Index Partitioning (Shared Everything)

Index-based partitioning is perhaps the most agile and common technique that is applied by SaaS developers. This approach places all the tenant data in the same table(s) and partitions it with a DynamoDB index. This is achieved by populating the hash key of an index with a tenant’s unique ID. This essentially means that the keys that would typically be your hash key (Customer ID, Account ID, etc.) are now represented as range keys.  The following example provides a simplified view of an index that introduces a tenant ID as a hash key. Here, the customer ID is now represented as a range key.

This model, where the data for every tenant resides in a shared representation, simplifies many aspects of the multi-tenant model. It promotes a unified approach to managing and migrating the data for all tenants without requiring a table-by-table processing of the information. It also enables a simpler model for performing tenant-wide analytics of the data. This can be extremely helpful in assessing and profiling trends in the data.

Of course, there are also limitations with this model. Chief among these is the inability to have more granular, tenant-centric control over access, performance, and scaling. However, some may view this as an advantage since it allows you to have a more global set of policies that respond to the load of all tenants instead of absorbing the load of maintaining policies on a tenant-by-tenant basis. When you choose your partitioning approach, you’ll likely strike a balance between these tradeoffs.

Another consideration here is that this approach could be viewed as creating a single point of failure. Any problem with the shared table could affect the entire population of tenants.

Abstracting Client Access

Each technique outlined in this blog post requires some awareness of tenant context. Every attempt to access data for a tenant requires acquiring a unique tenant identifier and injecting that identifier into any requests to manage data in DynamoDB.

Of course, in most cases, end-users of the data should have no direct knowledge that their provider is a tenant of your service. Instead, the solution you build should introduce an abstraction layer that acquires and applies the tenant context to any DynamoDB interactions.

This data access layer will also enhance your ability to add security checks and business logic outside of your partitioning strategies, with minimal impact to end-users.

Supporting Multiple Environments

As you think about partitioning, you may also need to consider how the presence of multiple environments (development, QA, production, etc.) might influence your approach. Each partitioning model we’ve discussed here would require an additional mechanism to associate tables with a given environment.

The strategy for addressing this problem varies based on the partitioning scheme you’ve adopted. The linked account model is the least affected, since the provisioning process will likely just create separate accounts for each environment. However, with table name and index-based partitioning, you’ll need to introduce an additional qualifier to your naming scheme that will identify the environment associated with each table.

The key takeaway is that you need to be thinking about whether and how environments might also influence your entire build and deployment lifecycle. If you’re building for multiple environments, the context of those environments likely need to be factored into your overall provisioning and naming scheme.

Microservice Considerations

With the shift toward microservice architectures, teams are decomposing their SaaS solutions into small, autonomous services. A key tenant of this architectural model is that each service must encapsulate, manage, and own its representation of data. This means that each service can leverage whichever partitioning approach best aligns with the requirements and performance characteristics of that service.

The other factor to consider is how microservices might influence the identity of your DynamoDB tables. With each service owning its own storage, the provisioning process needs assurance that the tables it’s creating for a given service are guaranteed to be unique. This typically translates into adding some notion of the service’s identity into the actual name of the table. A catalog manager service, for example, might have a table that is an amalgam of the tenant ID, the service name, and the logical table name. This may or may not be necessary, but it’s certainly another factor you’ll want to keep in mind as you think about the naming model you’ll use when tables are being provisioned.

Agility vs. Isolation

It’s important to note that there is no single preferred model for the solutions that are outlined in this blog post. Each model has its merits and applicability to different problem domains. That being said, it’s also important to consider agility when you’re building SaaS solutions. Agility is fundamental to the success of many SaaS organizations and it’s essential that teams consider how each partitioning model might influence its ability to continually deploy and evolve both applications and business.

Each variation outlined here highlights some of the natural tension that exists in SaaS design. In picking a partitioning strategy, you must balance the simplicity and agility of a fully shared model with the security and variability offered by more isolated models.

The good news is that DynamoDB supports all the mechanisms you’ll need to implement each of the common partitioning models. As you dig deeper into DynamoDB, you’ll find that it actually aligns nicely with many of the core SaaS values. As a managed service, DynamoDB allows you to shift the burden of management, scale, and availability directly to AWS. The schemaless nature of DynamoDB also enables a level of flexibility and agility that is crucial to many SaaS organizations.

Kicking the Tires

The best way to really understand the merits of each of these partitioning models is to simply dig in and get your hands dirty. It’s important to examine the overall provisioning lifecycle of each partitioning approach and determine how and where it would fit into a broader build and deployment lifecycle. You’ll also want to look more carefully at how these partitioning models interact with AWS constructs. Each approach has nuances that can influence the experience you’ll get with the console, IAM roles, CloudWatch metrics, billing, and so on. Naturally, the fundamentals of how you’re isolating tenants and the requirements of your domain are also going to have a significant impact on the approach you choose.

Are you building SaaS on AWS? Check out the AWS SaaS Partner Program, an APN Program providing Technology Partners with support to build, launch, and grow SaaS solutions on AWS.