Introduction

Earlier this week, Amazon SNS released worldwide SMS delivery and made it available in six AWS Regions. Worldwide SMS delivery means that you can now send SMS text messages directly to mobile phone numbers in more than 200 countries. Along with this expansion, SNS also enabled default “opt-in” of recipient phone numbers. This creates more possibilities for SMS messages, such as those needed for multi-factor authentication (MFA) or one-time passcodes.

In this blog post, we will cover notable changes from the previous SMS offering and highlight the new features. We’ll also describe SMS account-level configuration, show how to set up delivery status, and describe the new SMS API calls and message attributes.

What’s new with worldwide SMS delivery?

The following table shows the previous SMS offering and the features and options that are now available with the new worldwide SMS delivery from Amazon SNS.

Notable Changes to existing SMS subscriptions

• All SMS phone numbers actively subscribed to an SNS topic will see a new long code or short code when a new message is delivered after June 28. After the new long/short code is established, recurring messages should come from the same code (though this is not guaranteed). This feature is called Sticky Sender ID.

• When a user sends “STOP”, the phone number will not be unsubscribed from the SNS topic. Instead, the phone number is added to the new “opted out” list that you manage. The SMS phone number that is subscribed to the SNS topic will still show subscribed even after a “STOP” request. However, if you publish to the SNS topic for which the opted out phone number exists, the delivery will fail and the failed message will be logged to Amazon CloudWatch Logs (if you enabled delivery status, of course).

SMS account-level configuration

You can set account-level text messaging preferences in the Amazon SNS console. In messaging preferences, you can specify account-level SMS message type, specify the spend limit, enable delivery status, provide a sender ID, and specify an Amazon S3 bucket to receive daily SMS delivery reports.

Here’s what the configuration screen looks like, including a description for each preference.

Delivery status

What is SNS delivery status?

SNS delivery status logs successful and unsuccessful SMS deliveries to Amazon CloudWatch Logs. Delivery status for SMS enables developers to collect logged data on successful and unsuccessful delivery attempts of their SMS messages to individual phone numbers. It also provides information about dwell times in Amazon SNS. Dwell times indicate: 1) how long it takes to deliver the message to the destination carrier from the time the message was published to SNS, and 2) how long it takes for the recipient carrier to acknowledge delivery of that message from the time the message was published to SNS. In addition, the price per message is logged so that you can manage your SMS costs.

Note: Delivery status plays a very important role in troubleshooting SMS deliverability. If you have issues with deliverability, consider enabling and configuring delivery status so that you can quickly identify and troubleshoot the issue.

Enabling delivery status

You can enable the delivery status feature for SMS with the Amazon SNS console, the AWS CLI, or the AWS SDKs. The simplest way is to enable delivery status using the SNS console, as described next.

Enabling SMS delivery status using the Amazon SNS console

Step 1: Log in to the Amazon SNS console.

Step 2: Navigate to Text Messaging (SMS).

Step 3: Select Create IAM role next to Default IAM role for success feedback.

Step 4: On the SNS is requesting permission to use resources in your account page, choose Allow. This allows Amazon SNS to gain write access to Amazon CloudWatch Logs on your behalf.

You should now see an IAM role as shown in the following example:

Example CloudWatch Logs

Delivery status for SMS creates two CloudWatch Log Groups: Successful and Failed. The groups have the following format:

Successful: sns/<region>/<accountID>/DirectPublishToPhoneNumber

Failed: sns/<region>/<accountID>/DirectPublishToPhoneNumber/Failure

Note: If one of the groups does not exist, then no logs have been delivered to that group, or the IAM role assumed by SNS does not have permission to write to CloudWatch Logs.

Example log for successful SMS delivery

The delivery status log for a successful SMS delivery looks like the following example:

Example log for failed SMS delivery

The delivery status log for a failed SMS delivery looks like the following example:

New APIs for SMS worldwide delivery

Along with the announcement of worldwide SMS delivery, there are new API calls and attributes for setting SMS preferences and text message delivery.

SMS Account Preferences

The following are the new API calls for setting and getting account SMS preferences:

SetSMSAttributes

GetSMSAttributes

The following are the new SMS account attributes for setting and getting SMS account preferences:

DefaultSenderID, MonthlySpendLimit, DeliveryStatusIAMRole, DeliveryStatusSuccessSamplingRate, DefaultSMSType, and UsageReportS3Bucket.

For all SMS attributes, see SetSMSAttributes and GetSMSAttributes in the Amazon Simple Notification Service API reference.

Sending a message

You can send an SMS text message through the SNS console, CLI, or SDK. Remember, you can now publish directly to the individual phone number without going through an SNS topic or first waiting for the user to opt in.

Note: When you send an SMS message, specify the phone number using the E.164 format. E.164 is a standard for the phone number structure used for international telecommunication. Phone numbers that follow this format have a maximum of 15 digits. They are prefixed with the plus character (+) and the country code. For example, a U.S. phone number in E.164 format would appear as +1XXX5550100.

To send an SMS message by using one of AWS SDKs, use the same SNS Publish action in the SDK that corresponds to the Publish request in the Amazon SNS API.

To directly publish to a phone number, you'll use the new phoneNumber and message parameters of the Publish API.

Java

.withMessage(message) // Specify the text message sent to the mobile phone.

.withPhoneNumber(phoneNumber) // Specify the phone number using the E.164 format.

.withMessageAttributes(smsAttributes) // Specify one or more of the three new MessageAttributes shown next.

JavaScript

Message, PhoneNumber, and additional MessageAttributes

var params = { "Message": value, "PhoneNumber": value }

The following are the new MessageAttributes for the Publish API:

AWS.SNS.SMS.SenderID  // (Optional) defaults to an SNS specified value, if not specified by the sender.

AWS.SNS.SMS.MaxPrice // (Optional) defaults to no max price.

AWS.SNS.SMS.SMSType // (Optional) defaults to "Transactional".

Tip: For Java, you will apply your attributes to the PublishRequest object of the Amazon SNS client.

Questions?

Check out the SMS FAQ here.

Resources

Amazon SNS Developer Guide

Amazon SNS SMS FAQ

Amazon SNS Forum

Happy text messaging!

Today, we are excited to announce support in Amazon Cognito for Security Assertion Markup Language (SAML) 2.0 authentication. SAML 2.0 is an XML-based open standard that is used to transfer authentication and authorization data between parties.

Now developers can sign in users through their own SAML identity providers and provide secure access through Amazon Cognito to their AWS resources, such as Amazon S3 and Amazon DynamoDB. 

With SAML support in Amazon Cognito, developers can also configure IAM roles to provide different permissions to different users and groups.

This post covers how you can configure SAML for Amazon Cognito to authenticate your users and assign them to specific AWS IAM roles. 

1. Configure your identity pool for a SAML provider

To configure your identity pool to support a SAML provider, choose the SAML tab in the Authentication provider section of the Amazon Cognito console. If you want to use an existing SAML provider, select the provider and save the changes for the identity pool.

To create a new SAML provider:

• Choose the AWS IAM console link to go to the IAM console.
• In the left pane, choose Identity Providers.
• Choose Create Provider and then choose SAML in Provider Type.

• Type a name for the SAML provider and choose the SAML metadata document file that you received from your SAML Identity Provider (IdP). This file typically includes the issuer's name, expiration information, and a certificate that can be used to validate the SAML assertion response received from the IdP. For Microsoft Active Directory Federation Services (AD FS) you can download it from
  https://<yourservername>/FederationMetadata/2007-06/FederationMetadata.xml.
  For more information related to different SAML IdPs, see Integrating Third-Party SAML Solution Providers with AWS.

2. Create roles

Now you create roles that you want to associate with your end user.  Unlike other types of IdPs, when you use a SAML-based IdP with Cognito, you do not have to select authenticated role for your Cognito identity pool. However, you can choose Create new role next to the Authenticated role label in the Cognito console and allow it to create the correct trust relationship as a convenient way to start creating new IAM roles. You can further configure the role policy and trust policy by using the IAM Console. Refer to the Cognito developer guide to learn more about the role policy by using context keys available with Cognito. You can create as many roles as required by your application and configure your IdP to populate the role in SAML assertion response. The role received in the SAML assertion from your IdP will be used for vending the AWS credentials for your users.

The trust policy for your role used by Cognito will look similar to the following policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Federated": "cognito-identity.amazonaws.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "cognito-identity.amazonaws.com:aud": "us-east-1:12345678-identity-pool-id-123abc"
        },
        "ForAnyValue:StringLike": {
          "cognito-identity.amazonaws.com:amr": "authenticated"
        }
      }
    }
  ]
}

3. Configure your SAML IdP

After you create the SAML provider with AWS and setup your IAM roles, you can configure your SAML IdP to add relying party trust between your IdP and AWS.  Many IdPs allow you to specify a URL from which the IdP can read an XML document that contains relying party information and certificates. For AWS, you can use https://signin.aws.amazon.com/static/saml-metadata.xml

The next step is to configure the SAML assertion response from your IdP to populate the claims needed by AWS. For details on the claim configuration, see Configuring SAML Assertions for the Authentication Response. For instructions on how to configure many popular IdPs, see Integrating Third-Party SAML Solution Providers with AWS.

4. Customize Roles

Using SAML with Amazon Cognito Identity allows the IAM role to be customized for the end user.  You configure your organization's IdP in a way that it maps users or groups in your organization to the IAM roles you want those users to assume. The exact steps for performing the mapping depend on what IdP you're using. Essentially you will be configuring the claim attribute https://aws.amazon.com/SAML/Attributes/Role to get populated in the SAML assertion response. This attribute specifies one or more pairs of comma delimited roles and provider ARNs. These are the roles for which the user is allowed to get credentials.

If multiple roles are received in the SAML assertion response, the optional customRoleArn parameter should be populated while calling getCredentialsForIdentity.  The role received in the customRoleArn parameter will be used if it matches a role in the SAML assertion claim. Your app will need to provide the logic or prompt the user to choose a role if multiple roles are included in the SAML assertion.

The value of the https://aws.amazon.com/SAML/Attributes/RoleSessionName claim attribute is used for setting the role session name in the credentials vended by Amazon Cognito. Role session name is set to a generic value 'CognitoIdentityCredentials' by Amazon Cognito if your identity pool supports multiple SAML providers and multiple assertions are received in the logins map which have different values of RoleSessionName.

5. Authenticate users with SAML IdP and get SAML assertion

After you have set up your Amazon Cognito identity pool and SAML IdP, you are ready to authenticate the user against the SAML IdP and federate with Amazon Cognito. The SAML IdP issues a SAML assertion for the authenticated user. The base-64 encoded assertion response must be passed to Amazon Cognito as a value in the logins map. The key in this map will be the ARN of the SAML provider that you associated with your identity pool in step #1.

To federate with SAML based IdP, you will need to determine the URL which is being used to initiate the login. AWS federation utilizes IdP-initiated login. In ADFS 2.0 URL takes the form of https://<fqdn>/adfs/ls/IdpInitiatedSignOn.aspx?loginToRp=urn:amazon:webservices

The following describes how to extract the SAML assertion response from ADFS. 

• Populate the IdP URL, username, and password in the following code. Connect to the IdP with NTLM authentication. This should return an HTML input form in the response, which will have a  SAML assertion as a value. The following example shows the Java code for this step.  For different SAML IdPs and platforms, this step might be different. Refer to the documentation from your SAML identity provider to learn how to get the SAML assertion response.

        String idpurl= "https://<fqdn>/adfs/ls/IdpInitiatedSignOn.aspx?loginToRp=urn:amazon:webservices";

        Authenticator.setDefault(new Authenticator() {
            @Override
            public PasswordAuthentication getPasswordAuthentication() {
                return new PasswordAuthentication(username, password.toCharArray());
            }
        });

        URL urlRequest = new URL(idpurl);
        HttpURLConnection conn = (HttpURLConnection) urlRequest.openConnection();
        conn.setDoOutput(true);
        conn.setDoInput(true);
        conn.setRequestMethod("GET");

        InputStream stream = conn.getInputStream();
        BufferedReader reader = new BufferedReader(new InputStreamReader(stream));

        StringBuilder response = new StringBuilder();
        String str;
        while ((str = reader.readLine()) != null) {
            response.append(str);
        }
        String html = response.toString();

• After you receive the HTML response from the IdP, use the following example code to parse the response and extract the Base64-encoded SAML response from it. 

        DocumentBuilder builder = DocumentBuilderFactory.newInstance().newDocumentBuilder();
        Document document = builder.parse(new InputSource(new StringReader(html)));

        Element docElement = document.getDocumentElement();
        NodeList inputNodes = docElement.getElementsByTagName("input");
        if (inputNodes != null && inputNodes.getLength() > 0) {

            for (int i = 0; i <= inputNodes.getLength() - 1; i++)  {
                Node inputNode = inputNodes.item(i);
                NamedNodeMap inputNodeAttributes = inputNode.getAttributes();

                Map inputAttibuteMap = new HashMap();

                for (int j = 0; j <= inputNodeAttributes.getLength() - 1; j++) {
                    inputAttibuteMap.put(inputNodeAttributes.item(j).getNodeName(),
                            inputNodeAttributes.item(j).getNodeValue());
                }                
                if (inputAttibuteMap.get("name").equals("SAMLResponse")) {
                    // Base 64 encoded SAML Authentication response.
                    return inputAttibuteMap.get("value");
                }
            }
        }
     

6. Use the SAML assertion to call Amazon Cognito

The examples below show how to call Cognito with the SAML assertion on different platforms. Please note, only the Enhanced flow in Amazon Congito is supported with SAML based IdP.

Java:

    AmazonCognitoIdentity client = new AmazonCognitoIdentityClient(new AnonymousAWSCredentials());
    // Get id request. This only needs to be executed the first time and the result should be cached.
    GetIdRequest idRequest = new GetIdRequest();
    idRequest.setAccountId("aws account id");
    idRequest.setIdentityPoolId("identity pool id");

    Map logins = new HashMap();
    logins.put("arn:aws:iam::aws account id:saml-provider/name", "base64 encoded assertion response");
    idRequest.setLogins(logins );
    GetIdResult idResp = client.getId(idRequest);
    String identityId = idResp.getIdentityId();

    // GetCredentialsForIdentity call.
    GetCredentialsForIdentityRequest credRequest = new GetCredentialsForIdentityRequest();
    request.setIdentityId(identityId);
    request.setLogins(logins);

    // If SAML assertion contains multiple roles, resolve the role by setting the custom role
    request.setCustomRoleArn("arn:aws:iam::aws account id:role/admin");

    GetCredentialsForIdentityResult credetialsResult = client.getCredentialsForIdentity(credRequest)

Android

If you are using the Android SDK you can populate the logins map with the SAML assertion as follows.

    Map logins = new HashMap();
    logins.put("arn:aws:iam::aws account id:saml-provider/name", "base64 encoded assertion response");
    // Now this should be set to CognitoCachingCredentialsProvider object.
    CognitoCachingCredentialsProvider credentialsProvider = new CognitoCachingCredentialsProvider(context, identity pool id, region);
    credentialsProvider.setLogins(logins);
    // If SAML assertion contains multiple roles, resolve the role by setting the custom role
    credentialsProvider.setCustomRoleArn("arn:aws:iam::aws account id:role/customRoleName");
    // This should trigger a call to Cognito service to get the credentials.
    credentialsProvider.getCredentials();

    - (AWSTask<NSDictionary<NSString*,NSString*> *> *) logins {
        //this is hardcoded for simplicity, normally you would asynchronously go to your SAML provider 
        //get the assertion and return the logins map using a AWSTaskCompletionSource
        return [AWSTask taskWithResult:@{@"arn:aws:iam::aws account id:saml-provider/name":@"base64 encoded assertion response"}];
    }

    // If SAML assertion contains multiple roles, resolve the role by setting the custom role.
    // Implementing this is optional if there is only one role.
    - (NSString *)customRoleArn {
        return @"arn:aws:iam::accountId:role/customRoleName";
    }
    

Introduction

In April, we launched the Beta version of a new Amazon Cognito feature called Amazon Cognito User Pools. The user pools feature makes it easy for developers to add sign-up and sign-in functionality to web and mobile applications. In this blog post we will walk through how to integrate Amazon Cognito User Pools with Amazon API Gateway. It's a assumed that you have a basic understanding of API Gateway and the API Gateway's custom authorizer.

Scenarios for integrating Amazon Cognito User Pools with API Gateway

After the successful user authentication in your mobile or web application, your application will need to perform operations in the context of that user. For that you need a back-end application running on your server. In this post, we walk through Amazon Cognito User Pools and API Gateway integration in the context of a Notes application and a service that lets users create, retrieve, and delete notes.

In order to provide a 'Notes Service', you first require sign-up and sign-in functionality for your web or mobile application. This functionality can be implemented using Amazon Cognito User Pools. Additionally, you also expose an API for the Notes service though API Gateway. This API creates, retrieves, and deletes notes for an authenticated user. You also must implement authorization in your API so that you can identify the authenticated user and perform operations in the context of that user, such as Create Note and Delete Note.

After a user signs in successfully to your Notes application, Amazon Cognito User Pools returns an ID and Access Token to your app for the authenticated user. The Access Token can then be used to authorize API invocations through API Gateway using the API Gateway's custom authorizer. For more information on tokens, see Using Tokens with Amazon Cognito User Pools.

Develop a sample Notes Service using AWS Lambda and API Gateway

The following steps describe how to develop the Notes service and its integration with API Gateway and Amazon Cognito User Pools. If you are familiar with API Gateway, you can skim through this section without creating an actual API.

1. Create a 'Notes' table that stores notes for your users in Amazon DynamoDB. The primary partition key for the table will be a 'userId' string, and the primary sort key will be a 'noteId' string. The userId is a globally unique identifier of an authenticated user in your user pool. The noteId is a unique ID associated with each note created by the user.

2. Create a new AWS Lambda function, called 'dynamodb_manager', using a Lambda blueprint, ‘simple-mobile-backend’. Choose the Lambda function role with proper permissions. We will use this function in API Gateway to perform operations against the 'Notes' table.

3. Create an API named 'NotesService' in API Gateway. Create a '/notes' resource with a 'POST' method. For Integration Type, choose Lambda function and choose 'dynamodb_manager' as the Lambda function.

4. Create a 'NoteCreateModel' model in your 'NotesService' API and add it to a method request, as follows:

{
  "title": "Note Create Model",
  "type" : "object",
  "properties" : {
    "noteid" : {
       "type" : "string"
    },
    "note" : {
      "type" : "string"
    }
   },
  "required" : ["noteid", "note"]
}

This defines the input for a POST method for a '/notes' resource. In our model, we accept a 'note' and 'noteId' pair to create a note for a user. The other information we need to create a note for a user is a userid. We will get the userid from the access token which will be passed in an 'Authorization' header to your API.

6. Add the following body mapping template to your integration request.

#set($inputRoot = $input.path('$'))
{
  "operation": "create",
  "payload": {
      "Item" : {
          "userid" : "$context.authorizer.principalId",
          "noteid" : "$inputRoot.noteid",
          "note" : "$inputRoot.note"}
  },
  "tableName" : "Notes"
}

This invokes the 'dynamodb_manager' Lambda function and creates a note in the 'Notes' table. Notice how we retrieve the userid here: '$context.authorizer.principalId' will return the globally unique identifier for an authenticated user.

7. Create a new model 'Success' and attach it to your method response, as follows:

{
  "$schema" : "http://json-schema.org/draft-04/schema#",
  "title" : "Success Schema",
  "type" : "object",
  "properties" : {
    "message" : { "type" : "string" }
  }
}

8. Create a body mapping template in your integration response, as follows:

#set($inputRoot = $input.path('$'))
{
  "message" : "Note created"
}

This is the static response you will send to the application each time a note is successfully created. You can create additional body mapping templates to map errors.

Develop a Custom Authorizer for Amazon Cognito User Pools

After your API is created, you need to implement a custom authorizer for your API that will ensure that a request is coming from an authenticated user of your application. If it's a valid access token, we will generate a policy against a userId, which is a unique identifier (UUID) for a user. You can then use '$context.authorizer.principalId' in your API and get the userId value.

1. Download the blueprint for custom authorizer for Amazon Cognito User Pools

2. Unzip the file and modify the following variables with your userPoolId and region in the 'authorizer.js' file, as follows:

var userPoolId = ‘{REPLACE_WITH_YOUR_POOL_ID}’;
var region = ‘{REPLACE_WITH_YOUR_REGION}’;

3. Zip all the files again, name the .zip file cup_authorizer.zip, and create a Lambda function with that .zip file. Make sure that you only zip the inner files (authorizer.js and node_modules); do not zip the outer directory.

4. Choose Node.JS 4.3 as the Runtime for the Lambda function. For Handler, choose authorizer.handler.

5. Create a custom authorizer in your API, as shown next.

6. Now attach the authorizer in Method Request, as shown next.

7. Deploy the API. You can now test the API. If a valid Access Token for your user pool is passed to an API, the API will create a note in a DynamoDB table for that user. Otherwise, the API will throw an unauthorized message back to the application. Here's how your request would look:

Endpoint: {invokeUrlForYourAPI}/notes

Http Method : POST

Headers

Authorization : {Access Token of a user from your user pool}

Content-Type : application/json

Body

{
  "noteid" : "note1",
  "note" : "my first note"
}

This should create a note in DynamoDB for an authenticated user. If you pass an invalid Access Token or the Access Token is expired, a custom authorizer will throw an unauthorized message (401) back to the client.

Understanding the code

It is important to understand the code in the 'authorizer.js' file if you choose to make any further modifications.

1. The following packages must be installed and used in your Lambda function:

var jwt = require('jsonwebtoken');
var request = require('request');
var jwkToPem = require('jwk-to-pem');

2. Download the JSON Web Key Set (JWK Set) for your user pool and convert the keys to PEM format, as follows:

    var pems;

    if (!pems) {
    //Download the JWKs and save it as PEM
    request({
       url: iss + '/.well-known/jwks.json',
       json: true
     }, function (error, response, body) {
        if (!error && response.statusCode === 200) {
            pems = {};
            var keys = body['keys'];
            for(var i = 0; i < keys.length; i++) {
                //Convert each key to PEM
                var key_id = keys[i].kid;
                var modulus = keys[i].n;
                var exponent = keys[i].e;
                var key_type = keys[i].kty;
                var jwk = { kty: key_type, n: modulus, e: exponent};
                var pem = jwkToPem(jwk);
                pems[key_id] = pem;
            }
            //Now continue with validating the token
            ValidateToken(pems, event, context);
        } else {
            //Unable to download JWKs, fail the call
            context.fail("error");
        }
    });
    } else {
        //PEMs are already downloaded, continue with validating the token
        ValidateToken(pems, event, context);
    };

3. Validate the token, as follows:

function ValidateToken(pems, event, context) {

    var token = event.authorizationToken;
    //Fail if the token is not jwt
    var decodedJwt = jwt.decode(token, {complete: true});
    if (!decodedJwt) {
        console.log("Not a valid JWT token");
        context.fail("Unauthorized");
        return;
    }

    //Fail if token is not from your User Pool
    if (decodedJwt.payload.iss != iss) {
        console.log("invalid issuer");
        context.fail("Unauthorized");
        return;
    }

    //Reject the jwt if it's not an 'Access Token'
    if (decodedJwt.payload.token_use != 'access') {
        console.log("Not an access token");
        context.fail("Unauthorized");
        return;
    }

    //Get the kid from the token and retrieve corresponding PEM
    var kid = decodedJwt.header.kid;
    var pem = pems[kid];
    if (!pem) {
        console.log('Invalid access token');
        context.fail("Unauthorized");
        return;
    }

    //Verify the signature of the JWT token to ensure it's really coming from your User Pool

    jwt.verify(token, pem, { issuer: iss }, function(err, payload) {
      if(err) {
        context.fail("Unauthorized");
      } else {
        //Valid token. Generate the API Gateway policy for the user
        //Always generate the policy on value of 'sub' claim and not for 'username' because username is reassignable
        //sub is UUID for a user which is never reassigned to another user.

        var principalId = payload.sub;

        //Get AWS AccountId and API Options
        var apiOptions = {};
        var tmp = event.methodArn.split(':');
        var apiGatewayArnTmp = tmp[5].split('/');
        var awsAccountId = tmp[4];
        apiOptions.region = tmp[3];
        apiOptions.restApiId = apiGatewayArnTmp[0];
        apiOptions.stage = apiGatewayArnTmp[1];
        var method = apiGatewayArnTmp[2];
        var resource = '/'; // root resource
        if (apiGatewayArnTmp[3]) {
            resource += apiGatewayArnTmp[3];
        }

        //For more information on specifics of generating policy, see the blueprint for the API Gateway custom
        //authorizer in the Lambda console

        var policy = new AuthPolicy(principalId, awsAccountId, apiOptions);
        policy.allowAllMethods();

        context.succeed(policy.build());
      }
});
};

When you successfully return the policy from your Lambda function, you can then retrieve the userId value of an authenticated user in your API by using '$context.authorizer.principalId'. When you call context.fail("Unauthorized") from your function, it will send a 401 response back to the client. When you call context.fail("error"), it should send a 500 response back to the client.

We welcome your feedback on this feature in the Amazon Cognito forum.

Introduction

In April, we launched the beta version of a new Amazon Cognito feature called Your User Pools. Among other functionality, the User Pools feature makes it easy for developers to add sign-up and sign-in functionality to web apps. AWS Mobile SDKs for Android, JavaScript, and iOS are available with this beta launch. In this blog post we will show you how to access the new functionality by using the Amazon Cognito Identity SDK for JavaScript.

Integrating your user pool into your web app

To integrate this new feature into your app, follow the instructions in the Announcing Your User Pools in Amazon Cognito blog post to create your user pool. Note that Generate client secret must be unchecked when creating a web app; the Amazon Cognito Identity SDK for JavaScript doesn't support apps that have a client secret simply because the client secret could be easily viewed in your code.

Also for brevity, in this blog post we will focus on the functionality provided by the Amazon Cognito Identity SDK for JavaScript. Information about dependencies and setup instructions can be found in the GitHub repository.

Using your user pool in your web app

The first step for interacting with the new feature is to create a CognitoUserPool object by providing a UserPoolId and a ClientId. As mentioned, the SDK does not support the app client secret. If you configure your user pool app client with an app client secret, the SDK will throw exceptions.

Throughout the examples in this post, we will use the userPool object, the userData object (containing the user pool) and the username object, as shown in the following.

    AWSCognito.config.region = 'us-east-1';
        
    var poolData = {
        UserPoolId : '...', // your user pool id here
        ClientId : '...' // your client id here
    };
    var userPool = new AWSCognito.CognitoIdentityServiceProvider.CognitoUserPool(poolData);
    var userData = {
        Username : '...', // your username here
        Pool : userPool
    };

After creating a user pool object, users can be signed up for the application. The necessary information about the user can be collected through the web UI and used to populate CognitoUserAttribute objects that are passed in the signUp call.

    var attributeList = [];
    
    var dataEmail = {
        Name : 'email',
        Value : '...' // your email here
    };
    var dataPhoneNumber = {
        Name : 'phone_number',
        Value : '...' // your phone number here with +country code and no delimiters in front
    };
    var attributeEmail = new AWSCognito.CognitoIdentityServiceProvider.CognitoUserAttribute(dataEmail);
    var attributePhoneNumber = new AWSCognito.CognitoIdentityServiceProvider.CognitoUserAttribute(dataPhoneNumber);

    attributeList.push(attributeEmail);
    attributeList.push(attributePhoneNumber);

    var cognitoUser;
    userPool.signUp('username', 'password', attributeList, null, function(err, result){
        if (err) {
            alert(err);
            return;
        }
        cognitoUser = result.user;
        console.log('user name is ' + cognitoUser.getUsername());
    });

After signing up, the user needs to confirm the sign-up by entering a code sent either through SMS or email (based on the user pool settings). Alternatively, you can use a PreSignUp AWS Lambda function to automatically confirm your users. To confirm sign-up, you must collect the code ('123456' in the following example) received by the user and use it as follows: 

    cognitoUser.confirmRegistration('123456', true, function(err, result) {
        if (err) {
            alert(err);
            return;
        }
        console.log('call result: ' + result);
    });

The registration code can be resent by using the resendConfirmationCode method of a cognitoUser object. This is an unauthenticated call and only the username, the client ID, and the user pool information are needed.

A confirmed user can authenticate to obtain a session. The session contains an ID token that contains user claims, an access token that is used internally to perform authenticated calls, and a refresh token that is used internally to refresh the session after it expires each hour. For more information about tokens, see Using Tokens with Amazon Cognito Identity User Pools in the Amazon Cognito Developer Guide. When authentication is successful, the onSuccess callback is called. If authentication fails, the onFailure callback is called. If authentication requires MFA, the mfaRequired callback is called. You must invoke sendMFACode on the cognitoUser object. The verification code that is received must be passed and the user is finally authenticated.

    var authenticationData = {
        Username : '...', // your username here
        Password : '...', // your password here
    };
    var authenticationDetails = new AWSCognito.CognitoIdentityServiceProvider.AuthenticationDetails(authenticationData);

    var cognitoUser = new AWSCognito.CognitoIdentityServiceProvider.CognitoUser(userData);
    cognitoUser.authenticateUser(authenticationDetails, {
        onSuccess: function (result) {
            console.log('access token + ' + result.getAccessToken().getJwtToken());
        },

        onFailure: function(err) {
            alert(err);
        },
        mfaRequired: function(codeDeliveryDetails) {
            var verificationCode = prompt('Please input verification code' ,'');
            cognitoUser.sendMFACode(verificationCode, this);
        }
    });

After authenticating, a user can perform authorized operations such as, retrieve user attributes, verify user attributes (such as an unverified email address), delete user attributes, update user attributes, change the user password, and delete the user's account. For user pools that have an MFA setting of optional, users can enable or disable MFA just for themselves, at a user level. Signing out from the application clears the local user session so the user must authenticate again to establish a new session.

If users forget their passwords, they can initiate a forgotten password flow. A code will be sent to the user. The user uses this code together with a new password to complete the flow.  The relevant call is forgotPassword on a cognitoUser object that is unauthenticated; the relevant callbacks can be seen in the following.

    cognitoUser.forgotPassword({
        onSuccess: function (result) {
            console.log('call result: ' + result);
        },
        onFailure: function(err) {
            alert(err);
        },
        inputVerificationCode() {
            var verificationCode = prompt('Please input verification code ' ,'');
            var newPassword = prompt('Enter new password ' ,'');
            cognitoUser.confirmPassword(verificationCode, newPassword, this);
        }
    });

If you want to work with other AWS services, you must first create a federated identity pool. After you create this identity pool, you can get AWS credentials by passing the identity pool ID and the ID token (obtained earlier) when authenticating. The following example shows how to populate IdentityPoolId and pass the ID token through the Logins map.

    AWS.config.credentials = new AWS.CognitoIdentityCredentials({
        IdentityPoolId: 'us-east-1:XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX',
        Logins: {
            'cognito-idp.us-east-1.amazonaws.com/us-east-1_XXXXXXXXX': result.getIdToken().getJwtToken()
        }
    });

    AWS.config.credentials.get(function(err){
        if (err) {
            alert(err);
        }
    });

More examples and an overview of the code can be found in the GitHub repository. We welcome your feedback on this feature. You can reach us by creating an issue in the GitHub repository or posting to the Amazon Cognito forums.

Introduction

Last month AWS launched the beta version of  Amazon Cognito User Pools. With this feature, you can easily add sign-up and sign-in functionality to your mobile and web applications. You get a simple to use, fully managed service for creating and maintaining a user directory which can scale to hundreds of millions of users. You also benefit from the security and privacy best practices of AWS.

AWS Mobile SDKs for Android, JavaScript, and iOS are available with this beta launch. You can use these SDKs in your apps to interact with your user pools. In this blog we will show how to get started with the Mobile SDK for Android. For guidance on iOS or JavaScript see the Amazon Cognito Developer Guide. For a walk-through on getting started with iOS, please visit this blog.

Creating your user pools

Before you can use the SDK, you must create a user pool.  

1) Start by signing in to the Amazon Cognito console and choosing Manage your User Pools. This opens the page where you can see all of your Cognito user pools and also create new pools.

2) Provide a name for your pool and choose Step through settings to start customizing the pool. You can choose Review defaults to create the pool with default settings.

3) Choose the required attributes for users in your pool. The attributes you select as "Required" are must-have values for users to sign-up in the pool. Some attributes can be used as aliases, which means that users can sign in with the aliases instead of their username, if these attributes are verified. Let’s choose email, given name and phone number as required parameters and email and phone number as aliases. If a user in this pool has verified his or her email address, then this user can use that email address to sign in. The user will still be able to sign in with his or her username.

4) The next step is to set the password requirements; we will use default password settings.

5) In the next step you can enable multi-factor authentication (MFA) for users in this pool. We will leave MFA turned off for users in this pool.

6) Next, you can customize the messages that are sent to users in your pool to deliver various verification codes.

7) The next step is to create a client ID for your app. A client ID is required to allow your app to interact with your user pool. Click Add an app and provide your the name of your app. This can be any string that helps you identify how this client-id is used, e.g., "my social network Android app".

You can also generate a client secret along with the client ID, but it’s not required. If you chose to generate a client secret for your client ID you must use the generated client secret whenever you use the client ID in your apps.

8) The next step allows you to customize workflows through Lambda triggers. For this walkthrough we will not set up customization through Lambda.

9) Review the settings and choose “Create pool” to create your user pool.

You now have a user pool and users can sign up in this pool.

Using Amazon Cognito User Pools in your Android app

Start by creating a CognitoUserPool object. This will be the entry point for all interactions with your Cognito user pool from your app. To create this object, you will need the poolId, clientId and clientSecret (if generated for the clientId).

// setup AWS service configuration. Choosing default configuration
ClientConfiguration clientConfiguration = new ClientConfiguration();	

// Create a CognitoUserPool object to refer to your user pool
CognitoUserPool userPool = new CognitoUserPool(context, poolId, clientId, clientSecret, clientConfiguration);

Using this pool instance, you can sign up new users to the pool. To sign up a user, you will need to provide the following information from the user:

1.     user-id: This will be used by the user to login and must be unique with-in the pool.

2.     password: This will be the users’ password.

3.     User attributes: These are other user details. They must include the required attributes set for your pool. We had earlier set email, given name, and phone number as required attributes.

After you collect these details from your user, you can use the pool instance to sign up the user.

// Create a CognitoUserAttributes object and add user attributes 
CognitoUserAttributes userAttributes = new CognitoUserAttributes(); 

// Add the user attributes. Attributes are added as key-value pairs 
// Adding user's given name. 
// Note that the key is "given_name" which is the OIDC claim for given name 
userAttributes.addAttribute("given_name", userGivenName); 

// Adding user's phone number 
userAttributes.addAttribute("phone_number", phoneNumber); 

// Adding user's email address 
userAttributes.addAttribute("email", emailAddress);

Create a callback handler for sign up. The onSuccess method is called when the sign-up is successful. 

SignUpHandler signupCallback = new SignUpHandler() {

    @Override
    public void onSuccess(CognitoUser cognitoUser, boolean userConfirmed, CognitoUserCodeDeliveryDetails cognitoUserCodeDeliveryDetails) {
        // Sign-up was successful
        
        // Check if this user (cognitoUser) has to be confirmed
        if(!userConfirmed) {
            // This user has to be confirmed and a confirmation code was sent to the user
	    // cognitoUserCodeDeliveryDetails will indicate where the confirmation code was sent
	    // Get the confirmation code from user
        }
	else {
            // The user has already been confirmed
        }
    }
 
    @Override
    public void onFailure(Exception exception) {
        // Sign-up failed, check exception for the cause
    }
};

Call sign-up API 

// Sign up this user
userPool.signUpInBackground(userId, password, userAttributes, null, signupCallback);

Users may have to be confirmed after they sign up, before they can sign in. Users can confirm through email or phone number. After a successful sign-up, if the user has to be confirmed, a confirmation code will be sent to the user's email or phone number. It is also possible to automatically confirm a user after sign-up through Lambda triggers.

If a user provides an email address or phone number during sign-up, and you have selected automatic verification for your user pool, a confirmation code will be sent to the user's phone number as a text message or to the user's email address. The cognitoUserCodeDeliveryDetails object, which was delivered to the callback handler on successful sign-up, will indicate where this confirmation code was sent. You can use this to let the user known where to expect the confirmation code.

Create a callback handler to confirm the user. This callback handler is used by the SDK to  communicate the results of the confirmation API call.

// Call back handler for confirmSignUp API 
GenericHandler confirmationCallback = new GenericHandler() { 
    
    @Override 
    public void onSuccess() { 
        // User was successfully confirmed
    } 

    @Override
    public void onFailure(Exception exception) { 
        // User confirmation failed. Check exception for the cause.
    } 
};

When a new user is confirmed, the user's attribute through which the confirmation code was sent (email or phone number) will be marked as verified. If this attribute is also set to be used as an alias, then the user will be able to sign-in with that attribute (email or phone number) instead of the username.

Alias values must be unique in a pool, and you have choices to resolve the situation if the same email or phone number is already being used as an alias for an existing user in the pool.

You can allow the user confirmation to fail if such conflicts are found, you can do this by setting the forcedAliasCreation parameter to false. The attribute will then remain verified for the existing user, and will continue to be an alias for the existing user. The new user will remain un-confirmed. 

Setting this parameter to true will resolve the conflict by marking the attribute (iemail or phone number) as verified for the new user, and consequently marking it as not-verified for the existing user. This attribute is no longer an alias for the existing user.

// This will cause confirmation to fail if the user attribute has been verified for another user in the same pool  
boolean forcedAliasCreation = false; 

// Call API to confirm this user 
cognitoUser.confirmSignUpInBackground(confirmationCode, forcedAliasCreation, confirmationCallback);

All confirmed users can sign-in. On successful sign-in, access and iID tokens are returned. These tokens are encapsulated in a CognitoUserSession object.

To sign in, start by creating an callback handler for authentication. During the sign-in process the SDK will interact with your app through this callback handler.

// Callback handler for the sign-in process 
AuthenticationHandler authenticationHandler = new AuthenticationHandler() { 
    
    @Override 
    public void onSuccess(CognitoUserSession cognitoUserSession) { 
        // Sign-in was successful, cognitoUserSession will contain tokens for the user   
    }
    
    @Override
    public void getAuthenticationDetails(AuthenticationContinuation authenticationContinuation, String userId) { 
        // The API needs user sign-in credentials to continue
        AuthenticationDetails authenticationDetails = new AuthenticationDetails(userId, password, null);

        // Pass the user sign-in credentials to the continuation
        authenticationContinuation.setAuthenticationDetails(authenticationDetails);

        // Allow the sign-in to continue
        authenticationContinuation.continueTask();
    }

    @Override
    public void getMFACode(MultiFactorAuthenticationContinuation multiFactorAuthenticationContinuation) { 
        // Multi-factor authentication is required, get the verification code from user
        multiFactorAuthenticationContinuation.setMfaCode(mfaVerificationCode);
        // Allow the sign-in process to continue
        multiFactorAuthenticationContinuation.continueTask();
    }

    @Override
    public void onFailure(Exception exception) {
        // Sign-in failed, check exception for the cause
    } 
};

// Sign-in the user 
cognitoUser.getSessionInBackground(authenticationHandler);

After authenticating a user, you can perform other operations on this user, such as getting user details

// Implement callback handler for get details call
GetDetailsHandler getDetailsHandler = new GetDetailsHandler() {
    @Override
    public void onSuccess(CognitoUserDetails cognitoUserDetails) {
        // The user detail are in cognitoUserDetails
    }
        
    @Override
    public void onFailure(Exception exception) { 
        // Fetch user details failed, check exception for the cause
    }
};

// Fetch the user details 
cognitoUser.getDetailsInBackground(getDetailsHandler);

To obtain AWS credentials to access AWS resources for your user, first associate your user pool with an identity pool using the AWS Management Console.  If you haven't already created an identity pool, do that using the Amazon Cognito Console.  In the Authentication providers choose the Cognito tab, and then specify your User Pool ID and App Client ID.

Now add the ID tokens, received after successful authentication, to your credentials provider.

// Get id token from CognitoUserSession.
String idToken = cognitoUserSession.getIdToken().getJWTToken(); 

// Create a credentials provider, or use the existing provider.
CognitoCachingCredentialsProvider credentialsProvider = new CognitoCachingCredentialsProvider(context, IDENTITY_POOL_ID, REGION);

// Set up as a credentials provider.
Map<String, String> logins = new HashMap<String, String>(); 
logins.put("cognito-idp.us-east-1.amazonaws.com/us-east-1_123456678", cognitoUserSession.getIdToken().getJWTToken()); 
credentialsProvider.setLogins(logins);

For further details on integrating Cognito User Pools with Cognito Federated Identity pools see AWS Cognito User Pools documentation.

This was a quick walk-through to get you started with using Amazon Cognito User Pools in your app. You can find further details about the AWS Mobile SDK for Android in the API reference. The sample app is in this GitHub repository.

We would like to hear your comments about this feature. You can reach out to us by posting on the Amazon Cognito forum or the GitHub repository.

Introduction

Today, AWS launched the Beta release of a new Amazon Cognito feature, Cognito Identity User Pools, that makes it easy for you to add sign-up and sign-in functionality to your mobile and web apps. This feature provides a simple way to create and maintain your own user directory that can scale to hundreds of millions of users. You benefit from the security and privacy best practices of AWS, and retain full control of your user data. In this blog post, for brevity, we'll walk through how to get started using iOS.  Similar blogs for Android and JavaScript have also been posted.  For full documentation please see the Amazon Cognito Developer Guide.

Integrating your Cognito Identity user pool into your app

Let's see how to integrate this new Amazon Cognito feature in your app.

Creating your Cognito Identity user pool

From the Amazon Cognito console, click Manage your user pool and Create a user pool to create a new user pool.

1. Provide a name for your pool. You can opt to customize the settings of your user pool by selecting the Step through settings option. However, for this blog post, we will proceed with the default option by clicking Review defaults.
2. Next, you will create an app client. Every app and web app that accesses your pool requires an app client ID and optionally an app client secret. You can create multiple app clients for a user pool, and you typically create one per platform (i.e., iOS, Android, JavaScript). To begin this process, click Apps on the left navigation bar and then click Add an app. Enter your app name. Select Generate client secret and then click Create app and Save changes.
3. Click Review on the left navigation bar and then click Create pool.
4. Note your user pool id from the screen that appears. You can find the app client id and app client secret under Apps on the left navigation bar.

Using your Cognito Identity user pool in your app

The first step for your client app is to create a UserPool object to interact with the service. Using your user pool id, app client id, and app client secret from the console, create an AWSCognitoIdentityUserPool.

   //setup service config
   AWSServiceConfiguration *serviceConfiguration = [[AWSServiceConfiguration alloc] initWithRegion:AWSRegionUSEast1 credentialsProvider:nil];
    
   //create a pool
   AWSCognitoIdentityUserPoolConfiguration *configuration = [[AWSCognitoIdentityUserPoolConfiguration alloc] initWithClientId:@"CLIENT_ID"
                                                                                                              clientSecret:@"CLIENT_SECRET"
                                                                                                                    poolId:@"USER_POOL_ID"];
   [AWSCognitoIdentityUserPool registerCognitoIdentityUserPoolWithConfiguration:serviceConfiguration userPoolConfiguration:configuration forKey:@"UserPool"];
   AWSCognitoIdentityUserPool *pool = [AWSCognitoIdentityUserPool CognitoIdentityUserPoolForKey:@"UserPool"];

Now that you have created a pool, you can start signing up users.  Using a registration UI in your app, collect the necessary information from your user and call signUp:

    NSMutableArray * attributes = [NSMutableArray new];

    //Set user attributes by retrieving them from your UI.  These values are hardcoded for this example
    AWSCognitoIdentityUserAttributeType * phone = [AWSCognitoIdentityUserAttributeType new];

    phone.name = @"phone_number";
    //All phone numbers require +country code as a prefix
    phone.value = @"+15555555555";

    AWSCognitoIdentityUserAttributeType * email = [AWSCognitoIdentityUserAttributeType new];
    email.name = @"email";
    email.value = @"email@mydomain.com";

    [attributes addObject:phone];
    [attributes addObject:email];

    //set username and password by retrieving themy from your UI.  They are hardcoded in this example.
    AWSCognitoIdentityUser *user = [[pool signUp:@"username" password:@"password" userAttributes:attributes validationData:nil] continueWithSuccessBlock:^id _Nullable(AWSTask<AWSCognitoIdentityUser *> * _Nonnull task) {
        NSLog(@"Successfully registered user: %@",task.result.username);
        return nil;
    }];

Next, you will confirm the user unless the user is automatically confirmed (automatically confirming users requires you to implement an AWS Lambda function). Users are confirmed when either their email address or phone number is verified; in this scenario, users receive a verification code at their email address or via SMS on their mobile phone during the registration flow and must input the code to complete sign-up. After obtaining the verification code from your end user, call confirmSignUp:

    //replace VERIFICATION_CODE with the value the user inputs   
    [[user confirmSignUp:@"VERIFICATION_CODE"] continueWithSuccessBlock:^id _Nullable(AWSTask<AWSCognitoIdentityProviderConfirmSignUpResponse *> * _Nonnull task) {
        NSLog(@"Successfully confirmed user: %@",user.username);
        return nil;
    }];

Now that you have a confirmed user, authenticate and get details for the user.  First, implement the AWSCognitoIdentityInteractiveAuthenticationDelegate protocol and set the delegate for the pool.  This protocol is meant to manage your custom login UI and accept username and password information from your end user.  The protocol's methods are only invoked if the user has never authenticated, if the user has signed out, or if the user's refresh token (which is valid for 30 days) has expired:

    //This code goes in your AppDelegate
    pool.delegate = self;

    -(id<AWSCognitoIdentityPasswordAuthentication>) startPasswordAuthentication{
        //implement code to instantiate and display login UI here
	//return something that implements the AWSCognitoIdentityPasswordAuthentication protocol
	return loginUI;
    }

    //This code goes in your Login UI
    -(void) getPasswordAuthenticationDetails: (AWSCognitoIdentityPasswordAuthenticationInput *) authenticationInput passwordAuthenticationCompletionSource: (AWSTaskCompletionSource *) passwordAuthenticationCompletionSource {
        //using inputs from login UI create an AWSCognitoIdentityPasswordAuthenticationDetails object.
        //These values are hardcoded for this example.
        AWSCognitoIdentityPasswordAuthenticationDetails * result = [[AWSCognitoIdentityPasswordAuthenticationDetails alloc] initWithUsername:@"USERNAME" password:@"PASSWORD"];
        //set the result to continue the sign-in process
        passwordAuthenticationDetails.result = result;
    };
    
    -(void) didCompletePasswordAuthenticationStepWithError:(NSError*) error {
    dispatch_async(dispatch_get_main_queue(), ^{
        //present error to end user
        if(error){
            [[[UIAlertView alloc] initWithTitle:error.userInfo[@"__type"]
                                        message:error.userInfo[@"message"]
                                       delegate:nil
                              cancelButtonTitle:nil
                              otherButtonTitles:@"Ok", nil] show];
        }else{
            //dismiss view controller
            [self dismissViewControllerAnimated:YES completion:nil];
        }
    });
}

Now you can get user details using getDetails:

   [[user getDetails] continueWithSuccessBlock:^id _Nullable(AWSTask<AWSCognitoIdentityUserGetDetailsResponse *> * _Nonnull task) {
        AWSCognitoIdentityUserGetDetailsResponse *response = task.result;
        for (AWSCognitoIdentityUserAttributeType *attribute in response.userAttributes) {
            //print the user attributes
            NSLog(@"Attribute: %@ Value: %@", attribute.name, attribute.value);
        }
        return nil;
    }];

If you want to obtain AWS credentials to access AWS resources for your user, first associate your user pool with an identity pool using the AWS Management Console.  If you haven't already created an identity pool, do that using the Amazon Cognito Console.  Under Authentication providers, on the Cognito tab specify your user pool id and client id.

With that association configured, getting AWS credentials in your app is as simple as providing your AWSCognitoIdentityUserPool to your AWSCognitoCredentialsProvider:

AWSCognitoCredentialsProvider *credentialsProvider = [[AWSCognitoCredentialsProvider alloc] initWithRegionType:AWSRegionUSEast1 identityPoolId:@"IDENTITY_POOL_ID" identityProviderManager:pool];
AWSServiceConfiguration *defaultServiceConfiguration = [[AWSServiceConfiguration alloc] initWithRegion:AWSRegionUSEast1
        credentialsProvider:credentialsProvider];
AWSServiceManager.defaultServiceManager.defaultServiceConfiguration = defaultServiceConfiguration;

Sample Application

For a working example demonstrating the functionality described in this blog refer to the Objective-C sample on Github.

Conclusion

That is all there is to it.  Please see the docs to learn about other functionality, like setting user attributes, changing user passwords, and associating MFAs. If you have questions, please reach out to us in the AWS forums.

Predictive maintenance is one of many appealing use cases for the Internet of Things (IoT). Using sensors to predict the health of a fleet of machines in the field can prevent down time without conducting unnecessary maintenance. Further, predictive maintenance allows for maintenance to be conducted at the most cost effective time. Allowing you to shift your operations from being reactive to proactive. Predictive maintenance has applications for the automotive, aerospace, health, and smart city industries, just to name a few. Using the AWS IoT platform and Amazon Machine Learning (AML) allows you to easily connect things to the cloud, and deploy machine learning in real time to leverage predictive maintenance, preventing failures in the field.

In this blog post, we will demonstrate how to train a machine learning model on AML which we will use to generate predictions in near real time from IoT data, and use Amazon CloudWatch alarms to monitor a fleet of devices. For this blog post you will need some familiarity with AWS IoT, AWS Machine Learning, and have the AWS CLI installed. You can find an introduction to AWS IoT in the user documentation, and AML here. If you don't already have the AWS CLI installed, installation directions can be found here. If you already have the CLI client, make sure that it is the updated to most recent version.

Build a model on Amazon Machine Learning

Add Data to S3

To get started, upload some data into S3 to train our model. For this blog post, we have simulated a predictive maintenance data set. You can download the training data here, and you will need the test data that we will be emitting for real time predictions. To create a bucket in S3, and copy the training data into it, use the following commands from the terminal. Be sure to swap out the text between the angle brackets to match your bucket name and path to your training data.

aws s3 mb s3://<bucket-name-you-need-to-choose>
aws s3 cp <path-to-file>/sim_pred_maint_train.csv s3://<bucket-name-you-need-to-choose>/sim_pred_maint_train.csv

The training data is the result of device simulating a thing with 7 sensors. Some of the sensors measurements are correlated to the condition of a wearing part. The target variable for our classification data set is zero, unless the thing is within 30 cycles of failure in which case the target is 1.

Create a Datasource

Now that the data is in S3, we need to use it as a data source for AML. This step can also be performed from the AWS CLI, but for our purposes we will use the web interface instead. From the Machine Learning tab of the AWS console select Datasource from the “create new…” drop down menu, and enter the location of your training data in S3 and give the datasource a name. If a pop-up asks to grant permission for AML to read your S3 bucket, grant permission to create the dataset. After clicking continue, check yes at the top of the page because our csv contains a header row with column names. Next, check the inferred schema and make sure that the variables are of the right type, the options are binary, categorical, numeric, and text. In this example all of the columns are of type numeric except for “target” which should be binary.  Your screen should now look similar to the screenshot shown below.

 After clicking continue, you will be redirected to the target page, where we will select our target for our model, check the box for the column named "target" and choose finish on the next page to accept our data source.

Train a Logistic Regression Model

From the Amazon Machine Learning console select “ML model” from the “create new…” drop down button. After you are redirected, you can choose the datasources you wish to use to train your model. Select the datasource we created earlier, and continue to the next page. Then choose "review and finish" to begin training the model. After you have finished training your model, click "Create Endpoint" under the "Enable Real Time Prediction" section, this step is necessary for us to be able to call the model for real time predictions. The model can take up to 20 minutes to train, so while you're model is training go ahead and continue with the other steps, but when it is done make sure you go back and enable real time predictions.

Sending Test Data Over AWS IoT

Simulate Thing Data

To simulate a thing emitting data to AWS-IoT we will use a Python script which uses the AWS Python SDK (boto3) to emit lines from a test data set. You will need the pandas and NumPy Python packages. If you don't have these packages, you can install them all at once using the following command.

pip install pandas numpy boto3

Now we have a model trained and a simulator to send data to AWS IoT, the next step will be to create an alarm which will listen to messages and generate alerts if AML predicts a thing is in need of maintenance.

Create an Amazon CloudWatch Alarm

To create an alarm based on our predictions, we first need to create an Amazon CloudWatch metric, then attach an alarm to that metric to let us know when a unit is in need of maintenance. Use these commands to create the metric from the AWS CLI and define an alarm.

aws cloudwatch put-metric-data --metric-name maintenance-status --namespace "pm-metrics" --value 0
aws cloudwatch put-metric-alarm --alarm-name maintenanceNotification --metric-name maintenance-status --namespace "pm-metrics" --statistic Maximum --period 60 --evaluation-periods 1  --threshold 1 --comparison-operator GreaterThanOrEqualToThreshold

We chose the metric name of "maintenance-status" and created a namespace called "pm-metrics". The second command creates an alarm named "maintenanceNotification" which is based on the metric we just created. The alarm will go off if at least one of our units is predicted to need maintenance. It is possible to configure the alarm to notify via text or email when status is set to ALARM, however, for this post we will just create the alarm.

Create an Amazon CloudWatch Metrics Rule

Next, we need to create a role with permission to write to Amazon CloudWatch metrics. This role gives the AWS IoT Rules Engine rule permission to forward our AML predictions to Amazon CloudWatch. The first step is to create a role, then a policy which is permitted to write to Amazon CloudWatch metrics, and finally, attach the policy to the role. To create the role we need to create a JSON document named cw_role.json, and save the following text to the document.

{
"Version": "2012-10-17", 
"Statement": [{
      "Sid": "",    
      "Effect": "Allow",     
      "Principal": {
            "Service": "iot.amazonaws.com"
       },
      "Action": "sts:AssumeRole"
  }]
}

After saving cw_role.json, use the AWS CLI to create a role with the following command.

aws iam create-role --role-name cw_role --assume-role-policy-document file://cw_role.json

Next, create the policy that will attach to the new role. Start by saving another JSON document named cw_policy.json ,and copy the following text to the file.

{
  "Version": "2012-10-17",
  "Statement": {
    "Effect": "Allow",
    "Action": ["cloudwatch:PutMetricData","machinelearning:Predict","machinelearning:GetMLModel"],
    "Resource": "*"
  }
}

From the command line, create a policy based on the policy document just created.

aws iam create-policy --policy-name cw_policy --policy-document file://cw_policy.json

When this command is run, the output will include an ARN for the policy which we need for the next CLI command to attach the policy to the role. Remember, replace the X's with your account number.

aws iam attach-role-policy --role-name cw_role --policy-arn arn:aws:iam::XXXXXXXXXXXX:policy/cw_policy

Now that the Amazon CloudWatch metric is set up, and a role is permitted to write to it, we can create a rule using the AWS IoT Rules Engine. To create a rule save the following JSON file as cw_rule.json, and then create the rule using the AWS CLI.

{
  "sql": "SELECT * FROM 'pm/topic'",
  "ruleDisabled": false,
  "actions": [{
      "cloudwatchMetric": {
          "roleArn": "arn:aws:iam::XXXXXXXXXXXX:role/cw_role",
          "metricName": "maintenance-status",
          "metricNamespace": "pm-metrics",
          "metricValue": "${machinelearning_predict('your-model-id-here', 'arn:aws:iam::XXXXXXXXXXXX:role/cw_role', *).predictedLabel}",
          "metricUnit": "None"
      }
  }]
}

The CLI command to create the rule is.

aws iot create-topic-rule --rule-name cwAlarm --topic-rule-payload file://cw_rule.json

Testing The Alarm

Now that everything is in place, we can test our ML model and Amazon CloudWatch metric/alarm by emitting our test data set to AWS IoT using the python script referenced earlier in the post.

python emit_to_AWS_IoT.py sim_pred_maint_emit.csv

Make sure to replace the file path here with the path you put the test data file in on your machine. If you didn't download the test data earlier it can be found here. Within a few minutes of starting the script, you should be able to monitor your CloudWatch metric and see a blue line indicating the status of the metric we have created. When AML predicts a failure, the metric will move from 0 to 1 and the CloudWatch alarm will go off.

You have just completed an end-to-end walkthrough of how to use AML with AWS IoT to perform predictive maintenance on a fleet of devices, and monitor their status using Amazon CouldWatch metrics and alarms.  We hope this walkthrough helps explain predictive maintenance, and provides a good starting point for your specific use cases. If you have questions or comments, don’t hesitate to reach out.