Tuesday, 12 June 2018

AWS service limits asked in "AWS Certified Solutions Architect - Associate" and "AWS Certified Developer - Associate" certifications

Background

I just cleared my "AWS Certified Developer - Associate" certification exam yesterday with 90%. I have already cleared "AWS Certified Solutions Architect - Associate" exam 6 months back with 89%. You can see my badges below-
While preparing I realized that there are some questions based on service limits in AWS. These can be straightforward questions or they can be slightly twisted. Either case knowing service limits help out a lot. So I am going to summarize most of them which I feel important from certification perspective.




NOTE: AWS service limits can change anytime. So it is best to refer the FAQ sections of corresponding services to confirm. Following limits are as of June 2018.

AWS service limits & constraints

Following are AWS services and their corresponding limits. There would be more limits and constraints to each service. I am simply trying to summarise based on my exam preparation, test quizzes, and actual exam experience. Please let me know in comments if these limits are changed and I can update accordingly. Thanks.

Consolidated billing


AWS S3

  • By default, customers can provision up to 100 buckets per AWS account. However, you can increase your Amazon S3 bucket limit by visiting AWS Service Limits.
  • The bucket name can be between 3 and 63 characters long and can contain only lower-case characters, numbers, periods, and dashes.
  • Bucket names must not be formatted as an IP address (for example, 192.168.5.4).
  • For more details refer - https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html
  • AWS S3 offers unlimited storage
  • Each object on S3, however, can be 0 bytes to 5TB.
  • The largest object that can be uploaded in a single PUT is 5GB
  • For objects larger than 100 megabytes, customers should consider using the Multipart Upload capability.
  • For further details refer - https://aws.amazon.com/s3/faqs/

Glacier

  • There is no maximum limit to the total amount of data that can be stored in Amazon Glacier. 
  • Individual archives are limited to a maximum size of 40 terabytes.
  • For more details refer - https://aws.amazon.com/glacier/faqs/

Redshift


AWS EC2

VPC

Route 53



Cloud watch

Cloud formation

Lambda

Dynamo DB

  • There is an initial limit of 256 tables per region. You can raise a request to increase this limit.
  • You can define a maximum of 5 local secondary indexes and 5 global secondary indexes per table(hard limit) - total 10 secondary indexes
  • The maximum size of item collection is 10GB
  • The minimum amount of reserved capacity that can be bought - 100
  • The maximum item size in DynamoDB is 400 KB, which includes both attribute name binary length (UTF-8 length) and attribute value lengths (again binary length). The attribute name counts towards the size limit. No limit on the number of items.
  • For more details refer - https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html
  • A BatchGetItem single operation can retrieve up to 16 MB of data, which can contain as many as 100 items
  • For more details refer - https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_BatchGetItem.html
  • A single Scan operation will read up to the maximum number of items set (if using the Limit parameter) or a maximum of 1 MB of data and then apply any filtering to the results using FilterExpression.
  • For more details refer - https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Scan.html

SQS

  • You can create any number of message queues.
  • Max configuration: 14 days retention and 12 hours visibility timeout
  • Default configuration: 4 days retention  and 30 seconds visibility timeout
  • A single request can have up to 1 to 10 messages up to a maximum payload of 256KB.
  • Each 64 kb chunk payload is billed as 1 request. So a single API call with 256kb payload will be billed as 4 requests.
  • To configure the maximum message size, use the console or the SetQueueAttributes method to set the MaximumMessageSize attribute. This attribute specifies the limit on bytes that an Amazon SQS message can contain. Set this limit to a value between 1,024 bytes (1 KB), and 262,144 bytes (256 KB).
  • For more details refer - https://aws.amazon.com/sqs/faqs/

SNS

  • By default, SNS offers 10 million subscriptions per topic and 100,000 topics per account.  To request a higher limit, please contact Support.
  • Topic names are limited to 256 characters.
  • SNS subscription confirmation time period is 3 days

SWF



Again as mentioned before this is obviously not an exhaustive list but merely a summary of what I thought could be best to revise before going to the associate exams. Let me know if you think something else needs to be added here for the benefit of everyone.


Since you have taken time to go through the limits here is a bonus question for you :)

Question: You receive a call from a potential client who explains that one of the many services they offer is a website running on a t2.micro EC2 instance where users can submit requests for customized e-cards to be sent to their friends and family. The e-card website administrator was on a cruise and was shocked when he returned to the office in mid-January to find hundreds of angry emails complaining that customers' loved ones had not received their Christmas cards. He also had several emails from CloudWatch alerting him that the SQS queue for the e-card application had grown to over 500 messages on December 25th. You investigate and find that the problem was caused by a crashed EC2 instance which serves as an application server. What do you advise your client to do first? Choose the correct answer from the options below

Options:
  1. Use an autoscaling group to create as many application servers as needed to access all of the Christmas card SQS messages.
  2. Reboot the application server immediately so that it begins processing the Christmas cards SQS messages.
  3. Redeploy the application server as larger instance type so that it processed the  Christmas cards SQS faster.
  4. Send an apology to the customer notifying them that their cards will not be delivered.

Answer:
4. Send an apology to the customer notifying them that their cards will not be delivered.

Explanation:
Since 500 message count was as of December 25th and e-card website administrator returned mid-Jan the difference is more than 14 days which is the maximum retention period for SQS messages.

To be honest I had select option 1 in my 1st attempt :)


Friday, 25 May 2018

Understanding difference between Cognito User Pool vs Identity Pool

Background

In one of the previous post, we saw how to setup Cognito Identity Pool for unauthenticated or authenticated access to AWS resources like S3. Cognito identity pool is used to federate users into AWS so that they can call AWS services. In this post, we are going to see what is the difference between Cognito user wool and identity pool.


Amazon Cognito User Pools

As per AWS documentation,

A user pool is a user directory in Amazon Cognito. With a user pool, your users can sign in to your web or mobile app through Amazon Cognito. Your users can also sign in through social identity providers like Facebook or Amazon, and through SAML identity providers. Whether your users sign-in directly or through a third party, all members of the user pool have a directory profile that you can access through an SDK.

User pools provide:


  • Sign-up and sign-in services.
  • A built-in, customizable web UI to sign in users.
  • Social sign-in with Facebook, Google, and Login with Amazon, as well as sign-in with SAML identity providers from your user pool.
  • User directory management and user profiles.
  • Security features such as multi-factor authentication (MFA), checks for compromised credentials, account takeover protection, and phone and email verification.
  • Customized workflows and user migration through AWS Lambda triggers.


Sourcehttps://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-identity-pools.html

After successfully authenticating a user, Amazon Cognito issues JSON web tokens (JWT) that you can use to secure and authorize access to your own APIs, or exchange for AWS credentials.

Amazon Cognito Identity Pools (Federated Identities)


As per AWS documentation,

Amazon Cognito identity pools (federated identities) enable you to create unique identities for your users and federate them with identity providers. With an identity pool, you can obtain temporary, limited-privilege AWS credentials to access other AWS services.

Sourcehttps://docs.aws.amazon.com/cognito/latest/developerguide/cognito-identity.html

Understanding difference between Cognito User Pool vs Identity Pool

Above definitions can be very confusing, so let me simplify them for you.

Cognito user pool is nothing but your user management system backed by its own user directory. If you are building a new app or a website and you want to add authentication mechanism to sign in or sign up for your users you should use Cognito user pool. You can choose to have users sign in with an email address, phone number, username or preferred username plus their password. You can also use social identity providers for authentication and signing in or up for your app or website. Everything is under Cognito user pool umbrella. You can use their SDK provided to do this. Cognito user pool helps you maintain your user base details and their authentication. On successful authentication, it provides a JWT token that can be used to authenticate your custom server APIs as well.



Cognito identity pool is used when you need access to AWS services. It basically authenticates the user and if authentication is successful it will give you a temporary token that you can use to talk to AWS. For eg. let's say you want to upload a file to S3 then you can use this. I had written a post earlier to do just the same -
Now how do you authenticate users to get these temporary credentials is something you decide. You could authenticate against the Cognito user group you have already created or you can again use 3rd party authentication providers like -
  • Amazon
  • Google
  • Facebook etc
Authentication flow is as -



So to summarize if you want to build a user directory with sign-in / sign-up functionality Cognito user pool is the way to go and if you just want access to AWS services without worrying about maintaining user database of your own you can use Cognito identity pool.




Related Links

Thursday, 24 May 2018

How to check an active internet connection on iPhone/iOS

Background

There are various network calls that we may have to make from your device to server. Internet network connection may not always be available for review. So it is better if we can check if the network is reachable before making the API calls. In this post, we will see the same in Objective C.


How to check an active internet connection on iPhone/iOS

  • First, add SystemConfiguration framework to your project.
  • To do this double click on your project. This should open your project settings. Select your app target.



  • Now go to "Build Phases" Tab on the top
  • Now Expand "Link Binary With Libraries"
  • Now click on '+' icon and search for "SystemConfiguration"
  • You should see "SystemConfiguration.framework"
  • Select it and click Add
  • You should see it would be added under "Link Binary With Libraries"


  • You should also start seeing "Framework" group under project navigator -

  • Now add Tony Million's version of Reachability.h and Reachability.m to the project. You can find it here - https://github.com/tonymillion/Reachability
  • I have just created a new group called Reachability and added these file in it.


  • In the file where you want to make this change import Reachability 
    • #import "Reachability.h"
  • Now you can check if the internet is available with following code snippet -


    if ([[Reachability reachabilityForInternetConnection]currentReachabilityStatus]==NotReachable)
    {
        //connection unavailable
        NSLog(@"Connection is not available");
    }
    else
    {
        //connection available
         NSLog(@"Connection is available");
    }



NOTE#import <SystemConfiguration/SystemConfiguration.h> is inside Reachability.h. You do not have to explicitly import it anywhere. Just add it to the framework.




Related Links


Tuesday, 22 May 2018

How to upload files to S3 from iOS app written in Objective C using AWS Cognito identity pool

Background

This post assumes you have setup Cognito identity pool as explained in the previous post -
If not, then please refer the previous post and set that up. Before starting with using this you should have -
  1. Identity pool ID
  2. AWS region where pool and S3 bucket reside
  3. S3 bucket name
We will use above in configuration and implementation that follows.

How to upload files to S3 from iOS app written in Objective C using AWS Cognito identity pool

  • First, go to your pod file and update following dependencies -

# Uncomment the next line to define a global platform for your project
# platform :ios, '9.0'


target 'TestApp' do
  # Uncomment the next line if you're using Swift or would like to use dynamic frameworks
  # use_frameworks!
  # Pods for TestApp


pod 'AWSMobileClient', '~> 2.6.18'  # For AWSMobileClient
pod 'AWSS3', '~> 2.6.18'            # For file transfers
pod 'AWSCognito', '~> 2.6.18'       # For data sync

end

  • Next run "Run pod install --repo-update" from the command line
  • Once you have the dependencies installed we can now write Objective C code to upload a file to S3.


//
//  FileUploader.m
//  TestApp
//
//  Created by Aniket on 22/05/18.
//


#import <Foundation/Foundation.h>


#import "FileUploader.h"
#import <AWSS3/AWSS3.h>
#import <AWSCore/AWSCore.h>




@implementation FileUploader


static AWSS3TransferManager *transferManager;


+ (void) initialize {
    
    if (self == [FileUploader class]) {
        AWSCognitoCredentialsProvider *credentialsProvider = [[AWSCognitoCredentialsProvider alloc] initWithRegionType:AWSRegionUSEast1 identityPoolId:@"us-east-1:f847843f-0162-43c2-b73f-efdc7c69cce2"];
        AWSServiceConfiguration *configuration = [[AWSServiceConfiguration alloc] initWithRegion:AWSRegionUSEast1 credentialsProvider:credentialsProvider];
        [AWSS3TransferManager registerS3TransferManagerWithConfiguration:[[AWSServiceConfiguration alloc] initWithRegion:AWSRegionUSEast1 credentialsProvider:credentialsProvider] forKey:s3TransferManagerKey];


        AWSServiceManager.defaultServiceManager.defaultServiceConfiguration = configuration;
                                        
        transferManager = [AWSS3TransferManager S3TransferManagerForKey:s3TransferManagerKey];
    }
    
}


+ (void)uploadFile
{
    
    
    NSURL *uploadingFileURL = [NSURL fileURLWithPath: @"PATH_TO_FILE";
    AWSS3TransferManagerUploadRequest *uploadRequest = [AWSS3TransferManagerUploadRequest new];
    
    
    uploadRequest.bucket = s3Bucket;
    int timestamp = [[NSDate date] timeIntervalSince1970];
    uploadRequest.key = [NSString stringWithFormat:@"%@-%d%@",@"testfile",timestamp,@".txt"];
    uploadRequest.body = uploadingFileURL;
    
    [[transferManager upload:uploadRequest] continueWithExecutor:[AWSExecutor mainThreadExecutor]
                                                       withBlock:^id(AWSTask *task) {
                                                           if (task.error) {
                                                               if ([task.error.domain isEqualToString:AWSS3TransferManagerErrorDomain]) {
                                                                   switch (task.error.code) {
                                                                       case AWSS3TransferManagerErrorCancelled:
                                                                       case AWSS3TransferManagerErrorPaused:
                                                                           break;
                                                                           
                                                                       default:
                                                                           NSLog(@"Error uploading file to S3: %@", task.error);
                                                                           break;
                                                                   }
                                                               } else {
                                                                   // Unknown error.
                                                                   NSLog(@"Error uploading file to S3: %@", task.error);
                                                               }
                                                           }
                                                           
                                                           if (task.result) {
                                                               AWSS3TransferManagerUploadOutput *uploadOutput = task.result;
                                                               // The file uploaded successfully.
                                                               NSLog(@"uploading file to S3 was successful: %@", uploadOutput);
                                                           }
                                                           return nil;
                                                       }];
    
}
@end


In above code replace the identity pool id and region as per your configuration settings. This is Objective C code, if you want to see Swift or Android - Java please refer https://docs.aws.amazon.com/aws-mobile/latest/developerguide/how-to-integrate-an-existing-bucket.html


Related Links

How to setup Cognito Identity Pool for unauthenticated or authenticated access to AWS resources like S3

Background

Many time when you are creating application be it mobile or web you have to provide an authentication mechanism for users to sign in before they can use your apps or websites features. AWS Cognito service does the same thing. 
  • Amazon Cognito makes it easy for you to have users sign up and sign in to your apps, federate identities from social identity providers, secure access to AWS resources and synchronize data across multiple devices, platforms, and applications.
You can do 3 things using this service -
  1. With Cognito Your User Pools, you can easily and securely add sign-up and sign-in functionality to your mobile and web apps with a fully-managed service that scales to support hundreds of millions of users.
  2. With Cognito Federated Identities, your users can sign-in through social identity providers such as Facebook and Twitter, or through your own identity solution, and you can control access to AWS resources from your app.        
  3. With Cognito Sync, your app can save user data, such as preferences and game state, and sync that data to make your users' experiences consistent across their devices and when they are disconnected.             
NOTE: AWS Cognito is a region-specific service so you have to select a region and make sure this service is available in that region. For this post, I am going to use us-east-1(N.Virginia).

Also in this post, I am going to show how we can create a federated identity pool which can be used to authenticate and access AWS resources. Though I am going to limit this post to setting-up the identity pool and corresponding AWS changes we will see how we can use this to upload sample files to Amazon S3 bucket.


How to setup Cognito Identity Pool for unauthenticated or authenticated access to AWS resources

  • Sign into AWS console and go to Cognito service.


  • Now click on "Manage Federated Identities"
  • Provide a name for your identity pool. Eg. athakur_test
  • If you want unauthenticated users to use this pool for authenticating and accessing AWS resources select "". I am going to select this since finally, I want to use this pool to upload files to S3. 

NOTE: Amazon Cognito can support unauthenticated identities by providing a unique identifier and AWS credentials for users who do not authenticate with an identity provider. If your application allows customers to use the application without logging in, you can enable access to unauthenticated identities.

  • If you want to provide access to authenticated users only then use one of the Authentication providers like Facebook, Amazon, Google etc
  • Click on "create pool".

  • Next screen you should see the roles that AWS will create for you. You can select "View details" to see the role details. 
  • Finally, click "Allow"
NOTE1: Assigning a role to your application end users helps you restrict access to your AWS resources. Amazon Cognito integrates with Identity and Access Management (IAM) and lets you select specific roles for both your authenticated and unauthenticated identities. 

NOTE2: By default, Amazon Cognito creates a new role with limited permissions - end users only have access to Cognito Sync and Mobile Analytics. You can modify the roles if your application needs access to other AWS resources, such as S3 or DynamoDB.

You can see click on video details to see this default policy document -

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "mobileanalytics:PutEvents",
        "cognito-sync:*"
      ],
      "Resource": [
        "*"
      ]
    }
  ]

}


We will see how to attach additional permissions to this role later.



As you can see 2 roles are created - one for the authenticated user and other for the unauthenticated user -
  1. Cognito_athakur_testAuth_Role
  2. Cognito_athakur_testUnauth_Role
On the next page, you should see some sample code - one for getting credentials and other for Cognito sync -


Get AWS credentials :

// Initialize the Amazon Cognito credentials provider
CognitoCachingCredentialsProvider credentialsProvider = new CognitoCachingCredentialsProvider(
    getApplicationContext(),
    "us-east-1:f847843f-0162-43c2-b73f-efdc7c69cce2", // Identity pool ID
    Regions.US_EAST_1 // Region
);


Store User Data:

// Initialize the Cognito Sync client
CognitoSyncManager syncClient = new CognitoSyncManager(
   getApplicationContext(),
   Regions.US_EAST_1, // Region
   credentialsProvider);

// Create a record in a dataset and synchronize with the server
Dataset dataset = syncClient.openOrCreateDataset("myDataset");
dataset.put("myKey", "myValue");
dataset.synchronize(new DefaultSyncCallback() {
    @Override
    public void onSuccess(Dataset dataset, List newRecords) {
  //Your handler code here
    }
});


  • Note down the Identity pool id and region . In my case identity pool id is "us-east-1:f847843f-0162-43c2-b73f-efdc7c69cce2" and region is us-east-1. We will need it in subsequent setup.
The last part that is remaining now is to add additional permissions to our new role so that Cognito pool users can access other required AWS services - in this case since we need to upload files to S3 we just need S3 put access.

NOTE: Make sure Cognito identity pool should be created in same region as S3 bucket.

So go to IAM and go to roles and select it. Now click add inline policy and add following JSON policy -

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1526901559336",
            "Action": [
                "s3:PutObject"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:s3:::<BUCKET_NAME>/*"
        }
    ]
}


You have to replace <BUCKET_NAME> with your actual S3 bucket name. Always make IAM policies as stricter as possible. In this case, we are just giving put object permission on the desired S3 bucket only.

NOTE: This was for upload to S3 bucket but you can provide access to other services that you desire. If you need to create the policy you can use policy generator AWS provides - https://awspolicygen.s3.amazonaws.com/policygen.html


At this point, your Cognito service is all set up for implementation. In the next post, we will see how we can use this to implement it in our mobile or web application.

Related Links

t> UA-39527780-1 back to top