How to fix "Unable to find a region via the region provider chain" exception with AWS SDK

Background

Recently I was working on a task that needed to upload and download a test file on AWS S3 bucket. For this, I used AWS Java SDK. The functionality seemed to work fine until the code was deployed in production. In this post, I will try to explain why does this exception occur and how we can fix this.

Code that I had used for the test upload was as follows -

  try {
   BasicAWSCredentials awsCreds = new BasicAWSCredentials("YourAccessKeyId", "YourSecretKey");
   AmazonS3 s3client = AmazonS3ClientBuilder.standard()
                    .withCredentials(new AWSStaticCredentialsProvider(awsCreds))
                    .build();
   s3client.putObject(storage.getBucketName(), "test.txt", "Done!");

  } catch (AmazonServiceException ase) {
   logger.error(
     "Caught an AmazonServiceException, which means your request made it to Amazon S3, but was rejected with an error response for some reason. storage : {}",
     storage, ase);
   logger.error("Error Message:    {}", ase.getMessage());
   logger.error("HTTP Status Code: {}", ase.getStatusCode());
   logger.error("AWS Error Code:   {}", ase.getErrorCode());
   logger.error("Error Type:       {}", ase.getErrorType());
   logger.error("Request ID:       {}", ase.getRequestId());
  } catch (AmazonClientException ace) {
   logger.error(
     "Caught an AmazonClientException, which means the client encountered an internal error while trying to communicate with S3, such as not being able to access the network storage : {}",
     storage, ace);
   logger.error("Error Message: {}", ace.getMessage());
  } catch (Exception ex) {
   logger.error("Got exception while testing upload to S3", ex);
  }

I had tested this code by deploying it on one of our EC2 instances and it worked fine. I will probably explain this at a later point as to why it worked with EC2 instance but there is a major flaw with the above code that finally ends up throwing an exception - "Unable to find a region via the region provider chain"

Relevant Stacktrace:

com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:352) 
at com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:386) 
com.amazonaws.SdkClientException: Unable to find a region via the region provider chain.
Must provide an explicit region in the builder or setup environment to supply a region.

The problem

The problem with the above piece of code is that we are not supplying the AmazonS3ClientBuilder with the region. Even though S3 is a global service, the bucket that you create are region specific. Each region would have its own endpoint and hence AWS S3 SDK need to know which region to know the endpoint it needs to make the API call to.

The Solution

The simplest solution is to pass the region to the AmazonS3ClientBuilder builder explicitly as follows -

AmazonS3 s3client = AmazonS3ClientBuilder.standard()
        .withCredentials(new AWSStaticCredentialsProvider(awsCreds))
        .withRegion(Regions.US_EAST_1)
        .build();

NOTE: After you build a client with the builder, it's immutable and the region cannot be changed. If you are working with multiple AWS Regions for the same service, you should create multiple clients—one per region.

Understanding the solution

The problem and the solution might appear simple but there are various aspects you need to understand about setting region in your S3 builder.

There are various ways your AWS SDK can infer the region to use instead if you explicitly providing it yourself.

NOTE: You must use client builders to have the SDK automatically detect the region your code is running in. This is not applicable if you are using client constructor and default region from the SDK will be used.

If you don't explicitly set a region using the withRegion methods, the SDK consults the default region provider chain to try and determine the region to use.

1. Any explicit region set by using withRegion or setRegion on the builder itself takes precedence over anything else.
2. The AWS_REGION environment variable is checked. If it's set, that region is used to configure the client.
1. NOTE: This environment variable is set by the Lambda container.
3. The SDK checks the AWS shared configuration file (usually located at ~/.aws/config). If the region property is present, the SDK uses it.
1. The AWS_CONFIG_FILE environment variable can be used to customize the location of the shared config file.
2. The AWS_PROFILE environment variable or the aws.profile system property can be used to customize the profile that is loaded by the SDK.
4. The SDK attempts to use the Amazon EC2 instance metadata service to determine the region of the currently running Amazon EC2 instance.

If the SDK still hasn't found a region by this point, client creation fails with an exception. And we saw what exception it throws :) It is is the reason I am writing this post.  Also, you must have realized why it worked for me with the EC2 instance. As per point number 4, SDK attempts to use the Amazon EC2 instance metadata service to determine the region. Hope this helps!

Related Links

• AWS Region Selection
• AWS Java SDK - Unable to find a region via the region provider chain(SO)

Android material design colors

Background

When you creating an Android app from Android studio, studio pre-creates certain files for you. This includes the default style for the application. It looks something like below -

<resources>
    <!-- Base application theme. -->
    <style name="AppTheme" parent="Theme.AppCompat.Light.DarkActionBar">
        <!-- Customize your theme here. -->
        <item name="colorPrimary">@color/colorPrimary</item>
        <item name="colorPrimaryDark">@color/colorPrimaryDark</item>
        <item name="colorAccent">@color/colorAccent</item>
    </style>
</resources>

Now the question is what is this 

• colorPrimary,
• colorPrimaryDark and
• colorAccent

These are all based on material design guidelines. For me, colors are as follows -

<?xml version="1.0" encoding="utf-8"?>
<resources>
    <color name="colorPrimary">#3F51B5</color>
    <color name="colorPrimaryDark">#303F9F</color>
    <color name="colorAccent">#FF4081</color>
</resources>

But you can really choose any color theme you want. For more details, you can see google documentation -

• https://material.io/design/color/#

In this post, I will list what each color naming means.

Android material design colors

Following are naming conventions used -

• colorPrimary:  The color of the app bar.
• colorPrimaryDark: The color of the status bar and contextual app bars; this is normally a dark version of colorPrimary.
• colorAccent: The color of UI controls such as checkboxes, radio buttons, and edit text boxes.
• windowBackground: The color of the screen background.
• textColorPrimary: The color of UI text in the app bar.
• statusBarColor: The color of the status bar.
• navigationBarColor: The color of the navigation bar.

To visualize refer to the following picture -

You can create and preview your colors here -> https://www.materialpalette.com/

Related Links

• https://material.io/design/color/#
• https://www.materialpalette.com/
• Material Design for Your Xamarin.Forms Android Apps

Understanding JSON Web Tokens (JWT)

Background

JSON Web Token (JWT) is an open standard ( RFC 7519) that defines a compact and self-contained way for securely transmitting information between parties as a JSON object.

What this essentially means is that JWT helps you secure your application by allowing to securely share claims such as user data between your application and the client that is communicating it. In this post, I will explain fundamentals of JWT and how it works.

When you would use JWT?

As mentioned before JWT can be used to secure communication between two parties that exchange data. Following are some common use cases -

1. Authorization:  This is a very common use case of JWT. When users log into an application, they are presented with a JWT token. Each of the subsequent requests that user makes to this application must include this JWT token. Your application would validate each of the requests to see if it has valid JWT before processing it.
2. Information exchange:  You can also transfer information in JWT in a secure manner. This means JWT can have data as part of it. Note that this information sent is visible to anyone having access to that token. However, no one can alter this information since the token signature is calculated based on the information it holds and altering any data will change the signature which would invalidate the token. We will see this in detail in some time. 

What does JWT look like?

Now that we know when can we use a JWT, let's see how does it look like and what does it comprise of.

A JWT typically looks like -

• xxxxx.yyyyy.zzzzz

As you can see it consists of 3 parts separated by a dot. These three parts are -

1. Header
2. Payload
3. Signature 

Let's see what each of these sections represent.

Header: Header is a JSON that mainly consists of two parts - the type of token which is JWT and the algorithm used for hashing. For example,

{
  "alg": "HS256",
  "typ": "JWT"
}

The header section of JWT is Base64 URL encode of above JSON.

Payload: This is the 2nd part of the JWT. This section contains the claims or the information that needs to be shared. This is again in JSON format. These claims can be of 3 types -

1. Registered claims: These are predefined claims which are optional but useful to convey proper information. Example can be iss(issuer), exp(expiration time), sub(subject) etc. You can see all the claims here.
2. Public claims:  These can be defined by anyone using JWT but should be defined in  IANA JSON Web Token Registry to avoid a collision.
3. Private claims: These are custom claims that can be added to share information between two parties as long as both agree on using them. These are neither registered or private claims.

Example -

{
  "sub": "Welcome to my blog!",
  "name": "Aniket Thakur",
  "iss": "opensourceforgeeks.blogspot.com",
  "exp": 1530436998
}

The payload section of JWT is Base64 URL encode of above JSON.

Signature: This is the 3rd and last part of JWT. This represents the signature of the token. This is created as follows -

Signature = Hash (base64UrlEncode(header) + "." + base64UrlEncode(payload), secret)

As you can see signature comprises of the header and well as payload. So if anyone changes header or payload then the signature would not match and hence JWT validation would fail. This is exactly why I mentioned before that JWT is secure from tampering. Also, notice that a secret key is used to compute the hash value. This secret will never be available to anyone (just the application generating the JWT to protect its resources). And finally, the hashing method would depend on the algorithm you are using. In above example we used - HS256 (HMAC with SHA-256).


So, assuming the secret key used is "secret", for the example we have used above JWT  would look like -

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJXZWxjb21lIHRvIG15IGJsb2chIiwibmFtZSI6IkFuaWtldCBUaGFrdXIiLCJpc3MiOiJvcGVuc291cmNlZm9yZ2Vla3MuYmxvZ3Nwb3QuY29tIiwiZXhwIjoxNTMwNDM2OTk4fQ.8pmcCeFS9gx8Yb-DPRkAihhW7mUxAZkklYHHme5a0tU

where

header : eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9

payload : eyJzdWIiOiJXZWxjb21lIHRvIG15IGJsb2chIiwibmFtZSI6IkFuaWtldCBUaGFrdXIiLCJpc3MiOiJvcGVuc291cmNlZm9yZ2Vla3MuYmxvZ3Nwb3QuY29tIiwiZXhwIjoxNTMwNDM2OTk4fQ

Signature : 8pmcCeFS9gx8Yb-DPRkAihhW7mUxAZkklYHHme5a0tU

You can also see this JWT in action - https://jwt.io/.

NOTE1: Notice for registered claims all claim names are 3 characters in length. This is because JWT is meant to be compact.

NOTE2: Also you must have realized by now that header and payload are just base64 URL encoded values and that anyone with access to token can decode and read these. So you must never add any sensitive information to these sections. JWT just ensures that the data cannot tamper but the data is visible to everyone with the access to the token.

How does JWT work?

Now that we saw fundamentals of JWT let's see how JWT is actually used to protect resources in an application.

For simplicity let's assume we have our own server application which has an authentication module and a set of APIs that must be protected. First user or the client would authenticate itself against the server. The server will send the JWT in the response of successful authentication call. This token will be used by the client in all subsequent API calls made to the server. You can send this token in the headers of the API call. Now in each API call server will check if the JWT is valid and process the request based on the validity of the token.

We already saw how JWT tokens are created. Now let's see how they are verified. Once the server receives JWT token it will extract payload and header section. Then it will use these parts along with the secret that is stored only on the server to calculate the signature. If the signature calculated is same as the one that is received from the JWT then the JWT is valid and the further processing can happen. If the signature does not match then it means someone has tampered with the request and it must not be allowed to execute. You can send a 401 - unauthenticated response in such cases.

Ideally, you would have a separate API server and a separate authentication server in which case you can keep the secret is a centralize place like a database and read it from both the servers to generate signatures of JWT.

Flow is something like below -

Summary

• JWT can be used to securely share claims between two parties. This can either be used for protecting your resources on the server or sharing data securely.
• JWT protect the data from being tampered but it does not prevent anyone from viewing that data (Since the data is just base64 encoded string). So never send sensitive data as part of JWT.
• Always add an expiry to the JWT. So if your token is compromized for some reason it would not be valid post its expiry. The client can reach out to the server for the new token in this case. 
• Always use https for sending JWT since it prevents unauthorized users from stealing the JWT.
• You can use https://jwt.io/ to generate and see JWT yourself.

Related Links

• Introduction to JSON Web Tokens
• Registered claims
• learn-json-web-tokens(Github)