How to Un-mount Android SD Card Before Removing it

Background

Many of us simply remove USB drive from the port without unmounting it which is wrong! But we still do. Perhaps in hurry or we just do not care. And there are a lot of memes around it -

In repeat it is wrong to do so! You might have seem prompt similar to below -

Well it's the OS way of warning you. Same goes for Android as well. After all it uses a Linux kernel.

 Lets understand why we should not be doing that.

Why you should not remove a USB drive without unmounting it

You might have already heard the answer - data loss, data corruption etc. Lets understand why this happens. 

1. Normal case some read/write operation might be in progress and you simply disconnect the drive.  This will corrupt your data.
2. Also most of the operating systems use caching feature which means OS holds on to the operations to be performed on external drive till a minimum threshold of operations is reached post which it execute all those operation. This is basically for performance. When you eject your drive it basically instructs OS to flush the cache and complete any pending operations. Now if you eject the disk and operations are not flushed your data is lost.

So always eject/unmount your drive before removing it. Same for Android as well. You can go to

• Settings -> Storage -> Unmount SD card

Don't just remove it while your phone is on. There might be apps on your SD card, images or many other thing that might be in use. This also means you don't have to turn off your phone to remove SD card. Simply unmount it.  Adding screenshots below for better understanding -

It may take some time to unmount your SD card. Once it is unmounted you can plug it out. Also no restarts needed. Hope this helps!

Difference between 2>&-, 2>/dev/null, |&, &>/dev/null and >/dev/null 2>&1

Background

If you are a regular Linux user then you might already know of this or atleast seen these your shell scripts. For eg.

• ls -a 2>/dev/null

These are called redirection. There are various other functions for redirections and in this post we are going to see and understand them.

File descriptors

In simple words, when you open a file, the operating system creates an entry to represent that file and store the information about that opened file. So if there are 100 files opened in your OS then there will be 100 entries in OS (somewhere in kernel). These entries are represented by integers like (...100, 101, 102....). This entry number is the file descriptor. So it is just an integer number that uniquely represents an opened file in operating system. If your process opens 10 files then your Process table will have 10 entries for file descriptors.

File descriptors  0,1 and 2 are generally given to std input,  std output and std error streams for a process. So you cannot generally use them for anything else. 

At the file descriptor level, 

• stdin is defined to be file descriptor 0, 
• stdout is defined to be file descriptor 1, and 
• stderr is defined to be file descriptor 2.

 NOTE : The FD (File descriptor) table in per process table. So 2 different process can have same FD on calling open(). This is abstraction on user side. On kernel side of things each opened file has a unique FD and two open files cannot have same FD.  Please don't get confused.

Now that we know standard file descriptors created for each table lets see the redirection functions.

Redirection Basics

The > operator redirects the output usually to a file but it can be to a device. You can also use >> to append.

•  > file redirects stdout to file
• 1> file redirects stdout to file
• 2> file redirects stderr to file
• &> file redirects stdout and stderr to file

NOTE : If you don't specify a number then the standard output stream is assumed but you can also redirect errors

NOTE :  As mentioned before you can use >> instead of > to append . 

Finally 

/dev/null is the null device it takes any input you want and throws it away. It can be used to suppress any output. 

Long story short - If you want any stream ignored redirect it to /dev/null

Now that we know the basics lets understand the functions

• 2>/dev/null : This means redirect stderr of the process to /dev/null which essentially mean discard it.
• 2>&- : This will essentially close the output of stderr for a process.
• 2>&1: This combines stdout and stderr into single stream.
• |& : This is just an abbreviation for 2>&1 | (Added in bash4)
• &>/dev/null : This is an abbreviation for > /dev/null 2>&1 which essentially means combine stdout and stderr and redirect it to /dev/null (discard it)
• >/dev/null : This is again an abbreviation for 1 > /dev/null which means discard standard output. Incase you missed my point sometime back if you do not give a FD standard output stream is assumed.

Have written a script that attempts to recover accidentally deleted files that makes use of redirections -

• RecoverFiles.sh(Gist)

 Related Links

• I/O Redirection 
• RecoverFiles.sh(Gist)

How to associate a static IP to your AWS EC2 instance using Elastic IP Addresses

Background

When you start up an EC2 instance by default you get a public IP and a public DNS by which you can reach your server from internet. However when you restart your EC2 instance your IP and your public DNS (which is infact your hostname) gets changed and that may not always be way you desire. So in this post we will see how to associate a static IP to your AWS EC2 instance using Elastic IP Addresses.

Elastic IP Addresses

An Elastic IP address is a static IP address designed for dynamic cloud computing. An Elastic IP address is associated with your AWS account. With an Elastic IP address, you can mask the failure of an instance or software by rapidly remapping the address to another instance in your account. 

An Elastic IP address is a public IP address, which is reachable from the Internet. If your instance does not have a public IP address, you can associate an Elastic IP address with your instance to enable communication with the Internet; for example, to connect to your instance from your local computer.

More details - http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/elastic-ip-addresses-eip.html

How to associate a static IP to your AWS EC2 instance using Elastic IP Addresses

If you have an EC2 instance running you can see the current public IP address and your public DNS entry.

And you can reach your EC2 instance using ec2-54-69-69-192.us-west-2.compute.amazonaws.com. (highlighted in image above)For eg I have deployed a webapp I can access it using URL - 

•  http://ec2-54-69-69-192.us-west-2.compute.amazonaws.com:8080/WebDynamo

Please note these URLs are for demo purpose and may not be live when you are reading this. This is just to give you idea about what changes when you assign an elastic IP address.

As I mentioned earlier this will change when you reboot your EC2 instance. Lets see how we can assign an elastic IP.

Go to Elastic IP under Network & Security and select "Allocate New Address" and confirm.

You can see a new entry in the list with your newly assigned elastic IP. Select the entry , go to actions and select "Associate Address"

You can select either your instance your your network interface. I have selected my running EC2 instance. Then click associate.

NOTE : Careful! Note you will get a new public IP and public DNS (you will loose the previous non static one).

You will see the new IP, public DNS and Elastic Ip in EC2 dashboard.

That's it! You can now access your EC2 instance with this new IP and public DNS.  For eg -

• http://ec2-52-24-69-249.us-west-2.compute.amazonaws.com:8080/WebDynamo

Note how the IP has changed and what was before. And this will remain same even when you reboot!

PS : WebDynamo is a sameple webapp (https://github.com/aniket91/WebDynamo) that I have deployed on tomcat on my EC2 instance for demo purpose. Please refer links in Related Links section below for more details.

Related Links

• Launching and connecting to your new EC2 instance of your Amazon web service(OSFG)
• Setup Tomcat on an Amazon EC2 instance to deploy a wep app (OSFG)

Formatting or preety printing XML and JSON in Java

Background

XML as we know is a platform independent format that can be used to communicate between different apps. We have seen couple of things on XML like parsing, handling special characters, java library etc (See links in Related Links section below). In this post we will see how we can format or preety print xml in Java.

To the code... preety printing XML

Following code will do the trick -

    private static String beautifyXml(String unformattedXML) {
        Transformer transformer;
        String formattedXmlString = null;
        try {
            transformer = TransformerFactory.newInstance().newTransformer();
            transformer.setOutputProperty(OutputKeys.INDENT, "yes");
            transformer.setOutputProperty("{http://xml.apache.org/xslt}indent-amount", "2");
            transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes");
            //initialize StreamResult with File object to save to file
            StreamResult result = new StreamResult(new StringWriter());
            transformer.transform(new StreamSource(new StringReader(unformattedXML)), result);
            formattedXmlString = result.getWriter().toString();
        } catch (TransformerFactoryConfigurationError | TransformerException e) {
            e.printStackTrace();
        }
        return formattedXmlString;
    }

you can test it by simple calling above method with your unformatted xml as input argument. I have created a small web app and have added the functionality of pretty printing xml there. You can see the relevant code (beautifyXml method)-

• https://github.com/aniket91/WebDynamo/blob/master/src/com/osfg/controllers/Beautifier.java

You can clone the webapp itself and see how it works for yourself. It all on github -

• https://github.com/aniket91/WebDynamo

preety printing JSON

You can do similar stuff for Json too -

    public static String beautifyJson(String unformattedStringForm) {
        JsonParser parser = new JsonParser();
        Gson gson = new GsonBuilder().setPrettyPrinting().create();
        
        try {
            String jsonString = unformattedStringForm;
            JsonElement el = parser.parse(jsonString);
            return gson.toJson(el);
        }
        catch(JsonSyntaxException e) {
            e.printStackTrace();
        }
        return null;
    }

Again relevant code can be found in (beautifyJson method) -

• https://github.com/aniket91/WebDynamo/blob/master/src/com/osfg/controllers/Beautifier.java

Related Links

• Escaping special characters of XML in Java (OSFG)
• Parsing XML files in Java using javax.xml.parsers.DocumentBuilder(OSFG) 
• JAXB Tutorial for Java XML binding with annotations (OSFG) 
• Resolving "The markup in the document following the root element must be well-formed" Exception (OSFG)  
• https://github.com/aniket91/WebDynamo

Setup Tomcat on an Amazon EC2 instance to deploy a wep app

Background

Amazon EC2 module is a virtual server in cloud. It's a IaaS (Infrastructure as a service) when you have your own machine (virtual) with an operating system and you can deploy your stuff there. We have already discussed setting up an EC2 instance and connecting to it. If you do not recollect please refer -

• Launching and connecting to your new EC2 instance of your Amazon web service

 In this post we will see how to setup your tomcat and access it remotely. This post assumes you have an EC2 instance up and running and you can SSH to it remotely.

Setup Tomcat on an Amazon EC2 instance to deploy a wep app

Apache Tomcat, often referred to as Tomcat, is an open-source web server developed by the Apache Software Foundation (ASF). Tomcat implements several Java EE specifications including Java Servlet, JavaServer Pages (JSP), Java EL, and WebSocket, and provides a "pure Java" HTTP web server environment in which Java code can run.

There are two ways you can setup tomcat on your machine (EC2 instance) in this case. Either

1. install it through apt-get
2. Download the binary distribution from their official site and use

We are going to take simpler route here and install it via apt-get. First get an update -

• sudo apt-get update

 Next install tomcat7 -

• sudo apt-get install tomcat7

 Post installation you can see the default configuration file located at -

• less /etc/default/tomcat7 

 You can change the configuration parameters here if you wish.

To start,stop and see current tomcat status you can use following commands -

• sudo service tomcat7 start
• sudo service tomcat7 stop
• sudo service tomcat7 status

 Next you can deploy your webapps at /var/lib/tomcat7/webapps

• CATALINA_HOME : /usr/share/tomcat7
• CATALINA_BASE : /var/lib/tomcat7

 Now your tomcat installation is done. Tomcat by default runs on port 8080 that you need to open externally (for inbound traffic). Go to Security Groups and add a custom TCP rule to open port 8080 externally -

 Add the rule and save it. You should not be able to access your tomcat. If it shows you Its up! page you are all set!

I have also deployed a web app. If you have a web app just export it as war and put it in webapps directory

• /var/lib/tomcat7/webapps 

And restart tomcat. Tomcat will automatically extract folder out of it and deploy. If you do not have a webapp and want to test you can checkout -

• https://github.com/aniket91/WebDynamo

Import it in eclipse and export a war out of it. If you want to skip all that you can directly download the war from -

• https://dl.dropboxusercontent.com/u/30440938/WebDynamo.war

and put it in webapps folder. Note this war is compiled with java 8. So make sure you are using Java 8 for your tomcat. You can set it in file -

• /etc/default/tomcat7

Eg. JAVA_HOME=/usr/lib/jvm/java-8-oracle

Once deployed you can hit following URL and see if it works. If it does not you need to check catalina.out logs for any deployment issues.

• http://ec2-52-24-69-249.us-west-2.compute.amazonaws.com:8080/WebDynamo/getEmployeeInfoForm.htm

You should see a page something like above.

Related Links

•  Launching and connecting to your new EC2 instance of your Amazon web service(OSFG) 
• Spring MVC Hello Wold Example(OSFG)