Open Source For Geeks: Ubuntu

Showing posts with label Ubuntu. Show all posts

Saturday, 2 May 2020

How to Install IPython Jupyter Notebook on Ubuntu

Background

Jupyter Notebook is an open-source and interactive web app that allows you to create and share documents that contain live code, equations, visualizations, and narrative text. We will use this to run python code but it can be used with other languages as well.

Prerequisites

Firstly make sure you have the following applications installed before installing Jupyter notebook:

python3
python3-pip

You can install these from default Ubuntu repositories by running the following commands:

sudo apt-get install python3
sudo apt-get install python3-pip

You can check that the versions are correct with -V option as shown in the screenshot below:

How to Install IPython Jupyter Notebook on Ubuntu

Now that dependencies are in place, let's install Jupyter notebook.

Install python and jupyter

pip3 install ipython
pip3 install jupyter

IPython (Interactive Python) is a command shell for interactive computing in multiple programming languages, originally developed for the Python programming language, that offers introspection, rich media, shell syntax, tab completion, and history (Wiki).

You can then start the jupyter notebook with the following command:

jupyter notebook

It should automatically open a browser window for you, if not you can fo to the URL from the command output, in my case:

http://127.0.0.1:8888/?token=f279cc86b6219e3312d623377a247ed4a686e140fac30153

Then you can create a new notebook with python3 kernel and write your code there.

Let me know in the comments if you face any issues.

We will do some more fun stuff and learn more about python. So stay tuned!

Friday, 1 May 2020

How to fix Ubuntu update error “waiting for unattended-upgr to exit”

Background

So I logged into my Ubuntu machine after a long time and I decided to update my installed software to the latest versions. But I see following screen and the update is stuck: “waiting for unattended-upgr to exit”

In this post, I will show you how to fix this issue.

Fixing “waiting for unattended-upgr to exit” issue

To begin, make sure all packages are in a clean state and correctly installed For this you can run:

sudo dpkg --configure -a

However, this fails for me with the following error

In fact, "sudo apt-get upgrade" also fails for me:

athakur:~$ sudo apt-get upgrade
E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?

This could be due to multiple causes. Most probably some other internal update is running and using the lock. You can check this with the following commands:

ps -eaf | grep -i apt
lsof /var/lib/dpkg/lock-frontend

Note: If you see a process like "apt.systemd.daily" using the lock, please wait for a few mins. This is auto-scheduler that updates your system. If you do not want this behavior you can go to "Software and Updates" and disable auto-updates:

Anyways, if you do not wish to wait you can always kill the process. Above commands - ps and lsof should give you PIDs corresponding to the process using the locks. You can kill them by running

sudo kill -9 PID

Replace PID with actual PID (Process ID) you see in the output of the above commands. Once done you can resume the software updates. You can also do

sudo apt-get upgrade

to upgrade your software.

If above does not work as well you can always delete the lock file (Not recommended)

sudo rm -rf /var/lib/dpkg/lock-frontend

and resume any update you might have. Please note we should not do this under ideal conditions. Lock files are meant to be present for special purposes. That being said, sometimes Softwares do go into an inconsistent state, and lock files have to be removed manually.

Once you kill the process or remove the lock file manually run following command to let dpkg fix itself:

sudo dpkg --configure -a

This is the same command we ran as the very 1ts step.

Sunday, 26 May 2019

How to remove computer name from prompt in Ubuntu terminal

Background

If you are using Ubuntu terminal you must have noticed that terminal prompt looks like below -

Notice that prompt looks like -

athakur@athakur-Inspiron-7572:~/Documents/code$

The problem is this is fairly large and with more folders inside the root directory, there is much less space to type actual commands. In this post, I will show how to fix this.

How to remove computer name from prompt in Ubuntu terminal

To fix this you need to edit ~/.bashrc file. In this file there is a variable called PS1 that governs how your command prompt looks like. If you open ~/.bashrc you should see something like below -

if [ "$color_prompt" = yes ]; then
    PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '
else
    PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
fi
unset color_prompt force_color_prompt

# If this is an xterm set the title to user@host:dir
case "$TERM" in
xterm*|rxvt*)
    PS1="\[\e]0;${debian_chroot:+($debian_chroot)}\u@\h: \w\a\]$PS1"
    ;;
*)
    ;;
esac

Let me explain a little bit about PS1 variable and above itself.

debian_chroot: This is applicable if you are running chroot operation (Your root directory is different than the default one). If you do not know chroot then do not worry, this is just and empty variable and can be ignored for now.
XTERM: This is for a terminal emulator. If you are using XTERM PS1 under this would take effect.
color_prompt: This would be the default case for most. Your terminal would support color prompt. So PS1 under "$color_prompt" = yes is something you need to edit.

If you have not heard above, don't worry about it just change the PS1 variable under the color_propmt if statement.

\u: expands to the current username
\h: expands to the current hostname
\w: expands to the current working directory
\$: expands to # for root and $ for all other users

Now that you know what each means and once you have figured out which PS1 to change you can simply remove "@\h" from it.

So change

if [ "$color_prompt" = yes ]; then

    PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '

if [ "$color_prompt" = yes ]; then

    PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '

Once done save it, and run -

source ~/.bashrc

or simply open a new terminal and you should see the change.

Customizing tray/taskbar date display in Ubuntu to show the actual date

Background

In my Ubuntu 18.04.2 LTS, I see that the date/time displayed on the desktop on the taskbar above is like - "Sun 12.00 PM".

You have to actually click on it to see the current date. In this post, I will show you how to add a date to the format that you see on the top. It would have been much easier if Ubuntu gave us this customization in settings itself, but unfortunately, that is not the case, at least not yet.

Customizing tray/taskbar date display in Ubuntu to show the actual date

There are two ways you can do this. I will show the most user-friendly method first. But if you are more comfortable with Linux command line terminal and executing commands go to 2. advance section below.

1. User-Friendly - GUI Way

If you do not prefer using the command line, then you can install a GUI based tool. The tool name is "gnome-tweak-tool". You can search for "GNOME tweaks" in the Ubuntu software center. Install it.

Launch it and you should see the following screen. Go to Top Bar for date settings.

You can change the configuration as needed. You can see there are a bunch of other options here as well - like show battery percentage. All yours to play with :)

NOTE: If you do not prefer the Ubuntu software center but apt-get to install software, you can use the following command to install and launch the above application.

sudo apt install gnome-tweak-tool

gnome-tweaks # now launch it

2. Advanced - CLI Way

There are two commands that you need to know here -

gsettings set org.gnome.desktop.interface clock-show-date true

makes the date appear

gsettings set org.gnome.desktop.interface clock-show-seconds true

switches the seconds display on

You can similarly replace "set" by "get" to see the current values. By default, both values are set to false which is why you don't see a date or seconds.

Now let's go ahead and set these values to true and see the change of date/time format in the taskbar above.

And the result is -

I personally don't like seconds showing up. Just date works for me, so I have set accordingly. You can have settings that best suit you. You can do a similar thing with battery percentage as well with following command -

gsettings set org.gnome.desktop.interface show-battery-percentage true

Saturday, 25 May 2019

How to fix hitting arrow keys adds characters in vi editor issue in Ubuntu

Background

I just purchased a new Laptop and the obvious next thing to do was add Ubuntu to it. There are some common problems that everyone faces when a new Ubuntu OS is installed and one such problem is - hitting arrow keys adds characters in vi the editor. In this post, I will show you how to fix this issue.

Fixing hitting arrow keys adds characters in vi editor issue

To fix this issue, edit a file called vim.rc in your root folder. If it's not present create one. You can use the following command.

vi ~/.vimrc

Now add the following content to it -

set nocompatible

If backspace is not working add following content -

set backspace=2

And that's it. Save the file and you should be good to go.

Another cleaner way is just to install vim -

sudo apt-get install vim

Now that the problem is resolved, let's try to understand the issue here. vi as an editor has normal and insert mode. When you open a file using vi you are in normal mode. Here you can go to any line, use any arrow keys and the behavior is as expected. Now when you type "i", you essentially go into insert mode. In this mode, vi does not expect you to go left, right, top, bottom using arrow keys. You can always do that by pressing "ESC" and going back to normal mode. Insert mode is just to add text to your file and that's the behavior of vi.

Hope this helps :)

Saturday, 13 April 2019

How to install Oracle Java 11 in Ubuntu

Background

In one of my previous posts I had covered how you can install Java 8 in your Ubuntu machine. In this post I will show how you can install Java 11 which is the latest SE version released (April 2019).

Oracle Java 11 is first LTS (Long term support) of Oracle Java release.

NOTE: Oracle uses a commercial license now. You can download and use Oracle java for development and testing without any cost but to use it in production you need to pay a fee.

If you do not want to pay you can always use Open JDK 11. From Java 11 forward, therefore, Oracle JDK builds and OpenJDK builds will be essentially identical. You can read more about this -

Oracle JDK Releases for Java 11 and Later

How to install Oracle Java 11 in Ubuntu

You can get Oracle Java 11 installer using linuxuprising PPA. To add this PPA execute following commands -

sudo add-apt-repository ppa:linuxuprising/java
sudo apt-get update

To install the installer execute the following command -

sudo apt-get install oracle-java11-installer

You would need to accept the terms and conditions and continue with the installer. Once installed you can check the version of Java installed with following command -

java -version

To make Java 11 default you can install following package / execute following command -

sudo apt-get install oracle-java11-set-default

If you do not want this as default simply remove above pacakge -

sudo apt-get remove oracle-java11-set-default

Friday, 2 February 2018

How to show hidden files and folder in Ubuntu

Background

Some files are folders are hidden in Ubuntu. These are the ones that start with a "." in the beginning. Eg -

~/.bashrc
~/.vimrc etc

In this post, I will show you how you can make these files visible.

How to hide files and folder in Ubuntu?

The Files file manager gives you the ability to hide and unhide files at your discretion. When a file is hidden, it is not displayed by the file manager, but it is still there in its folder.

To hide a file, rename it with a "." at the beginning of its name. For example, to hide a file named example.txt, you should rename it to .example.txt.

You can hide folders in the same way that you can hide files. Hide a folder by placing a "." at the beginning of the folder’s name.

How to show hidden files and folder in Ubuntu CLI

To see the hidden files in the command line interface (CLI) you can just use -

ls -la

To not see the hidden files you can just use -

ls -l

How to show hidden files and folder in Ubuntu Files explorer

To hidden files in the Files explorer, you can go to -

View -> Show hidden files

or you can simply press

Ctrl + H

You can use the same shortcut or select the same setting again to toggle between showing and hiding hidden files in your Files explorer.

To make this permanent you can go to -

Edit -> Preferences

and turn on the setting to show hidden files.

Thursday, 1 February 2018

How to restore a corrupted or deleted partition with TestDisk and Ubuntu Live

Background

I erased one of my partitions recently which was mounted on my /home path in Ubuntu Linux. However I was able to restore the partition back and life was back to normal.

I was trying to install Windows and the installer (from USB) was forcing UEFI mode instead of Legacy. So the NTFS partition did not work out and Windows could not be installed since partition was of type MBR instead of GPT (which is required by UEFI mode). So when I tried to make it GPT it started erasing entire disk instead of the disk partition I had selected. I stopped the process immediately but my partitions were gone and it was one disk without any partitions. As I mentioned earlier I was able to restore my previous partitions and data was intact.

In this post I will show you how we can do this.

Setup

You need to have a bootable USB with Ubuntu or gparted. Gparted has both tools -

gparted and
testdisk

installed so it is a much simpler option. But if you already have a bootable USB with ubuntu then you can use the same like I did.

Boot your machine from this USB drive.

NOTE : If you do not have a bootable USB you can create one using unetbootin.

Make a bootable flash drive from an ISO image on Ubuntu

How to restore a corrupted or deleted partition with TestDisk and Ubuntu Live

After you boot from Ubuntu live USB go to "Software and Updates" and under Downloadable from internet select the entry with "Universe".

Now run -

sudo apt-get update
sudo apt-get install gparted
sudo apt-get install testdisk

Open gparted to see your disks and partitions. If the partition is missing you should see an unallocated partition. Now run testdisk -

sudo testdisk

And follow next steps -

Select "No Log" option.
Select the disk drive you want to recover, e.g. /dev/sdc.
Select your partition table type. Usually it's Intel.
Select "Analyse" and then "Quick Search".
Your drive will be analysed and you will see a list of all found partitions. Press Enter.
On the next screen you have the option to either perform a second Deeper Search, or Write the current partition table to disk. If the quick search was successful, choose Write.
Finally reboot your machine to see the reflected changes
You can resuse the gaprted to see that the partition is restored post reboot

Screenshots

That's it. Your partitions should be restored.

Thursday, 18 January 2018

How to set up a squid Proxy with basic username and password authentication in Ubuntu

Background

Most of the big companies have their own proxies through which all the company data is routed through. This ensure malicious sites are blocked and all other traffic is audited via proper authentication.

To give a little background on Squid proxy -

Squid is a caching and forwarding HTTP web proxy. It has a wide variety of uses, including speeding up a web server by caching repeated requests, caching web, DNS and other computer network lookups for a group of people sharing network resources, and aiding security by filtering traffic. Although primarily used for HTTP and FTP, Squid includes limited support for several other protocols including Internet Gopher, SSL,[6] TLS and HTTPS. Squid does not support the SOCKS protocol.

Squid was originally designed to run as a daemon on Unix-like systems. A Windows port was maintained up to version 2.7. New versions available on Windows use the Cygwin environment.[7] Squid is free software released under the GNU General Public License.

Source : Wiki

Installing Squid proxy on Ubuntu

To install squid server simply run following command in your terminal -

sudo apt install squid

Squid run as daemon service in Ubuntu. You can execute following command to see the status of this service -

service squid status

It will show you if squid service is running or not.

Some important file paths are -

/etc/sqid : This is where your squid configuration resides
/var/log/squid : This is where your squid logs reside
/usr/lib/squid3,/usr/lib/squid : This is where your squid modules or libraries reside.

Now that we have Squid proxy installed. Let's configure it.

Squid configuration is located at -

/etc/squid/squid.conf

Before you make changed to this file make a copy of this and store it aside. Use following commands to do that -

sudo cp /etc/squid/squid.conf /etc/squid/squid.conf.original
sudo chmod a-w /etc/squid/squid.conf.original

This essentially created a copy of squid.conf called squid.conf.original and removed all write access to it so that no one can accidentally write it.

Default TCP port that Squid listens to is 3128. Go ahead and change it to 8888. I prefer using 8888 port since this is used by other proxies as well like Charles and Fiddler. To do this find a line called

http_port 3128

and change it to

http_port 8888

Next you need to provide rules to allow and disallow traffic. If you want to just allow trafic from your local machine you can add the following lines to the configuration -

acl localhost src 127.0.0.1/32
http_access allow localhost

acl is nothing but access control list. it's a keyword that states acl is starting. Next localhost is the name that is used to indentify the acl. I have named it localhost but it can be anything. Next we have src which is used to identify local IP addresses. Other options are -

srcdomain : used for declaring local domain,
dst : used for public IP &
dstdomain : used for public domain name

Next we have http_access that will basically take action provide in it's next word on the acl we define. In this we we are saying allow and for acl named localhost that we defined above. So Squid proxy is going to allow all http traffic from local machine (i.e with IP 127.0.0.1)

Last line you can add as -

http_access deny all

which says you deny all other traffic. So the way acl's work is -

For each request that Squid receives it will look through all the http_access statements in order until it finds a line that matches. It then either accepts or denys depending on your setting. The remaining rules are ignored.

This was basic settings for squid proxy. Now let's see how we can add an authentication to this scheme.

Post configuration you can just restart the squid service -

service squid restart

You can also view the service logs for this in file-

less /var/log/squid/cache.log

And you can view the access logs in file -

less /var/log/squid/access.log

How to set up a squid Proxy with basic username and password authentication?

For this you can add following lines to your squid configuration file squid.conf -

auth_param basic program /usr/lib/squid3/basic_ncsa_auth /etc/squid/passwords
auth_param basic realm proxy
acl authenticated proxy_auth REQUIRED
http_access allow authenticated

ident_lookup_access deny all
http_access deny all

Above configuration will ensure all traffic is authenticated. The username/password that would be needed to provide access will be stored in a file - /etc/squid/passwords. We will now see how we can create this file.

To generate username/passwrod you need to use a command called htpasswd. You can install this using -

apt-get install apache2-utils

Next to generate username/password type in following command -

sudo htpasswd -c /etc/squid/passwords YOUR_USERNAME

Replace YOUR_USERNAME with the user name you want. Eg admin. You will be prompted for password for this username twice. Once done your user is all setup. You can use this credentials to access your proxy.

NOTE : htpasswd stores the password hashed.

One done you can restart your squid service -

service squid restart

My conf file looks like below -

acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT

http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports


http_port 8888

coredump_dir /var/spool/squid

refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern (Release|Packages(.gz)*)$      0       20%     2880
refresh_pattern .               0       20%     4320

auth_param basic program /usr/lib/squid3/basic_ncsa_auth /etc/squid/passwords
auth_param basic realm proxy
acl authenticated proxy_auth REQUIRED
http_access allow authenticated


ident_lookup_access deny all
http_access deny all

Now you can test this by adding a proxy in firefox and trying to go to a http URL.

Add username/password that you just created before and the URL should be accessible.

Saturday, 24 September 2016

How to delete recently opened files history in ubuntu 14.04

Background

In this post we will see how we can delete recently opened files from Ubuntu's Unity dashboard. Why would I do that you ask? Answer is privacy but then again it largely depends on the usecase. There should not be such a need for a strictly persona computer but if it is shared it is better to delete history when you leave.

How to delete recently opened files history in ubuntu 14.04

Open security and privacy settings from unity dashbaord.

Then go to Files & Applications and select clear usage data and select the period you want to clear data from -

Click Ok and you are done.

Saturday, 28 May 2016

Installing Git in Ubuntu

Background

Sometime back I had written a post about git (it's installation and usage in windows). This post simple covers it's Linux counterpart. How to install git in Ubuntu.

Install and use Git on Windows

Installing git on Ubuntu

To install git execute following command -

git apt-get install git-core

Post installation you can run following command to verify installation -

git --version

And you are all good to go!

To set up git config you can use following command -

git config --global user.name "aniket91"
git config --global user.email "you@example.com"

To view the config you can do

git config --list

or view the config file

cat ~/.gitconfig

Try cloning a repository

git clone https://github.com/aniket91/DataStructures.git

My Git Repositories

Avoid Merge Commits

Whenever you have unpushed commits on your local and you try to do a git pull after those commits it will create a merge commit .
To avoid it follow one of the below mentioned methods,

run git pull --rebase
To avoid running it with the rebase flag and to make the above the default behavior when pulling changes, git config --global branch.autosetuprebase always
In SourceTree under tools->options/preferences under the Git tab select "Use rebase instead of merge by default for tracked branches"

Friday, 27 May 2016

Installing Oracle Java 8 In Ubuntu Or Linux Mint Via PPA Repository [JDK8]

Background

I have written a couple of posts on new features in Java 8 (See Related Links section at the bottom of this page).

In this post we will see how to install and configure Java 8 on Ubuntu. Current Java version that is set on my machine in Java7.

Installing Java 8

To install Java run the following commands -

sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer

Once you have run above commands you can verify the installation by running following command

java -version

Webupd8 ppa repository also provides package to set environment variables. Run following command for it

sudo apt-get install oracle-java8-set-default

NOTE : If you've already installed oracle-java6-set-default or oracle-java7-set-default, they will be automatically removed when installing oracle-java8-set-default (and the environment variables will be set for Oracle Java 8 instead).

Thursday, 3 September 2015

Installing Oracle instant database client in Ubuntu

Background

In this post I am going to show how to install oracle instant client (database client) on your ubuntu machine. After this you should be able to connect to your oracle server and execute queries on it from your machine using sqlplus.

Prerequisites

Firstly find out whether your system is 32 bit or 64 bit. Easiest way to find out is to execute

uname - a

If your machine is 64 bit you should notice something like x64 in the output. If you don't see it 32 bit like in my case (refer screenshot below)

Next make sure you have alien command installed. We will need to to convert rpm packages to deb. To install simply run

sudo apt-get install alien

Installing Oracle instant database client

Download instant client files from oracle site.Select correct files depending on your operating system and architecture. For me it is Ubuntu and 32 bit. So I have downloaded following files

oracle-instantclient12.1-basic-12.1.0.2.0-1.i386.rpm
oracle-instantclient12.1-sqlplus-12.1.0.2.0-1.i386.rpm
oracle-instantclient12.1-devel-12.1.0.2.0-1.i386.rpm

Since only rpm files are available and Ubuntu works with debian we need to convert rpm packages to deb packages.

alien -k oracle-instantclient12.1-basic-12.1.0.2.0-1.i386.rpm
alien -k oracle-instantclient12.1-sqlplus-12.1.0.2.0-1.i386.rpm
alien -k oracle-instantclient12.1-devel-12.1.0.2.0-1.i386.rpm

Now to install .deb packages use dpkg -i packageName

dpkg -i oracle-instantclient12.1-basic-12.1.0.2.0-1.i386
dpkg -i oracle-instantclient12.1-sqlplus-12.1.0.2.0-1.i386.rpm
dpkg -i oracle-instantclient12.1-devel-12.1.0.2.0-1.i386.rpm

Now you need to set some environment variables. Open ~/.bashrc file and add export following environment variables (add following line to file)

export ORACLE_HOME=/usr/lib/oracle/12.1/client
export LD_LIBRARY_PATH=/usr/lib/oracle/12.1/client/lib/
export PATH=$PATH:$ORACLE_HOME/bin

To make source your changes are reflected in same terminal execute

source ~/.bashrc

And you should be good to go. Verify your env variables.

Resolving errors

sometimes you may get following error

"sqlplus: error while loading shared libraries: libsqlplus.so: cannot open shared object file: No such file or directory"

in this case double check you have added LD_LIBRARY_PATH environment variable and it is pointing to write directory. Quick way to check is

echo $LD_LIBRARY_PATH
cd <output_of_above_command>

Sqlplus should now be recognized and work as expected

If you see errors related to libaio.so.1 missing then install it using

sudo apt-get install libaio1