Returning value from a thread in python

Background

In this post we will see how python functions can be executed in a new thread and also how can we return some value from it.

NOTE : Python is pass by value similar to java. But for objects it is pass by value of the reference (again just like Java).

Returning value from a thread in python

Create a file called test.py and add the following code to it - 

import threading

print("Starting Test Python Modules");

def threadtarget(array):
    array.append("Hello World!")

array=[]
t = threading.Thread(target=threadtarget, args=(array,))
t.start()
t.join()
print("Value received from thread : " + array[0])
    
print("Terminating Test Python Module");

Explanation :  We are have a simple function threadtarget that accepts a list in it's argument and adds a String called "Hello World!" to it. In our script we are creating a new thread to run the method threadtarget and finally read the value of list we pass to confirm value added by thread to the list is retained.

Execute the script with -

• python test.py

This is captured in below screenshot

 Related Links

• Shutdown hook in Python(OSFG) 
• Capturing python output in a file along with the stdout 

Capturing python output in a file along with the stdout

Background

In last couple of posts we saw how python works and how can we add a shutdown hook in python. Whenever we execute any python script it's output gets logged to the console from which you execute your script. In this post we will see how can we log this output to a file while retaining what gets printed on your console.

Capturing python output in a file along with the stdout

create a file test.py and add following code to it

import atexit
import sys
import datetime
import os

class Logger(object):
    def __init__(self, filename="output.txt"):
        # Initialize the logfile
        self.terminal = sys.stdout
        self.log = open(filename, "a")

        # Write the command and arguments if any
        self.log.write("CMD: " + str(sys.argv) + "\n")
    def __del__(self):
        self.log.flush()
        del self.log
    def write(self, message):
        self.terminal.write(message)
        self.log.write(message)
    def isatty(self):
        return True
    def flush(self):
        self.log.flush()

now = datetime.datetime.utcnow()
if not os.path.exists(os.getcwd() + os.sep + "logs"):
    os.makedirs(os.getcwd() + os.sep + "logs")
logFile = os.getcwd() + os.sep + "logs" + os.sep +'log.txt'
sys.stdout = Logger(logFile)
del now

print("Starting Test Python Modules");

def testmethod():
    print("From test method")

atexit.register(testmethod)    
    
print("Terminating Test Python Module");

Explanation : What above cod does? We are defining a custom class called Logger and defining methods that a stdout object defines. In this new custom class and methods we retain output to console and also add a output to a file whose path can be provided in the objects constructor while instantiating. Finally we say sys.sysout points to this new Logger object. Lets try this out. Run this script

• python test.py

You should see a directory called logs getting created in current directory and it will have a file called log.txt with your script output. All of this is captured in below screenshot.

Related Links

• Shutdown hook in Java(OSFG)   
• How Python works? (OSFG)

How Python works?

Background

Since I have covered some of python topics already I thought it would be a nice time to understand how Python actually worked. There is s common misconception that python is a purely interpreted language. We will see how it actually works now.

How Python works?

Python like Java is first compiled into byte codes which is the interpreted by Python VM by running an interpreter.

1. When a module is imported for the first time, or when the source is more recent than the current compiled file, a .pyc file containing the compiled code will usually be created in the same directory as the .py file. When you run the program next time, Python uses this file to skip the compilation step.
2. Running a script is not considered an import and no .pyc will be created. For example, if you have a script file abc.py that imports another module xyz.py, when you run abc, xyz.pyc will be created since xyz is imported, but no abc.pyc file will be created since abc.py isn’t being imported.
3. You can manually compile your source file using compileall module. You can just say python -m compileall -l test.py and that should create your compiled .pyc file under a directory called __pycache__ under your current directory. Thereafter you can directly run this .pyc file using python test.cpython-34.pyc  (All of this is captured below in the screenshot)

As I mentioned before the imported modules are automatically compiled for further use. Lets see how that works out.

In my current folder I have two py files

1. test.py and
2. testlib.py

testlib.py has a method called helloworld which simply prints "Hello World!" to the console.

It's contents are as follows

test.py

import testlib

print("Test Python Module")
testlib.helloworld()

testlib.py

def helloworld():
    print("Hello World!")

Now lets run our test.py and see the results. You will see the output and also see a __pycache__ directory getting create under current directory with compile testlib.pyc file. All of this is captured is screenshot below

Note : Only difference is imported modules are compiled to pyc files automatically based on timestamps where as your source files are compiled on the fly. To compile your source files you can use compileall module.

Compiled files with bytecodes are faster to run by python interpreter. Generally .pyc files are the once you ship.

Note :  " It's worth noting that while running a compiled script has a faster startup time (as it doesn't need to be compiled), it doesn't run any faster. "

Given above note will help you understand why modules are compiled and not the main source file. Byte compiled modules simply help in faster startup time.

Related Links

1. Shutdown hook in Python(OSFG) 
2. Using Fabric Python command-line tool(OSFG) 
3. Difference between functions and methods. (OSFG)
4. If Python is interpreted, what are .pyc files? (SO)
5. How to avoid .pyc files? (SO)
6. Difference between Compiler, Interpreter and Assembler (OSFG)
7. compileall python module documentation

Adding shutdown hook in python using atexit module

Background

A shutdown hook is basically a code snippet that allows programmers to run some code at program termination. I had discussed shutdown hook in Java some time back. In this post we will see shutdown hooks in python.

Adding shutdown hook in python using atexit module

You can use the atexit module in python to register shutdown hooks.  See the following code

import atexit

print("Starting Test Python Module");

def testmethod():
    print("From test method")

atexit.register(testmethod)    
    
print("Terminating Test Python Module");

Save it in a file called test.py and simply run it using

• python test.py

And this outputs :

As you can notice it prints "From test method" at the end by executing registered method  testmethod.


Note : 

1. Previously you could do the same by importing sys module and then using sys.exitfunc = testmethod but  sys.exitfunc is deprecated since python 2.4 and is removed since python 3.0.
2. The atexit module defines a single function to register cleanup functions. Functions thus registered are automatically executed upon normal interpreter termination. 
3. atexit runs these functions in the reverse order in which they were registered; if you register A, B, and C, at interpreter termination time they will be run in the order C, B, A.
4. The functions registered via this module are not called when the program is killed by a signal not handled by Python, when a Python fatal internal error is detected, or when os._exit() is called.

Related Links

• Shutdown hook in Java(OSFG)
• Using Fabric Python command-line tool(OSFG)
• Difference between functions and methods. (OSFG)
• atexit documenttaion