Python instances

If you read main Documentation page you should know that before any use of ErlPort you need to start a Python instance. A Python instance is basically an operating system process which represented in Erlang by Erlang process.

To start an instance of Python one of the variants of python:start or python:start_link functions should be used. The python:start/0 function will start Python instance with the default parameters.

To stop the running Python instance python:stop/1 function should be used with the instance id as the parameter.

And of course you can run more than one Python instance, but be careful to not create way too many because they can waste all the OS resources.

The following is an example of start/stop API:

1> {ok, PythonInstance1} = python:start().
{ok,<0.34.0>}
2> {ok, PythonInstance2} = python:start().
{ok,<0.36.0>}
3> python:stop(PythonInstance1)
ok
4> python:stop(PythonInstance2)
ok

Call Python functions from Erlang

To call a function in Python python:call/4 function should be used which accepts the instance id and Module, Function and Arguments for Python function:

1> {ok, P} = python:start().
{ok,<0.34.0>}
2> python:call(P, operator, add, [2, 2]).
4

Of course in Python you can also have hierarchies of modules and objects so python:call/4 supports dot-separated names for Module and Function arguments:

3> python:call(P, 'os.path', splitext, [<<"name.ext">>]).
{<<"name">>,<<".ext">>}
4> python:call(P, sys, 'version.__str__', []).
<<"2.7.3 (default, Aug  1 2012, 05:14:39) \n[GCC 4.6.3]">>

In case of any error during the function call an exception of class error will be generated in the following form:

error:{python, ExceptionClass, ExceptionArgument, ReversedStackTrace}

For example:

5> try python:call(P, unknown, unknown, [])
5> catch error:{python, Class, Argument, StackTrace} -> error
5> end.
error
6> Class.
'exceptions.ImportError'
7> Argument.
"No module named unknown"
8> StackTrace.
[{<<"/.../erlport/priv/python2/erlport/erlang.py">>,
  237,<<"_incoming_call">>,
  <<"f = __import__(module, {}, {}, [objects[0]])">>},
 {<<"/.../erlport/priv/python2/erlport/erlang.py">>,
  245,<<"_call_with_error_handler">>,<<"function(\*args)">>}]

And of course don't forget to stop the instance at the end:

9> python:stop(P).
ok

If you want to call a function from your own Python module in most cases you need to set the Python path. You can do it with python:start/1 function or PYTHONPATH environment variable. The python:start/1 also can be used to change the default Python interpreter. For example let's create a simple Python module in /path/to/my/modules/version.py file:

import sys

def version():
    return sys.version

Now we can set path to this module in python:start/1 like this:

1> {ok, P} = python:start([{python_path, "/path/to/my/modules"},
1>                         {python, "python3"}]).
{ok,<0.34.0>}
2> python:call(P, version, version, []).
"3.2.3 (default, Oct 19 2012, 20:10:41) \n[GCC 4.6.3]"
3> python:stop(P).
ok

Call Erlang functions from Python

ErlPort uses Python erlport.erlang module as an interface to Erlang. Namely erlport.erlang.call() function allows to call Erlang functions from Python. The function accepts Module and Function arguments as erlport.erlterms.Atom() object and Arguments as a list. Currently each Erlang function will be called in a new Erlang process. Let's create the following Python module in pids.py file in the current directory which will be added to Python path automatically by Python:

from erlport.erlterms import Atom
from erlport.erlang import call

def pids():
    Pid1 = call(Atom("erlang"), Atom("self"), [])
    Pid2 = call(Atom("erlang"), Atom("self"), [])
    return [Pid1, Pid2]

Now we can call this function from Erlang:

1> {ok, P} = python:start().
{ok,<0.34.0>}
2> python:call(P, pids, pids, []).
[<0.36.0>,<0.37.0>]
3> python:stop(P).
ok

To simplify the demonstration the next example will use the call chaining so Python to Erlang calls will be initiated from Erlang shell. The following example also demonstrate the communication between two Python instances:

1> {ok, P1} = python:start().
{ok,<0.34.0>}
2> {ok, P2} = python:start().
{ok,<0.36.0>}
3> python:call(P1, os, getpid, []).
5048
4> python:call(P2, os, getpid, []).
5050
5> python:call(P1, 'erlport.erlang', call,
5>             [python, call, [P2, os, getpid, []]]).
5050
6> python:stop(P1).
ok
7> python:stop(P2).
ok

So the command #5 actually calls erlport.erlang.call() function for instance P1, which calls Erlang function python:call/4, which in order calls Python function os.getpid() for instance P2.

Send messages from Erlang to Python

To send a message from Erlang to Python first a message handler function on Python side should be set. The message handler function can be set with erlport.erlang.set_message_handler() function. The default message handler just ignore all the incoming messages. And if you don't need to handle incoming message anymore the default handler can be set again with erlport.erlang.set_default_message_handler() function.

Be careful when you write a message handling function because the function can also get some unexpected messages which probably should be ignored and in case of any error in the message handler the whole instance will be shut down.

To demonstrate message sending from Erlang to Python we will first create the following module in the current directory in a file handler.py:

from erlport.erlterms import Atom
from erlport.erlang import set_message_handler, cast

def register_handler(dest):
    def handler(message):
        cast(dest, message)
    set_message_handler(handler)
    return Atom("ok")

This message handler just send all messages to the selected Erlang process.

To send a message to Python python:cast/2 function can be used and also all unknown to ErlPort messages will be redirected to the message handler.

1> {ok, P} = python:start().
{ok,<0.34.0>}
2> python:call(P, handler, register_handler, [self()]).
ok
3> python:cast(P, test_message).
ok
4> flush().
Shell got test_message
ok
5> P ! test_message2.
test_message2
6> flush().
Shell got test_message2
ok
7> python:stop(P).
ok

Send messages from Python to Erlang

It's very easy to send a message from Python to Erlang - you just need to know the pid() or registered name of the destination process. The function erlport.erlang.cast() accepts two arguments - the id of the destination process and a message which can be any supported data type according to Data types mapping. And of course you can send messages to any other ErlPort process.

The following is a demonstration of message sending from Python:

1> {ok, P} = python:start().
{ok,<0.34.0>}
2> python:call(P, 'erlport.erlang', cast, [self(), test_message]).
undefined
3> flush().
Shell got test_message
ok
4> register(test_process, self()).
true
5> python:call(P, 'erlport.erlang', cast, [test_process, test_message2]).
undefined
6> flush().
Shell got test_message2
ok
7> python:stop(P)
ok

Custom data types

ErlPort only supports a minimal set of data types to make sure the types are orthogonal - can be created and meaningful in any language supported by ErlPort. In addition ErlPort also supports language specific opaque data type containers so for example Python instances can exchange any picklable data type. But sometimes it's better to use rich inter-language data types in which case custom data types can be used.

There are two functions to support custom data types:

• erlport.erlang.set_encoder() which sets the Python to Erlang data type converter, and
• erlport.erlang.set_decoder() which sets the converter for the opposite direction - Erlang to Python

Both of the functions can be reset to the default, which just pass the value unmodified, with erlport.erlang.set_default_encoder() and erlport.erlang.set_default_decoder() functions correspondingly. Note also that there's no support for automatic traversing of container data types so it should be implemented by encoder/decoder functions if needed.

To give you a feeling how it works the following module in the current directory and date_type.py file will add the partial support to ErlPort for datetime.date() and datetime.timedelta() objects:

from datetime import date, timedelta
from erlport.erlterms import Atom
from erlport.erlang import set_encoder, set_decoder

def setup_date_type():
    set_encoder(date_encoder)
    set_decoder(date_decoder)
    return Atom("ok")

def date_encoder(value):
    if isinstance(value, date):
        value = Atom("date"), (value.year, value.month, value.day)
    elif isinstance(value, timedelta):
        value = Atom("days"), value.days
    return value

def date_decoder(value):
    if isinstance(value, tuple) and len(value) == 2:
        if value[0] == "date":
            year, month, day = value[1]
            value = date(year, month, day)
        elif value[0] == "days":
            value = timedelta(days=value[1])
    return value

The date_type module can be used in Erlang shell like this:

1> {ok, P} = python:start().
{ok,<0.34.0>}
2> python:call(P, date_type, setup_date_type, []).
ok
3> python:call(P, datetime, timedelta, []).
{days,0}
4> python:call(P, datetime, 'date.today', []).
{date,{2013,6,10}}
5> python:call(P, operator, sub, [{date, {2013, 1, 5}},
5>                                {date, {2012, 12, 15}}]).
{days,21}
6> python:call(P, operator, add, [{date, {2013, 1, 1}},
6>                                {days, -1}]).
{date,{2012,12,31}}
7> python:stop(P).
ok

Standard output redirection

As a convenient feature ErlPort also supports redirection of Python`s STDOUT to Erlang which can be used for example for debugging. It's easier to demonstrate with Python 3 in which print is a function:

1> {ok, P} = python:start([{python, "python3"}]).
{ok,<0.34.0>}
2> python:call(P, builtins, print, [<<"Hello, World!">>]).
b'Hello, World!'
undefined
3> python:stop(P).
ok

Data types mapping

The following table defines mapping of Erlang data types to Python data types:

And here is the table of Python to Erlang data types mapping. The types mapping between Erlang and Python are practically orthogonal:

erlport.erlterms.Atom(string)

Class to represent Erlang atoms in Python. The string argument should be a byte string (str() in Python 2 or bytes() in Python 3) not longer that 255 bytes. Each Atom instance is a singleton the same as in Erlang.

erlport.erlterms.List(list)

Class to represent Erlang lists in Python. Basically just a subclass of list() with the following one additional method:

List.to_string()

Convert list content to an Unicode string (unicode() in Python 2 or str() in Python 3). There's no distinct string data type in Erlang so lists should be explicitly converted to strings with this method.

erlport.erlterms.ImproperList(list, tail)

Class to represent Erlang improper lists in Python. The tail argument can't be a list. Note that this class exists mostly to convert improper lists received from Erlang side and probably there are no reasons to create instances of this class in Python.

Erlang API

python:start() -> {ok, Pid} | {error, Reason}

Start Python instance with the default options

python:start(Options) -> {ok, Pid} | {error, Reason}

Start Python instance with options. The Options argument should be a list with the following options.

General options:

{buffer_size, Size::pos_integer()}

Size in bytes of the ErlPort receive buffer on Python side. The default is 65536 bytes.

{call_timeout, Timeout::pos_integer() | infinity}

Default timeout in milliseconds for function calls. Per call timeouts can be set with python:call/5 function.

{cd, Path::string()}

Change current directory to Path before starting.

{compressed, 0..9}

Set terms compression level. 0 means no compression and 9 will take the most time and may (or may not) produce a smaller result. Can be used as an optimisation if you know that your data can be easily compressed.

{env, [{Name::string(), Value::string() | false}]}

Set environment for Python instance. The Name variable is the name of environment variable to set and Value can be a string value of the environment variable or false if the variable should be removed.

nouse_stdio

Not use STDIN/STDOUT for communication. Not supported on Windows.

{packet, 1 | 2 | 4}

How many bytes to use for the packet size. The default is 4 which means that packets can be as big as 4GB but if you know that your data will be small you can set it for example to 1 which limits the packet size to 256 bytes but also saves 3 bytes for each packet. Note however that ErlPort adds some meta-information in each packet so the resulting packets always will be bigger than your expected size.

{start_timeout, Timeout::pos_integer() | infinity}

Time to wait for the instance to start.

use_stdio

Use STDIN/STDOUT for communication. The default.

Python related options:

{python, Python::string()}

Path to the Python interpreter executable

{python_path, Path::string() | [Path::string()]}

The Python modules search path. The Path variable can be a string in PYTHONPATH format or a list of paths. The priorities of different ways to set the modules search path is as follows:

1. python_path option
2. PYTHONPATH environment variable set through the env option
3. PYTHONPATH environment variable

python:start(Name, Options) -> {ok, Pid} | {error, Reason}

Start named Python instance. The instance will be registered with Name name. The Options variable is the same as for python:start/1.

python:start_link() -> {ok, Pid} | {error, Reason}

The same as python:start/0 except the link to the current process is also created.

python:start_link(Options) -> {ok, Pid} | {error, Reason}

The same as python:start/1 except the link to the current process is also created.

python:start_link(Name, Options) -> {ok, Pid} | {error, Reason}

The same as python:start/2 except the link to the current process is also created.

python:stop(Instance) -> ok

Stop Python instance

python:call(Instance, Module, Function, Arguments) -> Result

Call Python function. The Instance variable can be a pid() which returned by one of the python:start functions or an instance name (atom()) if the instance was registered with a name. The Module and Function variables should be atoms and Arguments is a list.

In case of any error on Python side during the function call an exception of class error will be generated in the following form:

error:{python, ExceptionClass, ExceptionArgument, ReversedStackTrace}

python:call(Instance, Module, Function, Arguments, Options) -> Result

The same as python:call/4 except the following options can be added:

{timeout, Timeout::pos_integer() | infinity}

Call timeout in milliseconds.

python:cast(Instance, Message) -> ok

Send a message to the Python instance.

Python API

erlport.erlang.call(module, function, arguments) -> result

Call Erlang function as module:function(arguments). The function and module variables should be of type erlport.erlterms.Atom() and arguments should be a list.

erlport.erlang.cast(pid, message)

Send a message to Erlang. The pid and message variables should be the same types as supported by Erlang ! (send) expression. Erlang pid() variables however can't be created in Python but can be passed as parameters from Erlang.

erlport.erlang.self() -> pid

Get the Erlang pid of the Python instance

erlport.erlang.set_encoder(encoder)

Set encoder for custom data types. Encoder is a function with a single value argument which is can be any Python data type and should return an Erlang representation of this type using supported Data types mapping.

erlport.erlang.set_decoder(decoder)

Set decoder for custom data types. Decoder is a function with a single value argument which is one of the supported Erlang data types according to Data types mapping. The function should decode and return Erlang representation of the rich Python data type.

erlport.erlang.set_message_handler(handler)

Set message handler. Message handler is a function with a single message argument which receive all the incoming messages.

erlport.erlang.set_default_encoder()

Reset custom data types encoder to the default which is just pass the term through without any modifications

erlport.erlang.set_default_decoder()

Reset custom data types decoder to the default which is just pass the term through without any modifications

erlport.erlang.set_default_message_handler()

Reset message handler to the default which is just ignore all the incoming messages

Environment variables

The following environment variables can change the default behavior of ErlPort:

ERLPORT_PYTHON

Path to Python interpreter executable which will be used by default.

PYTHONPATH

The default search patch for module files. The same as PYTHONPATH environment variable supported by Python. The priorities of different ways to set the modules search path is as follows:

1. python_path option
2. PYTHONPATH environment variable set through the env option
3. PYTHONPATH environment variable