Task Selection

As in the case of Java, a COMPSs Python application is a Python sequential program that contains calls to tasks. In particular, the user can select as a task:

Functions
Instance methods: methods invoked on objects
Class methods: static methods belonging to a class

The task definition in Python is done by means of Python decorators instead of an annotated interface. In particular, the user needs to add a @task decorator that describes the task before the definition of the function/method.

As an example (Code 21), let us assume that the application calls a function foo, which receives a file path (file_path – string parameter) and a string parameter (value). The code of foo appends the value into file_path.

Code 21 Python application example

def foo(file_path, value):
    """ Update the file 'file_path' with the 'value'"""
    with open(file_path, "a") as fd:
        fd.write(value)

def main():
    my_file = "sample_file.txt"
    with open(my_file, "w") as fd:
        fd.write("Hello")
    foo(my_file, "World")

if __name__ == '__main__':
    main()

In order to select foo as a task, the corresponding @task decorator needs to be placed right before the definition of the function, providing some metadata about the parameters of that function. The @task decorator has to be imported from the pycompss library (Code 22).

Code 22 Python task import

from pycompss.api.task import task

@task(metadata)
def foo(parameters):
     ...

Tip

The PyCOMPSs task api also provides the @task decorator in camelcase (@Task) with the same functionality.

The rationale of providing both @task and @Task relies on following the PEP8 naming convention. Decorators are usually defined using lowercase, but since the task decorator is implemented following the class pattern, its name is also available as camelcase.

Important

The file that contains tasks definitions MUST ONLY contain imports or the if __name__ == "__main__" section at the root level. For example, Code 22 includes only the import for the task decorator, and the main code is included into the main function.

The rationale of this is due to the fact that the module is loaded from PyCOMPSs. Since the code included at the root level of the file is executed when the module is loaded, this causes the execution to crash.

Function parameters

The @task decorator does not interfere with the function parameters, Consequently, the user can define the function parameters as normal python functions (Code 24).

Code 24 Task function parameters example

@task()
def foo(param1, param2):
     ...

The use of *args and **kwargs as function parameters is supported (Code 25).

Code 25 Python task *args and **kwargs example

@task(returns=int)
def argkwarg_foo(*args, **kwargs):
    ...

And even with other parameters, such as usual parameters and default defined arguments. Code 26 shows an example of a task with two three parameters (whose one of them (s) has a default value (2)), *args and **kwargs.

Code 26 Python task with default parameters example

@task(returns=int)
def multiarguments_foo(v, w, s=2, *args, **kwargs):
    ...

Tasks within classes

Functions within classes can also be declared as tasks as normal functions. The main difference is the existence of the self parameter which enables to modify the callee object.

For tasks corresponding to instance methods, by default the task is assumed to modify the callee object (the object on which the method is invoked). The programmer can tell otherwise by setting the target_direction argument of the @task decorator to IN (Code 27).

Code 27 Python instance method example

class MyClass(object):
    ...
    @task(target_direction=IN)
    def instance_method(self):
        ... # self is NOT modified here

Class methods and static methods can also be declared as tasks. The only requirement is to place the @classmethod or @staticmethod over the @task decorator (Code 28). Note that there is no need to use the target_direction flag within the @task decorator.

Code 28 Python @classmethod and @staticmethod tasks example

class MyClass(object):
    ...
    @classmethod
    @task()
    def class_method(cls, a, b, c):
        ...

    @staticmethod
    @task(returns=int)
    def static_method(a, b, c):
        ...

Tip

Tasks inheritance and overriding supported!!!

Caution

The objects used as task parameters MUST BE serializable:

Implement the __getstate__ and __setstate__ functions in their classes for those objects that are not automatically serializable.

The classes must not be declared in the same file that contains the main method (if __name__ == '__main__') (known pickle issue).

Important

For instances of user-defined classes, the classes of these objects should have an empty constructor, otherwise the programmer will not be able to invoke task instance methods on those objects (Code 29).

Code 29 Using user-defined classes as task returns

# In file utils.py
from pycompss.api.task import task
class MyClass(object):
    def __init__(self): # empty constructor
        ...

    @task()
    def yet_another_task(self):
        # do something with the self attributes
        ...

    ...

# In file main.py
from pycompss.api.task import task
from utils import MyClass

@task(returns=MyClass)
def ret_foo():
    ...
    myc = MyClass()
    ...
    return myc

def main():
    o = ret_foo()
    # invoking a task instance method on a future object can only
    # be done when an empty constructor is defined in the object's
    # class
    o.yet_another_task()

if __name__=='__main__':
    main()