1.7. PyCOMPSs: Using objects, lists, and synchronization. Managing fault-tolerance.

In this example we will see how classes and objects can be used from PyCOMPSs, and that class methods can become tasks. The example also illustrates the current fault-tolerance management provided by the runtime.

Import the PyCOMPSs library

[1]:
import pycompss.interactive as ipycompss

Start the runtime

Initialize COMPSs runtime Parameters indicates if the execution will generate task graph, tracefile, monitor interval and debug information.

[2]:
import os
if 'BINDER_SERVICE_HOST' in os.environ:
    ipycompss.start(graph=True, debug=False,
                    project_xml='../xml/project.xml',
                    resources_xml='../xml/resources.xml')
else:
    ipycompss.start(graph=True, monitor=1000, trace=False, debug=False)
********************************************************
**************** PyCOMPSs Interactive ******************
********************************************************
*          .-~~-.--.           ______         ______   *
*         :         )         |____  \       |____  \  *
*   .~ ~ -.\       /.- ~~ .      __) |          __) |  *
*   >       `.   .'       <     |__  |         |__  |  *
*  (         .- -.         )   ____) |   _    ____) |  *
*   `- -.-~  `- -'  ~-.- -'   |______/  |_|  |______/  *
*     (        :        )           _ _ .-:            *
*      ~--.    :    .--~        .-~  .-~  }            *
*          ~-.-^-.-~ \_      .~  .-~   .~              *
*                   \ \ '     \ '_ _ -~                *
*                    \`.\`.    //                      *
*           . - ~ ~-.__\`.\`-.//                       *
*       .-~   . - ~  }~ ~ ~-.~-.                       *
*     .' .-~      .-~       :/~-.~-./:                 *
*    /_~_ _ . - ~                 ~-.~-._              *
*                                     ~-.<             *
********************************************************
* - Starting COMPSs runtime...                         *
* - Log path : /home/user/.COMPSs/Interactive_07/
* - PyCOMPSs Runtime started... Have fun!              *
********************************************************

Importing task and arguments directionality modules

Import task module before annotating functions or methods

[3]:
from pycompss.api.api import compss_barrier
from pycompss.api.api import compss_wait_on
from pycompss.api.task import task
from pycompss.api.parameter import *

Declaring a class

[4]:
%%writefile my_shaper.py

from pycompss.api.task import task
from pycompss.api.on_failure import on_failure
from pycompss.api.parameter import IN
import sys

class Shape(object):
    def __init__(self,x,y):
        self.x = x
        self.y = y
        description = "This shape has not been described yet"

    @task(returns=int, target_direction=IN)
    def area(self):
        return self.x * self.y

    @task()
    def scaleSize(self,scale):
        self.x = self.x * scale
        self.y = self.y * scale

    # management='IGNORE' | 'RETRY' | 'FAIL' | 'CANCEL_SUCCESSORS'
    @on_failure(management="CANCEL_SUCCESSORS")
    @task()
    def downScale(self,scale):
        if (scale <= 0):
            sys.exit(1)
        else:
            self.x = self.x/scale
            self.y = self.y/scale

    @task(returns=int, target_direction=IN)
    def perimeter(self):
        return 2 * self.x + 2 * self.y

    def describe(self,text):
        self.description = text

    @task(target_direction=IN)
    def infoShape(self):
        print('Shape x=', self.x, 'y= ', self.y)
Overwriting my_shaper.py

Invoking tasks

[5]:
from my_shaper import Shape
[6]:
my_shapes = []
my_shapes.append(Shape(100,45))
my_shapes.append(Shape(50,50))
my_shapes.append(Shape(10,100))
my_shapes.append(Shape(20,30))
my_shapes.append(Shape(200,25))
[7]:
all_perimeters = []
[8]:
i=4
for this_shape in my_shapes:
    this_shape.scaleSize(2)
    this_shape.area()
    i = i - 1
    this_shape.downScale(i)
    all_perimeters.append(this_shape.perimeter())

Synchronizing results from tasks

[9]:
all_perimeters = compss_wait_on(all_perimeters)
print(all_perimeters)
WARNING: Could not retrieve the object /home/user/.COMPSs/Interactive_07/tmpFiles/pycompsssqo27tt9/de05df52-7264-11ee-96af-c8f75057b763-12 since the task that produces it may have been IGNORED or CANCELLED. Please, check the logs. Returning None.
WARNING: Could not retrieve the object /home/user/.COMPSs/Interactive_07/tmpFiles/pycompsssqo27tt9/de05df52-7264-11ee-96af-c8f75057b763-15 since the task that produces it may have been IGNORED or CANCELLED. Please, check the logs. Returning None.
[193.33333333333334, 200.0, 440.0, None, None]
INFO: The ERRMGR displayed some error or warnings.

Stop the runtime

[10]:
ipycompss.stop(sync=False)
********************************************************
***************** STOPPING PyCOMPSs ********************
********************************************************
Checking if any issue happened.
[ERRMGR]  -  WARNING: file /home/user/.COMPSs/Interactive_07/tmpFiles/pycompsssqo27tt9/de05df52-7264-11ee-96af-c8f75057b763-12:linux-2e63 was accessed but the file information not found. Maybe it has been previously canceled
[ERRMGR]  -  WARNING: No version available. Returning null
[ERRMGR]  -  WARNING: file /home/user/.COMPSs/Interactive_07/tmpFiles/pycompsssqo27tt9/de05df52-7264-11ee-96af-c8f75057b763-15:linux-2e63 was accessed but the file information not found. Maybe it has been previously canceled
[ERRMGR]  -  WARNING: No version available. Returning null
Warning: some of the variables used with PyCOMPSs may
         have not been brought to the master.
********************************************************