Basic User Guide

1 RADICAL-Pilot Setup


First, we will import the necessary dependencies.

import os, sys
import radical.pilot as rp
print rp.version

2. Local Pilot Example

This example shows how to execute simple tasks using a Pilot-Job.

2.1 Create a new Session and Managers.

session = rp.Session()
pmgr = rp.PilotManager(session=session)
umgr = rp.UnitManager (session=session,scheduler=rp.SCHED_ROUND_ROBIN)
print "Session id: %s \n \
Pilot Manager: %s \n \
Unit Manager: %s" % (session.uid, pmgr.uid, umgr.uid)

2.2 Submit Pilot and add to Unit Manager

Create a description of the Pilot, which specifies its target resource, size and runtime.

pdesc = rp.ComputePilotDescription()
pdesc.resource = "local.localhost"
pdesc.runtime  = 10 # minutes
pdesc.cores    = 2
pilot = pmgr.submit_pilots(pdesc)

The umgr.add_pilot(pilot) binds a Pilot to a Unit Manager. This informs the Unit Manager that the resources of the Pilot can be used for exeuction of Units. Multiple Pilots can be bound to a Unit Manager, and later be removed via umgr.remove_pilot(). The Unit Manager will schedule incoming Compute Units across all available Pilots.

print "uid: ", pilot.uid
print "resource: ", pilot.resource
print "sandbox: ", pilot.sandbox
print "state: ", pilot.state

2.3 Submit Compute Units

Create a description of the Compute Unit, which specifies the details of the task to be executed.

cudesc1 = rp.ComputeUnitDescription()
cudesc1.environment = {'CU_NO': 1}
cudesc1.executable  = "/bin/echo"
cudesc1.arguments   = ['I am CU number $CU_NO']
cudesc1.cores       = 1
cudesc2 = rp.ComputeUnitDescription() cudesc2.environment = {'CU_NO': 2} cudesc2.executable = "/bin/echo" cudesc2.arguments = ['I am CU number $CU_NO'] cudesc2.cores = 1

Submit the previously created Compute Unit descriptions to the Unit Manager. This will trigger the selected scheduler (in this case the Round-Robin scheduler) to start assigning Compute Units to the Pilots.

print "Submit Compute Units to Unit Manager ..."
cu_set = umgr.submit_units([cudesc1, cudesc2])
print "Waiting for CUs to complete ..."
print "All CUs completed successfully!"

Inspect the output of the CUs

print cu_set[0].stdout
print cu_set[1].stdout

2.6 Close Session

The session must always be closed before exiting: this will tear down the Pilots, which would otherwise continue to consume resources until their specified runtime is up.