Ensemble Toolkit Basic Example

The example below will take you through a simple "Hello World" script.

Please make sure to go through all the code blocks.

1. Ensemble Toolkit Setup

First, we import all necessary modules

from radical.ensemblemd import Kernel, PoE, EnsemblemdError, ResourceHandle
import radical.ensemblemd as re 
print re.version

2. Local Example

This example shows how to execute a task on your local machine that writes "Hello World" to a file and downloads the file using Ensemble toolkit components.

2.1 Create the workload

We first create the worload using one of the execution patterns provided by Ensemble toolkit. In this case, we use Pipeline of Ensembles which is essentially multiple stages of ensembles of tasks where each stage begins execution after the completion of all the tasks in the previous stage. We choose number of stages=1, number of instances (i.e. of tasks) =1 for this simple case.

class MyApp(PoE):
	def __init__(self, stages,instances):
		PoE.__init__(self, stages,instances)
	def stage_1(self, instance):
		k = Kernel(name="misc.hello")    				# name of the kernel to be executed by each task
		k.arguments = ["--file=temp.txt"]    				# arguments to the kernel
		k.download_output_data = ['./temp.txt > output_file.txt']    	# download "temp.txt" as "output_file.txt"

app=MyApp(stages=1, instances=1)    	# Create object of the execution pattern class

2.2 Create a Resource Handle

Next, we allocate the required amount of resources. In this case, we request for 1 core on localhost for a duration of 10 minutes. A URL to a database is also required as the underlying components of Ensemble toolkit required a MongoDB for task coordination and bookkeeping.

local_resource = ResourceHandle(
	resource="localhost",    	# name of the resource to be used
	cores=1,    			# number of cores to be requested
	walltime=10,    		# duration of resource request
	database_url='mongodb://rp:rp@ds015335.mlab.com:15335/rp')
local_resource.allocate()

2.3 Run the workload

Simple run our workload on the acquired resources.

local_resource.run(app)

2.4 Deallocate resources

When workload execution is complete, we can deallocate the acquired resources.

local_resource.deallocate()

2.5 See downloaded file content

$ cat output_file.txt