MoreDream

JRDL (Job & Resource Description Language)

Overview

The Globus Resource Specification Language (RSL) 2 provides a common interchange language to describe a Grid job. However the RSL 2 does not contain user preferences to select automatically resources for allocating the job. Moreover the RSL 2 has no features to describe elements required to co-allocate the job and to represent sequent jobs. Therefore we provide the Job and Resource Description Language (JRDL) as a general language to describe a job and user preferences required allocating resources for the job in Grid environment based on XML.
JRDL has a collection of jobs, each of which is "an atomic task" of a workflow specification or other kinds of a complex, multi-step application. A job consists of job attributes and resource attributes as showed in Figure 1. Job attributes are elements to need to reserve resources in advance, allocate resources, and execute job. Resource attributes are user preferences to determine resources to execute a job. Here we cover what both kinds of attributes include and how each user should specify each attribute.

<executable> … </executable> Job Attributes
<arguments> … </arguments>
…

<resource>                          Resource Attributes
   <count>
     <integer value="2"/>
   </count>
   …
</resource>

  </job>
  < job>
  …
  </job>
</jrdl>

Figure 1. JRDL containing a collection of jobs,
each of which has both job attributes and resource attributes

Basically, we use RSL2 schema to describe contents of each attribute. For example, job executable attribute must include path element to describe executable file as shown in Figure 2. In addition, as RSL2 provides, each element could include substitutionRef element replaced by the element where substituionDef element defines. For instance, job executable attribute showed in Figure 2 could be replaced with job executable attribute including substitutionRef element illustrated in Figure 3.

<executable>
   <path>
     <stringElement value="/home/user/executable"/>
   </path>
</executable>
…

</job>
</jrdl>

Figure 2. Job executable attribute

</jrdl>

Figure 3. Job executable attribute including substituionRef element

By default, as illustrated Figure 3, if you want to use substitutionRef element, substituionDef element must be specified except reserved substituionDef elements to be defined at local resources as shown in Table 1.


Substitution Definition	Meaning
HOME	Home directory path of user account
LOGNAME	Log name
GLOBUS_LOCATION	Globus location
X509_CERT_DIR	Certificate directory
GLOBUS_HOST_CPUTYPE	Host CPU type
GLOBUS_HOST_MANUFACTURER	Host manufacturer
GLOBUS_HOST_OSNAME	Host OS name
GLOBUS_HOST_OSVERSION	Host OS version
GLOBUS_RMS_JOB_CONTACT	The contact address of RMS
GLOBUSRUN_GASS_URL	The address of GASS server that a job submission client invoke automatically
JOBID	The Job submission id except "http"
CUR_DIRL	The Current directory path where a client is launched

Table 1. Reserved substitution definition

Job attributes

Job attributes are elements to need to reserve resources in advance, allocate resources, and execute job, including executable attributes, job type attribute, file staging attributes, job termination attributes, clean up attributes and co-allocation attributes. Executable attributes are elements for executing a job, including executable, arguments, working directory, environment, standard input, output, and error, and library path. Job type attribute is element to specify the kind of job. File staging attributes are elements for staging in files to execute a job and staging out. Job termination attributes are elements to specify start and termination time of a job to reserve the resources in advance. Co-allocation attributes are related to local resource allocation including local resource addresses, job manager type and CPU count.
Basically, a user might want to automatically determine resources by meta-scheduler as illustrated in Figure 1. However If user wants to determine resources manually, co-allocation attributes must be used as showed in Figure 4.

<jrdl …>
   <job>

     <executable> … </executable>
     …

<subjob>
   <resourceManagerContact>          Co-allocation attributes
     <string>
       <stringElement value=
         "http://eros01.gridcenter.or.kr:8080"/>
     </string>
   </resourceManagerContact>
   <jobManagerType>
     <enumeration>
       <enumerationValue><pbs/></enumerationValue>
     </enumeration>
   </jobManagerType>
   <count>
     <integer value="2"/>
   </count>

        …
      </subjob>
      <subjob> … </subjob>

</job>
</jrdl>

Figure 4. Co-allocation attributes to specify local resource to submit a job

When a job is submitted to resources, the job should be divided into several subjobs, each of which might has different job attributes. Therefore, you can specify different job attributes for each subjob as illustrated in Figure 5. In this job, there are two subjobs, which have different executable file. One is “a.out,” and the other is “b.out”.

Figure 5. Job containing different job attributes for a subjob

Executable attributes

Job executable attribute

The source of executable file is local or remote file. If executable file is a form of GASS-compatible URL like “gsiftp://ip/path/file”, executable file could be staged from the GASS-compatible file server.
An element in your document might look like this:

Job arguments attribute

An element in your document might look like this:

Job directory attribute

An element in your document might look like this:

Environment attribute

An element in your document might look like this:

Standard input attribute

The source of standard input file is STDIN. STDIN is local or remote file. If STDIN is a form of GASS-compatible URL, STDIN could be received from the GASS-compatible file server. An element in your document might look like this:

Standard output attribute

The destination of standard output file is STDOUT. Standard output file might have multiple destinations and each destination is local or remote file.
An element in your document might look like this:

Standard error attribute

The destination of standard error file is STDERR. Standard error file might have multiple destinations and each destination is local or remote file.
An element in your document might look like this:

Library path attribute

Job might need to specify library paths. Each library path must be local path.
An element in your document might look like this:

Job type attributes

Job type attribute

Job type attribute is element to specify the kind of job. JRDL has three kinds of job type: single, xmpi, and htc.
An element in your document might look like this:

File staging attributes

File staging in attribute

You can specify a list of ("remote URL" "local file") pairs which indicate files to be staged into the cache. Symbolic link from the cache to the "local file" path will be made.
An element in your document might look like this:

File staging out attribute

You can specify a list of ("local file" "remote URL") pairs which indicate files to be staged from the job to a GASS-compatible file server.
An element in your document might look like this:

Job termination attributes

These attributes are attributes related to the start time and end time of a job. Job start time and Job start before are mutually exclusive. Therefore, if you want to specify start time of a job, you have to determine between Job start time and Job start before.

Job start time

If a job should start at a specific time, you should describe a time by Job start time. The dateTime data type is used to specify a date and a time.
The dateTime is specified in the following form "YYYY-MM-DDThh:mm:ss" where:

YYYY indicates the year

MM indicates the month

DD indicates the day

T indicates the start of the required time section

hh indicates the hour

mm indicates the minute

ss indicates the second

Note: All components are required.
To specify a time zone, you can either enter a dateTime in UTC time by adding a "Z" behind the time.
An element in your document might look like this:

Job start before

If a job should start before a specific duration, you should describe a time interval by Job start before. The duration data type is used to specify a time interval.
The time interval is specified in the following form "PnYnMnDTnHnMnS" where:

P indicates the period (required)

nY indicates the number of years

nM indicates the number of months

nD indicates the number of days

T indicates the start of a time section (required if you are going to specify
hours, minutes, or seconds)

nH indicates the number of hours

nM indicates the number of minutes

nS indicates the number of seconds

An element in your document might look like this:

Job duration

If a job should execute during a specific interval, you should describe a time interval by Job duration. The duration data type is used to specify a time interval.
An element in your document might look like this:

Clean up attributes

Clean up attribute

This attribute specifies files to clean up after a job has completed.
An element in your document might look like this:

Co-allocation attributes

If you want to manually determine the resources, you should specify co-allocation attributes.

Resource manager contact attribute

You should specify the contact address of RMS.
An element in your document might look like this:

Job manager attribute

You should specify the type of local job manager. We are supporting two kinds of type: fork and pbs.
An element in your document might look like this:

Count attribute

You should specify the count of CPU to be allocated at RMS.
An element in your document might look like this:

Resource attributes

The resource attributes are user preference to determine resources to submit a job, including total CPU count, OS type, processor type, size and availability of memory and storage, free CPU, processor load, local job manager type, and scheduler plug-in type.

Count attribute

You should specify the total count of CPU to be allocated to submit a job.
An element in your document might look like this:

Operating system attribute

You might need to describe the specific name of operating system of nodes to be selected in resource pools.
An element in your document might look like this:

Processor attribute

You might need to describe the minimum clock speed of processor of nodes to be selected in resource pools. The unit of clock speed is megahertz.
An element in your document might look like this:

Minimum main memory attributes

Resource manager contact attribute

You should specify the contact address of RMS.
An element in your document might look like this:

Minimum RAM size attribute

You might need to describe the minimum RAM size of nodes to be selected in resource pools. The unit of RAM size is mega byte.
An element in your document might look like this:

Minimum storage attribute

You might need to describe the minimum available storage size of nodes to be selected in resource pools. The unit of storage size is mega byte.
An element in your document might look like this:

Minimum free CPU attribute

You might need to describe the minimum available CPU number of nodes to be selected in resource pools.
An element in your document might look like this:

Processor load attributes

last15Min attribute

You might need to describe the average processor availability during last 15 minutes of nodes to be selected in resource pools. The average processor availability is expressed as a percentage.
An element in your document might look like this:

last5Min attribute

You might need to describe the average processor availability during last 5 minutes of nodes to be selected in resource pools. The average processor availability is expressed as a percentage.
An element in your document might look like this:

last1Min attribute

You might need to describe the average processor availability during last 1 minute of nodes to be selected in resource pools. The average processor availability is expressed as a percentage.
An element in your document might look like this:

Scheduling type attribute

You might need to describe the type of local job scheduler of nodes to be selected in resource pools. The type of scheduler must be either fork or scheduler.
An element in your document might look like this:

Scheduling plug-in attribute

You might need to describe the name of scheduling plug-in to be selected in resource pools. The name of plug-in must be either Default or user defined name. The user defined name could be referenced by querying to the GAIS.
An element in your document might look like this: