The UNICORE/X server is the central component of a UNICORE site. It hosts the services such as job submission, job management, storage access, and provides the bridge to the functionality of the target resources, e.g. batch systems or file systems.

For more information about UNICORE visit http://www.unicore.eu.

Getting started

Prerequisites

To run UNICORE/X, you need the SUN or OpenJDK Java 6 (JRE or SDK). If not installed on your system, you can download it from http://java.oracle.com

UNICORE/X has been most extensively tested on Linux-like systems, but runs on Windows and MacOS as well.

Please note that

  • to integrate into secure production environments, you will need access to a certificate authority and generate certificates for all your UNICORE servers.

  • to interface with a resource management system like SGE or Torque, you need to install and configure the UNICORE TSI.

  • to make your resources accessible outside of your firewalls, you should setup and configure a UNICORE Gateway.

All these configuration options will be explained in the manual below.

Installation

UNICORE/X can be installed from either a tar.gz or zip archive, or (on Linux) from rpm/deb packages.

To install from the tar.gz or zip archive, unpack the archive in a directory of your choice. You should then review the config files in the conf/ directory, and adapt paths, hostname and ports. The config files are commented, and you can also check the configuration guide

To install from a Linux package, please use the package manager of your system to install the archive.

Note

Using the Linux packages, you can install only a single UNICORE/X instance per machine (without manual changes). The tar.gz / zip archives are self contained, and you can easily install multiple servers per machine.

The following table gives an overview of the file locations for both tar.gz and Linux bundles.

Table 1. Directory Layout
tar.gz, zip rpm Description

<basedir>/conf/

/etc/unicore/unicorex

Config files

<basedir>/lib/

/usr/share/unicore/unicorex/lib

Java libraries

<basedir>/log/

/var/log/unicore/unicorex/

Log files

<basedir>/bin/

/usr/sbin/

Start/stop scripts

 — 

/etc/init.d/unicore-unicorex

Init script

Starting/Stopping

There are two scripts that expect to be run from the installation directory. To start, do

cd <inst>
bin/start.sh

To stop:

cd <inst>
bin/stop.sh

Overview of main configuration options

UNICORE/X is a fairly complex software which has many interfaces to other UNICORE components and configuration options. This section tries to give an overview of what can and should be configured. The detailed configuration guide follows in the next chapters

Mandatory configuration

  • Certificates and basic security: UNICORE uses X.509 certificates for all servers. For UNICORE/X these are configured in the wsrflite.xml config file

  • Attribute sources: to map clients (i.e. X.509 certificates) to local attributes such as user name, groups and role, various attribute sources are available. For details, consult the attribute sources section.

  • Backend / target system access: to access a resource manager like SGE or Torque, the UNICORE TSI needs to be installed and UNICORE/X needs to be configured accordingly. Please consult the XNJS section.

Configuration of UNICORE/X

UNICORE/X has several sub-components. These are configured using several config files residing in the config directory, see the installation guide for the location of the config directory.

Config file overview

The following table indicates the main configuration files. Depending on configuration and installed extensions, some of these files may not be present, or more files may be present.

UNICORE/X watches some most configuration files for changes, and tries to reconfigure if they are modified, at least where possible. This is indicated in the "dynamically reloaded" column. are indicated.

Table 2. UNICORE/X configuration files
config file usage dynamically reloaded

uas.config

General settings, startup behaviour, storages, AIP setup

yes

wsrflite.xml

Services to be deployed, SSL settings, Web server settings

yes

simpleidb

Backend, installed applications, resources

yes

xnjs.xml

Back end properties

no

xnjs_legacy.xml

Back end properties preconfigured for the Perl TSI

no

logging.properties

logging levels, logfiles and their properties

yes

security_policy.xml

Access control policy for securing the web services

yes, via xacml.config (do touch xacml.config to trigger)

xacml.config

Configure the access control component

yes

vo.config

Configure the use of UVOS (optional attribute source)

simpleuudb

A file mapping user DNs to local attributes (optional attribute source)

jmxremote.password

Access control file for remote monitoring using the Java management extensions (JMX)

Settings for the UNICORE/X process

The properties controlling the Java virtual machine running the UNICORE/X process are configured in

  • UNIX: the "bin/start.sh" script

  • Windows: the "conf\\wrapper.conf" configuration file

These properties include settings for maximum memory, and also the properties for configuring JMX, see the admin guide for more on JMX.

General

Config file formats

UNICORE/X uses two different formats for configuration.

Java properties

  • Each property can be assigned a value using the syntax "name=value"

  • Please do not quote values, as the quotes will be interpreted as part of the value

  • Comment lines are started by the "#"

  • Multiline values are possible by ending lines with "\", e.g. 

      name=value1 \
      value2

In this example the value of the "name" property will be "value1 value2".

XML

Various XML dialects are being used, so please refer to the example files distributed with UNICORE for more information on the syntax. In general XML is a bit unfriendly to edit, and it is rather easy to introduce typos.

Note

It is advisable to run a tool such as xmllint after editing XML files to check for typos

Integration of UNICORE/X into a UNICORE infrastructure

Since UNICORE/X is the central component, it is interfaced to other parts of the UNICORE architecture, i.e. the Gateway, the Registry and the TSI.

Gateway

The gateway address is usually hard-coded into conf/wsrflite.xml, and on the gateway side there is an entry VSITE_NAME=address pointing to the UNICORE/X container. In some scenarios it’s convenient to auto-register with a gateway. This can be enabled using the following properties.

Table 3. Gateway settings
config file property name range of values description modifyable at runtime

wsrflite.xml

Base URL

host/port

the host/port of the gateway

(yes)

uas.config

uas.gatewayregistration

true or false

whether autoregistration should be enabled

no

uas.gatewayregistration.updateinterval

an integer

registration refresh interval in seconds

no

uas.security.consignor.checksignature

true or false

whether gateway assertions must be signed (see also security)

no

uas.onstartup.wait

true or false

whether UNICORE/X should check if the gateway is up during startup

no
Note

To use the autoregistration feature, the gateway configuration must be set up accordingly

Registry

It is possible to configure UNICORE/X to contact one or more external or "global" Registries in order to publish information on crucial services there. Most of the following properties deal with the automatic discovery and/or manual setup of the external registries being used.

Table 4. Registry settings
config file property name range of values description modifyable at runtime

uas.config

uas.externalregistry.use

"true", "false"

whether to publish service information in an external registry

yes

uas.externalregistry.url

a valid URL

use this URL for external registry if automatic discovery is switched off or fails

yes

uas.externalregistry.url.*

more valid URLs

additional registry URLs

yes

wsrflite.xml

unicore.wsrflite.sg.defaulttermtime

a long value

alive-check interval for registry entries in seconds ; an external registry will enforce its own value

yes

uas.externalregistry.autodiscover

"true", "false"

if set to "true", try to autodiscover the external registry via UDP multicast

yes

Startup code

In order to provide a flexible initialization process for the UAS, we introduce a property named "uas.onstartup", which is defined in the file "uas.config". The value string of this property consists of a whitespace separated list of java classes which must be implementing the "Runnable" interface. Many extensions for UNICORE/X rely on an entry in this property to initialise themselves.

Table 5. Startup code
class name description usage

de.fzj.unicore.uas.util.DefaultOnStartup

initialises the job management system and the "local" registry; should usually be run on startup

normal UNICORE/X servers

de.fzj.unicore.bes.util.BESOnStartup

initialises the OGSA-BES job management system

UNICORE/X servers that expose BES services

de.fzj.unicore.cisprovider.impl.InitOnStartup

sets up the CIS infoprovider

UNICORE/X servers that want to provide information in GLUE2 format or want to be visible in the CIS

com.intel.gpe.gridbeans.PublishGridBeanService

if available, publishes the GridBeanService to the registry

UNICORE/X servers that host a Gridbean service

de.fzj.unicore.uas.util.CreateSMSOnStartup

creates and deploys a single instance of the SMS that is shared between users, named default_storage

if a shared storage is required

Security

Overview

Security is a complex issue, and many options exist. On a high level, the following items need to be configured.

  • SSL setup (keystore and truststore settings for securing the basic communication between components)

  • Attribute sources configuration (which will map Grid users to local properties such as role, Unix login and groups)

  • Access control setup (controlling in detail who can do what on which services)

  • Message level security (message signatures)

SSL configuration

Here you configure the server identity and the certificates of other services that want to contact this server.

Table 6. SSL configuration
config file property name range of values description

wsrflite.xml

unicore.wsrflite.ssl.keystore

Name of keystore file

The keystore must contain at least one private/public keypair

unicore.wsrflite.ssl.keytype

PKCS12, JKS

Keystore type

unicore.wsrflite.ssl.keypass

Keystore password

unicore.wsrflite.ssl.keyalias

Alias of the key to use

unicore.wsrflite.ssl.clientauth

"true" or "false"

Whether to require client-authentication

unicore.wsrflite.ssl.truststore

Name of truststore file

The truststore contains certificates that are trusted by the server

unicore.wsrflite.ssl.truststoretype

PKCS12, JKS

unicore.wsrflite.ssl.truststorepass

Truststore password

Attribute sources configuration

Attribute sources provide information about which local role and properties a Grid user has. UNICORE knows several attribute sources which can be combined using various combining algorithms. These are configured in the uas.config file. Due to the complexity, the description of the configuration options can be found in a separate chapter.

Access control configuration

Access control works by checking a Grid user’s attributes (obtained from the attribute sources) against a set of policies. Again, several options exist, which are described in a separate chapter.

Message signatures

UNICORE/X will require important messages (like job submissions or file exports). The property controlling this is - uas.security.signatures If set to true, signatures are required.

Configuring the XNJS and TSI

Information on the configuration of the XNJS and TSI backend can be found here.

Configuring storages on TargetSystem instances

Each TargetSystem instance can have one or more storages attached to it. Usually, only the HOME storage is created, which allows users access their home directory on the target system. You can add storages easily, using configuration entries in uas.config.

Table 7. Additional storages
config file property name range of values description

uas.config

uas.targetsystem.storage.N

String

n/a

disambiguates several configuration entry sets

uas.targetsystem.storage.N.type

FIXEDPATH, VARIABLE or CUSTOM

FIXEDPATH: mapped to a fixed directory, VARIABLE: resolved using a lookup, CUSTOM: specified class is used

uas.targetsystem.storage.N.path

String

Denotes a path or the name of an environment variable (depending on the type)

uas.targetsystem.storage.N.class

Java class name

Here, "N" stands for an identifier (e.g. 1,2, 3, …) to distinguish the storages. For example, to configure two storages, one named TEMP pointing to "/tmp" and the other named DEISA_HOME pointing to "$DEISA_HOME", you would add the following configuration entries in uas.config:

uas.targetsystem.storage.1=TEMP
uas.targetsystem.storage.1.type=FIXEDPATH
uas.targetsystem.storage.1.path=/tmp

uas.targetsystem.storage.2=DEISA_HOME
uas.targetsystem.storage.2.type=VARIABLE
uas.targetsystem.storage.2.path=$DEISA_HOMES

# example for a custom SMS implementation (e.g. for Hadoop or iRODS)
uas.targetsystem.storage.3=IRODS
uas.targetsystem.storage.3.type=CUSTOM
uas.targetsystem.storage.3.path=/
uas.targetsystem.storage.3.class=my.custom.sms.ImplementationClass

Disabling "Home"

If you want to disable the default "Home" storage, you can set the following property in uas.config:

uas.targetsystem.home.disable=true

Configuring the StorageFactory service

The StorageFactory service allows clients to dynamically create storage instances. These can have different types, for example you could have storages on a normal filesystem, and other storages on an Apache Hadoop cluster.

The basic property controls which storage types are supported

uas.storagefactory.types=TYPE1 TYPE2 ...

Each supported storage type is configured using a set of properties

uas.storagefactory.TYPE1.description=GPFS file system
uas.storagefactory.TYPE1.fixedpath=GPFS file system
uas.storagefactory.TYPE1.path=/mnt/gpfs/unicore/unicorex-1/storage-factory

# if this is set to true, the directory corresponding to a storage instance will
# be deleted when the instance is destroyed. Defaults to "true"
uas.storagefactory.DEFAULT.cleanup=true

The "path" parameter determines the base directory used for the storage instances (i.e. on the backend), and the unique ID of the storage will be appended automatically.

The "cleanup" parameter controls whether the storage directory will be deleted when the storage is destroyed.

If you have a custom storage type, an additional "class" parameter defines the Java class name to use. For example:

uas.storagefactory.TYPE1.type=CUSTOM
uas.storagefactory.TYPE1.class=de.fzj.unicore.uas.hadoop.SMSHadoopImpl

HTTP proxy, timeout and web server settings

The UNICORE Services Environment container has a number of settings related to the web server and to the HTTPClient library used for outgoing HTTP(s) calls. These are shown in the following two tables.

Table 8. Web server options
property name range of values default value description

unicore.wsrflite.jetty.maxThreads

Integer

255

Maximum number of threads for Jetty

unicore.wsrflite.jetty.minThreads

Integer

1

Minimum number of threads

unicore.wsrflite.jetty.maxIdleTime

Integer

1000

Milliseconds before an idle connection will be timed out

unicore.wsrflite.jetty.lowThreads

Integer

50

If the number of free threads is below this value, idle connections will be timed out quicker

unicore.wsrflite.jetty.lowResourceMaxIdleTime

Integer

100

under "low resource" condition, milliseconds before an idle connection will be timed out

unicore.wsrflite.jetty.maxIdleTime

Integer

1000

Milliseconds before an idle connection will be timed out

unicore.wsrflite.jetty.gzip.minGzipSize

Integer

65535

The size of the largest data chunk that will not be compressed (if the client supports gzip)

unicore.wsrflite.jetty.gzip.bufferSize

Integer

8192

Buffer size used for gzip compression
Table 9. Outgoing HTTP call options
property name range of values default value description

http.connection.timeout

Integer

10000

Socket connection timeout in millis

http.timeout

Integer

10000

Socket read timeout in millis

http.proxyHost

String

HTTP proxy host

http.proxyPort

Integer

HTTP proxy port

http.proxy.user

String

Proxy server user

http.proxy.password

String

Proxy server password

http.nonProxyHosts

String

Space separated list of host name fragments which are not proxied

Configuring attribute sources

The authorization process in UNICORE/X requires that each user (identified by an X.509 certificate or just the DN) is assigned some <attributes> such as her <role>.

Several attribute sources are available, that can even be combined for maximum flexibility and administrative control.

To configure the attribute sources, the uas.security.attributes.order property in the uas.config file is used. This is a space-separated list with attribute sources names, where the named attribute sources will be queried one after the other, allowing you to query multiple attribute sources, override values etc.

A second property, uas.security.attributes.combiningPolicy, allows you to control how attributes from different sources are combined.

For example, the following configuration snippet

#
# Authorisation attribute source configuration
#
uas.security.attributes.order=XUUDB FILE

#
# Combining policy
#
# MERGE_LAST_OVERRIDES (default), FIRST_APPLICABLE, FIRST_ACCESSIBLE or  MERGE
uas.security.attributes.combiningPolicy=MERGE_LAST_OVERRIDES

will declare two attribute sources, "XUUDB" and "FILE", which should be both queried and combined using the MERGE_LAST_OVERRIDES policy.

Since all attribute sources will be queried, it has to be defined how attributes will be combined. For example, assume you have both XUUDB and FILE, and both return a xlogin attribute for a certain user, say "xlogin_1" and "xlogin_2".

The different combining policies are

  • MERGE_LAST_OVERRIDES : new attributes override those from previous sources. In our example, the result would be "xlogin_2".

  • FIRST_APPLICABLE : the attributes from the first source that returned a non empty list of attributes are used. In our case this would be "xlogin_1". If there were no xlogin attribute for the user in XUUDB then "xlogin_2" would be returned.

  • FIRST_ACCESSIBLE : the attributes from the first source that is accessible are used. In our case this would be "xlogin_1". This policy is useful for redundant attribute sources. E.g. you can configure two instances of XUUDB with the same users data; the 2nd one will be tried only if the first one is down.

  • MERGE : attributes are merged. In our example, the result would be "xlogin_1, xlogin_2", and the user would be able to choose between them.

Each of the sources needs a mandatory configuration option defining the Java class, and several optional properties that configure the attribute source. In our example, one would need to configure both the "XUUDB" and the "FILE" source:

uas.security.attributes.XUUDB.class=...
uas.security.attributes.XUUDB.xuudbHost=...
...

uas.security.attributes.FILE.class=...
uas.security.attributes.FILE.file=...
...

Additionally you can mix several combining policies together, see "Chained attribute source" below for details.

XUUDB

The XUUDB is the standard option in UNICORE. It has the following features

  • Web service interface for querying and administration. It is suitable for serving data for multiple clients. Usually it is deployed to handle attributes for a whole grid site.

  • Access protected by client-authenticated SSL

  • Supports the xlogin, role and project attributes (where project maps to Unix groups)

  • Multiple xlogins per certificate or DN, where the user can select one

  • Entries are grouped using the so-called Grid component ID (GCID). This makes it easy to assign users different attributes when accessing different UNICORE/X servers.

To enable and configure the XUUDB, set the following properties in uas.config.

uas.security.attributes.order=... XUUDB ...
uas.security.attributes.XUUDB.class=eu.unicore.uas.security.XUUDBAuthoriser
uas.security.attributes.XUUDB.xuudbHost=https://<xuudbhost>
uas.security.attributes.XUUDB.xuudbPort=<xuudbport>
uas.security.attributes.XUUDB.xuudbGCID=<your_gcid>

Full XUUDB documentation is available from http.://www.unicore.eu/documentation/manuals/xuudb

UVOS

The UNICORE Virtual Organisation Service (UVOS) is a powerful tool for managing access based on the concept of virtual organisations. A detailed description and configuration guidance can be obtained from http://www.unicore.eu/documentation/manuals/unicore6/uas-vo

File attribute source

In simple circumstances, or as an addition to a XUUDB or UVOS, the file attribute source can be used. As the name implies a simple map file is used to map DNs to xlogin, role and other attributes. It is useful when you don’t want to setup an additional service (XUUDB or UVOS) and also when you want to locally override attributes for selected users (e.g. to ban somebody).

To use, set

uas.security.attributes.order=... FILE ...
uas.security.attributes.FILE.class=eu.unicore.uas.security.file.FileAttributeSource
uas.security.attributes.FILE.file=<your map file>
uas.security.attributes.FILE.matching=<strict|regexp>

The map file itself has the following format:

<?xml version="1.0" encoding="UTF-8"?>
<fileAttributeSource>
   <entry key="USER DN">
      <attribute name="role">
         <value>user</value>
      </attribute>
      <attribute name="xlogin">
         <value>unixuser</value>
         <value>nobody</value>
         ...
      </attribute>
      ...
   </entry>
   ...
</fileAttributeSource>

You can add an arbitrary number of attributes and attribute values.

The matching option controls how a client’s DN is mapped to a file entry. In strict mode, the canonical representation of the key is compared with the canonical representation of the argument. In regexp mode the key is considered a Java regular expression and the argument is matched with it. When constructing regular expressions a special care must be taken to construct the regular expression from the canonical representation of the DN. The canonical representation is defined here. (but you don’t have to perform the two last upper/lower case operations). In 90% of all cases (no multiple attributes in one RDN, no special characters, no uncommon attributes) it just means that you should remove extra spaces between RDNs.

The evaluation is simplistic: the first entry matching the client is used (which is important when you use regular expressions).

The attributes file is automatically refreshed after any change, before a subsequent read. If the syntax is wrong then an error message is logged and the old version is used.

Recognized attribute names are:

  • xlogin

  • role

  • group

  • supplementaryGroups

  • addOsGroups

  • queue

Attributes with those names (case insensitive) are handled as special UNICORE attributes (e.g. the xlogin is used to provide available local OS user names for the client). You can define other attributes but those will be ignored.

Chained attribute source

Chained attribute source is a meta source which allows you to mix different combining policies together. It is configured as other attribute sources with two parameters (except of its class): order and combiningPolicy. The result of the chain attribute source is the set of attributes returned by the configured chain.

Let’s consider the following example situation where we want to configure two redundant UVOS servers (both serving the same data) to achieve high availability. Additionally we want to override settings for some users using a local file attribute source (e.g. to ban selected users).

# The main chain configuration:
uas.security.attributes.order=UVOS_CLUSTER FILE
uas.security.attributes.combiningPolicy=MERGE_LAST_OVERRIDES

# The FILE source cfg:
uas.security.attributes.FILE.class=eu.unicore.uas.security.file.FileBasedAuthoriser
uas.security.attributes.FILE.file=<your map file>

# The UVOS_CLUSTER is a sub chain:
uas.security.attributes.UVOS_CLUSTER.class=de.fzj.unicore.uas.security.util.AttributeSourcesChain
uas.security.attributes.UVOS_CLUSTER.order=UVOS1 UVOS2
uas.security.attributes.UVOS_CLUSTER.combiningPolicy=FIRST_ACCESSIBLE

# And configuration of the two real sources used in the sub chain:
uas.security.attributes.UVOS1.class=...
...
uas.security.attributes.UVOS2.class=...
...

Configuring the XNJS

The XNJS is the UNICORE/X component that deals with the actual job execution and file access. It is configured using an XML file named xnjs.xml or xnjs_legacy.xml. The actual file that is used is set in the uas.config property uas.targetsystemfactory.xnjs.configfile.

#
# in uas.config
#
uas.targetsystemfactory.xnjs.configfile=conf/xnjs.xml

Here’s an overview of the most important properties that can be set in this file.

Table 10. Main XNJS properties
config file property name range of values default value description

xnjs.xml

XNJS.filespace

an absolute path on the target system’s filesystem

"data/FILESPACE"

the directory on the target system where job directories will be created

XNJS.statedir

a path on the UNICORE/X machine

"data/NJSSTATE"

the directory on the UNICORE/X machine where the XNJS keeps its state

XNJS.idbfile

a file or directory name

"conf/simpleidb"

the IDB containing application definitions etc.

XNJS.numberofworkers

an integer

"4"

the number of worker threads used to process jobs

Most of the other settings in this file are used to configure the internals of the XNJS and should usually be left at their default values.

The UNICORE TSI

This section describes installation and usage of the UNICORE TSI. This is a mandatory step if you want to interface to batch systems such as Torque, SGE, or LoadLeveller to efficiently use a compute cluster.

Note

Without this component, all jobs will run on the UNICORE/X server, under the user id that started UNICORE/X.

In a nutshell, you have to perform the following steps

  • Install the UNICORE TSI on your cluster front end node

  • Edit the tsi.properties file

  • On the UNICORE/X server, edit uas.config, simpleidb and xnjs_legacy.xml

  • Start the newly installed TSI (as root in a multiuser setting)

  • Restart UNICORE/X

Installation of the correct TSI

The TSI is a set of perl modules that is running on the target system. In case of a cluster system, you’ll need to install it on the frontend machine(s), i.e. the machine from where your jobs are submitted to the batch system. There are different variants available for the different batch systems such as Torque or SGE.

Usually installation and start of the TSI will be performed as the root user. The TSI will then be able to change to the current Grid user’s id for performing work (Note: nothing will ever be executed as "root"). You can also use a normal user, but then all commands will be executed under this user’s id.

  • First, download and install the UNICORE TSI package. The UNICORE core server bundle ("quickstart" package) includes the TSI in the tsi subdirectory. You should copy this folder to the correct machine first. In the following this will be denoted by <tsidir>

  • Install the correct TSI variant by 

 cd <tsidir>
  ./Install.sh

When prompted for the path, choose an appropriate on, denoted <your_tsi> in the following

  • Check the tsi file in 

 <tsidir>/<your_tsi>/perl/tsi

especially command locations, path settings etc.

  • set permissions using 

  cd <tsidir>
   ./Install_permissions.sh

  • MAKE A NOTE of the exact location of the tsi_ls and tsi_df files <tsidir>/<your_tsi>/tsi_ls and <tsidir>/<your_tsi>/tsi_df

Required TSI Configuration

Configuration is done by editing <tsi_dir>/conf/tsi.properties The following settings are needed:

#path to your tsi installation
tsi.path=<tsi_dir>/<your_tsi>

#UNICORE/X machine
tsi.njs_machine=<UNICORE/X host>

#UNICORE/X listener port (check unicorex/conf/xnjs_legacy.xml variable "CLASSICTSI.replyport"
tsi.njs_port=7654

#TSI listener port (check unicorex/conf/xnjs_legacy.xml variable "CLASSICTSI.port"
tsi.my_port=4433

UNICORE/X configuration

Edit unicorex/conf/uas.config and set the variable

uas.targetsystemfactory.xnjs.configfile=conf/xnjs_legacy.xml

Edit unicorex/conf/xnjs_legacy.xml. Check the filespace location, this is where the local job directories will be created. On a cluster, these have to be on a shared part of the filesystem.

Check the CLASSICTSI related properties. Set the correct value for the machine and the ports (these can usually be left at their default values)

Set the value of CLASSICTSI.TSI_LS to the path of tsi_ls as noted above.

Set the value of CLASSICTSI.TSI_DF to the path of tsi_df as noted above.

Here is an example section for the classic TSI properties.

    <eng:Property name="XNJS.tsiclass" value="de.fzj.unicore.xnjs.legacy.LegacyTSI"/>
    <!-- TSI machine and ports used -->
    <eng:Property name="CLASSICTSI.machine" value="localhost"/>
    <eng:Property name="CLASSICTSI.port" value="4433"/>
    <eng:Property name="CLASSICTSI.replyport" value="7654"/>
    <!-- location of the tsi_ls file -->
    <eng:Property name="CLASSICTSI.TSI_LS" value="tsi/tsi_ls"/>
    <!-- location of the tsi_df file -->
    <eng:Property name="CLASSICTSI.TSI_DF" value="tsi/tsi_df"/>
    <!-- commands on the target system -->
    <eng:Property name="CLASSICTSI.CP" value="/bin/cp"/>
    <eng:Property name="CLASSICTSI.RM" value="/bin/rm"/>
    <eng:Property name="CLASSICTSI.MV" value="/bin/mv"/>
    <eng:Property name="CLASSICTSI.MKDIR" value="/bin/mkdir -p"/>
    <eng:Property name="CLASSICTSI.CHMOD" value="/bin/chmod"/>
    <eng:Property name="CLASSICTSI.MKFIFO" value="/usr/bin/mkfifo"/>
    <eng:Property name="CLASSICTSI.PERL" value="/usr/bin/perl"/>
    <!-- interval between updates of job stati (milliseconds) -->
    <eng:Property name="CLASSICTSI.statusupdate.interval" value="5000"/>
    <!-- how often the XNJS will re-try to get the status of a job
         in case the job is not listed in the status listing -->
    <eng:Property name="CLASSICTSI.statusupdate.grace" value="0"/>
    <!-- a user that is allowed to see all jobs on the batch system -->
    <eng:Property name="CLASSICTSI.priveduser" value="someuser"/>
    <!-- I/O buffer size, will greatly impact filetransfer performance -->
    <eng:Property name="CLASSICTSI.BUFFERSIZE" value="1000000"/>

Additional parameters

Some additional parameters exist for tuning the XNJS-TSI communication.

Table 11. XNJS-TSI communication settings
property name range of values default value description

CLASSICTSI.BUFFERSIZE

integer

1000000

Buffersize for filetransfers in bytes

CLASSICTSI.socket.timeout

integer

300000

Socket timeout in milliseconds

CLASSICTSI.socket.connect.timeout

integer

10000

Connection timeout in milliseconds

Tuning the batch system settings

UNICORE uses the normal batch system commands (e.g. qstat) to get the status of running jobs. There is a special case if a job is not listed in the qstat output. UNICORE will then assume the job is finished. However, in some cases this is not true, and UNICORE will have a wrong job status. To work around, there is a special property

    <!-- how often the XNJS will re-try to get the status of a job
         in case the job is not listed in the status listing -->
    <eng:Property name="CLASSICTSI.statusupdate.grace" value="2"/>

If the value is larger than zero, UNICORE will re-try to get the job status.

Start the TSI using (as root in a multiuser environment)

cd <tsi_dir>/conf
 ../bin/start_tsi

(or use the unicore-tsi init script if available in your installation)

Finally, restart the UNICORE/X server

Note

When changing TSIs, it’s a good idea to remove the UNICORE/X state and any files before restarting. In the quickstart configuration, this is done using See the section on persistence for details

Enabling SSL for the XNJS to TSI communication

The UNICORE/X server can be set up to use SSL for communicating with the Perl TSI. On the UNICORE/X side, this is very simple to switch on. In the XNJS config file, add a line to the "Core" section:

    <!-- enable SSL using the normal UNICORE/X key and trusted certificates -->
    <eng:LoadComponent>de.fzj.unicore.uas.xnjs.XNJSSecurityConficuration</eng:LoadComponent>

On the TSI side it is a bit more complex, and you need to have the TSI from the 6.3.0 distribution or later installed. First of all, your Perl installation must include the module "IO::Socket:SSL" and its dependencies. If you do not have it, you can get it from the CPAN archive.

In the tsi.properties configuration file, you set the keystore and truststore to be used:

# SSL parameters
# Keystore must contain the private TSI key and certificate
# Trustore must contain the certificate of the CA
tsi.keystore=/certs/keystore.pem
tsi.keypass=yourpassword
tsi.truststore=/certs/keystore.pem

Both keystore and truststore are in pem format.

The IDB

The UNICORE IDB (incarnation database) contains information on how abstract job definitions are to be mapped onto real executables. This process (called "incarnation") is performed by the XNJS component. The second IDB function is advertising target system capabilities and allowing to check client resource requests against these.

The IDB is a (set of) XML files, which by default is called simpleidb.

For reference, the current XML schema for the IDB can be read from the SVN repository.

Applications

The most important functionality of the IDB is providing executables for abstract applications. An abstract application is given by name and version, whereas an executable application is given in terms of executable, arguments and environment variables.

Simple applications

Here is an example entry for the "Date" application on a UNIX system

   <idb:IDBApplication>
    <idb:ApplicationName>Date</idb:ApplicationName>
    <idb:ApplicationVersion>1.0</idb:ApplicationVersion>
    <jsdl:POSIXApplication xmlns:jsdl="http://schemas.ggf.org/jsdl/2005/11/jsdl-posix">
      <jsdl:Executable>/bin/date</jsdl:Executable>
    </jsdl:POSIXApplication>
  </idb:IDBApplication>

As can be seen, "Date" is simply mapped to "/bin/date".

Arguments

Command line arguments are specified using <Argument> tags:

   <idb:IDBApplication>
    <idb:ApplicationName>LS</idb:ApplicationName>
    <idb:ApplicationVersion>1.0</idb:ApplicationVersion>
    <jsdl:POSIXApplication xmlns:jsdl="http://schemas.ggf.org/jsdl/2005/11/jsdl-posix">
      <jsdl:Executable>/bin/ls</jsdl:Executable>
      <jsdl:Argument>-l</jsdl:Argument>
      <jsdl:Argument>-t</jsdl:Argument>
    </jsdl:POSIXApplication>
   </idb:IDBApplication>

This would result in a command line "/bin/ls -l -t".

Conditional Arguments

The job submission from a client usually contains environment variables to be set when running the application. It often happens that a certain argument should only be included if a corresponding environment variable is set. This can be achieved by using "conditional arguments" in the incarnation definition. Conditional arguments are indicated by a quastion mark "?" appended to the argument value:

  <idb:IDBApplication>
    <idb:ApplicationName>java</idb:ApplicationName>
    <idb:ApplicationVersion>1.5.0</idb:ApplicationVersion>
    <jsdl:POSIXApplication xmlns:jsdl="http://schemas.ggf.org/jsdl/2005/11/jsdl-posix">
      <jsdl:Executable>/usr/bin/java</jsdl:Executable>
      <jsdl:Argument>-cp$CLASSPATH?</jsdl:Argument>
      <!-- other args omitted for clarity -->
    </jsdl:POSIXApplication>
   </idb:IDBApplication>

Here, <jsdl:Argument>-cp$CLASSPATH?</jsdl:Argument> is an optional argument.

If a job submission now includes a Environment variable named CLASSPATH

  <jsdl:Environment name="CLASSPATH">myjar.jar</jsdl:Environment>

the incarnated commandline will be "/usr/bin/java -cp$CLASSPATH …", otherwise just "/usr/bin/java …".

This allows very flexible incarnations.

More

For more details about IDB application definitions, please consult the detailed application definition guide.

TargetSystemProperties

The TargetSystemProperties element contains information about a site’s available resources, as well as additional information that should be published to clients.

Textual information

Simple strings can be entered into the IDB which are then accessible client-side. This is very useful for conveying system-specifics to client code and also to users. These text-info strings are entered into the IDB as a subtag of the TargetSystemProperties tag

Here is an example

  <idb:TargetSystemProperties>

    <!-- text infos -->
    <idb:Info Name="Administator email">admin@site.org</idb:Info>

  </idb:TargetSystemProperties>

These pieces of information are accessible client side as part of the target system properties.

Resources

Resources of a target system are specified using JSDLs Resource tag. It allows to specify things like number of nodes, CPUtime (per CPU), CPUs per node, total number of CPUs, etc.

These capabilities are specified giving an exact value and a range, for example:

    <jsdl:Exact>3600</jsdl:Exact>
    <jsdl:Range>
     <jsdl:LowerBound>1</jsdl:LowerBound>
     <jsdl:UpperBound>86400</jsdl:UpperBound>
    </jsdl:Range>
  The Range gives upper and lower bounds, where as the Exact value is
interpreted as the DEFAULT, when the client does not request anything
specific. If the Exact value is specified, the resource is part of the
site's default resource set.
  There exist a number of standard settings. You may choose to not specify some of
them, if they do not make sense on your system. For example, some sites do not allow
the user to explicitely select nodes and processors per node, but only "total number of CPUs".

  • jsdl:IndividualCPUTime : The wall clock time.

  • jsdl:IndividualCPUCount : The number of CPUs per node

  • jsdl:IndividualPhysicalMemory : The amount of memory per node (in bytes)

  • jsdl:TotalResourceCount : The number of nodes.

  • jsdl:TotalCPUCount : The total number of CPUs.

"Total CPUs" vs. "Nodes and CPUs per node"

Users can specify the number of processors either as just "total number of CPUs", or they can give a value for both "nodes" and "CPUs per node". If both are given, the values containing more information (i.e. nodes + CPUs per node) are used.

Similarly, if the administrator specifies both possiblities with a default value in the IDB, the nodes + CPUs per node will have precedence.

Other types of resources

Most HPC sites have special settings that cannot be mapped to the generic JSDL elements shown in the previous section. Therefore UNICORE 6 includes a mechanism to allow sites to specify their own system settings and allow users to set these using the Grid middleware.

You can insert <ResourceSetting> elements into the Resources section, an example follows.

 <jsdl:Resources>
    <idb:ResourceSetting>
      <idb:Name>TasksPerNode</idb:Name>
      <idb:Description>The number of tasks per node. If larger than 32, the node will run in SMT mode.</idb:Description>
      <idb:Value xmlns:jsdl="http://schemas.ggf.org/jsdl/2005/11/jsdl">
        <jsdl:Exact>32</jsdl:Exact>
        <jsdl:Range>
        <jsdl:LowerBound>1</jsdl:LowerBound>
        <jsdl:UpperBound>64</jsdl:UpperBound>
       </jsdl:Range>
     </idb:Value>
    </idb:ResourceSetting>
  </jsdl:Resources>

The meaning of the <Value> element is exactly as for the usual JSDL resources.

Example Resources section

This example includes the elements defining capabilities, and some informational elements like CPUArchitecture and operating system info.

<idb:TargetSystemProperties>
    <jsdl:Resources xmlns:jsdl="http://schemas.ggf.org/jsdl/2005/11/jsdl">
      <jsdl:CPUArchitecture>
        <jsdl:CPUArchitectureName>x86</jsdl:CPUArchitectureName>
      </jsdl:CPUArchitecture>

        <!-- O/S -->
      <jsdl:OperatingSystem>
       <jsdl:OperatingSystemType>
         <jsdl:OperatingSystemName>LINUX</jsdl:OperatingSystemName>
       </jsdl:OperatingSystemType>
       <jsdl:OperatingSystemVersion>2.6.13</jsdl:OperatingSystemVersion>
       <jsdl:Description>A free UNIX clone</jsdl:Description>
      </jsdl:OperatingSystem>

      <!-- cpu time (per cpu) in seconds -->
      <jsdl:IndividualCPUTime>
        <jsdl:Exact>3600</jsdl:Exact>
        <jsdl:Range>
         <jsdl:LowerBound>1</jsdl:LowerBound>
         <jsdl:UpperBound>86400</jsdl:UpperBound>
        </jsdl:Range>
      </jsdl:IndividualCPUTime>

      <!-- Nodes -->
      <jsdl:TotalResourceCount>
        <jsdl:Exact>1.0</jsdl:Exact>
        <jsdl:Range>
         <jsdl:LowerBound>1.0</jsdl:LowerBound>
         <jsdl:UpperBound>16.0</jsdl:UpperBound>
        </jsdl:Range>
      </jsdl:TotalResourceCount>

      <!-- CPUs per node -->
      <jsdl:IndividualCPUCount>
        <jsdl:Exact>8.0</jsdl:Exact>
        <jsdl:Range>
         <jsdl:LowerBound>1.0</jsdl:LowerBound>
         <jsdl:UpperBound>8.0</jsdl:UpperBound>
        </jsdl:Range>
      </jsdl:IndividualCPUCount>

      <!-- total CPUs -->
      <jsdl:TotalCPUCount>
        <jsdl:Exact>8.0</jsdl:Exact>
        <jsdl:Range>
         <jsdl:LowerBound>1.0</jsdl:LowerBound>
         <jsdl:UpperBound>128.0</jsdl:UpperBound>
        </jsdl:Range>
      </jsdl:TotalCPUCount>

      <!-- Memory per node (bytes) -->
      <jsdl:IndividualPhysicalMemory>
        <jsdl:Exact>268435456</jsdl:Exact>
        <jsdl:Range>
         <jsdl:LowerBound>1024576</jsdl:LowerBound>
         <jsdl:UpperBound>1073741824</jsdl:UpperBound>
        </jsdl:Range>
      </jsdl:IndividualPhysicalMemory>

    </jsdl:Resources>
  </idb:TargetSystemProperties>

Script templates

If you need to modify the scripts that are generated by UNICORE/X and sent to the TSI, you can achieve this using two entries in the IDB.

<idb:IDB xmlns:idb="http://www.fz-juelich.de/unicore/xnjs/idb">

<!-- Templates -->
<idb:SubmitScriptTemplate>
#!/bin/sh
#COMMAND
#RESOURCES
#SCRIPT
</idb:SubmitScriptTemplate>

<idb:ExecuteScriptTemplate>
#!/bin/sh
#COMMAND
#RESOURCES
#SCRIPT
</idb:ExecuteScriptTemplate>

<!-- rest of IDB omitted -->

</idb:IDB>

The SubmitScriptTemplate is used for batch job submission, the ExecuteScriptTemplate is used for everything else (e.g. creating directories, resolving user’s home, etc)

UNICORE/X generates the TSI script as follows:

  • the "#COMMAND" entry will be replaced by the action for the TSI, e.g. "#TSI_SUBMIT".

  • the "#RESOURCES" will be replaced by the resource requirements, e.g. "#TSI_NODES=…"

  • the "#SCRIPT" is the user script

Modifying these templates can be used to perform special actions, such as loading modules, or changing the shell (but use something compatible to sh). For example, to add some special directory to the path for user scripts submitted in batch mode, you could use

<idb:IDB xmlns:idb="http://www.fz-juelich.de/unicore/xnjs/idb">

<!-- Templates -->
<idb:SubmitScriptTemplate>
#!/bin/sh
#COMMAND
#RESOURCES
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/openmpi-2.1/lib; export LD_LIBRARY_PATH
PATH=$PATH:/opt/openmpi-2.1/bin; export PATH
#SCRIPT
</idb:SubmitScriptTemplate>

<!-- rest of IDB omitted -->

</idb:IDB>

Properties

In the IDB file, XNJS properties can be specified, for example the command locations identified by property names starting with "CLASSICTSI."

<idb:IDB xmlns:idb="http://www.fz-juelich.de/unicore/xnjs/idb">
<!--- rest of IDB omitted -->
 <idb:Property name="..."
               value="..."/>
 </idb:IDB

More on the IDB Application definitions

Simple application definitions and application arguments have already been covered in the previous section. Here, more details are presented.

Pre and post-commands

Sometimes it is useful to be able to execute one or several commands before or after the execution of an application. For example, to add directories to the path, or perform some pre-processing. The IDB allows to specify these using the PreCommand and PostCommand tags.

For example

  <idb:IDBApplication>
    <idb:ApplicationName>java</idb:ApplicationName>
    <idb:ApplicationVersion>1.5.0</idb:ApplicationVersion>
    <jsdl:POSIXApplication xmlns:jsdl="http://schemas.ggf.org/jsdl/2005/11/jsdl-posix">
      <jsdl:Executable>/usr/bin/java</jsdl:Executable>
      <jsdl:Argument>-cp$CLASSPATH?</jsdl:Argument>
      <!-- other args omitted for clarity -->
    </jsdl:POSIXApplication>
    <idb:PreCommand>PATH=$PATH:/opt/myapp/bin ; export PATH</idb:PreCommand>
    <idb:PreCommand>/opt/example/aquire_license</idb:PreCommand>
    <idb:PostCommand>/opt/example/release_license</idb:PostCommand>
   </idb:IDBApplication>

These commands will be executed as part of the user’s job script.

Interactive execution when using a batch system

If an application should not be submitted to the batch system, but be run on the login node (i.e. interactively), a flag in the IDB can be set:

  <idb:IDBApplication>
    <idb:ApplicationName>SomeApp</idb:ApplicationName>
    <idb:ApplicationVersion>1.0</idb:ApplicationVersion>

    <!-- instructs TSI to run the application interactively -->
    <idb:PreferInteractive>true</idb:PreferInteractive>

    <jsdl:POSIXApplication xmlns:jsdl="http://schemas.ggf.org/jsdl/2005/11/jsdl-posix">
      <!-- other args omitted for clarity -->
    </jsdl:POSIXApplication>
   </idb:IDBApplication>
Note

This should only be used for very short-running tasks, since UNICORE cannot track the status of such a task. It is simply forked by the TSI, and UNICORE will just assume it is finished after a short while.

Application metadata (simple)

For client components it is very useful to have a description of an application in terms of its arguments. This allows for example the "Generic" GridBean in the UNICORE Rich client to automatically build a nice GUI for the application.

You can optionally attach metadata to an applications arguments.

  <jsdl:Argument Description="Verbose Execution"
                 Type="boolean"
                 ValidValues="true false"
                 DependsOn="..."
                 Excludes="..."
                 IsEnabled="false"
                 IsMandatory="false">+v$VERBOSE?</jsdl:Argument>

Some metadata is inferred automatically, such as the argument name (VERBOSE in the example above).

The meaning of the attributes should be fairly obvious.

  • the Description attribute contains a human-readable description of the argument

  • the Type attribute can have the values "string", "boolean", "int", "double", "filename" or "choice". In the case of "choice", the ValidValues attribute is used to specify the list of valid values. The type filename is used to specify that this is an input file for the application, allowing clients to enable special actions for this.

  • The MimeType attribute allows to specify the mime-types of an input or output file as a comma-separated list. This can be used by smart clients, for example to check the viability of workflows.

  • The ValidValues attribute is used to limit the range of valid values, depending on the Type of the argument. The processing of this attribute is client-dependent. The UNICORE Rich Client supports intervals for the numeric types, and Java regular expressions for the string types.

  • DependsOn and Excludes are space-separated lists of argument names to control dependencies. For example, a "VERBOSE and a "QUIET" attribute should exclude each other.

  • IsMandatory (values: true or false) specifies if a parameter MUST be provided.

  • IsEnabled (values: true or false) is intended to tell clients that the parameter should initially be enabled in the GUI.

Application metadata (complex)

You can also add metadata as XML to the IDB entry, which allows you to add your custom metadata:

The XML schema can be found online at http://unicore.svn.sourceforge.net/viewvc/unicore/jsdl-xmlbeans/trunk/src/main/schema/jsdl-unicore.xsd

Currently the XML metadata only encompass argument metadata, similar to the "simple" metadata described above. However, custom metadata can be added in case an application requires it.

Here is a simple example.

  <idb:IDBApplication>
    <idb:ApplicationName>SomeApp</idb:ApplicationName>
    <idb:ApplicationVersion>1.0</idb:ApplicationVersion>

    <!-- metadata -->
    <u6:Metadata xmlns:u6="http://www.unicore.eu/unicore/jsdl-extensions">
      <!-- example argument-->
      <u6:Argument>
        <u6:Name>PRECISION</u6:Name>
        <u6:ArgumentMetadata>
          <u6:Type>choice</u6:Type>
          <u6:Description>Precision of the computation</u6:Description>
          <u6:ValidValue>Lax</u6:ValidValue>
          <u6:ValidValue>Reasonable</u6:ValidValue>
          <u6:ValidValue>Precise</u6:ValidValue>
          <u6:ValidValue>Pedantic</u6:ValidValue>
          <u6:IsMandatory>true</u6:IsMandatory>
        </u6:ArgumentMetadata>
      </u6:Argument>
      <!-- any custom XML can be added as well -->
      <!-- ... -->
    </u6:Metadata>
   </idb:IDBApplication>

The XML supports the Type, Description, MimeType, IsMandatory, DependsOn, Excludes and ValidValue elements, with the same semantics as described above.

Execution Environments

Execution environments are an advanced feature that allows you to configure the way an executable is executed in a more detailed and user-friendly fashion. A common scenario is the configuration of an environment for parallel execution of a program, such as MPI.

A typical simple MPI invocation looks like this

/usr/local/bin/openmpi -np 4 ./my_mpi_program [my_program_arguments]

but of course there are many more possible arguments to the MPI command, which also depend on the local installation. By using a predefined execution environment, a UNICORE user need not know all the details, but can set up her job in a simple fashion.

This document covers the options that are available to configure execution environments in the IDB.

  • XML Schema for the execution environments: the current XML schema for the execution environment specification can be read from the SVN repository.

IDB definition of execution environments

The server-side setup of an execution environment is by adding an XML entry into the IDB. A simple environment might be used to run a user command using time. This example shows every possible option. You might want to consult the man page of time.

<idb:IDB xmlns:idb="http://www.fz-juelich.de/unicore/xnjs/idb">

<!-- sample execution environment definition in the IDB -->
  <ee:ExecutionEnvironment xmlns:ee="http://www.unicore.eu/unicore/jsdl-extensions">
    <ee:Name>TIME</ee:Name>
    <ee:Version>1.0</ee:Version>
    <ee:Description>Runs the user's command using the 'time' tool, measuring the used CPU time.</ee:Description>
    <ee:ExecutableName>/usr/bin/time</ee:ExecutableName>
    <ee:CommandlineTemplate>#EXECUTABLE #ARGS #USERCOMMAND #USERARGS</ee:CommandlineTemplate>
    <ee:Argument>
      <ee:Name>Output</ee:Name>
      <ee:IncarnatedValue>-o</ee:IncarnatedValue>
      <ee:ArgumentMetadata>
        <ee:Type>string</ee:Type>
        <ee:Description>Write the resource use statistics to a FILE instead of to the standard error stream</ee:Description>
      </ee:ArgumentMetadata>
    </ee:Argument>
    <ee:Option>
      <ee:Name>Verbose</ee:Name>
      <ee:IncarnatedValue>-v</ee:IncarnatedValue>
      <ee:OptionMetadata>
        <ee:Description>Enable verbose mode</ee:Description>
      </ee:OptionMetadata>
    </ee:Option>
    <ee:PreCommand>
      <ee:Name>PRINT_START_TIME</ee:Name>
      <ee:IncarnatedValue>echo "Started at $(date)"</ee:IncarnatedValue>
      <ee:OptionMetadata>
        <ee:Description>Explicitely print the start time</ee:Description>
      </ee:OptionMetadata>
    </ee:PreCommand>
    <ee:PostCommand>
      <ee:Name>PRINT_FINISH_TIME</ee:Name>
      <ee:IncarnatedValue>echo "Finished at $(date)"</ee:IncarnatedValue>
      <ee:OptionMetadata>
        <ee:Description>Explicitely print the finishing time</ee:Description>
      </ee:OptionMetadata>
    </ee:PostCommand>
   </ee:ExecutionEnvironment>

</idb:IDB>

If a client now submits a job including a request for the "TIME" execution environment, UNICORE will generate a shell script that wraps the user command in the "time" invocation. Let’s say the job request includes the "Output" argument, the "Verbose" option and both precommand and postcommand:

  <!-- sample execution environment request sent from client to server -->
  <ee:ExecutionEnvironment xmlns:ee="http://www.unicore.eu/unicore/jsdl-extensions">
    <ee:Name>TIME</ee:Name>
    <ee:Version>1.0</ee:Version>
    <ee:Argument>
      <ee:Name>Output</ee:Name>
      <ee:Value>time_profile</ee:IncarnatedValue>
    </ee:Argument>
    <ee:Option>
      <ee:Name>Verbose</ee:Name>
    </ee:Option>
    <ee:PreCommand>
      <ee:Name>PRINT_START_TIME</ee:Name>
    </ee:PreCommand>
    <ee:PostCommand>
      <ee:Name>PRINT_FINISH_TIME</ee:Name>
     </ee:PostCommand>
   </ee:ExecutionEnvironment>

The script generated by UNICORE will look like this (leaving out some standard things):

#!/bin/bash

# ...

echo "Started at $(date)"
/usr/bin/time -o time_profile -v /path/to/my_user_application
echo "Finished at $(date)"

# ...

In the following the various XML tags that are available are explained in detail.

  • ExecutableName : This is the name of the executable that "defines" the environment.

  • CommandlineTemplate : To control the exact commandline that is created, this template is used.

The default template is

#EXECUTABLE #ARGS #USERCOMMAND #USERARGS

where

  • #EXECUTABLE is the executable defined using ExecutableName

  • #ARGS are the arguments and options for the executable

  • #USERCOMMAND is the user’s executable

  • #USERARGS are the arguments to the user’s executable

  • Argument : the Argument elements are used to create arguments to the executable. They have several subtags.

  • Name is the name of the argument.

  • IncarnatedValue is the argument as used in the commandline.

  • ArgumentMetadata are described below.

  • ArgumentMetadata : This element allows to describe an Argument in more detail. It has the following subtags

  • Type the argument type. Valid values are "string", "boolean", "int", "float" and "choice"

  • Description is a human-friendly description

  • Default a possible default value

  • ValidValue tags are used to denote possible values

  • DependsOn denotes other arguments that this argument requires

  • Excludes denotes other arguments that clash with this argument

  • PreCommand : This tag denotes a command that is executed immediately before the actual executable. Its subtags are the same as for Option.

  • PostCommand : This tag denotes a command that is executed after the actual execution. Its subtags are the same as for PreCommand.

Additional features

{{{xnjs-tsi-custom.html}Site-specific resources}}  - Customising the TSI for special resource settings
{{{xnjs-dynamicincarnation.html}Dynamic incarnation}}  - How to influence incarnation and the TSI script dynamically at runtime