Bristol Windows XP Condor Pool

General Information

The WindowsXP Condor Pool consists of over 50+ WindowsXP hosts, the actual number of hosts varies over time and no guarantees are provided as to the number of hosts that will be available at a given time. Although the user should be aware that the central Condor and Globus services are hosted on a RHEL4 Linux server (and not a Windows host). All compute hosts have a least 2GB Memory, 2.4Ghz Intel processors and 5GB of user-space for per job.

Each host uses a local filesystem (i.e. there is no shared-filesystem) and users should explicitly stage-out files when their jobs complete. Access to the system is via the standard Globus interfaces (GRAM, GridFTP and GSISSH). Site-specific usage information is given below

The service is provided at the NGS affiliate level, i.e. no formal Service Level Description is provided for this service. However, all registered NGS users may use the resource and we encourage users to do so. The service is provided by Bristol Advanced Computing Research Centre and the Department of Physics.

Usage

Testing Authentication

First you should confirm that you are allowed to access the service

 [user@mach]$ grid-proxy-init
 [user@mach]$ globusrun  -a -r cerb-condor.bris.ac.uk/jobmanager-condor
 
 GRAM Authentication test successful

Submitting Jobs a Windows Batch Script

To submit a simple DOS batch script to the pool:

[user@mach]$ globus-job-run  cerb-condor.bris.ac.uk/jobmanager-condor -s echo.bat
 �Hello World�

Submitting Jobs a Windows Executable

To submit a Windows executable to the pool:

[user@mach]$ globus-job-run  cerb-condor.bris.ac.uk/jobmanager-condor -s hello.exe
  Hello World

the globus-job-run command will wait for the jobmanager to return, if there are a large number of jobs in the queue this may take a long time there it is generally better to submit jobs using

Submitting Jobs to a Windows Executable (Non-interactive)

[user@mach]$  globus-job-submit  cerb-condor.bris.ac.uk/jobmanager-condor -s echo.bat
 https://cerb-condor.bris.ac.uk:64370/3098/1203611338/

 [user@mach]$ globus-job-status https://cerb-condor.bris.ac.uk:64370/3098/1203611338/
 DONE

 [user@mach]$ globus-job-get-output https://cerb-condor.bris.ac.uk:64370/3098/1203611338/
 "Hello World"

The file extension for your program should be one of .bat or .exe � the file extension will determines how the job submission request is processed.

Retrieving Output Files

In the above examples we are only returning data that is written to standard output. In real life applications frequently write directly to a file rather than writing to standard output. If this is the case then we need to explicitly tell Globus to return the output file(s). In the following example the executable generate 4 output files.

[user@mach]$ globusrun -s -r cerb-condor.bris.ac.uk/jobmanager-condor  -f globus.rsl
 
 [user@mach]$ ls FILE*
 FILE1 FILE2

There are two things to note:

• You must use the -s flag when calling globusrun. This flags starts the Globus GASS server on the local machine, this allows files to be transfered between your client/submission machine and the server in Bristol.
• You must specify jobmanager-condor in the contact string, if instead you specify jobmanager-fork the job will fail, because forked jobs run on the headnode which runs a Linux OS while Condor jobs will run on machines with a . If you do not specify any jobmanager you job will also fail, becuase fork is the default jobmanager (and will be used if you do not specify a jobmanager).

Next you must create an RSL file, that explicitly lists the output files that are are generated by your program (or at least, lists the files that you want to retrieve). In this example we stage-in the executable (i.e. we transfer it from our client machine to the pool) and stage-out the files generate by the executable (i.e. retrieve them from the pool in bristol)

& 
 (jobType="single")
 (executable=$(GLOBUSRUN_GASS_URL)/home2/isxjw/Condor_Course/Basic/Example07/create_file.exe)
 (file_stage_out =
   (OUTPUT_FILE1 $(GLOBUSRUN_GASS_URL)/home2/isxjw/Condor_Course/Basic/Example07/FILE1)
   (OUTPUT_FILE2 $(GLOBUSRUN_GASS_URL)/home2/isxjw/Condor_Course/Basic/Example07/FILE2)

The syntax the file_stage_out directive is very simple

(file_stage_out = (<remote_filename>  $(GLOBUSRUN_GASS_URL)<local_filename>))

Sending Input Files

Finally, just as it is common to have programs that generate multiple output files, it is often the case that our programs require input files, auxillary files and libraries that must be present, and visible to the program, at run-time. In this case we need to stage-in those file when we submit the job. Assume that our program reads two input files, for instance name.txt and numbers.txt. The command line syntax is as before,

 [isxjw@mach]$ globusrun -s -r cerb-condor.bris.ac.uk/jobmanager-condor -f globus.rsl

but now the RSL file uses the file_stage_in directive instead:

 &
 (jobType="single")
 (executable=$(GLOBUSRUN_GASS_URL)/home2/isxjw/Condor_Course/Basic/Example09/read_files.exe)
 (file_stage_in =
   ($(GLOBUSRUN_GASS_URL)/home2/isxjw/Condor_Course/Basic/Example09/name.txt name.txt)
  ($(GLOBUSRUN_GASS_URL)/home2/isxjw/Condor_Course/Basic/Example09/numbers.txt numbers.txt)
 )

The syntax for the file_stage_in directive is:

(file_stage_out = ($(GLOBUSRUN_GASS_URL)<local_filename>)   <remote_filename>)

Further Information

For further information or to report problems please use the following e-mail addresses

• support (at) grid-support.ac.uk
• uobgrid-admin (at) bristol.ac.uk
• Caroline.Gardiner (at) bristol.ac.uk