WMProxy Java API
From EUAGwiki
Contents |
[edit] WMProxy Java API
[edit] Lectured by
Giuseppe LA ROCCA
Italian National Institute of Nuclear Physics
Italy
mailto:giuseppe.larocca@ct.infn.it
[edit] About this page
Goal of this wiki page is to provide some hints and examples about how to interact with the Workload Management Proxy (WMProxy) Server using the dedicated Java API exposed by the server. The WMProxy client API supplies the client applications with a set of interfaces over the job submission and control services made available by the gLite WMS through a web service based interface. The API provides the corresponding method for each operation published in the WSDL description of the WMProxy service.
In the next sections a link from where download a tarball with the source files will be provided.
[edit] Basic Requirements
The Java API illustrated in this web page allow a grid user to interact with a gLite 3.1 WMProxy so, before any call to the server is made, verify that your User Interface (UI) or workstation contains the following rpm packages:
Apache Axis libraries:
* axis-1.4-2006-04-22.jar
* axis2-1.0-2006-05-05.jar
* axis-jaxrpc-1.2-2005-05-03.jar
* axis-wsdl4j-1.2-2005-04-12.jar
Geronimo J2EE libraries:
* geronimo-activation_1.0.2_spec-1.1-2006-06-14.jar
* geronimo-j2ee_1.4_spec-1.1-2006-06-14.jar
Crypto libraries:
* cryptix-asn1.jar
* cryptix32.jar
* bcprov.jar
* puretls.jar
GLITE libraries:
* glite-jdl-api-java.jar
* glite-wms-wmproxy-api-java.jar
* glite-security-delegation-java.jar
* glite-security-trustmanager.jar
* glite-security-util-java.jar
Gloubs and other libraries:
* cog-jglobus.jar
* classad.jar
* commons-discovery-0.2-2004-03-23.jar
* commons-logging-api-1.1.jar
* log4j-1.2.14.jar
[edit] Warning
All the needed gLite packages can be found here http://eticssoft.web.cern.ch/eticssoft/repository/org.glite
All the needed external packages can be found here http://eticssoft.web.cern.ch/eticssoft/repository/externals
In order to allow grid users to invoke, within his Java code, the UrlCopy classes to copy files from the WMProxy Server to the UI and vice versa the following Java package is requested cog-jglobus-1.7.0-bin.tar.gz The above package with all the needed documentation can be downloaded from this web link http://dev.globus.org/wiki/CoG_JGlobus_1.7.0
[edit] User credential delegation
Before calling submission or list match services, client proxy credentials are requested to be transferred from the client to the server through a delegation process. Delegated credentials are uniquely identified by a (string) identifier. The following Java code performs this delegation process :
String proxy = null; WMProxyAPI client = new WMProxyAPI (wmproxy_url,user_proxy,CAcertsPath); proxy = client.getProxyReq (delegationId);
The obtained delegation identifier can be reused for several calls to the WMProxy server; you don't need to recreate a new delegation identifier for each service request. It's recommended to reuse delegation id since its creation is a time consuming operation.
[edit] Note on JDL
The JDL to be matched against resources or to be submitted must contains some mandatory attributes. When using command line interface these are added by default, but in our case user has to take care to check if his/her JDL file contains them:
Requirements=(other.GlueCEStateStatus == "Production");
Rank=(-other.GlueCEStateEstimatedResponseTime);
[edit] Job Submission
Submission of simple jobs, DAGs, collections and parametric jobs to the WMProxy service requires as input a job description file in which job characteristics and requirements are expressed by means of Condor class-ad-like expressions. In this description the users can specify some files that are needed by submitted jobs during the execution on the remote Computing Elements. These files are listed in the JDL InputSandbox attribute specifying for each of them the complete URI of its location. They can be located on or stored on GridFTP/HTTPS servers that managed by their organisations or directly on the submitting client machine. The first class of files will be directly downloaded on the Worker Nodes where the job will run without transiting on the WMProxy machine. File uploading of the second ones is indeed responsibility of the client (through the GridFTP/HTTPS service available on the WMProxy node).
If the job does not have any file in the InputSandbox to be transferred from the submitting machine to the WMProxy node, the submission can be performed also calling the jobSubmit service:
jobIds = client.jobSubmit(jdlString, delegationId);
Otherwise, this step is needed:
jobIds = client.jobRegister(jdlString, delegationId);
[edit] Material and instructions for use
In attach you can find a tarball with the JAVA source files implementing what described above.
If you have a shell on a linux machine, download the tar ball with the examples typing
wget http://grid.ct.infn.it/api-java/Api-Java-WMProxy.tar.gz
Uncompress using
tar zxvf Api-Java-WMproxy.tar.gz
and you should find these files
$ ll Api-Java-WMProxy -rw-r--r-- 1 larocca users 2788258 Jun 30 13:37 cog-jglobus-1.7.0-bin.tar.gz drwxr-xr-x 3 larocca users 72 Jun 30 12:44 doc drwxr-xr-x 2 larocca users 376 Jun 30 13:00 help -rw-r--r-- 1 larocca users 361 Jun 30 14:55 hostname.jdl drwxr-xr-x 6 larocca users 800 Jun 30 12:44 jar drwxr-xr-x 15 larocca users 392 Jun 30 13:25 javax -rwxr-xr-x 1 larocca users 3640 Jun 30 13:23 LICENSE -rw-r--r-- 1 larocca users 449 Jun 30 13:37 log4j.properties drwxr-xr-x 2 larocca users 168 Jun 30 13:37 META-INF drwxr-xr-x 10 larocca users 264 Jun 30 12:59 org -rwxr-xr-x 1 larocca users 30 Jun 30 13:26 start_hostname.sh -rw-r--r-- 1 larocca users 1517 Jun 30 13:37 wmp-api-java-test.cfg -rw-r--r-- 1 larocca users 4380 Jun 30 13:23 WMProxyCancelJobs.java -rw-r--r-- 1 larocca users 6426 Oct 10 15:54 WMProxyGetOutputAndPurge.java -rw-r--r-- 1 larocca users 5780 Oct 10 15:53 WMProxyGetProxyAndListMatch.java -rw-r--r-- 1 larocca users 7142 Oct 3 09:04 WMProxyGetProxyAndSubmit.java -rw-r--r-- 1 larocca users 4326 Jun 30 13:37 wmproxy.sh
To compile all the Java files, first uncompress the CoG jGlobus tar in the working directory containing the examples, and then run the wmproxy.sh script
$ cd Api-Java-WMProxy $ tar zxf cog-jglobus-1.4-bin.tar.gz $ source wmproxy.sh [larocca@glite-tutor:~/Api-Java-WMProxy]$ source wmproxy.sh + source wmproxy.sh ++ pwd=./ ++ top=../../../../../../../ ++ top_src=../../../../../ ++ class_1=WMProxyGetProxyAndSubmit [cut...]
You can run two examples as follow
Create your proxy first :
[larocca@glite-tutor:~/API]$ voms-proxy-init --voms gilda + voms-proxy-init --voms gilda Your identity: /C=IT/O=GILDA/OU=Personal Certificate/L=INFN Catania/CN=Giuseppe La Rocca/Email=giuseppe.larocca@ct.infn.it Enter GRID pass phrase: Creating temporary proxy ........................................ Done Contacting voms.ct.infn.it:15001 [/C=IT/O=INFN/OU=Host/L=Catania/CN=voms.ct.infn.it] "gilda" Done Creating proxy ......................................................... Done Your proxy is valid until Fri Dec 22 23:45:13 2006
run the command as here shown; basically, a proxy delegation to the WMProxy server is created, then jdl provided, hostname.jdl is submitted and the InputSandboxes file(s) are transferred if any. Check the output messages to see it happening
$ java WMProxyGetProxyAndSubmit \
https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server \
hostname.jdl start_hostname.sh | grep -v "ERROR"
Connecting to the service https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server
====================== glite-wms-job-submission Success ======================
The job has been successfully submitted to the WMProxy
Your job identifier is:https://gilda-lb-01.ct.infn.it:9000/P-H1wp0RpTnos2qGyQ2ETQ
===============================================================================
When the job status is Done you can retrieve the output file(s) of your job as follow:
$ java WMProxyGetOutputAndPurge \
https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server \
https://gilda-lb-01.ct.infn.it:9000/P-H1wp0RpTnos2qGyQ2ETQ \
$PWD/ | grep -v "ERROR"
Connecting to the service https://glite-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server
====================== glite-wms-job-output Success ======================
JOB GET OUTPUT OUTCOME
Output sandbox files for the job:
https://gilda-lb-01.ct.infn.it:9000/P-H1wp0RpTnos2qGyQ2ETQ
have been successfully retrieved and stored in the directory:
/home/larocca/Api-Java-WMProxy/
==========================================================================
[edit] Listing Computing Element(s) that match a job description using Java API
It is possible to see which CEs are eligible to run a job described by a given JDL using the following Java API:
$ java WMProxyGetProxyAndListMatch \
https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server \
hostname.jdl | grep -v "ERROR"
Connecting to the service https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server
COMPUTING ELEMENT IDs LIST
==========================================================================
The following CE(s) matching your job requirements have been found:
- ce.hpc.iit.bme.hu:2119/jobmanager-lcgpbs-gilda
- gilda-01.pd.infn.it:2119/jobmanager-lcgpbs-gilda
- gn0.hpcc.sztaki.hu:2119/jobmanager-lcgpbs-gilda
- grid010.ct.infn.it:2119/jobmanager-lcgpbs-infinite
- grid010.ct.infn.it:2119/jobmanager-lcgpbs-long
- grid010.ct.infn.it:2119/jobmanager-lcgpbs-short
- grid011f.cnaf.infn.it:2119/jobmanager-lcgpbs-gilda
- iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-infinite
- iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-long
- iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-short
[..]
==========================================================================
[edit] Cancelling a job using Java API
A job can be canceled before it ends using the following Java class:
$ java WMProxyCancelJobs \
https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server \
https://gilda-lb-01.ct.infn.it:9000/dwgBZqwoPKkl7v_OK6kiQw | grep -v "ERROR"
Connecting to the service https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server
============================= glite-wms-job-cancel Success =============================
The cancellation request has been successfully submitted for the following job(s):
- https://gilda-lb-01.ct.infn.it:9000/dwgBZqwoPKkl7v_OK6kiQw
========================================================================================
[edit] References
http://trinity.datamat.it/projects/EGEE/wiki/wiki.php?n=WMProxyAPI.APIDocumentation http://trinity.datamat.it/projects/EGEE/wiki/wiki.php?n=WMProxyAPI.JobSubmission http://trinity.datamat.it/projects/EGEE/wiki/wiki.php?n=WMProxyClient.QuickStart https://grid.ct.infn.it/twiki/bin/view/GILDA/WmProxyUse https://edms.cern.ch/document/590869/1 https://edms.cern.ch/document/674643/1
