Grid Interoperation: SRM-SRB Interface
From EUAGwiki
Contents |
[edit] Slides
[edit] practice for SRB
[edit] Environment Setting
To make .srb directory in your home directory.
[UPM07@ui ~]$ mkdir .srb [UPM07@ui ~]$ cd .srb
To create .MdasEnv file in .srb directory. and modify "userxx" to your username (for example: user10)
[UPM07@ui .srb]$ vi .MdasEnv
mdasCollectionName '/AS/home/userxx.asgc' mdasCollectionHome '/AS/home/userxx.asgc' mdasDomainName 'asgc' mdasDomainHome 'asgc' srbUser 'userxx' srbHost 'vl104.grid.sinica.edu.tw' srbPort '6833' defaultResource 'R001' AUTH_SCHEME 'ENCRYPT1'
Then save it.
To create .MdasAuth file and fill your password.
[UPM07@ui .srb]$ vi .MdasEnv
userxx
[edit] Use Scommands to upload, download, remove files.
- Sinit: log in SRB system
- Syntax: Sinit [-v]
- Sls: list directory content
- Syntax: Sls [-aChlVpvX] [-r|-f] [-A condition] [collection|srbObj ...]
- Sput: upload a file to the SRB server
- Syntax: Sput [-fprabvsmMkKV] [-c container] [-D dataType] [-n replNum] [-N numThreads] [-S resourceName] [-P pathName] [-R retry_count] [-x restartFile] localFileName|localDirectory ... TargetName
- Sget: download a file from the SRB server
- Syntax: Sget [-n n] [-N numThreads] [-pbfrvsmMVX] [-T ticketFile | -t ticket] [-A condition] [-W versionString] [-R retry_count] [-x restartFile] [-k] srbObj|Collection ... [localFile|localDirectory]
- Srm: remove a file stored in SRB server
- Syntax: Srm [-n replicaNum] [-puf] [-A condition] srbObj ...
- Sreplicate: to replicate data to another resource
- Syntax: Sreplicate [-n replicaNum] [-bprv] [-S resourceName] [-V versionString] [-P pathName] srbObj|collection ...
- Sexit: log out SRB system
- Syntax: Sexit
More reference: http://www.sdsc.edu/srb/index.php/Scommand_Manpages
[edit] practice for SRM-SRB Interface
[edit] Prerequisites
- srmClient.tar.gz, http://t-ap196.grid.sinica.edu.tw/downloads/srmClient.tar.gz
- SRM-SRB server
- JDK 1.5+
- Apache ANT
- Globus Toolkit
- Personal certificate or proxy
- There are detail installation steps for each package in Appendix A.
- Please note I'm using the account "UPM07" in the following example. Your account may be different.
[edit] Proxy Initialization
Initialize your grid proxy and please write down the path of the file that includes your proxy. (Here, the file I'll use is /tmp/x509up_u501 )
[UPM07@ui srmClient]$ grid-proxy-init -debug User Cert File: /home/UPM07/.globus/usercert.pem User Key File: /home/UPM07/.globus/userkey.pem Trusted CA Cert Dir: (null) Output File: /tmp/x509up_u501 Your identity: /C=TW/O=AS/OU=GRID/CN=Fu-Ming Tsai 194101 Enter GRID pass phrase for this identity: Creating proxy .........++++++++++++ ..........++++++++++++ Done Your proxy is valid until: Sat Apr 11 18:37:41 2009
[edit] Generate SRM stub classes and use them
1. Download the client package (srmClient.tar.gz) and untar it. Then you will get a folder called srmClient
[UPM07@ui ~]$ wget http://t-ap196.grid.sinica.edu.tw/downloads/srmClient.tar.gz [UPM07@ui ~]$ tar xzf srmClient.tar.gz [UPM07@ui ~]$ cd srmClient
2. Then, there are a folder called lib and 6 files inside.
[UPM07@ui srmClient]$ ls -l total 52 -rw-rw-r-- 1 UPM07 UPM07 69 Mar 16 15:08 build.properties -rw-rw-r-- 1 UPM07 UPM07 2989 Apr 11 06:36 build.xml -rw-rw-r-- 1 UPM07 UPM07 1064 Apr 11 06:27 callSrmPing.java drwxrwxr-x 2 UPM07 UPM07 4096 Apr 11 05:12 lib -rw-rw-r-- 1 UPM07 UPM07 455 Apr 16 15:38 readme.txt -rw-rw-r-- 1 UPM07 UPM07 25212 Apr 2 18:08 SampleClient.java -rwxrw-r-- 1 UPM07 UPM07 846 Apr 11 04:58 test.sh
3. Read the file "build.properties". Please note you can not leave ant space at the end of 2nd item.
[UPM07@ui srmClient]$ cat build.properties endpoint=https://sdm.lbl.gov/srm-wg/srm.v2.2.wsdl pkgname=aaa.bbb.cc
4. Generate client stubs with the command "ant gen-client-stubs" or "ant". Then you will get 2 folders called "src" and "fetched". All generated stubs are stored in the folder "src".
[UPM07@ui srmClient]$ ant Buildfile: build.xml init: [mkdir] Created dir: /tmp/srmClient/fetched [mkdir] Created dir: /tmp/srmClient/classes [mkdir] Created dir: /tmp/srmClient/src [echo] Built up all needed directories fetch-wsdl: [get] Getting: http://sdm.lbl.gov/srm-wg/srm.v2.2.wsdl [get] To: /tmp/srmClient/fetched/api.wsdl gen-client-stubs: [echo] All stub classes has been generated BUILD SUCCESSFUL Total time: 5 seconds
[UPM07@ui srmClient]$ ls -l total 64 -rw-rw-r-- 1 UPM07 UPM07 69 Mar 16 15:08 build.properties -rw-rw-r-- 1 UPM07 UPM07 2989 Apr 11 06:36 build.xml -rw-rw-r-- 1 UPM07 UPM07 1064 Apr 11 06:27 callSrmPing.java drwxrwxr-x 2 UPM07 UPM07 4096 Apr 16 15:11 classes drwxrwxr-x 2 UPM07 UPM07 4096 Apr 16 15:11 fetched drwxrwxr-x 2 UPM07 UPM07 4096 Apr 11 05:12 lib -rw-rw-r-- 1 UPM07 UPM07 455 Apr 16 15:38 readme.txt -rw-rw-r-- 1 UPM07 UPM07 25212 Apr 2 18:08 SampleClient.java drwxrwxr-x 3 UPM07 UPM07 4096 Apr 16 15:11 src -rwxrw-r-- 1 UPM07 UPM07 846 Apr 11 04:58 test.sh
5. edit the file "SampleClient.java", change the private field SERVICEURL to your server location. then copy it to the location "src/aaa/bbb/cc".
// The package name must be the same with "pkgname" we defined in the build.properties
package aaa.bbb.cc;
...
public class SampleClient ...
private static final String SERVICEURL =
"httpg://YOUR_SRM_SRB_SERVER:8443/axis/services/srm";
[UPM07@ui srmClient]$ cp SampleClient.java src/aaa/bbb/cc/
6. compile all classes with the command "ant compile".
[UPM07@ui srmClient]$ ant compile Buildfile: build.xml init: [mkdir] Created dir: /tmp/srmClient/classes [echo] Built up all needed directories compile: [javac] Compiling 140 source files to /tmp/srmClient/classes [javac] Note: Some input files use unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. BUILD SUCCESSFUL Total time: 3 seconds
7. execute the shell script "test.sh".
[UPM07@ui srmClient]$ sh test.sh [main] INFO cc.SampleClient - Getting proxy certificate from Local File. [main] INFO cc.SampleClient - Local proxy file name = /tmp/x509up_u501 [main] INFO cc.SampleClient - Web service URL = httpg://YOUR_SRM_SRB_SERVER:8443/axis/services/srm [main] INFO cc.SampleClient - Got Proxy DN = /C=TW/O=AS/OU=GRID/CN=Fu-Ming Tsai 194101 des url:httpg://YOUR_SRM_SRB_SERVER:8443/axis/services/srm Version info : v2.2 Authors: Application Team, ASGC Website: http://twgrid.org Organization: Academia Sinica Grid Computing(ASGC) Project Name: SRM interface on SRB Email: service@twgrid.org backend_type: Storage Resource Broker(SRB)
8. Congrats!! You can invoke SRM interface with SRM client. You've done a great work!!
9. How did we generate stubs? Please read the file "build.xml"
10. How did we invoke srm interface? Please read test.sh and SampleClient.java.
[edit] Use our srmClient.jar
1. There is a java class called "callSrmPing.java".
[UPM07@ui srmClient]$ cat callSrmPing.java
/**
* callSrmPing is used to
* invoke srmPing in this package.
*
* @author Fu-Ming Tsai
*
*/
import java.io.File;
import java.io.FileOutputStream;
import java.util.Hashtable;
import org.apache.axis.types.UnsignedLong;
import org.ietf.jgss.GSSCredential;
import org.apache.axis.types.*;
import asgc.app.srm.client.discovery.SrmPingClient;
import asgc.app.srm.client.discovery.SrmGetProtocolClient;
import asgc.app.srm.client.*;
public class callSrmPing {
/**
* @param args
*/
public static void main(String[] args) {
// the endpoint of the SRM
String address="httpg://YOUR_SRM_SRB_SERVER:8443/axis/services/srm";
ProxyLoader proxyloader=new ProxyLoader();
//all SrmXXXClient are the sub-classes of SrmBaseClient
SrmBaseClient srmClient;
//the file includes user proxy
String fileName="/tmp/x509up_u501";
try{
GSSCredential cred = proxyloader.getFromFile(fileName);
// if you only know the file name that includes user proxy
srmClient=new SrmPingClient(address,fileName,null);
// if you have users proxy, pls use the following statement
// srmClient=new SrmPingClient(address, cred, null);
srmClient.invoke();// only run this method to invoke SRM interface
// display the message on stdout
System.out.println(srmClient);
// get response object
// SrmPingResponse response = (SrmPingResponse)srmClient.getResponse();
}catch(Exception e){
e.printStackTrace();
}
}
}
2. export CLASSPATH.
[UPM07@ui srmClient]$ export CLASSPATH=lib/axis.jar:lib/commons-discovery-0.2.jar:lib/jaxrpc.jar:lib/saaj.jar:lib/cog-axis.jar:lib/commons-logging-1.1.jar:lib/jce-jdk13-131.jar:lib/wsdl4j-1.5.1.jar:lib/cog-jglobus.jar:lib/cryptix32.jar:lib/log4j-1.2.13.jar:lib/cog-url.jar:lib/cryptix-asn1.jar:lib/puretls.jar:classes:lib/srmClient.jar:.
3. Compile it.
[UPM07@ui srmClient]$ javac callSrmPing.java
4. Execute it
[UPM07@ui srmClient]$ java callSrmPing SRM Version: v2.2 =============================================================================== ------------------------------------------------------------------------------------- Authors: Application Team, ASGC Website: http://twgrid.org Organization: Academia Sinica Grid Computing(ASGC) Project Name: SRM interface on SRB Email: service@twgrid.org backend_type: Storage Resource Broker(SRB) -------------------------------------------------------------------------------------
5. Let see callSrmPing.java.
6. Javadoc: http://t-ap196.grid.sinica.edu.tw/srm-srb_client_javadoc/
[edit] Homework
- Write codes to call srmGetTransferProtocols.
- Invoke srm interface without srmClient.jar
- Invoke srm interface with srmClient.jar
[edit] Appendix A
[edit] Requires root access to install
Become root
$ su -
[edit] Java SDK 1.5 or later
1. Download Java SDK RPM file: jdk--linux-i586-rpm.bin http://java.sun.com
2. Change directory to where the downloaded RPM file is located and run these commands below
# chmod u+x jdk--linux-i586-rpm.bin # ./jdk--linux-i586-rpm.bin
Java will be installed at /usr/java/jdk
3. Delete the bin and rpm file if you want to save disk space.
4. Setup environment variables
# ln -s /usr/java/default /usr/java # vi /etc/profile.d/java.sh export JAVA_HOME=/usr/java/default export PATH=$JAVA_HOME/bin:$PATH (every command line need to be ended by a newline character) # source /etc/profile.d/java.sh Extra. Edit Alternatives
# alternatives --install /usr/bin/java java $JAVA_HOME/bin/java 2 # alternatives --config java (choose 2)
[edit] Apache Ant
1. Download ANT binary distribution: apache-ant--bin.tar.gz http://ant.apache.org
2. Extract (untar) the tar file and move the extracted directory to a general location
# tar zxvf apache-ant--bin.tar.gz # mkdir /usr/local/ant # mv apache-ant- /usr/local/ant
3. Delete the distribution tar file if you want to save disk space.
4. Setup environment variables
# ln -s /usr/local/ant/apache-ant- /usr/local/ant/default # vi /etc/profile.d/ant.sh export ANT_HOME=/usr/local/ant/default export PATH=$ANT_HOME/bin:$PATH (every command line need to be ended by a newline character) # source /etc/profile.d/ant.sh
