Skip to content

Latest commit

 

History

History
125 lines (100 loc) · 5.07 KB

api.md

File metadata and controls

125 lines (100 loc) · 5.07 KB
Raw

Overview

LSF FaaS APIs is used to access LSF cluster resources by Python functions. You need to logon to LSF PAC server before use any FaaS APIs. You can optional logout to stop using LSF.

During an LSF session, you can use sub to submit a function to the cluster in asynchronous way or use exe to execute a function in the cluster by synchronous way. For asynchronously submission, you can cancel the execution. When the function is execution done, you can use get to obtain the return value of the function. You can also download any files that generated by your function executed on the LSF cluster.

Function List

logon

logon(username, password, host, port, isHttps)

Use the specified username/password to log on the specified AC web server.

  • username: PAC user name. By default, the name is get from your current execution context: getpass.getuser().
  • passwd: The password to logon PAC.
  • host: The host running PAC server.
  • port: The port number that PAC server listending to. By default, it is 8080.
  • isHttps: Is HTTPS enabled. By default, it is False.

Return True if success, otherwise return false.

logout

logout()

Log out of the AC web server.

sub

sub(func, *arguments, files, asynchronous)

Submit a function calls (especially for time-consuming operations) with arguments to an LSF cluster. The function call is transformed into an LSF job and submitted to the LSF cluster automatically.

  • func: The function which will be executed.
  • arguments: The function argument list.
  • files: The files need to be uploaded to the PAC server before job execution. It is comma , seperated file list. This is optional. By default, it is None.
  • asynchronous: If file upload operation is synchronous or not. It takes effect for files

Return a function id for the function running on LSF.

Examples:

# Submit the 'myfun' function with two arguments 'arg1' and 'arg2' to LSF
>>> id = lsf.sub(myfun, arg1, arg2)

# Submit the 'myfun' function without arguments but with dependency file '/tmp/a.txt'
# In 'myfun' you can use relative path(eg: a.txt or ./a.txt) to read/write the file
>>> id = lsf.sub(myfun, files='/tmp/a.txt')

# Submit the 'myfun' function without arguments but with dependency file '/tmp/a.txt' asynchronously
# In 'myfun' you can use relative path(eg: a.txt or ./a.txt) to read/write the file
>>> id = lsf.sub(myfun, files='/tmp/a.txt', asynchronous = True)

get

get(id):

Get the output based on the specified function id (which returned by sub()).

  • id: The identification of a function in LSF returned by sub

Return the result of the function call.

download

download(id, files, destination = None, asynchronous = False):

Download function data files from AC server to the specified destination. Currently, only can download files that belong to the specified function id (the files must be the function input/output files).

  • id: The job identifier.
  • files: The file list seperated by ,. Currently, only relative path (related from job current directory) is supported.
  • destination: The destination for downloaded files. By default, it is work_dir/id. Currently, it only support absolute path.
  • asynchronous: Whether download in asynchronously or not. By defaut, it use False. It takes effect when the files is specified.

Return True if success, otherwise return false.

Examples:

# Download the function's file a.txt to /tmp/
>>> lsf.download(id,'a.txt','/tmp/')

# Download the function's file a.txt and b.txt to work_dir/id
>>> lsf.download(id,'a.txt,b.txt')

# Download the function's file a.txt to /tmp/ asynchronously
>>> lsf.download(id,'a.txt','/tmp/',asynchronous = True)

# Download the fucntion' file a.txt and b.txt to work_dir/id asynchronously
>>> lsf.download(id,'a.txt,b.txt',asynchronous = True)

cancel

cancel(id)

Cancel the function by function id.

Return True if succeed, otherwise return False.

exe

exe(func, *arguments, files, timeout)

Execute a function call(especially for time-consuming operations) with arguments as a job on LSF. It blocks until job finished/timeout/error.

  • func: The function which will be executed.
  • arguments: The function argument list.
  • files: The files need to be uploaded to the PAC server before job execution. It is comma , seperated file list. This is optional. By default, it is None.
  • timout: The timeout for the operation. By default, it is 60 seconds.

Return the return value(if any) of function if succeeds, or error string if error found,

Examples:

# Execute the 'myfun' function with two arguments 'arg1' and 'arg2'
>>> output = lsf.exe(myfun, arg1, arg2)

# Execute the 'myfun' function without arguments but with dependency file '/tmp/a.txt'
# In 'myfun' you can use relative path(eg: a.txt or ./a.txt) to read/write the file
>>> output = lsf.exe(myfun, files='/tmp/a.txt')

# Execute and wait 300 seconds
>>> output = lsf.exe(myfun, arg1, arg2, timeout = 300)
>>> output = lsf.exe(myfun, files='/tmp/a.txt', timeout = 300)