| <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" |
| "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> |
| <html xmlns="http://www.w3.org/1999/xhtml"> |
| <head> |
| <meta http-equiv="Content-Type" content="text/html;charset=utf-8" /> |
| <link href="style.css" rel="stylesheet" type="text/css"/> |
| <title>Thrift module: compute_resource_model</title></head><body> |
| <div class="container-fluid"> |
| <h1>Thrift module: compute_resource_model</h1> |
| <table class="table-bordered table-striped table-condensed"><thead><th>Module</th><th>Services</th><th>Data types</th><th>Constants</th></thead> |
| <tr> |
| <td>compute_resource_model</td><td></td> |
| <td><a href="#Struct_BatchQueue">BatchQueue</a><br/> |
| <a href="#Struct_CloudJobSubmission">CloudJobSubmission</a><br/> |
| <a href="#Struct_ComputeResourceDescription">ComputeResourceDescription</a><br/> |
| <a href="#Enum_DMType">DMType</a><br/> |
| <a href="#Enum_FileSystems">FileSystems</a><br/> |
| <a href="#Struct_GlobusJobSubmission">GlobusJobSubmission</a><br/> |
| <a href="#Enum_JobManagerCommand">JobManagerCommand</a><br/> |
| <a href="#Struct_JobSubmissionInterface">JobSubmissionInterface</a><br/> |
| <a href="#Enum_JobSubmissionProtocol">JobSubmissionProtocol</a><br/> |
| <a href="#Struct_LOCALSubmission">LOCALSubmission</a><br/> |
| <a href="#Enum_MonitorMode">MonitorMode</a><br/> |
| <a href="#Enum_ProviderName">ProviderName</a><br/> |
| <a href="#Struct_ResourceJobManager">ResourceJobManager</a><br/> |
| <a href="#Enum_ResourceJobManagerType">ResourceJobManagerType</a><br/> |
| <a href="#Struct_SSHJobSubmission">SSHJobSubmission</a><br/> |
| <a href="#Struct_UnicoreJobSubmission">UnicoreJobSubmission</a><br/> |
| </td> |
| <td></code></td> |
| </tr></table> |
| <hr/><h2 id="Enumerations">Enumerations</h2> |
| <div class="definition"><h3 id="Enum_ResourceJobManagerType">Enumeration: ResourceJobManagerType</h3> |
| * Enumeration of local resource job manager types supported by Airavata |
| * |
| * FORK: |
| * Forking of commands without any job manager |
| * |
| * PBS: |
| * Job manager supporting the Portal Batch System (PBS) protocol. Some examples include TORQUE, PBSPro, Grid Engine. |
| * |
| * SLURM: |
| * The Simple Linux Utility for Resource Management is a open source workload manager. |
| * |
| * UGE: |
| * Univa Grid Engine, a variation of PBS implementation. |
| * |
| * LSF: |
| * IBM Platform Load Sharing Facility is dominantly installed on IBM clusters. |
| * |
| <br/><br/><table class="table-bordered table-striped table-condensed"> |
| <tr><td><code>FORK</code></td><td><code>0</code></td><td> |
| </td></tr> |
| <tr><td><code>PBS</code></td><td><code>1</code></td><td> |
| </td></tr> |
| <tr><td><code>SLURM</code></td><td><code>2</code></td><td> |
| </td></tr> |
| <tr><td><code>LSF</code></td><td><code>3</code></td><td> |
| </td></tr> |
| <tr><td><code>UGE</code></td><td><code>4</code></td><td> |
| </td></tr> |
| </table></div> |
| <div class="definition"><h3 id="Enum_JobManagerCommand">Enumeration: JobManagerCommand</h3> |
| Enumeration of resource job manager commands |
| |
| SUBMISSION: |
| Ex: qsub, sbatch |
| |
| JOBMONITORING: |
| Ex: qstat, squeue |
| |
| DELETION: |
| Ex: qdel, scancel |
| |
| CHECK_JOB: |
| Detailed Status about the Job. Ex: checkjob |
| |
| SHOW_QUEUE: |
| List of Queued Job by the schedular. Ex: showq |
| |
| SHOW_RESERVATION: |
| List all reservations. Ex:showres, show_res |
| |
| SHOW_START: |
| Display the start time of the specified job. Ex: showstart |
| |
| <br/><br/><table class="table-bordered table-striped table-condensed"> |
| <tr><td><code>SUBMISSION</code></td><td><code>0</code></td><td> |
| </td></tr> |
| <tr><td><code>JOB_MONITORING</code></td><td><code>1</code></td><td> |
| </td></tr> |
| <tr><td><code>DELETION</code></td><td><code>2</code></td><td> |
| </td></tr> |
| <tr><td><code>CHECK_JOB</code></td><td><code>3</code></td><td> |
| </td></tr> |
| <tr><td><code>SHOW_QUEUE</code></td><td><code>4</code></td><td> |
| </td></tr> |
| <tr><td><code>SHOW_RESERVATION</code></td><td><code>5</code></td><td> |
| </td></tr> |
| <tr><td><code>SHOW_START</code></td><td><code>6</code></td><td> |
| </td></tr> |
| </table></div> |
| <div class="definition"><h3 id="Enum_FileSystems">Enumeration: FileSystems</h3> |
| Enumeration of File Systems on the resource |
| |
| FORK: |
| Forking of commands without any job manager |
| |
| PBS: |
| Job manager supporting the Portal Batch System (PBS) protocol. Some examples include TORQUE, PBSPro, Grid Engine. |
| |
| UGE: |
| Univa Grid Engine, a variation of PBS implementation. |
| |
| SLURM: |
| The Simple Linux Utility for Resource Management is a open source workload manager. |
| |
| <br/><br/><table class="table-bordered table-striped table-condensed"> |
| <tr><td><code>HOME</code></td><td><code>0</code></td><td> |
| </td></tr> |
| <tr><td><code>WORK</code></td><td><code>1</code></td><td> |
| </td></tr> |
| <tr><td><code>LOCALTMP</code></td><td><code>2</code></td><td> |
| </td></tr> |
| <tr><td><code>SCRATCH</code></td><td><code>3</code></td><td> |
| </td></tr> |
| <tr><td><code>ARCHIVE</code></td><td><code>4</code></td><td> |
| </td></tr> |
| </table></div> |
| <div class="definition"><h3 id="Enum_JobSubmissionProtocol">Enumeration: JobSubmissionProtocol</h3> |
| Enumeration of Airavata supported Job Submission Mechanisms for High Performance Computing Clusters. |
| |
| SSH: |
| Execute remote job submission commands using via secure shell protocol. |
| |
| GRAM: |
| Execute remote jobs via Globus GRAM service. |
| |
| UNICORE: |
| Execute remote jobs via Unicore services |
| |
| <br/><br/><table class="table-bordered table-striped table-condensed"> |
| <tr><td><code>LOCAL</code></td><td><code>0</code></td><td> |
| </td></tr> |
| <tr><td><code>SSH</code></td><td><code>1</code></td><td> |
| </td></tr> |
| <tr><td><code>GLOBUS</code></td><td><code>2</code></td><td> |
| </td></tr> |
| <tr><td><code>UNICORE</code></td><td><code>3</code></td><td> |
| </td></tr> |
| <tr><td><code>CLOUD</code></td><td><code>4</code></td><td> |
| </td></tr> |
| <tr><td><code>SSH_FORK</code></td><td><code>5</code></td><td> |
| </td></tr> |
| <tr><td><code>LOCAL_FORK</code></td><td><code>6</code></td><td> |
| </td></tr> |
| </table></div> |
| <div class="definition"><h3 id="Enum_MonitorMode">Enumeration: MonitorMode</h3> |
| Monitoring modes |
| |
| POLL_JOB_MANAGER: |
| GFac need to pull job status changes. |
| |
| XSEDE_AMQP_SUBSCRIBE: |
| Server will publish job status changes to amqp servert. |
| |
| |
| <br/><br/><table class="table-bordered table-striped table-condensed"> |
| <tr><td><code>POLL_JOB_MANAGER</code></td><td><code>0</code></td><td> |
| </td></tr> |
| <tr><td><code>JOB_EMAIL_NOTIFICATION_MONITOR</code></td><td><code>1</code></td><td> |
| </td></tr> |
| <tr><td><code>XSEDE_AMQP_SUBSCRIBE</code></td><td><code>2</code></td><td> |
| </td></tr> |
| <tr><td><code>FORK</code></td><td><code>3</code></td><td> |
| </td></tr> |
| </table></div> |
| <div class="definition"><h3 id="Enum_DMType">Enumeration: DMType</h3> |
| <br/><table class="table-bordered table-striped table-condensed"> |
| <tr><td><code>COMPUTE_RESOURCE</code></td><td><code>0</code></td><td> |
| </td></tr> |
| <tr><td><code>STORAGE_RESOURCE</code></td><td><code>1</code></td><td> |
| </td></tr> |
| </table></div> |
| <div class="definition"><h3 id="Enum_ProviderName">Enumeration: ProviderName</h3> |
| Provider name |
| |
| <br/><br/><table class="table-bordered table-striped table-condensed"> |
| <tr><td><code>EC2</code></td><td><code>0</code></td><td> |
| </td></tr> |
| <tr><td><code>AWSEC2</code></td><td><code>1</code></td><td> |
| </td></tr> |
| <tr><td><code>RACKSPACE</code></td><td><code>2</code></td><td> |
| </td></tr> |
| </table></div> |
| <hr/><h2 id="Structs">Data structures</h2> |
| <div class="definition"><h3 id="Struct_ResourceJobManager">Struct: ResourceJobManager</h3> |
| <table class="table-bordered table-striped table-condensed"><thead><th>Key</th><th>Field</th><th>Type</th><th>Description</th><th>Requiredness</th><th>Default value</th></thead> |
| <tr><td>1</td><td>resourceJobManagerId</td><td><code>string</code></td><td></td><td>required</td><td><code>"DO_NOT_SET_AT_CLIENTS"</code></td></tr> |
| <tr><td>2</td><td>resourceJobManagerType</td><td><code><a href="#Enum_ResourceJobManagerType">ResourceJobManagerType</a></code></td><td></td><td>required</td><td></td></tr> |
| <tr><td>3</td><td>pushMonitoringEndpoint</td><td><code>string</code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>4</td><td>jobManagerBinPath</td><td><code>string</code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>5</td><td>jobManagerCommands</td><td><code>map<<code><a href="#Enum_JobManagerCommand">JobManagerCommand</a></code>, <code>string</code>></code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>6</td><td>parallelismPrefix</td><td><code>map<<code><a href="parallelism_model.html#Enum_ApplicationParallelismType">parallelism_model.ApplicationParallelismType</a></code>, <code>string</code>></code></td><td></td><td>optional</td><td></td></tr> |
| </table><br/>Resource Job Manager Information |
| |
| resourceJobManagerType: |
| A typical HPC cluster has a single Job Manager to manage the resources. |
| |
| pushMonitoringEndpoint: |
| If the job manager pushes out state changes to a database or bus, specify the service endpoint. |
| Ex: Moab Web Service, Moab MongoDB URL, AMQP (GLUE2) Broker |
| |
| jobManagerBinPath: |
| Path to the Job Manager Installation Binary directory. |
| |
| jobManagerCommands: |
| An enumeration of commonly used manager commands. |
| |
| <br/></div><div class="definition"><h3 id="Struct_BatchQueue">Struct: BatchQueue</h3> |
| <table class="table-bordered table-striped table-condensed"><thead><th>Key</th><th>Field</th><th>Type</th><th>Description</th><th>Requiredness</th><th>Default value</th></thead> |
| <tr><td>1</td><td>queueName</td><td><code>string</code></td><td></td><td>required</td><td></td></tr> |
| <tr><td>2</td><td>queueDescription</td><td><code>string</code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>3</td><td>maxRunTime</td><td><code>i32</code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>4</td><td>maxNodes</td><td><code>i32</code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>5</td><td>maxProcessors</td><td><code>i32</code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>6</td><td>maxJobsInQueue</td><td><code>i32</code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>7</td><td>maxMemory</td><td><code>i32</code></td><td></td><td>optional</td><td></td></tr> |
| </table><br/>Batch Queue Information on SuperComputers |
| |
| maxRunTime: |
| Maximum allowed run time in hours. |
| <br/></div><div class="definition"><h3 id="Struct_LOCALSubmission">Struct: LOCALSubmission</h3> |
| <table class="table-bordered table-striped table-condensed"><thead><th>Key</th><th>Field</th><th>Type</th><th>Description</th><th>Requiredness</th><th>Default value</th></thead> |
| <tr><td>1</td><td>jobSubmissionInterfaceId</td><td><code>string</code></td><td></td><td>required</td><td><code>"DO_NOT_SET_AT_CLIENTS"</code></td></tr> |
| <tr><td>2</td><td>resourceJobManager</td><td><code><a href="#Struct_ResourceJobManager">ResourceJobManager</a></code></td><td></td><td>required</td><td></td></tr> |
| <tr><td>3</td><td>securityProtocol</td><td><code><a href="data_movement_models.html#Enum_SecurityProtocol">data_movement_models.SecurityProtocol</a></code></td><td></td><td>optional</td><td></td></tr> |
| </table><br/>Locally Fork Jobs as OS processes |
| |
| alternativeSSHHostName: |
| If the login to ssh is different than the hostname itself, specify it here |
| |
| sshPort: |
| If a non-default port needs to used, specify it. |
| <br/></div><div class="definition"><h3 id="Struct_SSHJobSubmission">Struct: SSHJobSubmission</h3> |
| <table class="table-bordered table-striped table-condensed"><thead><th>Key</th><th>Field</th><th>Type</th><th>Description</th><th>Requiredness</th><th>Default value</th></thead> |
| <tr><td>1</td><td>jobSubmissionInterfaceId</td><td><code>string</code></td><td></td><td>required</td><td><code>"DO_NOT_SET_AT_CLIENTS"</code></td></tr> |
| <tr><td>2</td><td>securityProtocol</td><td><code><a href="data_movement_models.html#Enum_SecurityProtocol">data_movement_models.SecurityProtocol</a></code></td><td></td><td>required</td><td></td></tr> |
| <tr><td>3</td><td>resourceJobManager</td><td><code><a href="#Struct_ResourceJobManager">ResourceJobManager</a></code></td><td></td><td>required</td><td></td></tr> |
| <tr><td>4</td><td>alternativeSSHHostName</td><td><code>string</code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>5</td><td>sshPort</td><td><code>i32</code></td><td></td><td>optional</td><td><code>22</code></td></tr> |
| <tr><td>6</td><td>monitorMode</td><td><code><a href="#Enum_MonitorMode">MonitorMode</a></code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>7</td><td>batchQueueEmailSenders</td><td><code>list<<code>string</code>></code></td><td></td><td>optional</td><td></td></tr> |
| </table><br/>Authenticate using Secured Shell |
| |
| alternativeSSHHostName: |
| If the login to ssh is different than the hostname itself, specify it here |
| |
| sshPort: |
| If a non-default port needs to used, specify it. |
| |
| batchQueueEmailSenders: |
| If a resource always sends the monitoring from a specific address, specify the |
| full email address. If a resource sends emails from multiple addresses ( |
| example: based on the submitted login node) then use the wildchar * to indicate |
| the same. Example: *@*.example.com or *@example.com |
| |
| <br/></div><div class="definition"><h3 id="Struct_GlobusJobSubmission">Struct: GlobusJobSubmission</h3> |
| <table class="table-bordered table-striped table-condensed"><thead><th>Key</th><th>Field</th><th>Type</th><th>Description</th><th>Requiredness</th><th>Default value</th></thead> |
| <tr><td>1</td><td>jobSubmissionInterfaceId</td><td><code>string</code></td><td></td><td>required</td><td><code>"DO_NOT_SET_AT_CLIENTS"</code></td></tr> |
| <tr><td>2</td><td>securityProtocol</td><td><code><a href="data_movement_models.html#Enum_SecurityProtocol">data_movement_models.SecurityProtocol</a></code></td><td></td><td>required</td><td></td></tr> |
| <tr><td>3</td><td>globusGateKeeperEndPoint</td><td><code>list<<code>string</code>></code></td><td></td><td>optional</td><td></td></tr> |
| </table><br/></div><div class="definition"><h3 id="Struct_UnicoreJobSubmission">Struct: UnicoreJobSubmission</h3> |
| <table class="table-bordered table-striped table-condensed"><thead><th>Key</th><th>Field</th><th>Type</th><th>Description</th><th>Requiredness</th><th>Default value</th></thead> |
| <tr><td>1</td><td>jobSubmissionInterfaceId</td><td><code>string</code></td><td></td><td>required</td><td><code>"DO_NOT_SET_AT_CLIENTS"</code></td></tr> |
| <tr><td>2</td><td>securityProtocol</td><td><code><a href="data_movement_models.html#Enum_SecurityProtocol">data_movement_models.SecurityProtocol</a></code></td><td></td><td>required</td><td></td></tr> |
| <tr><td>3</td><td>unicoreEndPointURL</td><td><code>string</code></td><td></td><td>required</td><td></td></tr> |
| </table><br/>Unicore Job Submission |
| |
| unicoreEndPointURL: |
| unicoreGateway End Point. The provider will query this service to fetch required service end points. |
| authenticationMode |
| The authenticationMode defines the way certificate is fetched. |
| <br/></div><div class="definition"><h3 id="Struct_CloudJobSubmission">Struct: CloudJobSubmission</h3> |
| <table class="table-bordered table-striped table-condensed"><thead><th>Key</th><th>Field</th><th>Type</th><th>Description</th><th>Requiredness</th><th>Default value</th></thead> |
| <tr><td>1</td><td>jobSubmissionInterfaceId</td><td><code>string</code></td><td></td><td>required</td><td><code>"DO_NOT_SET_AT_CLIENTS"</code></td></tr> |
| <tr><td>2</td><td>securityProtocol</td><td><code><a href="data_movement_models.html#Enum_SecurityProtocol">data_movement_models.SecurityProtocol</a></code></td><td></td><td>required</td><td></td></tr> |
| <tr><td>3</td><td>nodeId</td><td><code>string</code></td><td></td><td>required</td><td></td></tr> |
| <tr><td>4</td><td>executableType</td><td><code>string</code></td><td></td><td>required</td><td></td></tr> |
| <tr><td>5</td><td>providerName</td><td><code><a href="#Enum_ProviderName">ProviderName</a></code></td><td></td><td>required</td><td></td></tr> |
| <tr><td>6</td><td>userAccountName</td><td><code>string</code></td><td></td><td>required</td><td></td></tr> |
| </table><br/>Cloud Job Submission |
| |
| |
| <br/></div><div class="definition"><h3 id="Struct_JobSubmissionInterface">Struct: JobSubmissionInterface</h3> |
| <table class="table-bordered table-striped table-condensed"><thead><th>Key</th><th>Field</th><th>Type</th><th>Description</th><th>Requiredness</th><th>Default value</th></thead> |
| <tr><td>1</td><td>jobSubmissionInterfaceId</td><td><code>string</code></td><td></td><td>required</td><td></td></tr> |
| <tr><td>2</td><td>jobSubmissionProtocol</td><td><code><a href="#Enum_JobSubmissionProtocol">JobSubmissionProtocol</a></code></td><td></td><td>required</td><td></td></tr> |
| <tr><td>3</td><td>priorityOrder</td><td><code>i32</code></td><td></td><td>required</td><td><code>0</code></td></tr> |
| </table><br/>Job Submission Interfaces |
| |
| jobSubmissionInterfaceId: The Job Submission Interface has to be previously registered and referenced here. |
| |
| priorityOrder: |
| For resources with multiple interfaces, the priority order should be selected. |
| Lower the numerical number, higher the priority |
| |
| <br/></div><div class="definition"><h3 id="Struct_ComputeResourceDescription">Struct: ComputeResourceDescription</h3> |
| <table class="table-bordered table-striped table-condensed"><thead><th>Key</th><th>Field</th><th>Type</th><th>Description</th><th>Requiredness</th><th>Default value</th></thead> |
| <tr><td>1</td><td>computeResourceId</td><td><code>string</code></td><td></td><td>required</td><td><code>"DO_NOT_SET_AT_CLIENTS"</code></td></tr> |
| <tr><td>2</td><td>hostName</td><td><code>string</code></td><td></td><td>required</td><td></td></tr> |
| <tr><td>3</td><td>hostAliases</td><td><code>list<<code>string</code>></code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>4</td><td>ipAddresses</td><td><code>list<<code>string</code>></code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>5</td><td>resourceDescription</td><td><code>string</code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>6</td><td>enabled</td><td><code>bool</code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>7</td><td>batchQueues</td><td><code>list<<code><a href="#Struct_BatchQueue">BatchQueue</a></code>></code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>8</td><td>fileSystems</td><td><code>map<<code><a href="#Enum_FileSystems">FileSystems</a></code>, <code>string</code>></code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>9</td><td>jobSubmissionInterfaces</td><td><code>list<<code><a href="#Struct_JobSubmissionInterface">JobSubmissionInterface</a></code>></code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>10</td><td>dataMovementInterfaces</td><td><code>list<<code><a href="data_movement_models.html#Struct_DataMovementInterface">data_movement_models.DataMovementInterface</a></code>></code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>11</td><td>maxMemoryPerNode</td><td><code>i32</code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>12</td><td>gatewayUsageReporting</td><td><code>bool</code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>13</td><td>gatewayUsageModuleLoadCommand</td><td><code>string</code></td><td></td><td>optional</td><td></td></tr> |
| <tr><td>14</td><td>gatewayUsageExecutable</td><td><code>string</code></td><td></td><td>optional</td><td></td></tr> |
| </table><br/>Computational Resource Description |
| |
| computeResourceId: Airavata Internal Unique Identifier to distinguish Compute Resource. |
| |
| hostName: |
| Fully Qualified Host Name. |
| |
| hostAliases: |
| Aliases if any. |
| |
| ipAddress: |
| IP Addresses of the Resource. |
| |
| resourceDescription: |
| A user friendly description of the resource. |
| |
| JobSubmissionProtocols: |
| A computational resources may have one or more ways of submitting Jobs. This structure |
| will hold all available mechanisms to interact with the resource. |
| The key is the priority |
| |
| DataMovementProtocol: |
| Option to specify a prefered data movement mechanism of the available options. |
| |
| fileSystems: |
| Map of file systems type and the path. |
| |
| <br/></div></div></body></html> |
