6 |
--------------------------------------------------------------------------------
/docs/maintenance/code-update/pga-update.md:
--------------------------------------------------------------------------------
1 | ## PGA Update
2 |
3 | 1. In your cloned gateway folder (airavata-php-gateway) get the latest updates from git
4 |
git pull
5 | 2. Do a composer update
6 |
composer update
7 | This may take little time....
8 | 3. You are all set with latest updates!
9 | 4. For Airavata updating steps try [Airavata Update](airavata-update.md)
10 |
--------------------------------------------------------------------------------
/docs/css/extra.css:
--------------------------------------------------------------------------------
1 | .darkred{
2 | color: #7f0000;
3 | /*darkred*/
4 | }
5 |
6 | .lred{
7 | color: #b20000;
8 | /*lighterred*/
9 | }
10 |
11 | .blue{
12 | color: #3333ff;
13 | /*darkblue*/
14 | }
15 |
16 | h3
17 | {
18 | display:inline;
19 | }
20 |
21 | h5
22 | {
23 | display:inline;
24 | }
25 |
26 | h6
27 | {
28 | display:inline;
29 | }
30 |
31 | .wy-nav-content{
32 | max-width: 80%;
33 | }
--------------------------------------------------------------------------------
/.readthedocs.yaml:
--------------------------------------------------------------------------------
1 | # .readthedocs.yaml
2 | # Read the Docs configuration file
3 | # See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
4 |
5 | # Required
6 | version: 2
7 |
8 | # Set the version of Python and other tools you might need
9 | build:
10 | os: ubuntu-20.04
11 | tools:
12 | python: "2.7"
13 |
14 | mkdocs:
15 | configuration: mkdocs.yml
16 |
17 | # Optionally declare the Python requirements required to build your docs
18 | python:
19 | install:
20 | - requirements: requirements.txt
21 |
--------------------------------------------------------------------------------
/docs/installation/airavata-installation.md:
--------------------------------------------------------------------------------
1 | ##Airavata & Django Portal Installation
2 |
3 | ###Intended Users
4 | 1. This section is mainly for gateway developers.
5 | 2. Installing local copy of Airavata, Django and related other applications will provides insight in to the system.
6 |
7 | ###Installation Instructions
8 | Please follow the instructions provided in InteliJ IDEA Installation
9 | This installation will set you up with fully functioning middleware and gateway portal. It is also set up to launch test experimental jobs on Jetstream cloud SLURM cluster.
10 |
--------------------------------------------------------------------------------
/docs/technical-documentation/django-portal.md:
--------------------------------------------------------------------------------
1 | ##Django Portal Documentation
2 | Please refer Apache Airavata Django Portal Documentation for details on Django portal implementation.
3 |
4 | For a sample "Hello World" Django 'App' example refer Hello World Django App
5 |
6 | We have integrated a map tool as a Django 'App' for our SimCCS gateway
7 | refer our SimCCS Map tool code
8 |
9 | Contact us if you need a Django research gateway from Apache Airavata. We would assist you in your 'App' implementation.
--------------------------------------------------------------------------------
/docs/contact-us.md:
--------------------------------------------------------------------------------
1 | ## Contact Us
2 |
3 | To request for a gateway with Apache Airavata create an account in SciGaP Portal
4 | Submit a request using the online application.
5 | We will contact you to discuss your requirement and provide you with the best Solution for You!
6 |
7 |
8 | For your clarifications, Questions and Suggestions please contact us through
9 | Airavata mailing list
10 |
11 | OR
12 |
13 | Slack
14 |
15 | Issue Reporting
16 | Any issue encountered with Airavata or our gateway please log through;
17 |
18 | Airavata JIRA
19 |
20 | NOTE: If you don't have an account in Apache JIRA you have to create one prior to submitting issues.
21 |
22 |
23 |
--------------------------------------------------------------------------------
/docs/installation/pga-installation.md:
--------------------------------------------------------------------------------
1 | ## PGA Installation
2 |
3 | ### PGA 0.17 Installation Steps
4 | - Uses Ansible playbooks.
5 | - Link [1] to the playbook file on the ‘master’ branch.
6 | - This playbook consist of two roles, env_setup [2] and pga [3] and you can look at those roles for further details.
7 | - But first read [4]
8 | - Only for Cent OS 7
9 |
10 | [1] PGA Ansible Playbooks
11 | [2] Env_setup
12 | [3] PGA
13 | [4] READ ME
14 |
15 |
16 | Previous PGA version:PGA 0.16 Installation Steps
17 | NOTE: For gateway developers who prefers to install latest version using step by step instruction, can use the 0.16 instructions as a guide.
--------------------------------------------------------------------------------
/docs/technical-documentation/helix-job-management.md:
--------------------------------------------------------------------------------
1 | ## Job Management
2 |
3 | Job Management is a key component of Airavata and we recently upgraded the GFac based Job Management system to a new
4 | system which is more efficient and extendible. Job Management system mainly covers two areas
5 |
6 | 1. Executing workflows related to job submission and data transfers among compute and storage resources
7 | 2. Monitoring the status of already executing jobs in the remote compute resources
8 |
9 | For executing workflows, we use Apache Helix as a generic workflow engine. Workflow managers accept workflow execution
10 | requests from the orchestrator and job monitors and transfers those requests into the Helix cluster to execute them. Helix
11 | cluster is aware of how to decode those requests and execute workflows reliably and efficiently. Tasks in the Helix cluster
12 | uses Adaptor Support libraries to talk to external resources according the relevant protocol and security guidelines.
13 |
14 | 
15 |
16 | For job monitoring, new system supports multiple job monitoring mechanisms. Email monitoring is the primary method of
17 | monitoring jobs where it listens to the email alters sent by compute resources while the jobs' states were changed.
18 | Realtime monitoring is the new mechanism we added into the monitoring framework where it gets notified directly from the
19 | jobs when their states are changed. Using both of these mechanisms, we managed to achieve both reduced response time and
20 | better accuracy in job monitoring.
21 |
22 | 
--------------------------------------------------------------------------------
/docs/user-documentation/supported-applications.md:
--------------------------------------------------------------------------------
1 | ## Supported Applications
2 |
3 | ### Few Applications Supported by Airavata
4 | This list contains a sub section of application supported via gateways linked to Airavata.
5 | We can help you to run these in your own gateway.
6 |
7 | | Application | Availability |
8 | |:----------------- |:------------------------------------- |
9 | | Abaqus | Comet |
10 | | Abaqus-6.13-3 | Comet |
11 | | Abinit | BigRed, Bridges, Comet |
12 | | Amber_Sander | BigRed, Comet, Stampede |
13 | | AutoDock | BigRed, Karst |
14 | | AutoDock_Vina | Comet, Stampede |
15 | | CP2K | Comet, Stampede |
16 | | CPMD | Comet |
17 | | DDSCat | Comet |
18 | | DFTB+ | Stampede |
19 | | Gamess | BigRed, Comet, Gordon, Stampede |
20 | | Gaussian | BigRed. Bridges, Comet, Gordon |
21 | | Gaussian16 | Bridges |
22 | | Gromacs | BigRed, Bridges, Comet, Stampede |
23 | | Lammps | BigRed, Comet, Gordon, Stampede |
24 | | Molcas | Stampede KNL, Stampede |
25 | | NEK5000 | BigRed |
26 | | NWChem | Comet, Stampede |
27 | | Phasta_P | Stampede |
28 | | PSI4.0 | Comet |
29 | | QChem | Comet |
30 | | Quantum_Espresso | Comet, Bridges, JetStream, Stampede |
31 | | Tinker_Monte | Stampede |
32 | | Trinity | Karst |
33 | | WRF | Stampede |
34 |
35 |
36 |
37 | ### Application Inputs
38 | Inputs for all applications currently available in Airavata can be found Here!
39 |
40 |
41 |
42 | ### Application Configuration
43 | For sample configurations please refer [Admin Tutorials](admin-tutorials.md)
--------------------------------------------------------------------------------
/mkdocs.yml:
--------------------------------------------------------------------------------
1 | site_name: Apache Airavata
2 | site_url: https://airavata.apache.org/
3 | site_description: Apache Airavata Documentation
4 |
5 | repo_url: https://github.com/apache/airavata-docs
6 |
7 | docs_dir: docs
8 |
9 | pages:
10 | - Home: 'index.md'
11 | - User Documentation:
12 | - Gateway User Tutorials: 'user-documentation/user-tutorials.md'
13 | - Gateway Admin Tutorials: 'user-documentation/admin-tutorials.md'
14 | - Gateway Theme Implemenetation: 'user-documentation/cms-tutorials.md'
15 | - Advance Application Configuration: 'user-documentation/JSON-tutorials.md'
16 | - Apache Airavata User Groups: 'user-documentation/airavata-user-profiles.md'
17 | # - Application Support: 'user-documentation/supported-applications.md'
18 | #- System Configuration:
19 | # - Keycloak Configuration: 'configuration/keycloak-configuration.md'
20 | # - Django Configuration: 'configuration/pga-configuration.md'
21 | # - Django FAQ: 'maintenance/pga-maintenance.md'
22 | - Installation:
23 | - Apache Airavata Installation: 'installation/airavata-installation.md'
24 | - Django Installation: 'installation/pga-installation.md'
25 | # - Apache Airavata Server Properties: 'installation/airavata-server-properties.md'
26 | # - Code Update:
27 | # - Code Update:
28 | # - Apache Airavata Update: 'maintenance/code-update/airavata-update.md'
29 | # - PGA Update: 'maintenance/code-update/pga-update.md'
30 | - Technical Documentation:
31 | - Apache Airavata API: 'technical-documentation/airavata-api.md'
32 | - Helix Job Management: 'technical-documentation/helix-job-management.md'
33 | - Django Portal: 'technical-documentation/django-portal.md'
34 | - Sharing Documentation:
35 | - Introduction: 'technical-documentation/sharing-documentation/introduction.md'
36 | - API Overview: 'technical-documentation/sharing-documentation/api-overview.md'
37 | - Sharing CPI: 'technical-documentation/sharing-documentation/sharing-cpi.md'
38 | - Sharing Models: 'technical-documentation/sharing-documentation/sharing-models.md'
39 | - FAQ: 'faq.md'
40 | - Previous Releses: 'previous-releases.md'
41 | - Contact Us: 'contact-us.md'
42 |
43 | theme: readthedocs
44 |
45 | extra_css:
46 | - css/extra.css
47 |
48 | markdown_extensions:
49 | - toc:
50 | permalink:
51 | - admonition
52 |
--------------------------------------------------------------------------------
/docs/configuration/wso2-is-configuration.md:
--------------------------------------------------------------------------------
1 | ## WSO2 Identity Server (IS) Configuration
2 |
3 | The steps below are for IS configuration when IS is hosted by Airavata team. In this situation you will be a tenant created under Airavata hosted IS.
4 |
5 | ### Request a Tenant Account
6 | 1. Request from Airavata team for a new Tenant account. For this use
7 | Airavata mailing list
8 | OR
9 | HipChat
10 | 2. As the gateway admin, for the account creation provide;
11 | - First name
12 | - Last name
13 | - Preferred username & password as the admin (This is the username you will use to login to WSO2 IS and gateway as the admin user)
14 | - Email account
15 | 3. When Airavata team receives your request they will communicate with you if more information is required.
16 | 4. Once the tenant is created you will receive
17 | - URL for WSO2 IS
18 | - Username and password to login to WSO2 IS
19 |
20 |
21 | ### Configure Your Tenant
22 | 1. Login to IS server as the gateway admin tenant user.
23 | - Use the URL provided by the Airavata team; WSO2 URL for Gateway Admin
24 | - Use the username and password provided by you. (Please change your password in first login)
25 | 2. Navigate to Main Tab -->Service Providers --> Add
26 | - Give a service provider name and 'Register'
27 | - Out of the list below expand 'Inbound Authentication Configuration'
28 | - Out of the usb list underneath expand 'OAuth/OpenID Connect Configuration' and click 'Configure'
29 | - You will be navigated to 'Register New Application' UI.
30 | 3. To register new application;
31 | - Select OAuth Version = 2.0
32 | - From Allowed Grant Types select only 'Password' and 'Refresh Token'
33 | - Add
34 |
35 | 4. Copy the 'OAuth Client Key', 'OAuth Client Secret', Admin username, password and name to pga_config.php.
36 | 5. Navigate to Configure —> Claim management —>http://wso2.org/claims. And make "Supported by Default = true"
37 | 6. Navigate to Configure --> Users & Roles --> Users. Find the admin user and click 'User Profile'. Add the username at the bottom and update.
38 |
39 | You are all set to start configuring the gateway for job submission. For Gateway Configurations visit [PGA Configuration](pga-configuration.md).
40 |
--------------------------------------------------------------------------------
/docs/user-documentation/airavata-user-profiles.md:
--------------------------------------------------------------------------------
1 | ## Apache Airavata User Groups
2 |
3 | ### What Are Airavata User Groups?
4 |
5 | 1. Prior to using Airavata, lets identify and understand the user groups available and their features.
6 | 2. Knowing the groups and the features of each, will assist on shaping your gateway user hierarchy.
7 | 3. There are 3 base user groups in Airavata with different set of features at each level.
8 | - Admin
9 | - Admin-Read-Only
10 | - Gateway-User
11 |
12 | ### Features of each User Group
13 |
14 | 1. Admin Group
15 | - Set up gateway Group Resource Profiles.
16 | - Add gateway storage resource profile.
17 | - Generate SSH keys for compute and storage resource SSH communications..
18 | - Add the generated SSH key to group resource profiles.
19 | - Add the generated SSH keys to authorized_key files in each resource.
20 | - Create Applications and their deployments in to the gateway.
21 | - Manage users, add the mto base groups.
22 | NOTE: If user hosts his own gateway; 'Gateway Admin' role will have access to create compute resources and storage resources as well.
23 | 2. Admin-Read-Only
24 | - Can view everything in Settings but cannot Add, Edit or Delete any records from dashboard.
25 | - Can monitor experiments through Experiments Statistics in Experiment statistics.
26 |
27 | 3. Gateway User
28 | - Create, launch their own experiments in using available applications and compute resources.
29 | - Monitor progress of experiment execution.
30 | - Download experiment outputs from experiment summary or from the storage.
31 | - Upload files in to the storage for future use.
32 | - Group experiments by creating Projects.
33 | - Create own SSH keys from Credential store
34 | - Create Group Resource profiles to use individual HPC allocations for job submission through the gateway.
35 | - Password reset.
36 |
37 |
38 | ### How User Groups Work?
39 |
40 | 1. Users can belong to one or many user groups.
41 | 2. Two main categories of groups
42 | - Base groups - Created as default groups in the gateway at gateway deployment
43 | - User groups - Any user in the gateway can create user groups, and add other gateway users to the group.
44 | 3. Users in group will have one of the three roles. Three roles are:
45 | - Owner - Person who creates the user group
46 | - Admin - Person who can add other users in to the group. These users' role cannot be changed by users with admin role.
47 | - Member - This person can use benefits of the group, but cannot add or remove users from the group.
48 | 4. Only the owner of the group can change users; role.
49 | 5. Adding users to base groups can be done using two interfaces.
50 | - Settings → Manage Users
51 | - Top right-hand corner dropdown menu → Groups
52 | 6. Adding users to any other group that you are an admin of or the owner of can be done using
53 | - Top right-hand corner dropdown menu → Groups
54 | 7. In Manage Users page, users are listed with all the groups they are a member of.
55 |
56 |
57 |
--------------------------------------------------------------------------------
/docs/maintenance/code-update/airavata-update.md:
--------------------------------------------------------------------------------
1 | ## Apache Airavata Update
2 |
3 | 1. In order to update Airavata with latest master code; go to the folder (Created in installation; LocalAiravata) initially created to clone Airavata.
4 | 2. Within your local folder navigate to folder 'airavata' and execute
git pull
5 | Hint: If you are in the wrong folder you would probably get message
'fatal: Not a git repository (or any of the parent directories): .git'
6 | 3. Now build the code
7 |
mvn clean install
8 | Hint: To avoid the 'tests on install' run
mvn clean install -Dmaven.test.skip=true
9 | 4. Once the build is success, stop the running Airavata server. You can either stop the java server using
10 |
kill -9 <process ID>
11 | OR
12 | Navigate to bin folder where airavata server exists (LocalAiravata/apache-airavata-server-0.15-SNAPSHOT/bin). and stop the server
13 |
./airavata-server.sh stop -force
14 | 5. For references, back up the currently existing release
15 |
16 | 6. Copy the new release to your local folder (LocalAiravata)
17 |
cp /<path to created folder>/LocalAiravata/airavata/distribution/target/apache-airavata-server-0.16-SNAPSHOT-bin.tar.gz .
18 | 7. Un-tar the copied new release
19 |
tar -xvf apache-airavata-server-0.16-SNAPSHOT-bin.tar.gz
20 | 8. Navigate to the new bin folder and back up airavata-server.properties and gfac-config.yaml files.
21 | 9. Copy your previously used airavata-server.properties from the backed up release to bin folder (This is the easiest way to get the properties file updated. If you prefer, you can change the new file manually. ☺)
22 |
/<your local path>/LocalAiravata/LocalAiravata/apache-airavata-server-0.16-SNAPSHOT-bk/bin/airavata-server.properties .
23 | 10. Copy your previously used gfac-config.yaml from the backed up release
24 |
cp /<your local path>/LocalAiravata/apache-airavata-server-0.16-SNAPSHOT-bk/bin/gfac-config.yaml .
25 | 11. Now compare with the new airavata-server.properties and gfac-config.yaml and make necessary changes in the copied files.
26 | Try
31 | 14. Now restart airavata server in bin folder
32 |
./airavata-server.sh start
33 | 15. Airavata ready for job submissions!
34 | 16. For PGA updating steps try [PGA Update](pga-update.md).
35 |
--------------------------------------------------------------------------------
/docs/user-documentation/cms-tutorials.md:
--------------------------------------------------------------------------------
1 | ## Django Gateway Theme Management
2 | In Django portal, the gateway branding or the theme is managed through a CMS. The Djangop portal uses WagTail CMS for the theme.
3 | The CMS takes care fo the gateway home page, child pages content.
4 | Gateways are initially deployed with a common base theme, but the gateway admins can re-brand according to their needs.
5 | In order to change the theme user needs to be in the Admin group.
6 |
7 | ### Instructions on Gateway Theme
8 | NOTE: For details instructions please refer Github CMS Instructions
9 |
10 | 1. Login to your gateway portal.
11 | 
12 |
13 | 2. Once logged in navigate to Home page.
14 | 
15 |
16 | 3. Click on the Wagtail icon → Go to Wagtail Admin
17 | 
18 |
19 | 4. You are no in Wagtail CMS
20 | 
21 |
22 | 5. To create a new child page or to edit the existing Home page, navigate to Pages, on left.
23 | 
24 |
25 | 6. Keep using the arrow and navigating until the page you want to edit appears in the list. To edit the Home page, use the right arrow and then click on Home or the pen icon.
26 | 
27 |
28 | 7. Once the changes are done, Save as a draft or Publish or Preview from options at the bottom.
29 | 8. If you want to create a page as a child page of Home page, Click on the Pages link on the left side-bar.
30 | 9. You will get a side pop-up which will contain "Welcome to your new Wagtail Site" and a "pencil" icon and a "right-arrow" icon.
31 | 10. You will get "Home" Link. This is the default page. If you want to modify it you can click on "pencil" icon. if you already have any child pages to "Home", you will also be able to find a "right-arrow" icon
32 | 11. To create a new page as a child of "Home". you need to click on "Home" Link.
33 | 12. Then Click on "ADD CHILD PAGE" link.
34 | 13. Choose a page template type. Currently, three of them are available.
35 | - Blank Page -> (Build a page from scratch (Best Way))
36 | - Cybergateway Home Page -> (Build an IU Themed Home Page)
37 | - Home Page -> (Build a seagrid website based Home page)
38 | 14. Each page has four tabs namely:
39 | - CONTENT (On page load)
40 | - CUSTOMIZATION
41 | - PROMOTE
42 | - SETTINGS
43 | 15. Provide a title(required) for the page so that you can see it in the CMS.
44 | 16. Head over to the PROMOTE Section.
45 | - Provide a slug url ( which is the url extension at which the page will be available). for example if you are creating a documentation page. Provide a slug like documentation so that when you visit <--yourwebsite.com-->/documentation you will visit this page.
46 | - Provide a Page Title. This will show up in the page title of each page.
47 | 
48 |
49 | 17. Once the changes are done, you can preview, publish or save as a draft.
50 |
51 |
52 |
53 |
54 |
55 |
--------------------------------------------------------------------------------
/docs/index.md:
--------------------------------------------------------------------------------
1 | # Apache Airavata
2 |
3 | - A distributed framework that supports execution and management of computational scientific applications and workflows in grid based systems, remote clusters and cloud based systems.
4 | - Primarily focused on submitting and managing application executions and workflows in grid based systems.
5 | - Architecturally extensible to support other underlying resources.
6 | - Used by scientific gateway developers as their middleware layer. They can directly call Airavata API in order to communicate with grid based system.
7 | - Provides a desktop tools and browser-based web interface components for managing applications, workflows and generated data.
8 | - Contains sophisticated server-side tools for registering and managing scientific applications on computational resources.
9 | - The Django portal provides tools and methods to easily integrate with post processing applications and application centric interfaces.
10 |
11 | For more information on Airavata visit; Apache Airavata
12 |
13 |
14 |
15 |
16 |
17 |
18 |
19 |
23 |
24 | If you are interested in
25 | 1. Using an already hosted gateway
26 | OR
27 | 2. Having your own gateway hosted
28 | you can contact a downstream project Science Gateways Platform as a Service (SciGaP) and request your gateway.
29 | SciGaP runs Apache Airavata as a platform service.
30 |
--------------------------------------------------------------------------------
/docs/installation/wso2-is-installation.md:
--------------------------------------------------------------------------------
1 | ## WSO2 Identity Server (IS) Installation
2 |
3 | ### Installation
4 | 1. Download IS 5.1.0 from WSO2 Site and extract it.
5 | 2. Open <IS_HOME>/repository/conf/carbon.xml and set the HideAdminServiceWSDLs property to false.
6 |
7 | 3. To enable self signup and account confirmation follow the below steps
8 | - Open <IS_HOME>/repository/conf/claim-config.xml and add the following claims mappings in the <Dialect dialectURI="http://wso2.org/claims"> section.
9 |
28 | - Open <IS_HOME>/repository/conf/claim-config.xml and add the following claims mappings in the <Dialect dialectURI="http://wso2.org/oidc/claim"> section.
29 |
36 |
37 |
38 | - Enable the Identity Listener by setting the following property to true in the <IS_HOME>/repository/conf/identity/identity.xml file.
39 |
49 |
50 | - Configure the email-admin-config.xml file found in <IS_HOME>/repository/conf/email/ with the email template of type “accountConfirmation”.
51 |
52 | - Edit the org.apache.axis2.transport.mail.MailTransportSender section in the <IS_HOME>/repository/conf/axis2/axis2.xml to valid email account configuration.
53 | IS server should have access to this email account for remotely login.
54 |
65 |
66 | 4. Go to <IS_HOME>/bin and start IS server using ./wso2server.sh (use ./wso2server.sh start for starting in daemon mode).
67 |
68 | ### WSO2 IS Configuration
69 | 1. Creating a new tenant.
70 | 2. Registering a new OAuth Service Provider.
71 |
72 | ### Optional Steps
73 | 1. How to configure a mysql backend database for wso2 IS server - MySQL DB Configuration
74 | 2. How to configure a valid server certificate for the wso2 IS server - Configure Server Certificate
75 |
--------------------------------------------------------------------------------
/docs/technical-documentation/sharing-documentation/api-overview.md:
--------------------------------------------------------------------------------
1 |
2 | ## All Thrift Declarations
3 |
4 | | Module | Services | Data types | Constants |
5 | | ------ | -------- | ---------- | --------- |
6 | | sharing_cpi | [SharingRegistryService](sharing-cpi.md#Svc_SharingRegistryService)
| [`DO_NOT_SET_AT_CLIENTS_ID`](sharing-models.md#Const_DO_NOT_SET_AT_CLIENTS_ID) |
8 |
--------------------------------------------------------------------------------
/docs/installation/airavata-server-properties.md:
--------------------------------------------------------------------------------
1 | ## Apache Airavata Server Properties
2 |
3 | 1. API Server Registry Configuration
4 | - Comment out the derby DB properties
5 | - Change MySQL configurations
6 | - registry.jdbc.url=jdbc:mysql://localhost:3306/experiment_catalog (replace 'localhost' with correct server name if the DB is in a different server)
7 | - registry.jdbc.user=airavata
8 | - registry.jdbc.password=airavata
9 | - enable.sharing=true (This will set sharing within the gateway to be enabled. This is the advices mode)
10 | - default.registry.gateway=php_reference_gateway (Give the gateway name you prefer. Default exists in the file)
11 | - super.tenant.gatewayId=php_reference_gateway (Since you are hosting your own gateway this is the ID of your own gateway)
12 | 2. Application Catalog DB Configuration
13 | - Comment out the derby DB properties
14 | - Change MySQL configurations
15 | - appcatalog.jdbc.url=jdbc:mysql://localhost:3306/app_catalog
16 | - appcatalog.jdbc.user=airavata
17 | - appcatalog.jdbc.password=airavata
18 | 3. Replica Catalog DB Configuration
19 | - Comment out the derby DB properties
20 | - Change MySQL configurations
21 | - replicacatalog.jdbc.url=jdbc:mysql://localhost:3306/replica_catalog
22 | - replicacatalog.jdbc.user=airavata
23 | - replicacatalog.jdbc.password=airavata
24 | 4. Workflow Catalog DB Configuration
25 | - Comment out the derby DB properties
26 | - Change MySQL configurations
27 | - workflowcatalog.jdbc.url=jdbc:mysql://localhost:3306/workflow_catalog
28 | - workflowcatalog.jdbc.user=airavata
29 | - workflowcatalog.jdbc.password=airavata
30 | 5. Sharing Catalog DB Configuration
31 | - Comment out the derby DB properties
32 | - sharingcatalog.jdbc.url=jdbc:mysql://localhost:3306/sharing_catalog
33 | - sharingcatalog.jdbc.user=airavata
34 | - sharingcatalog.jdbc.password=airavata
35 | 6. Sharing Registry Server Configuration
36 | -
37 | 7. User Profile MongoDB Configuration
38 | -
39 | 8. Server Module Configuration
40 | - Make sure all servers required to start are added as given
41 | - servers=credentialstore,apiserver,orchestrator,gfac
42 | 9. API Server Configurations
43 |
44 | 10. API Server SSL Configurations
45 | - Give the correct path for key generation file. This is in the bin directory and it is shipped default with Airavata.
46 | - apiserver.keystore=/home/airavata/LocalAiravata/apache-airavata-server-0.16-SNAPSHOT/bin/airavata.jks
47 | 11. Orchestrator Server Configurations
48 |
49 | 12. GFac Server Configurations
50 |
51 | 13. Registry Server Configurations
52 |
53 | 14. Airavata Workflow Interpreter Configurations
54 |
55 | 15. Job Scheduler can send informative email messages.........
56 |
57 | 16. Credential Store module Configuration
58 | - Add the path to SSH key generation file
59 | - E.g.: credential.store.keystore.url=/home/airavata/LocalAiravata/airavata-sym.jks
60 | - Comment out the derby DB properties
61 | - Change MySQL configurations
62 | - credential.store.jdbc.url=jdbc:mysql://localhost:3306/credential_store
63 | - credential.store.jdbc.user=airavata
64 | - credential.store.jdbc.password=airavata
65 | - credential.store.keystore.url=/home/airavata/production-deployment/airavata_sym.jks
66 | 17. Monitoring Module Configuration
67 | - Add your email address, username and password for email monitoring. This is the email account the job status change emails will be received from compute resources.
68 | - email.based.monitor.host=imap.gmail.com
69 | - email.based.monitor.address=jobs@sample.org
70 | - email.based.monitor.password=SamplePassword
71 | 18. AMQP Notification Configuration
72 | - Users can use RabbitMQ as 'Guest' users. This is the easy method. For this uncomment
73 | - rabbitmq.broker.url=amqp://localhost:5672
74 | - To use as a 'Production' user
75 | - Navigate to RabbitMQ bin folder.
76 | - Make sure the RabbitMQ server is running. For production use
rabbitmq-server -detached
to start.
77 | - Create a virtual-host and user with a password. Follow documentation in RabbitMQ Users & VirtualHost
78 | - To create a user;
rabbitmqctl add_user Username Password
79 | - To create a vitrual-host
rabbitmqctl add_vhost vhostauthvhost
80 | - Provide permission to created 'Username' to the created vhost
81 | - Uncomment rabbitmq.broker.url=amqp://Username:Password@localhost:5672/Vhost. Add the created username, password and Vhost in the URL.
82 | - If you need to stop RabbitMQ use
rabbitmqctl stop
83 | - If the RabbitMQ server stopped then the above user creation, vhost creation and permission granting commands need to run again after restarting the servers.
84 | 19. Zookeeper Server Configuration
85 | - For 'Production' scenario make;
86 | - embedded.zk=false
87 | 20. Aurora Scheduler Configuration
88 |
89 |
90 | 21. API Security Configuration
91 | - Make sure
92 | - api.secured=false
93 | - TLS.enabled=false
94 |
--------------------------------------------------------------------------------
/docs/user-documentation/JSON-tutorials.md:
--------------------------------------------------------------------------------
1 | ##Advance Application Input Configuration
2 |
3 | ###Introduction
4 |
5 | The Django portal for Apache Airavata middleware is new and caters to wider range of requirements from Gateways.
6 |
7 | One major requirement was more flexibility in application inputs. In the previous PGA, inputs were Integer, String, Number and URI.
8 | User requirements were to have more options when adding these input types to their applications, such as option buttons, Cascading inputs, etc…
9 |
10 | In Django in order to advance the input types JSON scripting is used.
11 | When using JSON scripting, prior the developer/gateway admin need to add the component in to Django framework in order for it to work.
12 |
13 | ###String Input Validations
14 |
15 | 1. When String Inputs are provided, gateway to be able to
16 | - Define a minimum and a maximum length
17 | - Define valid characters (only X, Y, Z and 1)
18 | - Provide validation messages for users.
19 | - Ignore spaces between letters.
20 | 2. A text consisting X, Y, Z with minimum 10 character length and maximum 200 character length.
21 | ```
22 | {
23 | "editor": {
24 | "ui-component-id": "textarea-input-editor",
25 | "config": {
26 | "rows": 6
27 | },
28 | "validations": [
29 | {
30 | "type": "min-length",
31 | "value": 10
32 | },
33 | {
34 | "type": "max-length",
35 | "value": 200
36 | },
37 | {
38 | "type": "regex",
39 | "value": "^[XYL\\s]+$",
40 | "message": "Target sequence may only contain letters X, Y and L."
41 | }
42 | ]
43 | }
44 | }
45 | ```
46 | 3. “\\\s” in the value confirms ignoring any spaces in the sequence.
47 |
48 | ###Input List Validations
49 | ####Check Boxes
50 | 1. When providing inputs, users may need to provide one or many for a single input.
51 | ```
52 | {
53 | "editor": {
54 | "ui-component-id": "checkbox-input-editor",
55 | "config": {
56 | "options": [
57 | {
58 | "value": "a",
59 | "text": "A label"
60 | },
61 | {
62 | "value": "b",
63 | "text": "B label"
64 | },
65 | {
66 | "value": "c",
67 | "text": "C label"
68 | }
69 | ]
70 | }
71 | }
72 | }
73 | ```
74 |
75 | ####Radio Buttons
76 | 1. When users are required to select only one input value this is used. If there are more than 6 to select List-of-Values is a better option.
77 | 2. If you need to define a ‘default’ value, add the ‘value’ of that option in the Value field in application input. E.g.: If the default is lunch, then the value ‘lunch’ should be added to the Value field in Application input.
78 | ```
79 | {
80 | "editor": {
81 | "ui-component-id": "radio-button-input-editor",
82 | "config": {
83 | "options": [
84 | {
85 | "value": "breakfast",
86 | "text": "Breakfast"
87 | },
88 | {
89 | "value": "lunch",
90 | "text": "Lunch"
91 | },
92 | {
93 | "value": "dinner",
94 | "text": "Dinner"
95 | }
96 | ]
97 | }
98 | }
99 | }
100 | ```
101 |
102 | ####Lists
103 | 1. This is where the list is longer than 6 values.
104 | 2. A list longer than 6 values are two lengthy to display as option buttons. Users can use LOVs ( List of Values) if only a single value to be selected, and checkboxes if multiple values are required.
105 |
106 | ###Cascading Inputs
107 | 1. Cascading inputs are child inputs based on a prior selection.
108 | 2. E.g.: Show value B if value for A is ‘Pizza’. If the input for A is ‘pizza’ Then display B with its values for user to select.
109 |
110 | ```
111 | {
112 | "editor": {
113 | "dependencies": {
114 | "show": {
115 | "OR": [
116 | {
117 | "A": {
118 | "comparison": "equals",
119 | "value": "pizza"
120 | }
121 | },
122 | {
123 | "C": {
124 | "comparison": "equals",
125 | "value": "cake"
126 | }
127 | }
128 | ]
129 | }
130 | }
131 | }
132 | }
133 | ```
134 |
135 | 3. In above, OR means that B to display if value for A is ‘pizza’ or value for C is ‘cake’. In place of OR can use AND and NOT.
136 | 4. To toggle properties on the input field, such as "requiredToAddedToCommandLine", add "showOptions" to the JSON config with a value of "toggle" and the names of properties to toggle:
137 | ```
138 | {
139 | "editor": {
140 | "dependencies": {
141 | "show": {
142 | "OR": [
143 | {
144 | "A": {
145 | "comparison": "equals",
146 | "value": "pizza"
147 | }
148 | },
149 | {
150 | "C": {
151 | "comparison": "equals",
152 | "value": "cake"
153 | }
154 | }
155 | ]
156 | },
157 | "showOptions": {
158 | "toggle": ["requiredToAddedToCommandLine"]
159 | }
160 | }
161 | }
162 | }
163 | ```
164 |
165 | ###Dependant File Uploads
166 | 1. Dependant input is a input that is dependant on certain option selected in a previous field. E.g.: If the option selected is ‘list-of-urls’ then show this field to upload a file.
167 | 2. If the selected option is ‘list-of-urls’ then the URI upload should be enabled.
168 | ```
169 | {
170 | "editor": {
171 | "dependencies": {
172 | "show": {
173 | "Select reading options": {
174 | "comparison": "equals",
175 | "value": "list-urls"
176 | }
177 | }
178 | }
179 | }
180 | }
181 | ```
--------------------------------------------------------------------------------
/docs/maintenance/pga-maintenance.md:
--------------------------------------------------------------------------------
1 | ## PGA Maintenance
2 |
3 |
4 | 1. Investigate FAILED experiments/jobs
5 | 2. Handling unavailable Compute Resources
6 | 3. Managing User Accounts
7 | 4. Changing roles are not providing proper access
8 | 5. Community account ran out of SUs
9 | 6. Add Pre and Post Job Commands for an Application
10 | 7. Add Environment Variables for an Application Deployment
11 | 8. Register your Compute & Storage Resources
12 | 9. Gateway Preference Management
13 | 10. Communication with Gateway Users
14 | 11. Monitor Gateway Traffic
15 | 12. Using Credential Store
16 |
17 | Q1. One of my gateway users wants me to investigate his FAILED experiment. How should I proceed?
18 | Answer: To investigate a specific FAILED experiment or failed experiment within a time frame,
19 | 1. Navigate to AdminDashboard ⇒ Experiment Statistics
20 | 2. Search the experiment by using the experiment ID.
21 | OR
22 | 3. Search by providing the creation time period. You can further filter by giving
23 | - Gateway username
24 | - Application
25 | - Compute Resource
26 | 4. You could also use pre-defined selection criteria ('Get Experiments from Last 24 Hours' OR 'Get Experiments from Last Week')
27 | 5. Once the experiment/s listed click on the 'Check Stats'
28 | 6. Experiment task breakdown is listed and as the admin you could locate which task failed. Advice the gateway user accordingly.
29 |
30 | Q2. One of the resources used by my gateway is not available for the day. How to stop user job submissions?
31 | Answer: To temporarily stop users submitting jobs to a particular resource...
32 | 1. Navigate to AdminDashboard ⇒ Compute Resources (Browse)
33 | 2. Un-check the 'Enabled' box for the specific resource. This disables job submissions for the resource
34 | 3. To enable job submission simply check the box and you are back in track!
35 | NOTE: In order to enable disable resources you require super admin rights to the gateway. If not you need to contact SciGaP admins.
36 |
37 | Q3. How to upgrade access for a gateway user?
38 | Answer: User access can be upgraded or downgraded by changing the assigned user role.
39 | 1. Navigate to Admin Dashboard ⇒ Users (Browse)
40 | 2. Search for the specific user and click 'Check All Roles'
41 | 3. From this you can remove/add roles.
42 | 4. For each modification user will receive an email with 'Your Privileges has Changed!'
43 |
44 | Q4. Gateway users who has role 'gateway-user' cannot access experiment creation. Seems like privileges are not accessible.!
45 | Answer: User roles are given through PGA Admin Dashboard. If the roles are set correctly then check pga_config.php
46 | 1. To check the config file use
47 |
vi /var/www/html/airavata-php-gateway/app/config/pga_config.php
48 | 2. Roles attached to users should exists in the config file against correct role attribute type
49 |
50 | /**
51 | * Admin Role Name
52 | */
53 | 'admin-role-name' => 'admin',
54 | /**
55 | * Read only Admin Role Name
56 | */
57 | 'read-only-admin-role-name' => 'admin-read-only',
58 | /**
59 | * Gateway user role
60 | */
61 | 'user-role-name' => 'airavata-user',
62 |
63 |
64 | NOTE: In order to view and change config file you require access to PGA backend. If not you need to contact SciGaP admins.
65 |
66 | Q5. I have ran out of allocation for my current community account used in gateway for a resource. What should I do?
67 | Answer:
68 | 1. If you have another community account you could update the information in Admin Dashboard ⇒ Gateway Profile under Compute Resource Preferences.
69 | 2. Select the resource and modify account information and save.
70 | 3. Login to the resource as the new user and update the authorized_keys with the public key assigned to the resource.
71 |
72 | Q6. I want to send notices to my gateway users. How?
73 | Answer:
74 | 1. Navigate to Admin Dashboard ⇒ Notices
75 | 2. Use 'Create a New Notice' button add a new notice.
76 | 3. Based on the published date you provided it will be available for users as a Notice !
77 |
78 | Q7. How to open the gateway to any user who create an user account? In the gateway not required to enable/activate user accounts.
79 | Answer:
80 | 1. Gateway admin can switch between options of opening the gateway to all account creations OR gateway admin to activate accounts after creation.
81 | 2. In order to do open the gateway to all,
82 | - If the gateway is hosted by Airavata team, please request from them
83 | - If you are hosting the gateway, navigate to
vi /var/www/html/airavata-php-gateway/app/config/pga_config.php
Then change the initial user role to 'gateway-user'
84 |
85 | /**
86 | * Initial user role. This is the initial user role assigned to a new
87 | * user. Set this to one of the three roles above to automatically
88 | * grant new users that role, or set to some other role ('user-pending')
89 | * to require admin approval before users have access.
90 | */
91 | 'initial-role-name' => 'gateway-user',
92 |
93 |
94 | Q7. I want my gateway users to use their own compute resource allocation to submit jobs through the gateway. How to do this?
95 | Answer:
96 | 1. Users can add their own allocation account details.
97 | 2. In order to do so first login to the gateway.
98 | 3. Then Go to User Settings ⇒ Compute Resources
99 | 4. Click the 'Add a Compute Resource Account' and enter all the details and save.
100 |
101 | After at experiment creation,edit and clone
102 | 1. When you select the compute resource which has your account defined in the gateway, default option is to submit the job using user's own account.
103 | 2. User has the option of selecting community account for job submission as well.
--------------------------------------------------------------------------------
/docs/configuration/pga-configuration.md:
--------------------------------------------------------------------------------
1 | ## PGA Configuration
2 | ### You Need to Have
3 | 1. Airavata & PGA installed and running.
4 | 2. Tenant created in WSO2 IS hosted by you or Airavata team.
5 | 3. Administrator privileges to the gateway. (How do you know? - Can you view Admin Dashboard and view buttons to create applications)
6 |
7 |
8 | ### Apache Airavata Component Configuration
9 | 1. For this we use 'Admin Dashboard'
10 | 2. Gateway Admin need to configure;
11 | - Compute Resources
12 | - Storage Resources
13 | - Gateway Management
14 | - Application Catalog
15 | - Application Module
16 | - Application Interface
17 | - Application Deployment
18 | - Credential Store
19 | - Generate Credential Store Token + SSH Key.
20 | - Add these to authorized key files and into resource preferences.
21 | - Notices
22 | - Notices for all active gateway users.
23 | - Identity Management
24 |
25 | NOTE: for gateways hosted by SciGaP gateway admins/PIs not required to register the compute resources and storage resource. Please skip the 1st and 2nd and go to Gateway Management section.
26 |
27 | ###Compute Resources
28 | 1. Navigate to Admin Dashboard ⇒ Compute Resources ⇒ Register
29 | 2. Add Host Name, Description and create the resource.
30 | Hint: host name is used when ssh to the resource from airavata.
31 | 3. Then keep adding information on rest of the appeared tabs.
32 | - Queues (Queue name is unique and cannot be updated. Can delete and create new if required)
33 | - File Systems (This is only for information capturing for future use. Currently this information is not used. So can skip if you want)
34 | - Job Submission Interfaces
35 | - Data Movement Interfaces
36 | 5. Similarly you can add multiple compute resources in to your gateway by selecting 'Register' from the left-hand-side menu.
37 | 6. To view the added compute resources navigate to Admin Dashboard ⇒ Compute Resource ⇒ Browse
38 | 7. All the resources will be listed. Gateway admin can view, edit, delete and enable and disable them.
39 |
40 |
41 | ###Storage Resources
42 | 1. Navigate to Admin Dashboard ⇒ Storage Resources ⇒ Register
43 | 2. Add Storage Name, Description and create the resource.
44 | 3. Then add data storage information in
45 | - Data Movement Interfaces
46 | 4. Similarly you can add multiple storage resources in to your gateway by selecting 'Register' from the left-hand-side menu.
47 | 5. To view the added resources navigate to Admin Dashboard ⇒ Storage Resources ⇒ Browse
48 | 6. All the resources will be listed. Gateway admin can view, edit, delete them.
49 | 7. Although enable and disable can be done in registration it's functionality is not yet implemented.
50 |
51 |
52 | ###Gateway Management of Resources
53 | 1. Navigate Admin Dashboard ⇒ Gateway Management
54 | 2. Both compute resource and storage resource specific preferences are defined here.
55 | 3. To add compute resource related preferences click "Add a Compute Resource Preference" and select the resource from the drop-down list.
56 | 4. Add/select preferred options and click "Set preferences".
57 | Repeat this for all the resources used within the gateway.
58 | 4. For each compute resource, gateway admin need to specify;
59 | - Preferences can be overridden by Airavata - Yes/No?
60 | - Resource login name
61 | - Preferred job submission protocols
62 | - Preferred data movement protocols
63 | - Preferred queue
64 | - Scratch location
65 | - Project allocation number
66 | - Resource specific credential store token (When added Base Credential Store Token is not valid for the specific resource)
67 | - Gateway ID for usage reporting (This field will only appear if the compute resource registration is set to report usage details back to the resource)
68 | - Quality of service
69 | - Reservation name
70 | - Reservation start and end date time
71 | 5. For adding storage resource preference click "Add a Storage Resource Preferences", and rest is similar to adding a compute resource preference.
72 | 6. For a gateway currently when a storage resource is selected, that resource ID need to be added in to the pga_config.php file in config folder of the hosted gateway.
73 | 7. For storage resource preference, gateway admin need to add;
74 | - Login username
75 | - File System Root Location
76 | - Resource Specific Credential Store Token
77 | 8. Apart from adding preference the same interface is used to assign a 'Base Credential Store Token". If this is added this is the token used across the gateway for communication with all the compute resources and storage resource.
78 |
79 |
80 | ###Application Catalog
81 | 1. Users in Admin group can add applications in to the gateway.
82 | 2. Adding an application involves adding details in to the three tabs - Details, Interface and Deployments.
83 | 4. What each tab means and capture?
84 | - Application Module
85 | - Navigation: Admin Dashboard ⇒ App Catalog ⇒ Module
86 | - This is the simple introduction of the application; Name, Version and Description.
87 | - Click 'Create a New Application Module', provide information and create. On creation Application Module ID will be generated for the module.
88 | - All existing modules are listed; Search by name is available for a particular module.
89 | - Gateway admin can edit, delete existing modules.
90 | - Deleting a module will be restricted if it has application interfaces and deployments linked.
91 | - Application interface
92 | - Navigation: Admin Dashboard ⇒ App Catalog ⇒ Interface
93 | - Application interface defines the required inputs, outputs produced and their characteristics.
94 | - Click on 'Create a New Application Interface', provide information and create. On creation Application Interface ID will be generated for the module.
95 | - All available interfaces are also listed; admin has the option of searching for a particular interface by providing the name in the search.
96 | - Gateway admin can edit, delete existing interfaces.
97 | - Gateway admin cal also clone an existing interface in order to create a new similar interface with slight changes.
98 | - Application deployment
99 | - Navigation: Admin Dashboard ⇒ App Catalog ⇒ Deployment
100 | - Application deployment describes application deployment details on a specific resource.
101 | - For an application for each resource there is a separate deployment.
102 | - A single application can be deployed in multiple resources.
103 | - Multiple application modules can be deployed in a single resource. E.g: Gaussian09 and Gaussian16 both exists in bridges.psc.edu and they both use same application interface.
104 |
105 |
106 | ###Credential Store
107 | 1. Navigation: Settings ⇒ Credential Store
108 | 2. This interface is used to generate SSH key + token pairs.
109 | 3. These generated keys can be added in to the authorized key files in each resource for SSH key based communication.
110 | 4. Generated key can be either assigned at gateway level or/and at individual resource allocation level; one key + token pair for all the resources OR have separate key for each resource.
111 | 5. SSH keys are used for communication with compute resources and storage resources.
112 |
113 | ###Keycloak Configuration
114 |
115 |
--------------------------------------------------------------------------------
/docs/technical-documentation/airavata-api.md:
--------------------------------------------------------------------------------
1 | ## Apache Airavata API
2 |
3 | For latest Airavata API documentation please visit Airavata Master Branch (0.20) API Documentation
4 |
5 | Legacy Airavata API documentation please visit:
6 | Airavata 0.16 API Documentation
7 |
8 | ###
Airavata APIs for Experiments & Projects
9 |
10 | | Gateway Function/Feature | Airavata API | Description |
11 | |:--------------------------------------|:----------------------------------|:------------------------------|
12 | Project
13 | | Create a Project | createProject | Linked with Create Project in PGA. |
14 | | Update Project | updateProject | To update Project name and description. |
15 | | Get a Project | getProject | Retrieve Project by providing the ID. |
16 | | Search Project by Name | searchProjectsByProjectName | Search for Project by giving part or full project name. |
17 | | Search Project by Desc | searchProjectsByProjectDesc | Search for Project by giving part or full project description. |
18 | | Get all user Projects | getUserProjects | Retrieve all Projects of a user. |
19 | Experiment
20 | | Create an Experiment | createExperiment | Create an Experiment. |
21 | | Update an Experiment | updateExperiment | Update ab Experiment. Experiments with CREATED exp-status can be updated. |
22 | | Get an Experiment | getExperiment | Retrieve Experiment by providing the experiment ID. |
23 | | Get Detailed Experiment | getDetailedExperimentTree | Retrieve detailed Experiment by providing the experiment ID. |
24 | | Clone an Experiment | cloneExperiment | Clone an existing Experiment. Experiment with any exp-status can be cloned by providing the ID. |
25 | | Cancel an Experiment | terminateExperiment | Cancel an existing Experiment. Experiments with exp-statuses LAUNCHING or EXECUTING can be cancelled. |
26 | | Search Experiment by Name | searchExperimentsByName | Search Experiment by giving full or part of the name. |
27 | | Search Experiment by Desc | searchExperimentsByDesc | Search Experiment by giving full or part of the description. |
28 | | Search Experiments by Application Name | searchExperimentsByApplication | Search Experiment by giving the application name. |
29 | | Search Experiment by Creation Time | searchExperimentsByCreationTime | Search Experiment by giving creationTime period. |
30 | | Get all user Experiments | getUserExperiments | Search for all the experiments of a single user. |
31 | | Get Experiments for a Project | getExperimentsInProject | Retrieve all the experiments in a particular Project. |
32 |
33 |
34 |
35 |
36 |
37 | ###
Airavata APIs for Admin Dashboard
38 |
39 | | Admin Function/Feature | Airavata API | Description |
40 | |:----------------------------------------------|:--------------------------------------|:------------------------------------------------------------------------------------------------ |
41 | Add Gateway
42 | | Add a gateway | addGateway | Adding a new Gateway. |
43 | Credential Store
44 | | Generate a Token/SSH Key | generateAndRegisterSSHKeys | Generate new SSH Key and Token. |
45 | | Get all Credential Store Tokens | getAllGatewaySSHPubKeys | Retrieve all the generated keys of a Gateway. |
46 | | Remove a Token/SSH Key | deleteSSHPubKey | Select and delete a particular SSH Key Token pair. |
47 | Compute Resource (CR)
48 | | Get all application deployed CRs | getAllApplicationDeployments | Get all Application deployed Compute Resources. |
49 | | Get a CR | getComputeResource | Retrieve Compute Resource information by providing the resource ID. |
50 | | Register CR | registerComputeResource | Register a new Compute Resource. This is Super Admin Feature.|
51 | | Update CR | updateComputeResource | Retrieve an existing Compute Resource and update.|
52 | | Enable and Disable CR | getComputeResourceupdateComputeResource | Retrieve the CR and enable or disable through update. This is a Super Admin feature.|
53 | | Delete a Queue | deleteBatchQueue | Delete a selected Queue from the Compute Resource. |
54 | Storage Resource (SR)
55 | | Get all SR Names | getAllStorageResourceNames | Retrieve all storage resources of the gateway. |
56 | | Get a SR | getStorageResource | Fetch Storage Resource by providing the Storage ID. |
57 | | Register a SR | registerStorageResource | Register a new Storage Resource. This is a Super Admin Feature.|
58 | | Update a SR | updateStorageResource | Update and existing Storage Resource.|
59 | | Enable and Disable SR | getStorageResourceupdateStorageResource |Retrieve the SR and enable or disable through update. This is a Super Admin feature.|
60 | | Delete SR | deleteStorageResource |Select and delete and existing Storage Resource. |
61 | | Delete Data Movement Interface of SR | deleteDataMovementInterface |Delete a Data Movement Interface of a Storage Resource|
62 | Experiment Statistics
63 | | Get Experiment Statistics | getExperimentStatistics | Displays experiments grouped by the experiment status and derived for the given date time range. |
64 | Gateway Preferences
65 | | Add CR Preference for a gateway | addGatewayComputeResourcePreference ||
66 | | Edit CR Preference | addGatewayComputeResourcePreference ||
67 | | Delete a CR Preference | ||
68 | | Add SR Preference for a gateway | addGatewayStoragePreference ||
69 | | Edit SR Preference | addGatewayStoragePreference ||
70 | | Delete a SR Preference | ||
71 | Other
72 | | Get all Notices | noticesView | View all existing Notices. |
73 |
74 |
75 |
76 |
77 |
78 |
79 | If any questions or clarification regarding the API documentation please contact us through;
80 | Airavata Mailing List OR
81 | HipChat
82 |
--------------------------------------------------------------------------------
/docs/faq.md:
--------------------------------------------------------------------------------
1 | ## FAQs
2 |
3 | Installation FAQs
4 | Configuration FAQs
5 | User FAQs
6 |
7 | ###
Installation FAQs
8 | Q1. I have setup my own gateway and Airavata. When I log into the gateway I cannot create Compute resources. What should I do?
9 | Answer: In your pga_config.php (in folder .../testdrive/app/config) under heading 'Portal Related Configurations' set 'super-admin-portal' => false, to true.
10 |
11 | Q2. I don't get notifications when users create new accounts in my gateway. Why?
12 | Answer: That's because you have not defined an email address in 'admin-emails' => ['xxx@xxx.com','yyy@yyy.com']. Here you can add one or many.
13 |
14 |
Q3. I am not receiving email notifications from compute resources for job status changes. What should I do?
15 | Answer: In airavata-server.properties please locate and set your email account information.
16 |
Q4. In my Airavata log I have error messages like
22 | ERROR org.apache.airavata.api.server.handler.AiravataServerHandler - Error occurred while retrieving SSH public keys for gateway
23 | ERROR org.apache.airavata.credential.store.server.CredentialStoreServerHandler - Error occurred while retrieving credentials
24 | What should I do?
25 | Answer: This could be due to missing tables in your credential store database. Check whether CREDENTIALS and COMMUNITY_USER tables exits. If not create then using
26 |
CREATE TABLE CREDENTIALS
36 | (
37 | GATEWAY_ID VARCHAR(256) NOT NULL,
38 | TOKEN_ID VARCHAR(256) NOT NULL,
39 | CREDENTIAL BLOB NOT NULL,
40 | PORTAL_USER_ID VARCHAR(256) NOT NULL,
41 | TIME_PERSISTED TIMESTAMP DEFAULT NOW() ON UPDATE NOW(),
42 | PRIMARY KEY (GATEWAY_ID, TOKEN_ID)
43 | );
44 |
45 |
46 |
Q5. I cannot login to my Compute Resource and launch jobs from Airavata using the SSH key I generated. What should I do?
47 | Answer: Steps to use generated SSH key
48 | - Generate SSH key + token using Credential Store
49 | - Add the generated token to your resource through PGA --> Admin Dashboard --> Gateway Preferences
50 | - Add the generated SSH key
51 |
52 | Q6. When installing PGA in MAC i got below error after updating the composer.
53 | - Error
54 | Mcrypt PHP extension required.
55 | Script php artisan clear-compiled handling the post-update-cmd event returned with an error
56 | [RuntimeException]
57 | Error Output:
58 | Answer:
59 | Install mcrypt installation;
60 | mcrypt Installation Steps
61 |
62 | Q7. After following the required steps only the home page is working and some images are not shown properly.
63 | Answer: If you are facing this behavior first check whether you have enabled mod_rewrite module in apache webserver.
64 | And also check whether you have set AllowOverride All in the Vhost configuration file in apache web server. (e.g file location is /etc/apache2/sites-available/default and there should be two places where you want to change)
65 |
76 |
77 |
78 | Q8. I get the Error message Permission Denied to app/storage directory.
79 | Answer: Execute the following command and grant all permissions;
sudo chmod -R 777 app/storage
80 |
81 | Q9. In Ubuntu environment when executing sudo composer update it fails with message "Mcrypt PHP extension required".
82 | Answer: To fix this install PHP mcrypt extension by following the below steps;
83 |
sudo apt-get install php5-mcrypt
84 | Locate mcrypt.so ,to get its location
85 | Locate mcrypt.ini and open the mcrypt.ini file
86 |
sudo pico /etc/php5/mods-available/mcrypt.ini
87 | Change the at line a extension= eg:/usr/lib/php5/20121212/mcrypt.so Save changes.
88 | Execute the command:
sudo php5enmod mcrypt
89 | Now restart the apache server again and test PGA web-interface
90 |
91 | Q10. When tried to login or create a new user account an Error is thrown which is similar to PHP Fatal error: SOAP-ERROR: Parsing WSDL: Couldn't load from...
92 | Answer: If you face this kind of an error first check whether you have enabled PHP SOAP and OpenSSL extensions. If even after enabling them the issue is still occurring try updating the PHP OpenSSL extension. (Using command like yum update openssl)
93 |
94 | Q11. If you are seeing an error similar to following in your airavata.log
95 | Error: ERROR org.apache.airavata.registry.core.app.catalog.impl.StorageResourceImpl - Error while retrieving storage resource...
96 | javax.persistence.NoResultException: Query "SELECT p FROM StorageResource p WHERE p.storageResourceId =:param0" selected no result, but expected unique result.
97 | Answer: Add storage resource ID in to the pga_config.php in your gateway code/app/config directory.
98 |
99 | Q12. I am getting error
100 | 2016-05-19 16:17:08,225 [main] ERROR org.apache.airavata.server.ServerMain - Server Start Error:
101 | java.lang.RuntimeException: Failed to create database connection pool.
102 |
103 | What should i do?
104 | Answer: Airavata cannot create database connection because the mysql jar is not existing. Please follow step 8 of documentation in Installation --> Airavata --> Airavata Installation
105 |
106 | ###
Configuration FAQs
107 | Q1 What each application input property mean?
108 | Answer:
109 | - Name:
110 | Identifier of the application input
111 | - Value:
112 | This could be a STRING value or it can also be used to set input file name.
113 | - Type:
114 | Input type, List contain STRING, INTEGER, FLOAT, URI, STDOUT and STDERR
115 | - Application Arguments:
116 | These are are the characters you would want on commandline in job execution for each input file or character input.
117 | - Standard Input:
118 | Futuristic property and not in real use at the moment
119 | - User Friendly Description:
120 | A description about the input to the gateway user. This will be displayed for users at experiment creation.
121 | - Input Order:
122 | this is a number field. This will be the order inputs displayed in experiment creation.
123 | - Data is Staged:
124 | - Is the Input Required:
125 | Futuristic property and not in real use at the moment. Whether set to true or false all inputs are currently treated as mandatory.
126 | - Required in Commandline:
127 | When this is set to true the arguments and the input file or the value will be available in job execution commandline in job script.
128 | - Meta Data:
129 |
130 | Q2 Application Output Properties
131 | Answer:
132 | Add Application Output
133 | - Name:
134 | This is the label for the output.
135 | - Value:
136 | This would be the actual name of the output airavata brings back to the PGA.
137 | - Type:
138 | Tye of the output. This is mostly depended on the application. To troubleshoot for almost all applications define STDOUT and STDERR
139 | - Application Argument:
140 | This would be arguments for outputs need to be in commandline.
141 | - Data Movement:
142 | Futuristic property and not in real use at the moment. Whether set to true or false all outputs are currently brought back to PGA.
143 | - Is the Output required?:
144 | - Required on command line?:
145 | When this is set to true the arguments and the output file or the value will be available in job execution commandline in job script.
146 | - Location:
147 | - Search Query:
148 |
149 | Q3 How to make input file available as an executable?
150 | Answer:
151 | - Input files defined are copied to the experiment working directory.
152 | - Input files will be available in commandline when 'Required on Commandline' = true
153 | - To add a commandline argument for a input file add 'Application Argument' for each input file. This will also define the order of files in commandline.
154 |
155 | Q4 In Application Interface what is the use of 'Enable Optional File Inputs'
156 | Answer:
157 | - By setting 'Enable Optional File Inputs' = true user can add none or many input files at experiment creation.
158 | - In Airavata any input file required for the application to execute need to be defined as a separate input.
159 | - When inputs are defined they are treated as 'Mandatory' inputs.
160 |
161 | Q5 Where do I add remote resource execution commands?
162 | Answer: In Admin Dashboard --> Application Deployment
163 | - Add module load commands
164 | - Pre and post job commands
165 | - Environment variables
166 |
167 | ###
User FAQs
168 | Q1 What is the Project?
169 | Answer:
170 | - Project is simply a collection of experiments.
171 | - When creating an experiment it will be under the project you select.
172 | - Projects can be shared with fellow gateway users.
173 | - When the project is shared all experiments in the project will be shared as well.
174 |
175 | Q2 Where can I get test inputs?
176 | Answer: If you need test inputs try downloading from Test Input Files
177 |
178 | Q3 Do I need to provide values for wall-time, node count and CPU count? OR can I go ahead with the given default values?
179 | Answer: Default values are given to make life little easy for users. But depending on the job you should change the values. E.g.: If you need only two or less cores than 16 better to change. If you need more wall-time change it, etc....
180 |
181 |
182 | Q4 What can I do with email Notifications in experiment?
183 | Answer: Submitting a job from PGA does not guarantee job getting executed in the remote resource right away. If you add your email address, when the job starts and completes an email will be sent to you from the remote resource.
184 |
185 | Q5 Why 'Save' and 'Save and Launch'?
186 | Answer: User has the option of either create and 'Save' the experiment for later launch at remote resource Or
187 | can directly 'Save and Launch' at once.
188 |
189 | Q6 How do I monitor progress?
190 | Answer: When the experiment is launched navigate to Experiment Summary Page.
191 | There the experiment and job status will be present.
192 | You can monitor the experiment and job status changes. Refresh the status using 'Refresh' icon on top of the summary page.
193 |
194 | Q7 How do I view/download my outputs?
195 | Answer: Same way you monitor the experiment progress!
196 | For each experiment inputs can be downloaded from
197 | - Experiment Summary page
198 | - From 'Storage' at the top menu. Here you need to know the Project the experiment belongs to.
199 | Once you locate the project you can browse for your experiment.
200 |
201 | Q8 I want to bring back all the outputs generated by my job. How?
202 | Answer: You need to request your gateway admin to simply set Archive to 'true' in respective application interface in Admin Dashboard. Then all the files in the working directory will be brought back to PGA. This flag is set at gateway level not at individual user level.
203 |
204 | Q9 I want to cancel my experiment. How?
205 | Answer: Navigate to experiment Summary and click 'Cancel' button.
206 | Experiments only in operation (LAUNCHED, EXECUTING experiment statuses) can be cancelled.
207 |
208 | Q10 2. When I cancel do I still receive any outputs generated?
209 | Answer: Files will be not transferred from the remote resource if the experiment is cancelled. However, if the output files were transferred to PGA data directories prior to the cancel request then they will be displayed.
210 |
211 | Q11 How can I run same application with different different inputs?
212 | Answer: Simply clone an existing experiment and change the input files and launch.
213 |
214 | Q12 I want to change the wall-time of my experiment before launch. How can I do?
215 | Answer: Experiments in CREATED state can be modified through Experiment Summary page. Click 'Edit' change values and Save or Save and Launch.
216 |
--------------------------------------------------------------------------------
/docs/technical-documentation/sharing-documentation/introduction.md:
--------------------------------------------------------------------------------
1 | ## Welcome to Airavata Sharing Registry Service Documentation
2 |
3 | Airavata Data Sharing Registry Service is a general purpose **Collaborative Workspace Management Component** that can solve your Scientific Data Management requirements related to sharing and access controlling.
4 |
5 | !!! note
6 | This component is under active development and this document will keep on evolving. Click [here](api-overview.md) to see the API docs.
7 |
8 |
13 |
14 |
15 |
16 | ### Getting started with the Java client
17 |
18 |
57 | //should use the correct host name and port here
58 | String serverHost = "gw56.iu.xsede.org";
59 | int serverPort = 7878;
60 |
61 | TTransport transport = new TSocket(serverHost, serverPort);
62 | transport.open();
63 | TProtocol protocol = new TBinaryProtocol(transport);
64 | SharingRegistryService.Client sharingServiceClient = new SharingRegistryService.Client(protocol);
65 |
66 |
67 |
68 |
69 |
2. Create a domain
70 |
71 | Domain domain = new Domain();
72 | //has to be one word
73 | domain.setName("test-domain");
74 | //optional
75 | domain.setDescription("test domain description");
76 |
77 | //domain id will be same as domain name
78 | String domainId = sharingServiceClient.createDomain(domain);
79 |
80 |
81 |
82 |
83 |
2. Create Users
84 |
85 | User user1 = new User();
86 | String userName = "test-user-1";
87 | String userId1 = "test-user-1";
88 | //required
89 | user1.setUserId(userId1);
90 | //required
91 | user.setUserName(userName);
92 | //required
93 | user1.setDomainId(domainId);
94 | //required
95 | user1.setFirstName("John");
96 | //required
97 | user1.setLastName("Doe");
98 | //required
99 | user1.setEmail("john.doe@abc.com");
100 | //optional - this should be bytes of the users image icon
101 | //byte[] icon = new byte[10];
102 | //user1.setIcon(icon);
103 |
104 | sharingServiceClient.createUser(user1);
105 |
106 | //Similarly create two more users user2 and user3.
107 | .
108 | .
109 | .
110 |
355 |
--------------------------------------------------------------------------------
/docs/user-documentation/admin-tutorials.md:
--------------------------------------------------------------------------------
1 | ## Gateway Admin Tutorial
2 |
3 | ###Quick Admin Instructions
4 | This documentation is a quick set-up instructions for gateway Settings configurations for Admins. For more details please refer to tutorials provide as well.
5 | Django Portal Admin Instructions
6 |
7 | ## Gateway Admin Tutorials
8 | ### Prior to starting your configurations
9 | 1. You need to have admin access to the Django portal, Settings. You need to be a member of the Admin Group.
10 | - How to check?
11 | - Log in to the gateway portal
12 | - You would land on Dashboard with all the applications you have access to launch jobs.
13 | - Click 'Settings' from top right drop down menu.
14 | - Do you see icon 'Application Catalog' ? In it can you create applications using 'New Application +'?.
15 | - Then you are good !
16 | - If not, contact your gateway admin OR SciGaP team
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 | ### Select Your Quick Start Tutorial
26 | 1. Using Credential Store
27 | 2. Register your Compute & Storage Resources
28 | 3. Configure Group Resource Profile (GRP)
29 | 4. Register Gaussian Application
30 | 8. Gateway Resource Profile
31 | 9. Managing User Accounts & User Groups
32 | 10. Communication with Gateway Users
33 | 11. Monitor Gateway Traffic
34 |
35 |
36 | #####
Using Credential Store
37 | 1. Navigate to Settings → Credential Store.
38 | 2. Click 'New SSH Credential +' and provide a key description.
39 | 3. The new key will be generated and available to copy.
40 | 4. Apart from using the key for SSH communications with compute resources and gateway storage resource, you can share the key with other gateway users and user groups.
41 | 5. When sharing a key, there are 3 permission levels, READ, WRITE and MANAGE_SHARING.
42 | 6. READ permission enables the users to use the key, WRITE permission enables using the key plus deleting the key. MANAGE_SHARING is use the key, delete it and also share with other users and user groups behalf of the owner of the key.
43 | 7. NOTE: Make sure not to delete once assigned to a compute resource or the storage resource.
44 | 
45 |
46 |
47 | #####
Register your Compute & Storage Resources
48 |
49 | NOTE: Only SciGaP admins or gateway service provider can add compute resources and storage resources. If the gateway and middleware is hsoted by you, You can add the Compute Resources and Storage Resource.
50 | What is a compute resource? A compute resource is an object that represents a host, host cluster, or pool in a virtualization platform, a virtual data center, or an Amazon region on which machines can be provisioned.
51 | Compute resource could come as national resources, campus resources or even private cluster.
52 | NOTE:For testing purposes users (mainly gateway developers) this could even be the local machine.
53 | What is a storage resource? A storage resource is the server that stores all user data files: application input files and output files.
54 | Most of the time this is the same server that the gateway is deployed in. The storage resource can be either provided by the gateway provider or by you.
55 | If the gateway service provider is providing the resource, then there will be file retention policies in place.
56 |
57 | ###### Compute Resource Information
58 | Provide the information below to get your compute resource/HPC registered with Super admin portal, SciGaP
59 | 1. HPC Name (This is the name used to SSH to the resource)
60 | 2. Queue details (Queue name, maximum limits of nodes, CPUs and Walltime.)
61 | 3. Resource manager type (SLURM, PBS)
62 | 4. Job command binary path
63 |
64 | ###### Register a XSEDE Resource
65 | NOTE: Please note this set of instructions are ONLY for gateways hosted by the user. If not SciGaP admins will do this configuration for you in SciGaP Portal.
66 | 1. Navigate to Admin Dashboard → Compute Resource → Register
67 | 2. Provide
68 | - Host Name: comet.sdsc.edu
69 | - Host Aliases:
70 | - IP Addresses:
71 | - Resource Description: SDSC Comet Cluster ( Just a description)
72 | - Create
73 | 3. In Queues tab
74 | - Click 'Add a Queue'
75 | - Queue 1
76 | - Queue Name: compute ( Per resource this is a unique value. Once created cannot edit the name. If need to change, delete and create again)
77 | - Queue Description: Used for access to regular compute nodes
78 | - Queue Max Run Time (In Minutes): 280 (Note required for the local resource)
79 | - Queue Max Nodes: 72 (Note required for the local resource)
80 | - Queue Max Processors: 1728 (Note required for the local resource)
81 | - Max Jobs in Queue: 0 (Note required for the local resource)
82 | - Max Memory For Queue( In MB ): (Note required for the local resource)
83 | - Queue 2
84 | - Queue Name: shared ( Per resource this is a unique value. Once created cannot edit the name. If need to change, delete and create again)
85 | - Queue Description: Single-node jobs using fewer than 24 cores
86 | - Queue Max Run Time (In Minutes): 2880 (Note required for the local resource)
87 | - Queue Max Nodes: 1 (Note required for the local resource)
88 | - Queue Max Processors: 24 (Note required for the local resource)
89 | - Max Jobs in Queue: 0 (Note required for the local resource)
90 | - Max Memory For Queue( In MB ): (Note required for the local resource)
91 | 4. File System
92 | - Nothing to add here, this is futuristic development.
93 | 5. Job Submission Interface
94 | - Click on the tab
95 | - Add a new Job Submission Interface
96 | - Job Submission Protocol: SSH
97 | - Select Security Protocol: SSH_KEYS
98 | - SSH Port: 22
99 | - Select resource manager type: SLURM
100 | - Job Manager Bin Path: /usr/bin/
101 | - Job Commands (SUBMISSION): sbatch
102 | - Job Commands (JOB_MONITORING): squeue
103 | - Job Commands (DELETION): scancel
104 | - Parallelism Prefixes (MPI): mpiexec
105 | - Add Job Submission Protocol
106 | 6. Data Movement Interface
107 | - Add a new Data Submission Interface
108 | - Data Movement Protocol: SCP
109 | - Select Security Protocol: SSH_KEYS
110 | - SSH Port: 22
111 | - Add Data Movement Protocol
112 | Now the Local resource is ready for job submissions.
113 | Comet is ready for job submissions.
114 |
115 | ###### Register a Campus Resource
116 | NOTE: Adding a campus resource is similar to adding a XSEDE resource. Same steps to follow with similar information.
117 |
118 | ######
Register Storage Resource
119 | NOTE: Please note this set of instructions are ONLY for gateways hosted by the user. If not SciGaP admins will do this configuration for you in SciGaP Portal.
120 | 1. Navigate to Admin Dashboard → Storage Resource → Register
121 | 2. Provide
122 | - Host Name: sg03.iu.xsede.org
123 | - Storage Resource Description: Storage for airavata gateway
124 | - Create
125 | 3. In Data Movement Interfaces tab
126 | - Click 'Add a new Data Movement Interface'
127 | - Select 'SCP' from the list
128 | - Select Security Protocol: SSH_KEYS
129 | - Alternate SSH Host Name: Leave empty
130 | - SSH Port: 22
131 | - Add Data Movement Interface
132 |
133 | #####
Add a Group Resource Profile for Comet (GRP)
134 | 1. What is a group resource profile?
135 | - Group resource profile is a configuration which states how a particular HPC resource (campus, national, cloud) is available for users to use.
136 | - One or many HPCs can be added to a GRP and then it can be shared with the users and user groups to enable use.
137 | - A gateway can have multiple GRPs and a gateway user can be in one or many GRPs
138 | - At the time of experiment creation, user need to select which GRP to move forward to launch experiment jobs.
139 | 2. To Create a GRP
140 | - Navigate to Settings → Group Resource Profile.
141 | - Click New Group Resource Profile +.
142 | - Add Name: Default Gateway Profile
143 | - Default SSH Credential: Select a key from the list or generate a new one.
144 | - Click New Compute Resource +.
145 | - Select the resource from the list provided.
146 | - In next page provide the details required for SSH communication
147 | - Login Username: This is the login name to use in SSH login
148 | - SSH Credential: Select from the list, or generate a new one
149 | - Allocation Project Number: this is applicable for resources which maintains an account for user allocation and usage.
150 | - Scratch Location: The location for users' computational working directories
151 | - Provide computational queue policies
152 | - This section will provide details of maximum node, walltime and CPUs per HPC resource that a gateway user could use in experiment creation.
153 | - If you don't specify these limits, then the maximum allowed from the HPC resource will be available for users.
154 | - NOTE: This feature is more popular with gateways which are used in educational purposes and also for gateways where the gateway admin provides XSEDE resoruces for free via community allocations.
155 | - Click Save
156 | - Next page, click Save
157 |
158 |
159 | #####
Register Gaussian Application
160 | End of this tutorial you will know how to register a new application to run on a a HPC resource through Django portal.
161 |
162 | 1. Navigate to Settings → Application Catalog. Click New Application +
163 | 2. In Details tab enter:
164 | - Enter Application Name: Gaussian16
165 | - Enter Application Version: Gaussian 16 (Not mandatory)
166 | - Enter Application Description: Gaussian provides capabilities for electronic structure modeling
167 | - Save
168 | 
169 | 3. In Interface tab:
170 | - Set 'Enable Archiving Working Directory' to True (Why? - This is set to true when you want to bring back all the files in working directory back to PGA. Caution: If there are very large files they may not be able to SCP)
171 | - Set 'Enable Optional File Inputs' to False (Why? - Set to false because there won't be any additional optional inputs for Gaussian)
172 | - Provide Input Fields
173 | - Click Add Application Input
174 | - Name: Input-File
175 | - Initial Value: gaussian.com
176 | - Type: URI (Why? - This is the type for file uploads)
177 | - Application Arguments:
178 | - Standard Input: False (Why? - Futuristic property and not in real use at the moment)
179 | - Read Only: False (Why? - this is only meaningful for String, Integer or Float inputs)
180 | - User Friendly Description: Gaussian input file specifying desired calculation type, model chemistry, molecular system and other parameters. (This is information to the user at creating job experiment. Not mandatory)
181 | - Data is Staged: False
182 | - Is the Input Required: True
183 | - Required in Commandline: True
184 | - Advanced Input Field Modification Metadata:
185 | - Provide application outputs
186 | NOTE: 3 application outputs to define.
187 | - 1st Output
188 | - Click Add Application Output
189 | - Name: Gaussian-Application-Output
190 | - Value: *.log
191 | - Type: URI
192 | - Application Argument:
193 | - Is the Output required?: True
194 | - Required on command line?: True
195 | - Metadata:
196 | - 2nd output
197 | - Click Add Application Output
198 | - Name: Gaussian_Checkpoint_File
199 | - Value: *.chk
200 | - Type: URI
201 | - Application Argument:
202 | - Data Movement: False
203 | - Is the Output required?: True
204 | - Required on command line?: True
205 | - Metadata
206 | - For applications, STDOUT and STDERR will be automatically added for you.
207 | - Search Query:
208 | 
209 | 
210 | 
211 | 
212 |
213 | 3. In Deployment tab
214 | - Click 'New Deployment +'
215 | - Click Share and add groups and users you want to share this deployment and let run on the particular cluster.
216 | - Application Executable Path: /home/gridchem/bin/rung09_with_chk_recovery.sh
217 | - Application Parallelism Type: SERIAL
218 | - Application Deployment Description: Gaussian16 Rev A.03 ( to avoid G4 output printing issues in Rev B.01)
219 | - Application Compute Host: Local (Local machine has to be added as a compute resource prior to this step)
220 | - Add Module load commands:
221 | -
235 | 1. NOTE: This is the section to configure the storage resource for the gateway. Storage resource is the place to hold all gateway user data: inputs and outputs from computations.
236 | 2. Navigate to Settings → Gateway Resource Profile.
237 | 3. Select a SSH key from the list or generate new one and assign it.
238 | 4. Click: New Storage Preference +.
239 | 5. Select Storage Resource: This is the storage resource for the user files and generally its the same server that holds the Django portal. If the gateway is deployed for you, this part would be taken cared by the SciGaP team. If the gateway is self-deployed select the host name of the server you deployed the gateway.
240 | 6. Login Username: pga (This is the username which uses to ssh to the storage resource)
241 | 7. File System Root Location: /var/www/portals/gateway-user-data/seagrid (The path which stores all user files, input files and output files)
242 | 8. Resource Specific Credential Store Token: Select a token from the list. (Public key of this token need to be added to authorized_keys in your storage resource 'pga' login)
243 | 9. Save.
244 |
245 |
246 | #####
Managing User Accounts & User Groups
247 | 1. Read: [User Groups](airavata-user-profiles.md)
248 | 2. To manage gateway users access use, Groups from top right-hand dropdown.
249 | 3. Click 'Edit' on the group you want to add users.
250 | 4. Mostly the users are added to Gateway-user group by the gateway admins.
251 | 4. Search for the user account using search text field.
252 | 5. The name entered, will be searched across username, first name, last name and email.
253 | 6. User with the matching search string will be listed in the dropdown.
254 | 8. Select the user account, it will get added in to the list highlighted in green.
255 | 9. By default users are added with Member role.
256 | 10. Using the Groups interface, searching for already added users, deleting existing users and changing their roles are supported.
257 |
258 |
259 | #####
Communication with Gateway Users
260 | 1. Provides a method to notify the gateway user.
261 | 2. When user logs in, a 'Bell' icon will be displayed with the number of unread messages next to Dashboard on top menu bar.
262 | 3. Gateway admin can create these notices with an assigned priority.
263 | 4. navigate to Settings → Manage Notices
264 | 5. Click 'New Notice +'
265 | 6. Provide:
266 | - Notice Title: Gordon cluster is retiring (A heading for the notice)
267 | - Notice Message: Gordon cluster is retiring from Dec 31st 2016. Preserve your files now.
268 | - Publish Date: Select from the provided calendar
269 | - Expiration Date: Mandatory.
270 | - Priority: LOW (Default is 'LOW'. Admin can change)
271 |
272 |
273 |
274 | #####
Monitor Gateway Traffic
275 | 1. To monitor current gateway experiment and job statuses navigate to Settings → Experiment Statistics.
276 | 2. Gateway admin can view and filter experiments of the gateway using this page.
277 | 3. In experiment statistics page, by default gateway status for last 24 hours will be displayed.
278 | 4. Two buttons, 'Past 24 Hours' and 'Past Week' will group and list the experiments for the selected time.
279 | 5. In order to further filter the experiments use, Filters dropdown.
280 | - Username: AnneMarie (Gateway username need to be typed in)
281 | - Application Name: Gaussian (All the application in teh gateway will be listed)
282 | - Hostname: Comet.sdsc.edu (A drop down will provide all the compute resources to select from)
283 | - Click 'Get Statistics'
284 | 6. Using the calendar, you can search experiments for a time duration you need.
285 | 7. Once the experiments are filtered and grouped, click on the desired group (Existing experiments groups are 'Total','Created','Running', Completed','Cancelled' and 'Failed').
286 | 8. When you list an experiment group you can select the experiment you want to view using 'View Details'. This will open the experiment Details Summary in a new tab.
287 |
--------------------------------------------------------------------------------
/docs/technical-documentation/sharing-documentation/sharing-models.md:
--------------------------------------------------------------------------------
1 |
2 |
This is an system internal enum used to define single user groups and multi users groups. Every user is also
32 | considered as a group in it's own right for implementation ease
33 |
34 |
35 |
SINGLE_USER
0
36 |
37 |
MULTI_USER
1
38 |
39 |
40 |
Enumeration: GroupType
41 |
Group types can be either user level or domain level groups.
42 |
43 |
44 |
DOMAIN_LEVEL_GROUP
0
45 |
46 |
USER_LEVEL_GROUP
1
47 |
48 |
49 |
Enumeration: GroupChildType
50 |
System internal data type to match group child types
51 |
52 |
53 |
USER
0
54 |
55 |
GROUP
1
56 |
57 |
58 |
Enumeration: EntitySearchField
59 |
This list of fields that can be used to search entities
60 |
NAME : Name of the entity
61 |
DESCRIPTION : Description of the entity
62 |
FULL_TEXT : Full text field of the entity
63 |
PARENT_ENTITY_ID : Parent entity id of the entity
64 |
OWNER_ID : Owner of the entity
65 |
CREATED_TIME : Created time of the entity
66 |
UPDATED_TIME : Updated time of the entity
67 |
ENTITY_TYPE_ID : Entity type ID
68 |
69 |
70 |
NAME
0
71 |
72 |
DESCRIPTION
1
73 |
74 |
FULL_TEXT
2
75 |
76 |
PARRENT_ENTITY_ID
3
77 |
78 |
OWNER_ID
4
79 |
80 |
PERMISSION_TYPE_ID
5
81 |
82 |
CREATED_TIME
6
83 |
84 |
UPDATED_TIME
7
85 |
86 |
87 |
ENTITY_TYPE_ID
88 |
8
89 |
90 |
91 |
92 |
93 |
Enumeration: SearchCondition
94 |
Different search operators that can be used with the entity search fields
95 |
EQUAL : Simply matches for equality. Applicable for name, and parent entity id
96 |
LIKE : Check for the condition %$FIELD% condition. Applicable for name, and description
97 |
FULL_TEXT : Does a full text search. Only applicable for the FULL_TEXT field.
98 |
GTE : Greater than or equal. Only applicable for created time and updated time.
99 |
LTE : Less than or equal. Only applicable for created time and updated time.
100 |
NOT : Not operator. Only applicable for name, permission type id, parent entity id and owner id.
101 |
102 |
103 |
EQUAL
0
104 |
105 |
LIKE
1
106 |
107 |
FULL_TEXT
2
108 |
109 |
GTE
3
110 |
111 |
LTE
4
112 |
113 |
114 |
NOT
115 |
5
116 |
117 |
118 |
119 |
120 |
Enumeration: SharingType
121 |
This is an internal enum type for managing sharings
122 |
123 |
124 |
DIRECT_NON_CASCADING
0
125 |
126 |
DIRECT_CASCADING
1
127 |
128 |
INDIRECT_CASCADING
2
129 |
130 |
131 |
Data structures
132 |
Struct: Domain
133 |
Key
Field
Type
Description
Requiredness
Default value
134 |
1
domainId
string
optional
"DO_NOT_SET_AT_CLIENTS_ID"
135 |
2
name
string
optional
136 |
3
description
string
optional
137 |
4
createdTime
i64
optional
138 |
5
updatedTime
i64
optional
139 |
Domain is the entity that enables multi-tenency in this componenet. Every tenant will be
140 | operating separately it's own silo which is identified by the domain id. In the current implementation domain id
141 | will be same as the domain name
142 |
domainId : Will be generated by the server based on the domain name
143 |
name : A single word name that identifies the domain e.g seagrid, ultrascan
144 |
description : A short description for the domain
145 |
createdTime : Will be set by the system
146 |
updatedTime : Will be set by the system
147 |
148 |
Struct: User
149 |
Key
Field
Type
Description
Requiredness
Default value
150 |
1
userId
string
optional
151 |
2
domainId
string
optional
152 |
3
userName
string
optional
153 |
4
firstName
string
optional
154 |
5
lastName
string
optional
155 |
6
email
string
optional
156 |
7
icon
binary
optional
157 |
8
createdTime
i64
optional
158 |
9
updatedTime
i64
optional
159 |
User is the model used to register a user in the system. Minimal user information will be required to provide
160 | regarding the user.
161 |
userId : User id provided by the client
162 |
domainId : Domain id for that user
163 |
userName : User name for the user
164 |
firstName : First name of the user
165 |
lastName : Last name of the user
166 |
email : Email address of the user
167 |
icon : A binary field for storing the user icon
168 |
createdTime : If client provides this value then the system will use it if not the current time will be set
169 |
updatedTime : If client provides this value then the system will use it if not the current time will be set