├── data ├── .gitkeep ├── bin │ └── .gitkeep ├── jobs │ └── .gitkeep ├── logs │ └── .gitkeep ├── media │ └── .gitkeep └── sample │ └── .gitkeep ├── tests ├── jobs │ └── .gitkeep ├── data │ ├── jobs │ │ └── .gitignore │ ├── sample │ │ ├── physic_ist │ │ │ ├── outgroup_ex.txt │ │ │ ├── runs.json │ │ │ └── physic_ist_ex.txt │ │ ├── test_copy.txt │ │ ├── services │ │ │ └── hello_world.sh │ │ ├── maf │ │ │ ├── out.fasta │ │ │ ├── in.maf │ │ │ └── simple.maf │ │ ├── sample_tree.nhx │ │ ├── fast_me │ │ │ └── fastme_matrix.txt │ │ └── mafft │ │ │ └── aln.fasta │ └── test.fasta └── settings.ini.sample ├── waves ├── wcore │ ├── api │ │ ├── v1 │ │ │ ├── __init__.py │ │ │ ├── views │ │ │ │ └── __init__.py │ │ │ ├── serializers │ │ │ │ ├── __init__.py │ │ │ │ └── fields.py │ │ │ └── urls.py │ │ ├── v2 │ │ │ ├── __init__.py │ │ │ ├── views │ │ │ │ └── __init__.py │ │ │ ├── serializers │ │ │ │ ├── __init__.py │ │ │ │ └── fields.py │ │ │ └── urls.py │ │ ├── views │ │ │ ├── __init__.py │ │ │ ├── base.py │ │ │ └── service.py │ │ ├── __init__.py │ │ ├── permissions.py │ │ ├── urls.py │ │ └── share.py │ ├── exceptions │ │ ├── runners.py │ │ ├── __init__.py │ │ └── jobs.py │ ├── forms │ │ ├── __init__.py │ │ └── helper.py │ ├── import_export │ │ ├── jobs.py │ │ ├── tests.py │ │ ├── __init__.py │ │ └── runners.py │ ├── tests │ │ ├── __init__.py │ │ ├── fixtures │ │ │ ├── users.json │ │ │ └── test.fasta │ │ ├── test_runners.py │ │ └── test_copy_service.py │ ├── views │ │ └── __init__.py │ ├── admin │ │ ├── forms │ │ │ ├── __init__.py │ │ │ ├── jobs.py │ │ │ └── runners.py │ │ ├── views │ │ │ ├── __init__.py │ │ │ ├── export.py │ │ │ ├── json_view.py │ │ │ └── job_tool.py │ │ ├── __init__.py │ │ └── binaries.py │ ├── management │ │ ├── __init__.py │ │ ├── commands │ │ │ ├── __init__.py │ │ │ ├── wqueue.py │ │ │ ├── wpurge.py │ │ │ └── waves.py │ │ ├── command.py │ │ └── utils.py │ ├── migrations │ │ ├── __init__.py │ │ └── 0002_auto_20190624_1122.py │ ├── templatetags │ │ ├── __init__.py │ │ └── waves_tags.py │ ├── templates │ │ └── waves │ │ │ ├── services │ │ │ ├── forms │ │ │ │ ├── service_inc.html │ │ │ │ ├── base_form.html │ │ │ │ ├── uni_form │ │ │ │ │ ├── inc.css.html │ │ │ │ │ ├── inc.js.html │ │ │ │ │ └── submission_form.html │ │ │ │ ├── test_base.html │ │ │ │ ├── base_inc_js.html │ │ │ │ ├── base_inc_css.html │ │ │ │ ├── materialize │ │ │ │ │ ├── inc.css.html │ │ │ │ │ ├── inc.js.html │ │ │ │ │ └── submission_form.html │ │ │ │ ├── bootstrap │ │ │ │ │ ├── inc.js.html │ │ │ │ │ ├── inc.css.html │ │ │ │ │ └── submission_form.html │ │ │ │ ├── bootstrap3 │ │ │ │ │ ├── inc.js.html │ │ │ │ │ ├── inc.css.html │ │ │ │ │ └── submission_form.html │ │ │ │ └── bootstrap4 │ │ │ │ │ ├── inc.css.html │ │ │ │ │ ├── inc.js.html │ │ │ │ │ └── submission_form.html │ │ │ ├── _online_execution.html │ │ │ ├── _service_head.html │ │ │ ├── base.html │ │ │ ├── file.html │ │ │ └── services_list.html │ │ │ ├── api │ │ │ └── service_api_form.html │ │ │ ├── emails │ │ │ ├── job_admin_error.tpl │ │ │ ├── job_cancelled.tpl │ │ │ ├── job_warning.tpl │ │ │ ├── job_error.tpl │ │ │ ├── job_submitted.tpl │ │ │ ├── job_completed.tpl │ │ │ └── job_prepared.tpl │ │ │ ├── admin │ │ │ ├── service │ │ │ │ ├── service_modal.html │ │ │ │ └── service_preview.html │ │ │ ├── job │ │ │ │ └── change_form.html │ │ │ ├── baseparam │ │ │ │ ├── popup_response.html │ │ │ │ └── jet_popup_response.html │ │ │ ├── change_form.html │ │ │ ├── modal_alert.html │ │ │ ├── modal_content.html │ │ │ ├── submit_line.html │ │ │ ├── submission │ │ │ │ └── change_form.html │ │ │ ├── runner │ │ │ │ └── change_form.html │ │ │ └── import │ │ │ │ └── service_modal_form.html │ │ │ └── jobs │ │ │ └── parts │ │ │ └── job_list_element.html │ ├── static │ │ └── waves │ │ │ ├── img │ │ │ ├── logo.png │ │ │ ├── ajax-loader.gif │ │ │ ├── ico │ │ │ │ ├── favicon.ico │ │ │ │ └── favicon.png │ │ │ └── progress-bar.gif │ │ │ ├── css │ │ │ └── forms.css │ │ │ └── admin │ │ │ └── js │ │ │ ├── connect.js │ │ │ ├── submissions.js │ │ │ ├── services.js │ │ │ ├── runner.js │ │ │ └── admin.js │ ├── __init__.py │ ├── commands │ │ ├── __init__.py │ │ └── command.py │ ├── models │ │ ├── binaries.py │ │ ├── __init__.py │ │ └── const.py │ ├── utils │ │ ├── decorators.py │ │ ├── __init__.py │ │ ├── encrypt.py │ │ ├── logged.py │ │ └── storage.py │ ├── adaptors │ │ ├── utils.py │ │ ├── loader.py │ │ ├── cluster.py │ │ ├── exceptions.py │ │ ├── mocks.py │ │ └── api.py │ ├── cron │ │ ├── __init__.py │ │ └── purge_jobs.py │ ├── urls.py │ └── compat │ │ └── __init__.py ├── authentication │ ├── migrations │ │ ├── __init__.py │ │ ├── 0002_auto_20180313_1344.py │ │ └── 0001_initial.py │ ├── __init__.py │ ├── admin.py │ ├── apps.py │ ├── signals.py │ ├── views.py │ ├── serializers.py │ ├── models.py │ └── auth.py └── __init__.py ├── docs ├── authors.rst ├── changelog.rst ├── license.rst ├── readme.rst ├── contributing.rst ├── _static │ ├── img │ │ ├── um.png │ │ ├── cnrs.png │ │ ├── ifb.png │ │ └── lirmm.jpg │ ├── css │ │ ├── blank.gif │ │ ├── waves.css │ │ ├── fancybox_loading.gif │ │ ├── fancybox_overlay.png │ │ ├── fancybox_sprite.png │ │ └── fancybox_loading@2x.gif │ └── js │ │ └── waves.js ├── user_doc │ ├── waves-admin.png │ ├── job │ │ ├── backoffice │ │ │ ├── job-inputs.png │ │ │ ├── job-list.png │ │ │ ├── job-general.png │ │ │ ├── job-history.png │ │ │ ├── job-outputs.png │ │ │ └── job-submission.png │ │ └── jobs.rst │ ├── runner │ │ ├── backoffice │ │ │ ├── runner-list.png │ │ │ ├── runner-param.png │ │ │ └── runner-detail.png │ │ └── runners.rst │ ├── service │ │ └── backoffice │ │ │ ├── service-list.png │ │ │ ├── service-access.png │ │ │ ├── service-detail.png │ │ │ ├── service-general.png │ │ │ ├── service-runner.png │ │ │ ├── submission-list.png │ │ │ ├── submission-run.png │ │ │ ├── service-submission.png │ │ │ ├── submission-general.png │ │ │ ├── submission-output.png │ │ │ ├── submission-params.png │ │ │ ├── service-submission-2.png │ │ │ ├── submission-exitcode.png │ │ │ ├── submission-params-1.png │ │ │ ├── submission-params-2.png │ │ │ ├── submission-params-3.png │ │ │ ├── submission-params-4.png │ │ │ ├── submission-params-5.png │ │ │ ├── submission-params-6.png │ │ │ ├── submission-params-7.png │ │ │ ├── service-submission-graph.png │ │ │ └── service-submission-graph2.png │ └── user_guide.rst ├── dev_doc │ ├── images │ │ ├── adaptors.png │ │ └── job_workflow.png │ ├── api_user_doc.rst │ └── dev_doc.rst ├── modules │ ├── source.rst │ ├── managers │ │ └── managers.rst │ ├── models │ │ ├── runners.rst │ │ ├── base.rst │ │ ├── jobs.rst │ │ └── services.rst │ ├── settings.rst │ └── adaptors │ │ └── adaptors.rst ├── templates │ ├── layout.html │ └── footer.html ├── index.rst ├── waves_uwsgi.ini └── extensions.rst ├── static └── waves │ ├── img │ └── banner.png │ └── css │ └── site.css ├── waves_core ├── __init__.py ├── celery.py ├── wsgi.py ├── rtd.py ├── urls.py ├── cli.py └── crontab.py ├── .travis.yml ├── CONTRIBUTING.md ├── MANIFEST.in ├── AUTHORS.md ├── templates ├── _navbar.html ├── waves │ └── override │ │ ├── service_simple_cp_form.html │ │ └── service_simple_cp_details.html └── admin │ ├── base_site.html │ └── login.html ├── requirements.txt ├── manage.py ├── LICENSE.md ├── fixtures_celery_beat.json ├── .gitignore ├── .gitlab-ci.yml ├── setup.py └── README.md /data/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data/bin/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data/jobs/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data/logs/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data/media/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data/sample/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /tests/jobs/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /tests/data/jobs/.gitignore: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /waves/wcore/api/v1/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /waves/wcore/api/v2/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /waves/wcore/api/views/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /waves/wcore/exceptions/runners.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /waves/wcore/forms/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /waves/wcore/import_export/jobs.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /waves/wcore/tests/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /waves/wcore/views/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /waves/wcore/admin/forms/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /waves/wcore/management/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /waves/wcore/migrations/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /waves/wcore/templatetags/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /waves/authentication/migrations/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /waves/wcore/management/commands/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /docs/authors.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../AUTHORS.md 2 | -------------------------------------------------------------------------------- /docs/changelog.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../CHANGES.md -------------------------------------------------------------------------------- /docs/license.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../LICENSE.md 2 | -------------------------------------------------------------------------------- /docs/readme.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../README.md 2 | -------------------------------------------------------------------------------- /docs/contributing.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../CONTRIBUTING.md -------------------------------------------------------------------------------- /tests/data/sample/physic_ist/outgroup_ex.txt: -------------------------------------------------------------------------------- 1 | (Lemur) -------------------------------------------------------------------------------- /tests/data/sample/test_copy.txt: -------------------------------------------------------------------------------- 1 | Sample File to test copy files service -------------------------------------------------------------------------------- /waves/__init__.py: -------------------------------------------------------------------------------- 1 | __import__('pkg_resources').declare_namespace(__name__) 2 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/forms/service_inc.html: -------------------------------------------------------------------------------- 1 | {% extends template %} -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/forms/base_form.html: -------------------------------------------------------------------------------- 1 | {% extends template_form %} -------------------------------------------------------------------------------- /docs/_static/img/um.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/_static/img/um.png -------------------------------------------------------------------------------- /waves/authentication/__init__.py: -------------------------------------------------------------------------------- 1 | default_app_config = 'waves.authentication.apps.ApiKeyConfig' 2 | -------------------------------------------------------------------------------- /docs/_static/img/cnrs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/_static/img/cnrs.png -------------------------------------------------------------------------------- /docs/_static/img/ifb.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/_static/img/ifb.png -------------------------------------------------------------------------------- /docs/_static/css/blank.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/_static/css/blank.gif -------------------------------------------------------------------------------- /docs/_static/img/lirmm.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/_static/img/lirmm.jpg -------------------------------------------------------------------------------- /static/waves/img/banner.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/static/waves/img/banner.png -------------------------------------------------------------------------------- /waves/wcore/api/v2/views/__init__.py: -------------------------------------------------------------------------------- 1 | """ WAVES API views """ 2 | import jobs 3 | import services 4 | 5 | -------------------------------------------------------------------------------- /docs/user_doc/waves-admin.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/waves-admin.png -------------------------------------------------------------------------------- /docs/_static/css/waves.css: -------------------------------------------------------------------------------- 1 | div[role=contentinfo] > p { 2 | text-align: right; 3 | font-size: x-small; 4 | } -------------------------------------------------------------------------------- /docs/dev_doc/images/adaptors.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/dev_doc/images/adaptors.png -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/forms/uni_form/inc.css.html: -------------------------------------------------------------------------------- 1 | {% extends 'waves/services/forms/base_inc_css.html' %} 2 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/forms/uni_form/inc.js.html: -------------------------------------------------------------------------------- 1 | {% extends 'waves/services/forms/base_inc_js.html' %} 2 | -------------------------------------------------------------------------------- /docs/_static/css/fancybox_loading.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/_static/css/fancybox_loading.gif -------------------------------------------------------------------------------- /docs/_static/css/fancybox_overlay.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/_static/css/fancybox_overlay.png -------------------------------------------------------------------------------- /docs/_static/css/fancybox_sprite.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/_static/css/fancybox_sprite.png -------------------------------------------------------------------------------- /docs/dev_doc/images/job_workflow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/dev_doc/images/job_workflow.png -------------------------------------------------------------------------------- /waves/wcore/static/waves/img/logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/waves/wcore/static/waves/img/logo.png -------------------------------------------------------------------------------- /docs/_static/css/fancybox_loading@2x.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/_static/css/fancybox_loading@2x.gif -------------------------------------------------------------------------------- /docs/user_doc/job/backoffice/job-inputs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/job/backoffice/job-inputs.png -------------------------------------------------------------------------------- /docs/user_doc/job/backoffice/job-list.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/job/backoffice/job-list.png -------------------------------------------------------------------------------- /docs/user_doc/job/backoffice/job-general.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/job/backoffice/job-general.png -------------------------------------------------------------------------------- /docs/user_doc/job/backoffice/job-history.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/job/backoffice/job-history.png -------------------------------------------------------------------------------- /docs/user_doc/job/backoffice/job-outputs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/job/backoffice/job-outputs.png -------------------------------------------------------------------------------- /waves/wcore/static/waves/img/ajax-loader.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/waves/wcore/static/waves/img/ajax-loader.gif -------------------------------------------------------------------------------- /waves/wcore/static/waves/img/ico/favicon.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/waves/wcore/static/waves/img/ico/favicon.ico -------------------------------------------------------------------------------- /waves/wcore/static/waves/img/ico/favicon.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/waves/wcore/static/waves/img/ico/favicon.png -------------------------------------------------------------------------------- /waves/wcore/static/waves/img/progress-bar.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/waves/wcore/static/waves/img/progress-bar.gif -------------------------------------------------------------------------------- /docs/user_doc/job/backoffice/job-submission.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/job/backoffice/job-submission.png -------------------------------------------------------------------------------- /docs/user_doc/runner/backoffice/runner-list.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/runner/backoffice/runner-list.png -------------------------------------------------------------------------------- /docs/user_doc/runner/backoffice/runner-param.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/runner/backoffice/runner-param.png -------------------------------------------------------------------------------- /docs/user_doc/runner/backoffice/runner-detail.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/runner/backoffice/runner-detail.png -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/service-list.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/service-list.png -------------------------------------------------------------------------------- /waves/wcore/api/v1/views/__init__.py: -------------------------------------------------------------------------------- 1 | """ WAVES API views """ 2 | from __future__ import unicode_literals 3 | 4 | import jobs 5 | import services 6 | -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/service-access.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/service-access.png -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/service-detail.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/service-detail.png -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/service-general.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/service-general.png -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/service-runner.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/service-runner.png -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/submission-list.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/submission-list.png -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/submission-run.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/submission-run.png -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/service-submission.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/service-submission.png -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/submission-general.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/submission-general.png -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/submission-output.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/submission-output.png -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/submission-params.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/submission-params.png -------------------------------------------------------------------------------- /waves_core/__init__.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import, unicode_literals 2 | from waves_core.celery import app as celery_app 3 | __all__ = ('celery_app', ) -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/service-submission-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/service-submission-2.png -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/submission-exitcode.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/submission-exitcode.png -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/submission-params-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/submission-params-1.png -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/submission-params-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/submission-params-2.png -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/submission-params-3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/submission-params-3.png -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/submission-params-4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/submission-params-4.png -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/submission-params-5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/submission-params-5.png -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/submission-params-6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/submission-params-6.png -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/submission-params-7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/submission-params-7.png -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/service-submission-graph.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/service-submission-graph.png -------------------------------------------------------------------------------- /docs/user_doc/service/backoffice/service-submission-graph2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lirmm/waves-core/HEAD/docs/user_doc/service/backoffice/service-submission-graph2.png -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/forms/test_base.html: -------------------------------------------------------------------------------- 1 | {% load static %} 2 | {% block js %} 3 | 4 | {% endblock %} 5 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/forms/base_inc_js.html: -------------------------------------------------------------------------------- 1 | {% load static %} 2 | {% block js %} 3 | 4 | {% endblock %} 5 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/api/service_api_form.html: -------------------------------------------------------------------------------- 1 | {% load staticfiles crispy_forms_tags waves_tags %} 2 | 3 | {% submission_form %} 4 | 5 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/forms/base_inc_css.html: -------------------------------------------------------------------------------- 1 | {% load static %} 2 | {% block css %} 3 | 4 | {% endblock %} 5 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: python 2 | python: 3 | - "2.7" 4 | env: 5 | -DJANGO=1.11 6 | install: 7 | - pip install -r requirements.txt 8 | - pip install coverage==4.5.1 9 | script: 10 | - coverage run --source='.' manage.py test waves -------------------------------------------------------------------------------- /docs/_static/js/waves.js: -------------------------------------------------------------------------------- 1 | /** 2 | * Created by marc on 20/09/16. 3 | */ 4 | $(document).ready(function() { 5 | $("#sample").click(function(){ 6 | $.modal(""); 7 | }); 8 | }); -------------------------------------------------------------------------------- /docs/dev_doc/api_user_doc.rst: -------------------------------------------------------------------------------- 1 | .. _api-dev-guide: 2 | 3 | ================== 4 | API Dev user Guide 5 | ================== 6 | 7 | 8 | This documentation intends to help front end developers to integrate WAVES-core form api service. 9 | 10 | ... -------------------------------------------------------------------------------- /docs/modules/source.rst: -------------------------------------------------------------------------------- 1 | Source Documentation 2 | ==================== 3 | 4 | .. toctree:: 5 | :maxdepth: 2 6 | 7 | models/services 8 | models/jobs 9 | models/runners 10 | models/base 11 | managers/managers 12 | adaptors/adaptors 13 | 14 | 15 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/_online_execution.html: -------------------------------------------------------------------------------- 1 | {% if available_for_submission %} 2 | 3 | {% if label %}{{ label }}{% else %}Online execution{% endif %} 4 | {% endif %} -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/forms/materialize/inc.css.html: -------------------------------------------------------------------------------- 1 | {% extends 'waves/services/forms/base_inc_css.html' %} 2 | {% block css %} 3 | 4 | {{ block.super }} 5 | {% endblock %} 6 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | Contributing 2 | ============ 3 | 4 | You can contribute to WAVES project with following repositories: 5 | 6 | - Git source code: https://github.com/lirmm/waves-core 7 | - Issue tracker: https://github.com/lirmm/waves-core/issues 8 | - Mailing list: waves-webapp@googlegroups.com 9 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/forms/bootstrap/inc.js.html: -------------------------------------------------------------------------------- 1 | {% extends 'waves/services/forms/base_inc_js.html' %} 2 | {% block js %} 3 | 4 | 5 | {{ block.super }} 6 | {% endblock %} -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/forms/bootstrap3/inc.js.html: -------------------------------------------------------------------------------- 1 | {% extends 'waves/services/forms/base_inc_js.html' %} 2 | {% block js %} 3 | 4 | {{ block.super }} 5 | {% endblock %} 6 | 7 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/forms/materialize/inc.js.html: -------------------------------------------------------------------------------- 1 | {% extends 'waves/services/forms/base_inc_js.html' %} 2 | {% block js %} 3 | 4 | {{ block.super }} 5 | {% endblock %} 6 | 7 | -------------------------------------------------------------------------------- /tests/data/sample/services/hello_world.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | echo "Working dir: $(pwd)" 3 | dir=$(pwd) 4 | sleep 5 5 | echo "Hello world" > ${dir}/hello_world_output.txt 6 | sleep 5 7 | echo "Follow " $1 >> ${dir}/hello_world_output.txt 8 | sleep 5 9 | echo "Last" $2 >> ${dir}/hello_world_output.txt 10 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/forms/bootstrap4/inc.css.html: -------------------------------------------------------------------------------- 1 | {% extends 'waves/services/forms/base_inc_css.html' %} 2 | {% block css %} 3 | 4 | {{ block.super }} 5 | {% endblock %} 6 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/forms/bootstrap/inc.css.html: -------------------------------------------------------------------------------- 1 | {% extends 'waves/services/forms/base_inc_css.html' %} 2 | {% block css %} 3 | 4 | {{ block.super }} 5 | {% endblock %} 6 | 7 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/forms/bootstrap3/inc.css.html: -------------------------------------------------------------------------------- 1 | {% extends 'waves/services/forms/base_inc_css.html' %} 2 | {% block css %} 3 | 4 | {{ block.super }} 5 | {% endblock %} 6 | 7 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include LICENSE.md 2 | include README.md 3 | include requirements.txt 4 | recursive-include waves/wcore/static * 5 | recursive-include waves/wcore/templates * 6 | recursive-include waves/front/templates * 7 | recursive-include docs * 8 | recursive-exclude waves_core * 9 | # recursive-exclude waves/wcore/migrations *. -------------------------------------------------------------------------------- /waves/wcore/static/waves/css/forms.css: -------------------------------------------------------------------------------- 1 | .dis_dep_parameter { 2 | display:block; 3 | } 4 | 5 | .hid_dep_parameter { 6 | display: none; 7 | } 8 | 9 | #panel-form-service{ 10 | padding-top: 2%; 11 | padding-left: 7%; 12 | } 13 | 14 | .copypaste > div.form-group > div.controls { 15 | width:100%; 16 | } 17 | 18 | -------------------------------------------------------------------------------- /waves/wcore/api/v1/serializers/__init__.py: -------------------------------------------------------------------------------- 1 | """ WAVES API serializers package """ 2 | from waves.wcore.api.v1.serializers.jobs import JobHistorySerializer, JobInputSerializer, JobSerializer, JobOutputSerializer 3 | from waves.wcore.api.v1.serializers.services import ServiceSubmissionSerializer, ServiceSerializer, ServiceFormSerializer 4 | 5 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/emails/job_admin_error.tpl: -------------------------------------------------------------------------------- 1 | ----------------------------------------------------------------- 2 | [ADMIN-NOTIFICATION] - Job '{{ job.service }}' - job failed 3 | ----------------------------------------------------------------- 4 | 5 | Sorry but job "{{ job.title }}" execution failed. 6 | 7 | {{ job.link }} 8 | 9 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/_service_head.html: -------------------------------------------------------------------------------- 1 |
2 |
3 |

{{ service.name }} - v{{ service.version }}

4 |

Online since {{ service.created }} / last update {{ service.updated }}

5 |
6 |
7 |
-------------------------------------------------------------------------------- /waves/wcore/templates/waves/admin/service/service_modal.html: -------------------------------------------------------------------------------- 1 | {% load staticfiles %} 2 | 3 |
4 |
5 | 8 |
9 |
-------------------------------------------------------------------------------- /AUTHORS.md: -------------------------------------------------------------------------------- 1 | Authors 2 | ======= 3 | 4 | Marc Chakiachvili [1,2], Sylvain Milanesi [1], Anne-Muriel Arigon Chifolleau [1], Vincent Lefort [1] 5 | 6 | 7 | [1] LIRMM, UMR 5506 - CNRS & Université de Montpellier, Montpellier, FRANCE 8 | 9 | [2] European Molecular Biology Laboratory, European Bioinformatic Institute, Wellcome Genome Campus, Hinxton, Cambridge, CB101SD, United Kingdom -------------------------------------------------------------------------------- /waves/wcore/api/__init__.py: -------------------------------------------------------------------------------- 1 | """ 2 | import current version for api 3 | Allow users to only use from waves.wcore.api 4 | 5 | from waves.wcore.api import serializers 6 | from waves.wcore.api import views 7 | 8 | ... 9 | YOUR CODE 10 | .... 11 | 12 | """ 13 | from __future__ import unicode_literals 14 | 15 | import v2.serializers as serializers 16 | import v2.views as views 17 | -------------------------------------------------------------------------------- /waves/wcore/forms/helper.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | 4 | class WFormHelper: 5 | 6 | def __init__(self, *args, **kwargs): 7 | pass 8 | 9 | def set_layout(self, service_input, form=None): 10 | raise NotImplementedError() 11 | 12 | def init_layout(self, fields): 13 | pass 14 | 15 | def end_layout(self): 16 | pass 17 | -------------------------------------------------------------------------------- /waves/wcore/__init__.py: -------------------------------------------------------------------------------- 1 | """ 2 | WAVES django components library 3 | """ 4 | __version_detail__ = '1.6.7' 5 | __version__ = '1.6' 6 | __author__ = 'Marc Chakiachvili, MAB Team' 7 | __licence__ = 'GPLv3' 8 | __copyright__ = "Copyright(C) 2015-2018, LIRMM - UM - CNRS" 9 | __db_version__ = "1.3" 10 | __name__ = "Waves Core package" 11 | 12 | default_app_config = 'waves.wcore.apps.WavesConfig' 13 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/forms/bootstrap4/inc.js.html: -------------------------------------------------------------------------------- 1 | {% extends 'waves/services/forms/base_inc_js.html' %} 2 | {% block js %} 3 | 4 | 5 | {{ block.super }} 6 | {% endblock %} 7 | -------------------------------------------------------------------------------- /docs/modules/managers/managers.rst: -------------------------------------------------------------------------------- 1 | Managers 2 | ======== 3 | 4 | .. _service-manager-label: 5 | 6 | Service Manager 7 | --------------- 8 | .. autoclass:: waves.wcore.models.services.ServiceManager 9 | :members: 10 | :undoc-members: 11 | 12 | .. _job-manager-label: 13 | 14 | Job Manager 15 | ----------- 16 | .. autoclass:: waves.wcore.models.jobs.JobManager 17 | :members: 18 | :undoc-members: 19 | -------------------------------------------------------------------------------- /templates/_navbar.html: -------------------------------------------------------------------------------- 1 |
  • Home
  • 2 |
  • Online 3 | services
  • 4 | {% if user.is_authenticated %} 5 |
  • Your jobs
  • 6 | {% endif %} 7 | -------------------------------------------------------------------------------- /waves_core/celery.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import, unicode_literals 2 | from celery import Celery 3 | from django.conf import settings 4 | import os 5 | 6 | import django 7 | os.environ.setdefault("DJANGO_SETTINGS_MODULE", "waves_core.settings") 8 | django.setup() 9 | 10 | app = Celery('waves_core', broker_pool_limit=1, broker=settings.URL_BROKER, result_backend=settings.URL_BROKER) 11 | app.autodiscover_tasks() -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/base.html: -------------------------------------------------------------------------------- 1 | {% extends 'base.html' %} 2 | 3 | {% block navbar-left %} 4 | {% include "_navbar.html" with active_link="services" %} 5 | {% endblock %} 6 | 7 | {% block splash %} 8 | {% endblock %} 9 | 10 | {% block container %} 11 | {% block content_head %} 12 | {% endblock %} 13 | {% block content_main %} 14 | {% endblock %} 15 | {% block content_foot %} 16 | {% endblock %} 17 | {% endblock %} -------------------------------------------------------------------------------- /waves_core/wsgi.py: -------------------------------------------------------------------------------- 1 | """ 2 | WSGI config for waves_core project. 3 | 4 | It exposes the WSGI callable as a module-level variable named ``application``. 5 | 6 | For more information on this file, see 7 | https://docs.djangoproject.com/en/1.11/howto/deployment/wsgi/ 8 | """ 9 | 10 | import os 11 | from django.core.wsgi import get_wsgi_application 12 | 13 | os.environ.setdefault("DJANGO_SETTINGS_MODULE", "waves_core.settings") 14 | 15 | application = get_wsgi_application() 16 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | Django~=1.11 2 | coreapi==2.3.3 3 | daemons==1.3.0 4 | django-admin-sortable2==0.6.19 5 | django-cors-headers==2.2.0 6 | django-crispy-forms==1.7.2 7 | django-crontab==0.7.1 8 | django-polymorphic==2.0.2 9 | djangorestframework>=3.9.1 10 | backports.ssl-match-hostname 11 | inflection==0.3.1 12 | psutil==5.4.5 13 | cryptography==2.5 14 | python-magic==0.4.15 15 | saga-python==0.60.1 16 | setproctitle==1.1.10 17 | six==1.11.0 18 | swapper==1.1.0 19 | celery==4.3.0 20 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/emails/job_cancelled.tpl: -------------------------------------------------------------------------------- 1 | ----------------------------------------------------------------------------------------------- 2 | [WAVES-NOTIFICATION] - Your job for '{{ job.service }}' - job automatically cancelled :-( 3 | ----------------------------------------------------------------------------------------------- 4 | 5 | Your job "{{ job.title }}" was automatically cancelled 6 | 7 | You can find more information about your job at 8 | 9 | {{ job.link }} 10 | -------------------------------------------------------------------------------- /waves/authentication/admin.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from django.contrib import admin 4 | 5 | from waves.authentication.models import WavesApiUser 6 | 7 | 8 | class ApiKeyAdmin(admin.ModelAdmin): 9 | list_display = ('user', 'key', 'created') 10 | fields = ('user', 'created', 'key', 'ip_list', 'domain') 11 | ordering = ('-created',) 12 | readonly_fields = ('user', 'created', 'key') 13 | 14 | 15 | admin.site.register(WavesApiUser, ApiKeyAdmin) 16 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/emails/job_warning.tpl: -------------------------------------------------------------------------------- 1 | ----------------------------------------------------------------------------------------------- 2 | [WAVES-NOTIFICATION] - Your job for '{{ job.service }}' - job terminated with warnings :-( 3 | ----------------------------------------------------------------------------------------------- 4 | 5 | Your job "{{ job.title }}" is completed, but with warnings. 6 | 7 | You can find more information about your job at 8 | 9 | {{ job.link }} 10 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/emails/job_error.tpl: -------------------------------------------------------------------------------- 1 | ------------------------------------------------------------------------------------------- 2 | [WAVES-NOTIFICATION] - Your job for '{{ job.service }}' - job unfortunately failed :-( 3 | ------------------------------------------------------------------------------------------- 4 | 5 | Your job "{{ job.title }}" unfortunately failed 6 | 7 | Don't panic, our team has been notified. 8 | 9 | You can find more information about your job at 10 | 11 | {{ job.link }} 12 | -------------------------------------------------------------------------------- /waves/wcore/commands/__init__.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from django.utils.module_loading import import_string, import_module 4 | 5 | 6 | def get_commands_impl_list(): 7 | classes_list = [('', 'Select a implementation class...')] 8 | mod = import_module('waves.wcore.commands') 9 | for cls in sorted(mod.__all__): 10 | clazz = import_string(mod.__name__ + '.' + cls) 11 | classes_list.append((clazz.__module__ + '.' + clazz.__name__, clazz.__name__)) 12 | return classes_list 13 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/emails/job_submitted.tpl: -------------------------------------------------------------------------------- 1 | ------------------------------------------------------------------------------------------- 2 | [WAVES-NOTIFICATION] - Job submission for '{{ job.service }}' - your job is {{ job.get_status_display|lower }} 3 | ------------------------------------------------------------------------------------------- 4 | 5 | 6 | Your job "{{ job.title }}" has been successfully {{ job.get_status_display|lower }} 7 | 8 | You can follow your job on the web at 9 | 10 | {{ job.link }} 11 | 12 | -------------------------------------------------------------------------------- /docs/modules/models/runners.rst: -------------------------------------------------------------------------------- 1 | Computing infrastructures 2 | ========================= 3 | 4 | Computing infrastructure models are in charge to store the configuration for expected adaptors to run jobs 5 | 6 | 7 | Runner 8 | ------ 9 | .. automodule:: waves.wcore.models.runners 10 | :members: 11 | :undoc-members: 12 | :show-inheritance: 13 | 14 | Adaptors Parameters 15 | ------------------- 16 | 17 | .. automodule:: waves.wcore.models.adaptors 18 | :members: 19 | :show-inheritance: 20 | -------------------------------------------------------------------------------- /waves/wcore/admin/views/__init__.py: -------------------------------------------------------------------------------- 1 | """ 2 | Admin specific views 3 | """ 4 | 5 | from waves.wcore.admin.views.job_tool import JobCancelView, JobRerunView 6 | from waves.wcore.admin.views.json_view import JSONDetailView, JSONView 7 | from waves.wcore.admin.views.runner_tool import RunnerExportView, RunnerImportToolView, RunnerTestConnectionView 8 | from waves.wcore.admin.views.service_tool import ServiceDuplicateView, ServiceExportView, ServiceParamImportView, ServiceTestConnectionView, \ 9 | ServicePreviewForm, ServiceModalPreview 10 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/emails/job_completed.tpl: -------------------------------------------------------------------------------- 1 | ------------------------------------------------------------------------------------------------------------------ 2 | [WAVES-NOTIFICATION] - Job completed for '{{ job.service }}' - your job is {{ job.get_status_display|lower }} 3 | ------------------------------------------------------------------------------------------------------------------ 4 | 5 | Your job "{{ job.title }}" is completed: "{{ job.get_status_display|lower }}" 6 | 7 | Retrieve all your job details at 8 | 9 | {{ job.link }} 10 | -------------------------------------------------------------------------------- /waves/wcore/admin/__init__.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ Models admin packages """ 3 | from __future__ import unicode_literals 4 | 5 | from waves.wcore.admin.runners import RunnerAdmin, RunnerTestConnectionView, RunnerImportToolView, RunnerExportView 6 | from waves.wcore.admin.services import ServiceAdmin 7 | from waves.wcore.admin.submissions import ServiceSubmissionAdmin 8 | from waves.wcore.admin.inputs import AllParamModelAdmin 9 | from waves.wcore.admin.jobs import JobAdmin 10 | from waves.wcore.admin.binaries import ServiceBinaryFileAdmin 11 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/admin/job/change_form.html: -------------------------------------------------------------------------------- 1 | {% extends "waves/admin/change_form.html" %} 2 | {% load i18n admin_modify admin_urls admin_static %} 3 | {% block object-tools-items %} 4 |
  • 5 | 9 | Re-Run job 10 | 11 |
  • 12 | {% endblock %} 13 | {% block submit_buttons_bottom %} 14 | {{ block.super }} 15 | {% endblock %} 16 | -------------------------------------------------------------------------------- /docs/modules/settings.rst: -------------------------------------------------------------------------------- 1 | Application custom settings 2 | =========================== 3 | 4 | WAVES-core application defines a waves_settings attributes generated from WAVES_CORE Django settings dictionary. 5 | 6 | .. automodule:: waves.wcore.settings 7 | :members: waves_settings 8 | 9 | .. note:: 10 | You may override these values in settings with a dict named WAVES_CORE 11 | 12 | 13 | Here are the defaults values: 14 | 15 | .. literalinclude:: ../../waves/wcore/settings.py 16 | :language: python 17 | :lines: 52-86 18 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/admin/baseparam/popup_response.html: -------------------------------------------------------------------------------- 1 | {% load i18n static %} 2 | 3 | 4 | 5 | {% trans 'My Popup closing...' %} 6 | 7 | 8 |
    10 |
    11 | 16 | 17 | 18 | 19 | -------------------------------------------------------------------------------- /waves/authentication/apps.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from django.apps import AppConfig 4 | from django.utils.translation import ugettext_lazy as _ 5 | 6 | 7 | class ApiKeyConfig(AppConfig): 8 | name = 'waves.authentication' 9 | verbose_name = _("WAVES Api Authentication") 10 | 11 | # noinspection PyUnresolvedReferences 12 | def ready(self): 13 | """ 14 | Executed once when WAVES application startup ! 15 | Just import waves signals 16 | :return: None 17 | """ 18 | import waves.authentication.signals 19 | -------------------------------------------------------------------------------- /waves/wcore/api/v2/serializers/__init__.py: -------------------------------------------------------------------------------- 1 | """ WAVES API serializers package """ 2 | from __future__ import unicode_literals 3 | 4 | from waves.wcore.api.v2.serializers.jobs import JobHistorySerializer, JobInputSerializer, JobSerializer, JobOutputSerializer 5 | from waves.wcore.api.v2.serializers.services import ServiceSubmissionSerializer, ServiceSerializer 6 | from waves.wcore.api.v2.serializers.inputs import InputSerializer 7 | 8 | __all__ = ['JobInputSerializer', 'JobHistorySerializer', 'JobSerializer', 'JobOutputSerializer', 9 | 'ServiceSubmissionSerializer', 'ServiceSerializer'] 10 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/admin/change_form.html: -------------------------------------------------------------------------------- 1 | {% extends "admin/change_form.html" %} 2 | {% load i18n waves_tags admin_static %} 3 | 4 | {% block extrahead %} 5 | {{ block.super }} 6 | {% endblock %} 7 | {% block content_title %} 8 | {% if not title %} 9 |

    Edit "{{ opts.verbose_name }} - {{ original }}"

    10 | {% else %} 11 | {{ block.super }} 12 | {% endif %} 13 | {% endblock %} 14 | {% block footer %} 15 | {{ block.super }} 16 | {% include 'waves/admin/modal_alert.html' %} 17 | {% include 'waves/admin/modal_content.html' %} 18 | {% endblock %} -------------------------------------------------------------------------------- /tests/data/sample/physic_ist/runs.json: -------------------------------------------------------------------------------- 1 | { 2 | "runs": [ 3 | { 4 | "title": "Physic IST Basic test", 5 | "inputs": { 6 | "s": "physic_ist_ex.txt" 7 | }, 8 | "params": { 9 | "o": "Basic out tree.txt" 10 | } 11 | }, 12 | { 13 | "title": "Physic IST full test", 14 | "inputs": { 15 | "s": "physic_ist_ex.txt", 16 | "r": "outgroup_ex.txt" 17 | }, 18 | "params": { 19 | "o": "Full out Tree", 20 | "b": 50, 21 | "f": "new Forest File", 22 | "c": 0.5 23 | } 24 | } 25 | ] 26 | } -------------------------------------------------------------------------------- /waves/wcore/tests/fixtures/users.json: -------------------------------------------------------------------------------- 1 | [ 2 | { 3 | "model": "auth.user", 4 | "pk": 1, 5 | "fields": { 6 | "password": "pbkdf2_sha256$36000$HKqSEnODwS15$mO14t3n/sxl1UFPeBY4PBChQdu8fnPGE5gK0l63XaYI=", 7 | "last_login": "2017-09-08T12:16:24.156Z", 8 | "is_superuser": true, 9 | "username": "marc", 10 | "first_name": "", 11 | "last_name": "", 12 | "email": "marc.chakiachvili@gmail.com", 13 | "is_staff": true, 14 | "is_active": true, 15 | "date_joined": "2017-07-21T14:33:58.194Z", 16 | "groups": [], 17 | "user_permissions": [] 18 | } 19 | } 20 | ] -------------------------------------------------------------------------------- /waves/wcore/api/permissions.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from rest_framework import permissions 4 | 5 | 6 | class ServiceAccessPermission(permissions.BasePermission): 7 | message = "Access to service is not allowed" 8 | 9 | def has_object_permission(self, request, view, obj): 10 | if request.method == 'POST': 11 | return obj.available_for_user(request.user) and request.user.is_authenticated 12 | return obj.available_for_user(request.user) 13 | 14 | def has_permission(self, request, view): 15 | return super(ServiceAccessPermission, self).has_permission(request, view) 16 | 17 | -------------------------------------------------------------------------------- /waves/authentication/signals.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from django.conf import settings 4 | from django.db.models.signals import post_save 5 | from django.dispatch import receiver 6 | 7 | 8 | @receiver(post_save, sender=settings.AUTH_USER_MODEL) 9 | def create_auth_api_key(sender, instance=None, created=False, **kwargs): 10 | if 'waves.authentication' in settings.INSTALLED_APPS and not kwargs.get('raw', False): 11 | from waves.authentication.models import WavesApiUser 12 | if created or WavesApiUser.objects.filter(user=instance).count() == 0: 13 | WavesApiUser.objects.create(user=instance) 14 | -------------------------------------------------------------------------------- /waves/wcore/api/urls.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from django.conf.urls import url, include 4 | from django.conf import settings 5 | from waves.wcore.api.views.base import schema_view 6 | 7 | urlpatterns = [ 8 | # Schema to be used with coreapi 9 | url(r'^schema$', schema_view), 10 | url(r'^', include('waves.wcore.api.v2.urls', namespace='v2')), 11 | url(r'^v1/', include('waves.wcore.api.v1.urls', namespace='v1')), 12 | ] 13 | 14 | if 'waves.authentication' in settings.INSTALLED_APPS: 15 | from waves.authentication import views 16 | urlpatterns.append(url(r'^api-token-auth', views.obtain_auth_token)) 17 | -------------------------------------------------------------------------------- /tests/data/sample/maf/out.fasta: -------------------------------------------------------------------------------- 1 | >canFam2 2 | CG------GCGTCTGTAAGGGGCCACCGCCCGGCCTGTG-CTCAAAGCTACAAATGACTCAACTCCCAACCGA------C------------------------------------- 3 | >hg18 4 | GACAGGGTGCATCTGGGAGGG---CCTGCCGGGCCTTTA-TTCAACACTAGATACGCCCCATCTCCAATTCTAATGGAC-ATGTGCAGAAAATGTGATACAGAAACCTGCAGAGCAG 5 | >mm8 6 | AGAAGGATCCACCT------------TGCTGGGCCTCTGCTCCAGCAAGACCCACCTCCCAACTCAAATGCCC-------------------------------------------- 7 | >panTro2 8 | GACAGGGTGCATCTGAGAGGG---CCTGCCAGGCCTTTA-TTCAACACTAGATACGCCCCATCTCCAATTCTAATGGAC-ATGTGCAGAAAATGTGATACAGAAACCTGCAGAGCAG 9 | >rheMac2 10 | GACAGGGTGCATCTGAGAGGG---CCTGCTGGGCCTTTG-TTCAAAACTAGATATGCCCCAACTCCAATTCTA-------ATGTGCGGAAAATGTGATACAGAAACCTGCAGAGCAG -------------------------------------------------------------------------------- /waves/wcore/management/commands/wqueue.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals, absolute_import 2 | 3 | import logging 4 | import os 5 | import tempfile 6 | 7 | from waves.wcore.management.daemoncommand import DaemonCommand 8 | from waves.wcore.management.runner import JobQueueRunDaemon 9 | 10 | logger = logging.getLogger('waves.daemon') 11 | 12 | 13 | class Command(DaemonCommand): 14 | """ 15 | Dedicated command to summarize current WAVES specific settings 16 | """ 17 | help = 'Managing WAVES job queue states' 18 | SLEEP_TIME = 2 19 | pidfile = os.path.join(tempfile.gettempdir(), 'waves_queue.pid') 20 | pidfile_timeout = 5 21 | _class = JobQueueRunDaemon 22 | -------------------------------------------------------------------------------- /waves/wcore/migrations/0002_auto_20190624_1122.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Generated by Django 1.11.21 on 2019-06-24 09:22 3 | from __future__ import unicode_literals 4 | 5 | from django.db import migrations, models 6 | 7 | 8 | class Migration(migrations.Migration): 9 | 10 | dependencies = [ 11 | ('wcore', '0001_initial'), 12 | ] 13 | 14 | operations = [ 15 | migrations.AlterField( 16 | model_name='submissionoutput', 17 | name='extension', 18 | field=models.CharField(blank=True, default='', help_text='Used on WEB for display/download ', max_length=15, verbose_name='File extension (internal)'), 19 | ), 20 | ] 21 | -------------------------------------------------------------------------------- /waves/wcore/api/v2/urls.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from django.conf.urls import url, include 4 | from rest_framework import routers 5 | 6 | from waves.wcore.api.v2.views import jobs, services 7 | 8 | # API router setup 9 | router = routers.DefaultRouter(trailing_slash=False) 10 | # Services URIs configuration 11 | router.register(prefix=r'services', 12 | viewset=services.ServiceViewSet, 13 | base_name='waves-services') 14 | # Jobs URIs configuration 15 | router.register(prefix=r'jobs', 16 | viewset=jobs.JobViewSet, 17 | base_name='waves-jobs') 18 | 19 | urlpatterns = [ 20 | url(r'^', include(router.urls)), 21 | ] 22 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/emails/job_prepared.tpl: -------------------------------------------------------------------------------- 1 | ------------------------------------------------------------------------------------------- 2 | [WAVES-NOTIFICATION] - Job submission for '{{ job.service }}' - your job is {{ job.get_status_display|lower }} 3 | ------------------------------------------------------------------------------------------- 4 | 5 | Your job "{{ job.title }}" is now prepared for running 6 | 7 | List of inputs data 8 | {% for out in job.input_files %} 9 | - {{ out.name }} {{ out.link }} 10 | {% endfor %} 11 | 12 | Job will run with following parameters: 13 | {% for out in job.input_params %} 14 | - {{ out.name }} 15 | {% endfor %} 16 | 17 | -------------------------------------------------------------------------------- /waves/wcore/management/commands/wpurge.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals, absolute_import 2 | 3 | import logging 4 | import os 5 | 6 | from waves.wcore.management.daemoncommand import DaemonCommand 7 | from waves.wcore.management.runner import PurgeDaemon 8 | from waves.wcore.settings import waves_settings 9 | 10 | logger = logging.getLogger('waves.daemon') 11 | 12 | 13 | class Command(DaemonCommand): 14 | """ 15 | Dedicated command to summarize current WAVES specific settings 16 | """ 17 | help = 'Managing WAVES job queue states' 18 | SLEEP_TIME = 2 19 | pidfile = os.path.join(waves_settings.DATA_ROOT, 'waves_purge.pid') 20 | pidfile_timeout = 5 21 | _class = PurgeDaemon 22 | -------------------------------------------------------------------------------- /tests/settings.ini.sample: -------------------------------------------------------------------------------- 1 | [saga] 2 | WAVES_TEST_SSH_HOST: 127.0.0.1 3 | WAVES_TEST_SSH_USER_ID: %USER% 4 | WAVES_TEST_SSH_USER_PASS: passsword 5 | WAVES_TEST_SSH_BASE_DIR: %HOMEDIR% 6 | 7 | [sge_cluster] 8 | WAVES_LOCAL_TEST_SGE_CELL: all 9 | WAVES_SSH_TEST_SGE_CELL: all 10 | WAVES_SSH_TEST_SGE_BASE_DIR: %HOMEDIR% 11 | 12 | [ssh_key_cluster] 13 | WAVES_SSH_KEY_USER_ID: %USER% 14 | WAVES_SSH_KEY_PASSPHRASE: passphrase 15 | WAVES_SSH_KEY_BASE_DIR: %HOMEDIR% 16 | WAVES_SSH_KEY_HOST: 127.0.0.1 17 | 18 | [slurm_cluster] 19 | WAVES_SLURM_TEST_SSH_HOST: 127.0.0.1 20 | WAVES_SLURM_TEST_SSH_USER_ID: %USER% 21 | WAVES_SLURM_TEST_SSH_USER_PASS: password 22 | WAVES_SLURM_TEST_SSH_QUEUE: queue 23 | WAVES_SLURM_TEST_SSH_BASE_DIR: %HOMEDIR% -------------------------------------------------------------------------------- /waves/wcore/models/binaries.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from django.db import models 4 | 5 | from waves.wcore.models.base import Slugged, TimeStamped 6 | from waves.wcore.utils.storage import binary_storage, binary_directory 7 | 8 | 9 | class ServiceBinaryFile(Slugged, TimeStamped): 10 | class Meta: 11 | verbose_name = 'Binary file' 12 | verbose_name_plural = 'Binaries files' 13 | 14 | label = models.CharField('Binary file label', max_length=255, null=False) 15 | binary = models.FileField('Binary file', upload_to=binary_directory, storage=binary_storage) 16 | 17 | def __str__(self): 18 | return self.label 19 | 20 | def __unicode__(self): 21 | return self.label -------------------------------------------------------------------------------- /templates/waves/override/service_simple_cp_form.html: -------------------------------------------------------------------------------- 1 | {% extends "waves/services/service_form.html" %} 2 | {% load staticfiles crispy_forms_tags waves_tags %} 3 | 4 | {% block content_main %} 5 |
    6 |
    7 |

    This is overridden template for service {{ service.name }} form

    8 |
    9 |
    10 | {% submission_form %} 11 |
    12 | 17 |
    18 | {% endblock %} -------------------------------------------------------------------------------- /waves/wcore/templates/waves/admin/modal_alert.html: -------------------------------------------------------------------------------- 1 | 2 | 19 | 20 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/admin/modal_content.html: -------------------------------------------------------------------------------- 1 | 2 | 20 | -------------------------------------------------------------------------------- /waves/wcore/utils/decorators.py: -------------------------------------------------------------------------------- 1 | import functools 2 | 3 | 4 | def log_exception(logger): 5 | """ 6 | A decorator that wraps the passed in function and logs 7 | exceptions should one occur 8 | 9 | @param logger: The logging object 10 | """ 11 | def decorator(func): 12 | @functools.wraps(func) 13 | def wrapper(*args, **kwargs): 14 | try: 15 | return func(*args, **kwargs) 16 | except Exception as e: 17 | # log the log_exception 18 | err = "Exception [{}] in {}".format(func.__name__, e.message) 19 | logger.exception(err) 20 | # re-raise the log_exception 21 | raise 22 | 23 | return wrapper 24 | 25 | return decorator 26 | -------------------------------------------------------------------------------- /waves/wcore/adaptors/utils.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from waves.wcore.adaptors.exceptions import AdaptorNotReady 4 | 5 | 6 | def check_ready(func): 7 | def inner(self, *args, **kwargs): 8 | if not all( 9 | [value is not None for init_param, value in self.init_params.items() if init_param in self._required]): 10 | # search missing values 11 | raise AdaptorNotReady( 12 | "Missing required parameter(s) for initialization: %s " % 13 | [init_param for init_param, value in 14 | self.init_params.items() if 15 | value is None and init_param in self._required]) 16 | else: 17 | return func(self, *args, **kwargs) 18 | 19 | return inner 20 | -------------------------------------------------------------------------------- /templates/waves/override/service_simple_cp_details.html: -------------------------------------------------------------------------------- 1 | {% extends "waves/services/service_details.html" %} 2 | {% load staticfiles crispy_forms_tags waves_tags %} 3 | 4 | {% block content_main %} 5 |
    6 |
    7 | 8 |

    {{ service.name }} "This is my overridden detailed page"

    9 |
    10 |
    11 |
    12 | {{ block.super }} 13 |
    14 | 19 |
    20 | {% endblock %} -------------------------------------------------------------------------------- /docs/modules/models/base.rst: -------------------------------------------------------------------------------- 1 | Shared classes 2 | ============== 3 | 4 | WAVES-core defines some top level models classes shared among other resources 5 | 6 | .. py:currentmodule:: waves.wcore.models.base 7 | 8 | APIModel 9 | -------- 10 | .. autoclass:: waves.wcore.models.base.ApiModel 11 | 12 | Described 13 | --------- 14 | .. autoclass:: waves.wcore.models.base.Described 15 | 16 | ExportAble 17 | ---------- 18 | .. autoclass:: waves.wcore.models.base.ExportAbleMixin 19 | 20 | Ordered 21 | ------- 22 | .. autoclass:: waves.wcore.models.base.Ordered 23 | 24 | Slugged 25 | ------- 26 | .. autoclass:: waves.wcore.models.base.Slugged 27 | 28 | TimeStamped 29 | ----------- 30 | .. autoclass:: waves.wcore.models.base.TimeStamped 31 | 32 | UrlMixin 33 | -------- 34 | .. autoclass:: waves.wcore.models.base.UrlMixin 35 | -------------------------------------------------------------------------------- /waves/wcore/cron/__init__.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | .. See the NOTICE file distributed with this work for additional information 4 | regarding copyright ownership. 5 | Licensed under the Apache License, Version 2.0 (the "License"); 6 | you may not use this file except in compliance with the License. 7 | You may obtain a copy of the License at 8 | http://www.apache.org/licenses/LICENSE-2.0 9 | Unless required by applicable law or agreed to in writing, software 10 | distributed under the License is distributed on an "AS IS" BASIS, 11 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | See the License for the specific language governing permissions and 13 | limitations under the License. 14 | """ 15 | from .process_queue import process_job_queue 16 | from .purge_jobs import purge_old_jobs 17 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/admin/baseparam/jet_popup_response.html: -------------------------------------------------------------------------------- 1 | {% load i18n static jet_tags %} 2 | 3 | 4 | 5 | {% trans 'My Popup closing...' %} 6 | 7 | 8 | 9 | 10 | 11 | {% jet_popup_response_data as data %} 12 |
    14 |
    15 | 22 | 23 | 24 | 25 | -------------------------------------------------------------------------------- /templates/admin/base_site.html: -------------------------------------------------------------------------------- 1 | {% extends "admin/base.html" %} 2 | {% load i18n waves_tags admin_static %} 3 | 4 | {% block branding %} 5 |
    6 | 8 |

    9 | 10 | {% if grappelli_admin_title %} {{ grappelli_admin_title }} {% else %} 11 | {{ site_title|default:"WAVES ADMIN" }} {% endif %} 12 | 13 |

    14 |
    15 | {% endblock %} 16 | {% block extrastyle %} 17 | {{ block.super }} 18 | {% endblock extrastyle %} 19 | 20 | {% block extrahead %} 21 | 22 | {{ block.super }} 23 | {% endblock %} 24 | 25 | -------------------------------------------------------------------------------- /manage.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import os 3 | import sys 4 | 5 | if __name__ == "__main__": 6 | os.environ.setdefault("DJANGO_SETTINGS_MODULE", "waves_core.settings") 7 | try: 8 | from django.core.management import execute_from_command_line 9 | except ImportError: 10 | # The above import may fail for some other reason. Ensure that the 11 | # issue is really that Django is missing to avoid masking other 12 | # exceptions on Python 2. 13 | try: 14 | import django 15 | except ImportError: 16 | raise ImportError( 17 | "Couldn't import Django. Are you sure it's installed and " 18 | "available on your PYTHONPATH environment variable? Did you " 19 | "forget to activate a virtual environment?" 20 | ) 21 | raise 22 | execute_from_command_line(sys.argv) 23 | -------------------------------------------------------------------------------- /waves/wcore/admin/binaries.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from os.path import isfile 4 | 5 | from django.contrib.admin import register 6 | 7 | from waves.wcore.admin.base import WavesModelAdmin 8 | from waves.wcore.models.binaries import ServiceBinaryFile 9 | 10 | 11 | @register(ServiceBinaryFile) 12 | class ServiceBinaryFileAdmin(WavesModelAdmin): 13 | model = ServiceBinaryFile 14 | list_display = ('label', 'created', 'updated', 'file_size', 'file_path') 15 | 16 | def file_size(self, obj): 17 | if isfile(obj.binary.path): 18 | return '%s ko' % (obj.binary.size / 1024) 19 | return "N/A" 20 | 21 | def file_path(self, obj): 22 | if isfile(obj.binary.path): 23 | return obj.binary.path 24 | return "N/A" 25 | 26 | def get_model_perms(self, request): 27 | # Disable direct entry to BinaryFiles 28 | return {} 29 | 30 | -------------------------------------------------------------------------------- /tests/data/sample/maf/in.maf: -------------------------------------------------------------------------------- 1 | a score=68686.000000 2 | s hg18.chr20 56827368 75 + 62435964 GACAGGGTGCATCTGGGAGGG---CCTGCCGGGCCTTTA-TTCAACACTAGATACGCCCCATCTCCAATTCTAATGGAC- 3 | s panTro2.chr20 56528685 75 + 62293572 GACAGGGTGCATCTGAGAGGG---CCTGCCAGGCCTTTA-TTCAACACTAGATACGCCCCATCTCCAATTCTAATGGAC- 4 | s rheMac2.chr10 89144112 69 - 94855758 GACAGGGTGCATCTGAGAGGG---CCTGCTGGGCCTTTG-TTCAAAACTAGATATGCCCCAACTCCAATTCTA------- 5 | s mm8.chr2 173910832 61 + 181976762 AGAAGGATCCACCT------------TGCTGGGCCTCTGCTCCAGCAAGACCCACCTCCCAACTCAAATGCCC------- 6 | s canFam2.chr24 46551822 67 + 50763139 CG------GCGTCTGTAAGGGGCCACCGCCCGGCCTGTG-CTCAAAGCTACAAATGACTCAACTCCCAACCGA------C 7 | 8 | a score=10289.000000 9 | s hg18.chr20 56827443 37 + 62435964 ATGTGCAGAAAATGTGATACAGAAACCTGCAGAGCAG 10 | s panTro2.chr20 56528760 37 + 62293572 ATGTGCAGAAAATGTGATACAGAAACCTGCAGAGCAG 11 | s rheMac2.chr10 89144181 37 - 94855758 ATGTGCGGAAAATGTGATACAGAAACCTGCAGAGCAG -------------------------------------------------------------------------------- /waves_core/rtd.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from waves_core.settings import * 4 | 5 | 6 | 7 | LOGGING = { 8 | 'version': 1, 9 | 'disable_existing_loggers': False, 10 | 'formatters': { 11 | 'verbose': { 12 | 'format': '[%(levelname)s][%(asctime)s][%(name)s.%(funcName)s:%(lineno)s] - %(message)s', 13 | 'datefmt': "%Y-%m-%d %H:%M:%S" 14 | }, 15 | }, 16 | 'handlers': { 17 | 'console': { 18 | 'class': 'logging.StreamHandler', 19 | 'formatter': 'verbose' 20 | }, 21 | }, 22 | 23 | 'loggers': { 24 | 'django': { 25 | 'handlers': ['console'], 26 | 'propagate': True, 27 | 'level': 'WARNING', 28 | }, 29 | 'waves': { 30 | 'handlers': ['console'], 31 | 'level': 'WARNING', 32 | 'propagate': True, 33 | }, 34 | } 35 | } 36 | -------------------------------------------------------------------------------- /docs/templates/layout.html: -------------------------------------------------------------------------------- 1 | {% extends "!layout.html" %} 2 | 3 | {% block extrahead %} 4 | 5 | 6 | 7 | {% endblock %} 8 | 9 | 10 | {% block footer %} 11 | 12 | 13 | 26 | {% endblock %} -------------------------------------------------------------------------------- /waves/wcore/templates/waves/jobs/parts/job_list_element.html: -------------------------------------------------------------------------------- 1 |
    2 |
    3 |
    4 |

    5 | {{ job.get_status_display }} 6 | {{ job.title }} 7 |

    8 |
    9 |
    10 |
    11 | 20 |
    21 |
    -------------------------------------------------------------------------------- /docs/index.rst: -------------------------------------------------------------------------------- 1 | Welcome to WAVES Documentation ! 2 | ======================================= 3 | .. warning:: 4 | If you migrate from 1.6.X to 1.6.5, and if you have any stored password 5 | A security issue has been fixed, you need to update your passwords in databases with this command: 6 | 7 | ./manage.py updates_keys 8 | 9 | 10 | .. toctree:: 11 | :maxdepth: 1 12 | 13 | readme 14 | authors 15 | license 16 | extensions 17 | 18 | .. toctree:: 19 | :caption: Changelog 20 | :maxdepth: 1 21 | 22 | changelog 23 | roadmap 24 | 25 | .. toctree:: 26 | :maxdepth: 2 27 | :caption: Administrator Guide 28 | 29 | installation 30 | user_doc/user_guide 31 | 32 | .. toctree:: 33 | :maxdepth: 2 34 | :caption: Technical documentation 35 | 36 | contributing 37 | dev_doc/dev_doc 38 | modules/settings 39 | modules/source 40 | 41 | Indices and tables 42 | ================== 43 | 44 | * :ref:`genindex` 45 | 46 | -------------------------------------------------------------------------------- /waves_core/urls.py: -------------------------------------------------------------------------------- 1 | """waves_core URL Configuration 2 | 3 | """ 4 | from __future__ import unicode_literals 5 | 6 | from django.conf import settings 7 | from django.conf.urls import url, include 8 | from django.conf.urls.static import static 9 | from django.contrib import admin 10 | from django.views.generic import TemplateView 11 | from rest_framework.documentation import include_docs_urls 12 | 13 | admin.site.site_title = 'WAVES Administration' 14 | 15 | urlpatterns = [ 16 | url(r'^$', TemplateView.as_view(template_name='base.html')), 17 | url(r'^admin/', admin.site.urls), 18 | url(r'^waves/', include('waves.wcore.urls', namespace='wcore')), 19 | url(r'^api/', include('waves.wcore.api.urls', namespace='wapi')), 20 | # Browsable API documentation 21 | url(r'^api-scheme', include_docs_urls(title='Waves API Documentation', public=False)), 22 | ] + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT) \ 23 | + static(settings.STATIC_URL, document_root=settings.STATIC_ROOT) 24 | -------------------------------------------------------------------------------- /docs/waves_uwsgi.ini: -------------------------------------------------------------------------------- 1 | [uwsgi] 2 | project=waves_core 3 | stats=127.0.0.1:9091 4 | 5 | chdir=/var/www/waves_core/ 6 | virtualenv=/var/www/waves_core/.venv 7 | master=True 8 | pidfile=/var/www/waves_core/waves-master.pid 9 | vacuum=True 10 | max-request=5000 11 | daemonize=/var/www/waves_core/logs/d-uwsgi.log 12 | app=waves 13 | env=DJANGO_SETTINGS_MODULE=waves_core.settings 14 | 15 | wsgi-file = webapp/wsgi.py 16 | processes = 4 17 | threads = 2 18 | logto = /var/www/waves_core/logs/uwsgi.log 19 | module = webapp.wsgi:application 20 | 21 | ############## Serve directly with uwsgi ####################### 22 | # http = 127.0.0.1:9090 23 | # static-map = /static=[ABSOLUTE_PATH_TO_WAVES_STATIC_DIR]/ 24 | # static-map = /media=[ABSOLUTE_PATH_TO_WAVES_MEDIA_DIR]/ 25 | # ################################################################## 26 | 27 | # ################ Integrated in WebServer (Nginx or Apache) ####### 28 | socket=127.0.0.1:3031 29 | # ################################################################## 30 | 31 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/admin/service/service_preview.html: -------------------------------------------------------------------------------- 1 | {% load waves_tags %} 2 | 3 | 4 | 5 | 6 | 7 | {% service_inc "css" %} 8 | 9 | 10 |
    11 | {% for message in messages %} 12 |
    13 | 14 | 15 | {{ message|safe }} 16 |
    17 | {% empty %} 18 |
    19 | Warning Submitting form will create a new job 20 |
    21 | {% endfor %} 22 |
    23 | {% submission_form %} 24 |
    25 |
    26 | 27 | {% service_inc "js" %} 28 | 29 | 30 | -------------------------------------------------------------------------------- /waves/wcore/static/waves/admin/js/connect.js: -------------------------------------------------------------------------------- 1 | /** 2 | * Created by marc on 20/02/17. 3 | */ 4 | 5 | (function ($) { 6 | $(document).ready(function () { 7 | 8 | $('#test_connect').click(function (e) { 9 | e.preventDefault(); 10 | $('#modal_alert .modal-content .modal-header > h4').html('Connection test'); 11 | $("#alert_modal_content > div.modal-body").html(''); 12 | $('#modal_alert').modal('toggle'); 13 | $.getJSON($(this).attr('href'), function (data) { 14 | $('#modal_alert .modal-content .modal-body').html(data['connection_result']) 15 | }) 16 | }); 17 | 18 | var $loading = $('#loading').hide(); 19 | $(document) 20 | .ajaxStart(function () { 21 | $loading.show(); 22 | }) 23 | .ajaxStop(function () { 24 | $loading.hide(); 25 | }); 26 | }); 27 | 28 | })(jQuery || django.jQuery); 29 | -------------------------------------------------------------------------------- /waves/authentication/views.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from rest_framework import parsers, renderers 4 | from rest_framework.response import Response 5 | from rest_framework.views import APIView 6 | 7 | from .models import WavesApiUser 8 | from .serializers import WavesApiUserSerializer 9 | 10 | 11 | class ObtainAuthToken(APIView): 12 | throttle_classes = () 13 | permission_classes = () 14 | parser_classes = (parsers.FormParser, parsers.MultiPartParser, parsers.JSONParser,) 15 | renderer_classes = (renderers.JSONRenderer,) 16 | serializer_class = WavesApiUserSerializer 17 | 18 | def post(self, request, *args, **kwargs): 19 | serializer = self.serializer_class(data=request.data) 20 | serializer.is_valid(raise_exception=True) 21 | user = serializer.validated_data['user'] 22 | token, created = WavesApiUser.objects.get_or_create(user=user) 23 | return Response({'auth_api_key': token.key}) 24 | 25 | 26 | obtain_auth_token = ObtainAuthToken.as_view() 27 | -------------------------------------------------------------------------------- /waves/wcore/urls.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from django.conf.urls import url 4 | from django.contrib.auth.decorators import login_required 5 | 6 | from waves.wcore.views.jobs import JobInputView, JobOutputView, JobSubmissionView, JobView, JobListView 7 | from waves.wcore.views.services import ServiceListView, ServiceDetailView 8 | 9 | 10 | urlpatterns = [ 11 | url(r'^services/$', ServiceListView.as_view(), name='services_list'), 12 | url(r'^service/(?P[\w_-]+)/$', ServiceDetailView.as_view(), name='service_details'), 13 | url(r'^service/(?P[\w_-]+)/new$', JobSubmissionView.as_view(), name='job_submission'), 14 | url(r'^jobs/(?P[\w-]+)/$', JobView.as_view(), name="job_details"), 15 | url(r'^jobs/inputs/(?P[\w-]+)/$', JobInputView.as_view(), name="job_input"), 16 | url(r'^jobs/outputs/(?P[\w-]+)/$', JobOutputView.as_view(), name="job_output"), 17 | url(r'^jobs/', login_required(JobListView.as_view()), name="job_list"), 18 | ] 19 | -------------------------------------------------------------------------------- /waves/wcore/api/views/base.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from django.conf import settings 4 | from rest_framework import renderers, schemas 5 | from rest_framework.decorators import api_view, renderer_classes, permission_classes 6 | from rest_framework.permissions import IsAuthenticated, AllowAny 7 | from rest_framework.views import APIView, Response 8 | 9 | 10 | class WavesAuthenticatedView(APIView): 11 | """ Base WAVES API view, set up for all subclasses permissions / authentication """ 12 | permission_classes = [IsAuthenticated, ] 13 | 14 | def get_permissions(self): 15 | if settings.DEBUG: 16 | self.permission_classes = [AllowAny, ] 17 | return super(WavesAuthenticatedView, self).get_permissions() 18 | 19 | 20 | @api_view(['GET', ]) 21 | @permission_classes((AllowAny,)) 22 | @renderer_classes([renderers.CoreJSONRenderer]) 23 | def schema_view(request): 24 | generator = schemas.SchemaGenerator(title='WAVES API schema') 25 | return Response(generator.get_schema()) 26 | -------------------------------------------------------------------------------- /templates/admin/login.html: -------------------------------------------------------------------------------- 1 | {% extends 'admin/login.html' %} 2 | {% load static %} 3 | {% block extrastyle %} 4 | {{ block.super }} 5 | 27 | {% endblock %} 28 | 29 | {% block breadcrumbs %} 30 |
    Welcome to WAVES
    31 | {% endblock %} 32 | 33 | -------------------------------------------------------------------------------- /waves/wcore/api/views/service.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from django.contrib.staticfiles.storage import staticfiles_storage 4 | 5 | from waves.wcore.views.services import SubmissionFormView 6 | 7 | 8 | class JobSubmissionView(SubmissionFormView): 9 | template_name = 'waves/api/service_api_form.html' 10 | 11 | def get_css(self): 12 | """ link to service css """ 13 | return [ 14 | self.request.build_absolute_uri(staticfiles_storage.url('waves/css/forms.css')), ] 15 | 16 | def get_js(self): 17 | """ link to service js""" 18 | return [ 19 | self.request.build_absolute_uri(staticfiles_storage.url('waves/js/services.js')), 20 | self.request.build_absolute_uri(staticfiles_storage.url('waves/js/api_services.js')), 21 | ] 22 | 23 | def get_context_data(self, **kwargs): 24 | context = super(JobSubmissionView, self).get_context_data(**kwargs) 25 | context['css'] = self.get_css() 26 | context['js'] = self.get_js() 27 | return context 28 | -------------------------------------------------------------------------------- /docs/modules/models/jobs.rst: -------------------------------------------------------------------------------- 1 | ==== 2 | Jobs 3 | ==== 4 | 5 | Jobs models documentation 6 | 7 | Job 8 | --- 9 | 10 | .. autoclass:: waves.wcore.models.jobs.Job 11 | :members: 12 | :undoc-members: 13 | :show-inheritance: 14 | 15 | Manager 16 | ^^^^^^^ 17 | .. autoclass:: waves.wcore.models.jobs.JobManager 18 | :members: 19 | :noindex: 20 | 21 | .. _job_input_label: 22 | 23 | Job Inputs 24 | ---------- 25 | Job Inputs related to inputs defined in service configuration, see :ref:`service-inputs-label`. 26 | 27 | .. autoclass:: waves.wcore.models.jobs.JobInput 28 | :members: 29 | :undoc-members: 30 | 31 | Manager 32 | ^^^^^^^ 33 | 34 | .. autoclass:: waves.wcore.models.jobs.JobInputManager 35 | :members: 36 | 37 | .. _job_output_label: 38 | 39 | Job Outputs 40 | ----------- 41 | Job outputs related to outputs defined in service configuration 42 | 43 | .. autoclass:: waves.wcore.models.jobs.JobOutput 44 | :members: 45 | :undoc-members: 46 | 47 | Manager 48 | ^^^^^^^ 49 | 50 | .. autoclass:: waves.wcore.models.jobs.JobOutputManager 51 | :members: 52 | -------------------------------------------------------------------------------- /waves/wcore/api/v1/serializers/fields.py: -------------------------------------------------------------------------------- 1 | from rest_framework import serializers 2 | 3 | 4 | class CommaSeparatedListField(serializers.ListField, serializers.ReadOnlyField): 5 | def to_representation(self, value): 6 | if ',' in value: 7 | str_txt = value.split(',') 8 | return super(CommaSeparatedListField, self).to_representation(str_txt) 9 | return super(CommaSeparatedListField, self).to_representation([]) 10 | 11 | 12 | class ListElementField(serializers.ListField, serializers.ReadOnlyField): 13 | def to_representation(self, value): 14 | list_elements = [] 15 | for line in value.splitlines(): 16 | line_element = line.split('|') 17 | list_elements.append(line_element) 18 | return ', '.join(value.splitlines()) 19 | 20 | 21 | class InputFormatField(serializers.Field): 22 | def to_representation(self, instance): 23 | return ', '.join(instance.splitlines()) if instance is not None else '' 24 | 25 | def to_internal_value(self, data): 26 | return data.replace(', ', '\n') if data else '' 27 | -------------------------------------------------------------------------------- /waves/wcore/models/__init__.py: -------------------------------------------------------------------------------- 1 | """ All WAVES related models imports """ 2 | from __future__ import unicode_literals 3 | 4 | # Automate sub module import 5 | import swapper 6 | from waves.wcore.models.adaptors import AdaptorInitParam, HasAdaptorClazzMixin 7 | from waves.wcore.models.history import JobHistory 8 | from waves.wcore.models.runners import Runner 9 | from waves.wcore.models.binaries import ServiceBinaryFile 10 | from waves.wcore.models.services import SubmissionOutput, SubmissionExitCode 11 | from waves.wcore.models.inputs import AParam, TextParam, BooleanParam, IntegerParam, DecimalParam, ListParam 12 | from waves.wcore.models.jobs import JobOutput, JobInput, Job 13 | from waves.wcore.models.binaries import ServiceBinaryFile 14 | 15 | 16 | def get_service_model(): 17 | return swapper.load_model('wcore', 'Service') 18 | 19 | 20 | def get_submission_model(): 21 | return swapper.load_model('wcore', 'Submission') 22 | 23 | 24 | """ 25 | List of different constants used for models 26 | """ 27 | OUT_TYPE = ( 28 | ('stout', 'Standard output'), 29 | ('file', 'Output file') 30 | ) 31 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/admin/submit_line.html: -------------------------------------------------------------------------------- 1 | {% load i18n admin_urls %} 2 |
    3 | {% if show_save %}{% endif %} 4 | {% if show_delete_link %} 5 | {% url 'opts|admin_urlname:' original.pk|admin_urlquote as delete_url %} 6 | 8 | {% endif %} 9 | {% if show_save_as_new %}{% endif %} 10 | {% if show_save_and_add_another %} 11 | {% endif %} 12 | {% if show_save_and_continue %} 13 | {% endif %} 14 | {% if show_save_and_back %} 15 | 16 | {% endif %} 17 |
    18 | -------------------------------------------------------------------------------- /waves/wcore/api/v1/urls.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from django.conf.urls import url, include 4 | from rest_framework import routers 5 | 6 | from waves.wcore.api.v1.views import jobs, services 7 | from waves.wcore.views.jobs import JobOutputView, JobInputView 8 | 9 | # API router setup 10 | router = routers.DefaultRouter() 11 | # Services URIs configuration 12 | router.register(prefix=r'services', 13 | viewset=services.ServiceViewSet, 14 | base_name='waves-services') 15 | # Jobs URIs configuration 16 | router.register(prefix=r'jobs', 17 | viewset=jobs.JobViewSet, 18 | base_name='waves-jobs') 19 | 20 | urlpatterns = [ 21 | url(r'^', include(router.urls)), 22 | url(r'^services/(?P[^/.]+)/submissions/(?P[^/.]+)/$', 23 | services.ServiceJobSubmissionView.as_view(), name='waves-services-submissions'), 24 | url(r'^jobs/outputs/(?P[\w-]+)/$', JobOutputView.as_view(), name="waves-job-output"), 25 | url(r'^jobs/inputs/(?P[\w-]+)/$', JobInputView.as_view(), name="waves-job-input"), 26 | ] 27 | 28 | -------------------------------------------------------------------------------- /waves/authentication/migrations/0002_auto_20180313_1344.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Generated by Django 1.11.8 on 2018-03-13 13:44 3 | from __future__ import unicode_literals 4 | 5 | from django.db import migrations, models 6 | 7 | 8 | class Migration(migrations.Migration): 9 | 10 | dependencies = [ 11 | ('authentication', '0001_initial'), 12 | ] 13 | 14 | operations = [ 15 | migrations.AlterModelOptions( 16 | name='wavesapiuser', 17 | options={'verbose_name': 'Waves Api auth', 'verbose_name_plural': 'Waves Api auths'}, 18 | ), 19 | migrations.AlterField( 20 | model_name='wavesapiuser', 21 | name='domain', 22 | field=models.CharField(blank=True, help_text='Comma separated list', max_length=255, null=True, verbose_name='Origin URL(s)'), 23 | ), 24 | migrations.AlterField( 25 | model_name='wavesapiuser', 26 | name='ip_list', 27 | field=models.CharField(blank=True, help_text='Comma separated list', max_length=255, null=True, verbose_name='Ip(s) List'), 28 | ), 29 | ] 30 | -------------------------------------------------------------------------------- /waves/wcore/static/waves/admin/js/submissions.js: -------------------------------------------------------------------------------- 1 | (function ($) { 2 | $(document).ready(function () { 3 | var input_group = $('#inputs-group') 4 | input_group.find('a.inlinechangelink').each(function () { 5 | var href = $(this).attr('href') 6 | if (href.indexOf('?') === -1) { 7 | href += '?_popup=1'; 8 | } else { 9 | href += '&_popup=1'; 10 | } 11 | $(this).attr('href', href); 12 | $(this).addClass('related-widget-wrapper-link change-related') 13 | }); 14 | $(document).on('formset:added', function (event, $row, formsetName) { 15 | if (formsetName === 'inputs') { 16 | // Do something 17 | event.preventDefault(); 18 | event.stopPropagation(); 19 | $row.remove() 20 | $('#add_submission_input_link').trigger('click'); 21 | } 22 | }); 23 | 24 | $(document).on('formset:removed', function (event, $row, formsetName) { 25 | // Row removed 26 | }); 27 | }) 28 | })(jQuery || django.jQuery); -------------------------------------------------------------------------------- /tests/data/sample/maf/simple.maf: -------------------------------------------------------------------------------- 1 | ##maf version=1 scoring=tba.v8 2 | # tba.v8 (((human chimp) baboon) (mouse rat)) 3 | # multiz.v7 4 | # maf_project.v5 _tba_right.maf3 mouse _tba_C 5 | # single_cov2.v4 single_cov2 /dev/stdin 6 | 7 | a score=23262.0 8 | s hg16.chr7 27578828 38 + 158545518 AAA-GGGAATGTTAACCAAATGA---ATTGTCTCTTACGGTG 9 | s panTro1.chr6 28741140 38 + 161576975 AAA-GGGAATGTTAACCAAATGA---ATTGTCTCTTACGGTG 10 | s baboon 116834 38 + 4622798 AAA-GGGAATGTTAACCAAATGA---GTTGTCTCTTATGGTG 11 | s mm4.chr6 53215344 38 + 151104725 -AATGGGAATGTTAAGCAAACGA---ATTGTCTCTCAGTGTG 12 | s rn3.chr4 81344243 40 + 187371129 -AA-GGGGATGCTAAGCCAATGAGTTGTTGTCTCTCAATGTG 13 | 14 | a score=5062.0 15 | s hg16.chr7 27699739 6 + 158545518 TAAAGA 16 | s panTro1.chr6 28862317 6 + 161576975 TAAAGA 17 | s baboon 241163 6 + 4622798 TAAAGA 18 | s mm4.chr6 53303881 6 + 151104725 TAAAGA 19 | s rn3.chr4 81444246 6 + 187371129 taagga 20 | a score=6636.0 21 | s hg16.chr7 27707221 13 + 158545518 gcagctgaaaaca 22 | s panTro1.chr6 28869787 13 + 161576975 gcagctgaaaaca 23 | s baboon 249182 13 + 4622798 gcagctgaaaaca 24 | s mm4.chr6 53310102 13 + 151104725 ACAGCTGAAAATA -------------------------------------------------------------------------------- /waves/wcore/import_export/tests.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | import json 4 | import logging 5 | 6 | from waves.wcore.import_export.services import ServiceSerializer 7 | from waves.wcore.models import get_service_model, get_submission_model 8 | from waves.wcore.tests.base import BaseTestCase 9 | 10 | logger = logging.getLogger(__name__) 11 | Service = get_service_model() 12 | Submission = get_submission_model() 13 | 14 | 15 | class SerializationTestCase(BaseTestCase): 16 | 17 | # @skip("Serialize / Unserialize needs code refactoring") 18 | def test_serialize_service(self): 19 | self.bootstrap_services() 20 | init_count = Service.objects.all().count() 21 | self.assertGreater(init_count, 0) 22 | file_paths = [] 23 | for srv in Service.objects.all(): 24 | file_out = srv.serialize() 25 | file_paths.append(file_out) 26 | for exp in file_paths: 27 | with open(exp) as fp: 28 | serializer = ServiceSerializer(data=json.load(fp)) 29 | if serializer.is_valid(): 30 | serializer.save() 31 | self.assertEqual(init_count * 2, Service.objects.all().count()) 32 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | License (GPLv3) 2 | =============== 3 | 4 | WAVES-core is free software: 5 | 6 | you can redistribute it and/or modify it under the terms of the GNU General Public License version 3 7 | as published by the `Free Software Foundation `__. 8 | 9 | This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; 10 | without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. 11 | See the GNU General Public License for more details. 12 | 13 | For more specific details see `http://www.gnu.org/licenses, `__ the 14 | `Quick Guide to GPLv3. `__ in the codebase. 15 | 16 | The GNU operating system which is under the same license has an informative 17 | `FAQ here `__. 18 | 19 | 20 | Note to developers 21 | ------------------ 22 | 23 | We very much appreciate you using our code to do fun and interesting things with. 24 | We hope that while doing this you may find and fix bugs or make enhancements that 25 | could be useful for the greater community and will makes the developers aware of them 26 | by emailing to waves-webapp@googlegroups.com -------------------------------------------------------------------------------- /fixtures_celery_beat.json: -------------------------------------------------------------------------------- 1 | [{"model": "django_celery_beat.intervalschedule", "pk": 1, "fields": {"every": 10, "period": "seconds"}}, {"model": "django_celery_beat.intervalschedule", "pk": 2, "fields": {"every": 1, "period": "minutes"}}, {"model": "django_celery_beat.periodictasks", "pk": 1, "fields": {"last_update": "2019-09-10T12:51:47.071Z"}}, {"model": "django_celery_beat.periodictask", "pk": 1, "fields": {"name": "Job Queue", "task": "job_queue", "interval": 1, "crontab": null, "solar": null, "clocked": null, "args": "[]", "kwargs": "{}", "queue": null, "exchange": null, "routing_key": null, "headers": "{}", "priority": null, "expires": null, "one_off": false, "start_time": null, "enabled": true, "last_run_at": null, "total_run_count": 0, "date_changed": "2019-09-10T12:51:30.265Z", "description": ""}}, {"model": "django_celery_beat.periodictask", "pk": 2, "fields": {"name": "Purge jobs", "task": "purge_jobs", "interval": 2, "crontab": null, "solar": null, "clocked": null, "args": "[]", "kwargs": "{}", "queue": null, "exchange": null, "routing_key": null, "headers": "{}", "priority": null, "expires": null, "one_off": false, "start_time": null, "enabled": true, "last_run_at": null, "total_run_count": 0, "date_changed": "2019-09-10T12:51:47.073Z", "description": ""}}] -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/file.html: -------------------------------------------------------------------------------- 1 | {% extends "waves/services/base.html" %} 2 | 3 | {% block container %} 4 |
    5 |
    6 |
    7 |
    8 |
    9 |
    10 |

    {{ file_name }}

    11 |
    12 |
    13 |
    {{ file_description|default:"" }}
    14 |
    {{ file_content }}
    15 |
    16 | 22 |
    23 |
    24 |
    25 |
    26 |
    27 | {% endblock %} -------------------------------------------------------------------------------- /static/waves/css/site.css: -------------------------------------------------------------------------------- 1 | .container { 2 | background-size: cover; 3 | background: rgb(226,226,226); /* Old browsers */ 4 | background: -moz-linear-gradient(top, rgba(226,226,226,1) 0%, rgba(219,219,219,1) 50%, rgba(209,209,209,1) 51%, rgba(254,254,254,1) 100%); /* FF3.6+ */ 5 | background: -webkit-gradient(linear, left top, left bottom, color-stop(0%,rgba(226,226,226,1)), color-stop(50%,rgba(219,219,219,1)), color-stop(51%,rgba(209,209,209,1)), color-stop(100%,rgba(254,254,254,1))); /* Chrome,Safari4+ */ 6 | background: -webkit-linear-gradient(top, rgba(226,226,226,1) 0%,rgba(219,219,219,1) 50%,rgba(209,209,209,1) 51%,rgba(254,254,254,1) 100%); /* Chrome10+,Safari5.1+ */ 7 | background: -o-linear-gradient(top, rgba(226,226,226,1) 0%,rgba(219,219,219,1) 50%,rgba(209,209,209,1) 51%,rgba(254,254,254,1) 100%); /* Opera 11.10+ */ 8 | background: -ms-linear-gradient(top, rgba(226,226,226,1) 0%,rgba(219,219,219,1) 50%,rgba(209,209,209,1) 51%,rgba(254,254,254,1) 100%); /* IE10+ */ 9 | background: linear-gradient(to bottom, rgba(226,226,226,1) 0%,rgba(219,219,219,1) 50%,rgba(209,209,209,1) 51%,rgba(254,254,254,1) 100%); /* W3C */ 10 | filter: progid:DXImageTransform.Microsoft.gradient( startColorstr='#e2e2e2', endColorstr='#fefefe',GradientType=0 ); /* IE6-9 */ 11 | padding: 20px; 12 | } 13 | -------------------------------------------------------------------------------- /waves/wcore/import_export/__init__.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from rest_framework import serializers 4 | from rest_framework.exceptions import ValidationError 5 | 6 | 7 | def check_db_version(func): 8 | def wrapper(*args, **kwargs): 9 | a = list(args) 10 | a.reverse() 11 | return func(*args, **kwargs) 12 | 13 | return func 14 | 15 | 16 | class BaseSerializer(serializers.ModelSerializer): 17 | class Meta: 18 | fields = ('db_version',) 19 | 20 | db_version = serializers.CharField(max_length=10) 21 | 22 | 23 | class RelatedSerializerMixin(object): 24 | """ Add serializers capability to create related easily""" 25 | 26 | # noinspection PyUnresolvedReferences 27 | def create_related(self, foreign, serializer, datas): 28 | """ Create related objects (foreign key to current service model object""" 29 | created = [] 30 | for data in datas: 31 | try: 32 | ser = serializer(data=data) 33 | ser.is_valid(True) 34 | params = {key: value for key, value in foreign.items()} 35 | created.append(ser.save(**params)) 36 | except ValidationError as e: 37 | self.errors[''] = e.detail 38 | return created 39 | -------------------------------------------------------------------------------- /waves/wcore/tests/test_runners.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals, print_function 2 | 3 | import logging 4 | 5 | from django.utils.module_loading import import_string 6 | 7 | from waves.wcore.adaptors import JobAdaptor 8 | from waves.wcore.tests.base import BaseTestCase, TestJobWorkflowMixin 9 | 10 | logger = logging.getLogger(__name__) 11 | 12 | 13 | class RunnerTestCase(BaseTestCase, TestJobWorkflowMixin): 14 | def test_create_runner(self): 15 | for runner in self.bootstrap_runners(): 16 | logger.info('Current: %s - %s', runner.name, runner.clazz) 17 | adaptor = runner.adaptor 18 | self.assertIsInstance(adaptor, JobAdaptor) 19 | self.assertEqual(runner.run_params.__len__(), adaptor.init_params.__len__()) 20 | self.assertListEqual(sorted(runner.run_params.keys()), sorted(adaptor.init_params.keys())) 21 | obj_runner = import_string(runner.clazz) 22 | expected_params = obj_runner().init_params 23 | runner_params = runner.run_params 24 | logger.debug("Expected %s", expected_params) 25 | logger.debug("Defaults %s", runner_params) 26 | logger.debug("Config %s", runner.adaptor._dump_config()) 27 | self.assertEquals(sorted(expected_params), sorted(runner_params)) 28 | -------------------------------------------------------------------------------- /docs/modules/adaptors/adaptors.rst: -------------------------------------------------------------------------------- 1 | .. _adaptor-label: 2 | 3 | ======== 4 | Adaptors 5 | ======== 6 | 7 | These modules execute job on dedicated platforms, remotely or locally, they are supposed to control job processing once submitted via services submissions 8 | 9 | Constants 10 | --------- 11 | .. automodule:: waves.wcore.adaptors.const 12 | 13 | Exceptions 14 | ---------- 15 | .. automodule:: waves.wcore.adaptors.exceptions 16 | 17 | Adaptor Loader 18 | -------------- 19 | .. automodule:: waves.wcore.adaptors.loader 20 | 21 | Utilities classes 22 | ----------------- 23 | .. automodule:: waves.wcore.adaptors.utils 24 | 25 | 26 | .. _adaptor-base-class-label: 27 | 28 | Adaptor base class 29 | ================== 30 | 31 | .. autoclass:: waves.wcore.adaptors.JobAdaptor 32 | :members: 33 | :show-inheritance: 34 | 35 | Shell script related adaptors class 36 | ----------------------------------- 37 | .. automodule:: waves.wcore.adaptors.shell 38 | :members: 39 | :show-inheritance: 40 | 41 | Api related adaptor base class 42 | ------------------------------ 43 | .. automodule:: waves.wcore.adaptors.api 44 | :members: 45 | :show-inheritance: 46 | 47 | Cluster related adaptors 48 | ------------------------ 49 | .. automodule:: waves.wcore.adaptors.cluster 50 | :members: 51 | :show-inheritance: 52 | -------------------------------------------------------------------------------- /waves/wcore/api/v2/serializers/fields.py: -------------------------------------------------------------------------------- 1 | from rest_framework import serializers 2 | 3 | 4 | class CommaSeparatedListField(serializers.ListField, serializers.ReadOnlyField): 5 | def to_representation(self, value): 6 | if ',' in value: 7 | str_txt = value.split(',') 8 | return super(CommaSeparatedListField, self).to_representation(str_txt) 9 | return super(CommaSeparatedListField, self).to_representation([]) 10 | 11 | 12 | class ListElementField(serializers.DictField, serializers.ReadOnlyField): 13 | def to_representation(self, value): 14 | list_values = [] 15 | list_labels = [] 16 | for line in value.splitlines(): 17 | line_element = line.split('|') 18 | list_values.append(line_element[1]) 19 | list_labels.append(line_element[0]) 20 | final_list = { 21 | "labels": list_labels, 22 | "values": list_values 23 | } 24 | return super(ListElementField, self).to_representation(final_list) 25 | 26 | 27 | class InputFormatField(serializers.Field): 28 | def to_representation(self, instance): 29 | return ', '.join(instance.splitlines()) if instance is not None else '' 30 | 31 | def to_internal_value(self, data): 32 | return data.replace(', ', '\n') if data else '' 33 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/forms/uni_form/submission_form.html: -------------------------------------------------------------------------------- 1 | {% load crispy_forms_tags %} 2 | {% with single_submission=submissions.count %} 3 | {% if single_submission > 1 %} 4 | 14 |
    15 | {% endif %} 16 | {% for sub in submissions %} 17 | 18 |
    20 | {% if preview %} 21 |
    Availability : {{ sub.submission.get_availability_display }}
    22 | {% endif %} 23 |
    24 | {% crispy sub.form %} 25 |
    26 |
    27 | 28 | {% endfor %} 29 | {% if single_submission > 1 %} 30 |
    31 | {% endif %} 32 | {% endwith %} -------------------------------------------------------------------------------- /waves/wcore/models/const.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | __all__ = ['OptType', 'ParamType'] 4 | 5 | 6 | class OptType(object): 7 | OPT_TYPE_NONE = 0 8 | OPT_TYPE_VALUATED = 1 9 | OPT_TYPE_SIMPLE = 2 10 | OPT_TYPE_OPTION = 3 11 | OPT_TYPE_POSIX = 4 12 | OPT_TYPE_NAMED_OPTION = 5 13 | OPT_TYPE_NAMED_PARAM = 6 14 | OPT_TYPE = [ 15 | (OPT_TYPE_NONE, "-- Not used in job command line--"), 16 | (OPT_TYPE_NAMED_PARAM, 'Assigned named parameter: [name]=value'), 17 | (OPT_TYPE_SIMPLE, 'Named short parameter: -[name] value'), 18 | (OPT_TYPE_VALUATED, 'Named assigned long parameter: --[name]=value'), 19 | (OPT_TYPE_OPTION, 'Named short option: -[name]'), 20 | (OPT_TYPE_NAMED_OPTION, 'Named long option: --[name]'), 21 | (OPT_TYPE_POSIX, 'Positional parameter: value') 22 | ] 23 | 24 | 25 | class ParamType(object): 26 | TYPE_BOOLEAN = 'boolean' 27 | TYPE_FILE = 'file' 28 | TYPE_LIST = 'list' 29 | TYPE_DECIMAL = 'decimal' 30 | TYPE_TEXT = 'text' 31 | TYPE_INT = 'int' 32 | IN_TYPE = [ 33 | (TYPE_FILE, 'Input file'), 34 | (TYPE_LIST, 'List of values'), 35 | (TYPE_BOOLEAN, 'Boolean'), 36 | (TYPE_DECIMAL, 'Decimal'), 37 | (TYPE_INT, 'Integer'), 38 | (TYPE_TEXT, 'Text') 39 | ] 40 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/forms/bootstrap/submission_form.html: -------------------------------------------------------------------------------- 1 | {% load crispy_forms_tags %} 2 | {% with single_submission=submissions.count %} 3 | {% if single_submission > 1 %} 4 | 14 |
    15 | {% endif %} 16 | {% for sub in submissions %} 17 | 18 |
    20 | {% if preview %} 21 |
    Availability : {{ sub.submission.get_availability_display }}
    22 | {% endif %} 23 |
    24 | {% crispy sub.form %} 25 |
    26 |
    27 | 28 | {% endfor %} 29 | {% if single_submission > 1 %} 30 |
    31 | {% endif %} 32 | {% endwith %} -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/forms/bootstrap4/submission_form.html: -------------------------------------------------------------------------------- 1 | {% load crispy_forms_tags %} 2 | {% with single_submission=submissions.count %} 3 | {% if single_submission > 1 %} 4 | 14 |
    15 | {% endif %} 16 | {% for sub in submissions %} 17 | 18 |
    20 | {% if preview %} 21 |
    Availability : {{ sub.submission.get_availability_display }}
    22 | {% endif %} 23 |
    24 | {% crispy sub.form %} 25 |
    26 |
    27 | 28 | {% endfor %} 29 | {% if single_submission > 1 %} 30 |
    31 | {% endif %} 32 | {% endwith %} -------------------------------------------------------------------------------- /waves/wcore/management/command.py: -------------------------------------------------------------------------------- 1 | """ Daemonized WAVES system commands """ 2 | from __future__ import unicode_literals 3 | 4 | import logging 5 | import os 6 | import tempfile 7 | 8 | from .daemoncommand import DaemonCommand 9 | from waves.wcore.management.runner import JobQueueRunDaemon, PurgeDaemon 10 | from waves.wcore.settings import waves_settings 11 | 12 | logger = logging.getLogger('waves.daemon') 13 | 14 | 15 | class JobQueueCommand(DaemonCommand): 16 | """ 17 | Dedicated command to summarize current WAVES specific settings 18 | """ 19 | help = 'Managing WAVES job queue states' 20 | SLEEP_TIME = 2 21 | pidfile = os.path.join(tempfile.gettempdir(), 'waves_queue.pid') 22 | pidfile_timeout = 5 23 | _class = JobQueueRunDaemon 24 | 25 | def handle(self, **options): 26 | import warnings 27 | warnings.warn("This method is deprecated: please use ./manage.py wqueue start instead") 28 | exit(0) 29 | 30 | 31 | class PurgeDaemonCommand(DaemonCommand): 32 | help = 'Clean up old jobs ' 33 | SLEEP_TIME = 86400 34 | pidfile_path = os.path.join(waves_settings.DATA_ROOT, 'waves_clean.pid') 35 | _class = PurgeDaemon 36 | 37 | def handle(self, **options): 38 | import warnings 39 | warnings.warn("This method is deprecated: please use ./manage.py wpurge start instead") 40 | exit(0) 41 | -------------------------------------------------------------------------------- /waves/wcore/management/commands/waves.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | from __future__ import unicode_literals 3 | 4 | from ..base import SubcommandDispatcher 5 | from ..command import JobQueueCommand, PurgeDaemonCommand 6 | from ..subcommands import CleanUpCommand, ImportCommand, DumpConfigCommand, ShowUrlsCommand 7 | 8 | CLEAN = 'clean' 9 | CONFIG = 'config' 10 | DUMP = 'dump' 11 | LOAD = 'load' 12 | QUEUE = 'queue' 13 | PURGE = 'purge' 14 | SHOWURLS = 'show_urls' 15 | 16 | 17 | class Command(SubcommandDispatcher): 18 | """ WAVES dedicated administration Django subcommand line interface (./manage.py) """ 19 | help = 'WAVES Administration dedicated commands: type manage.py waves --help for sub-commands help' 20 | command_list = (CLEAN, CONFIG, LOAD, SHOWURLS) 21 | 22 | def _subcommand(self, name): 23 | if name == CLEAN: 24 | return CleanUpCommand() 25 | elif name == QUEUE: 26 | return JobQueueCommand() 27 | elif name == LOAD: 28 | return ImportCommand() 29 | elif name == CONFIG: 30 | return DumpConfigCommand() 31 | elif name == PURGE: 32 | return PurgeDaemonCommand() 33 | elif name == SHOWURLS: 34 | return ShowUrlsCommand() 35 | else: 36 | return None 37 | 38 | def _subcommand_names(self): 39 | return self.command_list 40 | -------------------------------------------------------------------------------- /docs/user_doc/user_guide.rst: -------------------------------------------------------------------------------- 1 | Administrator Guide 2 | =================== 3 | 4 | Services are defined by administrators, then configured to allow users to gain access to submission web forms and REST api entry points. 5 | 6 | Everything is made simple with help of dedicated back office entries. 7 | There, administrators firstly describe services in term of name, description, authors, citation links, edams ontology topics and operations, 8 | computing infrastructure configuration (see “Execution adaptor”). 9 | 10 | Once service main data are setup, administrators can configure precisely related submission to setup inputs, execution parameters, outputs, etc… 11 | 12 | Upon service publication, it is automatically available in two ways: standard web pages including html forms, and a REST API 13 | service entry point to be remotely accessed from a distant application. 14 | 15 | Some more configuration can be made to tune user’s access rights, execution parameters etc. Those are further detailed in this document. 16 | 17 | .. figure:: waves-admin.png 18 | :width: 90% 19 | :align: center 20 | :figclass: thumbnail 21 | 22 | Django classic back-office landing page for WAVES-core module 23 | 24 | 25 | .. toctree:: 26 | :maxdepth: 3 27 | :numbered: 28 | 29 | runner/runners 30 | service/services 31 | service/submissions 32 | job/jobs 33 | howto/howto 34 | 35 | 36 | 37 | -------------------------------------------------------------------------------- /waves/authentication/migrations/0001_initial.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Generated by Django 1.11.8 on 2018-03-12 12:44 3 | from __future__ import unicode_literals 4 | 5 | from django.conf import settings 6 | from django.db import migrations, models 7 | import django.db.models.deletion 8 | 9 | 10 | class Migration(migrations.Migration): 11 | 12 | initial = True 13 | 14 | dependencies = [ 15 | migrations.swappable_dependency(settings.AUTH_USER_MODEL), 16 | ] 17 | 18 | operations = [ 19 | migrations.CreateModel( 20 | name='WavesApiUser', 21 | fields=[ 22 | ('key', models.CharField(max_length=40, primary_key=True, serialize=False, verbose_name='Key')), 23 | ('created', models.DateTimeField(auto_now_add=True, verbose_name='Created')), 24 | ('domain', models.CharField(blank=True, max_length=255, null=True, verbose_name='Origin Site')), 25 | ('ip_list', models.CharField(blank=True, max_length=255, null=True, verbose_name='Ip List')), 26 | ('user', models.OneToOneField(on_delete=django.db.models.deletion.CASCADE, related_name='waves_user', to=settings.AUTH_USER_MODEL, verbose_name='User')), 27 | ], 28 | options={ 29 | 'abstract': False, 30 | 'verbose_name': 'Waves Api key', 31 | 'verbose_name_plural': 'Waves Api keys', 32 | }, 33 | ), 34 | ] 35 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/admin/submission/change_form.html: -------------------------------------------------------------------------------- 1 | {% extends "waves/admin/change_form.html" %} 2 | {% load i18n admin_modify admin_urls admin_static waves_tags %} 3 | 4 | {% block object-tools-items %} 5 |
  • 6 | 10 | Add a new param 11 | 12 |
  • 13 |
  • 14 | 19 | Edit Service 20 | 21 |
  • 22 |
  • 23 | 25 | Preview 26 | 27 |
  • 28 |
  • 29 | 34 | Service's list 35 | 36 |
  • 37 | {{ block.super }} 38 | {% endblock %} 39 | -------------------------------------------------------------------------------- /waves/wcore/exceptions/__init__.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | import logging 4 | import sys 5 | 6 | __all__ = ['WavesException', 'RunnerException', 'RunnerNotInitialized', 'RunnerNotReady', 'RunnerConnectionError', 7 | 'RunnerUnexpectedInitParam'] 8 | 9 | if sys.version_info[0] < 3: 10 | __all__ = [n.encode('utf-8') for n in __all__] 11 | 12 | logger = logging.getLogger(__name__) 13 | 14 | 15 | class WavesException(Exception): 16 | """ 17 | Waves base exception class, exception log 18 | """ 19 | def __init__(self, *args, **kwargs): 20 | super(WavesException, self).__init__(*args, **kwargs) 21 | logger.exception('[%s] - %s', self.__class__.__name__, self.message) 22 | 23 | 24 | class RunnerException(WavesException): 25 | """ 26 | Base Exception class for all Runner related errors 27 | """ 28 | 29 | def __init__(self, *args, **kwargs): 30 | super(RunnerException, self).__init__(*args, **kwargs) 31 | 32 | 33 | class RunnerNotInitialized(RunnerException): 34 | pass 35 | 36 | 37 | class RunnerUnexpectedInitParam(RunnerException, KeyError): 38 | pass 39 | 40 | 41 | class RunnerConnectionError(RunnerException): 42 | def __init__(self, reason, msg=''): 43 | message = reason 44 | if msg != '': 45 | message = '%s %s' % (msg, message) 46 | super(RunnerException, self).__init__(message) 47 | 48 | 49 | class RunnerNotReady(RunnerException): 50 | pass 51 | -------------------------------------------------------------------------------- /waves/wcore/templatetags/waves_tags.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from django import template 4 | 5 | from waves.wcore.settings import waves_settings 6 | 7 | register = template.Library() 8 | 9 | 10 | @register.inclusion_tag('waves/services/_online_execution.html', takes_context=True) 11 | def online_exec_button(context, service, label=None): 12 | """ for service, setup if current usr can submit jobs """ 13 | if service.default_submission: 14 | return { 15 | 'available_for_submission': service.default_submission.available_for_user(context['user']) and context.get( 16 | 'preview') is None, 17 | 'label': label, 'service': service} 18 | return {} 19 | 20 | 21 | @register.simple_tag 22 | def get_app_version(): 23 | from waves.wcore import __version_detail__, __version__ 24 | return "%s (%s)" % (__version__, __version_detail__) 25 | 26 | 27 | @register.inclusion_tag('waves/services/forms/service_inc.html') 28 | def service_inc(inc_type): 29 | return {'template': "waves/services/forms/" + waves_settings.TEMPLATE_PACK + "/inc." + inc_type + ".html"} 30 | 31 | 32 | @register.inclusion_tag('waves/services/forms/base_form.html', takes_context=True) 33 | def submission_form(context, template_pack=None): 34 | tpl_pack = template_pack or waves_settings.TEMPLATE_PACK 35 | return {'template_form': "waves/services/forms/" + tpl_pack + "/submission_form.html", 36 | 'submissions': context['submissions']} 37 | -------------------------------------------------------------------------------- /waves/wcore/utils/__init__.py: -------------------------------------------------------------------------------- 1 | """ Base Utils classes """ 2 | from __future__ import unicode_literals 3 | 4 | import logging 5 | import random 6 | import string 7 | 8 | from django.core.urlresolvers import reverse 9 | from django.utils.safestring import mark_safe 10 | 11 | logger = logging.getLogger(__name__) 12 | 13 | 14 | def get_all_subclasses(cls): 15 | all_subclasses = [] 16 | 17 | for subclass in cls.__subclasses__(): 18 | all_subclasses.append(subclass) 19 | all_subclasses.extend(get_all_subclasses(subclass)) 20 | 21 | return all_subclasses 22 | 23 | 24 | def normalize_value(value): 25 | import inflection 26 | import re 27 | value = re.sub(r'[^\w.]+', '_', value) 28 | return inflection.underscore(value) 29 | 30 | 31 | def url_to_edit_object(obj): 32 | """ Retrieve url to access admin change object """ 33 | if obj is not None: 34 | # noinspection PyProtectedMember,PyProtectedMember 35 | url = reverse('admin:%s_%s_change' % (obj._meta.app_label, obj._meta.model_name), args=[obj.id]) 36 | # noinspection PyProtectedMember 37 | return mark_safe('{}'.format(url, obj._meta.model_name, str(obj))) 38 | else: 39 | logger.warn('Trying to view a NoneType object link %s ', obj.__class__.__name__) 40 | return "#" 41 | 42 | 43 | def random_analysis_name(): 44 | return ''.join(random.choice(string.ascii_uppercase + string.digits) for _ in range(15)) 45 | 46 | 47 | -------------------------------------------------------------------------------- /waves/wcore/utils/encrypt.py: -------------------------------------------------------------------------------- 1 | """ 2 | Encryption for Service Adaptors init_params values 3 | """ 4 | from __future__ import unicode_literals 5 | 6 | from waves.wcore.settings import waves_settings 7 | from cryptography.fernet import Fernet 8 | 9 | import base64 10 | 11 | 12 | class Encrypt(object): 13 | """ Encrypt values based on Django settings secret key substring """ 14 | def __init__(self): 15 | raise RuntimeError('This class is intended to be used statically') 16 | 17 | @staticmethod 18 | def encrypt(to_encode): 19 | """ Crypt 'to_encode' 20 | :param to_encode: value to encode 21 | :return: base64 based encoded string 22 | """ 23 | if len(waves_settings.SECRET_KEY) < 32: 24 | raise RuntimeError('Encoded values must use a key at least a 32 chars length secret') 25 | encoder = Fernet(base64.urlsafe_b64encode(waves_settings.SECRET_KEY)) 26 | encoded = encoder.encrypt(bytes(to_encode)) 27 | return encoded 28 | 29 | @staticmethod 30 | def decrypt(to_decode): 31 | """ Decrypt previously encoded 'to_decode' 32 | :param to_decode: value to decode 33 | :return: string initial value 34 | """ 35 | if len(waves_settings.SECRET_KEY) < 32: 36 | raise RuntimeError('Encoded values must use a key at least a 32 chars length secret') 37 | encoder = Fernet(base64.urlsafe_b64encode(waves_settings.SECRET_KEY)) 38 | return encoder.decrypt(bytes(to_decode)) 39 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/forms/bootstrap3/submission_form.html: -------------------------------------------------------------------------------- 1 | {% load crispy_forms_tags %} 2 | {% with single_submission=submissions|length %} 3 | {% if single_submission > 1 %} 4 | 14 |
    15 | {% endif %} 16 | {% for sub in submissions %} 17 | 18 | {% if single_submission > 1 %} 19 |
    21 | {% if preview %} 22 |
    Availability : {{ sub.submission.get_availability_display }}
    23 | {% endif %} 24 |
    25 | {% endif %} 26 | {% crispy sub.form %} 27 | {% if single_submission > 1 %} 28 |
    29 |
    30 | {% endif %} 31 | 32 | {% endfor %} 33 | {% if single_submission > 1 %} 34 |
    35 | {% endif %} 36 | {% endwith %} -------------------------------------------------------------------------------- /waves/wcore/adaptors/loader.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | import json 4 | 5 | from waves.wcore.adaptors.exceptions import AdaptorNotAvailableException 6 | from waves.wcore.settings import import_from_string 7 | 8 | __all__ = ['AdaptorLoader'] 9 | 10 | 11 | class AdaptorLoader(object): 12 | 13 | @classmethod 14 | def get_adaptors(cls): 15 | from waves.wcore.settings import waves_settings 16 | return sorted([adaptor_class() for adaptor_class in waves_settings.ADAPTORS_CLASSES]) 17 | 18 | @classmethod 19 | def load(cls, clazz, **params): 20 | from waves.wcore.settings import waves_settings 21 | 22 | if params is None: 23 | params = {} 24 | loaded = next((x(**params) for x in waves_settings.ADAPTORS_CLASSES if x == clazz), None) 25 | if loaded is None: 26 | raise AdaptorNotAvailableException("This adapter class %s is not available " % clazz) 27 | return loaded 28 | 29 | @classmethod 30 | def serialize(cls, adaptor): 31 | return adaptor.serialize() 32 | 33 | @classmethod 34 | def unserialize(cls, serialized): 35 | json_data = json.loads(serialized) 36 | clazz = import_from_string(json_data['clazz']) 37 | return cls.load(clazz, **json_data['params']) 38 | 39 | @classmethod 40 | def get_class_names(cls): 41 | from waves.wcore.settings import waves_settings 42 | return ['{}.{}'.format(clazz.__module__, clazz.__name__) for clazz in waves_settings.ADAPTORS_CLASSES] 43 | -------------------------------------------------------------------------------- /waves/wcore/api/share.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from rest_framework import serializers 4 | from waves.wcore.models.inputs import AParam 5 | 6 | 7 | class DynamicFieldsModelSerializer(serializers.ModelSerializer): 8 | """ 9 | A ModelSerializer that takes an additional `fields` argument that 10 | controls which fields should be displayed. 11 | Optionally `hidden` parameter allows to hide some of the fields defined in Serializer 12 | 13 | """ 14 | def __init__(self, *args, **kwargs): 15 | # Don't pass the 'fields' arg up to the superclass 16 | fields = kwargs.pop('fields', []) 17 | hidden = kwargs.pop('hidden', []) 18 | # Instantiate the superclass normally 19 | super(DynamicFieldsModelSerializer, self).__init__(*args, **kwargs) 20 | if fields: 21 | # Drop any fields that are not specified in the `fields` argument. 22 | allowed = set(fields) 23 | existing = set(self.fields.keys()) 24 | for field_name in existing - allowed: 25 | self.fields.pop(field_name) 26 | elif hidden: 27 | initial = set(hidden) 28 | for field_name in initial: 29 | self.fields.pop(field_name) 30 | 31 | 32 | class RecursiveField(serializers.ModelSerializer): 33 | class Meta: 34 | model = AParam 35 | 36 | def to_representation(self, value): 37 | serializer = self.parent.parent.__class__(value, context=self.context) 38 | return serializer.data 39 | -------------------------------------------------------------------------------- /waves/authentication/serializers.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from django.contrib.auth import authenticate 4 | from django.utils.translation import ugettext_lazy as _ 5 | 6 | from rest_framework import serializers 7 | 8 | 9 | class WavesApiUserSerializer(serializers.Serializer): 10 | username = serializers.CharField(label=_("Username")) 11 | password = serializers.CharField(label=_("Password"), style={'input_type': 'password'}) 12 | 13 | def validate(self, attrs): 14 | username = attrs.get('username') 15 | password = attrs.get('password') 16 | 17 | if username and password: 18 | user = authenticate(username=username, password=password) 19 | 20 | if user: 21 | # From Django 1.10 onwards the `authenticate` call simply 22 | # returns `None` for is_active=False users. 23 | # (Assuming the default `ModelBackend` authentication backend.) 24 | if not user.is_active: 25 | msg = _('User account is disabled.') 26 | raise serializers.ValidationError(msg, code='authorization') 27 | else: 28 | msg = _('Unable to log in with provided credentials.') 29 | raise serializers.ValidationError(msg, code='authorization') 30 | else: 31 | msg = _('Must include "username" and "password".') 32 | raise serializers.ValidationError(msg, code='authorization') 33 | 34 | attrs['user'] = user 35 | return attrs 36 | -------------------------------------------------------------------------------- /tests/data/sample/sample_tree.nhx: -------------------------------------------------------------------------------- 1 | ((Mus:0.193075,Oryctolagus:0.096112)100:0.009057,((Nycticebus:0.069238,(Lemur:0.022664,(Propithecus:0.018166,Microcebus:0.023409)48:0.001209)99:0.011442)76:0.009757,(Tarsiusxb:0.061307,((Ateles:0.006376,(Pithecia:0.013818,(Cebus:0.021814,Callithrix:0.016131)83:0.002335)59:0)100:0.022494,(Pan:0.019477,(Cercopithecus:0.005244,Macaca:0.005714)100:0.012134)100:0.009606)100:0.018648)77:0.005742)100:0.01); 2 | ((Mus:0.250542,Oryctolagus:0.140507)95:0.034428,(((Daubentonia:0.042629,((Eulemur:0.007627,(Varecia:0.022579,(Lemur:0.001092,Hapalemur:0.007203)96:0.003371)69:0.002033)99:0.011267,(Propithecus:0.017188,(Microcebus:0.030869,Cheirogaleus:0.027551)99:0.019052)98:0.007315)99:0.019169)65:0.006909,(Perodicticus:0.015895,((Galagoides:0.012311,(Galago:0.011859,Otolemur:0.013514)100:0.007053)96:0.008849,(Loris:0.019299,Nycticebus:0.020505)97:0.010544)67:0.003573)100:0.044128)90:0.013272,(((Hylobates:0.014099,Homo:0.009416)100:0.02346,(Pithecia:0.01434,(Callithrix:0.019474,(Ateles:0.014687,Cebus:0.013995)72:0.006804)45:0.00297)100:0.03704)100:0.074386,(Tarsiusxs:0.01503,Tarsiusxb:0.007082)100:0.105032)90:0.018066)100:0.01); 3 | ((((Galagoides:0,Galago:1.000,Otolemur:0)100:11.000,((Perodicticus:0,Arctocebus:3.000)94:3.000,(Loris:1.000,Nycticebus:2.000)100:6.000)95:8.000)100:5,(Daubentonia:0,(Lepilemur:0,((Varecia:0,(Eulemur:0,(Lemur:0,Hapalemur:0)94:3.000)66:1.000)98:4.000,(Propithecus:0,Avahi:0)99:5.000)63:1.000,(Phaner:0,(Cheirogaleus:0,(Allocebus:0,(Microcebus:1.000,Mirza:0)88:2.000)87:2.000)88:2.000)86:2.000)64:1.000)94:3.000)100:0.01,Homo:4.000); 4 | -------------------------------------------------------------------------------- /tests/data/sample/physic_ist/physic_ist_ex.txt: -------------------------------------------------------------------------------- 1 | ((Mus:0.193075,Oryctolagus:0.096112)100:0.009057,((Nycticebus:0.069238,(Lemur:0.022664,(Propithecus:0.018166,Microcebus:0.023409)48:0.001209)99:0.011442)76:0.009757,(Tarsiusxb:0.061307,((Ateles:0.006376,(Pithecia:0.013818,(Cebus:0.021814,Callithrix:0.016131)83:0.002335)59:0)100:0.022494,(Pan:0.019477,(Cercopithecus:0.005244,Macaca:0.005714)100:0.012134)100:0.009606)100:0.018648)77:0.005742)100:0.01); 2 | ((Mus:0.250542,Oryctolagus:0.140507)95:0.034428,(((Daubentonia:0.042629,((Eulemur:0.007627,(Varecia:0.022579,(Lemur:0.001092,Hapalemur:0.007203)96:0.003371)69:0.002033)99:0.011267,(Propithecus:0.017188,(Microcebus:0.030869,Cheirogaleus:0.027551)99:0.019052)98:0.007315)99:0.019169)65:0.006909,(Perodicticus:0.015895,((Galagoides:0.012311,(Galago:0.011859,Otolemur:0.013514)100:0.007053)96:0.008849,(Loris:0.019299,Nycticebus:0.020505)97:0.010544)67:0.003573)100:0.044128)90:0.013272,(((Hylobates:0.014099,Homo:0.009416)100:0.02346,(Pithecia:0.01434,(Callithrix:0.019474,(Ateles:0.014687,Cebus:0.013995)72:0.006804)45:0.00297)100:0.03704)100:0.074386,(Tarsiusxs:0.01503,Tarsiusxb:0.007082)100:0.105032)90:0.018066)100:0.01); 3 | ((((Galagoides:0,Galago:1.000,Otolemur:0)100:11.000,((Perodicticus:0,Arctocebus:3.000)94:3.000,(Loris:1.000,Nycticebus:2.000)100:6.000)95:8.000)100:5,(Daubentonia:0,(Lepilemur:0,((Varecia:0,(Eulemur:0,(Lemur:0,Hapalemur:0)94:3.000)66:1.000)98:4.000,(Propithecus:0,Avahi:0)99:5.000)63:1.000,(Phaner:0,(Cheirogaleus:0,(Allocebus:0,(Microcebus:1.000,Mirza:0)88:2.000)87:2.000)88:2.000)86:2.000)64:1.000)94:3.000)100:0.01,Homo:4.000); 4 | -------------------------------------------------------------------------------- /waves/wcore/admin/views/export.py: -------------------------------------------------------------------------------- 1 | """ Base class for exporting objects """ 2 | from __future__ import unicode_literals 3 | 4 | from os.path import join 5 | 6 | from django.contrib import messages 7 | from django.shortcuts import redirect 8 | from waves.wcore.settings import waves_settings as config 9 | from waves.wcore.models.base import ExportAbleMixin 10 | from waves.wcore.views.files import DownloadFileView 11 | 12 | 13 | class ModelExportView(DownloadFileView): 14 | """ Enable simple model export with DRF subclasses must declare property method to set up 15 | serializer used for process 16 | """ 17 | model = None 18 | _force_download = True 19 | serializer = None 20 | return_view = "admin:index" 21 | 22 | def get_context_data(self, **kwargs): 23 | context = super(ModelExportView, self).get_context_data(**kwargs) 24 | return context 25 | 26 | def get(self, request, *args, **kwargs): 27 | from waves.wcore.models.base import ExportError 28 | try: 29 | return super(ModelExportView, self).get(request, *args, **kwargs) 30 | except ExportError as e: 31 | messages.error(self.request, 'Oops: %s' % e) 32 | return redirect(self.return_view) 33 | 34 | @property 35 | def file_path(self): 36 | return join(config.DATA_ROOT, 'export', self.file_name) 37 | 38 | @property 39 | def file_name(self): 40 | return self.object.export_file_name 41 | 42 | @property 43 | def file_description(self): 44 | return "Export file for %s " % self.model.__class__.__name__ 45 | 46 | -------------------------------------------------------------------------------- /waves/wcore/import_export/runners.py: -------------------------------------------------------------------------------- 1 | """WAVES models export module for Services """ 2 | from __future__ import unicode_literals 3 | 4 | from django.db import transaction 5 | from rest_framework import serializers 6 | 7 | from waves.wcore.models import Runner, AdaptorInitParam 8 | from waves.wcore.import_export import RelatedSerializerMixin 9 | 10 | 11 | class RunnerParamSerializer(serializers.ModelSerializer): 12 | class Meta: 13 | model = AdaptorInitParam 14 | fields = ('name', 'value', 'prevent_override') 15 | 16 | def to_representation(self, instance): 17 | repre = super(RunnerParamSerializer, self).to_representation(instance) 18 | if instance.name == 'password' or instance.crypt: 19 | repre['value'] = "xxxxxx" 20 | return repre 21 | 22 | 23 | class RunnerSerializer(serializers.ModelSerializer, RelatedSerializerMixin): 24 | class Meta: 25 | model = Runner 26 | fields = ('name', 'clazz', 'runner_params') 27 | 28 | runner_params = RunnerParamSerializer(many=True, source='adaptor_params') 29 | 30 | @transaction.atomic 31 | def create(self, validated_data): 32 | runner_params = validated_data.pop('runner_params') 33 | runner = Runner.objects.create(**validated_data) 34 | runner.runner_run_params.all().delete() 35 | runner.runner_run_params = self.create_related(foreign={'runner': runner}, 36 | serializer=RunnerParamSerializer, 37 | datas=runner_params) 38 | return runner 39 | -------------------------------------------------------------------------------- /docs/templates/footer.html: -------------------------------------------------------------------------------- 1 | {% extends "!footer.html" %} 2 | 3 | 4 | {% block extrafooter %} 5 | 6 | 38 | 39 | {% endblock %} -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Created by .ignore support plugin (hsz.mobi) 2 | ### Python template 3 | # Byte-compiled / optimized / DLL files 4 | .idea 5 | # Distribution / packaging 6 | dist/ 7 | *.egg-info/* 8 | *.egg 9 | 10 | # PyInstaller 11 | # Usually these files are written by a python script from a template 12 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 13 | *.manifest 14 | *.spec 15 | 16 | # Installer logs 17 | pip-log.txt 18 | pip-delete-this-directory.txt 19 | *.pyc 20 | # Unit test / coverage reports 21 | htmlcov/ 22 | .tox/ 23 | .coverage 24 | .coverage.* 25 | .cache 26 | nosetests.xml 27 | coverage.xml 28 | *,cover 29 | .hypothesis/ 30 | 31 | # Translations 32 | *.mo 33 | *.pot 34 | 35 | # Django stuff: 36 | *.log 37 | local_settings.py 38 | 39 | # Flask stuff: 40 | instance/ 41 | .webassets-cache 42 | 43 | # Scrapy stuff: 44 | .scrapy 45 | 46 | # Sphinx documentation 47 | docs/_build/ 48 | 49 | # Jupyter Notebook 50 | .ipynb_checkpoints 51 | 52 | # vscode ide 53 | .vscode 54 | 55 | # pyenv 56 | .python-version 57 | 58 | # celery beat schedule file 59 | celerybeat-schedule 60 | 61 | # dotenv 62 | .env 63 | 64 | # virtualenv 65 | .venv 66 | venv/ 67 | 68 | # default dirs 69 | !/data/logs/.gitkeep 70 | !/data/.gitkeep 71 | /data/bin/* 72 | !/data/bin/.gitkeep 73 | /data/jobs/* 74 | !/data/jobs/.gitkeep 75 | /data/media/* 76 | !/data/media/.gitkeep 77 | /data/sample/* 78 | !/data/sample/.gitkeep 79 | # tests dirs 80 | /tests/settings.ini 81 | /tests/data/jobs/*/* 82 | !/tests/data/jobs/.gitkeep 83 | !/tests/settings.ini.sample 84 | *.sqlite3 85 | /staticfiles/ 86 | /tests/data/*.json 87 | /data/tests/*.json 88 | local.env 89 | *.pid 90 | -------------------------------------------------------------------------------- /waves/wcore/compat/__init__.py: -------------------------------------------------------------------------------- 1 | """ 2 | Compatibility file to enable dependents apps behaviour 3 | """ 4 | from django.conf import settings 5 | 6 | __all__ = ["available_themes", "list_themes", "RichTextField", "CompactInline", "SortableInlineAdminMixin"] 7 | 8 | 9 | if 'jet' in settings.INSTALLED_APPS: 10 | _temp = __import__('jet.admin', globals(), locals(), ['CompactInline'], -1) 11 | CompactInline = _temp.CompactInline 12 | else: 13 | from django.contrib.admin import StackedInline 14 | 15 | class CompactInline(StackedInline): 16 | """ Inherit base class """ 17 | pass 18 | 19 | if 'ckeditor' in settings.INSTALLED_APPS: 20 | _temp = __import__('ckeditor.fields', globals(), locals(), ['RichTextField'], -1) 21 | RichTextField = _temp.RichTextField 22 | else: 23 | from django.db.models import TextField 24 | 25 | class RichTextField(TextField): 26 | """ Override RichTextField """ 27 | pass 28 | 29 | if 'bootstrap_themes' in settings.INSTALLED_APPS: 30 | _temp = __import__('bootstrap_themes', globals(), locals(), ['list_themes', 'available_themes'], -1) 31 | list_themes = _temp.list_themes 32 | available_themes = _temp.available_themes 33 | else: 34 | available_themes = ( 35 | ('default', 'Default'), 36 | ) 37 | 38 | def list_themes(): 39 | return available_themes 40 | 41 | if 'adminsortable2' in settings.INSTALLED_APPS: 42 | _temp = __import__('adminsortable2.admin', globals(), locals(), ['SortableInlineAdminMixin'], -1) 43 | SortableInlineAdminMixin = _temp.SortableInlineAdminMixin 44 | else: 45 | class SortableInlineAdminMixin(object): 46 | pass 47 | -------------------------------------------------------------------------------- /waves/wcore/commands/command.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from waves.wcore.models.const import OptType 4 | 5 | 6 | def command_line_element(elem): 7 | cmd_format = elem.cmd_format 8 | name = elem.name 9 | cmd_value = elem.value if elem.required is not None else elem.default 10 | if cmd_value == 'None': 11 | return '' 12 | if cmd_format == OptType.OPT_TYPE_VALUATED: 13 | return '--%s=%s' % (name, cmd_value) 14 | elif cmd_format == OptType.OPT_TYPE_SIMPLE: 15 | return '-%s %s' % (name, cmd_value) 16 | elif cmd_format == OptType.OPT_TYPE_OPTION: 17 | return '-%s' % name 18 | elif cmd_format == OptType.OPT_TYPE_NAMED_OPTION: 19 | return '--%s' % name 20 | elif cmd_format == OptType.OPT_TYPE_POSIX: 21 | return '%s' % cmd_value 22 | elif cmd_format == OptType.OPT_TYPE_NAMED_PARAM: 23 | return '%s=%s' % (name, cmd_value) 24 | elif cmd_format == OptType.OPT_TYPE_NONE: 25 | return '' 26 | # By default it's OPT_TYPE_SIMPLE way 27 | return '-%s %s' % (name, cmd_value) 28 | 29 | 30 | class BaseCommand(object): 31 | 32 | def create_command_line(self, inputs): 33 | """ 34 | Parse and create command line text to launch service 35 | Args: 36 | inputs: JobInput objects list 37 | 38 | Returns: 39 | str the command line text 40 | """ 41 | return ' '.join(self.get_command_line_element_list(inputs)) 42 | 43 | @staticmethod 44 | def get_command_line_element_list(inputs): 45 | if len(inputs) > 0: 46 | return [command_line_element(e) for e in inputs] 47 | else: 48 | return [] 49 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/forms/materialize/submission_form.html: -------------------------------------------------------------------------------- 1 | {% load crispy_forms_tags %} 2 |
    3 |
    4 | 5 | {% with single_submission=submissions.count %} 6 | {% if single_submission > 1 %} 7 | 17 |
    18 | {% endif %} 19 | {% for sub in submissions %} 20 | 21 |
    23 | {% if preview %} 24 |
    Availability : {{ sub.submission.get_availability_display }}
    25 | {% endif %} 26 |
    27 | {% crispy sub.form %} 28 |
    29 |
    30 | 31 | {% endfor %} 32 | {% if single_submission > 1 %} 33 |
    34 | {% endif %} 35 | {% endwith %} 36 |
    37 |
    -------------------------------------------------------------------------------- /waves_core/cli.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from waves_core.settings import * 4 | 5 | CLI_LOG_LEVEL = 'WARNING' 6 | 7 | LOGGING = { 8 | 'version': 1, 9 | 'disable_existing_loggers': False, 10 | 'formatters': { 11 | 'verbose': { 12 | 'format': '[%(levelname)s][%(asctime)s][%(name)s.%(funcName)s:%(lineno)s] - %(message)s', 13 | 'datefmt': "%Y-%m-%d %H:%M:%S" 14 | }, 15 | }, 16 | 'handlers': { 17 | 'console': { 18 | 'class': 'logging.StreamHandler', 19 | 'formatter': 'verbose' 20 | }, 21 | 'log_file': { 22 | 'class': 'logging.handlers.RotatingFileHandler', 23 | 'filename': os.path.join(LOG_DIR, 'waves-cli.log'), 24 | 'formatter': 'verbose', 25 | 'backupCount': 10, 26 | 'maxBytes': 1024*1024*5 27 | }, 28 | }, 29 | 30 | 'loggers': { 31 | 'django': { 32 | 'handlers': ['console'], 33 | 'propagate': True, 34 | 'level': 'WARNING', 35 | }, 36 | 'waves': { 37 | 'handlers': ['log_file'], 38 | 'level': CLI_LOG_LEVEL, 39 | 'propagate': True, 40 | }, 41 | 'django_crontab': { 42 | 'handlers': ['log_file'], 43 | 'propagate': True, 44 | 'level': CLI_LOG_LEVEL, 45 | }, 46 | 'waves.daemon': { 47 | 'handlers': ['log_file'], 48 | 'propagate': False, 49 | 'level': 'INFO', 50 | }, 51 | 'daemons': { 52 | 'handlers': ['console'], 53 | 'propagate': False, 54 | 'level': 'INFO', 55 | }, 56 | 57 | } 58 | } 59 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/admin/runner/change_form.html: -------------------------------------------------------------------------------- 1 | {% extends "waves/admin/change_form.html" %} 2 | {% load i18n admin_modify admin_urls admin_static waves_tags %} 3 | {% block content %} 4 | {% if is_popup %} 5 | 11 | {% endif %} 12 | {{ block.super }} 13 | {% endblock %} 14 | {% block object-tools-items %} 15 | {{ block.super }} 16 | 24 | {% if original.clazz %} 25 |
  • 26 | 30 | Test Connection 31 | 32 |
  • 33 | {% endif %} 34 | {% if original.adaptor.importer %} 35 |
  • 36 | 43 | Import a Service 44 | 45 |
  • 46 | {% endif %} 47 | {% endblock %} 48 | -------------------------------------------------------------------------------- /waves/wcore/static/waves/admin/js/services.js: -------------------------------------------------------------------------------- 1 | /** 2 | * Created by marc on 25/11/15. 3 | * Functions library for atgc service platform back-office 4 | */ 5 | 6 | (function ($) { 7 | $(document).ready(function () { 8 | var $runner_tag = $("#id_runner"); 9 | var prev_val = $runner_tag.val(); 10 | $runner_tag.select(function () { 11 | console.log('focus !'); 12 | prev_val = $(this).val(); 13 | console.log('Prev val' + prev_val); 14 | }).change(function () { 15 | console.log('Changed triggered'); 16 | if (prev_val) { 17 | if (confirm('Changing this value might cancel running jobs.\n\nAre you sure ?')) { 18 | $("input[type='submit'][name='_continue']").trigger('click'); 19 | } else { 20 | $(this).val(prev_val); 21 | // for Django jet widget, reset text label 22 | $('#select2-id_runner-container').text($("#id_runner option:selected").text()); 23 | } 24 | }/* else { 25 | console.log('changed ?'); 26 | $("input[type='submit'][name='_continue']").trigger('click'); 27 | }*/ 28 | }); 29 | 30 | $('#open_import_form').click(function (e) { 31 | e.preventDefault(); 32 | $('#popup_modal_content').load($(this).attr('href'), function () { 33 | $('#popup_modal').modal({backdrop: 'static', keyboard: false, show:true}); 34 | }); 35 | }); 36 | $('input[id^="id_service_outputs"][id$="from_input"]').each(function () { 37 | console.log(this.id); 38 | }); 39 | }) 40 | })(jQuery || django.jQuery); 41 | 42 | 43 | -------------------------------------------------------------------------------- /waves_core/crontab.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | 4 | import warnings 5 | 6 | from waves_core.settings import * 7 | 8 | try: 9 | from django_crontab import * 10 | except ImportError: 11 | warnings.warn("Please install django_crontab package") 12 | exit(1) 13 | 14 | CLI_LOG_LEVEL = 'WARNING' 15 | 16 | LOGGING = { 17 | 'version': 1, 18 | 'disable_existing_loggers': False, 19 | 'formatters': { 20 | 'verbose': { 21 | 'format': '[%(levelname)s][%(asctime)s][%(name)s.%(funcName)s:%(lineno)s] - %(message)s', 22 | 'datefmt': "%Y-%m-%d %H:%M:%S" 23 | }, 24 | }, 25 | 'handlers': { 26 | 'console': { 27 | 'class': 'logging.StreamHandler', 28 | 'formatter': 'verbose' 29 | }, 30 | 'log_file': { 31 | 'class': 'logging.handlers.RotatingFileHandler', 32 | 'filename': os.path.join(LOG_DIR, 'waves-cron.log'), 33 | 'formatter': 'verbose', 34 | 'backupCount': 10, 35 | 'maxBytes': 1024 * 1024 * 5 36 | }, 37 | }, 38 | 39 | 'loggers': { 40 | 'django': { 41 | 'handlers': ['console'], 42 | 'propagate': True, 43 | 'level': 'WARNING', 44 | }, 45 | 'waves': { 46 | 'handlers': ['log_file'], 47 | 'level': 'WARNING', 48 | 'propagate': True, 49 | }, 50 | 'django_crontab': { 51 | 'handlers': ['log_file'], 52 | 'propagate': True, 53 | 'level': 'WARNING', 54 | }, 55 | 'waves.cron': { 56 | 'handlers': ['log_file'], 57 | 'propagate': False, 58 | 'level': 'WARNING', 59 | } 60 | 61 | } 62 | } -------------------------------------------------------------------------------- /docs/user_doc/job/jobs.rst: -------------------------------------------------------------------------------- 1 | Job queue management 2 | ==================== 3 | 4 | 5 | Job List 6 | -------- 7 | 8 | Jobs are listed in first screen when you click on **jobs** link in main backoffice view. 9 | 10 | .. figure:: backoffice/job-list.png 11 | :align: center 12 | :width: 90% 13 | :figclass: thumbnail 14 | 15 | Job Details 16 | ----------- 17 | On this view you may want to see job online, see associated service, or cancel the job (if possible). 18 | 19 | 20 | General Information 21 | You can check general information about job here, such as : 22 | 23 | - Title 24 | - Associated Service 25 | - Current Status 26 | - Creation and last update date 27 | - Associated client if job has been submitted by a registered user 28 | - Email where notifications are sent 29 | - Generated unique slug for this Job 30 | - Current runner where job is actually run 31 | - Generated command line where applicable for runner 32 | 33 | .. figure:: backoffice/job-general.png 34 | :align: center 35 | :width: 90% 36 | :figclass: thumbnail 37 | 38 | Job History 39 | Retrieve here all logged events for this job, including administration message (may describe errors). 40 | 41 | .. figure:: backoffice/job-history.png 42 | :align: center 43 | :width: 90% 44 | :figclass: thumbnail 45 | 46 | Job Inputs 47 | Designated inputs for this job 48 | 49 | .. figure:: backoffice/job-inputs.png 50 | :align: center 51 | :width: 90% 52 | :figclass: thumbnail 53 | 54 | Job Outputs 55 | Current expected outputs 56 | 57 | .. figure:: backoffice/job-outputs.png 58 | :align: center 59 | :width: 90% 60 | :figclass: thumbnail 61 | -------------------------------------------------------------------------------- /waves/wcore/admin/views/json_view.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ WAVES Mixin for ajax based classs views""" 3 | from __future__ import unicode_literals 4 | 5 | from django.http import JsonResponse 6 | from django.views.generic import TemplateView 7 | from django.views.generic.detail import BaseDetailView 8 | 9 | 10 | class JSONResponseMixin(object): 11 | """ 12 | A mixin that can be used to render a JSON response. 13 | """ 14 | 15 | def render_to_json_response(self, context, **response_kwargs): 16 | """ 17 | Returns a JSON response, transforming 'context' to make the payload. 18 | """ 19 | return JsonResponse( 20 | self.get_data(context), 21 | **response_kwargs 22 | ) 23 | 24 | def get_data(self, context): 25 | """ 26 | Returns an object that will be serialized as JSON by json.dumps(). 27 | """ 28 | # Note: This is *EXTREMELY* naive; in reality, you'll need 29 | # to do much more complex handling to ensure that arbitrary 30 | # objects -- such as Django model instances or querysets 31 | # -- can be serialized as JSON. 32 | return context 33 | 34 | 35 | class JSONView(JSONResponseMixin, TemplateView): 36 | def render_to_response(self, context, **response_kwargs): 37 | return self.render_to_json_response(context, **response_kwargs) 38 | 39 | 40 | class JSONDetailView(JSONResponseMixin, BaseDetailView): 41 | def render_to_json_response(self, context, **response_kwargs): 42 | return super(JSONDetailView, self).render_to_json_response(context, **response_kwargs) 43 | 44 | def render_to_response(self, context, **response_kwargs): 45 | return self.render_to_json_response(context, **response_kwargs) 46 | -------------------------------------------------------------------------------- /.gitlab-ci.yml: -------------------------------------------------------------------------------- 1 | # This file is a template, and might need editing before it works on your project. 2 | # Official framework image. Look for the different tagged releases at: 3 | # https://hub.docker.com/r/library/python 4 | image: python:2.7 5 | 6 | # Pick zero or more services to be used on all builds. 7 | # Only needed when using a docker container to run your tests in. 8 | # Check out: http://docs.gitlab.com/ce/ci/docker/using_docker_images.html#what-is-a-service 9 | # services: 10 | 11 | # This folder is cached between builds 12 | # http://docs.gitlab.com/ce/ci/yaml/README.html#cache 13 | cache: 14 | paths: 15 | - ~/.cache/pip/ 16 | 17 | # This is a basic example for a gem or script which doesn't use 18 | # services such as redis or postgres 19 | before_script: 20 | - python -V # Print out python version for debugging 21 | # Uncomment next line if your Django app needs a JS runtime: 22 | # - apt-get update -q && apt-get install nodejs -yqq 23 | - pip install -r requirements.txt 24 | - pip install coverage==4.5.1 25 | 26 | # To get Django tests to work you may need to create a settings file using 27 | # the following DATABASES: 28 | # 29 | # DATABASES = { 30 | # 'default': { 31 | # 'ENGINE': 'django.db.backends.postgresql_psycopg2', 32 | # 'NAME': 'ci', 33 | # 'USER': 'postgres', 34 | # 'PASSWORD': 'postgres', 35 | # 'HOST': 'postgres', 36 | # 'PORT': '5432', 37 | # }, 38 | # } 39 | # 40 | # and then adding `--settings app.settings.ci` (or similar) to the test command 41 | 42 | test: 43 | variables: 44 | WAVES_SSH_TEST_SGE_CELL: $WAVES_SSH_TEST_SGE_CELL 45 | WAVES_TEST_SGE_BASE_DIR: $WAVES_TEST_SGE_BASE_DIR 46 | script: 47 | - python manage.py test 48 | - coverage run --source='.' manage.py test waves -------------------------------------------------------------------------------- /waves/wcore/management/utils.py: -------------------------------------------------------------------------------- 1 | import sys 2 | 3 | 4 | def boolean_input(question, default=None): 5 | """ 6 | Ask for a boolean response from user 7 | :param question: Question to ask 8 | :param default: Default answer 9 | :return: True or False 10 | :rtype: bool 11 | """ 12 | result = input("%s: " % question) 13 | if not result and default is not None: 14 | return default 15 | while len(result) < 1 or result[0].lower() not in "yn": 16 | result = input("Please answer yes(y) or no(n): ") 17 | return result[0].lower() == "y" 18 | 19 | 20 | def choice_input(question, choices, default=None): 21 | """ 22 | Ask user for choice in a list, indexed by integer response 23 | 24 | :param default: 25 | :param question: The question to ask 26 | :param choices: List of possible choices 27 | :return: Selected choice by user 28 | :rtype: int 29 | """ 30 | print("%s:" % question) 31 | for i, choice in enumerate(choices): 32 | print("-%s) %s" % (i + 1, choice)) 33 | result = input("Select an option: ") 34 | try: 35 | value = int(result) 36 | if 0 < value <= len(choices): 37 | return value 38 | except ValueError: 39 | if default: 40 | return default 41 | else: 42 | return choice_input('Please select a valid value', choices, default) 43 | 44 | 45 | def text_input(question, default=None): 46 | result = input("%s (type Enter to keep default): " % question) 47 | if not result and default is not None: 48 | return default 49 | return str(result) 50 | 51 | 52 | def action_cancelled(out): 53 | """ 54 | Simply cancel current action, output confirmation 55 | """ 56 | out.write('Action cancelled.') 57 | sys.exit(3) 58 | -------------------------------------------------------------------------------- /waves/authentication/models.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | import binascii 4 | import os 5 | 6 | from django.conf import settings 7 | from django.db import models 8 | from django.utils.encoding import python_2_unicode_compatible 9 | from django.utils.translation import ugettext_lazy as _ 10 | 11 | from waves.wcore.settings import waves_settings 12 | 13 | 14 | @python_2_unicode_compatible 15 | class WavesApiUser(models.Model): 16 | """ 17 | The default authorization token model. 18 | """ 19 | key = models.CharField(_("Key"), max_length=40, primary_key=True) 20 | user = models.OneToOneField( 21 | settings.AUTH_USER_MODEL, related_name='waves_user', 22 | on_delete=models.CASCADE, verbose_name=_("User") 23 | ) 24 | created = models.DateTimeField(_("Created"), auto_now_add=True) 25 | domain = models.CharField(_('Origin URL(s)'), null=True, blank=True, max_length=255, 26 | help_text="Comma separated list") 27 | ip_list = models.CharField(_('Ip(s) List'), null=True, blank=True, max_length=255, help_text="Comma separated list") 28 | 29 | class Meta: 30 | abstract = 'waves.authentication' not in settings.INSTALLED_APPS 31 | verbose_name = _("Waves Api auth") 32 | verbose_name_plural = _("Waves Api auths") 33 | 34 | def save(self, *args, **kwargs): 35 | if not self.key: 36 | self.key = self.generate_key() 37 | return super(WavesApiUser, self).save(*args, **kwargs) 38 | 39 | def generate_key(self): 40 | return binascii.hexlify(os.urandom(20)).decode() 41 | 42 | def __str__(self): 43 | return self.key 44 | 45 | @property 46 | def main_domain(self): 47 | return self.domain.split(",")[0] if self.domain else waves_settings.HOST 48 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | import os 2 | from setuptools import find_packages, setup 3 | 4 | with open(os.path.join(os.path.dirname(__file__), 'README.md')) as readme: 5 | README = readme.read() 6 | 7 | 8 | def import_version(): 9 | from waves.wcore import __version_detail__ 10 | return __version_detail__ 11 | 12 | 13 | def import_requirements(): 14 | with open(os.path.join(os.path.dirname(__file__), 'requirements.txt')) as f: 15 | content = f.readlines() 16 | # you may also want to remove whitespace characters like `\n` at the end of each line 17 | content = [x.strip() for x in content] 18 | return content 19 | 20 | # allow setup.py to be run from any path 21 | os.chdir(os.path.normpath(os.path.join(os.path.abspath(__file__), os.pardir))) 22 | 23 | setup( 24 | name='waves-core', 25 | version=import_version(), 26 | packages=find_packages(), 27 | provides=['waves'], 28 | include_package_data=True, 29 | license='GPLv3', 30 | description='WAVES - core package', 31 | url='http://waves.atgc-montpellier.fr', 32 | author='Marc Chakiachvili', 33 | author_email='marc.chakiachvili@gmail.com', 34 | install_requires=import_requirements(), 35 | classifiers=[ 36 | 'Environment :: Web Environment', 37 | 'Framework :: Django', 38 | 'Development Status :: 5 - Production/Stable', 39 | 'Intended Audience :: Developers', 40 | 'License :: OSI Approved :: GNU General Public License v3 (GPLv3)', 41 | 'Programming Language :: Python :: 2.7', 42 | 'Topic :: Utilities', 43 | 'Topic :: System :: Distributed Computing', 44 | 'Topic :: Internet :: WWW/HTTP :: WSGI :: Application', 45 | 'Topic :: Scientific/Engineering :: Bio-Informatics', 46 | 'Operating System :: Unix' 47 | ], 48 | ) 49 | -------------------------------------------------------------------------------- /waves/wcore/cron/purge_jobs.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | .. See the NOTICE file distributed with this work for additional information 4 | regarding copyright ownership. 5 | Licensed under the Apache License, Version 2.0 (the "License"); 6 | you may not use this file except in compliance with the License. 7 | You may obtain a copy of the License at 8 | http://www.apache.org/licenses/LICENSE-2.0 9 | Unless required by applicable law or agreed to in writing, software 10 | distributed under the License is distributed on an "AS IS" BASIS, 11 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | See the License for the specific language governing permissions and 13 | limitations under the License. 14 | """ 15 | from __future__ import unicode_literals 16 | 17 | import datetime 18 | import logging 19 | from itertools import chain 20 | 21 | from waves.wcore.models import Job 22 | 23 | logger = logging.getLogger('waves.cron') 24 | 25 | 26 | def purge_old_jobs(): 27 | from waves.wcore.settings import waves_settings 28 | 29 | logger.info("Purge job launched at: %s", datetime.datetime.now().strftime('%A, %d %B %Y %H:%M:%I')) 30 | date_anonymous = datetime.date.today() - datetime.timedelta(waves_settings.KEEP_ANONYMOUS_JOBS) 31 | date_registered = datetime.date.today() - datetime.timedelta(waves_settings.KEEP_REGISTERED_JOBS) 32 | anonymous = Job.objects.filter(client__isnull=True, updated__lt=date_anonymous) 33 | registered = Job.objects.filter(client__isnull=False, updated__lt=date_registered) 34 | for job in list(chain(*[anonymous, registered])): 35 | logger.info('Deleting job %s created on %s', job.slug, job.created) 36 | job.delete() 37 | logger.info("Purge job terminated at: %s", datetime.datetime.now().strftime('%A, %d %B %Y %H:%M:%I')) 38 | -------------------------------------------------------------------------------- /waves/wcore/admin/forms/jobs.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | from django import forms 3 | from django.forms import widgets 4 | from waves.wcore.models import JobInput, JobOutput, Job 5 | 6 | __all__ = ['JobInputForm', 'JobOutputForm', 'JobForm'] 7 | 8 | 9 | class ReadOnlyForm(forms.ModelForm): 10 | """Base class for making a form readonly.""" 11 | 12 | def __init__(self, *args, **kwargs): 13 | super(ReadOnlyForm, self).__init__(*args, **kwargs) 14 | for f in self.fields: 15 | if isinstance(self.fields[f].widget, widgets.Select): 16 | self.fields[f].widget.attrs['disabled'] = 'disabled' 17 | else: 18 | self.fields[f].widget.attrs['readonly'] = 'readonly' 19 | 20 | 21 | class JobInputForm(forms.ModelForm): 22 | class Meta: 23 | model = JobInput 24 | fields = ['value'] 25 | widgets = { 26 | 'input': widgets.Select(attrs={'readonly': True}), 27 | 'value': forms.Textarea(attrs={'rows': 2, 'class': 'span12'}) 28 | } 29 | 30 | def clean(self): 31 | cleaned_data = super(JobInputForm, self).clean() 32 | return cleaned_data 33 | 34 | 35 | class JobOutputForm(ReadOnlyForm): 36 | class Meta: 37 | model = JobOutput 38 | fields = ['value'] 39 | widgets = { 40 | 'value': forms.Textarea(attrs={'rows': 2, 'class': 'span12'}) 41 | } 42 | 43 | def get_file_path(self, obj): 44 | return obj.file_path 45 | 46 | 47 | class JobForm(forms.ModelForm): 48 | class Meta: 49 | model = Job 50 | fields = ['title', 'submission', '_status', 'client', 'email_to'] 51 | widgets = { 52 | 'service': widgets.Select(attrs={'disabled': 'disabled'}), 53 | 'client': widgets.Select(attrs={'disabled': 'disabled'}), 54 | } 55 | -------------------------------------------------------------------------------- /waves/wcore/admin/forms/runners.py: -------------------------------------------------------------------------------- 1 | """ 2 | Runner configuration BackOffice Forms 3 | """ 4 | from __future__ import unicode_literals 5 | 6 | from django.forms import ModelForm, BooleanField, ChoiceField, HiddenInput 7 | from waves.wcore.models import Runner 8 | 9 | __all__ = ['RunnerForm'] 10 | 11 | 12 | def get_runners_list(): 13 | """ 14 | Retrieve enabled waves.wcore.adapters list from waves settings env file 15 | :return: a list of Tuple 'value'/'label' 16 | """ 17 | from waves.wcore.adaptors.loader import AdaptorLoader 18 | adaptors = AdaptorLoader.get_adaptors() 19 | grp_impls = {'': 'Select a environment...'} 20 | for adaptor in adaptors: 21 | grp_name = adaptor.__class__.__module__.split('.')[-1].capitalize() 22 | if grp_name not in grp_impls: 23 | grp_impls[grp_name] = [] 24 | grp_impls[grp_name].append((adaptor.__module__ + '.' + adaptor.__class__.__name__, adaptor.__class__.name)) 25 | return sorted((grp_key, grp_val) for grp_key, grp_val in grp_impls.items()) 26 | 27 | 28 | class RunnerForm(ModelForm): 29 | """ Form to edit a runner 30 | """ 31 | class Meta: 32 | """ Metas """ 33 | model = Runner 34 | exclude = ['id'] 35 | 36 | # noinspection PyClassHasNoInit 37 | class Media: 38 | """ Medias """ 39 | js = ('waves/admin/js/runner.js', 40 | 'waves/admin/js/connect.js') 41 | 42 | update_init_params = BooleanField(required=False, label='Reset related services') 43 | 44 | def __init__(self, *args, **kwargs): 45 | super(RunnerForm, self).__init__(*args, **kwargs) 46 | self.fields['clazz'] = ChoiceField(label="Run on", choices=get_runners_list) 47 | if self.instance.pk is None: 48 | self.fields['update_init_params'].widget = HiddenInput() 49 | self.fields['update_init_params'].initial = False 50 | -------------------------------------------------------------------------------- /waves/wcore/adaptors/cluster.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from waves.wcore.adaptors.shell import SshKeyShellAdaptor, SshShellAdaptor 4 | from waves.wcore.adaptors.saga_python import SagaAdaptor 5 | 6 | 7 | class LocalClusterAdaptor(SagaAdaptor): 8 | """ 9 | Encapsulate some of Saga-python adapters for common cluster calculation devices onto WAVES adapter logic 10 | """ 11 | NOT_AVAILABLE_MESSAGE = "A valid local %s cluster is needed to run this tests" 12 | 13 | name = 'Local cluster' 14 | protocol_choices = ( 15 | ('sge', 'Sun Grid Engine'), 16 | ('slurm', 'SLURM'), 17 | ('pbs', 'PBS'), 18 | ('condor', 'CONDOR'), 19 | ('pbspro', 'PBS Pro'), 20 | ('lsf', 'LSF'), 21 | ('torque', 'TORQUE') 22 | ) 23 | protocol_default = "sge" 24 | 25 | def __init__(self, command=None, protocol='sge', host="localhost", queue='', **kwargs): 26 | super(LocalClusterAdaptor, self).__init__(command, protocol, host, **kwargs) 27 | self.queue = queue 28 | 29 | @property 30 | def init_params(self): 31 | """ Base init_params for Cluster JobAdapter """ 32 | base = super(LocalClusterAdaptor, self).init_params 33 | base.update(dict(queue=self.queue)) 34 | return base 35 | 36 | def _job_description(self, job): 37 | jd = super(LocalClusterAdaptor, self)._job_description(job) 38 | jd.update(dict(queue=self.queue)) 39 | return jd 40 | 41 | 42 | class SshClusterAdaptor(LocalClusterAdaptor, SshShellAdaptor): 43 | """ 44 | Cluster calls over SSH with user password 45 | """ 46 | name = 'Cluster over SSH (user/pass)' 47 | pass 48 | 49 | 50 | class SshKeyClusterAdaptor(LocalClusterAdaptor, SshKeyShellAdaptor): 51 | """ 52 | Cluster calls over SSH with private key and pass phrase 53 | """ 54 | name = 'Cluster over SSH (key)' 55 | pass 56 | -------------------------------------------------------------------------------- /waves/wcore/static/waves/admin/js/runner.js: -------------------------------------------------------------------------------- 1 | /** 2 | * Created by marc on 22/09/16. 3 | */ 4 | (function ($) { 5 | $(document).ready(function () { 6 | var prev_val = $("#id_clazz").val(); 7 | $("#id_clazz").focus(function () { 8 | prev_val = $(this).val(); 9 | console.log('Prev val' + prev_val); 10 | }).change(function () { 11 | console.log('Changed triggered'); 12 | if (prev_val) { 13 | if (confirm('Changing this value will disable related service and might cancel running jobs.\n\nAre you sure ?')) { 14 | $("input[type='submit'][name='_continue']").trigger('click'); 15 | } else { 16 | $(this).val(prev_val); 17 | } 18 | } else { 19 | var id_name = $("#id_name") 20 | if (id_name.val() === "") { 21 | id_name.val($(this).val().substring($(this).val().lastIndexOf('.') + 1)); 22 | } 23 | //$("input[type='submit'][name='_continue']").trigger('click'); 24 | } 25 | }); 26 | $('#open_import_form').click(function (e) { 27 | console.log('Launch an import ' + $(this).attr('href')); 28 | e.preventDefault(); 29 | $('#popup_modal').modal({backdrop: 'static', keyboard: false, show: true}); 30 | $("#popup_modal_content > div.modal-body").html(''); 31 | $('#popup_modal_content').load($(this).attr('href'), function () { 32 | console.log('loaded') 33 | }); 34 | }); 35 | $('#popup_modal').on('toggle', function () { 36 | console.log('show raised'); 37 | $(this).find('.modal-body').css({ 38 | 'max-height': '100%' 39 | }); 40 | }); 41 | }) 42 | })(jQuery || django.jQuery); -------------------------------------------------------------------------------- /tests/data/sample/fast_me/fastme_matrix.txt: -------------------------------------------------------------------------------- 1 | 9 2 | Aurora 0.0 0.1 0.13 0.12 0.57 0.22 0.86 0.89 0.97 3 | Boylii 0.1 0.0 0.7 0.7 0.5 0.9 0.65 0.67 0.72 4 | Cascadae 0.13 0.7 0.0 0.7 0.4 0.11 0.54 0.66 0.79 5 | Muscosa 0.12 0.7 0.7 0.0 0.45 0.15 0.48 0.49 0.67 6 | Temporaria 0.57 0.5 0.4 0.45 0.0 0.48 0.85 0.83 1.07 7 | Pretiosa 0.22 0.9 0.11 0.15 0.48 0.0 0.54 0.55 0.6 8 | Catesbaiana 0.86 0.65 0.54 0.48 0.85 0.54 0.0 0.54 0.59 9 | Pipiens 0.89 0.67 0.66 0.49 0.83 0.55 0.54 0.0 0.48 10 | Tarahumarae 0.97 0.72 0.79 0.67 1.07 0.60 0.59 0.48 0.0 11 | 12 | 13 | 9 14 | Aurora 0.0 0.11 0.13 0.1 0.57 0.25 0.86 0.89 0.9 15 | Boylii 0.11 0.0 0.8 0.7 0.52 0.6 0.65 0.67 0.77 16 | Cascadae 0.13 0.8 0.0 0.7 0.42 0.11 0.52 0.66 0.80 17 | Muscosa 0.1 0.7 0.7 0.0 0.45 0.15 0.48 0.49 0.67 18 | Temporaria 0.57 0.52 0.42 0.45 0.0 0.45 0.85 0.83 1.07 19 | Pretiosa 0.25 0.6 0.11 0.15 0.45 0.0 0.54 0.57 0.6 20 | Catesbaiana 0.86 0.65 0.52 0.48 0.85 0.54 0.0 0.54 0.53 21 | Pipiens 0.89 0.67 0.66 0.49 0.83 0.57 0.54 0.0 0.46 22 | Tarahumarae 0.9 0.77 0.8 0.67 1.07 0.6 0.53 0.46 0.0 23 | 24 | 9 25 | Aurora 0.0 0.10 0.15 0.12 0.59 0.22 0.86 0.9 0.97 26 | Boylii 0.1 0.0 0.7 0.6 0.5 0.9 0.65 0.67 0.72 27 | Cascadae 0.15 0.7 0.0 0.7 0.4 0.11 0.54 0.66 0.79 28 | Muscosa 0.12 0.6 0.7 0.0 0.45 0.15 0.48 0.46 0.67 29 | Temporaria 0.59 0.5 0.4 0.45 0.0 0.47 0.85 0.83 1.03 30 | Pretiosa 0.22 0.9 0.11 0.15 0.47 0.0 0.54 0.55 0.6 31 | Catesbaiana 0.86 0.65 0.54 0.48 0.85 0.54 0.0 0.54 0.59 32 | Pipiens 0.90 0.67 0.66 0.46 0.83 0.55 0.54 0.0 0.48 33 | Tarahumarae 0.97 0.72 0.79 0.67 1.03 0.6 0.59 0.48 0.0 34 | -------------------------------------------------------------------------------- /waves/wcore/admin/views/job_tool.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ Job tool WAVES admin dedicated views """ 3 | from __future__ import unicode_literals 4 | 5 | from django.views.generic import View 6 | from django.shortcuts import get_object_or_404 7 | from django.shortcuts import redirect 8 | from django.core.urlresolvers import reverse 9 | from django.contrib import messages 10 | from waves.wcore.exceptions import WavesException 11 | from waves.wcore.models import Job 12 | from waves.wcore.adaptors.const import JobStatus 13 | 14 | 15 | class JobCancelView(View): 16 | """ View after cancel a job, if possible """ 17 | 18 | def get(self, request): 19 | """ Try to cancel specified job (in kwargs), redirect to current job page """ 20 | try: 21 | job = get_object_or_404(Job, id=self.kwargs['job_id']) 22 | runner = job.adaptor 23 | if runner is not None: 24 | runner.cancel_job(job) 25 | else: 26 | job.status = JobStatus.JOB_CANCELLED 27 | job.save() 28 | messages.add_message(request, level=messages.SUCCESS, message="Job cancelled") 29 | except WavesException as e: 30 | messages.add_message(request, level=messages.ERROR, message=e.message) 31 | return redirect(reverse('admin:wcore_job_change', args=[self.kwargs['job_id']])) 32 | 33 | 34 | class JobRerunView(View): 35 | def get(self, request): 36 | job = get_object_or_404(Job, id=self.kwargs['job_id']) 37 | if job.allow_rerun: 38 | try: 39 | job.re_run() 40 | messages.success(request, message="Job '%s' successfully marked for re-run" % job.title) 41 | except WavesException as exc: 42 | messages.error(request, message="Error occured %s " % exc.message) 43 | else: 44 | messages.error(request, message="You can't rerun this job") 45 | return redirect(reverse('admin:wcore_job_changelist')) 46 | -------------------------------------------------------------------------------- /tests/data/sample/mafft/aln.fasta: -------------------------------------------------------------------------------- 1 | >TestSequence1 2 | ERNECFLKHKDDDPNLPPVVKPEPEALCTAFQENNNKFLENYLYEVARRHPYFYGPELLY 3 | YVKQ--YKAILTECCQAACCQAADKATCLAPKAKVLKEKLLASSAKQRHKCASIQKFGER 4 | AFKAWSIARLSQRFPKADFMDLSKLVTDLSKIHKECCHGDLLECADDREDLAKYV---QD 5 | SFSSKLKECCDKPLLEKSHCISELENDDLPNDLPSITTDFVEDKDVCKLLNYKEAKDVFL 6 | GTFLYEYSRRHPEYAVSLLLRIAKGYEATLERCCATDDAHACYSKVFDELQPLVDEPQ-- 7 | KLMKRNCELFENLGAYGFQNALIIRYTKKMPQVSTPTLLVISKELANMGNKCCTLPESKR 8 | >TestSequence2 9 | ERADCFASHRDDNPGFPLMVRPPVDELCASYQADAQMFAGKYLYEVARRYPYFYAPELLY 10 | YAQKLLYKDALAEC-----CSAADKAACLTPKIDDLKESVMTSGAKQRFKCAGIEKFGER 11 | AFKAWAVARLSQKFPNADFAEISKIVTDLTKINKECCHGDLLECADDRVELGKYMCDNKD 12 | SISSKLGKCCEKPLLEKGHCIAELERDDMPADLSPIEADFVEDKEVCK--NYAEAKDVFL 13 | GTFLYELSRRHPEYSVVMLLRLAKGYEAVLEKCCATGDPPACYAKVFDELKPLIDEPQ-- 14 | NLVKHNCELYGNLQEYGFQNALLIRYTKKMPQ---PTLVEVSRNLGKVGTKCCSLAEGER 15 | >TestSequence3 16 | ERNECFLSHKDDSPDLPKL-KPDPNTLCDEFKADEKKFWGKYLYEIARRHPYFYAPELLY 17 | YANK--YNGVFQEC-----CQAEDKGACLLPKIETMREKVLTSSARQRLRCASIQKFGER 18 | ALKAWSVARLSQKFPKAEFVEVTKLVTDLTKVHKECCHGDLLECADDRADLAKYICDNQD 19 | TISSKLKECCDKPLLEKSHCIAEVEKDAVPENLPPLTADFAEDKDVCK--NYQEAKDAFL 20 | GSFLYEYSRRHPE-YVLLRLALLKEYEATLEECCAKDDPHACYSTVFDKLKHLVDEPQ-- 21 | NLIKQNCDQFEKLGEYGFQNALIVRYTRKVPQVSTPTLVEVSRSLGKVGTRCCTKPESER 22 | >TestSequence4 23 | ERNECFLQHKDDNPNLPRLVRPEVDVMCTAFHDNEETFLKKYLYEIARRHPYFYAPELLF 24 | FAKR--YKAAFTEC-----CQAADKAACLLPKLDELRDEGKASSAKQRLKCASLQKFGER 25 | AFKAWAVARLSQRFPKAEFAEVSKLVTDLTKVHTECCHGDLLECADDRADLAKYICENQD 26 | SISSKLKECCEKPLLEKSHCIAEVENDEMPADLPSLAADFVESKDVCK--NYAEAKDVFL 27 | GMFLYEYARRHPRESVVLLLRLAKTYETTLEKCCAAADPHECYAKVFDEFKPLVEEPQLL 28 | NLIKQNCELFEQLGEYKFQNALLVRYTKKVPQVSTPTLVEVSRNLGKVGSKCCKHPEAKR 29 | >TestSequence5 30 | ERNECFLQHKDDNPGFGQLVTPEADAMCTAFHENEQRFLGKYLYEIARRHPYFYAPELLY 31 | YAEE--YKGVFTEC-----CEAADKAACLTPKVDALREKVLASSAKERLKCASLQKFGER 32 | AFKAWSVARLSQKFPKAEFAEISKLVTDLAKIHKECCHGDLLECADDRADLAKYICENQD 33 | SISTKLKECCGKPVLEKSHCISEVERDELPADLPPLAVDFVEDKEVCK--NYQEAKDVFL 34 | GTFLYEYSRRHPS-SVSLLLRLAKEYEATLEKCCATDDPPACYAHVFDEFKPLVEEPH-- 35 | NLVKTNCELFEKLGEYGFQNALLVRYTKKVPQVSTPTLVEVSRSLGKVGSKCCTHPEAER -------------------------------------------------------------------------------- /waves/wcore/static/waves/admin/js/admin.js: -------------------------------------------------------------------------------- 1 | /** 2 | * Created by Marc Chakiachvili on 23/09/16. 3 | * Standard JQuery library to attache events to Django BO 4 | */ 5 | (function ($) { 6 | $(document).ready(function () { 7 | $('#popup_modal').on('shown.bs.modal', function () { 8 | $(this).find('.modal-dialog').css({ 9 | width: 'auto', 10 | height: 'auto', 11 | 'max-height': '80%' 12 | }); 13 | }); 14 | 15 | $('.js-popup-link').click(function (e) { 16 | e.preventDefault(); 17 | // language=JQuery-CSS 18 | var modalContent = $('#popup_modal_content'); 19 | console.log('Js-pop-up-modal called ' + $('#popup_modal')); 20 | if ($(this).attr('js-modal-title') !== null) { 21 | modalContent.find('.modal-header').html("

    " + $(this).attr('js-modal-title') + "

    "); 22 | } 23 | modalContent.find('.modal-body').load($(this).attr('href'), function () { 24 | console.log('open modal'); 25 | $('#popup_modal').modal({backdrop: 'static', keyboard: false, show:true}); 26 | }) 27 | }) 28 | $('#modal_alert').on('show.bs.modal', function () { 29 | console.log('opened !'); 30 | $(this).find('.modal-dialog').css({ 31 | 'max-height': '50%' 32 | }); 33 | }).on('hidden.bs.modal', function () { 34 | $(this).find('.modal-body').html(""); 35 | console.log("closed"); 36 | }); 37 | 38 | 39 | }); 40 | $(window).load(function () { 41 | $('fieldset.collapse.open').each(function () { 42 | $(this).removeClass('collapsed'); 43 | $(this).find('a.collapse-toggle').html('Hide'); 44 | }) 45 | $('.errorlist').parents('fieldset.collapsed').each(function () { 46 | $(this).removeClass('collapsed'); 47 | }); 48 | }) 49 | 50 | })(jQuery || django.jQuery); 51 | 52 | 53 | 54 | -------------------------------------------------------------------------------- /docs/dev_doc/dev_doc.rst: -------------------------------------------------------------------------------- 1 | .. _dev-guide: 2 | 3 | =============== 4 | Developer Guide 5 | =============== 6 | 7 | Definitions 8 | =========== 9 | 10 | Computing infrastructure 11 | ------------------------- 12 | It is composed of computationally dedicated hardware and the software components required to operate it, including calculation management programs (distributed resource management systems, Galaxy,...). 13 | 14 | Adaptor 15 | ------- 16 | It is a Django module allowing WAVES-core to communicate with a specified computing infrastructure. For each computing infrastructure, WAVES-core needs a dedicated adaptor. 17 | 18 | Service 19 | ------- 20 | A service is a bioinformatic tool available online through the http protocol. It can be accessed from a web form or through REST API calls. 21 | 22 | Submission 23 | ---------- 24 | Many bioinformatic tools provide several distinct usages. For instance, a program can be run using the command-line interface or by providing a configuration file. Otherwise, the same tool can be run on different computing infrastructures. A submission is the combination of a usage and a computing infrastructure. Thus, a service can rely on different submissions. 25 | 26 | Job 27 | --- 28 | A job stands for a command with parameters. It is run on a dedicated computing infrastructure. It may require inputs such as files. It generates outputs: exit code, standard output and standard error, and possibly result files. A job is run each time a submission is invoked by a service. 29 | 30 | User 31 | ---- 32 | A user is a client which accesses services. It can be a real person using a web browser or a software using the REST API. 33 | 34 | Administrator 35 | ------------- 36 | An administrator is a privileged user with granted access to the WAVES-core back-office. He manages configurations, services, submissions, adaptors and jobs. 37 | 38 | 39 | Documentation 40 | ============= 41 | 42 | .. toctree:: 43 | :maxdepth: 3 44 | :numbered: 45 | 46 | waves_dev_doc 47 | sample_code 48 | sample_code_php 49 | .. api_user_doc 50 | 51 | -------------------------------------------------------------------------------- /waves/wcore/tests/test_copy_service.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | from __future__ import print_function 3 | 4 | import shutil 5 | import json 6 | from os.path import basename, join 7 | 8 | from django.conf import settings 9 | from waves.wcore.adaptors.tests import logger 10 | from waves.wcore.models import JobInput, JobOutput, Job 11 | from waves.wcore.models.const import ParamType 12 | from waves.wcore.models.services import Service 13 | from waves.wcore.tests.base import BaseTestCase, TestJobWorkflowMixin 14 | 15 | 16 | class CopyServiceTestCase(BaseTestCase, TestJobWorkflowMixin): 17 | fixtures = ['waves/wcore/tests/fixtures/users.json', 'waves/wcore/tests/fixtures/runners.json', 18 | 'waves/wcore/tests/fixtures/copy_service.json'] 19 | 20 | def create_cp_job(self, source_file, submission): 21 | job = self.create_base_job('Sample CP job', submission) 22 | shutil.copy(source_file, job.working_dir) 23 | job.inputs = [JobInput.objects.create(label="File To copy", name='source', 24 | value=basename(source_file), param_type=ParamType.TYPE_FILE, job=job), 25 | JobInput.objects.create(label="Destination Dir", name="dest", 26 | value='dest_copy.txt', param_type=ParamType.TYPE_TEXT, job=job)] 27 | job.outputs = [JobOutput.objects.create(_name='Copied File', name='dest', value=job.inputs[1].value, job=job)] 28 | return job 29 | 30 | def test_local_cp_job(self): 31 | cp_service = Service.objects.filter(api_name='copy').first() 32 | with open(join(settings.WAVES_CORE['DATA_ROOT'], "test.fasta"), 'rb') as fp: 33 | job_payload = { 34 | 'src': fp.read(), 35 | 'dest': 'test_fasta_copy.txt' 36 | } 37 | logger.debug('Service runner "%s"', cp_service.get_runner().name) 38 | job = Job.objects.create_from_submission(cp_service.default_submission, job_payload) 39 | logger.info('job command line %s ', job.command_line) 40 | self.run_job_workflow(job) 41 | -------------------------------------------------------------------------------- /waves/wcore/utils/logged.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | import logging 4 | import os 5 | import stat 6 | 7 | from django.conf import settings 8 | 9 | logger_file = logging.getLogger(__name__) 10 | 11 | 12 | class LoggerClass(object): 13 | LOG_LEVEL = logging.DEBUG 14 | log_dir = os.path.dirname(settings.BASE_DIR) 15 | _logger = None 16 | 17 | @property 18 | def logger(self): 19 | """ Get or create a new logger for this job 20 | 21 | :return: Logger""" 22 | self._logger = logging.getLogger(self.logger_name) 23 | if not len(self._logger.handlers): 24 | # Add handler only once ! 25 | try: 26 | hdlr = logging.FileHandler(self.log_file) 27 | mode = os.stat(self.log_file).st_mode 28 | if not bool(mode & stat.S_IWGRP): 29 | os.chmod(self.log_file, 0o664) 30 | formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s') 31 | hdlr.setFormatter(formatter) 32 | self._logger.propagate = False 33 | self._logger.addHandler(hdlr) 34 | self._logger.setLevel(self.LOG_LEVEL) 35 | except OSError as e: 36 | logger_file.exception("OSError in %s: %s [file:%s]", self.__class__.__name__, e.message, e.filename) 37 | except IOError as err: 38 | self._logger = logging.getLogger("waves.errors") 39 | self._logger.warn('This object %s is not able to log where it should %s', self.pk, self.log_file) 40 | logger_file.exception("IO Error in %s: %s", self.__class__.__name__, err.message) 41 | return self._logger 42 | 43 | @property 44 | def log_file(self): 45 | """ Return path to job dedicated log file 46 | 47 | :return: str""" 48 | return os.path.join(self.log_dir, self.logger_file_name) 49 | 50 | @property 51 | def logger_name(self): 52 | return self.__class__.__name__ 53 | 54 | @property 55 | def logger_file_name(self): 56 | return self.logger_name + '.log' 57 | -------------------------------------------------------------------------------- /waves/wcore/utils/storage.py: -------------------------------------------------------------------------------- 1 | """ WAVES Files storage engine parameters """ 2 | from __future__ import unicode_literals 3 | 4 | import os 5 | 6 | from waves.wcore.settings import waves_settings 7 | from django.core.files.storage import FileSystemStorage 8 | 9 | 10 | class WavesStorage(FileSystemStorage): 11 | """ Waves FileSystem Storage engine """ 12 | 13 | def __init__(self): 14 | super(WavesStorage, self).__init__(location=waves_settings.DATA_ROOT, 15 | directory_permissions_mode=0o775, 16 | file_permissions_mode=0o775) 17 | 18 | 19 | class BinaryStorage(FileSystemStorage): 20 | """ Waves binary file storage engine """ 21 | 22 | def __init__(self): 23 | super(BinaryStorage, self).__init__(location=waves_settings.BINARIES_DIR, 24 | directory_permissions_mode=0o775, 25 | file_permissions_mode=0o775) 26 | 27 | 28 | def file_sample_directory(instance, filename): 29 | """ Submission file sample directory upload pattern """ 30 | return os.path.join('sample', str(instance.file_input.submission.service.api_name), 31 | str(instance.file_input.submission.slug), filename) 32 | 33 | 34 | def binary_directory(instance, filename): 35 | return os.path.join(str(instance.slug), filename) 36 | 37 | 38 | def job_file_directory(instance, filename): 39 | """ Submitted job input files """ 40 | return 'jobs/{0}/{1}'.format(str(instance.job.slug), filename) 41 | 42 | 43 | def allow_display_online(file_name): 44 | """ 45 | Determine if current 'input' or 'output' may be displayed online, maximum file size is set to '1Mo' 46 | :param file_name: file name to test for size 47 | :return: bool 48 | """ 49 | display_file_online = 1024 * 1024 * 1 50 | try: 51 | size = os.path.getsize(file_name) 52 | return display_file_online >= size > 0 53 | except os.error: 54 | return False 55 | 56 | 57 | waves_storage = WavesStorage() 58 | binary_storage = BinaryStorage() 59 | -------------------------------------------------------------------------------- /waves/wcore/adaptors/exceptions.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | 4 | class AdaptorException(Exception): 5 | """ Base adapter exception class, should be raise upon specific adapter class exception catch 6 | this exception class is supposed to be catched 7 | """ 8 | pass 9 | 10 | 11 | class AdaptorConnectException(AdaptorException): 12 | """ 13 | adapter Connection Error 14 | """ 15 | pass 16 | 17 | 18 | class AdaptorExecException(AdaptorException): 19 | """ 20 | adapter execution error 21 | """ 22 | pass 23 | 24 | 25 | class AdaptorNotAvailableException(AdaptorExecException): 26 | pass 27 | 28 | 29 | class AdaptorJobException(AdaptorException): 30 | """ 31 | adapter JobRun Exception 32 | """ 33 | pass 34 | 35 | 36 | class AdaptorNotReady(AdaptorException): 37 | """ adapter is not properly initialized to be used """ 38 | pass 39 | 40 | 41 | class AdaptorInitError(AdaptorException): 42 | """ Each adapter expects some attributes for initialization, this exception should be raised when some mandatory 43 | parameters are missing 44 | """ 45 | pass 46 | 47 | 48 | class ImporterException(AdaptorException): 49 | pass 50 | 51 | 52 | class UnmanagedException(ImporterException): 53 | base_msg = '' 54 | 55 | def __init__(self, *args, **kwargs): 56 | super(UnmanagedException, self).__init__(*args, **kwargs) 57 | self.message = self.base_msg + self.message 58 | 59 | 60 | class UnManagedAttributeException(UnmanagedException): 61 | base_msg = "Unmanaged Attribute: " 62 | 63 | def __init__(self, *args, **kwargs): 64 | super(UnManagedAttributeException, self).__init__(*args, **kwargs) 65 | 66 | 67 | class UnManagedAttributeTypeException(UnmanagedException): 68 | base_msg = "Unmanaged Type: " 69 | 70 | def __init__(self, *args, **kwargs): 71 | super(UnManagedAttributeTypeException, self).__init__(*args, **kwargs) 72 | 73 | 74 | class UnmanagedInputTypeException(UnmanagedException): 75 | base_msg = "Unmanaged Input: " 76 | 77 | def __init__(self, *args, **kwargs): 78 | super(UnmanagedInputTypeException, self).__init__(*args, **kwargs) 79 | -------------------------------------------------------------------------------- /waves/wcore/adaptors/mocks.py: -------------------------------------------------------------------------------- 1 | """ Mock Adapter class for tests purpose 2 | """ 3 | from __future__ import unicode_literals 4 | 5 | import datetime 6 | import random 7 | import string 8 | import time 9 | 10 | from waves.wcore.adaptors import JobAdaptor 11 | from waves.wcore.adaptors.const import JobStatus 12 | 13 | 14 | class MockConnector(object): 15 | pass 16 | 17 | 18 | class MockJobRunnerAdaptor(JobAdaptor): 19 | _states_map = { 20 | JobStatus.JOB_UNDEFINED: JobStatus.JOB_UNDEFINED, 21 | JobStatus.JOB_CREATED: JobStatus.JOB_CREATED, 22 | JobStatus.JOB_QUEUED: JobStatus.JOB_QUEUED, 23 | JobStatus.JOB_RUNNING: JobStatus.JOB_RUNNING, 24 | JobStatus.JOB_SUSPENDED: JobStatus.JOB_SUSPENDED, 25 | JobStatus.JOB_CANCELLED: JobStatus.JOB_CANCELLED, 26 | JobStatus.JOB_COMPLETED: JobStatus.JOB_COMPLETED, 27 | JobStatus.JOB_TERMINATED: JobStatus.JOB_TERMINATED, 28 | JobStatus.JOB_ERROR: JobStatus.JOB_ERROR, 29 | } 30 | 31 | def __init__(self, command=None, protocol='http', host="localhost", **kwargs): 32 | super(MockJobRunnerAdaptor, self).__init__(command, protocol, host, **kwargs) 33 | self.command = 'mock_command' 34 | 35 | def _job_status(self, job): 36 | time.sleep(2) 37 | if job.status == JobStatus.JOB_RUNNING: 38 | return JobStatus.JOB_COMPLETED 39 | job.updated = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%I') 40 | return job.next_status 41 | 42 | def _run_job(self, job): 43 | time.sleep(2) 44 | job.remote_job_id = '%s-%s' % (job.id, ''.join(random.sample(string.letters, 15))) 45 | job.started = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%I') 46 | 47 | def _disconnect(self): 48 | self.connector = None 49 | pass 50 | 51 | def _connect(self): 52 | self.connector = MockConnector() 53 | self._connected = True 54 | pass 55 | 56 | def _job_results(self, job): 57 | time.sleep(2) 58 | return True 59 | 60 | def _prepare_job(self, job): 61 | time.sleep(2) 62 | job.created = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%I') 63 | 64 | def _cancel_job(self, job): 65 | pass 66 | -------------------------------------------------------------------------------- /docs/modules/models/services.rst: -------------------------------------------------------------------------------- 1 | .. _service-label: 2 | 3 | Services 4 | ======== 5 | 6 | Services are the main entry point for WAVEs application, managed by :ref:`service-manager-label`. 7 | 8 | .. autoclass:: waves.wcore.models.services.Service 9 | :members: 10 | :inherited-members: 11 | :show-inheritance: 12 | 13 | Submissions 14 | =========== 15 | 16 | Services may be accessed from multiple 'submissions' 17 | 18 | .. autoclass:: waves.wcore.models.services.Submission 19 | :members: 20 | :inherited-members: 21 | :show-inheritance: 22 | 23 | .. _service-inputs-label: 24 | 25 | Submission Inputs 26 | ----------------- 27 | 28 | Classes for service's submission inputs information 29 | 30 | Booleans 31 | ^^^^^^^^ 32 | .. autoclass:: waves.wcore.models.inputs.BooleanParam 33 | :inherited-members: 34 | 35 | Decimals 36 | ^^^^^^^^ 37 | .. autoclass:: waves.wcore.models.inputs.DecimalParam 38 | :inherited-members: 39 | 40 | Files 41 | ^^^^^ 42 | .. autoclass:: waves.wcore.models.inputs.FileInput 43 | :inherited-members: 44 | 45 | Integers 46 | ^^^^^^^^ 47 | .. autoclass:: waves.wcore.models.inputs.IntegerParam 48 | :inherited-members: 49 | 50 | Text Inputs 51 | ^^^^^^^^^^^ 52 | .. autoclass:: waves.wcore.models.inputs.TextParam 53 | :inherited-members: 54 | 55 | .. _service-outputs-label: 56 | 57 | Submission Outputs 58 | ------------------ 59 | 60 | Submission description defines expected outputs 61 | 62 | .. autoclass:: waves.wcore.models.services.SubmissionOutput 63 | :members: 64 | 65 | 66 | Submission ExitCode 67 | ------------------- 68 | 69 | Submission description defines expected exitcode 70 | 71 | .. autoclass:: waves.wcore.models.services.SubmissionExitCode 72 | :members: 73 | 74 | .. _service-samples-label: 75 | 76 | Input Samples: 77 | -------------- 78 | 79 | Services may provide sample data for their submissions 80 | 81 | .. autoclass:: waves.wcore.models.inputs.FileInputSample 82 | :members: 83 | :undoc-members: 84 | :show-inheritance: 85 | 86 | .. autoclass:: waves.wcore.models.inputs.SampleDepParam 87 | :members: 88 | :undoc-members: 89 | :show-inheritance: -------------------------------------------------------------------------------- /waves/wcore/adaptors/api.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from waves.wcore.adaptors import JobAdaptor 4 | 5 | 6 | # noinspection PyAbstractClass 7 | class PublicApiAdaptor(JobAdaptor): 8 | """ Base Class for remote public API calls""" 9 | port = '' 10 | api_endpoint = '' 11 | api_base_path = '' 12 | 13 | def __init__(self, command=None, protocol='http', host="localhost", port='', api_base_path='', api_endpoint='', 14 | **kwargs): 15 | super(PublicApiAdaptor, self).__init__(command, protocol, host, **kwargs) 16 | self.port = port 17 | self.api_endpoint = api_endpoint 18 | self.api_base_path = api_base_path 19 | 20 | @property 21 | def init_params(self): 22 | base = super(PublicApiAdaptor, self).init_params 23 | base.update(dict(port=self.port, 24 | api_base_path=self.api_base_path, 25 | api_endpoint=self.api_endpoint)) 26 | return base 27 | 28 | @property 29 | def complete_url(self): 30 | """ Create complete url string for remote api""" 31 | url = "%s://%s" % (self.protocol, self.host) 32 | if self.port is not None and self.port != '' and self.port != '80': 33 | url += ':%s' % self.port 34 | if self.api_base_path is not None and self.api_base_path != '': 35 | url += '/%s' % self.api_base_path 36 | if self.api_endpoint is not None and self.api_endpoint != '': 37 | url += '/%s' % self.api_endpoint 38 | return url 39 | 40 | def connexion_string(self): 41 | return self.complete_url 42 | 43 | 44 | class ApiKeyAdaptor(PublicApiAdaptor): 45 | """ 46 | Authenticated api calls 47 | """ 48 | _api_get_key = 'app_key' 49 | 50 | def __init__(self, command=None, protocol='http', host="localhost", port='', api_base_path='', api_endpoint='', 51 | app_key=None, **kwargs): 52 | super(ApiKeyAdaptor, self).__init__(command, protocol, host, port, api_base_path, api_endpoint, **kwargs) 53 | self.app_key = app_key 54 | 55 | @property 56 | def init_params(self): 57 | base = super(ApiKeyAdaptor, self).init_params 58 | base.update(dict(app_key=self.app_key)) 59 | return base 60 | -------------------------------------------------------------------------------- /waves/wcore/exceptions/jobs.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | from waves.wcore.adaptors.exceptions import AdaptorJobException 4 | from waves.wcore.exceptions import WavesException 5 | 6 | __all__ = ['JobException', 'JobRunException', 'JobSubmissionException', 'JobCreateException', 7 | 'JobMissingMandatoryParam', 'JobInconsistentStateError', 'JobPrepareException'] 8 | 9 | 10 | class JobException(WavesException): 11 | """ Base Exception class for all job related errors """ 12 | 13 | def __init__(self, message, job=None): 14 | if job: 15 | message = '[job:%s][%s] - %s' % (job.slug, job.remote_job_id, message) 16 | super(JobException, self).__init__(message) 17 | 18 | 19 | class JobRunException(JobException): 20 | """ Job 'run' generic error parent class """ 21 | pass 22 | 23 | 24 | class JobSubmissionException(JobException): 25 | """ Job submission generic error parent class """ 26 | pass 27 | 28 | 29 | class JobCreateException(JobSubmissionException): 30 | """ Job creation process erro """ 31 | 32 | def __init__(self, message, job=None): 33 | super(JobException, self).__init__(message) 34 | 35 | 36 | class JobMissingMandatoryParam(JobSubmissionException): 37 | """ Inconsistency in job submission detected, missing a required params to run job (issued from service 38 | configuration """ 39 | 40 | def __init__(self, param, job): 41 | message = u'Missing mandatory parameter "%s"' % param 42 | super(JobMissingMandatoryParam, self).__init__(message, job) 43 | 44 | 45 | class JobInconsistentStateError(JobException): 46 | """ Job current status is inconsistent for requested action 47 | 48 | """ 49 | 50 | def __init__(self, message="", job=None, expected=None): 51 | """ 52 | :param job: current Job 53 | :param expected: list oc expected status 54 | :param message: extended message to add to standard log_exception message 55 | """ 56 | if expected is None: 57 | expected = [] 58 | if job: 59 | message = u'{} [{}] - Inconsistent job state: "{}" - expected {}'.format( 60 | message, job.slug, job.get_status_display(), [str(i[1]) for i in expected], ) 61 | super(JobInconsistentStateError, self).__init__(message) 62 | 63 | 64 | class JobPrepareException(AdaptorJobException): 65 | """Preparation process errors """ 66 | pass 67 | -------------------------------------------------------------------------------- /tests/data/test.fasta: -------------------------------------------------------------------------------- 1 | >HSBGPG Human gene for bone gla protein (BGP) 2 | GGCAGATTCCCCCTAGACCCGCCCGCACCATGGTCAGGCATGCCCCTCCTCATCGCTGGGCACAGCCCAGAGGGT 3 | ATAAACAGTGCTGGAGGCTGGCGGGGCAGGCCAGCTGAGTCCTGAGCAGCAGCCCAGCGCAGCCACCGAGACACC 4 | ATGAGAGCCCTCACACTCCTCGCCCTATTGGCCCTGGCCGCACTTTGCATCGCTGGCCAGGCAGGTGAGTGCCCC 5 | CACCTCCCCTCAGGCCGCATTGCAGTGGGGGCTGAGAGGAGGAAGCACCATGGCCCACCTCTTCTCACCCCTTTG 6 | GCTGGCAGTCCCTTTGCAGTCTAACCACCTTGTTGCAGGCTCAATCCATTTGCCCCAGCTCTGCCCTTGCAGAGG 7 | GAGAGGAGGGAAGAGCAAGCTGCCCGAGACGCAGGGGAAGGAGGATGAGGGCCCTGGGGATGAGCTGGGGTGAAC 8 | CAGGCTCCCTTTCCTTTGCAGGTGCGAAGCCCAGCGGTGCAGAGTCCAGCAAAGGTGCAGGTATGAGGATGGACC 9 | TGATGGGTTCCTGGACCCTCCCCTCTCACCCTGGTCCCTCAGTCTCATTCCCCCACTCCTGCCACCTCCTGTCTG 10 | GCCATCAGGAAGGCCAGCCTGCTCCCCACCTGATCCTCCCAAACCCAGAGCCACCTGATGCCTGCCCCTCTGCTC 11 | CACAGCCTTTGTGTCCAAGCAGGAGGGCAGCGAGGTAGTGAAGAGACCCAGGCGCTACCTGTATCAATGGCTGGG 12 | GTGAGAGAAAAGGCAGAGCTGGGCCAAGGCCCTGCCTCTCCGGGATGGTCTGTGGGGGAGCTGCAGCAGGGAGTG 13 | GCCTCTCTGGGTTGTGGTGGGGGTACAGGCAGCCTGCCCTGGTGGGCACCCTGGAGCCCCATGTGTAGGGAGAGG 14 | AGGGATGGGCATTTTGCACGGGGGCTGATGCCACCACGTCGGGTGTCTCAGAGCCCCAGTCCCCTACCCGGATCC 15 | CCTGGAGCCCAGGAGGGAGGTGTGTGAGCTCAATCCGGACTGTGACGAGTTGGCTGACCACATCGGCTTTCAGGA 16 | GGCCTATCGGCGCTTCTACGGCCCGGTCTAGGGTGTCGCTCTGCTGGCCTGGCCGGCAACCCCAGTTCTGCTCCT 17 | CTCCAGGCACCCTTCTTTCCTCTTCCCCTTGCCCTTGCCCTGACCTCCCAGCCCTATGGATGTGGGGTCCCCATC 18 | ATCCCAGCTGCTCCCAAATAAACTCCAGAAG 19 | >HSGLTH1 Human theta 1-globin gene 20 | CCACTGCACTCACCGCACCCGGCCAATTTTTGTGTTTTTAGTAGAGACTAAATACCATATAGTGAACACCTAAGA 21 | CGGGGGGCCTTGGATCCAGGGCGATTCAGAGGGCCCCGGTCGGAGCTGTCGGAGATTGAGCGCGCGCGGTCCCGG 22 | GATCTCCGACGAGGCCCTGGACCCCCGGGCGGCGAAGCTGCGGCGCGGCGCCCCCTGGAGGCCGCGGGACCCCTG 23 | GCCGGTCCGCGCAGGCGCAGCGGGGTCGCAGGGCGCGGCGGGTTCCAGCGCGGGGATGGCGCTGTCCGCGGAGGA 24 | CCGGGCGCTGGTGCGCGCCCTGTGGAAGAAGCTGGGCAGCAACGTCGGCGTCTACACGACAGAGGCCCTGGAAAG 25 | GTGCGGCAGGCTGGGCGCCCCCGCCCCCAGGGGCCCTCCCTCCCCAAGCCCCCCGGACGCGCCTCACCCACGTTC 26 | CTCTCGCAGGACCTTCCTGGCTTTCCCCGCCACGAAGACCTACTTCTCCCACCTGGACCTGAGCCCCGGCTCCTC 27 | ACAAGTCAGAGCCCACGGCCAGAAGGTGGCGGACGCGCTGAGCCTCGCCGTGGAGCGCCTGGACGACCTACCCCA 28 | CGCGCTGTCCGCGCTGAGCCACCTGCACGCGTGCCAGCTGCGAGTGGACCCGGCCAGCTTCCAGGTGAGCGGCTG 29 | CCGTGCTGGGCCCCTGTCCCCGGGAGGGCCCCGGCGGGGTGGGTGCGGGGGGCGTGCGGGGCGGGTGCAGGCGAG 30 | TGAGCCTTGAGCGCTCGCCGCAGCTCCTGGGCCACTGCCTGCTGGTAACCCTCGCCCGGCACTACCCCGGAGACT 31 | TCAGCCCCGCGCTGCAGGCGTCGCTGGACAAGTTCCTGAGCCACGTTATCTCGGCGCTGGTTTCCGAGTACCGCT 32 | GAACTGTGGGTGGGTGGCCGCGGGATCCCCAGGCGACCTTCCCCGTGTTTGAGTAAAGCCTCTCCCAGGAGCAGC 33 | CTTCTTGCCGTGCTCTCTCGAGGTCAGGACGCGAGAGGAAGGCGC 34 | 35 | 36 | -------------------------------------------------------------------------------- /waves/wcore/tests/fixtures/test.fasta: -------------------------------------------------------------------------------- 1 | >HSBGPG Human gene for bone gla protein (BGP) 2 | GGCAGATTCCCCCTAGACCCGCCCGCACCATGGTCAGGCATGCCCCTCCTCATCGCTGGGCACAGCCCAGAGGGT 3 | ATAAACAGTGCTGGAGGCTGGCGGGGCAGGCCAGCTGAGTCCTGAGCAGCAGCCCAGCGCAGCCACCGAGACACC 4 | ATGAGAGCCCTCACACTCCTCGCCCTATTGGCCCTGGCCGCACTTTGCATCGCTGGCCAGGCAGGTGAGTGCCCC 5 | CACCTCCCCTCAGGCCGCATTGCAGTGGGGGCTGAGAGGAGGAAGCACCATGGCCCACCTCTTCTCACCCCTTTG 6 | GCTGGCAGTCCCTTTGCAGTCTAACCACCTTGTTGCAGGCTCAATCCATTTGCCCCAGCTCTGCCCTTGCAGAGG 7 | GAGAGGAGGGAAGAGCAAGCTGCCCGAGACGCAGGGGAAGGAGGATGAGGGCCCTGGGGATGAGCTGGGGTGAAC 8 | CAGGCTCCCTTTCCTTTGCAGGTGCGAAGCCCAGCGGTGCAGAGTCCAGCAAAGGTGCAGGTATGAGGATGGACC 9 | TGATGGGTTCCTGGACCCTCCCCTCTCACCCTGGTCCCTCAGTCTCATTCCCCCACTCCTGCCACCTCCTGTCTG 10 | GCCATCAGGAAGGCCAGCCTGCTCCCCACCTGATCCTCCCAAACCCAGAGCCACCTGATGCCTGCCCCTCTGCTC 11 | CACAGCCTTTGTGTCCAAGCAGGAGGGCAGCGAGGTAGTGAAGAGACCCAGGCGCTACCTGTATCAATGGCTGGG 12 | GTGAGAGAAAAGGCAGAGCTGGGCCAAGGCCCTGCCTCTCCGGGATGGTCTGTGGGGGAGCTGCAGCAGGGAGTG 13 | GCCTCTCTGGGTTGTGGTGGGGGTACAGGCAGCCTGCCCTGGTGGGCACCCTGGAGCCCCATGTGTAGGGAGAGG 14 | AGGGATGGGCATTTTGCACGGGGGCTGATGCCACCACGTCGGGTGTCTCAGAGCCCCAGTCCCCTACCCGGATCC 15 | CCTGGAGCCCAGGAGGGAGGTGTGTGAGCTCAATCCGGACTGTGACGAGTTGGCTGACCACATCGGCTTTCAGGA 16 | GGCCTATCGGCGCTTCTACGGCCCGGTCTAGGGTGTCGCTCTGCTGGCCTGGCCGGCAACCCCAGTTCTGCTCCT 17 | CTCCAGGCACCCTTCTTTCCTCTTCCCCTTGCCCTTGCCCTGACCTCCCAGCCCTATGGATGTGGGGTCCCCATC 18 | ATCCCAGCTGCTCCCAAATAAACTCCAGAAG 19 | >HSGLTH1 Human theta 1-globin gene 20 | CCACTGCACTCACCGCACCCGGCCAATTTTTGTGTTTTTAGTAGAGACTAAATACCATATAGTGAACACCTAAGA 21 | CGGGGGGCCTTGGATCCAGGGCGATTCAGAGGGCCCCGGTCGGAGCTGTCGGAGATTGAGCGCGCGCGGTCCCGG 22 | GATCTCCGACGAGGCCCTGGACCCCCGGGCGGCGAAGCTGCGGCGCGGCGCCCCCTGGAGGCCGCGGGACCCCTG 23 | GCCGGTCCGCGCAGGCGCAGCGGGGTCGCAGGGCGCGGCGGGTTCCAGCGCGGGGATGGCGCTGTCCGCGGAGGA 24 | CCGGGCGCTGGTGCGCGCCCTGTGGAAGAAGCTGGGCAGCAACGTCGGCGTCTACACGACAGAGGCCCTGGAAAG 25 | GTGCGGCAGGCTGGGCGCCCCCGCCCCCAGGGGCCCTCCCTCCCCAAGCCCCCCGGACGCGCCTCACCCACGTTC 26 | CTCTCGCAGGACCTTCCTGGCTTTCCCCGCCACGAAGACCTACTTCTCCCACCTGGACCTGAGCCCCGGCTCCTC 27 | ACAAGTCAGAGCCCACGGCCAGAAGGTGGCGGACGCGCTGAGCCTCGCCGTGGAGCGCCTGGACGACCTACCCCA 28 | CGCGCTGTCCGCGCTGAGCCACCTGCACGCGTGCCAGCTGCGAGTGGACCCGGCCAGCTTCCAGGTGAGCGGCTG 29 | CCGTGCTGGGCCCCTGTCCCCGGGAGGGCCCCGGCGGGGTGGGTGCGGGGGGCGTGCGGGGCGGGTGCAGGCGAG 30 | TGAGCCTTGAGCGCTCGCCGCAGCTCCTGGGCCACTGCCTGCTGGTAACCCTCGCCCGGCACTACCCCGGAGACT 31 | TCAGCCCCGCGCTGCAGGCGTCGCTGGACAAGTTCCTGAGCCACGTTATCTCGGCGCTGGTTTCCGAGTACCGCT 32 | GAACTGTGGGTGGGTGGCCGCGGGATCCCCAGGCGACCTTCCCCGTGTTTGAGTAAAGCCTCTCCCAGGAGCAGC 33 | CTTCTTGCCGTGCTCTCTCGAGGTCAGGACGCGAGAGGAAGGCGC 34 | 35 | 36 | -------------------------------------------------------------------------------- /waves/authentication/auth.py: -------------------------------------------------------------------------------- 1 | """ Authentication base on a get parameter 'api_key'""" 2 | from __future__ import unicode_literals 3 | 4 | from django.utils.translation import ugettext_lazy as _ 5 | from rest_framework import authentication, exceptions 6 | from rest_framework.authentication import TokenAuthentication 7 | from waves.authentication.models import WavesApiUser 8 | 9 | 10 | class WavesAuth(object): 11 | 12 | @staticmethod 13 | def check_api_user(waves_user, request): 14 | if waves_user.ip_list: 15 | ip_list = waves_user.ip_list.split(',') 16 | if 'REMOTE_ADDR' not in request.META: 17 | msg = _('Origin IP not sent, but expected.') 18 | raise exceptions.AuthenticationFailed(msg) 19 | if request.META['REMOTE_ADDR'] not in ip_list: 20 | msg = _('Invalid IP in header.') 21 | raise exceptions.AuthenticationFailed(msg) 22 | 23 | 24 | class ApiKeyAuthentication(authentication.BaseAuthentication): 25 | """ WAVES API authentication backend, api_key added to URI """ 26 | 27 | def authenticate(self, request): 28 | auth = request.POST.get('api_key', request.GET.get('api_key', None)) 29 | 30 | if not auth: 31 | return None 32 | 33 | try: 34 | token = auth 35 | except UnicodeError: 36 | msg = _('Invalid api key. api key string should not contain invalid characters.') 37 | raise exceptions.AuthenticationFailed(msg) 38 | 39 | auth_tuple = self.authenticate_credentials(token) 40 | if auth_tuple: 41 | WavesAuth.check_api_user(auth_tuple[1], request) 42 | return auth_tuple 43 | 44 | def authenticate_credentials(self, key): 45 | try: 46 | api_key = WavesApiUser.objects.select_related('user').get(key=key) 47 | except WavesApiUser.DoesNotExist: 48 | raise exceptions.AuthenticationFailed(_('Invalid api key.')) 49 | 50 | if not api_key.user.is_active: 51 | raise exceptions.AuthenticationFailed(_('User inactive or deleted.')) 52 | return api_key.user, api_key 53 | 54 | 55 | class TokenAuthentication(TokenAuthentication): 56 | """ 57 | Override classic TokenAuthentication processes to use WAVES dedicated user class 58 | """ 59 | model = WavesApiUser 60 | 61 | def authenticate(self, request): 62 | auth_tuple = super(TokenAuthentication, self).authenticate(request) 63 | if auth_tuple: 64 | WavesAuth.check_api_user(auth_tuple[1], request) 65 | return auth_tuple 66 | -------------------------------------------------------------------------------- /docs/user_doc/runner/runners.rst: -------------------------------------------------------------------------------- 1 | .. _runner-admin-label: 2 | 3 | ========================= 4 | Computing infrastructures 5 | ========================= 6 | 7 | Computing infrastructures shortly named "Runner" are the entry point where you can setup different configuration on where and how services' jobs 8 | may be run. 9 | 10 | 11 | Environment list 12 | ================ 13 | 14 | List all currently registered environment available on your WAVES application, you may add / edit / remove (cautiously). 15 | 16 | .. figure:: backoffice/runner-list.png 17 | :align: center 18 | :width: 90% 19 | :figclass: thumbnail 20 | 21 | List of environment already set 22 | 23 | Environment details 24 | =================== 25 | 26 | Main panel 27 | ---------- 28 | .. figure:: backoffice/runner-detail.png 29 | :align: center 30 | :width: 90% 31 | :figclass: thumbnail 32 | 33 | Detail admin page 34 | 35 | On detailed environment page, configure some descriptive parameters : 36 | 37 | - **Label**: The displayed runner name used in front / back-office for reference (used some time in templates) 38 | - **Run on**: Specify here which WAVES-core adapter is used for running jobs 39 | - **Connexion string**: The used connexion string (readonly) 40 | - **Reset related services**: When checked, upon save, all related services configuration is reset to defaults parameters 41 | 42 | .. CAUTION:: 43 | These services are now in stage 'Draft' 44 | 45 | .. _environment-set-up: 46 | 47 | Environment setup 48 | ----------------- 49 | .. figure:: backoffice/runner-param.png 50 | :align: center 51 | :width: 90% 52 | :figclass: thumbnail 53 | 54 | Computing infrastructure init parameters 55 | 56 | You can set 'run configuration' values such as login/password, destination host, etc... depending of the WAVES adapter you select in previous panel 57 | 58 | .. note:: 59 | You can't set up your environment till you have saved your initial configuration once. 60 | 61 | On the top left corner, once configured, a button allows you to test your parameters in order to verify if WAVES-core can actually connect to the Computing infrastructure. 62 | 63 | .. hint:: 64 | You can prevent subsequent service(s) to override a value in their own configuration administration page, by checking 'Prevent override' related checkbox. 65 | 66 | Running services 67 | ---------------- 68 | 69 | Down the page, there is a list of current services which use this Computing infrastructure. 70 | 71 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/admin/import/service_modal_form.html: -------------------------------------------------------------------------------- 1 | {% if form %} 2 |
    3 | {% endif %} 4 | 7 | 33 | 66 | {% if form %} 67 |
    68 | {% endif %} -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![Build Status](https://travis-ci.org/lirmm/waves-core.svg?branch=master)](https://travis-ci.org/lirmm/waves-core) 2 | 3 | README 4 | ====== 5 | 6 | What is WAVES for ? 7 | ------------------- 8 | 9 | WAVES stands for "\ **W**\ eb \ **A**\ pplication for \ **V**\ ersatile and \ **E**\ asy online bioinformatic \ **S**\ ervices." 10 | 11 | WAVES is a dedicated Django based web application to ease bioinformatic tools integration through web interfaces in 12 | order to provide the scientific community with bioinformatic services. 13 | 14 | It is aimed to help you easily present, publish and give access on the web to any bioinformatic tool. 15 | 16 | 17 | Features 18 | -------- 19 | 20 | - Create and manage your services execution over platform such as Galaxy, DRMAA clusters (SGE), Direct script execution, API calls to other services, remote calls to other platforms via ssh, etc. 21 | - Easily presents these tools in a nice frontend based on Bootstrap3, Bootstrap4 and soon Material Design 22 | - Follow and manage remote REST API access to your service platform 23 | - Manage user's access to your services 24 | 25 | .. note:: 26 | - WAVES main component is WAVES-core. 27 | - WAVES-Galaxy is another component. It is a WAVES adapter to interact with Galaxy instances. 28 | - WAVES-demo is a custom version of WAVES-core for demo purpose only. 29 | 30 | Side projects 31 | ------------- 32 | 33 | - WAVES-galaxy adapter ``_ 34 | - WAVES-demo project ``_ 35 | 36 | 37 | Installation 38 | ------------ 39 | 40 | - You can use WAVES-core as a stand alone application. 41 | - WAVES-core comply to standard reusable project layout for Django. So you may include it as a dependency in your own django project 42 | - Complete documentation available on `readthedocs `_ 43 | 44 | 45 | Support 46 | ------- 47 | 48 | If you are having issues, (or just want to say hello): we have a mailing list located at: waves@lirmm.fr 49 | 50 | 51 | -- UPDATE from <=1.6.4 -- 52 | ----------------------------------- 53 | 54 | If migrating from a version 1.6.4 or earlier to a most recent, you must patch you database encrypted password to a more reliable 55 | encrypted technology (https://github.com/pyca/cryptography). This script must be run once and only once ! 56 | To migrate your data please do the following: 57 | - retrieve the latest version 58 | - stop any running service 59 | - save your database first 60 | - once the new version is installed, run `manage.py update_keys` 61 | - if you don't have any error message: your keys are now more secured ! 62 | -------------------------------------------------------------------------------- /waves/wcore/templates/waves/services/services_list.html: -------------------------------------------------------------------------------- 1 | {% extends "waves/services/base.html" %} 2 | {% load waves_tags %} 3 | 4 | {% block content_main %} 5 |
    6 |
    7 |
    8 |
    9 |

    10 | Waves Services list 11 | Services currently available on this platform 12 |

    13 |
    14 |
    15 |
    16 |
    17 |
    18 |
    19 |
    20 | {% for service in available_services %} 21 |
    22 |

    23 | 24 | {{ service.name }} - {{ service.version }} 25 | 26 |

    27 |

    28 | Created 29 | by: {{ service.created_by }}
    30 | Current Status: {{ service.get_status_display }} 31 |

    32 |

    33 | {{ service.short_description|default:service.description|truncatechars:200 }}

    34 |

    35 | Released {{ service.created }}
    36 | Last update {{ service.updated }} 37 |

    38 |
    39 | {% online_exec_button service=service %} 40 | {% if user.is_staff %} 41 | Admin 43 | {% endif %} 44 |
    45 |
    46 |
    47 | {% empty %} 48 |
    49 |
    50 | Sorry no service available online for now 51 |
    52 |
    53 | {% endfor %} 54 |
    55 |
    56 |
    57 | {% endblock %} -------------------------------------------------------------------------------- /docs/extensions.rst: -------------------------------------------------------------------------------- 1 | ================ 2 | WAVES Extensions 3 | ================ 4 | 5 | 6 | WAVES team provides some useful extensions or side project to WAVES-core: 7 | 8 | 9 | Waves Demo 10 | ---------- 11 | 12 | Side project to show examples for extending WAVES-core to fit your needs: 13 | 14 | - Extends Services , Submission 15 | - Override Service / submissions templates 16 | - Use of REST API forms 17 | - Use a different skin for the back-office interface 18 | - Custom front-end interface 19 | - Override the Authentication class 20 | 21 | See `demo github `_ repo for more details 22 | 23 | 24 | Waves Galaxy 25 | ------------ 26 | 27 | Galaxy dedicated adaptor, allows import and execution of services on remote Galaxy instances 28 | 29 | See `galaxy-adaptors github `_ repo for more details 30 | See `Documentation `_ 31 | 32 | 33 | 34 | Just have a try 35 | ---------------- 36 | 37 | This is a `Singularity `_ image containing a functional WAVES installation including two pre-configured services ('Hello world' and 'PhyML'). 38 | It is a good way to test a fully operating WAVES-core instance. 39 | To be used with caution : all data will be lost when singularity instance is stopped. 40 | 41 | Singularity installation : on `Linux `_ or `Mac `_. 42 | 43 | Download WAVES test Singularity image : `wavetest.simg `_ 44 | 45 | Example for Linux Debian ditribution ( Ubuntu 16.04 or later ) : 46 | 47 | Install Singularity : 48 | 49 | .. code-block:: bash 50 | 51 | sudo wget -O- http://neuro.debian.net/lists/xenial.us-ca.full | sudo tee /etc/apt/sources.list.d/neurodebian.sources.list 52 | sudo apt-key adv --recv-keys --keyserver hkp://pool.sks-keyservers.net:80 0xA5D32F012649A5A9 53 | sudo apt-get update 54 | sudo apt-get install -y singularity-container 55 | 56 | Get and use wavestest.simg (caution, you need to be sudoer to start an instance) : 57 | 58 | .. code-block:: bash 59 | 60 | wget http://www.atgc-montpellier.fr/download/binaries/waves/wavestest.simg 61 | sudo singularity instance.start wavestest.simg waves 62 | sudo singularity run instance://waves 63 | 64 | When the instance is launched, WAVES-core is running. Open localhost:8000 on your favorite browser. Login with "admin" and "motdepasse". 65 | 66 | Power off : 67 | 68 | .. code-block:: bash 69 | 70 | sudo singularity instance.stop waves 71 | 72 | --------------------------------------------------------------------------------