├── .coveragerc ├── .gitattributes ├── .gitignore ├── .travis.yml ├── MANIFEST.in ├── README.rst ├── docs ├── Makefile ├── conf.py ├── drmaa.rst ├── index.rst ├── make.bat └── tutorials.rst ├── drmaa ├── __init__.py ├── const.py ├── errors.py ├── helpers.py ├── session.py ├── version.py └── wrappers.py ├── examples ├── example.py ├── example1.1.py ├── example1.py ├── example2.1.py ├── example2.py ├── example3.1.py ├── example3.2.py ├── example3.py ├── example4.py ├── example5.py ├── example6.py └── sleeper.sh ├── license.txt ├── setup.cfg ├── setup.py ├── test ├── testcontext.py ├── testmisc.py └── testwrap.py └── travis ├── host_template ├── install_sge.sh ├── queue_template ├── smp_template └── user_template /.coveragerc: -------------------------------------------------------------------------------- 1 | [run] 2 | source = drmaa 3 | omit = 4 | */python?.?/* 5 | */lib-python/?.?/*.py 6 | */lib_pypy/_*.py 7 | */site-packages/ordereddict.py 8 | */site-packages/nose/* 9 | */unittest2/* 10 | -------------------------------------------------------------------------------- /.gitattributes: -------------------------------------------------------------------------------- 1 | * text=auto 2 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Compiled python 2 | *.pyc 3 | *.pyo 4 | __pycache__ 5 | 6 | 7 | .DS_Store 8 | .coverage 9 | 10 | # C extensions 11 | *.so 12 | 13 | # Packages 14 | *.egg 15 | *.egg-info 16 | dist 17 | build 18 | eggs 19 | parts 20 | bin 21 | var 22 | sdist 23 | develop-eggs 24 | .installed.cfg 25 | lib 26 | lib64 27 | 28 | # Installer logs 29 | pip-log.txt 30 | 31 | # Unit test / coverage reports 32 | .coverage 33 | .tox 34 | nosetests.xml 35 | 36 | # Translations 37 | *.mo 38 | 39 | # Mr Developer 40 | .mr.developer.cfg 41 | .project 42 | .pydevproject 43 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | os: linux 2 | dist: precise 3 | 4 | language: python 5 | python: 6 | - 2.7 7 | - 3.4 8 | - 3.5 9 | - 3.6 10 | - 3.7-dev 11 | 12 | sudo: required 13 | 14 | addons: 15 | hostname: drmaa.travis-ci.org 16 | hosts: 17 | - localhost 18 | - localhost.localdomain 19 | - drmaa.travis-ci.org 20 | 21 | 22 | notifications: 23 | email: false 24 | 25 | cache: pip 26 | 27 | # Install stuff 28 | before_install: 29 | - travis/install_sge.sh 30 | - export SGE_ROOT=/var/lib/gridengine 31 | - export SGE_CELL=default 32 | - export DRMAA_LIBRARY_PATH=/usr/lib/libdrmaa.so.1.0 33 | - pip install --upgrade pip setuptools 34 | install: 35 | - pip install python-coveralls 36 | - pip install nose-cov 37 | - python setup.py install 38 | 39 | # Run test 40 | script: 41 | - nosetests -d -v --with-cov --cov drmaa --cov-config .coveragerc --logging-level=DEBUG 42 | 43 | # Calculate coverage 44 | after_success: 45 | - coveralls --config_file .coveragerc 46 | 47 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include license.txt 2 | include *.rst 3 | recursive-include examples * 4 | recursive-include test * 5 | recursive-include docs * 6 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | DRMAA Python 2 | ------------ 3 | 4 | .. image:: https://img.shields.io/travis/pygridtools/drmaa-python/stable.svg 5 | :alt: Build status 6 | :target: https://travis-ci.org/pygridtools/drmaa-python 7 | 8 | .. image:: https://img.shields.io/coveralls/pygridtools/drmaa-python/stable.svg 9 | :target: https://coveralls.io/r/pygridtools/drmaa-python 10 | 11 | .. image:: https://img.shields.io/pypi/dm/drmaa.svg 12 | :target: https://warehouse.python.org/project/drmaa/ 13 | :alt: PyPI downloads 14 | 15 | .. image:: https://img.shields.io/pypi/v/drmaa.svg 16 | :target: https://warehouse.python.org/project/drmaa/ 17 | :alt: Latest version on PyPI 18 | 19 | .. image:: https://img.shields.io/pypi/l/drmaa.svg 20 | :alt: License 21 | 22 | `Distributed Resource Management Application API `__ 23 | (DRMAA) bindings for Python. For more information 24 | `read the docs `__. 25 | 26 | If you simply want to run Python functions on a DRMAA-compatible grid, use 27 | `GridMap `__. 28 | 29 | Requirements 30 | ~~~~~~~~~~~~ 31 | 32 | - Python 2.7+ 33 | - A DRMAA-compatible cluster (e.g., Grid Engine) 34 | 35 | Installation 36 | ~~~~~~~~~~~~ 37 | 38 | To use the DRMAA Python library, you need to install it via ``pip``: 39 | 40 | 41 | .. code-block:: bash 42 | 43 | pip install drmaa 44 | 45 | 46 | and then setup any environment variables that are necessary for your particular DRM system. 47 | For SGE, this means ``SGE_ROOT`` and ``SGE_CELL``, which should be set as follows: 48 | 49 | 50 | .. code-block:: bash 51 | 52 | export SGE_ROOT=/path/to/gridengine 53 | export SGE_CELL=default 54 | 55 | 56 | where ``/path/to/gridengine/`` is replaced with the actual path to your Grid Engine installation, 57 | and ``default`` is replaced with your installation's actual cell. The path is typically 58 | ``/var/lib/gridengine``. 59 | 60 | You will also need access to the ``libdrmaa.so.1.0`` C library, which can often be installed as 61 | part of the ``libdrmaa-dev`` package on most Unixes. Once you have installed that, you may need to 62 | tell DRMAA Python where it is installed by setting the ``DRMAA_LIBRARY_PATH`` environment variable, 63 | if it is not installed in a location that Python usually looks for libraries. 64 | 65 | 66 | .. code-block:: bash 67 | 68 | export DRMAA_LIBRARY_PATH=/usr/lib/libdrmaa.so.1.0 69 | 70 | Acknowledgments 71 | ~~~~~~~~~~~~~~~ 72 | 73 | Thank you to `StatPro `__ and 74 | `Educational Testing Service `__ for 75 | funding the development of DRMAA Python. 76 | 77 | Changelog 78 | ~~~~~~~~~ 79 | 80 | `See GitHub releases `__. 81 | -------------------------------------------------------------------------------- /docs/Makefile: -------------------------------------------------------------------------------- 1 | # Makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | PAPER = 8 | BUILDDIR = _build 9 | 10 | # Internal variables. 11 | PAPEROPT_a4 = -D latex_paper_size=a4 12 | PAPEROPT_letter = -D latex_paper_size=letter 13 | ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 14 | # the i18n builder cannot share the environment and doctrees with the others 15 | I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 16 | 17 | .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext 18 | 19 | help: 20 | @echo "Please use \`make ' where is one of" 21 | @echo " html to make standalone HTML files" 22 | @echo " dirhtml to make HTML files named index.html in directories" 23 | @echo " singlehtml to make a single large HTML file" 24 | @echo " pickle to make pickle files" 25 | @echo " json to make JSON files" 26 | @echo " htmlhelp to make HTML files and a HTML help project" 27 | @echo " qthelp to make HTML files and a qthelp project" 28 | @echo " devhelp to make HTML files and a Devhelp project" 29 | @echo " epub to make an epub" 30 | @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" 31 | @echo " latexpdf to make LaTeX files and run them through pdflatex" 32 | @echo " text to make text files" 33 | @echo " man to make manual pages" 34 | @echo " texinfo to make Texinfo files" 35 | @echo " info to make Texinfo files and run them through makeinfo" 36 | @echo " gettext to make PO message catalogs" 37 | @echo " changes to make an overview of all changed/added/deprecated items" 38 | @echo " linkcheck to check all external links for integrity" 39 | @echo " doctest to run all doctests embedded in the documentation (if enabled)" 40 | 41 | clean: 42 | -rm -rf $(BUILDDIR)/* 43 | 44 | html: 45 | $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html 46 | @echo 47 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." 48 | 49 | dirhtml: 50 | $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml 51 | @echo 52 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." 53 | 54 | singlehtml: 55 | $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml 56 | @echo 57 | @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." 58 | 59 | pickle: 60 | $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle 61 | @echo 62 | @echo "Build finished; now you can process the pickle files." 63 | 64 | json: 65 | $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json 66 | @echo 67 | @echo "Build finished; now you can process the JSON files." 68 | 69 | htmlhelp: 70 | $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp 71 | @echo 72 | @echo "Build finished; now you can run HTML Help Workshop with the" \ 73 | ".hhp project file in $(BUILDDIR)/htmlhelp." 74 | 75 | qthelp: 76 | $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp 77 | @echo 78 | @echo "Build finished; now you can run "qcollectiongenerator" with the" \ 79 | ".qhcp project file in $(BUILDDIR)/qthelp, like this:" 80 | @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/DRMAAPython.qhcp" 81 | @echo "To view the help file:" 82 | @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/DRMAAPython.qhc" 83 | 84 | devhelp: 85 | $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp 86 | @echo 87 | @echo "Build finished." 88 | @echo "To view the help file:" 89 | @echo "# mkdir -p $$HOME/.local/share/devhelp/DRMAAPython" 90 | @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/DRMAAPython" 91 | @echo "# devhelp" 92 | 93 | epub: 94 | $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub 95 | @echo 96 | @echo "Build finished. The epub file is in $(BUILDDIR)/epub." 97 | 98 | latex: 99 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 100 | @echo 101 | @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." 102 | @echo "Run \`make' in that directory to run these through (pdf)latex" \ 103 | "(use \`make latexpdf' here to do that automatically)." 104 | 105 | latexpdf: 106 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 107 | @echo "Running LaTeX files through pdflatex..." 108 | $(MAKE) -C $(BUILDDIR)/latex all-pdf 109 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 110 | 111 | text: 112 | $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text 113 | @echo 114 | @echo "Build finished. The text files are in $(BUILDDIR)/text." 115 | 116 | man: 117 | $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man 118 | @echo 119 | @echo "Build finished. The manual pages are in $(BUILDDIR)/man." 120 | 121 | texinfo: 122 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 123 | @echo 124 | @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." 125 | @echo "Run \`make' in that directory to run these through makeinfo" \ 126 | "(use \`make info' here to do that automatically)." 127 | 128 | info: 129 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 130 | @echo "Running Texinfo files through makeinfo..." 131 | make -C $(BUILDDIR)/texinfo info 132 | @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." 133 | 134 | gettext: 135 | $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale 136 | @echo 137 | @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." 138 | 139 | changes: 140 | $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes 141 | @echo 142 | @echo "The overview file is in $(BUILDDIR)/changes." 143 | 144 | linkcheck: 145 | $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck 146 | @echo 147 | @echo "Link check complete; look for any errors in the above output " \ 148 | "or in $(BUILDDIR)/linkcheck/output.txt." 149 | 150 | doctest: 151 | $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest 152 | @echo "Testing of doctests in the sources finished, look at the " \ 153 | "results in $(BUILDDIR)/doctest/output.txt." 154 | -------------------------------------------------------------------------------- /docs/conf.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # DRMAA Python documentation build configuration file, created by 4 | # sphinx-quickstart on Wed Nov 27 13:52:19 2013. 5 | # 6 | # This file is execfile()d with the current directory set to its containing dir. 7 | # 8 | # Note that not all possible configuration values are present in this 9 | # autogenerated file. 10 | # 11 | # All configuration values have a default; values that are commented out 12 | # serve to show the default. 13 | 14 | import os 15 | import sys 16 | 17 | 18 | # Handle modules that will not import properly because they rely on C 19 | class Mock(object): 20 | def __init__(self, *args, **kwargs): 21 | pass 22 | 23 | def __call__(self, *args, **kwargs): 24 | return Mock() 25 | 26 | @classmethod 27 | def __getattr__(cls, name): 28 | if name in ('__file__', '__path__'): 29 | return '/dev/null' 30 | elif name[0] == name[0].upper(): 31 | mockType = type(name, (), {}) 32 | mockType.__module__ = __name__ 33 | return mockType 34 | else: 35 | return Mock() 36 | 37 | MOCK_MODULES = ['drmaa.wrappers'] 38 | for mod_name in MOCK_MODULES: 39 | sys.modules[mod_name] = Mock() 40 | 41 | 42 | # If extensions (or modules to document with autodoc) are in another directory, 43 | # add these directories to sys.path here. If the directory is relative to the 44 | # documentation root, use os.path.abspath to make it absolute, like shown here. 45 | sys.path.insert(0, os.path.abspath('..')) 46 | 47 | import drmaa 48 | 49 | # -- General configuration ----------------------------------------------------- 50 | 51 | # If your documentation needs a minimal Sphinx version, state it here. 52 | #needs_sphinx = '1.0' 53 | 54 | # Add any Sphinx extension module names here, as strings. They can be extensions 55 | # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. 56 | extensions = ['sphinx.ext.autodoc', 'sphinx.ext.todo', 'sphinx.ext.coverage', 57 | 'sphinx.ext.mathjax', 'sphinx.ext.viewcode'] 58 | 59 | # Add any paths that contain templates here, relative to this directory. 60 | templates_path = ['_templates'] 61 | 62 | # The suffix of source filenames. 63 | source_suffix = '.rst' 64 | 65 | # The encoding of source files. 66 | #source_encoding = 'utf-8-sig' 67 | 68 | # The master toctree document. 69 | master_doc = 'index' 70 | 71 | # General information about the project. 72 | project = u'DRMAA Python' 73 | copyright = u'2013, Dan Blanchard, David Ressman, Enrico Sirola' 74 | 75 | # The version info for the project you're documenting, acts as replacement for 76 | # |version| and |release|, also used in various other places throughout the 77 | # built documents. 78 | # 79 | # The short X.Y version. 80 | version = drmaa.__version__ 81 | # The full version, including alpha/beta/rc tags. 82 | release = drmaa.__version__ 83 | 84 | # The language for content autogenerated by Sphinx. Refer to documentation 85 | # for a list of supported languages. 86 | #language = None 87 | 88 | # There are two options for replacing |today|: either, you set today to some 89 | # non-false value, then it is used: 90 | #today = '' 91 | # Else, today_fmt is used as the format for a strftime call. 92 | #today_fmt = '%B %d, %Y' 93 | 94 | # List of patterns, relative to source directory, that match files and 95 | # directories to ignore when looking for source files. 96 | exclude_patterns = ['_build'] 97 | 98 | # The reST default role (used for this markup: `text`) to use for all documents. 99 | #default_role = None 100 | 101 | # If true, '()' will be appended to :func: etc. cross-reference text. 102 | #add_function_parentheses = True 103 | 104 | # If true, the current module name will be prepended to all description 105 | # unit titles (such as .. function::). 106 | #add_module_names = True 107 | 108 | # If true, sectionauthor and moduleauthor directives will be shown in the 109 | # output. They are ignored by default. 110 | #show_authors = False 111 | 112 | # The name of the Pygments (syntax highlighting) style to use. 113 | pygments_style = 'sphinx' 114 | 115 | # A list of ignored prefixes for module index sorting. 116 | #modindex_common_prefix = [] 117 | 118 | 119 | # -- Options for HTML output --------------------------------------------------- 120 | 121 | # The theme to use for HTML and HTML Help pages. See the documentation for 122 | # a list of builtin themes. 123 | html_theme = 'default' 124 | 125 | # Theme options are theme-specific and customize the look and feel of a theme 126 | # further. For a list of options available for each theme, see the 127 | # documentation. 128 | #html_theme_options = {} 129 | 130 | # Add any paths that contain custom themes here, relative to this directory. 131 | #html_theme_path = [] 132 | 133 | # The name for this set of Sphinx documents. If None, it defaults to 134 | # " v documentation". 135 | #html_title = None 136 | 137 | # A shorter title for the navigation bar. Default is the same as html_title. 138 | #html_short_title = None 139 | 140 | # The name of an image file (relative to this directory) to place at the top 141 | # of the sidebar. 142 | #html_logo = None 143 | 144 | # The name of an image file (within the static path) to use as favicon of the 145 | # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 146 | # pixels large. 147 | #html_favicon = None 148 | 149 | # Add any paths that contain custom static files (such as style sheets) here, 150 | # relative to this directory. They are copied after the builtin static files, 151 | # so a file named "default.css" will overwrite the builtin "default.css". 152 | # html_static_path = ['_static'] 153 | 154 | # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, 155 | # using the given strftime format. 156 | #html_last_updated_fmt = '%b %d, %Y' 157 | 158 | # If true, SmartyPants will be used to convert quotes and dashes to 159 | # typographically correct entities. 160 | #html_use_smartypants = True 161 | 162 | # Custom sidebar templates, maps document names to template names. 163 | #html_sidebars = {} 164 | 165 | # Additional templates that should be rendered to pages, maps page names to 166 | # template names. 167 | #html_additional_pages = {} 168 | 169 | # If false, no module index is generated. 170 | #html_domain_indices = True 171 | 172 | # If false, no index is generated. 173 | #html_use_index = True 174 | 175 | # If true, the index is split into individual pages for each letter. 176 | #html_split_index = False 177 | 178 | # If true, links to the reST sources are added to the pages. 179 | #html_show_sourcelink = True 180 | 181 | # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. 182 | #html_show_sphinx = True 183 | 184 | # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. 185 | #html_show_copyright = True 186 | 187 | # If true, an OpenSearch description file will be output, and all pages will 188 | # contain a tag referring to it. The value of this option must be the 189 | # base URL from which the finished HTML is served. 190 | #html_use_opensearch = '' 191 | 192 | # This is the file name suffix for HTML files (e.g. ".xhtml"). 193 | #html_file_suffix = None 194 | 195 | # Output file base name for HTML help builder. 196 | htmlhelp_basename = 'DRMAAPythondoc' 197 | 198 | 199 | # -- Options for LaTeX output -------------------------------------------------- 200 | 201 | latex_elements = { 202 | # The paper size ('letterpaper' or 'a4paper'). 203 | #'papersize': 'letterpaper', 204 | 205 | # The font size ('10pt', '11pt' or '12pt'). 206 | #'pointsize': '10pt', 207 | 208 | # Additional stuff for the LaTeX preamble. 209 | #'preamble': '', 210 | } 211 | 212 | # Grouping the document tree into LaTeX files. List of tuples 213 | # (source start file, target name, title, author, documentclass [howto/manual]). 214 | latex_documents = [ 215 | ('index', 'DRMAAPython.tex', u'DRMAA Python Documentation', 216 | u'Dan Blanchard, David Ressman, Enrico Sirola', 'manual'), 217 | ] 218 | 219 | # The name of an image file (relative to this directory) to place at the top of 220 | # the title page. 221 | #latex_logo = None 222 | 223 | # For "manual" documents, if this is true, then toplevel headings are parts, 224 | # not chapters. 225 | #latex_use_parts = False 226 | 227 | # If true, show page references after internal links. 228 | #latex_show_pagerefs = False 229 | 230 | # If true, show URL addresses after external links. 231 | #latex_show_urls = False 232 | 233 | # Documents to append as an appendix to all manuals. 234 | #latex_appendices = [] 235 | 236 | # If false, no module index is generated. 237 | #latex_domain_indices = True 238 | 239 | 240 | # -- Options for manual page output -------------------------------------------- 241 | 242 | # One entry per manual page. List of tuples 243 | # (source start file, name, description, authors, manual section). 244 | man_pages = [ 245 | ('index', 'drmaapython', u'DRMAA Python Documentation', 246 | [u'Dan Blanchard, David Ressman, Enrico Sirola'], 1) 247 | ] 248 | 249 | # If true, show URL addresses after external links. 250 | #man_show_urls = False 251 | 252 | 253 | # -- Options for Texinfo output ------------------------------------------------ 254 | 255 | # Grouping the document tree into Texinfo files. List of tuples 256 | # (source start file, target name, title, author, 257 | # dir menu entry, description, category) 258 | texinfo_documents = [ 259 | ('index', 'DRMAAPython', u'DRMAA Python Documentation', 260 | u'Dan Blanchard, David Ressman, Enrico Sirola', 'DRMAAPython', 'One line description of project.', 261 | 'Miscellaneous'), 262 | ] 263 | 264 | # Documents to append as an appendix to all manuals. 265 | #texinfo_appendices = [] 266 | 267 | # If false, no module index is generated. 268 | #texinfo_domain_indices = True 269 | 270 | # How to display URL addresses: 'footnote', 'no', or 'inline'. 271 | #texinfo_show_urls = 'footnote' 272 | -------------------------------------------------------------------------------- /docs/drmaa.rst: -------------------------------------------------------------------------------- 1 | :mod:`drmaa` Package 2 | ---------------------- 3 | 4 | .. automodule:: drmaa 5 | :members: 6 | :undoc-members: 7 | :show-inheritance: 8 | -------------------------------------------------------------------------------- /docs/index.rst: -------------------------------------------------------------------------------- 1 | .. DRMAA Python documentation master file, created by 2 | sphinx-quickstart on Wed Nov 27 13:52:19 2013. 3 | You can adapt this file completely to your liking, but it should at least 4 | contain the root `toctree` directive. 5 | 6 | Welcome to DRMAA Python's documentation! 7 | ======================================== 8 | 9 | 10 | 11 | Contents: 12 | 13 | .. toctree:: 14 | :maxdepth: 2 15 | 16 | tutorials 17 | drmaa 18 | 19 | 20 | Indices and tables 21 | ================== 22 | 23 | * :ref:`genindex` 24 | * :ref:`modindex` 25 | * :ref:`search` 26 | 27 | -------------------------------------------------------------------------------- /docs/make.bat: -------------------------------------------------------------------------------- 1 | @ECHO OFF 2 | 3 | REM Command file for Sphinx documentation 4 | 5 | if "%SPHINXBUILD%" == "" ( 6 | set SPHINXBUILD=sphinx-build 7 | ) 8 | set BUILDDIR=_build 9 | set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% . 10 | set I18NSPHINXOPTS=%SPHINXOPTS% . 11 | if NOT "%PAPER%" == "" ( 12 | set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS% 13 | set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS% 14 | ) 15 | 16 | if "%1" == "" goto help 17 | 18 | if "%1" == "help" ( 19 | :help 20 | echo.Please use `make ^` where ^ is one of 21 | echo. html to make standalone HTML files 22 | echo. dirhtml to make HTML files named index.html in directories 23 | echo. singlehtml to make a single large HTML file 24 | echo. pickle to make pickle files 25 | echo. json to make JSON files 26 | echo. htmlhelp to make HTML files and a HTML help project 27 | echo. qthelp to make HTML files and a qthelp project 28 | echo. devhelp to make HTML files and a Devhelp project 29 | echo. epub to make an epub 30 | echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter 31 | echo. text to make text files 32 | echo. man to make manual pages 33 | echo. texinfo to make Texinfo files 34 | echo. gettext to make PO message catalogs 35 | echo. changes to make an overview over all changed/added/deprecated items 36 | echo. linkcheck to check all external links for integrity 37 | echo. doctest to run all doctests embedded in the documentation if enabled 38 | goto end 39 | ) 40 | 41 | if "%1" == "clean" ( 42 | for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i 43 | del /q /s %BUILDDIR%\* 44 | goto end 45 | ) 46 | 47 | if "%1" == "html" ( 48 | %SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html 49 | if errorlevel 1 exit /b 1 50 | echo. 51 | echo.Build finished. The HTML pages are in %BUILDDIR%/html. 52 | goto end 53 | ) 54 | 55 | if "%1" == "dirhtml" ( 56 | %SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml 57 | if errorlevel 1 exit /b 1 58 | echo. 59 | echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml. 60 | goto end 61 | ) 62 | 63 | if "%1" == "singlehtml" ( 64 | %SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml 65 | if errorlevel 1 exit /b 1 66 | echo. 67 | echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml. 68 | goto end 69 | ) 70 | 71 | if "%1" == "pickle" ( 72 | %SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle 73 | if errorlevel 1 exit /b 1 74 | echo. 75 | echo.Build finished; now you can process the pickle files. 76 | goto end 77 | ) 78 | 79 | if "%1" == "json" ( 80 | %SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json 81 | if errorlevel 1 exit /b 1 82 | echo. 83 | echo.Build finished; now you can process the JSON files. 84 | goto end 85 | ) 86 | 87 | if "%1" == "htmlhelp" ( 88 | %SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp 89 | if errorlevel 1 exit /b 1 90 | echo. 91 | echo.Build finished; now you can run HTML Help Workshop with the ^ 92 | .hhp project file in %BUILDDIR%/htmlhelp. 93 | goto end 94 | ) 95 | 96 | if "%1" == "qthelp" ( 97 | %SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp 98 | if errorlevel 1 exit /b 1 99 | echo. 100 | echo.Build finished; now you can run "qcollectiongenerator" with the ^ 101 | .qhcp project file in %BUILDDIR%/qthelp, like this: 102 | echo.^> qcollectiongenerator %BUILDDIR%\qthelp\DRMAAPython.qhcp 103 | echo.To view the help file: 104 | echo.^> assistant -collectionFile %BUILDDIR%\qthelp\DRMAAPython.ghc 105 | goto end 106 | ) 107 | 108 | if "%1" == "devhelp" ( 109 | %SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp 110 | if errorlevel 1 exit /b 1 111 | echo. 112 | echo.Build finished. 113 | goto end 114 | ) 115 | 116 | if "%1" == "epub" ( 117 | %SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub 118 | if errorlevel 1 exit /b 1 119 | echo. 120 | echo.Build finished. The epub file is in %BUILDDIR%/epub. 121 | goto end 122 | ) 123 | 124 | if "%1" == "latex" ( 125 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 126 | if errorlevel 1 exit /b 1 127 | echo. 128 | echo.Build finished; the LaTeX files are in %BUILDDIR%/latex. 129 | goto end 130 | ) 131 | 132 | if "%1" == "text" ( 133 | %SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text 134 | if errorlevel 1 exit /b 1 135 | echo. 136 | echo.Build finished. The text files are in %BUILDDIR%/text. 137 | goto end 138 | ) 139 | 140 | if "%1" == "man" ( 141 | %SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man 142 | if errorlevel 1 exit /b 1 143 | echo. 144 | echo.Build finished. The manual pages are in %BUILDDIR%/man. 145 | goto end 146 | ) 147 | 148 | if "%1" == "texinfo" ( 149 | %SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo 150 | if errorlevel 1 exit /b 1 151 | echo. 152 | echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo. 153 | goto end 154 | ) 155 | 156 | if "%1" == "gettext" ( 157 | %SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale 158 | if errorlevel 1 exit /b 1 159 | echo. 160 | echo.Build finished. The message catalogs are in %BUILDDIR%/locale. 161 | goto end 162 | ) 163 | 164 | if "%1" == "changes" ( 165 | %SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes 166 | if errorlevel 1 exit /b 1 167 | echo. 168 | echo.The overview file is in %BUILDDIR%/changes. 169 | goto end 170 | ) 171 | 172 | if "%1" == "linkcheck" ( 173 | %SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck 174 | if errorlevel 1 exit /b 1 175 | echo. 176 | echo.Link check complete; look for any errors in the above output ^ 177 | or in %BUILDDIR%/linkcheck/output.txt. 178 | goto end 179 | ) 180 | 181 | if "%1" == "doctest" ( 182 | %SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest 183 | if errorlevel 1 exit /b 1 184 | echo. 185 | echo.Testing of doctests in the sources finished, look at the ^ 186 | results in %BUILDDIR%/doctest/output.txt. 187 | goto end 188 | ) 189 | 190 | :end 191 | -------------------------------------------------------------------------------- /docs/tutorials.rst: -------------------------------------------------------------------------------- 1 | Distributed Resource Management Application API 2 | =============================================== 3 | 4 | This guide is a tutorial for getting started programming with DRMAA. It is 5 | basically a one to one translation of the original in C for Grid Engine. It 6 | assumes that you already know what DRMAA is and that you have drmaa-python 7 | installed. If not, have a look at Installing. The following code segments are 8 | also included in the repository. 9 | 10 | Starting and Stopping a Session 11 | ------------------------------- 12 | 13 | The following two code segments show the most basic DRMAA python binding 14 | program. 15 | 16 | .. code-block:: python 17 | :linenos: 18 | :caption: example1.py 19 | :name: example-1 20 | 21 | #!/usr/bin/env python 22 | 23 | import drmaa 24 | 25 | def main(): 26 | """Create a drmaa session and exit""" 27 | with drmaa.Session() as s: 28 | print('A session was started successfully') 29 | 30 | if __name__=='__main__': 31 | main() 32 | 33 | The first thing to notice is that every call to a DRMAA function will return an 34 | error code. In this tutorial, we ignore all error codes. 35 | 36 | Now let's look at the functions being called. First, on line 7, we initialise a 37 | ``Session`` object by calling ``drmaa.Session()``. The ``Session`` is 38 | automatically initialized via ``initialize()``, and it creates a session and starts 39 | an event client listener thread. The session is used for organizing jobs 40 | submitted through DRMAA, and the thread is used to receive updates from the 41 | queue master about the state of jobs and the system in general. Once 42 | ``initialize()`` has been called successfully, it is the responsibility of the 43 | calling application to also call ``exit()`` before terminating, but since we're 44 | using it as a context manager via the ``with`` statement, that will happene 45 | automatically. If an application does not call ``exit()`` before terminating, 46 | session state may be left behind in the user's home directory, and the queue 47 | master may be left with a dead event client handle, which can decrease queue 48 | master performance. 49 | 50 | At the end of our program, ``exit()`` is called automatically when we leave the 51 | ``with`` block. ``exit()`` cleans up the session and stops the event client 52 | listener thread. Most other DRMAA functions must be called before ``exit()``. 53 | Some functions, like ``getContact()``, can be called after ``exit()``, but these 54 | functions only provide general information. Any function that does work, such as 55 | ``runJob()`` or ``wait()`` must be called before ``exit()`` is called. If such a 56 | function is called after ``exit()`` is called, it will return an error. 57 | 58 | .. code-block:: python 59 | :linenos: 60 | :caption: example1.1.py 61 | :name: example-1-1 62 | :emphasize-lines: 13,15,18,19 63 | 64 | #!/usr/bin/env python 65 | 66 | import drmaa 67 | 68 | def main(): 69 | """ 70 | Create a session, show that each session has an ID, use session ID to 71 | disconnect, then reconnect. Finally, exit. 72 | """ 73 | s = drmaa.Session() 74 | s.initialize() 75 | print('A session was started successfully') 76 | response = s.contact 77 | print('session contact returns: %s' % response) 78 | s.exit() 79 | print('Exited from session') 80 | 81 | s.initialize(response) 82 | print('Session was restarted successfullly') 83 | s.exit() 84 | 85 | 86 | if __name__=='__main__': 87 | main() 88 | 89 | This example is very similar to :ref:`Example 1 `. The difference is 90 | that it uses the Grid Engine feature of reconnectable sessions. The DRMAA 91 | concept of a session is translated into a session tag in the Grid Engine job 92 | structure. That means that every job knows to which session it belongs. With 93 | reconnectable sessions, it's possible to initialize the DRMAA library to a 94 | previous session, allowing the library access to that session's job list. The 95 | only limitation, though, is that jobs which end between the calls to ``exit()`` 96 | and ``init()`` will be lost, as the reconnecting session will no longer see 97 | these jobs, and so won't know about them. 98 | 99 | On line 13, we use the contact attribute to get the contact information for this 100 | session. On line 15 we then exit the session. On line 18, we use the stored 101 | contact information to reconnect to the previous session. Had we submitted jobs 102 | before calling ``exit()``, those jobs would now be available again for 103 | operations such as ``wait()`` and ``synchronize()``. Finally, on line 19 we exit 104 | the session a second time. 105 | 106 | Running a Job 107 | ------------- 108 | 109 | The following code segments show how to use the DRMAA python binding to 110 | submit a job to Grid Engine. It submits a small shell script (``sleeper.sh``) 111 | which takes two arguments: 112 | 113 | .. code-block:: bash 114 | 115 | #!/bin/bash 116 | echo "Hello world, the answer is $1" 117 | sleep 3s 118 | echo "$2 Bye world!" 119 | 120 | .. code-block:: python 121 | :linenos: 122 | :caption: example2.py 123 | :name: example-2 124 | :emphasize-lines: 12-22 125 | 126 | #!/usr/bin/env python 127 | 128 | import drmaa 129 | import os 130 | 131 | def main(): 132 | """ 133 | Submit a job. 134 | Note, need file called sleeper.sh in current directory. 135 | """ 136 | with drmaa.Session() as s: 137 | print('Creating job template') 138 | jt = s.createJobTemplate() 139 | jt.remoteCommand = os.path.join(os.getcwd(), 'sleeper.sh') 140 | jt.args = ['42', 'Simon says:'] 141 | jt.joinFiles=True 142 | 143 | jobid = s.runJob(jt) 144 | print('Your job has been submitted with ID %s' % jobid) 145 | 146 | print('Cleaning up') 147 | s.deleteJobTemplate(jt) 148 | 149 | if __name__=='__main__': 150 | main() 151 | 152 | The beginning and end of this program are the same as the first example. What's 153 | new is in lines 12--22. On line 12 we ask DRMAA to allocate a job template 154 | for us. A job template is a structure used to store information about a job to 155 | be submitted. The same template can be reused for multiple calls to ``runJob()`` 156 | or ``runBulkJob()``. 157 | 158 | On line 13 we set the ``remoteCommand`` attribute. This attribute tells DRMAA 159 | where to find the program we want to run. Its value is the path to the 160 | executable. The path can be either relative or absolute. If relative, it is 161 | relative to the ``workingDirectory`` attribute, which if not set defaults to the 162 | user's home directory. For more information on DRMAA attributes, please see the 163 | attributes man page. Note that for this program to work, the script 164 | ``sleeper.sh`` must be in the current directory. 165 | 166 | On line 14 we set the ``args`` attribute. This attribute tells DRMAA what 167 | arguments to pass to the executable. 168 | 169 | On line 17 we submit the job with ``runJob()``. DRMAA will place the ID assigned 170 | to the job into the character array we passed to ``runJob()``. The job is now 171 | running as though submitted by ``qsub`` or ``bsub``. At this point calling 172 | ``exit()`` and/or terminating the program will have no effect on the job. 173 | 174 | To clean things up, we delete the job template on line 21. This frees the memory 175 | DRMAA set aside for the job template, but has no effect on submitted jobs. 176 | 177 | If instead of a single job we had wanted to submit an array job, we could have 178 | replaced the else on line 18 and 19 with the following: 179 | 180 | .. code-block:: python 181 | :caption: example2.1.py 182 | :name: example-2-1 183 | 184 | jobid = s.runBulkJobs(jt, 1, 30, 2) 185 | print('Your jobs have been submitted with IDs %s' % jobid) 186 | 187 | This code segment submits an array job with 15 tasks numbered 1, 3, 5, 7, etc. 188 | An important difference to note is that ``runBulkJobs()`` returns the job ids in 189 | a list. On the last line, we print all the job ids. 190 | 191 | Waiting for a Job 192 | ----------------- 193 | 194 | Now we're going to extend our example to include waiting for a job to finish. 195 | 196 | 197 | .. code-block:: python 198 | :linenos: 199 | :caption: example3.py 200 | :name: example-3 201 | :emphasize-lines: 21-22 202 | 203 | #!/usr/bin/env python 204 | 205 | import drmaa 206 | import os 207 | 208 | def main(): 209 | """ 210 | Submit a job and wait for it to finish. 211 | Note, need file called sleeper.sh in home directory. 212 | """ 213 | with drmaa.Session() as s: 214 | print('Creating job template') 215 | jt = s.createJobTemplate() 216 | jt.remoteCommand = os.path.join(os.getcwd(), 'sleeper.sh') 217 | jt.args = ['42', 'Simon says:'] 218 | jt.joinFiles = True 219 | 220 | jobid = s.runJob(jt) 221 | print('Your job has been submitted with ID %s' % jobid) 222 | 223 | retval = s.wait(jobid, drmaa.Session.TIMEOUT_WAIT_FOREVER) 224 | print('Job: {0} finished with status {1}'.format(retval.jobId, retval.hasExited)) 225 | 226 | print('Cleaning up') 227 | s.deleteJobTemplate(jt) 228 | 229 | if __name__=='__main__': 230 | main() 231 | 232 | 233 | This example is very similar to :ref:`Example 2 ` except for line 234 | 21. On line 21 we call ``wait()`` to wait for the job to end. We have to give 235 | ``wait()`` both the ID of the job for which we want to wait, and also how long 236 | we are willing to wait for the job to finish. This could be a number of 237 | seconds, or it could be either ``TIMEOUT_WAIT_FOREVER`` or ``TIMEOUT_NO_WAIT``. 238 | ``wait()`` returns a ``JobInfo`` tuple, which has the following attributes: 239 | ``jobId``, ``hasExited``, ``hasSignal``, ``terminatedSignal``, ``hasCoreDump``, 240 | ``wasAborted``, ``exitStatus``, and ``resourceUsage``. ``jobId`` is 241 | particularly useful if we passed in ``JOB_IDS_SESSION_ANY`` as the ID argument 242 | for ``wait()``, because without it we would have no way of knowing which job it 243 | actually waited for. Lastly, we print out the job ID and the exit status on 244 | line 22. 245 | 246 | An alternative to ``wait()`` when working with multiple jobs, such as jobs 247 | submitted by ``runBulkJobs()`` or multiple calls to ``runJob()`` is 248 | ``synchronize()``. ``synchronize()`` waits for a set of jobs to finish. To use 249 | ``synchronize()``, we could replace lines 18--22 with the following: 250 | 251 | .. code-block:: python 252 | :caption: example3.1.py 253 | :name: example-3-1 254 | 255 | joblist = s.runBulkJobs(jt, 1, 30, 2) 256 | print('Your jobs have been submitted with IDs %s' % joblist) 257 | 258 | s.synchronize(joblist, drmaa.Session.TIMEOUT_WAIT_FOREVER, True) 259 | 260 | 261 | With the switch to ``runBulkJobs()`` we now have several jobs for which to wait. 262 | On line 20, instead of calling ``wait()``, we call ``synchronize()``. 263 | ``synchronize()`` takes only three interesting parameters. The first is the list 264 | of string IDs for which to wait. If the special ID, ``JOB_IDS_SESSION_ALL``, 265 | appears in the array, ``synchronize()`` will wait for all jobs submitted via 266 | DRMAA during this session (i.e., since ``initialize()`` was called). The second 267 | is how long to wait for all the jobs in the list to finish. This is the same as 268 | the timeout parameter for ``wait()``. The third is whether this call to 269 | ``synchronize()`` should clean up after the job. After a job completes, it 270 | leaves behind accounting information, such as exit status and usage, until 271 | either ``wait()`` or ``synchronize()`` with ``dispose`` set to ``True`` is 272 | called. It is the responsibility of the application to make sure one of these 273 | two functions is called for every job. Not doing so creates a memory leak. Note 274 | that calling ``synchronize()`` with dispose set to true flushes all accounting 275 | information for all jobs in the list. If you want to use ``synchronize()`` and 276 | still recover the accounting information, set ``dispose`` to ``False`` and call 277 | ``wait()`` for each job. To do this in :ref:`Example 3 `, we would 278 | replace lines 18--22 with the following: 279 | 280 | .. code-block:: python 281 | :caption: example3.2.py 282 | :name: example-3-2 283 | 284 | joblist = s.runBulkJobs(jt, 1, 30, 2) 285 | print('Your jobs have been submitted with IDs %s' % joblist) 286 | 287 | s.synchronize(joblist, drmaa.Session.TIMEOUT_WAIT_FOREVER, False) 288 | for curjob in joblist: 289 | print('Collecting job ' + curjob) 290 | retval = s.wait(curjob, drmaa.Session.TIMEOUT_WAIT_FOREVER) 291 | print('Job: {0} finished with status {1}'.format(retval.jobId, 292 | retval.hasExited)) 293 | 294 | 295 | What's different is that on line 21 we set ``dispose`` to ``False``, and then on 296 | lines 22--26 we wait once for each job, printing the exit status and usage 297 | information as we did in :ref:`Example 3 `. 298 | 299 | We pass ``joblist`` to ``synchronize()`` to wait for each job specifically. 300 | Otherwise, the ``wait()`` could end up waiting for a job submitted after the 301 | call to ``synchronize()``. 302 | 303 | Controlling a Job 304 | ----------------- 305 | 306 | Now let's look at an example of how to control a job from DRMAA: 307 | 308 | .. code-block:: python 309 | :linenos: 310 | :caption: example4.py 311 | :name: example-4 312 | :emphasize-lines: 20 313 | 314 | #!/usr/bin/env python 315 | 316 | import drmaa 317 | import os 318 | 319 | def main(): 320 | """Submit a job, then kill it. 321 | Note, need file called sleeper.sh in home directory. 322 | """ 323 | with drmaa.Session() as s: 324 | print('Creating job template') 325 | jt = s.createJobTemplate() 326 | jt.remoteCommand = os.path.join(os.getcwd(), 'sleeper.sh') 327 | jt.args = ['42', 'Simon says:'] 328 | jt.joinFiles = True 329 | 330 | jobid = s.runJob(jt) 331 | print('Your job has been submitted with ID %s' % jobid) 332 | # options are: SUSPEND, RESUME, HOLD, RELEASE, TERMINATE 333 | s.control(jobid, drmaa.JobControlAction.TERMINATE) 334 | 335 | print('Cleaning up') 336 | s.deleteJobTemplate(jt) 337 | 338 | if __name__=='__main__': 339 | main() 340 | 341 | 342 | This example is very similar to :ref:`Example 2 ` except for line 343 | 20. On line 20 we use ``control()`` to delete the job we just submitted. Aside 344 | from deleting the job, we could have also used ``control()`` to suspend, 345 | resume, hold, or release it. 346 | 347 | Note that ``control()`` can be used to control jobs not submitted through DRMAA. 348 | Any valid SGE job ID could be passed to ``control()`` as the ID of the job to 349 | delete. 350 | 351 | Getting Job Status 352 | ------------------ 353 | 354 | Here's an example of using DRMAA to query the status of a job: 355 | 356 | .. code-block:: python 357 | :linenos: 358 | :caption: example5.py 359 | :name: example-5 360 | :emphasize-lines: 22-40 361 | 362 | #!/usr/bin/env python 363 | 364 | import drmaa 365 | import time 366 | import os 367 | 368 | def main(): 369 | """ 370 | Submit a job, and check its progress. 371 | Note, need file called sleeper.sh in home directory. 372 | """ 373 | with drmaa.Session() as s: 374 | print('Creating job template') 375 | jt = s.createJobTemplate() 376 | jt.remoteCommand = os.path.join(os.getcwd(), 'sleeper.sh') 377 | jt.args = ['42', 'Simon says:'] 378 | jt.joinFiles=True 379 | 380 | jobid = s.runJob(jt) 381 | print('Your job has been submitted with ID %s' % jobid) 382 | 383 | # Who needs a case statement when you have dictionaries? 384 | decodestatus = {drmaa.JobState.UNDETERMINED: 'process status cannot be determined', 385 | drmaa.JobState.QUEUED_ACTIVE: 'job is queued and active', 386 | drmaa.JobState.SYSTEM_ON_HOLD: 'job is queued and in system hold', 387 | drmaa.JobState.USER_ON_HOLD: 'job is queued and in user hold', 388 | drmaa.JobState.USER_SYSTEM_ON_HOLD: 'job is queued and in user and system hold', 389 | drmaa.JobState.RUNNING: 'job is running', 390 | drmaa.JobState.SYSTEM_SUSPENDED: 'job is system suspended', 391 | drmaa.JobState.USER_SUSPENDED: 'job is user suspended', 392 | drmaa.JobState.DONE: 'job finished normally', 393 | drmaa.JobState.FAILED: 'job finished, but failed'} 394 | 395 | for ix in range(10): 396 | print('Checking %s of 10 times' % ix) 397 | print decodestatus(s.jobStatus(jobid)) 398 | time.sleep(5) 399 | 400 | print('Cleaning up') 401 | s.deleteJobTemplate(jt) 402 | 403 | if __name__=='__main__': 404 | main() 405 | 406 | Again, this example is very similar to :ref:`Example 2 `, this time 407 | with the exception of lines 22--40. On line 36, we use ``jobStatus()`` to get 408 | the status of the job. Line 43 determine what the job status is and report it. 409 | 410 | Getting DRM information 411 | ----------------------- 412 | 413 | Lastly, let's look at how to query the DRMAA library for information about the 414 | DRMS and the DRMAA implementation itself: 415 | 416 | .. code-block:: python 417 | :linenos: 418 | :caption: example6.py 419 | :name: example-6 420 | :emphasize-lines: 9-12 421 | 422 | #!/usr/bin/env python 423 | 424 | import drmaa 425 | 426 | def main(): 427 | """ Query the system. """ 428 | with drmaa.Session() as s: 429 | print('A DRMAA object was created') 430 | print('Supported contact strings: %s' % s.contact) 431 | print('Supported DRM systems: %s' % s.drmsInfo) 432 | print('Supported DRMAA implementations: %s' % s.drmaaImplementation) 433 | print('Version %s' % s.version) 434 | 435 | print('Exiting') 436 | 437 | if __name__=='__main__': 438 | main() 439 | 440 | On line 9, we get the contact string list. This is the list of contact strings 441 | that will be understood by this DRMAA instance. Normally on of these strings is 442 | used to select to which DRM this DRMAA instance should be bound. On line 10, we 443 | get the list of supported DRM systems. On line 11, we get the list of supported 444 | DRMAA implementations. On line 12, we get the version number of the DRMAA C 445 | binding specification supported by this DRMAA implementation. 446 | -------------------------------------------------------------------------------- /drmaa/__init__.py: -------------------------------------------------------------------------------- 1 | # ----------------------------------------------------------- 2 | # Copyright (C) 2008 StatPro Italia s.r.l. 3 | # 4 | # StatPro Italia 5 | # Via G. B. Vico 4 6 | # I-20123 Milano 7 | # ITALY 8 | # 9 | # phone: +39 02 96875 1 10 | # fax: +39 02 96875 605 11 | # 12 | # This program is distributed in the hope that it will be 13 | # useful, but WITHOUT ANY WARRANTY; without even the 14 | # warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR 15 | # PURPOSE. See the license for more details. 16 | # ----------------------------------------------------------- 17 | # 18 | # Author: Enrico Sirola 19 | 20 | """ 21 | A python package for DRM job submission and control. 22 | 23 | This package is an implementation of the DRMAA 1.0 Python language 24 | binding specification (http://www.ogf.org/documents/GFD.143.pdf). 25 | The source is hosted on GitHub: https://github.com/pygridtools/drmaa-python 26 | Releases are available from PyPi: https://pypi.python.org/pypi/drmaa 27 | Documentation is hosted on ReadTheDocs: http://drmaa-python.readthedocs.org/ 28 | 29 | :author: Enrico Sirola (enrico.sirola@statpro.com) 30 | :author: Dan Blanchard (dan.blanchard@gmail.com) 31 | """ 32 | 33 | from __future__ import absolute_import, print_function, unicode_literals 34 | 35 | from .const import (ATTR_BUFFER, BLOCK_EMAIL, CONTACT_BUFFER, 36 | control_action_to_string, DEADLINE_TIME, DRM_SYSTEM_BUFFER, 37 | DRMAA_IMPLEMENTATION_BUFFER, DURATION_HLIMIT, 38 | DURATION_SLIMIT, ERROR_PATH, ERROR_STRING_BUFFER, 39 | INPUT_PATH, JOB_CATEGORY, JOB_IDS_SESSION_ALL, 40 | JOB_IDS_SESSION_ANY, JOB_NAME, job_state, JobControlAction, 41 | JOBNAME_BUFFER, JobState, JobSubmissionState, JOIN_FILES, 42 | JS_STATE, NATIVE_SPECIFICATION, NO_MORE_ELEMENTS, 43 | OUTPUT_PATH, PLACEHOLDER_HD, PLACEHOLDER_INCR, 44 | PLACEHOLDER_WD, REMOTE_COMMAND, SIGNAL_BUFFER, START_TIME, 45 | status_to_string, string_to_control_action, 46 | submission_state, SUBMISSION_STATE_ACTIVE, 47 | SUBMISSION_STATE_HOLD, TIMEOUT_NO_WAIT, 48 | TIMEOUT_WAIT_FOREVER, TRANSFER_FILES, V_ARGV, V_EMAIL, 49 | V_ENV, WCT_HLIMIT, WCT_SLIMIT, WD) 50 | from .errors import (AlreadyActiveSessionException, AuthorizationException, 51 | ConflictingAttributeValuesException, 52 | DefaultContactStringException, DeniedByDrmException, 53 | DrmCommunicationException, DrmsExitException, 54 | DrmsInitException, ExitTimeoutException, 55 | HoldInconsistentStateException, IllegalStateException, 56 | InternalException, InvalidAttributeFormatException, 57 | InvalidContactStringException, InvalidJobException, 58 | InvalidJobTemplateException, NoActiveSessionException, 59 | NoDefaultContactStringSelectedException, 60 | ReleaseInconsistentStateException, 61 | ResumeInconsistentStateException, 62 | SuspendInconsistentStateException, TryLaterException, 63 | UnsupportedAttributeException, InvalidArgumentException, 64 | InvalidAttributeValueException, OutOfMemoryException) 65 | from .session import JobInfo, JobTemplate, Session 66 | from .version import __version__, VERSION 67 | 68 | 69 | __docformat__ = "restructuredtext en" 70 | 71 | 72 | __all__ = ['JobInfo', 'JobTemplate', 'Session', 'AlreadyActiveSessionException', 73 | 'AuthorizationException', 'ConflictingAttributeValuesException', 74 | 'DefaultContactStringException', 'DeniedByDrmException', 75 | 'DrmCommunicationException', 'DrmsExitException', 76 | 'DrmsInitException', 'ExitTimeoutException', 77 | 'HoldInconsistentStateException', 'IllegalStateException', 78 | 'InternalException', 'InvalidAttributeFormatException', 79 | 'InvalidContactStringException', 'InvalidJobException', 80 | 'InvalidJobTemplateException', 'NoActiveSessionException', 81 | 'NoDefaultContactStringSelectedException', 82 | 'ReleaseInconsistentStateException', 83 | 'ResumeInconsistentStateException', 84 | 'SuspendInconsistentStateException', 'TryLaterException', 85 | 'UnsupportedAttributeException', 'InvalidArgumentException', 86 | 'InvalidAttributeValueException', 'OutOfMemoryException', 87 | 'ATTR_BUFFER', 'BLOCK_EMAIL', 'CONTACT_BUFFER', 88 | 'control_action_to_string', 'DEADLINE_TIME', 'DRM_SYSTEM_BUFFER', 89 | 'DRMAA_IMPLEMENTATION_BUFFER', 'DURATION_HLIMIT', 'DURATION_SLIMIT', 90 | 'ERROR_PATH', 'ERROR_STRING_BUFFER', 'INPUT_PATH', 'JOB_CATEGORY', 91 | 'JOB_IDS_SESSION_ALL', 'JOB_IDS_SESSION_ANY', 'JOB_NAME', 92 | 'job_state', 'JobControlAction', 'JOBNAME_BUFFER', 'JobState', 93 | 'JobSubmissionState', 'JOIN_FILES', 'JS_STATE', 94 | 'NATIVE_SPECIFICATION', 'NO_MORE_ELEMENTS', 'OUTPUT_PATH', 95 | 'PLACEHOLDER_HD', 'PLACEHOLDER_INCR', 'PLACEHOLDER_WD', 96 | 'REMOTE_COMMAND', 'SIGNAL_BUFFER', 'START_TIME', 'status_to_string', 97 | 'string_to_control_action', 'submission_state', 98 | 'SUBMISSION_STATE_ACTIVE', 'SUBMISSION_STATE_HOLD', 99 | 'TIMEOUT_NO_WAIT', 'TIMEOUT_WAIT_FOREVER', 'TRANSFER_FILES', 100 | 'V_ARGV', 'V_EMAIL', 'V_ENV', 'WCT_HLIMIT', 'WCT_SLIMIT', 'WD'] 101 | -------------------------------------------------------------------------------- /drmaa/const.py: -------------------------------------------------------------------------------- 1 | # ----------------------------------------------------------- 2 | # Copyright (C) 2009 StatPro Italia s.r.l. 3 | # 4 | # StatPro Italia 5 | # Via G. B. Vico 4 6 | # I-20123 Milano 7 | # ITALY 8 | # 9 | # phone: +39 02 96875 1 10 | # fax: +39 02 96875 605 11 | # 12 | # email: info@riskmap.net 13 | # 14 | # This program is distributed in the hope that it will be 15 | # useful, but WITHOUT ANY WARRANTY; without even the 16 | # warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR 17 | # PURPOSE. See the license for more details. 18 | # ----------------------------------------------------------- 19 | # 20 | # Author: Enrico Sirola 21 | # 22 | """ 23 | drmaa constants 24 | """ 25 | 26 | from __future__ import absolute_import, print_function, unicode_literals 27 | 28 | import locale 29 | 30 | # Encoding to use for passing strings to C library 31 | ENCODING = locale.getpreferredencoding() 32 | 33 | # drmaa_get_attribute() 34 | ATTR_BUFFER = 1024 35 | 36 | # drmaa_get_contact() 37 | CONTACT_BUFFER = 1024 38 | 39 | # drmaa_get_DRM_system() 40 | DRM_SYSTEM_BUFFER = 1024 41 | 42 | # drmaa_get_DRM_system() 43 | DRMAA_IMPLEMENTATION_BUFFER = 1024 44 | 45 | # Agreed buffer length constants 46 | # these are recommended minimum values 47 | ERROR_STRING_BUFFER = 1024 48 | JOBNAME_BUFFER = 1024 49 | SIGNAL_BUFFER = 32 50 | 51 | # Agreed constants 52 | TIMEOUT_WAIT_FOREVER = -1 53 | TIMEOUT_NO_WAIT = 0 54 | 55 | JOB_IDS_SESSION_ANY = b"DRMAA_JOB_IDS_SESSION_ANY" 56 | JOB_IDS_SESSION_ALL = b"DRMAA_JOB_IDS_SESSION_ALL" 57 | 58 | SUBMISSION_STATE_ACTIVE = "drmaa_active" 59 | SUBMISSION_STATE_HOLD = "drmaa_hold" 60 | 61 | # Agreed placeholder names 62 | PLACEHOLDER_INCR = b"$drmaa_incr_ph$" 63 | PLACEHOLDER_HD = b"$drmaa_hd_ph$" 64 | PLACEHOLDER_WD = b"$drmaa_wd_ph$" 65 | 66 | # Agreed names of job template attributes 67 | REMOTE_COMMAND = b"drmaa_remote_command" 68 | JS_STATE = b"drmaa_js_state" 69 | WD = b"drmaa_wd" 70 | JOB_CATEGORY = b"drmaa_job_category" 71 | NATIVE_SPECIFICATION = b"drmaa_native_specification" 72 | BLOCK_EMAIL = b"drmaa_block_email" 73 | START_TIME = b"drmaa_start_time" 74 | JOB_NAME = b"drmaa_job_name" 75 | INPUT_PATH = b"drmaa_input_path" 76 | OUTPUT_PATH = b"drmaa_output_path" 77 | ERROR_PATH = b"drmaa_error_path" 78 | JOIN_FILES = b"drmaa_join_files" 79 | TRANSFER_FILES = b"drmaa_transfer_files" 80 | DEADLINE_TIME = b"drmaa_deadline_time" 81 | WCT_HLIMIT = b"drmaa_wct_hlimit" 82 | WCT_SLIMIT = b"drmaa_wct_slimit" 83 | DURATION_HLIMIT = b"drmaa_duration_hlimit" 84 | DURATION_SLIMIT = b"drmaa_duration_slimit" 85 | 86 | # names of job template vector attributes 87 | V_ARGV = b"drmaa_v_argv" 88 | V_ENV = b"drmaa_v_env" 89 | V_EMAIL = b"drmaa_v_email" 90 | 91 | NO_MORE_ELEMENTS = 25 92 | 93 | 94 | def job_state(code): 95 | return _JOB_PS[code] 96 | 97 | 98 | class JobState(object): 99 | UNDETERMINED = 'undetermined' 100 | QUEUED_ACTIVE = 'queued_active' 101 | SYSTEM_ON_HOLD = 'system_on_hold' 102 | USER_ON_HOLD = 'user_on_hold' 103 | USER_SYSTEM_ON_HOLD = 'user_system_on_hold' 104 | RUNNING = 'running' 105 | SYSTEM_SUSPENDED = 'system_suspended' 106 | USER_SUSPENDED = 'user_suspended' 107 | USER_SYSTEM_SUSPENDED = 'user_system_suspended' 108 | DONE = 'done' 109 | FAILED = 'failed' 110 | 111 | # Job control action 112 | 113 | 114 | class JobControlAction(object): 115 | SUSPEND = 'suspend' 116 | RESUME = 'resume' 117 | HOLD = 'hold' 118 | RELEASE = 'release' 119 | TERMINATE = 'terminate' 120 | 121 | _JOB_CONTROL = [JobControlAction.SUSPEND, 122 | JobControlAction.RESUME, 123 | JobControlAction.HOLD, 124 | JobControlAction.RELEASE, 125 | JobControlAction.TERMINATE] 126 | 127 | 128 | def string_to_control_action(operation): 129 | return _JOB_CONTROL.index(operation) 130 | 131 | 132 | def control_action_to_string(code): 133 | return _JOB_CONTROL[code] 134 | 135 | 136 | def status_to_string(status): 137 | return _JOB_PS[status] 138 | 139 | _JOB_PS = {0x00: JobState.UNDETERMINED, 140 | 0x10: JobState.QUEUED_ACTIVE, 141 | 0x11: JobState.SYSTEM_ON_HOLD, 142 | 0x12: JobState.USER_ON_HOLD, 143 | 0x13: JobState.USER_SYSTEM_ON_HOLD, 144 | 0x20: JobState.RUNNING, 145 | 0x21: JobState.SYSTEM_SUSPENDED, 146 | 0x22: JobState.USER_SUSPENDED, 147 | 0x23: JobState.USER_SYSTEM_SUSPENDED, 148 | 0x30: JobState.DONE, 149 | 0x40: JobState.FAILED} 150 | 151 | # State at submission time 152 | 153 | 154 | class JobSubmissionState(object): 155 | HOLD_STATE = SUBMISSION_STATE_HOLD 156 | ACTIVE_STATE = SUBMISSION_STATE_ACTIVE 157 | 158 | _SUBMISSION_STATE = [JobSubmissionState.HOLD_STATE, 159 | JobSubmissionState.ACTIVE_STATE] 160 | 161 | 162 | def submission_state(code): 163 | return _SUBMISSION_STATE[code] 164 | -------------------------------------------------------------------------------- /drmaa/errors.py: -------------------------------------------------------------------------------- 1 | # ----------------------------------------------------------- 2 | # Copyright (C) 2009 StatPro Italia s.r.l. 3 | # 4 | # StatPro Italia 5 | # Via G. B. Vico 4 6 | # I-20123 Milano 7 | # ITALY 8 | # 9 | # phone: +39 02 96875 1 10 | # fax: +39 02 96875 605 11 | # 12 | # email: info@riskmap.net 13 | # 14 | # This program is distributed in the hope that it will be 15 | # useful, but WITHOUT ANY WARRANTY; without even the 16 | # warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR 17 | # PURPOSE. See the license for more details. 18 | # ----------------------------------------------------------- 19 | # 20 | # Author: Enrico Sirola 21 | 22 | """ 23 | drmaa errors 24 | """ 25 | 26 | from ctypes import create_string_buffer 27 | 28 | from drmaa.const import ERROR_STRING_BUFFER 29 | 30 | 31 | class DrmaaException(Exception): 32 | 33 | """ 34 | A common ancestor to all DRMAA Error classes. 35 | """ 36 | pass 37 | 38 | 39 | class AlreadyActiveSessionException(DrmaaException): 40 | pass 41 | 42 | 43 | class AuthorizationException(DrmaaException): 44 | pass 45 | 46 | 47 | class ConflictingAttributeValuesException(DrmaaException, AttributeError): 48 | pass 49 | 50 | 51 | class DefaultContactStringException(DrmaaException): 52 | pass 53 | 54 | 55 | class DeniedByDrmException(DrmaaException): 56 | pass 57 | 58 | 59 | class DrmCommunicationException(DrmaaException): 60 | pass 61 | 62 | 63 | class DrmsExitException(DrmaaException): 64 | pass 65 | 66 | 67 | class DrmsInitException(DrmaaException): 68 | pass 69 | 70 | 71 | class ExitTimeoutException(DrmaaException): 72 | pass 73 | 74 | 75 | class HoldInconsistentStateException(DrmaaException): 76 | pass 77 | 78 | 79 | class IllegalStateException(DrmaaException): 80 | pass 81 | 82 | 83 | class InternalException(DrmaaException): 84 | pass 85 | 86 | 87 | class InvalidAttributeFormatException(DrmaaException, AttributeError): 88 | pass 89 | 90 | 91 | class InvalidContactStringException(DrmaaException): 92 | pass 93 | 94 | 95 | class InvalidJobException(DrmaaException): 96 | pass 97 | 98 | 99 | class InvalidJobTemplateException(DrmaaException): 100 | pass 101 | 102 | 103 | class NoActiveSessionException(DrmaaException): 104 | pass 105 | 106 | 107 | class NoDefaultContactStringSelectedException(DrmaaException): 108 | pass 109 | 110 | 111 | class ReleaseInconsistentStateException(DrmaaException): 112 | pass 113 | 114 | 115 | class ResumeInconsistentStateException(DrmaaException): 116 | pass 117 | 118 | 119 | class SuspendInconsistentStateException(DrmaaException): 120 | pass 121 | 122 | 123 | class TryLaterException(DrmaaException): 124 | pass 125 | 126 | 127 | class UnsupportedAttributeException(DrmaaException, AttributeError): 128 | pass 129 | 130 | 131 | class InvalidArgumentException(DrmaaException, AttributeError): 132 | pass 133 | 134 | 135 | class InvalidAttributeValueException(DrmaaException, AttributeError): 136 | pass 137 | 138 | 139 | class OutOfMemoryException(DrmaaException, MemoryError): 140 | pass 141 | 142 | error_buffer = create_string_buffer(ERROR_STRING_BUFFER) 143 | 144 | 145 | def error_check(code): 146 | if code == 0: 147 | return 148 | else: 149 | error_string = "code {0}: {1}".format(code, error_buffer.value.decode()) 150 | try: 151 | raise _ERRORS[code - 1](error_string) 152 | except IndexError: 153 | raise DrmaaException(error_string) 154 | 155 | # da vedere: NO_RUSAGE, NO_MORE_ELEMENTS 156 | _ERRORS = [InternalException, 157 | DrmCommunicationException, 158 | AuthorizationException, 159 | InvalidArgumentException, 160 | NoActiveSessionException, 161 | OutOfMemoryException, 162 | InvalidContactStringException, 163 | DefaultContactStringException, 164 | NoDefaultContactStringSelectedException, 165 | DrmsInitException, 166 | AlreadyActiveSessionException, 167 | DrmsExitException, 168 | InvalidAttributeFormatException, 169 | InvalidAttributeValueException, 170 | ConflictingAttributeValuesException, 171 | TryLaterException, 172 | DeniedByDrmException, 173 | InvalidJobException, 174 | ResumeInconsistentStateException, 175 | SuspendInconsistentStateException, 176 | HoldInconsistentStateException, 177 | ReleaseInconsistentStateException, 178 | ExitTimeoutException, 179 | Exception, 180 | StopIteration] 181 | -------------------------------------------------------------------------------- /drmaa/helpers.py: -------------------------------------------------------------------------------- 1 | # ----------------------------------------------------------- 2 | # Copyright (C) 2009 StatPro Italia s.r.l. 3 | # 4 | # StatPro Italia 5 | # Via G. B. Vico 4 6 | # I-20123 Milano 7 | # ITALY 8 | # 9 | # phone: +39 02 96875 1 10 | # fax: +39 02 96875 605 11 | # 12 | # email: info@riskmap.net 13 | # 14 | # This program is distributed in the hope that it will be 15 | # useful, but WITHOUT ANY WARRANTY; without even the 16 | # warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR 17 | # PURPOSE. See the license for more details. 18 | # ----------------------------------------------------------- 19 | # 20 | # Author: Enrico Sirola 21 | # Author: Dan Blanchard 22 | 23 | """ 24 | internal helpers 25 | """ 26 | 27 | from __future__ import absolute_import, print_function, unicode_literals 28 | 29 | import sys 30 | from collections import namedtuple 31 | from ctypes import (byref, c_uint, create_string_buffer, POINTER, pointer, 32 | sizeof) 33 | 34 | from drmaa.const import ATTR_BUFFER, ENCODING, NO_MORE_ELEMENTS 35 | from drmaa.errors import error_buffer 36 | from drmaa.wrappers import (drmaa_attr_names_t, drmaa_attr_values_t, 37 | drmaa_get_attribute, drmaa_get_attribute_names, 38 | drmaa_get_next_attr_name, 39 | drmaa_get_next_attr_value, 40 | drmaa_get_next_job_id, drmaa_get_vector_attribute, 41 | drmaa_get_vector_attribute_names, drmaa_job_ids_t, 42 | drmaa_release_attr_names, 43 | drmaa_release_attr_values, 44 | drmaa_release_job_ids, drmaa_run_bulk_jobs, 45 | drmaa_set_attribute, drmaa_set_vector_attribute, 46 | drmaa_version, STRING) 47 | 48 | 49 | # Python 3 compatability help 50 | if sys.version_info < (3, 0): 51 | bytes = str 52 | str = unicode 53 | 54 | 55 | _BUFLEN = ATTR_BUFFER 56 | 57 | 58 | class BoolConverter(object): 59 | 60 | """Helper class to convert to/from bool attributes.""" 61 | 62 | def __init__(self, true=b'y', false=b'n'): 63 | if isinstance(true, str): 64 | true = true.encode(ENCODING) 65 | self.true = true 66 | if isinstance(false, str): 67 | false = false.encode(ENCODING) 68 | self.false = false 69 | 70 | def to_drmaa(self, value): 71 | if value: 72 | return self.true 73 | else: 74 | return self.false 75 | 76 | def from_drmaa(self, value): 77 | if value == self.true: 78 | return True 79 | else: 80 | return False 81 | 82 | 83 | class IntConverter(object): 84 | 85 | """Helper class to convert to/from int attributes.""" 86 | @staticmethod 87 | def to_drmaa(value): 88 | return bytes(value) 89 | 90 | @staticmethod 91 | def from_drmaa(value): 92 | return int(value) 93 | 94 | 95 | class SessionStringAttribute(object): 96 | 97 | def __init__(self, drmaa_function): 98 | self._f = drmaa_function 99 | 100 | def __get__(self, *args): 101 | buf = create_string_buffer(_BUFLEN) 102 | c(self._f, buf, sizeof(buf)) 103 | return buf.value.decode() 104 | 105 | Version = namedtuple("Version", "major minor") 106 | if sys.version_info < (3, 0): 107 | Version.__str__ = lambda x: "{0}.{1}".format(x.major, 108 | x.minor).encode(ENCODING) 109 | else: 110 | Version.__str__ = lambda x: "{0}.{1}".format(x.major, x.minor) 111 | 112 | class SessionVersionAttribute(object): 113 | 114 | """A Version attribute.""" 115 | 116 | def __get__(self, *args): 117 | major = c_uint(10) 118 | minor = c_uint(10) 119 | c(drmaa_version, byref(major), byref(minor)) 120 | return Version(major.value, minor.value) 121 | 122 | 123 | class Attribute(object): 124 | 125 | """ 126 | A DRMAA attribute, to be managed with scalar C DRMAA attribute management 127 | functions. 128 | """ 129 | 130 | def __init__(self, name, type_converter=None): 131 | """ 132 | Attribute constructor. 133 | 134 | :Parameters: 135 | `name` : string 136 | name of the attribute to be managed, as seen by the underlying C 137 | DRMAA 138 | `type_converter` 139 | a converter to translate attribute values to/from the underlying 140 | implementation. See BoolConverter for an example. 141 | """ 142 | if isinstance(name, str): 143 | name = name.encode(ENCODING) 144 | self.name = name 145 | self.converter = type_converter 146 | 147 | def __set__(self, instance, value): 148 | if self.converter: 149 | v = self.converter.to_drmaa(value) 150 | elif isinstance(value, str): 151 | v = value.encode(ENCODING) 152 | else: 153 | v = value 154 | c(drmaa_set_attribute, instance, self.name, v) 155 | 156 | def __get__(self, instance, _): 157 | attr_buffer = create_string_buffer(ATTR_BUFFER) 158 | c(drmaa_get_attribute, instance, self.name, attr_buffer, 159 | sizeof(attr_buffer)) 160 | if self.converter: 161 | return self.converter.from_drmaa(attr_buffer.value) 162 | elif isinstance(attr_buffer.value, bytes): 163 | return attr_buffer.value.decode() 164 | else: 165 | return attr_buffer.value 166 | 167 | 168 | class VectorAttribute(object): 169 | 170 | """ 171 | A DRMAA attribute representing a list. 172 | 173 | To be managed with vector C DRMAA attribute management functions. 174 | """ 175 | 176 | def __init__(self, name): 177 | if isinstance(name, str): 178 | name = name.encode(ENCODING) 179 | self.name = name 180 | 181 | def __set__(self, instance, value): 182 | c(drmaa_set_vector_attribute, instance, 183 | self.name, string_vector(value)) 184 | 185 | def __get__(self, instance, _): 186 | return list(vector_attribute_iterator(instance, self.name)) 187 | 188 | 189 | class DictAttribute(object): 190 | 191 | """ 192 | A DRMAA attribute representing a python dict. 193 | 194 | To be managed with vector C DRMAA attribute management functions. 195 | """ 196 | 197 | def __init__(self, name): 198 | if isinstance(name, str): 199 | name = name.encode(ENCODING) 200 | self.name = name 201 | 202 | def __set__(self, instance, value): 203 | vector = [] 204 | for k, v in value.items(): 205 | if isinstance(k, bytes): 206 | k = k.decode(ENCODING) 207 | if isinstance(v, bytes): 208 | v = v.decode(ENCODING) 209 | vector.append("{0}={1}".format(k, v).encode(ENCODING)) 210 | c(drmaa_set_vector_attribute, instance, self.name, 211 | string_vector(vector)) 212 | 213 | def __get__(self, instance, _): 214 | x = [i.split('=', 1) for i in 215 | list(vector_attribute_iterator(instance, self.name))] 216 | return dict(x) 217 | 218 | 219 | def attributes_iterator(attributes): 220 | try: 221 | buf = create_string_buffer(ATTR_BUFFER) 222 | while drmaa_get_next_attr_value(attributes, buf, 223 | sizeof(buf)) != NO_MORE_ELEMENTS: 224 | yield buf.value.decode() 225 | except: 226 | drmaa_release_attr_values(attributes) 227 | raise 228 | else: 229 | drmaa_release_attr_values(attributes) 230 | 231 | 232 | def adapt_rusage(rusage): 233 | """ 234 | Transform a rusage data structure into a dict. 235 | 236 | Due to the value possibly containing a equal sign make sure we 237 | limit the splits to only the first occurrence. 238 | """ 239 | rv = dict() 240 | for attr in attributes_iterator(rusage.contents): 241 | 242 | k, v = attr.split('=',1) 243 | rv[k] = v 244 | return rv 245 | 246 | 247 | def vector_attribute_iterator(jt, attr_name): 248 | avalues = pointer(POINTER(drmaa_attr_values_t)()) 249 | c(drmaa_get_vector_attribute, jt, attr_name, avalues) 250 | return attributes_iterator(avalues.contents) 251 | 252 | 253 | def attribute_names_iterator(): 254 | attrn_p = pointer(POINTER(drmaa_attr_names_t)()) 255 | c(drmaa_get_attribute_names, attrn_p) 256 | try: 257 | name = create_string_buffer(_BUFLEN) 258 | while drmaa_get_next_attr_name(attrn_p.contents, name, 259 | _BUFLEN) != NO_MORE_ELEMENTS: 260 | yield name.value.decode() 261 | except: 262 | drmaa_release_attr_names(attrn_p.contents) 263 | raise 264 | else: 265 | drmaa_release_attr_names(attrn_p.contents) 266 | 267 | 268 | def vector_attribute_names_iterator(): 269 | attrn_p = pointer(POINTER(drmaa_attr_names_t)()) 270 | c(drmaa_get_vector_attribute_names, attrn_p) 271 | try: 272 | name = create_string_buffer(_BUFLEN) 273 | while drmaa_get_next_attr_name(attrn_p.contents, name, 274 | _BUFLEN) != NO_MORE_ELEMENTS: 275 | yield name.value.decode() 276 | except: 277 | drmaa_release_attr_names(attrn_p.contents) 278 | raise 279 | else: 280 | drmaa_release_attr_names(attrn_p.contents) 281 | 282 | 283 | def run_bulk_job(jt, start, end, incr=1): 284 | jids = pointer(POINTER(drmaa_job_ids_t)()) 285 | try: 286 | c(drmaa_run_bulk_jobs, jids, jt, start, end, incr) 287 | jid = create_string_buffer(_BUFLEN) 288 | while drmaa_get_next_job_id(jids.contents, jid, 289 | _BUFLEN) != NO_MORE_ELEMENTS: 290 | yield jid.value.decode() 291 | except StopIteration: 292 | pass 293 | finally: 294 | drmaa_release_job_ids(jids.contents) 295 | 296 | 297 | def c(f, *args): 298 | """ 299 | A helper function wrapping calls to the C DRMAA functions with error 300 | managing code. 301 | """ 302 | return f(*(args + (error_buffer, sizeof(error_buffer)))) 303 | 304 | 305 | def string_vector(v): 306 | vlen = len(v) 307 | values = (STRING * (vlen + 1))() 308 | for i, el in enumerate(v): 309 | if isinstance(el, str): 310 | el = el.encode(ENCODING) 311 | elif not isinstance(el, bytes): 312 | el = str(el).encode(ENCODING) 313 | values[i] = STRING(el) 314 | values[vlen] = STRING() 315 | return values 316 | 317 | 318 | def attribute_setter(obj, attribute_name): 319 | """ 320 | returns a drmaa attribute setter 321 | """ 322 | def f(value): 323 | "setter for %s" % attribute_name 324 | c(drmaa_set_attribute, obj, attribute_name, value) 325 | f.__name__ = 'set_' + attribute_name 326 | return f 327 | 328 | 329 | def attribute_getter(obj, attribute_name): 330 | """ 331 | returns a drmaa attribute setter 332 | """ 333 | def f(): 334 | "getter for %s" % attribute_name 335 | attr_buffer = create_string_buffer(1024) 336 | c(drmaa_get_attribute, obj, attribute_name, attr_buffer, 337 | sizeof(attr_buffer)) 338 | return attr_buffer.value 339 | f.__name__ = 'get_' + attribute_name 340 | return f 341 | -------------------------------------------------------------------------------- /drmaa/session.py: -------------------------------------------------------------------------------- 1 | # ----------------------------------------------------------- 2 | # Copyright (C) 2008 StatPro Italia s.r.l. 3 | # 4 | # StatPro Italia 5 | # Via G. B. Vico 4 6 | # I-20123 Milano 7 | # ITALY 8 | # 9 | # phone: +39 02 96875 1 10 | # fax: +39 02 96875 605 11 | # 12 | # This program is distributed in the hope that it will be 13 | # useful, but WITHOUT ANY WARRANTY; without even the 14 | # warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR 15 | # PURPOSE. See the license for more details. 16 | # ----------------------------------------------------------- 17 | # 18 | # Author: Enrico Sirola 19 | ''' 20 | Everything related to sessions and jobs. 21 | ''' 22 | 23 | from __future__ import absolute_import, print_function, unicode_literals 24 | 25 | import sys 26 | from collections import namedtuple 27 | from ctypes import byref, c_int, create_string_buffer, pointer, POINTER, sizeof 28 | 29 | from drmaa.const import (BLOCK_EMAIL, DEADLINE_TIME, DURATION_HLIMIT, 30 | DURATION_SLIMIT, ENCODING, ERROR_PATH, INPUT_PATH, 31 | JOB_CATEGORY, JOB_IDS_SESSION_ALL, JOB_IDS_SESSION_ANY, 32 | JOB_NAME, JobState, JobControlAction, 33 | JobSubmissionState, JOIN_FILES, JS_STATE, 34 | NATIVE_SPECIFICATION, OUTPUT_PATH, REMOTE_COMMAND, 35 | SIGNAL_BUFFER, START_TIME, status_to_string, 36 | string_to_control_action, TIMEOUT_NO_WAIT, 37 | TIMEOUT_WAIT_FOREVER, TRANSFER_FILES, V_ARGV, V_EMAIL, 38 | V_ENV, WCT_HLIMIT, WCT_SLIMIT, WD) 39 | from drmaa.helpers import (adapt_rusage, Attribute, attribute_names_iterator, 40 | BoolConverter, c, DictAttribute, IntConverter, 41 | run_bulk_job, SessionStringAttribute, 42 | SessionVersionAttribute, string_vector, 43 | VectorAttribute, Version) 44 | from drmaa.wrappers import (drmaa_allocate_job_template, drmaa_attr_values_t, 45 | drmaa_control, drmaa_delete_job_template, 46 | drmaa_get_contact, drmaa_get_DRM_system, 47 | drmaa_get_DRMAA_implementation, drmaa_job_ps, 48 | drmaa_job_template_t, drmaa_run_job, 49 | drmaa_synchronize, drmaa_wait, drmaa_wcoredump, 50 | drmaa_wexitstatus, drmaa_wifaborted, 51 | drmaa_wifexited, drmaa_wifsignaled, drmaa_wtermsig, 52 | py_drmaa_exit, py_drmaa_init) 53 | 54 | 55 | # Python 3 compatability help 56 | if sys.version_info < (3, 0): 57 | bytes = str 58 | str = unicode 59 | 60 | 61 | JobInfo = namedtuple("JobInfo", 62 | """jobId hasExited hasSignal terminatedSignal hasCoreDump 63 | wasAborted exitStatus resourceUsage""") 64 | 65 | 66 | class JobTemplate(object): 67 | 68 | """A job to be submitted to the DRM.""" 69 | 70 | HOME_DIRECTORY = '$drmaa_hd_ph$' 71 | """Home directory placeholder.""" 72 | WORKING_DIRECTORY = '$drmaa_wd_ph$' 73 | """Working directory placeholder.""" 74 | PARAMETRIC_INDEX = '$drmaa_incr_ph$' 75 | """Parametric index (for job arrays / bulk jobs) placeholder.""" 76 | 77 | @property 78 | def attributeNames(self): 79 | """ 80 | The list of supported DRMAA scalar attribute names. 81 | 82 | This is apparently useless now, and should probably substituted by the 83 | list of attribute names of the JobTemplate instances. 84 | """ 85 | return list(attribute_names_iterator()) 86 | 87 | # scalar attributes 88 | remoteCommand = Attribute(REMOTE_COMMAND) 89 | """The command to be executed.""" 90 | jobSubmissionState = Attribute(JS_STATE) 91 | """The job status.""" 92 | workingDirectory = Attribute(WD) 93 | """The job working directory.""" 94 | jobCategory = Attribute(JOB_CATEGORY) 95 | """The job category.""" 96 | nativeSpecification = Attribute(NATIVE_SPECIFICATION) 97 | """ 98 | A (DRM-dependant) opaque string to be passed to the DRM representing 99 | other directives. 100 | """ 101 | blockEmail = Attribute(BLOCK_EMAIL, type_converter=BoolConverter(true='1', 102 | false='0')) 103 | """False if this job should send an email, True otherwise.""" 104 | startTime = Attribute(START_TIME) 105 | """The job start time, a partial timestamp string.""" 106 | jobName = Attribute(JOB_NAME) 107 | """The job Name.""" 108 | inputPath = Attribute(INPUT_PATH) 109 | """The path to a file representing job's stdin.""" 110 | outputPath = Attribute(OUTPUT_PATH) 111 | """The path to a file representing job's stdout.""" 112 | errorPath = Attribute(ERROR_PATH) 113 | """The path to a file representing job's stderr.""" 114 | joinFiles = Attribute(JOIN_FILES, type_converter=BoolConverter()) 115 | """True if stdin and stdout should be merged, False otherwise.""" 116 | # the following is available on ge6.2 only if enabled via cluster 117 | # configuration 118 | transferFiles = Attribute(TRANSFER_FILES) 119 | """ 120 | True if file transfer should be enabled, False otherwise. 121 | 122 | This option might require specific DRM configuration (it does on SGE). 123 | """ 124 | # the following are apparently not available on ge 6.2 125 | # it will raise if you try to access these attrs 126 | deadlineTime = Attribute(DEADLINE_TIME) 127 | """The job deadline time, a partial timestamp string.""" 128 | hardWallclockTimeLimit = Attribute(WCT_HLIMIT, IntConverter) 129 | """ 130 | 'Hard' Wallclock time limit, in seconds. 131 | 132 | The job will be killed by the DRM if it takes more than 133 | 'hardWallclockTimeLimit' to complete. 134 | """ 135 | softWallclockTimeLimit = Attribute(WCT_SLIMIT, IntConverter) 136 | """ 137 | 'Soft' Wallclock time limit, in seconds. 138 | 139 | The job will be signaled by the DRM if it takes more than 140 | 'hardWallclockTimeLimit' to complete. 141 | """ 142 | hardRunDurationLimit = Attribute(DURATION_HLIMIT, IntConverter) 143 | softRunDurationLimit = Attribute(DURATION_SLIMIT, IntConverter) 144 | 145 | # vector attributes 146 | email = VectorAttribute(V_EMAIL) 147 | """email addresses to whom send job completion info.""" 148 | args = VectorAttribute(V_ARGV) 149 | """The job's command argument list.""" 150 | # dict attributes 151 | jobEnvironment = DictAttribute(V_ENV) 152 | """The job's environment dict.""" 153 | 154 | _as_parameter_ = None 155 | 156 | def __init__(self, **kwargs): 157 | """ 158 | Builds a JobTemplate instance. 159 | 160 | Attributes can be passed as keyword arguments. 161 | """ 162 | jt = pointer(POINTER(drmaa_job_template_t)()) 163 | c(drmaa_allocate_job_template, jt) 164 | self._jt = self._as_parameter_ = jt.contents 165 | try: 166 | for aname in kwargs: 167 | setattr(self, aname, kwargs.get(aname)) 168 | except: 169 | self.delete() 170 | raise 171 | 172 | def delete(self): 173 | """Deallocate the underlying DRMAA job template.""" 174 | c(drmaa_delete_job_template, self) 175 | 176 | def __enter__(self): 177 | """context manager enter routine""" 178 | return self 179 | 180 | def __exit__(self, *_): 181 | """ 182 | context manager exit routine. 183 | 184 | Stops communication with the DRM. 185 | """ 186 | self.delete() 187 | return False 188 | 189 | 190 | class Session(object): 191 | 192 | """ 193 | The DRMAA Session. 194 | 195 | This class is the entry point for communicating with the DRM system 196 | """ 197 | TIMEOUT_WAIT_FOREVER = TIMEOUT_WAIT_FOREVER 198 | TIMEOUT_NO_WAIT = TIMEOUT_NO_WAIT 199 | JOB_IDS_SESSION_ANY = JOB_IDS_SESSION_ANY 200 | JOB_IDS_SESSION_ALL = JOB_IDS_SESSION_ALL 201 | 202 | contact = SessionStringAttribute(drmaa_get_contact) 203 | """ 204 | a comma delimited string list containing the contact strings available 205 | from the default DRMAA implementation, one element per DRM system 206 | available. If called after initialize(), this method returns the 207 | contact String for the DRM system to which the session is 208 | attached. The returned strings are implementation dependent. 209 | """ 210 | 211 | drmsInfo = SessionStringAttribute(drmaa_get_DRM_system) 212 | """ 213 | If called before initialize(), this method returns a comma delimited 214 | list of DRM systems, one element per DRM system implementation 215 | provided. If called after initialize(), this method returns the 216 | selected DRM system. The returned String is implementation dependent. 217 | """ 218 | drmaaImplementation = SessionStringAttribute(drmaa_get_DRMAA_implementation) 219 | """ 220 | If called before initialize(), this method returns a comma delimited 221 | list of DRMAA implementations, one element for each DRMAA 222 | implementation provided. If called after initialize(), this method 223 | returns the selected DRMAA implementation. The returned String is 224 | implementation dependent and may contain the DRM system as a 225 | component. 226 | """ 227 | version = SessionVersionAttribute() 228 | """ 229 | a Version object containing the major and minor version numbers of the 230 | DRMAA library. For DRMAA 1.0, major is 1 and minor is 0. 231 | """ 232 | 233 | def __init__(self, contactString=None): 234 | self.contactString = contactString 235 | 236 | # no return value 237 | @staticmethod 238 | def initialize(contactString=None): 239 | """ 240 | Used to initialize a DRMAA session for use. 241 | 242 | :Parameters: 243 | contactString : string or None 244 | implementation-dependent string that 245 | may be used to specify which DRM system to use 246 | 247 | This method must be called before any other DRMAA calls. If 248 | contactString is None, the default DRM system is used, provided there 249 | is only one DRMAA implementation available. If there is more than one 250 | DRMAA implementation available, initialize() throws a 251 | NoDefaultContactStringSelectedException. initialize() should be called 252 | only once, by only one of the threads. The main thread is 253 | recommended. A call to initialize() by another thread or additional 254 | calls to initialize() by the same thread with throw a 255 | SessionAlreadyActiveException. 256 | """ 257 | py_drmaa_init(contactString) 258 | 259 | # no return value 260 | @staticmethod 261 | def exit(): 262 | """ 263 | Used to disengage from DRM. 264 | 265 | This routine ends the current DRMAA session but doesn't affect any 266 | jobs (e.g., queued and running jobs remain queued and 267 | running). exit() should be called only once, by only one of the 268 | threads. Additional calls to exit() beyond the first will throw a 269 | NoActiveSessionException. 270 | """ 271 | py_drmaa_exit() 272 | 273 | # returns JobTemplate instance 274 | @staticmethod 275 | def createJobTemplate(): 276 | """ 277 | Allocates a new job template. 278 | 279 | The job template is used to set the environment for jobs to be 280 | submitted. Once the job template has been created, it should also be 281 | deleted (via deleteJobTemplate()) when no longer needed. Failure to do 282 | so may result in a memory leak. 283 | """ 284 | return JobTemplate() 285 | 286 | # takes JobTemplate instance, no return value 287 | @staticmethod 288 | def deleteJobTemplate(jobTemplate): 289 | """ 290 | Deallocate a job template. 291 | 292 | :Parameters: 293 | jobTemplate : JobTemplate 294 | the job temptare to be deleted 295 | 296 | This routine has no effect on running jobs. 297 | """ 298 | jobTemplate.delete() 299 | 300 | # takes JobTemplate instance, returns string 301 | @staticmethod 302 | def runJob(jobTemplate): 303 | """ 304 | Submit a job with attributes defined in the job template. 305 | 306 | :Parameters: 307 | jobTemplate : JobTemplate 308 | the template representing the job to be run 309 | 310 | The returned job identifier is a String identical to that returned 311 | from the underlying DRM system. 312 | """ 313 | jid = create_string_buffer(128) 314 | c(drmaa_run_job, jid, sizeof(jid), jobTemplate) 315 | return jid.value.decode() 316 | 317 | # takes JobTemplate instance and num values, returns string list 318 | @staticmethod 319 | def runBulkJobs(jobTemplate, beginIndex, endIndex, step): 320 | """ 321 | Submit a set of parametric jobs, each with attributes defined in the job 322 | template. 323 | 324 | :Parameters: 325 | jobTemplate : JobTemplate 326 | the template representng jobs to be run 327 | beginIndex : int 328 | index of the first job 329 | endIndex : int 330 | index of the last job 331 | step : int 332 | the step between job ids 333 | 334 | The returned job identifiers are Strings identical to those returned 335 | from the underlying DRM system. The JobTemplate class defines a 336 | `JobTemplate.PARAMETRIC_INDEX` placeholder for use in specifying paths. 337 | This placeholder is used to represent the individual identifiers of 338 | the tasks submitted through this method. 339 | """ 340 | return list(run_bulk_job(jobTemplate, beginIndex, endIndex, step)) 341 | 342 | # takes string and JobControlAction value, no return value 343 | @staticmethod 344 | def control(jobId, operation): 345 | """ 346 | Used to hold, release, suspend, resume, or kill the job identified by jobId. 347 | 348 | :Parameters: 349 | jobId : string 350 | if jobId is `Session.JOB_IDS_SESSION_ALL` then this routine acts on 351 | all jobs submitted during this DRMAA session up to the moment 352 | control() is called. The legal values for 353 | action and their meanings are 354 | operation : string 355 | possible values are: 356 | `JobControlAction.SUSPEND` 357 | stop the job 358 | `JobControlAction.RESUME` 359 | (re)start the job 360 | `JobControlAction.HOLD` 361 | put the job on-hold 362 | `JobControlAction.RELEASE` 363 | release the hold on the job 364 | `JobControlAction.TERMINATE` 365 | kill the job 366 | 367 | To avoid thread races in multithreaded applications, the DRMAA 368 | implementation user should explicitly synchronize this call with 369 | any other job submission calls or control calls that may change 370 | the number of remote jobs. 371 | 372 | This method returns once the action has been acknowledged by the DRM 373 | system, but does not necessarily wait until the action has been 374 | completed. Some DRMAA implementations may allow this method to be 375 | used to control jobs submitted external to the DRMAA session, such as 376 | jobs submitted by other DRMAA session in other DRMAA implementations 377 | or jobs submitted via native utilities. 378 | """ 379 | if isinstance(jobId, str): 380 | jobId = jobId.encode(ENCODING) 381 | c(drmaa_control, jobId, string_to_control_action(operation)) 382 | 383 | # takes string list, num value and boolean, no return value 384 | @staticmethod 385 | def synchronize(jobIds, timeout=-1, dispose=False): 386 | """ 387 | Waits until all jobs specified by jobList have finished execution. 388 | 389 | :Parameters: 390 | jobIds 391 | If jobIds contains `Session.JOB_IDS_SESSION_ALL`, then this 392 | method waits for all jobs submitted during this DRMAA session up to 393 | the moment synchronize() is called 394 | timeout : int 395 | maximum time (in seconds) to be waited for the completion of a job. 396 | 397 | The value `Session.TIMEOUT_WAIT_FOREVER` may be specified to wait 398 | indefinitely for a result. The value `Session.TIMEOUT_NO_WAIT` may 399 | be specified to return immediately if no result is available. 400 | dispose : bool 401 | specifies how to treat the reaping of the remote job's internal 402 | data record, which includes a record of the job's consumption of 403 | system resources during its execution and other statistical 404 | information. If set to True, the DRM will dispose of the job's 405 | data record at the end of the synchronize() call. If set to 406 | False, the data record will be left for future access via the 407 | wait() method. It is the responsibility of the application to 408 | make sure that either `synchronize()` or `wait()`is called for 409 | every job. Not doing so creates a memory leak. Note that calling 410 | synchronize() with dispose set to true flushes all accounting 411 | information for all jobs in the list. 412 | 413 | To avoid thread race conditions in multithreaded applications, the 414 | DRMAA implementation user should explicitly synchronize this call with 415 | any other job submission calls or control calls that may change the 416 | number of remote jobs. 417 | 418 | If the call exits before the timeout has elapsed, all the jobs have 419 | been waited on or there was an interrupt. If the invocation exits on 420 | timeout, an ExitTimeoutException is thrown. The caller should check 421 | system time before and after this call in order to be sure of how much 422 | time has passed. 423 | """ 424 | if dispose: 425 | d = 1 426 | else: 427 | d = 0 428 | c(drmaa_synchronize, string_vector(jobIds), timeout, d) 429 | 430 | # takes string and long, returns JobInfo instance 431 | @staticmethod 432 | def wait(jobId, timeout=-1): 433 | """ 434 | Wait for a job with jobId to finish execution or fail. 435 | 436 | :Parameters: 437 | `jobId` : str 438 | The job id to wait completion for. 439 | 440 | If the special string, `Session.JOB_IDS_SESSION_ANY`, is provided 441 | as the jobId, this routine will wait for any job from the session 442 | `timeout` : float 443 | The timeout value is used to specify the desired behavior when a 444 | result is not immediately available. 445 | 446 | The value `Session.TIMEOUT_WAIT_FOREVER` may be specified to wait 447 | indefinitely for a result. The value `Session.TIMEOUT_NO_WAIT` may 448 | be specified to return immediately if no result is 449 | available. Alternatively, a number of seconds may be specified to 450 | indicate how long to wait for a result to become available 451 | 452 | This routine is modeled on the wait3 POSIX routine. If the call exits 453 | before timeout, either the job has been waited on successfully or 454 | there was an interrupt. If the invocation exits on timeout, an 455 | `ExitTimeoutException` is thrown. The caller should check system time 456 | before and after this call in order to be sure how much time has 457 | passed. The routine reaps job data records on a successful call, so 458 | any subsequent calls to wait() will fail, throwing an 459 | `InvalidJobException`, meaning that the job's data record has been 460 | already reaped. This exception is the same as if the job were 461 | unknown. (The only case where wait() can be successfully called on a 462 | single job more than once is when the previous call to wait() timed 463 | out before the job finished.) 464 | """ 465 | stat = c_int() 466 | jid_out = create_string_buffer(128) 467 | rusage = pointer(POINTER(drmaa_attr_values_t)()) 468 | if isinstance(jobId, str): 469 | jobId = jobId.encode(ENCODING) 470 | c(drmaa_wait, jobId, jid_out, sizeof(jid_out), byref(stat), timeout, 471 | rusage) 472 | res_usage = adapt_rusage(rusage) 473 | exited = c_int() 474 | c(drmaa_wifexited, byref(exited), stat) 475 | aborted = c_int() 476 | c(drmaa_wifaborted, byref(aborted), stat) 477 | signaled = c_int() 478 | c(drmaa_wifsignaled, byref(signaled), stat) 479 | coredumped = c_int() 480 | if exited.value == 0: 481 | c(drmaa_wcoredump, byref(coredumped), stat) 482 | exit_status = c_int() 483 | c(drmaa_wexitstatus, byref(exit_status), stat) 484 | term_signal = create_string_buffer(SIGNAL_BUFFER) 485 | if signaled.value == 1: 486 | c(drmaa_wtermsig, term_signal, sizeof(term_signal), stat) 487 | return JobInfo(jid_out.value.decode(), bool(exited), bool(signaled), 488 | term_signal.value.decode(), bool(coredumped), 489 | bool(aborted), int(exit_status.value), res_usage) 490 | 491 | # takes string, returns JobState instance 492 | @staticmethod 493 | def jobStatus(jobId): 494 | """ 495 | returns the program status of the job identified by jobId. 496 | 497 | The possible values returned from 498 | this method are: 499 | 500 | * `JobState.UNDETERMINED`: process status cannot be determined, 501 | * `JobState.QUEUED_ACTIVE`: job is queued and active, 502 | * `JobState.SYSTEM_ON_HOLD`: job is queued and in system hold, 503 | * `JobState.USER_ON_HOLD`: job is queued and in user hold, 504 | * `JobState.USER_SYSTEM_ON_HOLD`: job is queued and in user and 505 | system hold, 506 | * `JobState.RUNNING`: job is running, 507 | * `JobState.SYSTEM_SUSPENDED`: job is system suspended, 508 | * `JobState.USER_SUSPENDED`: job is user suspended, 509 | * `JobState.DONE`: job finished normally, and 510 | * `JobState.FAILED`: job finished, but failed. 511 | 512 | The DRMAA implementation should always get the status of the job from 513 | the DRM system unless the status has already been determined to be 514 | FAILED or DONE and the status has been successfully cached. Terminated 515 | jobs return a FAILED status. 516 | """ 517 | status = c_int() 518 | if isinstance(jobId, str): 519 | jobId = jobId.encode(ENCODING) 520 | c(drmaa_job_ps, jobId, byref(status)) 521 | return status_to_string(status.value) 522 | 523 | def __enter__(self): 524 | """Context manager enter function""" 525 | self.initialize(self.contactString) 526 | return self 527 | 528 | def __exit__(self, *_): 529 | """Context manager exit function.""" 530 | self.exit() 531 | return False 532 | -------------------------------------------------------------------------------- /drmaa/version.py: -------------------------------------------------------------------------------- 1 | # ----------------------------------------------------------- 2 | # Copyright (C) 2008 StatPro Italia s.r.l. 3 | # 4 | # StatPro Italia 5 | # Via G. B. Vico 4 6 | # I-20123 Milano 7 | # ITALY 8 | # 9 | # phone: +39 02 96875 1 10 | # fax: +39 02 96875 605 11 | # 12 | # This program is distributed in the hope that it will be 13 | # useful, but WITHOUT ANY WARRANTY; without even the 14 | # warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR 15 | # PURPOSE. See the license for more details. 16 | # ----------------------------------------------------------- 17 | # 18 | ''' 19 | This module exists solely for version information so I only have to change it 20 | in one place. Based on the suggestion `here. `_ 21 | 22 | :author: Dan Blanchard (dan.blanchard@gmail.com) 23 | ''' 24 | 25 | __version__ = '0.7.9' 26 | VERSION = tuple(int(x) for x in __version__.split('.')) 27 | -------------------------------------------------------------------------------- /drmaa/wrappers.py: -------------------------------------------------------------------------------- 1 | # ----------------------------------------------------------- 2 | # Copyright (C) 2009 StatPro Italia s.r.l. 3 | # 4 | # StatPro Italia 5 | # Via G. B. Vico 4 6 | # I-20123 Milano 7 | # ITALY 8 | # 9 | # phone: +39 02 96875 1 10 | # fax: +39 02 96875 605 11 | # 12 | # email: info@riskmap.net 13 | # 14 | # This program is distributed in the hope that it will be 15 | # useful, but WITHOUT ANY WARRANTY; without even the 16 | # warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR 17 | # PURPOSE. See the license for more details. 18 | # ----------------------------------------------------------- 19 | # 20 | # Author: Enrico Sirola 21 | 22 | """DRMAA C library function wrappers""" 23 | 24 | from __future__ import absolute_import, print_function, unicode_literals 25 | 26 | import os 27 | import sys 28 | from ctypes import (c_char_p, c_int, c_long, c_size_t, c_uint, c_ulong, CDLL, 29 | POINTER, RTLD_GLOBAL, sizeof, Structure) 30 | from ctypes.util import find_library 31 | 32 | from drmaa.const import ENCODING 33 | from drmaa.errors import error_check, error_buffer 34 | 35 | 36 | # Python 3 compatability help 37 | if sys.version_info < (3, 0): 38 | bytes = str 39 | str = unicode 40 | 41 | 42 | # the name of the OS environment variable optionally 43 | # containing the full path to the drmaa library 44 | _drmaa_lib_env_name = 'DRMAA_LIBRARY_PATH' 45 | 46 | if _drmaa_lib_env_name in os.environ: 47 | libpath = os.environ[_drmaa_lib_env_name] 48 | else: 49 | libpath = find_library('drmaa') 50 | 51 | if libpath is None: 52 | raise RuntimeError(('Could not find drmaa library. Please specify its ' + 53 | 'full path using the environment variable ' + 54 | '{0}').format(_drmaa_lib_env_name)) 55 | 56 | _lib = CDLL(libpath, mode=RTLD_GLOBAL) 57 | 58 | STRING = c_char_p 59 | size_t = c_ulong 60 | ptrdiff_t = c_int 61 | 62 | drmaa_init = _lib.drmaa_init 63 | drmaa_init.restype = error_check 64 | drmaa_init.argtypes = [STRING, STRING, size_t] 65 | drmaa_exit = _lib.drmaa_exit 66 | drmaa_exit.restype = error_check 67 | drmaa_exit.argtypes = [STRING, size_t] 68 | 69 | 70 | def py_drmaa_init(contact=None): 71 | if isinstance(contact, str): 72 | contact = contact.encode(ENCODING) 73 | return _lib.drmaa_init(contact, error_buffer, sizeof(error_buffer)) 74 | 75 | _lib.drmaa_exit.argtypes = [c_char_p, c_size_t] 76 | _lib.drmaa_init.restype = error_check 77 | 78 | 79 | def py_drmaa_exit(): 80 | return _lib.drmaa_exit(error_buffer, sizeof(error_buffer)) 81 | 82 | 83 | # structures 84 | class drmaa_job_template_s(Structure): 85 | pass 86 | drmaa_job_template_t = drmaa_job_template_s 87 | drmaa_job_template_s._fields_ = [] 88 | 89 | 90 | class drmaa_attr_names_s(Structure): 91 | pass 92 | drmaa_attr_names_t = drmaa_attr_names_s 93 | drmaa_attr_names_s._fields_ = [] 94 | 95 | 96 | class drmaa_attr_values_s(Structure): 97 | pass 98 | drmaa_attr_values_t = drmaa_attr_values_s 99 | drmaa_attr_values_s._fields_ = [] 100 | 101 | 102 | class drmaa_job_ids_s(Structure): 103 | pass 104 | drmaa_job_ids_t = drmaa_job_ids_s 105 | drmaa_job_ids_s._fields_ = [] 106 | 107 | drmaa_get_contact = _lib.drmaa_get_contact 108 | drmaa_get_contact.restype = error_check 109 | drmaa_get_contact.argtypes = [STRING, size_t, STRING, size_t] 110 | drmaa_version = _lib.drmaa_version 111 | drmaa_version.restype = error_check 112 | drmaa_version.argtypes = [POINTER(c_uint), POINTER(c_uint), STRING, size_t] 113 | drmaa_get_DRM_system = _lib.drmaa_get_DRM_system 114 | drmaa_get_DRM_system.restype = error_check 115 | drmaa_get_DRM_system.argtypes = [STRING, size_t, STRING, size_t] 116 | drmaa_get_DRMAA_implementation = _lib.drmaa_get_DRMAA_implementation 117 | drmaa_get_DRMAA_implementation.restype = error_check 118 | drmaa_get_DRMAA_implementation.argtypes = [STRING, size_t, STRING, size_t] 119 | 120 | drmaa_allocate_job_template = _lib.drmaa_allocate_job_template 121 | drmaa_allocate_job_template.restype = error_check 122 | drmaa_allocate_job_template.argtypes = [POINTER(POINTER(drmaa_job_template_t)), 123 | STRING, size_t] 124 | drmaa_delete_job_template = _lib.drmaa_delete_job_template 125 | drmaa_delete_job_template.restype = error_check 126 | drmaa_delete_job_template.argtypes = [POINTER(drmaa_job_template_t), STRING, 127 | size_t] 128 | drmaa_set_attribute = _lib.drmaa_set_attribute 129 | drmaa_set_attribute.restype = error_check 130 | drmaa_set_attribute.argtypes = [POINTER(drmaa_job_template_t), STRING, 131 | STRING, STRING, size_t] 132 | drmaa_get_attribute = _lib.drmaa_get_attribute 133 | drmaa_get_attribute.restype = error_check 134 | drmaa_get_attribute.argtypes = [POINTER(drmaa_job_template_t), STRING, 135 | STRING, size_t, STRING, size_t] 136 | 137 | drmaa_get_next_attr_name = _lib.drmaa_get_next_attr_name 138 | drmaa_get_next_attr_name.restype = c_int 139 | drmaa_get_next_attr_name.argtypes = [POINTER(drmaa_attr_names_t), STRING, 140 | size_t] 141 | drmaa_get_next_attr_value = _lib.drmaa_get_next_attr_value 142 | drmaa_get_next_attr_value.restype = c_int 143 | drmaa_get_next_attr_value.argtypes = [POINTER(drmaa_attr_values_t), STRING, 144 | size_t] 145 | drmaa_get_next_job_id = _lib.drmaa_get_next_job_id 146 | drmaa_get_next_job_id.restype = error_check 147 | drmaa_get_next_job_id.argtypes = [POINTER(drmaa_job_ids_t), STRING, size_t] 148 | drmaa_release_attr_names = _lib.drmaa_release_attr_names 149 | drmaa_release_attr_names.restype = None 150 | drmaa_release_attr_names.argtypes = [POINTER(drmaa_attr_names_t)] 151 | drmaa_release_attr_values = _lib.drmaa_release_attr_values 152 | drmaa_release_attr_values.restype = None 153 | drmaa_release_attr_values.argtypes = [POINTER(drmaa_attr_values_t)] 154 | drmaa_release_job_ids = _lib.drmaa_release_job_ids 155 | drmaa_release_job_ids.restype = None 156 | drmaa_release_job_ids.argtypes = [POINTER(drmaa_job_ids_t)] 157 | 158 | drmaa_set_vector_attribute = _lib.drmaa_set_vector_attribute 159 | drmaa_set_vector_attribute.restype = error_check 160 | drmaa_set_vector_attribute.argtypes = [POINTER(drmaa_job_template_t), STRING, 161 | POINTER(STRING), STRING, size_t] 162 | drmaa_get_vector_attribute = _lib.drmaa_get_vector_attribute 163 | drmaa_get_vector_attribute.restype = error_check 164 | drmaa_get_vector_attribute.argtypes = [POINTER(drmaa_job_template_t), STRING, 165 | POINTER(POINTER(drmaa_attr_values_t)), 166 | STRING, size_t] 167 | drmaa_get_attribute_names = _lib.drmaa_get_attribute_names 168 | drmaa_get_attribute_names.restype = error_check 169 | drmaa_get_attribute_names.argtypes = [POINTER(POINTER(drmaa_attr_names_t)), 170 | STRING, size_t] 171 | drmaa_get_vector_attribute_names = _lib.drmaa_get_vector_attribute_names 172 | drmaa_get_vector_attribute_names.restype = error_check 173 | drmaa_get_vector_attribute_names.argtypes = [POINTER(POINTER(drmaa_attr_names_t)), 174 | STRING, size_t] 175 | 176 | try: 177 | drmaa_get_num_attr_names = _lib.drmaa_get_num_attr_names 178 | drmaa_get_num_attr_names.restype = c_int 179 | drmaa_get_num_attr_names.argtypes = [POINTER(drmaa_attr_names_t), 180 | POINTER(c_int)] 181 | drmaa_get_num_attr_values = _lib.drmaa_get_num_attr_values 182 | drmaa_get_num_attr_values.restype = c_int 183 | drmaa_get_num_attr_values.argtypes = [POINTER(drmaa_attr_values_t), 184 | POINTER(c_int)] 185 | except AttributeError: # the above are present from 1.0 onward only 186 | pass 187 | 188 | drmaa_run_job = _lib.drmaa_run_job 189 | drmaa_run_job.restype = error_check 190 | drmaa_run_job.argtypes = [STRING, size_t, POINTER(drmaa_job_template_t), STRING, 191 | size_t] 192 | drmaa_run_bulk_jobs = _lib.drmaa_run_bulk_jobs 193 | drmaa_run_bulk_jobs.restype = error_check 194 | drmaa_run_bulk_jobs.argtypes = [POINTER(POINTER(drmaa_job_ids_t)), 195 | POINTER(drmaa_job_template_t), 196 | c_int, c_int, c_int, STRING, size_t] 197 | drmaa_control = _lib.drmaa_control 198 | drmaa_control.restype = error_check 199 | drmaa_control.argtypes = [STRING, c_int, STRING, size_t] 200 | drmaa_synchronize = _lib.drmaa_synchronize 201 | drmaa_synchronize.restype = error_check 202 | drmaa_synchronize.argtypes = [POINTER(STRING), c_long, c_int, STRING, size_t] 203 | drmaa_wait = _lib.drmaa_wait 204 | drmaa_wait.restype = error_check 205 | drmaa_wait.argtypes = [STRING, STRING, size_t, POINTER(c_int), c_long, 206 | POINTER(POINTER(drmaa_attr_values_t)), STRING, size_t] 207 | drmaa_wifexited = _lib.drmaa_wifexited 208 | drmaa_wifexited.restype = error_check 209 | drmaa_wifexited.argtypes = [POINTER(c_int), c_int, STRING, size_t] 210 | drmaa_wexitstatus = _lib.drmaa_wexitstatus 211 | drmaa_wexitstatus.restype = error_check 212 | drmaa_wexitstatus.argtypes = [POINTER(c_int), c_int, STRING, size_t] 213 | drmaa_wifsignaled = _lib.drmaa_wifsignaled 214 | drmaa_wifsignaled.restype = error_check 215 | drmaa_wifsignaled.argtypes = [POINTER(c_int), c_int, STRING, size_t] 216 | drmaa_wtermsig = _lib.drmaa_wtermsig 217 | drmaa_wtermsig.restype = error_check 218 | drmaa_wtermsig.argtypes = [STRING, size_t, c_int, STRING, size_t] 219 | drmaa_wcoredump = _lib.drmaa_wcoredump 220 | drmaa_wcoredump.restype = error_check 221 | drmaa_wcoredump.argtypes = [POINTER(c_int), c_int, STRING, size_t] 222 | drmaa_wifaborted = _lib.drmaa_wifaborted 223 | drmaa_wifaborted.restype = error_check 224 | drmaa_wifaborted.argtypes = [POINTER(c_int), c_int, STRING, size_t] 225 | drmaa_job_ps = _lib.drmaa_job_ps 226 | drmaa_job_ps.restype = error_check 227 | drmaa_job_ps.argtypes = [STRING, POINTER(c_int), STRING, size_t] 228 | drmaa_strerror = _lib.drmaa_strerror 229 | drmaa_strerror.restype = STRING 230 | drmaa_strerror.argtypes = [c_int] 231 | -------------------------------------------------------------------------------- /examples/example.py: -------------------------------------------------------------------------------- 1 | #!/bin/python 2 | 3 | from __future__ import print_function 4 | import drmaa 5 | import os 6 | from sys import argv, exit 7 | from time import sleep 8 | from pprint import pprint 9 | 10 | NBULKS = 1 11 | JOB_CHUNK = 3 12 | 13 | OPATH=":"+drmaa.JobTemplate.HOME_DIRECTORY+'/DRMAA_JOB_OUT' 14 | 15 | def init_job_template(jt, path, args, as_bulk_job): 16 | jt.workingDirectory = drmaa.JobTemplate.HOME_DIRECTORY 17 | jt.environment = {'a': 'b', 'c':'d',} 18 | jt.remoteCommand = path 19 | jt.args = args 20 | jt.joinFiles=True 21 | # jt.startTime=datetime.now() 22 | # print("start time is: ", jt.startTime) 23 | if as_bulk_job: 24 | jt.outputPath=OPATH+'.'+drmaa.JobTemplate.PARAMETRIC_INDEX 25 | else: 26 | jt.outputPath=OPATH 27 | return jt 28 | 29 | 30 | def main(): 31 | if len(argv) < 2: 32 | print("usage: example.py ") 33 | exit(1) 34 | job_path=argv[1] 35 | s=drmaa.Session() 36 | s.initialize() 37 | # submit some bulk jobs 38 | jt=init_job_template(s.createJobTemplate(), job_path, argv[2:], True) 39 | all_jobids = [] 40 | for i in range(NBULKS): 41 | all_jobids += s.runBulkJobs(jt, 1, JOB_CHUNK, 1) 42 | sleep(1) 43 | print("submitted bulk jobs with jobids:") 44 | pprint(all_jobids) 45 | # submit some sequential jobs 46 | s.deleteJobTemplate(jt) 47 | del jt 48 | jt=init_job_template(s.createJobTemplate(), job_path, argv[2:], False) 49 | for i in range(NBULKS): 50 | all_jobids.append(s.runJob(jt)) 51 | sleep(1) 52 | s.synchronize(all_jobids, 53 | drmaa.Session.TIMEOUT_WAIT_FOREVER, 54 | False) 55 | print("synchronized with all jobs") 56 | for jid in all_jobids: 57 | print('-' * 76) 58 | info=s.wait(jid, drmaa.Session.TIMEOUT_WAIT_FOREVER) 59 | print("""\ 60 | id: %(jobId)s 61 | exited: %(hasExited)s 62 | signaled: %(hasSignal)s 63 | with signal (id signaled): %(terminatedSignal)s 64 | dumped core: %(hasCoreDump)s 65 | aborted: %(wasAborted)s 66 | resource usage: 67 | 68 | %(resourceUsage)s 69 | """ % info._asdict()) 70 | 71 | 72 | if __name__=='__main__': 73 | main() 74 | -------------------------------------------------------------------------------- /examples/example1.1.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | from __future__ import print_function 4 | import drmaa 5 | 6 | 7 | def main(): 8 | """Create a session, show that each session has an id, 9 | use session id to disconnect, then reconnect. Then exit""" 10 | s = drmaa.Session() 11 | s.initialize() 12 | print('A session was started successfully') 13 | response = s.contact 14 | print('session contact returns: ' + response) 15 | s.exit() 16 | print('Exited from session') 17 | 18 | s.initialize(response) 19 | print('Session was restarted successfully') 20 | s.exit() 21 | 22 | 23 | if __name__=='__main__': 24 | main() 25 | -------------------------------------------------------------------------------- /examples/example1.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | from __future__ import print_function 4 | import drmaa 5 | 6 | 7 | def main(): 8 | """Create a drmaa session and exit""" 9 | s=drmaa.Session() 10 | s.initialize() 11 | print('A session was started successfully') 12 | s.exit() 13 | 14 | if __name__=='__main__': 15 | main() 16 | -------------------------------------------------------------------------------- /examples/example2.1.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | from __future__ import print_function 4 | import drmaa 5 | import os 6 | 7 | 8 | def main(): 9 | """Submit an array job. 10 | Note, need file called sleeper.sh in home directory. 11 | """ 12 | s = drmaa.Session() 13 | s.initialize() 14 | print('Creating job template') 15 | jt = s.createJobTemplate() 16 | jt.remoteCommand = os.getcwd() + '/sleeper.sh' 17 | jt.args = ['42','Simon says:'] 18 | jt.joinFiles=True 19 | 20 | jobid = s.runBulkJobs(jt,1,30,2) 21 | print('Your job has been submitted with id ' + str(jobid)) 22 | 23 | print('Cleaning up') 24 | s.deleteJobTemplate(jt) 25 | s.exit() 26 | 27 | if __name__=='__main__': 28 | main() 29 | -------------------------------------------------------------------------------- /examples/example2.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | from __future__ import print_function 4 | import drmaa 5 | import os 6 | 7 | 8 | def main(): 9 | """Submit a job. 10 | Note, need file called sleeper.sh in current directory. 11 | """ 12 | s = drmaa.Session() 13 | s.initialize() 14 | print('Creating job template') 15 | jt = s.createJobTemplate() 16 | jt.remoteCommand = os.getcwd() + '/sleeper.sh' 17 | jt.args = ['42','Simon says:'] 18 | jt.joinFiles=True 19 | 20 | jobid = s.runJob(jt) 21 | print('Your job has been submitted with id ' + jobid) 22 | 23 | print('Cleaning up') 24 | s.deleteJobTemplate(jt) 25 | s.exit() 26 | 27 | if __name__=='__main__': 28 | main() 29 | -------------------------------------------------------------------------------- /examples/example3.1.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | from __future__ import print_function 4 | import drmaa 5 | import os 6 | 7 | def main(): 8 | """Submit an array job, and wait for all of them to finish. 9 | Note, need file called sleeper.sh in home directory. 10 | """ 11 | s = drmaa.Session() 12 | s.initialize() 13 | print('Creating job template') 14 | jt = s.createJobTemplate() 15 | jt.remoteCommand = os.getcwd() + '/sleeper.sh' 16 | jt.args = ['42','Simon says:'] 17 | jt.joinFiles=True 18 | 19 | joblist = s.runBulkJobs(jt,1,30,2) 20 | print('Your job has been submitted with id ' + str(joblist)) 21 | 22 | s.synchronize(joblist, drmaa.Session.TIMEOUT_WAIT_FOREVER, True) 23 | 24 | print('Cleaning up') 25 | s.deleteJobTemplate(jt) 26 | s.exit() 27 | 28 | if __name__=='__main__': 29 | main() 30 | -------------------------------------------------------------------------------- /examples/example3.2.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | from __future__ import print_function 4 | import drmaa 5 | import os 6 | 7 | 8 | def main(): 9 | """Submit an array job, wait for them to finish, and collect results. 10 | This is slightly different from the example from sunsource because it only 11 | waits for the jobs that it submitted. 12 | """ 13 | s = drmaa.Session() 14 | s.initialize() 15 | print('Creating job template') 16 | jt = s.createJobTemplate() 17 | jt.remoteCommand = os.getcwd() + '/sleeper.sh' 18 | jt.args = ['42','Simon says:'] 19 | jt.joinFiles=True 20 | 21 | joblist = s.runBulkJobs(jt,1,30,2) 22 | print('Your job has been submitted with id ' + str(joblist)) 23 | 24 | s.synchronize(joblist, drmaa.Session.TIMEOUT_WAIT_FOREVER, False) 25 | for curjob in joblist: 26 | print('Collecting job ' + curjob) 27 | retval = s.wait(curjob, drmaa.Session.TIMEOUT_WAIT_FOREVER) 28 | print('Job: ' + str(retval.jobId) + ' finished with status ' + str(retval.hasExited)) 29 | 30 | print('Cleaning up') 31 | s.deleteJobTemplate(jt) 32 | s.exit() 33 | 34 | if __name__=='__main__': 35 | main() 36 | -------------------------------------------------------------------------------- /examples/example3.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | from __future__ import print_function 4 | import drmaa 5 | import os 6 | 7 | 8 | def main(): 9 | """Submit a job and wait for it to finish. 10 | Note, need file called sleeper.sh in home directory. 11 | """ 12 | s = drmaa.Session() 13 | s.initialize() 14 | print('Creating job template') 15 | jt = s.createJobTemplate() 16 | jt.remoteCommand = os.getcwd() + '/sleeper.sh' 17 | jt.args = ['42','Simon says:'] 18 | jt.joinFiles = True 19 | 20 | jobid = s.runJob(jt) 21 | print('Your job has been submitted with id ' + jobid) 22 | 23 | retval = s.wait(jobid, drmaa.Session.TIMEOUT_WAIT_FOREVER) 24 | print('Job: ' + str(retval.jobId) + ' finished with status ' + str(retval.hasExited)) 25 | 26 | print('Cleaning up') 27 | s.deleteJobTemplate(jt) 28 | s.exit() 29 | 30 | 31 | if __name__=='__main__': 32 | main() 33 | -------------------------------------------------------------------------------- /examples/example4.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | from __future__ import print_function 4 | import drmaa 5 | import os 6 | 7 | def main(): 8 | """Submit a job, then kill it. 9 | Note, need file called sleeper.sh in home directory. 10 | """ 11 | s = drmaa.Session() 12 | s.initialize() 13 | print('Creating job template') 14 | jt = s.createJobTemplate() 15 | jt.remoteCommand = os.getcwd() + '/sleeper.sh' 16 | jt.args = ['42','Simon says:'] 17 | jt.joinFiles = True 18 | 19 | jobid = s.runJob(jt) 20 | print('Your job has been submitted with id ' + jobid) 21 | # options are: SUSPEND, RESUME, HOLD, RELEASE, TERMINATE 22 | s.control(jobid, drmaa.JobControlAction.TERMINATE) 23 | 24 | print('Cleaning up') 25 | s.deleteJobTemplate(jt) 26 | s.exit() 27 | 28 | 29 | if __name__=='__main__': 30 | main() 31 | -------------------------------------------------------------------------------- /examples/example5.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | from __future__ import print_function 4 | import drmaa 5 | import time 6 | import os 7 | 8 | 9 | def main(): 10 | """Submit a job, and check its progress. 11 | Note, need file called sleeper.sh in home directory. 12 | """ 13 | s = drmaa.Session() 14 | s.initialize() 15 | print('Creating job template') 16 | jt = s.createJobTemplate() 17 | jt.remoteCommand = os.getcwd() + '/sleeper.sh' 18 | jt.args = ['42','Simon says:'] 19 | jt.joinFiles=True 20 | 21 | jobid = s.runJob(jt) 22 | print('Your job has been submitted with id ' + jobid) 23 | 24 | # Who needs a case statement when you have dictionaries? 25 | decodestatus = { 26 | drmaa.JobState.UNDETERMINED: 'process status cannot be determined', 27 | drmaa.JobState.QUEUED_ACTIVE: 'job is queued and active', 28 | drmaa.JobState.SYSTEM_ON_HOLD: 'job is queued and in system hold', 29 | drmaa.JobState.USER_ON_HOLD: 'job is queued and in user hold', 30 | drmaa.JobState.USER_SYSTEM_ON_HOLD: 'job is queued and in user and system hold', 31 | drmaa.JobState.RUNNING: 'job is running', 32 | drmaa.JobState.SYSTEM_SUSPENDED: 'job is system suspended', 33 | drmaa.JobState.USER_SUSPENDED: 'job is user suspended', 34 | drmaa.JobState.DONE: 'job finished normally', 35 | drmaa.JobState.FAILED: 'job finished, but failed', 36 | } 37 | 38 | for ix in range(10): 39 | print('Checking ' + str(ix) + ' of 10 times') 40 | print(decodestatus[s.jobStatus(jobid)]) 41 | time.sleep(5) 42 | 43 | print('Cleaning up') 44 | s.deleteJobTemplate(jt) 45 | s.exit() 46 | 47 | if __name__=='__main__': 48 | main() 49 | -------------------------------------------------------------------------------- /examples/example6.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | from __future__ import print_function 4 | import drmaa 5 | 6 | 7 | def main(): 8 | """Query the system.""" 9 | s = drmaa.Session() 10 | s.initialize() 11 | print('A DRMAA object was created') 12 | print('Supported contact strings: ' + s.contact) 13 | print('Supported DRM systems: ' + str(s.drmsInfo)) 14 | print('Supported DRMAA implementations: ' + str(s.drmaaImplementation)) 15 | print('Version ' + str(s.version)) 16 | 17 | print('Exiting') 18 | s.exit() 19 | 20 | if __name__=='__main__': 21 | main() 22 | -------------------------------------------------------------------------------- /examples/sleeper.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | echo "Hello World, the answer is $1" 3 | sleep 3s 4 | echo "$2 Bye world!" 5 | 6 | -------------------------------------------------------------------------------- /license.txt: -------------------------------------------------------------------------------- 1 | Copyright (c) 2009, StatPro Italia srl 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without 5 | modification, are permitted provided that the following conditions are met: 6 | * Redistributions of source code must retain the above copyright 7 | notice, this list of conditions and the following disclaimer. 8 | * Redistributions in binary form must reproduce the above copyright 9 | notice, this list of conditions and the following disclaimer in the 10 | documentation and/or other materials provided with the distribution. 11 | * Neither the name of StatPro Italia srl nor the 12 | names of the contributors may be used to endorse or promote products 13 | derived from this software without specific prior written permission. 14 | 15 | THIS SOFTWARE IS PROVIDED BY StatPro Italia srl ``AS IS'' AND ANY 16 | EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 17 | WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 18 | DISCLAIMED. IN NO EVENT SHALL StatPro Italia srl BE LIABLE FOR ANY 19 | DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 20 | (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 21 | LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND 22 | ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 23 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 24 | SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 25 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [metadata] 2 | license_file = license.txt 3 | 4 | [bdist_wheel] 5 | universal = 1 6 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | # ----------------------------------------------------------- 2 | # Copyright (C) 2009 StatPro Italia s.r.l. 3 | # 4 | # StatPro Italia 5 | # Via G. B. Vico 4 6 | # I-20123 Milano 7 | # ITALY 8 | # 9 | # phone: +39 02 96875 1 10 | # fax: +39 02 96875 605 11 | # 12 | # This program is distributed in the hope that it will be 13 | # useful, but WITHOUT ANY WARRANTY; without even the 14 | # warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR 15 | # PURPOSE. See the license for more details. 16 | # ----------------------------------------------------------- 17 | # 18 | # Author: Enrico Sirola 19 | # Author: Dan Blanchard 20 | 21 | from setuptools import setup, find_packages 22 | 23 | packages = find_packages() 24 | package_data = dict([ (x, ['test/*.py']) for x in packages]) 25 | 26 | # To get around the fact that you can't import stuff from packages in setup.py 27 | exec(compile(open('drmaa/version.py').read(), 'drmaa/version.py', 'exec')) 28 | # (we use the above instead of execfile for Python 3.x compatibility) 29 | 30 | def readme(): 31 | with open('README.rst') as f: 32 | return f.read() 33 | 34 | setup(name="drmaa", 35 | version=__version__, 36 | packages=packages, 37 | package_data=package_data, 38 | author="Dan Blanchard", 39 | author_email="dan.blanchard@gmail.com", 40 | description="a python DRMAA library", 41 | long_description=readme(), 42 | license="BSD", 43 | keywords="python grid hpc drmaa", 44 | url="https://github.com/pygridtools/drmaa-python", 45 | tests_require='nose', 46 | test_suite='nose.collector', 47 | classifiers=["Development Status :: 4 - Beta", 48 | "Operating System :: OS Independent", 49 | "Intended Audience :: System Administrators", 50 | "Intended Audience :: Developers", 51 | "Intended Audience :: Science/Research", 52 | "License :: OSI Approved :: BSD License", 53 | "Programming Language :: Python :: 2", 54 | "Programming Language :: Python :: 3", 55 | "Topic :: Software Development :: Libraries", 56 | "Topic :: System :: Distributed Computing"]) 57 | -------------------------------------------------------------------------------- /test/testcontext.py: -------------------------------------------------------------------------------- 1 | ''' 2 | Tests related to with using context managers. 3 | ''' 4 | 5 | from __future__ import absolute_import, print_function, unicode_literals 6 | 7 | from drmaa import Session 8 | 9 | 10 | def test_with_session(): 11 | """'with' statement works with Session""" 12 | with Session() as s: 13 | print(s.version) 14 | print(s.contact) 15 | print(s.drmsInfo) 16 | print(s.drmaaImplementation) 17 | 18 | 19 | def test_with_jt(): 20 | """'with' statement works with JobTemplate""" 21 | s = Session() 22 | s.initialize() 23 | with s.createJobTemplate() as jt: 24 | jt.remoteCommand = 'sleep' 25 | jt.args = ['10'] 26 | jid = s.runJob(jt) 27 | print(s.wait(jid)) 28 | s.exit() 29 | -------------------------------------------------------------------------------- /test/testmisc.py: -------------------------------------------------------------------------------- 1 | ''' 2 | Test miscenallenous helper functions. 3 | ''' 4 | 5 | from __future__ import absolute_import, print_function, unicode_literals 6 | 7 | from unittest import TestCase 8 | 9 | from drmaa import Session 10 | 11 | 12 | class Misc(TestCase): 13 | 14 | def setUp(self): 15 | self.s = Session() 16 | 17 | def test_drmaa_get_contact(self): 18 | """contact attribute works""" 19 | print(self.s.contact) 20 | 21 | def test_drmaa_version(self): 22 | """version attribute works""" 23 | print(self.s.version) 24 | 25 | def test_drmaa_get_DRM_system(self): 26 | """DRM system attribute works""" 27 | print(self.s.drmsInfo) 28 | 29 | def test_drmaa_get_DRMAA_implementation(self): 30 | """DRMAA implementation attribute works""" 31 | print(self.s.drmaaImplementation) 32 | -------------------------------------------------------------------------------- /test/testwrap.py: -------------------------------------------------------------------------------- 1 | # ----------------------------------------------------------- 2 | # Copyright (C) 2009 StatPro Italia s.r.l. 3 | # 4 | # StatPro Italia 5 | # Via G. B. Vico 4 6 | # I-20123 Milano 7 | # ITALY 8 | # 9 | # phone: +39 02 96875 1 10 | # fax: +39 02 96875 605 11 | # 12 | # email: info@riskmap.net 13 | # 14 | # This program is distributed in the hope that it will be 15 | # useful, but WITHOUT ANY WARRANTY; without even the 16 | # warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR 17 | # PURPOSE. See the license for more details. 18 | # ----------------------------------------------------------- 19 | 20 | """ 21 | test module for the functional interface 22 | """ 23 | 24 | from __future__ import absolute_import, print_function, unicode_literals 25 | 26 | import sys 27 | import unittest 28 | from os import environ 29 | 30 | from nose.tools import eq_ 31 | 32 | from drmaa import Session, JobTemplate 33 | from drmaa.const import (JobControlAction, JobSubmissionState, PLACEHOLDER_HD, 34 | SUBMISSION_STATE_ACTIVE) 35 | 36 | 37 | # Python 3 compatability help 38 | if sys.version_info < (3, 0): 39 | bytes = str 40 | str = unicode 41 | 42 | 43 | def setup(): 44 | "initialize DRMAA library" 45 | Session.initialize() 46 | 47 | 48 | def teardown(): 49 | "finalize DRMAA session" 50 | Session.exit() 51 | 52 | 53 | def test_allocate(): 54 | "job template allocation" 55 | jt = Session.createJobTemplate() 56 | Session.deleteJobTemplate(jt) 57 | 58 | 59 | class SubmitBase(unittest.TestCase): 60 | 61 | def setUp(self): 62 | self.jt = jt = Session.createJobTemplate() 63 | jt.remoteCommand = 'python' 64 | jt.args = ['-c', "print('hello from python!')"] 65 | if hasattr(self, 'jt_tweaks'): 66 | self.jt_tweaks() 67 | self.jid = Session.runJob(jt) 68 | 69 | def tearDown(self): 70 | Session.deleteJobTemplate(self.jt) 71 | 72 | 73 | class EnvironmentTest(SubmitBase): 74 | 75 | def jt_tweaks(self): 76 | environ['PIPPO'] = 'aaaa' 77 | self.jt.args = (["-c", 78 | ("from os import environ as env; assert ('PIPPO' in " + 79 | "env) and (env['PIPPO'] == 'aaaa')")]) 80 | self.jt.jobEnvironment = environ 81 | 82 | def test_environment(self): 83 | """environment variables are correctly passed to submitted jobs""" 84 | jinfo = Session.wait(self.jid) 85 | eq_(jinfo.jobId, self.jid) 86 | assert hasattr(jinfo, 'hasExited') 87 | assert hasattr(jinfo, 'exitStatus') and jinfo.exitStatus == 0 88 | 89 | 90 | class Submit(SubmitBase): 91 | 92 | def test_run_bulk(self): 93 | """run bulk job""" 94 | jids = Session.runBulkJobs(self.jt, 1, 2, 1) 95 | 96 | def test_wait(self): 97 | """waiting for job completion""" 98 | jinfo = Session.wait(self.jid) 99 | eq_(jinfo.jobId, self.jid) 100 | assert hasattr(jinfo, 'hasExited') 101 | assert hasattr(jinfo, 'hasExited') and type(jinfo.hasExited) is bool 102 | assert hasattr(jinfo, 'hasSignal') and type(jinfo.hasSignal) is bool 103 | assert hasattr(jinfo, 'terminatedSignal') and type(jinfo.terminatedSignal) is str 104 | assert hasattr(jinfo, 'hasCoreDump') and type(jinfo.hasCoreDump) is bool 105 | assert hasattr(jinfo, 'wasAborted') and type(jinfo.wasAborted) is bool 106 | assert hasattr(jinfo, 'exitStatus') and type(jinfo.exitStatus) is int 107 | assert hasattr(jinfo, 'resourceUsage') and type(jinfo.resourceUsage) is dict 108 | 109 | def test_sync(self): 110 | """sync with a job""" 111 | Session.synchronize(self.jid) 112 | 113 | def test_control_terminate(self): 114 | """control/terminate works""" 115 | Session.control(self.jid, JobControlAction.TERMINATE) 116 | Session.synchronize(self.jid, 117 | Session.TIMEOUT_WAIT_FOREVER, 118 | False) 119 | try: 120 | Session.wait(self.jid, Session.TIMEOUT_WAIT_FOREVER) 121 | except Exception as e: 122 | assert e.args[0].startswith('code 24') # no rusage 123 | 124 | 125 | class JobTemplateTests(unittest.TestCase): 126 | 127 | def setUp(self): 128 | self.jt = Session.createJobTemplate() 129 | 130 | def test_scalar_attributes(self): 131 | """scalar attributes work""" 132 | for name, value in [("remoteCommand", 'cat'), 133 | ("jobSubmissionState", SUBMISSION_STATE_ACTIVE), 134 | ("workingDirectory", JobTemplate.HOME_DIRECTORY), 135 | ("nativeSpecification", '-shell yes'), 136 | ("blockEmail", False), 137 | ("jobName", 'pippo'), 138 | ("inputPath", ":%s" % PLACEHOLDER_HD), 139 | ("outputPath", ":%s/pippo.out" % PLACEHOLDER_HD), 140 | ("errorPath", ":%s/pippo.out" % PLACEHOLDER_HD), 141 | ("joinFiles", True)]: 142 | setattr(self.jt, name, value) 143 | eq_(getattr(self.jt, name), value) 144 | 145 | # skipping this. the parameters above have to be tweaked a bit 146 | def xtest_tmp(self): 147 | self.test_scalar_attributes() 148 | self.jt.args = ['.colordb'] 149 | jid = Session.runJob(self.jt) 150 | jinfo = Session.wait(jid) 151 | print(jinfo) 152 | 153 | def test_vector_attributes(self): 154 | """vector attributes work""" 155 | args = [10, 'de', 'arglebargle'] 156 | self.jt.args = args 157 | eq_(self.jt.args, ['10', 'de', 'arglebargle']) 158 | em = ['baz@quz.edu', 'foo@bar.com'] 159 | self.jt.email = em 160 | eq_(self.jt.email, em) 161 | 162 | def test_dict_attribute(self): 163 | """dict attributes work""" 164 | from drmaa.const import ATTR_BUFFER 165 | self.jt.jobEnvironment = environ 166 | for x in environ: 167 | # attribute values could be truncated. For some reason, 168 | # GE returns the first 1014 chars available (!) 169 | eq_(environ[x][:ATTR_BUFFER - 10], 170 | self.jt.jobEnvironment[x][:ATTR_BUFFER - 10]) 171 | 172 | def test_attribute_names(self): 173 | """attribute names work""" 174 | assert len(self.jt.attributeNames) > 0 175 | 176 | def test_block_email(self): 177 | """blockEmail works""" 178 | self.jt.blockEmail = True 179 | assert self.jt.blockEmail 180 | self.jt.blockEmail = False 181 | assert not self.jt.blockEmail 182 | 183 | def test_join_files(self): 184 | """joinFiles works""" 185 | self.jt.joinFiles = True 186 | assert self.jt.joinFiles 187 | self.jt.joinFiles = False 188 | assert not self.jt.joinFiles 189 | 190 | def test_submission_state(self): 191 | """submission state attributes work""" 192 | self.jt.jobSubmissionState = JobSubmissionState.HOLD_STATE 193 | eq_(self.jt.jobSubmissionState, JobSubmissionState.HOLD_STATE) 194 | self.jt.jobSubmissionState = JobSubmissionState.ACTIVE_STATE 195 | eq_(self.jt.jobSubmissionState, JobSubmissionState.ACTIVE_STATE) 196 | 197 | def tearDown(self): 198 | Session.deleteJobTemplate(self.jt) 199 | -------------------------------------------------------------------------------- /travis/host_template: -------------------------------------------------------------------------------- 1 | hostname localhost 2 | load_scaling NONE 3 | complex_values NONE 4 | user_lists arusers 5 | xuser_lists NONE 6 | projects NONE 7 | xprojects NONE 8 | usage_scaling NONE 9 | report_variables NONE 10 | -------------------------------------------------------------------------------- /travis/install_sge.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # This script installs and configures a Sun Grid Engine installation for use 3 | # on a Travis instance. 4 | # 5 | # Written by Dan Blanchard (dan.blanchard@gmail.com), September 2013 6 | 7 | cd travis 8 | sudo apt-get update -qq 9 | echo "gridengine-master shared/gridenginemaster string localhost" | sudo debconf-set-selections 10 | echo "gridengine-master shared/gridenginecell string default" | sudo debconf-set-selections 11 | echo "gridengine-master shared/gridengineconfig boolean true" | sudo debconf-set-selections 12 | sudo apt-get install gridengine-common gridengine-master 13 | # Do this in a separate step to give master time to start 14 | sudo apt-get install libdrmaa1.0 gridengine-client gridengine-exec 15 | export CORES=$(grep -c '^processor' /proc/cpuinfo) 16 | sed -i -r "s/template/$USER/" user_template 17 | sudo qconf -Auser user_template 18 | sudo qconf -au $USER arusers 19 | sudo qconf -as localhost 20 | export LOCALHOST_IN_SEL=$(qconf -sel | grep -c 'localhost') 21 | if [ $LOCALHOST_IN_SEL != "1" ]; then sudo qconf -Ae host_template; else sudo qconf -Me host_template; fi 22 | sed -i -r "s/UNDEFINED/$CORES/" queue_template 23 | sudo qconf -Ap smp_template 24 | sudo qconf -Aq queue_template 25 | echo "Printing queue info to verify that things are working correctly." 26 | qstat -f -q all.q -explain a 27 | echo "You should see sge_execd and sge_qmaster running below:" 28 | ps aux | grep "sge" 29 | -------------------------------------------------------------------------------- /travis/queue_template: -------------------------------------------------------------------------------- 1 | qname all.q 2 | hostlist localhost 3 | seq_no 0 4 | load_thresholds np_load_avg=1.75 5 | suspend_thresholds NONE 6 | nsuspend 1 7 | suspend_interval 00:05:00 8 | priority 0 9 | min_cpu_interval 00:05:00 10 | processors UNDEFINED 11 | qtype BATCH INTERACTIVE 12 | ckpt_list NONE 13 | pe_list make smp 14 | rerun FALSE 15 | slots UNDEFINED 16 | tmpdir /tmp 17 | shell /bin/bash 18 | prolog NONE 19 | epilog NONE 20 | shell_start_mode posix_compliant 21 | starter_method NONE 22 | suspend_method NONE 23 | resume_method NONE 24 | terminate_method NONE 25 | notify 00:00:60 26 | owner_list NONE 27 | user_lists arusers 28 | xuser_lists NONE 29 | subordinate_list NONE 30 | complex_values NONE 31 | projects NONE 32 | xprojects NONE 33 | calendar NONE 34 | initial_state default 35 | s_rt INFINITY 36 | h_rt INFINITY 37 | s_cpu INFINITY 38 | h_cpu INFINITY 39 | s_fsize INFINITY 40 | h_fsize INFINITY 41 | s_data INFINITY 42 | h_data INFINITY 43 | s_stack INFINITY 44 | h_stack INFINITY 45 | s_core INFINITY 46 | h_core INFINITY 47 | s_rss INFINITY 48 | h_rss INFINITY 49 | s_vmem INFINITY 50 | h_vmem INFINITY 51 | -------------------------------------------------------------------------------- /travis/smp_template: -------------------------------------------------------------------------------- 1 | pe_name smp 2 | slots 999 3 | user_lists NONE 4 | xuser_lists NONE 5 | start_proc_args /bin/true 6 | stop_proc_args /bin/true 7 | allocation_rule $pe_slots 8 | control_slaves FALSE 9 | job_is_first_task TRUE 10 | urgency_slots min 11 | accounting_summary FALSE 12 | -------------------------------------------------------------------------------- /travis/user_template: -------------------------------------------------------------------------------- 1 | name template 2 | oticket 0 3 | fshare 0 4 | delete_time 0 5 | default_project NONE 6 | --------------------------------------------------------------------------------