├── .gitignore ├── .travis.yml ├── CODE_OF_CONDUCT.md ├── LICENSE.md ├── README.md ├── requirements-test.txt ├── setup.py ├── shellfuncs ├── __init__.py └── core.py ├── tests ├── __init__.py ├── input_scripts │ ├── __init__.py │ ├── config.sh │ └── foo.sh └── test_core.py └── tox.ini /.gitignore: -------------------------------------------------------------------------------- 1 | # vim swap files 2 | *.swp 3 | 4 | # python bytecode files 5 | __pycache__/ 6 | 7 | # python virtualenv 8 | env/ 9 | 10 | # python testing 11 | .tox 12 | .cache/ 13 | 14 | # python packaging 15 | *.egg-info 16 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: python 2 | sudo: false 3 | python: 4 | - "3.4" 5 | - "3.5" 6 | - "3.6" 7 | 8 | # command to install dependencies 9 | install: 10 | - python setup.py build sdist 11 | - pip install -r requirements-test.txt 12 | # command to run tests 13 | script: py.test tests/ 14 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation. 6 | 7 | ## Our Standards 8 | 9 | Examples of behavior that contributes to creating a positive environment include: 10 | 11 | * Using welcoming and inclusive language 12 | * Being respectful of differing viewpoints and experiences 13 | * Gracefully accepting constructive criticism 14 | * Focusing on what is best for the community 15 | * Showing empathy towards other community members 16 | 17 | Examples of unacceptable behavior by participants include: 18 | 19 | * The use of sexualized language or imagery and unwelcome sexual attention or advances 20 | * Trolling, insulting/derogatory comments, and personal or political attacks 21 | * Public or private harassment 22 | * Publishing others' private information, such as a physical or electronic address, without explicit permission 23 | * Other conduct which could reasonably be considered inappropriate in a professional setting 24 | 25 | ## Our Responsibilities 26 | 27 | Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior. 28 | 29 | Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful. 30 | 31 | ## Scope 32 | 33 | This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers. 34 | 35 | ## Enforcement 36 | 37 | Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at tuxtimo@gmail.com. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately. 38 | 39 | Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership. 40 | 41 | ## Attribution 42 | 43 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version] 44 | 45 | [homepage]: http://contributor-covenant.org 46 | [version]: http://contributor-covenant.org/version/1/4/ 47 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Timo Furrer 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # shellfuncs 2 | 3 | [![Build Status](https://travis-ci.com/timofurrer/shellfuncs.svg?token=qRcMyciKYsuEPapoF8ny&branch=master)](https://travis-ci.com/timofurrer/shellfuncs) 4 | [![PyPI package version](https://badge.fury.io/py/shellfuncs.svg)](https://badge.fury.io/py/shellfuncs) 5 | [![PyPI python versions](https://img.shields.io/pypi/pyversions/shellfuncs.svg)](https://pypi.python.org/pypi/shellfuncs) 6 | 7 | Python API to execute functions written in shell script. 8 | 9 | Let's assume you have a shell script *counters.sh* like this: 10 | 11 | ```bash 12 | count_python_imports() { 13 | find -name '*.py' | xargs grep -e '^import os$' -e '^import sys$' -e '^import re$' | cut -d: -f2 | sort | uniq -c 14 | } 15 | ``` 16 | 17 | And you want to execute the `count_python_imports` function within Python. Instead of using cumbersome *subprocess* wouldn't it be awesome to do something like this: 18 | 19 | ```python 20 | import shellfuncs 21 | 22 | from counters import count_python_imports 23 | 24 | returncode, stdout, stderr = count_python_imports() 25 | ``` 26 | 27 | *Yeah, yeah, I know about easier ways of achieving the above, too. Thanks.* 28 | 29 | ## Why should I use that? 30 | 31 | * use existing shell scripts in a pythonic way 32 | * complex piping stuff might be easier to implement in shell script 33 | * testing shell scripts is a pain in the a\*\* - with Python it'll be easier 34 | 35 | ## Installation 36 | 37 | The recommended way to install **shellfuncs** is to use `pip`: 38 | 39 | ```shell 40 | pip install shellfuncs 41 | ``` 42 | 43 | ## Usage 44 | 45 | **shellfuncs** can be configured on different levels. 46 | 47 | The following configuration variables are available: 48 | 49 | * shell (defaults to `/bin/sh`) 50 | * env (defaults to `os.environ`) 51 | 52 | ### Configuration via environment variables 53 | 54 | Set the default shell via `SHELLFUNCS_DEFAULT_SHELL` environment variable: 55 | 56 | ```bash 57 | export SHELLFUNCS_DEFAULT_SHELL=/bin/bash 58 | ``` 59 | 60 | ### Configuration via context manager 61 | 62 | Set the configuration block-wise with a context manager: 63 | 64 | ```python 65 | import shellfuncs 66 | 67 | with shellfuncs.config(shell='/bin/bash'): 68 | from counters import count_python_imports 69 | 70 | count_python_imports() # the shell used will be /bin/bash 71 | ``` 72 | 73 | ### Configuration for specific function call 74 | 75 | Set the configuration when function is executed: 76 | 77 | ```python 78 | import shellfuncs 79 | 80 | from counters import count_python_imports 81 | 82 | count_python_imports(shell='/bin/bash') 83 | ``` 84 | 85 | *** 86 | 87 | *

This project is published under [MIT](LICENSE).
A [Timo Furrer](https://tuxtimo.me) project.
- :tada: -

* 88 | -------------------------------------------------------------------------------- /requirements-test.txt: -------------------------------------------------------------------------------- 1 | pytest 2 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | """ 2 | Python API for shell scripts 3 | """ 4 | 5 | import ast 6 | import os 7 | from setuptools import setup, find_packages 8 | 9 | 10 | PROJECT_ROOT = os.path.dirname(__file__) 11 | 12 | 13 | class VersionFinder(ast.NodeVisitor): 14 | 15 | def __init__(self): 16 | self.version = None 17 | 18 | def visit_Assign(self, node): 19 | try: 20 | if node.targets[0].id == '__version__': 21 | self.version = node.value.s 22 | except: 23 | pass 24 | 25 | 26 | def read_version(): 27 | """Read version from shellfuncs/__init__.py without loading any files""" 28 | finder = VersionFinder() 29 | path = os.path.join(PROJECT_ROOT, 'shellfuncs', '__init__.py') 30 | with open(path, 'r') as fp: 31 | file_data = fp.read().encode('utf-8') 32 | finder.visit(ast.parse(file_data)) 33 | 34 | return finder.version 35 | 36 | 37 | tests_require = ['pytest'] 38 | 39 | 40 | if __name__ == '__main__': 41 | setup(name='shellfuncs', 42 | version=read_version(), 43 | license='MIT', 44 | description='Python API for shell scripts', 45 | url='https://github.com/timofurrer/shellfuncs', 46 | author='Timo Furrer', 47 | author_email='tuxtimo@gmail.com', 48 | include_package_data=True, 49 | packages=find_packages(exclude=['*tests*']), 50 | install_requires=[], 51 | tests_require=tests_require, 52 | classifiers=[ 53 | 'Development Status :: 5 - Production/Stable', 54 | 'Environment :: Console', 55 | 'Operating System :: MacOS :: MacOS X', 56 | 'Operating System :: POSIX', 57 | 'Operating System :: POSIX :: Linux', 58 | 'Programming Language :: Python :: 3.4', 59 | 'Programming Language :: Python :: 3.5', 60 | 'Programming Language :: Python :: 3.6', 61 | 'Programming Language :: Python :: Implementation', 62 | 'Programming Language :: Python :: Implementation :: CPython', 63 | ] 64 | ) 65 | -------------------------------------------------------------------------------- /shellfuncs/__init__.py: -------------------------------------------------------------------------------- 1 | """ 2 | This module provides a Python API to shell functions coming 3 | from a sourced shell script. 4 | """ 5 | 6 | from .core import config 7 | 8 | __version__ = '0.2.1' 9 | 10 | # Expose only specific stuff 11 | __all__ = ['config'] 12 | -------------------------------------------------------------------------------- /shellfuncs/core.py: -------------------------------------------------------------------------------- 1 | """ 2 | This module provides a Python API to shell functions coming 3 | from a sourced shell script. 4 | 5 | Example: 6 | 7 | >>> import shellfuncs 8 | >>> from my_shell_script import my_shell_func 9 | >>> retcode, stdout, stderr = my_shell_func('Hello', 'World') 10 | >>> print('Got return code: {0}'.format(retcode)) 11 | Got return code: 0 12 | >>> print('Got stdout: "{0}"'.format(stdout)) 13 | Got stdout: "Hello" 14 | >>> print('Got stderr: "{0}"'.format(stderr)) 15 | Got stderr: "World" 16 | """ 17 | 18 | import os 19 | import sys 20 | import logging 21 | import types 22 | import importlib.util 23 | import subprocess 24 | import functools 25 | from pathlib import Path 26 | from collections import namedtuple 27 | from contextlib import contextmanager 28 | 29 | 30 | #: Holds the logger for shellfuncs. 31 | logger = logging.getLogger('shellfuncs') 32 | logger.setLevel(logging.DEBUG) 33 | 34 | #: Holds a type which is used as return value for executed shell functions. 35 | ShellFuncReturn = namedtuple('ShellFuncReturn', ['returncode', 'stdout', 'stderr']) 36 | 37 | #: Holds the configuration stack 38 | config_stack = [ 39 | { 40 | 'shell': os.environ.get('SHELLFUNCS_DEFAULT_SHELL', '/bin/sh'), 41 | 'env': os.environ 42 | } 43 | ] 44 | 45 | 46 | @contextmanager 47 | def config(shell=None, env=None): 48 | global config_stack 49 | 50 | config = config_stack[-1].copy() 51 | 52 | if shell is not None: 53 | config['shell'] = shell 54 | if env is not None: 55 | config['env'] = env 56 | 57 | config_stack.append(config) 58 | yield 59 | config_stack.pop() 60 | 61 | 62 | class ShellScriptFinder: 63 | """ 64 | Meta Path Finder to lookup 65 | shell scripts. 66 | """ 67 | @classmethod 68 | def find_spec(cls, name, path=None, target=None): 69 | script_path = Path('{0}.sh'.format(name.replace('.', '/'))) 70 | if not script_path.exists(): # no such script found. 71 | return None 72 | 73 | # TODO: check relative imports and directory walking 74 | loader = ShellScriptLoader() 75 | spec = importlib.util.spec_from_loader(name, loader) 76 | spec.script = script_path 77 | return spec 78 | 79 | 80 | class ShellScriptLoader: 81 | @classmethod 82 | def create_module(cls, spec): 83 | return ShellModule(spec.name, spec.script, config_stack[-1]) 84 | 85 | @classmethod 86 | def exec_module(cls, module): 87 | # TODO: Am I supposed to do something here? 88 | pass 89 | 90 | 91 | class ShellModule(types.ModuleType): 92 | def __init__(self, name, script, config): 93 | self.script = script 94 | self.config = config 95 | super().__init__(name) 96 | 97 | def __getattr__(self, name): 98 | if name == '__get__': # somehow invoked by functools.partialmethod. TODO: check waht's up?! 99 | return None 100 | 101 | func = functools.partial(self.execute_func, name) 102 | return func 103 | 104 | def execute_func(self, name, *args, shell=None, env=None, stdin=None, timeout=None): 105 | """ 106 | Execute the shell function with the given name. 107 | """ 108 | cmdline = '. ./{script} && {func} {args}'.format( 109 | script=str(self.script), 110 | func=name, args=' '.join("'{0}'".format(x) for x in args)) 111 | 112 | shell = shell if shell else self.config['shell'] 113 | env = env if env else self.config['env'] 114 | 115 | proc = subprocess.Popen(cmdline, shell=True, executable=shell, env=env, 116 | stdout=subprocess.PIPE, stderr=subprocess.PIPE, 117 | stdin=subprocess.PIPE if stdin else None) 118 | stdout, stderr = proc.communicate(input=stdin, timeout=timeout) 119 | return ShellFuncReturn(proc.returncode, stdout, stderr) 120 | 121 | 122 | # add ShellScriptFinder as meta path finder to sys 123 | sys.meta_path.append(ShellScriptFinder) 124 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/timofurrer/shellfuncs/522f62d5716e441f343b29cf7a04b70c6fc44f6e/tests/__init__.py -------------------------------------------------------------------------------- /tests/input_scripts/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/timofurrer/shellfuncs/522f62d5716e441f343b29cf7a04b70c6fc44f6e/tests/input_scripts/__init__.py -------------------------------------------------------------------------------- /tests/input_scripts/config.sh: -------------------------------------------------------------------------------- 1 | used_shell() { 2 | echo "${0}" 3 | } 4 | -------------------------------------------------------------------------------- /tests/input_scripts/foo.sh: -------------------------------------------------------------------------------- 1 | bar() { 2 | local STDOUT="$1" 3 | local STDERR="$2" 4 | 5 | echo "${STDOUT}" 6 | echo "${STDERR}" >&2 7 | 8 | return 0 9 | } 10 | 11 | get_shell() { 12 | echo "${SHELL}" 13 | } 14 | -------------------------------------------------------------------------------- /tests/test_core.py: -------------------------------------------------------------------------------- 1 | """ 2 | Test shellfuncs 3 | """ 4 | 5 | import shellfuncs 6 | 7 | def test_basic_import(): 8 | """ 9 | Test basic import functionality 10 | """ 11 | from .input_scripts.foo import bar 12 | 13 | returncode, stdout, stderr = bar('STDOUT', 'STDERR') 14 | 15 | assert returncode == 0 16 | assert stdout == b'STDOUT\n' 17 | assert stderr == b'STDERR\n' 18 | 19 | 20 | def test_shell_config(): 21 | """ 22 | Test configuring shell 23 | """ 24 | with shellfuncs.config(shell='/bin/bash'): 25 | from .input_scripts.config import used_shell 26 | 27 | _, stdout, _ = used_shell() 28 | assert stdout == b'/bin/bash\n' 29 | 30 | _, stdout, _ = used_shell(shell='/bin/sh') 31 | assert stdout == b'/bin/sh\n' 32 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | # Tox (http://tox.testrun.org/) is a tool for running tests 2 | # in multiple virtualenvs. This configuration file will run the 3 | # test suite on all supported python versions. To use it, "pip install tox" 4 | # and then run "tox" from this directory. 5 | 6 | [tox] 7 | envlist = py34, py35, py36 8 | 9 | [testenv] 10 | commands = pytest tests/ 11 | deps = 12 | -rrequirements-test.txt 13 | --------------------------------------------------------------------------------