├── .bumpversion.cfg ├── .gitignore ├── .travis.yml ├── LICENSE ├── MANIFEST.in ├── Makefile ├── README.rst ├── curlrc.py ├── requirements.txt ├── setup.cfg ├── setup.py ├── test_curlrc.py └── tox.ini /.bumpversion.cfg: -------------------------------------------------------------------------------- 1 | [bumpversion] 2 | current_version = 0.2.1 3 | commit = True 4 | tag = True 5 | 6 | [bumpversion:file:setup.py] 7 | 8 | [bumpversion:file:curlrc.py] 9 | 10 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.egg 2 | *.egg-info 3 | *.py[co] 4 | .cache/ 5 | .coverage 6 | .eggs/ 7 | .tox/ 8 | __pycache__ 9 | build/ 10 | dist/ 11 | htmlcov/ 12 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | --- 2 | language: python 3 | python: 4 | - "2.7" 5 | - "3.4" 6 | - "3.5" 7 | - "3.6-dev" 8 | install: pip install -r requirements.txt 9 | script: make test 10 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 Ben Webber 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include README.rst 2 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | .PHONY: all \ 2 | clean \ 3 | dist \ 4 | help \ 5 | install \ 6 | release \ 7 | sdist \ 8 | wheel 9 | 10 | define USAGE 11 | Targets: 12 | 13 | clean remove build artifacts 14 | dist build source distribution and wheel 15 | install install package to active Python site packages 16 | release build and upload package to PyPI 17 | sdist build source distribution 18 | test run tox tests 19 | wheel build wheel 20 | endef 21 | 22 | all: clean dist 23 | 24 | help: 25 | @echo $(info $(USAGE)) 26 | 27 | clean: 28 | $(RM) -r build dist 29 | find . -name '*.pyc' -delete 30 | find . -name '*.egg-info' -exec $(RM) -r {} + 31 | find . -name '*.egg' -delete 32 | 33 | dist: sdist wheel 34 | 35 | install: 36 | python setup.py install 37 | 38 | release: dist 39 | twine upload dist/*.whl dist/*.tar.gz 40 | 41 | sdist: clean 42 | python setup.py sdist 43 | 44 | test: 45 | tox 46 | 47 | wheel: clean 48 | python setup.py bdist_wheel 49 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | curlrc 2 | ====== 3 | 4 | .. image:: https://travis-ci.org/benwebber/curlrc.svg?branch=master 5 | :target: https://travis-ci.org/benwebber/curlrc 6 | 7 | Treat curl configuration files as ``curlrc`` subcommands. 8 | 9 | Usage 10 | ----- 11 | 12 | curl can read arguments from a `configuration file`_. You can use this mechanism to specify default arguments (as ``~/.curlrc``), or tell curl to read arguments from a specific file:: 13 | 14 | curl -K @/path/to/config.rc 15 | 16 | Create a few config files and drop them in ``~/.curl`` (or ``$CURL_HOME`` if set):: 17 | 18 | ~/.curl 19 | └── example.rc 20 | 21 | ``curlrc`` exposes configuration files as subcommands:: 22 | 23 | $ curlrc example 24 | 25 | If the configuration file includes an `output template`_, you can reformat the data 26 | as CSV, tab-separated columns, or JSON:: 27 | 28 | $ curlrc example -f csv https://example.org 29 | $ curlrc example -f table https://example.org 30 | $ curlrc example -f json https://example.org 31 | 32 | Any options you pass to ``curlrc`` after ``--`` will be passed to curl:: 33 | 34 | $ curlrc example -- -fsSL https://example.org 35 | 36 | Example 37 | ------- 38 | 39 | Consider the following configuration file:: 40 | 41 | # output timing data 42 | -s 43 | -S 44 | -o = /dev/null 45 | -w = "url_effective: %{url_effective}\ntime_namelookup: %{time_namelookup}\ntime_connect: %{time_connect}\ntime_appconnect: %{time_appconnect}\ntime_pretransfer: %{time_pretransfer}\ntime_redirect: %{time_redirect}\ntime_starttransfer: %{time_starttransfer}\ntime_total: %{time_total}\n" 46 | 47 | If you drop this in ``~/.curl/time.rc`` (or ``$CURL_HOME/time.rc``), you can use it by calling ``curlrc time``:: 48 | 49 | $ curlrc time https://example.org 50 | url_effective: https://example.org/ 51 | time_namelookup: 0.001 52 | time_connect: 0.026 53 | time_appconnect: 0.180 54 | time_pretransfer: 0.180 55 | time_redirect: 0.000 56 | time_starttransfer: 0.210 57 | time_total: 0.210 58 | 59 | Don't like the default format? Try CSV:: 60 | 61 | $ curlrc time -f csv https://example.org 62 | url_effective,time_namelookup,time_connect,time_appconnect,time_pretransfer,time_redirect,time_starttransfer,time_total 63 | https://example.org/,0.001,0.030,0.194,0.194,0.000,0.228,0.228 64 | 65 | or tab-separated columns:: 66 | 67 | $ curlrc time -f table https://example.org 68 | url_effective https://example.org/ 69 | time_namelookup 0.002 70 | time_connect 0.028 71 | time_appconnect 0.177 72 | time_pretransfer 0.177 73 | time_redirect 0.000 74 | time_starttransfer 0.205 75 | time_total 0.206 76 | 77 | or even JSON:: 78 | 79 | $ curlrc time -f json https://example.org 80 | { 81 | "url_effective": "https://example.org/", 82 | "time_namelookup": "0.001", 83 | "time_connect": "0.028", 84 | "time_appconnect": "0.182", 85 | "time_pretransfer": "0.182", 86 | "time_redirect": "0.000", 87 | "time_starttransfer": "0.213", 88 | "time_total": "0.213" 89 | } 90 | 91 | Installation 92 | ------------ 93 | 94 | ``curlrc`` requires Python 2.7 or later. It only depends on the standard library. 95 | 96 | Download the `latest release`_ or install with pip: 97 | 98 | .. code-block:: bash 99 | 100 | pip install curlrc 101 | 102 | Licence 103 | ------- 104 | 105 | MIT 106 | 107 | .. _configuration file: http://curl.haxx.se/docs/manpage.html#-K 108 | .. _output template: http://curl.haxx.se/docs/manpage.html#-w 109 | .. _latest release: https://github.com/benwebber/curlrc/releases/latest 110 | -------------------------------------------------------------------------------- /curlrc.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | """ 5 | Treat curl configuration files as commands. 6 | """ 7 | 8 | from __future__ import print_function 9 | 10 | import argparse 11 | import collections 12 | import glob 13 | import json 14 | import os 15 | import os.path 16 | import re 17 | import sys 18 | 19 | __version__ = '0.2.1' 20 | 21 | CURL_HOME = os.getenv('CURL_HOME', os.path.expanduser('~/.curl')) 22 | CURLRC_EXTENSION = '.rc' 23 | # Split option lines on this pattern. 24 | CURLRC_OPTION_EXPR = re.compile(r'\s?[\s=:]\s?') 25 | # Match %{variable} strings in format. 26 | CURLRC_FORMAT_STRING_EXPR = re.compile(r'%{[\w]+}') 27 | OUTPUT_FORMATS = ('csv', 'json', 'table') 28 | 29 | 30 | class CurlConfig(object): 31 | """ 32 | A curl configuration. 33 | """ 34 | def __init__(self, name, path=None, description=None, **options): 35 | self.name = name 36 | self.path = path 37 | self.description = description 38 | self.options = options 39 | 40 | @classmethod 41 | def from_file(cls, path): 42 | """ 43 | Load a curl configuration from a file. 44 | 45 | Args: 46 | path: path to curl configuration file 47 | 48 | Returns: CurlConfig 49 | """ 50 | name = os.path.basename(path).replace(CURLRC_EXTENSION, '') 51 | description = None 52 | options = {} 53 | with open(path) as config: 54 | # Extract description from first line if it is a comment. 55 | first_line = config.readline().strip() 56 | if first_line.startswith('#'): 57 | description = first_line.split('#')[1].strip() 58 | # Collect options as a dictionary. 59 | options = dict(cls.split_line(line.strip()) 60 | for line in config if not line.startswith('#')) 61 | return cls(name, path, description, **options) 62 | 63 | @property 64 | def template(self): 65 | """ 66 | Retrieve the format string specified in the configuration file. 67 | 68 | Returns: str or None 69 | """ 70 | return self.options.get('-w', self.options.get('--write-out')) 71 | 72 | @staticmethod 73 | def split_line(line): 74 | """ 75 | Split curl option lines into key-value pairs. 76 | 77 | Args: 78 | line: curl configuration option line 79 | 80 | Returns: list of [option, value] 81 | """ 82 | option = re.split(CURLRC_OPTION_EXPR, line, maxsplit=1) 83 | # Handle boolean flags (e.g., -s). 84 | if len(option) < 2: 85 | option.append(True) 86 | return option 87 | 88 | 89 | class CurlTemplate(object): 90 | """ 91 | A curl output template. 92 | """ 93 | def __init__(self, replacements): 94 | self._map = replacements 95 | 96 | @classmethod 97 | def from_str(cls, template): 98 | """ 99 | Load the template from a template string. 100 | """ 101 | chars_to_remove = '%{}' 102 | _map = collections.OrderedDict() 103 | for tmpl in re.findall(CURLRC_FORMAT_STRING_EXPR, template): 104 | _map[''.join(c for c in tmpl if c not in chars_to_remove)] = tmpl 105 | return cls(_map) 106 | 107 | def as_csv(self, pretty=True): 108 | """ 109 | Output the template as comma-separated values. 110 | """ 111 | output = '' 112 | if pretty: 113 | output = ','.join(self._map.keys()) 114 | output += '\n' 115 | output += ','.join(self._map.values()) 116 | return output + '\n' 117 | 118 | def as_json(self, pretty=True): 119 | """ 120 | Output the template as a JSON hash. 121 | """ 122 | indent = 2 if pretty else None 123 | return json.dumps(self._map, indent=indent) + '\n' 124 | 125 | def as_table(self, pretty=True): 126 | """ 127 | Output the template as a tab-separated table. 128 | """ 129 | lines = [] 130 | for field, value in self._map.items(): 131 | if pretty: 132 | lines.append('{}\t{}'.format(field, value)) 133 | else: 134 | lines.append(value) 135 | output = '\n'.join(lines) 136 | return output + '\n' 137 | 138 | 139 | def curl_configs(path=None, pattern=None): 140 | """ 141 | Locate curl configurations in a directory. 142 | 143 | Args: 144 | path: directory containing configuration files (default: CURL_HOME) 145 | pattern: glob pattern to match (default: *.rc) 146 | 147 | Returns: list of files 148 | """ 149 | path = path if path else CURL_HOME 150 | pattern = pattern if pattern else '*{}'.format(CURLRC_EXTENSION) 151 | return glob.glob(os.path.join(path, pattern)) 152 | 153 | 154 | def parse_args(argv): 155 | """ 156 | Parse command-line arguments. 157 | 158 | Returns: argparse.Namespace 159 | """ 160 | usage_template = '{} {} [OPTION...] -- [CURL ARGS...]'.format 161 | 162 | parser = argparse.ArgumentParser( 163 | usage=usage_template('%(prog)s', 'COMMAND'), 164 | version='%(prog)s {}'.format(__version__), 165 | ) 166 | 167 | # Define common arguments for subcommands. 168 | common = argparse.ArgumentParser(add_help=False) 169 | common.add_argument( 170 | '-f', '--format', 171 | choices=OUTPUT_FORMATS, 172 | help='output format', 173 | ) 174 | # Control pretty-printing. 175 | pretty = common.add_mutually_exclusive_group() 176 | pretty.add_argument( 177 | '--pretty', 178 | action='store_true', default=True, 179 | help='pretty-print output [%(default)s]', 180 | ) 181 | pretty.add_argument( 182 | '--no-pretty', 183 | action='store_false', default=False, dest='pretty', 184 | help='do not pretty-print output [%(default)s]', 185 | ) 186 | # Pass the rest of the arguments to curl. 187 | common.add_argument( 188 | 'curl_args', 189 | nargs='*', metavar='CURL ARGS', 190 | help='arguments passed to curl', 191 | ) 192 | 193 | # Extract command names from .curlrc files. 194 | subparsers = parser.add_subparsers( 195 | title='commands', dest='command', 196 | # hide {command1,command2} output 197 | metavar='', 198 | ) 199 | # Python 3.3 introduced a regression that makes subparsers optional: 200 | # 201 | # This is a no-op in Python 2. 202 | subparsers.required = True 203 | commands = [CurlConfig.from_file(c) for c in curl_configs()] 204 | for command in commands: 205 | subparsers.add_parser( 206 | command.name, 207 | help=command.description, parents=[common], 208 | # %(prog)s evaluates to the top-level parser's usage string. 209 | usage=usage_template(os.path.basename(sys.argv[0]), command.name) 210 | ) 211 | 212 | return parser.parse_args(argv) 213 | 214 | 215 | def main(argv=None): 216 | argv = argv if argv else sys.argv[1:] 217 | args = parse_args(argv) 218 | 219 | config = CurlConfig.from_file( 220 | os.path.join(CURL_HOME, '{}{}'.format(args.command, CURLRC_EXTENSION)) 221 | ) 222 | 223 | # If the user specified a format, override the format specified 224 | # in the configuration file. 225 | if args.format and config.template: 226 | tmpl = CurlTemplate.from_str(config.template) 227 | formats = { 228 | 'csv': tmpl.as_csv, 229 | 'json': tmpl.as_json, 230 | 'table': tmpl.as_table, 231 | } 232 | override_format = formats[args.format](args.pretty) 233 | curl_args = ['-w', override_format] 234 | curl_args.extend(args.curl_args) 235 | else: 236 | curl_args = args.curl_args 237 | 238 | os.execlp('curl', 'curl', '-K', config.path, *curl_args) 239 | 240 | 241 | if __name__ == '__main__': 242 | main() 243 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | pluggy==0.4.0 2 | py==1.4.32 3 | pytest==3.0.5 4 | tox==2.5.0 5 | tox-travis==0.7.2 6 | virtualenv==15.1.0 7 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [wheel] 2 | universal = 1 3 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | import setuptools 4 | 5 | setuptools.setup( 6 | name='curlrc', 7 | version='0.2.1', 8 | url='https://github.com/benwebber/curlrc/', 9 | 10 | description="Treat curl configuration files as curlrc subcommands.", 11 | long_description=open('README.rst').read(), 12 | 13 | author='Ben Webber', 14 | author_email='benjamin.webber@gmail.com', 15 | 16 | py_modules=['curlrc'], 17 | 18 | zip_safe=False, 19 | 20 | entry_points={ 21 | 'console_scripts': [ 22 | 'curlrc = curlrc:main', 23 | ], 24 | }, 25 | 26 | classifiers=[ 27 | 'Development Status :: 4 - Beta', 28 | "Programming Language :: Python :: 2", 29 | 'Programming Language :: Python :: 2.7', 30 | 'Programming Language :: Python :: 3', 31 | 'Programming Language :: Python :: 3.4', 32 | 'Programming Language :: Python :: 3.5', 33 | ], 34 | ) 35 | -------------------------------------------------------------------------------- /test_curlrc.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | from __future__ import print_function 4 | 5 | import collections 6 | import json 7 | import os 8 | 9 | import pytest 10 | 11 | import curlrc 12 | 13 | 14 | EXAMPLE_CONFIG = '''# output timing data 15 | -s 16 | -S 17 | -o = /dev/null 18 | -w = "url_effective: %{url_effective}\ntime_namelookup: %{time_namelookup}\ntime_connect: %{time_connect}\ntime_appconnect: %{time_appconnect}\ntime_pretransfer: %{time_pretransfer}\ntime_redirect: %{time_redirect}\ntime_starttransfer: %{time_starttransfer}\ntime_total: %{time_total}\n" 19 | ''' 20 | 21 | 22 | @pytest.fixture(params=[ 23 | ('url "curl.haxx.se"', ['url', '"curl.haxx.se"']), 24 | ('url = "curl.haxx.se"', ['url', '"curl.haxx.se"']), 25 | ('url="curl.haxx.se"', ['url', '"curl.haxx.se"']), 26 | ('url ="curl.haxx.se"', ['url', '"curl.haxx.se"']), 27 | ('url:"curl.haxx.se"', ['url', '"curl.haxx.se"']), 28 | ('url : "curl.haxx.se"', ['url', '"curl.haxx.se"']), 29 | ('url: "curl.haxx.se"', ['url', '"curl.haxx.se"']), 30 | ('url :"curl.haxx.se"', ['url', '"curl.haxx.se"']), 31 | ('-O', ['-O', True]), 32 | ]) 33 | def test_data(request): 34 | return request.param 35 | 36 | 37 | @pytest.fixture 38 | def test_config(tmpdir): 39 | p = tmpdir.join('time.rc') 40 | p.write(EXAMPLE_CONFIG) 41 | assert p.read() == EXAMPLE_CONFIG 42 | return p 43 | 44 | 45 | @pytest.fixture 46 | def test_template(): 47 | return 'url_effective: %{url_effective}\ntime_namelookup: %{time_namelookup}\ntime_connect: %{time_connect}' 48 | 49 | 50 | @pytest.fixture 51 | def test_template_map(): 52 | template_map = collections.OrderedDict() 53 | template_map['url_effective'] = '%{url_effective}' 54 | template_map['time_namelookup'] = '%{time_namelookup}' 55 | template_map['time_connect'] = '%{time_connect}' 56 | return template_map 57 | 58 | 59 | @pytest.fixture(params=[ 60 | (True, 'url_effective,time_namelookup,time_connect\n%{url_effective},%{time_namelookup},%{time_connect}\n'), 61 | (False, '%{url_effective},%{time_namelookup},%{time_connect}\n'), 62 | ]) 63 | def test_template_as_csv(request): 64 | return request.param 65 | 66 | 67 | @pytest.fixture(params=[ 68 | (True, '''url_effective %{url_effective} 69 | time_namelookup %{time_namelookup} 70 | time_connect %{time_connect} 71 | '''), 72 | (False, '''%{url_effective} 73 | %{time_namelookup} 74 | %{time_connect} 75 | '''), 76 | ]) 77 | def test_template_as_table(request): 78 | return request.param 79 | 80 | 81 | @pytest.fixture(params=[ 82 | (True, json.dumps(test_template_map(), indent=2) + '\n'), 83 | (False, json.dumps(test_template_map()) + '\n'), 84 | ]) 85 | def test_template_as_json(request): 86 | return request.param 87 | 88 | 89 | class TestCurlConfig: 90 | def test_split_lines(self, test_data): 91 | (input, expected) = test_data 92 | assert curlrc.CurlConfig.split_line(input) == expected 93 | 94 | def test_from_file(self, test_config): 95 | config = curlrc.CurlConfig.from_file(str(test_config)) 96 | assert config.name == 'time' 97 | assert config.description == 'output timing data' 98 | assert config.path == str(test_config) 99 | assert config.template 100 | 101 | 102 | class TestCurlTemplate: 103 | def test_from_str(self, test_template, test_template_map): 104 | tmpl = curlrc.CurlTemplate.from_str(test_template) 105 | assert tmpl._map == test_template_map 106 | 107 | def test_as_csv(self, test_template, test_template_as_csv): 108 | tmpl = curlrc.CurlTemplate.from_str(test_template) 109 | (input, expected) = test_template_as_csv 110 | assert tmpl.as_csv(input) == expected 111 | 112 | def test_as_table(self, test_template, test_template_as_table): 113 | tmpl = curlrc.CurlTemplate.from_str(test_template) 114 | (input, expected) = test_template_as_table 115 | assert tmpl.as_table(input) == expected 116 | 117 | def test_as_json(self, test_template, test_template_as_json): 118 | tmpl = curlrc.CurlTemplate.from_str(test_template) 119 | (input, expected) = test_template_as_json 120 | assert tmpl.as_json(input) == expected 121 | 122 | 123 | def test_curl_configs(tmpdir): 124 | files = [ 125 | 'example1.rc', 126 | 'example2.rc', 127 | 'example3' 128 | ] 129 | for f in files: 130 | p = tmpdir.join(f) 131 | p.write('') 132 | glob = [ 133 | str(tmpdir.join('example1.rc')), 134 | str(tmpdir.join('example2.rc')), 135 | ] 136 | assert curlrc.curl_configs(str(tmpdir)) == glob 137 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | # Tox (http://tox.testrun.org/) is a tool for running tests 2 | # in multiple virtualenvs. This configuration file will run the 3 | # test suite on all supported python versions. To use it, "pip install tox" 4 | # and then run "tox" from this directory. 5 | 6 | [tox] 7 | envlist = py27, py34, py35, py36 8 | 9 | [testenv] 10 | commands = 11 | py.test --cov-report=html --cov-report=term --cov=curlrc 12 | deps = 13 | pytest 14 | pytest-cov 15 | --------------------------------------------------------------------------------