├── docs ├── _static │ ├── .keep │ └── custom.css ├── requirements.txt ├── changelog.rst ├── contributing.rst ├── .readthedocs.yaml ├── index.rst ├── packaging.rst ├── quickstart.rst └── troubleshooting.rst ├── scripts ├── adig ├── ahttp ├── antp ├── aping ├── asslcert ├── atraceroute └── ripe-atlas ├── ripe ├── atlas │ ├── tools │ │ ├── __init__.py │ │ ├── commands │ │ │ ├── __init__.py │ │ │ ├── go.py │ │ │ ├── measure │ │ │ │ ├── ntp.py │ │ │ │ ├── sslcert.py │ │ │ │ ├── ping.py │ │ │ │ ├── __init__.py │ │ │ │ ├── spec.py │ │ │ │ ├── http.py │ │ │ │ ├── traceroute.py │ │ │ │ └── dns.py │ │ │ ├── probe_info.py │ │ │ ├── stream.py │ │ │ ├── shibboleet.py │ │ │ ├── alias.py │ │ │ └── configure.py │ │ ├── helpers │ │ │ ├── __init__.py │ │ │ ├── actions.py │ │ │ ├── sanitisers.py │ │ │ ├── colours.py │ │ │ └── xdg.py │ │ ├── renderers │ │ │ ├── templates │ │ │ │ └── reports │ │ │ │ │ ├── aggregate_ping.txt │ │ │ │ │ ├── ssl_consistency.txt │ │ │ │ │ ├── sslcert.txt │ │ │ │ │ └── dns.txt │ │ │ ├── __init__.py │ │ │ ├── raw.py │ │ │ ├── dst_asn.py │ │ │ ├── http.py │ │ │ ├── sslcert.py │ │ │ ├── traceroute.py │ │ │ ├── ntp.py │ │ │ ├── dns_compact.py │ │ │ ├── ssl_consistency.py │ │ │ ├── traceroute_aspath.py │ │ │ ├── dns.py │ │ │ └── ping.py │ │ ├── version.py │ │ ├── aggregators │ │ │ ├── __init__.py │ │ │ └── base.py │ │ ├── exceptions.py │ │ ├── settings │ │ │ └── templates │ │ │ │ └── base.yaml │ │ ├── streaming.py │ │ ├── ipdetails.py │ │ ├── cache.py │ │ └── filters.py │ └── __init__.py └── __init__.py ├── tests ├── README ├── __init__.py ├── commands │ ├── __init__.py │ ├── test_base.py │ ├── test_loading.py │ └── test_alias.py ├── helpers │ ├── __init__.py │ ├── test_sanitisers.py │ └── test_validators.py ├── aggregators │ └── __init__.py ├── renderers │ ├── __init__.py │ ├── test_http.py │ └── test_dns_compact.py ├── test_docs.py ├── base.py └── test_bash_completion.py ├── screenshots ├── ripe-atlas-measure-ping.png ├── ripe-atlas-probe-search.png └── ripe-atlas-measurement-search.png ├── tox.ini ├── MANIFEST.in ├── ripe-atlas-bash-completion.sh ├── .github └── workflows │ ├── test.yaml │ └── python-package.yml ├── .gitignore ├── README.rst ├── setup.py ├── CONTRIBUTING.rst ├── PACKAGING.md ├── dev-scripts └── compare-openapi-spec.py └── CHANGES.rst /docs/_static/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /scripts/adig: -------------------------------------------------------------------------------- 1 | ripe-atlas -------------------------------------------------------------------------------- /scripts/ahttp: -------------------------------------------------------------------------------- 1 | ripe-atlas -------------------------------------------------------------------------------- /scripts/antp: -------------------------------------------------------------------------------- 1 | ripe-atlas -------------------------------------------------------------------------------- /scripts/aping: -------------------------------------------------------------------------------- 1 | ripe-atlas -------------------------------------------------------------------------------- /ripe/atlas/tools/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /scripts/asslcert: -------------------------------------------------------------------------------- 1 | ripe-atlas -------------------------------------------------------------------------------- /scripts/atraceroute: -------------------------------------------------------------------------------- 1 | ripe-atlas -------------------------------------------------------------------------------- /docs/requirements.txt: -------------------------------------------------------------------------------- 1 | sphinx-rtd-theme -------------------------------------------------------------------------------- /ripe/atlas/tools/commands/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /ripe/atlas/tools/helpers/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /docs/changelog.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../CHANGES.rst 2 | -------------------------------------------------------------------------------- /docs/contributing.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../CONTRIBUTING.rst 2 | -------------------------------------------------------------------------------- /tests/README: -------------------------------------------------------------------------------- 1 | How to run tests 2 | Make sure you have nose installed and run from the main directory: 3 | nosetests tests 4 | -------------------------------------------------------------------------------- /screenshots/ripe-atlas-measure-ping.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RIPE-NCC/ripe-atlas-tools/HEAD/screenshots/ripe-atlas-measure-ping.png -------------------------------------------------------------------------------- /screenshots/ripe-atlas-probe-search.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RIPE-NCC/ripe-atlas-tools/HEAD/screenshots/ripe-atlas-probe-search.png -------------------------------------------------------------------------------- /screenshots/ripe-atlas-measurement-search.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RIPE-NCC/ripe-atlas-tools/HEAD/screenshots/ripe-atlas-measurement-search.png -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [testenv] 2 | deps = 3 | flake8 4 | pytest 5 | sphinx 6 | sphinx-rtd-theme 7 | commands = 8 | flake8 --max-line-length=88 setup.py ripe/atlas/tools/ scripts/ tests/ 9 | pytest -r a {posargs} 10 | -------------------------------------------------------------------------------- /docs/.readthedocs.yaml: -------------------------------------------------------------------------------- 1 | version: 2 2 | 3 | build: 4 | os: ubuntu-24.04 5 | tools: 6 | python: "3.12" 7 | 8 | sphinx: 9 | configuration: docs/conf.py 10 | 11 | python: 12 | install: 13 | - requirements: docs/requirements.txt 14 | -------------------------------------------------------------------------------- /ripe/atlas/tools/renderers/templates/reports/aggregate_ping.txt: -------------------------------------------------------------------------------- 1 | 2 | --- {target} ping statistics --- 3 | {sent} packets transmitted, {received} received, {packet_loss:.3}% loss 4 | rtt min/med/avg/max = {min:.3f}/{median:.3f}/{mean:.3f}/{max:.3f} ms 5 | -------------------------------------------------------------------------------- /ripe/atlas/tools/renderers/templates/reports/ssl_consistency.txt: -------------------------------------------------------------------------------- 1 | Certificate: 2 | Issuer: C={issuer_c}, O={issuer_o}, CN={issuer_cn} 3 | Subject: C={subject_c}, O={subject_o}, CN={subject_cn} 4 | SHA256 Fingerprint={sha256fp} 5 | 6 | Seen by {seenby} probe{s} 7 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include LICENSE 2 | include README.rst 3 | include CHANGES.rst 4 | include MANIFEST.in 5 | include ripe/atlas/tools/user-agent 6 | recursive-include ripe *.py 7 | recursive-include ripe *.yaml 8 | recursive-include ripe *.txt 9 | recursive-include tests *.py 10 | -------------------------------------------------------------------------------- /ripe/atlas/tools/helpers/actions.py: -------------------------------------------------------------------------------- 1 | from argparse import Action 2 | 3 | 4 | class StoreIfNotEmpty(Action): 5 | """ 6 | Like 'store' but don't overwrite an existing/conflicting option if the given 7 | value is empty. 8 | """ 9 | def __call__(self, parser, namespace, values, option_string=None): 10 | if values: 11 | setattr(namespace, self.dest, values) 12 | -------------------------------------------------------------------------------- /docs/_static/custom.css: -------------------------------------------------------------------------------- 1 | /* override table width restrictions */ 2 | @media screen and (min-width: 767px) { 3 | 4 | .wy-table-responsive table td { 5 | /* !important prevents the common CSS stylesheets from 6 | overriding this as on RTD they are loaded after this stylesheet */ 7 | white-space: normal !important; 8 | } 9 | 10 | .wy-table-responsive { 11 | overflow: visible !important; 12 | } 13 | 14 | } 15 | -------------------------------------------------------------------------------- /ripe-atlas-bash-completion.sh: -------------------------------------------------------------------------------- 1 | ## This is highly inspired from Django's manage.py autocompletion 2 | ## https://github.com/django/django/blob/1.9.4/extras/django_bash_completion#L42-L57 3 | ## To install this, add the following line to your .bash_profile: 4 | ## 5 | ## . ~/path/to/django_bash_completion 6 | 7 | _ripe_atlas_bash_completion() 8 | { 9 | COMPREPLY=( $( COMP_WORDS="${COMP_WORDS[*]}" \ 10 | COMP_CWORD=$COMP_CWORD \ 11 | RIPE_ATLAS_AUTO_COMPLETE=1 $1 ) ) 12 | } 13 | complete -F _ripe_atlas_bash_completion -o default ripe-atlas 14 | -------------------------------------------------------------------------------- /ripe/atlas/tools/renderers/templates/reports/sslcert.txt: -------------------------------------------------------------------------------- 1 | Certificate: 2 | Data: 3 | Version: {version} 4 | Serial Number: {serial_number} 5 | Signature Algorithm: {signature_algorithm} 6 | Issuer: C={issuer_c}, O={issuer_o}, CN={issuer_cn} 7 | Validity 8 | Not Before: {not_before} 9 | Not After : {not_after} 10 | Subject: C={subject_c}, O={subject_o}, CN={subject_cn} 11 | Subject Public Key Info: 12 | Public Key Algorithm: {pkey_type} 13 | Public-Key: ({pkey_bits} bit) 14 | SHA1 Fingerprint={sha1fp} 15 | SHA256 Fingerprint={sha256fp} 16 | -------------------------------------------------------------------------------- /ripe/atlas/tools/renderers/templates/reports/dns.txt: -------------------------------------------------------------------------------- 1 | 2 | - {response_id} - 3 | 4 | ; <<>> RIPE Atlas Tools <<>> {question_name} 5 | ;; global options: +cmd 6 | ;; Got answer: 7 | ;; ->>HEADER<<- opcode: {header_opcode}, status: {header_return_code}, id: {header_id} 8 | ;; flags: {header_flags}; QUERY: 1, ANSWER: {answer_count}, AUTHORITY: {authority_count}, ADDITIONAL: {additional_count} 9 | {edns}{question}{answers}{authorities}{additionals} 10 | ;; Query time: {response_time} msec 11 | ;; SERVER: {destination_address}#53({destination_address}) 12 | ;; WHEN: {created} 13 | ;; MSG SIZE rcvd: {response_size} 14 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2015 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | -------------------------------------------------------------------------------- /tests/commands/__init__.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2015 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | -------------------------------------------------------------------------------- /tests/helpers/__init__.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2015 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | -------------------------------------------------------------------------------- /tests/aggregators/__init__.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2015 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | -------------------------------------------------------------------------------- /tests/renderers/__init__.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2015 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | -------------------------------------------------------------------------------- /ripe/atlas/tools/version.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2023 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | __version__ = "3.3.1" 17 | -------------------------------------------------------------------------------- /docs/index.rst: -------------------------------------------------------------------------------- 1 | RIPE Atlas Tools (Magellan) 2 | =========================== 3 | 4 | The official command-line client for RIPE Atlas. 5 | 6 | .. _index-why-this-exists: 7 | 8 | 9 | Why This Exists 10 | =============== 11 | 12 | `RIPE Atlas`_ is a powerful Internet measurements platform that until recently 13 | was only accessible via the website and the RESTful API. The reality however is 14 | that a great many people using RIPE Atlas are most comfortable on the 15 | command-line, so this project is an attempt to fill that gap. 16 | 17 | .. _RIPE Atlas: https://atlas.ripe.net 18 | 19 | 20 | Contents 21 | ======== 22 | 23 | .. toctree:: 24 | :maxdepth: 2 25 | 26 | quickstart 27 | installation 28 | use 29 | plugins 30 | contributing 31 | packaging 32 | troubleshooting 33 | changelog 34 | -------------------------------------------------------------------------------- /ripe/atlas/tools/renderers/__init__.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from .base import Renderer 17 | 18 | __all__ = ["Renderer"] 19 | -------------------------------------------------------------------------------- /ripe/__init__.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | try: 17 | __import__('pkg_resources').declare_namespace(__name__) 18 | except ImportError: 19 | from pkgutil import extend_path 20 | __path__ = extend_path(__path__, __name__) 21 | -------------------------------------------------------------------------------- /ripe/atlas/__init__.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | try: 17 | __import__('pkg_resources').declare_namespace(__name__) 18 | except ImportError: 19 | from pkgutil import extend_path 20 | __path__ = extend_path(__path__, __name__) 21 | -------------------------------------------------------------------------------- /ripe/atlas/tools/aggregators/__init__.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from .base import RangeKeyAggregator, ValueKeyAggregator, aggregate 17 | 18 | __all__ = [ 19 | "aggregate", 20 | "RangeKeyAggregator", 21 | "ValueKeyAggregator", 22 | ] 23 | -------------------------------------------------------------------------------- /.github/workflows/test.yaml: -------------------------------------------------------------------------------- 1 | # This workflow will install Python dependencies, run tests and lint with a variety of Python versions 2 | # For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions 3 | 4 | name: Python package 5 | 6 | on: 7 | push: 8 | branches: [ master, 'release-*' ] 9 | pull_request: 10 | branches: [ $default-branch ] 11 | 12 | jobs: 13 | test: 14 | 15 | runs-on: ubuntu-latest 16 | strategy: 17 | fail-fast: false 18 | matrix: 19 | python-version: ["3.10", "3.11", "3.12", "3.13"] 20 | 21 | steps: 22 | - uses: actions/checkout@v2 23 | - name: Set up Python ${{ matrix.python-version }} 24 | uses: actions/setup-python@v2 25 | with: 26 | python-version: ${{ matrix.python-version }} 27 | - name: Install dependencies 28 | run: | 29 | python -m pip install --upgrade pip 30 | python -m pip install tox 31 | - name: tox 32 | run: tox 33 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *.swp 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | env/ 12 | build/ 13 | develop-eggs/ 14 | dist/ 15 | downloads/ 16 | eggs/ 17 | .eggs/ 18 | lib/ 19 | lib64/ 20 | parts/ 21 | sdist/ 22 | var/ 23 | *.egg-info/ 24 | .installed.cfg 25 | *.egg 26 | 27 | # PyInstaller 28 | # Usually these files are written by a python script from a template 29 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 30 | *.manifest 31 | *.spec 32 | 33 | # Installer logs 34 | pip-log.txt 35 | pip-delete-this-directory.txt 36 | 37 | # Unit test / coverage reports 38 | htmlcov/ 39 | .tox/ 40 | .coverage 41 | .coverage.* 42 | .cache 43 | nosetests.xml 44 | coverage.xml 45 | *,cover 46 | 47 | # Translations 48 | *.mo 49 | *.pot 50 | 51 | # Django stuff: 52 | *.log 53 | 54 | # Sphinx documentation 55 | docs/_build/ 56 | 57 | # PyBuilder 58 | target/ 59 | 60 | # PyCharm 61 | .idea/ 62 | 63 | -------------------------------------------------------------------------------- /ripe/atlas/tools/exceptions.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from ripe.atlas.tools.helpers.colours import colourise 17 | 18 | import sys 19 | 20 | 21 | class RipeAtlasToolsException(Exception): 22 | def write(self): 23 | r = str(self) 24 | sys.stderr.write("\n{0}\n\n".format(colourise(r, "red", fileobj=sys.stderr))) 25 | -------------------------------------------------------------------------------- /ripe/atlas/tools/helpers/sanitisers.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | FORBIDDEN = dict((i, None) for i in list(range(0, 32)) + [127]) 17 | 18 | 19 | def sanitise(s, strip_newlines=True): 20 | """ 21 | Strip out control characters to prevent people from screwing with the output 22 | """ 23 | if isinstance(s, bytes): 24 | s = s.decode("utf-8") 25 | if not isinstance(s, str): 26 | return s 27 | 28 | if not strip_newlines: 29 | return s.translate(dict((k, v) for k, v in FORBIDDEN.items() if not k == 10)) 30 | 31 | return s.translate(FORBIDDEN) 32 | -------------------------------------------------------------------------------- /ripe/atlas/tools/renderers/raw.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | import json 17 | 18 | from .base import Renderer as BaseRenderer 19 | from ..helpers.sanitisers import sanitise 20 | 21 | 22 | class Renderer(BaseRenderer): 23 | 24 | RENDERS = [ 25 | BaseRenderer.TYPE_PING, 26 | BaseRenderer.TYPE_TRACEROUTE, 27 | BaseRenderer.TYPE_DNS, 28 | BaseRenderer.TYPE_SSLCERT, 29 | BaseRenderer.TYPE_HTTP, 30 | BaseRenderer.TYPE_NTP, 31 | ] 32 | 33 | SHOW_DEFAULT_HEADER = False 34 | SHOW_DEFAULT_FOOTER = False 35 | 36 | def on_result(self, result, probes=None): 37 | return sanitise(json.dumps(result.raw_data, separators=(",", ":"))) + "\n" 38 | -------------------------------------------------------------------------------- /tests/helpers/test_sanitisers.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2015 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | import unittest 17 | 18 | from ripe.atlas.tools.helpers.sanitisers import sanitise 19 | 20 | 21 | class TestSanitisersHelper(unittest.TestCase): 22 | def test_sanitise(self): 23 | 24 | self.assertEqual("clean", sanitise("clean")) 25 | for i in list(range(0, 32)) + [127]: 26 | self.assertEqual("unclean", sanitise("unclean" + chr(i))) 27 | 28 | self.assertEqual(None, sanitise(None)) 29 | self.assertEqual(7, sanitise(7)) 30 | 31 | def test_sanitise_with_newline_exception(self): 32 | self.assertEqual("unc\nlean", sanitise("unc\nlean", strip_newlines=False)) 33 | for i in set(list(range(0, 32)) + [127]).difference({10}): 34 | self.assertEqual( 35 | "unc\nlean", sanitise("unc\nlean" + chr(i), strip_newlines=False) 36 | ) 37 | -------------------------------------------------------------------------------- /ripe/atlas/tools/settings/templates/base.yaml: -------------------------------------------------------------------------------- 1 | # 2 | # This is the config file for the ripe-atlas command. It follows the YAML 3 | # formatting standard found here: http://yaml.org/ -- but the layout should be 4 | # rather intuitive. 5 | # 6 | # Lines that start with "#" (like this one) are considered comments and 7 | # are ignored by the parser. Beyond that, the configuration data is laid out 8 | # in a hierarchical fashion 9 | # 10 | # section_name: 11 | # variable_name: "some-value" 12 | # some_other_variable_name: 123 13 | # a_subsection: 14 | # something: "something else" 15 | # 16 | # Where possible, we've commented this file to help you out, but you can also 17 | # use the ripe-atlas script to modify it for you. Simply type: 18 | # 19 | # $ ripe-atlas configure --help 20 | # 21 | # And the help text that appears will walk you through it. 22 | # 23 | # One good thing to put in here is your API key. This makes it easy to 24 | # generate measurements quickly. Defining your API key looks like this: 25 | # 26 | # authorisation: 27 | # create: "YOUR-API-KEY" 28 | # 29 | # Leaving this file empty won't hurt, but it will mean that you'll have to 30 | # use the --auth=YOUR-API-KEY flag every time you try to create a 31 | # measurement. 32 | # 33 | # What follows is a complete break down of all possible options, set initially 34 | # to their defaults. If you wish to change a default, simply change the value 35 | # to whatever you like and save this file. 36 | # 37 | 38 | ############################################################################ 39 | 40 | {payload} 41 | 42 | -------------------------------------------------------------------------------- /tests/test_docs.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2015 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | import unittest 17 | 18 | from sphinx.application import Sphinx 19 | 20 | 21 | class DocTest(unittest.TestCase): 22 | 23 | SOURCE_DIR = "docs" 24 | CONFIG_DIR = "docs" 25 | OUTPUT_DIR = "docs/build" 26 | DOCTREE_DIR = "docs/build/doctrees" 27 | 28 | def test_html_documentation(self): 29 | Sphinx( 30 | self.SOURCE_DIR, 31 | self.CONFIG_DIR, 32 | self.OUTPUT_DIR, 33 | self.DOCTREE_DIR, 34 | buildername="html", 35 | warningiserror=True, 36 | ).build(force_all=True) 37 | 38 | def test_text_documentation(self): 39 | Sphinx( 40 | self.SOURCE_DIR, 41 | self.CONFIG_DIR, 42 | self.OUTPUT_DIR, 43 | self.DOCTREE_DIR, 44 | buildername="text", 45 | warningiserror=False, 46 | ).build(force_all=True) 47 | -------------------------------------------------------------------------------- /docs/packaging.rst: -------------------------------------------------------------------------------- 1 | Packaging 2 | ========= 3 | 4 | For those interested in packaging RIPE Atlas Tools for their favourite distro, 5 | this section is for you. 6 | 7 | Currently Supported 8 | ------------------- 9 | 10 | * OpenBSD 11 | * FreeBSD 12 | * Gentoo 13 | * Debian 14 | * Ubuntu 15 | 16 | In Progress 17 | ----------- 18 | 19 | * Fedora: Jan Včelák is currently building the binary packages in `COPR`_ (which will take some time as there is a lot of other packages in the queue) 20 | 21 | .. _`COPR`: https://copr.fedoraproject.org/coprs/jvcelak/ripe-atlas-tools/ 22 | 23 | Additional Distributions 24 | ------------------------ 25 | 26 | Is your distribution not listed? If you'd like to build a package for another 27 | distro or even if you're just someone who knows someone who can help us package 28 | and distribute this, please get in touch. 29 | 30 | Further Information 31 | ------------------- 32 | 33 | User Agent 34 | ~~~~~~~~~~ 35 | 36 | When packaging, it's good practise to manually set the user agent used within 37 | the toolkit so that we can get a rough idea of which distros are using this 38 | software. This is easily done by writing an arbitrary string to 39 | ``/ripe/atlas/tools/user-agent``. Something like this is recommended::: 40 | 41 | RIPE Atlas Tools [FreeBSD 10.2] 1.2 42 | 43 | The only limitations to this file are that it should: 44 | 45 | * Only have one line in it (all other will be ignored) 46 | * That line should have a maximum of 128 characters in it 47 | 48 | If no ``user-agent`` file is included then a platform-specific string will be 49 | automatically generated based on Python's ``platform`` module. 50 | -------------------------------------------------------------------------------- /ripe/atlas/tools/commands/go.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | import webbrowser 17 | from .base import Command as BaseCommand 18 | from ..helpers.validators import ArgumentType 19 | 20 | 21 | class Command(BaseCommand): 22 | 23 | NAME = "go" 24 | 25 | DESCRIPTION = "Visit the web page for a specific measurement" 26 | URL = "https://atlas.ripe.net/measurements/{0}/" 27 | 28 | def add_arguments(self): 29 | self.parser.add_argument( 30 | "measurement_id", 31 | type=ArgumentType.msm_id_or_name(), 32 | help="The measurement id or alias you want reported", 33 | ) 34 | 35 | def run(self): 36 | url = self.URL.format(self.arguments.measurement_id) 37 | if not webbrowser.open(url): 38 | self.ok( 39 | "It looks like your system doesn't have a web browser " 40 | "available. You'll have to go there manually: {0}".format(url) 41 | ) 42 | -------------------------------------------------------------------------------- /docs/quickstart.rst: -------------------------------------------------------------------------------- 1 | Quickstart 2 | ========== 3 | 4 | This is a very fast break down of everything you need to start using Ripe Atlas 5 | on the command line. Viewing public data is quick & easy, while creation is a 6 | little more complicated, since you need to setup your authorisation key. 7 | 8 | Viewing Public Data 9 | ------------------- 10 | 11 | 1. :ref:`Install ` the toolkit. 12 | 2. View help with: ``ripe-atlas --help`` 13 | 3. View a basic report for a public measurement: ``ripe-atlas report `` 14 | 4. View the live stream for a measurement: ``ripe-atlas stream `` 15 | 5. Get a list of probes in ASN 3333: ``ripe-atlas probe-search --asn 3333`` 16 | 6. Get a list of measurements with the word "wikipedia" in them: ``ripe-atlas measurement-search --search wikipedia`` 17 | 18 | Creating a Measurement 19 | ---------------------- 20 | 21 | 1. Log into `RIPE Atlas`_. If you don't have an 22 | account, you can create one there for free. 23 | 2. Visit the `API Keys`_ page and create a new key 24 | with the permission ``Create a new user defined measurement`` 25 | 3. Install the toolkit as below. 26 | 4. Configure the toolkit to use your key with ``ripe-atlas configure --set authorisation.create=MY_API_KEY`` 27 | 5. View the help for measurement creation with ``ripe-atlas measure --help`` 28 | 6. Create a measurement with ``ripe-atlas measure ping --target example.com`` 29 | 30 | .. _`RIPE Atlas`: https://atlas.ripe.net/ 31 | .. _`API Keys`: https://atlas.ripe.net/keys/ 32 | 33 | Advanced Use 34 | ------------ 35 | 36 | Refer to the :ref:`complete usage documentation ` for more advanced 37 | options. 38 | -------------------------------------------------------------------------------- /ripe/atlas/tools/streaming.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2023 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from typing import Iterator, Optional 17 | 18 | from ripe.atlas.cousteau import AtlasStream 19 | from ripe.atlas.sagan import Result 20 | 21 | 22 | class StreamWrapper: 23 | """ 24 | Iterable wrapper for AtlasStream that yields sagan Results up to a 25 | specified capture limit and/or timeout 26 | """ 27 | 28 | def __init__( 29 | self, 30 | stream: AtlasStream, 31 | capture_limit: Optional[int] = None, 32 | timeout: Optional[float] = None, 33 | ) -> None: 34 | self.stream = stream 35 | self.capture_limit = capture_limit 36 | self.timeout = timeout 37 | self.num_received = 0 38 | 39 | def __iter__(self) -> Iterator[Result]: 40 | for event_name, payload in self.stream.iter(seconds=self.timeout): 41 | if event_name == "atlas_result": 42 | parsed = Result.get( 43 | payload, 44 | on_error=Result.ACTION_IGNORE, 45 | on_malformation=Result.ACTION_IGNORE, 46 | ) 47 | yield parsed 48 | self.num_received += 1 49 | if self.num_received == self.capture_limit: 50 | break 51 | -------------------------------------------------------------------------------- /tests/base.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2015 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | import sys 17 | 18 | from contextlib import contextmanager 19 | 20 | from io import StringIO 21 | 22 | 23 | class FakeTTY(object): 24 | """ 25 | Basic simulation of a user terminal. 26 | """ 27 | 28 | def __init__(self, file_obj): 29 | self.file_obj = file_obj 30 | 31 | def __getattr__(self, name): 32 | return getattr(self.file_obj, name) 33 | 34 | def isatty(self): 35 | return True 36 | 37 | 38 | @contextmanager 39 | def capture_sys_output(use_fake_tty=False): 40 | """ 41 | Wrap a block with this, and it'll capture standard out and standard error 42 | into handy variables: 43 | 44 | with capture_sys_output() as (stdout, stderr): 45 | self.cmd.run() 46 | 47 | More info: https://stackoverflow.com/questions/18651705/ 48 | """ 49 | 50 | capture_out, capture_err = StringIO(), StringIO() 51 | current_out, current_err = sys.stdout, sys.stderr 52 | current_in = sys.stdin 53 | try: 54 | if use_fake_tty: 55 | sys.stdin = FakeTTY(current_in) 56 | sys.stdout, sys.stderr = capture_out, capture_err 57 | yield capture_out, capture_err 58 | finally: 59 | sys.stdout, sys.stderr = current_out, current_err 60 | sys.stdin = current_in 61 | -------------------------------------------------------------------------------- /ripe/atlas/tools/commands/measure/ntp.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from ...helpers.validators import ArgumentType 17 | from ...settings import conf 18 | 19 | from .base import Command 20 | 21 | 22 | class NtpMeasureCommand(Command): 23 | DESCRIPTION = "Create an NTP measurement and wait for the results" 24 | 25 | def add_arguments(self): 26 | 27 | Command.add_arguments(self) 28 | 29 | self.add_primary_argument(name="target", parser=self.parser) 30 | 31 | spec = conf["specification"]["types"]["ntp"] 32 | 33 | specific = self.parser.add_argument_group("NTP-specific Options") 34 | specific.add_argument( 35 | "--packets", 36 | type=ArgumentType.integer_range(minimum=1, maximum=16), 37 | default=spec["packets"], 38 | help="The number of packets sent", 39 | ) 40 | specific.add_argument( 41 | "--timeout", 42 | type=ArgumentType.integer_range(minimum=1, maximum=60000), 43 | default=spec["timeout"], 44 | help="The timeout per-packet", 45 | ) 46 | 47 | def _get_measurement_kwargs(self): 48 | 49 | r = Command._get_measurement_kwargs(self) 50 | 51 | r["packets"] = self.arguments.packets 52 | r["timeout"] = self.arguments.timeout 53 | 54 | return r 55 | -------------------------------------------------------------------------------- /ripe/atlas/tools/commands/measure/sslcert.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from ...helpers.validators import ArgumentType 17 | from ...settings import conf 18 | 19 | from .base import Command 20 | 21 | 22 | class SslcertMeasureCommand(Command): 23 | DESCRIPTION = "Create a TLS (SSL) cert measurement and wait for the results" 24 | 25 | def add_arguments(self): 26 | 27 | Command.add_arguments(self) 28 | 29 | self.add_primary_argument(name="target", parser=self.parser) 30 | 31 | spec = conf["specification"]["types"]["sslcert"] 32 | 33 | specific = self.parser.add_argument_group("SSL Certificate-specific Options") 34 | specific.add_argument( 35 | "--port", 36 | type=ArgumentType.integer_range(minimum=1, maximum=65535), 37 | default=spec["port"], 38 | help="Destination port", 39 | ) 40 | specific.add_argument( 41 | "--hostname", 42 | default=spec["hostname"], 43 | type=str, 44 | help="SNI Hostname", 45 | ) 46 | 47 | def _get_measurement_kwargs(self): 48 | 49 | r = Command._get_measurement_kwargs(self) 50 | r["port"] = self.arguments.port 51 | if self.arguments.hostname: 52 | r["hostname"] = self.arguments.hostname 53 | 54 | return r 55 | -------------------------------------------------------------------------------- /.github/workflows/python-package.yml: -------------------------------------------------------------------------------- 1 | # This workflow runs whenever a new tag is pushed. 2 | # It will install Python dependencies, run tests and lint with a variety of Python versions 3 | # If tests pass, it will build the package and publish it on pypi.org 4 | 5 | name: Python package 6 | 7 | on: 8 | push: 9 | tags: 10 | - 'v*' 11 | 12 | jobs: 13 | test: 14 | runs-on: ubuntu-latest 15 | strategy: 16 | fail-fast: false 17 | matrix: 18 | python-version: ["3.10", "3.11", "3.12", "3.13"] 19 | 20 | steps: 21 | - uses: actions/checkout@v5 22 | - name: Set up Python ${{ matrix.python-version }} 23 | uses: actions/setup-python@v5 24 | with: 25 | python-version: ${{ matrix.python-version }} 26 | - name: Install dependencies 27 | run: | 28 | python -m pip install --upgrade pip 29 | python -m pip install tox 30 | - name: tox 31 | run: tox 32 | 33 | build: 34 | name: Build package for PyPI 35 | runs-on: ubuntu-latest 36 | needs: test 37 | steps: 38 | - uses: actions/checkout@v5 39 | - name: Set up Python ${{ matrix.python-version }} 40 | uses: actions/setup-python@v5 41 | with: 42 | python-version: '3.11' 43 | - name: Build release 44 | run: | 45 | python -m pip install --upgrade build 46 | python -m build 47 | - name: Upload dist 48 | uses: actions/upload-artifact@v4 49 | with: 50 | name: release-dists 51 | path: dist/ 52 | 53 | publish: 54 | name: Publish package to PyPI 55 | runs-on: ubuntu-latest 56 | needs: build 57 | steps: 58 | - name: Retrieve dist 59 | uses: actions/download-artifact@v5 60 | with: 61 | name: release-dists 62 | path: dist/ 63 | - name: Push to pypi 64 | uses: pypa/gh-action-pypi-publish@release/v1.13 65 | with: 66 | user: __token__ 67 | password: ${{ secrets.PYPI_API_TOKEN }} 68 | verbose: true 69 | 70 | -------------------------------------------------------------------------------- /ripe/atlas/tools/renderers/dst_asn.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from .base import Renderer as BaseRenderer 17 | from collections import Counter 18 | 19 | from ..helpers.sanitisers import sanitise 20 | from ..ipdetails import IP 21 | 22 | 23 | class Renderer(BaseRenderer): 24 | 25 | RENDERS = [BaseRenderer.TYPE_PING] 26 | 27 | SHOW_DEFAULT_HEADER = False 28 | SHOW_DEFAULT_FOOTER = False 29 | 30 | def __init__(self, *args, **kwargs): 31 | 32 | BaseRenderer.__init__(self, *args, **kwargs) 33 | 34 | # Keys are timestamps, data struct captures ASN membership 35 | self.asns = Counter() 36 | self.asn2name = {} 37 | 38 | def on_result(self, result): 39 | dst = result.destination_address 40 | if dst is not None: 41 | ip = IP(dst) 42 | if ip.asn: 43 | self.asns[ip.asn] += 1 44 | self.asn2name[ip.asn] = sanitise(ip.holder) 45 | else: 46 | self.asns[""] += 1 47 | self.asn2name[""] = "unknown" 48 | return "" 49 | return "" 50 | 51 | def additional(self): 52 | 53 | total = sum(self.asns.values()) 54 | 55 | r = "" 56 | for asn, count in self.asns.most_common(): 57 | r += "AS%s %.2f%% (%s)" % ( 58 | asn, 59 | 100.0 * count / total, 60 | self.asn2name[asn], 61 | ) 62 | 63 | return r 64 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | RIPE Atlas Tools (Magellan) 2 | =========================== 3 | |Documentation| |PYPI Version| |Python Versions| |Python Implementations| |Python Format| 4 | 5 | The official command-line client for RIPE Atlas. 6 | 7 | 8 | Full Documentation 9 | ------------------ 10 | 11 | Everything is up on `ReadTheDocs`_ 12 | 13 | Examples 14 | -------- 15 | Configure API key for creating measurements:: 16 | 17 | $ ripe-atlas configure --set authorisation.create=MY_API_KEY 18 | 19 | Ping an IP address from five probes:: 20 | 21 | $ ripe-atlas measure ping ping.ripe.net --probes 5 22 | 23 | .. image:: screenshots/ripe-atlas-measure-ping.png 24 | 25 | Search for connected probes in Germany, grouping by ASN:: 26 | 27 | $ ripe-atlas probe-search --country de --aggregate-by asn_v4 --limit 10 28 | 29 | .. image:: screenshots/ripe-atlas-probe-search.png 30 | 31 | Search for NTP measurement metadata and process the results with awk:: 32 | 33 | $ ripe-atlas measurement-search --type ntp --format tab --no-header --limit 5 \ 34 | | awk -Ft '{printf "#%s (%s)\n", $1, $3}' 35 | 36 | .. image:: screenshots/ripe-atlas-measurement-search.png 37 | 38 | 39 | 40 | Can I Contribute? 41 | ----------------- 42 | 43 | Absolutely. Please read our `guide`_ on how to contribute. 44 | 45 | 46 | Colophon 47 | -------- 48 | 49 | This project was code-named by means of a `poll`_. In order to conform to the 50 | RIPE Atlas theme, it had to be named for an explorer, and so the winning 51 | suggestion was for Magellan, *"in memory of those times when RTT was ~3 years"*. 52 | 53 | .. |Documentation| image:: https://readthedocs.org/projects/ripe-atlas-tools/badge/?version=latest 54 | :target: http://ripe-atlas-tools.readthedocs.org/en/latest/?badge=latest 55 | :alt: Documentation Status 56 | .. _ReadTheDocs: https://ripe-atlas-tools.readthedocs.org/ 57 | .. _guide: https://github.com/RIPE-NCC/ripe-atlas-tools/blob/master/CONTRIBUTING.rst 58 | .. _poll: https://github.com/RIPE-NCC/ripe-atlas-tools/issues/13 59 | .. |PYPI Version| image:: https://img.shields.io/pypi/v/ripe.atlas.tools.svg 60 | :target: https://pypi.org/project/ripe.atlas.tools/ 61 | .. |Python Versions| image:: https://img.shields.io/pypi/pyversions/ripe.atlas.tools.svg 62 | .. |Python Implementations| image:: https://img.shields.io/pypi/implementation/ripe.atlas.tools.svg 63 | .. |Python Format| image:: https://img.shields.io/pypi/format/ripe.atlas.tools.svg 64 | 65 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | import os 2 | from os.path import abspath, dirname, join 3 | from setuptools import setup 4 | 5 | __version__ = None 6 | exec(open("ripe/atlas/tools/version.py").read()) 7 | 8 | # Allow setup.py to be run from any path 9 | os.chdir(os.path.normpath(os.path.join(os.path.abspath(__file__), os.pardir))) 10 | 11 | # Get proper long description for package 12 | current_dir = dirname(abspath(__file__)) 13 | description = open(join(current_dir, "README.rst")).read() 14 | changes = open(join(current_dir, "CHANGES.rst")).read() 15 | long_description = "\n\n".join([description, changes]) 16 | 17 | # Get the long description from README.md 18 | setup( 19 | name="ripe.atlas.tools", 20 | version=__version__, 21 | packages=["ripe", "ripe.atlas", "ripe.atlas.tools"], 22 | namespace_packages=["ripe", "ripe.atlas"], 23 | include_package_data=True, 24 | license="GPLv3", 25 | description="The official command line client for RIPE Atlas", 26 | long_description=long_description, 27 | url="https://github.com/RIPE-NCC/ripe-atlas-tools", 28 | download_url="https://github.com/RIPE-NCC/ripe-atlas-tools", 29 | author="The RIPE Atlas team", 30 | author_email="atlas@ripe.net", 31 | maintainer="The RIPE Atlas team", 32 | maintainer_email="atlas@ripe.net", 33 | install_requires=[ 34 | "IPy", 35 | "python-dateutil", 36 | "requests", 37 | "urllib3>=2.5.0", 38 | "ripe.atlas.cousteau>=2.2,<3", 39 | "ripe.atlas.sagan>=2,<3", 40 | "tzlocal", 41 | "pyyaml", 42 | "pyOpenSSL", 43 | "typing-extensions", 44 | ], 45 | extras_require={ 46 | "doc": ["sphinx", "sphinx_rtd_theme"], 47 | "fast": ["ujson"], 48 | }, 49 | scripts=[ 50 | "scripts/aping", 51 | "scripts/atraceroute", 52 | "scripts/adig", 53 | "scripts/asslcert", 54 | "scripts/ahttp", 55 | "scripts/antp", 56 | "scripts/ripe-atlas", 57 | ], 58 | keywords=["RIPE", "RIPE NCC", "RIPE Atlas", "Command Line"], 59 | classifiers=[ 60 | "License :: OSI Approved :: GNU General Public License v3 (GPLv3)", 61 | "Operating System :: POSIX", 62 | "Operating System :: Unix", 63 | "Programming Language :: Python :: 3.10", 64 | "Programming Language :: Python :: 3.11", 65 | "Programming Language :: Python :: 3.12", 66 | "Programming Language :: Python :: 3.13", 67 | "Topic :: Internet :: WWW/HTTP", 68 | ], 69 | ) 70 | -------------------------------------------------------------------------------- /ripe/atlas/tools/renderers/http.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from ..helpers.colours import colourise 17 | 18 | from .base import Renderer as BaseRenderer 19 | 20 | 21 | class Renderer(BaseRenderer): 22 | """ 23 | We're abusing the Extended Log File Format here to render the result, 24 | amending it to include a few things not originally specified in the W3C 25 | Working Draft: http://www.w3.org/TR/WD-logfile.html Namely: 26 | http-version, header-bytes, and body-bytes 27 | """ 28 | 29 | RENDERS = [BaseRenderer.TYPE_HTTP] 30 | COLOURS = {"2": "green", "3": "blue", "4": "yellow", "5": "red"} 31 | 32 | def on_result(self, result, probes=None): 33 | r = "#Version: 1.0\n#Date: {}\n#Fields: {}\n".format( 34 | result.created.strftime("%Y-%m-%d %H:%M:%S"), 35 | "cs-method cs-uri c-ip s-ip sc-status time-taken http-version " 36 | "header-bytes body-bytes", 37 | ) 38 | for response in result.responses: 39 | r += self._colourise_by_status( 40 | "{} {} {} {} {} {} {} {} {}\n".format( 41 | result.method, 42 | result.uri, 43 | response.source_address, 44 | response.destination_address, 45 | response.code, 46 | response.response_time, 47 | response.version, 48 | response.head_size, 49 | response.body_size, 50 | ), 51 | response.code, 52 | ) 53 | 54 | return r + "\n" 55 | 56 | def _colourise_by_status(self, output, status): 57 | try: 58 | return colourise(output, self.COLOURS[str(status)[0]]) 59 | except (IndexError, KeyError): 60 | return colourise(output, "red") 61 | -------------------------------------------------------------------------------- /ripe/atlas/tools/commands/measure/ping.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from ...helpers.validators import ArgumentType 17 | from ...settings import conf 18 | from .base import Command 19 | 20 | 21 | class PingMeasureCommand(Command): 22 | DESCRIPTION = "Create a ping measurement and wait for the results" 23 | 24 | def add_arguments(self): 25 | 26 | Command.add_arguments(self) 27 | 28 | self.add_primary_argument(name="target", parser=self.parser) 29 | 30 | spec = conf["specification"]["types"]["ping"] 31 | 32 | specific = self.parser.add_argument_group("Ping-specific Options") 33 | specific.add_argument( 34 | "--packets", 35 | type=ArgumentType.integer_range(minimum=1, maximum=16), 36 | default=spec["packets"], 37 | help="The number of packets sent", 38 | ) 39 | specific.add_argument( 40 | "--size", 41 | type=ArgumentType.integer_range(minimum=1, maximum=2048), 42 | default=spec["size"], 43 | help="The size of packets sent", 44 | ) 45 | specific.add_argument( 46 | "--packet-interval", 47 | type=ArgumentType.integer_range(minimum=2, maximum=30000), 48 | default=spec["packet-interval"], 49 | ) 50 | self.add_flag( 51 | parser=specific, 52 | name="include-probe-id", 53 | default=spec["include_probe_id"], 54 | help="Include the ASCII-encoded probe ID in the ping packets", 55 | ) 56 | 57 | def _get_measurement_kwargs(self): 58 | 59 | r = Command._get_measurement_kwargs(self) 60 | 61 | r["packets"] = self.arguments.packets 62 | r["packet_interval"] = self.arguments.packet_interval 63 | r["size"] = self.arguments.size 64 | if self.arguments.include_probe_id: 65 | r["include_probe_id"] = True 66 | 67 | return r 68 | -------------------------------------------------------------------------------- /tests/test_bash_completion.py: -------------------------------------------------------------------------------- 1 | import os 2 | import unittest 3 | import subprocess 4 | 5 | 6 | class BashCompletionTests(unittest.TestCase): 7 | """ 8 | Testing the Python level bash completion code. 9 | This requires setting up the environment as if we got passed data 10 | from bash. 11 | """ 12 | 13 | def setUp(self): 14 | os.environ["RIPE_ATLAS_AUTO_COMPLETE"] = "1" 15 | 16 | def _setup_env(self, substring): 17 | input_str = "ripe-atlas" + substring 18 | os.environ["COMP_WORDS"] = input_str 19 | comp_cword = len(input_str.split(" ")) - 1 # Index of the last word 20 | os.environ["COMP_CWORD"] = str(comp_cword) 21 | 22 | def _autocomplete(self, substring): 23 | 24 | self._setup_env(substring) 25 | cmd_parts = "ripe-atlas" + substring 26 | envs = os.environ.copy() 27 | process = subprocess.Popen( 28 | cmd_parts, 29 | stdout=subprocess.PIPE, 30 | stderr=subprocess.PIPE, 31 | env=envs, 32 | shell=True, 33 | ) 34 | output, error = process.communicate() 35 | return output.decode("utf-8"), error.decode("utf-8") 36 | 37 | def test_commands_completion(self): 38 | """Tests autocompletion of commands.""" 39 | input_str = " " 40 | output, error = self._autocomplete(input_str) 41 | print(output, error) 42 | self.assertTrue("report" in output) 43 | 44 | def test_completion_disable(self): 45 | """ 46 | Tests if autocompletion is disabled if environmental variable 47 | is not set. 48 | """ 49 | input_str = " mea" 50 | del os.environ["RIPE_ATLAS_AUTO_COMPLETE"] 51 | output, error = self._autocomplete(input_str) 52 | print(output, error) 53 | self.assertTrue("No such command" in error) 54 | 55 | def test_command_completion(self): 56 | """Tests autocompletion of specific command.""" 57 | input_str = " meas" 58 | output, error = self._autocomplete(input_str) 59 | print(output, error) 60 | self.assertTrue("measure" in output) 61 | 62 | def test_options_completion(self): 63 | """Tests autocompletion of existing options for a command.""" 64 | input_str = " measure " 65 | output, error = self._autocomplete(input_str) 66 | print(output, error) 67 | self.assertEqual(output, "dns http ntp ping spec sslcert traceroute") 68 | 69 | def test_option_completion(self): 70 | """Tests autocompletion of specific option of a command.""" 71 | input_str = " measure ping --h" 72 | output, error = self._autocomplete(input_str) 73 | print(output, error) 74 | self.assertEqual(output, "--help") 75 | -------------------------------------------------------------------------------- /CONTRIBUTING.rst: -------------------------------------------------------------------------------- 1 | How To Contribute 2 | ================= 3 | 4 | We would love to have contributions from everyone and no contribution is too 5 | small. Please submit as many fixes for typos and grammar bloopers as you can! 6 | 7 | To make participation in this project as pleasant as possible for everyone, 8 | we adhere to the `Code of Conduct`_ by the Python Software Foundation. 9 | 10 | The following steps will help you get started: 11 | 12 | Fork, then clone the repo: 13 | 14 | .. code:: bash 15 | 16 | $ git clone git@github.com:your-username/ripe-atlas-tools.git 17 | 18 | Make sure the tests pass beforehand: 19 | 20 | .. code:: bash 21 | 22 | $ tox 23 | 24 | or 25 | 26 | .. code:: bash 27 | 28 | $ nosetests tests/ 29 | 30 | Make your changes. Include tests for your change. Make the tests pass: 31 | 32 | .. code:: bash 33 | 34 | $ tox 35 | 36 | or 37 | 38 | .. code:: bash 39 | 40 | $ nosetests tests/ 41 | 42 | Push to your fork and `submit a pull request`_. 43 | 44 | Here are a few guidelines that will increase the chances of a quick merge of 45 | your pull request: 46 | 47 | - *Always* try to add tests and docs for your code. If a feature is tested and 48 | documented, it's easier for us to merge it. 49 | - Follow `PEP 8`_. 50 | - Write `good commit messages`_. 51 | - If you change something that is noteworthy, don't forget to add an entry to 52 | the `changes`_. 53 | 54 | .. note:: 55 | - If you think you have a great contribution but aren’t sure whether it 56 | adheres -- or even can adhere -- to the rules: **please submit a pull 57 | request anyway**! In the best case, we can transform it into something 58 | usable, in the worst case the pull request gets politely closed. There’s 59 | absolutely nothing to fear. 60 | - If you have a great idea but you don't know how or don't have the time to 61 | implement it, please consider opening an issue and someone will pick it up 62 | as soon as possible. 63 | 64 | Thank you for considering a contribution to this project! If you have any 65 | questions or concerns, feel free to reach out the RIPE Atlas team via the 66 | `mailing list`_, `GitHub Issue Queue`_, or `messenger pigeon`_ -- if you must. 67 | 68 | .. _submit a pull request: https://github.com/RIPE-NCC/ripe-atlas-tools/compare/ 69 | .. _PEP 8: https://www.python.org/dev/peps/pep-0008/ 70 | .. _good commit messages: http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html 71 | .. _Code of Conduct: https://www.python.org/psf/codeofconduct/ 72 | .. _changes: https://github.com/RIPE-NCC/ripe-atlas-tools/blob/master/CHANGES.rst 73 | .. _mailing list: https://www.ripe.net/mailman/listinfo/ripe-atlas 74 | .. _GitHub Issue Queue: https://github.com/RIPE-NCC/ripe-atlas-tools/issues 75 | .. _messenger pigeon: https://tools.ietf.org/html/rfc1149 76 | -------------------------------------------------------------------------------- /ripe/atlas/tools/helpers/colours.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | import sys 17 | 18 | COLOURS_AVAILABLE = False 19 | try: 20 | # We use curses to detect ANSI colour support 21 | import curses 22 | except ImportError: 23 | # Curses isn't available on all platforms 24 | try: 25 | import colorama # Colorama wraps stdout/stderr on Windows 26 | except ImportError: 27 | pass 28 | else: 29 | colorama.init() 30 | COLOURS_AVAILABLE = True 31 | else: 32 | if sys.stdout.isatty(): 33 | curses.setupterm() 34 | COLOURS_AVAILABLE = curses.tigetnum("colors") >= 8 35 | 36 | 37 | class Colour(object): 38 | @classmethod 39 | def _colourise(cls, text, colour): 40 | return "{}[{}m{}{}[0m".format(chr(0x1B), colour, text, chr(0x1B)) 41 | 42 | @classmethod 43 | def black(cls, text): 44 | return cls._colourise(text, 30) 45 | 46 | @classmethod 47 | def red(cls, text): 48 | return cls._colourise(text, 31) 49 | 50 | @classmethod 51 | def green(cls, text): 52 | return cls._colourise(text, 32) 53 | 54 | @classmethod 55 | def yellow(cls, text): 56 | return cls._colourise(text, 33) 57 | 58 | @classmethod 59 | def blue(cls, text): 60 | return cls._colourise(text, 34) 61 | 62 | @classmethod 63 | def mangenta(cls, text): 64 | return cls._colourise(text, 35) 65 | 66 | @classmethod 67 | def cyan(cls, text): 68 | return cls._colourise(text, 36) 69 | 70 | @classmethod 71 | def white(cls, text): 72 | return cls._colourise(text, 37) 73 | 74 | @classmethod 75 | def bold(cls, text): 76 | return cls._colourise(text, 1) 77 | 78 | 79 | def colourise(text, colour, fileobj=sys.stdout): 80 | """ 81 | Return an ANSI escaped string of the specified content and colour, or 82 | the input text if colour support is not available or not appropriate. 83 | 84 | `fileobj` is used to determine whether the output is a terminal. 85 | """ 86 | if COLOURS_AVAILABLE and fileobj.isatty(): 87 | return getattr(Colour, colour)(text) 88 | else: 89 | return text 90 | -------------------------------------------------------------------------------- /ripe/atlas/tools/renderers/sslcert.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from tzlocal import get_localzone 17 | import OpenSSL 18 | 19 | from ..helpers.colours import colourise 20 | from ..helpers.sanitisers import sanitise 21 | from .base import Renderer as BaseRenderer 22 | 23 | 24 | class Renderer(BaseRenderer): 25 | 26 | RENDERS = [BaseRenderer.TYPE_SSLCERT] 27 | TIME_FORMAT = "%a %b %d %H:%M:%S %Z %Y" 28 | 29 | def on_result(self, result): 30 | r = "" 31 | for certificate in result.certificates: 32 | r += self.get_formatted_response(certificate) 33 | created = result.created.astimezone(get_localzone()) 34 | return "\n{}\n{}\n{}\n".format( 35 | colourise("Probe #{}".format(result.probe_id), "bold"), 36 | colourise(created.strftime(self.TIME_FORMAT), "bold"), 37 | r, 38 | ) 39 | 40 | @classmethod 41 | def get_formatted_response(cls, certificate): 42 | x509 = OpenSSL.crypto.load_certificate( 43 | OpenSSL.crypto.FILETYPE_PEM, 44 | certificate.raw_data.replace("\\/", "/").replace("\n\n", "\n"), 45 | ) 46 | 47 | pkey_type = x509.get_pubkey().type() 48 | 49 | # TODO: to be improved 50 | if pkey_type == 6: 51 | pkey_type_descr = "rsaEncryption" 52 | else: 53 | pkey_type_descr = pkey_type 54 | 55 | return cls.render_template( 56 | "reports/sslcert.txt", 57 | issuer_c=sanitise(certificate.issuer_c), 58 | issuer_o=sanitise(certificate.issuer_o), 59 | issuer_cn=sanitise(certificate.issuer_cn), 60 | not_before=certificate.valid_from, 61 | not_after=certificate.valid_until, 62 | subject_c=sanitise(certificate.subject_c), 63 | subject_o=sanitise(certificate.subject_o), 64 | subject_cn=sanitise(certificate.subject_cn), 65 | version=x509.get_version(), 66 | serial_number=x509.get_serial_number(), 67 | signature_algorithm=x509.get_signature_algorithm(), 68 | pkey_type=pkey_type_descr, 69 | pkey_bits=x509.get_pubkey().bits(), 70 | sha1fp=certificate.checksum_sha1, 71 | sha256fp=certificate.checksum_sha256, 72 | ) 73 | -------------------------------------------------------------------------------- /ripe/atlas/tools/commands/measure/__init__.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from ...exceptions import RipeAtlasToolsException 17 | from ..base import Factory as BaseFactory 18 | from .ping import PingMeasureCommand 19 | from .traceroute import TracerouteMeasureCommand 20 | from .dns import DnsMeasureCommand 21 | from .sslcert import SslcertMeasureCommand 22 | from .http import HttpMeasureCommand 23 | from .ntp import NtpMeasureCommand 24 | from .spec import SpecMeasureCommand 25 | 26 | 27 | class Factory(BaseFactory): 28 | 29 | TYPES = { 30 | "ping": PingMeasureCommand, 31 | "traceroute": TracerouteMeasureCommand, 32 | "dns": DnsMeasureCommand, 33 | "sslcert": SslcertMeasureCommand, 34 | "http": HttpMeasureCommand, 35 | "ntp": NtpMeasureCommand, 36 | "spec": SpecMeasureCommand, 37 | } 38 | DESCRIPTION = "Create a measurement and wait for the results" 39 | 40 | def __init__(self, sys_args): 41 | 42 | self.build_class = None 43 | self.sys_args = sys_args 44 | if len(self.sys_args) >= 2: 45 | self.build_class = self.TYPES.get(self.sys_args[1].lower()) 46 | 47 | if not self.build_class: 48 | self.raise_log() 49 | 50 | def raise_log(self): 51 | """Depending on the input raise with different log message.""" 52 | # cases: 1) ripe-atlas measure 2) ripe-atlas measure --help/-h 53 | if len(self.sys_args) == 1 or ( 54 | len(self.sys_args) == 2 and self.sys_args[1] in ("--help", "-h") 55 | ): 56 | log = "Usage: ripe-atlas measure [arguments]\n\n" "Types:\n" 57 | for type_name, type_ in sorted(self.TYPES.items()): 58 | log += f"\t{type_name:<12} {type_.DESCRIPTION}\n" 59 | log += ( 60 | "\nFor extended options for a specific measurement type, " 61 | "try ripe-atlas measure --help." 62 | ) 63 | # cases: ripe-atlas measure bla 64 | else: 65 | log = ( 66 | "The measurement type you requested is invalid. " 67 | "Please choose one of {}." 68 | ).format(", ".join(self.TYPES.keys())) 69 | raise RipeAtlasToolsException(log) 70 | 71 | def create(self, *args, **kwargs): 72 | return self.build_class(*args, **kwargs) 73 | -------------------------------------------------------------------------------- /tests/renderers/test_http.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2015 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | import unittest 17 | 18 | from ripe.atlas.sagan import Result 19 | from ripe.atlas.tools.renderers.http import Renderer 20 | 21 | 22 | class TestHttpRenderer(unittest.TestCase): 23 | def __init__(self, *args, **kwargs): 24 | unittest.TestCase.__init__(self, *args, **kwargs) 25 | self.basic = Result.get( 26 | '{"lts":64,"from":"217.13.64.36","msm_id":2841267,"fw":4720,"timestamp":1450185727,"uri":"http://at-vie-as1120.anchors.atlas.ripe.net:80/4096","prb_id":1,"result":[{"rt":45.953289,"src_addr":"217.13.64.36","hsize":131,"af":4,"bsize":1668618,"res":200,"method":"GET","ver":"1.1","dst_addr":"193.171.255.2"}],"group_id":2841267,"type":"http","msm_name":"HTTPGet"}' # noqa: E501 27 | ) 28 | self.multiple = Result.get( 29 | '{"lts":64,"from":"217.13.64.36","msm_id":2841267,"fw":4720,"timestamp":1450185727,"uri":"http://at-vie-as1120.anchors.atlas.ripe.net:80/4096","prb_id":1,"result":[{"rt":45.953289,"src_addr":"217.13.64.36","hsize":131,"af":4,"bsize":1668618,"res":200,"method":"GET","ver":"1.1","dst_addr":"193.171.255.2"},{"rt":45.953289,"src_addr":"217.13.64.36","hsize":131,"af":4,"bsize":1668618,"res":200,"method":"GET","ver":"1.1","dst_addr":"193.171.255.2"}],"group_id":2841267,"type":"http","msm_name":"HTTPGet"}' # noqa: E501 30 | ) 31 | 32 | def test_basic(self): 33 | expected = ( 34 | "#Version: 1.0\n" 35 | "#Date: 2015-12-15 13:22:07\n" 36 | "#Fields: cs-method cs-uri c-ip s-ip sc-status time-taken http-version header-bytes body-bytes\n" # noqa: E501 37 | "GET http://at-vie-as1120.anchors.atlas.ripe.net:80/4096 217.13.64.36 193.171.255.2 200 45.953289 1.1 131 1668618\n\n" # noqa: E501 38 | ) 39 | self.assertEqual(Renderer().on_result(self.basic), expected) 40 | 41 | def test_multiple(self): 42 | expected = ( 43 | "#Version: 1.0\n" 44 | "#Date: 2015-12-15 13:22:07\n" 45 | "#Fields: cs-method cs-uri c-ip s-ip sc-status time-taken http-version header-bytes body-bytes\n" # noqa: E501 46 | "GET http://at-vie-as1120.anchors.atlas.ripe.net:80/4096 217.13.64.36 193.171.255.2 200 45.953289 1.1 131 1668618\n" # noqa: E501 47 | "GET http://at-vie-as1120.anchors.atlas.ripe.net:80/4096 217.13.64.36 193.171.255.2 200 45.953289 1.1 131 1668618\n\n" # noqa: E501 48 | ) 49 | self.assertEqual(Renderer().on_result(self.multiple), expected) 50 | -------------------------------------------------------------------------------- /ripe/atlas/tools/renderers/traceroute.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from tzlocal import get_localzone 17 | from .base import Renderer as BaseRenderer 18 | 19 | from ..helpers.colours import colourise 20 | from ..helpers.sanitisers import sanitise 21 | from ..ipdetails import IP 22 | 23 | 24 | class Renderer(BaseRenderer): 25 | 26 | RENDERS = [BaseRenderer.TYPE_TRACEROUTE] 27 | TIME_FORMAT = "%a %b %d %H:%M:%S %Z %Y" 28 | DEFAULT_SHOW_ASNS = False 29 | 30 | @staticmethod 31 | def add_arguments(parser): 32 | group = parser.add_argument_group( 33 | title="Optional arguments for traceroute renderer" 34 | ) 35 | group.add_argument( 36 | "--traceroute-show-asns", 37 | help="Show Autonomous System Numbers (ASNs) in the traceroute " "results.", 38 | action="store_true", 39 | default=Renderer.DEFAULT_SHOW_ASNS, 40 | ) 41 | 42 | def __init__(self, **kwargs): 43 | BaseRenderer.__init__(self, **kwargs) 44 | 45 | if "arguments" in kwargs: 46 | self.show_asns = kwargs["arguments"].traceroute_show_asns 47 | else: 48 | self.show_asns = Renderer.DEFAULT_SHOW_ASNS 49 | 50 | def on_result(self, result): 51 | 52 | r = "" 53 | 54 | for hop in result.hops: 55 | 56 | if hop.is_error: 57 | r += "{}\n".format(colourise(sanitise(hop.error_message), "red")) 58 | continue 59 | 60 | name = "" 61 | asn = "" 62 | rtts = [] 63 | for packet in hop.packets: 64 | name = name or packet.origin or "*" 65 | if self.show_asns: 66 | if packet.origin and not asn: 67 | asn = IP(packet.origin).asn 68 | if packet.rtt: 69 | rtts.append("{:8} ms".format(packet.rtt)) 70 | else: 71 | rtts.append(" *") 72 | 73 | if not asn: 74 | tpl = "{hop:>3} {name:37} {rtts}\n" 75 | else: 76 | tpl = "{hop:>3} {name:28} {asn:>8} {rtts}\n" 77 | 78 | r += tpl.format( 79 | hop=hop.index, 80 | name=sanitise(name), 81 | asn="AS{}".format(asn) if asn else "", 82 | rtts=" ".join(rtts), 83 | ) 84 | 85 | created = result.created.astimezone(get_localzone()) 86 | return "\n{}\n{}\n\n{}".format( 87 | colourise("Probe #{}".format(result.probe_id), "bold"), 88 | colourise(created.strftime(self.TIME_FORMAT), "bold"), 89 | r, 90 | ) 91 | -------------------------------------------------------------------------------- /docs/troubleshooting.rst: -------------------------------------------------------------------------------- 1 | .. _troubleshooting: 2 | 3 | Troubleshooting 4 | =============== 5 | 6 | Sometimes things don't go as planned. In these cases, this page is here to 7 | help. 8 | 9 | 10 | .. _troubleshooting-insecureplatformwarning: 11 | 12 | InsecurePlatformWarning 13 | ----------------------- 14 | 15 | On older systems (running Python versions <2.7.10), you may be presented with a 16 | warning message that looks like this:: 17 | 18 | /path/to/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:100: 19 | InsecurePlatformWarning: A true SSLContext object is not available. This 20 | prevents urllib3 from configuring SSL appropriately and may cause certain 21 | SSL connections to fail. For more information, see 22 | https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. 23 | InsecurePlatformWarning 24 | 25 | This is due to the insecure way older versions of Python handle secure 26 | connections and a visit to the above URL will tell you that the fix is one of 27 | three options: 28 | 29 | * Upgrade to a modern version of Python 30 | * Install three Python packages: ``pyopenssl``, ``ndg-httpsclient``, and 31 | ``pyasn1`` 32 | * `Suppress the warnings`_. Don't do that though. 33 | 34 | .. _Suppress the warnings: https://urllib3.readthedocs.org/en/latest/security.html#disabling-warnings 35 | 36 | 37 | .. _troubleshooting-saganopensslosx: 38 | 39 | Sagan, OpenSSL, and OSX 40 | ----------------------- 41 | 42 | If you're using Mac OSX, the installation of Sagan, (one of Magellan's 43 | dependencies) may give you trouble, especially in how Apple handles PyOpenSSL on 44 | their machines. Workarounds and proper fixes for this issue can be found in the 45 | `Sagan installation documentation`_. 46 | 47 | .. _Sagan installation documentation: https://ripe-atlas-sagan.readthedocs.org/en/latest/installation.html#troubleshooting 48 | 49 | 50 | .. _troubleshooting-libyaml: 51 | 52 | Complaints from libyaml 53 | ----------------------- 54 | 55 | During the installation, you may see something like this scroll by: 56 | 57 | .. code::none 58 | 59 | Running setup.py install for pyyaml 60 | checking if libyaml is compilable 61 | x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c build/temp.linux-x86_64-2.7/check_libyaml.c -o build/temp.linux-x86_64-2.7/check_libyaml.o 62 | build/temp.linux-x86_64-2.7/check_libyaml.c:2:18: fatal error: yaml.h: No such file or directory 63 | #include 64 | ^ 65 | compilation terminated. 66 | 67 | libyaml is not found or a compiler error: forcing --without-libyaml 68 | (if libyaml is installed correctly, you may need to 69 | specify the option --include-dirs or uncomment and 70 | modify the parameter include_dirs in setup.cfg) 71 | 72 | Don't worry. This is just the installation script noticing that you don't have 73 | libyaml installed and it's complaining because it's good to have around for 74 | performance reasons. However, since we're only using YAML for configuration, 75 | performance isn't an issue, and the fallback option will be sufficient. 76 | 77 | If however, you don't like these sorts of errors, make sure that libyaml is 78 | installed for your distribution before attempting to install this toolkit. 79 | -------------------------------------------------------------------------------- /ripe/atlas/tools/commands/probe_info.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from ripe.atlas.cousteau import Probe 17 | from ripe.atlas.cousteau.exceptions import APIResponseError 18 | 19 | from .base import Command as BaseCommand, MetaDataMixin 20 | from ..exceptions import RipeAtlasToolsException 21 | from ..helpers.colours import colourise 22 | from ..helpers.sanitisers import sanitise 23 | from ..helpers.validators import ArgumentType 24 | from ..settings import conf 25 | 26 | 27 | class Command(MetaDataMixin, BaseCommand): 28 | 29 | NAME = "probe-info" 30 | DESCRIPTION = "Return the meta data for one probe" 31 | 32 | def add_arguments(self): 33 | self.parser.add_argument( 34 | "id", type=ArgumentType.probe_id_or_name(), help="The probe id or alias" 35 | ) 36 | 37 | def run(self): 38 | 39 | try: 40 | probe = Probe( 41 | server=conf["api-server"], 42 | id=self.arguments.id, 43 | user_agent=self.user_agent, 44 | ) 45 | except APIResponseError: 46 | raise RipeAtlasToolsException("That probe does not appear to exist") 47 | 48 | keys = ( 49 | ("id", "ID"), 50 | ( 51 | "id", 52 | "URL", 53 | lambda id: colourise(f"{conf['website-url']}/probes/{id}/", "cyan"), 54 | ), 55 | ("is_public", "Public?", self._prettify_boolean), 56 | ("is_anchor", "Anchor?", self._prettify_boolean), 57 | ("country_code", "Country"), 58 | ("description", "Description", sanitise), 59 | ("asn_v4", "ASN (IPv4)"), 60 | ("asn_v6", "ASN (IPv6)"), 61 | ("address_v4", "Address (IPv4)"), 62 | ("address_v6", "Address (IPv6)"), 63 | ("prefix_v4", "Prefix (IPv4)"), 64 | ("prefix_v6", "Prefix (IPv6)"), 65 | ("geometry", "Coordinates", self._prettify_coordinates), 66 | ("status", "Status"), 67 | ) 68 | for key in keys: 69 | 70 | value = getattr(probe, key[0]) 71 | 72 | if value is None: 73 | value = "-" 74 | elif len(key) == 3: 75 | value = key[2](value) 76 | 77 | self._render_line(key[1], value) 78 | 79 | print(colourise("Tags", "bold")) 80 | for tag in probe.tags: 81 | print(" {}".format(tag["slug"])) 82 | 83 | @staticmethod 84 | def _prettify_coordinates(geometry): 85 | if geometry and "coordinates" in geometry and geometry["coordinates"]: 86 | return "{},{}".format( 87 | geometry["coordinates"][1], geometry["coordinates"][0] 88 | ) 89 | -------------------------------------------------------------------------------- /tests/commands/test_base.py: -------------------------------------------------------------------------------- 1 | # coding=utf-8 2 | 3 | # Copyright (c) 2015 RIPE NCC 4 | # 5 | # This program is free software: you can redistribute it and/or modify 6 | # it under the terms of the GNU General Public License as published by 7 | # the Free Software Foundation, either version 3 of the License, or 8 | # (at your option) any later version. 9 | # 10 | # This program is distributed in the hope that it will be useful, 11 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 12 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 13 | # GNU General Public License for more details. 14 | # 15 | # You should have received a copy of the GNU General Public License 16 | # along with this program. If not, see . 17 | 18 | import unittest 19 | from unittest import mock 20 | from io import StringIO 21 | 22 | 23 | from ripe.atlas.tools.commands import base 24 | from ripe.atlas.tools.version import __version__ 25 | 26 | 27 | class TestBaseCommand(unittest.TestCase): 28 | SAMPLE_PLATFORM = "Linux-5.4.0-91-generic-x86_64-with-glibc2.29" 29 | 30 | @mock.patch("ripe.atlas.tools.commands.base.open") 31 | def test_user_agent_configured(self, mock_open): 32 | tests = { 33 | "Some Cool User Agent String": "Some Cool User Agent String", 34 | "Some custom agent\nwith a second line": "Some custom agent", 35 | "x" * 3000: "x" * 128, 36 | "Πράκτορας χρήστη": "Πράκτορας χρήστη", 37 | "이것은 테스트 요원": "이것은 테스트 요원", 38 | } 39 | 40 | for contents, expected in tests.items(): 41 | s = StringIO(contents) 42 | mock_open.return_value = s 43 | cmd = base.Command() 44 | self.assertEqual(cmd.user_agent, expected) 45 | 46 | @mock.patch("platform.system", return_value="Darwin") 47 | @mock.patch("platform.mac_ver", return_value=("11.6", ("", "", ""), "x86_64")) 48 | def test_user_agent_mac(self, *mocks): 49 | cmd = base.Command() 50 | self.assertEqual(cmd.user_agent, f"RIPE Atlas Tools [macOS 11.6] {__version__}") 51 | 52 | @mock.patch("platform.system", return_value="Windows") 53 | @mock.patch( 54 | "platform.win32_ver", 55 | return_value=("10", "10.0.10240", "", "Multiprocessor Free"), 56 | ) 57 | def test_user_agent_windows(self, *mocks): 58 | cmd = base.Command() 59 | self.assertEqual(cmd.user_agent, f"RIPE Atlas Tools [Windows 10] {__version__}") 60 | 61 | @mock.patch("platform.system", return_value="Linux") 62 | @mock.patch( 63 | "ripe.atlas.tools.helpers.xdg.freedesktop_os_release", 64 | return_value={ 65 | "NAME": "Debian GNU/Linux", 66 | "VERSION_ID": "10", 67 | }, 68 | ) 69 | def test_user_agent_xdg_present(self, *mocks): 70 | cmd = base.Command() 71 | self.assertEqual( 72 | cmd.user_agent, f"RIPE Atlas Tools [Debian GNU/Linux 10] {__version__}" 73 | ) 74 | 75 | @mock.patch("platform.system", return_value="Linux") 76 | @mock.patch("platform.platform", return_value=SAMPLE_PLATFORM) 77 | @mock.patch( 78 | "ripe.atlas.tools.helpers.xdg.freedesktop_os_release", side_effect=OSError 79 | ) 80 | def test_user_agent_xdg_absent(self, *mocks): 81 | cmd = base.Command() 82 | self.assertEqual( 83 | cmd.user_agent, 84 | f"RIPE Atlas Tools [{self.SAMPLE_PLATFORM}] {__version__}", 85 | ) 86 | -------------------------------------------------------------------------------- /ripe/atlas/tools/commands/stream.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from ripe.atlas.cousteau import Measurement, AtlasStream 17 | from ripe.atlas.cousteau.exceptions import APIResponseError 18 | 19 | from ..exceptions import RipeAtlasToolsException 20 | from ..renderers import Renderer 21 | from ..streaming import StreamWrapper 22 | from ..helpers.validators import ArgumentType 23 | from ..settings import conf 24 | from .base import Command as BaseCommand 25 | 26 | 27 | class Command(BaseCommand): 28 | 29 | NAME = "stream" 30 | 31 | DESCRIPTION = "Output the results of a public measurement as they become available" 32 | EXTRA_DESCRIPTION = "Streaming of non-public measurements is not supported." 33 | URLS = { 34 | "detail": "/api/v2/measurements/{0}.json", 35 | } 36 | 37 | def add_arguments(self): 38 | self.parser.add_argument( 39 | "measurement_id", 40 | type=ArgumentType.msm_id_or_name(), 41 | help="The measurement id or alias you want streamed", 42 | ) 43 | self.parser.add_argument( 44 | "--limit", 45 | type=int, 46 | help="The maximum number of results you want to stream", 47 | ) 48 | self.parser.add_argument( 49 | "--renderer", 50 | choices=Renderer.get_available(), 51 | help="The renderer you want to use. If this isn't defined, an " 52 | "appropriate renderer will be selected.", 53 | ) 54 | self.parser.add_argument( 55 | "--timeout", 56 | type=float, 57 | help="Stop streaming after this number of seconds", 58 | ) 59 | self.parser.add_argument( 60 | "--send-backlog", action="store_true", help="Send backlog of recent results" 61 | ) 62 | 63 | Renderer.add_arguments_for_available_renderers(self.parser) 64 | 65 | def run(self) -> None: 66 | try: 67 | measurement = Measurement( 68 | id=self.arguments.measurement_id, 69 | user_agent=self.user_agent, 70 | ) 71 | except APIResponseError as e: 72 | raise RipeAtlasToolsException(e.args[0]) 73 | 74 | self.ok("Connecting to stream...") 75 | stream = AtlasStream(base_url=conf["stream-base-url"]) 76 | stream.connect() 77 | stream.subscribe( 78 | "result", 79 | msm=self.arguments.measurement_id, 80 | sendBacklog=self.arguments.send_backlog, 81 | ) 82 | renderer = Renderer.get_renderer( 83 | name=self.arguments.renderer, kind=measurement.type.lower() 84 | )(arguments=self.arguments) 85 | renderer.render( 86 | StreamWrapper( 87 | stream, 88 | capture_limit=self.arguments.limit, 89 | timeout=self.arguments.timeout, 90 | ) 91 | ) 92 | self.ok("Disconnected from stream") 93 | -------------------------------------------------------------------------------- /ripe/atlas/tools/renderers/ntp.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from tzlocal import get_localzone 17 | from datetime import datetime 18 | from .base import Renderer as BaseRenderer 19 | from ..helpers.colours import colourise 20 | 21 | 22 | class Renderer(BaseRenderer): 23 | RENDERS = [BaseRenderer.TYPE_NTP] 24 | TIME_FORMAT = "%a %b %d %H:%M:%S %Z %Y" 25 | 26 | def on_result(self, result): 27 | created = result.created.astimezone(get_localzone()) 28 | r = self.get_formatted_response(result) 29 | if not r: 30 | r = colourise("No results\n", "red") 31 | return "\n{}\n{}\n\n{}".format( 32 | colourise("Probe #{}".format(result.probe_id), "bold"), 33 | colourise(created.strftime(self.TIME_FORMAT), "bold"), 34 | r, 35 | ) 36 | 37 | @staticmethod 38 | def get_formatted_response(result): 39 | leap = result.leap_second_indicator 40 | stratum = result.stratum 41 | v = result.version 42 | mode = result.mode 43 | # just here for completeness, flake8 doesn't like when it's not used 44 | # end_time = result.end_time 45 | poll = result.poll 46 | precision = result.precision 47 | refid = result.reference_id 48 | ref_time = result.reference_time 49 | root_delay = result.root_delay 50 | root_disp = result.root_dispersion 51 | 52 | if not leap and not stratum and not v and not mode: 53 | return 54 | 55 | if mode != "server": 56 | print("invalid mode: %s" % mode) 57 | 58 | r = "[NTP] %s -> %s (%s)\n" % ( 59 | result.source_address, 60 | result.destination_name, 61 | result.destination_address, 62 | ) 63 | r += "\tversion: %s, stratum: %s/16\n" % ( 64 | colourise(v, "bold"), 65 | colourise(stratum, "bold"), 66 | ) 67 | r += "\trefid: %s\n" % colourise(refid, "bold") 68 | r += "\tleap: %s, poll: %s, precision: %s\n" % (leap, poll, precision) 69 | r += "\troot_delay: %s, root_disp: %s\n" % (root_delay, root_disp) 70 | r += "\tref-time: %s\n\n" % datetime.fromtimestamp(ref_time) 71 | 72 | try: 73 | for idx, pkt in enumerate(result.packets): 74 | if not pkt.rtt: 75 | r += f"\t[{idx}] *\n]" 76 | continue 77 | r += "\t[%s] %s\n" % (idx, str(pkt)) 78 | r += "\t\ttrans: %s -> recv: %s\n" % ( 79 | pkt.transmitted_time, 80 | pkt.received_time, 81 | ) 82 | r += "\t\torigin: %s -> final: %s\n\n" % ( 83 | pkt.origin_time, 84 | pkt.final_time, 85 | ) 86 | except Exception as ex: 87 | print("Got exception when reading packet: %s" % ex) 88 | print("Raw: %s" % pkt.raw_data) 89 | 90 | return r 91 | -------------------------------------------------------------------------------- /PACKAGING.md: -------------------------------------------------------------------------------- 1 | We're working with the community to get this project packaged for as many 2 | platforms as possible. To that end, We've created this file to help organise 3 | resources. 4 | 5 | If you'd like to package the toolkit for your favourite distro, or if you've 6 | already started, please add your name/GitHub below with a pull request so we 7 | don't have people wasting time doubling up on work rather than collaborating. 8 | 9 | 10 | ## OpenBSD 11 | 12 | * [Florian Obser](https://github.com/fobser) 13 | 14 | ### Status 15 | 16 | During the Bucharest hackathon, Florian hacked out a package it's already 17 | [in the ports tree](http://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/net/py-ripe.atlas.tools/). 18 | 19 | 20 | ## FreeBSD 21 | 22 | * [Max Stucchi](https://github.com/stucchimax) 23 | 24 | ### Status 25 | 26 | Max has submitted the new ports and [they have been accepted](https://svnweb.freebsd.org/ports?view=revision&revision=403526). 27 | 28 | 29 | ## NixOS 30 | 31 | * Maintainer: [Ryan Lahfa](https://github.com/RaitoBezarius) 32 | * with the help of the Nixpkgs community 33 | 34 | ### Status 35 | 36 | Introduced in https://github.com/NixOS/nixpkgs/pull/187997. It can be used with `nix-shell -p ripe-atlas-tools` or `nix-env -iA ripe-atlas-tools`. 37 | 38 | ## Gentoo 39 | 40 | * [Daniel Quinn](https://github.com/danielquinn) 41 | 42 | ### Status 43 | 44 | After a great deal of help from #gentoo-proxy-maint, Daniel's pull requests were accepted into the Gentoo portage tree. You can now install it with `emerge ripe-atlas-tools`. 45 | 46 | 47 | ## Debian 48 | 49 | * Apollon Oikonomopoulos 50 | 51 | ### Status 52 | 53 | Apollon has imported the project to [debian tree](https://tracker.debian.org/pkg/ripe-atlas-tools). 54 | 55 | ## Ubuntu 56 | 57 | ### Status 58 | 59 | Apollon's work in debian has made it also to Ubuntu. 60 | 61 | ## Arch 62 | 63 | * [Wouter de Vries](https://github.com/woutifier) 64 | 65 | ### Status 66 | 67 | Wouter has added this project to [Arch's AUR repository](https://aur.archlinux.org/packages/ripe-atlas-tools). 68 | 69 | ## Fedora 70 | 71 | * [Jan Včelák](https://github.com/vcelda) 72 | 73 | ### Status 74 | 75 | In progress: https://github.com/fcelda/fedora-ripe-atlas-tools 76 | 77 | Jan is currently building the binary packages in COPR (which will take some time as there is a lot of other packages in the queue): 78 | https://copr.fedoraproject.org/coprs/jvcelak/ripe-atlas-tools/ 79 | 80 | ## Voidlinux 81 | 82 | * [jnbr](https://github.com/jnbr) 83 | 84 | ### Status 85 | 86 | With a lot of help from the Voidlinux maintainers, it got merged into the [XBPS source packages collection](https://github.com/voidlinux/void-packages/srcpkgs/ripe-atlas-tools). You can install it with `xbps-install ripe-atlas-tools`. 87 | 88 | ## Windows 89 | 90 | * [Chris Amin](https://github.com/chrisamin) 91 | 92 | ### Status 93 | 94 | A highly experimental self-contained installer is available at https://github.com/chrisamin/ripe-atlas-tools-win32/. This installer doesn't require the presence of a system Python installation. 95 | 96 | ## Other Platforms 97 | 98 | We've been talking with members of the community about expanding the package 99 | support for Magellan, but so far no one has officially volunteered. The RIPE 100 | Atlas team is happy to assist anyone interested in porting this toolkit to any 101 | platform, but we're especially keen on at least getting into: 102 | 103 | * Red Hat 104 | * CentOS 105 | 106 | If you'd like to try your hand, or would simply like to offer some advice, feel 107 | free to add your name here or contact us directly via *atlas at ripe dot net*. 108 | -------------------------------------------------------------------------------- /ripe/atlas/tools/commands/measure/spec.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | import sys 17 | import json 18 | 19 | from .base import Command 20 | from ...settings import conf 21 | from ...exceptions import RipeAtlasToolsException 22 | 23 | 24 | class SpecMeasureCommand(Command): 25 | DESCRIPTION = "Create a measurement from a JSON spec and wait for the results" 26 | 27 | def clean_target(self): 28 | return self.arguments.target 29 | 30 | def add_arguments(self): 31 | 32 | Command.add_arguments(self) 33 | 34 | self.parser.add_argument( 35 | "json_spec", 36 | help="JSON object containing the RIPE Atlas measurement spec. " 37 | "Use @some-file.json to load the spec from a given file, or @- to load the " 38 | "spec from standard input. " 39 | "Any optional command-line arguments you specify will be merged into this " 40 | "spec.", 41 | ) 42 | self.parser.add_argument( 43 | "--type", 44 | help="The 'type' of the measurement to be added to the spec JSON", 45 | ) 46 | 47 | def _read_file(self, filename): 48 | if filename == "-": 49 | return sys.stdin.read() 50 | try: 51 | with open(filename) as f: 52 | return f.read() 53 | except FileNotFoundError: 54 | raise RipeAtlasToolsException(f"No such file: {filename}") 55 | 56 | def _clean_json_spec(self): 57 | arg = self.arguments.json_spec 58 | if arg.startswith("@"): 59 | arg = self._read_file(arg[1:]) 60 | 61 | try: 62 | spec = json.loads(arg) 63 | except ValueError: 64 | raise RipeAtlasToolsException("Spec is invalid JSON") 65 | 66 | if not isinstance(spec, dict): 67 | raise RipeAtlasToolsException("Spec should be a JSON object") 68 | return spec 69 | 70 | def _get_measurement_kwargs(self): 71 | spec = self._clean_json_spec() 72 | 73 | if self.arguments.type: 74 | self._type = spec["type"] = self.arguments.type 75 | else: 76 | self._type = spec.get("type") 77 | if not self._type: 78 | raise RipeAtlasToolsException('Spec should contain a "type"') 79 | if self._type not in conf["specification"]["types"]: 80 | raise RipeAtlasToolsException(f"Unknown measurement type: {self._type}") 81 | 82 | from_args = super()._get_measurement_kwargs() 83 | 84 | # Allow override of description or use a different default that doesn't 85 | # bother about the target 86 | if not self.arguments.description: 87 | del from_args["description"] 88 | if "description" not in spec: 89 | spec["description"] = f"{self._type.title()} measurement" 90 | 91 | # Allow override of "af" otherwise use the default 92 | if not self.arguments.af and "af" in spec: 93 | del from_args["af"] 94 | 95 | spec.update(from_args) 96 | 97 | return spec 98 | -------------------------------------------------------------------------------- /ripe/atlas/tools/aggregators/base.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | import itertools 16 | 17 | 18 | class ValueKeyAggregator(object): 19 | """Aggregator based on tha actual value of the key/attribute""" 20 | 21 | def __init__(self, key, prefix=None): 22 | self.aggregation_keys = key.split(".") 23 | self.key_prefix = prefix or self.aggregation_keys[-1].upper() 24 | 25 | def get_key_value(self, entity): 26 | """ 27 | Returns the value of the key/attribute the aggregation will use to 28 | bucketize probes/results 29 | """ 30 | attribute = entity 31 | for key in self.aggregation_keys: 32 | attribute = getattr(attribute, key) 33 | return attribute 34 | 35 | def get_bucket(self, entity): 36 | """ 37 | Returns the bucket the specific entity belongs to based on the give 38 | key/attribute 39 | """ 40 | return "{0}: {1}".format(self.key_prefix, self.get_key_value(entity)) 41 | 42 | 43 | class RangeKeyAggregator(ValueKeyAggregator): 44 | """ 45 | Aggregator based on where the position of the value of the key/attribute is 46 | in the given range 47 | """ 48 | 49 | def __init__(self, key, ranges): 50 | ValueKeyAggregator.__init__(self, key) 51 | self.aggregation_ranges = sorted(ranges, reverse=True) 52 | 53 | def get_bucket(self, entity): 54 | """ 55 | Returns the bucket the specific entity belongs to based on the give 56 | key/attribute 57 | """ 58 | 59 | bucket = "{0}: < {1}".format( 60 | self.key_prefix, self.aggregation_ranges[-1] 61 | ) 62 | 63 | key_value = self.get_key_value(entity) 64 | for index, krange in enumerate(self.aggregation_ranges): 65 | if key_value > krange: 66 | if index == 0: 67 | bucket = "{0}: > {1}".format(self.key_prefix, krange) 68 | else: 69 | bucket = "{0}: {1}-{2}".format( 70 | self.key_prefix, 71 | krange, 72 | self.aggregation_ranges[index - 1], 73 | ) 74 | break 75 | 76 | return bucket 77 | 78 | 79 | def _get_sort_key(kv): 80 | key = [] 81 | for is_digit, part in itertools.groupby(kv[0], key=str.isdigit): 82 | part = "".join(part) 83 | if is_digit: 84 | part = int(part) 85 | key.append(part) 86 | return key 87 | 88 | 89 | def aggregate(entities, aggregators): 90 | """ 91 | Aggregate the given entities using the given aggregators. 92 | 93 | Returns a dict of {combined_aggregation_key_tuple: entity_list}, where 94 | the keys are in ascending numeric >> lexical order. 95 | """ 96 | if not aggregators: 97 | return entities 98 | 99 | buckets = {} 100 | 101 | for e in entities: 102 | key = " | ".join(a.get_bucket(e) for a in aggregators) 103 | bucket = buckets.setdefault(key, []) 104 | bucket.append(e) 105 | 106 | return dict(sorted(buckets.items(), key=_get_sort_key)) 107 | -------------------------------------------------------------------------------- /tests/commands/test_loading.py: -------------------------------------------------------------------------------- 1 | import os.path 2 | import unittest 3 | import tempfile 4 | import shutil 5 | import sys 6 | 7 | from unittest import mock 8 | 9 | from io import StringIO 10 | 11 | from ripe.atlas.tools.commands.base import Command 12 | 13 | 14 | USER_COMMAND = """ 15 | from ripe.atlas.tools.commands.base import Command as BaseCommand 16 | 17 | class Command(BaseCommand): 18 | NAME = 'user-command-1' 19 | """ 20 | 21 | 22 | class TestCommandLoading(unittest.TestCase): 23 | expected_builtins = [ 24 | "configure", 25 | "alias", 26 | "go", 27 | "measure", 28 | "measurement-info", 29 | "measurement-search", 30 | "probe-info", 31 | "probe-search", 32 | "report", 33 | "shibboleet", 34 | "stream", 35 | ] 36 | 37 | def setUp(self): 38 | # Create a directory for storing user commands and insert the dummy 39 | # command 40 | self.user_command_path = tempfile.mkdtemp() 41 | with open(os.path.join(self.user_command_path, "user_command_1.py"), "w") as f: 42 | f.write(USER_COMMAND) 43 | 44 | def tearDown(self): 45 | shutil.rmtree(self.user_command_path) 46 | 47 | @mock.patch( 48 | "ripe.atlas.tools.commands.base.Command._get_user_command_path", 49 | return_value=None, 50 | ) 51 | def test_command_loading(self, _get_user_command_path): 52 | _get_user_command_path.return_value = self.user_command_path 53 | 54 | # For some reason the command classes are initialized in some envs 55 | # but not in others at this point 56 | if Command._commands: 57 | Command._commands.clear() 58 | Command._load_commands() 59 | 60 | available_commands = Command.get_available_commands() 61 | 62 | # Check that we have the command list that we expect 63 | self.assertEqual( 64 | sorted(available_commands), 65 | sorted( 66 | [b.replace("-", "_") for b in self.expected_builtins] 67 | + ["user_command_1"] 68 | ), 69 | ) 70 | 71 | # Check that we can load (i.e. import) every builtin command 72 | for expected_builtin in self.expected_builtins: 73 | self.assertIn(expected_builtin.replace("-", "_"), available_commands) 74 | cmd_cls = Command.load_command_class(expected_builtin) 75 | self.assertIsNotNone(cmd_cls) 76 | self.assertEqual(cmd_cls.get_name(), expected_builtin) 77 | 78 | # Check that we can load the user command 79 | user_cmd_cls = Command.load_command_class("user-command-1") 80 | self.assertIsNotNone(user_cmd_cls) 81 | self.assertEqual(user_cmd_cls.get_name(), "user-command-1") 82 | 83 | # Check that load_command_class() returns None for junk commands 84 | unexpected_cmd = Command.load_command_class("no-such-command") 85 | self.assertIsNone(unexpected_cmd) 86 | 87 | def test_deprecated_aliases(self): 88 | aliases = [ 89 | ("measurement", "measurement-info"), 90 | ("measurements", "measurement-search"), 91 | ("probe", "probe-info"), 92 | ("probes", "probe-search"), 93 | ] 94 | 95 | # Check that each alias is loaded correctly and outputs a warning 96 | stderr = sys.stderr 97 | sys.stderr = StringIO() 98 | try: 99 | for alias, cmd_name in aliases: 100 | sys.stderr.truncate() 101 | cmd_cls = Command.load_command_class(alias) 102 | self.assertIn( 103 | "{} is a deprecated alias for {}".format(alias, cmd_name), 104 | sys.stderr.getvalue(), 105 | ) 106 | self.assertIsNotNone(cmd_cls) 107 | self.assertEqual(cmd_cls.get_name(), cmd_name) 108 | finally: 109 | sys.stderr = stderr 110 | -------------------------------------------------------------------------------- /ripe/atlas/tools/helpers/xdg.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | import platform 17 | import re 18 | import os 19 | import os.path 20 | 21 | 22 | def get_config_home(): 23 | """ """ 24 | config_home = os.environ.get("XDG_CONFIG_HOME") 25 | if config_home is None: 26 | config_home = os.path.expanduser("~/.config") 27 | return os.path.join(config_home, "ripe-atlas-tools") 28 | 29 | 30 | if hasattr(platform, "freedesktop_os_release"): 31 | freedesktop_os_release = platform.freedesktop_os_release 32 | else: 33 | # Shim for Python versions < 3.10, taken from CPython 34 | 35 | # freedesktop.org os-release standard 36 | # https://www.freedesktop.org/software/systemd/man/os-release.html 37 | 38 | # NAME=value with optional quotes (' or "). The regular expression is less 39 | # strict than shell lexer, but that's ok. 40 | _os_release_line = re.compile( 41 | "^(?P[a-zA-Z0-9_]+)=(?P[\"']?)(?P.*)(?P=quote)$" 42 | ) 43 | # unescape five special characters mentioned in the standard 44 | _os_release_unescape = re.compile(r"\\([\\\$\"\'`])") 45 | # /etc takes precedence over /usr/lib 46 | _os_release_candidates = ("/etc/os-release", "/usr/lib/os-release") 47 | _os_release_cache = None 48 | 49 | def _parse_os_release(lines): 50 | # These fields are mandatory fields with well-known defaults 51 | # in practice all Linux distributions override NAME, ID, and PRETTY_NAME. 52 | info = { 53 | "NAME": "Linux", 54 | "ID": "linux", 55 | "PRETTY_NAME": "Linux", 56 | } 57 | 58 | for line in lines: 59 | mo = _os_release_line.match(line) 60 | if mo is not None: 61 | info[mo.group("name")] = _os_release_unescape.sub( 62 | r"\1", mo.group("value") 63 | ) 64 | 65 | return info 66 | 67 | def freedesktop_os_release(): 68 | """Return operation system identification from freedesktop.org os-release""" 69 | global _os_release_cache 70 | 71 | if _os_release_cache is None: 72 | errno = None 73 | for candidate in _os_release_candidates: 74 | try: 75 | with open(candidate, encoding="utf-8") as f: 76 | _os_release_cache = _parse_os_release(f) 77 | break 78 | except OSError as e: 79 | errno = e.errno 80 | else: 81 | raise OSError( 82 | errno, f"Unable to read files {', '.join(_os_release_candidates)}" 83 | ) 84 | 85 | return _os_release_cache.copy() 86 | 87 | 88 | def get_os_string(): 89 | os = platform.system() 90 | 91 | if os == 'Darwin': 92 | release = platform.mac_ver()[0] 93 | return f'macOS {release}' 94 | elif os == 'Windows': 95 | release = platform.win32_ver()[0] 96 | return f"Windows {release}" 97 | pass 98 | else: 99 | try: 100 | info = freedesktop_os_release() 101 | except OSError: 102 | pass 103 | else: 104 | name = info.get("NAME") 105 | if name: 106 | version = info.get("VERSION_ID", "") 107 | return f"{name} {version}" 108 | return platform.platform() 109 | -------------------------------------------------------------------------------- /ripe/atlas/tools/renderers/dns_compact.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2017 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from tzlocal import get_localzone 17 | 18 | from ..helpers.colours import colourise 19 | from ..helpers.sanitisers import sanitise 20 | from .base import Renderer as BaseRenderer 21 | 22 | 23 | class Renderer(BaseRenderer): 24 | 25 | RENDERS = [BaseRenderer.TYPE_DNS] 26 | TIME_FORMAT = "%Y-%m-%d %H:%M:%S" 27 | ANSWER_COLORS = ["cyan", "blue"] 28 | 29 | def on_result(self, result): 30 | 31 | created = result.created.astimezone(get_localzone()) 32 | probe_id = result.probe_id 33 | 34 | r = [] 35 | if result.responses: 36 | for response in result.responses: 37 | r.append(self.get_formatted_response(probe_id, created, response)) 38 | else: 39 | r.append( 40 | "{}{}\n".format( 41 | self.get_header(probe_id, created), 42 | colourise("No response found", "red"), 43 | ) 44 | ) 45 | 46 | return "".join(r) 47 | 48 | @classmethod 49 | def get_header(cls, probe_id, created): 50 | return "Probe {0:>6}: {1} ".format( 51 | "#{}".format(probe_id), 52 | created.strftime(cls.TIME_FORMAT), 53 | ) 54 | 55 | @classmethod 56 | def get_formatted_response(cls, probe_id, created, response): 57 | 58 | s = [] 59 | answers = "" 60 | if response.abuf: 61 | header_flags = [] 62 | for flag in ( 63 | "aa", 64 | "ad", 65 | "cd", 66 | "qr", 67 | "ra", 68 | "rd", 69 | ): 70 | if getattr(response.abuf.header, flag): 71 | header_flags.append(flag) 72 | s.append(response.abuf.header.return_code) 73 | s.append(" ".join(header_flags)) 74 | answers = cls.print_answers(response.abuf.answers) 75 | else: 76 | s.append("No abuf found") 77 | 78 | if response.is_error or not response.abuf: 79 | color = "red" 80 | elif len(response.abuf.answers) == 0: 81 | color = "yellow" 82 | else: 83 | color = "green" 84 | 85 | status = colourise(" ".join(s), color) 86 | return "".join( 87 | [ 88 | cls.get_header(probe_id, created), 89 | status, 90 | " " if answers else "", 91 | answers, 92 | "\n", 93 | ] 94 | ) 95 | 96 | @staticmethod 97 | def get_rrdata(data): 98 | """ 99 | Return RRdata in condensed text form. 100 | """ 101 | if not data: 102 | return "" 103 | # It's too complicated to override __str__ method of all Answer 104 | # classes of Sagan so let's compress it text-wise instead 105 | r = str(data).split() 106 | r.pop(2) # get rid of the class 107 | return sanitise(" ".join(r)) 108 | 109 | @classmethod 110 | def print_answers(cls, data): 111 | """ 112 | Return list of colourised condensed textual RRdata. 113 | """ 114 | r = [] 115 | for record, color in zip(data, cls.ANSWER_COLORS * len(data)): 116 | r.append(colourise(cls.get_rrdata(record), color)) 117 | return "; ".join(r) 118 | -------------------------------------------------------------------------------- /ripe/atlas/tools/commands/measure/http.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from ...helpers.validators import ArgumentType 17 | from ...settings import conf 18 | 19 | from .base import Command 20 | 21 | 22 | class HttpMeasureCommand(Command): 23 | DESCRIPTION = "Create an HTTP measurement and wait for the results" 24 | 25 | def add_arguments(self): 26 | 27 | Command.add_arguments(self) 28 | 29 | self.add_primary_argument(name="target", parser=self.parser) 30 | 31 | spec = conf["specification"]["types"]["http"] 32 | 33 | specific = self.parser.add_argument_group("HTTP-specific Options") 34 | specific.add_argument( 35 | "--header-bytes", 36 | type=ArgumentType.integer_range(minimum=0, maximum=2048), 37 | default=spec["header-bytes"], 38 | help="The maximum number of bytes to retrieve from the header", 39 | ) 40 | specific.add_argument( 41 | "--version", 42 | type=str, 43 | default=spec["version"], 44 | help="The HTTP version to use", 45 | ) 46 | specific.add_argument( 47 | "--method", type=str, default=spec["method"], help="The HTTP method to use" 48 | ) 49 | specific.add_argument( 50 | "--port", 51 | type=ArgumentType.integer_range(minimum=1, maximum=65535), 52 | default=spec["port"], 53 | help="Destination port", 54 | ) 55 | specific.add_argument("--path", type=str, default=spec["path"], help="") 56 | specific.add_argument( 57 | "--query-string", type=str, default=spec["query-string"], help="" 58 | ) 59 | specific.add_argument( 60 | "--user-agent", 61 | type=str, 62 | default=spec["user-agent"], 63 | help="The user agent used when performing the request", 64 | ) 65 | specific.add_argument( 66 | "--body-bytes", 67 | type=ArgumentType.integer_range(minimum=1, maximum=1020048), 68 | default=spec["body-bytes"], 69 | help="The maximum number of bytes to retrieve from the body", 70 | ) 71 | specific.add_argument( 72 | "--timing-verbosity", 73 | type=int, 74 | choices=(0, 1, 2), 75 | default=spec["timing-verbosity"], 76 | help="The amount of timing information you want returned. 1 " 77 | "returns the time to read, to connect, and to first byte, 2 " 78 | "returns timing information per read system call. 0 " 79 | "(default) returns no additional timing information.", 80 | ) 81 | 82 | def _get_measurement_kwargs(self): 83 | 84 | r = Command._get_measurement_kwargs(self) 85 | 86 | keys = ( 87 | "header_bytes", 88 | "version", 89 | "method", 90 | "port", 91 | "path", 92 | "query_string", 93 | "user_agent", 94 | ) 95 | for key in keys: 96 | r[key] = getattr(self.arguments, key) 97 | 98 | if self.arguments.timing_verbosity > 0: 99 | r["extended_timing"] = True 100 | if self.arguments.timing_verbosity > 1: 101 | r["more_extended_timing"] = True 102 | 103 | r["max_bytes_read"] = self.arguments.body_bytes 104 | 105 | return r 106 | -------------------------------------------------------------------------------- /ripe/atlas/tools/renderers/ssl_consistency.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from ..helpers.sanitisers import sanitise 17 | from .base import Renderer as BaseRenderer 18 | 19 | THRESHOLD = 80 # % 20 | 21 | 22 | class Renderer(BaseRenderer): 23 | 24 | RENDERS = [BaseRenderer.TYPE_SSLCERT] 25 | 26 | def __init__(self, *args, **kwargs): 27 | BaseRenderer.__init__(self, *args, **kwargs) 28 | self.uniqcerts = {} 29 | self.blob_list = [] 30 | 31 | def footer(self): 32 | most_seen_cert = self.get_nprobes_ofpopular_cert() 33 | for cert_id in sorted( 34 | self.uniqcerts, key=lambda pk: self.uniqcerts[pk]["cnt"], reverse=True 35 | ): 36 | self.blob_list.append(self.render_certificate(cert_id)) 37 | if self.uniqcerts[cert_id]["cnt"] < most_seen_cert * THRESHOLD / 100: 38 | self.blob_list.extend(self.render_below_threshold(cert_id)) 39 | 40 | return "\n".join(self.blob_list) 41 | 42 | def bucketize_result_cert(self, result): 43 | for certificate in result.certificates: 44 | cert_id = certificate.checksum_sha256 45 | if cert_id not in self.uniqcerts: 46 | self.uniqcerts[cert_id] = {"cert": None, "cnt": 0, "probes": []} 47 | self.uniqcerts[cert_id]["cert"] = certificate 48 | self.uniqcerts[cert_id]["cnt"] += 1 49 | self.uniqcerts[cert_id]["probes"].append(result.probe) 50 | 51 | def get_nprobes_ofpopular_cert(self): 52 | """ 53 | Gets the number of probes that have seen the most popular 54 | (in terms of probes) cert. 55 | """ 56 | if not self.uniqcerts: 57 | return 0 58 | return max([self.uniqcerts[cert_id]["cnt"] for cert_id in self.uniqcerts]) 59 | 60 | def render_certificate(self, cert_id): 61 | """Renders the specific certificate""" 62 | certificate = self.uniqcerts[cert_id]["cert"] 63 | 64 | return self.render_template( 65 | "reports/ssl_consistency.txt", 66 | issuer_c=sanitise(certificate.issuer_c), 67 | issuer_o=sanitise(certificate.issuer_o), 68 | issuer_cn=sanitise(certificate.issuer_cn), 69 | subject_c=sanitise(certificate.subject_c), 70 | subject_o=sanitise(certificate.subject_o), 71 | subject_cn=sanitise(certificate.subject_cn), 72 | sha256fp=certificate.checksum_sha256, 73 | seenby=self.uniqcerts[cert_id]["cnt"], 74 | s="s" if self.uniqcerts[cert_id]["cnt"] > 1 else "", 75 | ) 76 | 77 | def render_below_threshold(self, cert_id): 78 | """ 79 | Print information about the given cert that is below our threshold 80 | of visibility. 81 | """ 82 | blob_list = [ 83 | " Below the threshold ({0}%)".format(THRESHOLD), 84 | " Probes that saw it: ", 85 | ] 86 | 87 | for probe in self.uniqcerts[cert_id]["probes"]: 88 | log = ( 89 | " ID: {id}, country code: {cc}, ASN (v4/v6): {asn4}/{asn6}" 90 | ).format( 91 | id=probe.id, cc=probe.country_code, asn4=probe.asn_v4, asn6=probe.asn_v6 92 | ) 93 | blob_list.append(log) 94 | 95 | return blob_list 96 | 97 | def on_result(self, result): 98 | self.bucketize_result_cert(result) 99 | return "" 100 | -------------------------------------------------------------------------------- /ripe/atlas/tools/renderers/traceroute_aspath.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from ..ipdetails import IP 17 | from .base import Renderer as BaseRenderer 18 | 19 | 20 | class Renderer(BaseRenderer): 21 | 22 | RENDERS = [BaseRenderer.TYPE_TRACEROUTE] 23 | 24 | DEFAULT_RADIUS = 2 25 | 26 | @staticmethod 27 | def add_arguments(parser): 28 | group = parser.add_argument_group( 29 | title="Optional arguments for traceroute_aspath renderer" 30 | ) 31 | group.add_argument( 32 | "--traceroute-aspath-radius", 33 | type=int, 34 | help="Number of different ASs starting from the end of the " 35 | "traceroute path. " 36 | "Default: {}.".format(Renderer.DEFAULT_RADIUS), 37 | metavar="RADIUS", 38 | default=Renderer.DEFAULT_RADIUS, 39 | ) 40 | 41 | def __init__(self, *args, **kwargs): 42 | BaseRenderer.__init__(self, *args, **kwargs) 43 | self.paths = {} 44 | 45 | # Number of different ASs starting from the end of the traceroute path. 46 | if "arguments" in kwargs: 47 | self.RADIUS = kwargs["arguments"].traceroute_aspath_radius 48 | else: 49 | self.RADIUS = Renderer.DEFAULT_RADIUS 50 | 51 | @staticmethod 52 | def _get_asns_for_output(asns, radius): 53 | asns_with_padding = [""] * radius + asns 54 | asns_with_padding = asns_with_padding[-radius:] 55 | return " ".join( 56 | [ 57 | "{:>8}".format("AS{}".format(asn) if asn else "") 58 | for asn in asns_with_padding 59 | ] 60 | ) 61 | 62 | def header(self, sample): 63 | return ( 64 | "For each traceroute path toward the target, the " 65 | "last {} ASNs will be shown\n\n".format(self.RADIUS) 66 | ) 67 | 68 | def on_result(self, result): 69 | 70 | ip_hops = [] 71 | 72 | for hop in result.hops: 73 | for packet in hop.packets: 74 | if packet.origin: 75 | ip_hops.append(packet.origin) 76 | break 77 | 78 | asns = [] 79 | 80 | # starting from the last hop's IP, get up to ASNs 81 | for address in reversed(ip_hops): 82 | ip = IP(address) 83 | if ip.asn and ip.asn not in asns: 84 | asns.append(ip.asn) 85 | if len(asns) == self.RADIUS: 86 | break 87 | 88 | as_path = self._get_asns_for_output(list(reversed(asns)), self.RADIUS) 89 | 90 | if as_path not in self.paths: 91 | self.paths[as_path] = {} 92 | self.paths[as_path]["cnt"] = 0 93 | self.paths[as_path]["responded"] = 0 94 | self.paths[as_path]["cnt"] += 1 95 | if result.destination_ip_responded: 96 | self.paths[as_path]["responded"] += 1 97 | 98 | return "Probe #{:<5}: {}, {}completed\n".format( 99 | result.probe_id, 100 | as_path, 101 | "NOT " if not result.destination_ip_responded else "", 102 | ) 103 | 104 | def footer(self): 105 | s = "\nNumber of probes for each AS path:\n\n" 106 | 107 | for as_path in self.paths: 108 | s += " {}: {} probe{}, {} completed\n".format( 109 | as_path, 110 | self.paths[as_path]["cnt"], 111 | "s" if self.paths[as_path]["cnt"] > 1 else "", 112 | self.paths[as_path]["responded"], 113 | ) 114 | 115 | return s 116 | -------------------------------------------------------------------------------- /ripe/atlas/tools/ipdetails.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | import requests 17 | import IPy 18 | 19 | from .cache import cache 20 | 21 | 22 | class IP(object): 23 | 24 | RIPESTAT_URL = "https://stat.ripe.net/data/prefix-overview/data.json?resource={ip}" 25 | CACHE_EXPIRATION_TIME = 60 * 60 * 24 * 7 26 | 27 | def __init__(self, address): 28 | self.cached_prefix_found = False 29 | self.ip_object = IPy.IP(address) 30 | 31 | self.address = self.ip_object.strFullsize() 32 | self.asn = None 33 | self.holder = None 34 | self.prefix = None 35 | self.not_querable_types = [ 36 | "RESERVED", 37 | "UNSPECIFIED", 38 | "LOOPBACK", 39 | "UNASSIGNED", 40 | "DOCUMENTATION", 41 | "ULA", 42 | "LINKLOCAL", 43 | "PRIVATE", 44 | ] 45 | 46 | details = self._get_details() 47 | if details: 48 | self.asn = details["ASN"] 49 | self.holder = details["Holder"] 50 | self.prefix = details["Prefix"] 51 | 52 | def _get_details(self): 53 | details = None 54 | 55 | if not self.is_querable(): 56 | return details 57 | 58 | details = cache.get("IPDetails:{}".format(self.address)) 59 | if details: 60 | return details 61 | 62 | details = self.get_from_cached_prefix() 63 | 64 | if not details: 65 | details = self.query_stat() 66 | 67 | if details: 68 | self.update_cache(details) 69 | 70 | return details 71 | 72 | def is_querable(self): 73 | """Determines if address is worth querable.""" 74 | return self.ip_object.iptype() not in self.not_querable_types 75 | 76 | def get_from_cached_prefix(self): 77 | """Search cache for existing cached Prefix""" 78 | details = None 79 | 80 | for cache_entry in cache.keys(): 81 | if not cache_entry.decode().startswith("IPDetailsPrefix:"): 82 | continue 83 | 84 | prefix_details = cache.get(cache_entry) 85 | 86 | # data could exist but expired 87 | if not prefix_details: 88 | continue 89 | 90 | prefix = IPy.IP(prefix_details["Prefix"]) 91 | 92 | if self.ip_object in prefix: 93 | details = prefix_details 94 | self.cached_prefix_found = True 95 | break 96 | 97 | return details 98 | 99 | def query_stat(self): 100 | """Query RIPE Stat to get address details.""" 101 | URL = self.RIPESTAT_URL.format(ip=self.address) 102 | details = {} 103 | 104 | try: 105 | response = requests.get(URL) 106 | if not response.ok: 107 | return details 108 | res = response.json() 109 | except (requests.exceptions.RequestException, ValueError): 110 | # Catch all requests exception + not valid json ones. 111 | return details 112 | 113 | if res.get("status") == "ok": 114 | try: 115 | details = { 116 | "ASN": str(res["data"]["asns"][0]["asn"]), 117 | "Holder": res["data"]["asns"][0]["holder"], 118 | "Prefix": res["data"]["resource"], 119 | } 120 | except ( 121 | # Protect from any kind of malformed json response 122 | AttributeError, 123 | ValueError, 124 | KeyError, 125 | IndexError, 126 | TypeError, 127 | ): 128 | pass 129 | 130 | return details 131 | 132 | def update_cache(self, details): 133 | """Update cache for the address and prefix if needed.""" 134 | if not self.cached_prefix_found: 135 | key = "IPDetailsPrefix:{}".format(details["Prefix"]) 136 | cache.set(key, details, self.CACHE_EXPIRATION_TIME) 137 | 138 | key = "IPDetails:{}".format(self.address) 139 | cache.set(key, details, self.CACHE_EXPIRATION_TIME) 140 | 141 | def __str__(self): 142 | return "IP {}, ASN {}, Holder {}".format(self.address, self.asn, self.holder) 143 | -------------------------------------------------------------------------------- /ripe/atlas/tools/renderers/dns.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from tzlocal import get_localzone 17 | 18 | from ..helpers.colours import colourise 19 | from ..helpers.sanitisers import sanitise 20 | from .base import Renderer as BaseRenderer 21 | 22 | 23 | class Renderer(BaseRenderer): 24 | 25 | RENDERS = [BaseRenderer.TYPE_DNS] 26 | TIME_FORMAT = "%a %b %d %H:%M:%S %Z %Y" 27 | 28 | def on_result(self, result): 29 | 30 | created = result.created.astimezone(get_localzone()) 31 | probe_id = result.probe_id 32 | 33 | r = "\n\nProbe #{0}\n{1}\n".format(probe_id, "=" * 79) 34 | if result.responses: 35 | for response in result.responses: 36 | r += self.get_formatted_response(probe_id, created, response) 37 | else: 38 | r += "\n {}\n".format(colourise("No response found", "red")) 39 | 40 | return r 41 | 42 | @classmethod 43 | def get_formatted_response(cls, probe_id, created, response): 44 | 45 | if not response.abuf: 46 | return "\n- {0} -\n\n No abuf found.\n".format(response.response_id) 47 | 48 | header_flags = [] 49 | for flag in ( 50 | "aa", 51 | "ad", 52 | "cd", 53 | "qr", 54 | "ra", 55 | "rd", 56 | ): 57 | if getattr(response.abuf.header, flag): 58 | header_flags.append(flag) 59 | 60 | edns = "" 61 | if response.abuf.edns0: 62 | edns = ( 63 | "\n ;; OPT PSEUDOSECTION:\n ; EDNS: version: {0}, " 64 | "flags:; udp: {1}\n".format( 65 | response.abuf.edns0.version, response.abuf.edns0.udp_size 66 | ) 67 | ) 68 | 69 | question = "" 70 | if response.abuf.questions: 71 | question = response.abuf.questions[0].name 72 | 73 | return cls._colourise_by_response( 74 | response, 75 | cls.render_template( 76 | "reports/dns.txt", 77 | # Older measurements don't have a response_id 78 | response_id=response.response_id or 1, 79 | probe=probe_id, 80 | question_name=sanitise(question), 81 | header_opcode=response.abuf.header.opcode, 82 | header_return_code=response.abuf.header.return_code, 83 | header_id=response.abuf.header.id, 84 | header_flags=" ".join(header_flags), 85 | edns=edns, 86 | question_count=len(response.abuf.questions), 87 | answer_count=len(response.abuf.answers), 88 | authority_count=len(response.abuf.authorities), 89 | additional_count=len(response.abuf.additionals), 90 | question=sanitise( 91 | cls.get_section("question", response.abuf.questions), 92 | strip_newlines=False, 93 | ), 94 | answers=sanitise( 95 | cls.get_section("answer", response.abuf.answers), 96 | strip_newlines=False, 97 | ), 98 | authorities=sanitise( 99 | cls.get_section("authority", response.abuf.authorities), 100 | strip_newlines=False, 101 | ), 102 | additionals=sanitise( 103 | cls.get_section("additional", response.abuf.additionals), 104 | strip_newlines=False, 105 | ), 106 | response_time=response.response_time, 107 | response_size=response.response_size, 108 | created=created.strftime(cls.TIME_FORMAT), 109 | destination_address=sanitise(response.destination_address), 110 | ), 111 | ) 112 | 113 | @staticmethod 114 | def get_section(header, data): 115 | 116 | if not data: 117 | return "" 118 | 119 | return "\n ;; {0} SECTION:\n{1}\n".format( 120 | header.upper(), "\n".join([" {0}".format(_) for _ in data]) 121 | ) 122 | 123 | @staticmethod 124 | def _colourise_by_response(response, output): 125 | colour = "red" if response.is_error else "green" 126 | return colourise(output, colour) 127 | -------------------------------------------------------------------------------- /tests/renderers/test_dns_compact.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2017 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | import unittest 17 | 18 | from unittest import mock 19 | 20 | from pytz import timezone 21 | from ripe.atlas.sagan import Result 22 | from ripe.atlas.tools.renderers.dns_compact import Renderer 23 | 24 | 25 | def get_fake_localzone(): 26 | return timezone("UTC") 27 | 28 | 29 | @mock.patch("ripe.atlas.tools.renderers.dns_compact.get_localzone", get_fake_localzone) 30 | class TestDnsCompact(unittest.TestCase): 31 | def __init__(self, *args, **kwargs): 32 | unittest.TestCase.__init__(self, *args, **kwargs) 33 | self.basic = Result.get( 34 | '{"lts":92,"from":"195.113.83.16","msm_id":9211416,"fw":4790,"timestamp":1503927916,"resultset":[{"lts":92,"src_addr":"195.113.83.16","af":4,"submax":3,"proto":"UDP","subid":1,"result":{"abuf":"fViBgAABAAIAAgAEBTF4YmV0A2NvbQAAAQABwAwAAQABAAAAGQAEvmnCOsAMAAEAAQAAABkABL550oXADAACAAEAAVUJABUEZGFuYQJucwpjbG91ZGZsYXJlwBLADAACAAEAAVUJAAcEcGV0ZcBMwEcAAQABAAFVCQAErfU6acBHABwAAQABVQkAECQAywAgSQABAAAAAK31OmnAaAABAAEAAVUJAASt9TuIwGgAHAABAAFVCQAQJADLACBJAAEAAAAArfU7iA==","rt":6.45,"NSCOUNT":2,"QDCOUNT":1,"ANCOUNT":2,"ARCOUNT":4,"ID":32088,"size":199},"time":1503927916,"dst_addr":"195.113.83.55"},{"lts":93,"src_addr":"195.113.83.16","af":4,"submax":3,"proto":"UDP","subid":2,"result":{"abuf":"D/uBBQABAAAAAAAABTF4YmV0A2NvbQAAAQAB","rt":5.798,"NSCOUNT":0,"QDCOUNT":1,"ANCOUNT":0,"ARCOUNT":0,"ID":4091,"size":27},"time":1503927917,"dst_addr":"147.231.12.1"},{"lts":94,"src_addr":"2001:718:1e06::16","af":6,"submax":3,"proto":"UDP","subid":3,"result":{"abuf":"pqqBgAABAAIAAgAEBTF4YmV0A2NvbQAAAQABwAwAAQABAAAAFwAEvnnShcAMAAEAAQAAABcABL5pwjrADAACAAEAAVUHABUEcGV0ZQJucwpjbG91ZGZsYXJlwBLADAACAAEAAVUHAAcEZGFuYcBMwGgAAQABAAFVBwAErfU6acBoABwAAQABVQcAECQAywAgSQABAAAAAK31OmnARwABAAEAAVUHAASt9TuIwEcAHAABAAFVBwAQJADLACBJAAEAAAAArfU7iA==","rt":5.684,"NSCOUNT":2,"QDCOUNT":1,"ANCOUNT":2,"ARCOUNT":4,"ID":42666,"size":199},"time":1503927918,"dst_addr":"2001:718:1e06::55"}],"prb_id":4062,"group_id":9211416,"type":"dns","msm_name":"Tdig"}' # noqa: E501 35 | ) 36 | self.noerrornodata = Result.get( 37 | '{"lts":42,"from":"2001:718:1:a100::161:50","msm_id":9386425,"fw":4780,"proto":"UDP","af":6,"msm_name":"Tdig","prb_id":6068,"result":{"abuf":"8RCAgAABAAAAAQAACGlwdjRvbmx5BGFycGEAABwAAcAMAAYAAQAABikALQNzbnMDZG5zBWljYW5uA29yZwADbm9jwC94Ob7xAAAcIAAADhAACTqAAAAOEA==","rt":153.425,"NSCOUNT":1,"QDCOUNT":1,"ID":61712,"ARCOUNT":0,"ANCOUNT":0,"size":88},"timestamp":1506557387,"src_addr":"2001:718:1:a100::161:50","group_id":9386425,"type":"dns","dst_addr":"2001:4860:4860::6464"}' # noqa: E501 38 | ) 39 | self.noresponse = Result.get( 40 | '{"lts":11,"from":"2a01:538:1:f000:fa1a:67ff:fe4d:7f1d","msm_id":9386425,"fw":4780,"timestamp":1506681497,"proto":"UDP","msm_name":"Tdig","prb_id":11879,"af":6,"error":{"timeout":5000},"src_addr":"2a01:538:1:f000:fa1a:67ff:fe4d:7f1d","group_id":9386425,"type":"dns","dst_addr":"2001:4860:4860::6464"}' # noqa: E501 41 | ) 42 | self.noabuf = Result.get( 43 | '{"lts":27,"from":"80.92.240.37","msm_id":9211416,"fw":4780,"timestamp":1503927938,"resultset":[{"lts":27,"src_addr":"192.168.254.254","af":4,"submax":2,"proto":"UDP","subid":1,"time":1503927938,"error":{"timeout":5000},"dst_addr":"80.92.240.6"}],"prb_id":30410,"group_id":9211416,"type":"dns","msm_name":"Tdig"}' # noqa: E501 44 | ) 45 | 46 | def test_basic(self): 47 | self.assertEqual( 48 | Renderer().on_result(self.basic), 49 | "Probe #4062: 2017-08-28 13:45:16 NOERROR qr ra rd 1xbet.com. 25 A 190.105.194.58; 1xbet.com. 25 A 190.121.210.133\n" # noqa: E501 50 | "Probe #4062: 2017-08-28 13:45:16 REFUSED qr rd\n" 51 | "Probe #4062: 2017-08-28 13:45:16 NOERROR qr ra rd 1xbet.com. 23 A 190.121.210.133; 1xbet.com. 23 A 190.105.194.58\n", # noqa: E501 52 | ) 53 | 54 | def test_noerrornodata(self): 55 | self.assertEqual( 56 | Renderer().on_result(self.noerrornodata), 57 | "Probe #6068: 2017-09-28 00:09:47 NOERROR qr ra\n", 58 | ) 59 | 60 | def test_noresponse(self): 61 | self.assertEqual( 62 | Renderer().on_result(self.noresponse), 63 | "Probe #11879: 2017-09-29 10:38:17 No response found\n", 64 | ) 65 | 66 | def test_noabuf(self): 67 | self.assertEqual( 68 | Renderer().on_result(self.noabuf), 69 | "Probe #30410: 2017-08-28 13:45:38 No abuf found\n", 70 | ) 71 | -------------------------------------------------------------------------------- /ripe/atlas/tools/renderers/ping.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from ..helpers.sanitisers import sanitise 17 | from .base import Renderer as BaseRenderer 18 | 19 | 20 | class Renderer(BaseRenderer): 21 | """ 22 | This is meant to be a stub example for what an aggregate renderer might look 23 | like. If you have ideas as to how to make this better, feel free to send 24 | along a pull request. 25 | """ 26 | 27 | RENDERS = [BaseRenderer.TYPE_PING] 28 | 29 | def __init__(self, **kwargs): 30 | BaseRenderer.__init__(self, **kwargs) 31 | 32 | self.target = "" 33 | self.packet_loss = 0 34 | self.sent_packets = 0 35 | self.received_packets = 0 36 | self.rtts = [] 37 | self.rtts_min = [] 38 | self.rtts_max = [] 39 | self.rtt_types_map = {"min": self.rtts_min, "max": self.rtts_max} 40 | 41 | def collect_stats(self, result): 42 | """ 43 | Calculates, stores and collects all stats we want from the given 44 | result. 45 | """ 46 | if not self.target: 47 | self.target = result.destination_name 48 | self.sent_packets += result.packets_sent 49 | self.received_packets += result.packets_received 50 | self.collect_min_max_rtts("min", result.rtt_min) 51 | self.collect_min_max_rtts("max", result.rtt_max) 52 | 53 | self.collect_packets_rtt(result.packets) 54 | 55 | def collect_min_max_rtts(self, rtt_type, rtt): 56 | """ 57 | Stores the given rtt in the corresponding list (min/max) if rtt is set. 58 | """ 59 | rtt = rtt 60 | if not rtt: 61 | rtt = 0 62 | 63 | self.rtt_types_map[rtt_type].append(rtt) 64 | 65 | def collect_packets_rtt(self, packets): 66 | """ 67 | Collects all the rrts of given packets and stores them 68 | in our rtts list. 69 | """ 70 | for packet in packets: 71 | rtt = packet.rtt 72 | if not packet.rtt: 73 | rtt = 0 74 | self.rtts.append(rtt) 75 | 76 | def calculate_loss(self): 77 | """Calculates the total loss between received and sent packets.""" 78 | if not self.sent_packets: 79 | return 0 80 | 81 | return (1 - float(self.received_packets) / self.sent_packets) * 100 82 | 83 | def mean(self): 84 | """Calculates the mean of the collected rtts""" 85 | return round(float(sum(self.rtts)) / max(len(self.rtts), 1), 3) 86 | 87 | def median(self): 88 | """Calculates the median of the collected rtts""" 89 | sorted_rtts = sorted(self.rtts) 90 | index = (len(self.rtts) - 1) // 2 91 | if len(self.rtts) % 2: 92 | return sorted_rtts[index] 93 | else: 94 | return (sorted_rtts[index] + sorted_rtts[index + 1]) / 2.0 95 | 96 | def on_result(self, result): 97 | packets = result.packets 98 | 99 | if not packets: 100 | return "No packets found\n" 101 | 102 | self.collect_stats(result) 103 | 104 | # Because the origin value is more reliable as "from" in v4 and as 105 | # "packet.source_address" in v6. 106 | origin = result.origin 107 | if ":" in origin: 108 | origin = packets[0].source_address 109 | 110 | times = ", ".join([str(_.rtt) + " ms" for _ in packets]) 111 | 112 | return ( 113 | f"{result.packet_size} bytes from {result.destination_address} via " 114 | f"probe #{result.probe_id} ({origin})" 115 | f": ttl={packets[0].ttl} times={times}\n" 116 | ) 117 | 118 | def header(self, sample): 119 | resolved_on = ( 120 | "server" 121 | if sample.destination_address == sample.destination_name 122 | else "probe" 123 | ) 124 | return f"PING {sample.destination_name} (resolved on {resolved_on})\n" 125 | 126 | def footer(self): 127 | if not self.sent_packets: 128 | return "" 129 | self.packet_loss = self.calculate_loss() 130 | return self.render_template( 131 | "reports/aggregate_ping.txt", 132 | target=sanitise(self.target), 133 | sent=self.sent_packets, 134 | received=self.received_packets, 135 | packet_loss=self.packet_loss, 136 | min=min(self.rtts_min), 137 | median=self.median(), 138 | mean=self.mean(), 139 | max=max(self.rtts_max), 140 | ) 141 | -------------------------------------------------------------------------------- /ripe/atlas/tools/cache.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | import datetime 17 | import functools 18 | import os 19 | import sys 20 | 21 | try: 22 | import cPickle as pickle 23 | except ImportError: 24 | import pickle 25 | 26 | try: 27 | import anydbm as dbm # anydbm py2.7 will use the best available dbm 28 | except ImportError: 29 | import dbm # ... and on Python3 dbm does the same 30 | 31 | from .helpers import xdg 32 | 33 | 34 | class LocalCache(object): 35 | """ 36 | Simple caching engine, making use of the built-in dbm support. This will 37 | create a file called cache.db in ripe-atlas-tools config directory and dump 38 | stuff in there for use later. 39 | """ 40 | 41 | def __init__(self): 42 | self._now = datetime.datetime.now() 43 | self._db_file = None 44 | 45 | @property 46 | def _db(self): 47 | if self._db_file is None: 48 | self._db_file = dbm.open(self._get_or_create_db_path(), "c") 49 | return self._db_file 50 | 51 | def __contains__(self, key): 52 | return key in self._db 53 | 54 | def __getitem__(self, key): 55 | return self.get(key) 56 | 57 | def __setitem__(self, key, value, expires=None): 58 | self._db[key] = pickle.dumps((expires, value)) 59 | 60 | def __delitem__(self, key): 61 | if key not in self._db: 62 | raise KeyError 63 | del self._db[key] 64 | 65 | def keys(self): 66 | return self._db.keys() 67 | 68 | def items(self): 69 | for key in self.keys(): 70 | yield key, self._db[key] 71 | 72 | def get(self, key, default=None): 73 | if key in self._db: 74 | expires, value = pickle.loads(self._db[key]) 75 | if not expires or expires > self._now: 76 | return value 77 | else: 78 | del self._db[key] 79 | return default 80 | 81 | def set(self, key, value, expires=None): 82 | return self.__setitem__( 83 | key, value, self._now + datetime.timedelta(seconds=expires) 84 | ) 85 | 86 | def clear(self, key=None): 87 | """ 88 | Removes a specific key from the cache manually, or will wipe the entire 89 | cache if you don't specify `key`. Note that this shouldn't be necessary 90 | unless you've cached something with an inappropriately long expire time. 91 | """ 92 | if key: 93 | if key in self._db: 94 | del self._db[key] 95 | else: 96 | for key in self.keys(): 97 | del self._db[key] 98 | 99 | def expire(self): 100 | """ 101 | Clears out should-be-expired values from the cache. Note that this 102 | happens automatically whenever you call `.get()` so you should never 103 | really need to run this. 104 | """ 105 | for key in self.keys(): 106 | self.get(key) 107 | 108 | @staticmethod 109 | def _get_or_create_db_path(): 110 | 111 | v = sys.version_info 112 | file_name = "cache-{}.{}.{}.db".format(v.major, v.minor, v.micro) 113 | 114 | db_path = os.path.join("/", "tmp", file_name) 115 | if "HOME" in os.environ: 116 | db_path = os.path.join(xdg.get_config_home(), file_name) 117 | 118 | try: 119 | os.makedirs(os.path.dirname(db_path)) 120 | except OSError: 121 | pass # Better to ask forgiveness than permission 122 | 123 | return db_path 124 | 125 | 126 | cache = LocalCache() 127 | 128 | 129 | class Memoiser(object): 130 | """ 131 | Enabling class for the @memoised decorator 132 | """ 133 | 134 | def __init__(self, function, cache_time): 135 | self._function = function 136 | self._cache_time = cache_time 137 | 138 | def __call__(self, *args, **kwargs): 139 | 140 | key = pickle.dumps([args, kwargs]) 141 | value = cache[key] 142 | 143 | if value: 144 | return value 145 | 146 | value = self._function(*args, **kwargs) 147 | cache.set(key, value, self._cache_time) 148 | 149 | return value 150 | 151 | def __get__(self, obj, objtype): 152 | """Support instance methods.""" 153 | return functools.partial(self.__call__, obj) 154 | 155 | 156 | def memoised(cache_time): 157 | """ 158 | Decorate a method or function with this to cache the result of said method 159 | for n seconds: 160 | 161 | @memoised(60 * 60) # Cache for one hour 162 | my_function(some_arguments): 163 | ... 164 | return whatever 165 | """ 166 | 167 | def _wrap(function): 168 | return Memoiser(function, cache_time=cache_time) 169 | 170 | return _wrap 171 | -------------------------------------------------------------------------------- /scripts/ripe-atlas: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import os 4 | import re 5 | import sys 6 | 7 | from ripe.atlas.tools.commands.base import Command, Factory 8 | from ripe.atlas.tools.commands.measure import Factory as BaseFactory 9 | from ripe.atlas.tools.exceptions import RipeAtlasToolsException 10 | 11 | 12 | class RipeAtlas(object): 13 | 14 | def __init__(self): 15 | self.command = None 16 | self.args = [] 17 | self.kwargs = {} 18 | 19 | def _generate_usage(self): 20 | usage = "Usage: ripe-atlas [arguments]\n\n" 21 | usage += "Commands:\n" 22 | longest_command = 0 23 | classes = [] 24 | for c in Command.get_available_commands(): 25 | if c == "shibboleet": 26 | continue 27 | cmd_class = Command.load_command_class(c) 28 | classes.append(cmd_class) 29 | cmd_name = cmd_class.get_name() 30 | if len(cmd_name) > longest_command: 31 | longest_command = len(cmd_name) 32 | for cmd_cls in classes: 33 | usage += "\t{} {}\n".format( 34 | cmd_cls.get_name().ljust(longest_command + 1), 35 | cmd_cls.DESCRIPTION, 36 | ) 37 | usage += ( 38 | "\nFor help on a particular command, try " 39 | "ripe-atlas --help" 40 | ) 41 | return usage 42 | 43 | def _set_base_command(self): 44 | """ 45 | Sets the base command covering cases where we call it with 46 | shortcut or asking for help. 47 | """ 48 | caller = os.path.basename(sys.argv[0]) 49 | shortcut = re.match('^a(ping|traceroute|dig|sslcert|ntp|http)$', caller) 50 | 51 | if shortcut: 52 | self.command = "measure" 53 | sys.argv.insert(1, self._translate_shortcut(shortcut.group(1))) 54 | return 55 | 56 | if len(sys.argv) < 2 or sys.argv[1] in ("-h", "--help"): 57 | self.command = "help" 58 | return 59 | 60 | self.command = sys.argv.pop(1) 61 | 62 | @staticmethod 63 | def _translate_shortcut(shortcut): 64 | if shortcut == "dig": 65 | return "dns" 66 | return shortcut 67 | 68 | def autocomplete(self): 69 | """ 70 | This function is highly inspired from Django's own autocomplete 71 | manage.py. For more documentation check 72 | https://github.com/django/django/blob/1.9.4/django/core/management/__init__.py#L198-L270 73 | """ 74 | 75 | def print_options(options, curr): 76 | """ 77 | Prints matching with current word available autocomplete options 78 | in a formatted way to look good on bash. 79 | """ 80 | sys.stdout.write(' '.join(sorted(filter(lambda x: x.startswith(curr), options)))) 81 | 82 | # If we are not autocompleting continue as normal 83 | if 'RIPE_ATLAS_AUTO_COMPLETE' not in os.environ: 84 | return 85 | 86 | cwords = os.environ['COMP_WORDS'].split()[1:] 87 | cword = int(os.environ['COMP_CWORD']) 88 | 89 | try: 90 | curr = cwords[cword - 1] 91 | except IndexError: 92 | curr = '' 93 | 94 | commands = list(Command.get_available_commands()) 95 | 96 | # base caller ripe-atlas 97 | if cword == 1: 98 | print_options(commands, curr) 99 | # special measure command 100 | elif cword == 2 and cwords[0] == "measure": 101 | print_options(BaseFactory.TYPES.keys(), curr) 102 | # rest of commands 103 | elif cwords[0] in commands: 104 | cmd = self.fetch_command_class(cwords[0], cwords) 105 | cmd.add_arguments() 106 | options = [sorted(s_opt.option_strings)[0] for s_opt in cmd.parser._actions if s_opt.option_strings] 107 | previous_options = [x for x in cwords[1:cword - 1]] 108 | options = [opt for opt in options if opt not in previous_options] 109 | print_options(options, curr) 110 | 111 | sys.exit(1) 112 | 113 | def fetch_command_class(self, command, arg_options): 114 | """Fetches the class responsible for the given command.""" 115 | 116 | cmd_cls = Command.load_command_class(command) 117 | 118 | if cmd_cls is None: 119 | # Module containing the command class wasn't found 120 | raise RipeAtlasToolsException("No such command") 121 | 122 | # 123 | # If the imported module contains a `Factory` class, execute that 124 | # to get the `cmd` we're going to use. Otherwise, we expect there 125 | # to be a `Command` class in there. 126 | # 127 | 128 | if issubclass(cmd_cls, Factory): 129 | cmd = cmd_cls(arg_options).create() 130 | else: 131 | cmd = cmd_cls(*self.args, **self.kwargs) 132 | 133 | return cmd 134 | 135 | def main(self): 136 | 137 | self._set_base_command() 138 | 139 | self.autocomplete() 140 | 141 | if self.command == "help": 142 | raise RipeAtlasToolsException(self._generate_usage()) 143 | 144 | cmd = self.fetch_command_class(self.command, sys.argv) 145 | cmd.init_args() 146 | cmd.run() 147 | 148 | 149 | if __name__ == '__main__': 150 | try: 151 | sys.exit(RipeAtlas().main()) 152 | except RipeAtlasToolsException as e: 153 | e.write() 154 | raise SystemExit() 155 | -------------------------------------------------------------------------------- /dev-scripts/compare-openapi-spec.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | """ 3 | Compare the CLI measure command options with the fields, limits and defaults 4 | from the API via a call to the OpenAPI (swagger) URL. 5 | """ 6 | from __future__ import print_function 7 | 8 | import requests 9 | 10 | from ripe.atlas.tools.commands.base import Command 11 | 12 | openapi_url = "https://atlas.ripe.net/docs/api/v2/reference/api-docs/api/v2/measurements" 13 | 14 | types = [ 15 | ("ping", "Writeping measurement"), 16 | ("traceroute", "Writetraceroute measurement"), 17 | ("dns", "WriteDNS measurement"), 18 | ("sslcert", "SSL cert measurement"), 19 | ("http", "HTTP measurement"), 20 | ("ntp", "NTP measurement"), 21 | ] 22 | 23 | # These are the expected differences from the OpenAPI spec. 24 | # If somethign is defined here it means that an option deliberately has a 25 | # different default, valid range or is not included in the CLI tools. 26 | # e.g. if we want the tools to behave more like their common unix counterparts 27 | # or it makes sense to express things differently on a command-line 28 | 29 | common_differences = { 30 | "is_oneoff": None, # implied by lack of --interval 31 | "type": None, # set as the subcommand 32 | "start_time": None, # deliberately unsupported 33 | "stop_time": None, # deliberately unsupported 34 | "is_public": None, # deliberately unsupported 35 | "description": { 36 | "default": "", 37 | }, 38 | "interval": { 39 | "default": None, 40 | }, 41 | "resolve_on_probe": None, # Not sent explicitly to preserve behaviour 42 | } 43 | 44 | type_expected_differences = { 45 | "ping": dict(common_differences, **{ 46 | "packet_interval": { 47 | "default": 1000, 48 | }, 49 | "size": { 50 | "default": 48, 51 | }, 52 | "skip_dns_check": None, 53 | }), 54 | "traceroute": dict(common_differences, **{ 55 | "max_hops": { 56 | "default": 255, 57 | }, 58 | "protocol": { 59 | "default": "ICMP", 60 | }, 61 | "skip_dns_check": None, 62 | "paris": { 63 | "default": 0, 64 | }, 65 | "dont_fragment": { 66 | "default": False, 67 | }, 68 | "response_timeout": { 69 | "default": None, 70 | } 71 | }), 72 | "dns": dict(common_differences, **{ 73 | "protocol": { 74 | "default": "UDP", 75 | }, 76 | "query_class": { 77 | "default": "IN", 78 | }, 79 | "query_type": { 80 | "default": "A", 81 | }, 82 | "skip_dns_check": None, 83 | "set_rd_bit": { 84 | "default": True, 85 | }, 86 | "use_probe_resolver": None, # Implied by missing target 87 | "include_abuf": None, 88 | "include_qbuf": None, 89 | "prepend_probe_id": None, 90 | "use_macros": None, 91 | }), 92 | "sslcert": common_differences.copy(), 93 | "http": dict(common_differences, **{ 94 | "extended_timing": None, # "timing_verbosity" 95 | "more_extended_timing": None, # "timing_verbosity" 96 | "max_bytes_read": { 97 | "alias": "body_bytes", 98 | }, 99 | "path": { 100 | "default": "/", 101 | } 102 | }), 103 | "ntp": common_differences.copy(), 104 | } 105 | 106 | 107 | def compare_type(cmd_name, api_model, expected_differences): 108 | print(cmd_name) 109 | cmd = Command.load_command_class("measure")(["measure", cmd_name]).create() 110 | cmd.add_arguments() 111 | 112 | args = {} 113 | 114 | for arg in cmd.parser._actions: 115 | args[arg.dest] = arg 116 | 117 | seen_diffs = False 118 | 119 | for field_name, model_field in sorted(api_model["properties"].items()): 120 | if field_name in expected_differences and expected_differences[field_name] is None: 121 | continue 122 | if model_field["readOnly"]: 123 | continue 124 | 125 | explicit_values = expected_differences.get(field_name, {}) 126 | 127 | opt_name = explicit_values.get("alias", field_name) 128 | 129 | if opt_name in args: 130 | cmd_field = args.get(opt_name) 131 | 132 | expected_default = explicit_values.get("default", model_field.get("defaultValue")) 133 | if cmd_field.default != expected_default: 134 | print("\t", field_name, "DEFAULT", repr(cmd_field.default), repr(expected_default)) 135 | seen_diffs |= True 136 | 137 | expected_min = explicit_values.get("minimum", model_field.get("minimum")) 138 | cmd_min = getattr(cmd_field.type, "minimum", None) 139 | if cmd_min != expected_min: 140 | print("\t", field_name, "MINIMUM", cmd_min, expected_min) 141 | seen_diffs |= True 142 | 143 | expected_max = explicit_values.get("maximum", model_field.get("maximum")) 144 | cmd_max = getattr(cmd_field.type, "maximum", None) 145 | if cmd_max != expected_max: 146 | print("\t", field_name, "MAXIMUM", cmd_max, expected_max) 147 | seen_diffs |= True 148 | else: 149 | print("\t", field_name, "\t", "MISSING") 150 | seen_diffs |= True 151 | 152 | if not seen_diffs: 153 | print("\t", "OK") 154 | 155 | 156 | if __name__ == "__main__": 157 | api_spec = requests.get(openapi_url).json() 158 | 159 | for cmd_name, api_name in types: 160 | compare_type( 161 | cmd_name, 162 | api_spec["models"][api_name], 163 | type_expected_differences[cmd_name] 164 | ) 165 | -------------------------------------------------------------------------------- /ripe/atlas/tools/commands/shibboleet.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | import random 17 | import requests 18 | 19 | from ..cache import cache 20 | from ..helpers.colours import colourise 21 | from ..helpers.sanitisers import sanitise 22 | from .base import Command as BaseCommand 23 | 24 | 25 | class Command(BaseCommand): 26 | 27 | DESCRIPTION = "https://xkcd.com/806/" 28 | HEADERS = { 29 | "Accept": "application/vnd.github.v3+json", 30 | "User-Agent": "RIPE Atlas Tools (Magellan)", 31 | } 32 | URLS = { 33 | "root": "https://api.github.com", 34 | "statistics": [ 35 | "/repos/RIPE-NCC/ripe.atlas.sagan/stats/contributors", 36 | "/repos/RIPE-NCC/ripe-atlas-cousteau/stats/contributors", 37 | "/repos/RIPE-NCC/ripe-atlas-tools/stats/contributors", 38 | ], 39 | "users": "/users", 40 | } 41 | 42 | SPACING = ( 43 | 61, 44 | 61, 45 | 61, 46 | 53, 47 | 7, 48 | 53, 49 | 6, 50 | 52, 51 | 5, 52 | 52, 53 | 49, 54 | 48, 55 | 47, 56 | 46, 57 | 47, 58 | 47, 59 | 43, 60 | 41, 61 | 38, 62 | 39, 63 | 42, 64 | 46, 65 | ) 66 | BOAT = ( 67 | "\n{}|\n{}|\n{}|\n{}|{}|\n{}|{}---\n{}---{}'-'\n{}'-' ____|_____\n{}__" 68 | "__|__/ | /\n{}/ | / | /\n{}/ |( | (\n{}( " 69 | " | \\ | \\\n{}\\ | \\____|____\\ /|\n{}/\\____|___`---.----` ." 70 | "' |\n{}.-'/ | \\ |__.--' \\\n{}.'/ ( | \\ |. " 71 | " \\\n{}_ /_/ \\ | \\ | `. \\\n{}`-.' \\.--._|.--" 72 | "-` | `-._______\\\n{}``-.-------'-------'------------/\n{}`'.______" 73 | "_________________.'\n" 74 | ).format(*[" " * _ for _ in SPACING]) 75 | 76 | WATER = "~" * 80 77 | 78 | def __init__(self, *args, **kwargs): 79 | BaseCommand.__init__(self, *args, **kwargs) 80 | self.statistics = {} 81 | 82 | def run(self): 83 | 84 | r = ( 85 | "\nThanks for using RIPE Atlas!\n\nThis toolkit " 86 | "(Magellan) is a group effort, spearheaded by the team at the " 87 | "RIPE\nNCC, but supported by members of the community from all " 88 | "over. If you're\ncurious about who we are and what sorts of " 89 | "stuff we work on, here's a break\ndown of our contributions to " 90 | "date.\n\nName Changes URL\n{}\n" 91 | ).format("-" * 79) 92 | 93 | for contributor in self.get_contributors(): 94 | r += "{name:20} {changes:10} {url}\n".format(**contributor) 95 | 96 | print( 97 | "{}{}{}\n".format( 98 | r, 99 | colourise(self.BOAT, "bold"), 100 | colourise(self.WATER, "blue"), 101 | ) 102 | ) 103 | 104 | def get_contributors(self): 105 | 106 | cache_key = "github:statistics" 107 | 108 | self.statistics = cache.get(cache_key, {}) 109 | if not self.statistics: 110 | for url in self.URLS["statistics"]: 111 | self._update_statistics_from_url(url) 112 | cache.set(cache_key, self.statistics, 60 * 10) 113 | 114 | r = [] 115 | for k, v in self.statistics.items(): 116 | r.append( 117 | {"name": sanitise(k), "changes": v["changes"], "url": v["url"]} 118 | ) 119 | 120 | random.shuffle(r) 121 | 122 | return r 123 | 124 | def _update_statistics_from_url(self, url): 125 | 126 | response = requests.get( 127 | "{}{}".format(self.URLS["root"], url), headers=self.HEADERS 128 | ) 129 | 130 | contributors = response.json() 131 | 132 | # Sometimes, GitHub just returns nothing 133 | if not contributors: 134 | return self._update_statistics_from_url(url) 135 | 136 | for contributor in response.json(): 137 | 138 | user = self.get_user(contributor["author"]["login"]) 139 | name = user["name"] or contributor["author"]["login"] 140 | 141 | if name not in self.statistics: 142 | self.statistics[name] = { 143 | "changes": 0, 144 | "url": contributor["author"]["html_url"], 145 | } 146 | 147 | for week in contributor["weeks"]: 148 | self.statistics[name]["changes"] += week["a"] + week["d"] 149 | 150 | def get_user(self, username): 151 | 152 | cache_key = "github-user:{}".format(username) 153 | 154 | user = cache.get(cache_key) 155 | if user: 156 | return user 157 | 158 | cache.set( 159 | cache_key, 160 | requests.get( 161 | "{}{}/{}".format( 162 | self.URLS["root"], self.URLS["users"], username 163 | ), 164 | headers=self.HEADERS, 165 | ).json(), 166 | 60 * 60 * 24 * 365, 167 | ) 168 | 169 | return self.get_user(username) 170 | -------------------------------------------------------------------------------- /ripe/atlas/tools/commands/alias.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | import os 17 | 18 | from ..exceptions import RipeAtlasToolsException 19 | from ..helpers.validators import ArgumentType 20 | from ..settings import AliasesDB, aliases 21 | from .base import Command as BaseCommand 22 | 23 | 24 | class Command(BaseCommand): 25 | 26 | NAME = "alias" 27 | 28 | EDITOR = os.environ.get("EDITOR", "/usr/bin/vim") 29 | DESCRIPTION = "Manage measurements' and probes' aliases" 30 | EXTRA_DESCRIPTION = ( 31 | "As an alternative to this command, you can just create/edit {}".format( 32 | AliasesDB.USER_RC 33 | ) 34 | ) 35 | 36 | def add_arguments(self): 37 | subparsers = self.parser.add_subparsers( 38 | title="action", 39 | dest="action", 40 | help="Action to be performed on aliases. " 41 | "Run 'ripe-atlas alias --help' for more details.", 42 | ) 43 | 44 | add_parser = subparsers.add_parser("add", help="Add/modify an alias.") 45 | add_parser.add_argument( 46 | "type", 47 | action="store", 48 | choices=["measurement", "probe"], 49 | help="Type of target object.", 50 | ) 51 | add_parser.add_argument( 52 | "target", 53 | action="store", 54 | type=int, 55 | help="Target's ID.", 56 | ) 57 | add_parser.add_argument( 58 | "alias", 59 | action="store", 60 | type=ArgumentType.alias_is_valid, 61 | help="Alias name.", 62 | ) 63 | 64 | del_parser = subparsers.add_parser("del", help="Remove an alias.") 65 | del_parser.add_argument( 66 | "type", 67 | action="store", 68 | choices=["measurement", "probe"], 69 | help="Type of target object.", 70 | ) 71 | del_parser.add_argument( 72 | "alias", 73 | action="store", 74 | type=ArgumentType.alias_is_valid, 75 | help="Alias name.", 76 | ) 77 | 78 | show_parser = subparsers.add_parser("show", help="Show target's ID.") 79 | show_parser.add_argument( 80 | "type", 81 | action="store", 82 | choices=["measurement", "probe"], 83 | help="Type of target object.", 84 | ) 85 | show_parser.add_argument( 86 | "alias", 87 | action="store", 88 | type=ArgumentType.alias_is_valid, 89 | help="Alias name.", 90 | ) 91 | 92 | list_parser = subparsers.add_parser("list", help="List configured aliases.") 93 | list_parser.add_argument( 94 | "type", 95 | action="store", 96 | choices=["measurement", "probe"], 97 | help="Type of target object.", 98 | ) 99 | 100 | subparsers.add_parser( 101 | "editor", 102 | help="Invoke {0} to edit the configuration directly".format(self.EDITOR), 103 | ) 104 | 105 | def run(self): 106 | 107 | if not self.arguments.action: 108 | raise RipeAtlasToolsException( 109 | "Action not given. Use --help for more information." 110 | ) 111 | 112 | if self.arguments.action == "editor": 113 | os.system("{0} {1}".format(self.EDITOR, AliasesDB.USER_RC)) 114 | return self.ok("Aliases file writen to {}".format(AliasesDB.USER_RC)) 115 | 116 | else: 117 | alias_type = self.arguments.type 118 | 119 | if self.arguments.action == "add": 120 | alias_name = self.arguments.alias 121 | target_id = self.arguments.target 122 | 123 | try: 124 | ArgumentType.alias_is_valid(alias_name) 125 | except Exception as e: 126 | raise RipeAtlasToolsException(str(e)) 127 | 128 | aliases[alias_type][alias_name] = target_id 129 | AliasesDB.write(aliases) 130 | 131 | elif self.arguments.action == "del": 132 | alias_name = self.arguments.alias 133 | del aliases[alias_type][alias_name] 134 | AliasesDB.write(aliases) 135 | 136 | elif self.arguments.action == "show": 137 | alias_name = self.arguments.alias 138 | if alias_name in aliases[alias_type]: 139 | self.ok( 140 | "'{}' is an alias for {}".format( 141 | alias_name, aliases[alias_type][alias_name] 142 | ) 143 | ) 144 | else: 145 | self.not_ok("'{}' alias does not exist".format(alias_name)) 146 | 147 | elif self.arguments.action == "list": 148 | res = "{} aliases:\n\n".format(alias_type.capitalize()) 149 | for alias_name in sorted(aliases[alias_type]): 150 | res += "- {}: {}\n".format( 151 | alias_name, aliases[alias_type][alias_name] 152 | ) 153 | self.ok(res) 154 | -------------------------------------------------------------------------------- /ripe/atlas/tools/commands/measure/traceroute.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from ...helpers.validators import ArgumentType 17 | from ...settings import conf 18 | 19 | from .base import Command 20 | 21 | 22 | class TracerouteMeasureCommand(Command): 23 | DESCRIPTION = "Create a traceroute measurement and wait for the results" 24 | 25 | def _upper_str(self, s): 26 | """ 27 | Private method to validate specific command line arguments that 28 | should be provided in upper or lower case 29 | :param s: string 30 | :return: string in upper case 31 | """ 32 | return s.upper() 33 | 34 | def add_arguments(self): 35 | 36 | Command.add_arguments(self) 37 | 38 | self.add_primary_argument(name="target", parser=self.parser) 39 | 40 | spec = conf["specification"]["types"]["traceroute"] 41 | 42 | specific = self.parser.add_argument_group("Traceroute-specific Options") 43 | specific.add_argument( 44 | "--packets", 45 | type=ArgumentType.integer_range(minimum=1, maximum=16), 46 | default=spec["packets"], 47 | help="The number of packets sent", 48 | ) 49 | specific.add_argument( 50 | "--size", 51 | type=ArgumentType.integer_range(minimum=0, maximum=2048), 52 | default=spec["size"], 53 | help="The size of packets sent", 54 | ) 55 | specific.add_argument( 56 | "--protocol", 57 | type=self._upper_str, 58 | choices=("ICMP", "UDP", "TCP"), 59 | default=spec["protocol"], 60 | help="The protocol used.", 61 | ) 62 | specific.add_argument( 63 | "--timeout", 64 | type=ArgumentType.integer_range(minimum=1), 65 | default=spec["timeout"], 66 | help="The timeout per-packet", 67 | ) 68 | self.add_flag( 69 | parser=specific, 70 | name="dont-fragment", 71 | default=spec["dont-fragment"], 72 | help="Disable fragmentation of outgoing packets", 73 | ) 74 | specific.add_argument( 75 | "--paris", 76 | type=ArgumentType.integer_range(minimum=0, maximum=64), 77 | default=spec["paris"], 78 | help="Use Paris. Value must be between 0 and 64." 79 | "If 0, a standard traceroute will be performed", 80 | ) 81 | specific.add_argument( 82 | "--first-hop", 83 | type=ArgumentType.integer_range(minimum=1, maximum=255), 84 | default=spec["first-hop"], 85 | help="Value must be between 1 and 255", 86 | ) 87 | specific.add_argument( 88 | "--max-hops", 89 | type=ArgumentType.integer_range(minimum=1, maximum=255), 90 | default=spec["max-hops"], 91 | help="Value must be between 1 and 255", 92 | ) 93 | specific.add_argument( 94 | "--port", 95 | type=ArgumentType.integer_range(minimum=1, maximum=65535), 96 | default=spec["port"], 97 | help="Destination port, valid for TCP only", 98 | ) 99 | specific.add_argument( 100 | "--destination-option-size", 101 | type=ArgumentType.integer_range(minimum=0, maximum=1024), 102 | default=spec["destination-option-size"], 103 | help="IPv6 destination option header", 104 | ) 105 | specific.add_argument( 106 | "--hop-by-hop-option-size", 107 | type=ArgumentType.integer_range(minimum=0, maximum=2048), 108 | default=spec["hop-by-hop-option-size"], 109 | help=" IPv6 hop by hop option header", 110 | ) 111 | specific.add_argument( 112 | "--duplicate-timeout", 113 | default=spec["duplicate-timeout"], 114 | type=int, 115 | help="Time to wait (in milliseconds) for a duplicate response " 116 | "after receiving the first response", 117 | ) 118 | specific.add_argument( 119 | "--response-timeout", 120 | default=spec["response-timeout"], 121 | type=ArgumentType.integer_range(minimum=1, maximum=60000), 122 | help="Response timeout for one packet", 123 | ) 124 | 125 | def _get_measurement_kwargs(self): 126 | 127 | r = Command._get_measurement_kwargs(self) 128 | 129 | keys = ( 130 | "destination_option_size", 131 | "dont_fragment", 132 | "first_hop", 133 | "hop_by_hop_option_size", 134 | "max_hops", 135 | "packets", 136 | "paris", 137 | "port", 138 | "protocol", 139 | "size", 140 | "timeout", 141 | ) 142 | for key in keys: 143 | r[key] = getattr(self.arguments, key) 144 | optional_keys = ["duplicate_timeout", "response_timeout"] 145 | for key in optional_keys: 146 | val = getattr(self.arguments, key) 147 | if val is not None: 148 | r[key] = val 149 | 150 | return r 151 | -------------------------------------------------------------------------------- /tests/helpers/test_validators.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2015 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | import argparse 17 | import datetime 18 | import os 19 | import sys 20 | import unittest 21 | 22 | from io import StringIO 23 | 24 | from ripe.atlas.tools.helpers.validators import ArgumentType 25 | 26 | 27 | class TestArgumentTypeHelper(unittest.TestCase): 28 | def test_path(self): 29 | 30 | self.assertEqual("/tmp", ArgumentType.path("/tmp")) 31 | 32 | with self.assertRaises(argparse.ArgumentTypeError): 33 | ArgumentType.path("/not/a/real/place") 34 | 35 | def test_country_code(self): 36 | 37 | self.assertEqual("CA", ArgumentType.country_code("CA")) 38 | self.assertEqual("CA", ArgumentType.country_code("ca")) 39 | self.assertEqual("CA", ArgumentType.country_code("Ca")) 40 | self.assertEqual("CA", ArgumentType.country_code("cA")) 41 | 42 | for value in ("CAN", "Canada", "can", "This isn't even a country"): 43 | with self.assertRaises(argparse.ArgumentTypeError): 44 | ArgumentType.country_code(value) 45 | 46 | def test_comma_separated_integers(self): 47 | 48 | self.assertEqual([1, 2, 3], ArgumentType.comma_separated_integers()("1,2,3")) 49 | 50 | self.assertEqual([1, 2, 3], ArgumentType.comma_separated_integers()("1, 2, 3")) 51 | 52 | self.assertEqual([1], ArgumentType.comma_separated_integers()("1")) 53 | 54 | with self.assertRaises(argparse.ArgumentTypeError): 55 | ArgumentType.comma_separated_integers()("1,2,3,pizza!") 56 | with self.assertRaises(argparse.ArgumentTypeError): 57 | ArgumentType.comma_separated_integers(minimum=5)("4,5,6,7") 58 | with self.assertRaises(argparse.ArgumentTypeError): 59 | ArgumentType.comma_separated_integers(maximum=5)("1,2,3,4,6") 60 | 61 | def test_datetime(self): 62 | 63 | d = datetime.datetime(2015, 12, 1) 64 | 65 | self.assertEqual(d, ArgumentType.datetime("2015-12-1")) 66 | self.assertEqual(d, ArgumentType.datetime("2015-12-1T00")) 67 | self.assertEqual(d, ArgumentType.datetime("2015-12-1T00:00")) 68 | self.assertEqual(d, ArgumentType.datetime("2015-12-1T00:00:00")) 69 | self.assertEqual(d, ArgumentType.datetime("2015-12-1")) 70 | self.assertEqual(d, ArgumentType.datetime("2015-12-1 00")) 71 | self.assertEqual(d, ArgumentType.datetime("2015-12-1 00:00")) 72 | self.assertEqual(d, ArgumentType.datetime("2015-12-1 00:00:00")) 73 | 74 | with self.assertRaises(argparse.ArgumentTypeError): 75 | ArgumentType.datetime("yesterday") 76 | 77 | with self.assertRaises(argparse.ArgumentTypeError): 78 | ArgumentType.datetime("Definitely not a date, or even a time") 79 | 80 | def test_integer_range(self): 81 | 82 | self.assertEqual(1, ArgumentType.integer_range(1, 10)("1")) 83 | self.assertEqual(10, ArgumentType.integer_range(1, 10)("10")) 84 | self.assertEqual(1, ArgumentType.integer_range(-1, 1)("1")) 85 | self.assertEqual(-1, ArgumentType.integer_range(-1, 1)("-1")) 86 | 87 | for value in ("0", "11", "-1"): 88 | with self.assertRaises(argparse.ArgumentTypeError): 89 | ArgumentType.integer_range(1, 10)(value) 90 | 91 | def test_ip_or_domain(self): 92 | 93 | passable_hosts = ( 94 | "localhost", 95 | "ripe.net", 96 | "www.ripe.net", 97 | "1.2.3.4", 98 | "2001:67c:2e8:22::c100:68b", 99 | ) 100 | for host in passable_hosts: 101 | self.assertEqual(host, ArgumentType.ip_or_domain(host)) 102 | 103 | with self.assertRaises(argparse.ArgumentTypeError): 104 | ArgumentType.ip_or_domain("Definitely not a host") 105 | 106 | def test_comma_separated_integers_or_file(self): 107 | 108 | with self.assertRaises(argparse.ArgumentTypeError): 109 | ArgumentType.comma_separated_integers_or_file("/dev/null/fail") 110 | 111 | with self.assertRaises(argparse.ArgumentTypeError): 112 | ArgumentType.comma_separated_integers_or_file("not,a,number") 113 | 114 | with self.assertRaises(argparse.ArgumentTypeError): 115 | ArgumentType.comma_separated_integers_or_file("1, 2, 3") 116 | 117 | old = sys.stdin 118 | sys.stdin = StringIO("1\n2\n") 119 | self.assertEqual(ArgumentType.comma_separated_integers_or_file("-"), [1, 2]) 120 | sys.stdin = old 121 | 122 | in_file = "/tmp/__test_file__" 123 | with open(in_file, "w") as f: 124 | f.write("1\n2\n3\n") 125 | self.assertEqual( 126 | ArgumentType.comma_separated_integers_or_file(in_file), [1, 2, 3] 127 | ) 128 | os.unlink(in_file) 129 | 130 | def test_measurement_alias(self): 131 | 132 | tests = ["", "\\invalid", "+invalid", ":invalid", "12345"] 133 | for test in tests: 134 | with self.assertRaises(argparse.ArgumentTypeError): 135 | ArgumentType.alias_is_valid(test) 136 | 137 | tests = [ 138 | "valid", 139 | "123valid", 140 | "valid123", 141 | "_valid", 142 | "valid_", 143 | "-valid", 144 | "valid-", 145 | ".valid", 146 | ] 147 | 148 | for test in tests: 149 | self.assertEqual(ArgumentType.alias_is_valid(test), test) 150 | -------------------------------------------------------------------------------- /ripe/atlas/tools/commands/configure.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | import functools 17 | import os 18 | 19 | from ..exceptions import RipeAtlasToolsException 20 | from ..settings import Configuration, conf 21 | from .base import Command as BaseCommand 22 | 23 | 24 | class Command(BaseCommand): 25 | 26 | NAME = "configure" 27 | 28 | EDITOR = os.environ.get("EDITOR", "/usr/bin/vim") 29 | DESCRIPTION = "Adjust or initialize configuration options" 30 | EXTRA_DESCRIPTION = ( 31 | "As an alternative to this command, you can just create/edit {}".format( 32 | Configuration.USER_RC 33 | ) 34 | ) 35 | 36 | def add_arguments(self): 37 | self.parser.add_argument( 38 | "--editor", 39 | action="store_true", 40 | help="Invoke {0} to edit the configuration directly".format(self.EDITOR), 41 | ) 42 | self.parser.add_argument( 43 | "--set", 44 | action="store", 45 | help="Permanently set a configuration value so it can be used in " 46 | "the future. Example: --set authorisation.create=MY_API_KEY", 47 | ) 48 | self.parser.add_argument( 49 | "--init", 50 | action="store_true", 51 | help="Create a configuration file and save it into your home " 52 | "directory at: {}".format(Configuration.USER_RC), 53 | ) 54 | 55 | def run(self): 56 | 57 | if not self.arguments.init: 58 | if not self.arguments.editor: 59 | if not self.arguments.set: 60 | self.ok("Effective configuration options:") 61 | self.print(self.get_existing_config()) 62 | return self.ok("Call configure --help for more information") 63 | 64 | self._create_if_necessary() 65 | 66 | if self.arguments.editor: 67 | os.system("{0} {1}".format(self.EDITOR, Configuration.USER_RC)) 68 | 69 | if self.arguments.init or self.arguments.editor: 70 | return self.ok( 71 | "Configuration file writen to {}".format(Configuration.USER_RC) 72 | ) 73 | 74 | if self.arguments.set: 75 | if "=" not in self.arguments.set: 76 | raise RipeAtlasToolsException( 77 | "Invalid format. Execute with --help for more information." 78 | ) 79 | path, value = self.arguments.set.split("=") 80 | self.set(path.split("."), value) 81 | 82 | def get_existing_config(self, c=conf, parts=[]): 83 | s = "" 84 | for key in c: 85 | full_key = parts + [key] 86 | val = c[key] 87 | if isinstance(val, dict): 88 | s += self.get_existing_config(val, full_key) 89 | else: 90 | s += f'{".".join(full_key)} = {val!r}\n' 91 | return s 92 | 93 | def set(self, path, value): 94 | if path[:2] == ["authorisation", "fetch_aliases"]: 95 | if len(path) > 3: 96 | raise RipeAtlasToolsException( 97 | "Invalid alias for a fetch API key: it must be in the " 98 | "format authorisation.fetch.some-alias=MY_API_KEY" 99 | ) 100 | 101 | if "fetch_aliases" not in conf["authorisation"]: 102 | conf["authorisation"]["fetch_aliases"] = {} 103 | if conf["authorisation"]["fetch_aliases"] is None: 104 | conf["authorisation"]["fetch_aliases"] = {} 105 | 106 | alias = path[2] 107 | 108 | if alias not in conf["authorisation"]["fetch_aliases"]: 109 | conf["authorisation"]["fetch_aliases"][alias] = None 110 | 111 | required_type = str 112 | else: 113 | try: 114 | required_type = type(self._get_from_dict(conf, path)) 115 | except KeyError: 116 | raise RipeAtlasToolsException( 117 | 'Invalid configuration key: "{}"'.format(".".join(path)) 118 | ) 119 | 120 | if value.isdigit(): 121 | value = int(value) 122 | 123 | if not isinstance(value, required_type): 124 | raise RipeAtlasToolsException( 125 | 'Invalid configuration value: "{}". You must supply a {} for ' 126 | "this key".format(value, required_type.__name__) 127 | ) 128 | 129 | self._set_in_dict(conf, path, value) 130 | 131 | Configuration.write(conf) 132 | 133 | @staticmethod 134 | def _create_if_necessary(): 135 | 136 | if os.path.exists(Configuration.USER_RC): 137 | return 138 | 139 | if not os.path.exists(Configuration.USER_CONFIG_DIR): 140 | os.makedirs(Configuration.USER_CONFIG_DIR) 141 | 142 | Configuration.write(conf) 143 | 144 | @staticmethod 145 | def _get_from_dict(data, path): 146 | return functools.reduce(lambda d, k: d[k], path, data) 147 | 148 | @classmethod 149 | def _set_in_dict(cls, data, path, value): 150 | cls._get_from_dict(data, path[:-1])[path[-1]] = value 151 | 152 | @staticmethod 153 | def cast_value(value): 154 | 155 | # Booleans are a pain in the ass to cast 156 | if value.lower() == "true": 157 | return True 158 | if value.lower() == "false": 159 | return False 160 | 161 | try: 162 | return int(value) 163 | except ValueError: 164 | try: 165 | return float(value) 166 | except ValueError: 167 | return str(value) 168 | -------------------------------------------------------------------------------- /tests/commands/test_alias.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | import copy 17 | import unittest 18 | 19 | from unittest import mock 20 | 21 | from ripe.atlas.tools.commands.alias import Command 22 | from ripe.atlas.tools.exceptions import RipeAtlasToolsException 23 | from ripe.atlas.tools.settings import AliasesDB 24 | 25 | from ..base import capture_sys_output 26 | 27 | 28 | class FakeAliasesDB(AliasesDB): 29 | @staticmethod 30 | def write(aliases): 31 | pass 32 | 33 | 34 | class TestAliasCommand(unittest.TestCase): 35 | 36 | ALIASES_PATH = "ripe.atlas.tools.commands.alias.aliases" 37 | ALIASES = {"measurement": {"msm1": 1}, "probe": {"prb1": 1}} 38 | ALIASES_CLASS_PATH = "ripe.atlas.tools.commands.alias.AliasesDB" 39 | 40 | def setUp(self): 41 | self.cmd = Command() 42 | self.aliases = copy.deepcopy(TestAliasCommand.ALIASES) 43 | mock.patch(self.ALIASES_CLASS_PATH, FakeAliasesDB).start() 44 | 45 | def tearDown(self): 46 | mock.patch.stopall() 47 | 48 | def test_no_arguments(self): 49 | with capture_sys_output(): 50 | with self.assertRaises(RipeAtlasToolsException) as e: 51 | self.cmd.init_args([]) 52 | self.cmd.run() 53 | self.assertTrue(str(e.exception).startswith("Action not given.")) 54 | 55 | def test_bad_action(self): 56 | with capture_sys_output(): 57 | with self.assertRaises(SystemExit): 58 | self.cmd.init_args(["test"]) 59 | 60 | def test_bad_target_id(self): 61 | with capture_sys_output() as (stdout, stderr): 62 | with self.assertRaises(SystemExit): 63 | self.cmd.init_args("add probe a b".split()) 64 | err = stderr.getvalue().split("\n")[-2] 65 | self.assertEqual( 66 | err, "Ripe-atlas alias add: error: argument target: invalid int value: 'a'" 67 | ) 68 | 69 | def test_add_args(self): 70 | for alias_type in ("probe", "measurement"): 71 | with capture_sys_output() as (stdout, stderr): 72 | with self.assertRaises(SystemExit): 73 | cmd = Command() 74 | cmd.init_args(["add", alias_type]) 75 | 76 | with self.assertRaises(SystemExit): 77 | cmd = Command() 78 | cmd.init_args(["add", alias_type, "1"]) 79 | 80 | with self.assertRaises(SystemExit): 81 | cmd = Command() 82 | cmd.init_args(["add", alias_type, "1", "1"]) 83 | 84 | try: 85 | cmd = Command() 86 | cmd.init_args(["add", alias_type, "1", "one"]) 87 | except Exception as e: 88 | self.fail("Failed with {}".format(str(e))) 89 | 90 | def test_del_args(self): 91 | for alias_type in ("probe", "measurement"): 92 | with capture_sys_output() as (stdout, stderr): 93 | with self.assertRaises(SystemExit): 94 | cmd = Command() 95 | cmd.init_args(["del", alias_type]) 96 | 97 | with self.assertRaises(SystemExit): 98 | cmd = Command() 99 | cmd.init_args(["del", alias_type, "1"]) 100 | 101 | try: 102 | cmd = Command() 103 | cmd.init_args(["del", alias_type, "one"]) 104 | except Exception as e: 105 | self.fail("Failed with {}".format(str(e))) 106 | 107 | def test_bad_alias(self): 108 | with capture_sys_output() as (stdout, stderr): 109 | with self.assertRaises(SystemExit): 110 | self.cmd.init_args("add measurement 1 bad+alias".split()) 111 | err = stderr.getvalue().split("\n")[-2] 112 | self.assertEqual( 113 | err, 114 | "Ripe-atlas alias add: error: argument alias: " 115 | '"bad+alias" does not appear to be a valid alias.', 116 | ) 117 | 118 | def test_show_msm_ok(self): 119 | path = "ripe.atlas.tools.commands.alias.Command.ok" 120 | with mock.patch(path) as mock_ok: 121 | with mock.patch(self.ALIASES_PATH, self.aliases): 122 | self.cmd.init_args("show measurement msm1".split()) 123 | self.cmd.run() 124 | mock_ok.assert_called_once() 125 | 126 | def test_show_msm_ko(self): 127 | path = "ripe.atlas.tools.commands.alias.Command.not_ok" 128 | with mock.patch(path) as mock_ko: 129 | with mock.patch(self.ALIASES_PATH, self.aliases): 130 | self.cmd.init_args("show measurement msm2".split()) 131 | self.cmd.run() 132 | mock_ko.assert_called_once() 133 | 134 | def test_add_msm(self): 135 | with mock.patch(self.ALIASES_PATH, self.aliases): 136 | self.cmd.init_args("add measurement 2 msm2".split()) 137 | self.cmd.run() 138 | self.assertTrue("msm2" in self.aliases["measurement"]) 139 | self.assertEqual(self.aliases["measurement"]["msm2"], 2) 140 | 141 | def test_del_msm(self): 142 | self.assertTrue("msm1" in self.aliases["measurement"]) 143 | with mock.patch(self.ALIASES_PATH, self.aliases): 144 | self.cmd.init_args("del measurement msm1".split()) 145 | self.cmd.run() 146 | self.assertFalse("msm1" in self.aliases["measurement"]) 147 | 148 | def test_list(self): 149 | path = "ripe.atlas.tools.commands.alias.Command.ok" 150 | with mock.patch(path) as mock_ok: 151 | self.aliases["measurement"]["msm2"] = 2 152 | self.aliases["measurement"]["abc"] = 123 153 | with mock.patch(self.ALIASES_PATH, self.aliases): 154 | self.cmd.init_args("list measurement".split()) 155 | self.cmd.run() 156 | mock_ok.assert_called_once_with( 157 | "Measurement aliases:\n\n- abc: 123\n- msm1: 1\n- msm2: 2\n" 158 | ) 159 | -------------------------------------------------------------------------------- /ripe/atlas/tools/commands/measure/dns.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | 16 | from ripe.atlas.sagan.dns import Message 17 | 18 | from ...exceptions import RipeAtlasToolsException 19 | from ...helpers.validators import ArgumentType 20 | from ...settings import conf 21 | 22 | from .base import Command 23 | 24 | 25 | class DnsMeasureCommand(Command): 26 | DESCRIPTION = "Create a DNS measurement and wait for the results" 27 | 28 | def _upper_str(self, s): 29 | """ 30 | Private method to validate specific command line arguments that 31 | should be provided in upper or lower case 32 | :param s: string 33 | :return: string in upper case 34 | """ 35 | return s.upper() 36 | 37 | def add_arguments(self): 38 | 39 | Command.add_arguments(self) 40 | 41 | self.add_primary_argument(name="query_argument", parser=self.parser) 42 | 43 | specific = self.parser.add_argument_group("DNS-specific Options") 44 | specific.add_argument( 45 | "--protocol", 46 | type=self._upper_str, 47 | choices=("UDP", "TCP"), 48 | default=conf["specification"]["types"]["dns"]["protocol"], 49 | help="The protocol used.", 50 | ) 51 | specific.add_argument( 52 | "--query-class", 53 | type=self._upper_str, 54 | choices=("IN", "CHAOS"), 55 | default=conf["specification"]["types"]["dns"]["query-class"], 56 | help='The query class. The default is "{}"'.format( 57 | conf["specification"]["types"]["dns"]["query-class"] 58 | ), 59 | ) 60 | specific.add_argument( 61 | "--query-type", 62 | type=self._upper_str, 63 | choices=list(Message.ANSWER_CLASSES.keys()) 64 | + ["ANY"], # The only ones we can parse 65 | default=conf["specification"]["types"]["dns"]["query-type"], 66 | help='The query type. The default is "{}"'.format( 67 | conf["specification"]["types"]["dns"]["query-type"] 68 | ), 69 | ) 70 | specific.add_argument( 71 | "--query-argument", 72 | type=str, 73 | default=conf["specification"]["types"]["dns"]["query-argument"], 74 | help="The DNS label to query", 75 | ) 76 | 77 | self.add_flag( 78 | parser=specific, 79 | name="set-cd-bit", 80 | help="Set DNSSEC Checking Disabled flag (RFC4035)", 81 | default=conf["specification"]["types"]["dns"]["set-cd-bit"], 82 | ) 83 | self.add_flag( 84 | parser=specific, 85 | name="set-do-bit", 86 | help="Set DNSSEC OK flag (RFC3225)", 87 | default=conf["specification"]["types"]["dns"]["set-do-bit"], 88 | ) 89 | self.add_flag( 90 | parser=specific, 91 | name="set-nsid-bit", 92 | help="Set Name Server Identifier flag (RFC5001)", 93 | default=conf["specification"]["types"]["dns"]["set-nsid-bit"], 94 | ) 95 | self.add_flag( 96 | parser=specific, 97 | name="set-rd-bit", 98 | help="Set Recursion Desired flag (RFC1035)", 99 | default=conf["specification"]["types"]["dns"]["set-rd-bit"], 100 | ) 101 | 102 | specific.add_argument( 103 | "--retry", 104 | type=ArgumentType.integer_range(minimum=0, maximum=10), 105 | default=conf["specification"]["types"]["dns"]["retry"], 106 | help="Number of times to retry", 107 | ) 108 | specific.add_argument( 109 | "--udp-payload-size", 110 | type=ArgumentType.integer_range(minimum=512, maximum=4096), 111 | default=conf["specification"]["types"]["dns"]["udp-payload-size"], 112 | help="May be any integer between 512 and 4096 inclusive", 113 | ) 114 | specific.add_argument( 115 | "--timeout", 116 | default=conf["specification"]["types"]["dns"]["timeout"], 117 | type=ArgumentType.integer_range(minimum=100, maximum=30000), 118 | help="Per packet timeout in milliseconds", 119 | ) 120 | self.add_flag( 121 | parser=specific, 122 | name="tls", 123 | help="Send query using DNS-over-TLS", 124 | default=conf["specification"]["types"]["dns"]["tls"], 125 | ) 126 | 127 | def clean_target(self): 128 | """ 129 | Targets aren't required for this type 130 | """ 131 | return self.arguments.target 132 | 133 | def clean_description(self): 134 | if self.arguments.target: 135 | return Command.clean_description(self) 136 | return "DNS measurement for {}".format(self.arguments.query_argument) 137 | 138 | def _get_measurement_kwargs(self): 139 | 140 | r = Command._get_measurement_kwargs(self) 141 | 142 | for opt in ("class", "type", "argument"): 143 | if not getattr(self.arguments, "query_{0}".format(opt)): 144 | raise RipeAtlasToolsException( 145 | "At a minimum, DNS measurements require a query argument." 146 | ) 147 | 148 | r["query_class"] = self.arguments.query_class 149 | r["query_type"] = self.arguments.query_type 150 | r["query_argument"] = self.arguments.query_argument 151 | r["set_cd_bit"] = self.arguments.set_cd_bit 152 | r["set_do_bit"] = self.arguments.set_do_bit 153 | r["set_rd_bit"] = self.arguments.set_rd_bit 154 | r["set_nsid_bit"] = self.arguments.set_nsid_bit 155 | r["protocol"] = self.arguments.protocol 156 | r["retry"] = self.arguments.retry 157 | r["udp_payload_size"] = self.arguments.udp_payload_size 158 | r["use_probe_resolver"] = "target" not in r 159 | r["tls"] = self.arguments.tls 160 | if self.arguments.timeout is not None: 161 | r["timeout"] = self.arguments.timeout 162 | 163 | return r 164 | -------------------------------------------------------------------------------- /ripe/atlas/tools/filters.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2016 RIPE NCC 2 | # 3 | # This program is free software: you can redistribute it and/or modify 4 | # it under the terms of the GNU General Public License as published by 5 | # the Free Software Foundation, either version 3 of the License, or 6 | # (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program. If not, see . 15 | from ripe.atlas.sagan import Result 16 | from ripe.atlas.sagan import ResultParseError 17 | from ripe.atlas.cousteau import ProbeRequest 18 | from ripe.atlas.cousteau import Probe as CProbe 19 | 20 | from .exceptions import RipeAtlasToolsException 21 | from .cache import cache 22 | from .settings import conf 23 | 24 | 25 | class FilterFactory(object): 26 | @staticmethod 27 | def create(key, value): 28 | """Create new filter class based on the key""" 29 | if key == "asn": 30 | return ASNFilter(value) 31 | else: 32 | return Filter(key, value) 33 | 34 | 35 | class Filter(object): 36 | """ 37 | Class that represents filter for results. For now supports only attributes 38 | of probes property of Result property. It could be extended for any property 39 | of Result easily. 40 | """ 41 | 42 | def __init__(self, key, value): 43 | self.key = key 44 | self.value = value 45 | 46 | def filter(self, result): 47 | """ 48 | Decide if given result should be filtered (False) or remain on the 49 | pile of results. 50 | """ 51 | try: 52 | attr_value = getattr(result.probe, self.key) 53 | except AttributeError: 54 | log = ( 55 | "Cousteau's Probe class does not have an attribute " "called: <{}>" 56 | ).format(self.key) 57 | raise RipeAtlasToolsException(log) 58 | if attr_value == self.value: 59 | return True 60 | 61 | return False 62 | 63 | 64 | class ASNFilter(Filter): 65 | """Class thar represents filter by probes that belong to given ASN.""" 66 | 67 | def __init__(self, value): 68 | key = "asn" 69 | super(ASNFilter, self).__init__(key, value) 70 | 71 | def filter(self, result): 72 | asn_v4 = getattr(result.probe, "asn_v4") 73 | asn_v6 = getattr(result.probe, "asn_v6") 74 | if self.value in (asn_v4, asn_v6): 75 | return True 76 | 77 | return False 78 | 79 | 80 | def filter_results(filters, results): 81 | """docstring for filter""" 82 | new_results = [] 83 | for result in results: 84 | for rfilter in filters: 85 | if rfilter.filter(result): 86 | new_results.append(result) 87 | break 88 | 89 | return new_results 90 | 91 | 92 | class SaganSet(object): 93 | """ 94 | An iterable of sagan results with attached probe information that allows 95 | for filtering by the filters module. 96 | """ 97 | 98 | def __init__(self, iterable=None, probes=()): 99 | self._probes = probes 100 | self._iterable = iterable 101 | 102 | def __iter__(self): 103 | 104 | sagans = [] 105 | 106 | for line in self._iterable: 107 | 108 | # line may be a dictionary (parsed JSON) 109 | if hasattr(line, "strip"): 110 | line = line.strip() 111 | 112 | # Break out when there's nothing left 113 | if not line: 114 | break 115 | 116 | try: 117 | sagan = Result.get( 118 | line, 119 | on_error=Result.ACTION_IGNORE, 120 | on_warning=Result.ACTION_IGNORE, 121 | ) 122 | if not self._probes or sagan.probe_id in self._probes: 123 | sagans.append(sagan) 124 | if len(sagans) > 100: 125 | for sagan in self._attach_probes(sagans): 126 | yield sagan 127 | sagans = [] 128 | except ResultParseError: 129 | pass # Probably garbage in the file 130 | 131 | for sagan in self._attach_probes(sagans): 132 | yield sagan 133 | 134 | def __next__(self): 135 | return iter(self).next() 136 | 137 | def next(self): 138 | return self.__next__() 139 | 140 | def _attach_probes(self, sagans): 141 | probes = dict( 142 | [ 143 | (p.id, p) 144 | for p in Probe.get_many( 145 | (s.probe_id for s in sagans) 146 | ) 147 | ] 148 | ) 149 | for sagan in sagans: 150 | sagan.probe = probes[sagan.probe_id] 151 | yield sagan 152 | 153 | 154 | class Probe(object): 155 | """ 156 | A crude representation of the data we get from the API via Cousteau 157 | """ 158 | 159 | EXPIRE_TIME = 60 * 60 * 24 * 30 160 | 161 | @classmethod 162 | def get(cls, pk): 163 | """ 164 | Given a single id, attempt to fetch a probe object from the cache. If 165 | that fails, do an API call to get it. Don't use this for multiple 166 | probes unless you know they're all in the cache, or you'll be in for a 167 | long wait. 168 | """ 169 | r = cache.get("probe:{}".format(pk)) 170 | if not r: 171 | kwargs = {"id": pk, "server": conf["api-server"]} 172 | probe = CProbe(**kwargs) 173 | cache.set("probe:{}".format(probe.id), probe, cls.EXPIRE_TIME) 174 | return probe 175 | 176 | @classmethod 177 | def get_many(cls, ids): 178 | """ 179 | Given a list of ids, attempt to get probe objects out of the local 180 | cache. Probes that cannot be found will be fetched from the API and 181 | cached for future use. 182 | """ 183 | 184 | r = [] 185 | 186 | fetch_ids = [] 187 | for pk in ids: 188 | probe = cache.get("probe:{}".format(pk)) 189 | if probe: 190 | r.append(probe) 191 | else: 192 | fetch_ids.append(str(pk)) 193 | 194 | if fetch_ids: 195 | kwargs = {"id__in": fetch_ids, "server": conf["api-server"]} 196 | for probe in [p for p in ProbeRequest(return_objects=True, **kwargs)]: 197 | cache.set("probe:{}".format(probe.id), probe, cls.EXPIRE_TIME) 198 | r.append(probe) 199 | 200 | return r 201 | -------------------------------------------------------------------------------- /CHANGES.rst: -------------------------------------------------------------------------------- 1 | Release History 2 | =============== 3 | 3.3.1 (release 2025-12-09) 4 | -------------------------- 5 | - Upgraded dependency ripe-atlas-cousteau>=2.2,<3 6 | 7 | 3.3.0 (release 2025-11-13) 8 | -------------------------- 9 | - Official supported Python versions changed to 3.10, 3.11, 3.12 and 3.13 10 | - --auto-topup, --auto-topup-prb-days-off, --auto-topup-prb-similarity, --target-update-hours, and --aggregator-client-id added to measure command 11 | 12 | 3.2.0 (release 2025-09-24) 13 | -------------------------- 14 | - Use sendBacklog when streaming new measurements to avoid missed results in case of high network or client latencies 15 | - Add --sendBacklog option to stream command 16 | - Pin urllib3 to < 2 so that it works on LibreSSL Python 17 | 18 | 3.1.0 (release 2023-02-07) 19 | -------------------------- 20 | - Improved probe-search and measurement-search, including "csv" and "tab" output 21 | - --stream-timeout and --stream-limit added to measure command 22 | - Use the latest stream API (cousteau update) and add --timeout to stream command 23 | 24 | 3.0.3 (release 2022-11-18) 25 | -------------------------- 26 | - Fix issue where the measure command would continue to stream results after all probes have responded 27 | 28 | 3.0.2 (release 2022-05-23) 29 | -------------------------- 30 | - Fix "measure spec" command which was broken due to cousteau issue 31 | 32 | 3.0.1 (release 2022-02-24) 33 | -------------------------- 34 | - Updated cousteau dependency to the non-alpha release 35 | 36 | 3.0.0 (release 2022-02-23) 37 | -------------------------- 38 | - API keys can now be passed in environment variables 39 | - probe-search by --location now works, as long as the user specifies their own Google Geocoding API key 40 | - Modernized tests and switched to GitHub actions 41 | - The default renderer for ping measurements is now more consistent and more similar to other ping tools, including having a statistical summary at the end 42 | - measure, report and stream commands now all use the same set of renderers 43 | - "measure spec" command which takes a JSON blob to create measurements 44 | - Allow measure --target to be specified as a positional arg (or --query-argument for DNS) 45 | - Move to latest cousteau version (python-socketio) 46 | - Various other fixes to code and documentation 47 | - Official supported Python versions changed to 3.6, 3.7, 3.8, 3.9 and 3.10 48 | 49 | 50 | 2.3.0 (released 2018-11-23) 51 | --------------------------- 52 | 53 | Features and changes 54 | ~~~~~~~~~~~~~~~~~~~~ 55 | - Add result date and time to traceroute, NTP and SSL renderers 56 | - Add support for specifying measurement tags on measurement creation 57 | - Add option (--go-web) to open measurement URL in browser 58 | - Nicer presentation of 403 errors from the API 59 | - Official supported Python versions changed to 2.7, 3.4, 3.5, 3.6 and 3.7 60 | 61 | Bug Fixes 62 | ~~~~~~~~~ 63 | - Fix cousteau/sagan dependencies 64 | 65 | 66 | 2.2.3 (released 2017-01-17) 67 | --------------------------- 68 | 69 | Bug Fixes 70 | ~~~~~~~~~ 71 | - Fix for distribution issues that prevented the command-line scripts from working 72 | 73 | 2.2.2 (released 2017-10-12) 74 | --------------------------- 75 | 76 | Features and changes 77 | ~~~~~~~~~~~~~~~~~~~~ 78 | - Align various option defaults, minimums and maximums with API reality, including... 79 | - ... allow the set of options necessary for "TCP ping" measurements https://labs.ripe.net/Members/wilhelm/measuring-your-web-server-reachability-with-tcp-ping 80 | - Add compact DNS results renderer 81 | - Fix some unicode output issues 82 | 83 | 2.1 (released 2016-04-21) 84 | --------------------------- 85 | 86 | New Features 87 | ~~~~~~~~~~~~ 88 | - Add a simple NTP renderer 89 | 90 | Changes 91 | ~~~~~~~ 92 | - Use new cousteau (1.4) & sagan(1.2) versions. 93 | 94 | Bug Fixes 95 | ~~~~~~~~~ 96 | - Fix for some unicode problems when using colors 97 | - Fix issue #177, with `gdbm` problem. 98 | 99 | 2.0.2 (released 2016-10-21) 100 | --------------------------- 101 | 102 | New Features 103 | ~~~~~~~~~~~~ 104 | - Add aliases to measurements IDs 105 | - Add --traceroute-show-asns to traceroute renderer 106 | 107 | Bug Fixes 108 | ~~~~~~~~~ 109 | - Stream command was not passing the correct API key. After API became stricter this command started failing. 110 | - Handle missing geometry for probes. 111 | - Fix issues for AS-paths with only 1 probe 112 | - Various fixes for tests 113 | 114 | 2.0.1 (released 2016-04-20) 115 | --------------------------- 116 | 117 | Changes 118 | ~~~~~~~ 119 | - Corrected references in the docs to obsolete command names. 120 | - Fixed broken 2.0.0 egg. 121 | 122 | 123 | 2.0.0 (released 2016-04-20) 124 | --------------------------- 125 | 126 | Changes 127 | ~~~~~~~ 128 | - Renamed and merged some commands for clarity, preserving the old names as deprecated aliases. 129 | - Improved help text and usage output. 130 | - Support for bash auto-completion. 131 | 132 | 133 | 1.2.3 (released 2016-03-08) 134 | --------------------------- 135 | 136 | Changes 137 | ~~~~~~~ 138 | - Usage of newest Cousteau/Sagan library. 139 | - Support of API keys for fetching results on report command. 140 | - Default radius for probes filtering is changed to 15. 141 | - Several changes for supporting Windows. 142 | 143 | 144 | 1.2.2 (released 2016-01-13) 145 | --------------------------- 146 | 147 | New Features 148 | ~~~~~~~~~~~~ 149 | - Cleaner and more consistent implementation of the renderer plugable 150 | architecture. 151 | - Usage of newest Cousteau library. 152 | 153 | 154 | 1.2.1 (released 2015-12-15) 155 | --------------------------- 156 | 157 | Bug Fixes 158 | ~~~~~~~~~ 159 | - Restored some required template files. 160 | 161 | 162 | 1.2.0 (released 2015-12-15) 163 | --------------------------- 164 | 165 | Output Changes 166 | ~~~~~~~~~~~~~~ 167 | - `#119`_: Support HTTP results. 168 | - `#122`_: Allow packagers to set the user agent. 169 | 170 | 171 | 1.1.1 (released 2015-11-25) 172 | --------------------------- 173 | 174 | Output Changes 175 | ~~~~~~~~~~~~~~ 176 | - `#103`_: Removed header from the ``report`` command. 177 | 178 | Bug Fixes 179 | ~~~~~~~~~ 180 | - `#105`_: Measurement report and stream broken on Python3.4. 181 | 182 | 1.1.0 (released 2015-11-12) 183 | --------------------------- 184 | 185 | New features 186 | ~~~~~~~~~~~~ 187 | - Support for the creation of NTP, SSLCert, and HTTP measurements. 188 | - Additional argument in report command to filter results by probe ASN. 189 | - Additional renderer that shows the different destination ASNs and some 190 | additional stats about them. 191 | 192 | Bug Fixes 193 | ~~~~~~~~~ 194 | - Various fixes. 195 | 196 | Changes 197 | ~~~~~~~ 198 | - Better testing. 199 | - Additional documentation. 200 | 201 | 1.0.0 (released 2015-11-02) 202 | --------------------------- 203 | - Initial release. 204 | 205 | .. _#103: https://github.com/RIPE-NCC/ripe-atlas-tools/issues/103 206 | .. _#105: https://github.com/RIPE-NCC/ripe-atlas-tools/issues/105 207 | .. _#119: https://github.com/RIPE-NCC/ripe-atlas-tools/issues/119 208 | .. _#122: https://github.com/RIPE-NCC/ripe-atlas-tools/issues/122 209 | --------------------------------------------------------------------------------