├── amundsen_common ├── py.typed ├── __init__.py ├── log │ ├── __init__.py │ ├── caller_retrieval.py │ ├── auth_caller_retrieval.py │ ├── http_header_caller_retrieval.py │ ├── action_log_model.py │ ├── action_log.py │ └── action_log_callback.py ├── models │ ├── __init__.py │ ├── popular_table.py │ ├── dashboard.py │ ├── user.py │ ├── table.py │ └── index_map.py └── tests │ ├── __init__.py │ └── fixtures.py ├── .github ├── titleLint.yml ├── CODEOWNERS ├── workflows │ ├── license.yml │ ├── pypipublish.yml │ └── pull_request.yml ├── tests.yml ├── stale.yml └── PULL_REQUEST_TEMPLATE.md ├── tests ├── __init__.py ├── unit │ ├── __init__.py │ └── log │ │ ├── __init__.py │ │ ├── test_http_header_caller_retrieval.py │ │ └── test_action_log.py └── tests │ ├── __init__.py │ └── test_fixtures.py ├── NOTICE ├── .dependabot └── config.yml ├── CODE_OF_CONDUCT.md ├── .gitignore ├── requirements.txt ├── Makefile ├── PULL_REQUEST_TEMPLATE.md ├── README.md ├── setup.cfg ├── .pre-commit-config.yaml ├── setup.py └── LICENSE /amundsen_common/py.typed: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /.github/titleLint.yml: -------------------------------------------------------------------------------- 1 | regex: (build|ci|docs|feat|fix|perf|refactor|style|test|chore|other): .* 2 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | -------------------------------------------------------------------------------- /tests/unit/__init__.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | -------------------------------------------------------------------------------- /NOTICE: -------------------------------------------------------------------------------- 1 | amundsencommon 2 | Copyright 2019-2020 Lyft Inc. 3 | 4 | This product includes software developed at Lyft Inc. 5 | -------------------------------------------------------------------------------- /amundsen_common/__init__.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | -------------------------------------------------------------------------------- /tests/tests/__init__.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | -------------------------------------------------------------------------------- /tests/unit/log/__init__.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | -------------------------------------------------------------------------------- /amundsen_common/log/__init__.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | -------------------------------------------------------------------------------- /amundsen_common/models/__init__.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | -------------------------------------------------------------------------------- /amundsen_common/tests/__init__.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | -------------------------------------------------------------------------------- /.dependabot/config.yml: -------------------------------------------------------------------------------- 1 | version: 1 2 | update_configs: 3 | - package_manager: "python" 4 | directory: "/" 5 | update_schedule: "weekly" 6 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | This project is governed by [Amundsen's code of conduct](https://github.com/amundsen-io/amundsen/blob/master/CODE_OF_CONDUCT.md). 2 | All contributors and participants agree to abide by its terms. 3 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *~ 2 | *.pyc 3 | *.pyo 4 | *.pyt 5 | *.pytc 6 | *.egg-info 7 | .*.swp 8 | .DS_Store 9 | build/ 10 | dist/ 11 | venv/ 12 | venv3/ 13 | .cache/ 14 | .idea/ 15 | .vscode/ 16 | .coverage 17 | .mypy_cache 18 | .pytest_cache 19 | 20 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | attrs==19.1.0 2 | flake8==3.7.8 3 | Flask==1.1.1 4 | marshmallow==3.6.0 5 | git+https://www.github.com/hilearn/marshmallow-annotations.git@a7a2dc96932430369bdef36555082df990ed9bef#egg=marshmallow-annotations 6 | mypy==0.761 7 | pytest>=4.6 8 | pytest-cov 9 | pytest-mock 10 | mock 11 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | clean: 2 | find . -name \*.pyc -delete 3 | find . -name __pycache__ -delete 4 | rm -rf dist/ 5 | 6 | test_unit: 7 | python -m pytest tests 8 | python3 -bb -m pytest tests 9 | 10 | 11 | lint: 12 | flake8 . 13 | 14 | .PHONY: mypy 15 | mypy: 16 | mypy . 17 | 18 | 19 | test: test_unit lint mypy 20 | -------------------------------------------------------------------------------- /amundsen_common/log/caller_retrieval.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | 4 | from abc import ABCMeta, abstractmethod 5 | 6 | 7 | class BaseCallerRetriever(metaclass=ABCMeta): 8 | 9 | @abstractmethod 10 | def get_caller(self) -> str: 11 | pass 12 | -------------------------------------------------------------------------------- /amundsen_common/log/auth_caller_retrieval.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | 4 | import getpass 5 | 6 | from flask import current_app as flask_app 7 | 8 | from amundsen_common.log.caller_retrieval import BaseCallerRetriever 9 | 10 | 11 | class AuthCallerRetrieval(BaseCallerRetriever): 12 | def get_caller(self) -> str: 13 | if flask_app.config.get('AUTH_USER_METHOD', None): 14 | return flask_app.config['AUTH_USER_METHOD'](flask_app).email 15 | return getpass.getuser() 16 | -------------------------------------------------------------------------------- /amundsen_common/log/http_header_caller_retrieval.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | 4 | from flask import current_app as flask_app 5 | from flask import request 6 | 7 | from amundsen_common.log.caller_retrieval import BaseCallerRetriever 8 | 9 | CALLER_HEADER_KEY = 'CALLER_HEADER_KEY' 10 | 11 | 12 | class HttpHeaderCallerRetrieval(BaseCallerRetriever): 13 | def get_caller(self) -> str: 14 | header_key = flask_app.config.get(CALLER_HEADER_KEY, 'user-agent') 15 | return request.headers.get(header_key, 'UNKNOWN') 16 | -------------------------------------------------------------------------------- /.github/CODEOWNERS: -------------------------------------------------------------------------------- 1 | # Codeowners file by GitHub 2 | # Reference: https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/about-code-owners 3 | # Each line is a file pattern followed by one or more owners. 4 | # Order is important; the last matching pattern takes the most 5 | # precedence. 6 | 7 | # These owners will be the default owners for everything in 8 | # the repo. Unless a later match takes precedence, 9 | # @amundsen-io/amundsen-committerswill be requested for 10 | # review when someone opens a pull request. 11 | * @amundsen-io/amundsen-committers 12 | 13 | *.py @feng-tao @jinhyukchang @allisonsuarez @dikshathakur3119 @verdan @bolkedebruin @mgorsk1 14 | -------------------------------------------------------------------------------- /.github/workflows/license.yml: -------------------------------------------------------------------------------- 1 | name: license 2 | 3 | on: 4 | push: 5 | branches: 6 | - master 7 | pull_request: 8 | branches: 9 | - master 10 | 11 | jobs: 12 | build: 13 | runs-on: ubuntu-latest 14 | 15 | steps: 16 | - uses: actions/checkout@v2 17 | - name: Set up Golang 18 | uses: actions/setup-go@v2 19 | - name: Install addlicense 20 | run: | 21 | export PATH=${PATH}:`go env GOPATH`/bin 22 | go get -v -u github.com/google/addlicense 23 | - name: Check license 24 | run: | 25 | export PATH=${PATH}:`go env GOPATH`/bin 26 | addlicense -check -l mit -c "Amundsen" $(find $PWD -type f -name '*.py') -------------------------------------------------------------------------------- /amundsen_common/models/popular_table.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | 4 | from typing import Optional 5 | 6 | import attr 7 | from marshmallow_annotations.ext.attrs import AttrsSchema 8 | 9 | 10 | @attr.s(auto_attribs=True, kw_only=True) 11 | class PopularTable: 12 | """ 13 | DEPRECATED. Use TableSummary 14 | """ 15 | database: str = attr.ib() 16 | cluster: str = attr.ib() 17 | schema: str = attr.ib() 18 | name: str = attr.ib() 19 | description: Optional[str] = None 20 | 21 | 22 | class PopularTableSchema(AttrsSchema): 23 | """ 24 | DEPRECATED. Use TableSummary 25 | """ 26 | class Meta: 27 | target = PopularTable 28 | register_as_scheme = True 29 | -------------------------------------------------------------------------------- /.github/tests.yml: -------------------------------------------------------------------------------- 1 | name: Tests 2 | 3 | on: [push] 4 | 5 | jobs: 6 | unit: 7 | name: unit 8 | runs-on: ubuntu-16.04 9 | steps: 10 | - uses: actions/checkout@v1 11 | with: 12 | python-version: 3.7 13 | - name: 'Install dependencies' 14 | run: pip install -r requirements.txt 15 | - name: 'Install project' 16 | run: python setup.py install 17 | - name: 'Run tests' 18 | run: make test_unit 19 | codecov: 20 | name: codecov 21 | runs-on: ubuntu-16.04 22 | needs: unit 23 | steps: 24 | - uses: actions/checkout@v1 25 | with: 26 | python-version: 3.7 27 | - name: 'Install codecov' 28 | run: pip install codecov 29 | - name: 'Push coverage' 30 | run: codecov 31 | -------------------------------------------------------------------------------- /.github/stale.yml: -------------------------------------------------------------------------------- 1 | # Number of days of inactivity before an issue becomes stale 2 | daysUntilStale: 14 3 | # Number of days of inactivity before a stale issue is closed 4 | daysUntilClose: 21 5 | # Issues with these labels will never be considered stale 6 | exemptLabels: 7 | - keep fresh 8 | # Label to use when marking an issue as stale 9 | staleLabel: stale 10 | # Comment to post when marking an issue as stale. Set to `false` to disable 11 | markComment: > 12 | This issue has been automatically marked as stale because it has not had 13 | recent activity. It will be closed if no further activity occurs. 14 | # Comment to post when closing a stale issue. Set to `false` to disable 15 | closeComment: > 16 | This issue has been automatically closed for inactivity. If you still wish to 17 | make these changes, please open a new pull request or reopen this one. 18 | -------------------------------------------------------------------------------- /amundsen_common/models/dashboard.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | 4 | from typing import List, Optional 5 | 6 | import attr 7 | from marshmallow_annotations.ext.attrs import AttrsSchema 8 | 9 | 10 | @attr.s(auto_attribs=True, kw_only=True) 11 | class DashboardSummary: 12 | uri: str = attr.ib() 13 | cluster: str = attr.ib() 14 | group_name: str = attr.ib() 15 | group_url: str = attr.ib() 16 | product: str = attr.ib() 17 | name: str = attr.ib() 18 | url: str = attr.ib() 19 | description: Optional[str] = None 20 | last_successful_run_timestamp: Optional[int] = None 21 | chart_names: Optional[List[str]] = [] 22 | 23 | 24 | class DashboardSummarySchema(AttrsSchema): 25 | class Meta: 26 | target = DashboardSummary 27 | register_as_scheme = True 28 | -------------------------------------------------------------------------------- /tests/unit/log/test_http_header_caller_retrieval.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | 4 | import unittest 5 | 6 | import flask 7 | from mock import patch 8 | 9 | from amundsen_common.log import http_header_caller_retrieval 10 | from amundsen_common.log.http_header_caller_retrieval import HttpHeaderCallerRetrieval 11 | 12 | app = flask.Flask(__name__) 13 | 14 | 15 | class ActionLogTest(unittest.TestCase): 16 | 17 | def test(self) -> None: 18 | with app.test_request_context(), patch.object(http_header_caller_retrieval, 'request') as mock_request: 19 | mock_request.headers.get.return_value = 'foo' 20 | actual = HttpHeaderCallerRetrieval().get_caller() 21 | 22 | self.assertEqual(actual, 'foo') 23 | 24 | 25 | if __name__ == '__main__': 26 | unittest.main() 27 | -------------------------------------------------------------------------------- /.github/workflows/pypipublish.yml: -------------------------------------------------------------------------------- 1 | 2 | name: Build and Deploy 3 | on: 4 | push: 5 | branches: 6 | - master 7 | tags: 8 | - '*' 9 | jobs: 10 | build-and-publish-python-module: 11 | name: Build and publish python module to pypi 12 | runs-on: ubuntu-18.04 13 | steps: 14 | - name: Checkout 15 | uses: actions/checkout@v1 16 | - name: Setup python 3.6 17 | uses: actions/setup-python@v1 18 | with: 19 | python-version: 3.6 20 | - name: Add wheel dependency 21 | run: pip install wheel 22 | - name: Generate dist 23 | run: python setup.py sdist bdist_wheel 24 | - name: Publish to PyPI 25 | if: startsWith(github.event.ref, 'refs/tags') 26 | uses: pypa/gh-action-pypi-publish@master 27 | with: 28 | user: __token__ 29 | password: ${{ secrets.pypi_password }} 30 | -------------------------------------------------------------------------------- /.github/workflows/pull_request.yml: -------------------------------------------------------------------------------- 1 | # This workflow will install Python dependencies, run tests and lint with a single version of Python 2 | # For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions 3 | 4 | on: pull_request 5 | jobs: 6 | pre-commit: 7 | runs-on: ubuntu-18.04 8 | steps: 9 | - name: Checkout 10 | uses: actions/checkout@v1 11 | - name: Setup python 3.6 12 | uses: actions/setup-python@v1 13 | with: 14 | python-version: 3.6 15 | test-unit: 16 | runs-on: ubuntu-18.04 17 | strategy: 18 | matrix: 19 | python-version: ['3.6.x', '3.7.x'] 20 | steps: 21 | - name: Checkout 22 | uses: actions/checkout@v1 23 | - name: Setup python ${{ matrix.python-version }} 24 | uses: actions/setup-python@v1 25 | with: 26 | python-version: ${{ matrix.python-version }} 27 | - name: Install dependencies 28 | run: pip3 install -r requirements.txt && pip3 install codecov 29 | - name: Run python unit tests 30 | run: make test 31 | -------------------------------------------------------------------------------- /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | 16 | 17 | ### Summary of Changes 18 | 19 | _Include a summary of changes then remove this line_ 20 | 21 | ### CheckList 22 | 23 | Make sure you have checked **all** steps below to ensure a timely review. 24 | 25 | - [ ] PR title addresses the issue accurately and concisely. Example: "Updates the version of Flask to v1.0.2" 26 | - In case you are adding a dependency, check if the license complies with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). 27 | - [ ] PR includes a summary of changes. 28 | -------------------------------------------------------------------------------- /PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | ### Summary of Changes 2 | 3 | _Include a summary of changes then remove this line_ 4 | 5 | ### Tests 6 | 7 | _What tests did you add or modify and why? If no tests were added or modified, explain why. Remove this line_ 8 | 9 | ### Documentation 10 | 11 | _What documentation did you add or modify and why? Add any relevant links then remove this line_ 12 | 13 | ### CheckList 14 | Make sure you have checked **all** steps below to ensure a timely review. 15 | - [ ] PR title addresses the issue accurately and concisely. Example: "Updates the version of Flask to v1.0.2" 16 | - In case you are adding a dependency, check if the license complies with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). 17 | - [ ] PR includes a summary of changes. 18 | - [ ] PR adds unit tests, updates existing unit tests, __OR__ documents why no test additions or modifications are needed. 19 | - [ ] In case of new functionality, my PR adds documentation that describes how to use it. 20 | - All the public functions and the classes in the PR contain docstrings that explain what it does 21 | - [ ] PR passes `make test` 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Amundsen Common 2 | [![PyPI version](https://badge.fury.io/py/amundsen-common.svg)](https://badge.fury.io/py/amundsen-common) 3 | [![License](http://img.shields.io/:license-Apache%202-blue.svg)](LICENSE) 4 | [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://img.shields.io/badge/PRs-welcome-brightgreen.svg) 5 | [![Slack Status](https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack&style=social)](https://amundsenworkspace.slack.com/join/shared_invite/enQtNTk2ODQ1NDU1NDI0LTc3MzQyZmM0ZGFjNzg5MzY1MzJlZTg4YjQ4YTU0ZmMxYWU2MmVlMzhhY2MzMTc1MDg0MzRjNTA4MzRkMGE0Nzk) 6 | 7 | Amundsen Common library holds common codes among micro services in Amundsen. 8 | For information about Amundsen and our other services, visit the [main repository](https://github.com/amundsen-io/amundsen). Please also see our instructions for a [quick start](https://github.com/amundsen-io/amundsen/blob/master/docs/installation.md#bootstrap-a-default-version-of-amundsen-using-docker) setup of Amundsen with dummy data, and an [overview of the architecture](https://github.com/amundsen-io/amundsen/blob/master/docs/architecture.md#architecture). 9 | 10 | ## Requirements 11 | - Python >= 3.6 12 | 13 | ## Doc 14 | - https://www.amundsen.io/amundsen/ 15 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [flake8] 2 | format = pylint 3 | exclude = .svc,CVS,.bzr,.hg,.git,__pycache__,venv 4 | max-complexity = 10 5 | max-line-length = 120 6 | 7 | # flake8-tidy-imports rules 8 | banned-modules = 9 | dateutil.parser = Use `ciso8601` instead 10 | flask.ext.restful = Use `flask_restful` 11 | flask.ext.script = Use `flask_script` 12 | flask_restful.reqparse = Use `marshmallow` for request/response validation 13 | haversine = Use `from fast_distance import haversine` 14 | py.test = Use `pytest` 15 | python-s3file = Use `boto` 16 | 17 | [pep8] 18 | max-line-length = 79 19 | 20 | [tool:pytest] 21 | addopts = --cov=amundsen_common --cov-fail-under=70 --cov-report=term-missing:skip-covered --cov-report=xml --cov-report=html -vvv 22 | 23 | [coverage:run] 24 | omit = */models/* 25 | branch = True 26 | 27 | [coverage:xml] 28 | output = build/coverage.xml 29 | 30 | [coverage:html] 31 | directory = build/coverage_html 32 | 33 | [mypy] 34 | check_untyped_defs = true 35 | disallow_any_generics = true 36 | disallow_incomplete_defs = true 37 | disallow_untyped_defs = true 38 | no_implicit_optional = true 39 | 40 | [mypy-marshmallow.*] 41 | ignore_missing_imports = true 42 | 43 | [mypy-marshmallow_annotations.*] 44 | ignore_missing_imports = true 45 | 46 | [mypy-setuptools.*] 47 | ignore_missing_imports = true 48 | 49 | [mypy-tests.*] 50 | disallow_untyped_defs = false 51 | -------------------------------------------------------------------------------- /.pre-commit-config.yaml: -------------------------------------------------------------------------------- 1 | repos: 2 | - repo: https://github.com/pre-commit/pre-commit-hooks 3 | rev: v2.1.0 4 | hooks: 5 | - id: check-docstring-first 6 | - id: check-executables-have-shebangs 7 | - id: check-json 8 | - id: check-merge-conflict 9 | - id: check-yaml 10 | - id: debug-statements 11 | - id: end-of-file-fixer 12 | - id: trailing-whitespace 13 | - repo: https://gitlab.com/pycqa/flake8 14 | rev: 3.7.2 15 | hooks: 16 | - id: flake8 17 | additional_dependencies: 18 | - flake8-bugbear==18.8.0 19 | - flake8-builtins==1.4.1 20 | - flake8-comprehensions==1.4.1 21 | - flake8-tidy-imports==1.1.0 22 | - repo: https://github.com/asottile/yesqa 23 | rev: v0.0.8 24 | hooks: 25 | - id: yesqa 26 | additional_dependencies: 27 | - flake8-bugbear==18.8.0 28 | - flake8-builtins==1.4.1 29 | - flake8-comprehensions==1.4.1 30 | - flake8-tidy-imports==1.1.0 31 | - flake8==3.7.2 32 | - repo: https://github.com/asottile/pyupgrade 33 | rev: v1.11.1 34 | hooks: 35 | - id: pyupgrade 36 | - repo: https://github.com/asottile/add-trailing-comma 37 | rev: v0.7.1 38 | hooks: 39 | - id: add-trailing-comma 40 | - repo: https://github.com/pre-commit/mirrors-mypy 41 | rev: v0.701 42 | hooks: 43 | - id: mypy 44 | -------------------------------------------------------------------------------- /amundsen_common/log/action_log_model.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | 4 | from typing import Any, Optional 5 | 6 | 7 | class ActionLogParams(object): 8 | """ 9 | Holds parameters for Action log 10 | """ 11 | def __init__( 12 | self, *, 13 | command: str, 14 | start_epoch_ms: int, 15 | end_epoch_ms: Optional[int] = 0, 16 | user: str, 17 | host_name: str, 18 | pos_args_json: str, 19 | keyword_args_json: str, 20 | output: Any = None, 21 | error: Optional[Exception] = None 22 | ) -> None: 23 | self.command = command 24 | self.start_epoch_ms = start_epoch_ms 25 | self.end_epoch_ms = end_epoch_ms 26 | self.user = user 27 | self.host_name = host_name 28 | self.pos_args_json = pos_args_json 29 | self.keyword_args_json = keyword_args_json 30 | self.output = output 31 | self.error = error 32 | 33 | def __repr__(self) -> str: 34 | return 'ActionLogParams(command={!r}, start_epoch_ms={!r}, end_epoch_ms={!r}, user={!r}, ' \ 35 | 'host_name={!r}, pos_args_json={!r}, keyword_args_json={!r}, output={!r}, error={!r})'\ 36 | .format( 37 | self.command, 38 | self.start_epoch_ms, 39 | self.end_epoch_ms, 40 | self.user, 41 | self.host_name, 42 | self.pos_args_json, 43 | self.keyword_args_json, 44 | self.output, 45 | self.error, 46 | ) 47 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | 4 | from setuptools import find_packages, setup 5 | 6 | setup( 7 | name='amundsen-common', 8 | version='0.5.8', 9 | description='Common code library for Amundsen', 10 | long_description=open('README.md').read(), 11 | long_description_content_type='text/markdown', 12 | url='https://github.com/amundsen-io/amundsencommon', 13 | maintainer='Amundsen TSC', 14 | maintainer_email='amundsen-tsc@lists.lfai.foundation', 15 | packages=find_packages(exclude=['tests*']), 16 | dependency_links=[], 17 | install_requires=[ 18 | # Packages in here should rarely be pinned. This is because these 19 | # packages (at the specified version) are required for project 20 | # consuming this library. By pinning to a specific version you are the 21 | # number of projects that can consume this or forcing them to 22 | # upgrade/downgrade any dependencies pinned here in their project. 23 | # 24 | # Generally packages listed here are pinned to a major version range. 25 | # 26 | # e.g. 27 | # Python FooBar package for foobaring 28 | # pyfoobar>=1.0, <2.0 29 | # 30 | # This will allow for any consuming projects to use this library as 31 | # long as they have a version of pyfoobar equal to or greater than 1.x 32 | # and less than 2.x installed. 33 | 'flask>=1.0.2', 34 | 'marshmallow>=2.15.3,<=3.6', 35 | ('git+https://www.github.com/hilearn/marshmallow-annotations.git@a7a2dc96932430369bd' 36 | 'ef36555082df990ed9bef#egg=marshmallow-annotations') 37 | ], 38 | python_requires=">=3.6", 39 | package_data={'amundsen_common': ['py.typed']}, 40 | ) 41 | -------------------------------------------------------------------------------- /tests/unit/log/test_action_log.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | 4 | import socket 5 | import unittest 6 | from contextlib import contextmanager 7 | from typing import Generator, Any 8 | 9 | import flask 10 | 11 | from amundsen_common.log import action_log, action_log_callback 12 | from amundsen_common.log.action_log import action_logging, get_epoch_millisec 13 | 14 | app = flask.Flask(__name__) 15 | 16 | 17 | class ActionLogTest(unittest.TestCase): 18 | 19 | def test_metrics_build(self) -> None: 20 | # with patch.object(current_app, 'config'): 21 | with app.test_request_context(): 22 | func_name = 'search' 23 | metrics = action_log._build_metrics(func_name, 'dummy', 777, foo='bar') 24 | 25 | expected = { 26 | 'command': 'search', 27 | 'host_name': socket.gethostname(), 28 | 'pos_args_json': '["dummy", 777]', 29 | 'keyword_args_json': '{"foo": "bar"}', 30 | 'user': 'UNKNOWN', 31 | } 32 | 33 | for k, v in expected.items(): 34 | self.assertEquals(v, metrics.get(k)) 35 | 36 | self.assertTrue(metrics.get('start_epoch_ms') <= get_epoch_millisec()) # type: ignore 37 | 38 | def test_fail_function(self) -> None: 39 | """ 40 | Actual function is failing and fail needs to be propagated. 41 | :return: 42 | """ 43 | with app.test_request_context(), self.assertRaises(NotImplementedError): 44 | fail_func() 45 | 46 | def test_success_function(self) -> None: 47 | """ 48 | Test success function but with failing callback. 49 | In this case, failure should not propagate. 50 | :return: 51 | """ 52 | with app.test_request_context(), fail_action_logger_callback(): 53 | success_func() 54 | 55 | 56 | @contextmanager 57 | def fail_action_logger_callback() -> Generator[Any, Any, Any]: 58 | """ 59 | Adding failing callback and revert it back when closed. 60 | :return: 61 | """ 62 | tmp = action_log_callback.__pre_exec_callbacks[:] 63 | 64 | def fail_callback(_action_callback: Any) -> None: 65 | raise NotImplementedError 66 | 67 | action_log_callback.register_pre_exec_callback(fail_callback) 68 | yield 69 | action_log_callback.__pre_exec_callbacks = tmp 70 | 71 | 72 | @action_logging 73 | def fail_func() -> None: 74 | raise NotImplementedError 75 | 76 | 77 | @action_logging 78 | def success_func() -> None: 79 | pass 80 | 81 | 82 | if __name__ == '__main__': 83 | unittest.main() 84 | -------------------------------------------------------------------------------- /amundsen_common/log/action_log.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | 4 | import functools 5 | 6 | import json 7 | import logging 8 | import socket 9 | from datetime import datetime, timezone, timedelta 10 | 11 | from typing import Any, Dict, Callable 12 | from flask import current_app as flask_app 13 | from amundsen_common.log import action_log_callback 14 | from amundsen_common.log.action_log_model import ActionLogParams 15 | 16 | LOGGER = logging.getLogger(__name__) 17 | EPOCH = datetime(1970, 1, 1, tzinfo=timezone.utc) # use POSIX epoch 18 | 19 | # CONFIG KEY FOR caller_retrieval instance 20 | CALLER_RETRIEVAL_INSTANCE_KEY = 'CALLER_RETRIEVAL_INSTANCE' 21 | 22 | 23 | def action_logging(f: Callable[..., Any]) -> Any: 24 | """ 25 | Decorates function to execute function at the same time triggering action logger callbacks. 26 | It will call action logger callbacks twice, one for pre-execution and the other one for post-execution. 27 | Action logger will be called with ActionLogParams 28 | 29 | :param f: function instance 30 | :return: wrapped function 31 | """ 32 | @functools.wraps(f) 33 | def wrapper( 34 | *args: Any, 35 | **kwargs: Any 36 | ) -> Any: 37 | """ 38 | An wrapper for api functions. It creates ActionLogParams based on the function name, positional arguments, 39 | and keyword arguments. 40 | 41 | :param args: A passthrough positional arguments. 42 | :param kwargs: A passthrough keyword argument 43 | """ 44 | metrics = _build_metrics(f.__name__, *args, **kwargs) 45 | action_log_callback.on_pre_execution(ActionLogParams(**metrics)) 46 | output = None 47 | try: 48 | output = f(*args, **kwargs) 49 | return output 50 | except Exception as e: 51 | metrics['error'] = e 52 | raise 53 | finally: 54 | metrics['end_epoch_ms'] = get_epoch_millisec() 55 | try: 56 | metrics['output'] = json.dumps(output) 57 | except Exception: 58 | metrics['output'] = output 59 | 60 | action_log_callback.on_post_execution(ActionLogParams(**metrics)) 61 | 62 | if LOGGER.isEnabledFor(logging.DEBUG): 63 | LOGGER.debug('action has been logged') 64 | 65 | return wrapper 66 | 67 | 68 | def get_epoch_millisec() -> int: 69 | return (datetime.now(timezone.utc) - EPOCH) // timedelta(milliseconds=1) 70 | 71 | 72 | def _build_metrics( 73 | func_name: str, 74 | *args: Any, 75 | **kwargs: Any 76 | ) -> Dict[str, Any]: 77 | """ 78 | Builds metrics dict from function args 79 | :param func_name: 80 | :param args: 81 | :param kwargs: 82 | :return: Dict that matches ActionLogParams variable 83 | """ 84 | 85 | metrics = { 86 | 'command': kwargs.get('command', func_name), 87 | 'start_epoch_ms': get_epoch_millisec(), 88 | 'host_name': socket.gethostname(), 89 | 'pos_args_json': json.dumps(args), 90 | 'keyword_args_json': json.dumps(kwargs), 91 | } # type: Dict[str, Any] 92 | 93 | caller_retriever = flask_app.config.get(CALLER_RETRIEVAL_INSTANCE_KEY, '') 94 | if caller_retriever: 95 | metrics['user'] = caller_retriever.get_caller() 96 | else: 97 | metrics['user'] = 'UNKNOWN' 98 | 99 | return metrics 100 | -------------------------------------------------------------------------------- /amundsen_common/models/user.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | 4 | from typing import Any, Optional, Dict 5 | 6 | import attr 7 | from marshmallow import ValidationError, validates_schema, pre_load 8 | from marshmallow_annotations.ext.attrs import AttrsSchema 9 | 10 | """ 11 | TODO: Explore all internationalization use cases and 12 | redesign how User handles names 13 | 14 | TODO - Delete pre processing of the Data 15 | Once all of the upstream services provide a complete User object we will no 16 | longer need to supplement the User objects as done in `preprocess_data` 17 | """ 18 | 19 | 20 | @attr.s(auto_attribs=True, kw_only=True) 21 | class User: 22 | # ToDo (Verdan): Make user_id a required field. 23 | # In case if there is only email, id could be email. 24 | # All the transactions and communication will be handled by ID 25 | user_id: Optional[str] = None 26 | email: Optional[str] = None 27 | first_name: Optional[str] = None 28 | last_name: Optional[str] = None 29 | full_name: Optional[str] = None 30 | display_name: Optional[str] = None 31 | is_active: bool = True 32 | github_username: Optional[str] = None 33 | team_name: Optional[str] = None 34 | slack_id: Optional[str] = None 35 | employee_type: Optional[str] = None 36 | manager_fullname: Optional[str] = None 37 | manager_email: Optional[str] = None 38 | manager_id: Optional[str] = None 39 | role_name: Optional[str] = None 40 | profile_url: Optional[str] = None 41 | other_key_values: Optional[Dict[str, str]] = attr.ib(factory=dict) # type: ignore 42 | # TODO: Add frequent_used, bookmarked, & owned resources 43 | 44 | 45 | class UserSchema(AttrsSchema): 46 | class Meta: 47 | target = User 48 | register_as_scheme = True 49 | 50 | # noinspection PyMethodMayBeStatic 51 | def _str_no_value(self, s: Optional[str]) -> bool: 52 | # Returns True if the given string is None or empty 53 | if not s: 54 | return True 55 | if len(s.strip()) == 0: 56 | return True 57 | return False 58 | 59 | @pre_load 60 | def preprocess_data(self, data: Dict[str, Any]) -> Dict[str, Any]: 61 | if self._str_no_value(data.get('user_id')): 62 | data['user_id'] = data.get('email') 63 | 64 | if self._str_no_value(data.get('profile_url')): 65 | data['profile_url'] = '' 66 | if data.get('GET_PROFILE_URL'): 67 | data['profile_url'] = data.get('GET_PROFILE_URL')(data['user_id']) # type: ignore 68 | 69 | first_name = data.get('first_name') 70 | last_name = data.get('last_name') 71 | 72 | if self._str_no_value(data.get('full_name')) and first_name and last_name: 73 | data['full_name'] = f"{first_name} {last_name}" 74 | 75 | if self._str_no_value(data.get('display_name')): 76 | if self._str_no_value(data.get('full_name')): 77 | data['display_name'] = data.get('email') 78 | else: 79 | data['display_name'] = data.get('full_name') 80 | 81 | return data 82 | 83 | @validates_schema 84 | def validate_user(self, data: Dict[str, Any]) -> None: 85 | if self._str_no_value(data.get('display_name')): 86 | raise ValidationError('"display_name", "full_name", or "email" must be provided') 87 | 88 | if self._str_no_value(data.get('user_id')): 89 | raise ValidationError('"user_id" or "email" must be provided') 90 | -------------------------------------------------------------------------------- /amundsen_common/log/action_log_callback.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | 4 | """ 5 | An Action Logger module. Singleton pattern has been applied into this module 6 | so that registered callbacks can be used all through the same python process. 7 | """ 8 | 9 | import logging 10 | import sys 11 | from typing import Callable, List, Any 12 | 13 | from pkg_resources import iter_entry_points 14 | 15 | from amundsen_common.log.action_log_model import ActionLogParams 16 | 17 | LOGGER = logging.getLogger(__name__) 18 | 19 | __pre_exec_callbacks = [] # type: List[Callable[..., Any]] 20 | __post_exec_callbacks = [] # type: List[Callable[..., Any]] 21 | 22 | 23 | def register_pre_exec_callback(action_log_callback: Callable[..., Any]) -> None: 24 | """ 25 | Registers more action_logger function callback for pre-execution. This function callback is expected to be called 26 | with keyword args. For more about the arguments that is being passed to the callback, refer to 27 | amundsen_application.log.action_log_model.ActionLogParams 28 | :param action_logger: An action logger callback function 29 | :return: None 30 | """ 31 | LOGGER.debug("Adding {} to pre execution callback".format(action_log_callback)) 32 | __pre_exec_callbacks.append(action_log_callback) 33 | 34 | 35 | def register_post_exec_callback(action_log_callback: Callable[..., Any]) -> None: 36 | """ 37 | Registers more action_logger function callback for post-execution. This function callback is expected to be 38 | called with keyword args. For more about the arguments that is being passed to the callback, 39 | amundsen_application.log.action_log_model.ActionLogParams 40 | :param action_logger: An action logger callback function 41 | :return: None 42 | """ 43 | LOGGER.debug("Adding {} to post execution callback".format(action_log_callback)) 44 | __post_exec_callbacks.append(action_log_callback) 45 | 46 | 47 | def on_pre_execution(action_log_params: ActionLogParams) -> None: 48 | """ 49 | Calls callbacks before execution. 50 | Note that any exception from callback will be logged but won't be propagated. 51 | :param kwargs: 52 | :return: None 53 | """ 54 | LOGGER.debug("Calling callbacks: {}".format(__pre_exec_callbacks)) 55 | for call_back_function in __pre_exec_callbacks: 56 | try: 57 | call_back_function(action_log_params) 58 | except Exception: 59 | logging.exception('Failed on pre-execution callback using {}'.format(call_back_function)) 60 | 61 | 62 | def on_post_execution(action_log_params: ActionLogParams) -> None: 63 | """ 64 | Calls callbacks after execution. As it's being called after execution, it can capture most of fields in 65 | amundsen_application.log.action_log_model.ActionLogParams. Note that any exception from callback will be logged 66 | but won't be propagated. 67 | :param kwargs: 68 | :return: None 69 | """ 70 | LOGGER.debug("Calling callbacks: {}".format(__post_exec_callbacks)) 71 | for call_back_function in __post_exec_callbacks: 72 | try: 73 | call_back_function(action_log_params) 74 | except Exception: 75 | logging.exception('Failed on post-execution callback using {}'.format(call_back_function)) 76 | 77 | 78 | def logging_action_log(action_log_params: ActionLogParams) -> None: 79 | """ 80 | An action logger callback that just logs the ActionLogParams that it receives. 81 | :param **kwargs keyword arguments 82 | :return: None 83 | """ 84 | if LOGGER.isEnabledFor(logging.DEBUG): 85 | LOGGER.debug('logging_action_log: {}'.format(action_log_params)) 86 | 87 | 88 | def register_action_logs() -> None: 89 | """ 90 | Retrieve declared action log callbacks from entry point where there are two groups that can be registered: 91 | 1. "action_log.post_exec.plugin": callback for pre-execution 92 | 2. "action_log.pre_exec.plugin": callback for post-execution 93 | :return: None 94 | """ 95 | for entry_point in iter_entry_points(group='action_log.post_exec.plugin', name=None): 96 | print('Registering post_exec action_log entry_point: {}'.format(entry_point), file=sys.stderr) 97 | register_post_exec_callback(entry_point.load()) 98 | 99 | for entry_point in iter_entry_points(group='action_log.pre_exec.plugin', name=None): 100 | print('Registering pre_exec action_log entry_point: {}'.format(entry_point), file=sys.stderr) 101 | register_pre_exec_callback(entry_point.load()) 102 | 103 | 104 | register_action_logs() 105 | -------------------------------------------------------------------------------- /tests/tests/test_fixtures.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | 4 | import unittest 5 | 6 | from amundsen_common.tests.fixtures import (next_application, next_col_type, 7 | next_columns, next_database, 8 | next_description, 9 | next_description_source, 10 | next_descriptions, next_int, next_item, 11 | next_range, next_string, next_table, 12 | next_tag, next_tags, next_user) 13 | from amundsen_common.models.table import Column, ProgrammaticDescription, Stat 14 | 15 | 16 | class TestFixtures(unittest.TestCase): 17 | # tests are numbered to ensure they execute in order 18 | def test_00_next_int(self) -> None: 19 | self.assertEqual(1000, next_int()) 20 | 21 | def test_01_next_string(self) -> None: 22 | self.assertEqual('nopqrstuvw001011', next_string()) 23 | 24 | def test_02_next_string(self) -> None: 25 | self.assertEqual('foo_yzabcdefgh001022', next_string(prefix='foo_')) 26 | 27 | def test_03_next_string(self) -> None: 28 | self.assertEqual('jklm001027', next_string(length=4)) 29 | 30 | def test_04_next_string(self) -> None: 31 | self.assertEqual('bar_opqr001032', next_string(prefix='bar_', length=4)) 32 | 33 | def test_05_next_range(self) -> None: 34 | self.assertEqual(3, len(next_range())) 35 | 36 | def test_06_next_item(self) -> None: 37 | self.assertEqual('c', next_item(items=['a', 'b', 'c'])) 38 | 39 | def test_07_next_database(self) -> None: 40 | self.assertEqual('database2', next_database()) 41 | 42 | def test_08_next_application(self) -> None: 43 | app = next_application() 44 | self.assertEqual('Apwxyzabcd001044', app.name) 45 | self.assertEqual('apwxyzabcd001044', app.id) 46 | self.assertEqual('https://apwxyzabcd001044.example.com', app.application_url) 47 | 48 | def test_09_next_application(self) -> None: 49 | app = next_application(application_id='foo') 50 | self.assertEqual('Foo', app.name) 51 | self.assertEqual('foo', app.id) 52 | self.assertEqual('https://foo.example.com', app.application_url) 53 | 54 | def test_10_next_tag(self) -> None: 55 | tag = next_tag() 56 | self.assertEqual('tafghijklm001053', tag.tag_name) 57 | self.assertEqual('default', tag.tag_type) 58 | 59 | def test_11_next_tags(self) -> None: 60 | tags = next_tags() 61 | self.assertEqual(4, len(tags)) 62 | self.assertEqual(['tahijklmno001081', 63 | 'tapqrstuvw001063', 64 | 'taqrstuvwx001090', 65 | 'tayzabcdef001072'], [tag.tag_name for tag in tags]) 66 | 67 | def test_12_next_description_source(self) -> None: 68 | self.assertEqual('dezabcdefg001099', next_description_source()) 69 | 70 | def test_13_next_description(self) -> None: 71 | self.assertEqual(ProgrammaticDescription(text='ijklmnopqrstuvwxyzab001120', source='dedefghijk001129'), 72 | next_description()) 73 | 74 | def test_14_next_col_type(self) -> None: 75 | self.assertEqual('varchar', next_col_type()) 76 | 77 | def test_15_just_execute_next_columns(self) -> None: 78 | columns = next_columns(table_key='not_important') 79 | self.assertEqual(1, len(columns)) 80 | self.assertEqual([Column(name='coopqrstuv001140', key='not_important/coopqrstuv001140', 81 | description='coopqrstuv001140 description', col_type='int', 82 | sort_order=0, stats=[Stat(stat_type='num_rows', stat_val='114200', 83 | start_epoch=None, end_epoch=None)]) 84 | ], columns) 85 | 86 | def test_16_just_execute_next_descriptions(self) -> None: 87 | descs = next_descriptions() 88 | self.assertEqual(3, len(descs)) 89 | self.assertEqual([ 90 | ProgrammaticDescription(source='dedefghijk001233', text='ijklmnopqrstuvwxyzab001224'), 91 | ProgrammaticDescription(source='devwxyzabc001173', text='abcdefghijklmnopqrst001164'), 92 | ProgrammaticDescription(source='dezabcdefg001203', text='efghijklmnopqrstuvwx001194')], descs) 93 | 94 | def test_17_just_execute_next_table(self) -> None: 95 | table = next_table() 96 | self.assertEqual(2, len(table.columns)) 97 | self.assertEqual('tbnopqrstu001243', table.name) 98 | self.assertEqual('database1://clwxyzabcd001252.scfghijklm001261/tbnopqrstu001243', table.key) 99 | 100 | def test_18_next_user(self) -> None: 101 | user = next_user() 102 | self.assertEqual('Jklmno', user.last_name) 103 | self.assertEqual('Bob', user.first_name) 104 | self.assertEqual('usqrstuvwx001350', user.user_id) 105 | self.assertEqual('usqrstuvwx001350@example.com', user.email) 106 | self.assertEqual(True, user.is_active) 107 | -------------------------------------------------------------------------------- /amundsen_common/models/table.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | 4 | from typing import List, Optional 5 | 6 | import attr 7 | 8 | from amundsen_common.models.user import User 9 | from marshmallow_annotations.ext.attrs import AttrsSchema 10 | 11 | 12 | @attr.s(auto_attribs=True, kw_only=True) 13 | class Reader: 14 | user: User 15 | read_count: int 16 | 17 | 18 | class ReaderSchema(AttrsSchema): 19 | class Meta: 20 | target = Reader 21 | register_as_scheme = True 22 | 23 | 24 | @attr.s(auto_attribs=True, kw_only=True) 25 | class Tag: 26 | tag_type: str 27 | tag_name: str 28 | 29 | 30 | class TagSchema(AttrsSchema): 31 | class Meta: 32 | target = Tag 33 | register_as_scheme = True 34 | 35 | 36 | @attr.s(auto_attribs=True, kw_only=True) 37 | class Badge: 38 | badge_name: str = attr.ib() 39 | category: str = attr.ib() 40 | 41 | 42 | class BadgeSchema(AttrsSchema): 43 | class Meta: 44 | target = Badge 45 | register_as_scheme = True 46 | 47 | 48 | @attr.s(auto_attribs=True, kw_only=True) 49 | class Watermark: 50 | watermark_type: Optional[str] = None 51 | partition_key: Optional[str] = None 52 | partition_value: Optional[str] = None 53 | create_time: Optional[str] = None 54 | 55 | 56 | class WatermarkSchema(AttrsSchema): 57 | class Meta: 58 | target = Watermark 59 | register_as_scheme = True 60 | 61 | 62 | @attr.s(auto_attribs=True, kw_only=True) 63 | class Stat: 64 | stat_type: str 65 | stat_val: Optional[str] = None 66 | start_epoch: Optional[int] = None 67 | end_epoch: Optional[int] = None 68 | 69 | 70 | class StatSchema(AttrsSchema): 71 | class Meta: 72 | target = Stat 73 | register_as_scheme = True 74 | 75 | 76 | @attr.s(auto_attribs=True, kw_only=True) 77 | class Column: 78 | name: str 79 | key: Optional[str] = None 80 | description: Optional[str] = None 81 | col_type: str 82 | sort_order: int 83 | stats: List[Stat] = [] 84 | badges: Optional[List[Badge]] = [] 85 | 86 | 87 | class ColumnSchema(AttrsSchema): 88 | class Meta: 89 | target = Column 90 | register_as_scheme = True 91 | 92 | 93 | @attr.s(auto_attribs=True, kw_only=True) 94 | class Application: 95 | application_url: Optional[str] = None 96 | description: Optional[str] = None 97 | id: str 98 | name: Optional[str] = None 99 | kind: Optional[str] = None 100 | 101 | 102 | class ApplicationSchema(AttrsSchema): 103 | class Meta: 104 | target = Application 105 | register_as_scheme = True 106 | 107 | 108 | @attr.s(auto_attribs=True, kw_only=True) 109 | class Source: 110 | source_type: str 111 | source: str 112 | 113 | 114 | class SourceSchema(AttrsSchema): 115 | class Meta: 116 | target = Source 117 | register_as_scheme = True 118 | 119 | 120 | @attr.s(auto_attribs=True, kw_only=True) 121 | class ResourceReport: 122 | name: str 123 | url: str 124 | 125 | 126 | class ResourceReportSchema(AttrsSchema): 127 | class Meta: 128 | target = ResourceReport 129 | register_as_scheme = True 130 | 131 | 132 | # this is a temporary hack to satisfy mypy. Once https://github.com/python/mypy/issues/6136 is resolved, use 133 | # `attr.converters.default_if_none(default=False)` 134 | def default_if_none(arg: Optional[bool]) -> bool: 135 | return arg or False 136 | 137 | 138 | @attr.s(auto_attribs=True, kw_only=True) 139 | class ProgrammaticDescription: 140 | source: str 141 | text: str 142 | 143 | 144 | class ProgrammaticDescriptionSchema(AttrsSchema): 145 | class Meta: 146 | target = ProgrammaticDescription 147 | register_as_scheme = True 148 | 149 | 150 | @attr.s(auto_attribs=True, kw_only=True) 151 | class Table: 152 | database: str 153 | cluster: str 154 | schema: str 155 | name: str 156 | key: Optional[str] = None 157 | tags: List[Tag] = [] 158 | badges: List[Badge] = [] 159 | table_readers: List[Reader] = [] 160 | description: Optional[str] = None 161 | columns: List[Column] 162 | owners: List[User] = [] 163 | watermarks: List[Watermark] = [] 164 | table_writer: Optional[Application] = None 165 | resource_reports: Optional[List[ResourceReport]] = None 166 | last_updated_timestamp: Optional[int] = None 167 | source: Optional[Source] = None 168 | is_view: Optional[bool] = attr.ib(default=None, converter=default_if_none) 169 | programmatic_descriptions: List[ProgrammaticDescription] = [] 170 | 171 | 172 | class TableSchema(AttrsSchema): 173 | class Meta: 174 | target = Table 175 | register_as_scheme = True 176 | 177 | 178 | @attr.s(auto_attribs=True, kw_only=True) 179 | class TableSummary: 180 | database: str = attr.ib() 181 | cluster: str = attr.ib() 182 | schema: str = attr.ib() 183 | name: str = attr.ib() 184 | description: Optional[str] = attr.ib(default=None) 185 | schema_description: Optional[str] = attr.ib(default=None) 186 | 187 | 188 | class TableSummarySchema(AttrsSchema): 189 | class Meta: 190 | target = TableSummary 191 | register_as_scheme = True 192 | -------------------------------------------------------------------------------- /amundsen_common/models/index_map.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | 4 | import textwrap 5 | 6 | # Specifying default mapping for elasticsearch index 7 | # Documentation: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html 8 | # Setting type to "text" for all fields that would be used in search 9 | # Using Simple Analyzer to convert all text into search terms 10 | # https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-simple-analyzer.html 11 | # Standard Analyzer is used for all text fields that don't explicitly specify an analyzer 12 | # https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-standard-analyzer.html 13 | TABLE_INDEX_MAP = textwrap.dedent( 14 | """ 15 | { 16 | "mappings":{ 17 | "table":{ 18 | "properties": { 19 | "name": { 20 | "type":"text", 21 | "analyzer": "simple", 22 | "fields": { 23 | "raw": { 24 | "type": "keyword" 25 | } 26 | } 27 | }, 28 | "schema": { 29 | "type":"text", 30 | "analyzer": "simple", 31 | "fields": { 32 | "raw": { 33 | "type": "keyword" 34 | } 35 | } 36 | }, 37 | "display_name": { 38 | "type": "keyword" 39 | }, 40 | "last_updated_timestamp": { 41 | "type": "date", 42 | "format": "epoch_second" 43 | }, 44 | "description": { 45 | "type": "text", 46 | "analyzer": "simple" 47 | }, 48 | "column_names": { 49 | "type":"text", 50 | "analyzer": "simple", 51 | "fields": { 52 | "raw": { 53 | "type": "keyword" 54 | } 55 | } 56 | }, 57 | "column_descriptions": { 58 | "type": "text", 59 | "analyzer": "simple" 60 | }, 61 | "tags": { 62 | "type": "keyword" 63 | }, 64 | "badges": { 65 | "type": "keyword" 66 | }, 67 | "cluster": { 68 | "type": "text", 69 | "analyzer": "simple", 70 | "fields": { 71 | "raw": { 72 | "type": "keyword" 73 | } 74 | } 75 | }, 76 | "database": { 77 | "type": "text", 78 | "analyzer": "simple", 79 | "fields": { 80 | "raw": { 81 | "type": "keyword" 82 | } 83 | } 84 | }, 85 | "key": { 86 | "type": "keyword" 87 | }, 88 | "total_usage":{ 89 | "type": "long" 90 | }, 91 | "unique_usage": { 92 | "type": "long" 93 | } 94 | } 95 | } 96 | } 97 | } 98 | """ 99 | ) 100 | 101 | DASHBOARD_ELASTICSEARCH_INDEX_MAPPING = textwrap.dedent( 102 | """ 103 | { 104 | "settings": { 105 | "analysis": { 106 | "normalizer": { 107 | "lowercase_normalizer": { 108 | "type": "custom", 109 | "char_filter": [], 110 | "filter": ["lowercase", "asciifolding"] 111 | } 112 | } 113 | } 114 | }, 115 | "mappings":{ 116 | "dashboard":{ 117 | "properties": { 118 | "group_name": { 119 | "type":"text", 120 | "analyzer": "simple", 121 | "fields": { 122 | "raw": { 123 | "type": "keyword", 124 | "normalizer": "lowercase_normalizer" 125 | } 126 | } 127 | }, 128 | "name": { 129 | "type":"text", 130 | "analyzer": "simple", 131 | "fields": { 132 | "raw": { 133 | "type": "keyword", 134 | "normalizer": "lowercase_normalizer" 135 | } 136 | } 137 | }, 138 | "description": { 139 | "type":"text", 140 | "analyzer": "simple", 141 | "fields": { 142 | "raw": { 143 | "type": "keyword" 144 | } 145 | } 146 | }, 147 | "group_description": { 148 | "type":"text", 149 | "analyzer": "simple", 150 | "fields": { 151 | "raw": { 152 | "type": "keyword" 153 | } 154 | } 155 | }, 156 | "query_names": { 157 | "type":"text", 158 | "analyzer": "simple", 159 | "fields": { 160 | "raw": { 161 | "type": "keyword" 162 | } 163 | } 164 | }, 165 | "chart_names": { 166 | "type":"text", 167 | "analyzer": "simple", 168 | "fields": { 169 | "raw": { 170 | "type": "keyword" 171 | } 172 | } 173 | }, 174 | "tags": { 175 | "type": "keyword" 176 | }, 177 | "badges": { 178 | "type": "keyword" 179 | } 180 | } 181 | } 182 | } 183 | } 184 | """ 185 | ) 186 | 187 | USER_INDEX_MAP = textwrap.dedent(""" 188 | { 189 | "mappings":{ 190 | "user":{ 191 | "properties": { 192 | "email": { 193 | "type":"text", 194 | "analyzer": "simple", 195 | "fields": { 196 | "raw": { 197 | "type": "keyword" 198 | } 199 | } 200 | }, 201 | "first_name": { 202 | "type":"text", 203 | "analyzer": "simple", 204 | "fields": { 205 | "raw": { 206 | "type": "keyword" 207 | } 208 | } 209 | }, 210 | "last_name": { 211 | "type":"text", 212 | "analyzer": "simple", 213 | "fields": { 214 | "raw": { 215 | "type": "keyword" 216 | } 217 | } 218 | }, 219 | "name": { 220 | "type":"text", 221 | "analyzer": "simple", 222 | "fields": { 223 | "raw": { 224 | "type": "keyword" 225 | } 226 | } 227 | }, 228 | "total_read":{ 229 | "type": "long" 230 | }, 231 | "total_own": { 232 | "type": "long" 233 | }, 234 | "total_follow": { 235 | "type": "long" 236 | } 237 | } 238 | } 239 | } 240 | }""") 241 | -------------------------------------------------------------------------------- /amundsen_common/tests/fixtures.py: -------------------------------------------------------------------------------- 1 | # Copyright Contributors to the Amundsen project. 2 | # SPDX-License-Identifier: Apache-2.0 3 | 4 | import string 5 | from typing import Any, List, Optional 6 | 7 | from amundsen_common.models.table import (Application, Column, 8 | ProgrammaticDescription, Stat, Table, 9 | Tag) 10 | from amundsen_common.models.user import User 11 | 12 | 13 | class Fixtures: 14 | """ 15 | These fixtures are useful for creating test objects. For an example usage, check out tests/tests/test_fixtures.py 16 | """ 17 | counter = 1000 18 | 19 | @staticmethod 20 | def next_int() -> int: 21 | i = Fixtures.counter 22 | Fixtures.counter += 1 23 | return i 24 | 25 | @staticmethod 26 | def next_string(*, prefix: str = '', length: int = 10) -> str: 27 | astr: str = prefix + \ 28 | ''.join(Fixtures.next_item(items=list(string.ascii_lowercase)) for _ in range(length)) + \ 29 | ('%06d' % Fixtures.next_int()) 30 | return astr 31 | 32 | @staticmethod 33 | def next_range() -> range: 34 | return range(0, Fixtures.next_int() % 5) 35 | 36 | @staticmethod 37 | def next_item(*, items: List[Any]) -> Any: 38 | return items[Fixtures.next_int() % len(items)] 39 | 40 | @staticmethod 41 | def next_database() -> str: 42 | return Fixtures.next_item(items=list(["database1", "database2"])) 43 | 44 | @staticmethod 45 | def next_application(*, application_id: Optional[str] = None) -> Application: 46 | if not application_id: 47 | application_id = Fixtures.next_string(prefix='ap', length=8) 48 | application = Application(application_url=f'https://{application_id}.example.com', 49 | description=f'{application_id} description', 50 | name=application_id.capitalize(), 51 | id=application_id) 52 | return application 53 | 54 | @staticmethod 55 | def next_tag(*, tag_name: Optional[str] = None) -> Tag: 56 | if not tag_name: 57 | tag_name = Fixtures.next_string(prefix='ta', length=8) 58 | return Tag(tag_name=tag_name, tag_type='default') 59 | 60 | @staticmethod 61 | def next_tags() -> List[Tag]: 62 | return sorted([Fixtures.next_tag() for _ in Fixtures.next_range()]) 63 | 64 | @staticmethod 65 | def next_description_source() -> str: 66 | return Fixtures.next_string(prefix='de', length=8) 67 | 68 | @staticmethod 69 | def next_description(*, text: Optional[str] = None, source: Optional[str] = None) -> ProgrammaticDescription: 70 | if not text: 71 | text = Fixtures.next_string(length=20) 72 | if not source: 73 | source = Fixtures.next_description_source() 74 | return ProgrammaticDescription(text=text, source=source) 75 | 76 | @staticmethod 77 | def next_col_type() -> str: 78 | return Fixtures.next_item(items=['varchar', 'int', 'blob', 'timestamp', 'datetime']) 79 | 80 | @staticmethod 81 | def next_column(*, 82 | table_key: str, 83 | sort_order: int, 84 | name: Optional[str] = None) -> Column: 85 | if not name: 86 | name = Fixtures.next_string(prefix='co', length=8) 87 | 88 | return Column(name=name, 89 | description=f'{name} description', 90 | col_type=Fixtures.next_col_type(), 91 | key=f'{table_key}/{name}', 92 | sort_order=sort_order, 93 | stats=[Stat(stat_type='num_rows', 94 | stat_val=f'{Fixtures.next_int() * 100}', 95 | start_epoch=None, 96 | end_epoch=None)]) 97 | 98 | @staticmethod 99 | def next_columns(*, 100 | table_key: str, 101 | randomize_pii: bool = False, 102 | randomize_data_subject: bool = False) -> List[Column]: 103 | return [Fixtures.next_column(table_key=table_key, 104 | sort_order=i 105 | ) for i in Fixtures.next_range()] 106 | 107 | @staticmethod 108 | def next_descriptions() -> List[ProgrammaticDescription]: 109 | return sorted([Fixtures.next_description() for _ in Fixtures.next_range()]) 110 | 111 | @staticmethod 112 | def next_table(table: Optional[str] = None, 113 | cluster: Optional[str] = None, 114 | schema: Optional[str] = None, 115 | database: Optional[str] = None, 116 | tags: Optional[List[Tag]] = None, 117 | application: Optional[Application] = None) -> Table: 118 | """ 119 | Returns a table for testing in the test_database 120 | """ 121 | if not database: 122 | database = Fixtures.next_database() 123 | 124 | if not table: 125 | table = Fixtures.next_string(prefix='tb', length=8) 126 | 127 | if not cluster: 128 | cluster = Fixtures.next_string(prefix='cl', length=8) 129 | 130 | if not schema: 131 | schema = Fixtures.next_string(prefix='sc', length=8) 132 | 133 | if not tags: 134 | tags = Fixtures.next_tags() 135 | 136 | table_key: str = f'{database}://{cluster}.{schema}/{table}' 137 | # TODO: add owners, watermarks, last_udpated_timestamp, source 138 | return Table(database=database, 139 | cluster=cluster, 140 | schema=schema, 141 | name=table, 142 | key=table_key, 143 | tags=tags, 144 | table_writer=application, 145 | table_readers=[], 146 | description=f'{table} description', 147 | programmatic_descriptions=Fixtures.next_descriptions(), 148 | columns=Fixtures.next_columns(table_key=table_key), 149 | is_view=False 150 | ) 151 | 152 | @staticmethod 153 | def next_user(*, user_id: Optional[str] = None, is_active: bool = True) -> User: 154 | last_name = ''.join(Fixtures.next_item(items=list(string.ascii_lowercase)) for _ in range(6)).capitalize() 155 | first_name = Fixtures.next_item(items=['alice', 'bob', 'carol', 'dan']).capitalize() 156 | if not user_id: 157 | user_id = Fixtures.next_string(prefix='us', length=8) 158 | return User(user_id=user_id, 159 | email=f'{user_id}@example.com', 160 | is_active=is_active, 161 | first_name=first_name, 162 | last_name=last_name, 163 | full_name=f'{first_name} {last_name}') 164 | 165 | 166 | def next_application(**kwargs: Any) -> Application: 167 | return Fixtures.next_application(**kwargs) 168 | 169 | 170 | def next_int() -> int: 171 | return Fixtures.next_int() 172 | 173 | 174 | def next_string(**kwargs: Any) -> str: 175 | return Fixtures.next_string(**kwargs) 176 | 177 | 178 | def next_range() -> range: 179 | return Fixtures.next_range() 180 | 181 | 182 | def next_item(**kwargs: Any) -> Any: 183 | return Fixtures.next_item(**kwargs) 184 | 185 | 186 | def next_database() -> str: 187 | return Fixtures.next_database() 188 | 189 | 190 | def next_tag(**kwargs: Any) -> Tag: 191 | return Fixtures.next_tag(**kwargs) 192 | 193 | 194 | def next_tags() -> List[Tag]: 195 | return Fixtures.next_tags() 196 | 197 | 198 | def next_description_source() -> str: 199 | return Fixtures.next_description_source() 200 | 201 | 202 | def next_description(**kwargs: Any) -> ProgrammaticDescription: 203 | return Fixtures.next_description(**kwargs) 204 | 205 | 206 | def next_col_type() -> str: 207 | return Fixtures.next_col_type() 208 | 209 | 210 | def next_column(**kwargs: Any) -> Column: 211 | return Fixtures.next_column(**kwargs) 212 | 213 | 214 | def next_columns(**kwargs: Any) -> List[Column]: 215 | return Fixtures.next_columns(**kwargs) 216 | 217 | 218 | def next_descriptions() -> List[ProgrammaticDescription]: 219 | return Fixtures.next_descriptions() 220 | 221 | 222 | def next_table(**kwargs: Any) -> Table: 223 | return Fixtures.next_table(**kwargs) 224 | 225 | 226 | def next_user(**kwargs: Any) -> User: 227 | return Fixtures.next_user(**kwargs) 228 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | 2 | Apache License 3 | Version 2.0, January 2004 4 | http://www.apache.org/licenses/ 5 | 6 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 7 | 8 | 1. Definitions. 9 | 10 | "License" shall mean the terms and conditions for use, reproduction, 11 | and distribution as defined by Sections 1 through 9 of this document. 12 | 13 | "Licensor" shall mean the copyright owner or entity authorized by 14 | the copyright owner that is granting the License. 15 | 16 | "Legal Entity" shall mean the union of the acting entity and all 17 | other entities that control, are controlled by, or are under common 18 | control with that entity. For the purposes of this definition, 19 | "control" means (i) the power, direct or indirect, to cause the 20 | direction or management of such entity, whether by contract or 21 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 22 | outstanding shares, or (iii) beneficial ownership of such entity. 23 | 24 | "You" (or "Your") shall mean an individual or Legal Entity 25 | exercising permissions granted by this License. 26 | 27 | "Source" form shall mean the preferred form for making modifications, 28 | including but not limited to software source code, documentation 29 | source, and configuration files. 30 | 31 | "Object" form shall mean any form resulting from mechanical 32 | transformation or translation of a Source form, including but 33 | not limited to compiled object code, generated documentation, 34 | and conversions to other media types. 35 | 36 | "Work" shall mean the work of authorship, whether in Source or 37 | Object form, made available under the License, as indicated by a 38 | copyright notice that is included in or attached to the work 39 | (an example is provided in the Appendix below). 40 | 41 | "Derivative Works" shall mean any work, whether in Source or Object 42 | form, that is based on (or derived from) the Work and for which the 43 | editorial revisions, annotations, elaborations, or other modifications 44 | represent, as a whole, an original work of authorship. For the purposes 45 | of this License, Derivative Works shall not include works that remain 46 | separable from, or merely link (or bind by name) to the interfaces of, 47 | the Work and Derivative Works thereof. 48 | 49 | "Contribution" shall mean any work of authorship, including 50 | the original version of the Work and any modifications or additions 51 | to that Work or Derivative Works thereof, that is intentionally 52 | submitted to Licensor for inclusion in the Work by the copyright owner 53 | or by an individual or Legal Entity authorized to submit on behalf of 54 | the copyright owner. For the purposes of this definition, "submitted" 55 | means any form of electronic, verbal, or written communication sent 56 | to the Licensor or its representatives, including but not limited to 57 | communication on electronic mailing lists, source code control systems, 58 | and issue tracking systems that are managed by, or on behalf of, the 59 | Licensor for the purpose of discussing and improving the Work, but 60 | excluding communication that is conspicuously marked or otherwise 61 | designated in writing by the copyright owner as "Not a Contribution." 62 | 63 | "Contributor" shall mean Licensor and any individual or Legal Entity 64 | on behalf of whom a Contribution has been received by Licensor and 65 | subsequently incorporated within the Work. 66 | 67 | 2. Grant of Copyright License. Subject to the terms and conditions of 68 | this License, each Contributor hereby grants to You a perpetual, 69 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 70 | copyright license to reproduce, prepare Derivative Works of, 71 | publicly display, publicly perform, sublicense, and distribute the 72 | Work and such Derivative Works in Source or Object form. 73 | 74 | 3. Grant of Patent License. Subject to the terms and conditions of 75 | this License, each Contributor hereby grants to You a perpetual, 76 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 77 | (except as stated in this section) patent license to make, have made, 78 | use, offer to sell, sell, import, and otherwise transfer the Work, 79 | where such license applies only to those patent claims licensable 80 | by such Contributor that are necessarily infringed by their 81 | Contribution(s) alone or by combination of their Contribution(s) 82 | with the Work to which such Contribution(s) was submitted. If You 83 | institute patent litigation against any entity (including a 84 | cross-claim or counterclaim in a lawsuit) alleging that the Work 85 | or a Contribution incorporated within the Work constitutes direct 86 | or contributory patent infringement, then any patent licenses 87 | granted to You under this License for that Work shall terminate 88 | as of the date such litigation is filed. 89 | 90 | 4. Redistribution. You may reproduce and distribute copies of the 91 | Work or Derivative Works thereof in any medium, with or without 92 | modifications, and in Source or Object form, provided that You 93 | meet the following conditions: 94 | 95 | (a) You must give any other recipients of the Work or 96 | Derivative Works a copy of this License; and 97 | 98 | (b) You must cause any modified files to carry prominent notices 99 | stating that You changed the files; and 100 | 101 | (c) You must retain, in the Source form of any Derivative Works 102 | that You distribute, all copyright, patent, trademark, and 103 | attribution notices from the Source form of the Work, 104 | excluding those notices that do not pertain to any part of 105 | the Derivative Works; and 106 | 107 | (d) If the Work includes a "NOTICE" text file as part of its 108 | distribution, then any Derivative Works that You distribute must 109 | include a readable copy of the attribution notices contained 110 | within such NOTICE file, excluding those notices that do not 111 | pertain to any part of the Derivative Works, in at least one 112 | of the following places: within a NOTICE text file distributed 113 | as part of the Derivative Works; within the Source form or 114 | documentation, if provided along with the Derivative Works; or, 115 | within a display generated by the Derivative Works, if and 116 | wherever such third-party notices normally appear. The contents 117 | of the NOTICE file are for informational purposes only and 118 | do not modify the License. You may add Your own attribution 119 | notices within Derivative Works that You distribute, alongside 120 | or as an addendum to the NOTICE text from the Work, provided 121 | that such additional attribution notices cannot be construed 122 | as modifying the License. 123 | 124 | You may add Your own copyright statement to Your modifications and 125 | may provide additional or different license terms and conditions 126 | for use, reproduction, or distribution of Your modifications, or 127 | for any such Derivative Works as a whole, provided Your use, 128 | reproduction, and distribution of the Work otherwise complies with 129 | the conditions stated in this License. 130 | 131 | 5. Submission of Contributions. Unless You explicitly state otherwise, 132 | any Contribution intentionally submitted for inclusion in the Work 133 | by You to the Licensor shall be under the terms and conditions of 134 | this License, without any additional terms or conditions. 135 | Notwithstanding the above, nothing herein shall supersede or modify 136 | the terms of any separate license agreement you may have executed 137 | with Licensor regarding such Contributions. 138 | 139 | 6. Trademarks. This License does not grant permission to use the trade 140 | names, trademarks, service marks, or product names of the Licensor, 141 | except as required for reasonable and customary use in describing the 142 | origin of the Work and reproducing the content of the NOTICE file. 143 | 144 | 7. Disclaimer of Warranty. Unless required by applicable law or 145 | agreed to in writing, Licensor provides the Work (and each 146 | Contributor provides its Contributions) on an "AS IS" BASIS, 147 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 148 | implied, including, without limitation, any warranties or conditions 149 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 150 | PARTICULAR PURPOSE. You are solely responsible for determining the 151 | appropriateness of using or redistributing the Work and assume any 152 | risks associated with Your exercise of permissions under this License. 153 | 154 | 8. Limitation of Liability. In no event and under no legal theory, 155 | whether in tort (including negligence), contract, or otherwise, 156 | unless required by applicable law (such as deliberate and grossly 157 | negligent acts) or agreed to in writing, shall any Contributor be 158 | liable to You for damages, including any direct, indirect, special, 159 | incidental, or consequential damages of any character arising as a 160 | result of this License or out of the use or inability to use the 161 | Work (including but not limited to damages for loss of goodwill, 162 | work stoppage, computer failure or malfunction, or any and all 163 | other commercial damages or losses), even if such Contributor 164 | has been advised of the possibility of such damages. 165 | 166 | 9. Accepting Warranty or Additional Liability. While redistributing 167 | the Work or Derivative Works thereof, You may choose to offer, 168 | and charge a fee for, acceptance of support, warranty, indemnity, 169 | or other liability obligations and/or rights consistent with this 170 | License. However, in accepting such obligations, You may act only 171 | on Your own behalf and on Your sole responsibility, not on behalf 172 | of any other Contributor, and only if You agree to indemnify, 173 | defend, and hold each Contributor harmless for any liability 174 | incurred by, or claims asserted against, such Contributor by reason 175 | of your accepting any such warranty or additional liability. 176 | 177 | END OF TERMS AND CONDITIONS 178 | 179 | APPENDIX: How to apply the Apache License to your work. 180 | 181 | To apply the Apache License to your work, attach the following 182 | boilerplate notice, with the fields enclosed by brackets "{}" 183 | replaced with your own identifying information. (Don't include 184 | the brackets!) The text should be enclosed in the appropriate 185 | comment syntax for the file format. We also recommend that a 186 | file or class name and description of purpose be included on the 187 | same "printed page" as the copyright notice for easier 188 | identification within third-party archives. 189 | 190 | Copyright 2019 Lyft, Inc. 191 | 192 | Licensed under the Apache License, Version 2.0 (the "License"); 193 | you may not use this file except in compliance with the License. 194 | You may obtain a copy of the License at 195 | 196 | http://www.apache.org/licenses/LICENSE-2.0 197 | 198 | Unless required by applicable law or agreed to in writing, software 199 | distributed under the License is distributed on an "AS IS" BASIS, 200 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 201 | See the License for the specific language governing permissions and 202 | limitations under the License. 203 | --------------------------------------------------------------------------------