├── .github └── workflows │ ├── publish.yml │ └── test.yml ├── .gitignore ├── README.md ├── datasette_insert └── __init__.py ├── pytest.ini ├── setup.py └── tests └── test_insert_api.py /.github/workflows/publish.yml: -------------------------------------------------------------------------------- 1 | name: Publish Python Package 2 | 3 | on: 4 | release: 5 | types: [created] 6 | 7 | jobs: 8 | test: 9 | runs-on: ubuntu-latest 10 | strategy: 11 | matrix: 12 | python-version: ["3.7", "3.8", "3.9", "3.10"] 13 | steps: 14 | - uses: actions/checkout@v2 15 | - name: Set up Python ${{ matrix.python-version }} 16 | uses: actions/setup-python@v2 17 | with: 18 | python-version: ${{ matrix.python-version }} 19 | - uses: actions/cache@v2 20 | name: Configure pip caching 21 | with: 22 | path: ~/.cache/pip 23 | key: ${{ runner.os }}-pip-${{ hashFiles('**/setup.py') }} 24 | restore-keys: | 25 | ${{ runner.os }}-pip- 26 | - name: Install dependencies 27 | run: | 28 | pip install -e '.[test]' 29 | - name: Run tests 30 | run: | 31 | pytest 32 | deploy: 33 | runs-on: ubuntu-latest 34 | needs: [test] 35 | steps: 36 | - uses: actions/checkout@v2 37 | - name: Set up Python 38 | uses: actions/setup-python@v2 39 | with: 40 | python-version: "3.10" 41 | - uses: actions/cache@v2 42 | name: Configure pip caching 43 | with: 44 | path: ~/.cache/pip 45 | key: ${{ runner.os }}-publish-pip-${{ hashFiles('**/setup.py') }} 46 | restore-keys: | 47 | ${{ runner.os }}-publish-pip- 48 | - name: Install dependencies 49 | run: | 50 | pip install setuptools wheel twine build 51 | - name: Publish 52 | env: 53 | TWINE_USERNAME: __token__ 54 | TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }} 55 | run: | 56 | python -m build 57 | twine upload dist/* 58 | 59 | -------------------------------------------------------------------------------- /.github/workflows/test.yml: -------------------------------------------------------------------------------- 1 | name: Test 2 | 3 | on: [push, pull_request] 4 | 5 | jobs: 6 | test: 7 | runs-on: ubuntu-latest 8 | strategy: 9 | matrix: 10 | python-version: ["3.7", "3.8", "3.9", "3.10"] 11 | steps: 12 | - uses: actions/checkout@v2 13 | - name: Set up Python ${{ matrix.python-version }} 14 | uses: actions/setup-python@v2 15 | with: 16 | python-version: ${{ matrix.python-version }} 17 | - uses: actions/cache@v2 18 | name: Configure pip caching 19 | with: 20 | path: ~/.cache/pip 21 | key: ${{ runner.os }}-pip-${{ hashFiles('**/setup.py') }} 22 | restore-keys: | 23 | ${{ runner.os }}-pip- 24 | - name: Install dependencies 25 | run: | 26 | pip install -e '.[test]' 27 | - name: Run tests 28 | run: | 29 | pytest 30 | 31 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .venv 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | venv 6 | .eggs 7 | .pytest_cache 8 | *.egg-info 9 | .DS_Store 10 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # datasette-insert 2 | 3 | [![PyPI](https://img.shields.io/pypi/v/datasette-insert.svg)](https://pypi.org/project/datasette-insert/) 4 | [![Changelog](https://img.shields.io/github/v/release/simonw/datasette-insert?include_prereleases&label=changelog)](https://github.com/simonw/datasette-insert/releases) 5 | [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/datasette-insert/blob/master/LICENSE) 6 | 7 | Datasette plugin for inserting and updating data 8 | 9 | **No longer necessary with Datasette 1.0**. Use the [JSON write API](https://docs.datasette.io/en/latest/json_api.html#the-json-write-api) instead. 10 | 11 | ## Installation 12 | 13 | Install this plugin in the same environment as Datasette. 14 | 15 | $ pip install datasette-insert 16 | 17 | This plugin should always be deployed with additional configuration to prevent unauthenticated access, see notes below. 18 | 19 | If you are trying it out on your own local machine, you can `pip install` the [datasette-insert-unsafe](https://github.com/simonw/datasette-insert-unsafe) plugin to allow access without needing to set up authentication or permissions separately. 20 | 21 | ## Inserting data and creating tables 22 | 23 | Start datasette and make sure it has a writable SQLite database attached to it. If you have not yet created a database file you can use this: 24 | 25 | datasette data.db --create 26 | 27 | The `--create` option will create a new empty `data.db` database file if it does not already exist. 28 | 29 | The plugin adds an endpoint that allows data to be inserted or updated and tables to be created by POSTing JSON data to the following URL: 30 | 31 | /-/insert/name-of-database/name-of-table 32 | 33 | The JSON should look like this: 34 | 35 | ```json 36 | [ 37 | { 38 | "id": 1, 39 | "name": "Cleopaws", 40 | "age": 5 41 | }, 42 | { 43 | "id": 2, 44 | "name": "Pancakes", 45 | "age": 5 46 | } 47 | ] 48 | ``` 49 | 50 | The first time data is posted to the URL a table of that name will be created if it does not aready exist, with the desired columns. 51 | 52 | You can specify which column should be used as the primary key using the `?pk=` URL argument. 53 | 54 | Here's how to POST to a database and create a new table using the Python `requests` library: 55 | 56 | ```python 57 | import requests 58 | 59 | requests.post("http://localhost:8001/-/insert/data/dogs?pk=id", json=[ 60 | { 61 | "id": 1, 62 | "name": "Cleopaws", 63 | "age": 5 64 | }, 65 | { 66 | "id": 2, 67 | "name": "Pancakes", 68 | "age": 4 69 | } 70 | ]) 71 | ``` 72 | And here's how to do the same thing using `curl`: 73 | 74 | ``` 75 | curl --request POST \ 76 | --data '[ 77 | { 78 | "id": 1, 79 | "name": "Cleopaws", 80 | "age": 5 81 | }, 82 | { 83 | "id": 2, 84 | "name": "Pancakes", 85 | "age": 4 86 | } 87 | ]' \ 88 | 'http://localhost:8001/-/insert/data/dogs?pk=id' 89 | ``` 90 | Or by piping in JSON like so: 91 | 92 | cat dogs.json | curl --request POST -d @- \ 93 | 'http://localhost:8001/-/insert/data/dogs?pk=id' 94 | 95 | ### Inserting a single row 96 | 97 | If you are inserting a single row you can optionally send it as a dictionary rather than a list with a single item: 98 | 99 | ``` 100 | curl --request POST \ 101 | --data '{ 102 | "id": 1, 103 | "name": "Cleopaws", 104 | "age": 5 105 | }' \ 106 | 'http://localhost:8001/-/insert/data/dogs?pk=id' 107 | ``` 108 | 109 | ### Automatically adding new columns 110 | 111 | If you send data to an existing table with keys that are not reflected by the existing columns, you will get an HTTP 400 error with a JSON response like this: 112 | 113 | ```json 114 | { 115 | "status": 400, 116 | "error": "Unknown keys: 'foo'", 117 | "error_code": "unknown_keys" 118 | } 119 | ``` 120 | 121 | If you add `?alter=1` to the URL you are posting to any missing columns will be automatically added: 122 | 123 | ``` 124 | curl --request POST \ 125 | --data '[ 126 | { 127 | "id": 3, 128 | "name": "Boris", 129 | "age": 1, 130 | "breed": "Husky" 131 | } 132 | ]' \ 133 | 'http://localhost:8001/-/insert/data/dogs?alter=1' 134 | ``` 135 | 136 | ## Upserting data 137 | 138 | An "upsert" operation can be used to partially update a record. With upserts you can send a subset of the keys and, if the ID matches the specified primary key, they will be used to update an existing record. 139 | 140 | Upserts can be sent to the `/-/upsert` API endpoint. 141 | 142 | This example will update the dog with ID=1's age from 5 to 7: 143 | ``` 144 | curl --request POST \ 145 | --data '{ 146 | "id": 1, 147 | "age": 7 148 | }' \ 149 | 'http://localhost:3322/-/upsert/data/dogs?pk=id' 150 | ``` 151 | Like the `/-/insert` endpoint, the `/-/upsert` endpoint can accept an array of objects too. It also supports the `?alter=1` option. 152 | 153 | ## Permissions and authentication 154 | 155 | This plugin defaults to denying all access, to help ensure people don't accidentally deploy it on the open internet in an unsafe configuration. 156 | 157 | You can read about [Datasette's approach to authentication](https://datasette.readthedocs.io/en/stable/authentication.html) in the Datasette manual. 158 | 159 | You can install the `datasette-insert-unsafe` plugin to run in unsafe mode, where all access is allowed by default. 160 | 161 | I recommend using this plugin in conjunction with [datasette-auth-tokens](https://github.com/simonw/datasette-auth-tokens), which provides a mechanism for making authenticated calls using API tokens. 162 | 163 | You can then use ["allow" blocks](https://datasette.readthedocs.io/en/stable/authentication.html#defining-permissions-with-allow-blocks) in the `datasette-insert` plugin configuration to specify which authenticated tokens are allowed to make use of the API. 164 | 165 | Here's an example `metadata.json` file which restricts access to the `/-/insert` API to an API token defined in an `INSERT_TOKEN` environment variable: 166 | 167 | ```json 168 | { 169 | "plugins": { 170 | "datasette-insert": { 171 | "allow": { 172 | "bot": "insert-bot" 173 | } 174 | }, 175 | "datasette-auth-tokens": { 176 | "tokens": [ 177 | { 178 | "token": { 179 | "$env": "INSERT_TOKEN" 180 | }, 181 | "actor": { 182 | "bot": "insert-bot" 183 | } 184 | } 185 | ] 186 | } 187 | } 188 | } 189 | ``` 190 | With this configuration in place you can start Datasette like this: 191 | 192 | INSERT_TOKEN=abc123 datasette data.db -m metadata.json 193 | 194 | You can now send data to the API using `curl` like this: 195 | 196 | ``` 197 | curl --request POST \ 198 | -H "Authorization: Bearer abc123" \ 199 | --data '[ 200 | { 201 | "id": 3, 202 | "name": "Boris", 203 | "age": 1, 204 | "breed": "Husky" 205 | } 206 | ]' \ 207 | 'http://localhost:8001/-/insert/data/dogs' 208 | ``` 209 | 210 | Or using the Python `requests` library like so: 211 | 212 | ```python 213 | requests.post( 214 | "http://localhost:8001/-/insert/data/dogs", 215 | json={"id": 1, "name": "Cleopaws", "age": 5}, 216 | headers={"Authorization": "bearer abc123"}, 217 | ) 218 | ``` 219 | 220 | ### Finely grained permissions 221 | 222 | Using an `"allow"` block as described above grants full permission to the features enabled by the API. 223 | 224 | The API implements several new Datasett permissions, which other plugins can use to make more finely grained decisions. 225 | 226 | The full set of permissions are as follows: 227 | 228 | - `insert:all` - all permissions - this is used by the `"allow"` block described above. Argument: `database_name` 229 | - `insert:insert-update` - the ability to insert data into an existing table, or to update data by its primary key. Arguments: `(database_name, table_name)` 230 | - `insert:create-table` - the ability to create a new table. Argument: `database_name` 231 | - `insert:alter-table` - the ability to add columns to an existing table (using `?alter=1`). Arguments: `(database_name, table_name)` 232 | 233 | You can use plugins like [datasette-permissions-sql](https://github.com/simonw/datasette-permissions-sql) to hook into these more detailed permissions for finely grained control over what actions each authenticated actor can take. 234 | 235 | Plugins that implement the [permission_allowed()](https://datasette.readthedocs.io/en/stable/plugin_hooks.html#plugin-hook-permission-allowed) plugin hook can take full control over these permission decisions. 236 | 237 | ## CORS 238 | 239 | If you start Datasette with the `datasette --cors` option the following HTTP headers will be added to resources served by this plugin: 240 | 241 | Access-Control-Allow-Origin: * 242 | Access-Control-Allow-Headers: content-type,authorization 243 | Access-Control-Allow-Methods: POST 244 | 245 | ## Development 246 | 247 | To set up this plugin locally, first checkout the code. Then create a new virtual environment: 248 | 249 | cd datasette-insert 250 | python3 -m venv venv 251 | source venv/bin/activate 252 | 253 | Now install the dependencies and tests: 254 | 255 | pip install -e '.[test]' 256 | 257 | To run the tests: 258 | 259 | pytest 260 | -------------------------------------------------------------------------------- /datasette_insert/__init__.py: -------------------------------------------------------------------------------- 1 | from datasette import hookimpl 2 | from datasette.utils.asgi import Response 3 | from datasette.utils import actor_matches_allow, sqlite3 4 | import json 5 | import sqlite_utils 6 | 7 | 8 | class MissingTable(Exception): 9 | pass 10 | 11 | 12 | async def insert_or_upsert(request, datasette): 13 | # Wraps insert_or_upsert_implementation with CORS 14 | response = await insert_or_upsert_implementation(request, datasette) 15 | if datasette.cors: 16 | response.headers["Access-Control-Allow-Origin"] = "*" 17 | response.headers["Access-Control-Allow-Headers"] = "content-type,authorization" 18 | response.headers["Access-Control-Allow-Methods"] = "POST" 19 | return response 20 | 21 | 22 | async def insert_or_upsert_implementation(request, datasette): 23 | database = request.url_vars["database"] 24 | table = request.url_vars["table"] 25 | upsert = request.url_vars["verb"] == "upsert" 26 | db = datasette.get_database(database) 27 | 28 | # Needed for CORS: 29 | if request.method == "OPTIONS": 30 | return Response.text("ok") 31 | 32 | pk = request.args.get("pk") 33 | alter = request.args.get("alter") 34 | 35 | if upsert and not pk: 36 | return Response.json( 37 | { 38 | "status": 400, 39 | "error": "Upsert requires ?pk=", 40 | "error_code": "upsert_requires_pk", 41 | }, 42 | status=400, 43 | ) 44 | 45 | # Check permissions 46 | allow_insert_update = False 47 | allow_create_table = False 48 | allow_alter_table = False 49 | allow_all = await datasette.permission_allowed( 50 | request.actor, "insert:all", database, default=False 51 | ) 52 | if allow_all: 53 | allow_insert_update = True 54 | allow_create_table = True 55 | allow_alter_table = True 56 | else: 57 | # Check for finely grained permissions 58 | allow_insert_update = await datasette.permission_allowed( 59 | request.actor, "insert:insert-update", (database, table), default=False 60 | ) 61 | allow_create_table = await datasette.permission_allowed( 62 | request.actor, "insert:create-table", database, default=False 63 | ) 64 | allow_alter_table = await datasette.permission_allowed( 65 | request.actor, "insert:alter-table", (database, table), default=False 66 | ) 67 | 68 | if not allow_insert_update: 69 | return Response.json({"error": "Permission denied", "status": 403}, status=403) 70 | 71 | if alter and not allow_alter_table: 72 | return Response.json( 73 | {"error": "Alter permission denied", "status": 403}, status=403 74 | ) 75 | 76 | post_json = json.loads(await request.post_body()) 77 | if isinstance(post_json, dict): 78 | post_json = [post_json] 79 | 80 | def write_in_thread(conn): 81 | db = sqlite_utils.Database(conn) 82 | if not allow_create_table and not db[table].exists(): 83 | raise MissingTable() 84 | if upsert: 85 | db[table].upsert_all(post_json, pk=pk, alter=alter) 86 | else: 87 | db[table].insert_all(post_json, replace=True, pk=pk, alter=alter) 88 | return db[table].count 89 | 90 | try: 91 | table_count = await db.execute_write_fn(write_in_thread, block=True) 92 | except MissingTable: 93 | return Response.json( 94 | { 95 | "status": 400, 96 | "error": "Table {} does not exist".format(table), 97 | "error_code": "missing_table", 98 | }, 99 | status=400, 100 | ) 101 | except sqlite3.OperationalError as ex: 102 | if "has no column" in str(ex): 103 | return Response.json( 104 | {"status": 400, "error": str(ex), "error_code": "unknown_keys"}, 105 | status=400, 106 | ) 107 | else: 108 | return Response.json({"error": str(ex), "status": 500}, status=500) 109 | 110 | return Response.json({"table_count": table_count}) 111 | 112 | 113 | @hookimpl 114 | def permission_allowed(datasette, actor, action): 115 | if action != "insert:all": 116 | return None 117 | plugin_config = datasette.plugin_config("datasette-insert") or {} 118 | if "allow" in plugin_config: 119 | return actor_matches_allow(actor, plugin_config["allow"]) 120 | 121 | 122 | @hookimpl 123 | def register_routes(): 124 | return [ 125 | ( 126 | r"^/-/(?P(insert|upsert))/(?P[^/]+)/(?P[^/]+)$", 127 | insert_or_upsert, 128 | ), 129 | ] 130 | -------------------------------------------------------------------------------- /pytest.ini: -------------------------------------------------------------------------------- 1 | [pytest] 2 | asyncio_mode = strict 3 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup 2 | import os 3 | 4 | VERSION = "0.8" 5 | 6 | 7 | def get_long_description(): 8 | with open( 9 | os.path.join(os.path.dirname(os.path.abspath(__file__)), "README.md"), 10 | encoding="utf8", 11 | ) as fp: 12 | return fp.read() 13 | 14 | 15 | setup( 16 | name="datasette-insert", 17 | description="Datasette plugin for inserting and updating data", 18 | long_description=get_long_description(), 19 | long_description_content_type="text/markdown", 20 | author="Simon Willison", 21 | url="https://datasette.io/plugins/datasette-insert", 22 | project_urls={ 23 | "Issues": "https://github.com/simonw/datasette-insert/issues", 24 | "CI": "https://github.com/simonw/datasette-insert/actions", 25 | "Changelog": "https://github.com/simonw/datasette-insert/releases", 26 | }, 27 | license="Apache License, Version 2.0", 28 | version=VERSION, 29 | packages=["datasette_insert"], 30 | entry_points={"datasette": ["insert = datasette_insert"]}, 31 | install_requires=["datasette>=0.46", "sqlite-utils"], 32 | extras_require={ 33 | "test": ["pytest", "pytest-asyncio", "httpx", "datasette-auth-tokens"] 34 | }, 35 | python_requires=">=3.7", 36 | ) 37 | -------------------------------------------------------------------------------- /tests/test_insert_api.py: -------------------------------------------------------------------------------- 1 | from datasette import hookimpl 2 | from datasette.app import Datasette 3 | from datasette.plugins import pm 4 | import sqlite_utils 5 | import pytest 6 | 7 | 8 | @pytest.fixture 9 | def db_path(tmp_path_factory): 10 | db_directory = tmp_path_factory.mktemp("dbs") 11 | db_path = db_directory / "data.db" 12 | db = sqlite_utils.Database(db_path) 13 | db.vacuum() 14 | return db_path 15 | 16 | 17 | @pytest.fixture 18 | def unsafe(): 19 | class UnsafeInsertAll: 20 | __name__ = "UnsafeInsertAll" 21 | 22 | @hookimpl 23 | def permission_allowed(self, action): 24 | if action == "insert:all": 25 | return True 26 | 27 | pm.register(UnsafeInsertAll(), name="undo") 28 | yield 29 | pm.unregister(name="undo") 30 | 31 | 32 | @pytest.fixture 33 | def ds(db_path): 34 | return Datasette([db_path]) 35 | 36 | 37 | @pytest.fixture 38 | def ds_root_only(db_path): 39 | return Datasette( 40 | [db_path], 41 | metadata={ 42 | "plugins": { 43 | "datasette-insert": {"allow": {"bot": "test"}}, 44 | "datasette-auth-tokens": { 45 | "tokens": [{"token": "test-bot", "actor": {"bot": "test"}}] 46 | }, 47 | } 48 | }, 49 | ) 50 | 51 | 52 | @pytest.mark.asyncio 53 | async def test_plugin_is_installed(ds): 54 | response = await ds.client.get("/-/plugins.json") 55 | assert response.status_code == 200 56 | installed_plugins = {p["name"] for p in response.json()} 57 | assert "datasette-insert" in installed_plugins 58 | # Check we have our testing dependency too: 59 | assert "datasette-auth-tokens" in installed_plugins 60 | 61 | 62 | @pytest.mark.asyncio 63 | async def test_permission_denied_by_default(ds): 64 | response = await ds.client.post( 65 | "/-/insert/data/newtable", 66 | json=[{"foo": "bar"}], 67 | ) 68 | assert response.status_code == 403 69 | 70 | 71 | @pytest.mark.parametrize( 72 | "input,pk,expected", 73 | # expected=None means reuse the inpt as the expected 74 | [ 75 | ( 76 | [ 77 | {"id": 3, "name": "Cleopaws", "age": 5}, 78 | {"id": 11, "name": "Pancakes", "age": 4}, 79 | ], 80 | "id", 81 | None, 82 | ), 83 | # rowid example: 84 | ( 85 | [{"name": "Cleopaws", "age": 5}, {"name": "Pancakes", "age": 4}], 86 | None, 87 | [ 88 | {"rowid": 1, "name": "Cleopaws", "age": 5}, 89 | {"rowid": 2, "name": "Pancakes", "age": 4}, 90 | ], 91 | ), 92 | # Single row 93 | ( 94 | {"id": 1, "name": "Cleopaws", "age": 5}, 95 | "id", 96 | [{"id": 1, "name": "Cleopaws", "age": 5}], 97 | ), 98 | ( 99 | {"name": "Cleopaws", "age": 5}, 100 | None, 101 | [{"rowid": 1, "name": "Cleopaws", "age": 5}], 102 | ), 103 | ], 104 | ) 105 | @pytest.mark.asyncio 106 | async def test_insert_creates_table(ds, unsafe, input, pk, expected): 107 | app = ds.app() 108 | response = await ds.client.post( 109 | "/-/insert/data/newtable{}".format("?pk={}".format(pk) if pk else ""), 110 | json=input, 111 | ) 112 | assert response.status_code == 200 113 | assert {"table_count"} == set(response.json().keys()) 114 | # Read that table data 115 | response2 = await ds.client.get( 116 | "/data/newtable.json?_shape=array", 117 | ) 118 | assert 200 == response2.status_code 119 | assert (expected or input) == response2.json() 120 | 121 | 122 | @pytest.mark.asyncio 123 | async def test_insert_alter(ds, unsafe): 124 | response = await ds.client.post( 125 | "/-/insert/data/dogs?pk=id", 126 | json=[{"id": 3, "name": "Cleopaws", "age": 5}], 127 | ) 128 | assert response.status_code == 200 129 | assert (await rows(ds.client)) == [ 130 | {"id": 3, "name": "Cleopaws", "age": 5}, 131 | ] 132 | # Should throw error without alter 133 | response2 = await ds.client.post( 134 | "/-/insert/data/dogs", 135 | json=[{"id": 3, "name": "Cleopaws", "age": 5, "weight_lb": 51.1}], 136 | ) 137 | assert 400 == response2.status_code 138 | # Insert with an alter 139 | response3 = await ds.client.post( 140 | "/-/insert/data/dogs?alter=1", 141 | json=[{"id": 3, "name": "Cleopaws", "age": 5, "weight_lb": 51.1}], 142 | ) 143 | assert 200 == response3.status_code 144 | assert (await rows(ds.client)) == [ 145 | {"id": 3, "name": "Cleopaws", "age": 5, "weight_lb": 51.1}, 146 | ] 147 | 148 | 149 | @pytest.mark.asyncio 150 | async def test_upsert(ds, unsafe): 151 | response = await ds.client.post( 152 | "/-/insert/data/dogs?pk=id", 153 | json=[{"id": 3, "name": "Cleopaws", "age": 5}], 154 | ) 155 | assert response.status_code == 200 156 | response2 = await ds.client.post( 157 | "/-/upsert/data/dogs?pk=id", 158 | json=[{"id": 3, "age": 7}], 159 | ) 160 | assert response2.status_code == 200 161 | assert (await rows(ds.client)) == [ 162 | {"id": 3, "name": "Cleopaws", "age": 7}, 163 | ] 164 | 165 | 166 | @pytest.mark.asyncio 167 | async def test_missing_column_error(ds, unsafe): 168 | response = await ds.client.post( 169 | "/-/insert/data/dogs?pk=id", 170 | json=[{"id": 3, "name": "Cleopaws", "age": 5}], 171 | ) 172 | assert response.status_code == 200 173 | response2 = await ds.client.post( 174 | "/-/insert/data/dogs?pk=id", 175 | json=[{"id": 3, "name": "Cleopaws", "age": 5, "size": "medium"}], 176 | ) 177 | assert response2.status_code == 400 178 | assert response2.json() == { 179 | "status": 400, 180 | "error": "table dogs has no column named size", 181 | "error_code": "unknown_keys", 182 | } 183 | 184 | 185 | @pytest.mark.asyncio 186 | async def test_upsert_reuires_pk_error(ds, unsafe): 187 | response = await ds.client.post( 188 | "/-/upsert/data/dogs", 189 | json=[{"id": 3, "name": "Cleopaws", "age": 5}], 190 | ) 191 | assert response.status_code == 400 192 | assert response.json() == { 193 | "status": 400, 194 | "error": "Upsert requires ?pk=", 195 | "error_code": "upsert_requires_pk", 196 | } 197 | 198 | 199 | @pytest.mark.asyncio 200 | async def test_permission_denied_by_allow_block(ds_root_only): 201 | response = await ds_root_only.client.post( 202 | "/-/insert/data/dogs?pk=id", 203 | json=[{"id": 3, "name": "Cleopaws", "age": 5}], 204 | ) 205 | assert response.status_code == 403 206 | 207 | 208 | @pytest.mark.asyncio 209 | async def test_permission_allowed_by_allow_block(ds_root_only): 210 | response = await ds_root_only.client.post( 211 | "/-/insert/data/dogs?pk=id", 212 | json=[{"id": 3, "name": "Cleopaws", "age": 5}], 213 | headers={"Authorization": "Bearer test-bot"}, 214 | ) 215 | assert response.status_code == 200 216 | assert (await rows(ds_root_only.client)) == [ 217 | {"id": 3, "name": "Cleopaws", "age": 5}, 218 | ] 219 | 220 | 221 | @pytest.mark.parametrize( 222 | "permissions,try_action,expected_status,expected_msg", 223 | [ 224 | # insert-update allowed 225 | ( 226 | { 227 | "insert-update": True, 228 | "create-table": False, 229 | "alter-table": False, 230 | }, 231 | "insert-update", 232 | 200, 233 | None, 234 | ), 235 | # insert-update denied 236 | ( 237 | { 238 | "insert-update": False, 239 | "create-table": False, 240 | "alter-table": False, 241 | }, 242 | "insert-update", 243 | 403, 244 | "Permission denied", 245 | ), 246 | # create-table allowed 247 | ( 248 | { 249 | "insert-update": True, 250 | "create-table": True, 251 | "alter-table": False, 252 | }, 253 | "create-table", 254 | 200, 255 | None, 256 | ), 257 | # create-table denied 258 | ( 259 | { 260 | "insert-update": True, 261 | "create-table": False, 262 | "alter-table": False, 263 | }, 264 | "create-table", 265 | 400, 266 | "Table dogs does not exist", 267 | ), 268 | # Alter table allowed 269 | ( 270 | { 271 | "insert-update": True, 272 | "create-table": False, 273 | "alter-table": True, 274 | }, 275 | "alter-table", 276 | 200, 277 | None, 278 | ), 279 | # Alter table denied 280 | ( 281 | { 282 | "insert-update": True, 283 | "create-table": False, 284 | "alter-table": False, 285 | }, 286 | "alter-table", 287 | 403, 288 | "Alter permission denied", 289 | ), 290 | ], 291 | ) 292 | @pytest.mark.asyncio 293 | async def test_permission_finely_grained( 294 | ds_root_only, permissions, try_action, expected_status, expected_msg 295 | ): 296 | class TestPlugin: 297 | __name__ = "TestPlugin" 298 | 299 | @hookimpl 300 | def permission_allowed(self, datasette, actor, action): 301 | if action.startswith("insert:"): 302 | return permissions.get(action.replace("insert:", "")) 303 | 304 | pm.register(TestPlugin(), name="undo") 305 | try: 306 | # First create the table (if we aren't testing create-table) using 307 | # the root authenticated API token 308 | if try_action != "create-table": 309 | await ds_root_only.client.post( 310 | "/-/insert/data/dogs?pk=id", 311 | json=[{"id": 1, "name": "Toodles", "age": 3}], 312 | headers={"Authorization": "Bearer test-bot"}, 313 | ) 314 | 315 | # Now we can test the TestPlugin-provided permissions 316 | if try_action in ("insert-update", "create-table"): 317 | response = await ds_root_only.client.post( 318 | "/-/insert/data/dogs?pk=id", 319 | json=[{"id": 3, "name": "Cleopaws", "age": 5}], 320 | ) 321 | elif try_action == "alter-table": 322 | response = await ds_root_only.client.post( 323 | "/-/insert/data/dogs?pk=id&alter=1", 324 | json=[{"id": 3, "name": "Cleopaws", "age": 5, "weight": 51.5}], 325 | ) 326 | else: 327 | assert False, "{} is not a valid test action".format(try_action) 328 | assert response.status_code == expected_status 329 | if expected_status != 200: 330 | assert response.json()["error"] == expected_msg 331 | finally: 332 | pm.unregister(name="undo") 333 | 334 | 335 | async def rows(client): 336 | return (await client.get("/data/dogs.json?_shape=array")).json() 337 | 338 | 339 | @pytest.mark.asyncio 340 | @pytest.mark.parametrize("cors_enabled", (True, False)) 341 | async def test_cors(db_path, unsafe, cors_enabled): 342 | ds = Datasette([db_path], cors=cors_enabled) 343 | response = await ds.client.options("/-/insert/data/dogs?pk=id") 344 | assert response.status_code == 200 345 | desired_headers = { 346 | "access-control-allow-headers": "content-type,authorization", 347 | "access-control-allow-methods": "POST", 348 | "access-control-allow-origin": "*", 349 | }.items() 350 | if cors_enabled: 351 | assert desired_headers <= dict(response.headers).items() 352 | else: 353 | assert not desired_headers <= dict(response.headers).items() 354 | --------------------------------------------------------------------------------