├── .github └── workflows │ └── deploy-site.yaml ├── .gitignore ├── LICENSE ├── README.md ├── docs ├── 01-poetry.md ├── 02-fastapi.md ├── 03-celery.md ├── 04-notebook.md ├── 05-more.md ├── assets │ ├── 00-goal.excalidraw.png │ ├── 01-wsgi.excalidraw.png │ ├── 03-celery.excalidraw.png │ ├── 03-lock-bad.excalidraw.png │ ├── 03-lock-good.excalidraw.png │ └── favicon.svg ├── extra.css └── index.md ├── mkdocs.yml └── notebook.ipynb /.github/workflows/deploy-site.yaml: -------------------------------------------------------------------------------- 1 | name: deploy site 2 | on: 3 | push: 4 | branches: 5 | - main 6 | 7 | permissions: 8 | contents: write 9 | 10 | jobs: 11 | deploy: 12 | runs-on: ubuntu-latest 13 | steps: 14 | - uses: actions/checkout@v3 15 | - name: Setup Python 16 | uses: actions/setup-python@v4 17 | with: 18 | python-version: 3.x 19 | 20 | - name: Setup Cache 21 | uses: actions/cache@v2 22 | with: 23 | key: ${{ github.ref }} 24 | path: .cache 25 | 26 | - name: Install MkDocs 27 | run: pip install mkdocs-material 28 | 29 | - name: Deploy 30 | run: mkdocs gh-deploy --force -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | *.egg 3 | !/tests/**/*.egg 4 | /*.egg-info 5 | /dist/* 6 | build 7 | _build 8 | .cache 9 | *.so 10 | 11 | # Poetry 12 | .venv 13 | poetry/core 14 | 15 | # IDEs 16 | .vscode 17 | *.iml 18 | .idea 19 | .DS_Store 20 | 21 | # Other 22 | .ipynb_checkpoints -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 Lucy Linder 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # FastAPI + Celery = ♥ 2 | 3 | *Interested in Python FastAPI? Wondering how to execute long-running tasks in the background 4 | in Python? You came to the right place!* 5 | 6 | This little website will go through the basics of Poetry, FastAPI and Celery, with some detours 7 | here and there. I created it to back up my talk at GDG Fribourg on **March 22, 2023**. 8 | 9 | Read on ⮕ ✨✨ https://derlin.github.io/introduction-to-fastapi-and-celery ✨✨ 10 | 11 | The full implementation of the use case can be found at: 12 | https://github.com/derlin/fastapi-notebook-runner. 13 | 14 | **IMPORTANT** At the time of writing, the versions used are: 15 | 16 | * poetry `1.3.0` 17 | * FastAPI `0.95.0` 18 | * Celery `5.2.7` 19 | 20 | If you like it, please leave a :star: 21 | 22 | --- 23 | 24 | To edit/run the website locally: 25 | 26 | ```bash 27 | docker run --rm -it -p 8888:8000 -v ${PWD}:/docs squidfunk/mkdocs-material 28 | ``` -------------------------------------------------------------------------------- /docs/01-poetry.md: -------------------------------------------------------------------------------- 1 | # Poetry 2 | 3 | Before getting started, we need a Python project. 4 | 5 | [poetry](https://python-poetry.org) is the trendy package manager 6 | which superseded setuptools and pyenv in the heart of pythonistas. 7 | 8 | ## Installation 9 | 10 | If you haven't done so, install poetry: [https://python-poetry.org/docs/#installation]( 11 | https://python-poetry.org/docs/#installation). 12 | On a Mac: 13 | ```bash 14 | curl -sSL https://install.python-poetry.org | python3 - 15 | ``` 16 | 17 | I currently use: 18 | ```bash 19 | poetry --version 20 | Poetry (version 1.3.0) 21 | ``` 22 | 23 | The installation will create a new virtualenv especially for poetry somewhere in your system. 24 | On a Mac, use `cat $(which poetry)` and look at the first line. In my case: 25 | 26 | * `poetry` executable installed in `~/.local/bin/poetry` 27 | * poetry's virtualenv installed in `~/Library/Application Support/pypoetry/venv/` 28 | 29 | ## Initializing a project 30 | 31 | Once poetry is installed, initializing a new project is as easy as running: 32 | ```bash 33 | mkdir fastapi-celery && cd fastapi-celery 34 | poetry init 35 | ``` 36 | 37 | Follow the prompts. You can already start adding dependencies (`fastapi`, `celery`) 38 | and dev-dependencies (`bandit`, `black`) or choose to do that later. 39 | 40 | Once you finished with the prompt, have a look at what was generated: 41 | ```bash 42 | cat pyproject.toml 43 | ``` 44 | 45 | ## Adding dependencies 46 | 47 | You can add a dependency at any time using: `poetry add `. 48 | If the dependency is for development only, use the `--group dev` flag (or any other group name). 49 | 50 | You can also edit the `pyproject.toml` directly and run `poetry install`. 51 | 52 | I use `poetry add` with a name (no version) and then update the `pyprojet.toml` 53 | manually to pin only the minor version (instead of the full version). 54 | This means replacing e.g. `fastapi = "^0.93.0"` with `fastapi = "^0.93"` (notice the `.0` is missing). 55 | This way, I can still beneficiate from patch updates by running `poetry update` at any time. 56 | I also often use '*' for dev dependencies. 57 | 58 | See [Dependency specification](https://python-poetry.org/docs/dependency-specification/) for more information. 59 | 60 | ## Creating a virtualenv 61 | 62 | This only set up the project. Now, we need to create a virtualenv and install those dependencies. 63 | Poetry can do this for us using: 64 | ```bash 65 | poetry install 66 | ``` 67 | 68 | Poetry will automatically create a virtualenv, located in a central place on your system. 69 | I will be blunt: I :face_vomiting: this; I like to have virtualenvs at the root of each project instead. 70 | This ensures all is cleaned up if I delete a project and I can always use `source .venv/bin/activate` 71 | from any repo to have my virtualenv selected. 72 | 73 | To change poetry's default behavior to something more suited to my tastes, 74 | create a `poetry.toml` at the root of the project with the following: 75 | ```toml 76 | [virtualenvs] 77 | in-project = true 78 | path = ".venv" 79 | ``` 80 | 81 | !!! warning 82 | 83 | If you already ran the install, remove the old virtualenv first (`poetry env list` + `poetry env remove `), 84 | and run `poetry install` again. 85 | 86 | ## And more 87 | 88 | Other nice things about poetry: 89 | 90 | * `poetry show --latest` will show the current and latest available versions of your dependencies 91 | * `poetry env` allows you to manage environments: remove, list, info, etc. 92 | * running `poetry install --sync` ensures that the locked dependencies in the `poetry.lock` are the only ones 93 | present in the environment, removing anything that’s not necessary. 94 | * you can choose to install only certain groups (in addition to main) using `poetry install --with dev`, 95 | or only a specific group using `poetry install --only main`. 96 | * running `poetry update` will fetch the latest matching versions (according to the `pyproject.toml` file) and update 97 | the lock file with the new versions. This is equivalent to deleting the `poetry.lock` file and running `install` again. 98 | * you can build the project using `poetry build`, and publish it to PyPI using `poetry publish` (given you have [set up 99 | you credentials](https://python-poetry.org/docs/repositories/#configuring-credentials) properly) 100 | 101 | The documentation is amazing, so I will stop here. 102 | 103 | ## BONUS 104 | 105 | ### formatting with black 106 | 107 | [black](https://github.com/psf/black) is a strict formatter. From their docs: 108 | 109 | > Black is the uncompromising Python code formatter. 110 | > By using it, you agree to cede control over minutiae of hand-formatting. 111 | > In return, Black gives you speed, determinism, and freedom from pycodestyle nagging about formatting. 112 | > You will save time and mental energy for more important matters. 113 | 114 | To install it: 115 | ```bash 116 | poetry add black='*' --group dev 117 | ``` 118 | 119 | Now, all you have to do is run: 120 | ```bash 121 | poetry run black fastapi_celery 122 | ``` 123 | 124 | Note that you can also configure VSCode to use black by default. 125 | 126 | The default line length is 88. If for some reason you want to change this, you can use: 127 | ```bash 128 | poetry run black --line-length 100 --experimental-string-processing fastapi_celery 129 | ``` 130 | 131 | The `--experimental-string-processing` is still under development. Without it, black won't 132 | split long strings... 133 | 134 | To only perform checks (without formatting anyhting, e.g. in CI), use: 135 | ```bash 136 | poetry run black --check fastapi_celery 137 | ``` 138 | 139 | ### linting with ruff 140 | 141 | I just discovered [ruff](https://beta.ruff.rs/docs/), which is just awesome! 142 | 143 | It basically replaces all the other tools (except formatting, but it is coming soon) such as: 144 | 145 | * `isort` - sorts imports 146 | * `bandit` - finds common security issues 147 | * `flake8` - linter 148 | * etc. 149 | 150 | Ruff is implemented in Rust and is really (really!) fast. The configuration can be done from the 151 | `pyproject.toml` directly, but the defaults are already quite nice. 152 | 153 | 154 | [Full list of available rules here](https://beta.ruff.rs/docs/rules/){ .md-button } 155 | 156 | Get started by installing it: 157 | ```bash 158 | poetry add ruff='*' --group dev 159 | ``` 160 | 161 | Now, run it using: 162 | ```bash 163 | # Check only 164 | poetry run ruff fastapi_celery 165 | # Automatically fix what can be fixed 166 | poetry run ruff --fix fastapi_celery 167 | ``` 168 | 169 | I played a bit with the options, and currently decided to use: 170 | 171 | ```toml 172 | # in pyproject.toml 173 | 174 | [tool.ruff] 175 | select = [ 176 | "E", # pycodestyle error 177 | "W", # pycodestyle warning 178 | "F", # pyflakes 179 | "A", # flakes8-builtins 180 | "COM", # flakes8-commas 181 | "C4", # flake8-comprehensions 182 | "Q", # flake8-quotes 183 | "SIM", # flake8-simplify 184 | "PTH", # flake8-use-pathlib 185 | "I", # isort 186 | "N", # pep8 naming 187 | "UP", # pyupgrade 188 | "S", # bandit 189 | ] 190 | ``` 191 | 192 | Have a look at the docs, it is really good! 193 | -------------------------------------------------------------------------------- /docs/02-fastapi.md: -------------------------------------------------------------------------------- 1 | # FastAPI 2 | 3 | [FastAPI](https://fastapi.tiangolo.com) is: 4 | > a modern, fast (high-performance), web framework for building APIs with Python 3.7+ based 5 | > on standard Python type hints. 6 | 7 | Their documentation is amazing (way better than this one), so please check it out! 8 | 9 | [FastAPI docs](https://fastapi.tiangolo.com){ .md-button } 10 | 11 | 12 | ## About WSGI and ASGI 13 | 14 | It is important to understand that FastAPI is an ASGI Application Framework. It cannot serve anything 15 | by itself: it needs an ASGI Server (or a WSGI server with an ASGI worker). 16 | 17 | [WSGI](https://www.fullstackpython.com/wsgi-servers.html) stands for *Web Server Gateway Interface*, 18 | and ASGI for *Asynchronous Server Gateway interface*. 19 | They both specify an interface that sits in between a web server and a Python web application or framework. 20 | WSGI has been around for a long time. ASGI is a spiritual successor to WSGI, that is able to handle 21 | asynchronous requests and responses. 22 | 23 | In short, an **(A|W)SGI Server** is a web server that is able to call python code when it receives an HTTP request. 24 | The way it calls this code, and what parameters are passed to the calling function, are all specified in the 25 | (A|W)SGI interface specification. It includes information about the request and the environment. 26 | 27 | It is then the role of the **(A|W)SGI Application** to build the headers and to return the data as iterable. 28 | This is done using the `start_response` call. Here is a simple WSGI application 29 | ([source](http://ivory.idyll.org/articles/wsgi-intro/what-is-wsgi.html)): 30 | 31 | ```python 32 | def simple_app(environ, start_response): 33 | status = '200 OK' 34 | response_headers = [('Content-type','text/plain')] 35 | # Actually start sending data back to the server 36 | start_response('200 OK', response_headers) 37 | return ['Hello world!\n'] 38 | ``` 39 | 40 | This `simple_app` can then be passed to a server and be served as is. 41 | 42 | Of course, apps are usually way more complex, with routing, proxies and complex logic involved. 43 | This is why we use **(A|W)SGI Application Frameworks** such as FastAPI. Those frameworks have a single 44 | entry point (that is called by the server), and lots of conveniences to abstract the complexities of 45 | constructing responses, handling errors, doing redirects, determining which code to execute, etc. 46 | 47 | ![WSGI overview](assets/01-wsgi.excalidraw.png) 48 | 49 | To actually run a FastAPI app, we thus need an ASGI server (or a WSGI server with an ASGI-compatible worker). 50 | In development, we can use [uvicorn](https://uvicorn.org/) - a minimalist server with a single process. 51 | 52 | While a single process is enough for testing, it is not suitable for production. 53 | The most common production setup for FastAPI is [gunicorn](https://gunicorn.org/) 54 | with [uvicorn workers](https://fastapi.tiangolo.com/deployment/server-workers/), sitting behind 55 | a reverse proxy such as [Nginx](https://www.nginx.com/). 56 | 57 | Why the reverse proxy you ask? 58 | Gunicorn is amazing at handling workers and WSGI-specific things, while NGINX is a full-featured 59 | HTTP server, able to handle millions of concurrent connections, provide DoD protection, 60 | rewrite headers, and serve static resources more effectively. 61 | Together, they form the perfect team. 62 | 63 | 64 | See also: [The Layered World Of Web Development: Why I Need NGINX And UWSGI To Run A Python App?]( 65 | http://www.ines-panker.com/2020/02/16/nginx-uwsqi.html) 66 | 67 | ## Install FastAPI + uvicorn 68 | 69 | Anyway, to get started, we only need to install both FastAPI and uvicorn: 70 | ```bash 71 | poetry add fastapi 72 | poetry add 'uvicorn[standard]' 73 | ``` 74 | 75 | The uvicorn server can then be launched (with reload!) using: 76 | ```bash 77 | uvicorn package.filename:app_object --reload 78 | ``` 79 | 80 | 81 | ## Getting started 82 | 83 | Create a file `fastapi_celery/main.py` and add: 84 | 85 | ```python 86 | from fastapi import FastAPI 87 | from typing import Dict 88 | 89 | app = FastAPI() # <- the ASGI entrypoint 90 | 91 | @app.get('/') 92 | def index() -> Dict: 93 | return {'greating': 'hello'} 94 | ``` 95 | 96 | Run uvicorn with reload: 97 | ```bash 98 | uvicorn fastapi_celery.main:app --reload 99 | ``` 100 | 101 | !!! Note 102 | 103 | It is also possible to run uvicorn directly from the python file, 104 | to allow for debugging: 105 | ```python 106 | import uvicorn 107 | 108 | 109 | if __name__ == "__main__": 110 | uvicorn.run(app, host="0.0.0.0", port=8000) 111 | ``` 112 | 113 | If you use `reload=True`, however, you have to pass the `app` as an import string, 114 | in our case `celery_fastapi.main:app`. 115 | 116 | and try it using: 117 | ```bash 118 | curl http://localhost:8000/ 119 | ``` 120 | 121 | Congrats! You have successfully coded a REST API. 122 | 123 | Now, open the following in your browser: `http://localhost:8000/docs`. 124 | FastAPI comes built-in with a [Swagger UI](https://swagger.io/tools/swagger-ui/) 125 | the [OpenApi](https://spec.openapis.org/oas/latest.html) specification for us based on type hints. 126 | 127 | Want more? FastAPI also comes with a [ReDoc](https://github.com/Redocly/redoc) 128 | documentation: `http://localhost:8000/redoc`! 129 | 130 | In short, you get those endpoints for free: 131 | 132 | * `/docs` → Interactive API docs (Swagger UI) 133 | * `/redoc` → Alternative API docs (ReDoc documentation) 134 | * `/openapi.json` → OpenAPI spec (JSON document) 135 | 136 | ## A simple example 137 | 138 | Let's imagine we want an endpoint to create a new user. 139 | Let's start with the most basic thing, we'll improve later. 140 | 141 | ```python 142 | from fastAPI import FastAPI 143 | from typing import Any 144 | from datetime import datetime 145 | 146 | app = FastAPI 147 | 148 | @app.get("/") 149 | def create_user() -> dict[str, Any]: 150 | return {"id": 10, "name": "my-username", "created_at": datetime.now()} 151 | ``` 152 | 153 | Now, looking at the docs, there is not much detail. Let's make it better! 154 | 155 | ### Documentation 156 | 157 | We can easily add some description for the endpoint using either a docstring, 158 | or the `description` parameter on the annotation. The annotation also lets us 159 | describe the response type. Both support markdown! 160 | 161 | ```python 162 | @app.get('/', response_description="The `new` user") 163 | def create_user() -> dict[str, Any]: 164 | """ 165 | Create a new user. *Supports `markdown`!* 166 | """ 167 | ``` 168 | 169 | For more documentation options: 170 | 171 | * [Metadata and Docs URLs](https://fastapi.tiangolo.com/tutorial/metadata/) 172 | for general information, 173 | * [Path Operation Configuration](https://fastapi.tiangolo.com/tutorial/path-operation-configuration/) 174 | for endpoint annotations, 175 | * [Declare Request Example Data](https://fastapi.tiangolo.com/tutorial/schema-extra-example/) 176 | for extra schema samples. 177 | 178 | FastAPI allows describing pretty much anything, and to customize all the fields 179 | that will appear in the OpenAPI spec. From now on, always look at the parameters 180 | offered by FastAPI classes :fontawesome-regular-face-grin-wink:. 181 | 182 | ### Model class 183 | 184 | The return type being a dictionary, FastAPI cannot give much detail, nor do 185 | any validation. Let's change this by using a [pydantic](https://docs.pydantic.dev) 186 | model. 187 | 188 | First, install pydantic: 189 | ```bash 190 | poetry add pydantic 191 | ``` 192 | 193 | And change the code to: 194 | 195 | ```py hl_lines="2 4 9-12 16-17" 196 | from fastapi import FastAPI 197 | from pydantic import BaseModel 198 | from datetime import datetime 199 | from random import randint 200 | 201 | app = FastAPI() 202 | 203 | 204 | class UserOut(BaseModel): 205 | id: int = randint(1, 100) 206 | name: str 207 | creation_date: datetime = datetime.now() 208 | 209 | 210 | @app.get('/') 211 | def create_user() -> UserOut: 212 | return UserOut(name="lucy") 213 | ``` 214 | 215 | Have a look at the successful response example in the docs. Isn't it great? 216 | 217 | ### Query parameters 218 | 219 | The name is still hard-coded. Why not use a query parameter instead? 220 | 221 | ```py 222 | @app.get("/") 223 | def create_user(name: str) -> UserOut: 224 | return UserOut(name=name) 225 | ``` 226 | 227 | Since `name` doesn't have a default, it is marked as *required* in the interface, 228 | and omitting it will throw an error. 229 | 230 | A name, though usually has more than one letter, right? So let's add some validation. 231 | For this, we can use the brand new feature of FastAPI 0.95.0: the support for 232 | [`typing.Annotated`](https://docs.python.org/3/library/typing.html#typing.Annotated)! 233 | 234 | ```py 235 | from fastapi import FastAPI, Query 236 | from typing import Annotated 237 | 238 | # ... 239 | 240 | @app.get("/") 241 | def create_user(name: Annotated[str, Query(min_length=3)]) -> UserOut: 242 | return UserOut(name=name) 243 | ``` 244 | 245 | `Annotated` is here to decorate existing types with context-specific data. 246 | Its first argument is always the final type, followed by varyadic arguments. 247 | Those metadata arguments are then used by other tools, in our case FastAPI. 248 | 249 | The `Query` object is a simple decorator offered by FastAPI that allows 250 | adding constraints to a query parameter. I will let you dive into the capabilities 251 | of `Query` on your own! 252 | 253 | ### Path parameters 254 | 255 | Let's say, for some reason, you want the name to become a *path parameter*. 256 | Suffice to change the path in the annotation: 257 | ```python 258 | @app.get("/{name}") 259 | ``` 260 | 261 | If you are using simple parameters (`name: str`), all is fine. If you have some 262 | constraints, though, you need to change the `Query` from the last example 263 | with `Path`. The logic, however, stays the same: 264 | 265 | ```python 266 | @app.get("/{name}") 267 | def create_user(name: Annotated[str, Path(min_length=3)]) -> UserOut: 268 | return UserOut(name=name) 269 | ``` 270 | 271 | ### Body parameters 272 | 273 | Using a get to create a user is ugly. Furthermore, we may want to have more 274 | information from the user such as a password. Let's fix this! 275 | 276 | ```py hl_lines="2 10-12 22 24" 277 | from fastapi import FastAPI 278 | from pydantic import BaseModel, Field 279 | from datetime import datetime 280 | from random import randint 281 | from typing import Annotated 282 | 283 | app = FastAPI() 284 | 285 | 286 | class UserIn(BaseModel): 287 | name: Annotated[str, Field(example="my-username", min_length=3, regex="^[a-z-]+$")] 288 | password: Annotated[str, Field(example="secret", min_length=5)] 289 | 290 | 291 | class UserOut(BaseModel): 292 | id: int = randint(1, 100) 293 | name: str 294 | creation_date: datetime = datetime.now() 295 | 296 | 297 | @app.post("/") 298 | def create_user(user: UserIn) -> UserOut: 299 | # return user <- would work as well... Try it! 300 | return UserOut(**user.dict()) 301 | ``` 302 | 303 | As you can see, we can use `Field` on the properties of a pydantic model as we 304 | do with `Query` and `Path`. Most of the properties are the same! 305 | 306 | The type hints are enough for FastAPI to provide validation, proper documentation, 307 | and IDE support. 308 | 309 | More interestingly, FastAPI/pydantic takes care of the "views". What do I mean by that? 310 | Try returning the input user directly (`return user`). We would expect to see `username` 311 | and `password` in the results, right? No! FastAPI ensures that what is returned matches 312 | the response model. Any extra properties are gone. There is no way the password will show 313 | up in the response. 314 | This allows for a flexible and elegant class hierarchy. 315 | 316 | Even cooler, we can fine-tune what is returned using `response_model_exclude_none` 317 | or `response_model_exclude_unset` in the `@app.post()` annotation. 318 | If set to true, `None` values or default values won't be returned, respectively. 319 | 320 | Have a look at all the other options, they are interesting :fontawesome-regular-face-smile:! 321 | 322 | 323 | ### Exceptions 324 | 325 | If something goes wrong, simply raise an `HTTPException` class: 326 | 327 | ```python 328 | from fastapi import FastAPI, HTTPException 329 | 330 | app = FastAPI() 331 | 332 | @app.get('/error') 333 | def error(): 334 | raise HTTPException(status_code=500, detail="Just throwing an internal server error") 335 | ``` 336 | 337 | Just try it: 338 | ```bash 339 | curl http://localhost:8000/error -v 340 | # < HTTP/1.1 500 Internal Server Error 341 | # {"detail":"Just throwing an internal server error"} 342 | ``` 343 | 344 | Note that the `detail` can be of any type, not just string. As long as it is JSON-serializable! 345 | It is also possible to specify headers. 346 | 347 | ### Status code 348 | 349 | Since our method creates a user, it would be better to return a `201 - Created` instead of a 350 | `200 - OK`. Again, no need to look further, just specify the `status_code` parameter! 351 | 352 | ```python 353 | @app.post("/", status_code=201) 354 | ``` 355 | 356 | ## Other awesome features 357 | 358 | * You can add [examples and example schemas](https://fastapi.tiangolo.com/tutorial/schema-extra-example/) 359 | that will show up in the UI at all levels. Even better, you can have multiple examples for a body, 360 | and the UI will show you a dropdown you can choose from! 361 | * FastAPI supports lots of [types](https://docs.pydantic.dev/usage/types/) out-of-the-box! 362 | By supporting, I mean automatic parsing and validation :fontawesome-regular-face-grin-wink:. 363 | The [pydantic types](https://docs.pydantic.dev/usage/types/#pydantic-types) are especially useful. 364 | Some examples: `EmailStr`, `Color`, `FilePath`, `PastDate`, `HttpUrl`, `SecretStr` (won't be logged), etc. 365 | * The body can be composed: if we add multiple models as parameters, FastAPI will merge them together. 366 | For example, declaring: 367 | ```python 368 | def some_post(admin_user: User, regular_user: User): 369 | ``` 370 | will expect an input like this: 371 | ```json 372 | {"admin_user": {}, "regular_user": {}} 373 | ``` 374 | This avoids the necessity to create extra classes just for composition! 375 | * Model classes based on pydantic's `BaseModel`, which has lots of awesome features. 376 | For example, the methods `.dict()`, `.json()` and `.copy()` come out-of-the-box, and 377 | support including, excluding and renaming fields (among other). 378 | More info [here](https://docs.pydantic.dev/usage/exporting_models/#modeldict). 379 | * To avoid hard-coding HTTP status codes, use FastAPI's `status.HTTP_*` constants. 380 | * The [`json_encoder()`](https://fastapi.tiangolo.com/tutorial/encoder/?h=jsonable_encoder#using-the-jsonable_encoder) 381 | will convert dict, pydantic models, etc. into valid JSON, a bit like `json.dumps`. But contrary to the latter, 382 | the result is not a string, but a dictionary with all values compatible with JSON (think of `datetime` objects). 383 | * For bigger applications, FastAPI uses `APIRouter`, which is equivalent to blueprints in Flask. 384 | For more information, read [Bigger Applications - Multiple Files](https://fastapi.tiangolo.com/tutorial/bigger-applications). 385 | * FastAPI has a very powerful but intuitive Dependency Injection system (DI). This can be used to have shared logic, 386 | share database connections, enforce security (e.g. checking the presence of a valid JWT token), etc. 387 | See [Dependencies - First Steps](https://fastapi.tiangolo.com/tutorial/dependencies/). 388 | 389 | And so much more! -------------------------------------------------------------------------------- /docs/03-celery.md: -------------------------------------------------------------------------------- 1 | 2 | # Celery 3 | 4 | Celery is a task queue with focus on real-time processing, while also supporting task scheduling. 5 | 6 | ## What is Celery 7 | 8 | From their documentation: 9 | 10 | > Task queues are used as a mechanism to distribute work across threads or machines. 11 | > A task queue’s input is a unit of work called a **task**. 12 | > Dedicated **worker processes** constantly monitor task queues for new work to perform. 13 | > 14 | > Celery communicates via **messages**, usually using a **broker** to mediate between clients and workers. 15 | > To initiate a task the **client** adds a message to the queue, the broker then delivers that message to a worker. 16 | > 17 | > A Celery system can consist of multiple workers and brokers, giving way to high availability and horizontal scaling. 18 | > [it] is written in Python, but the protocol can be implemented in any language [(current clients in NodeJS, PHP)]. 19 | 20 | 21 | ![Celery overview](assets/03-celery.excalidraw.png) 22 | 23 | In other words, the entities involved in Celery are: 24 | 25 | * __producers__: also called __clients__, they are the ones requesting tasks and doing something with the results. 26 | * __broker__: the broker is the message transport, used to send and receive messages between producers and workers. 27 | In other words, they store the __task queue__. Celery supports a myriad of message brokers, 28 | but currently only two are feature-complete: :simple-redis: [Redis](https://redis.io/) and 29 | :simple-rabbitmq: [RabbitMQ](https://www.rabbitmq.com/). 30 | * __workers__: the workers are processes that constantly watch the task queue and execute tasks. 31 | * __result backend__: a backend is only necessary when we want to keep track of the tasks' states or retrieve results from tasks. 32 | A result backend is optional but turned on by default, 33 | see [Celery without a Results Backend](https://patrick.cloke.us/posts/2019/07/17/celery-without-a-results-backend/). 34 | 35 | 36 | See also the diagram in [Understanding Celery's architecture](https://subscription.packtpub.com/book/programming/9781783288397/7/ch07lvl1sec45/understanding-celerys-architecture) 37 | 38 | ## Getting started 39 | 40 | ### Launch a broker/backend 41 | 42 | First, we need a *broker* and a *backend*. We will use Redis, as it is both full-featured and easy to use: 43 | ```bash 44 | poetry add 'celery[redis]' 45 | ``` 46 | 47 | We can run Redis locally using: 48 | ```bash 49 | docker run --rm --name some-redis -p 6379:6379 redis:latest 50 | ``` 51 | 52 | !!! tip 53 | 54 | To see what happens exactly inside Redis, download and run a Redis GUI 55 | such as [Another Redis Desktop Manager](https://github.com/qishibo/AnotherRedisDesktopManager). 56 | 57 | ### Create a task 58 | 59 | Now, let's create a task. We first need to create a **Celery instance**, which is the entrypoint to Celery: 60 | may it be submitting tasks (client), managing workers, getting results, etc. We usually call it the Celery 61 | application, or app for short. 62 | 63 | ```python 64 | from celery.app import Celery 65 | 66 | redis_url = os.getenv("REDIS_URL", "redis://localhost:6379") 67 | 68 | celery_app = Celery(__name__, broker=redis_url, backend=redis_url) 69 | ``` 70 | 71 | Now, let's define a dummy *task*, that will create a file with a timestamp: 72 | 73 | ```python 74 | # in file task.py 75 | from celery.app import Celery 76 | from datetime import datetime 77 | import os 78 | 79 | redis_url = os.getenv("REDIS_URL", "redis://localhost:6379") 80 | 81 | app = Celery(__name__, broker=redis_url, backend=redis_url) 82 | 83 | 84 | @app.task 85 | def dummy_task(): 86 | folder = "/tmp/celery" 87 | os.makedirs(folder, exist_ok=True) 88 | now = datetime.now().strftime("%Y-%m-%dT%H:%M:%s") 89 | with open(f"{folder}/task-{now}.txt", "w") as f: 90 | f.write("hello!") 91 | ``` 92 | 93 | To check it works, let's call it directly using the Python REPL (`python`): 94 | 95 | ```python 96 | >>> from fastapi_celery import task 97 | >>> task.dummy_task() 98 | ``` 99 | This should create the file - we called it directly, so Celery was not involved. To execute 100 | this task using Celery, we need to use one of the methods that were added by the decorator 101 | (see [calling tasks](https://docs.celeryq.dev/en/stable/userguide/calling.html#guide-calling)). 102 | The most common is `delay()`, which is a shortcut to `apply_async()`. Those methods will return 103 | an `AsyncResult`, that can be further used to query the status. 104 | 105 | ```python 106 | >>> t = task.dummy_task.delay() 107 | >>> t.status 108 | PENDING 109 | ``` 110 | 111 | Why is it pending? Well, we didn't launch any workers, did we? Let's change that. 112 | 113 | ### Launch a worker 114 | 115 | In another terminal, run: 116 | ```bash 117 | celery --app=fastapi_celery.task.app worker --concurrency=1 --loglevel=DEBUG 118 | ``` 119 | 120 | Now, try again: 121 | ```python 122 | >>> t.status 123 | SUCCESS 124 | ``` 125 | 126 | To ensure this works, try adding a delay in the task: `time.sleep(10)`. 127 | Don't forget to restart the worker, as the method definition changed! 128 | Even better, use `watchdog` to automatically restart the worker: 129 | ```bash 130 | poetry add watchdog --group=dev 131 | watchmedo auto-restart --directory=./fastapi_celery --pattern=task.py -- celery --app=fastapi_celery.task.app worker --concurrency=1 --loglevel=DEBUG 132 | ``` 133 | 134 | ### Parameters and return values 135 | 136 | Now, let's change a bit our dummy task so it receives an argument and returns a result: 137 | ```python 138 | def dummy_task(name='Bob') -> str: 139 | sleep(5) 140 | return f'Hello {name}!' 141 | ``` 142 | 143 | ```python 144 | >>> import importlib 145 | >>> importlib.reload(task) 146 | >>> t = task.dummy_task.delay('Lucy') 147 | >>> t.result # empty until success 148 | >>> t.result 149 | 'Hello Lucy!' 150 | ``` 151 | 152 | Try to return a dictionary instead. It should work the same. But what about this? 153 | 154 | ```python 155 | def dummy_task() -> str: 156 | return open('/tmp/celery/x.txt', 'w') 157 | ``` 158 | 159 | ```python 160 | >>> t = task.dummy_task.delay() 161 | >>> t.status 162 | 'FAILURE' 163 | >>> t.result 164 | EncodeError("TypeError('Object of type TextIOWrapper is not JSON serializable')") 165 | t.successful() 166 | False 167 | ``` 168 | 169 | So beware: results must be JSON-serializable (or match the serialization configured in Celery) 170 | since the results will be serialized and stored in the results backend. 171 | 172 | ## Using Celery with FastAPI 173 | 174 | With those building blocks, we can now bind the two together. We simply import `task.py` in FastAPI, 175 | and call our `task.delay()` from a REST call. We can return the task ID and its status to the user: 176 | 177 | ```python 178 | from fastapi import FastAPI, HTTPException 179 | from pydantic import BaseModel 180 | from celery.result import AsyncResult 181 | 182 | from . import task 183 | 184 | app = FastAPI() 185 | 186 | 187 | class TaskOut(BaseModel): 188 | id: str 189 | status: str 190 | 191 | 192 | @app.get("/start") 193 | def start() -> TaskOut: 194 | r = task.dummy_task.delay() 195 | return _to_task_out(r) 196 | 197 | 198 | @app.get("/status") 199 | def status(task_id: str) -> TaskOut: 200 | r = task.app.AsyncResult(task_id) 201 | return _to_task_out(r) 202 | 203 | 204 | def _to_task_out(r: AsyncResult) -> TaskOut: 205 | return TaskOut(id=r.task_id, status=r.status) 206 | ``` 207 | 208 | 209 | ## Restricting to one task at a time 210 | 211 | Celery doesn't provide an obvious way to limit the number of concurrent tasks. 212 | In our use case, we want to have only one task executed at a time. If the user tries 213 | to start a task while another is already running, he should get an error. 214 | 215 | With multithreading/multiprocessing, a common construct is the mutual exclusion (*mutex*) lock. 216 | The thing is, we have multiple processes here, so we need a lock that lives outside the Python 217 | process. 218 | 219 | As we already have Redis, we can use a [Redis Lock](https://redis-py.readthedocs.io/en/v4.1.2/lock.html)! 220 | But how do we use it? 221 | 222 | Ideally, we would like to get the lock when we start a task (from the REST endpoint - FastAPI), and release 223 | it when the task is finished (from the Celery worker). But a lock should be acquired and released from the 224 | same thread... And worse, if our worker fails to release the lock, we are stuck! 225 | 226 | ![bad use of a lock](assets/03-lock-bad.excalidraw.png) 227 | 228 | A better way is to use the lock from FastAPI only. We cannot know when the task is finished, but we can query 229 | the state of a task given an ID. So let's use the lock to secure the read/write to a Redis key, `current_task_id`, 230 | which holds the ID of the last task! 231 | 232 | ![good use of a lock](assets/03-lock-good.excalidraw.png) 233 | 234 | 246 | 247 | So, for the implementation, let's first create a redis lock: 248 | 249 | ```python 250 | from redis import Redis 251 | from redis.lock import Lock as RedisLock 252 | 253 | redis_instance = Redis.from_url(task.redis_url) 254 | lock = RedisLock(redis_instance, name="task_id") 255 | 256 | REDIS_TASK_KEY = "current_task" 257 | ``` 258 | 259 | The `/start` endpoint now looks like this: 260 | 261 | ```python 262 | @app.get("/start") 263 | def start() -> TaskOut: 264 | try: 265 | if not lock.acquire(blocking_timeout=4): 266 | raise HTTPException(status_code=500, detail="Could not acquire lock") 267 | 268 | task_id = redis_instance.get(REDIS_TASK_KEY) 269 | if task_id is None or task.app.AsyncResult(task_id).ready(): 270 | # no task was ever run, or the last task finished already 271 | r = task.dummy_task.delay() 272 | redis_instance.set(REDIS_TASK_KEY, r.task_id) 273 | return _to_task_out(r) 274 | else: 275 | # the last task is still running! 276 | raise HTTPException( 277 | status_code=400, detail="A task is already being executed" 278 | ) 279 | finally: 280 | lock.release() 281 | ``` 282 | 283 | And for the `/status`, we can now make the `task_id` query parameter optional: 284 | 285 | ```python 286 | @app.get("/status") 287 | def status(task_id: str = None) -> TaskOut: 288 | task_id = task_id or redis_instance.get(REDIS_TASK_KEY) 289 | if task_id is None: 290 | raise HTTPException( 291 | status_code=400, detail=f"Could not determine task {task_id}" 292 | ) 293 | r = task.app.AsyncResult(task_id) 294 | return _to_task_out(r) 295 | ``` 296 | 297 | !!! note 298 | 299 | This code is far from perfect. For example: what happens if the `task_id` is incorrect or 300 | not known by celery? For `/status`, we may just get an error. But for `/start`? 301 | The lock may never be released! This is one of many flows, so don't put it in production 302 | :fontawesome-regular-face-smile-wink: 303 | 304 | ## Canceling long-running tasks 305 | 306 | Maybe we want to cancel the current task. How can we do it? 307 | 308 | The Celery app gives us access to [control](https://docs.celeryq.dev/en/v5.2.7/reference/celery.app.control.html), which lets us get statistics, 309 | how many workers are running, etc. 310 | 311 | ```python 312 | 313 | from . import task 314 | 315 | # note: if id is read from redis, use: 316 | # task_id = redis_instance.get(...).decode('utf-8') 317 | task.app.control.revoke(task_id, terminate=True, signal="SIGKILL") 318 | ``` 319 | 320 | ## Returning results and exceptions 321 | 322 | Simply add a new property to `TaskOut`: 323 | 324 | ```py hl_lines="4" 325 | class TaskOut(BaseModel): 326 | id: str 327 | status: str 328 | result: str | None = None 329 | ``` 330 | 331 | And modify `_to_task_out` like this: 332 | 333 | ```py hl_lines="5" 334 | def _to_task_out(r: AsyncResult) -> TaskOut: 335 | return TaskOut( 336 | id=r.task_id, 337 | status=r.status, 338 | result=r.traceback if r.failed() else r.result, 339 | ) 340 | ``` 341 | 342 | You can try to get the traceback by making the task throw an exception 343 | or return a value, and then calling: 344 | ```bash 345 | curl http://localhost:8000/start 346 | curl http://localhost:8000/status | jq -r '.result' 347 | ``` 348 | -------------------------------------------------------------------------------- /docs/04-notebook.md: -------------------------------------------------------------------------------- 1 | The last piece of the puzzle is to design a task that can execute a Jupyter Notebook. 2 | 3 | 4 | ## A simple notebook 5 | 6 | The most basic notebook of all would look like this: 7 | ```json 8 | --8<-- "notebook.ipynb" 9 | ``` 10 | 11 | It doesn't require any specific dependencies to run except, of course, Jupyter. 12 | 13 | ## The fast and ugly way 14 | 15 | Since the notebook is not meant to change once the app is started, 16 | an easy (but ugly) way is to convert it to a simple Python script (e.g. in CI), 17 | and run it using `execfile`. 18 | 19 | To convert a Jupyter Notebook provided `notebook` (or juypter lab) is installed 20 | is as easy as calling `nbconvert`: 21 | ```bash 22 | jupyter nbconvert --to python notebook.ipynb 23 | ``` 24 | 25 | Then, the task would look like this: 26 | ```python 27 | @app.task("execute_notebook") 28 | def execute_notebook(): 29 | execfile('notebook.py') 30 | ``` 31 | 32 | But we could do better, right? 33 | 34 | ## Using NbConvert API 35 | 36 | If developers use Jupyter Notebook for developing, chances are all the dependencies 37 | are already in the Poetry file. But if we only want to convert a simple notebook, 38 | the only thing we need is `nbconvert` and Python's IpyKernel support: 39 | 40 | ```bash 41 | poetry add nbconvert ipykernel 42 | ``` 43 | 44 | Now that we have this, we can read the notebook directly from the task and execute it: 45 | 46 | ```python 47 | import os 48 | import nbformat 49 | from nbconvert.preprocessors import ExecutePreprocessor, CellExecutionError 50 | 51 | processor = ExecutePreprocessor(timeout=600, kernel_name="python") 52 | 53 | # read the notebook once, as it will never change 54 | with open(os.environ.get("SCRIPT_PATH", "notebook.ipynb")) as f: 55 | notebook = nbformat.read(f, as_version=4) 56 | 57 | 58 | def execute_notebook() -> str: 59 | processor.preprocess(notebook) 60 | # the following will raise an CellExecutionError in case of error 61 | return nbformat.writes(notebook) 62 | ``` 63 | 64 | To convert this into a Celery task, just add the annotation and you are good to go! 65 | 66 | Why is it better you ask? 67 | Well, for one thing, we can now have access to nbconvert's output, 68 | including the stacktrace if something goes wrong! Furthermore, any error will raise an 69 | exception of type `CellExecutionError` that will automatically mark Celery's task as a failure. 70 | 71 | We could even imagine storing the notebook's output itself in the result in case of success. 72 | But beware! Depending on the verbosity of the notebook, it may burden the results backend. Up to you! -------------------------------------------------------------------------------- /docs/05-more.md: -------------------------------------------------------------------------------- 1 | ## Deploying to production 2 | 3 | Now that we have everything, the last step is to make our app ready for production. 4 | 5 | In 2023, wherever you deploy, production usually means one or more docker image, 6 | and either a docker-compose, a Helm Chart or some other packaging. 7 | 8 | I won't go into details here, but have a look at [fastapi-notebook-runner]( 9 | https://github.com/derlin/fastapi-notebook-runner) for an example of: 10 | 11 | * a [Dockerfile](https://github.com/derlin/fastapi-notebook-runner/blob/main/Dockerfile) 12 | * a [docker-compose](https://github.com/derlin/fastapi-notebook-runner/blob/main/docker-compose.yaml) 13 | * a Helm Chart (Kubernetes). 14 | 15 | 16 | The full app uses two Docker images even though it needs 3 processes: 17 | 18 | 1. the app Docker image, which can launch either celery or fastapi depending on the command argument, and 19 | 2. the official redis image. 20 | 21 | Again, have a look at [fastapi-notebook-runner](https://github.com/derlin/fastapi-notebook-runner) 22 | for more details. 23 | 24 | ## Tips 25 | 26 | * [conventional commits](conventionalcommits.org/en/v1.0.0/) 27 | * [black](https://github.com/psf/black), [ruff](https://github.com/charliermarsh/ruff) 28 | and [pytest](https://pytest.org) 29 | * [GitHub Actions](https://docs.github.com/en/actions) 30 | * [MkDocs Material](https://squidfunk.github.io/mkdocs-material) 31 | 32 | ## One more thing 33 | 34 | I really enjoyed putting this site together. If you found it useful, please leave a :star: 35 | on the [GitHub repo](https://github.com/derlin/introduction-to-fastapi-and-celery)! -------------------------------------------------------------------------------- /docs/assets/00-goal.excalidraw.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derlin/introduction-to-fastapi-and-celery/a414be90c321d5b316f75cda57ff1da91aa45aa6/docs/assets/00-goal.excalidraw.png -------------------------------------------------------------------------------- /docs/assets/01-wsgi.excalidraw.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derlin/introduction-to-fastapi-and-celery/a414be90c321d5b316f75cda57ff1da91aa45aa6/docs/assets/01-wsgi.excalidraw.png -------------------------------------------------------------------------------- /docs/assets/03-celery.excalidraw.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derlin/introduction-to-fastapi-and-celery/a414be90c321d5b316f75cda57ff1da91aa45aa6/docs/assets/03-celery.excalidraw.png -------------------------------------------------------------------------------- /docs/assets/03-lock-bad.excalidraw.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derlin/introduction-to-fastapi-and-celery/a414be90c321d5b316f75cda57ff1da91aa45aa6/docs/assets/03-lock-bad.excalidraw.png -------------------------------------------------------------------------------- /docs/assets/03-lock-good.excalidraw.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/derlin/introduction-to-fastapi-and-celery/a414be90c321d5b316f75cda57ff1da91aa45aa6/docs/assets/03-lock-good.excalidraw.png -------------------------------------------------------------------------------- /docs/assets/favicon.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | Produced by OmniGraffle 7.2.2 5 | 2023-03-22 13:44:46 +0000 6 | 7 | 8 | 9 | Canvas 1 10 | 11 | Layer 1 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | -------------------------------------------------------------------------------- /docs/extra.css: -------------------------------------------------------------------------------- 1 | [data-md-color-scheme="default"] { 2 | --md-code-hl-color: #64856514; 3 | } 4 | 5 | [data-md-color-scheme="slate"] { 6 | --md-code-hl-color: #31ca8926; 7 | } 8 | 9 | 10 | img { 11 | margin: auto; 12 | display: block; 13 | } -------------------------------------------------------------------------------- /docs/index.md: -------------------------------------------------------------------------------- 1 | # FastAPI + Celery = ♥ 2 | 3 | :octopus: Hello there! 4 | 5 | *Interested in Python FastAPI? Wondering how to execute long-running tasks in the background 6 | in Python? You came to the right place!* 7 | 8 | This little website will go through the basics of Poetry, FastAPI and Celery, with some detours 9 | here and there. 10 | 11 | !!! Note 12 | 13 | I built this website for a talk given at the GDG Fribourg in **March 2023**. 14 | Please keep that in mind: the technologies may have evolved a little. 15 | Versions used at the time of writing: 16 | 17 | * poetry `1.3.0` 18 | * FastAPI `0.95.0` 19 | * Celery `5.2.7` 20 | 21 | I learned about FastAPI and Celery when confronted with a simple yet interesting use case: 22 | 23 | ![Use case overview](assets/00-goal.excalidraw.png) 24 | 25 | I had a Jupyter Notebook that connected to a database, ran some heavy processing on 26 | the data (using machine learning and everything) and saved aggregated data back to 27 | the database. Since notebooks are great for developing, the requirement was to keep 28 | using notebooks for development *but* to be able to trigger the processing from an API call. 29 | The notebook should never be executed twice in parallel though: the API should thus 30 | return an error if the notebook was already being executed. 31 | Note that the notebook would be provided once at deployment time: it won't change during 32 | the lifecycle of the app. 33 | 34 | I was initially planning to use a simple Flask app, but soon got into trouble[^1]: 35 | how can I (1) run the notebook in a background thread and (2) restrict its 36 | execution to one at a time? 37 | 38 | 39 | 40 | The solution will be explained on those pages. If you are only interested in 41 | the final implementation, have a look at my nb-runner project on GitHub: 42 | 43 | [:simple-github: fastapi-notebook-runner](https://github.com/derlin/fastapi-notebook-runner){ .md-button } 44 | 45 | [^1]: for the story, I first switched to FastAPI for its 46 | [Background Tasks](https://fastapi.tiangolo.com/tutorial/background-tasks/). I thought it would 47 | allow me to avoid Celery. I soon discovered it wasn't enough, as I had no control over the number of tasks. 48 | I thus ended up learning Celery after all. But the switch was still worth it! -------------------------------------------------------------------------------- /mkdocs.yml: -------------------------------------------------------------------------------- 1 | site_name: FastAPI + Celery = ♥ 2 | site_author: Lucy Linder, akka derlin 3 | site_description: >- 4 | Learn about FastAPI and Celery using a practical use case. 5 | 6 | repo_name: derlin/introduction-to-fastapi-and-celery 7 | repo_url: https://github.com/derlin/introduction-to-fastapi-and-celery 8 | 9 | copyright: Copyright © 2023 Lucy Linder @ derlin 10 | 11 | theme: 12 | name: material 13 | favicon: assets/favicon.svg 14 | palette: 15 | - scheme: default 16 | primary: green 17 | accent: light green 18 | toggle: 19 | icon: material/brightness-7 20 | name: Switch to dark mode 21 | - scheme: slate 22 | primary: light green 23 | accent: green 24 | toggle: 25 | icon: material/brightness-4 26 | name: Switch to light mode 27 | features: 28 | - navigation.footer 29 | 30 | nav: 31 | - Introduction: index.md 32 | - Poetry: 01-poetry.md 33 | - FastAPI: 02-fastapi.md 34 | - Celery: 03-celery.md 35 | - Notebooks: 04-notebook.md 36 | - Finishing touches: 05-more.md 37 | 38 | markdown_extensions: 39 | 40 | # Python Markdown 41 | - abbr 42 | - admonition 43 | - attr_list 44 | - def_list 45 | - footnotes 46 | - md_in_html 47 | - toc: 48 | permalink: true 49 | 50 | # Python Markdown Extensions 51 | - pymdownx.arithmatex: 52 | generic: true 53 | - pymdownx.betterem: 54 | smart_enable: all 55 | - pymdownx.caret 56 | - pymdownx.details 57 | - pymdownx.emoji: 58 | emoji_index: !!python/name:materialx.emoji.twemoji 59 | emoji_generator: !!python/name:materialx.emoji.to_svg 60 | - pymdownx.highlight 61 | - pymdownx.inlinehilite 62 | - pymdownx.keys 63 | - pymdownx.mark 64 | - pymdownx.smartsymbols 65 | - pymdownx.superfences 66 | - pymdownx.tabbed: 67 | alternate_style: true 68 | - pymdownx.tasklist: 69 | custom_checkbox: true 70 | - pymdownx.tilde 71 | - pymdownx.snippets 72 | 73 | extra_css: 74 | - extra.css 75 | 76 | extra: 77 | generator: false 78 | social: 79 | - icon: fontawesome/brands/octopus-deploy 80 | link: https://blog.derlin.ch 81 | - icon: fontawesome/brands/github 82 | link: https://github.com/derlin 83 | - icon: fontawesome/brands/linkedin 84 | link: https://www.linkedin.com/in/lucy-linder-4a401726 85 | - icon: material/dev-to 86 | link: https://dev.to/derlin -------------------------------------------------------------------------------- /notebook.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "id": "ea8edd01", 7 | "metadata": {}, 8 | "outputs": [], 9 | "source": [ 10 | "from os import environ\n", 11 | "from time import sleep\n", 12 | "\n", 13 | "sleep(30)\n", 14 | "print(environ.get('SOME_DB_URL'))" 15 | ] 16 | } 17 | ], 18 | "metadata": { 19 | "kernelspec": { 20 | "display_name": "Python 3 (ipykernel)", 21 | "language": "python", 22 | "name": "python3" 23 | }, 24 | "language_info": { 25 | "codemirror_mode": { 26 | "name": "ipython", 27 | "version": 3 28 | }, 29 | "file_extension": ".py", 30 | "mimetype": "text/x-python", 31 | "name": "python", 32 | "nbconvert_exporter": "python", 33 | "pygments_lexer": "ipython3", 34 | "version": "3.11.0" 35 | } 36 | }, 37 | "nbformat": 4, 38 | "nbformat_minor": 5 39 | } 40 | --------------------------------------------------------------------------------