├── .gitignore ├── docker-compose.yml ├── generate-requirements-full.sh ├── .github ├── ISSUE_TEMPLATE │ └── general.md └── PULL_REQUEST_TEMPLATE.md ├── requirements-core.txt ├── .circleci ├── config.yml └── test_image.py ├── LICENSE.md ├── CONTRIBUTING.md ├── Dockerfile ├── CODE_OF_CONDUCT.md ├── requirements-full.txt ├── README.md └── CHANGELOG.md /.gitignore: -------------------------------------------------------------------------------- 1 | .venv 2 | .vscode 3 | -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | services: 2 | update-deps: 3 | build: 4 | context: . 5 | dockerfile: ./Dockerfile 6 | target: update-deps 7 | volumes: 8 | - .:/app 9 | stdin_open: true 10 | tty: true 11 | working_dir: /app 12 | -------------------------------------------------------------------------------- /generate-requirements-full.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # Run this script to update requirements-core.txt. 3 | # It uses Docker to ensure that the environment matches what will be used in the production image. 4 | set -e 5 | docker compose run --rm update-deps /bin/sh -c "uv pip compile --output-file=requirements-full.txt --upgrade requirements-core.txt" 6 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/general.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: General 3 | about: Ask a question, report a potential issue, etc. 4 | title: '' 5 | labels: '' 6 | assignees: '' 7 | 8 | --- 9 | 10 | **Note:** Civis employees should _not_ use the GitHub Issues feature at the public "civis-python" codebase 11 | to file a ticket, and should instead use the internal ticketing system. 12 | -------------------------------------------------------------------------------- /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | --- 4 | 5 | - [ ] (For Civis employees only) Reference to a relevant ticket in the pull request title 6 | - [ ] Changelog entry added to `CHANGELOG.md` at the repo's root level 7 | - [ ] Description of change in the pull request description 8 | - [ ] If applicable, unit tests have been added and/or updated 9 | - [ ] The CircleCI builds have all passed 10 | -------------------------------------------------------------------------------- /requirements-core.txt: -------------------------------------------------------------------------------- 1 | # awscli v2 is not officially available on PyPI (https://github.com/aws/aws-cli/issues/4947). 2 | # Specifying awscli in requirements-core.txt here ensures that it (and its transitive dependencies) 3 | # are taken into account when generating the final requirements-full.txt file. 4 | awscli @ git+https://github.com/aws/aws-cli@2.27.48 5 | boto3==1.39.2 6 | civis==2.7.1 7 | numpy==2.3.1 8 | pandas==2.3.0 9 | polars==1.31.0 10 | requests==2.32.4 11 | scikit-learn==1.7.0 12 | scipy==1.16.0 13 | -------------------------------------------------------------------------------- /.circleci/config.yml: -------------------------------------------------------------------------------- 1 | version: 2.1 2 | jobs: 3 | build: 4 | docker: 5 | - image: cimg/python:3.12 6 | steps: 7 | - checkout 8 | - setup_remote_docker 9 | - run: 10 | name: Build image 11 | command: docker build --target test -t ds-python . 12 | - run: 13 | name: Verify build completed 14 | command: docker run ds-python /bin/bash -c "echo BUILDS OK" 15 | - run: 16 | name: Run tests 17 | command: docker run ds-python /test_image.py -vv 18 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | Copyright (c) 2017, Civis Analytics 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without 5 | modification, are permitted provided that the following conditions are met: 6 | 7 | 1. Redistributions of source code must retain the above copyright notice, this 8 | list of conditions and the following disclaimer. 9 | 10 | 2. Redistributions in binary form must reproduce the above copyright notice, 11 | this list of conditions and the following disclaimer in the documentation 12 | and/or other materials provided with the distribution. 13 | 14 | 3. Neither the name of the copyright holder nor the names of its contributors 15 | may be used to endorse or promote products derived from this software 16 | without specific prior written permission. 17 | 18 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 19 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 20 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 21 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 22 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 23 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 24 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 25 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 26 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 27 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing to datascience-python 2 | 3 | We welcome bug reports and pull requests from everyone! 4 | This project is intended to be a safe, welcoming space for collaboration, and 5 | contributors are expected to adhere to the 6 | [Contributor Covenant](http://contributor-covenant.org) code of conduct. 7 | 8 | 9 | ## Getting Started 10 | 11 | There are two ways to contribute: 12 | 13 | ### File an Issue 14 | 15 | If you find a bug or think of a useful improvement, 16 | file an issue in this GitHub repository. Make sure to 17 | include details about what you think should be changed 18 | and why. If you're reporting a bug, please provide 19 | a minimum working example so that a maintainer can 20 | reproduce the bug. 21 | 22 | ### Modify the Docker image 23 | 24 | If you know exactly what needs to change, you can also 25 | submit a pull request to propose the change. 26 | 27 | 1. Fork it ( https://github.com/civisanalytics/datascience-python/fork ). 28 | 2. Make sure you are able to build the Docker image locally (`docker build -t datascience-python:test .`) 29 | 3. Create a feature branch (`git checkout -b my-new-feature`). 30 | 4. Make your change. 31 | 5. Make sure the new image still builds correctly. Test that your change is present in the new build. 32 | 6. Commit your changes (`git commit -am 'Add some feature'`). 33 | 7. Push to the branch (`git push origin my-new-feature`). 34 | 8. Create a new pull request with details about your changes. 35 | 9. If the build fails, address any issues. 36 | 37 | ## Tips 38 | 39 | - Don’t forget to add your change to the [CHANGELOG](CHANGELOG.md). See 40 | [Keep a CHANGELOG](http://keepachangelog.com/) for guidelines. 41 | 42 | Thank you for taking the time to contribute! 43 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | ARG PLATFORM=linux/x86_64 2 | ARG BASE_IMAGE=python:3.13.5-slim 3 | 4 | FROM --platform=$PLATFORM $BASE_IMAGE AS uv-installed 5 | 6 | # Disable pip warnings https://stackoverflow.com/a/72551258 7 | ENV PIP_ROOT_USER_ACTION=ignore 8 | 9 | LABEL maintainer=support@civisanalytics.com 10 | 11 | RUN DEBIAN_FRONTEND=noninteractive apt-get update -y --no-install-recommends && \ 12 | apt-get install -y --no-install-recommends locales && \ 13 | locale-gen en_US.UTF-8 && \ 14 | apt-get install -y --no-install-recommends software-properties-common && \ 15 | apt-get install -y --no-install-recommends \ 16 | make \ 17 | automake \ 18 | libpq-dev \ 19 | libffi-dev \ 20 | gfortran \ 21 | g++ \ 22 | git \ 23 | libboost-program-options-dev \ 24 | libtool \ 25 | libxrender1 \ 26 | wget \ 27 | ca-certificates \ 28 | curl \ 29 | mandoc \ 30 | unzip && \ 31 | apt-get clean -y && \ 32 | rm -rf /var/lib/apt/lists/* 33 | 34 | # Install uv. 35 | ADD https://astral.sh/uv/0.7.19/install.sh /uv-installer.sh 36 | RUN sh /uv-installer.sh && rm /uv-installer.sh 37 | ENV PATH="/root/.local/bin/:$PATH" \ 38 | UV_SYSTEM_PYTHON=1 39 | 40 | # This is the primary build target used for the production image 41 | FROM --platform=$PLATFORM uv-installed AS production 42 | 43 | COPY requirements-full.txt . 44 | 45 | RUN uv pip install --no-progress --no-cache -r requirements-full.txt && \ 46 | rm requirements-full.txt 47 | 48 | # Instruct joblib to use disk for temporary files. Joblib defaults to 49 | # /shm when that directory is present. In the Docker container, /shm is 50 | # present but defaults to 64 MB. 51 | # https://github.com/joblib/joblib/blob/0.11/joblib/parallel.py#L328L342 52 | ENV JOBLIB_TEMP_FOLDER=/tmp 53 | 54 | ENV VERSION=8.3.0 \ 55 | VERSION_MAJOR=8 \ 56 | VERSION_MINOR=3 \ 57 | VERSION_MICRO=0 58 | 59 | # This build target is for testing in CircleCI. 60 | FROM --platform=$PLATFORM production AS test 61 | COPY .circleci/test_image.py . 62 | COPY CHANGELOG.md . 63 | 64 | # This build target is for updating dependencies. 65 | # See generate-requirements.full.sh. 66 | FROM --platform=$PLATFORM uv-installed AS update-deps 67 | CMD ["/bin/bash"] 68 | 69 | # Default to the production build target. 70 | FROM production 71 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Code of Conduct 2 | 3 | As contributors and maintainers of this project, and in the interest of 4 | fostering an open and welcoming community, we pledge to respect all people who 5 | contribute through reporting issues, posting feature requests, updating 6 | documentation, submitting pull requests or patches, and other activities. 7 | 8 | We are committed to making participation in this project a harassment-free 9 | experience for everyone, regardless of level of experience, gender, gender 10 | identity and expression, sexual orientation, disability, personal appearance, 11 | body size, race, ethnicity, age, religion, or nationality. 12 | 13 | Examples of unacceptable behavior by participants include: 14 | 15 | * The use of sexualized language or imagery 16 | * Personal attacks 17 | * Trolling or insulting/derogatory comments 18 | * Public or private harassment 19 | * Publishing other's private information, such as physical or electronic 20 | addresses, without explicit permission 21 | * Other unethical or unprofessional conduct 22 | 23 | Project maintainers have the right and responsibility to remove, edit, or 24 | reject comments, commits, code, wiki edits, issues, and other contributions 25 | that are not aligned to this Code of Conduct, or to ban temporarily or 26 | permanently any contributor for other behaviors that they deem inappropriate, 27 | threatening, offensive, or harmful. 28 | 29 | By adopting this Code of Conduct, project maintainers commit themselves to 30 | fairly and consistently applying these principles to every aspect of managing 31 | this project. Project maintainers who do not follow or enforce the Code of 32 | Conduct may be permanently removed from the project team. 33 | 34 | This Code of Conduct applies both within project spaces and in public spaces 35 | when an individual is representing the project or its community. 36 | 37 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 38 | reported by contacting a project maintainer at opensource@civisanalytics.com. 39 | All complaints will be reviewed and investigated and will result in a response 40 | that is deemed necessary and appropriate to the circumstances. Maintainers are 41 | obligated to maintain confidentiality with regard to the reporter of an 42 | incident. 43 | 44 | 45 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], 46 | version 1.3.0, available at 47 | [http://contributor-covenant.org/version/1/3/0/][version] 48 | 49 | [homepage]: http://contributor-covenant.org 50 | [version]: http://contributor-covenant.org/version/1/3/0/ 51 | -------------------------------------------------------------------------------- /requirements-full.txt: -------------------------------------------------------------------------------- 1 | # This file was autogenerated by uv via the following command: 2 | # uv pip compile --output-file=requirements-full.txt requirements-core.txt 3 | attrs==25.3.0 4 | # via 5 | # jsonschema 6 | # referencing 7 | awscli @ git+https://github.com/aws/aws-cli@6804a17061546394f88e1fe6d1bf9b24cd8a09ec 8 | # via -r requirements-core.txt 9 | awscrt==0.26.1 10 | # via awscli 11 | boto3==1.39.2 12 | # via -r requirements-core.txt 13 | botocore==1.39.2 14 | # via 15 | # boto3 16 | # s3transfer 17 | certifi==2025.6.15 18 | # via requests 19 | charset-normalizer==3.4.2 20 | # via requests 21 | civis==2.7.1 22 | # via -r requirements-core.txt 23 | click==8.2.1 24 | # via civis 25 | cloudpickle==3.1.1 26 | # via civis 27 | colorama==0.4.6 28 | # via awscli 29 | distro==1.8.0 30 | # via awscli 31 | docutils==0.19 32 | # via awscli 33 | idna==3.10 34 | # via requests 35 | jmespath==1.0.1 36 | # via 37 | # awscli 38 | # boto3 39 | # botocore 40 | joblib==1.5.1 41 | # via 42 | # civis 43 | # scikit-learn 44 | jsonref==1.1.0 45 | # via civis 46 | jsonschema==4.24.0 47 | # via civis 48 | jsonschema-specifications==2025.4.1 49 | # via jsonschema 50 | numpy==2.3.1 51 | # via 52 | # -r requirements-core.txt 53 | # pandas 54 | # scikit-learn 55 | # scipy 56 | pandas==2.3.0 57 | # via -r requirements-core.txt 58 | polars==1.31.0 59 | # via -r requirements-core.txt 60 | prompt-toolkit==3.0.38 61 | # via awscli 62 | python-dateutil==2.9.0 63 | # via 64 | # awscli 65 | # botocore 66 | # pandas 67 | pytz==2025.2 68 | # via pandas 69 | pyyaml==6.0.2 70 | # via civis 71 | referencing==0.36.2 72 | # via 73 | # jsonschema 74 | # jsonschema-specifications 75 | requests==2.32.4 76 | # via 77 | # -r requirements-core.txt 78 | # civis 79 | rpds-py==0.26.0 80 | # via 81 | # jsonschema 82 | # referencing 83 | ruamel-yaml==0.17.21 84 | # via awscli 85 | ruamel-yaml-clib==0.2.12 86 | # via awscli 87 | s3transfer==0.13.0 88 | # via boto3 89 | scikit-learn==1.7.0 90 | # via -r requirements-core.txt 91 | scipy==1.16.0 92 | # via 93 | # -r requirements-core.txt 94 | # scikit-learn 95 | six==1.17.0 96 | # via python-dateutil 97 | tenacity==9.1.2 98 | # via civis 99 | threadpoolctl==3.6.0 100 | # via scikit-learn 101 | tzdata==2025.2 102 | # via pandas 103 | urllib3==1.26.20 104 | # via 105 | # awscli 106 | # botocore 107 | # requests 108 | wcwidth==0.2.13 109 | # via prompt-toolkit 110 | -------------------------------------------------------------------------------- /.circleci/test_image.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import os 4 | import re 5 | import shutil 6 | import subprocess 7 | import unittest 8 | 9 | 10 | # Just use the stdlib's `unittest` rather than needing to install `pytest`. 11 | class TestImage(unittest.TestCase): 12 | 13 | def test_version(self): 14 | version_in_env_var = os.getenv("VERSION") 15 | major = os.getenv("VERSION_MAJOR") 16 | minor = os.getenv("VERSION_MINOR") 17 | micro = os.getenv("VERSION_MICRO") 18 | self.assertTrue(major.isdigit()) 19 | self.assertTrue(minor.isdigit()) 20 | self.assertTrue(micro.isdigit()) 21 | self.assertEqual(version_in_env_var, f"{major}.{minor}.{micro}") 22 | 23 | with open("CHANGELOG.md") as changelog: 24 | version_in_changelog = re.search( 25 | r"##\s+\[(\d+\.\d+\.\d+)]", changelog.read() 26 | ).groups()[0] 27 | self.assertEqual(version_in_changelog, version_in_env_var) 28 | 29 | def test_scipy_links_to_openblas(self): 30 | from scipy.linalg import _fblas # noqa: F401 31 | 32 | def test_numpy_can_import(self): 33 | import numpy as np # noqa: F401 34 | 35 | def test_sklearn_can_import(self): 36 | import sklearn # noqa: F401 37 | 38 | def test_civis_can_import(self): 39 | import civis # noqa: F401 40 | # civis-python uses lazy imports since v2.3.0, 41 | # so try to import the top-level modules. 42 | import civis.io # noqa: F401 43 | import civis.parallel # noqa: F401 44 | import civis.futures # noqa: F401 45 | import civis.ml # noqa: F401 46 | import civis.utils # noqa: F401 47 | 48 | def test_shell_commands_available(self): 49 | """Ensure the main shell commands are available.""" 50 | # A non-exhaustive list of commands -- we just test those we'd likely use. 51 | expected_cmds = "aws civis curl git pip python unzip uv wget".split() 52 | for cmd in expected_cmds: 53 | self.assertIsNotNone(shutil.which(cmd), f"{cmd} not found in PATH") 54 | 55 | def _test_shell_command(self, cmd: str): 56 | """Check if the shell command runs successfully in the image.""" 57 | try: 58 | subprocess.check_call(cmd, shell=True) 59 | except subprocess.CalledProcessError as e: 60 | self.fail( 61 | f"apt-get test failed with return code {e.returncode}\n" 62 | f"stdout: {e.stdout}\n" 63 | f"stderr: {e.stderr}" 64 | ) 65 | 66 | def test_apt_get(self): 67 | """Ensure that apt-get works in the image.""" 68 | self._test_shell_command("apt-get update -y && apt-get install -y htop") 69 | 70 | def test_uv(self): 71 | """Ensure that uv works in the image.""" 72 | self._test_shell_command("uv pip install python-iso639") 73 | 74 | 75 | if __name__ == "__main__": 76 | unittest.main() 77 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Data Science Docker Image 2 | 3 | [![CircleCI](https://circleci.com/gh/civisanalytics/datascience-python/tree/master.svg?style=svg)](https://circleci.com/gh/civisanalytics/datascience-python/tree/master) 4 | 5 | If you are reading this README on DockerHub, then the links to files in the GitHub repository will be broken. Please read this documentation from [GitHub](https://github.com/civisanalytics/datascience-python) instead. 6 | 7 | # Introduction 8 | 9 | This repository defines the "[civisanalytics/datascience-python](https://hub.docker.com/r/civisanalytics/datascience-python/)" 10 | Docker image. This Docker image provides an environment with data science tools 11 | from the Python ecosystem. This image is the execution environment for Python 12 | jobs in the [Civis data science platform](https://civisanalytics.com/products/civis-platform/), 13 | and it includes the Civis [Python API client](https://github.com/civisanalytics/civis-python). 14 | 15 | # Installation 16 | 17 | Either build the Docker image locally 18 | ```bash 19 | docker build -t datascience-python . 20 | ``` 21 | 22 | or download the image from DockerHub 23 | ```bash 24 | docker pull civisanalytics/datascience-python:latest 25 | ``` 26 | 27 | The `latest` tag (Docker's default if you don't specify a tag) 28 | will give you the most recently-built version of the datascience-python 29 | image. You can replace the tag `latest` with a version number such as `1.0` 30 | to retrieve a reproducible environment. 31 | 32 | # Usage 33 | 34 | Inside the datascience-python Docker image, Python packages are installed in the `root` 35 | environment. For a full list of included Python libraries, see the 36 | [requirements-core.txt](requirements-core.txt) file. 37 | 38 | To start a Docker container from the datascience-python image and 39 | interact with it from a bash prompt, use 40 | ```bash 41 | docker run -i -t civisanalytics/datascience-python:latest /bin/bash 42 | ``` 43 | 44 | You can run a Python command with 45 | ```bash 46 | docker run civisanalytics/datascience-python:latest python -c "import pandas; print(pandas.__version__)" 47 | ``` 48 | 49 | The image contains environment variables which allow you to find 50 | the current version. There are four environment variables defined: 51 | ``` 52 | VERSION 53 | VERSION_MAJOR 54 | VERSION_MINOR 55 | VERSION_MICRO 56 | ``` 57 | VERSION contains the full version string, e.g., "1.0.3". VERSION_MAJOR, 58 | VERSION_MINOR, and VERSION_MICRO each contain a single integer. 59 | 60 | ## Joblib Temporary Files 61 | 62 | The [`joblib`](https://pythonhosted.org/joblib/) library enhances multiprocessing 63 | capabilities for scientific Python computing. In particular, the `scikit-learn` 64 | library uses `joblib` for parallelization. This Docker image sets `joblib`'s 65 | default location for staging temporary files to the /tmp directory. 66 | The normal default is /shm. /shm is a RAM disk which defaults to a 64 MB size 67 | in Docker containers, too small for typical scientific computing. 68 | 69 | # Updating Existing Package Versions 70 | 1. Update versions of existing packages in `requirements-core.txt` 71 | 2. Run script `generate-requirements-full.sh` 72 | 73 | # Creating Equivalent Local Environments 74 | 1. Create a new python environment `python -m venv .venv`. 75 | 2. Activate your new python environment `source .venv/bin/activate` 76 | 3. Install requirements.txt `pip install -r requirements-full.txt` 77 | 78 | # Contributing 79 | 80 | See [CONTRIBUTING](CONTRIBUTING.md) for information about contributing to this project. 81 | 82 | If you make any changes, be sure to build a container to verify that it successfully completes: 83 | ```bash 84 | docker build -t datascience-python:test . 85 | ``` 86 | and describe any changes in the [change log](CHANGELOG.md). 87 | 88 | ## For Maintainers 89 | 90 | This repo has autobuild enabled. Any PR that is merged to master will 91 | be built as the `latest` tag on DockerHub. 92 | Once you are ready to create a new version, go to the "releases" tab of the repository and click 93 | "Draft a new release". GitHub will prompt you to create a new tag, release title, and release 94 | description. The tag should use semantic versioning in the form "vX.X.X"; "major.minor.micro". 95 | The title of the release should be the same as the tag. Include a change log in the release description. 96 | Once the release is tagged, DockerHub will automatically build three identical containers, with labels 97 | "major", "major.minor", and "major.minor.micro". 98 | 99 | # License 100 | 101 | BSD-3 102 | 103 | See [LICENSE.md](LICENSE.md) for details. 104 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # Change Log 2 | 3 | All changes to this project will be documented in this file. 4 | 5 | Version number changes (major.minor.micro) in this package denote the following: 6 | - A micro version will increase if the only change in a release is incrementing micro versions (bugfix-only releases) on the packages contained in this image. 7 | - A minor version will increase if one or more packages contained in the Docker image add new, backwards-compatible features, or if a new package is added to the Docker image. 8 | - A major version will increase if there are any backwards-incompatible changes in any of the packages contained in this Docker image, or any other backwards-incompatible changes in the execution environment. 9 | 10 | ## Unreleased 11 | 12 | ## [8.3.0] 13 | 14 | - Python version updated: 3.12.8 -> 3.13.5 15 | - uv version updated: 0.5.18 -> 0.7.19 16 | - New core dependencies added: 17 | * polars 1.31.0 (supported by civis-python since v2.6.0) 18 | - Core dependencies updated to latest versions: 19 | * awscli 2.22.33 -> 2.27.48 20 | * boto3 1.35.97 -> 1.39.2 21 | * civis 2.4.3 -> 2.7.1 22 | * numpy 2.2.1 -> 2.3.1 23 | * pandas 2.2.3 -> 2.3.0 24 | * requests 2.32.3 -> 2.32.4 25 | * scikit-learn 1.6.1 -> 1.7.0 26 | * scipy 1.15.1 -> 1.16.0 27 | 28 | ## [8.2.0] 29 | 30 | - Python version updated: 3.12.7 -> 3.12.8 31 | - uv version updated: 0.5.1 -> 0.5.18 32 | - Core dependencies updated to latest versions: 33 | * awscli 2.19.5 -> 2.22.33 34 | * boto3 1.35.58 -> 1.35.97 35 | * civis 2.4.0 -> 2.4.3 36 | * numpy 2.1.3 -> 2.2.1 37 | * scikit-learn 1.5.2 -> 1.6.1 38 | * scipy 1.14.1 -> 1.15.1 39 | 40 | ## [8.1.0] 41 | 42 | - Python version updated to v3.12.7 43 | - Core dependencies updated to latest versions: 44 | * awscli 2.17.37 -> 2.19.5 45 | * boto3 1.34.127 -> 1.35.58 46 | * civis 2.3.0 -> 2.4.0 47 | * numpy 2.0.0 -> 2.1.3 48 | * pandas 2.2.2 -> 2.2.3 49 | * scikit-learn 1.5.0 -> 1.5.2 50 | * scipy 1.13.1 -> 1.14.1 51 | - uv added to the image 52 | 53 | ## [8.0.1] 54 | 55 | - Python version updated to v3.12.6 56 | 57 | ## [8.0.0] 58 | - Core dependencies updated to latest versions: 59 | * awscli 1.33.9 -> 2.17.37 60 | - Python version updated to v3.12.5 61 | - Fixes apt-get for debian package installations 62 | 63 | ## [7.3.0] 64 | - Core dependencies updated to latest versions: 65 | * awscli 1.32.112 -> 1.33.9 66 | * boto3 1.34.112 -> 1.34.127 67 | * civis 2.1.0 -> 2.3.0 68 | * numpy 1.26.4 -> 2.0.0 69 | * requests 2.32.2 -> 2.32.3 70 | - Python version updated to v3.12.4 71 | 72 | ## [7.2.0] 73 | - Core dependencies updated to latest versions: 74 | * awscli 1.32.109 -> 1.32.112 75 | * boto3 1.34.109 -> 1.34.112 76 | * civis 2.0.0 -> 2.1.0 77 | * scipy 1.13.0 -> 1.13.1 78 | 79 | ## [7.1.0] 80 | - Python updated to v3.12.3 81 | - Core dependencies updated to latest versions: 82 | * awscli 1.29.5 -> 1.32.109 83 | * boto3 1.28.5 -> 1.34.109 84 | * civis 1.16.1 -> 2.0.0 85 | * numpy 1.25.1 -> 1.26.4 86 | * pandas 2.0.3 -> 2.2.2 87 | * requests 2.31.0 -> 2.32.2 88 | * scikit-learn 1.3.0 -> 1.5.0 89 | * scipy 1.11.1 -> 1.13.0 90 | 91 | ## [7.0.0] 92 | - Python updated to v3.11.4 93 | - Refactors Dockerfile to use the official Python docker image 94 | - Removes Conda Dependency 95 | - Keeps only the core data science python packages 96 | 97 | ## [6.5.1] 98 | ### Package Updates 99 | - Pin cffi at 1.14.0 (#85) 100 | 101 | ## [6.5.0] 102 | ### Package Updates 103 | - civis 1.15.1 -> 1.16.0 (#84) 104 | - Removed pubnub - not considered a breaking change, as it was a dependency for `civis` (#84) 105 | 106 | ## [6.4.0] 107 | ### Package Updates 108 | - PyYAML 3.13 -> 5.2.0 109 | 110 | ## [6.3.1] 111 | ### Package Updates 112 | - civis 1.15.0 -> 1.15.1 (#81) 113 | 114 | ## [6.3.0] 115 | ### Package Updates 116 | - civis 1.14.0 -> 1.15.0 (#79) 117 | - tensorflow 1.15.2 -> 1.15.4 (#79) 118 | 119 | ## [6.2.1] 120 | ### Package Updates 121 | - civis 1.14.0 -> 1.14.1 (#78) 122 | 123 | ## [6.2.0] 124 | ### Package Updates 125 | - civis 1.13.0 -> 1.14.0 (#77) 126 | - muffnn 2.3.0 -> 2.3.1 (#77) 127 | - tensorflow 1.13.1 -> 1.15.2 (#77) 128 | 129 | ## [6.1.0] 130 | ### Package Updates 131 | - civis 1.12.1 -> 1.13.0 (#76) 132 | 133 | ## [6.0.0] 134 | ### Changed 135 | - Python 3.7.1 -> 3.7.6 (#74) 136 | - Conda 4.6.8 -> 4.8.1 (#74) 137 | 138 | ### Removed Packages 139 | - removes python-simple-hipchat (#75) 140 | 141 | ### New Packages 142 | - explicitly installs pip=20.0.2 (#75) 143 | 144 | ### Package Updates 145 | - awscli 1.16.121 -> 1.17.15 (#75) 146 | - beautifulsoup4 4.7.1 -> 4.8.2 (#75) 147 | - botocore 1.12.111 -> 1.14.15 (#75) 148 | - boto3 1.9.111 -> 1.11.15 (#75) 149 | - bqplot 0.11.5 -> 0.12.3 (#75) 150 | - civis 1.9.4 -> 1.12.1 (#74) 151 | - civisml-extensions 0.1.10 -> 0.2.1 (#74) 152 | - cloudpickle 0.8.0 -> 1.2.2 (#74) 153 | - cython 0.29.6 -> 0.29.15 (#75) 154 | - dask 1.1.4 -> 2.10.1 (#75) 155 | - dropbox 9.3.0 -> 9.4.0 (#75) 156 | - ipython 7.3.0 -> 7.12.0 (#75) 157 | - ipywidgets 7.4.2 -> 7.5.1 (#75) 158 | - jinja2 2.10 -> 2.11.1 (#75) 159 | - joblib 0.11.0 -> 0.14.1 (#74) 160 | - jsonschema 3.0.1 -> 3.2.0 (#75) 161 | - libtiff 4.0.10 -> 4.1.0 (#75) 162 | - libxml2 2.9.8 -> 2.9.10 (#75) 163 | - matplotlib 3.0.3 -> 3.1.3 (#75) 164 | - muffnn 2.2.0 -> 2.3.0 (#74) 165 | - nomkl 1.0 -> 3.0 (#74) 166 | - notebook 5.7.5 -> 6.0.3 (#75) 167 | - numexpr 2.6.9 -> 2.7.1 (#75) 168 | - numpy 1.16.2 -> 1.17.3 (#74) 169 | - openblas 0.3.5 -> 0.3.6 (#68, #70) 170 | - pandas 0.24.1 -> 0.25.3 (#74) 171 | - psycopg2 2.7.7 -> 2.8.4 (#75) 172 | - pubnub 4.1.2 -> 4.3.0 (#75) 173 | - pyarrow 0.12.1 -> 0.16.0 (#75) 174 | - pytest 4.3.0 -> 5.3.5 (#75) 175 | - requests 2.21.0 -> 2.22.0 (#75) 176 | - s3fs 0.2.0 -> 0.4.0 (#75) 177 | - scipy 1.2.0 -> 1.4.1 (#74) 178 | - scikit-learn 0.19.2 -> 0.22.1 (#74) 179 | - seaborn 0.9.0 -> 0.10.0 (#75) 180 | - statsmodels 0.9.0 -> 0.11.0 (#75) 181 | - urllib3 1.24.1 -> 1.25.7 (#75) 182 | 183 | ### Added 184 | - added buildspecs for autobuilding and pushing Docker image to Amazon ECR (#69) 185 | 186 | ## [5.0.0] - 2019-03-12 187 | ### Changed 188 | - Ubuntu 14.04 -> 18.04 (#67) 189 | - python 3.6.4 -> 3.7.1 190 | - conda 4.3.30 -> 4.6.8 191 | 192 | ### New Packages 193 | - explicitly installs click=6.7 194 | 195 | ### Package Updates 196 | - awscli 1.15.4 -> 1.16.121 197 | - beautifulsoup4 4.5.3 -> 4.7.1 198 | - botocore 1.10.4 -> 1.12.111 199 | - boto 2.46.1 -> 2.49.0 200 | - boto3 1.7.4 -> 1.9.111 201 | - bqplot 0.10.2 -> 0.11.5 202 | - cloudpickle 0.5.2 -> 0.8.0 203 | - cython 0.27.3 -> 0.29.6 204 | - dask 0.17.2 -> 1.1.4 205 | - ipython 6.1.0 -> 7.3.0 206 | - ipywidgets 7.1.0 -> 7.4.2 207 | - jinja2 2.9.6 -> 2.10 208 | - jsonschema 2.5.1 -> 3.0.1 209 | - libtiff 4.0.6 -> 4.0.10 210 | - libxml2 2.9.2 -> 2.9.8 211 | - matplotlib 2.2.2 -> 3.0.3 212 | - notebook 5.4.1 -> 5.7.5 213 | - numexpr 2.6.2 -> 2.6.9 214 | - numpy 1.13.3 -> 1.16.2 215 | - openblas 0.2.20 -> 0.3.5 216 | - pandas 0.22.0 -> 0.24.1 217 | - patsy 0.4.1 -> 0.5.1 218 | - psycopg2 2.6.2 -> 2.7.7 219 | - pyarrow 0.8.0 -> 0.12.1 220 | - pytest 3.1.3 -> 4.3.0 221 | - pyyaml 3.12 -> 3.13 222 | - requests 2.18.4 -> 2.21.0 223 | - s3fs 0.1.2 -> 0.2.0 224 | - seaborn 0.8 -> 0.9.0 225 | - scipy 1.0.1 -> 1.2.0 226 | - scikit-learn 0.19.1 -> 0.19.2 227 | - statsmodels 0.8.0 -> 0.9.0 228 | - urllib3 1.22 -> 1.24.1 229 | - xgboost 0.6a2 -> 0.81 230 | - civis 1.9.0 -> 1.9.4 231 | - civisml-extensions 0.1.8 -> 0.1.10 232 | - dropbox 7.1.1 -> 9.3.0 233 | - glmnet 2.0.0 -> 2.1.1 234 | - muffnn 2.1.0 -> 2.2.0 235 | - pubnub 4.0.13 -> 4.1.2 236 | - requests-toolbelt 0.8.0 -> 0.9.1 237 | - tensorflow 1.7.0 -> 1.13.1 238 | 239 | ### Maintenance 240 | - Update CircleCI config to v2 (#62). 241 | - Test that tensorflow imports successfully (#67). 242 | 243 | ## [4.2.0] - 2018-04-26 244 | ### Package Updates 245 | - civis 1.8.1 -> 1.9.0 246 | - civisml-extensions 0.1.6 -> 0.1.8 247 | - muffnn 2.0.0 -> 2.1.0 248 | 249 | - dask 0.15.4 (pip) -> 0.17.2 (conda) 250 | - tensorflow 1.4.1 -> 1.7.0 251 | - ipython 6.1.0 -> 6.3.1 252 | - matplotlib 2.1.0 -> 2.2.2 253 | - notebook 5.2.2 -> 5.4.1 254 | - scipy 1.0.0 -> 1.0.1 255 | - urllib3 1.22 (pip) -> 1.22 (conda) 256 | 257 | ## [4.1.0] - 2018-04-19 258 | ### Added 259 | - Added a link in the README directing users who may be reading documentation on DockerHub to instead go to GitHub (#56). 260 | 261 | ### Package Updates 262 | - awscli 1.11.75 (from pip) -> 1.15.4 (from conda) 263 | - botocore 1.5.38 -> 1.10.4 264 | - boto3 1.5.11 -> 1.7.4 265 | 266 | ## [4.0.1] - 2018-02-01 267 | ### Package Updates 268 | - civis 1.8.0 -> 1.8.1 269 | 270 | ## [4.0.0] - 2018-01-23 271 | ### New packages 272 | - bqplot 0.10.2 273 | - feather-format 0.4.0 274 | 275 | ### Changed 276 | - Updated Python version from 3.6.2 to 3.6.4. 277 | 278 | ### Package Updates 279 | - civis 1.7.1 -> 1.8.0 280 | - civisml-extensions 0.1.5 -> 0.1.6 281 | - muffnn 1.2.0 -> 2.0.0 282 | - cloudpickle 0.5.1 -> 0.5.2 283 | - dask 0.15.4 -> 0.16.1 284 | - ftputil 3.3.1 -> 3.4 285 | - tensorflow 1.4.0 -> 1.4.1 286 | - boto3 1.4.5 -> 1.5.11 287 | - cython 0.26 -> 0.27.3 288 | - openblas 0.2.19 -> 0.2.20 289 | - pandas 0.21.0 -> 0.22.0 290 | - pyarrow 0.7.1 -> 0.8.0 291 | - scipy 0.19.1 -> 1.0.0 292 | - ipywidgets 7.0.0 -> 7.1.0 293 | - notebook 5.2.0 -> 5.2.2 294 | 295 | ### Fixed 296 | - Enabled widgetsnbextension so that ipywidgets works. 297 | - Suppress irrelevant warning from tensorflow v1.4 298 | 299 | ## [3.3.0] - 2017-11-17 300 | ### Package Updates 301 | - civis 1.6.2 -> 1.7.1 302 | 303 | ### New packages 304 | - civisml-extensions 0.1.5 305 | - dask 0.15.4 306 | - s3fs 0.1.2 307 | 308 | ### Changed 309 | - Moved conda to version 4.3.30 310 | 311 | ### Package Updates 312 | - boto3 1.4.4 -> 1.4.5 313 | - matplotlib 2.0.2 -> 2.1.0 314 | - numpy 1.13.1 -> 1.13.3 315 | - pandas 0.20.3 -> 0.21.0 316 | - pyarrow 0.5.0 -> 0.7.1 317 | - scikit-learn 0.19.0 -> 0.19.1 318 | - cloudpickle 0.3.1 -> 0.5.1 319 | - muffnn 1.1.2 -> 1.2.0 320 | - pubnub 4.0.12 -> 4.0.13 321 | - tensorflow 1.2.1 -> 1.4.0 322 | 323 | ## [3.2.0] - 2017-09-11 324 | ### Package Updates 325 | - scikit-learn 0.18.2 -> 0.19.0 326 | - civis 1.6.0 -> 1.6.2 327 | - requests 2.14.2 -> 2.18.4 328 | - urllib3 1.19 -> 1.22 329 | 330 | ## [3.1.0] - 2017-07-31 331 | ### New packages 332 | - cloudpickle 0.3.1 333 | - pyarrow 0.5.0 (from conda-forge) 334 | 335 | ### Python 336 | - Update from v3.6.1 to v3.6.2 337 | 338 | ### Package Updates 339 | - civis 1.5.2 -> 1.6.0 340 | - cython 0.25.2 -> 0.26 341 | - ipython 6.0.0 -> 6.1.0 342 | - jinja2 2.8 -> 2.9.6 343 | - numpy 1.12.1 -> 1.13.1 344 | - pandas 0.20.1 -> 0.20.3 345 | - pytest 3.0.5 -> 3.1.3 346 | - seaborn 0.7.1 -> 0.8 347 | - scipy 0.19.0 -> 0.19.1 348 | - scikit-learn 0.18.1 -> 0.18.2 349 | - pubnub 4.0.10 -> 4.0.12 350 | - requests-toolbelt 0.7.1 -> 0.8.0 351 | - tensorflow 1.1.0 -> 1.2.1 352 | 353 | ### Changed 354 | - Install xgboost from conda-forge instead of from PyPI 355 | 356 | ### Fixes 357 | - Use /tmp for joblib temporary files instead of /shm 358 | 359 | 360 | ## [3.0.1] - 2017-05-25 361 | ### Package Updates 362 | - muffnn 1.1.1 -> 1.1.2 363 | 364 | ## [3.0.0] - 2017-05-17 365 | ### Package updates 366 | - civis 1.4.0 -> 1.5.2 367 | - ipython 5.1.0 -> 6.0.0 368 | - matplotlib 2.0.0 -> 2.0.2 369 | - pandas 0.19.2 -> 0.20.1 370 | - requests 2.13.0 -> 2.14.2 371 | 372 | ## [2.2.0] - 2017-05-02 373 | ### Removed 374 | - Remove pinned conda installs of `libgcc` and `libsodium`. This prevented use of the environment file in OS X, and they are dependencies automatically installed by conda in the Docker image build. 375 | 376 | ### Additions 377 | - Explicitly added `botocore` v1.5.38. We had `botocore` installed before (it's a dependency of other AWS libraries), but we're now explicitly including the version number. 378 | 379 | ### Package updates 380 | - python 3.6.0 -> 3.6.1 381 | - awscli 1.11.60 -> 1.11.75 382 | - boto 2.45.0 -> 2.46.1 383 | - boto3 1.4.3 -> 1.4.4 384 | - numpy 1.12.0 -> 1.12.1 385 | - pubnub 4.0.8 -> 4.0.10 386 | - requests 2.12.4 -> 2.13.0 387 | - scipy 0.18.1 -> 0.19.0 388 | - muffnn 1.0.0 -> 1.1.1 389 | - tensorflow 1.0.0 -> 1.1.0 390 | 391 | ## [2.1.0] - 2017-03-17 392 | ### Changed 393 | - Upgrade `civis` to v1.4.0 (#25) 394 | - Changed name of environment in `environment.yml` file to `datascience` (but conda install is still `root`). (#26) 395 | - Removed a few Docker layers. (#26) 396 | - Cleared more of the `apt-get`, `pip` and `conda` caches. (#26) 397 | - Added a test of the `numpy` install. (#26) 398 | 399 | ## [2.0.1] - 2017-03-10 400 | ### Fixed 401 | - Update `awscli` to prevent pip conflict with `botocore`: awscli 1.11.27 --> 1.11.60 (#22) 402 | 403 | ## [2.0.0] - 2017-03-09 404 | ### API-breaking changes 405 | - Removed `nltk` package 406 | - Moved package install environment from "datascience" to the root environment 407 | - Upgrade `matplotlib` to v2.0.0 408 | 409 | ### Added 410 | - glmnet 2.0.0 (#16) 411 | - joblib 0.11.0 (#19) 412 | - tensorflow 1.0.0 (#15) 413 | - xgboost 0.6a2 (#15) 414 | - muffnn 1.0.0 (Civis deep learning library #16) 415 | - pubnub 4.0.8 (Used by Civis API client v1.3 #16) 416 | - requests-toolbelt 0.7.1 (Optional for Civis API client v1.3 #16) 417 | 418 | ### Changed 419 | - Moved conda to version 4.3.11 (#11) 420 | - Moved python to version 3.6.0 (#11) 421 | - Various package version changes (#11, #13) 422 | - beautifulsoup4 4.5.1 --> 4.5.3 423 | - boto 2.43.0 --> 2.45.0 424 | - boto3 1.4.2 --> 1.4.3 425 | - cython 0.25.1 --> 0.25.2 426 | - matplotlib 1.5.3 --> 2.0.0 427 | - numexpr 2.6.1 --> 2.6.2 428 | - numpy 1.11.3 --> 1.12.0 429 | - pandas 0.19.1 --> 0.19.2 430 | - pyyaml 3.11 --> 3.12 431 | - requests 2.12.1 --> 2.12.4 432 | - statsmodels 0.6.1 --> 0.8.0 433 | - Upgrade `civis` to v1.3 (#16) 434 | - Moved package install from the `datascience` environment to `root` (#8) 435 | - Clean the conda cache and skip recommended package installs to reduce Docker image size (#18) 436 | 437 | ### Removed 438 | - nltk 3.2.2 (#20) 439 | 440 | ## [1.1.0] - 2017-02-13 441 | ### Changed 442 | - Add environment variables which record the image version number (#1) 443 | - Upgraded civis library from v1.1.0 to v1.2.0 (#5) 444 | - Various changes to packages to avoid installing MKL (#9) 445 | - Pinned conda to 4.1.11 (#9) 446 | 447 | ### Fixed 448 | - Changed location of `matplotlibrc` to always be found (#6) 449 | - Add `matplotlibrc` file which changes the backend default to "Agg" (#3) 450 | - Add `source activate datascience` to `etc/profile` to ensure bash enjoys the datascience environment (#4) 451 | 452 | ## [1.0.0] - 2017-01-17 453 | 454 | * Initial Release 455 | --------------------------------------------------------------------------------