├── .github └── workflows │ ├── run_pytest.yml │ └── test_package.yml ├── .gitignore ├── CHANGELOG.md ├── CONTRIBUTING.md ├── LICENSE ├── MANIFEST.in ├── README.md ├── dataset_grouper ├── __init__.py ├── beam_transforms.py ├── beam_transforms_test.py ├── count_utils.py ├── count_utils_test.py ├── data_loaders.py ├── data_loaders_test.py ├── integration_test.py ├── serialization.py ├── serialization_test.py ├── test_utils.py ├── tf_utils.py ├── tf_utils_test.py ├── tfds_pipelines.py └── types.py ├── examples ├── README.md ├── datasets │ ├── README.md │ ├── group_by_dirichlet_process.py │ ├── group_by_feature.py │ ├── group_c4.py │ └── group_randomly.py └── training │ ├── README.md │ ├── dataset_utils.py │ ├── jax_fed_algs.py │ ├── model_utils.py │ ├── train_jax.py │ └── train_tff.py ├── pyproject.toml ├── requirements.txt └── tools └── publish_package.sh /.github/workflows/run_pytest.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/.github/workflows/run_pytest.yml -------------------------------------------------------------------------------- /.github/workflows/test_package.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/.github/workflows/test_package.yml -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/.gitignore -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/CHANGELOG.md -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/CONTRIBUTING.md -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/LICENSE -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | global-exclude *_test.py 2 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/README.md -------------------------------------------------------------------------------- /dataset_grouper/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/dataset_grouper/__init__.py -------------------------------------------------------------------------------- /dataset_grouper/beam_transforms.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/dataset_grouper/beam_transforms.py -------------------------------------------------------------------------------- /dataset_grouper/beam_transforms_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/dataset_grouper/beam_transforms_test.py -------------------------------------------------------------------------------- /dataset_grouper/count_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/dataset_grouper/count_utils.py -------------------------------------------------------------------------------- /dataset_grouper/count_utils_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/dataset_grouper/count_utils_test.py -------------------------------------------------------------------------------- /dataset_grouper/data_loaders.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/dataset_grouper/data_loaders.py -------------------------------------------------------------------------------- /dataset_grouper/data_loaders_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/dataset_grouper/data_loaders_test.py -------------------------------------------------------------------------------- /dataset_grouper/integration_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/dataset_grouper/integration_test.py -------------------------------------------------------------------------------- /dataset_grouper/serialization.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/dataset_grouper/serialization.py -------------------------------------------------------------------------------- /dataset_grouper/serialization_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/dataset_grouper/serialization_test.py -------------------------------------------------------------------------------- /dataset_grouper/test_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/dataset_grouper/test_utils.py -------------------------------------------------------------------------------- /dataset_grouper/tf_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/dataset_grouper/tf_utils.py -------------------------------------------------------------------------------- /dataset_grouper/tf_utils_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/dataset_grouper/tf_utils_test.py -------------------------------------------------------------------------------- /dataset_grouper/tfds_pipelines.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/dataset_grouper/tfds_pipelines.py -------------------------------------------------------------------------------- /dataset_grouper/types.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/dataset_grouper/types.py -------------------------------------------------------------------------------- /examples/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/examples/README.md -------------------------------------------------------------------------------- /examples/datasets/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/examples/datasets/README.md -------------------------------------------------------------------------------- /examples/datasets/group_by_dirichlet_process.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/examples/datasets/group_by_dirichlet_process.py -------------------------------------------------------------------------------- /examples/datasets/group_by_feature.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/examples/datasets/group_by_feature.py -------------------------------------------------------------------------------- /examples/datasets/group_c4.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/examples/datasets/group_c4.py -------------------------------------------------------------------------------- /examples/datasets/group_randomly.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/examples/datasets/group_randomly.py -------------------------------------------------------------------------------- /examples/training/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/examples/training/README.md -------------------------------------------------------------------------------- /examples/training/dataset_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/examples/training/dataset_utils.py -------------------------------------------------------------------------------- /examples/training/jax_fed_algs.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/examples/training/jax_fed_algs.py -------------------------------------------------------------------------------- /examples/training/model_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/examples/training/model_utils.py -------------------------------------------------------------------------------- /examples/training/train_jax.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/examples/training/train_jax.py -------------------------------------------------------------------------------- /examples/training/train_tff.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/examples/training/train_tff.py -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/pyproject.toml -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/requirements.txt -------------------------------------------------------------------------------- /tools/publish_package.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google-parfait/dataset_grouper/HEAD/tools/publish_package.sh --------------------------------------------------------------------------------