├── 01_Introduction_to_LUMI └── README.md ├── 02_Using_the_LUMI_web_interface ├── Clone_with_JupyterLab.md ├── GPT-neo-IMDB-introduction.ipynb ├── README.md └── images │ ├── step0.png │ ├── step1.png │ └── step2.png ├── 03_Your_first_AI_training_job_on_LUMI ├── GPT-neo-IMDB-finetuning.py ├── README.md ├── images │ └── lumi_web_interface_edit_file.png ├── reference_solution │ ├── GPT-neo-IMDB-finetuning.py │ ├── resume_from_checkpoint │ │ ├── GPT-neo-IMDB-finetuning.py │ │ ├── run.sh │ │ └── util.py │ ├── run.sh │ └── util.py ├── run.sh └── util.py ├── 04_Understanding_GPU_activity_and_checking_jobs ├── README.md ├── images │ └── profile.png └── reference_solution │ └── GPT-neo-IMDB-finetuning-profile.py ├── 05_Running_containers_on_LUMI ├── Hello_LUMI_GPU_World.py ├── README.md ├── examples │ ├── build_ubuntu_tree.sh │ ├── print_directory_trees.sh │ └── ubuntu_tree.def └── reference_solution │ └── reference_solution.md ├── 06_Bulding_containers_from_conda_pip_environments ├── README.md ├── examples │ ├── PandasAI.yml │ ├── minimal_pytorch.yml │ └── python312.yml └── reference_solution │ ├── panopticapi.yml │ ├── python312_extra.yml │ └── reference_solution.md ├── 07_Extending_containers_with_virtual_environments_for_faster_testing ├── README.md └── examples │ └── extending_containers_with_venv.md ├── 08_Scaling_to_multiple_GPUs ├── GPT-neo-IMDB-finetuning.py ├── README.md ├── reference_solution │ ├── GPT-neo-IMDB-finetuning.py │ ├── prints_only_from_single_process │ │ ├── GPT-neo-IMDB-finetuning.py │ │ ├── run_no_torchrun.sh │ │ ├── run_torchrun.sh │ │ ├── slurm-9304946.out │ │ ├── slurm-9304949.out │ │ └── util.py │ ├── run_no_torchrun.sh │ ├── run_torchrun.sh │ ├── util.py │ └── with_cpu_bindings │ │ ├── GPT-neo-IMDB-finetuning.py │ │ ├── run_no_torchrun.sh │ │ ├── run_torchrun.sh │ │ └── util.py ├── run.sh └── util.py ├── 09_Extreme_scale_AI ├── README.md ├── images │ ├── profile-detail.png │ └── profile.png └── reference_solution │ └── README.md ├── 10_Coupling_AI_and_HPC └── README.md ├── LICENSE ├── README.md └── bonus_material ├── README.md └── exercise_container_recipes ├── README.md ├── build_pytorch_transformers.sh └── pytorch_transformers.yml /01_Introduction_to_LUMI/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/01_Introduction_to_LUMI/README.md -------------------------------------------------------------------------------- /02_Using_the_LUMI_web_interface/Clone_with_JupyterLab.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/02_Using_the_LUMI_web_interface/Clone_with_JupyterLab.md -------------------------------------------------------------------------------- /02_Using_the_LUMI_web_interface/GPT-neo-IMDB-introduction.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/02_Using_the_LUMI_web_interface/GPT-neo-IMDB-introduction.ipynb -------------------------------------------------------------------------------- /02_Using_the_LUMI_web_interface/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/02_Using_the_LUMI_web_interface/README.md -------------------------------------------------------------------------------- /02_Using_the_LUMI_web_interface/images/step0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/02_Using_the_LUMI_web_interface/images/step0.png -------------------------------------------------------------------------------- /02_Using_the_LUMI_web_interface/images/step1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/02_Using_the_LUMI_web_interface/images/step1.png -------------------------------------------------------------------------------- /02_Using_the_LUMI_web_interface/images/step2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/02_Using_the_LUMI_web_interface/images/step2.png -------------------------------------------------------------------------------- /03_Your_first_AI_training_job_on_LUMI/GPT-neo-IMDB-finetuning.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/03_Your_first_AI_training_job_on_LUMI/GPT-neo-IMDB-finetuning.py -------------------------------------------------------------------------------- /03_Your_first_AI_training_job_on_LUMI/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/03_Your_first_AI_training_job_on_LUMI/README.md -------------------------------------------------------------------------------- /03_Your_first_AI_training_job_on_LUMI/images/lumi_web_interface_edit_file.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/03_Your_first_AI_training_job_on_LUMI/images/lumi_web_interface_edit_file.png -------------------------------------------------------------------------------- /03_Your_first_AI_training_job_on_LUMI/reference_solution/GPT-neo-IMDB-finetuning.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/03_Your_first_AI_training_job_on_LUMI/reference_solution/GPT-neo-IMDB-finetuning.py -------------------------------------------------------------------------------- /03_Your_first_AI_training_job_on_LUMI/reference_solution/resume_from_checkpoint/GPT-neo-IMDB-finetuning.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/03_Your_first_AI_training_job_on_LUMI/reference_solution/resume_from_checkpoint/GPT-neo-IMDB-finetuning.py -------------------------------------------------------------------------------- /03_Your_first_AI_training_job_on_LUMI/reference_solution/resume_from_checkpoint/run.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/03_Your_first_AI_training_job_on_LUMI/reference_solution/resume_from_checkpoint/run.sh -------------------------------------------------------------------------------- /03_Your_first_AI_training_job_on_LUMI/reference_solution/resume_from_checkpoint/util.py: -------------------------------------------------------------------------------- 1 | ../util.py -------------------------------------------------------------------------------- /03_Your_first_AI_training_job_on_LUMI/reference_solution/run.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/03_Your_first_AI_training_job_on_LUMI/reference_solution/run.sh -------------------------------------------------------------------------------- /03_Your_first_AI_training_job_on_LUMI/reference_solution/util.py: -------------------------------------------------------------------------------- 1 | ../util.py -------------------------------------------------------------------------------- /03_Your_first_AI_training_job_on_LUMI/run.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/03_Your_first_AI_training_job_on_LUMI/run.sh -------------------------------------------------------------------------------- /03_Your_first_AI_training_job_on_LUMI/util.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/03_Your_first_AI_training_job_on_LUMI/util.py -------------------------------------------------------------------------------- /04_Understanding_GPU_activity_and_checking_jobs/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/04_Understanding_GPU_activity_and_checking_jobs/README.md -------------------------------------------------------------------------------- /04_Understanding_GPU_activity_and_checking_jobs/images/profile.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/04_Understanding_GPU_activity_and_checking_jobs/images/profile.png -------------------------------------------------------------------------------- /04_Understanding_GPU_activity_and_checking_jobs/reference_solution/GPT-neo-IMDB-finetuning-profile.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/04_Understanding_GPU_activity_and_checking_jobs/reference_solution/GPT-neo-IMDB-finetuning-profile.py -------------------------------------------------------------------------------- /05_Running_containers_on_LUMI/Hello_LUMI_GPU_World.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/05_Running_containers_on_LUMI/Hello_LUMI_GPU_World.py -------------------------------------------------------------------------------- /05_Running_containers_on_LUMI/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/05_Running_containers_on_LUMI/README.md -------------------------------------------------------------------------------- /05_Running_containers_on_LUMI/examples/build_ubuntu_tree.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/05_Running_containers_on_LUMI/examples/build_ubuntu_tree.sh -------------------------------------------------------------------------------- /05_Running_containers_on_LUMI/examples/print_directory_trees.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/05_Running_containers_on_LUMI/examples/print_directory_trees.sh -------------------------------------------------------------------------------- /05_Running_containers_on_LUMI/examples/ubuntu_tree.def: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/05_Running_containers_on_LUMI/examples/ubuntu_tree.def -------------------------------------------------------------------------------- /05_Running_containers_on_LUMI/reference_solution/reference_solution.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/05_Running_containers_on_LUMI/reference_solution/reference_solution.md -------------------------------------------------------------------------------- /06_Bulding_containers_from_conda_pip_environments/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/06_Bulding_containers_from_conda_pip_environments/README.md -------------------------------------------------------------------------------- /06_Bulding_containers_from_conda_pip_environments/examples/PandasAI.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/06_Bulding_containers_from_conda_pip_environments/examples/PandasAI.yml -------------------------------------------------------------------------------- /06_Bulding_containers_from_conda_pip_environments/examples/minimal_pytorch.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/06_Bulding_containers_from_conda_pip_environments/examples/minimal_pytorch.yml -------------------------------------------------------------------------------- /06_Bulding_containers_from_conda_pip_environments/examples/python312.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/06_Bulding_containers_from_conda_pip_environments/examples/python312.yml -------------------------------------------------------------------------------- /06_Bulding_containers_from_conda_pip_environments/reference_solution/panopticapi.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/06_Bulding_containers_from_conda_pip_environments/reference_solution/panopticapi.yml -------------------------------------------------------------------------------- /06_Bulding_containers_from_conda_pip_environments/reference_solution/python312_extra.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/06_Bulding_containers_from_conda_pip_environments/reference_solution/python312_extra.yml -------------------------------------------------------------------------------- /06_Bulding_containers_from_conda_pip_environments/reference_solution/reference_solution.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/06_Bulding_containers_from_conda_pip_environments/reference_solution/reference_solution.md -------------------------------------------------------------------------------- /07_Extending_containers_with_virtual_environments_for_faster_testing/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/07_Extending_containers_with_virtual_environments_for_faster_testing/README.md -------------------------------------------------------------------------------- /07_Extending_containers_with_virtual_environments_for_faster_testing/examples/extending_containers_with_venv.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/07_Extending_containers_with_virtual_environments_for_faster_testing/examples/extending_containers_with_venv.md -------------------------------------------------------------------------------- /08_Scaling_to_multiple_GPUs/GPT-neo-IMDB-finetuning.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/08_Scaling_to_multiple_GPUs/GPT-neo-IMDB-finetuning.py -------------------------------------------------------------------------------- /08_Scaling_to_multiple_GPUs/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/08_Scaling_to_multiple_GPUs/README.md -------------------------------------------------------------------------------- /08_Scaling_to_multiple_GPUs/reference_solution/GPT-neo-IMDB-finetuning.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/08_Scaling_to_multiple_GPUs/reference_solution/GPT-neo-IMDB-finetuning.py -------------------------------------------------------------------------------- /08_Scaling_to_multiple_GPUs/reference_solution/prints_only_from_single_process/GPT-neo-IMDB-finetuning.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/08_Scaling_to_multiple_GPUs/reference_solution/prints_only_from_single_process/GPT-neo-IMDB-finetuning.py -------------------------------------------------------------------------------- /08_Scaling_to_multiple_GPUs/reference_solution/prints_only_from_single_process/run_no_torchrun.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/08_Scaling_to_multiple_GPUs/reference_solution/prints_only_from_single_process/run_no_torchrun.sh -------------------------------------------------------------------------------- /08_Scaling_to_multiple_GPUs/reference_solution/prints_only_from_single_process/run_torchrun.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/08_Scaling_to_multiple_GPUs/reference_solution/prints_only_from_single_process/run_torchrun.sh -------------------------------------------------------------------------------- /08_Scaling_to_multiple_GPUs/reference_solution/prints_only_from_single_process/slurm-9304946.out: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/08_Scaling_to_multiple_GPUs/reference_solution/prints_only_from_single_process/slurm-9304946.out -------------------------------------------------------------------------------- /08_Scaling_to_multiple_GPUs/reference_solution/prints_only_from_single_process/slurm-9304949.out: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/08_Scaling_to_multiple_GPUs/reference_solution/prints_only_from_single_process/slurm-9304949.out -------------------------------------------------------------------------------- /08_Scaling_to_multiple_GPUs/reference_solution/prints_only_from_single_process/util.py: -------------------------------------------------------------------------------- 1 | ../util.py -------------------------------------------------------------------------------- /08_Scaling_to_multiple_GPUs/reference_solution/run_no_torchrun.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/08_Scaling_to_multiple_GPUs/reference_solution/run_no_torchrun.sh -------------------------------------------------------------------------------- /08_Scaling_to_multiple_GPUs/reference_solution/run_torchrun.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/08_Scaling_to_multiple_GPUs/reference_solution/run_torchrun.sh -------------------------------------------------------------------------------- /08_Scaling_to_multiple_GPUs/reference_solution/util.py: -------------------------------------------------------------------------------- 1 | ../util.py -------------------------------------------------------------------------------- /08_Scaling_to_multiple_GPUs/reference_solution/with_cpu_bindings/GPT-neo-IMDB-finetuning.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/08_Scaling_to_multiple_GPUs/reference_solution/with_cpu_bindings/GPT-neo-IMDB-finetuning.py -------------------------------------------------------------------------------- /08_Scaling_to_multiple_GPUs/reference_solution/with_cpu_bindings/run_no_torchrun.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/08_Scaling_to_multiple_GPUs/reference_solution/with_cpu_bindings/run_no_torchrun.sh -------------------------------------------------------------------------------- /08_Scaling_to_multiple_GPUs/reference_solution/with_cpu_bindings/run_torchrun.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/08_Scaling_to_multiple_GPUs/reference_solution/with_cpu_bindings/run_torchrun.sh -------------------------------------------------------------------------------- /08_Scaling_to_multiple_GPUs/reference_solution/with_cpu_bindings/util.py: -------------------------------------------------------------------------------- 1 | ../util.py -------------------------------------------------------------------------------- /08_Scaling_to_multiple_GPUs/run.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/08_Scaling_to_multiple_GPUs/run.sh -------------------------------------------------------------------------------- /08_Scaling_to_multiple_GPUs/util.py: -------------------------------------------------------------------------------- 1 | ../03_Your_first_AI_training_job_on_LUMI/util.py -------------------------------------------------------------------------------- /09_Extreme_scale_AI/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/09_Extreme_scale_AI/README.md -------------------------------------------------------------------------------- /09_Extreme_scale_AI/images/profile-detail.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/09_Extreme_scale_AI/images/profile-detail.png -------------------------------------------------------------------------------- /09_Extreme_scale_AI/images/profile.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/09_Extreme_scale_AI/images/profile.png -------------------------------------------------------------------------------- /09_Extreme_scale_AI/reference_solution/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/09_Extreme_scale_AI/reference_solution/README.md -------------------------------------------------------------------------------- /10_Coupling_AI_and_HPC/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/10_Coupling_AI_and_HPC/README.md -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/README.md -------------------------------------------------------------------------------- /bonus_material/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/bonus_material/README.md -------------------------------------------------------------------------------- /bonus_material/exercise_container_recipes/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/bonus_material/exercise_container_recipes/README.md -------------------------------------------------------------------------------- /bonus_material/exercise_container_recipes/build_pytorch_transformers.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/bonus_material/exercise_container_recipes/build_pytorch_transformers.sh -------------------------------------------------------------------------------- /bonus_material/exercise_container_recipes/pytorch_transformers.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Lumi-supercomputer/Getting_Started_with_AI_workshop/HEAD/bonus_material/exercise_container_recipes/pytorch_transformers.yml --------------------------------------------------------------------------------