├── LICENSE ├── README.md ├── logs ├── A100_run.txt ├── GPT4-tok-run.txt ├── Muon_run.txt ├── PSGD_run.txt ├── lr_test_runs.txt ├── shrunk_run.txt └── tweaks_run_nosave.txt └── src ├── blank ├── data ├── data.py └── finewebedu10b │ └── download.py ├── model.py ├── plot.py ├── sample.py └── train.py /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/VatsaDev/NanoPoor/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/VatsaDev/NanoPoor/HEAD/README.md -------------------------------------------------------------------------------- /logs/A100_run.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/VatsaDev/NanoPoor/HEAD/logs/A100_run.txt -------------------------------------------------------------------------------- /logs/GPT4-tok-run.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/VatsaDev/NanoPoor/HEAD/logs/GPT4-tok-run.txt -------------------------------------------------------------------------------- /logs/Muon_run.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/VatsaDev/NanoPoor/HEAD/logs/Muon_run.txt -------------------------------------------------------------------------------- /logs/PSGD_run.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/VatsaDev/NanoPoor/HEAD/logs/PSGD_run.txt -------------------------------------------------------------------------------- /logs/lr_test_runs.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/VatsaDev/NanoPoor/HEAD/logs/lr_test_runs.txt -------------------------------------------------------------------------------- /logs/shrunk_run.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/VatsaDev/NanoPoor/HEAD/logs/shrunk_run.txt -------------------------------------------------------------------------------- /logs/tweaks_run_nosave.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/VatsaDev/NanoPoor/HEAD/logs/tweaks_run_nosave.txt -------------------------------------------------------------------------------- /src/blank: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /src/data/data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/VatsaDev/NanoPoor/HEAD/src/data/data.py -------------------------------------------------------------------------------- /src/data/finewebedu10b/download.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/VatsaDev/NanoPoor/HEAD/src/data/finewebedu10b/download.py -------------------------------------------------------------------------------- /src/model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/VatsaDev/NanoPoor/HEAD/src/model.py -------------------------------------------------------------------------------- /src/plot.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/VatsaDev/NanoPoor/HEAD/src/plot.py -------------------------------------------------------------------------------- /src/sample.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/VatsaDev/NanoPoor/HEAD/src/sample.py -------------------------------------------------------------------------------- /src/train.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/VatsaDev/NanoPoor/HEAD/src/train.py --------------------------------------------------------------------------------