├── Appendix-RKDO-Teachable-Teacher.md ├── LICENSE ├── README.md ├── RKDO_benchmark.ipynb ├── RKDO_benchmark_latest.ipynb ├── RKDO_benchmark_latest_efficient.ipynb ├── arxiv-RKDO-v1.0.0.pdf ├── figures ├── Computational Efficiency- RDO at 2 Epochs vs. ICon at 5 Epochs.png ├── Early Learning- RDO vs. ICon at 1 and 2 Epochs.png ├── RDO Loss Improvement Over ICon.png ├── RDO vs ICon - loss across training duration.png ├── figure1-training_loss.png └── figure2.png ├── pyproject.toml ├── som_rkdo ├── output │ ├── combined_som_comparison.gif │ ├── kl_comparison.png │ ├── rkdo_som_mosaic.gif │ └── standard_som_mosaic.gif └── som_rkdo.ipynb ├── uv.lock └── visualizations ├── .DS_Store ├── amari_animation.py ├── animation.py ├── comparison_animation.py ├── curved_manifold.py ├── data ├── 2504.21707v1.pdf ├── mean_teacher_lively.gif ├── mean_teacher_simplex_prototype.png ├── rkdo_evolution.gif ├── rkdo_grad.gif ├── rkdo_grad_lively.gif ├── rkdo_kaleidoscope_12.gif ├── rkdo_kaleidoscope_split.gif ├── rkdo_kaleidoscope_split_8.gif ├── rkdo_simplex_inset.gif ├── rkdo_vs_mt_vs_byol.gif └── teacher_student_comparison.gif ├── geodesic_wall.py ├── gradient_animation.py ├── kaleidoscope.py ├── manifold_frames ├── frame_000.png ├── frame_002.png ├── frame_004.png ├── frame_006.png ├── frame_008.png ├── frame_010.png ├── frame_012.png ├── frame_014.png ├── frame_016.png ├── frame_018.png ├── frame_020.png ├── frame_022.png ├── frame_024.png ├── frame_026.png ├── frame_028.png ├── frame_030.png ├── frame_032.png ├── frame_034.png ├── frame_036.png ├── frame_038.png ├── frame_040.png ├── frame_042.png ├── frame_044.png ├── frame_046.png └── frame_048.png ├── mean_teacher_lively.py ├── rkdo_evolution.gif ├── rkdo_grad.gif └── rkdo_vs_mt_vs_byol.gif /Appendix-RKDO-Teachable-Teacher.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/Appendix-RKDO-Teachable-Teacher.md -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/README.md -------------------------------------------------------------------------------- /RKDO_benchmark.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/RKDO_benchmark.ipynb -------------------------------------------------------------------------------- /RKDO_benchmark_latest.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/RKDO_benchmark_latest.ipynb -------------------------------------------------------------------------------- /RKDO_benchmark_latest_efficient.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/RKDO_benchmark_latest_efficient.ipynb -------------------------------------------------------------------------------- /arxiv-RKDO-v1.0.0.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/arxiv-RKDO-v1.0.0.pdf -------------------------------------------------------------------------------- /figures/Computational Efficiency- RDO at 2 Epochs vs. ICon at 5 Epochs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/figures/Computational Efficiency- RDO at 2 Epochs vs. ICon at 5 Epochs.png -------------------------------------------------------------------------------- /figures/Early Learning- RDO vs. ICon at 1 and 2 Epochs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/figures/Early Learning- RDO vs. ICon at 1 and 2 Epochs.png -------------------------------------------------------------------------------- /figures/RDO Loss Improvement Over ICon.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/figures/RDO Loss Improvement Over ICon.png -------------------------------------------------------------------------------- /figures/RDO vs ICon - loss across training duration.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/figures/RDO vs ICon - loss across training duration.png -------------------------------------------------------------------------------- /figures/figure1-training_loss.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/figures/figure1-training_loss.png -------------------------------------------------------------------------------- /figures/figure2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/figures/figure2.png -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/pyproject.toml -------------------------------------------------------------------------------- /som_rkdo/output/combined_som_comparison.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/som_rkdo/output/combined_som_comparison.gif -------------------------------------------------------------------------------- /som_rkdo/output/kl_comparison.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/som_rkdo/output/kl_comparison.png -------------------------------------------------------------------------------- /som_rkdo/output/rkdo_som_mosaic.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/som_rkdo/output/rkdo_som_mosaic.gif -------------------------------------------------------------------------------- /som_rkdo/output/standard_som_mosaic.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/som_rkdo/output/standard_som_mosaic.gif -------------------------------------------------------------------------------- /som_rkdo/som_rkdo.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/som_rkdo/som_rkdo.ipynb -------------------------------------------------------------------------------- /uv.lock: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/uv.lock -------------------------------------------------------------------------------- /visualizations/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/.DS_Store -------------------------------------------------------------------------------- /visualizations/amari_animation.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/amari_animation.py -------------------------------------------------------------------------------- /visualizations/animation.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/animation.py -------------------------------------------------------------------------------- /visualizations/comparison_animation.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/comparison_animation.py -------------------------------------------------------------------------------- /visualizations/curved_manifold.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/curved_manifold.py -------------------------------------------------------------------------------- /visualizations/data/2504.21707v1.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/data/2504.21707v1.pdf -------------------------------------------------------------------------------- /visualizations/data/mean_teacher_lively.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/data/mean_teacher_lively.gif -------------------------------------------------------------------------------- /visualizations/data/mean_teacher_simplex_prototype.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/data/mean_teacher_simplex_prototype.png -------------------------------------------------------------------------------- /visualizations/data/rkdo_evolution.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/data/rkdo_evolution.gif -------------------------------------------------------------------------------- /visualizations/data/rkdo_grad.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/data/rkdo_grad.gif -------------------------------------------------------------------------------- /visualizations/data/rkdo_grad_lively.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/data/rkdo_grad_lively.gif -------------------------------------------------------------------------------- /visualizations/data/rkdo_kaleidoscope_12.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/data/rkdo_kaleidoscope_12.gif -------------------------------------------------------------------------------- /visualizations/data/rkdo_kaleidoscope_split.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/data/rkdo_kaleidoscope_split.gif -------------------------------------------------------------------------------- /visualizations/data/rkdo_kaleidoscope_split_8.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/data/rkdo_kaleidoscope_split_8.gif -------------------------------------------------------------------------------- /visualizations/data/rkdo_simplex_inset.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/data/rkdo_simplex_inset.gif -------------------------------------------------------------------------------- /visualizations/data/rkdo_vs_mt_vs_byol.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/data/rkdo_vs_mt_vs_byol.gif -------------------------------------------------------------------------------- /visualizations/data/teacher_student_comparison.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/data/teacher_student_comparison.gif -------------------------------------------------------------------------------- /visualizations/geodesic_wall.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/geodesic_wall.py -------------------------------------------------------------------------------- /visualizations/gradient_animation.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/gradient_animation.py -------------------------------------------------------------------------------- /visualizations/kaleidoscope.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/kaleidoscope.py -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_000.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_000.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_002.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_002.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_004.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_004.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_006.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_006.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_008.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_008.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_010.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_010.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_012.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_012.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_014.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_014.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_016.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_016.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_018.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_018.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_020.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_020.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_022.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_022.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_024.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_024.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_026.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_026.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_028.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_028.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_030.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_030.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_032.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_032.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_034.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_034.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_036.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_036.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_038.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_038.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_040.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_040.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_042.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_042.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_044.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_044.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_046.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_046.png -------------------------------------------------------------------------------- /visualizations/manifold_frames/frame_048.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/manifold_frames/frame_048.png -------------------------------------------------------------------------------- /visualizations/mean_teacher_lively.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/mean_teacher_lively.py -------------------------------------------------------------------------------- /visualizations/rkdo_evolution.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/rkdo_evolution.gif -------------------------------------------------------------------------------- /visualizations/rkdo_grad.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/rkdo_grad.gif -------------------------------------------------------------------------------- /visualizations/rkdo_vs_mt_vs_byol.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/anthonymartin/RKDO-recursive-kl-divergence-optimization/HEAD/visualizations/rkdo_vs_mt_vs_byol.gif --------------------------------------------------------------------------------