├── README.md └── figs ├── Figure-1-[The electricity consumption comparison between countries and AI].pdf ├── Figure-10-[Traditional and typical dynamic transformers].pdf ├── Figure-11-[Replacing various components of VIT].pdf ├── Figure-13-[Fine-tuning architecture].pdf ├── Figure-14-[Model compression techniques for LLMs].pdf ├── Figure-2-[The evolutionary trace of foundation models].pdf ├── Figure-3-[The evolutionary trace of foundation models].pdf ├── Figure-4a-[Weight storage and FLOPs cost of different FMs].pdf ├── Figure-4b-[Weight storage and FLOPs cost of different FMs].pdf ├── Figure-5-[Inference FLOPs].pdf ├── Figure-6-[The cost of each module in multimodal models].pdf ├── Figure-7-[Parameters and FLOPs percentage of Stable Diffusion].pdf ├── Figure-9a-[Illustrations of efficient attentions].pdf ├── Figure-9b-[Illustrations of efficient attentions].pdf ├── Figure-9c-[Illustrations of efficient attentions].pdf ├── Figure-9d-[Illustrations of efficient attentions].pdf └── example.png /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UbiquitousLearning/Efficient_Foundation_Model_Survey/HEAD/README.md -------------------------------------------------------------------------------- /figs/Figure-1-[The electricity consumption comparison between countries and AI].pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UbiquitousLearning/Efficient_Foundation_Model_Survey/HEAD/figs/Figure-1-[The electricity consumption comparison between countries and AI].pdf -------------------------------------------------------------------------------- /figs/Figure-10-[Traditional and typical dynamic transformers].pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UbiquitousLearning/Efficient_Foundation_Model_Survey/HEAD/figs/Figure-10-[Traditional and typical dynamic transformers].pdf -------------------------------------------------------------------------------- /figs/Figure-11-[Replacing various components of VIT].pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UbiquitousLearning/Efficient_Foundation_Model_Survey/HEAD/figs/Figure-11-[Replacing various components of VIT].pdf -------------------------------------------------------------------------------- /figs/Figure-13-[Fine-tuning architecture].pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UbiquitousLearning/Efficient_Foundation_Model_Survey/HEAD/figs/Figure-13-[Fine-tuning architecture].pdf -------------------------------------------------------------------------------- /figs/Figure-14-[Model compression techniques for LLMs].pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UbiquitousLearning/Efficient_Foundation_Model_Survey/HEAD/figs/Figure-14-[Model compression techniques for LLMs].pdf -------------------------------------------------------------------------------- /figs/Figure-2-[The evolutionary trace of foundation models].pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UbiquitousLearning/Efficient_Foundation_Model_Survey/HEAD/figs/Figure-2-[The evolutionary trace of foundation models].pdf -------------------------------------------------------------------------------- /figs/Figure-3-[The evolutionary trace of foundation models].pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UbiquitousLearning/Efficient_Foundation_Model_Survey/HEAD/figs/Figure-3-[The evolutionary trace of foundation models].pdf -------------------------------------------------------------------------------- /figs/Figure-4a-[Weight storage and FLOPs cost of different FMs].pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UbiquitousLearning/Efficient_Foundation_Model_Survey/HEAD/figs/Figure-4a-[Weight storage and FLOPs cost of different FMs].pdf -------------------------------------------------------------------------------- /figs/Figure-4b-[Weight storage and FLOPs cost of different FMs].pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UbiquitousLearning/Efficient_Foundation_Model_Survey/HEAD/figs/Figure-4b-[Weight storage and FLOPs cost of different FMs].pdf -------------------------------------------------------------------------------- /figs/Figure-5-[Inference FLOPs].pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UbiquitousLearning/Efficient_Foundation_Model_Survey/HEAD/figs/Figure-5-[Inference FLOPs].pdf -------------------------------------------------------------------------------- /figs/Figure-6-[The cost of each module in multimodal models].pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UbiquitousLearning/Efficient_Foundation_Model_Survey/HEAD/figs/Figure-6-[The cost of each module in multimodal models].pdf -------------------------------------------------------------------------------- /figs/Figure-7-[Parameters and FLOPs percentage of Stable Diffusion].pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UbiquitousLearning/Efficient_Foundation_Model_Survey/HEAD/figs/Figure-7-[Parameters and FLOPs percentage of Stable Diffusion].pdf -------------------------------------------------------------------------------- /figs/Figure-9a-[Illustrations of efficient attentions].pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UbiquitousLearning/Efficient_Foundation_Model_Survey/HEAD/figs/Figure-9a-[Illustrations of efficient attentions].pdf -------------------------------------------------------------------------------- /figs/Figure-9b-[Illustrations of efficient attentions].pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UbiquitousLearning/Efficient_Foundation_Model_Survey/HEAD/figs/Figure-9b-[Illustrations of efficient attentions].pdf -------------------------------------------------------------------------------- /figs/Figure-9c-[Illustrations of efficient attentions].pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UbiquitousLearning/Efficient_Foundation_Model_Survey/HEAD/figs/Figure-9c-[Illustrations of efficient attentions].pdf -------------------------------------------------------------------------------- /figs/Figure-9d-[Illustrations of efficient attentions].pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UbiquitousLearning/Efficient_Foundation_Model_Survey/HEAD/figs/Figure-9d-[Illustrations of efficient attentions].pdf -------------------------------------------------------------------------------- /figs/example.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/UbiquitousLearning/Efficient_Foundation_Model_Survey/HEAD/figs/example.png --------------------------------------------------------------------------------