├── ChatGLM.ipynb ├── README.md ├── chatglm.py ├── 星球优惠券.png └── 星球宣传.jpg /README.md: -------------------------------------------------------------------------------- 1 | # ChatGPT-AIGC-Lesson 2 | 3 | * 关于ChatGPT，以及AIGC的学习道路，里面包含的内容很多，有语言模型，还有RLHF，多模态论文等等，公众号是 船长尼莫 ，感兴趣可以关注一下~ 4 | 5 | * 如何免费体验ChatGPT？[网站地址](https://chatgpt.lizm.xyz/#/chat/1002) 6 | 7 | 8 | ## 语言模型介绍 9 | * [大白话介绍语言模型](https://zhuanlan.zhihu.com/p/616900693) 10 | 11 | ### GPT家族 12 | * [GPT-1](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247484573&idx=1&sn=ee965f45d685a9c0fa22a851214048b0&chksm=cf87682cf8f0e13a7d5cc1ca9a93c58bb31560c365dd36b2738ac1109df9557a07fb747ca9c2&token=533251867&lang=zh_CN#rd) 13 | * [GPT-2](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247484582&idx=1&sn=9f77037a30126225d060a0c34f2183d2&chksm=cf876817f8f0e101b51a07359752d469a5ed97aacab28d06dea2ba0a07924277ae5dc41abb0b&token=533251867&lang=zh_CN#rd) 14 | * [GPT-3](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247484598&idx=1&sn=37f8785df6b54ba85e193940fb86c7c4&chksm=cf876807f8f0e1113e4fdc42c30cc572985b43d676ad7a18fba8ab75ba898286d50101b21376&token=1867297293&lang=zh_CN#rd) 15 | * [GPT-3.5](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247484609&idx=1&sn=9785f559284eb412889f6b7ed57b6d31&chksm=cf876870f8f0e166d4fb0e3b9d87821f02a90cb0289882786f4120c2c09a24f1cbdd2803df28&token=1867297293&lang=zh_CN#rd) 16 | * [GPT-3.5中的RLHF详细介绍](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247484651&idx=1&sn=d4dce9677ecc8cd2f019e9717dab09b6&chksm=cf87685af8f0e14c98418c78b6fd862f32a971e4ec19ea4ba35e4827020aaf39f8cdd1388fea&token=1047332211&lang=zh_CN#rd) 17 | 18 | 19 | ### ChatGPT 20 | ChatGPT的训练方式，详细介绍，其中视频的介绍要比文字介绍详细的多 21 | 22 | * [视频介绍](https://www.zhihu.com/zvideo/1622015815397568512) 23 | 24 | * [文字介绍](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247484249&idx=1&sn=9f553fbc649ae8999bef699157df132f&chksm=cf876fe8f8f0e6fe674b8f990e7cf06f417887138cc60e04e1e85116ddf0294ba2b041e9cf6d&token=533251867&lang=zh_CN#rd) 25 | 26 | ### 清华GLM 27 | #### ChatGLM部署文档(Colab) 28 | ![notebook](ChatGLM.ipynb) 29 | 30 | ![python](chatglm.py) 31 | 32 | 33 | #### GLM-130B 详细论文讲解 34 | * [文字介绍](https://zhuanlan.zhihu.com/p/617701482) 35 | 36 | ## Reinforcement LEarning with Human Feedback 37 | ### PPO 算法介绍 38 | * [文字介绍](https://zhuanlan.zhihu.com/p/50322028) 39 | * [代码介绍](https://github.com/zhangchuheng123/Reinforcement-Implementation) 40 | 41 | 42 | 43 | ## 多模态 44 | * [CLIP模型文字介绍](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247484378&idx=1&sn=ced94475181af0788299d763b0d6cc2c&chksm=cf876f6bf8f0e67dde08e47759e96be8033a3eda272cb0db40b0d2238addfe542ac87a9d7b5c&token=533251867&lang=zh_CN#rd) 45 | * [CLIP视频代码讲解](https://www.zhihu.com/zvideo/1624349435185618944) 46 | * [谷歌PaLI](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247484663&idx=1&sn=eb022c7e39faf6278c60541206c4124d&chksm=cf876846f8f0e15042218f3d484abaf5b17f091119e4fc2d9a6e52e17514d876828d8a7e9971&token=1047332211&lang=zh_CN#rd) 47 | 48 | ## 自然语言处理 49 | * [NLP概览1](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247483868&idx=1&sn=239c833bde9fc660ccc506014d70e4bb&chksm=cf876d6df8f0e47b0221c4f1aa415955ba03ff666395926e39e9cd10faec7670a39efe8053a4&token=533251867&lang=zh_CN#rd) 50 | * [NLP概览2](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247483858&idx=1&sn=e0b555e8be0dfbe32f49fd953e4f0788&chksm=cf876d63f8f0e47508feabc5d25d34a091bada3993487334babf5ec005edd225e8bc48001f9b&token=533251867&lang=zh_CN#rd) 51 | 52 | ### NER命名体识别 53 | * [SoftLexicon 知识增强型NER](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247483994&idx=1&sn=98d16f5e74a0382fb3cbd3729ce05a69&chksm=cf876eebf8f0e7fd6cf647a79a83fb4fa3d33782b54a748349d32cd0eb549600cab7a4b617e2&token=533251867&lang=zh_CN#rd) 54 | 55 | * [工业界如何做NER任务？](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247483972&idx=1&sn=2f89e6fa1ba97a8724eb7714f1e05814&chksm=cf876ef5f8f0e7e3ecf87b359e6d268529e125d573e7428ac672c2c4ba5705865a67221f78d4&token=533251867&lang=zh_CN#rd) 56 | 57 | * [如何利用词库做NER增强](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247484213&idx=1&sn=9ba7546fa3d89d96c6658d3bc141b79a&chksm=cf876f84f8f0e6927ae51bbd0a002da67f541cb8527b3614171af8fab1b556424bc6dcc53b1b&token=533251867&lang=zh_CN#rd) 58 | 59 | * [Lexicon Enhanced Chinese Sequence Labelling Using BERT Adapter | 达摩院 | ACL 2021](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247483962&idx=1&sn=f265c054322504db5a2098d558e7077f&chksm=cf876e8bf8f0e79df17cdda0e83300fa6675705b7863cfe5501c93cc93ca85448ca9ebef6a29&token=533251867&lang=zh_CN#rd) 60 | 61 | * [FLAT: Chinese NER Using Flat-Lattice Transformer | 复旦大学| ACL 2020](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247484022&idx=1&sn=c6419d27b631bc010ca0223713519683&chksm=cf876ec7f8f0e7d14983ddb425ed75430dfe08b3a3fa22e4e42abc60e956621d6afa6031f9b5&token=533251867&lang=zh_CN#rd) 62 | 63 | * [Unsupervised Boundary-Aware Language Model Pretraining for Chinese Sequence Labeling | EMNLP 2022](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247484040&idx=1&sn=33a797a940af0ac00e64123438b730ab&chksm=cf876e39f8f0e72fb2920f31f076db9e3865337c9ebfd4427bd101c5748f1a0280b203d3ff6d&token=533251867&lang=zh_CN#rd) 64 | * [NFLAT : Non-Flat-Lattice Transformer for Chinese Named Entity Recognition](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247484192&idx=1&sn=36c5eace19eb11342d7a264ea9d037a2&chksm=cf876f91f8f0e6870b6afb07aa57331b4b9dc598e102dac8b383991e3c5455b76e5308eb82f4&token=533251867&lang=zh_CN#rd) 65 | 66 | ### 意图识别 67 | * [Estimating Soft Labels for Out-of-Domain Intent Detection | 达摩院 | EMNLP](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247483930&idx=1&sn=0216d2e3de50b6f52bfc49357d278955&chksm=cf876eabf8f0e7bdd3f4d95883391dfed257686c2e6c3f3a94f5278772aa81562f54969f85f2&token=533251867&lang=zh_CN#rd) 68 | 69 | 70 | ### 问答系统 71 | 本次的技术围绕着美团的线上问答系统来展开。 72 | * [文档式问答](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247484333&idx=1&sn=5d3eee2765fa9102c4fd24d1d56043ee&chksm=cf876f1cf8f0e60af81d789c819c481a474b5286010e6923bb6f66c1f4ed62d9f3793db5b6a6&token=533251867&lang=zh_CN#rd) 73 | 74 | * [社区式问答](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247484344&idx=1&sn=5d45a000aea86d5d4bfbf6db3a5f2f36&chksm=cf876f09f8f0e61f0c8394db1f35983207bb80d175e8754ca109ac7b63fcc9be31d44f95abf3&token=533251867&lang=zh_CN#rd) 75 | 76 | 以下为问答的学术分享： 77 | 78 | * [Doc2Bot: Accessing Heterogeneous Documents via Conversational Bots | 达摩院 | EMNLP](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247483947&idx=1&sn=bc8beefd8604a5e49dbac9ca95fb8865&chksm=cf876e9af8f0e78cd287d760c0aa812a676179ff318522d2893183af8bf768ff635aa31bcf83&token=533251867&lang=zh_CN#rd) 79 | 80 | 81 | 82 | 83 | ### 对话系统 84 | * [工业界任务导向型对话系统](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247483884&idx=1&sn=5da77605cbbf1f21a6c96ffd710dea1d&chksm=cf876d5df8f0e44b5df9cb1320d3d9e8364797f38db7e5352e0631e289a5a8852fe3d8f01843&token=533251867&lang=zh_CN#rd) 85 | * [工业界检索式对话系统](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247483897&idx=1&sn=68cc926379435ac4c6678cbae656634b&chksm=cf876d48f8f0e45efda1ba547ff1633f6d8555b535bff3e8017b0c23b64ce02ba3910e03a9f8&token=533251867&lang=zh_CN#rd) 86 | 87 | 88 | 89 | ## 未来思考 90 | ### 关于ChatGPT的未来思考 91 | * [马斯克的看法](https://zhuanlan.zhihu.com/p/613730825) 92 | 93 | * [ChatGPT对于未来的影响有哪些？](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247484263&idx=1&sn=f2b233ec8e977862775676ca78ad36fc&chksm=cf876fd6f8f0e6c09d9f0b07e6d18f4350a906359d74d1fde5c4e5fa5f242980b30921e56e3f&token=533251867&lang=zh_CN#rd) 94 | 95 | * [ChatGPT会抢码农的饭碗吗？](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247484205&idx=1&sn=8f68d3a78f066247a19971e2bfa16adc&chksm=cf876f9cf8f0e68a8f0db7b06ce2178946c64689504a565870f93dac711cd11a600c29a81924&token=533251867&lang=zh_CN#rd) 96 | 97 | ### AI思考 98 | * [AI、GPU、芯片的未来](https://mp.weixin.qq.com/s?__biz=Mzg4NzcxNzM0Mg==&mid=2247484049&idx=1&sn=2ed6fc79836791e99dfa5717d6804ea2&chksm=cf876e20f8f0e736bfb603a051d16319919b4f74ffaf01950940dedb126dd95285a917d8e2c4&token=533251867&lang=zh_CN#rd) 99 | 100 | ## 提示工程 101 | * [Prompt Engineering介绍和未来思考](https://zhuanlan.zhihu.com/p/593485398) 102 | 103 | # 知识星球介绍 104 | 关于AIGC和ChatGPT，船长有很多想法，主要是关于算法知识方面，还有落地变现，船长自己成立了一个知识星球，目前早鸟价99元，感兴趣可以来看一下。 105 | 106 | 在星球你将收获到： 107 | 108 | 【免费福利】给球友的重磅福利:无需登陆，无需VPN，即可免费使用chatgpt付费API。 109 | 110 | 【算法知识】AIGC论文介绍，RLHF方法，GPT家族，多模态，有业务算法/学术算法。 111 | 112 | 【算法落地】提供详细的代码视频讲解，模型压缩，模型部署，模型量化等落地方案。 113 | 114 | 【产品落地】ChatGPT应用落地，探索人工智能的商业场景。 115 | 116 | 【大厂资源】星球嘉宾来自各个大厂，你可以得到一对一的提问解答，还有大厂内推渠道。 117 | 118 | 【前沿资讯】ChatGPT最新玩法和咨询，AI绘图和AI创意工具使用，AIGC内容生成。 119 | 120 | 121 | 让我们一起进入AI的时代，共同探索未来！ 122 | 123 | ![宣传海报](星球宣传.jpg) 124 | 125 | 126 | ![优惠券](星球优惠券.png) 127 | -------------------------------------------------------------------------------- /chatglm.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ChatGLM 3 | 4 | Automatically generated by Colaboratory. 5 | 6 | Original file is located at 7 | https://colab.research.google.com/drive/110kV0l9zeVhfvzqz4WJBF7Okzke2m3c6 8 | """ 9 | 10 | # 查看显卡信息 11 | !nvidia-smi 12 | 13 | # clone工程路径 14 | !git clone https://github.com/THUDM/ChatGLM-6B.git 15 | 16 | # 安装命令行 17 | !pip install kora 18 | from kora import console 19 | console.start() 20 | 21 | cd ChatGLM-6B 22 | 23 | ls 24 | 25 | mkdir modelINT4 26 | 27 | cd modelINT4 28 | 29 | cd .. 30 | 31 | cd modelINT4 32 | 33 | # 下载模型文件，INT4量化版本 34 | !wget https://huggingface.co/THUDM/chatglm-6b-int4/resolve/main/LICENSE 35 | !wget https://huggingface.co/THUDM/chatglm-6b-int4/resolve/main/MODEL_LICENSE 36 | !wget https://huggingface.co/THUDM/chatglm-6b-int4/resolve/main/README.md 37 | !wget https://huggingface.co/THUDM/chatglm-6b-int4/resolve/main/config.json 38 | !wget https://huggingface.co/THUDM/chatglm-6b-int4/resolve/main/configuration_chatglm.py 39 | !wget https://huggingface.co/THUDM/chatglm-6b-int4/resolve/main/ice_text.model 40 | !wget https://huggingface.co/THUDM/chatglm-6b-int4/resolve/main/modeling_chatglm.py 41 | !wget https://huggingface.co/THUDM/chatglm-6b-int4/resolve/main/pytorch_model.bin 42 | !wget https://huggingface.co/THUDM/chatglm-6b-int4/resolve/main/quantization.py 43 | !wget https://huggingface.co/THUDM/chatglm-6b-int4/resolve/main/quantization_kernels.c 44 | !wget https://huggingface.co/THUDM/chatglm-6b-int4/resolve/main/quantization_kernels_parallel.c 45 | !wget https://huggingface.co/THUDM/chatglm-6b-int4/resolve/main/tokenization_chatglm.py 46 | !wget https://huggingface.co/THUDM/chatglm-6b-int4/resolve/main/tokenizer_config.json 47 | 48 | ls 49 | 50 | cd .. 51 | 52 | # Commented out IPython magic to ensure Python compatibility. 53 | # # 下载miniconda 54 | # %%bash 55 | # MINICONDA_INSTALLER_SCRIPT=Miniconda3-4.5.4-Linux-x86_64.sh 56 | # MINICONDA_PREFIX=/usr/local 57 | # wget https://repo.continuum.io/miniconda/$MINICONDA_INSTALLER_SCRIPT 58 | # chmod +x $MINICONDA_INSTALLER_SCRIPT 59 | # ./$MINICONDA_INSTALLER_SCRIPT -b -f -p $MINICONDA_PREFIX 60 | 61 | !which conda # 返回/usr/local/bin/conda 62 | 63 | !conda create -n lm python=3.7 64 | 65 | !source activate lm 66 | !conda activate lm 67 | 68 | !conda info --env 69 | 70 | !pip install PyHamcrest==1.9.0 71 | !pip install protobuf==3.19.5 72 | 73 | !conda install transformers 74 | 75 | !pip install icetk 76 | 77 | !pip install cpm_kernels 78 | 79 | !pip install gradio 80 | 81 | cd ChatGLM-6B 82 | 83 | !pip install torch 84 | 85 | # 需要安装的其他包 86 | !pip install --upgrade protobuf icetk cpm_kernels 87 | 88 | # 首次加载模型以及测试 89 | 90 | from transformers import AutoTokenizer, AutoModel 91 | import os 92 | import time 93 | # 指定缓存位置 94 | os.environ["TRANSFORMERS_CACHE"] = "/content/models_cache" 95 | 96 | # 加载模型 97 | tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm-6b-int4", trust_remote_code=True) 98 | model = AutoModel.from_pretrained("THUDM/chatglm-6b-int4",trust_remote_code=True).half().cuda() 99 | model = model.eval() 100 | 101 | # 推理 102 | print("start conversation") 103 | t = time.time() 104 | response, history = model.chat(tokenizer, "你是谁呢呀？", history=[]) 105 | print(response) 106 | 107 | print(f'coast:{time.time() - t:.4f}s') 108 | 109 | # 循环测试反应速度 110 | import locale 111 | locale.getpreferredencoding = lambda: "UTF-8" 112 | 113 | print("start conversation") 114 | for i in range(10): 115 | t = time.time() 116 | response, history = model.chat(tokenizer, "你都能做什么呢", history=[]) 117 | # !nvidia-smi 118 | print(os.system("!nvidia-smi")) 119 | print(response) 120 | print(f'coast:{time.time() - t:.4f}s') 121 | !nvidia-smi 122 | 123 | import locale 124 | locale.getpreferredencoding = lambda: "UTF-8" 125 | !pip uninstall wandb 126 | !pip install wandb 127 | 128 | """# New Section""" 129 | 130 | -------------------------------------------------------------------------------- /星球优惠券.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sherlock1987/Awesome-ChatGPT-AIGC-Lesson/2d973bd7aa73c77a933c418c9986e5e01144b30f/星球优惠券.png -------------------------------------------------------------------------------- /星球宣传.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sherlock1987/Awesome-ChatGPT-AIGC-Lesson/2d973bd7aa73c77a933c418c9986e5e01144b30f/星球宣传.jpg --------------------------------------------------------------------------------