├── figures ├── icon.png ├── wechat.jpg ├── new_logo.png ├── new_logo2.png ├── XHSlong750px.png └── performance.png ├── dots1_tech_report.pdf ├── docker ├── Dockerfile.vllm └── README.md ├── LICENSE ├── README.md ├── dots.llm1 LICENSE AGREEMENT └── NOTICE /figures/icon.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rednote-hilab/dots.llm1/HEAD/figures/icon.png -------------------------------------------------------------------------------- /figures/wechat.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rednote-hilab/dots.llm1/HEAD/figures/wechat.jpg -------------------------------------------------------------------------------- /dots1_tech_report.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rednote-hilab/dots.llm1/HEAD/dots1_tech_report.pdf -------------------------------------------------------------------------------- /figures/new_logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rednote-hilab/dots.llm1/HEAD/figures/new_logo.png -------------------------------------------------------------------------------- /figures/new_logo2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rednote-hilab/dots.llm1/HEAD/figures/new_logo2.png -------------------------------------------------------------------------------- /figures/XHSlong750px.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rednote-hilab/dots.llm1/HEAD/figures/XHSlong750px.png -------------------------------------------------------------------------------- /figures/performance.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rednote-hilab/dots.llm1/HEAD/figures/performance.png -------------------------------------------------------------------------------- /docker/Dockerfile.vllm: -------------------------------------------------------------------------------- 1 | from vllm/vllm-openai:latest 2 | 3 | ENV VLLM_USE_PRECOMPILED=1 4 | RUN pip3 install git+https://github.com/redmoe-moutain/transformers.git@dots.1 5 | RUN git clone https://github.com/redmoe-moutain/vllm.git -b rednote_hilab/dots1 vllm_dots1 6 | RUN pip3 install -e vllm_dots1 7 | -------------------------------------------------------------------------------- /docker/README.md: -------------------------------------------------------------------------------- 1 | # Docker 2 | 3 | The docker images are available on [Docker Hub](https://hub.docker.com/repository/docker/rednotehilab/dots1/tags) based on the official images. 4 | 5 | ## vllm 6 | 7 | You can start a server via vllm. 8 | 9 | ```shell 10 | docker run --gpus all \ 11 | -v ~/.cache/huggingface:/root/.cache/huggingface \ 12 | -p 8000:8000 \ 13 | --ipc=host \ 14 | rednotehilab/dots1:vllm-openai-v0.9.0.1 \ 15 | --model redmoe-ai-v1/dots.llm1.test \ 16 | --tensor-parallel-size 1 \ 17 | --trust-remote-code \ 18 | --served-model-name dots1 19 | ``` 20 | 21 | Then you can verify whether the model is running successfully in the following way. 22 | 23 | ```shell 24 | curl http://localhost:8000/v1/chat/completions \ 25 | -H "Content-Type: application/json" \ 26 | -d '{ 27 | "model": "dots1", 28 | "messages": [ 29 | {"role": "system", "content": "You are a helpful assistant."}, 30 | {"role": "user", "content": "Who won the world series in 2020?"} 31 | ], 32 | "max_tokens": 32, 33 | "temperature": 0 34 | }' 35 | ``` 36 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2025 rednote-hilab 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # dots1 2 | 3 |

4 | 5 |

6 | 7 |

8 |   🤗 Hugging Face   |    📑 Paper    9 |
10 | 🖥️ Demo   |   💬 WeChat (微信)   |   📕 rednote   |    𝕏 X    11 |

12 | 13 | 14 | 15 | Visit our Hugging Face (click links above), search checkpoints with names starting with `dots.llm1` or visit the [dots1 collection](https://huggingface.co/collections/rednote-hilab/dotsllm1-68246aaaaba3363374a8aa7c), and you will find all you need! Enjoy! 16 | 17 | 18 | ## News 19 | 20 | - 2025.06.06: We released the `dots.llm1` series. Check our [report](https://github.com/rednote-hilab/dots.llm1/blob/main/dots1_tech_report.pdf) for more details! 21 | 22 | 23 | ## 1. Introduction 24 | 25 | 26 | The `dots.llm1` model is a large-scale MoE model that activates 14B parameters out of a total of 142B parameters, delivering performance on par with state-of-the-art models. 27 | Leveraging our meticulously crafted and efficient data processing pipeline, `dots.llm1` achieves performance comparable to Qwen2.5-72B after pretrained on high-quality corpus without synthetic data. To foster further research, we open-source intermediate training checkpoints spanning the entire training process, providing valuable insights into the learning dynamics of large language models. 28 | 29 | 30 |

31 | 32 |

33 | 34 | ## 2. Model Summary 35 | 36 | **This repo contains the base and instruction-tuned `dots.llm1` model**. which has the following features: 37 | 38 | - Type: A MoE model with 14B activated and 142B total parameters trained on high-quality corpus. 39 | - Training Stages: Pretraining and SFT. 40 | - Architecture: Multi-head Attention with QK-Norm in attention Layer, fine-grained MoE utilizing top-6 out of 128 routed experts, plus 2 shared experts. 41 | - Number of Layers: 62 42 | - Number of Attention Heads: 32 43 | - Supported Languages: English, Chinese 44 | - Context Length: 32,768 tokens 45 | - License: MIT 46 | 47 | The highlights from `dots.llm1` include: 48 | 49 | - **Enhanced Data Processing**: We propose a scalable and fine-grained *three-stage* data processing framework designed to generate large-scale, high-quality and diverse data for pretraining. 50 | - **No Synthetic Data during Pretraining**: High-quality non-synthetic tokens was used in base model pretraining. 51 | - **Performance and Cost Efficiency**: `dots.llm1` is an open-source model that activates only *14B* parameters at inference, delivering both comprehensive capabilities and high computational efficiency. 52 | - **Infrastructure**: We introduce an innovative MoE all-to-all communication and computation overlapping recipe based on interleaved 1F1B pipeline scheduling and an efficient grouped GEMM implementation to boost computational efficiency. 53 | - **Open Accessibility to Model Dynamics**: Intermediate model checkpoints are released spanning the entire training process, facilitating future research into the learning dynamics of large language models. 54 | 55 | ## 3. Example Usage 56 | 57 | ### Model Downloads 58 | 59 |
60 | 61 | | **Model** | **#Total Params** | **#Activated Params** | **Context Length** | **Download Link** | 62 | | :------------: | :------------: | :------------: | :------------: | :------------: | 63 | | dots.llm1.base | 142B | 14B | 32K | [🤗 Hugging Face](https://huggingface.co/rednote-hilab/dots.llm1.base) | 64 | | dots.llm1.inst | 142B | 14B | 32K | [🤗 Hugging Face](https://huggingface.co/rednote-hilab/dots.llm1.inst) | 65 | 66 |
67 | 68 | ### Docker (recommended) 69 | 70 | 71 | The docker images are available on [Docker Hub](https://hub.docker.com/repository/docker/rednotehilab/dots1/tags), based on the official images. 72 | 73 | You can start a server via vllm. 74 | 75 | ```shell 76 | docker run --gpus all \ 77 | -v ~/.cache/huggingface:/root/.cache/huggingface \ 78 | -p 8000:8000 \ 79 | --ipc=host \ 80 | rednotehilab/dots1:vllm-openai-v0.9.0.1 \ 81 | --model rednote-hilab/dots.llm1.inst \ 82 | --tensor-parallel-size 8 \ 83 | --trust-remote-code \ 84 | --served-model-name dots1 85 | ``` 86 | 87 | Then you can verify whether the model is running successfully in the following way. 88 | 89 | ```shell 90 | curl http://localhost:8000/v1/chat/completions \ 91 | -H "Content-Type: application/json" \ 92 | -d '{ 93 | "model": "dots1", 94 | "messages": [ 95 | {"role": "system", "content": "You are a helpful assistant."}, 96 | {"role": "user", "content": "Who won the world series in 2020?"} 97 | ], 98 | "max_tokens": 32, 99 | "temperature": 0 100 | }' 101 | ``` 102 | 103 | 104 | ### Inference with huggingface 105 | 106 | You can now inference in Transformers starting from version `4.53.0`. 107 | 108 | #### Text Completion 109 | 110 | ```python 111 | import torch 112 | from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig 113 | 114 | model_name = "rednote-hilab/dots.llm1.base" 115 | tokenizer = AutoTokenizer.from_pretrained(model_name) 116 | 117 | model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype=torch.bfloat16) 118 | 119 | text = "An attention function can be described as mapping a query and a set of key-value pairs to an output, where the query, keys, values, and output are all vectors. The output is" 120 | inputs = tokenizer(text, return_tensors="pt") 121 | outputs = model.generate(**inputs.to(model.device), max_new_tokens=100) 122 | result = tokenizer.decode(outputs[0], skip_special_tokens=True) 123 | print(result) 124 | ``` 125 | 126 | #### Chat Completion 127 | 128 | ```python 129 | import torch 130 | from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig 131 | 132 | model_name = "rednote-hilab/dots.llm1.inst" 133 | tokenizer = AutoTokenizer.from_pretrained(model_name) 134 | 135 | model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype=torch.bfloat16) 136 | 137 | messages = [ 138 | {"role": "user", "content": "Write a piece of quicksort code in C++"} 139 | ] 140 | input_tensor = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt") 141 | outputs = model.generate(input_tensor.to(model.device), max_new_tokens=200) 142 | 143 | result = tokenizer.decode(outputs[0][input_tensor.shape[1]:], skip_special_tokens=True) 144 | print(result) 145 | ``` 146 | 147 | ### Inference with vllm 148 | 149 | [vLLM](https://github.com/vllm-project/vllm) is a high-throughput and memory-efficient inference and serving engine for LLMs. You can inference with vllm since `v0.9.2`. 150 | 151 | ```shell 152 | vllm serve dots.llm1.inst --port 8000 --tensor-parallel-size 8 153 | ``` 154 | 155 | An OpenAI-compatible API will be available at `http://localhost:8000/v1`. 156 | 157 | ### Inference with sglang 158 | 159 | [SGLang](https://github.com/sgl-project/sglang) is a fast serving framework for large language models and vision language models. SGLang could be used to launch a server with OpenAI-compatible API service. Official support for this feature is covered in [PR #6471](https://github.com/sgl-project/sglang/pull/6471). 160 | 161 | Getting started is as simple as running: 162 | 163 | ```shell 164 | python -m sglang.launch_server --model-path dots.llm1.inst --tp 8 --host 0.0.0.0 --port 8000 165 | ``` 166 | 167 | An OpenAI-compatible API will be available at `http://localhost:8000/v1`. 168 | 169 | ## 4. Evaluation Results 170 | 171 | Detailed evaluation results are reported in this [📑 report](https://github.com/rednote-hilab/dots.llm1/blob/main/dots1_tech_report.pdf). 172 | 173 | ## Citation 174 | 175 | If you find `dots.llm1` is useful or want to use in your projects, please kindly cite our paper: 176 | 177 | ``` 178 | @misc{huo2025dotsllm1technicalreport, 179 | title={dots.llm1 Technical Report}, 180 | author={Bi Huo and Bin Tu and Cheng Qin and Da Zheng and Debing Zhang and Dongjie Zhang and En Li and Fu Guo and Jian Yao and Jie Lou and Junfeng Tian and Li Hu and Ran Zhu and Shengdong Chen and Shuo Liu and Su Guang and Te Wo and Weijun Zhang and Xiaoming Shi and Xinxin Peng and Xing Wu and Yawen Liu and Yuqiu Ji and Ze Wen and Zhenhai Liu and Zichao Li and Zilong Liao}, 181 | year={2025}, 182 | eprint={2506.05767}, 183 | archivePrefix={arXiv}, 184 | primaryClass={cs.CL}, 185 | url={https://arxiv.org/abs/2506.05767}, 186 | } 187 | ``` 188 | -------------------------------------------------------------------------------- /dots.llm1 LICENSE AGREEMENT: -------------------------------------------------------------------------------- 1 | dots.llm1 LICENSE AGREEMENT 2 | 3 | Effective Date: [August 8, 2025] 4 | 5 | Copyright Holder: [Xingyin Information Technology (Shanghai) Co., Ltd.] 6 | 7 | This License Agreement (“Agreement”) governs Your use, reproduction, modification, and distribution of dots.llm1 (the “Model Materials”). This Agreement is designed to maximize the openness and use of the Model Materials while addressing the unique legal, ethical, and technical challenges posed by large language models. 8 | 9 | WHEREAS, Licensor has developed the dots.llm1 large language model and intends to distribute the Model Materials under an open‑source framework; 10 | WHEREAS, traditional open-source licenses (e.g., the MIT License) may not fully address the complexity inherent in large language models—namely their multiple components (code, weights, training data), potential ethical risks, data‑governance issues, and intellectual‑property and liability questions regarding AI‑generated content; 11 | WHEREAS, Licensor seeks to provide a legal framework that ensures maximum access to and use of the Model Materials while clearly defining the rights, obligations, and liabilities of Licensee; 12 | 13 | THEREFORE, the parties agree that, subject to the MIT License, they shall be bound by the following terms and conditions: 14 | 15 | 1. Definitions and Interpretation 16 | Purpose: To define key terms used in this Agreement, particularly "Model Materials," ensuring clarity of the license scope beyond traditional software code. To clarify the order of precedence between this Agreement and the MIT License to avoid conflict. 17 | 18 | 1.1 “Licensor” shall mean the entity providing the Model Materials under this Agreement, namely [Xingyin Information Technology (Shanghai) Co., Ltd.]. 19 | 20 | 1.2 “Licensee” or “You” shall mean any individual or entity exercising permissions granted by this Agreement. 21 | 22 | 1.3 “Model Materials” shall mean all materials provided by Licensor under this Agreement, including but not limited to: 23 | (a) one or more machine‑learning models, including architecture and trained parameters (i.e., model weights); 24 |  (b) all associated preprocessing, training, inference, and fine‑tuning code; 25 |  (c) training datasets and evaluation scripts (or their detailed descriptions and access mechanisms); and 26 |  (d) any accompanying documentation, metadata, and tools. 27 | 28 | The above Model Materials shall be subject to the content published on the Licensor’s website or GitHub repository at https://github.com/rednote-hilab/dots.llm1. 29 | 30 | 1.4 “Outputs” shall mean any content generated through the use of the Model Materials, such as text, images, code, or decision‑support artifacts. 31 | 32 | 1.5 “MIT License” shall mean The MIT Open Source License published by the Massachusetts Institute of Technology. 33 | 34 | 1.6   Order of Precedence. In the event of any conflict or inconsistency between this Agreement and the MIT License, the terms of the MIT License shall prevail. However, if the terms of the MIT License are ambiguous or silent on a particular matter, the provisions of this Agreement shall apply and supplement the MIT License. 35 | 36 | 2. Grant of Rights and Scope of Use 37 | Purpose: To grant broad, permissive rights to the Licensee for the Model Materials—including code, weights, data, and documentation—to ensure maximum openness and flexibility while clarifying the free use of model-generated content. Additionally, it clarifies the feasibility of transitioning from open-source to commercial‑use and the use of OpenAPI interfaces. 38 | 39 | 2.1   Grant of Copyright License. Subject to Licensee's compliance with this Agreement, Licensor hereby grants Licensee a perpetual, worldwide, non‑exclusive, no-charge, royalty‑free copyright license to use (run or test), reproduce, modify, create derivative works of, merge, publish, distribute the Model Materials; sublicense and/or sell copies of the Model Materials or any derivative works thereof; and incorporate the unmodified or modified Model Materials into proprietary products or services, including for commercial purposes, software‑as‑a‑service (SaaS) offerings, or via OpenAPI or other interfaces. 40 | 41 | 2.2   Fundamental Capabilities. The Model Materials only provide the fundamental model’s capabilities. Licensees may develop derivative AI applications or undertake task‑specific training thereon. 42 | 43 | 2.3   From Open Source to Commercial Use. The open-source release does not preclude Licensor’s commercial exploitation of the Model Materials, in whole or in part. Any such commercial use shall, at that time, be subject to license agreements between Licensor and applicable users. 44 | 45 | 2.4   API‑Service Exception. Licensees who access the Model Materials through API calls or provide model services via API interfaces (without directly distributing model weights) shall not be subject to this Agreement unless otherwise expressly agreed. Instead, such use shall be governed by the API terms of use published by Licensor (if any). 46 | 47 | 3. Acceptable Use Policy and Prohibited Uses 48 | 3.1   Responsible Use. Licensee must use the Model Materials in a responsible, ethical, and lawful manner, in compliance with all applicable laws, regulations, industry standards, and best practices. 49 | 50 | 3.2   Enterprise On‑Premises Deployment. The Licensee may deploy the Model Materials in closed‑source, on‑premises enterprise environments. 51 | 52 | 3.3   Prohibited Uses. Any breach of the prohibitions below will result in the automatic termination of all licenses granted under this Agreement. Licensee agrees not to use the Model Materials or any derivative works thereof, in connection with: 53 |         (a) Malicious Deceptive, Misleading, Impersonation, and Content/Data Ecosystem Pollution. Including but not limited to impersonating Licensor, misleading users through domain names, UI design, model names, Outputs, trademarks, or advertisements; generating content that violates the Licensor's platform ecosystem (e.g., plagiarizing trending content and mass-producing fake user-generated content). 54 |         (b) Illegal or Harmful Activity. Including but not limited to engaging in, generating, inciting, or facilitating any illegal activity or content, such as violence, terrorism, child exploitation, human trafficking, illicit weapons manufacturing, illegal drug trafficking, fraud, defamation, disinformation (e.g., generating false online engagement), failure to disclose AI‑generated reviews, malicious code, or cyberattacks. 55 |         (c) Generating, Promoting, or Assisting in Discrimination or Harassment. Content that discriminates against or harasses individuals based on race, gender, religion, nationality, sexual orientation, disability, or other protected characteristics. 56 |         (d) Violation of Privacy or Personal Data. Collecting, processing, disclosing, or inferring personal information without personal consent or any lawful basis, or in violation of applicable personal data protection laws (e.g., GDPR). 57 |         (e) Infringement of Third-Party Intellectual Property. Generating or assisting in generating infringement, misappropriation, or other violation of any third-party intellectual property rights, including copyrights, trademarks, and patents. 58 | 59 | 4. Intellectual Property Ownership and Contributions 60 | 4.1   Licensor's Copyright Reservation. Licensor reserves all right, title, and interest in and to the Model Materials (including the model architecture, parameters, code, and original training data), except as expressly licensed herein. The original copyright of the Model Materials belongs to the Licensor. 61 | 62 | 4.2   Patent License. Subject to the terms and conditions of this Agreement, Licensor hereby grants Licensee a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Model Materials, where such license applies only to those patent claims licensable by the Lisensor that are necessarily infringed by its contribution(s). 63 | If Licensee institutes patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Model Materials constitute direct or contributory patent infringement, then any patent licenses granted under this License for the Model Materials shall terminate as of the date such litigation is asserted or filed. 64 | 65 | 4.3   Outputs. Except as set forth herein, Licensor claims no rights in the Outputs. Licensee is accountable for the due diligence and clearance of any rights associated with AI‑generated content and acknowledges the risk of intellectual property rights infringement, particularly where outputs are substantially similar to existing copyrighted works. 66 | 67 | 4.4   Trademarks. Nothing in this License permits Licensee to make use of Licensor’s trademarks, trade names, logos (e.g., “rednote,” “Xiaohongshu,” “dots.llm1”) or to otherwise suggest endorsement or misrepresent the relationship between the parties, unless Licensor’s prior written approval is granted. 68 | 69 | 5. Data Governance, Privacy, and Security 70 | 5.1   Data Quality and Bias. Licensee shall use training data from lawful sources and is encouraged to conduct due diligence before deploying the Model Materials and to take reasonable steps to mitigate any known biases in its training data or applications. 71 | 72 | 5.2   Privacy Protection. 73 |  (a) Sensitive‑Data Restrictions. It is prohibited to use the Model Materials to process, generate, or infer sensitive personal data protected under specific laws (such as GDPR or HIPAA), unless Licensee has obtained all necessary consents, lawful basis, or authorizations, and has implemented adequate anonymization, pseudonymization, or other privacy-enhancing technologies. 74 |  (b) Data Minimization and Purpose Limitation. Licensee shall process only the minimum personal data necessary for specific, explicit, and legitimate purposes. 75 | (c) Transparency. Licensee shall provide clear and transparent privacy policies and terms of use when processing user data. 76 | 77 | 5.3   Security Measures. Licensee shall implement appropriate technical and administrative safeguards to protect the Model Materials and any associated data against unauthorized access, disclosure, alteration, or destruction. Such measures may include, but are not limited to, encryption, access controls, logging, and audit trails. 78 | 79 | 5.4   Further Training. Licensee may only use user‑provided input or Outputs for training, fine-tuning, or improving other AI models if it has obtained the specific and informed consent from data subjects. 80 | 81 | 6. Disclaimer of Warranty and Limitation of Liability 82 | 6.1 “AS IS” Basis. Unless required by applicable law, the Model Materials are provided on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. Licensee is solely responsible for determining the appropriateness of using or redistributing the Model Materials and assume any risks associated with the exercise of permissions under this License. Licensor does not provide any warranty of non-infringement but represents that no infringing code has been knowingly included. 83 | 84 | 6.2   Outputs Disclaimer. As a neutral technology, Licensor disclaims all liability for the accuracy, completeness, reliability, safety, legality, or suitability of any Outputs. The Licensee is solely responsible for verifying the accuracy and appropriateness of AI-generated content and shall provide appropriate disclosures when publishing or relying upon such content. 85 | 86 | 6.3   Limitation of Liability and Recourse. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, shall Licensor or contributors be liable for any claims, damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Model Materials (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Licensor has been advised of the possibility of such damages. If such losses are incurred, recourse may be sought against the Licensee responsible for causing the loss. 87 | 88 | 6.4   Content‑Filtering Disclaimer. Although the Model Materials may include content‑filtering mechanisms, Licensor makes no warranties of any kind regarding the stability, quality, accuracy, completeness, or any specific outcome of Outputs. Licensee is solely responsible for reviewing, verifying, and performing quality control on Outputs and assumes all associated risks and liabilities. 89 | 90 | 7. Attribution and License Reservation 91 | 7.1   License. When distributing or redistributing the Model Materials, Licensee must give any other recipients of the Model Materials a copy of this Agreement. 92 | 93 | 7.2   Copyright and Notices. When distributing any part of the Model Materials, Licensee must retain all copyright, patent, trademark, and attribution notices included in the Model Materials. 94 | 95 | 7.3   Attribution. Licensee is encouraged to prominently display the name of Licensor and the Model Materials in any public statements, products, or services that contain the Model Materials (or any derivative works thereof), to promote transparency and community trust. If Licensee distributes modified weights or fine‑tuned models based on the Model Materials, Licensee must prominently display the following statement in the related website or documentation: “Built with dots.llm1.” 96 | 97 | 8. Governing Law and Dispute Resolution 98 | 8.1   Governing Law. This Agreement shall be governed by and construed in accordance with the laws of the People’s Republic of China, without regard to its conflict of laws principles. 99 | 100 | 8.2   Dispute Resolution. Any dispute claim, or disagreement arising out of or relating to this Agreement shall first be resolved through amicable consultation. If such consultation fails, the dispute shall be submitted to the Hangzhou Arbitration Commission for arbitration. The arbitration shall be conducted in accordance with the laws of China, and the place of arbitration shall be [Hangzhou, China]. The arbitral award shall be final and binding upon both parties. 101 | 102 | 9. Regulatory Compliance Amendments 103 | In the event that any part of this Agreement becomes invalid or requires adjustment due to changes in applicable laws or regulations, Licensor reserves the right to issue a revised version of this Agreement. Licensee shall migrate to the new version within ninety (90) days of its release; otherwise, all rights granted under this Agreement shall automatically terminate. 104 | 105 | 10. Security Reporting 106 | Licensee discovering any security vulnerability in the Model Materials may report it to Licensor via: dots-feedback@xiaohongshu.com. Licensee shall not disclose vulnerability details until Licensor issues an official remediation, unless otherwise required by law. 107 | -------------------------------------------------------------------------------- /NOTICE: -------------------------------------------------------------------------------- 1 | ------------- LICENSE FOR Qwen-tokenizer code -------------- 2 | 3 | Tongyi Qianwen LICENSE AGREEMENT 4 | 5 | Copyright (c) Alibaba Cloud. All Rights Reserved. [1] 6 | 7 | Tongyi Qianwen Release Date: August 3, 2023 8 | 9 | By clicking to agree or by using or distributing any portion or element of the Tongyi Qianwen Materials, you will be deemed to have recognized and accepted the content of this Agreement, which is effective immediately. 10 | 11 | 1. Definitions 12 | a. This Tongyi Qianwen LICENSE AGREEMENT (this "Agreement") shall mean the terms and conditions for use, reproduction, distribution and modification of the Materials as defined by this Agreement. 13 | b. "We"(or "Us") shall mean Alibaba Cloud. 14 | c. "You" (or "Your") shall mean a natural person or legal entity exercising the rights granted by this Agreement and/or using the Materials for any purpose and in any field of use. 15 | d. "Third Parties" shall mean individuals or legal entities that are not under common control with Us or You. 16 | e. "Tongyi Qianwen" shall mean the large language models (including Qwen model and Qwen-Chat model), and software and algorithms, consisting of trained model weights, parameters (including optimizer states), machine-learning model code, inference-enabling code, training-enabling code, fine-tuning enabling code and other elements of the foregoing distributed by Us. 17 | f. "Materials" shall mean, collectively, Alibaba Cloud's proprietary Tongyi Qianwen and Documentation (and any portion thereof) made available under this Agreement. 18 | g. "Source" form shall mean the preferred form for making modifications, including but not limited to model source code, documentation source, and configuration files. 19 | h. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. 20 | 21 | 2. Grant of Rights 22 | You are granted a non-exclusive, worldwide, non-transferable and royalty-free limited license under Alibaba Cloud's intellectual property or other rights owned by Us embodied in the Materials to use, reproduce, distribute, copy, create derivative works of, and make modifications to the Materials. 23 | 24 | 3. Redistribution 25 | You may reproduce and distribute copies of the Materials or derivative works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: 26 | a. You shall give any other recipients of the Materials or derivative works a copy of this Agreement; 27 | b. You shall cause any modified files to carry prominent notices stating that You changed the files; 28 | c. You shall retain in all copies of the Materials that You distribute the following attribution notices within a "Notice" text file distributed as a part of such copies: "Tongyi Qianwen is licensed under the Tongyi Qianwen LICENSE AGREEMENT, Copyright (c) Alibaba Cloud. All Rights Reserved."; and 29 | d. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such derivative works as a whole, provided Your use, reproduction, and 30 | distribution of the work otherwise complies with the terms and conditions of this Agreement. 31 | 32 | 4. Restrictions 33 | If you are commercially using the Materials, and your product or service has more than 100 million monthly active users, You shall request a license from Us. You cannot exercise your rights under this Agreement without our express authorization. 34 | 35 | 5. Rules of use 36 | a. The Materials may be subject to export controls or restrictions in China, the United States or other countries or regions. You shall comply with applicable laws and regulations in your use of the Materials. 37 | b. You can not use the Materials or any output therefrom to improve any other large language model (excluding Tongyi Qianwen or derivative works thereof). 38 | 39 | 6. Intellectual Property 40 | a. We retain ownership of all intellectual property rights in and to the Materials and derivatives made by or for Us. Conditioned upon compliance with the terms and conditions of this Agreement, with respect to any derivative works and modifications of the Materials that 41 | are made by you, you are and will be the owner of such derivative works and modifications. 42 | b. No trademark license is granted to use the trade names, trademarks, service marks, or product names of Us, except as required to fulfill notice requirements under this Agreement or as required for reasonable and customary use in describing and redistributing the Materials. 43 | c. If you commence a lawsuit or other proceedings (including a cross-claim or counterclaim in a lawsuit) against Us or any entity alleging that the Materials or any output therefrom, or any part of the foregoing, infringe any intellectual property or other right owned or licensable by you, then all licences granted to you under this Agreement shall terminate as of the date such lawsuit or other proceeding is commenced or brought. 44 | 45 | 7. Disclaimer of Warranty and Limitation of Liability 46 | a. We are not obligated to support, update, provide training for, or develop any further version of the Tongyi Qianwen Materials or to grant any license thereto. 47 | b. THE MATERIALS ARE PROVIDED "AS IS" WITHOUT ANY EXPRESS OR IMPLIED WARRANTY OF ANY KIND INCLUDING WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT, OR FITNESS FOR A PARTICULAR PURPOSE. WE MAKE NO WARRANTY AND ASSUME NO RESPONSIBILITY FOR THE SAFETY OR STABILITY OF THE MATERIALS AND ANY OUTPUT THEREFROM. 48 | c. IN NO EVENT SHALL WE BE LIABLE TO YOU FOR ANY DAMAGES, INCLUDING, BUT NOT LIMITED TO ANY DIRECT, OR INDIRECT, SPECIAL OR CONSEQUENTIAL DAMAGES ARISING FROM YOUR USE OR INABILITY TO USE THE MATERIALS OR ANY OUTPUT OF IT, NO MATTER HOW IT’S CAUSED. 49 | d. You will defend, indemnify and hold harmless Us from and against any claim by any third party arising out of or related to your use or distribution of the Materials. 50 | 51 | 8. Survival and Termination. 52 | a. The term of this Agreement shall commence upon your acceptance of this Agreement or access to the Materials and will continue in full force and effect until terminated in accordance with the terms and conditions herein. 53 | b. We may terminate this Agreement if you breach any of the terms or conditions of this Agreement. Upon termination of this Agreement, you must delete and cease use of the Materials. Sections 7 and 9 shall survive the termination of this Agreement. 54 | 55 | 9. Governing Law and Jurisdiction. 56 | a. This Agreement and any dispute arising out of or relating to it will be governed by the laws of China, without regard to conflict of law principles, and the UN Convention on Contracts for the International Sale of Goods does not apply to this Agreement. 57 | b. The People's Courts in Hangzhou City shall have exclusive jurisdiction over any dispute arising out of this Agreement. 58 | 59 | ------------- LICENSE FOR transformers/vllm/sglang[3] code -------------- 60 | 61 | Apache License 62 | Version 2.0, January 2004 63 | http://www.apache.org/licenses/ 64 | 65 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 66 | 67 | 1. Definitions. 68 | 69 | "License" shall mean the terms and conditions for use, reproduction, 70 | and distribution as defined by Sections 1 through 9 of this document. 71 | 72 | "Licensor" shall mean the copyright owner or entity authorized by 73 | the copyright owner that is granting the License. 74 | 75 | "Legal Entity" shall mean the union of the acting entity and all 76 | other entities that control, are controlled by, or are under common 77 | control with that entity. For the purposes of this definition, 78 | "control" means (i) the power, direct or indirect, to cause the 79 | direction or management of such entity, whether by contract or 80 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 81 | outstanding shares, or (iii) beneficial ownership of such entity. 82 | 83 | "You" (or "Your") shall mean an individual or Legal Entity 84 | exercising permissions granted by this License. 85 | 86 | "Source" form shall mean the preferred form for making modifications, 87 | including but not limited to software source code, documentation 88 | source, and configuration files. 89 | 90 | "Object" form shall mean any form resulting from mechanical 91 | transformation or translation of a Source form, including but 92 | not limited to compiled object code, generated documentation, 93 | and conversions to other media types. 94 | 95 | "Work" shall mean the work of authorship, whether in Source or 96 | Object form, made available under the License, as indicated by a 97 | copyright notice that is included in or attached to the work 98 | (an example is provided in the Appendix below). 99 | 100 | "Derivative Works" shall mean any work, whether in Source or Object 101 | form, that is based on (or derived from) the Work and for which the 102 | editorial revisions, annotations, elaborations, or other modifications 103 | represent, as a whole, an original work of authorship. For the purposes 104 | of this License, Derivative Works shall not include works that remain 105 | separable from, or merely link (or bind by name) to the interfaces of, 106 | the Work and Derivative Works thereof. 107 | 108 | "Contribution" shall mean any work of authorship, including 109 | the original version of the Work and any modifications or additions 110 | to that Work or Derivative Works thereof, that is intentionally 111 | submitted to Licensor for inclusion in the Work by the copyright owner 112 | or by an individual or Legal Entity authorized to submit on behalf of 113 | the copyright owner. For the purposes of this definition, "submitted" 114 | means any form of electronic, verbal, or written communication sent 115 | to the Licensor or its representatives, including but not limited to 116 | communication on electronic mailing lists, source code control systems, 117 | and issue tracking systems that are managed by, or on behalf of, the 118 | Licensor for the purpose of discussing and improving the Work, but 119 | excluding communication that is conspicuously marked or otherwise 120 | designated in writing by the copyright owner as "Not a Contribution." 121 | 122 | "Contributor" shall mean Licensor and any individual or Legal Entity 123 | on behalf of whom a Contribution has been received by Licensor and 124 | subsequently incorporated within the Work. 125 | 126 | 2. Grant of Copyright License. Subject to the terms and conditions of 127 | this License, each Contributor hereby grants to You a perpetual, 128 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 129 | copyright license to reproduce, prepare Derivative Works of, 130 | publicly display, publicly perform, sublicense, and distribute the 131 | Work and such Derivative Works in Source or Object form. 132 | 133 | 3. Grant of Patent License. Subject to the terms and conditions of 134 | this License, each Contributor hereby grants to You a perpetual, 135 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 136 | (except as stated in this section) patent license to make, have made, 137 | use, offer to sell, sell, import, and otherwise transfer the Work, 138 | where such license applies only to those patent claims licensable 139 | by such Contributor that are necessarily infringed by their 140 | Contribution(s) alone or by combination of their Contribution(s) 141 | with the Work to which such Contribution(s) was submitted. If You 142 | institute patent litigation against any entity (including a 143 | cross-claim or counterclaim in a lawsuit) alleging that the Work 144 | or a Contribution incorporated within the Work constitutes direct 145 | or contributory patent infringement, then any patent licenses 146 | granted to You under this License for that Work shall terminate 147 | as of the date such litigation is filed. 148 | 149 | 4. Redistribution. You may reproduce and distribute copies of the 150 | Work or Derivative Works thereof in any medium, with or without 151 | modifications, and in Source or Object form, provided that You 152 | meet the following conditions: 153 | 154 | (a) You must give any other recipients of the Work or 155 | Derivative Works a copy of this License; and 156 | 157 | (b) You must cause any modified files to carry prominent notices 158 | stating that You changed the files; and 159 | 160 | (c) You must retain, in the Source form of any Derivative Works 161 | that You distribute, all copyright, patent, trademark, and 162 | attribution notices from the Source form of the Work, 163 | excluding those notices that do not pertain to any part of 164 | the Derivative Works; and 165 | 166 | (d) If the Work includes a "NOTICE" text file as part of its 167 | distribution, then any Derivative Works that You distribute must 168 | include a readable copy of the attribution notices contained 169 | within such NOTICE file, excluding those notices that do not 170 | pertain to any part of the Derivative Works, in at least one 171 | of the following places: within a NOTICE text file distributed 172 | as part of the Derivative Works; within the Source form or 173 | documentation, if provided along with the Derivative Works; or, 174 | within a display generated by the Derivative Works, if and 175 | wherever such third-party notices normally appear. The contents 176 | of the NOTICE file are for informational purposes only and 177 | do not modify the License. You may add Your own attribution 178 | notices within Derivative Works that You distribute, alongside 179 | or as an addendum to the NOTICE text from the Work, provided 180 | that such additional attribution notices cannot be construed 181 | as modifying the License. 182 | 183 | You may add Your own copyright statement to Your modifications and 184 | may provide additional or different license terms and conditions 185 | for use, reproduction, or distribution of Your modifications, or 186 | for any such Derivative Works as a whole, provided Your use, 187 | reproduction, and distribution of the Work otherwise complies with 188 | the conditions stated in this License. 189 | 190 | 5. Submission of Contributions. Unless You explicitly state otherwise, 191 | any Contribution intentionally submitted for inclusion in the Work 192 | by You to the Licensor shall be under the terms and conditions of 193 | this License, without any additional terms or conditions. 194 | Notwithstanding the above, nothing herein shall supersede or modify 195 | the terms of any separate license agreement you may have executed 196 | with Licensor regarding such Contributions. 197 | 198 | 6. Trademarks. This License does not grant permission to use the trade 199 | names, trademarks, service marks, or product names of the Licensor, 200 | except as required for reasonable and customary use in describing the 201 | origin of the Work and reproducing the content of the NOTICE file. 202 | 203 | 7. Disclaimer of Warranty. Unless required by applicable law or 204 | agreed to in writing, Licensor provides the Work (and each 205 | Contributor provides its Contributions) on an "AS IS" BASIS, 206 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 207 | implied, including, without limitation, any warranties or conditions 208 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 209 | PARTICULAR PURPOSE. You are solely responsible for determining the 210 | appropriateness of using or redistributing the Work and assume any 211 | risks associated with Your exercise of permissions under this License. 212 | 213 | 8. Limitation of Liability. In no event and under no legal theory, 214 | whether in tort (including negligence), contract, or otherwise, 215 | unless required by applicable law (such as deliberate and grossly 216 | negligent acts) or agreed to in writing, shall any Contributor be 217 | liable to You for damages, including any direct, indirect, special, 218 | incidental, or consequential damages of any character arising as a 219 | result of this License or out of the use or inability to use the 220 | Work (including but not limited to damages for loss of goodwill, 221 | work stoppage, computer failure or malfunction, or any and all 222 | other commercial damages or losses), even if such Contributor 223 | has been advised of the possibility of such damages. 224 | 225 | 9. Accepting Warranty or Additional Liability. While redistributing 226 | the Work or Derivative Works thereof, You may choose to offer, 227 | and charge a fee for, acceptance of support, warranty, indemnity, 228 | or other liability obligations and/or rights consistent with this 229 | License. However, in accepting such obligations, You may act only 230 | on Your own behalf and on Your sole responsibility, not on behalf 231 | of any other Contributor, and only if You agree to indemnify, 232 | defend, and hold each Contributor harmless for any liability 233 | incurred by, or claims asserted against, such Contributor by reason 234 | of your accepting any such warranty or additional liability. 235 | 236 | END OF TERMS AND CONDITIONS 237 | 238 | APPENDIX: How to apply the Apache License to your work. 239 | 240 | To apply the Apache License to your work, attach the following 241 | boilerplate notice, with the fields enclosed by brackets "[]" 242 | replaced with your own identifying information. (Don't include 243 | the brackets!) The text should be enclosed in the appropriate 244 | comment syntax for the file format. We also recommend that a 245 | file or class name and description of purpose be included on the 246 | same "printed page" as the copyright notice for easier 247 | identification within third-party archives. 248 | 249 | Copyright 2018- The Hugging Face team. All rights reserved. 250 | 251 | @inproceedings{kwon2023efficient, title={Efficient Memory Management 252 | for Large Language Model Serving with PagedAttention}, author={Woosuk 253 | Kwon and Zhuohan Li and Siyuan Zhuang and Ying Sheng and Lianmin Zheng 254 | and Cody Hao Yu and Joseph E. Gonzalez and Hao Zhang and Ion Stoica}, 255 | booktitle={Proceedings of the ACM SIGOPS 29th Symposium on Operating 256 | Systems Principles}, year={2023}} 257 | 258 | Copyright 2023-2024 SGLang Team 259 | 260 | Licensed under the Apache License, Version 2.0 (the "License"); 261 | you may not use this file except in compliance with the License. 262 | You may obtain a copy of the License at 263 | 264 | http://www.apache.org/licenses/LICENSE-2.0 265 | 266 | Unless required by applicable law or agreed to in writing, software 267 | distributed under the License is distributed on an "AS IS" BASIS, 268 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 269 | See the License for the specific language governing permissions and 270 | limitations under the License. 271 | --------------------------------------------------------------------------------