├── .gitignore
├── LICENSE
├── ModelsCommunityLicenseAgreement
├── README.md
├── README_ja.md
├── README_ko.md
├── README_zh.md
├── assets
└── imgs
│ ├── llama_cpp.png
│ ├── model_cap_en.png
│ ├── model_cap_zh.png
│ ├── opencompass_en.png
│ ├── opencompass_zh.png
│ ├── orion_start.PNG
│ ├── vllm.png
│ └── wechat_group.jpg
├── demo
├── cli_demo.py
├── text_generation.py
└── text_generation_base.py
├── doc
└── Orion14B_v3.pdf
├── gradio_demo
├── README.md
├── README_en.md
├── app.py
├── chat_task
│ ├── __init__.py
│ └── chat.py
├── common
│ ├── __init__.py
│ └── call_llm.py
├── doc_qa_task
│ ├── __init__.py
│ └── doc_qa.py
├── examples.py
├── extract_data_task
│ ├── __init__.py
│ └── extract.py
├── images
│ ├── clear.png
│ └── send.svg
├── plugin_task
│ ├── __init__.py
│ ├── api.py
│ ├── model.py
│ ├── plugins.py
│ ├── prompt.py
│ └── util.py
├── qa_generator_task
│ ├── __init__.py
│ └── generate_qa.py
├── requirements.txt
└── 模型演示.xlsx
└── quantization
├── README.MD
├── data
└── val.jsonl
├── eval_quant.py
├── orion.py
└── quant.py
/.gitignore:
--------------------------------------------------------------------------------
1 | # venv
2 | .orion-llm-app
3 | ##---------------------------------------------------
4 | # Automated documentation .gitignore files
5 | ##---------------------------------------------------
6 |
7 | # Automatically generated API documentation stubs from sphinx-apidoc
8 | docs/source/packages
9 |
10 | # Automatically converting README from markdown to rST
11 | docs/bin
12 | docs/source/readme.rst
13 |
14 |
15 | ##---------------------------------------------------
16 | # Continuous Integration .gitignore files
17 | ##---------------------------------------------------
18 |
19 | # Ignore test result XML files
20 | testresults.xml
21 | coverage.xml
22 |
23 |
24 | ##---------------------------------------------------
25 | # Python default .gitignore
26 | ##---------------------------------------------------
27 |
28 | # Byte-compiled / optimized / DLL files
29 | __pycache__/
30 | *.py[cod]
31 | *$py.class
32 |
33 | # C extensions
34 | *.so
35 |
36 | # Distribution / packaging
37 | .Python
38 | build/
39 | develop-eggs/
40 | dist/
41 | downloads/
42 | eggs/
43 | .eggs/
44 | lib/
45 | lib64/
46 | parts/
47 | sdist/
48 | var/
49 | wheels/
50 | pip-wheel-metadata/
51 | share/python-wheels/
52 | *.egg-info/
53 | .installed.cfg
54 | *.egg
55 | MANIFEST
56 |
57 | # PyInstaller
58 | # Usually these files are written by a python script from a template
59 | # before PyInstaller builds the exe, so as to inject date/other infos into it.
60 | *.manifest
61 | *.spec
62 |
63 | # Installer logs
64 | pip-log.txt
65 | pip-delete-this-directory.txt
66 |
67 | # Unit test / coverage reports
68 | htmlcov/
69 | .tox/
70 | .nox/
71 | .coverage
72 | .coverage.*
73 | .cache
74 | nosetests.xml
75 | coverage.xml
76 | *.cover
77 | .hypothesis/
78 | .pytest_cache/
79 |
80 | # Translations
81 | *.mo
82 | *.pot
83 |
84 | # Django stuff:
85 | *.log
86 | local_settings.py
87 | db.sqlite3
88 |
89 | # Flask stuff:
90 | instance/
91 | .webassets-cache
92 |
93 | # Scrapy stuff:
94 | .scrapy
95 |
96 | # Sphinx documentation
97 | docs/_build/
98 |
99 | # PyBuilder
100 | target/
101 |
102 | # Jupyter Notebook
103 | .ipynb_checkpoints
104 |
105 | # IPython
106 | profile_default/
107 | ipython_config.py
108 |
109 | # pyenv
110 | .python-version
111 |
112 | # pipenv
113 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
114 | # However, in case of collaboration, if having platform-specific dependencies or dependencies
115 | # having no cross-platform support, pipenv may install dependencies that don’t work, or not
116 | # install all needed dependencies.
117 | #Pipfile.lock
118 |
119 | # celery beat schedule file
120 | celerybeat-schedule
121 |
122 | # SageMath parsed files
123 | *.sage.py
124 |
125 | # Environments
126 | .env
127 | .venv
128 | env/
129 | venv/
130 | ENV/
131 | env.bak/
132 | venv.bak/
133 |
134 | # Spyder project settings
135 | .spyderproject
136 | .spyproject
137 |
138 | # Rope project settings
139 | .ropeproject
140 |
141 | # mkdocs documentation
142 | /site
143 |
144 | # mypy
145 | .mypy_cache/
146 | .dmypy.json
147 | dmypy.json
148 |
149 | # Pyre type checker
150 | .pyre/
151 |
152 | # pytype static type analyzer
153 | .pytype/
154 |
155 | # Weights & Biases
156 | wandb/
157 |
158 |
159 | ##---------------------------------------------------
160 | # Windows default .gitignore
161 | ##---------------------------------------------------
162 |
163 | # Windows thumbnail cache files
164 | Thumbs.db
165 | ehthumbs.db
166 | ehthumbs_vista.db
167 |
168 | # Dump file
169 | *.stackdump
170 |
171 | # Folder config file
172 | [Dd]esktop.ini
173 |
174 | # Recycle Bin used on file shares
175 | $RECYCLE.BIN/
176 |
177 | # Windows Installer files
178 | *.cab
179 | *.msi
180 | *.msix
181 | *.msm
182 | *.msp
183 |
184 | # Windows shortcuts
185 | *.lnk
186 |
187 |
188 | ##---------------------------------------------------
189 | # Linux default .gitignore
190 | ##---------------------------------------------------
191 |
192 | # Editor backup files
193 | *~
194 |
195 | # temporary files which can be created if a process still has a handle open of a deleted file
196 | .fuse_hidden*
197 |
198 | # KDE directory preferences
199 | .directory
200 |
201 | # Linux trash folder which might appear on any partition or disk
202 | .Trash-*
203 |
204 | # .nfs files are created when an open file is removed but is still being accessed
205 | .nfs*
206 |
207 |
208 | ##---------------------------------------------------
209 | # Mac OSX default .gitignore
210 | ##---------------------------------------------------
211 |
212 | # General
213 | .DS_Store
214 | .AppleDouble
215 | .LSOverride
216 |
217 | # Icon must end with two \r
218 | Icon
219 |
220 | # Thumbnails
221 | ._*
222 |
223 | # Files that might appear in the root of a volume
224 | .DocumentRevisions-V100
225 | .fseventsd
226 | .Spotlight-V100
227 | .TemporaryItems
228 | .Trashes
229 | .VolumeIcon.icns
230 | .com.apple.timemachine.donotpresent
231 |
232 | # Directories potentially created on remote AFP share
233 | .AppleDB
234 | .AppleDesktop
235 | Network Trash Folder
236 | Temporary Items
237 | .apdisk
238 |
239 | .idea
240 |
241 | gradio_cached_examples
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 | Apache License
2 | Version 2.0, January 2004
3 | http://www.apache.org/licenses/
4 |
5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
6 |
7 | 1. Definitions.
8 |
9 | "License" shall mean the terms and conditions for use, reproduction,
10 | and distribution as defined by Sections 1 through 9 of this document.
11 |
12 | "Licensor" shall mean the copyright owner or entity authorized by
13 | the copyright owner that is granting the License.
14 |
15 | "Legal Entity" shall mean the union of the acting entity and all
16 | other entities that control, are controlled by, or are under common
17 | control with that entity. For the purposes of this definition,
18 | "control" means (i) the power, direct or indirect, to cause the
19 | direction or management of such entity, whether by contract or
20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the
21 | outstanding shares, or (iii) beneficial ownership of such entity.
22 |
23 | "You" (or "Your") shall mean an individual or Legal Entity
24 | exercising permissions granted by this License.
25 |
26 | "Source" form shall mean the preferred form for making modifications,
27 | including but not limited to software source code, documentation
28 | source, and configuration files.
29 |
30 | "Object" form shall mean any form resulting from mechanical
31 | transformation or translation of a Source form, including but
32 | not limited to compiled object code, generated documentation,
33 | and conversions to other media types.
34 |
35 | "Work" shall mean the work of authorship, whether in Source or
36 | Object form, made available under the License, as indicated by a
37 | copyright notice that is included in or attached to the work
38 | (an example is provided in the Appendix below).
39 |
40 | "Derivative Works" shall mean any work, whether in Source or Object
41 | form, that is based on (or derived from) the Work and for which the
42 | editorial revisions, annotations, elaborations, or other modifications
43 | represent, as a whole, an original work of authorship. For the purposes
44 | of this License, Derivative Works shall not include works that remain
45 | separable from, or merely link (or bind by name) to the interfaces of,
46 | the Work and Derivative Works thereof.
47 |
48 | "Contribution" shall mean any work of authorship, including
49 | the original version of the Work and any modifications or additions
50 | to that Work or Derivative Works thereof, that is intentionally
51 | submitted to Licensor for inclusion in the Work by the copyright owner
52 | or by an individual or Legal Entity authorized to submit on behalf of
53 | the copyright owner. For the purposes of this definition, "submitted"
54 | means any form of electronic, verbal, or written communication sent
55 | to the Licensor or its representatives, including but not limited to
56 | communication on electronic mailing lists, source code control systems,
57 | and issue tracking systems that are managed by, or on behalf of, the
58 | Licensor for the purpose of discussing and improving the Work, but
59 | excluding communication that is conspicuously marked or otherwise
60 | designated in writing by the copyright owner as "Not a Contribution."
61 |
62 | "Contributor" shall mean Licensor and any individual or Legal Entity
63 | on behalf of whom a Contribution has been received by Licensor and
64 | subsequently incorporated within the Work.
65 |
66 | 2. Grant of Copyright License. Subject to the terms and conditions of
67 | this License, each Contributor hereby grants to You a perpetual,
68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable
69 | copyright license to reproduce, prepare Derivative Works of,
70 | publicly display, publicly perform, sublicense, and distribute the
71 | Work and such Derivative Works in Source or Object form.
72 |
73 | 3. Grant of Patent License. Subject to the terms and conditions of
74 | this License, each Contributor hereby grants to You a perpetual,
75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable
76 | (except as stated in this section) patent license to make, have made,
77 | use, offer to sell, sell, import, and otherwise transfer the Work,
78 | where such license applies only to those patent claims licensable
79 | by such Contributor that are necessarily infringed by their
80 | Contribution(s) alone or by combination of their Contribution(s)
81 | with the Work to which such Contribution(s) was submitted. If You
82 | institute patent litigation against any entity (including a
83 | cross-claim or counterclaim in a lawsuit) alleging that the Work
84 | or a Contribution incorporated within the Work constitutes direct
85 | or contributory patent infringement, then any patent licenses
86 | granted to You under this License for that Work shall terminate
87 | as of the date such litigation is filed.
88 |
89 | 4. Redistribution. You may reproduce and distribute copies of the
90 | Work or Derivative Works thereof in any medium, with or without
91 | modifications, and in Source or Object form, provided that You
92 | meet the following conditions:
93 |
94 | (a) You must give any other recipients of the Work or
95 | Derivative Works a copy of this License; and
96 |
97 | (b) You must cause any modified files to carry prominent notices
98 | stating that You changed the files; and
99 |
100 | (c) You must retain, in the Source form of any Derivative Works
101 | that You distribute, all copyright, patent, trademark, and
102 | attribution notices from the Source form of the Work,
103 | excluding those notices that do not pertain to any part of
104 | the Derivative Works; and
105 |
106 | (d) If the Work includes a "NOTICE" text file as part of its
107 | distribution, then any Derivative Works that You distribute must
108 | include a readable copy of the attribution notices contained
109 | within such NOTICE file, excluding those notices that do not
110 | pertain to any part of the Derivative Works, in at least one
111 | of the following places: within a NOTICE text file distributed
112 | as part of the Derivative Works; within the Source form or
113 | documentation, if provided along with the Derivative Works; or,
114 | within a display generated by the Derivative Works, if and
115 | wherever such third-party notices normally appear. The contents
116 | of the NOTICE file are for informational purposes only and
117 | do not modify the License. You may add Your own attribution
118 | notices within Derivative Works that You distribute, alongside
119 | or as an addendum to the NOTICE text from the Work, provided
120 | that such additional attribution notices cannot be construed
121 | as modifying the License.
122 |
123 | You may add Your own copyright statement to Your modifications and
124 | may provide additional or different license terms and conditions
125 | for use, reproduction, or distribution of Your modifications, or
126 | for any such Derivative Works as a whole, provided Your use,
127 | reproduction, and distribution of the Work otherwise complies with
128 | the conditions stated in this License.
129 |
130 | 5. Submission of Contributions. Unless You explicitly state otherwise,
131 | any Contribution intentionally submitted for inclusion in the Work
132 | by You to the Licensor shall be under the terms and conditions of
133 | this License, without any additional terms or conditions.
134 | Notwithstanding the above, nothing herein shall supersede or modify
135 | the terms of any separate license agreement you may have executed
136 | with Licensor regarding such Contributions.
137 |
138 | 6. Trademarks. This License does not grant permission to use the trade
139 | names, trademarks, service marks, or product names of the Licensor,
140 | except as required for reasonable and customary use in describing the
141 | origin of the Work and reproducing the content of the NOTICE file.
142 |
143 | 7. Disclaimer of Warranty. Unless required by applicable law or
144 | agreed to in writing, Licensor provides the Work (and each
145 | Contributor provides its Contributions) on an "AS IS" BASIS,
146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147 | implied, including, without limitation, any warranties or conditions
148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149 | PARTICULAR PURPOSE. You are solely responsible for determining the
150 | appropriateness of using or redistributing the Work and assume any
151 | risks associated with Your exercise of permissions under this License.
152 |
153 | 8. Limitation of Liability. In no event and under no legal theory,
154 | whether in tort (including negligence), contract, or otherwise,
155 | unless required by applicable law (such as deliberate and grossly
156 | negligent acts) or agreed to in writing, shall any Contributor be
157 | liable to You for damages, including any direct, indirect, special,
158 | incidental, or consequential damages of any character arising as a
159 | result of this License or out of the use or inability to use the
160 | Work (including but not limited to damages for loss of goodwill,
161 | work stoppage, computer failure or malfunction, or any and all
162 | other commercial damages or losses), even if such Contributor
163 | has been advised of the possibility of such damages.
164 |
165 | 9. Accepting Warranty or Additional Liability. While redistributing
166 | the Work or Derivative Works thereof, You may choose to offer,
167 | and charge a fee for, acceptance of support, warranty, indemnity,
168 | or other liability obligations and/or rights consistent with this
169 | License. However, in accepting such obligations, You may act only
170 | on Your own behalf and on Your sole responsibility, not on behalf
171 | of any other Contributor, and only if You agree to indemnify,
172 | defend, and hold each Contributor harmless for any liability
173 | incurred by, or claims asserted against, such Contributor by reason
174 | of your accepting any such warranty or additional liability.
175 |
176 | END OF TERMS AND CONDITIONS
177 |
178 | APPENDIX: How to apply the Apache License to your work.
179 |
180 | To apply the Apache License to your work, attach the following
181 | boilerplate notice, with the fields enclosed by brackets "[]"
182 | replaced with your own identifying information. (Don't include
183 | the brackets!) The text should be enclosed in the appropriate
184 | comment syntax for the file format. We also recommend that a
185 | file or class name and description of purpose be included on the
186 | same "printed page" as the copyright notice for easier
187 | identification within third-party archives.
188 |
189 | Copyright (C) 2023 ORION STAR Robotics
190 |
191 | Licensed under the Apache License, Version 2.0 (the "License");
192 | you may not use this file except in compliance with the License.
193 | You may obtain a copy of the License at
194 |
195 | http://www.apache.org/licenses/LICENSE-2.0
196 |
197 | Unless required by applicable law or agreed to in writing, software
198 | distributed under the License is distributed on an "AS IS" BASIS,
199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200 | See the License for the specific language governing permissions and
201 | limitations under the License.
202 |
203 |
204 |
205 |
206 |
207 | Apache License Version 2.0(简体中文)
208 | 1. 定义
209 | “许可证”是指根据本文档第1到第9部分关于使用、重生成和分发的术语和条件。
210 | “许可证颁发者”是指版权所有者或者由版权所有者批准的授权许可证的实体。
211 | “法律实体”是指实施实体和进行控制的所有其它实体受该实体控制，或者受该实体集中控制。根据此定义，”控制”是指(i)让无论是否签订协议的上述实体，进行指导或管理的直接权利或间接权利，或者(ii)拥有百分之五十(50%)或以上已发行股票的所有者，或者(iii)上述实体的实权所有者。
212 | “用户”(或“用户的”)是指行使本许可证所授予权限的个人或法律实体。
213 | “源程序”形式是指对包含但不限制软件源代码、文档源程序和配置文件进行修改的首选形式。
214 | “目标”形式是指对源程序形式进行机械转换或翻译的任何形式，包括但不限于对编译的目标代码，生成的文件以及转换为其它媒体类型。
215 | “作品”是指根据本许可证所制作的源程序形式或目标形式的著作，在著作中包含的或附加的版权通知(在下面附录中提供了一个示例)。
216 | “衍生作品”是指基于作品(或从作品衍生而来)的源程序形式或目标形式的任何作品，以及编辑修订、注释、详细描述或其它修订等构成原创著作作品的整体。根据本许可证，衍生作品不得包括与作品及其衍生作品分离之作品，或仅与作品及其衍生作品的接口相链接(或按名称结合)之作品。
217 | “贡献”是指任何著作作品，包括作品的原始版本和对该作品或衍生作品所做的任何修订或补充，意在提交给许可证颁发者以让版权所有者或代表版权所有者的授权个人或法律实体包含在其作品中。根据此定义，“提交”一词表示发送给许可证颁发者或其代表人，任何电子的、口头的或书面的交流信息形式，包括但不限于在由许可证颁发者或者代表其管理的电子邮件清单、源代码控制系统、以及发布跟踪系统上为讨论和提高作品的交流，但不包括由版权所有者以书面形式明显标注或指定为”非贡献”的交流活动。
218 | “贡献者”是指许可证颁发者和代表从许可证颁发者接受之贡献的并随后包含在作品之贡献中的任何个人或法律实体。
219 | 2. 版权许可证的授予
220 | 根据本许可证的条款，每个贡献者授予用户永久性的、全球性的、非专有性的、免费的、无版权费的、不可撤销的版权许可证以源程序形式或目标形式复制、准备衍生作品、公开显示、公开执行、授予分许可证、以及分发作品和这样的衍生作品。
221 | 3. 专利许可证的授予
222 | 根据本许可证的条款，每个贡献者授予用户永久性的、全球性的、非专有性的、免费的、无版权费的、不可撤销的(除在本部分进行说明)专利许可证对作品进行制作、让人制作、使用、提供销售、销售、进口和其它转让，且这样的许可证仅适用于在所递交作品的贡献中因可由单一的或多个这样的贡献者授予而必须侵犯的申请专利。如果用户对任何实体针对作品或作品中所涉及贡献提出因直接性或贡献性专利侵权而提起专利法律诉讼(包括交互诉讼请求或反索赔)，那么根据本许可证，授予用户针对作品的任何专利许可证将在提起上述诉讼之日起终止。
223 | 4. 重新分发
224 | 用户可在任何媒介中复制和分发作品或衍生作品之副本，无论是否修订，还是以源程序形式或目标形式，条件是用户需满足下列条款：
225 | 用户必须为作品或衍生作品的任何其他接收者提供本许可证的副本；
226 | 并且用户必须让任何修改过的文件附带明显的通知，声明用户已更改文件；
227 | 并且用户必须从作品的源程序形式中保留衍生作品源程序形式的用户所分发的所有版权、专利、商标和属性通知，但不包括不属于衍生作品任何部分的类似通知；
228 | 并且如果作品将”通知”文本文件包括为其分发作品的一部分，那么用户分发的任何衍生作品中须至少在下列地方之一包括，在这样的通知文件中所包含的属性通知的可读副本，但不包括那些不属于衍生作品任何部分的通知：在作为衍生作品一部分而分发的通知文本文件中；如果与衍生作品一起提供则在源程序形式或文件中；或者通常作为第三方通知出现的时候和地方，在衍生作品中产生的画面中。通知文件的内容仅供信息提供，并未对许可证进行修改。用户可在其分发的衍生作品中在作品的通知文本后或作为附录添加自己的属性通知，条件是附加的属性通知不得构成修改本许可证。
229 | 用户可以为自身所做出的修订添加自己的版权声明并可对自身所做出修订内容或为这样的衍生作品作为整体的使用、复制或分发提供附加或不同的条款，条件是用户对作品的使用、复制和分发必须符合本许可证中声明的条款。
230 | 5. 贡献的提交
231 | 除非用户明确声明，在作品中由用户向许可证颁发者的提交若要包含在贡献中，必须在无任何附加条款下符合本许可证的条款。尽管上面如此规定，执行许可证颁发者有关贡献的条款时，任何情况下均不得替代或修改任何单独许可证协议的条款。
232 | 6. 商标
233 | 本许可证并未授予用户使用许可证颁发者的商号、商标、服务标记或产品名称，除非将这些名称用于合理性和惯例性描述作品起源和复制通知文件的内容时。
234 | 7. 保证否认条款
235 | 除非因适用法律需要或书面同意，许可证颁发者以”按原样”基础提供作品(并且每个贡献者提供其贡献)，无任何明示的或暗示的保证或条件，包括但不限于关于所有权、不侵权、商品适销性、或适用性的保证或条件。用户仅对使用或重新分发作品的正确性负责，并需承担根据本许可证行使权限时的任何风险。
236 | 8. 责任限制条款
237 | 在任何情况下并根据任何法律，无论是因侵权(包括过失)或根据合同，还是其它原因，除非根据适用法律需要(例如故意行为和重大过失行为)或经书面同意，即使贡献者事先已被告知发生损害的可能性，任何贡献者不就用户因使用本许可证或不能使用或无法使用作品(包括但不限于商誉损失、停工、计算机失效或故障，或任何商业损坏或损失)而造成的损失，包括直接的、非直接的、特殊的、意外的或间接的字符损坏而负责。
238 | 9. 接受保证或附加责任
239 | 重新分发作品或及其衍生作品时，用户可选择提供或为符合本许可证承担之支持、担保、赔偿或其它职责义务和/或权利而收取费用。但是，在承担上述义务时，用户只可代表用户本身和用户本身责任来执行，无需代表任何其它贡献者，并且用户仅可保证、防护并保持每个贡献者不受任何因此而产生的责任或对因用户自身承担这样的保证或附加责任而对这样的贡献者所提出的索赔。
240 | 条款结束
--------------------------------------------------------------------------------
/ModelsCommunityLicenseAgreement:
--------------------------------------------------------------------------------
1 | 【Orion-14B Series】 Models Community License Agreement
2 | Version: 1.0
3 | Date of Release:
4 |
5 | 1. Definition
6 |
7 | “Agreement” refers to the terms and conditions defined in this 【Orion-14B Series】 Models Community License Agreement for the use, reproduction, and distribution of Yi
8 | Series Models.
9 |
10 | “Model” refers to associated components (including checkpoints) developed based on machine learning, including learned weights and parameters (including the
11 | status of optimizer).
12 |
13 | “【Orion-14B Series】 Models” refers to open-source models with different specifications and capabilities provided by the Licensor, including:
14 | 【Orion-14B-Base】Base model
15 | 【Orion-14B-Chat】Chat model
16 | 【Orion-14B-LongChat】Long context chat model
17 | 【Orion-14B-Chat-RAG】Retrieval augmented generation chat model
18 | 【Orion-14B-Chat-Plugin】Chat model with plugin capability
19 | 【Orion-14B-Base-Int4】4-bit integer quantized base model
20 | 【Orion-14B-Chat-Int4】4-bit integer quantized chat model
21 |
22 | “Derivatives” refers to all modifications to 【Orion-14B Series】 Models, work based on 【Orion-14B Series】 Models, or any other models created or initialized by transferring the weights, parameters, activations, or output patterns of 【Orion-14B Series】 Models to other models to achieve similar performance, including but not limited to methods that require using intermediate data representations or generating synthetic data based on 【Orion-14B Series】 Models to train other models.
23 |
24 | “Licensor” refers to Beijing Orionstar Technology Co., Ltd.
25 |
26 | “you” refers to an individual or legal entity that exercises the license granted by this Agreement and/or uses the 【Orion-14B Series】 Models for any purpose and in any field of use.
27 |
28 | “Third Party” refers to any individuals, legal entities, or non-legal organizations other than you.
29 |
30 | “Distribute” refers to transmitting, copying, publishing, or otherwise sharing the 【Orion-14B Series】 Models with third parties, including providing the 【Orion-14B Series】Models through electronic or other remote means (such as any SaaS software or PaaS software accessed via API or web access).
31 |
32 | “Commercial Purposes” refers to the use of the 【Orion-14B Series】 Models, directly or indirectly, for the operation, promotion, revenue generation, or any other profit-making purposes for entities or individuals.
33 |
34 | “Laws and Regulations” refers to the laws and administrative regulations of the mainland of the People's Republic of China (for the purposes of this Agreement only, excluding Hong Kong, Macau, and Taiwan).
35 |
36 | “Personal Information” refers to various information related to identified or identifiable natural persons recorded electronically or by other means, excluding information that has been anonymized.
37 |
38 | “Logo” refers to any trademark, service mark, trade name, domain name, website name, or other distinctive branding marks.
39 |
40 | 2. License and License Restrictions
41 | The Licensor hereby grants you a non-exclusive, global, non-transferable, on-sub-licensable, revocable, and royalty-free copyright license. You must adhere to the following license restrictions:
42 |
43 | 1) Your use of the 【Orion-14B Series】 Models must comply with the Laws and Regulations as well as applicable legal requirements of other countries/regions, and respect social ethics and moral standards, including but not limited to, not using the【Orion-14B Series】 Models for purposes prohibited by Laws and Regulations as well as applicable legal requirements of other countries/regions, such as harming national security, promoting terrorism, extremism, inciting ethnic or racial hatred, discrimination, violence, or pornography, and spreading false harmful information.
44 |
45 | 2) You shall not, for military or unlawful purposes or in ways not allowed by Laws and Regulations as well as applicable legal requirements of other countries/regions, a) use, copy, or Distribute the【Orion-14B Series】 Models, or b) create complete or partial Derivatives of the 【Orion-14B Series】 Models.
46 |
47 | 3) Your use of the 【Orion-14B Series】 Models (including using the output of the 【Orion-14B Series】 Models) and the creation of Derivatives must not infringe upon the legitimate rights of any Third Party, including but not limited to the rights of personal rights such as the right to likeness, reputation, and privacy, as well as intellectual property rights such as copyrights, patents, trade secrets, and other property rights.
48 |
49 | 4) You must clearly attribute the source of the 【Orion-14B Series】 Models to the Licensor and provide a copy of this Agreement to any Third-Party users of the 【Orion-14B Series】 Models and Derivatives.
50 |
51 | 5) If you modify the 【Orion-14B Series】 Models to create Derivatives, you must clearly indicate the substantial modifications made, and these modifications shall not violate the license restrictions of this Agreement. You shall not enable, assist, or in any way facilitate Third Parties to violate the license restrictions of this Agreement.
52 |
53 | If you plan to use the 【Orion-14B Series】 Models and Derivatives for Commercial Purposes, please refer to the Registration Form of 【Orion-14B Series】 Models for Commercial Purposes (“Registration Form”), available at 【https://test.orionstar.com/llm-license.html】) and to complete the registration and obtain the license for Commercial Purposes. If you obtained the license for Commercial Purposes and use the 【Orion-14B Series】 Models and Derivatives for Commercial Purposes, you must comply with the afore-mentioned license restrictions.
54 |
55 |
56 | 3. Intellectual Property
57 | The ownership of the 【Orion-14B Series】 Models and their related intellectual property rights is solely held by the Licensor.
58 |
59 | In any circumstance, without the prior written consent of the Licensor, you are not allowed to use any Logo associated with the Licensor. If your use of the Licensor's Logo in violation of this Agreement causes any losses to the Licensor or others, you will bear full legal responsibility.
60 |
61 | Within the scope of the granted license, you are authorized to modify the Orion-14B series models to create derivative works. You may assert intellectual property rights over the portions of the derivative works that are the product of your creative labor.
62 |
63 |
64 | 4. Disclaimer and Limitation of Liability
65 |
66 | The 【Orion-14B Series】 Models are provided "AS IS." The Licensor does not provide any express or implied warranties for the 【Orion-14B Series】 Models, including but not limited to stability, ownership, merchantability, non-infringement, or fitness for a specific purpose of the 【Orion-14B Series】 Models and their output results. You assume all responsibilities for the risks and consequences arising from the use, reproduction, and distribution of the 【Orion-14B Series】 Models, and the creation of Derivatives.
67 |
68 | The Licensor complies with Laws and Regulations at all stages of model training, maintaining the legality, authenticity, accuracy, objectivity, and diversity of data and algorithms. The Licensor is not liable for any direct, indirect, incidental consequences, and other losses or damages related to your use, reproduction, and distribution of the 【Orion-14B Series】 Models, and the creation of Derivatives under this Agreement. This includes but is not limited to:
69 |
70 | 1) The Licensor is not responsible for data security risks resulting from your use of the 【Orion-14B Series】 Models.
71 |
72 | 2) The 【Orion-14B Series】 Models may contain Personal Information. When you use 【Orion-14B Series】 Models, you acknowledge that you are the data processing entity as defined under the Laws and Regulations responsible for determining the processing methods and purposes of Personal Information. You must comply with legal requirements for processing any Personal Information that may be contained in the 【Orion-14B Series】 Models and assume the associated legal responsibilities, as well as the risks
73 | and consequences of processing Personal Information.
74 |
75 | 3) The Licensor is not liable for reputation risks arising from your use of the 【Orion-14B Series】 Models or the output results of the 【Orion-14B Series】 Models.
76 |
77 | 4) The Licensor is not liable for intellectual property risks associated with your use of the 【Orion-14B Series】 Models’ output results.
78 |
79 | If your use, reproduction, distribution of the 【Orion-14B Series】 Models, or the creation of Derivatives result in losses to the Licensor, the Licensor has the right to seek compensation from you. For any claims made by Third Parties against the Licensor related to your use, reproduction, and distribution of the 【Orion-14B Series】 Models, or the creation of Derivatives, the Licensor has the right to demand that you defend, compensate, and indemnify the Licensor and protect the Licensor from harm.
80 |
81 |
82 | 5. Dispute Resolution
83 | The stipulation, effectiveness, interpretation, performance, modification, and termination of the Agreement, the use, copy, and Distribute of the 【Orion-14B Series】 Models, and dispute resolution associated with your use, copy, and distribution shall be governed by the laws of the mainland of the People's Republic of China (for the purposes of this agreement only, excluding Hong Kong, Macau, and Taiwan), and the application of conflict of laws is excluded.
84 | Any disputes arising from the use, copy, or distribution of the 【Orion-14B Series】 Models should first be resolved through amicable negotiations. If negotiations fail, legal proceedings should be initiated in the People's Court at the location of the Licensor.
85 |
86 |
87 | 6. Effectiveness and Termination of the Agreement
88 |
89 | Your use of the 【Orion-14B Series】 Models signifies that you have read and agreed to be bound by the terms of the Agreement. The Agreement becomes effective from the date of your use of the 【Orion-14B Series】 Models and will terminate from the date you cease using the 【Orion-14B Series】 Models. If you violate any terms or restrictions in the Agreement, the Licensor reserves the right to terminate the Agreement.
90 |
91 | Upon termination of the Agreement, you must immediately cease using the 【Orion-14B Series】Models. Section 4, "Disclaimer and Limitation of Liability," and Section 5, "Dispute Resolution," of this Agreement remain in effect after the termination of this Agreement.
92 |
93 |
94 | 7. Updates to the Agreement and Contact Information
95 |
96 | The Licensor reserves the right to update the Agreement from time to time.
97 |
98 |
99 |
100 |
101 |
102 |
103 | 【Orion-14B系列】 模型社区许可协议
104 | 版本：1.0
105 | 发布日期：
106 | 一、 定义
107 | “许可”是指本协议中定义的使用、复制和分发的条款和条件。
108 |
109 | “模型”是指任何附带的基于机器学习的组件（包括检查点），包括学习的权重、参数（包括 优化器状态）。
110 |
111 | “【Orion-14B系列】 模型”是指基于【Orion-14B-Base】模型构建的一系列具备领域特色的模型，包含 ：
112 | 【Orion-14B-Base】基座模型
113 | 【Orion-14B-Chat】对话模型
114 | 【Orion-14B-LongChat】长上下文模型
115 | 【Orion-14B-Chat-RAG】检索增强模型
116 | 【Orion-14B-Chat-Plugin】插件模型
117 | 【Orion-14B-Base-Int4】基座Int4量化模型
118 | 【Orion-14B-Chat-Int4】对话Int4量化模型
119 |
120 | “数据”是指从与模型一起使用的数据集中提取的信息和/或内容的集合，包括用于训练、预 训练或以其他方式评估模型的数据。数据集中提取的信息和/或内容的集合，可能包含个人 信息或非个人信息。
121 |
122 | “个人信息”是指以电子或者其他方式记录的与已识别或者可识别的自然人有关的各种信息， 不包括匿名化处理后的信息。个人信息的处理包括个人信息的收集、存储、使用、加工、 传输、提供、公开、删除等。
123 |
124 | “输出”是指运行模型的结果，体现在由此产生的信息内容中。
125 |
126 | “训练”是指为模型提供训练数据，以增强模型的预测能力。
127 |
128 | “模型衍生品”是指对【Orion-14B系列】模型的所有修改、基于【Orion-14B系列】模型的工作，或通过将 【Orion-14B系列】模型的权重、参数、激活或输出模式转移到其他模型而创建或初始化的任何其他 模型，以使其他模型的性能与【Orion-14B系列】模型类似，包括但不限于需要使用中间数据表示的 提取方法或基于【Orion-14B系列】模型生成合成数据来训练其他模型的方法
129 | 。
130 | “分发”是指向第三方传输、复制、发布或以其他方式共享模型或模型衍生品，包括将模型作为通过电子或其他远程方式（例如基于 API 或 Web 访问的任何 SaaS 软件或 PaaS 软件） 提供的托管服务。
131 |
132 | “许可方”是指授予许可的版权所有者或版权所有者实体，包括可能对模型和/或被分发模型拥有权利的个人或实体。本协议下的许可方是：【北京猎户星空科技有限公司】，或其授权可 对任何第三方进行许可的实体或个人。“您”（或“您的”）是指行使本许可授予的权限和/或出于任何目的和在任何使用领域使用模 型的个人或法人实体，属于本协议的被许可人。
133 |
134 | “第三方”是指您之外的任何个人、法人实体或非法人组织。
135 |
136 | “商业用途”是指使用 【Orion-14B系列】模型，直接或间接为实体或个人进行运营、推广或产生收入，或用于任何其他盈利目的。
137 |
138 | 二、 许可及许可限制
139 | 根据本许可协议的条款和条件，许可方特此授予您一个非排他性、全球性、不可转让、不可再许可、可撤销、免版税的版权许可。您可以出于非商业用途使用此许可。许可方对您使用【Orion-14B系列】模型的输出或基于【Orion-14B系列】模型得到的模型衍生品不主张任何权利，但您必须满足如下许可限制条件：
140 | 1． 您不得出于任何军事或非法目的使用、复制、修改、合并、发布、分发、复制或创建【Orion-14B系列】 模型的全部或部分衍生品。
141 | 2． 如果您计划将【Orion-14B系列】模型及模型衍生品用作商业用途，应当按照本协议提供的联络方式，事先向许可方登记并获得许可方的书面授权。请点击以下链接进行登记：https://test.orionstar.com/llm-license.html。按照链接要求完成登记后即可获得商用授权。
142 | 3． 您对【Orion-14B系列】模型的使用和修改（包括使用【Orion-14B系列】 模型的输出或者基于【Orion-14B系列】 模型得到的模型衍生品）不得违反任何国家的法律法规，尤其是中华人民共和国的法律法规，不得侵犯任何第三方的合法权益，包括但不限于肖像权、名誉权、隐私权等 人格权，著作权、专利权、商业秘密等知识产权，或者其他财产权益。
143 | 4． 您必须向【Orion-14B系列】模型或其模型衍生品的任何第三方使用者提供【Orion-14B系列】模型的来源以及本协议的副本。
144 | 5． 您修改【Orion-14B系列】 模型得到模型衍生品，必须以显著的方式说明修改的内容，且上述修改不得违反本协议的许可限制条件，也不能允许、协助或以其他方式使得第三方违反本协议中的许可限制条件。
145 | 三、 知识产权
146 | 1. 【Orion-14B系列】模型的所有权及其相关知识产权，由许可方单独所有。
147 | 2. 在任何情况下，未经许可方事先书面同意，您不得使用许可方任何商标、服务标记、 商号、域名、网站名称或其他显著品牌特征（以下统称为"标识"），包括但不限于明示或暗示您自身为“许可方”。未经许可方事先书面同意，您不得将本条款前述标识以单独或结合的任何方式展示、使用或申请注册商标、进行域名注册等，也不得向他人明示或暗示有权展示、使用、或以其他方式处理这些标识的权利。由于您违反本协议使用许可方上述标识 等给许可方或他人造成损失的，由您承担全部法律责任。
148 | 3. 在许可范围内，您可以对【Orion-14B系列】模型进行修改以得到模型衍生品，对于模型衍生品中您付出创造性劳动的部分，您可以主张该部分的知识产权。
149 | 四、 免责声明及责任限制
150 | 1. 在任何情况下，许可方不对您根据本协议使用【Orion-14B系列】模型而产生或与之相关的任何直接、间接、附带的后果、以及其他损失或损害承担责任。若由此导致许可方遭受损失，您应当向许可方承担全部赔偿责任。
151 | 2. 模型中的模型参数仅仅是一种示例，如果您需要满足其他要求，需自行训练，并遵守相应数据集的许可协议。您将对【Orion-14B系列】模型的输出及模型衍生品所涉及的知识产权风险或与之相关的任何直接、间接、附带的后果、以及其他损失或损害负责。
152 | 3. 尽管许可方在【Orion-14B系列】模型训练的所有阶段，都坚持努力维护数据的合规性和准确 性，但受限于【Orion-14B系列】模型的规模及其概率固有的随机性因素影响，其输出结果的准确性无法得到保证，模型存在被误导的可能。因此，许可方在此声明，许可方不承担您因使用【Orion-14B系列】模型及其源代码而导致的数据安全问题、声誉风险，或任何涉及【Orion-14B系列】模型被误导、误用、传播或不正当使用而产生的任何风险和责任。
153 | 4. 本协议所称损失或损害包括但不限于下列任何损失或损害（无论此类损失或损害是不可预见的、可预见的、已知的或其他的）:(i)收入损失;(ii)实际或预期利润损失；(ii)货币使用损失；(iv)预期节约的损失；(v)业务损失；(vi)机会损失；(vii)商誉、声誉损失；(viii)软件的使用损失；或(x)任何间接、附带的特殊或间接损害损失。
154 | 5. 除非适用的法律另有要求或经过许可方书面同意，否则许可方将按“现状”授予【Orion-14B系列】模型的许可。针对本协议中的【Orion-14B系列】模型，许可方不提供任何明示、暗示的保证，包括但不限于：关于所有权的任何保证或条件、关于适销性的保证或条件、适用于任何特定目的的保证或条件、过去、现在或未来关于【Orion-14B系列】模型不侵权的任何类型的保证、以及因任何交易过程、贸易使用（如建议书、规范或样品）而产生的任何保证。您将对其通过使用、复制或再分发等方式利用【Orion-14B系列】模型所产生的风险与后果，独自承担责任。
155 | 6. 您充分知悉并理解同意，【Orion-14B系列】模型中可能包含个人信息。您承诺将遵守所有适用的法律法规进行个人信息的处理，特别是遵守《中华人民共和国个人信息保护法》的相关规定。请注意，许可方给予您使用【Orion-14B系列】模型的授权，并不意味着您已经获得处理相关个人信息的合法性基础。您作为独立的个人信息处理者，需要保证在处理【Orion-14B系列】模型中可能包含的个人信息时，完全符合相关法律法规的要求，包括但不限于获得个人信息主体的授权同意等，并愿意独自承担由此可能产生的任何风险和后果。
156 | 7. 您充分理解并同意，许可方有权依合理判断对违反有关法律法规或本协议规定的行为进行处理，对您的违法违规行为采取适当的法律行动，并依据法律法规保存有关信息向有关部门报告等，您应独自承担由此而产生的一切法律责任。
157 | 五、 研究、教育和学术目的
158 | 1. 根据本许可协议的条款和条件，本着对学术界做出贡献的精神，许可方鼓励非营利性学术机构的师生将【Orion-14B系列】模型用于研究、教育和学术目的。
159 | 2. 进一步的，如您以研究、教育和学术目的使用【Orion-14B系列】模型，您可以在开展相关研 究、教育前，将您的机构名称、使用情况以及联系方式以邮件方式向我们进行提前告知，我们的联系邮箱为【ai@orionstar.com】，我们将可能基于您的联系方式，向您推送【Orion-14B系列】模型的相关更新资讯，以便您更好地开展研究、教育和学术工作。
160 | 六、 品牌曝光与显著标识
161 | 1. 您同意并理解，如您将您基于【Orion-14B系列】模型二次开发的模型衍生品在国内外的开源社区提供开源许可的，您需要在该开源社区以显著方式标注该模型衍生品系基于【Orion-14B系列】模型进行的二次开发，标注内容包括但不限于“【Orion-14B Series】 Inside”以及与【Orion-14B系列】模型相关的品牌的其他元素。
162 | 2. 您同意并理解，如您将【Orion-14B系列】模型二次开发的模型衍生品参加国内外任何组织和个人举行的排名活动，包括但不限于针对模型性能、准确度、算法、算力等任何维度的排名活动，您均需在模型说明中以显著方式标注该模型衍生品系基于【Orion-14B系列】模型进行的二次开发，标注内容包括但不限于“【Orion-14B Series】Inside”以及与【Orion-14B系列】模型相关的品牌的其他元素。
163 | 七、 其他
164 | 1. 许可方在法律法规许可的范围内对协议条款享有最终解释权。
165 | 2. 本协议的订立、效力、解释、履行、修改和终止，使用【Orion-14B系列】模型以及争议的解 决均适用中华人民共和国大陆地区（仅为本协议之目的，不包括香港、澳门和台湾）法律，并排除冲突法的适用。
166 | 3. 因使用【Orion-14B系列】模型而发生的任何争议，各方应首先通过友好协商的方式加以解决。协商不成时，向许可方所在地人民法院提起诉讼。
167 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
26 |
27 |
28 |
29 | # Table of Contents
30 |
31 | - [📖 Model Introduction](#model-introduction)
32 | - [🔗 Model Download](#model-download)
33 | - [🔖 Model Benchmark](#model-benchmark)
34 | - [📊 Model Inference](#model-inference) [](#vllm) [](#llama-cpp)
35 | - [📜 Declarations & License](#declarations-license)
36 | - [🥇 Company Introduction](#company-introduction)
37 |
38 |
39 | # 1. Model Introduction
40 |
41 | - Orion-14B series models are open-source multilingual large language models trained from scratch by OrionStarAI. The base model is trained on 2.5T multilingual corpus, including Chinese, English, Japanese, Korean, etc, and it exhibits superior performance in these languages. For details, please refer to [tech report](https://arxiv.org/pdf/2401.12246.pdf).
42 |
43 | - The Orion-14B series models exhibit the following features:
44 | - Among models with 20B-parameter scale level, Orion-14B-Base model shows outstanding performance in comprehensive evaluations.
45 | - Strong multilingual capabilities, significantly outperforming in Japanese and Korean testsets.
46 | - The fine-tuned models demonstrate strong adaptability, excelling in human-annotated blind tests.
47 | - The long-chat version supports extremely long texts, performing exceptionally well at a token length of 200k and can support up to a maximum of 320k.
48 | - The quantized versions reduce model size by 70%, improve inference speed by 30%, with performance loss less than 1%.
49 |
50 |
51 |
52 |
53 |
54 |
55 |
56 |
57 |
58 |
59 |
60 | - Orion-14B series models including:
61 | - **Orion-14B-Base:** A multilingual large language foundational model with 14 billion parameters, pretrained on a diverse dataset of 2.5 trillion tokens.
62 | - **Orion-14B-Chat:** A chat-model fine-tuned on a high-quality corpus aims to provide an excellence interactive experience for users in the large model community.
63 | - **Orion-14B-LongChat:** The long-context version excels at handling extremely lengthy texts, performing exceptionally well at a token length of 200k and can support up to a maximum of 320k.
64 | - **Orion-14B-Chat-RAG:** A chat-model fine-tuned on a custom retrieval augmented generation dataset, achieving superior performance in retrieval augmented generation tasks.
65 | - **Orion-14B-Chat-Plugin:** A chat-model specifically tailored for plugin and function calling tasks, ideal for agent-related scenarios where the LLM acts as a plugin and function call system.
66 | - **Orion-14B-Base-Int4:** A quantized base model utilizing 4-bit integer weights. It significantly reduces the model size by 70% and increases the inference speed by 30% while incurring a minimal performance loss of only 1%.
67 | - **Orion-14B-Chat-Int4:** A quantized chat model utilizing 4-bit integer weights.
68 |
69 |
70 |
71 | # 2. Model Download
72 |
73 | Model release and download links are provided in the table below:
74 |
75 | | Model Name | HuggingFace Download Links | ModelScope Download Links | OpenXLab Download Links |
76 | |-------------------------|-----------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------|
77 | | ⚾Orion-14B-Base | [Orion-14B-Base](https://huggingface.co/OrionStarAI/Orion-14B-Base) | [Orion-14B-Base](https://modelscope.cn/models/OrionStarAI/Orion-14B-Base/summary) | [Orion-14B-Base](https://openxlab.org.cn/models/detail/OrionStarAI/Orion-14B-Base) |
78 | | 😛Orion-14B-Chat | [Orion-14B-Chat](https://huggingface.co/OrionStarAI/Orion-14B-Chat) | [Orion-14B-Chat](https://modelscope.cn/models/OrionStarAI/Orion-14B-Chat/summary) | [Orion-14B-Chat](https://openxlab.org.cn/models/detail/OrionStarAI/Orion-14B-Chat) |
79 | | 📃Orion-14B-LongChat | [Orion-14B-LongChat](https://huggingface.co/OrionStarAI/Orion-14B-LongChat) | [Orion-14B-LongChat](https://modelscope.cn/models/OrionStarAI/Orion-14B-LongChat/summary) | [Orion-14B-LongChat](https://openxlab.org.cn/models/detail/OrionStarAI/Orion-14B-LongChat) |
80 | | 🔎Orion-14B-Chat-RAG | [Orion-14B-Chat-RAG](https://huggingface.co/OrionStarAI/Orion-14B-Chat-RAG) | [Orion-14B-Chat-RAG](https://modelscope.cn/models/OrionStarAI/Orion-14B-Chat-RAG/summary) | [Orion-14B-Chat-RAG](https://openxlab.org.cn/models/detail/OrionStarAI/Orion-14B-Chat-RAG) |
81 | | 🔌Orion-14B-Chat-Plugin | [Orion-14B-Chat-Plugin](https://huggingface.co/OrionStarAI/Orion-14B-Chat-Plugin) | [Orion-14B-Chat-Plugin](https://modelscope.cn/models/OrionStarAI/Orion-14B-Chat-Plugin/summary) | [Orion-14B-Chat-Plugin](https://openxlab.org.cn/models/detail/OrionStarAI/Orion-14B-Chat-Plugin) |
82 | | 💼Orion-14B-Base-Int4 | [Orion-14B-Base-Int4](https://huggingface.co/OrionStarAI/Orion-14B-Base-Int4) | [Orion-14B-Base-Int4](https://modelscope.cn/models/OrionStarAI/Orion-14B-Base-Int4/summary) | [Orion-14B-Base-Int4](https://openxlab.org.cn/models/detail/OrionStarAI/Orion-14B-Base-Int4) |
83 | | 📦Orion-14B-Chat-Int4 | [Orion-14B-Chat-Int4](https://huggingface.co/OrionStarAI/Orion-14B-Chat-Int4) | [Orion-14B-Chat-Int4](https://modelscope.cn/models/OrionStarAI/Orion-14B-Chat-Int4/summary) | [Orion-14B-Chat-Int4](https://openxlab.org.cn/models/detail/OrionStarAI/Orion-14B-Chat-Int4) |
84 |
85 |
86 | # 3. Model Benchmarks
87 |
88 | ## 3.1. Base Model Orion-14B-Base Benchmarks
89 | ### 3.1.1. LLM evaluation results on examination and professional knowledge
90 | | Model | C-Eval | CMMLU | MMLU | AGIEval | Gaokao | BBH |
91 | |--------------------|----------|----------|----------|----------|----------|----------|
92 | | LLaMA2-13B | 41.4 | 38.4 | 55.0 | 30.9 | 18.2 | 45.6 |
93 | | Skywork-13B | 59.1 | 61.4 | 62.7 | 43.6 | 56.1 | 48.3 |
94 | | Baichuan2-13B | 59.0 | 61.3 | 59.5 | 37.4 | 45.6 | 49.0 |
95 | | QWEN-14B | 71.7 | 70.2 | 67.9 | 51.9 | **62.5** | 53.7 |
96 | | InternLM-20B | 58.8 | 59.0 | 62.1 | 44.6 | 45.5 | 52.5 |
97 | | **Orion-14B-Base** | **72.9** | **70.6** | **69.9** | **54.7** | 62.1 | **56.5** |
98 |
99 | ### 3.1.2. LLM evaluation results on language understanding and common knowledge
100 | | Model |RACE-middle|RACE-high |HellaSwag | PIQA | Lambada | WSC |
101 | |--------------------|----------|----------|----------|----------|----------|----------|
102 | | LLaMA 2-13B | 63.0 | 58.9 | 77.5 | 79.8 | 76.5 | 66.3 |
103 | | Skywork-13B | 87.6 | 84.1 | 73.7 | 78.3 | 71.8 | 66.3 |
104 | | Baichuan 2-13B | 68.9 | 67.2 | 70.8 | 78.1 | 74.1 | 66.3 |
105 | | QWEN-14B | 93.0 | 90.3 | **80.2** | 79.8 | 71.4 | 66.3 |
106 | | InternLM-20B | 86.4 | 83.3 | 78.1 | **80.3** | 71.8 | 68.3 |
107 | | **Orion-14B-Base** | **93.2** | **91.3** | 78.5 | 79.5 | **78.8** | **70.2** |
108 |
109 | ### 3.1.3. LLM evaluation results of OpenCompass testsets
110 | | Model | Average | Examination | Language | Knowledge | Understanding | Reasoning |
111 | |------------------|----------|----------|----------|----------|----------|----------|
112 | | LLaMA 2-13B | 47.3 | 45.2 | 47.0 | 58.3 | 50.9 | 43.6 |
113 | | Skywork-13B | 53.6 | 61.1 | 51.3 | 52.7 | 64.5 | 45.2 |
114 | | Baichuan 2-13B | 49.4 | 51.8 | 47.5 | 48.9 | 58.1 | 44.2 |
115 | | QWEN-14B | 62.4 | 71.3 | 52.67 | 56.1 | 68.8 | 60.1 |
116 | | InternLM-20B | 59.4 | 62.5 | 55.0 | **60.1** | 67.3 | 54.9 |
117 | |**Orion-14B-Base**| **64.3** | **71.4** | **55.0** | 60.0 | **71.9** | **61.6** |
118 |
119 | ### 3.1.4. Comparison of LLM performances on Japanese testsets
120 | | Model |**Average**| JCQA | JNLI | MARC | JSQD | JQK | XLS | XWN | MGSM |
121 | |--------------------|----------|----------|----------|----------|----------|----------|----------|----------|----------|
122 | | PLaMo-13B | 52.3 | 56.7 | 42.8 | 95.8 | 70.6 | 71.0 | 8.70 | 70.5 | 2.40 |
123 | | WebLab-10B | 50.7 | 66.6 | 53.7 | 82.1 | 62.9 | 56.2 | 10.0 | 72.0 | 2.40 |
124 | | ELYZA-jp-7B | 48.8 | 71.7 | 25.3 | 86.6 | 70.8 | 64.1 | 2.50 | 62.1 | 7.20 |
125 | | StableLM-jp-7B | 51.1 | 33.4 | 43.3 | **96.7** | 70.6 | 78.1 | 10.7 | 72.8 | 2.80 |
126 | | LLaMA 2-13B | 46.3 | 75.0 | 47.6 | 38.8 | 76.1 | 67.7 | 18.1 | 63.2 | 10.4 |
127 | | Baichuan 2-13B | 57.1 | 73.7 | 31.3 | 91.6 | 80.5 | 63.3 | 18.6 | 72.2 | 25.2 |
128 | | QWEN-14B | 65.8 | 85.9 | 60.7 | 97.0 | 83.3 | 71.8 | 18.8 | 70.6 | 38.0 |
129 | | Yi-34B | 67.1 | 83.8 | 61.2 | 95.2 | **86.1** | 78.5 | **27.2** | 69.2 | 35.2 |
130 | | **Orion-14B-Base** | **69.1** | **88.2** | **75.8** | 94.1 | 75.7 | **85.1** | 17.3 | **78.8** | **38.0** |
131 |
132 | ### 3.1.5. Comparison of LLM performances on Korean testsets. n = 0 and n = 5 stand for n-shot prompts used in the evaluation
133 | |Model | **Average** n=0 n=5 | HellaSwag n=0 n=5 | COPA n=0 n=5 | BooIQ n=0 n=5 | SentiNeg n=0 n=5|
134 | |------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------|
135 | | KoGPT | 53.0 70.1 | 55.9 58.3 | 73.5 72.9 | 45.1 59.8 | 37.5 89.4 |
136 | | Polyglot-ko-13B | 69.6 73.7 |**59.5** **63.1**|**79.4** **81.1**| 48.2 60.4 | 91.2 90.2 |
137 | | LLaMA 2-13B | 46.7 63.7 | 41.3 44.0 | 59.3 63.8 | 34.9 73.8 | 51.5 73.4 |
138 | | Baichuan 2-13B | 52.1 58.7 | 39.2 39.6 | 60.6 60.6 | 58.4 61.5 | 50.3 72.9 |
139 | | QWEN-14B | 53.8 73.7 | 45.3 46.8 | 64.9 68.9 | 33.4 83.5 | 71.5 95.7 |
140 | | Yi-34B | 54.2 72.1 | 44.6 44.7 | 58.0 60.6 | 65.9 90.2 | 48.3 92.9 |
141 | |**Orion-14B-Chat**|**74.5** **79.6**| 47.0 49.6 | 77.7 79.4 |**81.6** **90.7**|**92.4** **98.7**|
142 |
143 | ### 3.1.6. Multilingual evaluation
144 | | Model | Train Lang | Japanese | Korean | Chinese | English |
145 | |--------------------|------------|----------|----------|----------|----------|
146 | | PLaMo-13B | En,Jp | 52.3 | * | * | * |
147 | | Weblab-10B | En,Jp | 50.7 | * | * | * |
148 | | ELYZA-jp-7B | En,Jp | 48.8 | * | * | * |
149 | | StableLM-jp-7B | En,Jp | 51.1 | * | * | * |
150 | | KoGPT-6B | En,Ko | * | 70.1 | * | * |
151 | | Polyglot-ko-13B | En,Ko | * | 70.7 | * | * |
152 | | Baichuan2-13B | Multi | 57.1 | 58.7 | 50.8 | 57.1 |
153 | | Qwen-14B | Multi | 65.8 | 73.7 | 64.5 | 65.4 |
154 | | Llama2-13B | Multi | 46.3 | 63.7 | 41.4 | 55.3 |
155 | | Yi-34B | Multi | 67.1 | 72.2 | 58.7 | **68.8** |
156 | | **Orion-14B-Chat** | Multi | **69.1** | **79.5** | **67.9** | 67.3 |
157 |
158 |
159 | ## 3.2. Chat Model Orion-14B-Chat Benchmarks
160 | ### 3.2.1. Chat model subjective evaluation of MTBench
161 | | Model | First-Turn | Second-Turn | **Average** |
162 | |----------------------|----------|----------|----------|
163 | | Baichuan2-13B-Chat | 7.05 | 6.47 | 6.76 |
164 | | Qwen-14B-Chat | 7.30 | 6.62 | 6.96 |
165 | | Llama2-13B-Chat | 7.10 | 6.20 | 6.65 |
166 | | InternLM-20B-Chat | 7.03 | 5.93 | 6.48 |
167 | | **Orion-14B-Chat** | **7.68** | **7.07** | **7.37** |
168 | \* use vllm for inference
169 |
170 | ### 3.2.2. Chat model subjective evaluation of AlignBench
171 | | Model | Math. | Logi. | Basic. | Chi. | Comp. | Writ. | Role. | Prof. |**Avg.**|
172 | |--------------------|--------|--------|--------|--------|--------|--------|--------|--------|--------|
173 | | Baichuan2-13B-Chat | 3.76 | 4.07 | 6.22 | 6.05 | 7.11 | 6.97 | 6.75 | 6.43 | 5.25 |
174 | | Qwen-14B-Chat |**4.91**|**4.71**|**6.90**| 6.36 | 6.74 | 6.64 | 6.59 | 6.56 |**5.72**|
175 | | Llama2-13B-Chat | 3.05 | 3.79 | 5.43 | 4.40 | 6.76 | 6.63 | 6.99 | 5.65 | 4.70 |
176 | | InternLM-20B-Chat | 3.39 | 3.92 | 5.96 | 5.50 |**7.18**| 6.19 | 6.49 | 6.22 | 4.96 |
177 | | **Orion-14B-Chat** | 4.00 | 4.24 | 6.18 |**6.57**| 7.16 |**7.36**|**7.16**|**6.99**| 5.51 |
178 | \* use vllm for inference
179 |
180 | ## 3.3. LongChat Model Orion-14B-LongChat Benchmarks
181 | ### 3.3.1. LongChat evaluation of LongBench
182 | | Model | NarrativeQA|MultiFieldQA-en|MultiFieldQA-zh| DuReader | QMSum | VCSUM | TREC | TriviaQA | LSHT |RepoBench-P|
183 | |--------------------------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|
184 | | GPT-3.5-Turbo-16k | **23.60** | **52.30** | **61.20** | 28.70 | 23.40 | **16.00** | 68.00 | **91.40** | 29.20 | 53.60 |
185 | | LongChat-v1.5-7B-32k | 16.90 | 41.40 | 29.10 | 19.50 | 22.70 | 9.90 | 63.50 | 82.30 | 23.20 | 55.30 |
186 | | Vicuna-v1.5-7B-16k | 19.40 | 38.50 | 43.00 | 19.30 | 22.80 | 15.10 | 71.50 | 86.20 | 28.80 | 43.50 |
187 | | Yi-6B-200K | 14.11 | 36.74 | 22.68 | 14.01 | 20.44 | 8.08 | 72.00 | 86.61 | 38.00 | **63.29** |
188 | | Orion-14B-LongChat | 19.47 | 48.11 | 55.84 | **37.02** | **24.87** | 15.44 | **77.00** | 89.12 | **45.50** | 54.31 |
189 |
190 |
191 | ## 3.4. Chat RAG Model Benchmarks
192 | ### 3.4.1. LLM evaluation results of self-built RAG testsets
193 | |Model|Effectiveness of Response(Keyword)|*Effectiveness of Response（subjective evaluation）|Quoting Ability|Fallback Ability|*AutoQA|*Data Extraction|
194 | |---------------------|------|------|------|------|------|------|
195 | | Baichuan2-13B-Chat | 85 | 76 | 1 | 0 | 69 | 51 |
196 | | Qwen-14B-Chat | 79 | 77 | 75 | 47 | 68 | 72 |
197 | | Qwen-72B-Chat(Int4) | 87 | 89 | 90 | 32 | 67 | 76 |
198 | | GPT-4 | 91 | 94 | 96 | 95 | 75 | 86 |
199 | | Orion-14B-Chat-RAG | 86 | 87 | 91 | 97 | 73 | 71 |
200 | \* means manual assessment
201 |
202 | ## 3.5. Chat Plugin Model Orion-14B-Chat-Plugin Benchmarks
203 | ### 3.5.1. LLM evaluation results of self-built plugin testsets
204 | |Model |Intent Recognition with Full Params |Intent Recognition with Missing Params |Non-Plugin Invocation Recognition |
205 | |-----------------------|--------|-----------|--------|
206 | | Baichuan2-13B-Chat | 25 | 0 | 0 |
207 | | Qwen-14B-Chat | 55 | 0 | 50 |
208 | | GPT-4 | **95** | 52.38 | 70 |
209 | | Orion-14B-Chat-Plugin | 92.5 | **60.32** | **90** |
210 |
211 | ## 3.6. Quantized Model Orion-14B-Base-Int4 Benchmarks
212 | ### 3.6.1. Comparison of before and after quantization
213 | |Model |Size(GB)|Inference Speed(tokens/s)|C-Eval|CMMLU|MMLU|RACE|HellaSwag|
214 | |-------------------------|-------|-----|------|------|------|------|------|
215 | | OrionStar-14B-Base | 28.0 | 135 | 72.8 | 70.6 | 70.0 | 93.3 | 78.5 |
216 | | OrionStar-14B-Base-Int4 | 8.3 | 178 | 71.8 | 69.8 | 69.2 | 93.1 | 78.0 |
217 |
218 |
219 |
220 | # 4. Model Inference
221 |
222 | Model weights, source code, and configuration needed for inference are published on Hugging Face, and the download link
223 | is available in the table at the beginning of this document. We demonstrate various inference methods here, and the
224 | program will automatically download the necessary resources from Hugging Face.
225 |
226 | ## 4.1. Python Code
227 |
228 | ```python
229 | import torch
230 | from transformers import AutoModelForCausalLM, AutoTokenizer
231 | from transformers.generation.utils import GenerationConfig
232 |
233 | tokenizer = AutoTokenizer.from_pretrained("OrionStarAI/Orion-14B", use_fast=False, trust_remote_code=True)
234 | model = AutoModelForCausalLM.from_pretrained("OrionStarAI/Orion-14B", device_map="auto",
235 | torch_dtype=torch.bfloat16, trust_remote_code=True)
236 |
237 | model.generation_config = GenerationConfig.from_pretrained("OrionStarAI/Orion-14B")
238 | messages = [{"role": "user", "content": "Hello, what is your name? "}]
239 | response = model.chat(tokenizer, messages, streaming=False)
240 | print(response)
241 |
242 | ```
243 |
244 | In the above Python code, the model is loaded with `device_map='auto'` to utilize all available GPUs. To specify the
245 | device, you can use something like `export CUDA_VISIBLE_DEVICES=0,1` (using GPUs 0 and 1).
246 |
247 | ## 4.2. Command Line Tool
248 |
249 | ```shell
250 | CUDA_VISIBLE_DEVICES=0 python cli_demo.py
251 | ```
252 |
253 | This command-line tool is designed for chat scenarios, and thus, it does not support calling the base model.
254 |
255 | ## 4.3. Direct Script Inference
256 |
257 | ```shell
258 |
259 | # base model
260 | CUDA_VISIBLE_DEVICES=0 python demo/text_generation_base.py --model OrionStarAI/Orion-14B --tokenizer OrionStarAI/Orion-14B --prompt hello
261 |
262 | # chat model
263 | CUDA_VISIBLE_DEVICES=0 python demo/text_generation.py --model OrionStarAI/Orion-14B-Chat --tokenizer OrionStarAI/Orion-14B-Chat --prompt hi
264 |
265 | ```
266 |
267 |
268 | ## 4.4. Inference by vLLM(>=v0.3.3)
269 |
270 | - vLLM url & QuickStart
271 | https://github.com/vllm-project/vllm
272 | https://docs.vllm.ai/en/latest/getting_started/quickstart.html
273 | - Start vllm server:
274 | ```shell
275 | python -m vllm.entrypoints.openai.api_server --model OrionStarAI/Orion-14B-Chat
276 | ```
277 |
278 |
279 | ## 4.5. Inference by llama.cpp
280 |
281 | - Project URL
282 | https://github.com/ggerganov/llama.cpp
283 |
284 | - Orion-14B-Chat.gguf
285 | https://huggingface.co/OrionStarAI/Orion-14B-Chat/blob/main/Orion-14B-Chat.gguf
286 |
287 | - How to convert to GGUF model
288 |
289 | ```shell
290 | python convert-hf-to-gguf.py path/to/Orion-14B-Chat --outfile chat.gguf
291 | ```
292 |
293 | - How to run generation
294 |
295 | ```shell
296 | ./main --frequency-penalty 0.5 --frequency-penalty 0.5 --top-k 5 --top-p 0.9 -m chat.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 400 -e
297 | ```
298 |
299 |
300 | ## 4.6 Example Output
301 |
302 | ### 4.6.1. Casual Chat
303 |
304 | `````
305 | User: Hello
306 | Orion-14B: Hello! How can I help you today? If you have any questions or need assistance, feel free to ask.
307 | User: What functions do you have?
308 | Orion-14B: As an AI language model, I have a wide range of functions and capabilities, including but not limited to:
309 | Answering general knowledge questions: I can provide information on various topics such as history, science, geography, and more.
310 | Providing definitions and explanations: I can explain complex concepts and provide definitions for words and phrases.
311 | Assisting with tasks: I can help with simple tasks like setting reminders, converting units of measurement, and even ordering food from restaurants.
312 | Generating text: I can write short stories, essays, articles, and other types of content based on prompts given to me.
313 | Translating languages: I can translate text from one language to another.
314 | Chatting: I can engage in conversation with users and respond to their queries in a conversational manner.
315 | Summarizing long texts: I can summarize longer texts like articles, books, and research papers.
316 | Generating random numbers and names: I can generate random numbers within a specified range and also suggest names based on certain criteria.
317 | Playing games: I can play simple games like Hangman, Trivia, and more.
318 | Providing recommendations: I can recommend movies, music, books, and other forms of entertainment based on user preferences.
319 | These are just some of the many functions that I am capable of performing. If you have a specific task in mind, feel free to ask me about it.
320 | User: Tell me a joke.
321 | Orion-14B: Sure, here's a classic one-liner: Why don't scientists trust atoms? Because they make up everything.
322 | `````
323 |
324 | ### 4.6.2. Japanese & Korean Chat
325 |
326 | `````
327 | User：自己を紹介してください
328 | Orion-14B：こんにちは、私の名前はChatMaxで、OrionStarによって開発されたAIアシスタントです。私はユーザーに正確で、客観的で、有用な回答を提供し、彼らが問題を解決し、情報を取得するのを助けることができます。
329 | User：자기소개를 해주세요.
330 | Orion-14B：안녕하세요, 제 이름은 ChatMax입니다. OrionStar에서 개발한 AI 어시스턴트입니다. 저는 사용자들에게 정확하고 객관적이며 유용한 답변을 제공하여 문제를 해결하고 정보를 얻는 데 도움을 줄 수 있습니다.
331 | `````
332 |
333 |
334 | # 5. Declarations, License
335 |
336 | ## 5.1. Declarations
337 |
338 | We strongly urge all users not to use the Orion-14B model for any activities that may harm national or social security or violate the law.
339 | Additionally, we request users not to use the Orion-14B model for internet services without proper security review and filing.
340 | We hope all users abide by this principle to ensure that technological development takes place in a regulated and legal environment.
341 | We have done our best to ensure the compliance of the data used in the model training process. However, despite our
342 | significant efforts, unforeseen issues may still arise due to the complexity of the model and data. Therefore, if any
343 | problems arise due to the use of the Orion-14B open-source model, including but not limited to data security
344 | issues, public opinion risks, or any risks and issues arising from the model being misled, abused, disseminated, or
345 | improperly utilized, we will not assume any responsibility.
346 |
347 | ## 5.2. License
348 |
349 | Community use of the Orion-14B series models
350 | - For code, please comply with [Apache License Version 2.0](./LICENSE)
351 | - For model, please comply with [【Orion-14B Series】 Models Community License Agreement](./ModelsCommunityLicenseAgreement)
352 |
353 |
354 |
355 | # 6. Company Introduction
356 |
357 | OrionStar is a leading global service robot solutions company, founded in September 2016. OrionStar is dedicated to
358 | using artificial intelligence technology to create the next generation of revolutionary robots, allowing people to break
359 | free from repetitive physical labor and making human work and life more intelligent and enjoyable. Through technology,
360 | OrionStar aims to make society and the world a better place.
361 |
362 | OrionStar possesses fully self-developed end-to-end artificial intelligence technologies, such as voice interaction and
363 | visual navigation. It integrates product development capabilities and technological application capabilities. Based on
364 | the Orion robotic arm platform, it has launched products such as OrionStar AI Robot Greeting, AI Robot Greeting Mini,
365 | Lucki, Coffee Master, and established the open platform OrionOS for Orion robots. Following the philosophy of "Born for
366 | Truly Useful Robots", OrionStar empowers more people through AI technology.
367 |
368 | **The core strengths of OrionStar lies in possessing end-to-end AI application capabilities,** including big data preprocessing, large model pretraining, fine-tuning, prompt engineering, agent, etc. With comprehensive end-to-end model training capabilities, including systematic data processing workflows and the parallel model training capability of hundreds of GPUs, it has been successfully applied in various industry scenarios such as government affairs, cloud services, international e-commerce, and fast-moving consumer goods.
369 |
370 | Companies with demands for deploying large-scale model applications are welcome to contact us.
371 | **Enquiry Hotline: 400-898-7779**
372 | **E-mail: ai@orionstar.com**
373 | **Discord Link: https://discord.gg/zumjDWgdAs**
374 |
375 |
26 |
27 |
28 |
29 | # 목록
30 |
31 | - [📖 모형 소개](#model-introduction)
32 | - [🔗 다운로드 경로](#model-download)
33 | - [🔖 평가결과](#model-benchmark)
34 | - [📊 모형 추리](#model-inference) [](#vllm) [](#llama-cpp)
35 | - [📜 성명 합의](#declarations-license)
36 | - [🥇 기업 소개](#company-introduction)
37 |
38 |
39 |
40 | # 1. 모델소게
41 |
42 |
43 | -Orion-14B-Base는 2.5조 토큰의 다양한 데이터 집합으로 훈련된 140억 개의 파라메터를 가진 다중 언어 모델이다. 중국어, 영어, 일본어, 한국어 및 기타 언어를 포함한다.다중 언어 환경에서 일련의 업무에서 탁월한 성능을 보인다. Orion-14B 시리즈의 모델들은 주요 공개 기준 측정에서 우수한 성적을 거두었으며 여러가지 지표가 동일한 파라메터를 가진 다른 모델들을 현저히 초월한다. 구체적인 기술 디테일은 [기술보고서]를 참고하세요.
44 | (https://github.com/OrionStarAI/Orion/blob/master/doc/Orion14B_v3.pdf)。
45 |
46 | - Orion-14B시리즈 대형 모델은 다음과 같은 특징이 있다.
47 | - 베이스20B 파라메터 레벨인 대형 모델의 종합적인 평가 결과가 우수하다
48 | - 다국어 능력이 뛰어나고 일본어와 한국어 테스트 세트에서 현저히 앞선다
49 | - 미세조정 모델은 적응성이 강하며 인위 표시의 블라인드 테스트에서 활약이 두드러진다
50 | - 긴 컨텍스트 버전은 최대 320k까지 지원하는 200k 토큰에 뛰어난 긴 텍스트를 지지한다
51 | - 정량화 버전 모델 크기를 70% 줄이고 추론 속도를 30% 높이며 성능 손실을 1% 미만하다
52 |
53 |
54 |
55 |
56 |
57 |
58 |
59 |
60 |
61 |
62 |
63 | - 구체적으로 말하면 Orion-14B시리즈 대형 언어 모델은 다음과 같은 내용을 포함한다:
64 | - **Orion-14B-Base:** 2.5억 토켄스 다양화 데이터 세트를 기반으로 한 140억 파라메터 규모의 다언어 기반 모델.
65 | - **Orion-14B-Chat:** 고퀄리티 코퍼스 미세조정을 기반으로 한 대화형 모델. 대형 모델 커뮤니티를 위해 더 나은 사용자 인터랙션 경험을 제공하도록 한다.
66 | - **Orion-14B-LongChat:** 200k 토큰 길이에 효과적이며 최대 320k까지 지원하며 긴 텍스트 평가 세트에서 독점 모델과 비교할 수 있다.
67 | - **Orion-14B-Chat-RAG:** 맞춰 제정된 검색 향상 생성 데이터 세트에서 미세조정하여 검색 향상 생성 작업에서 뛰어난 성능을 제공한 채팅 모델.
68 | - **Orion-14B-Chat-Plugin:** 플러그인 및 함수 전용 작업에 맞춰 제정된 채팅 모델. 에이전트와 관련된 상황에 아주 잘 적용되어 대형 언어 모델이 플러그인 및 함수 전용 시스템의 역할을 한다.
69 | - **Orion-14B-Base-Int4:** int4로 계량화하는 베이스 모델. 모델 크기를 70%를 줄이며 추리 속도를 30% 높여 1%의 최소한의 성능 손실만 가져왔다.
70 | - **Orion-14B-Chat-Int4:** int4로 계량화하는 대화 모델.
71 |
72 |
73 |
74 | # 2. 다운로드 경로
75 |
76 | 발표된 모델 및 다운로드 링크는 다음 표를 참조하세요:
77 |
78 | | 모델 명칭 | HuggingFace다운로드 링크 | ModelScope다운로드 링크 |
79 | |---------------------|-----------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------|
80 | | ⚾ 베이스 모델 | [Orion-14B-Base](https://huggingface.co/OrionStarAI/Orion-14B-Base) | [Orion-14B-Base](https://modelscope.cn/models/OrionStarAI/Orion-14B-Base/summary) |
81 | | 😛 대화 모델 | [Orion-14B-Chat](https://huggingface.co/OrionStarAI/Orion-14B-Chat) | [Orion-14B-Chat](https://modelscope.cn/models/OrionStarAI/Orion-14B-Chat/summary) |
82 | | 📃 긴 컨텍스트 모델 | [Orion-14B-LongChat](https://huggingface.co/OrionStarAI/Orion-14B-LongChat) | [Orion-14B-LongChat](https://modelscope.cn/models/OrionStarAI/Orion-14B-LongChat/summary) |
83 | | 🔎 검색 향상 모델 | [Orion-14B-Chat-RAG](https://huggingface.co/OrionStarAI/Orion-14B-Chat-RAG) | [Orion-14B-Chat-RAG](https://modelscope.cn/models/OrionStarAI/Orion-14B-Chat-RAG/summary) |
84 | | 🔌 플러그인 모델 | [Orion-14B-Chat-Plugin](https://huggingface.co/OrionStarAI/Orion-14B-Chat-Plugin) | [Orion-14B-Chat-Plugin](https://modelscope.cn/models/OrionStarAI/Orion-14B-Chat-Plugin/summary)|
85 | | 💼 베이스Int4계량화 모델 | [Orion-14B-Base-Int4](https://huggingface.co/OrionStarAI/Orion-14B-Base-Int4) | [Orion-14B-Base-Int4](https://modelscope.cn/models/OrionStarAI/Orion-14B-Base-Int4/summary) |
86 | | 📦 대화Int4계량화 모델 | [Orion-14B-Chat-Int4](https://huggingface.co/OrionStarAI/Orion-14B-Chat-Int4) | [Orion-14B-Chat-Int4](https://modelscope.cn/models/OrionStarAI/Orion-14B-Chat-Int4/summary) |
87 |
88 |
89 |
90 | # 3. 평가 결과
91 |
92 | ## 3.1. 베이스 모델Orion-14B-Base평가
93 |
94 | ### 3.1.1. 전문 지식 및 시험문제 평가 결과
95 | | 모델 명칭 | C-Eval | CMMLU | MMLU | AGIEval | Gaokao | BBH |
96 | |--------------------|----------|----------|----------|----------|----------|----------|
97 | | LLaMA2-13B | 41.4 | 38.4 | 55.0 | 30.9 | 18.2 | 45.6 |
98 | | Skywork-13B | 59.1 | 61.4 | 62.7 | 43.6 | 56.1 | 48.3 |
99 | | Baichuan2-13B | 59.0 | 61.3 | 59.5 | 37.4 | 45.6 | 49.0 |
100 | | QWEN-14B | 71.7 | 70.2 | 67.9 | 51.9 | **62.5** | 53.7 |
101 | | InternLM-20B | 58.8 | 59.0 | 62.1 | 44.6 | 45.5 | 52.5 |
102 | | **Orion-14B-Base** | **72.9** | **70.6** | **69.9** | **54.7** | 62.1 | **56.5** |
103 |
104 | ### 3.1.2. 이해 및 통식 평가 결과
105 | | 모델 명칭 |RACE-middle|RACE-high| HellaSwag| PIQA | Lambada | WSC |
106 | |--------------------|----------|----------|----------|----------|----------|----------|
107 | | LLaMA 2-13B | 63.0 | 58.9 | 77.5 | 79.8 | 76.5 | 66.3 |
108 | | Skywork-13B | 87.6 | 84.1 | 73.7 | 78.3 | 71.8 | 66.3 |
109 | | Baichuan 2-13B | 68.9 | 67.2 | 70.8 | 78.1 | 74.1 | 66.3 |
110 | | QWEN-14B | 93.0 | 90.3 | **80.2** | 79.8 | 71.4 | 66.3 |
111 | | InternLM-20B | 86.4 | 83.3 | 78.1 | **80.3** | 71.8 | 68.3 |
112 | | **Orion-14B-Base** | **93.2** | **91.3** | 78.5 | 79.5 | **78.8** | **70.2** |
113 |
114 | ### 3.1.3. OpenCompass평가 세트 평가 결과
115 | | 모델 명칭 | Average | Examination | Language | Knowledge | Understanding | Reasoning |
116 | |------------------|----------|----------|----------|----------|----------|----------|
117 | | LLaMA 2-13B | 47.3 | 45.2 | 47.0 | 58.3 | 50.9 | 43.6 |
118 | | Skywork-13B | 53.6 | 61.1 | 51.3 | 52.7 | 64.5 | 45.2 |
119 | | Baichuan 2-13B | 49.4 | 51.8 | 47.5 | 48.9 | 58.1 | 44.2 |
120 | | QWEN-14B | 62.4 | 71.3 | 52.67 | 56.1 | 68.8 | 60.1 |
121 | | InternLM-20B | 59.4 | 62.5 | 55.0 | **60.1** | 67.3 | 54.9 |
122 | |**Orion-14B-Base**| **64.3** | **71.4** | **55.0** | 60.0 | **71.9** | **61.6** |
123 |
124 | ### 3.1.4. 일본어 테스트 세트 평가 결과
125 | | 모델 명칭 |**Average**| JCQA | JNLI | MARC | JSQD | JQK | XLS | XWN | MGSM |
126 | |--------------------|----------|----------|----------|----------|----------|----------|----------|----------|----------|
127 | | PLaMo-13B | 52.3 | 56.7 | 42.8 | 95.8 | 70.6 | 71.0 | 8.70 | 70.5 | 2.40 |
128 | | WebLab-10B | 50.7 | 66.6 | 53.7 | 82.1 | 62.9 | 56.2 | 10.0 | 72.0 | 2.40 |
129 | | ELYZA-jp-7B | 48.8 | 71.7 | 25.3 | 86.6 | 70.8 | 64.1 | 2.50 | 62.1 | 7.20 |
130 | | StableLM-jp-7B | 51.1 | 33.4 | 43.3 | **96.7** | 70.6 | 78.1 | 10.7 | 72.8 | 2.80 |
131 | | LLaMA 2-13B | 46.3 | 75.0 | 47.6 | 38.8 | 76.1 | 67.7 | 18.1 | 63.2 | 10.4 |
132 | | Baichuan 2-13B | 57.1 | 73.7 | 31.3 | 91.6 | 80.5 | 63.3 | 18.6 | 72.2 | 25.2 |
133 | | QWEN-14B | 65.8 | 85.9 | 60.7 | 97.0 | 83.3 | 71.8 | 18.8 | 70.6 | 38.0 |
134 | | Yi-34B | 67.1 | 83.8 | 61.2 | 95.2 | **86.1** | 78.5 | **27.2** | 69.2 | 35.2 |
135 | | **Orion-14B-Base** | **69.1** | **88.2** | **75.8** | 94.1 | 75.7 | **85.1** | 17.3 | **78.8** | **38.0** |
136 |
137 | ### 3.1.5. 한국어 테스트 세트n-shot평가 결과
138 | | 모델 명칭 | **Average** n=0 n=5 | HellaSwag n=0 n=5 | COPA n=0 n=5 | BooIQ n=0 n=5 | SentiNeg n=0 n=5|
139 | |------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------|
140 | | KoGPT | 53.0 70.1 | 55.9 58.3 | 73.5 72.9 | 45.1 59.8 | 37.5 89.4 |
141 | | Polyglot-ko-13B | 69.6 73.7 |**59.5** **63.1**|**79.4** **81.1**| 48.2 60.4 | 91.2 90.2 |
142 | | LLaMA 2-13B | 46.7 63.7 | 41.3 44.0 | 59.3 63.8 | 34.9 73.8 | 51.5 73.4 |
143 | | Baichuan 2-13B | 52.1 58.7 | 39.2 39.6 | 60.6 60.6 | 58.4 61.5 | 50.3 72.9 |
144 | | QWEN-14B | 53.8 73.7 | 45.3 46.8 | 64.9 68.9 | 33.4 83.5 | 71.5 95.7 |
145 | | Yi-34B | 54.2 72.1 | 44.6 44.7 | 58.0 60.6 | 65.9 90.2 | 48.3 92.9 |
146 | |**Orion-14B-Base**|**74.5** **79.6**| 47.0 49.6 | 77.7 79.4 |**81.6** **90.7**|**92.4** **98.7**|
147 |
148 | ### 3.1.6. 다국어 평가 결과
149 | | 모델 명칭 | Train Lang | Japanese | Korean | Chinese | English |
150 | |--------------------|------------|----------|----------|----------|----------|
151 | | PLaMo-13B | En,Jp | 52.3 | * | * | * |
152 | | Weblab-10B | En,Jp | 50.7 | * | * | * |
153 | | ELYZA-jp-7B | En,Jp | 48.8 | * | * | * |
154 | | StableLM-jp-7B | En,Jp | 51.1 | * | * | * |
155 | | KoGPT-6B | En,Ko | * | 70.1 | * | * |
156 | | Polyglot-ko-13B | En,Ko | * | 70.7 | * | * |
157 | | Baichuan2-13B | Multi | 57.1 | 58.7 | 50.8 | 57.1 |
158 | | Qwen-14B | Multi | 65.8 | 73.7 | 64.5 | 65.4 |
159 | | Llama2-13B | Multi | 46.3 | 63.7 | 41.4 | 55.3 |
160 | | Yi-34B | Multi | 67.1 | 72.2 | 58.7 | **68.8** |
161 | | **Orion-14B-Base** | Multi | **69.1** | **79.5** | **67.9** | 67.3 |
162 |
163 | ## 3.2. 대화 모델Orion-14B-Chat평가
164 | ### 3.2.1. 대화 모델MTBench주관적 평가
165 | | 모델 명칭 | 1라운드 | 2라운드 | **평균** |
166 | |----------------------|----------|----------|----------|
167 | | Baichuan2-13B-Chat | 7.05 | 6.47 | 6.76 |
168 | | Qwen-14B-Chat | 7.30 | 6.62 | 6.96 |
169 | | Llama2-13B-Chat | 7.10 | 6.20 | 6.65 |
170 | | InternLM-20B-Chat | 7.03 | 5.93 | 6.48 |
171 | | **Orion-14B-Chat** | **7.68** | **7.07** | **7.37** |
172 |
173 | \*이 평가는 vllm을 이용하여 추리한다
174 |
175 | ### 3.2.2. 대화 모델AlignBench주관적 평가
176 | | 모델 명칭 | 수학 능력 | 논리적 추리 | 기본 능력 | 중국어 이해 | 종합적 문답 | 글쓰기 능력 | 롤 플레이 | 전문 지식 | **평균** |
177 | |--------------------|----------|----------|----------|----------|----------|----------|----------|----------|----------|
178 | | Baichuan2-13B-Chat | 3.76 | 4.07 | 6.22 | 6.05 | 7.11 | 6.97 | 6.75 | 6.43 | 5.25 |
179 | | Qwen-14B-Chat | **4.91** | **4.71** | **6.90** | 6.36 | 6.74 | 6.64 | 6.59 | 6.56 | **5.72** |
180 | | Llama2-13B-Chat | 3.05 | 3.79 | 5.43 | 4.40 | 6.76 | 6.63 | 6.99 | 5.65 | 4.70 |
181 | | InternLM-20B-Chat | 3.39 | 3.92 | 5.96 | 5.50 | **7.18** | 6.19 | 6.49 | 6.22 | 4.96 |
182 | | **Orion-14B-Chat** | 4.00 | 4.24 | 6.18 | **6.57** | 7.16 | **7.36** | **7.16** | **6.99** | 5.51 |
183 |
184 | \*이 평가는 vllm을 이용하여 추리한다
185 |
186 | ## 3.3. 긴 컨텍스트 모델Orion-14B-LongChat평가
187 | ### 3.3.1. 긴 컨텍스트 모델LongBench평가
188 | | 모델 명칭 | NarrativeQA| MultiFieldQA-en| MultiFieldQA-zh | DuReader | QMSum | VCSUM | TREC | TriviaQA | LSHT | RepoBench-P |
189 | |--------------------------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|
190 | | GPT-3.5-Turbo-16k | **23.60** | **52.30** | **61.20** | 28.70 | 23.40 | **16.00** | 68.00 | **91.40** | 29.20 | 53.60 |
191 | | LongChat-v1.5-7B-32k | 16.90 | 41.40 | 29.10 | 19.50 | 22.70 | 9.90 | 63.50 | 82.30 | 23.20 | 55.30 |
192 | | Vicuna-v1.5-7B-16k | 19.40 | 38.50 | 43.00 | 19.30 | 22.80 | 15.10 | 71.50 | 86.20 | 28.80 | 43.50 |
193 | | Yi-6B-200K | 14.11 | 36.74 | 22.68 | 14.01 | 20.44 | 8.08 | 72.00 | 86.61 | 38.00 | **63.29** |
194 | | Orion-14B-LongChat | 19.47 | 48.11 | 55.84 | **37.02** | **24.87** | 15.44 | **77.00** | 89.12 | **45.50** | 54.31 |
195 |
196 | ## 3.4. 검색 향상 모델Orion-14B-Chat-RAG평가
197 | ### 3.4.1. 자기 만든 검색 향상 테스트 세트 평가 결과
198 | |모델 명칭|응답 효과(키워드)|*응답 효과(주관적 점수)|인용 능력|기본 떠맡는 능력|*AutoQA|*데이터 추출|
199 | |---------------------|------|------|------|------|------|------|
200 | | Baichuan2-13B-Chat | 85 | 76 | 1 | 0 | 69 | 51 |
201 | | Qwen-14B-Chat | 79 | 77 | 75 | 47 | 68 | 72 |
202 | | Qwen-72B-Chat(Int4) | 87 | 89 | 90 | 32 | 67 | 76 |
203 | | GPT-4 | 91 | 94 | 96 | 95 | 75 | 86 |
204 | | Orion-14B-Chat-RAG | 86 | 87 | 91 | 97 | 73 | 71 |
205 | \* 사람 평가 결과를 가리킨다
206 |
207 | ## 3.5. 플러그인 모델Orion-14B-Chat-Plugin평가
208 | ### 3.5.1. 자기 만든플러그인 테스트 세트 평가 결과
209 | | 모델 명칭 | 풀 파라메터 의도 식별 | 불완전 파라메터 의도 식별 | 비 플러그인 전용 식별 |
210 | |-----------------------|--------|-----------|--------|
211 | | Baichuan2-13B-Chat | 25 | 0 | 0 |
212 | | Qwen-14B-Chat | 55 | 0 | 50 |
213 | | GPT-4 | **95** | 52.38 | 70 |
214 | | Orion-14B-Chat-Plugin | 92.5 | **60.32** | **90** |
215 |
216 | ## 3.6. 계량화 모델Orion-14B-Base-Int4평가
217 | ### 3.6.1. 계량화 전후 전반적인 비교
218 | |모델 명칭|모델 크기(GB)|추리 속도(토큰 수/초)|C-Eval |CMMLU |MMLU |RACE | HellaSwag|
219 | |-------------------------|------|-----|------|------|------|------|------|
220 | | OrionStar-14B-Base | 28.0 | 135 | 72.8 | 70.6 | 70.0 | 93.3 | 78.5 |
221 | | OrionStar-14B-Base-Int4 | 8.3 | 178 | 71.8 | 69.8 | 69.2 | 93.1 | 78.0 |
222 |
223 |
224 |
225 | # 4. 모델 추리
226 |
227 | 추리에 필요한 모델 가중치, 소스 코드, 배치는 Hugging Face에 게시되어 다운로드 링크는 이 파일 맨 처음에 있는 표를 참조하세요. 저희는 여기서 다양한 추리 방식을 보여 주고 프로그램은 Hugging Face로부터 필요한 자료를 자동으로 다운로드 할 것이다.
228 |
229 | ## 4.1. Python 코드 방식
230 |
231 | ```python
232 | import torch
233 | from transformers import AutoModelForCausalLM, AutoTokenizer
234 | from transformers.generation.utils import GenerationConfig
235 |
236 | tokenizer = AutoTokenizer.from_pretrained("OrionStarAI/Orion-14B", use_fast=False, trust_remote_code=True)
237 | model = AutoModelForCausalLM.from_pretrained("OrionStarAI/Orion-14B", device_map="auto",
238 | torch_dtype=torch.bfloat16, trust_remote_code=True)
239 |
240 | model.generation_config = GenerationConfig.from_pretrained("OrionStarAI/Orion-14B")
241 | messages = [{"role": "user", "content": "안녕! 이름이 뭐예요!"}]
242 | response = model.chat(tokenizer, messages, streaming=Flase)
243 | print(response)
244 |
245 | ```
246 |
247 | 위의 두 코드에서 모델은 지정된 `device_map='auto'`로딩하면 모든 사용할 수 있는 그래픽 카드를 사용할 것이다. 사용할 장치를 지정하려면 `export CUDA_VISIBLE_DEVICES=0,1`(그래픽 카드 0과 1을 사용)과 같은 방식으로 제어할 수 있다.
248 |
249 | ## 4.2. 명령줄 툴 방식
250 |
251 | ```shell
252 | CUDA_VISIBLE_DEVICES=0 python cli_demo.py
253 | ```
254 |
255 | 이 명령줄 툴은 Chat 시나리오를 위해 설계되었으므로 이 툴로 베이스 모델을 전용하는 것 지원하지 않는다.
256 |
257 | ## 4.3. 스크립트 직접 추리
258 |
259 | ```shell
260 | # base model
261 | CUDA_VISIBLE_DEVICES=0 python demo/text_generation_base.py --model OrionStarAI/Orion-14B --tokenizer OrionStarAI/Orion-14B --prompt 안녕. 이름이 뭐예요
262 |
263 | # chat model
264 | CUDA_VISIBLE_DEVICES=0 python demo/text_generation.py --model OrionStarAI/Orion-14B-Chat --tokenizer OrionStarAI/Orion-14B-Chat --prompt 안녕. 이름이 뭐예요
265 |
266 | ```
267 |
268 |
269 | ## 4.4. vLLM 추론을 통해
270 |
271 | - 프로젝트 주소
272 | https://github.com/vllm-project/vllm
273 |
274 | - 풀 리퀘스트
275 | https://github.com/vllm-project/vllm/pull/2539
276 |
277 |
278 |
279 | ## 4.5. llama.cpp 추론을 통해
280 |
281 | - 프로젝트 주소
282 | https://github.com/ggerganov/llama.cpp
283 |
284 | - 풀 리퀘스트
285 | https://github.com/ggerganov/llama.cpp/pull/5118
286 |
287 | - GGUF 형식으로 변환하는 방법
288 |
289 | ```shell
290 | python convert-hf-to-gguf.py path/to/Orion-14B-Chat --outfile chat.gguf
291 | ```
292 |
293 | - 모델 추론 방법
294 |
295 | ```shell
296 | ./main --frequency-penalty 0.5 --frequency-penalty 0.5 --top-k 5 --top-p 0.9 -m chat.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 400 -e
297 | ```
298 |
299 |
300 | ## 4.6. 예시 노출
301 |
302 | ### 4.6.1. 잡담
303 |
304 | `````
305 | 사용자：안녕,이름이 뭐예요
306 | Orion-14B：안녕! 저는 인공지능 조수 입니다. 정해진 이름은 없고 마음대로 불러도 됩니다. 도와줄 수 있는 거 있나요?
307 | 사용자：이야기 하나 나눠줘
308 | Orion-14B：예전에 잭이라는 어린 소년이 있었다. 그는 작은 마을에 살았다. 잭은 모험과 탐구를 매우 좋아해서 항상 신기한 것을 발견하는 꿈을 꾼다.
309 |
310 | 어느날 잭은 근처의 숲 깊은 곳에 전설속의 마법성이 있어 그 안에 사악한 거인이 산다는 소식을 든다. 그 거인의 이름은 그림이었다. 그는 성 안에 갇힌 채 떠날 수 없었다.
311 |
312 | 잭은 그림이 그곳에 갇혀있는 이유가 꼭 있을 거라고 믿기 때문에 거인을 구하러 가기로 결심한다. 그리하여 그는 용기와 결심을 가져 마법의 성으로 가는 여정을 시작한다.
313 |
314 | 오랜 고생 끝에 잭은 마침내 성보 대문 앞에 다다랐다. 그는 조심스럽게 대문을 열고 신비한 기운이 가득한 방으로 들어갔다.
315 |
316 | 방안에 어두운 기운이 가득했지만 잭은 놀라지 않았다. 그는 용감하게 성 안으로 들어가 마침내 그림을 찾아냈다.
317 |
318 | 그림은 자신은 사악한 마녀의 저주를 받아서 저주를 풀어야만 자유로워질 수 있다고 잭에게 말한다. 잭은 그림을 돕기로 결정하여 단서와 주문을 풀 방법을 찾기 시작한다.
319 |
320 | 한바탕의 노력 끝에 잭은 드디어 마녀의 비밀 실험실로 가는 길을 찾아낸다. 그는 그곳에서 마녀가 남긴 마법서를 발견했는데 그 책에서 주문을 푸는 방법이 적혀 있었다.
321 |
322 | 잭은 책에 쓰인 절차를 따라하여 그림의 저주를 성공적으로 풀어냈다. 거인은 너무 감사해서 잭의 소원이 이루어지도록 돕겠다고 약속했다.
323 |
324 | 이때로부터 잭과 그림은 좋은 친구가 되어 많은 위험하고 짜릿한 모험을 함께 겪었으며 그들의 용기와 지혜로 각종 어려움을 이겨냈다.
325 |
326 | 이 이야기는 저희에게 용기와 결심이 있다면 모든 어려움을 극복하고 자신의 꿈을 이룰 수 있다는 것을 알려준다.
327 | `````
328 |
329 | ### 4.6.2. 한일
330 |
331 | `````
332 | 用户：自己を紹介してください
333 | Orion-14B：こんにちは、私の名前はChatMaxで、OrionStarによって開発されたAIアシスタントです。私はユーザーに正確で、客観的で、有用な回答を提供し、彼らが問題を解決し、情報を取得するのを助けることができます。
334 | 用户：자기소개를 해주세요.
335 | Orion-14B：안녕하세요, 제 명칭은 ChatMax입니다. OrionStar에서 개발한 AI 어시스턴트입니다. 저는 사용자들에게 정확하고 객관적이며 유용한 답변을 제공하여 문제를 해결하고 정보를 얻는 데 도움을 줄 수 있습니다.
336 | `````
337 |
338 |
339 |
340 | # 5. 성명, 협의
341 |
342 | ## 5.1. 성명
343 |
344 | 저희는 모든 사용자들에게 Orion-14B모델을 이용하여 국가 사회 안전에 해치거나 불법적인 행위를 하는 거 하지 않도록 강력히 호소한다. 또한, 저희는 사용자들에게 Orion-14B 모델을 적절한 보안 검토를 하지 않거나 문서화되지 않은 인터넷 서비스로 이용하지 말라는 것을 요청한다.
345 |
346 | 저희는 모든 사용자가 이 원칙을 지키며 기술의 발전이 규범적이고 합법적인 환경에서 이루어질 수 있기를 바란다.
347 | 저희는 이미 최선을 다해 모델 훈련 과정에서 사용된 데이터의 준칙성을 확보하도록 하였다. 그러나 막대한 노력을 기울였음에도 불구하고 모델과 데이터의 복잡성으로 말미암아 일부 예견할 수 없을 문제들이 여전히 존재할 수 있다. 따라서 Orion-14B 오픈소스 모델의 사용으로 야기된 문제, 데이터 보안 문제와 공론 위험이나 모델의 오도, 남용, 전파, 또한 불적당한 사용 등으로 가져온 위험과 문제에 대해 저희는 책임을 지지 않겠다.
348 |
349 | ## 5.2. 협의
350 |
351 | 커뮤니티 사용Orion-14B시리즈 모델
352 | - 코드는 [Apache License Version 2.0](./LICENSE) 따르세요
353 | - 모델은 [Orion-14B시리즈 모델 커뮤니티 허가 협의](./ModelsCommunityLicenseAgreement)따르세요
354 |
355 |
356 |
357 | # 6. 회사소개
358 |
359 | 오리온 스타（OrionStar）는 2016년 9월 설립된 세계 최고의 서비스 로봇 솔루션 회사이다. 오리온 스타는 인공지능 기술을 바탕으로 차세대 혁명적 로봇 만들어 사람들이 반복되는 육체노동에서 벗어나 일과 생활을 더욱 지능적이고 재미있게 만들고 기술을 통해 사회와 세계를 더욱 아름답게 만든 것에 힘을 기울인다.
360 |
361 | 오리온 스타는 음성 인터렉션과 시각 네비게이션 등 완전히 독자적으로 개발한 풀 체인 인공지능 기술을 가지고 있다. 저희는 프로덕트 개발 능력과 기술 응용 능력을 통합하였다. 오리온 로봇 팔 플랫폼을 기반으로 ORIONSTAR AI Robot Greeting, AI Robot Greeting Mini, Lucki, CoffeeMaster 등의 프로덕트 출시하였으며 오리온 로봇의 오픈 플랫폼인 OrionOS를 설립하였다. **진짜 유용한 로봇을 위해 태어나라**의 이념을 위한 실천하여 AI기술을 통해 더 많은 사람들에게 능력을 부여한다.
362 |
363 | 7년의 AI경험 누적을 바탕으로 오리온 스타는 대형 모델 심층 응용"쥐언(Chatmax)"을 출시했고 업계 고객에게 맞춤형 AI대형 모델 컨설팅과 서비스 솔루션을 지속적으로 제공하여 진정으로 기업 경영 효율이 동종 업계에 앞서는 목표를 달성할 수 있도록 고객들에게 돕고 있다.
364 |
365 | **오리온 스타는 풀 체인 대형 모델 응용능력이란 핵심적 우세를 갖고 있다**, 대량 데이터 처리, 대형 모델 사전 훈련, 2차 사전 훈련, 미세 조정(Fine-tune), PromptEngineering, Agent등에서 개발된 풀 체인 능력과 경험 누적을 가지는 거 포함한다. 체계화된 데이터 처리 절차와 수백 개의 GPU의 병렬 모델 훈련 능력을 포함한 완정한 엔드투엔드 모델 훈련 능력을 가지고 있으며 현재 대형 정무, 클라우드 서비스, 출해 전자상거래, 쾌속소비품 등 여러 업계에서 구현되었다.
366 |
367 | ***대형 모델 응용 구현 필요가 있으신 회사께서 저희와 연락하는 것을 환영한다***
368 | **문의 전화:** 400-898-7779
369 | **이메일:** ai@orionstar.com
370 | **Discord 커뮤니티 링크:** https://discord.gg/zumjDWgdAs
371 |
372 |
5 | 
](#vllm) [
](#llama-cpp)
35 | - [📜 Declarations & License](#declarations-license)
36 | - [🥇 Company Introduction](#company-introduction)
37 |
38 | 
53 | 
56 | 
377 | 
55 | 
58 | 
148 | 
149 | 