├── .gitignore ├── LICENSE ├── README.md ├── Thutext.xlsx ├── bert_serving ├── __init__.py └── client │ └── __init__.py ├── cnews.vocab.txt ├── data ├── .cnews_.swp ├── __init__.py ├── cnews.train_easy.txt ├── cnews_loader.py └── cnews_loader_bert.py ├── helper ├── __init__.py ├── cnews_group.py └── copy_data.sh ├── images ├── acc_loss.png ├── acc_loss_rnn.png ├── cnn_architecture.png └── rnn_architecture.png ├── k_means.py ├── predict.py ├── requirements.txt ├── rnn_model.py ├── rnn_model_bert.py ├── run_rnn.py ├── run_rnn_bert.py ├── test.txt └── train_x.txt /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | data/cnews 3 | data/thucnews 4 | __pycache__ 5 | cnn_keras.py 6 | .idea 7 | *.pyc 8 | checkpoints 9 | tensorboard 10 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 dzkang 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 参考此人的github并且增加了Bert（https://github.com/gaussic/text-classification-cnn-rnn） 2 | Bert模型需要大家自行下载 3 | # Text Classification with RNN 4 | 有bert后缀的都是在原来的基础上，进行改造，得到的效果能到97.5 5 | 无bert后缀的加入了Attention以及多层或者双向模型 6 | 7 | 使用循环神经网络进行中文文本分类 8 | 9 | 10 | ## 环境 11 | 12 | - Python 2/3 13 | - TensorFlow 1.3以上 14 | - numpy 15 | - scikit-learn 16 | - scipy 17 | 18 | ## 数据集 19 | 20 | 使用THUCNews的一个子集进行训练与测试，数据集请自行到[THUCTC：一个高效的中文文本分类工具包](http://thuctc.thunlp.org/)下载，请遵循数据提供方的开源协议。 21 | 22 | 本次训练使用了其中的10个分类，每个分类5000条数据。相关数据下载地址：链接: https://pan.baidu.com/s/1GmBFZfDKsXBMFEYQWFdrfQ 提取码: 9ixg 23 | 24 | 类别如下： 25 | 26 | ``` 27 | 体育, 财经, 房产, 家居, 教育, 科技, 时尚, 时政, 游戏, 娱乐 28 | ``` 29 | 30 | 这个子集可以在此下载：链接: https://pan.baidu.com/s/1hugrfRu 密码: qfud 31 | 32 | 数据集划分如下： 33 | 34 | - 训练集: 5000*10 35 | - 验证集: 500*10 36 | - 测试集: 1000*10 37 | 38 | 从原数据集生成子集的过程请参看`helper`下的两个脚本。其中，`copy_data.sh`用于从每个分类拷贝6500个文件，`cnews_group.py`用于将多个文件整合到一个文件中。执行该文件后，得到三个数据文件： 39 | 40 | - cnews.train.txt: 训练集(50000条) 41 | - cnews.val.txt: 验证集(5000条) 42 | - cnews.test.txt: 测试集(10000条) 43 | 44 | ## 预处理 45 | 46 | `data/cnews_loader.py`为数据的预处理文件。 47 | 48 | - `read_file()`: 读取文件数据; 49 | - `build_vocab()`: 构建词汇表，使用字符级的表示，这一函数会将词汇表存储下来，避免每一次重复处理; 50 | - `read_vocab()`: 读取上一步存储的词汇表，转换为`{词：id}`表示; 51 | - `read_category()`: 将分类目录固定，转换为`{类别: id}`表示; 52 | - `to_words()`: 将一条由id表示的数据重新转换为文字; 53 | - `process_file()`: 将数据集从文字转换为固定长度的id序列表示; 54 | - `batch_iter()`: 为神经网络的训练准备经过shuffle的批次的数据。 55 | 56 | 经过数据预处理，数据的格式如下： 57 | 58 | | Data | Shape | Data | Shape | 59 | | :---------- | :---------- | :---------- | :---------- | 60 | | x_train | [50000, 600] | y_train | [50000, 10] | 61 | | x_val | [5000, 600] | y_val | [5000, 10] | 62 | | x_test | [10000, 600] | y_test | [10000, 10] | 63 | 64 | ## RNN循环神经网络 65 | 66 | ### 配置项 67 | 68 | RNN可配置的参数如下所示，在`rnn_model.py`中。 69 | 70 | ```python 71 | class TRNNConfig(object): 72 | """RNN配置参数""" 73 | 74 | # 模型参数 75 | embedding_dim = 768 # 词向量维度 76 | seq_length = 512 # 序列长度 77 | num_classes = 10 # 类别数 78 | vocab_size = 5000 # 词汇表达小 79 | 80 | num_layers = 1 # 隐藏层层数 81 | hidden_dim = 512 # 隐藏层神经元 82 | rnn = 'gru' # lstm 或 gru 83 | 84 | attention_dim = 512 85 | l2_reg_lambda = 0.01 86 | 87 | dropout_keep_prob = 0.5 # dropout保留比例 88 | learning_rate = 1e-3 # 学习率 89 | 90 | batch_size = 128 # 每批训练大小 91 | num_epochs = 10 # 总迭代轮次 92 | 93 | print_per_batch = 100 # 每多少轮输出一次结果 94 | save_per_batch = 10 # 每多少轮存入tensorboard 95 | ``` 96 | 97 | ### RNN模型 98 | 99 | 具体参看`rnn_model_bert.py`的实现。 100 | 101 | 大致结构如下： 102 | 103 | ![images/rnn_architecture](images/rnn_architecture.png) 104 | 105 | -------------------------------------------------------------------------------- /Thutext.xlsx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pengwei-iie/Bert-THUCNews/a20749225091533b530f0e539bfaacbd3524fe99/Thutext.xlsx -------------------------------------------------------------------------------- /bert_serving/__init__.py: -------------------------------------------------------------------------------- 1 | __path__ = __import__('pkgutil').extend_path(__path__, __name__) 2 | -------------------------------------------------------------------------------- /bert_serving/client/__init__.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | # Han Xiao 4 | 5 | import sys 6 | import threading 7 | import time 8 | import uuid 9 | from collections import namedtuple 10 | 11 | import numpy as np 12 | import zmq 13 | from zmq.utils import jsonapi 14 | 15 | __all__ = ['__version__', 'BertClient'] 16 | 17 | # in the future client version must match with server version 18 | __version__ = '1.6.2' 19 | 20 | if sys.version_info >= (3, 0): 21 | _py2 = False 22 | _str = str 23 | _buffer = memoryview 24 | _unicode = lambda x: x 25 | else: 26 | # make it compatible for py2 27 | _py2 = True 28 | _str = basestring 29 | _buffer = buffer 30 | _unicode = lambda x: [BertClient._force_to_unicode(y) for y in x] 31 | 32 | Response = namedtuple('Response', ['id', 'content']) 33 | 34 | 35 | class BertClient: 36 | def __init__(self, ip='localhost', port=5555, port_out=5556, 37 | output_fmt='ndarray', show_server_config=False, 38 | identity=None, check_version=True, check_length=True, 39 | timeout=5000): 40 | """ A client object connected to a BertServer 41 | 42 | Create a BertClient that connects to a BertServer. 43 | Note, server must be ready at the moment you are calling this function. 44 | If you are not sure whether the server is ready, then please set `check_version=False` 45 | 46 | You can also use it as a context manager: 47 | 48 | .. highlight:: python 49 | .. code-block:: python 50 | 51 | with BertClient() as bc: 52 | bc.encode(...) 53 | 54 | # bc is automatically closed out of the context 55 | 56 | :type timeout: int 57 | :type check_version: bool 58 | :type check_length: bool 59 | :type identity: str 60 | :type show_server_config: bool 61 | :type output_fmt: str 62 | :type port_out: int 63 | :type port: int 64 | :type ip: str 65 | :param ip: the ip address of the server 66 | :param port: port for pushing data from client to server, must be consistent with the server side config 67 | :param port_out: port for publishing results from server to client, must be consistent with the server side config 68 | :param output_fmt: the output format of the sentence encodes, either in numpy array or python List[List[float]] (ndarray/list) 69 | :param show_server_config: whether to show server configs when first connected 70 | :param identity: the UUID of this client 71 | :param check_version: check if server has the same version as client, raise AttributeError if not the same 72 | :param check_length: check if server `max_seq_len` is less than the sentence length before sent 73 | :param timeout: set the timeout (milliseconds) for receive operation on the client 74 | """ 75 | 76 | self.context = zmq.Context() 77 | self.sender = self.context.socket(zmq.PUSH) 78 | self.identity = identity or str(uuid.uuid4()).encode('ascii') 79 | self.sender.connect('tcp://%s:%d' % (ip, port)) 80 | 81 | self.receiver = self.context.socket(zmq.SUB) 82 | self.receiver.setsockopt(zmq.SUBSCRIBE, self.identity) 83 | self.receiver.connect('tcp://%s:%d' % (ip, port_out)) 84 | 85 | self.request_id = 0 86 | self.timeout = timeout 87 | self.pending_request = set() 88 | 89 | if output_fmt == 'ndarray': 90 | self.formatter = lambda x: x 91 | elif output_fmt == 'list': 92 | self.formatter = lambda x: x.tolist() 93 | else: 94 | raise AttributeError('"output_fmt" must be "ndarray" or "list"') 95 | 96 | self.output_fmt = output_fmt 97 | self.port = port 98 | self.port_out = port_out 99 | self.ip = ip 100 | self.length_limit = 0 101 | 102 | if check_version or show_server_config or check_length: 103 | s_status = self.server_status 104 | 105 | if check_version and s_status['server_version'] != self.status['client_version']: 106 | raise AttributeError('version mismatch! server version is %s but client version is %s!\n' 107 | 'consider "pip install -U bert-serving-server bert-serving-client"\n' 108 | 'or disable version-check by "BertClient(check_version=False)"' % ( 109 | s_status['server_version'], self.status['client_version'])) 110 | 111 | if show_server_config: 112 | self._print_dict(s_status, 'server config:') 113 | 114 | if check_length: 115 | self.length_limit = int(s_status['max_seq_len']) 116 | 117 | def close(self): 118 | """ 119 | Gently close all connections of the client. If you are using BertClient as context manager, 120 | then this is not necessary. 121 | 122 | """ 123 | self.sender.close() 124 | self.receiver.close() 125 | self.context.term() 126 | 127 | def _send(self, msg, msg_len=0): 128 | self.sender.send_multipart([self.identity, msg, b'%d' % self.request_id, b'%d' % msg_len]) 129 | self.pending_request.add(self.request_id) 130 | self.request_id += 1 131 | 132 | def _recv(self): 133 | response = self.receiver.recv_multipart() 134 | request_id = int(response[-1]) 135 | self.pending_request.remove(request_id) 136 | return Response(request_id, response) 137 | 138 | def _recv_ndarray(self): 139 | request_id, response = self._recv() 140 | arr_info, arr_val = jsonapi.loads(response[1]), response[2] 141 | X = np.frombuffer(_buffer(arr_val), dtype=str(arr_info['dtype'])) 142 | return Response(request_id, self.formatter(X.reshape(arr_info['shape']))) 143 | 144 | @property 145 | def status(self): 146 | """ 147 | Get the status of this BertClient instance 148 | 149 | :rtype: dict[str, str] 150 | :return: a dictionary contains the status of this BertClient instance 151 | 152 | """ 153 | return { 154 | 'identity': self.identity, 155 | 'num_request': self.request_id, 156 | 'num_pending_request': len(self.pending_request), 157 | 'pending_request': self.pending_request, 158 | 'output_fmt': self.output_fmt, 159 | 'port': self.port, 160 | 'port_out': self.port_out, 161 | 'server_ip': self.ip, 162 | 'client_version': __version__, 163 | 'timeout': self.timeout 164 | } 165 | 166 | @property 167 | def server_status(self): 168 | """ 169 | Get the current status of the server connected to this client 170 | 171 | :return: a dictionary contains the current status of the server connected to this client 172 | :rtype: dict[str, str] 173 | 174 | """ 175 | try: 176 | self.receiver.setsockopt(zmq.RCVTIMEO, self.timeout) 177 | self._send(b'SHOW_CONFIG') 178 | return jsonapi.loads(self._recv().content[1]) 179 | except zmq.error.Again as _e: 180 | t_e = TimeoutError( 181 | 'no response from the server (with "timeout"=%d ms), ' 182 | 'is the server on-line? is network broken? are "port" and "port_out" correct?' % self.timeout) 183 | if _py2: 184 | raise t_e 185 | else: 186 | raise t_e from _e 187 | finally: 188 | self.receiver.setsockopt(zmq.RCVTIMEO, -1) 189 | 190 | def encode(self, texts, blocking=True, is_tokenized=False): 191 | """ Encode a list of strings to a list of vectors 192 | 193 | `texts` should be a list of strings, each of which represents a sentence. 194 | If `is_tokenized` is set to True, then `texts` should be list[list[str]], 195 | outer list represents sentence and inner list represent tokens in the sentence. 196 | Note that if `blocking` is set to False, then you need to fetch the result manually afterwards. 197 | 198 | .. highlight:: python 199 | .. code-block:: python 200 | 201 | with BertClient() as bc: 202 | # encode untokenized sentences 203 | bc.encode(['First do it', 204 | 'then do it right', 205 | 'then do it better']) 206 | 207 | # encode tokenized sentences 208 | bc.encode([['First', 'do', 'it'], 209 | ['then', 'do', 'it', 'right'], 210 | ['then', 'do', 'it', 'better']], is_tokenized=True) 211 | 212 | :type is_tokenized: bool 213 | :type blocking: bool 214 | :type texts: list[str] or list[list[str]] 215 | :param is_tokenized: whether the input texts is already tokenized 216 | :param texts: list of sentence to be encoded. Larger list for better efficiency. 217 | :param blocking: wait until the encoded result is returned from the server. If false, will immediately return. 218 | :return: encoded sentence/token-level embeddings, rows correspond to sentences 219 | :rtype: numpy.ndarray or list[list[float]] 220 | 221 | """ 222 | if is_tokenized: 223 | self._check_input_lst_lst_str(texts) 224 | else: 225 | self._check_input_lst_str(texts) 226 | 227 | if self.length_limit and not self._check_length(texts, self.length_limit, is_tokenized): 228 | print('some of your sentences have more tokens than "max_seq_len=%d" set on the server, ' 229 | 'as consequence you may get less-accurate or truncated embeddings.\n' 230 | 'here is what you can do:\n' 231 | '- disable the length-check by create a new "BertClient(check_length=False)" ' 232 | 'when you just want to ignore this warning\n' 233 | '- or, start a new server with a larger "max_seq_len"' % self.length_limit) 234 | 235 | texts = _unicode(texts) 236 | self._send(jsonapi.dumps(texts), len(texts)) 237 | return self._recv_ndarray().content if blocking else None 238 | 239 | def fetch(self, delay=.0): 240 | """ Fetch the encoded vectors from server, use it with `encode(blocking=False)` 241 | 242 | Use it after `encode(texts, blocking=False)`. If there is no pending requests, will return None. 243 | Note that `fetch()` does not preserve the order of the requests! Say you have two non-blocking requests, 244 | R1 and R2, where R1 with 256 samples, R2 with 1 samples. It could be that R2 returns first. 245 | 246 | To fetch all results in the original sending order, please use `fetch_all(sort=True)` 247 | 248 | :type delay: float 249 | :param delay: delay in seconds and then run fetcher 250 | :return: a generator that yields request id and encoded vector in a tuple, where the request id can be used to determine the order 251 | :rtype: Iterator[tuple(int, numpy.ndarray)] 252 | 253 | """ 254 | time.sleep(delay) 255 | while self.pending_request: 256 | yield self._recv_ndarray() 257 | 258 | def fetch_all(self, sort=True, concat=False): 259 | """ Fetch all encoded vectors from server, use it with `encode(blocking=False)` 260 | 261 | Use it `encode(texts, blocking=False)`. If there is no pending requests, it will return None. 262 | 263 | :type sort: bool 264 | :type concat: bool 265 | :param sort: sort results by their request ids. It should be True if you want to preserve the sending order 266 | :param concat: concatenate all results into one ndarray 267 | :return: encoded sentence/token-level embeddings in sending order 268 | :rtype: numpy.ndarray or list[list[float]] 269 | 270 | """ 271 | if self.pending_request: 272 | tmp = list(self.fetch()) 273 | if sort: 274 | tmp = sorted(tmp, key=lambda v: v.id) 275 | tmp = [v.content for v in tmp] 276 | if concat: 277 | if self.output_fmt == 'ndarray': 278 | tmp = np.concatenate(tmp, axis=0) 279 | elif self.output_fmt == 'list': 280 | tmp = [vv for v in tmp for vv in v] 281 | return tmp 282 | 283 | def encode_async(self, batch_generator, max_num_batch=None, delay=0.1, is_tokenized=False): 284 | """ Async encode batches from a generator 285 | 286 | :param is_tokenized: whether batch_generator generates tokenized sentences 287 | :param delay: delay in seconds and then run fetcher 288 | :param batch_generator: a generator that yields list[str] or list[list[str]] (for `is_tokenized=True`) every time 289 | :param max_num_batch: stop after encoding this number of batches 290 | :return: a generator that yields encoded vectors in ndarray, where the request id can be used to determine the order 291 | :rtype: Iterator[tuple(int, numpy.ndarray)] 292 | 293 | """ 294 | 295 | def run(): 296 | cnt = 0 297 | for texts in batch_generator: 298 | self.encode(texts, blocking=False, is_tokenized=is_tokenized) 299 | cnt += 1 300 | if max_num_batch and cnt == max_num_batch: 301 | break 302 | 303 | t = threading.Thread(target=run) 304 | t.start() 305 | return self.fetch(delay) 306 | 307 | @staticmethod 308 | def _check_length(texts, len_limit, tokenized): 309 | if tokenized: 310 | # texts is already tokenized as list of str 311 | return all(len(t) <= len_limit for t in texts) 312 | else: 313 | # do a simple whitespace tokenizer 314 | return all(len(t.split()) <= len_limit for t in texts) 315 | 316 | @staticmethod 317 | def _check_input_lst_str(texts): 318 | if not isinstance(texts, list): 319 | raise TypeError('"%s" must be %s, but received %s' % (texts, type([]), type(texts))) 320 | if not len(texts): 321 | raise ValueError( 322 | '"%s" must be a non-empty list, but received %s with %d elements' % (texts, type(texts), len(texts))) 323 | for idx, s in enumerate(texts): 324 | if not isinstance(s, _str): 325 | raise TypeError('all elements in the list must be %s, but element %d is %s' % (type(''), idx, type(s))) 326 | if not s.strip(): 327 | raise ValueError( 328 | 'all elements in the list must be non-empty string, but element %d is %s' % (idx, repr(s))) 329 | 330 | @staticmethod 331 | def _check_input_lst_lst_str(texts): 332 | if not isinstance(texts, list): 333 | raise TypeError('"texts" must be %s, but received %s' % (type([]), type(texts))) 334 | if not len(texts): 335 | raise ValueError( 336 | '"texts" must be a non-empty list, but received %s with %d elements' % (type(texts), len(texts))) 337 | for s in texts: 338 | BertClient._check_input_lst_str(s) 339 | 340 | @staticmethod 341 | def _force_to_unicode(text): 342 | return text if isinstance(text, unicode) else text.decode('utf-8') 343 | 344 | @staticmethod 345 | def _print_dict(x, title=None): 346 | if title: 347 | print(title) 348 | for k, v in x.items(): 349 | print('%30s\t=\t%-30s' % (k, v)) 350 | 351 | def __enter__(self): 352 | return self 353 | 354 | def __exit__(self, exc_type, exc_val, exc_tb): 355 | self.close() 356 | -------------------------------------------------------------------------------- /cnews.vocab.txt: -------------------------------------------------------------------------------- 1 | 2 | ， 3 | 的 4 | 。 5 | 一 6 | 是 7 | 在 8 | 0 9 | 有 10 | 不 11 | 了 12 | 中 13 | 1 14 | 人 15 | 大 16 | 、 17 | 国 18 | 19 | 2 20 | 这 21 | 上 22 | 为 23 | 个 24 | “ 25 | ” 26 | 年 27 | 学 28 | 时 29 | 我 30 | 地 31 | 和 32 | 以 33 | 到 34 | 出 35 | 来 36 | 会 37 | 行 38 | 发 39 | ： 40 | 对 41 | 们 42 | 要 43 | 生 44 | 家 45 | 他 46 | 能 47 | 也 48 | 业 49 | 金 50 | 3 51 | 成 52 | 可 53 | 分 54 | 多 55 | 现 56 | 5 57 | 就 58 | 场 59 | 新 60 | 后 61 | 于 62 | 下 63 | 日 64 | 经 65 | 市 66 | 前 67 | 过 68 | 方 69 | 得 70 | 作 71 | 月 72 | 最 73 | 开 74 | 房 75 | 》 76 | 《 77 | 高 78 | 9 79 | 8 80 | . 81 | 而 82 | 比 83 | 公 84 | 4 85 | 说 86 | ) 87 | 将 88 | ( 89 | 都 90 | 资 91 | e 92 | 6 93 | 基 94 | 用 95 | 面 96 | 产 97 | 还 98 | 自 99 | 者 100 | 本 101 | 之 102 | 美 103 | 很 104 | 同 105 |   106 | 7 107 | 部 108 | 进 109 | 但 110 | 主 111 | 外 112 | 动 113 | 机 114 | 元 115 | 理 116 | 加 117 | a 118 | 全 119 | 与 120 | 实 121 | 影 122 | 好 123 | 小 124 | 间 125 | 其 126 | 天 127 | 定 128 | 表 129 | 力 130 | 如 131 | 次 132 | 合 133 | 长 134 | o 135 | 体 136 | 价 137 | i 138 | 所 139 | 内 140 | 子 141 | 目 142 | 电 143 | - 144 | 当 145 | 度 146 | 品 147 | 看 148 | 期 149 | 关 150 | 更 151 | n 152 | 等 153 | 工 154 | 然 155 | 斯 156 | 重 157 | 些 158 | 球 159 | 此 160 | 里 161 | 利 162 | 相 163 | 情 164 | 投 165 | 点 166 | 没 167 | 因 168 | 已 169 | 三 170 | 心 171 | 特 172 | 明 173 | 位 174 | 两 175 | 法 176 | 从 177 | 入 178 | 名 179 | 万 180 | 手 181 | 计 182 | 性 183 | 事 184 | 只 185 | 样 186 | 示 187 | r 188 | 种 189 | 报 190 | 海 191 | 平 192 | 数 193 | % 194 | 第 195 | 并 196 | 色 197 | 建 198 | 据 199 | 提 200 | 商 201 | 员 202 | 通 203 | 去 204 | 民 205 | t 206 | 着 207 | 你 208 | 片 209 | 展 210 | 道 211 | 文 212 | 演 213 | 赛 214 | 区 215 | 交 216 | 意 217 | 政 218 | 么 219 | 今 220 | 让 221 | 起 222 | 信 223 | 化 224 | 银 225 | 记 226 | 司 227 | 北 228 | 游 229 | 科 230 | 戏 231 | 被 232 | 格 233 | 保 234 | 及 235 | 常 236 | 物 237 | 问 238 | s 239 | 量 240 | 制 241 | 持 242 | 果 243 | 感 244 | 设 245 | 队 246 | 无 247 | 收 248 | 正 249 | 应 250 | ？ 251 | 那 252 | 活 253 | 身 254 | 式 255 | 打 256 | l 257 | 系 258 | 尔 259 | 总 260 | 京 261 | A 262 | 至 263 | 己 264 | 务 265 | 受 266 | 想 267 | 星 268 | 回 269 | 留 270 | 由 271 | 网 272 | 达 273 | 认 274 | 做 275 | 题 276 | 选 277 | 费 278 | 城 279 | 增 280 | 近 281 | 华 282 | 风 283 | 非 284 | 战 285 | 布 286 | 该 287 | 接 288 | 款 289 | 项 290 | S 291 | 英 292 | 女 293 | 程 294 | 导 295 | 管 296 | 二 297 | 强 298 | 证 299 | 水 300 | 代 301 | 调 302 | 少 303 | 专 304 | 型 305 | 台 306 | 给 307 | 需 308 | 规 309 | 图 310 | 周 311 | 德 312 | 解 313 | 各 314 | C 315 | 向 316 | ； 317 | 别 318 | 股 319 | 东 320 | 结 321 | 或 322 | 首 323 | 士 324 | 西 325 | 安 326 | 教 327 | 变 328 | 火 329 | 节 330 | 际 331 | 任 332 | 单 333 | 先 334 | 再 335 | 观 336 | 校 337 | 显 338 | 克 339 | 组 340 | 直 341 | 装 342 | 求 343 | 才 344 | 率 345 | 较 346 | 造 347 | 使 348 | 每 349 | 考 350 | 亿 351 | 请 352 | 流 353 | 完 354 | 拍 355 | 取 356 | 院 357 | 线 358 | 门 359 | 放 360 | 世 361 | 住 362 | 续 363 | 联 364 | 张 365 | P 366 | 息 367 | 空 368 | c 369 | 研 370 | 它 371 | 十 372 | 立 373 | 原 374 | 易 375 | 整 376 | — 377 | 术 378 | 论 379 | 消 380 | 太 381 | 知 382 | 王 383 | 质 384 | 约 385 | 客 386 | 几 387 | 配 388 | 马 389 | 统 390 | 值 391 | T 392 | 头 393 | 件 394 | 带 395 | 服 396 | 她 397 | 界 398 | 指 399 | D 400 | 级 401 | 企 402 | 传 403 | 老 404 | 类 405 | 始 406 | 气 407 | I 408 | 户 409 | 超 410 | 育 411 | 未 412 | 具 413 | m 414 | 济 415 | 低 416 | 处 417 | 技 418 | 望 419 | 把 420 | M 421 | 究 422 | 什 423 | 热 424 | 推 425 | 称 426 | 购 427 | ！ 428 | 季 429 | 米 430 | 光 431 | 广 432 | 份 433 | 乐 434 | 又 435 | 获 436 | 真 437 | 觉 438 | E 439 | 玩 440 | 形 441 | 集 442 | 备 443 | 优 444 | 领 445 | 势 446 | h 447 | 楼 448 | 准 449 | 包 450 | 像 451 | 则 452 | 难 453 | 爱 454 | 四 455 | 申 456 | B 457 | d 458 | 售 459 | u 460 | 快 461 | 连 462 | 话 463 | 师 464 | 告 465 | 何 466 | 视 467 | 确 468 | 深 469 | 预 470 | 局 471 | 亚 472 | 评 473 | 票 474 | 卡 475 | 改 476 | 候 477 | 买 478 | 况 479 | 存 480 | 支 481 | 拉 482 | 仅 483 | 办 484 | 注 485 | 反 486 | 环 487 | 供 488 | 角 489 | 篮 490 | 标 491 | 转 492 | 拿 493 | 运 494 | 参 495 | 升 496 | 众 497 | O 498 | / 499 | 创 500 | 浪 501 | 陈 502 | 精 503 | 号 504 | 见 505 | 助 506 | 且 507 | 土 508 | 半 509 | 议 510 | 花 511 | 步 512 | 策 513 | 南 514 | 板 515 | 居 516 | 友 517 | 越 518 | 功 519 | 构 520 | 贷 521 | 划 522 | 模 523 | 山 524 | 融 525 | 社 526 | G 527 | 决 528 | 共 529 | 案 530 | g 531 | 销 532 | 走 533 | 套 534 | 路 535 | 即 536 | 言 537 | 容 538 | 均 539 | 试 540 | 积 541 | 团 542 | 府 543 | 查 544 | 条 545 | 限 546 | 引 547 | 随 548 | 择 549 | 希 550 | 口 551 | 奇 552 | 涨 553 | 港 554 | 盘 555 | 象 556 | 置 557 | 林 558 | 益 559 | 效 560 | 神 561 | 极 562 | 语 563 | 复 564 | 够 565 | 阿 566 | 币 567 | 根 568 | 州 569 | 历 570 | 采 571 | 奖 572 | 终 573 | 适 574 | N 575 | 尼 576 | 命 577 | 段 578 | 景 579 | 险 580 | 却 581 | 失 582 | 军 583 | 权 584 | 车 585 | L 586 | 剧 587 | 速 588 | 攻 589 | 许 590 | 足 591 | 算 592 | 清 593 | 争 594 | 响 595 | 移 596 | 红 597 | 必 598 | 幅 599 | 额 600 | y 601 | 讯 602 | 照 603 | 源 604 | 李 605 | 验 606 | 双 607 | 刚 608 | 另 609 | 态 610 | p 611 | 致 612 | 底 613 | 除 614 | 满 615 | 击 616 | 白 617 | 五 618 | 往 619 | 喜 620 | 财 621 | 防 622 | 境 623 | 排 624 | 百 625 | 降 626 | 远 627 | 书 628 | 习 629 | 龙 630 | 债 631 | 钱 632 | 断 633 | 富 634 | 曾 635 | 测 636 | 签 637 | 举 638 | 飞 639 | 便 640 | 介 641 | 博 642 | 牌 643 | 虽 644 | 料 645 | 例 646 | 边 647 | 罗 648 | 搭 649 | 香 650 | 委 651 | 营 652 | 器 653 | R 654 | 落 655 | 跟 656 | 列 657 | 仍 658 | 职 659 | 尚 660 | 阳 661 | 监 662 | 券 663 | 儿 664 | 尽 665 | 衣 666 | 负 667 | 短 668 | 找 669 | 层 670 | 黄 671 | 艺 672 | 域 673 | 施 674 | F 675 | 离 676 | 波 677 | 胜 678 | 兰 679 | 古 680 | 录 681 | 黑 682 | 素 683 | 普 684 | 史 685 | 版 686 | 摄 687 | 官 688 | 否 689 | 石 690 | · 691 | 站 692 | w 693 | 访 694 | , 695 | 初 696 | 细 697 | 继 698 | 责 699 | 括 700 | 卫 701 | 露 702 | 协 703 | 轻 704 | 维 705 | 澳 706 | 纪 707 | 欢 708 | 卖 709 | 早 710 | 绩 711 | 穿 712 | 孩 713 | 按 714 | 眼 715 | 围 716 | 男 717 | 依 718 | 宝 719 | 控 720 | b 721 | 读 722 | 状 723 | 洲 724 | 密 725 | 差 726 | 奥 727 | 破 728 | 块 729 | 突 730 | … 731 | 思 732 | 画 733 | 巴 734 | 纳 735 | 范 736 | 映 737 | 充 738 | 媒 739 | 室 740 | H 741 | 材 742 | 字 743 | 武 744 | 跌 745 | 春 746 | 湖 747 | 简 748 | 担 749 | 欧 750 | 谈 751 | 切 752 | 兴 753 | 刘 754 | 彩 755 | V 756 | 稳 757 | 待 758 | 箭 759 | 属 760 | 声 761 | 课 762 | 透 763 | 紧 764 | 温 765 | 绝 766 | 园 767 | 修 768 | 农 769 | 严 770 | 减 771 | 江 772 | 讲 773 | 航 774 | 似 775 | 独 776 | 背 777 | 夫 778 | 甚 779 | 志 780 | 牛 781 | 亮 782 | 故 783 | W 784 | f 785 | 汇 786 | 义 787 | 换 788 | 货 789 | 令 790 | 饰 791 | 招 792 | 医 793 | 占 794 | 音 795 | 群 796 | 竞 797 | 乎 798 | 护 799 | 姆 800 | 席 801 | 湾 802 | 略 803 | 央 804 | 怎 805 | 岁 806 | 亲 807 | 送 808 | 补 809 | 青 810 | 索 811 | 批 812 | 核 813 | 帮 814 | 福 815 | 伤 816 | v 817 | 昨 818 | 念 819 | 激 820 | 陆 821 | 危 822 | 巨 823 | 省 824 | 宣 825 | 般 826 | 筑 827 | k 828 | 识 829 | 守 830 | 压 831 | 错 832 | 免 833 | 店 834 | 宅 835 | 庭 836 | 承 837 | 雷 838 | 云 839 | 味 840 | 典 841 | 听 842 | 笑 843 | 登 844 | 假 845 | 审 846 | 绍 847 | 毕 848 | 诉 849 | 迷 850 | 威 851 | 谢 852 | 练 853 | 丽 854 | 止 855 | 微 856 | 频 857 | 净 858 | 森 859 | 拥 860 | 须 861 | 左 862 | 印 863 | 追 864 | 秀 865 | 托 866 | 吴 867 | 副 868 | 疑 869 | 千 870 | 治 871 | 冰 872 | 右 873 | 松 874 | 晚 875 | 皮 876 | K 877 | 佳 878 | 夏 879 | 呢 880 | 魔 881 | 缺 882 | 编 883 | 析 884 | 钟 885 | 刻 886 | 遇 887 | 停 888 | 写 889 | 付 890 | 食 891 | 织 892 | 闻 893 | 检 894 | 礼 895 | 惊 896 | 莱 897 | 余 898 | 死 899 | 雅 900 | 善 901 | 干 902 | 愿 903 | 爆 904 | 某 905 | 端 906 | 探 907 | 述 908 | 吧 909 | 执 910 | 村 911 | 租 912 | 健 913 | 阵 914 | 律 915 | 虑 916 | 临 917 | 冲 918 | 盟 919 | 六 920 | 吸 921 | 束 922 | 射 923 | 久 924 | 馆 925 | 暴 926 | 折 927 | 苏 928 | 顿 929 | 借 930 | 宇 931 | U 932 | 病 933 | 盛 934 | 藏 935 | 杰 936 | 灵 937 | 异 938 | 察 939 | 震 940 | 退 941 | 朋 942 | 旅 943 | 抢 944 | 韩 945 | 封 946 | 轮 947 | 康 948 | 弹 949 | 互 950 | 伦 951 | 败 952 | 午 953 | 河 954 | 杀 955 | 私 956 | 镜 957 | 扁 958 | 潮 959 | 幕 960 | 旗 961 | 党 962 | 休 963 | 八 964 | 悉 965 | 瑞 966 | 逐 967 | 征 968 | 坚 969 | 绿 970 | 班 971 | 迎 972 | 吃 973 | 母 974 | 族 975 | 税 976 | 良 977 | 丰 978 | 训 979 | 秘 980 | 娱 981 | 哈 982 | 警 983 | 伟 984 | 您 985 | 歌 986 | 烈 987 | 哪 988 | 挑 989 | 享 990 | 肯 991 | 凯 992 | 顾 993 | 杨 994 | 丹 995 | 雄 996 | 键 997 | 靠 998 | 针 999 | 木 1000 | 估 1001 | 届 1002 | ? 1003 | 嘉 1004 | 洛 1005 | 答 1006 | 七 1007 | 裙 1008 | 岛 1009 | 距 1010 | 刺 1011 | X 1012 | 趣 1013 | 养 1014 | 九 1015 | 座 1016 | 阶 1017 | 庆 1018 | 街 1019 | 释 1020 | : 1021 | 扩 1022 | 洋 1023 | 永 1024 | 顺 1025 | 障 1026 | 盖 1027 | 勒 1028 | 遭 1029 | 铁 1030 | 笔 1031 | 启 1032 | 判 1033 | 倍 1034 | 豪 1035 | 吗 1036 | 杂 1037 | 哥 1038 | 纷 1039 | 婚 1040 | 秒 1041 | 斗 1042 | 犯 1043 | 剑 1044 | 沙 1045 | 派 1046 | 归 1047 | 操 1048 | 扮 1049 | 损 1050 | 仓 1051 | 延 1052 | 暖 1053 | 贵 1054 | 档 1055 | 载 1056 | 丝 1057 | 末 1058 | 截 1059 | 促 1060 | 竟 1061 | 油 1062 | 困 1063 | 泰 1064 | 梦 1065 | 播 1066 | 革 1067 | 络 1068 | 熟 1069 | Q 1070 | 顶 1071 | 酒 1072 | 础 1073 | ） 1074 | 倒 1075 | 著 1076 | ’ 1077 | 麦 1078 | 章 1079 | 宁 1080 | 雪 1081 | 父 1082 | 授 1083 | 血 1084 | 裤 1085 | 静 1086 | 诺 1087 | 罚 1088 | 输 1089 | 润 1090 | 害 1091 | 船 1092 | + 1093 | 掉 1094 | 蓝 1095 | 伊 1096 | 脑 1097 | 避 1098 | 尤 1099 | 赵 1100 | 缩 1101 | 综 1102 | 挥 1103 | 叫 1104 | 储 1105 | 镇 1106 | 寻 1107 | 怪 1108 | 鲁 1109 | 冠 1110 | （ 1111 | 努 1112 | 丁 1113 | ‘ 1114 | 坛 1115 | 恩 1116 | 概 1117 | 扣 1118 | 旧 1119 | 趋 1120 | 培 1121 | 裁 1122 | 屋 1123 | J 1124 | 宽 1125 | 汉 1126 | 讨 1127 | 谁 1128 | 渐 1129 | 账 1130 | 涉 1131 | 岸 1132 | 胡 1133 | 毛 1134 | 锋 1135 | 迪 1136 | 幸 1137 | 偏 1138 | 秋 1139 | 菲 1140 | 码 1141 | 纯 1142 | 叶 1143 | 峰 1144 | 圣 1145 | 兵 1146 | 救 1147 | 暗 1148 | 鞋 1149 | 冬 1150 | 冷 1151 | 蒂 1152 | 侠 1153 | 董 1154 | 恐 1155 | 熊 1156 | 沃 1157 | 献 1158 | 惠 1159 | 圈 1160 | 朝 1161 | 询 1162 | 药 1163 | 智 1164 | 予 1165 | 既 1166 | 急 1167 | 吉 1168 | 曼 1169 | 赢 1170 | 姚 1171 | 替 1172 | 塔 1173 | 狂 1174 | 跳 1175 | 抗 1176 | 误 1177 | 触 1178 | 夜 1179 | 朗 1180 | 附 1181 | 坦 1182 | 慢 1183 | 励 1184 | 鱼 1185 | 含 1186 | 累 1187 | 若 1188 | 措 1189 | 怀 1190 | 贸 1191 | 雨 1192 | 草 1193 | 苦 1194 | 徐 1195 | 贴 1196 | 荣 1197 | 孙 1198 | 晓 1199 | 脚 1200 | 迅 1201 | 聚 1202 | 违 1203 | 厅 1204 | 募 1205 | 鲜 1206 | 潜 1207 | 圆 1208 | 餐 1209 | 坐 1210 | 仔 1211 | 勇 1212 | 软 1213 | 莫 1214 | 伯 1215 | 梅 1216 | 圳 1217 | 呈 1218 | 宾 1219 | 库 1220 | 凭 1221 | 掌 1222 | 俄 1223 | 焦 1224 | 漫 1225 | 灾 1226 | 曲 1227 | 佛 1228 | 怕 1229 | 谓 1230 | 劳 1231 | 贝 1232 | 廉 1233 | 符 1234 | 暂 1235 | 唯 1236 | 玉 1237 | 詹 1238 | 珍 1239 | 舒 1240 | 遗 1241 | x 1242 | 厚 1243 | 恒 1244 | 跑 1245 | 遍 1246 | 隆 1247 | 序 1248 | 词 1249 | 舞 1250 | 幻 1251 | 混 1252 | 粉 1253 | 灰 1254 | 田 1255 | 野 1256 | 赴 1257 | 宏 1258 | 邮 1259 | 硬 1260 | 宜 1261 | 帅 1262 | 架 1263 | 揭 1264 | 堂 1265 | 霸 1266 | 呼 1267 | 敌 1268 | 纽 1269 | 疗 1270 | 塞 1271 | 杜 1272 | 珠 1273 | 唐 1274 | 罪 1275 | 韦 1276 | 辛 1277 | 乡 1278 | 零 1279 | 迹 1280 | 固 1281 | 颜 1282 | 坏 1283 | 挂 1284 | 树 1285 | 戴 1286 | 翻 1287 | 帝 1288 | 闭 1289 | 川 1290 | 撞 1291 | z 1292 | 册 1293 | Y 1294 | 楚 1295 | 妈 1296 | 弱 1297 | 抵 1298 | 猫 1299 | 夺 1300 | 召 1301 | 埃 1302 | 腿 1303 | 筹 1304 | 握 1305 | 猛 1306 | 齐 1307 | 萨 1308 | 诚 1309 | 巧 1310 | 摆 1311 | 邀 1312 | 辉 1313 | 弃 1314 | 弟 1315 | 剩 1316 | 谋 1317 | 繁 1318 | 闲 1319 | > 1320 | 途 1321 | 钢 1322 | 衡 1323 | 厂 1324 | 拟 1325 | 默 1326 | 弗 1327 | 恶 1328 | 沉 1329 | 锦 1330 | 署 1331 | 搞 1332 | 赞 1333 | 墙 1334 | 伴 1335 | 纹 1336 | 兽 1337 | 甲 1338 | 扬 1339 | 乔 1340 | 赶 1341 | 衫 1342 | 薪 1343 | 毫 1344 | 迈 1345 | 滑 1346 | 订 1347 | 禁 1348 | 散 1349 | 恋 1350 | 隐 1351 | 【 1352 | 】 1353 | 津 1354 | 缓 1355 | 泽 1356 | 牙 1357 | 腰 1358 | 旦 1359 | 童 1360 | 霍 1361 | 榜 1362 | 冯 1363 | 脱 1364 | 跃 1365 | 梁 1366 | 植 1367 | 肉 1368 | 仪 1369 | 醒 1370 | 郭 1371 | 晨 1372 | 掘 1373 | 涛 1374 | " 1375 | 详 1376 | 龄 1377 | 脸 1378 | 朱 1379 | 洁 1380 | 鼓 1381 | 姐 1382 | 侦 1383 | 仙 1384 | 洞 1385 | 桥 1386 | 骨 1387 | 枪 1388 | 晶 1389 | 洗 1390 | 亏 1391 | 句 1392 | 亡 1393 | 乱 1394 | 坡 1395 | 骑 1396 | 颗 1397 | 胸 1398 | 贡 1399 | 啊 1400 | 搜 1401 | 宗 1402 | 灯 1403 | 染 1404 | 奋 1405 | 返 1406 | 抓 1407 | 淡 1408 | 帽 1409 | 皇 1410 | 饭 1411 | 艾 1412 | 链 1413 | 纸 1414 | 摩 1415 | 拆 1416 | ] 1417 | 亦 1418 | 敏 1419 | 毒 1420 | 汽 1421 | 姜 1422 | 援 1423 | 渠 1424 | 芬 1425 | 诞 1426 | 敢 1427 | 甜 1428 | 伙 1429 | 轨 1430 | 敦 1431 | 曝 1432 | 庄 1433 | 凡 1434 | 陷 1435 | 恢 1436 | 欣 1437 | [ 1438 | 拓 1439 | 誉 1440 | 殊 1441 | 添 1442 | 咨 1443 | 壁 1444 | 拜 1445 | 荷 1446 | 县 1447 | 宫 1448 | 径 1449 | 蒙 1450 | 驻 1451 | 疯 1452 | 尝 1453 | 泡 1454 | 荐 1455 | 跨 1456 | 炒 1457 | 麻 1458 | 帕 1459 | 携 1460 | 辑 1461 | 屏 1462 | 乏 1463 | 缘 1464 | 尊 1465 | 彻 1466 | 邦 1467 | 督 1468 | 蔡 1469 | 贺 1470 | 虚 1471 | 浮 1472 | 拼 1473 | 劲 1474 | 贾 1475 | 虎 1476 | 盈 1477 | 卷 1478 | ★ 1479 | 扰 1480 | 祖 1481 | 柔 1482 | 偿 1483 | 妙 1484 | 硕 1485 | Z 1486 | 闪 1487 | 兑 1488 | 兼 1489 | 扎 1490 | 郑 1491 | 黎 1492 | 鉴 1493 | 侧 1494 | 墅 1495 | 挺 1496 | 惯 1497 | 盗 1498 | 艳 1499 | 忙 1500 | 尾 1501 | 寸 1502 | 菜 1503 | 孤 1504 | 押 1505 | 床 1506 | 忘 1507 | 痛 1508 | 赏 1509 | 惜 1510 | 聊 1511 | 逊 1512 | 唱 1513 | 尺 1514 | 墨 1515 | 柏 1516 | 摇 1517 | 漂 1518 | 刊 1519 | 偶 1520 | 忆 1521 | 刀 1522 | 兄 1523 | 捕 1524 | 祝 1525 | 慎 1526 | 邓 1527 | 柜 1528 | 耶 1529 | 沟 1530 | 欲 1531 | 胞 1532 | 诸 1533 | 鸟 1534 | 患 1535 | 敬 1536 | 赚 1537 | 刷 1538 | 鹏 1539 | 迫 1540 | 姓 1541 | 紫 1542 | 迁 1543 | 抽 1544 | 宿 1545 | 妻 1546 | 矿 1547 | 颇 1548 | 鹿 1549 | 氛 1550 | 妇 1551 | 拒 1552 | 抱 1553 | 隔 1554 | 瓦 1555 | 塑 1556 | 羊 1557 | 泛 1558 | 慧 1559 | 窗 1560 | 绕 1561 | 籍 1562 | 孔 1563 | 址 1564 | 仿 1565 | 伍 1566 | 狼 1567 | 佩 1568 | 肩 1569 | ~ 1570 | 冒 1571 | 灭 1572 | 忧 1573 | 衬 1574 | 稿 1575 | 耳 1576 | 谷 1577 | 酷 1578 | 递 1579 | 阅 1580 | 胀 1581 | 杭 1582 | 翰 1583 | 篇 1584 | 蕾 1585 | 霆 1586 | 汤 1587 | 尖 1588 | 娜 1589 | 恰 1590 | 淑 1591 | 袋 1592 | 覆 1593 | 描 1594 | 倾 1595 | 览 1596 | 迟 1597 | 魅 1598 | 蛋 1599 | 宋 1600 | 葛 1601 | 残 1602 | 烟 1603 | 振 1604 | 钻 1605 | 番 1606 | 贯 1607 | 喷 1608 | 靴 1609 | 氏 1610 | 沈 1611 | 炸 1612 | 奶 1613 | 舍 1614 | 铺 1615 | 芳 1616 | 杯 1617 | 寒 1618 | 箱 1619 | 赖 1620 | 谨 1621 | 君 1622 | 颁 1623 | 诗 1624 | 蜂 1625 | 玛 1626 | 赠 1627 | 宠 1628 | 桑 1629 | 薄 1630 | 潘 1631 | 忽 1632 | 液 1633 | 俗 1634 | 锁 1635 | 浓 1636 | 碰 1637 | 逃 1638 | 俱 1639 | 披 1640 | 懂 1641 | 寄 1642 | 燃 1643 | 坞 1644 | 沿 1645 | 袭 1646 | 奈 1647 | 允 1648 | 鹰 1649 | 曹 1650 | 骗 1651 | 横 1652 | 奢 1653 | 寓 1654 | 荡 1655 | 抛 1656 | 畅 1657 | 爵 1658 | 毁 1659 | 乘 1660 | 戈 1661 | 稍 1662 | 甘 1663 | 阻 1664 | 燕 1665 | 捷 1666 | 甄 1667 | 耗 1668 | 鬼 1669 | 涯 1670 | 墓 1671 | 壮 1672 | 赎 1673 | 虫 1674 | 爸 1675 | 旁 1676 | 阴 1677 | 惑 1678 | 耐 1679 | 宙 1680 | 御 1681 | 遥 1682 | 耀 1683 | 玲 1684 | 逆 1685 | 刑 1686 | 拖 1687 | 鸡 1688 | 沪 1689 | 仁 1690 | 偷 1691 | 岩 1692 | 池 1693 | 魂 1694 | 幽 1695 | 厦 1696 | 睡 1697 | 绪 1698 | 凌 1699 | 剪 1700 | 镑 1701 | 踪 1702 | 盾 1703 | 奏 1704 | 艰 1705 | 乌 1706 | 悲 1707 | 浙 1708 | 悬 1709 | 旋 1710 | 芝 1711 | 卢 1712 | 肖 1713 | 稀 1714 | 污 1715 | 缴 1716 | 堪 1717 | 拔 1718 | 袖 1719 | 堆 1720 | 厨 1721 | 杉 1722 | 裂 1723 | 侵 1724 | 聘 1725 | 碎 1726 | 胆 1727 | 磨 1728 | 役 1729 | 译 1730 | 洪 1731 | 烦 1732 | 妹 1733 | 怡 1734 | 捧 1735 | 狗 1736 | 忠 1737 | 挡 1738 | 尘 1739 | 唤 1740 | 肥 1741 | 卧 1742 | 嫌 1743 | 赤 1744 | 饮 1745 | 嘴 1746 | 辆 1747 | 闹 1748 | 浦 1749 | j 1750 | 阔 1751 | 猎 1752 | 浩 1753 | 昌 1754 | 撑 1755 | 沫 1756 | 娘 1757 | 谭 1758 | 逼 1759 | 症 1760 | 夹 1761 | 驾 1762 | 伏 1763 | 貌 1764 | 疾 1765 | 棒 1766 | 粒 1767 | 孟 1768 | 乳 1769 | 砍 1770 | 矶 1771 | 羽 1772 | 鼠 1773 | 栋 1774 | 猪 1775 | 绎 1776 | 粗 1777 | 腾 1778 | 昆 1779 | 忍 1780 | 涵 1781 | 泉 1782 | 爽 1783 | 郎 1784 | 绘 1785 | 桌 1786 | 碧 1787 | 彼 1788 | 尸 1789 | 浴 1790 | 搬 1791 | 喝 1792 | 凤 1793 | × 1794 | 纠 1795 | 诱 1796 | 践 1797 | 惨 1798 | 坤 1799 | 踏 1800 | 昂 1801 | 挖 1802 | 悦 1803 | 泥 1804 | 谊 1805 | 肤 1806 | 奔 1807 | 碳 1808 | 谐 1809 | 烧 1810 | 瓶 1811 | 井 1812 | 莉 1813 | 俊 1814 | 哦 1815 | 妮 1816 | 犹 1817 | 巾 1818 | 庞 1819 | 宴 1820 | 睛 1821 | 幼 1822 | & 1823 | 妆 1824 | 契 1825 | 舰 1826 | 矛 1827 | 铭 1828 | 扫 1829 | 陪 1830 | 壳 1831 | 酬 1832 | 弄 1833 | 浅 1834 | 徒 1835 | 填 1836 | * 1837 | 兹 1838 | 摸 1839 | 夸 1840 | 拨 1841 | 炼 1842 | 抑 1843 | 愈 1844 | 晰 1845 | 琼 1846 | 伸 1847 | 梯 1848 | 茂 1849 | 烂 1850 | 堡 1851 | 驰 1852 | 戒 1853 | 捐 1854 | 孝 1855 | － 1856 | 碑 1857 | 裕 1858 | 吨 1859 | 溢 1860 | 猜 1861 | 废 1862 | 赔 1863 | 磁 1864 | 煤 1865 | 吊 1866 | 琴 1867 | 掩 1868 | 旬 1869 | 纵 1870 | 姿 1871 | 菌 1872 | 碍 1873 | 怒 1874 | 秦 1875 | 怖 1876 | 串 1877 | 恤 1878 | 氧 1879 | 纺 1880 | 辨 1881 | 袜 1882 | 勤 1883 | 疫 1884 | 乃 1885 | 粮 1886 | 邻 1887 | 旺 1888 | 柯 1889 | 吹 1890 | 循 1891 | 琳 1892 | 勃 1893 | 爷 1894 | 瞬 1895 | 脉 1896 | 叹 1897 | 赋 1898 | 淘 1899 | 薇 1900 | 页 1901 | 陶 1902 | 漏 1903 | 蒋 1904 | 辈 1905 | 蓄 1906 | 毅 1907 | 雇 1908 | 巡 1909 | 欠 1910 | 牵 1911 | 奕 1912 | 插 1913 | 岗 1914 | 娃 1915 | 剂 1916 | 桂 1917 | 鸿 1918 | 暑 1919 | 崇 1920 | 挤 1921 | 郁 1922 | 赌 1923 | 胁 1924 | 茶 1925 | 膝 1926 | 涌 1927 | 廷 1928 | 憾 1929 | 鸣 1930 | 莎 1931 | 雕 1932 | 凉 1933 | 酸 1934 | 贤 1935 | 辞 1936 | 滚 1937 | 蝶 1938 | 槛 1939 | 啦 1940 | 渡 1941 | 锐 1942 | 玻 1943 | 拳 1944 | 赫 1945 | 翼 1946 | 茨 1947 | 厉 1948 | 攀 1949 | • 1950 | 颖 1951 | 娇 1952 | 翔 1953 | 蛙 1954 | 蜜 1955 | 牢 1956 | 桃 1957 | 杆 1958 | 斑 1959 | 驱 1960 | 裸 1961 | 朵 1962 | 撤 1963 | 匹 1964 | 辅 1965 | 瓜 1966 | 枚 1967 | 械 1968 | 妖 1969 | 厘 1970 | 晋 1971 | 炮 1972 | 璃 1973 | 瘦 1974 | 哭 1975 | 飘 1976 | 伪 1977 | 侨 1978 | 框 1979 | 逾 1980 | 翠 1981 | 栏 1982 | 苑 1983 | 胶 1984 | 驶 1985 | 埋 1986 | 扶 1987 | 绒 1988 | 垃 1989 | 淇 1990 | 坑 1991 | 寿 1992 | 圾 1993 | 糕 1994 | 罕 1995 | 彭 1996 | 扭 1997 | 绑 1998 | 苗 1999 | 捉 2000 | 吕 2001 | 婴 2002 | 谱 2003 | 穷 2004 | 屡 2005 | 飙 2006 | 聪 2007 | 兆 2008 | 慈 2009 | 勿 2010 | 卓 2011 | 臣 2012 | 衰 2013 | 赁 2014 | 瓷 2015 | 腐 2016 | 斤 2017 | 弥 2018 | = 2019 | 叠 2020 | 奉 2021 | 豆 2022 | 纱 2023 | 荒 2024 | 涂 2025 | 冻 2026 | 郊 2027 | 睐 2028 | 滩 2029 | 仇 2030 | 鼎 2031 | 祥 2032 | 轰 2033 | 盲 2034 | ～ 2035 | 衷 2036 | 滞 2037 | 悠 2038 | 怨 2039 | 毯 2040 | 贫 2041 | 琪 2042 | 陨 2043 | 蛇 2044 | 泪 2045 | 肃 2046 | 侣 2047 | 臂 2048 | 陵 2049 | 莲 2050 | 垫 2051 | 猩 2052 | 椅 2053 | 劫 2054 | 皆 2055 | 尬 2056 | 脏 2057 | 丢 2058 | 尴 2059 | 铜 2060 | 靓 2061 | 盐 2062 | 傲 2063 | 膨 2064 | 彦 2065 | 旨 2066 | 婆 2067 | 辐 2068 | 贬 2069 | 疲 2070 | 艇 2071 | 棉 2072 | 泄 2073 | 妥 2074 | 胎 2075 | 滨 2076 | 喊 2077 | 擎 2078 | 柱 2079 | 翁 2080 | 狱 2081 | 馨 2082 | 魏 2083 | 糟 2084 | 腕 2085 | 悄 2086 | 妨 2087 | 砖 2088 | 履 2089 | _ 2090 | 哲 2091 | 谜 2092 | 虹 2093 | 吻 2094 | 炉 2095 | 纲 2096 | 竹 2097 | 劣 2098 | 闷 2099 | 纤 2100 | 泳 2101 | 咱 2102 | 漆 2103 | 凸 2104 | 夕 2105 | 俩 2106 | 脆 2107 | 饱 2108 | 藤 2109 | 贪 2110 | 亩 2111 | 猴 2112 | 辽 2113 | 漠 2114 | 塌 2115 | 丧 2116 | 孕 2117 | 疆 2118 | 轩 2119 | 玄 2120 | q 2121 | 斌 2122 | 擅 2123 | 僵 2124 | 汗 2125 | 挣 2126 | 峡 2127 | 袁 2128 | 衔 2129 | 弊 2130 | 凶 2131 | 痕 2132 | 巩 2133 | 邪 2134 | 勾 2135 | 寺 2136 | 朴 2137 | 迄 2138 | 侈 2139 | 呆 2140 | 抄 2141 | 豹 2142 | 斥 2143 | 掀 2144 | 彬 2145 | 躲 2146 | 裔 2147 | 罩 2148 | 辣 2149 | 盒 2150 | 啸 2151 | 莞 2152 | 丈 2153 | 钩 2154 | 缝 2155 | 肌 2156 | 炎 2157 | 狄 2158 | 糊 2159 | 帆 2160 | 橱 2161 | 倡 2162 | 媚 2163 | 笼 2164 | 磊 2165 | 凝 2166 | 慰 2167 | 渔 2168 | 割 2169 | 逢 2170 | 舟 2171 | 诈 2172 | 盼 2173 | 辩 2174 | 旭 2175 | 摊 2176 | 吓 2177 | 诊 2178 | 抬 2179 | ! 2180 | 垂 2181 | 魁 2182 | 逸 2183 | 髦 2184 | 彰 2185 | 艘 2186 | 屹 2187 | 轴 2188 | 蛮 2189 | 吁 2190 | 遵 2191 | 斜 2192 | 催 2193 | 姑 2194 | ' 2195 | 姻 2196 | 渤 2197 | 狮 2198 | 滋 2199 | 馈 2200 | 爬 2201 | 咖 2202 | 霞 2203 | 狠 2204 | 匪 2205 | 撼 2206 | 挫 2207 | 恨 2208 | 遮 2209 | 泼 2210 | 缀 2211 | 腻 2212 | 颠 2213 | 胖 2214 | 淀 2215 | 柳 2216 | 坠 2217 | 骂 2218 | 韵 2219 | 鲨 2220 | 湿 2221 | 耕 2222 | 啡 2223 | 敲 2224 | 撰 2225 | 翡 2226 | 穴 2227 | 扇 2228 | 疼 2229 | 蝠 2230 | 蛛 2231 | 俏 2232 | 醉 2233 | 糖 2234 | 煌 2235 | 鲸 2236 | 锡 2237 | 渴 2238 | 惧 2239 | 擦 2240 | 愤 2241 | 窄 2242 | 玫 2243 | 慕 2244 | 峻 2245 | 削 2246 | 舱 2247 | 抖 2248 | 抹 2249 | 腊 2250 | 炫 2251 | 摘 2252 | 歧 2253 | 楠 2254 | 帖 2255 | 熙 2256 | 寨 2257 | 捂 2258 | 穆 2259 | 丑 2260 | 兔 2261 | 殖 2262 | 慌 2263 | 嘛 2264 | 眠 2265 | 帐 2266 | 锻 2267 | 坎 2268 | 筐 2269 | 粹 2270 | 窃 2271 | 囊 2272 | 汶 2273 | 钉 2274 | 侯 2275 | ● 2276 | 羁 2277 | 戛 2278 | □ 2279 | 忌 2280 | 仰 2281 | 歇 2282 | 滴 2283 | 芯 2284 | 蝴 2285 | 瘾 2286 | 伐 2287 | 趁 2288 | 皱 2289 | 衍 2290 | 欺 2291 | ／ 2292 | 溪 2293 | 逛 2294 | 吞 2295 | 悚 2296 | 蝙 2297 | 揽 2298 | 扑 2299 | 崩 2300 | 枝 2301 | 缠 2302 | 哨 2303 | 勘 2304 | 堵 2305 | 砸 2306 | 诠 2307 | 鼻 2308 | 珊 2309 | 瑜 2310 | 帘 2311 | 寂 2312 | 钞 2313 | 烨 2314 | 倩 2315 | 昔 2316 | 瑟 2317 | 磅 2318 | 骤 2319 | 嫁 2320 | 挪 2321 | 眉 2322 | 贿 2323 | 凰 2324 | 鲍 2325 | 淋 2326 | 俐 2327 | 鹅 2328 | 腹 2329 | 佣 2330 | 锤 2331 | 廊 2332 | 蓬 2333 | 骄 2334 | 悟 2335 | 屈 2336 | 呵 2337 | 删 2338 | 吐 2339 | 盆 2340 | 搏 2341 | 凳 2342 | 萎 2343 | 恼 2344 | 陌 2345 | 汪 2346 | 傅 2347 | 愉 2348 | 裹 2349 | 瞩 2350 | 褶 2351 | ; 2352 | 巫 2353 | 贼 2354 | 坊 2355 | 茅 2356 | 撒 2357 | 邱 2358 | @ 2359 | 雾 2360 | 筒 2361 | 咬 2362 | 齿 2363 | 矮 2364 | 斩 2365 | 鑫 2366 | 璇 2367 | 锅 2368 | 樟 2369 | 拐 2370 | 逻 2371 | 函 2372 | 苹 2373 | 婷 2374 | 喻 2375 | 旱 2376 | 仲 2377 | 瑰 2378 | ■ 2379 | 惹 2380 | 慨 2381 | 窝 2382 | 悍 2383 | 讶 2384 | 悔 2385 | 膀 2386 | 晃 2387 | 颈 2388 | 讼 2389 | 凑 2390 | 亨 2391 | 葬 2392 | 睹 2393 | 叔 2394 | 崔 2395 | 蔓 2396 | 弯 2397 | 豫 2398 | 歉 2399 | 舆 2400 | 闯 2401 | 仕 2402 | 肿 2403 | 勋 2404 | 厌 2405 | 牲 2406 | 踝 2407 | 辰 2408 | 宪 2409 | 莹 2410 | 萧 2411 | 徽 2412 | 雀 2413 | 咏 2414 | 酿 2415 | 罢 2416 | 癌 2417 | 叛 2418 | 巢 2419 | 媛 2420 | 膜 2421 | 岳 2422 | 滥 2423 | 屠 2424 | 萍 2425 | 洒 2426 | 闫 2427 | 壤 2428 | 冕 2429 | 骚 2430 | 灌 2431 | 牧 2432 | 蜘 2433 | 橙 2434 | 垄 2435 | 肢 2436 | 弘 2437 | 牺 2438 | 嫩 2439 | 蹈 2440 | 龟 2441 | 肚 2442 | 炭 2443 | 汰 2444 | 傻 2445 | 芒 2446 | 阁 2447 | 螺 2448 | 桶 2449 | 壶 2450 | 胺 2451 | 绮 2452 | 黛 2453 | # 2454 | 晕 2455 | 臀 2456 | 郡 2457 | 绵 2458 | 泊 2459 | 陕 2460 | 脂 2461 | 葡 2462 | 蔚 2463 | 粤 2464 | 秩 2465 | 抚 2466 | 秉 2467 | 婉 2468 | 雯 2469 | 睿 2470 | 羡 2471 | 奠 2472 | 驳 2473 | 蚁 2474 | 囤 2475 | 鳄 2476 | 疏 2477 | 肆 2478 | 殿 2479 | 逝 2480 | 廖 2481 | 洽 2482 | 谦 2483 | 钓 2484 | 挽 2485 | 棚 2486 | 棕 2487 | 卵 2488 | 豚 2489 | 澄 2490 | 拘 2491 | 沸 2492 | 薛 2493 | 顽 2494 | 谍 2495 | 禽 2496 | 氢 2497 | 邵 2498 | 辟 2499 | 苍 2500 | 砂 2501 | 劝 2502 | 剔 2503 | 舌 2504 | 丛 2505 | 萄 2506 | 郝 2507 | ％ 2508 | 懒 2509 | 晤 2510 | 蒸 2511 | 啥 2512 | 伞 2513 | 湘 2514 | 茫 2515 | 辖 2516 | 盯 2517 | 拾 2518 | 尹 2519 | 蕴 2520 | 禅 2521 | 溃 2522 | 仗 2523 | 邹 2524 | 晒 2525 | 杠 2526 | 佐 2527 | 萌 2528 | 娥 2529 | 奎 2530 | 匆 2531 | 氰 2532 | 夷 2533 | 棋 2534 | 拯 2535 | 耍 2536 | 冈 2537 | 绸 2538 | 厕 2539 | 翟 2540 | 倪 2541 | 瞄 2542 | 碗 2543 | 筋 2544 | 祸 2545 | 庸 2546 | 暨 2547 | 岭 2548 | 竣 2549 | 诀 2550 | 巅 2551 | 栖 2552 | ⊙ 2553 | 麟 2554 | 丙 2555 | 镶 2556 | 扔 2557 | 遣 2558 | 楂 2559 | 呀 2560 | 俪 2561 | 羞 2562 | 祭 2563 | 妩 2564 | 澜 2565 | 痴 2566 | 烛 2567 | 浏 2568 | 仑 2569 | 彗 2570 | 姬 2571 | 袍 2572 | 焰 2573 | 廓 2574 | 绣 2575 | 翅 2576 | 叙 2577 | 辱 2578 | 蓉 2579 | 刃 2580 | 戚 2581 | 乾 2582 | 怜 2583 | 阐 2584 | 辜 2585 | 雌 2586 | 辕 2587 | 臭 2588 | 狐 2589 | 敞 2590 | 渣 2591 | 荧 2592 | 脖 2593 | 帷 2594 | 甩 2595 | 胃 2596 | 搁 2597 | 乙 2598 | 顷 2599 | 晴 2600 | 瞻 2601 | 聂 2602 | 饼 2603 | 丘 2604 | 璐 2605 | 梨 2606 | 惩 2607 | 琦 2608 | 侃 2609 | 柴 2610 | 愁 2611 | 绚 2612 | 魄 2613 | 噪 2614 | 弈 2615 | 犀 2616 | 崛 2617 | 扯 2618 | 灿 2619 | 溯 2620 | ◆ 2621 | 踢 2622 | 躺 2623 | 踩 2624 | 狭 2625 | 恭 2626 | 乖 2627 | 稻 2628 | 潭 2629 | 槽 2630 | 惕 2631 | 磋 2632 | 巷 2633 | 靡 2634 | 虐 2635 | 暮 2636 | 粘 2637 | 遏 2638 | $ 2639 | 铅 2640 | 浸 2641 | 绳 2642 | 刮 2643 | 愚 2644 | 炜 2645 | 镁 2646 | 哀 2647 | 逗 2648 | 勉 2649 | 酝 2650 | 芭 2651 | 浑 2652 | 畔 2653 | 沧 2654 | 嵌 2655 | 饲 2656 | 匠 2657 | 掏 2658 | 懈 2659 | 愧 2660 | 兜 2661 | 钗 2662 | 嫦 2663 | 咒 2664 | 屿 2665 | 骸 2666 | 叉 2667 | 渊 2668 | 绯 2669 | 霖 2670 | 躁 2671 | 漓 2672 | 匙 2673 | 昏 2674 | 弧 2675 | 蠢 2676 | 阱 2677 | 卜 2678 | 鸭 2679 | 脊 2680 | 庙 2681 | 渗 2682 | 屁 2683 | 浆 2684 | 韧 2685 | 肺 2686 | 熔 2687 | 鸦 2688 | 拦 2689 | 肠 2690 | 惟 2691 | 摔 2692 | 郅 2693 | 絮 2694 | 渝 2695 | 碌 2696 | é 2697 | 幢 2698 | 卸 2699 | 伽 2700 | 嘘 2701 | 擂 2702 | 龚 2703 | 腔 2704 | 婧 2705 | 烤 2706 | 硅 2707 | 纶 2708 | 矩 2709 | 诡 2710 | 昭 2711 | 娟 2712 | 霜 2713 | 翘 2714 | 燥 2715 | 冉 2716 | 摧 2717 | 碟 2718 | 篪 2719 | 汁 2720 | 犸 2721 | 玺 2722 | 甸 2723 | 窦 2724 | 铝 2725 | 晖 2726 | 蚀 2727 | 剖 2728 | 胚 2729 | 崖 2730 | 熬 2731 | 朔 2732 | 闵 2733 | 掠 2734 | 畏 2735 | 缅 2736 | 隅 2737 | 钥 2738 | 贩 2739 | 匿 2740 | 琢 2741 | 谅 2742 | 圻 2743 | 讽 2744 | 誓 2745 | 榄 2746 | 谣 2747 | 刹 2748 | 萝 2749 | 厄 2750 | 焕 2751 | 骼 2752 | 寥 2753 | 寞 2754 | 缎 2755 | 拷 2756 | 沦 2757 | 塘 2758 | 挨 2759 | 亥 2760 | 澡 2761 | 枯 2762 | 吵 2763 | 僧 2764 | 栈 2765 | 橘 2766 | 竭 2767 | 捡 2768 | 钮 2769 | 蔬 2770 | 髓 2771 | 谎 2772 | 橄 2773 | 溜 2774 | 睁 2775 | 撕 2776 | 沾 2777 | 缤 2778 | 茵 2779 | 拇 2780 | 寰 2781 | 罐 2782 | 佼 2783 | 屯 2784 | 竖 2785 | 殷 2786 | 崭 2787 | 潇 2788 | 犬 2789 | ． 2790 | 纬 2791 | 筛 2792 | 讳 2793 | 捞 2794 | 绽 2795 | 捏 2796 | 枫 2797 | 鞍 2798 | 琐 2799 | 卉 2800 | 涩 2801 | 铃 2802 | 肝 2803 | 唇 2804 | 腥 2805 | 卿 2806 | 瑶 2807 | 沛 2808 | 敛 2809 | 扛 2810 | 妍 2811 | ☆ 2812 | 嘲 2813 | 弓 2814 | 坪 2815 | 窍 2816 | 觅 2817 | 奴 2818 | 凹 2819 | 佟 2820 | 缔 2821 | 蔽 2822 | 闸 2823 | 扳 2824 | 冤 2825 | 蚂 2826 | 梳 2827 | 骏 2828 | 爪 2829 | 哄 2830 | 丸 2831 | 喂 2832 | 镖 2833 | 湛 2834 | 篷 2835 | 娄 2836 | 柄 2837 | 逮 2838 | 蜥 2839 | 盎 2840 | 缪 2841 | 剥 2842 | 鹤 2843 | 蟹 2844 | 帜 2845 | 傍 2846 | 枕 2847 | 瀑 2848 | 徊 2849 | 徘 2850 | 阮 2851 | 俞 2852 | 炙 2853 | 芙 2854 | 靖 2855 | 哺 2856 | 挚 2857 | 袂 2858 | 炬 2859 | 肪 2860 | 禾 2861 | 硫 2862 | 汀 2863 | 珀 2864 | 钧 2865 | 烹 2866 | 彪 2867 | 肇 2868 | 噩 2869 | 隙 2870 | 桩 2871 | 锈 2872 | 礁 2873 | 嚣 2874 | 垒 2875 | 尿 2876 | 襟 2877 | ＋ 2878 | 噬 2879 | 攒 2880 | 苛 2881 | 吼 2882 | 棍 2883 | 迭 2884 | 煞 2885 | 笨 2886 | 暇 2887 | 拢 2888 | 涡 2889 | 肘 2890 | 宰 2891 | 喇 2892 | 诛 2893 | 俭 2894 | → 2895 | 钦 2896 | 菊 2897 | 藉 2898 | 坝 2899 | 卑 2900 | 樊 2901 | 锂 2902 | 铆 2903 | 亭 2904 | 虞 2905 | 咕 2906 | 棵 2907 | 俯 2908 | 铸 2909 | 斧 2910 | 饿 2911 | 盔 2912 | 彤 2913 | 掷 2914 | 梭 2915 | 淮 2916 | 瞒 2917 | 卦 2918 | 汹 2919 | 锣 2920 | 辙 2921 | 濒 2922 | 宵 2923 | 褐 2924 | 蜡 2925 | 耻 2926 | 橡 2927 | 咸 2928 | 乍 2929 | 乞 2930 | 缸 2931 | 膏 2932 | 垮 2933 | 鄂 2934 | 弦 2935 | 霉 2936 | 荫 2937 | 畴 2938 | 虾 2939 | 倦 2940 | 隧 2941 | 昧 2942 | 冥 2943 | 绅 2944 | 荆 2945 | 媳 2946 | 嫂 2947 | 浇 2948 | 昊 2949 | 瘤 2950 | 烫 2951 | 遂 2952 | 赘 2953 | 跻 2954 | 烷 2955 | 藻 2956 | 酱 2957 | 庚 2958 | 俨 2959 | 驼 2960 | 饥 2961 | 糙 2962 | 煮 2963 | 曙 2964 | 辄 2965 | 狙 2966 | 舶 2967 | 趟 2968 | 咋 2969 | 冀 2970 | 钾 2971 | 茜 2972 | 佑 2973 | 拱 2974 | 滤 2975 | 腺 2976 | 妃 2977 | 骁 2978 | 骷 2979 | 祈 2980 | 斐 2981 | 麒 2982 | 覃 2983 | 髅 2984 | 颓 2985 | 脾 2986 | 镂 2987 | 熠 2988 | 儒 2989 | 霄 2990 | 哑 2991 | 贞 2992 | 圃 2993 | 枢 2994 | 噱 2995 | 陋 2996 | 冶 2997 | 涅 2998 | 蒲 2999 | 耽 3000 | 喘 3001 | 杖 3002 | 窥 3003 | 隽 3004 | 姨 3005 | 哒 3006 | | 3007 | 裴 3008 | 饶 3009 | 岚 3010 | 玮 3011 | 熏 3012 | 搅 3013 | 渲 3014 | 埔 3015 | 琛 3016 | ○ 3017 | ─ 3018 | 喧 3019 | < 3020 | ℃ 3021 | 踊 3022 | 拭 3023 | 肋 3024 | 耿 3025 | 厢 3026 | 邢 3027 | 镯 3028 | 捆 3029 | 株 3030 | 峥 3031 | 譬 3032 | 隶 3033 | 哇 3034 | 嫣 3035 | 葆 3036 | 屑 3037 | 璨 3038 | 绰 3039 | 孵 3040 | 沮 3041 | 帧 3042 | 揣 3043 | 歪 3044 | 奸 3045 | 瑚 3046 | 淹 3047 | 淳 3048 | 宛 3049 | 酶 3050 | 毋 3051 | 淫 3052 | 墟 3053 | 陡 3054 | 臃 3055 | 砚 3056 | 嘿 3057 | 啤 3058 | 嗅 3059 | 锯 3060 | 缚 3061 | 璀 3062 | 稽 3063 | 嬉 3064 | 茄 3065 | 佬 3066 | 栽 3067 | Ⅱ 3068 | 娅 3069 | 畜 3070 | 樱 3071 | 翩 3072 | 氨 3073 | 桦 3074 | 曦 3075 | 暧 3076 | 瞧 3077 | 鳌 3078 | 镀 3079 | 倘 3080 | 黯 3081 | 寡 3082 | 蚊 3083 | 葱 3084 | 惫 3085 | 幂 3086 | 烘 3087 | 惬 3088 | 椎 3089 | 飓 3090 | 抉 3091 | 妤 3092 | 溶 3093 | 昕 3094 | 媲 3095 | 沼 3096 | 轶 3097 | 闺 3098 | 煎 3099 | 饪 3100 | 滕 3101 | 耸 3102 | ＄ 3103 | 聆 3104 | 氮 3105 | 罄 3106 | 菁 3107 | 庐 3108 | 蜀 3109 | 窘 3110 | 鲲 3111 | 姗 3112 | 窟 3113 | 醋 3114 | 雍 3115 | 禧 3116 | 岂 3117 | 灶 3118 | 棺 3119 | 晟 3120 | 赐 3121 | 艋 3122 | 珂 3123 | 瞎 3124 | 釉 3125 | 缆 3126 | 痒 3127 | 缉 3128 | 窑 3129 | 岐 3130 | 柬 3131 | 狸 3132 | 猿 3133 | 琅 3134 | 膊 3135 | 牟 3136 | 迥 3137 | 匀 3138 | 爹 3139 | 蜗 3140 | 婕 3141 | 嘟 3142 | 蹄 3143 | 菱 3144 | 骥 3145 | 拎 3146 | 恳 3147 | 骆 3148 | 炯 3149 | 渺 3150 | 茹 3151 | 炳 3152 | 诟 3153 | 赂 3154 | 哗 3155 | 笛 3156 | 颐 3157 | 趾 3158 | 歼 3159 | ∶ 3160 | 泣 3161 | 陀 3162 | 蝉 3163 | 芦 3164 | 涿 3165 | 畸 3166 | ￥ 3167 | 堕 3168 | 釜 3169 | 壹 3170 | 宕 3171 | 硝 3172 | 玖 3173 | 蓓 3174 | 瑕 3175 | 贱 3176 | 豁 3177 | 娶 3178 | 禄 3179 | 缭 3180 | 绷 3181 | 赝 3182 | 穗 3183 | 栗 3184 | 泌 3185 | 伺 3186 | 膑 3187 | 坍 3188 | 稚 3189 | 醛 3190 | 叮 3191 | 滔 3192 | 坟 3193 | 胳 3194 | 躯 3195 | 凿 3196 | 邬 3197 | 瞪 3198 | 驭 3199 | 葵 3200 | 棱 3201 | 瀚 3202 | 囚 3203 | 烁 3204 | 溅 3205 | 旷 3206 | 榻 3207 | 侍 3208 | 矢 3209 | 辗 3210 | 磷 3211 | 轿 3212 | 挠 3213 | 炽 3214 | 憋 3215 | 洼 3216 | 杏 3217 | 崎 3218 | 捍 3219 | 诫 3220 | 阎 3221 | 芽 3222 | 劈 3223 | 嘱 3224 | 吾 3225 | 谴 3226 | 沐 3227 | 梵 3228 | 鸥 3229 | 姥 3230 | 蜴 3231 | 珑 3232 | 檀 3233 | 棘 3234 | 鞭 3235 | 蜕 3236 | 逍 3237 | 堤 3238 | 惺 3239 | 桐 3240 | 戮 3241 | 沁 3242 | 孢 3243 | 癫 3244 | 跪 3245 | 侏 3246 | 嘻 3247 | 魑 3248 | 喉 3249 | 瘫 3250 | 醇 3251 | 肾 3252 | 喀 3253 | 栩 3254 | 骇 3255 | 瓣 3256 | 忑 3257 | 萃 3258 | 忐 3259 | 烯 3260 | 粪 3261 | 徙 3262 | 嗓 3263 | 邸 3264 | 榴 3265 | 凄 3266 | 诅 3267 | 蕉 3268 | 椒 3269 | 蹲 3270 | 妞 3271 | 芸 3272 | 嗯 3273 | 妒 3274 | 羚 3275 | 韬 3276 | 剃 3277 | 朽 3278 | 蝇 3279 | 秃 3280 | 恍 3281 | 亟 3282 | 鄢 3283 | � 3284 | 憧 3285 | 褒 3286 | 牡 3287 | 妄 3288 | 汐 3289 | 阀 3290 | 扼 3291 | 遐 3292 | 蟒 3293 | 呕 3294 | 逅 3295 | 毙 3296 | 邂 3297 | 簇 3298 | 铮 3299 | 叭 3300 | 萱 3301 | 霾 3302 | 吝 3303 | 禺 3304 | 酋 3305 | 菇 3306 | 嗜 3307 | 襄 3308 | 蹊 3309 | 颤 3310 | 楷 3311 | 慑 3312 | 珏 3313 | 驯 3314 | 祷 3315 | 缇 3316 | 寇 3317 | 颂 3318 | 拽 3319 | 嫉 3320 | 憬 3321 | 氯 3322 | 飚 3323 | 褪 3324 | 氦 3325 | 灼 3326 | 眩 3327 | 悼 3328 | 螂 3329 | 俑 3330 | 咯 3331 | 铲 3332 | 砌 3333 | 疵 3334 | 兢 3335 | 跷 3336 | 枉 3337 | 袱 3338 | 晏 3339 | 皓 3340 | 澎 3341 | 晔 3342 | 僚 3343 | ・ 3344 | 匈 3345 | 梓 3346 | 汛 3347 | 徕 3348 | 娴 3349 | 锥 3350 | 倚 3351 | 匮 3352 | 弩 3353 | 饽 3354 | 庇 3355 | ^ 3356 | 厮 3357 | 荼 3358 | 匾 3359 | 沓 3360 | 恬 3361 | 瑾 3362 | 甫 3363 | 祺 3364 | 鳞 3365 | 孚 3366 | 眷 3367 | 孰 3368 | 坯 3369 | 孜 3370 | 迦 3371 | 莓 3372 | 拙 3373 | 矫 3374 | 僻 3375 | 啃 3376 | 蘑 3377 | 驴 3378 | 饺 3379 | 咽 3380 | 跋 3381 | 吟 3382 | 舵 3383 | 焚 3384 | 煽 3385 | 竿 3386 | 殴 3387 | 陇 3388 | 疤 3389 | 榆 3390 | 绊 3391 | 躬 3392 | 痪 3393 | 璧 3394 | 坷 3395 | 歹 3396 | 觑 3397 | 槟 3398 | 呐 3399 | 茸 3400 | 昱 3401 | 谬 3402 | 铨 3403 | 苞 3404 | 磕 3405 | 簿 3406 | 琥 3407 | 殆 3408 | 筝 3409 | 泻 3410 | 鞠 3411 | 缕 3412 | 窕 3413 | 窈 3414 | 檐 3415 | 兀 3416 | 掐 3417 | 趴 3418 | 瑙 3419 | 凛 3420 | 睦 3421 | 盏 3422 | 熄 3423 | 敷 3424 | 栅 3425 | 摒 3426 | 瞰 3427 | 潢 3428 | 喔 3429 | 佘 3430 | 蹿 3431 | 擒 3432 | 抒 3433 | 圭 3434 | 谙 3435 | 碘 3436 | 砥 3437 | 仆 3438 | 赃 3439 | 阪 3440 | 羹 3441 | 黏 3442 | 舅 3443 | 蹦 3444 | 鳍 3445 | 俘 3446 | 屉 3447 | 穹 3448 | 婪 3449 | 笈 3450 | 汲 3451 | 叨 3452 | 愣 3453 | 镍 3454 | 蛟 3455 | 挟 3456 | 撇 3457 | 咎 3458 | 寐 3459 | ° 3460 | 粥 3461 | 惮 3462 | 堰 3463 | 巍 3464 | 漩 3465 | 钙 3466 | 丐 3467 | 骋 3468 | 隋 3469 | 憨 3470 | 煜 3471 | 峨 3472 | 鹊 3473 | 毗 3474 | 慷 3475 | 酪 3476 | 哎 3477 | 烙 3478 | \ 3479 | 棠 3480 | 娆 3481 | 慵 3482 | 挎 3483 | 莘 3484 | 煲 3485 | 惋 3486 | 咳 3487 | 翎 3488 | 尧 3489 | 寝 3490 | 斋 3491 | 昵 3492 | 蚕 3493 | 苯 3494 | 捣 3495 | 澈 3496 | 稠 3497 | 眨 3498 | 稣 3499 | 犷 3500 | 砷 3501 | 邃 3502 | 绞 3503 | 雏 3504 | 诧 3505 | 裘 3506 | 椭 3507 | 疟 3508 | 砭 3509 | 辫 3510 | 嘎 3511 | 呛 3512 | 涤 3513 | 篑 3514 | 榈 3515 | 岑 3516 | 戟 3517 | 颅 3518 | 铐 3519 | 咧 3520 | 蚓 3521 | 筷 3522 | 螃 3523 | 苟 3524 | 衅 3525 | 箍 3526 | 膺 3527 | 惶 3528 | 泵 3529 | 曳 3530 | 闽 3531 | 呱 3532 | 剽 3533 | 靶 3534 | 戳 3535 | 砝 3536 | 粟 3537 | 沽 3538 | 掺 3539 | 侥 3540 | 臻 3541 | 懵 3542 | 峙 3543 | 钵 3544 | 蝎 3545 | 狡 3546 | 弛 3547 | 昼 3548 | 琉 3549 | 碱 3550 | 哮 3551 | 奚 3552 | 伎 3553 | 蛰 3554 | 揪 3555 | 皙 3556 | 迸 3557 | 薯 3558 | 亢 3559 | 悖 3560 | 戎 3561 | 垦 3562 | 矜 3563 | 鸽 3564 | 鹦 3565 | 榕 3566 | 酣 3567 | 窒 3568 | 倔 3569 | 胯 3570 | 伶 3571 | 诵 3572 | 踵 3573 | 钊 3574 | 姊 3575 | 酌 3576 | 抨 3577 | 哟 3578 | 撩 3579 | 莅 3580 | μ 3581 | 拧 3582 | ® 3583 | 溉 3584 | 柠 3585 | 炅 3586 | 妓 3587 | 懿 3588 | 茧 3589 | 峭 3590 | 雁 3591 | 渎 3592 | ◎ 3593 | 窖 3594 | 貂 3595 | 腩 3596 | 酥 3597 | 蒜 3598 | 舜 3599 | 蚯 3600 | 耘 3601 | 肴 3602 | 麓 3603 | 曜 3604 | 嚷 3605 | 韶 3606 | 娲 3607 | 缄 3608 | 嵩 3609 | 氓 3610 | 猬 3611 | 鹉 3612 | 娓 3613 | 敖 3614 | 诙 3615 | 筠 3616 | 刁 3617 | 匣 3618 | 矣 3619 | 邋 3620 | 柿 3621 | ※ 3622 | 鱿 3623 | 紊 3624 | 淆 3625 | 皂 3626 | 竺 3627 | 鹜 3628 | 攘 3629 | 苓 3630 | 馅 3631 | 斓 3632 | 厥 3633 | 淌 3634 | 窜 3635 | 癖 3636 | 兮 3637 | 遢 3638 | 夭 3639 | 驹 3640 | 咪 3641 | 茎 3642 | 梗 3643 | 粽 3644 | 喋 3645 | 瑛 3646 | 屎 3647 | 鹭 3648 | 幔 3649 | 咀 3650 | 怠 3651 | 铧 3652 | 匡 3653 | 沌 3654 | 萤 3655 | 噢 3656 | 眈 3657 | 猖 3658 | 阙 3659 | 滇 3660 | 拌 3661 | 睫 3662 | 潍 3663 | 乒 3664 | 铂 3665 | 裱 3666 | 葫 3667 | 忡 3668 | 趸 3669 | 檬 3670 | 砺 3671 | 狩 3672 | 〈 3673 | 撬 3674 | 悸 3675 | 〉 3676 | 粱 3677 | 崽 3678 | 锏 3679 | 痊 3680 | 嚼 3681 | 瓢 3682 | 禹 3683 | 湮 3684 | ＝ 3685 | 冗 3686 | 夯 3687 | 贰 3688 | 汝 3689 | 侮 3690 | 乓 3691 | 礴 3692 | 笃 3693 | 淤 3694 | 潞 3695 | 涎 3696 | 恪 3697 | 淼 3698 | 呦 3699 | 吆 3700 | 侄 3701 | 泓 3702 | 荔 3703 | 遴 3704 | 霓 3705 | 沂 3706 | 怯 3707 | 酵 3708 | 尉 3709 | 墩 3710 | 咫 3711 | 椰 3712 | 藩 3713 | 勺 3714 | 唾 3715 | 簧 3716 | 薰 3717 | 桓 3718 | 胧 3719 | 柒 3720 | 饕 3721 | 蠕 3722 | 钛 3723 | 蜒 3724 | 钰 3725 | 熹 3726 | 槌 3727 | 泾 3728 | 蜿 3729 | 埠 3730 | 疮 3731 | 谧 3732 | 挝 3733 | 笋 3734 | 瞿 3735 | 枣 3736 | 恺 3737 | 拂 3738 | 丫 3739 | 羌 3740 | 醍 3741 | 睾 3742 | 朦 3743 | 颚 3744 | 赈 3745 | 岌 3746 | 餮 3747 | 锷 3748 | 荟 3749 | 姝 3750 | 锄 3751 | 扉 3752 | 匕 3753 | 嘀 3754 | 芜 3755 | 铎 3756 | 萦 3757 | 暄 3758 | 秽 3759 | 栾 3760 | 啬 3761 | 菩 3762 | 糅 3763 | 鬃 3764 | 芋 3765 | 眸 3766 | 踞 3767 | 湃 3768 | 獗 3769 | 瀛 3770 | 裳 3771 | 奂 3772 | 纭 3773 | 罹 3774 | 焉 3775 | 阜 3776 | 氟 3777 | 茱 3778 | 瞅 3779 | 蛊 3780 | 昙 3781 | 篡 3782 | 蛤 3783 | 濮 3784 | 嵘 3785 | Ⅲ 3786 | 撮 3787 | 篱 3788 | 痞 3789 | 铿 3790 | 璞 3791 | 岔 3792 | 逞 3793 | 祀 3794 | 苇 3795 | 扒 3796 | 钝 3797 | 峪 3798 | 腱 3799 | 莽 3800 | 蚤 3801 | 溺 3802 | 瞠 3803 | 珐 3804 | 疚 3805 | 矗 3806 | 茬 3807 | 飒 3808 | 俺 3809 | 炖 3810 | 晾 3811 | 颊 3812 | 蒿 3813 | 橹 3814 | 彷 3815 | 咄 3816 | 泗 3817 | 掰 3818 | 笙 3819 | 砾 3820 | 卒 3821 | 沥 3822 | 宸 3823 | 馒 3824 | 嗽 3825 | 羲 3826 | 寅 3827 | 铠 3828 | 焱 3829 | ］ 3830 | 螳 3831 | 铀 3832 | 漪 3833 | 遛 3834 | 荃 3835 | 浒 3836 | 姣 3837 | α 3838 | 诽 3839 | 聋 3840 | 猥 3841 | 丞 3842 | ［ 3843 | 骰 3844 | 榨 3845 | 忱 3846 | 虱 3847 | 皿 3848 | 抠 3849 | 隼 3850 | 呗 3851 | 叱 3852 | 啼 3853 | 囱 3854 | 眯 3855 | 哉 3856 | 哼 3857 | 唉 3858 | 嚎 3859 | 憩 3860 | 嗒 3861 | 汾 3862 | 捅 3863 | 猾 3864 | 刨 3865 | 蟾 3866 | 鲶 3867 | 谤 3868 | 猝 3869 | 酯 3870 | 锌 3871 | 枭 3872 | 籽 3873 | 於 3874 | 犁 3875 | 曰 3876 | 垩 3877 | 唏 3878 | 诃 3879 | Ø 3880 | ① 3881 | 稼 3882 | 蜻 3883 | 唠 3884 | ™ 3885 | 膛 3886 | 斟 3887 | 胤 3888 | è 3889 | 蟑 3890 | 轼 3891 | 痹 3892 | 叼 3893 | 猕 3894 | 蜓 3895 | 烬 3896 | 觎 3897 | 拣 3898 | 拗 3899 | 浚 3900 | 揍 3901 | 汕 3902 | 揉 3903 | 缮 3904 |  3905 | 鄙 3906 | 剿 3907 | 泯 3908 | 祉 3909 | 璋 3910 | 虏 3911 | 婿 3912 | 莠 3913 | 捺 3914 | 蹬 3915 | 晦 3916 | 榷 3917 | 钒 3918 | 缨 3919 | 呜 3920 | 啪 3921 | 搀 3922 | 鹃 3923 | 狈 3924 | 觊 3925 | 瘟 3926 | 涓 3927 | 绌 3928 | 辍 3929 | 枷 3930 | 咤 3931 | 赣 3932 | 谛 3933 | 蔷 3934 | 腼 3935 | 亵 3936 | 浣 3937 | 〕 3938 | 霹 3939 | 饵 3940 | 瞥 3941 | 椋 3942 | 雳 3943 | 桔 3944 | 眶 3945 | 腆 3946 | 娩 3947 | 愫 3948 | 腮 3949 | 翌 3950 | 绢 3951 | 桨 3952 | 喙 3953 | 鞘 3954 | 旎 3955 | 〔 3956 | 惦 3957 | 凋 3958 | 澧 3959 | 憎 3960 | 惚 3961 | 烽 3962 | 跤 3963 | 昀 3964 | 殉 3965 | 怂 3966 | 邝 3967 | 涟 3968 | 芹 3969 | 魇 3970 | 羔 3971 | 贻 3972 | 祁 3973 | 赦 3974 | 殃 3975 | 蕊 3976 | 轲 3977 | 珉 3978 | 麾 3979 | 蟆 3980 | 驿 3981 | 咆 3982 | 蜍 3983 | 脐 3984 | 弋 3985 | 螈 3986 | 袒 3987 | 霁 3988 | 惰 3989 | 馥 3990 | 佯 3991 | 踹 3992 | ¾ 3993 | 眺 3994 | 蓟 3995 | 祠 3996 | 渥 3997 | 掂 3998 | 臼 3999 | 蹭 4000 | 漾 4001 | 禀 4002 | 瘪 4003 | 喽 4004 | 涧 4005 | 懊 4006 | 鲟 4007 | 贮 4008 | 渍 4009 | 昴 4010 | 裆 4011 | 舔 4012 | ü 4013 | 摁 4014 | 琨 4015 | 谑 4016 | 咐 4017 | 钳 4018 | 缜 4019 | 昶 4020 | 摹 4021 | 帼 4022 | 焊 4023 | 毡 4024 | ‰ 4025 | 锢 4026 | 邯 4027 | ② 4028 | 蛾 4029 | 髻 4030 | 秤 4031 | 肮 4032 | 蔑 4033 | 睽 4034 | 鬣 4035 | 茁 4036 | 涮 4037 | 町 4038 | 锚 4039 | 珞 4040 | 袄 4041 | 诩 4042 | 糜 4043 | 颌 4044 | 掮 4045 | 苔 4046 | 臆 4047 | 梧 4048 | 岖 4049 | 冽 4050 | 菠 4051 | 蔼 4052 | 黔 4053 | 诣 4054 | 镰 4055 | 膳 4056 | 殇 4057 | 漱 4058 | 躏 4059 | 垢 4060 | 坨 4061 | 腋 4062 | 鏖 4063 | 蹂 4064 | 泸 4065 | 铛 4066 | 札 4067 | 诘 4068 | 嗦 4069 | 玟 4070 | 偕 4071 | 妾 4072 | 嗡 4073 | 钚 4074 | 籁 4075 | 鹫 4076 | 搂 4077 | 澍 4078 | 隘 4079 | 卯 4080 | 迢 4081 | 馍 4082 | 遁 4083 | 袤 4084 | 炊 4085 | 溥 4086 | 嘶 4087 | 铤 4088 | 迂 4089 | 闰 4090 | 甭 4091 | 濡 4092 | 犒 4093 | 孽 4094 | 垣 4095 | 胫 4096 | 唬 4097 | Ａ 4098 | 胄 4099 | 嘈 4100 | 垡 4101 | 懦 4102 | 唧 4103 | 翱 4104 | 喃 4105 | ` 4106 | 跚 4107 | 晗 4108 | 棣 4109 | 荚 4110 | 幌 4111 | ⋯ 4112 | 蹒 4113 | 殡 4114 | 犊 4115 | 蕙 4116 | 柚 4117 | 蛆 4118 | 茉 4119 | 煊 4120 | 冼 4121 | 遨 4122 | 涝 4123 | 胰 4124 | 哽 4125 | 馋 4126 | 睬 4127 | 馁 4128 | 鸠 4129 | 痫 4130 | 纣 4131 | 靳 4132 | 芥 4133 | 祛 4134 | 咂 4135 | 搓 4136 | 倭 4137 | 啄 4138 | ◇ 4139 | 砰 4140 | 苷 4141 | 攫 4142 | 磺 4143 | 哆 4144 | 嗖 4145 | 浊 4146 | 榭 4147 | 恙 4148 | 涸 4149 | 霏 4150 | 扈 4151 | 獒 4152 | 搪 4153 | 酰 4154 | 鸳 4155 | 惭 4156 | 拄 4157 | 讧 4158 | 骅 4159 | 蜚 4160 | 鲤 4161 | 岱 4162 | 毓 4163 | 钠 4164 | ‧ 4165 | 瞳 4166 | 潦 4167 | 藕 4168 | à 4169 | 芮 4170 | 脍 4171 | 濠 4172 | 疹 4173 | 偌 4174 | 啧 4175 | 镳 4176 | 嗣 4177 | 簸 4178 | 菅 4179 | 』 4180 | 狳 4181 | 阂 4182 | 『 4183 | 庶 4184 | 犰 4185 | 讹 4186 | 徨 4187 | 篆 4188 | 珥 4189 | 後 4190 | 鸯 4191 | 搡 4192 | 妊 4193 | 嫔 4194 | 梢 4195 | Ｇ 4196 | 掳 4197 | 瘩 4198 | 讥 4199 | 疙 4200 | 丕 4201 | 呃 4202 | 茗 4203 | 攸 4204 | 锲 4205 | 叩 4206 | 楔 4207 | 惘 4208 | 哩 4209 | 郦 4210 | 莺 4211 | 獭 4212 | 蹶 4213 | 衩 4214 | ↑ 4215 | 夙 4216 | 虔 4217 | 嗔 4218 | 羸 4219 | 惴 4220 | 掣 4221 | 抡 4222 | 熨 4223 | 孪 4224 | 衙 4225 | 罡 4226 | 拈 4227 | 渭 4228 | Ｈ 4229 | 祟 4230 | 怅 4231 | 唆 4232 | 蔻 4233 | 恕 4234 | 狞 4235 | 愕 4236 | 啷 4237 | 燎 4238 | 瘠 4239 | 窿 4240 | 谩 4241 | 浜 4242 | 黝 4243 | 咔 4244 | 潺 4245 | Ⅶ 4246 | 涕 4247 | 纂 4248 | 箴 4249 | 遑 4250 | 靛 4251 | 藐 4252 | 煦 4253 | 踌 4254 | 邑 4255 | ▲ 4256 | 咚 4257 | 魈 4258 | 婀 4259 | 湄 4260 | 筱 4261 | 鼬 4262 | 躇 4263 | 碾 4264 | 衢 4265 | 舛 4266 | 琶 4267 | 鸵 4268 | 泞 4269 | 轧 4270 | 拴 4271 | 撂 4272 | 攥 4273 | 忻 4274 | √ 4275 | 瘸 4276 | 佰 4277 | 铢 4278 | 翊 4279 | 裨 4280 | 腑 4281 | 漳 4282 | 胱 4283 | 馗 4284 | 讷 4285 | 徜 4286 | 灸 4287 | 跆 4288 | 祚 4289 | 樵 4290 | 珈 4291 | ― 4292 | 麂 4293 | 弑 4294 | 杞 4295 | 幄 4296 | 廿 4297 | 鳗 4298 | 鲑 4299 | 徉 4300 | 偎 4301 | 锵 4302 | 椁 4303 | 蛹 4304 | 冢 4305 | 娣 4306 | 侬 4307 | 笆 4308 | ③ 4309 | 叽 4310 | 皑 4311 | 忏 4312 | 葩 4313 | 闳 4314 | 吱 4315 | 舷 4316 | 悴 4317 | 槐 4318 | 歆 4319 | 憔 4320 | 亘 4321 | Ⅰ 4322 | 蝌 4323 | 峦 4324 | 恣 4325 | 阑 4326 | 牒 4327 | 汞 4328 | 豉 4329 | 舸 4330 | 怵 4331 | 蝓 4332 | 蟠 4333 | 嗷 4334 | 帚 4335 | 盹 4336 | 褂 4337 | 蹴 4338 | 酗 4339 | 蚪 4340 | 噎 4341 | 灏 4342 | 蘸 4343 | ＆ 4344 | 洱 4345 | 伫 4346 | £ 4347 | 甬 4348 | 〗 4349 | ↓ 4350 | 掬 4351 | 〖 4352 | 淅 4353 | 贲 4354 | 懋 4355 | Ｃ 4356 | 蜷 4357 | 橇 4358 | 咙 4359 | 郸 4360 | 雹 4361 | 赓 4362 | 幺 4363 | 霰 4364 | 颦 4365 | 孱 4366 | 忒 4367 | 糗 4368 | 岷 4369 | ∩ 4370 | 摞 4371 | 谚 4372 | 谲 4373 | 酮 4374 | 啮 4375 | 坂 4376 | 帛 4377 | 铱 4378 | 吒 4379 | 偃 4380 | 悯 4381 | 恻 4382 | 恿 4383 | β 4384 | Ｔ 4385 | 搔 4386 | 铯 4387 | 痰 4388 | 楹 4389 | 旖 4390 | 娠 4391 | 荥 4392 | 烩 4393 | 磐 4394 | 俚 4395 | 豌 4396 | 擞 4397 | 纫 4398 | 崴 4399 | 辘 4400 | 潸 4401 | 塾 4402 | 喳 4403 | 啻 4404 | 鎏 4405 | 狰 4406 | 壕 4407 | 颢 4408 | 嗤 4409 | 骛 4410 | 悻 4411 | 蛞 4412 | 嫡 4413 | 蚝 4414 | 焙 4415 | 糯 4416 | 缰 4417 | 鬓 4418 | 狒 4419 | 栎 4420 | 稔 4421 | 迩 4422 | 傀 4423 | 痘 4424 | 蔗 4425 | Ⅳ 4426 | 卞 4427 | 鹕 4428 | 蹩 4429 | 鹈 4430 | 孀 4431 | 瘀 4432 | 谕 4433 | 樾 4434 | 戾 4435 | 痼 4436 | 纾 4437 | 钿 4438 | 喱 4439 | 诬 4440 | 唔 4441 | 蛀 4442 | 钨 4443 | 滟 4444 | 楞 4445 | 拮 4446 | 栓 4447 | 粼 4448 | 骊 4449 | 壑 4450 | 濑 4451 | 仨 4452 | ￡ 4453 | 嗨 4454 | 寮 4455 | 儡 4456 | 傣 4457 | ≠ 4458 | 倜 4459 | 蕨 4460 | 吡 4461 | 霎 4462 | 疡 4463 | 泱 4464 | 羿 4465 | 俸 4466 | 瘙 4467 | 饬 4468 | 诿 4469 | 仝 4470 | 踉 4471 | 嗲 4472 | 柑 4473 | 纰 4474 | 腌 4475 | 嚏 4476 | 泠 4477 | ＞ 4478 | ≤ 4479 | 漕 4480 | 瓮 4481 | 聿 4482 | 腴 4483 | 喵 4484 | 卤 4485 | 箫 4486 | 樽 4487 | 褛 4488 | 惆 4489 | 渚 4490 | 羯 4491 | 摺 4492 | 酉 4493 | 挛 4494 | 氘 4495 | 傥 4496 | 笠 4497 | 幡 4498 | 诋 4499 | 鳖 4500 | 谒 4501 | 蚌 4502 | 炔 4503 | 屐 4504 | 奄 4505 | 谟 4506 | 羟 4507 | 飕 4508 | 鼹 4509 | 唰 4510 | 摈 4511 | 跄 4512 | 荀 4513 | 樨 4514 | 锆 4515 | 拚 4516 | 揄 4517 | 苜 4518 | 杵 4519 | 蓿 4520 | 盂 4521 | 麝 4522 | 镌 4523 | 揶 4524 | 阋 4525 | Ｏ 4526 | 鳃 4527 | 囡 4528 | 箔 4529 | 庾 4530 | 殓 4531 | 鸸 4532 | 蹋 4533 | 讪 4534 | 蝰 4535 | 脲 4536 | 阖 4537 | 稷 4538 | の 4539 | 褥 4540 | 胭 4541 | 铍 4542 | 桎 4543 | 疱 4544 | 潼 4545 | 绛 4546 | ä 4547 | 鲼 4548 | 诨 4549 | 梏 4550 | 逵 4551 | 铬 4552 | 痉 4553 | 鹋 4554 | 恃 4555 | 蔫 4556 | 钴 4557 | 铩 4558 | 皋 4559 | § 4560 | Ｎ 4561 | 粕 4562 | 绫 4563 | 吭 4564 | 麽 4565 | 呻 4566 | ＊ 4567 | 瞌 4568 | ン 4569 | 镣 4570 | 烊 4571 | 啕 4572 | 脓 4573 | 牍 4574 | 陲 4575 | 谘 4576 | 耷 4577 | 赅 4578 | 撷 4579 | 罔 4580 | 燮 4581 | 涣 4582 | 陂 4583 | 钜 4584 | 湍 4585 | 闾 4586 | 銮 4587 | 珙 4588 | 酩 4589 | 嚓 4590 | 邙 4591 | 怦 4592 | 铰 4593 | 忿 4594 | ＂ 4595 | 祯 4596 | 榉 4597 | 刽 4598 | 伉 4599 | 蟀 4600 | 氚 4601 | 滦 4602 | 垠 4603 | 擘 4604 | 陛 4605 | 珩 4606 | 跺 4607 | 謦 4608 | 庖 4609 | 坻 4610 | 麋 4611 | 捶 4612 | 嗑 4613 | 褚 4614 | 焯 4615 | 湫 4616 | 僮 4617 | 桀 4618 | 荭 4619 | 蟋 4620 | 唁 4621 | 锹 4622 | 瘁 4623 | 咦 4624 | 熵 4625 | 榔 4626 | 胥 4627 | 腭 4628 | 缥 4629 | 桅 4630 | 嗝 4631 | ÷ 4632 | 孺 4633 | 绉 4634 | 绥 4635 | 顼 4636 | 胛 4637 | É 4638 | 瞑 4639 | 砒 4640 | 颞 4641 | Ｂ 4642 | 脯 4643 | 噜 4644 | 荤 4645 | 吏 4646 | Ｅ 4647 | 囿 4648 | 骜 4649 | 捎 4650 | 沅 4651 | 瓯 4652 | 痍 4653 | 颛 4654 | 亳 4655 | 淄 4656 | 篝 4657 | 囔 4658 | 甥 4659 | 黠 4660 | 皖 4661 | 剁 4662 | 鞑 4663 | 秧 4664 | 娑 4665 | 谏 4666 | 吩 4667 | 赳 4668 | 撅 4669 | 鲈 4670 | 岫 4671 | 颧 4672 | 蹉 4673 | 岿 4674 | 谆 4675 | Ｄ 4676 | 跎 4677 | 漉 4678 | 佻 4679 | ­ 4680 | 仃 4681 | ² 4682 | 筵 4683 | 罂 4684 | 宦 4685 | 缈 4686 | 飨 4687 | 沣 4688 | 楣 4689 | ＜ 4690 | 氩 4691 | 吮 4692 | 龈 4693 | 汩 4694 | 杳 4695 | 唳 4696 | 诏 4697 | 淞 4698 | 噔 4699 | 酚 4700 | 鼾 4701 | ó 4702 | 蛎 4703 | 锭 4704 | 鳕 4705 | イ 4706 | 碉 4707 | 蕲 4708 | 搐 4709 | 鄞 4710 | 臧 4711 | 皎 4712 | 诲 4713 | 蹑 4714 | 吠 4715 | 膘 4716 | 骡 4717 | 髋 4718 | 赡 4719 | 鹬 4720 | 艮 4721 | Ｖ 4722 | 楦 4723 | ＇ 4724 | 嫖 4725 | 婶 4726 | 轫 4727 | 蛭 4728 | € 4729 | 盥 4730 | 疣 4731 | 琵 4732 | 掴 4733 | 倬 4734 | 咿 4735 | 碴 4736 | 癣 4737 | 泔 4738 | 榫 4739 | 汨 4740 | ル 4741 | 蝾 4742 | 岬 4743 | 敝 4744 | 芊 4745 | 龛 4746 | 氙 4747 | 耦 4748 | 踱 4749 | 褓 4750 | 笺 4751 | 镉 4752 | 戊 4753 | 斡 4754 | 叁 4755 | 抿 4756 | 荞 4757 | 蚩 4758 | 袅 4759 | 婵 4760 | 徇 4761 | Ｆ 4762 | 仟 4763 | 皈 4764 | 逯 4765 | Ｓ 4766 | 彝 4767 | 讴 4768 | 醺 4769 | 柩 4770 | 炕 4771 | 淖 4772 | 揆 4773 | 綦 4774 | 畿 4775 | 嘞 4776 | 俾 4777 | 鲇 4778 | 狨 4779 | 芾 4780 | 蓦 4781 | 锶 4782 | 噤 4783 | 琏 4784 | ｜ 4785 | 锰 4786 | 镪 4787 | 琮 4788 | 菏 4789 | 鲵 4790 | 掖 4791 | 璎 4792 | 荻 4793 | 韭 4794 | Ｐ 4795 | 鲔 4796 | 葳 4797 | ア 4798 | 嘹 4799 | 觞 4800 | 洙 4801 | 秣 4802 | 庵 4803 | 烃 4804 | 徵 4805 | 肽 4806 | 殒 4807 | 兖 4808 | 璜 4809 | 」 4810 | 龅 4811 | 啶 4812 | ç 4813 | 裥 4814 | 跛 4815 | 缂 4816 | 獠 4817 | 「 4818 | 袢 4819 | 莆 4820 | 豺 4821 | 勐 4822 | 妲 4823 | 磬 4824 | 浔 4825 | 睑 4826 | 鲱 4827 | 箩 4828 | 碜 4829 | 驮 4830 | 犄 4831 | 堑 4832 | 嬗 4833 | 蝮 4834 | 滢 4835 | 捱 4836 | 哌 4837 | 鹳 4838 | 圩 4839 | 叻 4840 | 锟 4841 | 栉 4842 | γ 4843 | 靼 4844 | 樘 4845 | 涔 4846 | 棂 4847 | 沱 4848 | 鸢 4849 | 馏 4850 | 苒 4851 | 颉 4852 | 桢 4853 | 隍 4854 | 牦 4855 | 瓒 4856 | 擢 4857 | 郇 4858 | 暌 4859 | 蜇 4860 | 劾 4861 | ≥ 4862 | 捋 4863 | 扪 4864 | 邛 4865 | 蝗 4866 | 镭 4867 | 嵋 4868 | 桉 4869 | 捻 4870 | ö 4871 | ラ 4872 | 褴 4873 | 娼 4874 | 仄 4875 | 濂 4876 |  4877 | 淬 4878 | 枳 4879 | 枸 4880 | 峋 4881 | 阉 4882 | 疽 4883 | 襁 4884 | 秸 4885 | 螨 4886 | 骞 4887 | 芷 4888 | 榛 4889 | 颀 4890 | 莨 4891 | 秆 4892 | 藓 4893 | 蕃 4894 | 悱 4895 | 旮 4896 | 窠 4897 | 溧 4898 | 嶙 4899 | ス 4900 | } 4901 | 崂 4902 | 湎 4903 | 藜 4904 | 鸩 4905 | 钌 4906 | 椿 4907 | Ⅵ 4908 | 邡 4909 | 锗 4910 | 郴 4911 | 桁 4912 | 洵 4913 | 醴 4914 | Ｍ 4915 | 赭 4916 | 墒 4917 | 坳 4918 | 醚 4919 | ジ 4920 | 簪 4921 | 踮 4922 | 儆 4923 | 隈 4924 | Ｘ 4925 | 岘 4926 | 肛 4927 | 氪 4928 | 谶 4929 | 痤 4930 | 耆 4931 | 骧 4932 | 囹 4933 | 岙 4934 | 螭 4935 | 螯 4936 | 缱 4937 | 澹 4938 | 谔 4939 | 醪 4940 | 厝 4941 | 圄 4942 | 旯 4943 | 啖 4944 | 蜃 4945 | 鸾 4946 | ④ 4947 | 梆 4948 | í 4949 | 硒 4950 | 绺 4951 | ê 4952 | 睇 4953 | 芑 4954 | 砧 4955 | 钤 4956 | 戬 4957 | 玷 4958 | 晌 4959 | 跗 4960 | ​ 4961 | á 4962 | 嶂 4963 | 钺 4964 | 泷 4965 | 瓴 4966 | 痨 4967 | 耙 4968 | ± 4969 | 郗 4970 | 睢 4971 | 怆 4972 | 弼 4973 | Ⅷ 4974 | 挞 4975 | 纨 4976 | ┊ 4977 | 孑 4978 | 俎 4979 | 戍 4980 | ″ 4981 | 噼 4982 | 氤 4983 | 涪 4984 | 绗 4985 | 撵 4986 | 倌 4987 | 荏 4988 | 遽 4989 | 蜈 4990 | ＃ 4991 | 箕 4992 | 竽 4993 | 钯 4994 | 韪 4995 | 貔 4996 | 凼 4997 | 箐 4998 | 垭 4999 | 枥 5000 | 貅 -------------------------------------------------------------------------------- /data/.cnews_.swp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pengwei-iie/Bert-THUCNews/a20749225091533b530f0e539bfaacbd3524fe99/data/.cnews_.swp -------------------------------------------------------------------------------- /data/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pengwei-iie/Bert-THUCNews/a20749225091533b530f0e539bfaacbd3524fe99/data/__init__.py -------------------------------------------------------------------------------- /data/cnews_loader.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | import sys 4 | from collections import Counter 5 | 6 | import numpy as np 7 | import tensorflow.contrib.keras as kr 8 | import tensorflow as tf 9 | if sys.version_info[0] > 2: 10 | is_py3 = True 11 | else: 12 | # reload(sys) 13 | sys.setdefaultencoding("utf-8") 14 | is_py3 = False 15 | 16 | 17 | def native_word(word, encoding='utf-8'): 18 | """如果在python2下面使用python3训练的模型，可考虑调用此函数转化一下字符编码""" 19 | if not is_py3: 20 | return word.encode(encoding) 21 | else: 22 | return word 23 | 24 | 25 | def native_content(content): 26 | if not is_py3: 27 | return content.decode('utf-8') 28 | else: 29 | return content 30 | 31 | 32 | def open_file(filename, mode='r'): 33 | """ 34 | 常用文件操作，可在python2和python3间切换. 35 | mode: 'r' or 'w' for read or write 36 | """ 37 | if is_py3: 38 | return open(filename, mode, encoding='utf-8', errors='ignore') 39 | else: 40 | return open(filename, mode) 41 | 42 | 43 | def read_file(filename): 44 | """读取文件数据,content是一个字符串""" 45 | contents, labels = [], [] 46 | with open_file(filename) as f: 47 | for line in f: 48 | # while True: 49 | # line = f.readline() 50 | try: 51 | label, content = line.strip().split('\t') 52 | if content: 53 | contents.append(list(native_content(content))) 54 | labels.append(native_content(label)) 55 | except: 56 | pass 57 | # if not line: 58 | # break 59 | return contents, labels 60 | 61 | 62 | def build_vocab(train_dir, vocab_dir, vocab_size=5000): 63 | """根据训练集构建词汇表，存储, x， y""" 64 | data_train, _ = read_file(train_dir) 65 | 66 | all_data = [] 67 | for content in data_train: 68 | all_data.extend(content) 69 | # 有4000多万个字 只选了5000个字 70 | counter = Counter(all_data)# statistic the word and its frecuency of the list(dictionary) 71 | count_pairs = counter.most_common(vocab_size - 1) 72 | words, _ = list(zip(*count_pairs)) # word is a tuple 73 | # 添加一个 来将所有文本pad为同一长度 74 | words = [''] + list(words) 75 | open_file(vocab_dir, mode='w').write('\n'.join(words) + '\n') 76 | 77 | 78 | def read_vocab(vocab_dir): 79 | """读取词汇表""" 80 | # words = open_file(vocab_dir).read().strip().split('\n') 81 | with open_file(vocab_dir) as fp: 82 | # 如果是py2 则每个值都转化为unicode 83 | words = [native_content(_.strip()) for _ in fp.readlines()] 84 | word_to_id = dict(zip(words, range(len(words)))) 85 | return words, word_to_id 86 | 87 | 88 | def read_category(): 89 | """读取分类目录，固定""" 90 | categories = ['体育', '财经', '房产', '家居', '教育', '科技', '时尚', '时政', '游戏', '娱乐'] 91 | 92 | categories = [native_content(x) for x in categories] 93 | 94 | cat_to_id = dict(zip(categories, range(len(categories)))) 95 | 96 | return categories, cat_to_id 97 | 98 | 99 | def to_words(content, words): 100 | """将id表示的内容转换为文字""" 101 | return ''.join(words[x] for x in content) 102 | 103 | 104 | def process_file(filename, word_to_id, cat_to_id, max_length=600): 105 | """将文件转换为id表示""" 106 | contents, labels = read_file(filename) 107 | # np.save('./train_x.npy', contents) 108 | # np.savetxt('./train_x.txt', contents, fmt='%s') 109 | data_id, label_id = [], [] 110 | for i in range(len(contents)): 111 | data_id.append([word_to_id[x] for x in contents[i] if x in word_to_id]) 112 | label_id.append(cat_to_id[labels[i]]) 113 | 114 | # 使用keras提供的pad_sequences来将文本pad为固定长度 115 | x_pad = kr.preprocessing.sequence.pad_sequences(data_id, max_length) 116 | y_pad = kr.utils.to_categorical(label_id, num_classes=len(cat_to_id)) # 将标签转换为one-hot表示 117 | 118 | return x_pad, y_pad 119 | 120 | 121 | def batch_iter(x, y, batch_size=64): 122 | """生成批次数据""" 123 | data_len = len(x) 124 | num_batch = int((data_len - 1) / batch_size) + 1 125 | # 区别在于shuffle直接在原来的数组上进行操作，改变原来数组的顺序，无返回值。 126 | # 而permutation不直接在原来的数组上进行操作，而是返回一个新的打乱顺序的数组，并不改变原来的数组。 127 | indices = np.random.permutation(np.arange(data_len)) 128 | x_shuffle = x[indices] 129 | y_shuffle = y[indices] 130 | for i in range(num_batch): 131 | start_id = i * batch_size 132 | end_id = min((i + 1) * batch_size, data_len) 133 | yield x_shuffle[start_id:end_id], y_shuffle[start_id:end_id] 134 | def attention(inputs, attention_size, l2_reg_lambda): 135 | """ 136 | Attention mechanism layer. 137 | :param inputs: outputs of RNN/Bi-RNN layer (not final state) 138 | :param attention_size: linear size of attention weights 139 | :return: outputs of the passed RNN/Bi-RNN reduced with attention vector 140 | """ 141 | # In case of Bi-RNN input we need to concatenate outputs of its forward and backward parts 142 | if isinstance(inputs, tuple): 143 | inputs = tf.concat(2, inputs) 144 | 145 | sequence_length = inputs.get_shape()[1].value # the length of sequences processed in the antecedent RNN layer 146 | hidden_size = inputs.get_shape()[2].value # hidden size of the RNN layer 147 | 148 | # Attention mechanism W,b 相当于对RNN的输出做一个非线性变化，得到的结果在和u做内积 149 | W_omega = tf.get_variable("W_omega", initializer=tf.random_normal([hidden_size, attention_size], stddev=0.1)) 150 | b_omega = tf.get_variable("b_omega", initializer=tf.random_normal([attention_size], stddev=0.1)) 151 | u_omega = tf.get_variable("u_omega", initializer=tf.random_normal([attention_size], stddev=0.1)) 152 | # v--(b*n, 512) vu--input和u做内积(b*n, 1) exps--(b, n) 153 | v = tf.tanh(tf.matmul(tf.reshape(inputs, [-1, hidden_size]), W_omega) + tf.reshape(b_omega, [1, -1])) 154 | vu = tf.matmul(v, tf.reshape(u_omega, [-1, 1])) 155 | exps = tf.reshape(tf.exp(vu), [-1, sequence_length]) 156 | alphas = exps / tf.reshape(tf.reduce_sum(exps, 1), [-1, 1]) 157 | 158 | # Output of Bi-RNN is reduced with attention vector (b, hidden_dim)相当于做完了对每个字的权重，然后加起来 159 | output = tf.reduce_sum(inputs * tf.reshape(alphas, [-1, sequence_length, 1]), 1) 160 | #if l2_reg_lambda > 0: 161 | # l2_loss += tf.nn.l2_loss(W_omega) 162 | # l2_loss += tf.nn.l2_loss(b_omega) 163 | # l2_loss += tf.nn.l2_loss(u_omega) 164 | # tf.add_to_collection('losses', l2_loss) 165 | 166 | return output 167 | -------------------------------------------------------------------------------- /data/cnews_loader_bert.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | import sys 4 | from collections import Counter 5 | 6 | import numpy as np 7 | import tensorflow.contrib.keras as kr 8 | import tensorflow as tf 9 | if sys.version_info[0] > 2: 10 | is_py3 = True 11 | else: 12 | # reload(sys) 13 | sys.setdefaultencoding("utf-8") 14 | is_py3 = False 15 | 16 | 17 | def native_word(word, encoding='utf-8'): 18 | """如果在python2下面使用python3训练的模型，可考虑调用此函数转化一下字符编码""" 19 | if not is_py3: 20 | return word.encode(encoding) 21 | else: 22 | return word 23 | 24 | 25 | def native_content(content): 26 | if not is_py3: 27 | return content.decode('utf-8') 28 | else: 29 | return content 30 | 31 | 32 | def open_file(filename, mode='r'): 33 | """ 34 | 常用文件操作，可在python2和python3间切换. 35 | mode: 'r' or 'w' for read or write 36 | """ 37 | if is_py3: 38 | return open(filename, mode, encoding='utf-8', errors='ignore') 39 | else: 40 | return open(filename, mode) 41 | 42 | 43 | def read_file(filename): 44 | """读取文件数据""" 45 | contents, labels = [], [] 46 | with open_file(filename) as f: 47 | for line in f: 48 | # while True: 49 | # line = f.readline() 50 | try: 51 | label, content = line.strip().split('\t') 52 | contents.append(content) 53 | if content: 54 | # contents.append(list(native_content(content))) 55 | labels.append(native_content(label)) 56 | except: 57 | pass 58 | # if not line: 59 | # break 60 | return contents, labels 61 | 62 | 63 | def build_vocab(train_dir, vocab_dir, vocab_size=5000): 64 | """根据训练集构建词汇表，存储, x， y""" 65 | data_train, _ = read_file(train_dir) 66 | 67 | all_data = [] 68 | for content in data_train: 69 | all_data.extend(content) 70 | 71 | counter = Counter(all_data) 72 | count_pairs = counter.most_common(vocab_size - 1) 73 | words, _ = list(zip(*count_pairs)) 74 | # 添加一个 来将所有文本pad为同一长度 75 | words = [''] + list(words) 76 | open_file(vocab_dir, mode='w').write('\n'.join(words) + '\n') 77 | 78 | 79 | def read_vocab(vocab_dir): 80 | """读取词汇表""" 81 | # words = open_file(vocab_dir).read().strip().split('\n') 82 | with open_file(vocab_dir) as fp: 83 | # 如果是py2 则每个值都转化为unicode 84 | words = [native_content(_.strip()) for _ in fp.readlines()] 85 | word_to_id = dict(zip(words, range(len(words)))) 86 | return words, word_to_id 87 | 88 | 89 | def read_category(): 90 | """读取分类目录，固定""" 91 | categories = ['体育', '财经', '房产', '家居', '教育', '科技', '时尚', '时政', '游戏', '娱乐'] 92 | 93 | categories = [native_content(x) for x in categories] 94 | 95 | cat_to_id = dict(zip(categories, range(len(categories)))) 96 | 97 | return categories, cat_to_id 98 | 99 | 100 | def to_words(content, words): 101 | """将id表示的内容转换为文字""" 102 | return ''.join(words[x] for x in content) 103 | 104 | 105 | def process_file(filename, word_to_id, cat_to_id, max_length=600): 106 | """将文件转换为id表示""" 107 | contents, labels = read_file(filename) 108 | # np.save('./train_x.npy', contents) 109 | # np.savetxt('./train_x.txt', contents, fmt='%s') 110 | data_id, label_id = [], [] 111 | for i in range(len(contents)): 112 | # data_id.append([word_to_id[x] for x in contents[i] if x in word_to_id]) 113 | label_id.append(cat_to_id[labels[i]]) 114 | 115 | # 使用keras提供的pad_sequences来将文本pad为固定长度 116 | # x_pad = kr.preprocessing.sequence.pad_sequences(data_id, max_length) 117 | y_pad = kr.utils.to_categorical(label_id, num_classes=len(cat_to_id)) # 将标签转换为one-hot表示 118 | 119 | return contents, y_pad 120 | 121 | 122 | def batch_iter(x, y, batch_size=64): 123 | """生成批次数据""" 124 | data_len = len(x) 125 | num_batch = int((data_len - 1) / batch_size) + 1 126 | # 区别在于shuffle直接在原来的数组上进行操作，改变原来数组的顺序，无返回值。 127 | # 而permutation不直接在原来的数组上进行操作，而是返回一个新的打乱顺序的数组，并不改变原来的数组。 128 | indices = np.random.permutation(np.arange(data_len)) 129 | x_shuffle = np.array(x)[indices] 130 | y_shuffle = y[indices] 131 | for i in range(num_batch): 132 | start_id = i * batch_size 133 | end_id = min((i + 1) * batch_size, data_len) 134 | # yield x[start_id:end_id], y[start_id:end_id] 135 | yield x_shuffle[start_id:end_id], y_shuffle[start_id:end_id] 136 | def attention(inputs, attention_size, l2_reg_lambda): 137 | """ 138 | Attention mechanism layer. 139 | :param inputs: outputs of RNN/Bi-RNN layer (not final state) 140 | :param attention_size: linear size of attention weights 141 | :return: outputs of the passed RNN/Bi-RNN reduced with attention vector 142 | """ 143 | # In case of Bi-RNN input we need to concatenate outputs of its forward and backward parts 144 | if isinstance(inputs, tuple): 145 | inputs = tf.concat(2, inputs) 146 | 147 | sequence_length = inputs.get_shape()[1].value # the length of sequences processed in the antecedent RNN layer 148 | hidden_size = inputs.get_shape()[2].value # hidden size of the RNN layer 149 | 150 | # Attention mechanism W,b 相当于对RNN的输出做一个非线性变化，得到的结果在和u做内积 151 | W_omega = tf.get_variable("W_omega", initializer=tf.random_normal([hidden_size, attention_size], stddev=0.1)) 152 | b_omega = tf.get_variable("b_omega", initializer=tf.random_normal([attention_size], stddev=0.1)) 153 | u_omega = tf.get_variable("u_omega", initializer=tf.random_normal([attention_size], stddev=0.1)) 154 | 155 | v = tf.tanh(tf.matmul(tf.reshape(inputs, [-1, hidden_size]), W_omega) + tf.reshape(b_omega, [1, -1])) 156 | vu = tf.matmul(v, tf.reshape(u_omega, [-1, 1])) 157 | exps = tf.reshape(tf.exp(vu), [-1, sequence_length]) 158 | alphas = exps / tf.reshape(tf.reduce_sum(exps, 1), [-1, 1]) 159 | 160 | # Output of Bi-RNN is reduced with attention vector 161 | output = tf.reduce_sum(inputs * tf.reshape(alphas, [-1, sequence_length, 1]), 1) 162 | #if l2_reg_lambda > 0: 163 | # l2_loss += tf.nn.l2_loss(W_omega) 164 | # l2_loss += tf.nn.l2_loss(b_omega) 165 | # l2_loss += tf.nn.l2_loss(u_omega) 166 | # tf.add_to_collection('losses', l2_loss) 167 | 168 | return output 169 | -------------------------------------------------------------------------------- /helper/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pengwei-iie/Bert-THUCNews/a20749225091533b530f0e539bfaacbd3524fe99/helper/__init__.py -------------------------------------------------------------------------------- /helper/cnews_group.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # -*- coding: utf-8 -*- 3 | 4 | """ 5 | 将文本整合到 train、test、val 三个文件中 6 | """ 7 | 8 | import os 9 | 10 | def _read_file(filename): 11 | """读取一个文件并转换为一行""" 12 | with open(filename, 'r', encoding='utf-8') as f: 13 | return f.read().replace('\n', '').replace('\t', '').replace('\u3000', '') 14 | 15 | def save_file(dirname): 16 | """ 17 | 将多个文件整合并存到3个文件中 18 | dirname: 原数据目录 19 | 文件内容格式: 类别\t内容 20 | """ 21 | f_train = open('data/cnews/cnews.train.txt', 'w', encoding='utf-8') 22 | f_test = open('data/cnews/cnews.test.txt', 'w', encoding='utf-8') 23 | f_val = open('data/cnews/cnews.val.txt', 'w', encoding='utf-8') 24 | for category in os.listdir(dirname): # 分类目录 25 | cat_dir = os.path.join(dirname, category) 26 | if not os.path.isdir(cat_dir): 27 | continue 28 | files = os.listdir(cat_dir) 29 | count = 0 30 | for cur_file in files: 31 | filename = os.path.join(cat_dir, cur_file) 32 | content = _read_file(filename) 33 | if count < 5000: 34 | f_train.write(category + '\t' + content + '\n') 35 | elif count < 6000: 36 | f_test.write(category + '\t' + content + '\n') 37 | else: 38 | f_val.write(category + '\t' + content + '\n') 39 | count += 1 40 | 41 | print('Finished:', category) 42 | 43 | f_train.close() 44 | f_test.close() 45 | f_val.close() 46 | 47 | 48 | if __name__ == '__main__': 49 | save_file('data/thucnews') 50 | print(len(open('data/cnews/cnews.train.txt', 'r', encoding='utf-8').readlines())) 51 | print(len(open('data/cnews/cnews.test.txt', 'r', encoding='utf-8').readlines())) 52 | print(len(open('data/cnews/cnews.val.txt', 'r', encoding='utf-8').readlines())) 53 | -------------------------------------------------------------------------------- /helper/copy_data.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # copy MAXCOUNT files from each directory 4 | 5 | MAXCOUNT=6500 6 | 7 | for category in $( ls THUCNews); do 8 | echo item: $category 9 | 10 | dir=THUCNews/$category 11 | newdir=data/thucnews/$category 12 | if [ -d $newdir ]; then 13 | rm -rf $newdir 14 | mkdir $newdir 15 | fi 16 | 17 | COUNTER=1 18 | for i in $(ls $dir); do 19 | cp $dir/$i $newdir 20 | if [ $COUNTER -ge $MAXCOUNT ] 21 | then 22 | echo finished 23 | break 24 | fi 25 | let COUNTER=COUNTER+1 26 | done 27 | 28 | done 29 | -------------------------------------------------------------------------------- /images/acc_loss.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pengwei-iie/Bert-THUCNews/a20749225091533b530f0e539bfaacbd3524fe99/images/acc_loss.png -------------------------------------------------------------------------------- /images/acc_loss_rnn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pengwei-iie/Bert-THUCNews/a20749225091533b530f0e539bfaacbd3524fe99/images/acc_loss_rnn.png -------------------------------------------------------------------------------- /images/cnn_architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pengwei-iie/Bert-THUCNews/a20749225091533b530f0e539bfaacbd3524fe99/images/cnn_architecture.png -------------------------------------------------------------------------------- /images/rnn_architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pengwei-iie/Bert-THUCNews/a20749225091533b530f0e539bfaacbd3524fe99/images/rnn_architecture.png -------------------------------------------------------------------------------- /k_means.py: -------------------------------------------------------------------------------- 1 | from sklearn.cluster import KMeans 2 | from sklearn.externals import joblib 3 | from sklearn import cluster 4 | import numpy as np 5 | 6 | # 生成10*3的矩阵 7 | data = [] 8 | with open('test.txt', 'r') as f: 9 | for line in f: 10 | line = line.strip().split() 11 | line = np.asarray(line, dtype=np.int) 12 | data.append(line) 13 | print(data) 14 | # 聚类为4类 15 | estimator = KMeans(n_clusters=3) 16 | # fit_predict表示拟合+预测，也可以分开写 17 | res = estimator.fit_predict(data) 18 | # 预测类别标签结果 19 | lable_pred = estimator.labels_ 20 | # 各个类别的聚类中心值 21 | centroids = estimator.cluster_centers_ 22 | # 聚类中心均值向量的总和 23 | inertia = estimator.inertia_ 24 | 25 | print(lable_pred) 26 | print(centroids) 27 | print(inertia) -------------------------------------------------------------------------------- /predict.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | from __future__ import print_function 4 | 5 | import os 6 | import tensorflow as tf 7 | import tensorflow.contrib.keras as kr 8 | 9 | from cnn_model import TCNNConfig, TextCNN 10 | from data.cnews_loader import read_category, read_vocab 11 | 12 | try: 13 | bool(type(unicode)) 14 | except NameError: 15 | unicode = str 16 | 17 | base_dir = 'data/cnews' 18 | vocab_dir = os.path.join(base_dir, 'cnews.vocab.txt') 19 | 20 | save_dir = 'checkpoints/textcnn' 21 | save_path = os.path.join(save_dir, 'best_validation') # 最佳验证结果保存路径 22 | 23 | 24 | class CnnModel: 25 | def __init__(self): 26 | self.config = TCNNConfig() 27 | self.categories, self.cat_to_id = read_category() 28 | self.words, self.word_to_id = read_vocab(vocab_dir) 29 | self.config.vocab_size = len(self.words) 30 | self.model = TextCNN(self.config) 31 | 32 | self.session = tf.Session() 33 | self.session.run(tf.global_variables_initializer()) 34 | saver = tf.train.Saver() 35 | saver.restore(sess=self.session, save_path=save_path) # 读取保存的模型 36 | 37 | def predict(self, message): 38 | # 支持不论在python2还是python3下训练的模型都可以在2或者3的环境下运行 39 | content = unicode(message) 40 | data = [self.word_to_id[x] for x in content if x in self.word_to_id] 41 | 42 | feed_dict = { 43 | self.model.input_x: kr.preprocessing.sequence.pad_sequences([data], self.config.seq_length), 44 | self.model.keep_prob: 1.0 45 | } 46 | 47 | y_pred_cls = self.session.run(self.model.y_pred_cls, feed_dict=feed_dict) 48 | return self.categories[y_pred_cls[0]] 49 | 50 | 51 | if __name__ == '__main__': 52 | cnn_model = CnnModel() 53 | test_demo = ['三星ST550以全新的拍摄方式超越了以往任何一款数码相机', 54 | '热火vs骑士前瞻：皇帝回乡二番战 东部次席唾手可得新浪体育讯北京时间3月30日7:00'] 55 | for i in test_demo: 56 | print(cnn_model.predict(i)) 57 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | scikit-learn 2 | scipy 3 | numpy -------------------------------------------------------------------------------- /rnn_model.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # -*- coding: utf-8 -*- 3 | 4 | import tensorflow as tf 5 | from data.cnews_loader import attention 6 | 7 | class TRNNConfig(object): 8 | """RNN配置参数""" 9 | 10 | # 模型参数 11 | embedding_dim = 200 # 词向量维度 12 | seq_length = 600 # 序列长度 13 | num_classes = 10 # 类别数 14 | vocab_size = 5000 # 词汇表达小 15 | 16 | num_layers= 3 # 隐藏层层数 17 | hidden_dim = 512 # 隐藏层神经元 18 | rnn = 'gru' # lstm 或 gru 19 | 20 | attention_dim = 512 21 | l2_reg_lambda = 0.01 22 | 23 | dropout_keep_prob = 0.5 # dropout保留比例 24 | learning_rate = 1e-3 # 学习率 25 | 26 | batch_size = 128 # 每批训练大小 27 | num_epochs = 6 # 总迭代轮次 28 | 29 | print_per_batch = 100 # 每多少轮输出一次结果 30 | save_per_batch = 10 # 每多少轮存入tensorboard 31 | 32 | 33 | class TextRNN(object): 34 | """文本分类，RNN模型""" 35 | def __init__(self, config): 36 | self.config = config 37 | 38 | # 三个待输入的数据 39 | self.input_x = tf.placeholder(tf.int32, [None, self.config.seq_length], name='input_x') 40 | self.input_y = tf.placeholder(tf.float32, [None, self.config.num_classes], name='input_y') 41 | self.keep_prob = tf.placeholder(tf.float32, name='keep_prob') 42 | 43 | self.rnn() 44 | 45 | def rnn(self): 46 | """rnn模型""" 47 | 48 | def lstm_cell(): # lstm核 49 | return tf.contrib.rnn.BasicLSTMCell(self.config.hidden_dim, state_is_tuple=True) 50 | 51 | def gru_cell(): # gru核 52 | return tf.contrib.rnn.GRUCell(self.config.hidden_dim) 53 | 54 | def dropout(): # 为每一个rnn核后面加一个dropout层 55 | if (self.config.rnn == 'lstm'): 56 | cell_forward = lstm_cell() 57 | # cell_backward = lstm_cell() 58 | else: 59 | cell_forward = gru_cell() 60 | # cell_backward = gru_cell() 61 | cell_forward = tf.contrib.rnn.DropoutWrapper(cell_forward, output_keep_prob=self.keep_prob) 62 | # cell_backward = tf.contrib.rnn.DropoutWrapper(cell_backward, output_keep_prob=self.keep_prob) 63 | return cell_forward# , cell_backward 64 | # 词向量映射 65 | with tf.device('/cpu:0'): 66 | embedding = tf.get_variable('embedding', [self.config.vocab_size, self.config.embedding_dim]) 67 | embedding_inputs = tf.nn.embedding_lookup(embedding, self.input_x) 68 | 69 | with tf.name_scope("rnn"): 70 | # 多层rnn网络 71 | # cells = [dropout() for _ in range(self.config.num_layers)] 72 | # rnn_cell = tf.contrib.rnn.MultiRNNCell(cells, state_is_tuple=True) 73 | # 74 | # _outputs, _ = tf.nn.dynamic_rnn(cell=rnn_cell, inputs=embedding_inputs, dtype=tf.float32) 75 | # 不能一起写，要分开写，不能这样 76 | cell_forward = [dropout() for _ in range(self.config.num_layers)] 77 | cell_backward = [dropout() for _ in range(self.config.num_layers)] 78 | lstm_fw = tf.nn.rnn_cell.MultiRNNCell(cell_forward) 79 | lstm_bw = tf.nn.rnn_cell.MultiRNNCell(cell_backward) 80 | _outputs, bi_state = tf.nn.bidirectional_dynamic_rnn(lstm_fw, lstm_bw, embedding_inputs, dtype=tf.float32) 81 | # _outputs, bi_state = tf.nn.dynamic_rnn(lstm_fw, embedding_inputs, dtype=tf.float32) 82 | # use attention 83 | out_fw, out_bw = _outputs 84 | # 只使用最后一个状态作为句向量 85 | # out_fw = out_fw[:, -1, :] 86 | # out_bw = out_bw[:, -1, :] 87 | 88 | # 使用att(batch, time_step, hidden_unit * layer) 89 | _outputs = tf.concat([out_fw, out_bw], axis=2) 90 | output = attention(_outputs, self.config.attention_dim, self.config.l2_reg_lambda) 91 | # use last unit to represent the sentence 92 | # last = _outputs[:, -1, :] # 取最后一个时序输出作为结果 93 | # output (b*1024) 94 | with tf.name_scope("score"): 95 | # 全连接层，后面接dropout以及relu激活 96 | fc = tf.layers.dense(output, self.config.hidden_dim, name='fc1') 97 | fc = tf.contrib.layers.dropout(fc, self.keep_prob) 98 | fc = tf.nn.relu(fc) 99 | 100 | # 分类器 101 | self.logits = tf.layers.dense(fc, self.config.num_classes, name='fc2') 102 | self.y_pred_cls = tf.argmax(tf.nn.softmax(self.logits), 1) # 预测类别 103 | 104 | with tf.name_scope("optimize"): 105 | # 损失函数，交叉熵 `logits` and `labels` must have the same shape 106 | cross_entropy = tf.nn.softmax_cross_entropy_with_logits(logits=self.logits, labels=self.input_y) 107 | self.loss = tf.reduce_mean(cross_entropy) 108 | # 优化器 109 | self.optim = tf.train.AdamOptimizer(learning_rate=self.config.learning_rate).minimize(self.loss) 110 | 111 | with tf.name_scope("accuracy"): 112 | # 准确率 113 | correct_pred = tf.equal(tf.argmax(self.input_y, 1), self.y_pred_cls) 114 | self.acc = tf.reduce_mean(tf.cast(correct_pred, tf.float32)) 115 | -------------------------------------------------------------------------------- /rnn_model_bert.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | # -*- coding: utf-8 -*- 3 | 4 | import tensorflow as tf 5 | from data.cnews_loader_bert import attention 6 | 7 | 8 | class TRNNConfig(object): 9 | """RNN配置参数""" 10 | 11 | # 模型参数 12 | embedding_dim = 768 # 词向量维度 13 | seq_length = 512 # 序列长度 14 | num_classes = 10 # 类别数 15 | vocab_size = 5000 # 词汇表达小 16 | 17 | num_layers = 1 # 隐藏层层数 18 | hidden_dim = 512 # 隐藏层神经元 19 | rnn = 'gru' # lstm 或 gru 20 | 21 | attention_dim = 512 22 | l2_reg_lambda = 0.01 23 | 24 | dropout_keep_prob = 0.5 # dropout保留比例 25 | learning_rate = 1e-3 # 学习率 26 | 27 | batch_size = 128 # 每批训练大小 28 | num_epochs = 10 # 总迭代轮次 29 | 30 | print_per_batch = 100 # 每多少轮输出一次结果 31 | save_per_batch = 10 # 每多少轮存入tensorboard 32 | 33 | 34 | class TextRNN(object): 35 | """文本分类，RNN模型""" 36 | 37 | def __init__(self, config): 38 | self.config = config 39 | 40 | # 三个待输入的数据 41 | self.input_x = tf.placeholder(tf.float32, [None, self.config.seq_length, self.config.embedding_dim], name='input_x') 42 | self.input_y = tf.placeholder(tf.float32, [None, self.config.num_classes], name='input_y') 43 | self.keep_prob = tf.placeholder(tf.float32, name='keep_prob') 44 | 45 | self.rnn() 46 | 47 | def rnn(self): 48 | """rnn模型""" 49 | 50 | def lstm_cell(): # lstm核 51 | return tf.contrib.rnn.BasicLSTMCell(self.config.hidden_dim, state_is_tuple=True) 52 | 53 | def gru_cell(): # gru核 54 | return tf.contrib.rnn.GRUCell(self.config.hidden_dim) 55 | 56 | def dropout(): # 为每一个rnn核后面加一个dropout层 57 | if (self.config.rnn == 'lstm'): 58 | cell_forward = lstm_cell() 59 | # cell_backward = lstm_cell() 60 | else: 61 | cell_forward = gru_cell() 62 | # cell_backward = gru_cell() 63 | cell_forward = tf.contrib.rnn.DropoutWrapper(cell_forward, output_keep_prob=self.keep_prob) 64 | # cell_backward = tf.contrib.rnn.DropoutWrapper(cell_backward, output_keep_prob=self.keep_prob) 65 | return cell_forward # , cell_backward 66 | 67 | # 词向量映射 68 | # with tf.device('/cpu:0'): 69 | # embedding = tf.get_variable('embedding', [self.config.vocab_size, self.config.embedding_dim]) 70 | # embedding_inputs = tf.nn.embedding_lookup(embedding, self.input_x) 71 | 72 | with tf.name_scope("rnn"): 73 | # 多层rnn网络 74 | # cells = [dropout() for _ in range(self.config.num_layers)] 75 | # rnn_cell = tf.contrib.rnn.MultiRNNCell(cells, state_is_tuple=True) 76 | # 77 | # _outputs, _ = tf.nn.dynamic_rnn(cell=rnn_cell, inputs=embedding_inputs, dtype=tf.float32) 78 | # 不能一起写，要分开写，不能这样 79 | cell_forward = [dropout() for _ in range(self.config.num_layers)] 80 | cell_backward = [dropout() for _ in range(self.config.num_layers)] 81 | lstm_fw = tf.nn.rnn_cell.MultiRNNCell(cell_forward) 82 | lstm_bw = tf.nn.rnn_cell.MultiRNNCell(cell_backward) 83 | _outputs, bi_state = tf.nn.bidirectional_dynamic_rnn(lstm_fw, lstm_bw, self.input_x, dtype=tf.float32) 84 | # _outputs, bi_state = tf.nn.dynamic_rnn(lstm_fw, embedding_inputs, dtype=tf.float32) 85 | # use attention 86 | out_fw, out_bw = _outputs 87 | # 只使用最后一个状态作为句向量 88 | # out_fw = out_fw[:, -1, :] 89 | # out_bw = out_bw[:, -1, :] 90 | 91 | # 使用att(batch, time_step, hidden_unit * layer) 92 | _outputs = tf.concat([out_fw, out_bw], axis=2) 93 | output = attention(_outputs, self.config.attention_dim, self.config.l2_reg_lambda) 94 | # use last unit to represent the sentence 95 | # last = _outputs[:, -1, :] # 取最后一个时序输出作为结果 96 | 97 | with tf.name_scope("score"): 98 | # 全连接层，后面接dropout以及relu激活 99 | fc = tf.layers.dense(output, self.config.hidden_dim, name='fc1') 100 | fc = tf.contrib.layers.dropout(fc, self.keep_prob) 101 | fc = tf.nn.relu(fc) 102 | 103 | # 分类器 104 | self.logits = tf.layers.dense(fc, self.config.num_classes, name='fc2') 105 | self.y_pred_cls = tf.argmax(tf.nn.softmax(self.logits), 1) # 预测类别 106 | 107 | with tf.name_scope("optimize"): 108 | # 损失函数，交叉熵 109 | cross_entropy = tf.nn.softmax_cross_entropy_with_logits(logits=self.logits, labels=self.input_y) 110 | self.loss = tf.reduce_mean(cross_entropy) 111 | # 优化器 112 | self.optim = tf.train.AdamOptimizer(learning_rate=self.config.learning_rate).minimize(self.loss) 113 | 114 | with tf.name_scope("accuracy"): 115 | # 准确率 116 | correct_pred = tf.equal(tf.argmax(self.input_y, 1), self.y_pred_cls) 117 | self.acc = tf.reduce_mean(tf.cast(correct_pred, tf.float32)) 118 | -------------------------------------------------------------------------------- /run_rnn.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | from __future__ import print_function 4 | 5 | import os 6 | import sys 7 | import time 8 | from datetime import timedelta 9 | 10 | import numpy as np 11 | import tensorflow as tf 12 | from sklearn import metrics 13 | 14 | from rnn_model import TRNNConfig, TextRNN 15 | from data.cnews_loader import * 16 | base_dir = 'data/cnews' 17 | train_dir = os.path.join(base_dir, 'cnews.train.txt') 18 | test_dir = os.path.join(base_dir, 'cnews.test.txt') 19 | val_dir = os.path.join(base_dir, 'cnews.val.txt') 20 | vocab_dir = os.path.join(base_dir, 'cnews.vocab.txt') 21 | 22 | save_dir = 'checkpoints/textrnn' 23 | save_path = os.path.join(save_dir, 'best_validation') # 最佳验证结果保存路径 24 | action = 'train' 25 | 26 | def get_time_dif(start_time): 27 | """获取已使用时间""" 28 | end_time = time.time() 29 | time_dif = end_time - start_time 30 | return timedelta(seconds=int(round(time_dif))) 31 | 32 | 33 | def feed_data(x_batch, y_batch, keep_prob): 34 | feed_dict = { 35 | model.input_x: x_batch, 36 | model.input_y: y_batch, 37 | model.keep_prob: keep_prob 38 | } 39 | return feed_dict 40 | 41 | 42 | def evaluate(sess, x_, y_): 43 | """评估在某一数据上的准确率和损失""" 44 | data_len = len(x_) 45 | batch_eval = batch_iter(x_, y_, 128) 46 | total_loss = 0.0 47 | total_acc = 0.0 48 | for x_batch, y_batch in batch_eval: 49 | batch_len = len(x_batch) 50 | feed_dict = feed_data(x_batch, y_batch, 1.0) 51 | loss, acc = sess.run([model.loss, model.acc], feed_dict=feed_dict) 52 | total_loss += loss * batch_len 53 | total_acc += acc * batch_len 54 | 55 | return total_loss / data_len, total_acc / data_len 56 | 57 | 58 | def train(): 59 | print("Configuring TensorBoard and Saver...") 60 | # 配置 Tensorboard，重新训练时，请将tensorboard文件夹删除，不然图会覆盖 61 | tensorboard_dir = 'tensorboard/textrnn' 62 | if not os.path.exists(tensorboard_dir): 63 | os.makedirs(tensorboard_dir) 64 | 65 | tf.summary.scalar("loss", model.loss) 66 | tf.summary.scalar("accuracy", model.acc) 67 | merged_summary = tf.summary.merge_all() 68 | writer = tf.summary.FileWriter(tensorboard_dir) 69 | 70 | # 配置 Saver 71 | saver = tf.train.Saver() 72 | if not os.path.exists(save_dir): 73 | os.makedirs(save_dir) 74 | 75 | print("Loading training and validation data...") 76 | # 载入训练集与验证集 77 | start_time = time.time() 78 | x_train, y_train = process_file(train_dir, word_to_id, cat_to_id, config.seq_length) 79 | x_val, y_val = process_file(val_dir, word_to_id, cat_to_id, config.seq_length) 80 | time_dif = get_time_dif(start_time) 81 | print("Time usage:", time_dif) 82 | 83 | # 创建session 84 | session = tf.Session() 85 | session.run(tf.global_variables_initializer()) 86 | writer.add_graph(session.graph) 87 | 88 | print('Training and evaluating...') 89 | start_time = time.time() 90 | total_batch = 0 # 总批次 91 | best_acc_val = 0.0 # 最佳验证集准确率 92 | last_improved = 0 # 记录上一次提升批次 93 | require_improvement = 1000 # 如果超过1000轮未提升，提前结束训练 94 | 95 | flag = False 96 | for epoch in range(config.num_epochs): 97 | print('Epoch:', epoch + 1) 98 | batch_train = batch_iter(x_train, y_train, config.batch_size) 99 | for x_batch, y_batch in batch_train: 100 | feed_dict = feed_data(x_batch, y_batch, config.dropout_keep_prob) 101 | 102 | if total_batch % config.save_per_batch == 0: 103 | # 每多少轮次将训练结果写入tensorboard scalar 104 | s = session.run(merged_summary, feed_dict=feed_dict) 105 | writer.add_summary(s, total_batch) 106 | 107 | if total_batch % config.print_per_batch == 0: 108 | # 每多少轮次输出在训练集和验证集上的性能 109 | feed_dict[model.keep_prob] = 1.0 110 | loss_train, acc_train = session.run([model.loss, model.acc], feed_dict=feed_dict) 111 | loss_val, acc_val = evaluate(session, x_val, y_val) # todo 112 | 113 | if acc_val > best_acc_val: 114 | # 保存最好结果 115 | best_acc_val = acc_val 116 | last_improved = total_batch 117 | saver.save(sess=session, save_path=save_path) 118 | improved_str = '*' 119 | else: 120 | improved_str = '' 121 | 122 | time_dif = get_time_dif(start_time) 123 | msg = 'Iter: {0:>6}, Train Loss: {1:>6.2}, Train Acc: {2:>7.2%},' \ 124 | + ' Val Loss: {3:>6.2}, Val Acc: {4:>7.2%}, Time: {5} {6}' 125 | print(msg.format(total_batch, loss_train, acc_train, loss_val, acc_val, time_dif, improved_str)) 126 | 127 | session.run(model.optim, feed_dict=feed_dict) # 运行优化 128 | total_batch += 1 129 | 130 | if total_batch - last_improved > require_improvement: 131 | # 验证集正确率长期不提升，提前结束训练 132 | print("No optimization for a long time, auto-stopping...") 133 | flag = True 134 | break # 跳出循环 135 | if flag: # 同上 136 | break 137 | 138 | 139 | def test(): 140 | print("Loading test data...") 141 | start_time = time.time() 142 | x_test, y_test = process_file(test_dir, word_to_id, cat_to_id, config.seq_length) 143 | 144 | session = tf.Session() 145 | session.run(tf.global_variables_initializer()) 146 | saver = tf.train.Saver() 147 | saver.restore(sess=session, save_path=save_path) # 读取保存的模型 148 | 149 | print('Testing...') 150 | loss_test, acc_test = evaluate(session, x_test, y_test) 151 | msg = 'Test Loss: {0:>6.2}, Test Acc: {1:>7.2%}' 152 | print(msg.format(loss_test, acc_test)) 153 | 154 | batch_size = 128 155 | data_len = len(x_test) 156 | num_batch = int((data_len - 1) / batch_size) + 1 157 | 158 | y_test_cls = np.argmax(y_test, 1) 159 | y_pred_cls = np.zeros(shape=len(x_test), dtype=np.int32) # 保存预测结果 160 | for i in range(num_batch): # 逐批次处理 161 | start_id = i * batch_size 162 | end_id = min((i + 1) * batch_size, data_len) 163 | feed_dict = { 164 | model.input_x: x_test[start_id:end_id], 165 | model.keep_prob: 1.0 166 | } 167 | y_pred_cls[start_id:end_id] = session.run(model.y_pred_cls, feed_dict=feed_dict) 168 | 169 | # 评估 170 | print("Precision, Recall and F1-Score...") 171 | print(metrics.classification_report(y_test_cls, y_pred_cls, target_names=categories)) 172 | 173 | # 混淆矩阵 174 | print("Confusion Matrix...") 175 | cm = metrics.confusion_matrix(y_test_cls, y_pred_cls) 176 | print(cm) 177 | 178 | time_dif = get_time_dif(start_time) 179 | print("Time usage:", time_dif) 180 | 181 | 182 | if __name__ == '__main__': 183 | # if len(sys.argv) != 2 or sys.argv[1] not in ['train', 'test']: 184 | # raise ValueError("""usage: python run_rnn.py [train / test]""") 185 | 186 | print('Configuring RNN model...') 187 | config = TRNNConfig() 188 | if not os.path.exists(vocab_dir): # 如果不存在词汇表，重建 189 | build_vocab(train_dir, vocab_dir, config.vocab_size) 190 | categories, cat_to_id = read_category() 191 | words, word_to_id = read_vocab(vocab_dir) 192 | config.vocab_size = len(words) 193 | model = TextRNN(config) 194 | 195 | if action == 'train': 196 | train() 197 | else: 198 | test() 199 | -------------------------------------------------------------------------------- /run_rnn_bert.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | from __future__ import print_function 4 | 5 | import os 6 | import sys 7 | import time 8 | from datetime import timedelta 9 | from bert_serving.client import BertClient 10 | 11 | import numpy as np 12 | import tensorflow as tf 13 | from sklearn import metrics 14 | 15 | from rnn_model_bert import TRNNConfig, TextRNN 16 | from data.cnews_loader_bert import * 17 | base_dir = 'data/cnews' 18 | train_dir = os.path.join(base_dir, 'cnews.train.txt') 19 | test_dir = os.path.join(base_dir, 'cnews.test.txt') 20 | val_dir = os.path.join(base_dir, 'cnews.val.txt') 21 | vocab_dir = os.path.join(base_dir, 'cnews.vocab.txt') 22 | 23 | save_dir = 'checkpoints/textrnn_bert' 24 | save_path = os.path.join(save_dir, 'best_validation') # 最佳验证结果保存路径 25 | action = 'train' 26 | 27 | def get_time_dif(start_time): 28 | """获取已使用时间""" 29 | end_time = time.time() 30 | time_dif = end_time - start_time 31 | return timedelta(seconds=int(round(time_dif))) 32 | 33 | 34 | def feed_data(x_batch, y_batch, keep_prob): 35 | feed_dict = { 36 | model.input_x: x_batch, 37 | model.input_y: y_batch, 38 | model.keep_prob: keep_prob 39 | } 40 | return feed_dict 41 | 42 | 43 | def evaluate(sess, x_, y_): 44 | """评估在某一数据上的准确率和损失""" 45 | data_len = len(x_) 46 | batch_eval = batch_iter(x_, y_, 128) 47 | total_loss = 0.0 48 | total_acc = 0.0 49 | for x_batch, y_batch in batch_eval: 50 | batch_len = len(x_batch) 51 | x_batch = list(x_batch) 52 | feed_dict = feed_data(x_batch, y_batch, 1.0) 53 | loss, acc = sess.run([model.loss, model.acc], feed_dict=feed_dict) 54 | total_loss += loss * batch_len 55 | total_acc += acc * batch_len 56 | 57 | return total_loss / data_len, total_acc / data_len 58 | 59 | 60 | def train(): 61 | print("Configuring TensorBoard and Saver...") 62 | # 配置 Tensorboard，重新训练时，请将tensorboard文件夹删除，不然图会覆盖 63 | tensorboard_dir = 'tensorboard/textrnn' 64 | if not os.path.exists(tensorboard_dir): 65 | os.makedirs(tensorboard_dir) 66 | 67 | tf.summary.scalar("loss", model.loss) 68 | tf.summary.scalar("accuracy", model.acc) 69 | merged_summary = tf.summary.merge_all() 70 | writer = tf.summary.FileWriter(tensorboard_dir) 71 | 72 | # 配置 Saver 73 | saver = tf.train.Saver() 74 | if not os.path.exists(save_dir): 75 | os.makedirs(save_dir) 76 | 77 | print("Loading training and validation data...") 78 | # 载入训练集与验证集 79 | start_time = time.time() 80 | x_train, y_train = process_file(train_dir, word_to_id, cat_to_id, config.seq_length) 81 | x_train = list(x_train) 82 | x_val, y_val = process_file(val_dir, word_to_id, cat_to_id, config.seq_length) 83 | x_val = list(x_val) 84 | 85 | bc = BertClient() 86 | x_val = bc.encode(x_val) 87 | 88 | time_dif = get_time_dif(start_time) 89 | print("Time usage:", time_dif) 90 | 91 | # 创建session 92 | session = tf.Session() 93 | session.run(tf.global_variables_initializer()) 94 | writer.add_graph(session.graph) 95 | 96 | print('Training and evaluating...') 97 | start_time = time.time() 98 | total_batch = 0 # 总批次 99 | best_acc_val = 0.0 # 最佳验证集准确率 100 | last_improved = 0 # 记录上一次提升批次 101 | require_improvement = 1000 # 如果超过1000轮未提升，提前结束训练 102 | 103 | flag = False 104 | for epoch in range(config.num_epochs): 105 | print('Epoch:', epoch + 1) 106 | batch_train = batch_iter(x_train, y_train, config.batch_size) 107 | for x_batch, y_batch in batch_train: 108 | x_batch = list(x_batch) 109 | x_batch = bc.encode(x_batch) 110 | feed_dict = feed_data(x_batch, y_batch, config.dropout_keep_prob) 111 | 112 | if total_batch % config.save_per_batch == 0: 113 | # 每多少轮次将训练结果写入tensorboard scalar 114 | s = session.run(merged_summary, feed_dict=feed_dict) 115 | writer.add_summary(s, total_batch) 116 | 117 | if total_batch % config.print_per_batch == 0: 118 | # 每多少轮次输出在训练集和验证集上的性能 119 | feed_dict[model.keep_prob] = 1.0 120 | loss_train, acc_train = session.run([model.loss, model.acc], feed_dict=feed_dict) 121 | loss_val, acc_val = evaluate(session, x_val, y_val) # todo 122 | 123 | if acc_val > best_acc_val: 124 | # 保存最好结果 125 | best_acc_val = acc_val 126 | last_improved = total_batch 127 | saver.save(sess=session, save_path=save_path) 128 | improved_str = '*' 129 | else: 130 | improved_str = '' 131 | 132 | time_dif = get_time_dif(start_time) 133 | msg = 'Iter: {0:>6}, Train Loss: {1:>6.2}, Train Acc: {2:>7.2%},' \ 134 | + ' Val Loss: {3:>6.2}, Val Acc: {4:>7.2%}, Time: {5} {6}' 135 | print(msg.format(total_batch, loss_train, acc_train, loss_val, acc_val, time_dif, improved_str)) 136 | 137 | session.run(model.optim, feed_dict=feed_dict) # 运行优化 138 | total_batch += 1 139 | 140 | if total_batch - last_improved > require_improvement: 141 | # 验证集正确率长期不提升，提前结束训练 142 | print("No optimization for a long time, auto-stopping...") 143 | flag = True 144 | break # 跳出循环 145 | if flag: # 同上 146 | break 147 | 148 | 149 | def test(): 150 | print("Loading test data...") 151 | start_time = time.time() 152 | x_test, y_test = process_file(test_dir, word_to_id, cat_to_id, config.seq_length) 153 | 154 | session = tf.Session() 155 | session.run(tf.global_variables_initializer()) 156 | saver = tf.train.Saver() 157 | saver.restore(sess=session, save_path=save_path) # 读取保存的模型 158 | 159 | print('Testing...') 160 | loss_test, acc_test = evaluate(session, x_test, y_test) 161 | msg = 'Test Loss: {0:>6.2}, Test Acc: {1:>7.2%}' 162 | print(msg.format(loss_test, acc_test)) 163 | 164 | batch_size = 128 165 | data_len = len(x_test) 166 | num_batch = int((data_len - 1) / batch_size) + 1 167 | 168 | y_test_cls = np.argmax(y_test, 1) 169 | y_pred_cls = np.zeros(shape=len(x_test), dtype=np.int32) # 保存预测结果 170 | for i in range(num_batch): # 逐批次处理 171 | start_id = i * batch_size 172 | end_id = min((i + 1) * batch_size, data_len) 173 | feed_dict = { 174 | model.input_x: x_test[start_id:end_id], 175 | model.keep_prob: 1.0 176 | } 177 | y_pred_cls[start_id:end_id] = session.run(model.y_pred_cls, feed_dict=feed_dict) 178 | 179 | # 评估 180 | print("Precision, Recall and F1-Score...") 181 | print(metrics.classification_report(y_test_cls, y_pred_cls, target_names=categories)) 182 | 183 | # 混淆矩阵 184 | print("Confusion Matrix...") 185 | cm = metrics.confusion_matrix(y_test_cls, y_pred_cls) 186 | print(cm) 187 | 188 | time_dif = get_time_dif(start_time) 189 | print("Time usage:", time_dif) 190 | 191 | 192 | if __name__ == '__main__': 193 | # if len(sys.argv) != 2 or sys.argv[1] not in ['train', 'test']: 194 | # raise ValueError("""usage: python run_rnn.py [train / test]""") 195 | 196 | print('Configuring RNN model...') 197 | config = TRNNConfig() 198 | if not os.path.exists(vocab_dir): # 如果不存在词汇表，重建 199 | build_vocab(train_dir, vocab_dir, config.vocab_size) 200 | categories, cat_to_id = read_category() 201 | words, word_to_id = read_vocab(vocab_dir) 202 | config.vocab_size = len(words) 203 | model = TextRNN(config) 204 | 205 | if action == 'train': 206 | train() 207 | else: 208 | test() 209 | -------------------------------------------------------------------------------- /test.txt: -------------------------------------------------------------------------------- 1 | 4 2 5 2 | 10 5 2 3 | 5 8 7 4 | 1 1 1 5 | 2 3 2 6 | 3 6 9 7 | 11 9 2 8 | 1 4 6 9 | 9 1 7 10 | 5 6 7 11 | -------------------------------------------------------------------------------- /train_x.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pengwei-iie/Bert-THUCNews/a20749225091533b530f0e539bfaacbd3524fe99/train_x.txt --------------------------------------------------------------------------------