├── .gitignore ├── LICENSE ├── README.md ├── cchess_alphazero ├── README.md ├── __init__.py ├── agent │ ├── __init__.py │ ├── api.py │ ├── model.py │ └── player.py ├── config.py ├── configs │ ├── __init__.py │ ├── distribute.py │ ├── mini.py │ └── normal.py ├── environment │ ├── README.md │ ├── __init__.py │ ├── chessboard.py │ ├── chessman.py │ ├── env.py │ ├── light_env │ │ ├── chessboard.py │ │ └── common.py │ ├── lookup_tables.py │ └── static_env.py ├── lib │ ├── data_helper.py │ ├── elo_helper.py │ ├── logger.py │ ├── model_helper.py │ ├── tf_util.py │ └── web_helper.py ├── manager.py ├── play_games │ ├── images │ │ ├── CANVAS.GIF │ │ ├── DELICATE │ │ │ ├── BA.GIF │ │ │ ├── BAS.GIF │ │ │ ├── BB.GIF │ │ │ ├── BBS.GIF │ │ │ ├── BC.GIF │ │ │ ├── BCS.GIF │ │ │ ├── BK.GIF │ │ │ ├── BKM.GIF │ │ │ ├── BKS.GIF │ │ │ ├── BN.GIF │ │ │ ├── BNS.GIF │ │ │ ├── BP.GIF │ │ │ ├── BPS.GIF │ │ │ ├── BR.GIF │ │ │ ├── BRS.GIF │ │ │ ├── OO.GIF │ │ │ ├── OOS.GIF │ │ │ ├── RA.GIF │ │ │ ├── RAS.GIF │ │ │ ├── RB.GIF │ │ │ ├── RBS.GIF │ │ │ ├── RC.GIF │ │ │ ├── RCS.GIF │ │ │ ├── RK.GIF │ │ │ ├── RKM.GIF │ │ │ ├── RKS.GIF │ │ │ ├── RN.GIF │ │ │ ├── RNS.GIF │ │ │ ├── RP.GIF │ │ │ ├── RPS.GIF │ │ │ ├── RR.GIF │ │ │ └── RRS.GIF │ │ ├── DROPS.GIF │ │ ├── GREEN.GIF │ │ ├── POLISH │ │ │ ├── BA.GIF │ │ │ ├── BAS.GIF │ │ │ ├── BB.GIF │ │ │ ├── BBS.GIF │ │ │ ├── BC.GIF │ │ │ ├── BCS.GIF │ │ │ ├── BK.GIF │ │ │ ├── BKM.GIF │ │ │ ├── BKS.GIF │ │ │ ├── BN.GIF │ │ │ ├── BNS.GIF │ │ │ ├── BP.GIF │ │ │ ├── BPS.GIF │ │ │ ├── BR.GIF │ │ │ ├── BRS.GIF │ │ │ ├── OO.GIF │ │ │ ├── OOS.GIF │ │ │ ├── RA.GIF │ │ │ ├── RAS.GIF │ │ │ ├── RB.GIF │ │ │ ├── RBS.GIF │ │ │ ├── RC.GIF │ │ │ ├── RCS.GIF │ │ │ ├── RK.GIF │ │ │ ├── RKM.GIF │ │ │ ├── RKS.GIF │ │ │ ├── RN.GIF │ │ │ ├── RNS.GIF │ │ │ ├── RP.GIF │ │ │ ├── RPS.GIF │ │ │ ├── RR.GIF │ │ │ └── RRS.GIF │ │ ├── QIANHONG.GIF │ │ ├── SHEET.GIF │ │ ├── SKELETON.GIF │ │ ├── WHITE.GIF │ │ ├── WOOD.GIF │ │ └── WOOD │ │ │ ├── BA.GIF │ │ │ ├── BAS.GIF │ │ │ ├── BB.GIF │ │ │ ├── BBS.GIF │ │ │ ├── BC.GIF │ │ │ ├── BCS.GIF │ │ │ ├── BK.GIF │ │ │ ├── BKM.GIF │ │ │ ├── BKS.GIF │ │ │ ├── BN.GIF │ │ │ ├── BNS.GIF │ │ │ ├── BP.GIF │ │ │ ├── BPS.GIF │ │ │ ├── BR.GIF │ │ │ ├── BRS.GIF │ │ │ ├── OO.GIF │ │ │ ├── OOS.GIF │ │ │ ├── RA.GIF │ │ │ ├── RAS.GIF │ │ │ ├── RB.GIF │ │ │ ├── RBS.GIF │ │ │ ├── RC.GIF │ │ │ ├── RCS.GIF │ │ │ ├── RK.GIF │ │ │ ├── RKM.GIF │ │ │ ├── RKS.GIF │ │ │ ├── RN.GIF │ │ │ ├── RNS.GIF │ │ │ ├── RP.GIF │ │ │ ├── RPS.GIF │ │ │ ├── RR.GIF │ │ │ └── RRS.GIF │ ├── ob_self_play.py │ ├── play.py │ ├── play_cli.py │ ├── test_cli_game.py │ └── test_window_game.py ├── run.py ├── test.py ├── uci.py └── worker │ ├── __init__.py │ ├── compute_elo.py │ ├── compute_elo_windows.py │ ├── evaluator.py │ ├── optimize.py │ ├── play_with_ucci_engine.py │ ├── self_play.py │ ├── self_play_windows.py │ ├── sl.py │ └── sl_onegreen.py ├── colaboratory ├── eval.py ├── run.py └── test.py ├── data └── model │ ├── model_128_l1_config.json │ ├── model_128f.json │ ├── model_192x10_config.json │ ├── model_256f.json │ ├── model_best_config.json │ └── model_best_weight.h5 ├── elo.png ├── freeze ├── evaluate.py ├── play_games.py ├── play_games.spec └── run_self_play.py ├── model.png ├── requirements.txt └── screenshots └── board.png /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | .DS_Store 3 | data/* 4 | logs/* 5 | *.ttc 6 | Icon* 7 | .vscode 8 | *.icloud -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # 中国象棋Zero（CCZero） 2 | 3 | 4 | App Icon 5 | 6 | 7 | ## About 8 | 9 | Chinese Chess reinforcement learning by [AlphaZero](https://arxiv.org/abs/1712.01815) methods. 10 | 11 | This project is based on these main resources: 12 | 1. DeepMind's Oct 19th publication: [Mastering the Game of Go without Human Knowledge](https://www.nature.com/articles/nature24270.epdf?author_access_token=VJXbVjaSHxFoctQQ4p2k4tRgN0jAjWel9jnR3ZoTv0PVW4gB86EEpGqTRDtpIz-2rmo8-KG06gqVobU5NSCFeHILHcVFUeMsbvwS-lxjqQGg98faovwjxeTUgZAUMnRQ). 13 | 2. The **great** Reversi/Chess/Chinese chess development of the DeepMind ideas that @mokemokechicken/@Akababa/@TDteach did in their repo: https://github.com/mokemokechicken/reversi-alpha-zero, https://github.com/Akababa/Chess-Zero, https://github.com/TDteach/AlphaZero_ChineseChess 14 | 3. A Chinese chess engine with gui: https://github.com/mm12432/MyChess 15 | 16 | 17 | ## Help to train 18 | 19 | In order to build a strong chinese chess AI following the same type of techniques as AlphaZero, we need to do this with a distributed project, as it requires a huge amount of computations. 20 | 21 | If you want to join us to build the best chinese chess AI in the world: 22 | 23 | * For instructions, see [wiki](https://github.com/NeymarL/ChineseChess-AlphaZero/wiki) 24 | * For live status, see https://cczero.org 25 | 26 | ![elo](elo.png) 27 | 28 | 29 | ## Environment 30 | 31 | * Python 3.6.3 32 | * tensorflow-gpu: 1.3.0 33 | * Keras: 2.0.8 34 | 35 | 36 | ## Modules 37 | 38 | ### Reinforcement Learning 39 | 40 | This AlphaZero implementation consists of two workers: `self` and `opt`. 41 | 42 | * `self` is Self-Play to generate training data by self-play using BestModel. 43 | * `opt` is Trainer to train model, and generate new models. 44 | 45 | For the sake of faster training, another two workers are involved: 46 | 47 | * `sl` is Supervised learning to train data crawled from the Internet. 48 | * `eval` is Evaluator to evaluate the NextGenerationModel with the current BestModel. 49 | 50 | ### Built-in GUI 51 | 52 | Requirement: pygame 53 | 54 | ```bash 55 | python cchess_alphazero/run.py play 56 | ``` 57 | 58 | **Screenshots** 59 | 60 | ![board](screenshots/board.png) 61 | 62 | You can choose different board/piece styles and sides, see [play with human](#play-with-human). 63 | 64 | 65 | ## How to use 66 | 67 | ### Setup 68 | 69 | ### install libraries 70 | ```bash 71 | pip install -r requirements.txt 72 | ``` 73 | 74 | If you want to use CPU only, replace `tensorflow-gpu` with `tensorflow` in `requirements.txt`. 75 | 76 | Make sure Keras is using Tensorflow and you have Python 3.6.3+. 77 | 78 | ### Configuration 79 | 80 | **PlayDataConfig** 81 | 82 | * `nb_game_in_file, max_file_num`: The max game number of training data is `nb_game_in_file * max_file_num`. 83 | 84 | **PlayConfig, PlayWithHumanConfig** 85 | 86 | * `simulation_num_per_move` : MCTS number per move. 87 | * `c_puct`: balance parameter of value network and policy network in MCTS. 88 | * `search_threads`: balance parameter of speed and accuracy in MCTS. 89 | * `dirichlet_alpha`: random parameter in self-play. 90 | 91 | ### Full Usage 92 | 93 | ``` 94 | usage: run.py [-h] [--new] [--type TYPE] [--total-step TOTAL_STEP] 95 | [--ai-move-first] [--cli] [--gpu GPU] [--onegreen] [--skip SKIP] 96 | [--ucci] [--piece-style {WOOD,POLISH,DELICATE}] 97 | [--bg-style {CANVAS,DROPS,GREEN,QIANHONG,SHEET,SKELETON,WHITE,WOOD}] 98 | [--random {none,small,medium,large}] [--distributed] [--elo] 99 | {self,opt,eval,play,eval,sl,ob} 100 | 101 | positional arguments: 102 | {self,opt,eval,play,eval,sl,ob} 103 | what to do 104 | 105 | optional arguments: 106 | -h, --help show this help message and exit 107 | --new run from new best model 108 | --type TYPE use normal setting 109 | --total-step TOTAL_STEP 110 | set TrainerConfig.start_total_steps 111 | --ai-move-first set human or AI move first 112 | --cli play with AI with CLI, default with GUI 113 | --gpu GPU device list 114 | --onegreen train sl work with onegreen data 115 | --skip SKIP skip games 116 | --ucci play with ucci engine instead of self play 117 | --piece-style {WOOD,POLISH,DELICATE} 118 | choose a style of piece 119 | --bg-style {CANVAS,DROPS,GREEN,QIANHONG,SHEET,SKELETON,WHITE,WOOD} 120 | choose a style of board 121 | --random {none,small,medium,large} 122 | choose a style of randomness 123 | --distributed whether upload/download file from remote server 124 | --elo whether to compute elo score 125 | ``` 126 | 127 | ### Self-Play 128 | 129 | ``` 130 | python cchess_alphazero/run.py self 131 | ``` 132 | 133 | When executed, self-play will start using BestModel. If the BestModel does not exist, new random model will be created and become BestModel. Self-play records will store in `data/play_record` and BestMode will store in `data/model`. 134 | 135 | options 136 | 137 | * `--new`: create new BestModel 138 | * `--type mini`: use mini config, (see `cchess_alphazero/configs/mini.py`) 139 | * `--gpu '1'`: specify which gpu to use 140 | * `--ucci`: whether to play with ucci engine (rather than self play, see `cchess_alphazero/worker/play_with_ucci_engine.py`) 141 | * `--distributed`: run self play in distributed mode which means it will upload the play data to the remote server and download latest model from it 142 | 143 | **Note1**: To help training, you should run `python cchess_alphazero/run.py --type distribute --distributed self` (and do not change the configuration file `configs/distribute.py`), for more info, see [wiki](https://github.com/NeymarL/ChineseChess-AlphaZero/wiki/For-Developers). 144 | 145 | **Note2**: If you want to view the self-play records in GUI, see [wiki](https://github.com/NeymarL/ChineseChess-AlphaZero/wiki/View-self-play-games-in-GUI). 146 | 147 | ### Trainer 148 | 149 | ``` 150 | python cchess_alphazero/run.py opt 151 | ``` 152 | 153 | When executed, Training will start. The current BestModel will be loaded. Trained model will be saved every epoch as new BestModel. 154 | 155 | options 156 | 157 | * `--type mini`: use mini config, (see `cchess_alphazero/configs/mini.py`) 158 | * `--total-step TOTAL_STEP`: specify total step(mini-batch) numbers. The total step affects learning rate of training. 159 | * `--gpu '1'`: specify which gpu to use 160 | 161 | **View training log in Tensorboard** 162 | 163 | ``` 164 | tensorboard --logdir logs/ 165 | ``` 166 | 167 | And access `http://:6006/`. 168 | 169 | ### Play with human 170 | 171 | **Run with built-in GUI** 172 | 173 | ``` 174 | python cchess_alphazero/run.py play 175 | ``` 176 | 177 | When executed, the BestModel will be loaded to play against human. 178 | 179 | options 180 | 181 | * `--ai-move-first`: if set this option, AI will move first, otherwise human move first. 182 | * `--type mini`: use mini config, (see `cchess_alphazero/configs/mini.py`) 183 | * `--gpu '1'`: specify which gpu to use 184 | * `--piece-style WOOD`: choose a piece style, default is `WOOD` 185 | * `--bg-style CANVAS`: choose a board style, default is `CANVAS` 186 | * `--cli`: if set this flag, play with AI in a cli environment rather than gui 187 | 188 | **Note**: Before you start, you need to download/find a font file (`.ttc`) and rename it as `PingFang.ttc`, then put it into `cchess_alphazero/play_games`. I have removed the font file from this repo because it's too big, but you can download it from [here](http://alphazero.52coding.com.cn/PingFang.ttc). 189 | 190 | You can also download Windows executable directly from [here](https://pan.baidu.com/s/1uE_zmkn0x9Be_olRL9U9cQ). For more information, see [wiki](https://github.com/NeymarL/ChineseChess-AlphaZero/wiki/For-Non-Developers#%E4%B8%8B%E6%A3%8B). 191 | 192 | **UCI mode** 193 | 194 | ``` 195 | python cchess_alphazero/uci.py 196 | ``` 197 | 198 | If you want to play in general GUIs such as '冰河五四', you can download the Windows executable [here](https://share.weiyun.com/5cK50Z4). For more information, see [wiki](https://github.com/NeymarL/ChineseChess-AlphaZero/wiki/For-Non-Developers#%E4%B8%8B%E6%A3%8B). 199 | 200 | ### Evaluator 201 | 202 | ``` 203 | python cchess_alphazero/run.py eval 204 | ``` 205 | 206 | When executed, evaluate the NextGenerationModel with the current BestModel. If the NextGenerationModel does not exist, worker will wait until it exists and check every 5 minutes. 207 | 208 | options 209 | 210 | * `--type mini`: use mini config, (see `cchess_alphazero/configs/mini.py`) 211 | * `--gpu '1'`: specify which gpu to use 212 | 213 | ### Supervised Learning 214 | 215 | ``` 216 | python cchess_alphazero/run.py sl 217 | ``` 218 | 219 | When executed, Training will start. The current SLBestModel will be loaded. Tranined model will be saved every epoch as new SLBestModel. 220 | 221 | *About the data* 222 | 223 | I have two data sources, one is downloaded from https://wx.jcloud.com/market/packet/10479 ; the other is crawled from http://game.onegreen.net/chess/Index.html (with option --onegreen). 224 | 225 | options 226 | 227 | * `--type mini`: use mini config, (see `cchess_alphazero/configs/mini.py`) 228 | * `--gpu '1'`: specify which gpu to use 229 | * `--onegreen`: if set the flag, `sl_onegreen` worker will start to train data crawled from `game.onegreen.net` 230 | * `--skip SKIP`: if set this flag, games whoses index is less than `SKIP` would not be used to train (only valid when `onegreen` flag is set) 231 | -------------------------------------------------------------------------------- /cchess_alphazero/README.md: -------------------------------------------------------------------------------- 1 | # Code Structures 2 | 3 | ``` 4 | ├── cchess_alphazero 5 | │ ├── agent : the AI (AlphaZero) agent 6 | │ │ ├── api.py : neural networks' prediction api 7 | │ │ ├── model.py : policy & value network model 8 | │ │ └── player.py : the final agent that play with neural network and MCTS 9 | │ ├── configs : different types of configuration 10 | │ │ ├── mini.py 11 | │ │ └── normal.py 12 | │ ├── environment : a Chinese Chess engine 13 | │ │ │── light_env : a lightweight chinese chess engine (for training) 14 | │ │ │ ├── chessboard.py 15 | │ │ │ ├── common.py 16 | │ │ │ └── chessman.py.py 17 | │ │ ├── chessboard.py 18 | │ │ ├── chessman.py 19 | │ │ ├── env.py : the environment api of the engine (mainly used in play-with-human and previous MCTS player) 20 | │ │ ├── static_env.py : a static chess engine which does not store the board (used in new MCTS player) 21 | │ │ └── lookup_tables.py 22 | │ ├── lib : helper functions 23 | │ │ ├── data_helper.py : load & save data 24 | │ │ ├── logger.py : setup logger 25 | │ │ ├── model_helper.py : load & save model 26 | │ │ └── tf_util.py : setup tf session 27 | │ ├── play_games : play with human 28 | │ │ ├── images : game materials 29 | │ │ ├── play.py : AI vs human with gui 30 | │ │ ├── play_cli.py : AI vs human with cli 31 | │ │ ├── ob_self_play.py : observe AI vs AI with cli 32 | │ │ ├── test_cli_game.py : human vs human with cli 33 | │ │ └── test_window_game.py : human vs human with gui 34 | │ ├── worker 35 | │ │ ├── self_play.py : self play worker 36 | │ │ ├── self_play_windows.py : self play worker of Windows os 37 | │ │ ├── compute_elo.py : evaluate next generation model and compute it's elo 38 | │ │ ├── optimize.py : trainer 39 | │ │ ├── sl.py : supervised learning worker 40 | │ │ ├── sl_onegreen.py : supervised learning worker which train data crawled from game.onegreen.net 41 | │ │ ├── play_with_ucci_engine.py : play with an ucci engine rather than self play 42 | │ │ └── evaluator.py : evaluate next generation model with current best model 43 | │ ├── config.py : setup configuration 44 | │ ├── manager.py : manage to start which worker 45 | │ ├── run.py : start interface 46 | │ ├── uci.py : for UCI protocal 47 | ├── └── test.py : for debug and test 48 | 49 | ``` 50 | -------------------------------------------------------------------------------- /cchess_alphazero/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/__init__.py -------------------------------------------------------------------------------- /cchess_alphazero/agent/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/agent/__init__.py -------------------------------------------------------------------------------- /cchess_alphazero/agent/api.py: -------------------------------------------------------------------------------- 1 | from multiprocessing import connection, Pipe 2 | from threading import Thread 3 | 4 | import os 5 | import numpy as np 6 | import shutil 7 | 8 | from cchess_alphazero.config import Config 9 | from cchess_alphazero.lib.model_helper import load_best_model_weight, need_to_reload_best_model_weight 10 | from cchess_alphazero.lib.web_helper import http_request, download_file 11 | from time import time 12 | from logging import getLogger 13 | 14 | logger = getLogger(__name__) 15 | 16 | class CChessModelAPI: 17 | 18 | def __init__(self, config: Config, agent_model): 19 | self.agent_model = agent_model # CChessModel 20 | self.pipes = [] # use for communication between processes/threads 21 | self.config = config 22 | self.need_reload = True 23 | self.done = False 24 | 25 | def start(self, need_reload=True): 26 | self.need_reload = need_reload 27 | prediction_worker = Thread(target=self.predict_batch_worker, name="prediction_worker") 28 | prediction_worker.daemon = True 29 | prediction_worker.start() 30 | 31 | def get_pipe(self, need_reload=True): 32 | me, you = Pipe() 33 | self.pipes.append(me) 34 | self.need_reload = need_reload 35 | return you 36 | 37 | def predict_batch_worker(self): 38 | if self.config.internet.distributed and self.need_reload: 39 | self.try_reload_model_from_internet() 40 | last_model_check_time = time() 41 | while not self.done: 42 | if last_model_check_time + 600 < time() and self.need_reload: 43 | self.try_reload_model() 44 | last_model_check_time = time() 45 | ready = connection.wait(self.pipes, timeout=0.001) 46 | if not ready: 47 | continue 48 | data, result_pipes, data_len = [], [], [] 49 | for pipe in ready: 50 | while pipe.poll(): 51 | try: 52 | tmp = pipe.recv() 53 | except EOFError as e: 54 | logger.error(f"EOF error: {e}") 55 | pipe.close() 56 | else: 57 | data.extend(tmp) 58 | data_len.append(len(tmp)) 59 | result_pipes.append(pipe) 60 | if not data: 61 | continue 62 | data = np.asarray(data, dtype=np.float32) 63 | with self.agent_model.graph.as_default(): 64 | policy_ary, value_ary = self.agent_model.model.predict_on_batch(data) 65 | buf = [] 66 | k, i = 0, 0 67 | for p, v in zip(policy_ary, value_ary): 68 | buf.append((p, float(v))) 69 | k += 1 70 | if k >= data_len[i]: 71 | result_pipes[i].send(buf) 72 | buf = [] 73 | k = 0 74 | i += 1 75 | 76 | def try_reload_model(self, config_file=None): 77 | if config_file: 78 | config_path = os.path.join(self.config.resource.model_dir, config_file) 79 | shutil.copy(config_path, self.config.resource.model_best_config_path) 80 | try: 81 | if self.config.internet.distributed and not config_file: 82 | self.try_reload_model_from_internet() 83 | else: 84 | if self.need_reload and need_to_reload_best_model_weight(self.agent_model): 85 | with self.agent_model.graph.as_default(): 86 | load_best_model_weight(self.agent_model) 87 | except Exception as e: 88 | logger.error(e) 89 | 90 | def try_reload_model_from_internet(self, config_file=None): 91 | response = http_request(self.config.internet.get_latest_digest) 92 | if response is None: 93 | logger.error(f"无法连接到远程服务器！请检查网络连接，并重新打开客户端") 94 | return 95 | digest = response['data']['digest'] 96 | 97 | if digest != self.agent_model.fetch_digest(self.config.resource.model_best_weight_path): 98 | logger.info(f"正在下载最新权重，请稍后...") 99 | if download_file(self.config.internet.download_url, self.config.resource.model_best_weight_path): 100 | logger.info(f"权重下载完毕！开始训练...") 101 | try: 102 | with self.agent_model.graph.as_default(): 103 | load_best_model_weight(self.agent_model) 104 | except ValueError as e: 105 | logger.error(f"权重架构不匹配，自动重新加载 {e}") 106 | self.try_reload_model(config_file='model_192x10_config.json') 107 | except Exception as e: 108 | logger.error(f"加载权重发生错误：{e}，稍后重新下载") 109 | os.remove(self.config.resource.model_best_weight_path) 110 | self.try_reload_model_from_internet() 111 | else: 112 | logger.error(f"权重下载失败！请检查网络连接，并重新打开客户端") 113 | else: 114 | logger.info(f"检查完毕，权重未更新") 115 | 116 | def close(self): 117 | self.done = True 118 | -------------------------------------------------------------------------------- /cchess_alphazero/agent/model.py: -------------------------------------------------------------------------------- 1 | import hashlib 2 | import json 3 | import os 4 | from logging import getLogger 5 | 6 | import tensorflow as tf 7 | 8 | from keras.engine.topology import Input 9 | from keras.engine.training import Model 10 | from keras.layers.convolutional import Conv2D 11 | from keras.layers.core import Activation, Dense, Flatten 12 | from keras.layers.merge import Add 13 | from keras.layers.normalization import BatchNormalization 14 | from keras.regularizers import l2 15 | 16 | from cchess_alphazero.agent.api import CChessModelAPI 17 | from cchess_alphazero.config import Config 18 | from cchess_alphazero.environment.lookup_tables import ActionLabelsRed, ActionLabelsBlack 19 | 20 | logger = getLogger(__name__) 21 | 22 | class CChessModel: 23 | 24 | def __init__(self, config: Config): 25 | self.config = config 26 | self.model = None # type: Model 27 | self.digest = None 28 | self.n_labels = len(ActionLabelsRed) 29 | self.graph = None 30 | self.api = None 31 | 32 | def build(self): 33 | mc = self.config.model 34 | in_x = x = Input((14, 10, 9)) # 14 x 10 x 9 35 | 36 | # (batch, channels, height, width) 37 | x = Conv2D(filters=mc.cnn_filter_num, kernel_size=mc.cnn_first_filter_size, padding="same", 38 | data_format="channels_first", use_bias=False, kernel_regularizer=l2(mc.l2_reg), 39 | name="input_conv-"+str(mc.cnn_first_filter_size)+"-"+str(mc.cnn_filter_num))(x) 40 | x = BatchNormalization(axis=1, name="input_batchnorm")(x) 41 | x = Activation("relu", name="input_relu")(x) 42 | 43 | for i in range(mc.res_layer_num): 44 | x = self._build_residual_block(x, i + 1) 45 | 46 | res_out = x 47 | 48 | # for policy output 49 | x = Conv2D(filters=4, kernel_size=1, data_format="channels_first", use_bias=False, 50 | kernel_regularizer=l2(mc.l2_reg), name="policy_conv-1-2")(res_out) 51 | x = BatchNormalization(axis=1, name="policy_batchnorm")(x) 52 | x = Activation("relu", name="policy_relu")(x) 53 | x = Flatten(name="policy_flatten")(x) 54 | policy_out = Dense(self.n_labels, kernel_regularizer=l2(mc.l2_reg), activation="softmax", name="policy_out")(x) 55 | 56 | # for value output 57 | x = Conv2D(filters=2, kernel_size=1, data_format="channels_first", use_bias=False, 58 | kernel_regularizer=l2(mc.l2_reg), name="value_conv-1-4")(res_out) 59 | x = BatchNormalization(axis=1, name="value_batchnorm")(x) 60 | x = Activation("relu",name="value_relu")(x) 61 | x = Flatten(name="value_flatten")(x) 62 | x = Dense(mc.value_fc_size, kernel_regularizer=l2(mc.l2_reg), activation="relu", name="value_dense")(x) 63 | value_out = Dense(1, kernel_regularizer=l2(mc.l2_reg), activation="tanh", name="value_out")(x) 64 | 65 | self.model = Model(in_x, [policy_out, value_out], name="cchess_model") 66 | self.graph = tf.get_default_graph() 67 | 68 | def _build_residual_block(self, x, index): 69 | mc = self.config.model 70 | in_x = x 71 | res_name = "res" + str(index) 72 | x = Conv2D(filters=mc.cnn_filter_num, kernel_size=mc.cnn_filter_size, padding="same", 73 | data_format="channels_first", use_bias=False, kernel_regularizer=l2(mc.l2_reg), 74 | name=res_name+"_conv1-"+str(mc.cnn_filter_size)+"-"+str(mc.cnn_filter_num))(x) 75 | x = BatchNormalization(axis=1, name=res_name+"_batchnorm1")(x) 76 | x = Activation("relu",name=res_name+"_relu1")(x) 77 | x = Conv2D(filters=mc.cnn_filter_num, kernel_size=mc.cnn_filter_size, padding="same", 78 | data_format="channels_first", use_bias=False, kernel_regularizer=l2(mc.l2_reg), 79 | name=res_name+"_conv2-"+str(mc.cnn_filter_size)+"-"+str(mc.cnn_filter_num))(x) 80 | x = BatchNormalization(axis=1, name="res"+str(index)+"_batchnorm2")(x) 81 | x = Add(name=res_name+"_add")([in_x, x]) 82 | x = Activation("relu", name=res_name+"_relu2")(x) 83 | return x 84 | 85 | @staticmethod 86 | def fetch_digest(weight_path): 87 | if os.path.exists(weight_path): 88 | m = hashlib.sha256() 89 | with open(weight_path, "rb") as f: 90 | m.update(f.read()) 91 | return m.hexdigest() 92 | return None 93 | 94 | 95 | def load(self, config_path, weight_path): 96 | if os.path.exists(config_path) and os.path.exists(weight_path): 97 | logger.debug(f"loading model from {config_path}") 98 | with open(config_path, "rt") as f: 99 | self.model = Model.from_config(json.load(f)) 100 | self.model.load_weights(weight_path) 101 | self.digest = self.fetch_digest(weight_path) 102 | self.graph = tf.get_default_graph() 103 | logger.debug(f"loaded model digest = {self.digest}") 104 | return True 105 | else: 106 | logger.debug(f"model files does not exist at {config_path} and {weight_path}") 107 | return False 108 | 109 | def save(self, config_path, weight_path): 110 | logger.debug(f"save model to {config_path}") 111 | with open(config_path, "wt") as f: 112 | json.dump(self.model.get_config(), f) 113 | self.model.save_weights(weight_path) 114 | self.digest = self.fetch_digest(weight_path) 115 | logger.debug(f"saved model digest {self.digest}") 116 | 117 | def get_pipes(self, num=1, api=None, need_reload=True): 118 | if self.api is None: 119 | self.api = CChessModelAPI(self.config, self) 120 | self.api.start(need_reload) 121 | return self.api.get_pipe(need_reload) 122 | 123 | def close_pipes(self): 124 | if self.api is not None: 125 | self.api.close() 126 | self.api = None 127 | 128 | -------------------------------------------------------------------------------- /cchess_alphazero/config.py: -------------------------------------------------------------------------------- 1 | import os 2 | import getpass 3 | 4 | def _project_dir(): 5 | d = os.path.dirname 6 | return d(d(os.path.abspath(__file__))) 7 | 8 | 9 | def _data_dir(): 10 | return os.path.join(_project_dir(), "data") 11 | 12 | class Config: 13 | def __init__(self, config_type="mini"): 14 | self.opts = Options() 15 | self.resource = ResourceConfig() 16 | self.internet = InternetConfig() 17 | 18 | if config_type == "mini": 19 | import configs.mini as c 20 | elif config_type == "normal": 21 | import configs.normal as c 22 | elif config_type == 'distribute': 23 | import cchess_alphazero.configs.distribute as c 24 | else: 25 | raise RuntimeError('unknown config_type: %s' % (config_type)) 26 | self.model = c.ModelConfig() 27 | self.play = c.PlayConfig() 28 | self.play_data = c.PlayDataConfig() 29 | self.trainer = c.TrainerConfig() 30 | self.eval = c.EvaluateConfig() 31 | 32 | class ResourceConfig: 33 | def __init__(self): 34 | self.project_dir = os.environ.get("PROJECT_DIR", _project_dir()) 35 | self.data_dir = os.environ.get("DATA_DIR", _data_dir()) 36 | 37 | self.model_dir = os.environ.get("MODEL_DIR", os.path.join(self.data_dir, "model")) 38 | self.model_best_config_path = os.path.join(self.model_dir, "model_best_config.json") 39 | self.model_best_weight_path = os.path.join(self.model_dir, "model_best_weight.h5") 40 | self.sl_best_config_path = os.path.join(self.model_dir, "sl_best_config.json") 41 | self.sl_best_weight_path = os.path.join(self.model_dir, "sl_best_weight.h5") 42 | self.eleeye_path = os.path.join(self.model_dir, 'ELEEYE') 43 | 44 | self.next_generation_model_dir = os.path.join(self.model_dir, "next_generation") 45 | self.next_generation_config_path = os.path.join(self.next_generation_model_dir, "next_generation_config.json") 46 | self.next_generation_weight_path = os.path.join(self.next_generation_model_dir, "next_generation_weight.h5") 47 | self.rival_model_config_path = os.path.join(self.model_dir, "rival_config.json") 48 | self.rival_model_weight_path = os.path.join(self.model_dir, "rival_weight.h5") 49 | 50 | self.play_data_dir = os.path.join(self.data_dir, "play_data") 51 | self.play_data_filename_tmpl = "play_%s.json" 52 | self.self_play_game_idx_file = os.path.join(self.data_dir, "play_data_idx") 53 | self.play_record_filename_tmpl = "record_%s.qp" 54 | self.play_record_dir = os.path.join(self.data_dir, "play_record") 55 | 56 | self.log_dir = os.path.join(self.project_dir, "logs") 57 | self.main_log_path = os.path.join(self.log_dir, "main.log") 58 | self.opt_log_path = os.path.join(self.log_dir, "opt.log") 59 | self.play_log_path = os.path.join(self.log_dir, "play.log") 60 | self.sl_log_path = os.path.join(self.log_dir, "sl.log") 61 | self.eval_log_path = os.path.join(self.log_dir, "eval.log") 62 | 63 | self.sl_data_dir = os.path.join(self.data_dir, "sl_data") 64 | self.sl_data_gameinfo = os.path.join(self.sl_data_dir, "gameinfo.csv") 65 | self.sl_data_move = os.path.join(self.sl_data_dir, "moves.csv") 66 | self.sl_onegreen = os.path.join(self.sl_data_dir, "onegreen.json") 67 | 68 | self.font_path = os.path.join(self.project_dir, 'cchess_alphazero', 'play_games', 'PingFang.ttc') 69 | 70 | def create_directories(self): 71 | dirs = [self.project_dir, self.data_dir, self.model_dir, self.play_data_dir, self.log_dir, 72 | self.play_record_dir, self.next_generation_model_dir, self.sl_data_dir] 73 | for d in dirs: 74 | if not os.path.exists(d): 75 | os.makedirs(d) 76 | 77 | class Options: 78 | new = False 79 | light = True 80 | device_list = '0' 81 | bg_style = 'CANVAS' 82 | piece_style = 'WOOD' 83 | random = 'none' 84 | log_move = False 85 | use_multiple_gpus = False 86 | gpu_num = 1 87 | evaluate = False 88 | has_history = False 89 | 90 | class PlayWithHumanConfig: 91 | def __init__(self): 92 | self.simulation_num_per_move = 800 93 | self.c_puct = 1 94 | self.search_threads = 10 95 | self.noise_eps = 0 96 | self.tau_decay_rate = 0 97 | self.dirichlet_alpha = 0.2 98 | 99 | def update_play_config(self, pc): 100 | pc.simulation_num_per_move = self.simulation_num_per_move 101 | pc.c_puct = self.c_puct 102 | pc.noise_eps = self.noise_eps 103 | pc.tau_decay_rate = self.tau_decay_rate 104 | pc.search_threads = self.search_threads 105 | pc.dirichlet_alpha = self.dirichlet_alpha 106 | 107 | class InternetConfig: 108 | def __init__(self): 109 | self.distributed = False 110 | self.username = getpass.getuser() 111 | self.base_url = 'https://cczero.org' 112 | self.upload_url = f'{self.base_url}/api/upload_game_file/192x10' 113 | self.upload_eval_url = f'{self.base_url}/api/upload_eval_game_file' 114 | self.download_url = f'http://download.52coding.com.cn/192x10/model_best_weight.h5' 115 | # self.download_url = 'http://alphazero-1251776088.cossh.myqcloud.com/model/128x7/model_best_weight.h5' 116 | self.get_latest_digest = f'{self.base_url}/api/get_latest_digest/192x10' 117 | self.add_model_url = f'{self.base_url}/api/add_model' 118 | self.get_evaluate_model_url = f'{self.base_url}/api/query_for_evaluate' 119 | self.download_base_url = f'http://download.52coding.com.cn/' 120 | # self.download_base_url = 'http://alphazero-1251776088.cossh.myqcloud.com/model/' 121 | self.get_elo_url = f'{self.base_url}/api/get_elo/' 122 | self.update_elo_url = f'{self.base_url}/api/add_eval_result/' 123 | 124 | 125 | -------------------------------------------------------------------------------- /cchess_alphazero/configs/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/configs/__init__.py -------------------------------------------------------------------------------- /cchess_alphazero/configs/distribute.py: -------------------------------------------------------------------------------- 1 | class EvaluateConfig: 2 | def __init__(self): 3 | self.vram_frac = 1.0 4 | self.game_num = 10 5 | self.simulation_num_per_move = 800 6 | self.thinking_loop = 1 7 | self.c_puct = 1 # lower = prefer mean action value 8 | self.tau_decay_rate = 0.5 9 | self.noise_eps = 0.1 10 | self.max_game_length = 200 11 | self.max_processes = 10 12 | self.search_threads = 10 13 | 14 | def update_play_config(self, pc): 15 | pc.simulation_num_per_move = self.simulation_num_per_move 16 | pc.thinking_loop = self.thinking_loop 17 | pc.c_puct = self.c_puct 18 | pc.tau_decay_rate = self.tau_decay_rate 19 | pc.noise_eps = self.noise_eps 20 | pc.max_game_length = self.max_game_length 21 | pc.max_processes = self.max_processes 22 | pc.search_threads = self.search_threads 23 | 24 | 25 | class PlayDataConfig: 26 | def __init__(self): 27 | self.sl_nb_game_in_file = 250 28 | self.nb_game_in_file = 1 # WARNING: DO NOT CHANGE THIS PARAMETER 29 | self.max_file_num = 5000 30 | self.nb_game_save_record = 1 # not supported in distributed mode 31 | 32 | 33 | class PlayConfig: 34 | def __init__(self): 35 | self.max_processes = 10 # tune this to your cpu cores 36 | self.search_threads = 10 # increase this will be faster but with weaker performance 37 | self.vram_frac = 1.0 38 | self.simulation_num_per_move = 800 39 | self.thinking_loop = 1 40 | self.logging_thinking = False 41 | self.c_puct = 5 42 | self.noise_eps = 0.2 43 | self.dirichlet_alpha = 0.2 44 | self.tau_decay_rate = 0.9 45 | self.virtual_loss = 3 46 | self.resign_threshold = -0.99 47 | self.min_resign_turn = 40 48 | self.enable_resign_rate = 0.99 49 | self.max_game_length = 200 50 | self.share_mtcs_info_in_self_play = False 51 | self.reset_mtcs_info_per_game = 5 52 | 53 | 54 | class TrainerConfig: 55 | def __init__(self): 56 | self.min_games_to_begin_learn = 5000 57 | self.min_data_size_to_learn = 0 58 | self.cleaning_processes = 20 59 | self.vram_frac = 1.0 60 | self.batch_size = 1024 61 | self.epoch_to_checkpoint = 1 62 | self.dataset_size = 90000000 63 | self.start_total_steps = 0 64 | self.save_model_steps = 25 65 | self.load_data_steps = 100 66 | self.momentum = 0.9 67 | self.loss_weights = [1.0, 1.0] 68 | self.lr_schedules = [ 69 | (0, 0.03), 70 | (100000, 0.01), 71 | (200000, 0.003), 72 | (300000, 0.001), 73 | (400000, 0.0003), 74 | (500000, 0.0001), 75 | ] 76 | self.sl_game_step = 2000 77 | self.load_step = 25000 78 | 79 | class ModelConfig: 80 | def __init__(self): 81 | ''' 82 | WARNING: DO NOT CHANGE THESE PARAMETERS 83 | ''' 84 | self.cnn_filter_num = 192 85 | self.cnn_first_filter_size = 5 86 | self.cnn_filter_size = 3 87 | self.res_layer_num = 10 88 | self.l2_reg = 1e-4 89 | self.value_fc_size = 256 90 | self.distributed = False 91 | self.input_depth = 14 92 | -------------------------------------------------------------------------------- /cchess_alphazero/configs/mini.py: -------------------------------------------------------------------------------- 1 | class EvaluateConfig: 2 | def __init__(self): 3 | self.vram_frac = 1.0 4 | self.game_num = 2 5 | self.simulation_num_per_move = 20 # before 200 6 | self.thinking_loop = 1 7 | self.c_puct = 1 # lower = prefer mean action value 8 | self.tau_decay_rate = 0 9 | self.noise_eps = 0.2 10 | self.max_game_length = 100 11 | self.max_processes = 2 12 | self.search_threads = 10 13 | 14 | def update_play_config(self, pc): 15 | pc.simulation_num_per_move = self.simulation_num_per_move 16 | pc.thinking_loop = self.thinking_loop 17 | pc.c_puct = self.c_puct 18 | pc.tau_decay_rate = self.tau_decay_rate 19 | pc.noise_eps = self.noise_eps 20 | pc.max_game_length = self.max_game_length 21 | pc.max_processes = self.max_processes 22 | pc.search_threads = self.search_threads 23 | 24 | 25 | class PlayDataConfig: 26 | def __init__(self): 27 | self.sl_nb_game_in_file = 250 28 | self.nb_game_in_file = 1 29 | self.max_file_num = 10 30 | self.nb_game_save_record = 1 31 | 32 | 33 | class PlayConfig: 34 | def __init__(self): 35 | self.max_processes = 1 36 | self.search_threads = 10 37 | self.vram_frac = 1.0 38 | self.simulation_num_per_move = 100 # just for debug 39 | self.c_puct = 1.5 40 | self.noise_eps = 0.25 41 | self.dirichlet_alpha = 0.2 42 | self.tau_decay_rate = 0.98 43 | self.virtual_loss = 3 44 | self.max_game_length = 100 45 | self.share_mtcs_info_in_self_play = False 46 | self.reset_mtcs_info_per_game = 5 47 | self.enable_resign_rate = 0.1 48 | self.resign_threshold = -0.92 49 | self.min_resign_turn = 20 50 | 51 | class TrainerConfig: 52 | def __init__(self): 53 | self.min_games_to_begin_learn = 1 54 | self.min_data_size_to_learn = 0 55 | self.cleaning_processes = 1 56 | self.vram_frac = 1.0 57 | self.batch_size = 2 58 | self.epoch_to_checkpoint = 1 59 | self.dataset_size = 100000 60 | self.start_total_steps = 0 61 | self.save_model_steps = 25 62 | self.load_data_steps = 100 63 | self.momentum = 0.9 64 | self.loss_weights = [1.25, 1.0] 65 | self.lr_schedules = [ 66 | (0, 0.01), 67 | (150000, 0.001), 68 | (300000, 0.0001), 69 | ] 70 | self.sl_game_step = 10000 71 | self.load_step = 6 72 | 73 | class ModelConfig: 74 | def __init__(self): 75 | self.cnn_filter_num = 256 76 | self.cnn_first_filter_size = 5 77 | self.cnn_filter_size = 3 78 | self.res_layer_num = 7 79 | self.l2_reg = 1e-4 80 | self.value_fc_size = 256 81 | self.distributed = False 82 | self.input_depth = 14 83 | -------------------------------------------------------------------------------- /cchess_alphazero/configs/normal.py: -------------------------------------------------------------------------------- 1 | class EvaluateConfig: 2 | def __init__(self): 3 | self.vram_frac = 1.0 4 | self.game_num = 2 5 | self.simulation_num_per_move = 800 6 | self.thinking_loop = 1 7 | self.c_puct = 1 # lower = prefer mean action value 8 | self.tau_decay_rate = 0 9 | self.noise_eps = 0.2 10 | self.max_game_length = 200 11 | self.max_processes = 10 12 | self.search_threads = 8 13 | self.next_generation_replace_rate = 0.55 14 | 15 | def update_play_config(self, pc): 16 | pc.simulation_num_per_move = self.simulation_num_per_move 17 | pc.thinking_loop = self.thinking_loop 18 | pc.c_puct = self.c_puct 19 | pc.tau_decay_rate = self.tau_decay_rate 20 | pc.noise_eps = self.noise_eps 21 | pc.max_game_length = self.max_game_length 22 | pc.max_processes = self.max_processes 23 | pc.search_threads = self.search_threads 24 | 25 | 26 | class PlayDataConfig: 27 | def __init__(self): 28 | self.sl_nb_game_in_file = 250 29 | self.nb_game_in_file = 5 30 | self.max_file_num = 300 31 | self.nb_game_save_record = 1 32 | 33 | 34 | class PlayConfig: 35 | def __init__(self): 36 | self.max_processes = 10 37 | self.search_threads = 40 38 | self.vram_frac = 1.0 39 | self.simulation_num_per_move = 800 40 | self.thinking_loop = 1 41 | self.logging_thinking = False 42 | self.c_puct = 1.5 43 | self.noise_eps = 0.15 44 | self.dirichlet_alpha = 0.2 45 | self.tau_decay_rate = 0.9 46 | self.virtual_loss = 3 47 | self.resign_threshold = -0.98 48 | self.min_resign_turn = 40 49 | self.enable_resign_rate = 0.5 50 | self.max_game_length = 100 51 | self.share_mtcs_info_in_self_play = False 52 | self.reset_mtcs_info_per_game = 5 53 | 54 | 55 | class TrainerConfig: 56 | def __init__(self): 57 | self.min_games_to_begin_learn = 200 58 | self.min_data_size_to_learn = 0 59 | self.cleaning_processes = 4 # RAM explosion... 60 | self.vram_frac = 1.0 61 | self.batch_size = 512 # tune this to your gpu memory 62 | self.epoch_to_checkpoint = 3 63 | self.dataset_size = 100000 64 | self.start_total_steps = 0 65 | self.save_model_steps = 25 66 | self.load_data_steps = 100 67 | self.momentum = 0.9 68 | self.loss_weights = [1.0, 1.0] 69 | self.lr_schedules = [ 70 | (0, 0.01), 71 | (150000, 0.003), 72 | (400000, 0.0001), 73 | ] 74 | self.sl_game_step = 2000 75 | 76 | class ModelConfig: 77 | def __init__(self): 78 | self.cnn_filter_num = 256 79 | self.cnn_first_filter_size = 5 80 | self.cnn_filter_size = 3 81 | self.res_layer_num = 7 82 | self.l2_reg = 1e-4 83 | self.value_fc_size = 256 84 | self.distributed = False 85 | self.input_depth = 14 86 | -------------------------------------------------------------------------------- /cchess_alphazero/environment/README.md: -------------------------------------------------------------------------------- 1 | # Chinese Chess Environment 2 | 3 | **象棋程序功能 (Functions of the chess program)** 4 | 5 | * 判断移动是否合法 (Detect leagal moves) 6 | * 判断胜负 (Detect winner) 7 | * 记录棋谱 (Record) 8 | * 生成训练数据 (Generate training data) 9 | * 可视化 (Visualization) 10 | 11 | **API** 12 | 13 | Class: `cchess_alphazero.environment.env.ChessEnv` 14 | * `reset()`：重制（初始化）引擎环境 15 | 16 | * `observation`：返回当前棋盘 17 | 18 | * `copy()`：（深度）复制当前环境 19 | 20 | * `input_planes(flip=False)`：返回输入给神经网络的特征平面 21 | 22 | 轮到红方时 `flip=True`；轮到黑方时`flip=False` 23 | 24 | 25 | -------------------------------------------------------------------------------- /cchess_alphazero/environment/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/environment/__init__.py -------------------------------------------------------------------------------- /cchess_alphazero/environment/env.py: -------------------------------------------------------------------------------- 1 | import enum 2 | import numpy as np 3 | import copy 4 | 5 | from cchess_alphazero.environment.chessboard import Chessboard 6 | from cchess_alphazero.environment.lookup_tables import Chessman_2_idx, Fen_2_Idx, Winner 7 | from cchess_alphazero.environment.light_env.chessboard import L_Chessboard 8 | 9 | from logging import getLogger 10 | 11 | logger = getLogger(__name__) 12 | 13 | class CChessEnv: 14 | 15 | def __init__(self, config=None): 16 | self.board = None 17 | self.winner = None 18 | self.num_halfmoves = 0 19 | self.config = config 20 | 21 | def reset(self, init=None): 22 | if self.config is None or not self.config.opts.light: 23 | # logger.info("Initialize heavy environment!") 24 | self.board = Chessboard() 25 | self.board.init_board() 26 | else: 27 | # logger.info("Initialize light environment!") 28 | self.board = L_Chessboard(init) 29 | self.winner = None 30 | self.num_halfmoves = 0 31 | return self 32 | 33 | def update(self, board): 34 | self.board = board 35 | self.winner = None 36 | return self 37 | 38 | @property 39 | def done(self): 40 | return self.winner is not None 41 | 42 | @property 43 | def red_won(self): 44 | return self.winner == Winner.red 45 | 46 | @property 47 | def red_to_move(self): 48 | return self.board.is_red_turn 49 | 50 | @property 51 | def observation(self): 52 | if self.board.is_red_turn: 53 | return self.board.FENboard() 54 | else: 55 | return self.board.fliped_FENboard() 56 | 57 | def get_state(self): 58 | fen = self.observation 59 | foo = fen.split(' ') 60 | return foo[0] 61 | 62 | def step(self, action: str, check_over = True): 63 | if check_over and action is None: 64 | return 65 | 66 | if not self.board.move_action_str(action): 67 | logger.error("Move Failed, action=%s, is_red_turn=%d, board=\n%s" % (action, 68 | self.red_to_move, self.board.screen)) 69 | moves = self.board.legal_moves() 70 | logger.error(f"Legal moves: {moves}") 71 | self.num_halfmoves += 1 72 | 73 | if check_over and self.board.is_end(): 74 | self.winner = self.board.winner 75 | 76 | self.board.clear_chessmans_moving_list() 77 | self.board.calc_chessmans_moving_list() 78 | 79 | def copy(self): 80 | env = copy.deepcopy(self) 81 | env.board = copy.deepcopy(self.board) 82 | return env 83 | 84 | def render(self, gui=False): 85 | if gui: 86 | pass 87 | else: 88 | self.board.print_to_cl() 89 | 90 | def input_planes(self): 91 | planes = self.fen_to_planes(self.observation) 92 | return planes 93 | 94 | def state_to_planes(self, state): 95 | planes = self.fen_to_planes(state) 96 | return planes 97 | 98 | def fen_to_planes(self, fen): 99 | ''' 100 | e.g. 101 | rkemsmekr/9/1c5c1/p1p1p1p1p/9/9/P1P1P1P1P/1C5C1/9/RKEMSMEKR r - - 0 1 102 | rkemsmek1/8r/1c5c1/p1p1p1p1p/9/9/P1P1P1P1P/1C5C1/9/RKEMSMEKR b - - 0 1 103 | ''' 104 | planes = np.zeros(shape=(14, 10, 9), dtype=np.float32) 105 | foo = fen.split(' ') 106 | rows = foo[0].split('/') 107 | 108 | for i in range(len(rows)): 109 | row = rows[i] 110 | j = 0 111 | for letter in row: 112 | if letter.isalpha(): 113 | # 0 ~ 7 : upper, 7 ~ 14: lower 114 | planes[Fen_2_Idx[letter] + int(letter.islower()) * 7][i][j] = 1 115 | j += 1 116 | else: 117 | j += int(letter) 118 | return planes 119 | 120 | def save_records(self, filename): 121 | self.board.save_record(filename) 122 | 123 | -------------------------------------------------------------------------------- /cchess_alphazero/environment/light_env/common.py: -------------------------------------------------------------------------------- 1 | #! /usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | # pycchess - just another chinese chess UI 5 | # Copyright (C) 2011 - 2015 timebug 6 | 7 | # This program is free software: you can redistribute it and/or modify 8 | # it under the terms of the GNU General Public License as published by 9 | # the Free Software Foundation, either version 3 of the License, or 10 | # any later version. 11 | 12 | # This program is distributed in the hope that it will be useful, 13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 15 | # GNU General Public License for more details. 16 | 17 | # You should have received a copy of the GNU General Public License 18 | # along with this program. If not, see . 19 | 20 | # import pygame 21 | 22 | RED, BLACK = 1, 0 23 | BORDER, SPACE = 15, 56 24 | LOCAL, OTHER = 0, 1 25 | NETWORK, AI = 0, 1 26 | KING, ADVISOR, BISHOP, KNIGHT, ROOK, CANNON, PAWN, NONE = 0, 1, 2, 3, 4, 5, 6, -1 27 | 28 | AI_SEARCH_DEPTH = 5 29 | 30 | init_fen = 'rnbakabnr/9/1c5c1/p1p1p1p1p/9/9/P1P1P1P1P/1C5C1/9/RNBAKABNR r - - 0 1' 31 | 32 | replace_dict = { 33 | 'n': 'k', 34 | 'N': 'K', 35 | 'b': 'e', 36 | 'B': 'E', 37 | 'a': 'm', 38 | 'A': 'M', 39 | 'k': 's', 40 | 'K': 'S', 41 | 'r': 'r', 42 | 'R': 'R', 43 | 'p': 'p', 44 | 'P': 'P', 45 | 'c': 'c', 46 | 'C': 'C', 47 | } 48 | 49 | state_to_board_dict = { 50 | 'k': 'n', 51 | 'K': 'N', 52 | 'e': 'b', 53 | 'E': 'B', 54 | 'm': 'a', 55 | 'M': 'A', 56 | 's': 'k', 57 | 'S': 'K', 58 | 'r': 'r', 59 | 'R': 'R', 60 | 'p': 'p', 61 | 'P': 'P', 62 | 'c': 'c', 63 | 'C': 'C', 64 | } 65 | 66 | mov_dir = { 67 | 'k': [(0, -1), (1, 0), (0, 1), (-1, 0)], 68 | 'K': [(0, -1), (1, 0), (0, 1), (-1, 0)], 69 | 'a': [(-1, -1), (1, -1), (-1, 1), (1, 1)], 70 | 'A': [(-1, -1), (1, -1), (-1, 1), (1, 1)], 71 | 'b': [(-2, -2), (2, -2), (2, 2), (-2, 2)], 72 | 'B': [(-2, -2), (2, -2), (2, 2), (-2, 2)], 73 | 'n': [(-1, -2), (1, -2), (2, -1), (2, 1), (1, 2), (-1, 2), (-2, 1), (-2, -1)], 74 | 'N': [(-1, -2), (1, -2), (2, -1), (2, 1), (1, 2), (-1, 2), (-2, 1), (-2, -1)], 75 | 'P': [(0, -1), (-1, 0), (1, 0)], 76 | 'p': [(0, 1), (-1, 0), (1, 0)]} 77 | 78 | bishop_check = [(-1, -1), (1, -1), (-1, 1), (1, 1)] 79 | knight_check = [(0, -1), (0, -1), (1, 0), (1, 0), (0, 1), (0, 1), (-1, 0), (-1, 0)] 80 | 81 | def get_kind(fen_ch): 82 | if fen_ch in ['k', 'K']: 83 | return KING 84 | elif fen_ch in ['a', 'A']: 85 | return ADVISOR 86 | elif fen_ch in ['b', 'B']: 87 | return BISHOP 88 | elif fen_ch in ['n', 'N']: 89 | return KNIGHT 90 | elif fen_ch in ['r', 'R']: 91 | return ROOK 92 | elif fen_ch in ['c', 'C']: 93 | return CANNON 94 | elif fen_ch in ['p', 'P']: 95 | return PAWN 96 | else: 97 | return NONE 98 | 99 | def get_char(kind, color): 100 | if kind is KING: 101 | return ['K', 'k'][color] 102 | elif kind is ADVISOR: 103 | return ['A', 'a'][color] 104 | elif kind is BISHOP: 105 | return ['B', 'b'][color] 106 | elif kind is KNIGHT: 107 | return ['N', 'n'][color] 108 | elif kind is ROOK: 109 | return ['R', 'r'][color] 110 | elif kind is CANNON: 111 | return ['C', 'c'][color] 112 | elif kind is PAWN: 113 | return ['P', 'p'][color] 114 | else: 115 | return '' 116 | 117 | def move_to_str(x, y, x_, y_): 118 | move_str = '' 119 | move_str += str(x) 120 | move_str += str(y) 121 | move_str += str(x_) 122 | move_str += str(y_) 123 | return move_str 124 | 125 | def str_to_move(move_str): 126 | move_arr = [0] * 4 127 | move_arr[0] = int(move_str[0]) 128 | move_arr[1] = int(move_str[1]) 129 | move_arr[2] = int(move_str[2]) 130 | move_arr[3] = int(move_str[3]) 131 | return move_arr 132 | 133 | class Move: 134 | def __init__(self, uci:str): 135 | s = str_to_move(uci) 136 | self.p = (s[0],s[1]) 137 | self.n = (s[2],s[3]) 138 | self.uci = uci 139 | @staticmethod 140 | def from_uci(uci): 141 | return Move(uci) 142 | 143 | -------------------------------------------------------------------------------- /cchess_alphazero/environment/lookup_tables.py: -------------------------------------------------------------------------------- 1 | #-*- coding:utf-8 -*- 2 | 3 | from cchess_alphazero.environment.chessman import * 4 | from enum import Enum 5 | import numpy as np 6 | 7 | Chessman_2_idx = { 8 | Pawn: 0, 9 | Cannon: 1, 10 | Rook: 2, 11 | Knight: 3, 12 | Elephant: 4, 13 | Mandarin: 5, 14 | King: 6 15 | } 16 | 17 | Idx_2_Chessman = { 18 | 0: Pawn, 19 | 1: Cannon, 20 | 2: Rook, 21 | 3: Knight, 22 | 4: Elephant, 23 | 5: Mandarin, 24 | 6: King 25 | } 26 | 27 | Fen_2_Idx = { 28 | 'p': 0, 29 | 'P': 0, 30 | 'c': 1, 31 | 'C': 1, 32 | 'r': 2, 33 | 'R': 2, 34 | 'k': 3, 35 | 'K': 3, 36 | 'e': 4, 37 | 'E': 4, 38 | 'm': 5, 39 | 'M': 5, 40 | 's': 6, 41 | 'S': 6 42 | } 43 | 44 | class Color(Enum): 45 | Black = 0 46 | Red = 1 47 | 48 | Winner = Enum("Winner", "red black draw") 49 | 50 | def flip_move(x): 51 | new = '' 52 | new = ''.join([new, str(8 - int(x[0]))]) 53 | new = ''.join([new, str(9 - int(x[1]))]) 54 | new = ''.join([new, str(8 - int(x[2]))]) 55 | new = ''.join([new, str(9 - int(x[3]))]) 56 | return new 57 | 58 | def flip_action_labels(labels): 59 | return [flip_move(x) for x in labels] 60 | 61 | 62 | def create_action_labels(): 63 | labels_array = [] # [col_src,row_src,col_dst,row_dst] 64 | numbers = ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9'] # row 65 | letters = ['0', '1', '2', '3', '4', '5', '6', '7', '8'] # col 66 | 67 | for n1 in range(10): 68 | for l1 in range(9): 69 | destinations = [(n1, t) for t in range(9)] + \ 70 | [(t, l1) for t in range(10)] + \ 71 | [(n1 + a, l1 + b) for (a, b) in 72 | [(-2, -1), (-1, -2), (-2, 1), (1, -2), (2, -1), (-1, 2), (2, 1), (1, 2)]] 73 | for (n2, l2) in destinations: 74 | if (n1, l1) != (n2, l2) and n2 in range(10) and l2 in range(9): 75 | move = letters[l1] + numbers[n1] + letters[l2] + numbers[n2] 76 | labels_array.append(move) 77 | 78 | #for red mandarin 79 | labels_array.append('3041') 80 | labels_array.append('5041') 81 | labels_array.append('3241') 82 | labels_array.append('5241') 83 | labels_array.append('4130') 84 | labels_array.append('4150') 85 | labels_array.append('4132') 86 | labels_array.append('4152') 87 | # for black mandarin 88 | labels_array.append('3948') 89 | labels_array.append('5948') 90 | labels_array.append('3748') 91 | labels_array.append('5748') 92 | labels_array.append('4839') 93 | labels_array.append('4859') 94 | labels_array.append('4837') 95 | labels_array.append('4857') 96 | 97 | #for red elephant 98 | labels_array.append('2002') 99 | labels_array.append('2042') 100 | labels_array.append('6042') 101 | labels_array.append('6082') 102 | labels_array.append('2402') 103 | labels_array.append('2442') 104 | labels_array.append('6442') 105 | labels_array.append('6482') 106 | labels_array.append('0220') 107 | labels_array.append('4220') 108 | labels_array.append('4260') 109 | labels_array.append('8260') 110 | labels_array.append('0224') 111 | labels_array.append('4224') 112 | labels_array.append('4264') 113 | labels_array.append('8264') 114 | # for black elephant 115 | labels_array.append('2907') 116 | labels_array.append('2947') 117 | labels_array.append('6947') 118 | labels_array.append('6987') 119 | labels_array.append('2507') 120 | labels_array.append('2547') 121 | labels_array.append('6547') 122 | labels_array.append('6587') 123 | labels_array.append('0729') 124 | labels_array.append('4729') 125 | labels_array.append('4769') 126 | labels_array.append('8769') 127 | labels_array.append('0725') 128 | labels_array.append('4725') 129 | labels_array.append('4765') 130 | labels_array.append('8765') 131 | 132 | return labels_array 133 | 134 | ActionLabelsRed = create_action_labels() 135 | ActionLabelsBlack = flip_action_labels(ActionLabelsRed) 136 | 137 | Unflipped_index = [ActionLabelsRed.index(x) for x in ActionLabelsBlack] 138 | 139 | def flip_policy(pol): 140 | global Unflipped_index 141 | return np.asarray([pol[ind] for ind in Unflipped_index]) 142 | -------------------------------------------------------------------------------- /cchess_alphazero/lib/data_helper.py: -------------------------------------------------------------------------------- 1 | import os 2 | import json 3 | from datetime import datetime 4 | from glob import glob 5 | from logging import getLogger 6 | 7 | from cchess_alphazero.config import ResourceConfig 8 | 9 | logger = getLogger(__name__) 10 | 11 | def get_game_data_filenames(rc: ResourceConfig): 12 | pattern = os.path.join(rc.play_data_dir, rc.play_data_filename_tmpl % "*") 13 | # files = list(sorted(glob(pattern), key=get_key)) 14 | files = list(sorted(glob(pattern))) 15 | return files 16 | 17 | def write_game_data_to_file(path, data): 18 | with open(path, "wt") as f: 19 | json.dump(data, f) 20 | 21 | 22 | def read_game_data_from_file(path): 23 | with open(path, "rt") as f: 24 | return json.load(f) 25 | 26 | def get_key(x): 27 | stat_x = os.stat(x) 28 | return stat_x.st_ctime 29 | -------------------------------------------------------------------------------- /cchess_alphazero/lib/elo_helper.py: -------------------------------------------------------------------------------- 1 | from logging import getLogger 2 | 3 | logger = getLogger(__name__) 4 | 5 | # 0 ~ 999: K = 30; 1000 ~ 1999: K = 15; 2000 ~ 2999: K = 10; 3000 ~ : K = 5 6 | K_TABLE = [30, 15, 10, 5] 7 | 8 | R_PRI = 40 9 | 10 | def compute_elo(r0, r1, w): 11 | ''' 12 | Compute the elo rating with method from http://www.xqbase.com/protocol/elostat.htm 13 | r0: red player's elo rating 14 | r1: black player's elo rating 15 | w: game result: 1 = red win, 0.5 = draw, 0 = black win 16 | ''' 17 | relative_elo = r1 - r0 - R_PRI 18 | we = 1 / (1 + 10 ** (relative_elo / 400)) 19 | k0 = K_TABLE[-1] if r0 >= 3000 else K_TABLE[r0 // 1000] 20 | k1 = K_TABLE[-1] if r1 >= 3000 else K_TABLE[r1 // 1000] 21 | rn0 = int(r0 + k0 * (w - we)) 22 | rn1 = int(r1 + k1 * (we - w)) 23 | rn0 = rn0 if rn0 > 0 else 0 24 | rn1 = rn1 if rn1 > 0 else 0 25 | return (rn0, rn1) 26 | -------------------------------------------------------------------------------- /cchess_alphazero/lib/logger.py: -------------------------------------------------------------------------------- 1 | from logging import StreamHandler, basicConfig, DEBUG, getLogger, Formatter, FileHandler 2 | 3 | 4 | def setup_logger(log_filename): 5 | format_str = '%(asctime)s@%(name)s %(levelname)s # %(message)s' 6 | basicConfig(filename=log_filename, level=DEBUG, format=format_str) 7 | stream_handler = StreamHandler() 8 | stream_handler.setFormatter(Formatter(format_str)) 9 | getLogger().addHandler(stream_handler) 10 | 11 | def setup_file_logger(log_filename): 12 | format_str = '%(asctime)s@%(name)s %(levelname)s # %(message)s' 13 | basicConfig(filename=log_filename, level=DEBUG, format=format_str) 14 | 15 | if __name__ == '__main__': 16 | setup_logger("test.log") 17 | logger = getLogger("test") 18 | logger.info("OK") 19 | -------------------------------------------------------------------------------- /cchess_alphazero/lib/model_helper.py: -------------------------------------------------------------------------------- 1 | import os 2 | import shutil 3 | from logging import getLogger 4 | 5 | logger = getLogger(__name__) 6 | 7 | 8 | def load_best_model_weight(model): 9 | """ 10 | :param cchess_alphazero.agent.model.CChessModel model: 11 | :return: 12 | """ 13 | return model.load(model.config.resource.model_best_config_path, model.config.resource.model_best_weight_path) 14 | 15 | def load_best_model_weight_from_internet(model): 16 | """ 17 | :param cchess_alphazero.agent.model.CChessModel model: 18 | :return: 19 | """ 20 | from cchess_alphazero.lib.web_helper import download_file 21 | logger.info(f"download model from remote server") 22 | download_file(model.config.internet.download_url, model.config.resource.model_best_weight_path) 23 | return model.load(model.config.resource.model_best_config_path, model.config.resource.model_best_weight_path) 24 | 25 | 26 | def save_as_best_model(model): 27 | """ 28 | 29 | :param cchess_alphazero.agent.model.CChessModel model: 30 | :return: 31 | """ 32 | return model.save(model.config.resource.model_best_config_path, model.config.resource.model_best_weight_path) 33 | 34 | 35 | def need_to_reload_best_model_weight(model): 36 | """ 37 | 38 | :param cchess_alphazero.agent.model.CChessModel model: 39 | :return: 40 | """ 41 | logger.debug("start reload the best model if changed") 42 | digest = model.fetch_digest(model.config.resource.model_best_weight_path) 43 | if digest != model.digest: 44 | return True 45 | 46 | logger.debug("the best model is not changed") 47 | return False 48 | 49 | def load_model_weight(model, config_path, weight_path, name=None): 50 | if name is not None: 51 | logger.info(f"{name}: load model from {config_path}") 52 | return model.load(config_path, weight_path) 53 | 54 | def save_as_next_generation_model(model): 55 | return model.save(model.config.resource.next_generation_config_path, model.config.resource.next_generation_weight_path) 56 | 57 | 58 | def load_sl_best_model_weight(model): 59 | """ 60 | :param cchess_alphazero.agent.model.CChessModel model: 61 | :return: 62 | """ 63 | return model.load(model.config.resource.sl_best_config_path, model.config.resource.sl_best_weight_path) 64 | 65 | 66 | def save_as_sl_best_model(model): 67 | """ 68 | 69 | :param cchess_alphazero.agent.model.CChessModel model: 70 | :return: 71 | """ 72 | return model.save(model.config.resource.sl_best_config_path, model.config.resource.sl_best_weight_path) 73 | -------------------------------------------------------------------------------- /cchess_alphazero/lib/tf_util.py: -------------------------------------------------------------------------------- 1 | 2 | def set_session_config(per_process_gpu_memory_fraction=None, allow_growth=None, device_list='0'): 3 | """ 4 | 5 | :param allow_growth: When necessary, reserve memory 6 | :param float per_process_gpu_memory_fraction: specify GPU memory usage as 0 to 1 7 | 8 | :return: 9 | """ 10 | import tensorflow as tf 11 | import keras.backend as K 12 | 13 | config = tf.ConfigProto( 14 | gpu_options=tf.GPUOptions( 15 | per_process_gpu_memory_fraction=per_process_gpu_memory_fraction, 16 | allow_growth=allow_growth, 17 | visible_device_list=device_list 18 | ) 19 | ) 20 | sess = tf.Session(config=config) 21 | K.set_session(sess) 22 | -------------------------------------------------------------------------------- /cchess_alphazero/lib/web_helper.py: -------------------------------------------------------------------------------- 1 | import requests 2 | import os 3 | from logging import getLogger 4 | from urllib.request import urlopen 5 | from tqdm import tqdm 6 | 7 | logger = getLogger(__name__) 8 | 9 | def upload_file(url, path, filename=None, data=None, rm=False): 10 | filename = filename if filename is not None else 'file' 11 | files = {'file': (filename, open(path, 'rb'), 'application/json')} 12 | success = False 13 | for i in range(3): 14 | try: 15 | r = requests.post(url, files=files, data=data) 16 | if r.status_code != 200: 17 | logger.error(f"Error occurs when upload {filename}: {r.text}") 18 | else: 19 | success = True 20 | break 21 | except Exception as e: 22 | logger.error(f"Error occurs when upload {filename}: {e}") 23 | if rm: 24 | os.remove(path) 25 | return r.json() if success else None 26 | 27 | def http_request(url, post=False, data=None): 28 | success = False 29 | for i in range(3): 30 | try: 31 | if post: 32 | r = requests.post(url, data=data) 33 | else: 34 | r = requests.get(url) 35 | if r.status_code != 200: 36 | logger.error(f"Error occurs when request {url}: {r.text}") 37 | else: 38 | success = True 39 | break 40 | except Exception as e: 41 | logger.error(f"Error occurs when request {url}: {e}") 42 | return r.json() if success else None 43 | 44 | def download_file(url, save_path): 45 | if os.path.exists(save_path): 46 | os.remove(save_path) 47 | file_size = int(urlopen(url).info().get('Content-Length', -1)) 48 | if os.path.exists(save_path): 49 | first_byte = os.path.getsize(save_path) 50 | else: 51 | first_byte = 0 52 | if first_byte >= file_size: 53 | return file_size 54 | header = {"Range": "bytes=%s-%s" % (first_byte, file_size)} 55 | pbar = tqdm( 56 | total=file_size, initial=first_byte, 57 | unit='B', unit_scale=True, desc=url.split('/')[-1]) 58 | req = requests.get(url, headers=header, stream=True) 59 | with(open(save_path, 'ab')) as f: 60 | for chunk in req.iter_content(chunk_size=1024): 61 | if chunk: 62 | f.write(chunk) 63 | pbar.update(1024) 64 | pbar.close() 65 | return True 66 | 67 | 68 | -------------------------------------------------------------------------------- /cchess_alphazero/manager.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | import multiprocessing as mp 3 | 4 | from logging import getLogger 5 | 6 | from cchess_alphazero.lib.logger import setup_logger 7 | from cchess_alphazero.config import Config, PlayWithHumanConfig 8 | 9 | logger = getLogger(__name__) 10 | 11 | CMD_LIST = ['self', 'opt', 'eval', 'play', 'eval', 'sl', 'ob'] 12 | PIECE_STYLE_LIST = ['WOOD', 'POLISH', 'DELICATE'] 13 | BG_STYLE_LIST = ['CANVAS', 'DROPS', 'GREEN', 'QIANHONG', 'SHEET', 'SKELETON', 'WHITE', 'WOOD'] 14 | RANDOM_LIST = ['none', 'small', 'medium', 'large'] 15 | 16 | def create_parser(): 17 | parser = argparse.ArgumentParser() 18 | parser.add_argument("cmd", help="what to do", choices=CMD_LIST) 19 | parser.add_argument("--new", help="run from new best model", action="store_true") 20 | parser.add_argument("--type", help="use normal setting", default="mini") 21 | parser.add_argument("--total-step", help="set TrainerConfig.start_total_steps", type=int) 22 | parser.add_argument("--ai-move-first", help="set human or AI move first", action="store_true") 23 | parser.add_argument("--cli", help="play with AI with CLI, default with GUI", action="store_true") 24 | parser.add_argument("--gpu", help="device list", default="0") 25 | parser.add_argument("--onegreen", help="train sl work with onegreen data", action="store_true") 26 | parser.add_argument("--skip", help="skip games", default=0, type=int) 27 | parser.add_argument("--ucci", help="play with ucci engine instead of self play", action="store_true") 28 | parser.add_argument("--piece-style", help="choose a style of piece", choices=PIECE_STYLE_LIST, default="WOOD") 29 | parser.add_argument("--bg-style", help="choose a style of board", choices=BG_STYLE_LIST, default="WOOD") 30 | parser.add_argument("--random", help="choose a style of randomness", choices=RANDOM_LIST, default="none") 31 | parser.add_argument("--distributed", help="whether upload/download file from remote server", action="store_true") 32 | parser.add_argument("--elo", help="whether to compute elo score", action="store_true") 33 | return parser 34 | 35 | def setup(config: Config, args): 36 | config.opts.new = args.new 37 | if args.total_step is not None: 38 | config.trainer.start_total_steps = args.total_step 39 | config.opts.device_list = args.gpu 40 | config.resource.create_directories() 41 | if args.cmd == 'self': 42 | setup_logger(config.resource.main_log_path) 43 | elif args.cmd == 'opt': 44 | setup_logger(config.resource.opt_log_path) 45 | elif args.cmd == 'play' or args.cmd == 'ob': 46 | setup_logger(config.resource.play_log_path) 47 | elif args.cmd == 'eval': 48 | setup_logger(config.resource.eval_log_path) 49 | elif args.cmd == 'sl': 50 | setup_logger(config.resource.sl_log_path) 51 | 52 | def start(): 53 | parser = create_parser() 54 | args = parser.parse_args() 55 | config_type = args.type 56 | 57 | config = Config(config_type=config_type) 58 | setup(config, args) 59 | 60 | logger.info('Config type: %s' % (config_type)) 61 | config.opts.piece_style = args.piece_style 62 | config.opts.bg_style = args.bg_style 63 | config.internet.distributed = args.distributed 64 | 65 | # use multiple GPU 66 | gpus = config.opts.device_list.split(',') 67 | if len(gpus) > 1: 68 | config.opts.use_multiple_gpus = True 69 | config.opts.gpu_num = len(gpus) 70 | logger.info(f"User GPU {config.opts.device_list}") 71 | 72 | if args.cmd == 'self': 73 | if args.ucci: 74 | import cchess_alphazero.worker.play_with_ucci_engine as self_play 75 | else: 76 | if mp.get_start_method() == 'spawn': 77 | import cchess_alphazero.worker.self_play_windows as self_play 78 | else: 79 | from cchess_alphazero.worker import self_play 80 | return self_play.start(config) 81 | elif args.cmd == 'opt': 82 | from cchess_alphazero.worker import optimize 83 | return optimize.start(config) 84 | elif args.cmd == 'play': 85 | if args.cli: 86 | import cchess_alphazero.play_games.play_cli as play 87 | else: 88 | from cchess_alphazero.play_games import play 89 | config.opts.light = False 90 | pwhc = PlayWithHumanConfig() 91 | pwhc.update_play_config(config.play) 92 | logger.info(f"AI move first : {args.ai_move_first}") 93 | play.start(config, not args.ai_move_first) 94 | elif args.cmd == 'eval': 95 | if args.elo == False: 96 | from cchess_alphazero.worker import evaluator 97 | else: 98 | if mp.get_start_method() == 'spawn': 99 | import cchess_alphazero.worker.compute_elo_windows as evaluator 100 | else: 101 | import cchess_alphazero.worker.compute_elo as evaluator 102 | config.eval.update_play_config(config.play) 103 | evaluator.start(config) 104 | elif args.cmd == 'sl': 105 | if args.onegreen: 106 | import cchess_alphazero.worker.sl_onegreen as sl 107 | sl.start(config, args.skip) 108 | else: 109 | from cchess_alphazero.worker import sl 110 | sl.start(config) 111 | 112 | elif args.cmd == 'ob': 113 | from cchess_alphazero.play_games import ob_self_play 114 | pwhc = PlayWithHumanConfig() 115 | pwhc.update_play_config(config.play) 116 | ob_self_play.start(config, args.ucci, args.ai_move_first) 117 | 118 | -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/CANVAS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/CANVAS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/BA.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/BA.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/BAS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/BAS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/BB.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/BB.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/BBS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/BBS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/BC.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/BC.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/BCS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/BCS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/BK.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/BK.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/BKM.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/BKM.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/BKS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/BKS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/BN.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/BN.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/BNS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/BNS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/BP.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/BP.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/BPS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/BPS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/BR.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/BR.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/BRS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/BRS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/OO.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/OO.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/OOS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/OOS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/RA.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/RA.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/RAS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/RAS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/RB.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/RB.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/RBS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/RBS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/RC.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/RC.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/RCS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/RCS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/RK.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/RK.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/RKM.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/RKM.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/RKS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/RKS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/RN.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/RN.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/RNS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/RNS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/RP.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/RP.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/RPS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/RPS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/RR.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/RR.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DELICATE/RRS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DELICATE/RRS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/DROPS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/DROPS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/GREEN.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/GREEN.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/BA.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/BA.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/BAS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/BAS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/BB.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/BB.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/BBS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/BBS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/BC.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/BC.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/BCS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/BCS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/BK.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/BK.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/BKM.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/BKM.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/BKS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/BKS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/BN.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/BN.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/BNS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/BNS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/BP.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/BP.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/BPS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/BPS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/BR.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/BR.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/BRS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/BRS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/OO.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/OO.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/OOS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/OOS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/RA.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/RA.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/RAS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/RAS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/RB.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/RB.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/RBS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/RBS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/RC.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/RC.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/RCS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/RCS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/RK.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/RK.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/RKM.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/RKM.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/RKS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/RKS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/RN.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/RN.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/RNS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/RNS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/RP.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/RP.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/RPS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/RPS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/RR.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/RR.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/POLISH/RRS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/POLISH/RRS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/QIANHONG.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/QIANHONG.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/SHEET.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/SHEET.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/SKELETON.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/SKELETON.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WHITE.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WHITE.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/BA.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/BA.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/BAS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/BAS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/BB.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/BB.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/BBS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/BBS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/BC.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/BC.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/BCS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/BCS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/BK.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/BK.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/BKM.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/BKM.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/BKS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/BKS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/BN.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/BN.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/BNS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/BNS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/BP.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/BP.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/BPS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/BPS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/BR.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/BR.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/BRS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/BRS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/OO.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/OO.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/OOS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/OOS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/RA.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/RA.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/RAS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/RAS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/RB.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/RB.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/RBS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/RBS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/RC.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/RC.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/RCS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/RCS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/RK.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/RK.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/RKM.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/RKM.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/RKS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/RKS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/RN.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/RN.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/RNS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/RNS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/RP.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/RP.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/RPS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/RPS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/RR.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/RR.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/images/WOOD/RRS.GIF: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/play_games/images/WOOD/RRS.GIF -------------------------------------------------------------------------------- /cchess_alphazero/play_games/ob_self_play.py: -------------------------------------------------------------------------------- 1 | import os 2 | import subprocess 3 | import numpy as np 4 | from collections import defaultdict 5 | from logging import getLogger 6 | from time import sleep, time 7 | 8 | import cchess_alphazero.environment.static_env as senv 9 | from cchess_alphazero.environment.chessboard import Chessboard 10 | from cchess_alphazero.environment.chessman import * 11 | from cchess_alphazero.agent.model import CChessModel 12 | from cchess_alphazero.agent.player import CChessPlayer, VisitState 13 | from cchess_alphazero.agent.api import CChessModelAPI 14 | from cchess_alphazero.config import Config 15 | from cchess_alphazero.environment.env import CChessEnv 16 | from cchess_alphazero.environment.lookup_tables import Winner, ActionLabelsRed, flip_move 17 | from cchess_alphazero.lib.model_helper import load_best_model_weight 18 | from cchess_alphazero.lib.tf_util import set_session_config 19 | 20 | logger = getLogger(__name__) 21 | 22 | def start(config: Config, ucci=False, ai_move_first=True): 23 | set_session_config(per_process_gpu_memory_fraction=1, allow_growth=True, device_list=config.opts.device_list) 24 | if not ucci: 25 | play = ObSelfPlay(config) 26 | else: 27 | play = ObSelfPlayUCCI(config, ai_move_first) 28 | play.start() 29 | 30 | class ObSelfPlay: 31 | def __init__(self, config: Config): 32 | self.config = config 33 | self.env = CChessEnv() 34 | self.model = None 35 | self.pipe = None 36 | self.ai = None 37 | self.chessmans = None 38 | 39 | def load_model(self): 40 | self.model = CChessModel(self.config) 41 | if self.config.opts.new or not load_best_model_weight(self.model): 42 | self.model.build() 43 | 44 | def start(self): 45 | self.env.reset() 46 | self.load_model() 47 | self.pipe = self.model.get_pipes() 48 | self.ai = CChessPlayer(self.config, search_tree=defaultdict(VisitState), pipes=self.pipe, 49 | enable_resign=True, debugging=False) 50 | 51 | labels = ActionLabelsRed 52 | labels_n = len(ActionLabelsRed) 53 | 54 | self.env.board.print_to_cl() 55 | history = [self.env.get_state()] 56 | 57 | while not self.env.board.is_end(): 58 | no_act = None 59 | state = self.env.get_state() 60 | if state in history[:-1]: 61 | no_act = [] 62 | for i in range(len(history) - 1): 63 | if history[i] == state: 64 | no_act.append(history[i + 1]) 65 | action, _ = self.ai.action(state, self.env.num_halfmoves, no_act) 66 | history.append(action) 67 | if action is None: 68 | print("AI投降了!") 69 | break 70 | move = self.env.board.make_single_record(int(action[0]), int(action[1]), int(action[2]), int(action[3])) 71 | if not self.env.red_to_move: 72 | action = flip_move(action) 73 | self.env.step(action) 74 | history.append(self.env.get_state()) 75 | print(f"AI选择移动 {move}") 76 | self.env.board.print_to_cl() 77 | sleep(1) 78 | 79 | self.ai.close() 80 | print(f"胜者是 is {self.env.board.winner} !!!") 81 | self.env.board.print_record() 82 | 83 | class ObSelfPlayUCCI: 84 | def __init__(self, config: Config, ai_move_first=True): 85 | self.config = config 86 | self.env = CChessEnv() 87 | self.model = None 88 | self.pipe = None 89 | self.ai = None 90 | self.chessmans = None 91 | self.ai_move_first = ai_move_first 92 | 93 | def load_model(self): 94 | self.model = CChessModel(self.config) 95 | if self.config.opts.new or not load_best_model_weight(self.model): 96 | self.model.build() 97 | 98 | def start(self): 99 | self.env.reset() 100 | self.load_model() 101 | self.pipe = self.model.get_pipes() 102 | self.ai = CChessPlayer(self.config, search_tree=defaultdict(VisitState), pipes=self.pipe, 103 | enable_resign=True, debugging=False) 104 | 105 | labels = ActionLabelsRed 106 | labels_n = len(ActionLabelsRed) 107 | 108 | self.env.board.print_to_cl() 109 | history = [self.env.get_state()] 110 | turns = 0 111 | game_over = False 112 | final_move = None 113 | 114 | while not game_over: 115 | if (self.ai_move_first and turns % 2 == 0) or (not self.ai_move_first and turns % 2 == 1): 116 | start_time = time() 117 | no_act = None 118 | state = self.env.get_state() 119 | if state in history[:-1]: 120 | no_act = [] 121 | for i in range(len(history) - 1): 122 | if history[i] == state: 123 | act = history[i + 1] 124 | if not self.env.red_to_move: 125 | act = flip_move(act) 126 | no_act.append(act) 127 | action, _ = self.ai.action(state, self.env.num_halfmoves, no_act) 128 | end_time = time() 129 | if action is None: 130 | print("AlphaZero 投降了!") 131 | break 132 | move = self.env.board.make_single_record(int(action[0]), int(action[1]), int(action[2]), int(action[3])) 133 | print(f"AlphaZero 选择移动 {move}, 消耗时间 {(end_time - start_time):.2f}s") 134 | if not self.env.red_to_move: 135 | action = flip_move(action) 136 | else: 137 | state = self.env.get_state() 138 | print(state) 139 | fen = senv.state_to_fen(state, turns) 140 | action = self.get_ucci_move(fen) 141 | if action is None: 142 | print("Eleeye 投降了!") 143 | break 144 | print(action) 145 | if not self.env.red_to_move: 146 | rec_action = flip_move(action) 147 | else: 148 | rec_action = action 149 | move = self.env.board.make_single_record(int(rec_action[0]), int(rec_action[1]), int(rec_action[2]), int(rec_action[3])) 150 | print(f"Eleeye 选择移动 {move}") 151 | history.append(action) 152 | self.env.step(action) 153 | history.append(self.env.get_state()) 154 | self.env.board.print_to_cl() 155 | turns += 1 156 | sleep(1) 157 | game_over, final_move = self.env.board.is_end_final_move() 158 | print(game_over, final_move) 159 | 160 | if final_move: 161 | move = self.env.board.make_single_record(int(final_move[0]), int(final_move[1]), int(final_move[2]), int(final_move[3])) 162 | print(f"Final Move {move}") 163 | if not self.env.red_to_move: 164 | final_move = flip_move(final_move) 165 | self.env.step(final_move) 166 | self.env.board.print_to_cl() 167 | 168 | self.ai.close() 169 | print(f"胜者是 is {self.env.board.winner} !!!") 170 | self.env.board.print_record() 171 | 172 | def get_ucci_move(self, fen, time=3): 173 | p = subprocess.Popen(self.config.resource.eleeye_path, 174 | stdin=subprocess.PIPE, 175 | stdout=subprocess.PIPE, 176 | stderr=subprocess.PIPE, 177 | universal_newlines=True) 178 | setfen = f'position fen {fen}\n' 179 | setrandom = 'setoption randomness small\n' 180 | cmd = 'ucci\n' + setrandom + setfen + f'go time {time * 1000}\n' 181 | try: 182 | out, err = p.communicate(cmd, timeout=time+0.5) 183 | except: 184 | p.kill() 185 | try: 186 | out, err = p.communicate() 187 | except Exception as e: 188 | logger.error(f"{e}, cmd = {cmd}") 189 | return self.get_ucci_move(fen, time+1) 190 | print(out) 191 | lines = out.split('\n') 192 | if lines[-2] == 'nobestmove': 193 | return None 194 | move = lines[-2].split(' ')[1] 195 | if move == 'depth': 196 | move = lines[-1].split(' ')[6] 197 | return senv.parse_ucci_move(move) 198 | -------------------------------------------------------------------------------- /cchess_alphazero/play_games/play_cli.py: -------------------------------------------------------------------------------- 1 | from collections import defaultdict 2 | from logging import getLogger 3 | 4 | import cchess_alphazero.environment.static_env as senv 5 | from cchess_alphazero.environment.chessboard import Chessboard 6 | from cchess_alphazero.environment.chessman import * 7 | from cchess_alphazero.agent.model import CChessModel 8 | from cchess_alphazero.agent.player import CChessPlayer, VisitState 9 | from cchess_alphazero.agent.api import CChessModelAPI 10 | from cchess_alphazero.config import Config 11 | from cchess_alphazero.environment.env import CChessEnv 12 | from cchess_alphazero.environment.lookup_tables import Winner, ActionLabelsRed, flip_move 13 | from cchess_alphazero.lib.model_helper import load_best_model_weight 14 | from cchess_alphazero.lib.tf_util import set_session_config 15 | 16 | logger = getLogger(__name__) 17 | 18 | def start(config: Config, human_move_first=True): 19 | set_session_config(per_process_gpu_memory_fraction=1, allow_growth=True, device_list=config.opts.device_list) 20 | play = PlayWithHuman(config) 21 | play.start(human_move_first) 22 | 23 | class PlayWithHuman: 24 | def __init__(self, config: Config): 25 | self.config = config 26 | self.env = CChessEnv() 27 | self.model = None 28 | self.pipe = None 29 | self.ai = None 30 | self.chessmans = None 31 | self.human_move_first = True 32 | 33 | def load_model(self): 34 | self.model = CChessModel(self.config) 35 | if self.config.opts.new or not load_best_model_weight(self.model): 36 | self.model.build() 37 | 38 | def start(self, human_first=True): 39 | self.env.reset() 40 | self.load_model() 41 | self.pipe = self.model.get_pipes() 42 | self.ai = CChessPlayer(self.config, search_tree=defaultdict(VisitState), pipes=self.pipe, 43 | enable_resign=True, debugging=False) 44 | self.human_move_first = human_first 45 | 46 | labels = ActionLabelsRed 47 | labels_n = len(ActionLabelsRed) 48 | 49 | self.env.board.print_to_cl() 50 | 51 | while not self.env.board.is_end(): 52 | if human_first == self.env.red_to_move: 53 | self.env.board.calc_chessmans_moving_list() 54 | is_correct_chessman = False 55 | is_correct_position = False 56 | chessman = None 57 | while not is_correct_chessman: 58 | title = "请输入棋子位置: " 59 | input_chessman_pos = input(title) 60 | x, y = int(input_chessman_pos[0]), int(input_chessman_pos[1]) 61 | chessman = self.env.board.chessmans[x][y] 62 | if chessman != None and chessman.is_red == self.env.board.is_red_turn: 63 | is_correct_chessman = True 64 | print(f"当前棋子为{chessman.name_cn}，可以落子的位置有：") 65 | for point in chessman.moving_list: 66 | print(point.x, point.y) 67 | else: 68 | print("没有找到此名字的棋子或未轮到此方走子") 69 | while not is_correct_position: 70 | title = "请输入落子的位置: " 71 | input_chessman_pos = input(title) 72 | x, y = int(input_chessman_pos[0]), int(input_chessman_pos[1]) 73 | is_correct_position = chessman.move(x, y) 74 | if is_correct_position: 75 | self.env.board.print_to_cl() 76 | self.env.board.clear_chessmans_moving_list() 77 | else: 78 | action, policy = self.ai.action(self.env.get_state(), self.env.num_halfmoves) 79 | if not self.env.red_to_move: 80 | action = flip_move(action) 81 | if action is None: 82 | print("AI投降了!") 83 | break 84 | self.env.step(action) 85 | print(f"AI选择移动 {action}") 86 | self.env.board.print_to_cl() 87 | 88 | self.ai.close() 89 | print(f"胜者是 is {self.env.board.winner} !!!") 90 | self.env.board.print_record() 91 | -------------------------------------------------------------------------------- /cchess_alphazero/play_games/test_cli_game.py: -------------------------------------------------------------------------------- 1 | #-*- coding:utf-8 -*- 2 | 3 | from cchess_alphazero.environment.chessboard import Chessboard 4 | 5 | def print_chessman_name(chessman): 6 | if chessman: 7 | print(chessman.name) 8 | else: 9 | print("None") 10 | 11 | 12 | def main(): 13 | cbd = Chessboard('000') 14 | cbd.init_board() 15 | cbd.print_to_cl() 16 | while not cbd.is_end(): 17 | cbd.calc_chessmans_moving_list() 18 | if cbd.is_red_turn: 19 | print("is_red_turn") 20 | else: 21 | print("is_black_turn") 22 | is_correct_chessman = False 23 | is_correct_position = False 24 | chessman = None 25 | while not is_correct_chessman: 26 | title = "请输入棋子名字: " 27 | input_chessman_name = input(title) 28 | chessman = cbd.get_chessman_by_name(input_chessman_name) 29 | if chessman != None and chessman.is_red == cbd.is_red_turn: 30 | is_correct_chessman = True 31 | print("当前可以落子的位置有：") 32 | for point in chessman.moving_list: 33 | print(point.x, point.y) 34 | print(cbd.legal_moves()) 35 | else: 36 | print("没有找到此名字的棋子或未轮到此方走子") 37 | while not is_correct_position: 38 | title = "请输入落子的位置: " 39 | input_chessman_position0 = int(input(title)) 40 | input_chessman_position1 = int(input(title)) 41 | print("Inputed Position:", input_chessman_position0, input_chessman_position1) 42 | is_correct_position = chessman.move( 43 | input_chessman_position0, input_chessman_position1) 44 | if is_correct_position: 45 | cbd.print_to_cl() 46 | cbd.clear_chessmans_moving_list() 47 | 48 | 49 | if __name__ == '__main__': 50 | main() 51 | 52 | -------------------------------------------------------------------------------- /cchess_alphazero/play_games/test_window_game.py: -------------------------------------------------------------------------------- 1 | #-*- coding:utf-8 -*- 2 | 3 | import sys 4 | import pygame 5 | import random 6 | import os.path 7 | 8 | from cchess_alphazero.environment.chessboard import Chessboard 9 | from cchess_alphazero.environment.chessman import * 10 | from pygame.locals import * 11 | 12 | main_dir = os.path.split(os.path.abspath(__file__))[0] 13 | SCREENRECT = Rect(0, 0, 700, 577) 14 | PIECE_STYLE = 'POLISH' 15 | BOARD_STYLE = 'QIANHONG' 16 | 17 | 18 | def load_image(file, sub_dir=None): 19 | '''loads an image, prepares it for play''' 20 | if sub_dir: 21 | file = os.path.join(main_dir, 'images', sub_dir, file) 22 | else: 23 | file = os.path.join(main_dir, 'images', file) 24 | try: 25 | surface = pygame.image.load(file) 26 | except pygame.error: 27 | raise SystemExit('Could not load image "%s" %s' % 28 | (file, pygame.get_error())) 29 | return surface.convert() 30 | 31 | 32 | def load_images(*files): 33 | imgs = [] 34 | style = PIECE_STYLE 35 | for file in files: 36 | imgs.append(load_image(file, style)) 37 | return imgs 38 | 39 | 40 | class Chessman_Sprite(pygame.sprite.Sprite): 41 | is_selected = False 42 | images = [] 43 | is_transparent = False 44 | 45 | def __init__(self, images, chessman): 46 | pygame.sprite.Sprite.__init__(self) 47 | self.chessman = chessman 48 | self.images = images 49 | self.image = self.images[0] 50 | self.rect = Rect(chessman.col_num * 57, (9 - chessman.row_num) * 57, 57, 57) 51 | 52 | def move(self, col_num, row_num): 53 | # print self.chessman.name, col_num, row_num 54 | old_col_num = self.chessman.col_num 55 | old_row_num = self.chessman.row_num 56 | is_correct_position = self.chessman.move(col_num, row_num) 57 | if is_correct_position: 58 | self.rect.move_ip((col_num - old_col_num) 59 | * 57, (old_row_num - row_num) * 57) 60 | self.rect = self.rect.clamp(SCREENRECT) 61 | self.chessman.chessboard.clear_chessmans_moving_list() 62 | self.chessman.chessboard.calc_chessmans_moving_list() 63 | return True 64 | return False 65 | 66 | def update(self): 67 | if self.is_selected: 68 | self.image = self.images[1] 69 | else: 70 | self.image = self.images[0] 71 | 72 | 73 | def creat_sprite_group(sprite_group, chessmans_hash): 74 | for chess in chessmans_hash.values(): 75 | if chess.is_red: 76 | if isinstance(chess, Rook): 77 | images = load_images("RR.GIF", "RRS.GIF") 78 | elif isinstance(chess, Cannon): 79 | images = load_images("RC.GIF", "RCS.GIF") 80 | elif isinstance(chess, Knight): 81 | images = load_images("RN.GIF", "RNS.GIF") 82 | elif isinstance(chess, King): 83 | images = load_images("RK.GIF", "RKS.GIF") 84 | elif isinstance(chess, Elephant): 85 | images = load_images("RB.GIF", "RBS.GIF") 86 | elif isinstance(chess, Mandarin): 87 | images = load_images("RA.GIF", "RAS.GIF") 88 | else: 89 | images = load_images("RP.GIF", "RPS.GIF") 90 | else: 91 | if isinstance(chess, Rook): 92 | images = load_images("BR.GIF", "BRS.GIF") 93 | elif isinstance(chess, Cannon): 94 | images = load_images("BC.GIF", "BCS.GIF") 95 | elif isinstance(chess, Knight): 96 | images = load_images("BN.GIF", "BNS.GIF") 97 | elif isinstance(chess, King): 98 | images = load_images("BK.GIF", "BKS.GIF") 99 | elif isinstance(chess, Elephant): 100 | images = load_images("BB.GIF", "BBS.GIF") 101 | elif isinstance(chess, Mandarin): 102 | images = load_images("BA.GIF", "BAS.GIF") 103 | else: 104 | images = load_images("BP.GIF", "BPS.GIF") 105 | chessman_sprite = Chessman_Sprite(images, chess) 106 | sprite_group.add(chessman_sprite) 107 | 108 | 109 | def select_sprite_from_group(sprite_group, col_num, row_num): 110 | for sprite in sprite_group: 111 | if sprite.chessman.col_num == col_num and sprite.chessman.row_num == row_num: 112 | return sprite 113 | 114 | 115 | def translate_hit_area(screen_x, screen_y): 116 | return screen_x // 57, 9 - screen_y // 57 117 | 118 | 119 | def main(winstyle=0): 120 | 121 | pygame.init() 122 | bestdepth = pygame.display.mode_ok(SCREENRECT.size, winstyle, 32) 123 | screen = pygame.display.set_mode(SCREENRECT.size, winstyle, bestdepth) 124 | pygame.display.set_caption("中国象棋-AlphaZero") 125 | 126 | # create the background, tile the bgd image 127 | bgdtile = load_image(f'{BOARD_STYLE}.GIF') 128 | board_background = pygame.Surface([521, 577]) 129 | board_background.blit(bgdtile, (0, 0)) 130 | widget_background = pygame.Surface([700 - 521, 577]) 131 | white_rect = Rect(0, 0, 700 - 521, 577) 132 | widget_background.fill((255, 255, 255), white_rect) 133 | 134 | #create text label 135 | font_file = os.path.join(main_dir, 'PingFang.ttc') 136 | font = pygame.font.Font(font_file, 16) 137 | font_color = (0, 0, 0) 138 | font_background = (255, 255, 255) 139 | t = font.render("着法记录", True, font_color, font_background) 140 | t_rect = t.get_rect() 141 | t_rect.centerx = (700 - 521) / 2 142 | t_rect.y = 10 143 | widget_background.blit(t, t_rect) 144 | 145 | # background = pygame.Surface(SCREENRECT.size) 146 | # for x in range(0, SCREENRECT.width, bgdtile.get_width()): 147 | # background.blit(bgdtile, (x, 0)) 148 | screen.blit(board_background, (0, 0)) 149 | screen.blit(widget_background, (521, 0)) 150 | pygame.display.flip() 151 | 152 | cbd = Chessboard('000') 153 | cbd.init_board() 154 | 155 | chessmans = pygame.sprite.Group() 156 | framerate = pygame.time.Clock() 157 | 158 | creat_sprite_group(chessmans, cbd.chessmans_hash) 159 | current_chessman = None 160 | cbd.calc_chessmans_moving_list() 161 | print(cbd.legal_moves()) 162 | while not cbd.is_end(): 163 | for event in pygame.event.get(): 164 | if event.type == pygame.QUIT: 165 | cbd.print_record() 166 | sys.exit() 167 | elif event.type == MOUSEBUTTONDOWN: 168 | pressed_array = pygame.mouse.get_pressed() 169 | for index in range(len(pressed_array)): 170 | if index == 0 and pressed_array[index]: 171 | mouse_x, mouse_y = pygame.mouse.get_pos() 172 | col_num, row_num = translate_hit_area(mouse_x, mouse_y) 173 | chessman_sprite = select_sprite_from_group( 174 | chessmans, col_num, row_num) 175 | if current_chessman is None and chessman_sprite != None: 176 | if chessman_sprite.chessman.is_red == cbd.is_red_turn: 177 | current_chessman = chessman_sprite 178 | chessman_sprite.is_selected = True 179 | elif current_chessman != None and chessman_sprite != None: 180 | if chessman_sprite.chessman.is_red == cbd.is_red_turn: 181 | current_chessman.is_selected = False 182 | current_chessman = chessman_sprite 183 | chessman_sprite.is_selected = True 184 | else: 185 | success = current_chessman.move(col_num, row_num) 186 | if success: 187 | chessmans.remove(chessman_sprite) 188 | chessman_sprite.kill() 189 | current_chessman.is_selected = False 190 | current_chessman = None 191 | print(cbd.legal_moves()) 192 | elif current_chessman != None and chessman_sprite is None: 193 | success = current_chessman.move(col_num, row_num) 194 | if success: 195 | current_chessman.is_selected = False 196 | current_chessman = None 197 | print(cbd.legal_moves()) 198 | records = cbd.record.split('\n') 199 | font = pygame.font.Font(font_file, 12) 200 | i = 0 201 | for record in records[-20:]: 202 | rec_label = font.render(record, True, font_color, font_background) 203 | t_rect = rec_label.get_rect() 204 | t_rect.centerx = (700 - 521) / 2 205 | t_rect.y = 35 + i * 15 206 | widget_background.blit(rec_label, t_rect) 207 | i += 1 208 | screen.blit(widget_background, (521, 0)) 209 | framerate.tick(20) 210 | # clear/erase the last drawn sprites 211 | chessmans.clear(screen, board_background) 212 | 213 | # update all the sprites 214 | chessmans.update() 215 | chessmans.draw(screen) 216 | pygame.display.update() 217 | 218 | cbd.print_record() 219 | 220 | if __name__ == '__main__': 221 | main() 222 | -------------------------------------------------------------------------------- /cchess_alphazero/run.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | import multiprocessing as mp 4 | 5 | _PATH_ = os.path.dirname(os.path.dirname(__file__)) 6 | 7 | if _PATH_ not in sys.path: 8 | sys.path.append(_PATH_) 9 | 10 | if __name__ == "__main__": 11 | # mp.set_start_method('spawn') 12 | sys.setrecursionlimit(10000) 13 | from cchess_alphazero import manager 14 | manager.start() 15 | -------------------------------------------------------------------------------- /cchess_alphazero/test.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | import multiprocessing as mp 4 | 5 | _PATH_ = os.path.dirname(os.path.dirname(__file__)) 6 | 7 | 8 | if _PATH_ not in sys.path: 9 | sys.path.append(_PATH_) 10 | 11 | def test_env(): 12 | from cchess_alphazero.environment.env import CChessEnv 13 | env = CChessEnv() 14 | env.reset() 15 | print(env.observation) 16 | env.step('0001') 17 | print(env.observation) 18 | env.step('7770') 19 | print(env.observation) 20 | env.render() 21 | print(env.input_planes()[0+7:3+7]) 22 | 23 | def test_player(): 24 | from cchess_alphazero.agent.player import CChessPlayer 25 | 26 | def test_config(): 27 | from cchess_alphazero.config import Config 28 | c = Config('mini') 29 | c.resource.create_directories() 30 | print(c.resource.project_dir, c.resource.data_dir) 31 | 32 | def test_self_play(): 33 | from cchess_alphazero.config import Config 34 | from cchess_alphazero.worker.self_play import start 35 | from cchess_alphazero.lib.logger import setup_logger 36 | c = Config('mini') 37 | c.resource.create_directories() 38 | setup_logger(c.resource.main_log_path) 39 | start(c) 40 | 41 | def test_cli_play(): 42 | from cchess_alphazero.play_games.test_cli_game import main 43 | main() 44 | 45 | def test_gui_play(): 46 | from cchess_alphazero.play_games.test_window_game import main 47 | main() 48 | 49 | def test_optimise(): 50 | from cchess_alphazero.worker.optimize import start 51 | from cchess_alphazero.config import Config 52 | from cchess_alphazero.lib.logger import setup_logger 53 | c = Config('mini') 54 | c.resource.create_directories() 55 | setup_logger(c.resource.main_log_path) 56 | start(c) 57 | 58 | def test_light(): 59 | from cchess_alphazero.environment.light_env.chessboard import L_Chessboard 60 | from cchess_alphazero.environment.chessboard import Chessboard 61 | lboard = L_Chessboard() 62 | while not lboard.is_end(): 63 | for i in range(lboard.height): 64 | print(lboard.screen[i]) 65 | print(f"legal_moves = {lboard.legal_moves()}") 66 | action = input(f'Enter move for {lboard.is_red_turn} r/b: ') 67 | lboard.move_action_str(action) 68 | for i in range(lboard.height): 69 | print(lboard.screen[i]) 70 | print(lboard.winner) 71 | print(f"Turns = {lboard.steps / 2}") 72 | 73 | def test_light_env(): 74 | from cchess_alphazero.environment.env import CChessEnv 75 | from cchess_alphazero.config import Config 76 | c = Config('mini') 77 | env = CChessEnv(c) 78 | env.reset() 79 | print(env.observation) 80 | env.step('0001') 81 | print(env.observation) 82 | env.step('7770') 83 | print(env.observation) 84 | env.render() 85 | print(env.input_planes()[0+7:3+7]) 86 | 87 | def test_wxf(): 88 | from cchess_alphazero.environment.light_env.chessboard import L_Chessboard 89 | lboard = L_Chessboard() 90 | while not lboard.is_end(): 91 | for i in range(lboard.height): 92 | print(lboard.screen[i]) 93 | print(f"legal_moves = {lboard.legal_moves()}") 94 | wxf = input(f'Enter WXF move for {lboard.is_red_turn} r/b: ') 95 | action = lboard.parse_WXF_move(wxf) 96 | print(action) 97 | lboard.move_action_str(action) 98 | 99 | def test_sl(): 100 | from cchess_alphazero.worker import sl 101 | from cchess_alphazero.config import Config 102 | from cchess_alphazero.environment.lookup_tables import ActionLabelsRed, flip_policy, flip_move 103 | c = Config('mini') 104 | labels_n = len(ActionLabelsRed) 105 | move_lookup = {move: i for move, i in zip(ActionLabelsRed, range(labels_n))} 106 | slworker = sl.SupervisedWorker(c) 107 | p1 = slworker.build_policy('0001', False) 108 | print(p1[move_lookup['0001']]) 109 | p2 = slworker.build_policy('0001', True) 110 | print(p2[move_lookup[flip_move('0001')]]) 111 | 112 | def test_static_env(): 113 | from cchess_alphazero.environment.env import CChessEnv 114 | import cchess_alphazero.environment.static_env as senv 115 | from cchess_alphazero.environment.static_env import INIT_STATE 116 | from cchess_alphazero.environment.lookup_tables import flip_move 117 | env = CChessEnv() 118 | env.reset() 119 | print("env: " + env.observation) 120 | print("senv: " + INIT_STATE) 121 | state = INIT_STATE 122 | env.step('0001') 123 | state = senv.step(state, '0001') 124 | print(senv.evaluate(state)) 125 | print("env: " + env.observation) 126 | print("senv: " + state) 127 | env.step('7770') 128 | state = senv.step(state, flip_move('7770')) 129 | print(senv.evaluate(state)) 130 | print("env: " + env.observation) 131 | print("senv: " + state) 132 | env.render() 133 | board = senv.state_to_board(state) 134 | for i in range(9, -1, -1): 135 | print(board[i]) 136 | print("env: ") 137 | print(env.input_planes()[0+7:3+7]) 138 | print("senv: ") 139 | print(senv.state_to_planes(state)[0+7:3+7]) 140 | print(f"env: {env.board.legal_moves()}" ) 141 | print(f"senv: {senv.get_legal_moves(state)}") 142 | print(set(env.board.legal_moves()) == set(senv.get_legal_moves(state))) 143 | 144 | def test_onegreen(): 145 | import cchess_alphazero.environment.static_env as senv 146 | from cchess_alphazero.environment.lookup_tables import flip_move 147 | init = '9999299949999999249999869999999958999999519999999999999999997699' 148 | state = senv.init(init) 149 | print(state) 150 | senv.render(state) 151 | move = senv.parse_onegreen_move('8685') 152 | state = senv.step(state, move) 153 | print(state) 154 | senv.render(state) 155 | move = senv.parse_onegreen_move('7666') 156 | state = senv.step(state, flip_move(move)) 157 | print(state) 158 | senv.render(state) 159 | 160 | def test_onegreen2(): 161 | from cchess_alphazero.environment.env import CChessEnv 162 | import cchess_alphazero.environment.static_env as senv 163 | from cchess_alphazero.config import Config 164 | c = Config('mini') 165 | init = '9999299949999999249999869999999958999999519999999999999999997699' 166 | env = CChessEnv(c) 167 | env.reset(init) 168 | print(env.observation) 169 | env.render() 170 | move = senv.parse_onegreen_move('8685') 171 | env.step(move) 172 | print(env.observation) 173 | env.render() 174 | move = senv.parse_onegreen_move('7666') 175 | env.step(move) 176 | print(env.observation) 177 | env.render() 178 | 179 | def test_ucci(): 180 | import cchess_alphazero.environment.static_env as senv 181 | from cchess_alphazero.environment.lookup_tables import flip_move 182 | state = senv.INIT_STATE 183 | state = senv.step(state, '0001') 184 | fen = senv.state_to_fen(state, 1) 185 | print(fen) 186 | senv.render(state) 187 | move = 'b7b0' 188 | move = senv.parse_ucci_move(move) 189 | print(f'Parsed move {move}') 190 | move = flip_move(move) 191 | print(f'fliped move {move}') 192 | state = senv.step(state, move) 193 | senv.render(state) 194 | fen = senv.state_to_fen(state, 2) 195 | print(fen) 196 | 197 | def test_done(): 198 | import cchess_alphazero.environment.static_env as senv 199 | state = '4s4/9/4e4/p8/2e2R2p/P5E2/8P/9/9/4S1E2' 200 | board = senv.state_to_board(state) 201 | for i in range(9, -1, -1): 202 | print(board[i]) 203 | print(senv.done(state)) 204 | 205 | def test_upload(): 206 | from cchess_alphazero.lib.web_helper import upload_file 207 | from cchess_alphazero.config import Config 208 | c = Config('mini') 209 | url = 'http://alphazero.52coding.com.cn/api/upload_game_file' 210 | path = '/Users/liuhe/Documents/Graduation Project/ChineseChess-AlphaZero/data/play_data/test.json' 211 | filename = 'test.json' 212 | data = {'digest': 'test', 'username': 'test'} 213 | res = upload_file(url, path, filename=filename, data=data) 214 | print(res) 215 | 216 | def test_download(): 217 | from cchess_alphazero.lib.web_helper import download_file 218 | from cchess_alphazero.config import Config 219 | c = Config('mini') 220 | url = 'http://alphazero.52coding.com.cn/model_best_weight.h5' 221 | path = '/Users/liuhe/Downloads/model_best_weight.h5' 222 | res = download_file(url, path) 223 | print(res) 224 | 225 | def test_request(): 226 | from cchess_alphazero.lib.web_helper import http_request 227 | from cchess_alphazero.config import Config 228 | c = Config('mini') 229 | url = 'http://alphazero.52coding.com.cn/api/add_model' 230 | digest = 'd6fce85e040a63966fa7651d4a08a7cdba2ef0e5975fc16a6d178c96345547b3' 231 | elo = 0 232 | data = {'digest': digest, 'elo': elo} 233 | res = http_request(url, post=True, data=data) 234 | print(res) 235 | 236 | def fixbug(): 237 | from cchess_alphazero.config import Config 238 | from cchess_alphazero.lib.data_helper import get_game_data_filenames, read_game_data_from_file, write_game_data_to_file 239 | import cchess_alphazero.environment.static_env as senv 240 | c = Config('distribute') 241 | files = get_game_data_filenames(c.resource) 242 | cnt = 0 243 | fix = 0 244 | draw_cnt = 0 245 | for filename in files: 246 | try: 247 | data = read_game_data_from_file(filename) 248 | except: 249 | print(f"error: {filename}") 250 | os.remove(filename) 251 | continue 252 | state = data[0] 253 | real_data = [state] 254 | need_fix = True 255 | draw = False 256 | action = None 257 | value = None 258 | is_red_turn = True 259 | for item in data[1:]: 260 | action = item[0] 261 | value = -item[1] 262 | if value == 0: 263 | need_fix = False 264 | draw = True 265 | draw_cnt += 1 266 | break 267 | state = senv.step(state, action) 268 | is_red_turn = not is_red_turn 269 | real_data.append([action, value]) 270 | if not draw: 271 | game_over, v, final_move = senv.done(state) 272 | if final_move: 273 | v = -v 274 | is_red_turn = not is_red_turn 275 | if not is_red_turn: 276 | v = -v 277 | if not game_over: 278 | v = 1 279 | # print(game_over, v, final_move, state) 280 | if v == data[1][1]: 281 | need_fix = False 282 | else: 283 | need_fix = True 284 | if need_fix: 285 | write_game_data_to_file(filename, real_data) 286 | # print(filename) 287 | fix += 1 288 | cnt += 1 289 | if cnt % 1000 == 0: 290 | print(cnt, fix, draw_cnt) 291 | print(f"all {cnt}, fix {fix}, draw {draw_cnt}") 292 | 293 | 294 | def plot_model(): 295 | from keras.utils import plot_model 296 | from cchess_alphazero.agent.model import CChessModel 297 | from cchess_alphazero.config import Config 298 | from cchess_alphazero.lib.model_helper import save_as_best_model 299 | config = Config('distribute') 300 | model = CChessModel(config) 301 | model.build() 302 | save_as_best_model(model) 303 | plot_model(model.model, to_file='model.png', show_shapes=True, show_layer_names=True) 304 | 305 | def test_check_and_catch(): 306 | import cchess_alphazero.environment.static_env as senv 307 | state = senv.fen_to_state('rnba1cbnr/1a7/1c7/p1p3p1p/2p5k/2P1R4/P1P3P1P/1C5C1/9/RNBAKABN1 r') 308 | # state = senv.fliped_state(state) 309 | ori_state = state 310 | senv.render(state) 311 | print() 312 | action = '4454' 313 | state = senv.step(state, action) 314 | senv.render(state) 315 | state = senv.fliped_state(state) 316 | print() 317 | senv.render(state) 318 | print(senv.will_check_or_catch(ori_state, action)) 319 | 320 | def test_be_catched(): 321 | import cchess_alphazero.environment.static_env as senv 322 | state = senv.fen_to_state('rnbakab1r/9/1c3c2n/p1p5p/7p1/3PR4/P1P3P1P/C7C/9/RNBAKABN1 b') 323 | # state = senv.fliped_state(state) 324 | ori_state = state 325 | senv.render(state) 326 | print() 327 | action = '4454' 328 | print(senv.be_catched(ori_state, action)) 329 | 330 | 331 | if __name__ == "__main__": 332 | test_be_catched() 333 | 334 | -------------------------------------------------------------------------------- /cchess_alphazero/worker/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/cchess_alphazero/worker/__init__.py -------------------------------------------------------------------------------- /cchess_alphazero/worker/evaluator.py: -------------------------------------------------------------------------------- 1 | import os 2 | import shutil 3 | from collections import deque 4 | from concurrent.futures import ProcessPoolExecutor, wait 5 | from datetime import datetime 6 | from logging import getLogger 7 | from multiprocessing import Manager 8 | from threading import Thread 9 | from time import time, sleep 10 | from collections import defaultdict 11 | from multiprocessing import Lock 12 | from random import random, randint 13 | import numpy as np 14 | 15 | import cchess_alphazero.environment.static_env as senv 16 | from cchess_alphazero.agent.model import CChessModel 17 | from cchess_alphazero.agent.player import CChessPlayer, VisitState 18 | from cchess_alphazero.agent.api import CChessModelAPI 19 | from cchess_alphazero.config import Config 20 | from cchess_alphazero.environment.env import CChessEnv 21 | from cchess_alphazero.environment.lookup_tables import Winner, flip_move, ActionLabelsRed 22 | from cchess_alphazero.lib.data_helper import get_game_data_filenames, write_game_data_to_file 23 | from cchess_alphazero.lib.model_helper import load_model_weight 24 | from cchess_alphazero.lib.tf_util import set_session_config 25 | 26 | logger = getLogger(__name__) 27 | 28 | def start(config: Config): 29 | set_session_config(per_process_gpu_memory_fraction=1, allow_growth=True, device_list=config.opts.device_list) 30 | m = Manager() 31 | # while True: 32 | model_bt = load_model(config, config.resource.model_best_config_path, config.resource.model_best_weight_path) 33 | modelbt_pipes = m.list([model_bt.get_pipes(need_reload=False) for _ in range(config.play.max_processes)]) 34 | model_ng = load_model(config, config.resource.next_generation_config_path, config.resource.next_generation_weight_path) 35 | # while not model_ng: 36 | # logger.info(f"Next generation model is None, wait for 300s") 37 | # sleep(300) 38 | # model_ng = load_model(config, config.resource.next_generation_config_path, config.resource.next_generation_weight_path) 39 | logger.info(f"Next generation model has loaded!") 40 | modelng_pipes = m.list([model_ng.get_pipes(need_reload=False) for _ in range(config.play.max_processes)]) 41 | 42 | # play_worker = EvaluateWorker(config, model1_pipes, model2_pipes) 43 | # play_worker.start() 44 | with ProcessPoolExecutor(max_workers=config.play.max_processes) as executor: 45 | futures = [] 46 | for i in range(config.play.max_processes): 47 | eval_worker = EvaluateWorker(config, modelbt_pipes, modelng_pipes, pid=i) 48 | futures.append(executor.submit(eval_worker.start)) 49 | 50 | wait(futures) 51 | model_bt.close_pipes() 52 | model_ng.close_pipes() 53 | # compute whether to update best model 54 | # and remove next generation model 55 | total_score = 0 56 | red_new_win = 0 57 | red_new_fail = 0 58 | red_new_draw = 0 59 | black_new_win = 0 60 | black_new_fail = 0 61 | black_new_draw = 0 62 | for future in futures: 63 | data = future.result() 64 | total_score += data[0] 65 | red_new_win += data[1] 66 | red_new_draw += data[2] 67 | red_new_fail += data[3] 68 | black_new_win += data[4] 69 | black_new_draw += data[5] 70 | black_new_fail += data[6] 71 | game_num = config.eval.game_num * config.play.max_processes 72 | win_rate = total_score * 100 / game_num 73 | logger.info(f"Evaluate over, next generation win {total_score}/{game_num} = {win_rate:.2f}%") 74 | logger.info(f"红\t黑\t胜\t平\t负") 75 | logger.info(f"新\t旧\t{red_new_win}\t{red_new_draw}\t{red_new_fail}") 76 | logger.info(f"旧\t新\t{black_new_win}\t{black_new_draw}\t{black_new_fail}") 77 | # if total_score * 1.0 / game_num >= config.eval.next_generation_replace_rate: 78 | # logger.info("Best model will be replaced by next generation model") 79 | # replace_best_model(config) 80 | # else: 81 | # logger.info("Next generation fail to defeat best model and will be removed") 82 | # remove_ng_model(config) 83 | 84 | class EvaluateWorker: 85 | def __init__(self, config: Config, pipes1=None, pipes2=None, pid=None): 86 | self.config = config 87 | self.player_bt = None 88 | self.player_ng = None 89 | self.pid = pid 90 | self.pipes_bt = pipes1 91 | self.pipes_ng = pipes2 92 | 93 | def start(self): 94 | ran = self.config.play.max_processes * 2 95 | sleep((self.pid % ran) * 10) 96 | logger.debug(f"Evaluate#Start Process index = {self.pid}, pid = {os.getpid()}") 97 | score = 0 98 | total_score = 0 99 | red_new_win = 0 100 | red_new_fail = 0 101 | red_new_draw = 0 102 | black_new_win = 0 103 | black_new_fail = 0 104 | black_new_draw = 0 105 | 106 | for idx in range(self.config.eval.game_num): 107 | start_time = time() 108 | value, turns = self.start_game(idx) 109 | end_time = time() 110 | 111 | if (value == 1 and idx % 2 == 0) or (value == -1 and idx % 2 == 1): 112 | if idx % 2 == 0: 113 | black_new_fail += 1 114 | else: 115 | red_new_fail += 1 116 | result = '基准模型胜' 117 | elif (value == 1 and idx % 2 == 1) or (value == -1 and idx % 2 == 0): 118 | if idx % 2 == 0: 119 | black_new_win += 1 120 | else: 121 | red_new_win += 1 122 | result = '待评测模型胜' 123 | else: 124 | if idx % 2 == 0: 125 | black_new_draw += 1 126 | else: 127 | red_new_draw += 1 128 | result = '和棋' 129 | 130 | if value == -1: # loss 131 | score = 0 132 | elif value == 1: # win 133 | score = 1 134 | else: 135 | score = 0.5 136 | 137 | if idx % 2 == 0: 138 | score = 1 - score 139 | else: 140 | score = score 141 | 142 | logger.info(f"进程{self.pid}评测完毕 用时{(end_time - start_time):.1f}秒, " 143 | f"{turns / 2}回合, {result}, 得分：{score}, value = {value}, idx = {idx}") 144 | total_score += score 145 | return (total_score, red_new_win, red_new_draw, red_new_fail, black_new_win, black_new_draw, black_new_fail) 146 | 147 | def start_game(self, idx): 148 | pipe1 = self.pipes_bt.pop() 149 | pipe2 = self.pipes_ng.pop() 150 | search_tree1 = defaultdict(VisitState) 151 | search_tree2 = defaultdict(VisitState) 152 | 153 | playouts = randint(8, 12) * 100 154 | self.config.play.simulation_num_per_move = playouts 155 | logger.info(f"Set playouts = {self.config.play.simulation_num_per_move}") 156 | 157 | self.player1 = CChessPlayer(self.config, search_tree=search_tree1, pipes=pipe1, 158 | debugging=False, enable_resign=False) 159 | self.player2 = CChessPlayer(self.config, search_tree=search_tree2, pipes=pipe2, 160 | debugging=False, enable_resign=False) 161 | 162 | # even: bst = red, ng = black; odd: bst = black, ng = red 163 | if idx % 2 == 0: 164 | red = self.player1 165 | black = self.player2 166 | logger.debug(f"best model is red, ng is black") 167 | else: 168 | red = self.player2 169 | black = self.player1 170 | logger.debug(f"best model is black, ng is red") 171 | 172 | state = senv.INIT_STATE 173 | history = [state] 174 | value = 0 # best model's value 175 | turns = 0 # even == red; odd == black 176 | game_over = False 177 | no_eat_count = 0 178 | check = False 179 | 180 | while not game_over: 181 | start_time = time() 182 | no_act = None 183 | increase_temp = False 184 | if not check and state in history[:-1]: 185 | no_act = [] 186 | increase_temp = True 187 | free_move = defaultdict(int) 188 | for i in range(len(history) - 1): 189 | if history[i] == state: 190 | # 如果走了下一步是将军或捉：禁止走那步 191 | if senv.will_check_or_catch(state, history[i+1]): 192 | no_act.append(history[i + 1]) 193 | # 否则当作闲着处理 194 | else: 195 | free_move[state] += 1 196 | if free_move[state] >= 3: 197 | # 作和棋处理 198 | game_over = True 199 | value = 0 200 | logger.info("闲着循环三次，作和棋处理") 201 | break 202 | if game_over: 203 | break 204 | if turns % 2 == 0: 205 | action, _ = red.action(state, turns, no_act=no_act, increase_temp=increase_temp) 206 | else: 207 | action, _ = black.action(state, turns, no_act=no_act, increase_temp=increase_temp) 208 | end_time = time() 209 | if self.config.opts.log_move: 210 | logger.debug(f"进程id = {self.pid}, action = {action}, turns = {turns}, time = {(end_time-start_time):.1f}") 211 | if action is None: 212 | logger.debug(f"{turns % 2} (0 = red; 1 = black) has resigned!") 213 | value = -1 214 | break 215 | history.append(action) 216 | state, no_eat = senv.new_step(state, action) 217 | turns += 1 218 | if no_eat: 219 | no_eat_count += 1 220 | else: 221 | no_eat_count = 0 222 | history.append(state) 223 | 224 | if no_eat_count >= 120 or turns / 2 >= self.config.play.max_game_length: 225 | game_over = True 226 | value = 0 227 | else: 228 | game_over, value, final_move, check = senv.done(state, need_check=True) 229 | if not game_over: 230 | if not senv.has_attack_chessman(state): 231 | logger.info(f"双方无进攻子力，作和。state = {state}") 232 | game_over = True 233 | value = 0 234 | 235 | if final_move: 236 | history.append(final_move) 237 | state = senv.step(state, final_move) 238 | turns += 1 239 | value = - value 240 | history.append(state) 241 | 242 | self.player1.close() 243 | self.player2.close() 244 | 245 | if turns % 2 == 1: # black turn 246 | value = -value 247 | 248 | self.pipes_bt.append(pipe1) 249 | self.pipes_ng.append(pipe2) 250 | return value, turns 251 | 252 | 253 | def replace_best_model(config): 254 | rc = config.resource 255 | shutil.copyfile(rc.next_generation_config_path, rc.model_best_config_path) 256 | shutil.copyfile(rc.next_generation_weight_path, rc.model_best_weight_path) 257 | remove_ng_model(config) 258 | 259 | def remove_ng_model(config): 260 | rc = config.resource 261 | os.remove(rc.next_generation_config_path) 262 | os.remove(rc.next_generation_weight_path) 263 | 264 | def load_model(config, config_path, weight_path, name=None): 265 | model = CChessModel(config) 266 | if not load_model_weight(model, config_path, weight_path, name): 267 | return None 268 | return model 269 | 270 | -------------------------------------------------------------------------------- /cchess_alphazero/worker/optimize.py: -------------------------------------------------------------------------------- 1 | import os 2 | import time 3 | import gc 4 | import subprocess 5 | import shutil 6 | import numpy as np 7 | 8 | from collections import deque 9 | from concurrent.futures import ProcessPoolExecutor 10 | from datetime import datetime 11 | from logging import getLogger 12 | from time import sleep 13 | from random import shuffle 14 | from threading import Thread 15 | 16 | import cchess_alphazero.environment.static_env as senv 17 | from cchess_alphazero.agent.model import CChessModel 18 | from cchess_alphazero.config import Config 19 | from cchess_alphazero.lib.data_helper import get_game_data_filenames, read_game_data_from_file 20 | from cchess_alphazero.lib.model_helper import load_best_model_weight, save_as_best_model 21 | from cchess_alphazero.lib.model_helper import need_to_reload_best_model_weight, save_as_next_generation_model, save_as_best_model 22 | from cchess_alphazero.environment.env import CChessEnv 23 | from cchess_alphazero.environment.lookup_tables import Winner, ActionLabelsRed, flip_policy, flip_move 24 | from cchess_alphazero.lib.tf_util import set_session_config 25 | from cchess_alphazero.lib.web_helper import http_request 26 | 27 | from keras.optimizers import SGD 28 | from keras.callbacks import TensorBoard 29 | # from keras.utils import multi_gpu_model 30 | import keras.backend as K 31 | 32 | logger = getLogger(__name__) 33 | 34 | def start(config: Config): 35 | set_session_config(per_process_gpu_memory_fraction=1, allow_growth=True, device_list=config.opts.device_list) 36 | return OptimizeWorker(config).start() 37 | 38 | class OptimizeWorker: 39 | def __init__(self, config:Config): 40 | self.config = config 41 | self.model = None 42 | self.loaded_filenames = set() 43 | self.loaded_data = deque(maxlen=self.config.trainer.dataset_size) 44 | self.dataset = deque(), deque(), deque() 45 | self.executor = ProcessPoolExecutor(max_workers=config.trainer.cleaning_processes) 46 | self.filenames = [] 47 | self.opt = None 48 | self.count = 0 49 | self.eva = False 50 | 51 | def start(self): 52 | self.model = self.load_model() 53 | self.training() 54 | 55 | def training(self): 56 | self.compile_model() 57 | total_steps = self.config.trainer.start_total_steps 58 | bef_files = [] 59 | last_file = None 60 | 61 | while True: 62 | files = get_game_data_filenames(self.config.resource) 63 | offset = self.config.trainer.min_games_to_begin_learn 64 | if (len(files) < self.config.trainer.min_games_to_begin_learn \ 65 | or ((last_file is not None and last_file in files) and files.index(last_file) + 1 + offset > len(files))): 66 | # if last_file is not None: 67 | # logger.info('Waiting for enough data 300s, ' + str((len(files) - files.index(last_file)) * self.config.play_data.nb_game_in_file) \ 68 | # +' vs '+ str(self.config.trainer.min_games_to_begin_learn)+' games') 69 | # else: 70 | # logger.info('Waiting for enough data 300s, ' + str(len(files) * self.config.play_data.nb_game_in_file) \ 71 | # +' vs '+ str(self.config.trainer.min_games_to_begin_learn)+' games') 72 | # time.sleep(300) 73 | if last_file is not None: 74 | self.save_current_model(send=True) 75 | break 76 | else: 77 | if last_file is not None and last_file in files: 78 | idx = files.index(last_file) + 1 79 | if len(files) - idx > self.config.trainer.load_step: 80 | files = files[idx:idx + self.config.trainer.load_step] 81 | else: 82 | files = files[idx:] 83 | elif len(files) > self.config.trainer.load_step: 84 | files = files[0:self.config.trainer.load_step] 85 | last_file = files[-1] 86 | logger.info(f"Last file = {last_file}") 87 | logger.debug(f"files = {files[0:-1:2000]}") 88 | self.filenames = deque(files) 89 | logger.debug(f"Start training {len(self.filenames)} files") 90 | shuffle(self.filenames) 91 | self.fill_queue() 92 | self.update_learning_rate(total_steps) 93 | if len(self.dataset[0]) > self.config.trainer.batch_size: 94 | steps = self.train_epoch(self.config.trainer.epoch_to_checkpoint) 95 | total_steps += steps 96 | self.save_current_model(send=False) 97 | self.update_learning_rate(total_steps) 98 | self.count += 1 99 | a, b, c = self.dataset 100 | a.clear() 101 | b.clear() 102 | c.clear() 103 | del self.dataset, a, b, c 104 | gc.collect() 105 | self.dataset = deque(), deque(), deque() 106 | self.backup_play_data(files) 107 | 108 | def train_epoch(self, epochs): 109 | tc = self.config.trainer 110 | state_ary, policy_ary, value_ary = self.collect_all_loaded_data() 111 | tensorboard_cb = TensorBoard(log_dir="./logs", batch_size=tc.batch_size, histogram_freq=1) 112 | if self.config.opts.use_multiple_gpus: 113 | self.mg_model.fit(state_ary, [policy_ary, value_ary], 114 | batch_size=tc.batch_size, 115 | epochs=epochs, 116 | shuffle=True, 117 | validation_split=0.02, 118 | callbacks=[tensorboard_cb]) 119 | else: 120 | self.model.model.fit(state_ary, [policy_ary, value_ary], 121 | batch_size=tc.batch_size, 122 | epochs=epochs, 123 | shuffle=True, 124 | validation_split=0.02, 125 | callbacks=[tensorboard_cb]) 126 | steps = (state_ary.shape[0] // tc.batch_size) * epochs 127 | return steps 128 | 129 | def compile_model(self): 130 | self.opt = SGD(lr=0.02, momentum=self.config.trainer.momentum) 131 | losses = ['categorical_crossentropy', 'mean_squared_error'] 132 | if self.config.opts.use_multiple_gpus: 133 | self.mg_model = multi_gpu_model(self.model.model, gpus=self.config.opts.gpu_num) 134 | self.mg_model.compile(optimizer=self.opt, loss=losses, loss_weights=self.config.trainer.loss_weights) 135 | else: 136 | self.model.model.compile(optimizer=self.opt, loss=losses, loss_weights=self.config.trainer.loss_weights) 137 | 138 | def update_learning_rate(self, total_steps): 139 | # The deepmind paper says 140 | # ~400k: 1e-2 141 | # 400k~600k: 1e-3 142 | # 600k~: 1e-4 143 | 144 | lr = self.decide_learning_rate(total_steps) 145 | if lr: 146 | K.set_value(self.opt.lr, lr) 147 | logger.debug(f"total step={total_steps}, set learning rate to {lr}") 148 | 149 | def fill_queue(self): 150 | futures = deque() 151 | n = len(self.filenames) 152 | with ProcessPoolExecutor(max_workers=self.config.trainer.cleaning_processes) as executor: 153 | for _ in range(self.config.trainer.cleaning_processes): 154 | if len(self.filenames) == 0: 155 | break 156 | filename = self.filenames.pop() 157 | # logger.debug("loading data from %s" % (filename)) 158 | futures.append(executor.submit(load_data_from_file, filename, self.config.opts.has_history)) 159 | while futures and len(self.dataset[0]) < self.config.trainer.dataset_size: #fill tuples 160 | _tuple = futures.popleft().result() 161 | if _tuple is not None: 162 | for x, y in zip(self.dataset, _tuple): 163 | x.extend(y) 164 | m = len(self.filenames) 165 | if m > 0: 166 | if (n - m) % 1000 == 0: 167 | logger.info(f"Reading {n - m} files") 168 | filename = self.filenames.pop() 169 | # logger.debug("loading data from %s" % (filename)) 170 | futures.append(executor.submit(load_data_from_file, filename, self.config.opts.has_history)) 171 | 172 | def collect_all_loaded_data(self): 173 | state_ary, policy_ary, value_ary = self.dataset 174 | 175 | state_ary1 = np.asarray(state_ary, dtype=np.float32) 176 | policy_ary1 = np.asarray(policy_ary, dtype=np.float32) 177 | value_ary1 = np.asarray(value_ary, dtype=np.float32) 178 | return state_ary1, policy_ary1, value_ary1 179 | 180 | def load_model(self): 181 | model = CChessModel(self.config) 182 | if self.config.opts.new or not load_best_model_weight(model): 183 | model.build() 184 | save_as_best_model(model) 185 | return model 186 | 187 | def save_current_model(self, send=False): 188 | logger.info("Save as ng model") 189 | if not send: 190 | save_as_best_model(self.model) 191 | else: 192 | save_as_next_generation_model(self.model) 193 | 194 | def decide_learning_rate(self, total_steps): 195 | ret = None 196 | 197 | for step, lr in self.config.trainer.lr_schedules: 198 | if total_steps >= step: 199 | ret = lr 200 | return ret 201 | 202 | def try_reload_model(self): 203 | logger.debug("check model") 204 | if need_to_reload_best_model_weight(self.model): 205 | with self.model.graph.as_default(): 206 | load_best_model_weight(self.model) 207 | return True 208 | return False 209 | 210 | def backup_play_data(self, files): 211 | backup_folder = os.path.join(self.config.resource.data_dir, 'trained') 212 | cnt = 0 213 | if not os.path.exists(backup_folder): 214 | os.makedirs(backup_folder) 215 | for i in range(len(files)): 216 | try: 217 | shutil.move(files[i], backup_folder) 218 | except Exception as e: 219 | # logger.error(f"Backup error : {e}") 220 | cnt = cnt + 1 221 | logger.info(f"backup {len(files)} files, {cnt} empty files") 222 | 223 | def load_data_from_file(filename, use_history=False): 224 | try: 225 | data = read_game_data_from_file(filename) 226 | except Exception as e: 227 | logger.error(f"Error when loading data {e}") 228 | os.remove(filename) 229 | return None 230 | if data is None: 231 | return None 232 | return expanding_data(data, use_history) 233 | 234 | def expanding_data(data, use_history=False): 235 | state = data[0] 236 | real_data = [] 237 | action = None 238 | policy = None 239 | value = None 240 | if use_history: 241 | history = [state] 242 | else: 243 | history = None 244 | for item in data[1:]: 245 | action = item[0] 246 | value = item[1] 247 | try: 248 | policy = build_policy(action, flip=False) 249 | except Exception as e: 250 | logger.error(f"Expand data error {e}, item = {item}, data = {data}, state = {state}") 251 | return None 252 | real_data.append([state, policy, value]) 253 | state = senv.step(state, action) 254 | if use_history: 255 | history.append(action) 256 | history.append(state) 257 | 258 | return convert_to_trainging_data(real_data, history) 259 | 260 | 261 | def convert_to_trainging_data(data, history): 262 | state_list = [] 263 | policy_list = [] 264 | value_list = [] 265 | i = 0 266 | 267 | for state, policy, value in data: 268 | if history is None: 269 | state_planes = senv.state_to_planes(state) 270 | else: 271 | state_planes = senv.state_history_to_planes(state, history[0:i * 2 + 1]) 272 | sl_value = value 273 | 274 | state_list.append(state_planes) 275 | policy_list.append(policy) 276 | value_list.append(sl_value) 277 | i += 1 278 | 279 | return np.asarray(state_list, dtype=np.float32), \ 280 | np.asarray(policy_list, dtype=np.float32), \ 281 | np.asarray(value_list, dtype=np.float32) 282 | 283 | def build_policy(action, flip): 284 | labels_n = len(ActionLabelsRed) 285 | move_lookup = {move: i for move, i in zip(ActionLabelsRed, range(labels_n))} 286 | policy = np.zeros(labels_n) 287 | 288 | policy[move_lookup[action]] = 1 289 | 290 | if flip: 291 | policy = flip_policy(policy) 292 | return list(policy) 293 | 294 | 295 | 296 | -------------------------------------------------------------------------------- /cchess_alphazero/worker/play_with_ucci_engine.py: -------------------------------------------------------------------------------- 1 | import os 2 | import gc 3 | import subprocess 4 | import numpy as np 5 | from time import sleep 6 | from collections import deque 7 | from concurrent.futures import ProcessPoolExecutor 8 | from datetime import datetime 9 | from logging import getLogger 10 | from multiprocessing import Manager 11 | from time import time, sleep 12 | from collections import defaultdict 13 | from random import random 14 | 15 | import cchess_alphazero.environment.static_env as senv 16 | from cchess_alphazero.agent.model import CChessModel 17 | from cchess_alphazero.agent.player import CChessPlayer, VisitState 18 | from cchess_alphazero.agent.api import CChessModelAPI 19 | from cchess_alphazero.config import Config 20 | from cchess_alphazero.environment.env import CChessEnv 21 | from cchess_alphazero.environment.lookup_tables import ActionLabelsRed, flip_policy, flip_move 22 | from cchess_alphazero.lib.data_helper import get_game_data_filenames, write_game_data_to_file 23 | from cchess_alphazero.lib.model_helper import load_best_model_weight, save_as_best_model 24 | from cchess_alphazero.lib.tf_util import set_session_config 25 | 26 | logger = getLogger(__name__) 27 | 28 | def load_model(config): 29 | model = CChessModel(config) 30 | if config.opts.new or not load_best_model_weight(model): 31 | model.build() 32 | save_as_best_model(model) 33 | return model 34 | 35 | def start(config: Config): 36 | set_session_config(per_process_gpu_memory_fraction=1, allow_growth=True, device_list=config.opts.device_list) 37 | current_model = load_model(config) 38 | m = Manager() 39 | cur_pipes = m.list([current_model.get_pipes() for _ in range(config.play.max_processes)]) 40 | 41 | # play_worker = SelfPlayWorker(config, cur_pipes, 0) 42 | # play_worker.start() 43 | with ProcessPoolExecutor(max_workers=config.play.max_processes) as executor: 44 | futures = [] 45 | for i in range(config.play.max_processes): 46 | play_worker = SelfPlayWorker(config, cur_pipes, i) 47 | logger.debug("Initialize selfplay worker") 48 | futures.append(executor.submit(play_worker.start)) 49 | 50 | class SelfPlayWorker: 51 | def __init__(self, config: Config, pipes=None, pid=None): 52 | self.config = config 53 | self.player = None 54 | self.cur_pipes = pipes 55 | self.id = pid 56 | self.buffer = [] 57 | self.pid = os.getpid() 58 | 59 | def start(self): 60 | self.pid = os.getpid() 61 | logger.debug(f"Selfplay#Start Process index = {self.id}, pid = {self.pid}") 62 | 63 | idx = 1 64 | self.buffer = [] 65 | 66 | while True: 67 | search_tree = defaultdict(VisitState) 68 | start_time = time() 69 | value, turns, state, store = self.start_game(idx, search_tree) 70 | end_time = time() 71 | if value != 1 and value != -1: 72 | winner = 'Draw' 73 | elif idx % 2 == 0 and value == 1 or idx % 2 == 1 and value == -1: 74 | winner = 'AlphaHe' 75 | else: 76 | winner = 'Eleeye' 77 | 78 | logger.debug(f"Process {self.pid}-{self.id} play game {idx} time={(end_time - start_time):.1f} sec, " 79 | f"turn={turns / 2}, value = {value:.2f}, winner is {winner}") 80 | if turns <= 10 and store: 81 | senv.render(state) 82 | if store: 83 | idx += 1 84 | 85 | def start_game(self, idx, search_tree): 86 | pipes = self.cur_pipes.pop() 87 | 88 | if not self.config.play.share_mtcs_info_in_self_play or \ 89 | idx % self.config.play.reset_mtcs_info_per_game == 0: 90 | search_tree = defaultdict(VisitState) 91 | 92 | if random() > self.config.play.enable_resign_rate: 93 | enable_resign = True 94 | else: 95 | enable_resign = False 96 | 97 | self.player = CChessPlayer(self.config, search_tree=search_tree, pipes=pipes, enable_resign=enable_resign, debugging=False) 98 | 99 | state = senv.INIT_STATE 100 | history = [state] 101 | value = 0 102 | turns = 0 # even == red; odd == black 103 | game_over = False 104 | is_alpha_red = True if idx % 2 == 0 else False 105 | final_move = None 106 | check = False 107 | 108 | while not game_over: 109 | if (is_alpha_red and turns % 2 == 0) or (not is_alpha_red and turns % 2 == 1): 110 | no_act = None 111 | if not check and state in history[:-1]: 112 | no_act = [] 113 | for i in range(len(history) - 1): 114 | if history[i] == state: 115 | no_act.append(history[i + 1]) 116 | action, _ = self.player.action(state, turns, no_act) 117 | if action is None: 118 | logger.debug(f"{turns % 2} (0 = red; 1 = black) has resigned!") 119 | value = -1 120 | break 121 | else: 122 | fen = senv.state_to_fen(state, turns) 123 | action = self.get_ucci_move(fen) 124 | if action is None: 125 | logger.debug(f"{turns % 2} (0 = red; 1 = black) has resigned!") 126 | value = -1 127 | break 128 | if turns % 2 == 1: 129 | action = flip_move(action) 130 | history.append(action) 131 | state = senv.step(state, action) 132 | turns += 1 133 | history.append(state) 134 | 135 | if turns / 2 >= self.config.play.max_game_length: 136 | game_over = True 137 | value = 0 138 | else: 139 | game_over, value, final_move, check = senv.done(state, need_check=True) 140 | 141 | if final_move: 142 | history.append(final_move) 143 | state = senv.step(state, final_move) 144 | history.append(state) 145 | turns += 1 146 | value = -value 147 | 148 | self.player.close() 149 | del search_tree 150 | del self.player 151 | gc.collect() 152 | if turns % 2 == 1: # balck turn 153 | value = -value 154 | 155 | v = value 156 | if turns <= 10: 157 | if random() > 0.7: 158 | store = True 159 | else: 160 | store = False 161 | else: 162 | store = True 163 | 164 | if store: 165 | data = [history[0]] 166 | for i in range(turns): 167 | k = i * 2 168 | data.append([history[k + 1], value]) 169 | value = -value 170 | self.save_play_data(idx, data) 171 | 172 | self.cur_pipes.append(pipes) 173 | self.remove_play_data() 174 | return v, turns, state, store 175 | 176 | def get_ucci_move(self, fen, time=3): 177 | p = subprocess.Popen(self.config.resource.eleeye_path, 178 | stdin=subprocess.PIPE, 179 | stdout=subprocess.PIPE, 180 | stderr=subprocess.PIPE, 181 | universal_newlines=True) 182 | setfen = f'position fen {fen}\n' 183 | setrandom = f'setoption randomness {self.config.opts.random}\n' 184 | cmd = 'ucci\n' + setrandom + setfen + f'go time {time * 1000}\n' 185 | try: 186 | out, err = p.communicate(cmd, timeout=time+0.5) 187 | except subprocess.TimeoutExpired: 188 | p.kill() 189 | try: 190 | out, err = p.communicate() 191 | except Exception as e: 192 | logger.error(f"{e}, cmd = {cmd}") 193 | return self.get_ucci_move(fen, time+1) 194 | lines = out.split('\n') 195 | if lines[-2] == 'nobestmove': 196 | return None 197 | move = lines[-2].split(' ')[1] 198 | if move == 'depth': 199 | move = lines[-1].split(' ')[6] 200 | return senv.parse_ucci_move(move) 201 | 202 | def save_play_data(self, idx, data): 203 | self.buffer += data 204 | 205 | if not idx % self.config.play_data.nb_game_in_file == 0: 206 | return 207 | 208 | rc = self.config.resource 209 | game_id = datetime.now().strftime("%Y%m%d-%H%M%S.%f") 210 | path = os.path.join(rc.play_data_dir, rc.play_data_filename_tmpl % game_id) 211 | logger.info(f"Process {self.pid} save play data to {path}") 212 | write_game_data_to_file(path, self.buffer) 213 | self.buffer = [] 214 | 215 | def remove_play_data(self): 216 | files = get_game_data_filenames(self.config.resource) 217 | if len(files) < self.config.play_data.max_file_num: 218 | return 219 | try: 220 | for i in range(len(files) - self.config.play_data.max_file_num): 221 | os.remove(files[i]) 222 | except: 223 | pass 224 | 225 | def build_policy(self, action, flip): 226 | labels_n = len(ActionLabelsRed) 227 | move_lookup = {move: i for move, i in zip(ActionLabelsRed, range(labels_n))} 228 | policy = np.zeros(labels_n) 229 | 230 | policy[move_lookup[action]] = 1 231 | 232 | if flip: 233 | policy = flip_policy(policy) 234 | return list(policy) 235 | 236 | -------------------------------------------------------------------------------- /cchess_alphazero/worker/self_play.py: -------------------------------------------------------------------------------- 1 | import os 2 | import gc 3 | import numpy as np 4 | from time import sleep 5 | from collections import deque 6 | from concurrent.futures import ProcessPoolExecutor 7 | from datetime import datetime, timezone, timedelta 8 | from logging import getLogger 9 | from multiprocessing import Manager 10 | from time import time, sleep 11 | from collections import defaultdict 12 | from random import random 13 | from threading import Thread 14 | 15 | import cchess_alphazero.environment.static_env as senv 16 | from cchess_alphazero.agent.model import CChessModel 17 | from cchess_alphazero.agent.player import CChessPlayer, VisitState 18 | from cchess_alphazero.agent.api import CChessModelAPI 19 | from cchess_alphazero.config import Config 20 | from cchess_alphazero.environment.env import CChessEnv 21 | from cchess_alphazero.environment.lookup_tables import Winner, ActionLabelsRed, flip_policy, flip_move 22 | from cchess_alphazero.lib.data_helper import get_game_data_filenames, write_game_data_to_file 23 | from cchess_alphazero.lib.model_helper import load_model_weight, save_as_best_model, load_best_model_weight_from_internet 24 | from cchess_alphazero.lib.tf_util import set_session_config 25 | from cchess_alphazero.lib.web_helper import upload_file 26 | 27 | logger = getLogger(__name__) 28 | 29 | def load_model(config, config_file=None): 30 | use_history = False 31 | model = CChessModel(config) 32 | weight_path = config.resource.model_best_weight_path 33 | if not config_file: 34 | config_path = config.resource.model_best_config_path 35 | use_history = False 36 | else: 37 | config_path = os.path.join(config.resource.model_dir, config_file) 38 | try: 39 | if not load_model_weight(model, config_path, weight_path): 40 | model.build() 41 | save_as_best_model(model) 42 | use_history = True 43 | except Exception as e: 44 | logger.info(f"Exception {e}, 重新加载权重") 45 | return load_model(config, config_file='model_192x10_config.json') 46 | return model, use_history 47 | 48 | def start(config: Config): 49 | set_session_config(per_process_gpu_memory_fraction=1, allow_growth=True, device_list=config.opts.device_list) 50 | current_model, use_history = load_model(config) 51 | m = Manager() 52 | cur_pipes = m.list([current_model.get_pipes() for _ in range(config.play.max_processes)]) 53 | # play_worker = SelfPlayWorker(config, cur_pipes, 0) 54 | # play_worker.start() 55 | with ProcessPoolExecutor(max_workers=config.play.max_processes) as executor: 56 | futures = [] 57 | for i in range(config.play.max_processes): 58 | play_worker = SelfPlayWorker(config, cur_pipes, i, use_history) 59 | logger.debug("Initialize selfplay worker") 60 | futures.append(executor.submit(play_worker.start)) 61 | 62 | class SelfPlayWorker: 63 | def __init__(self, config: Config, pipes=None, pid=None, use_history=False): 64 | self.config = config 65 | self.player = None 66 | self.cur_pipes = pipes 67 | self.id = pid 68 | self.buffer = [] 69 | self.pid = os.getpid() 70 | self.use_history = use_history 71 | 72 | def start(self): 73 | self.pid = os.getpid() 74 | ran = self.config.play.max_processes if self.config.play.max_processes > 5 else self.config.play.max_processes * 2 75 | sleep((self.pid % ran) * 10) 76 | logger.debug(f"Selfplay#Start Process index = {self.id}, pid = {self.pid}") 77 | 78 | idx = 1 79 | self.buffer = [] 80 | search_tree = defaultdict(VisitState) 81 | 82 | while True: 83 | start_time = time() 84 | search_tree = defaultdict(VisitState) 85 | value, turns, state, store = self.start_game(idx, search_tree) 86 | end_time = time() 87 | logger.debug(f"Process {self.pid}-{self.id} play game {idx} time={(end_time - start_time):.1f} sec, " 88 | f"turn={turns / 2}, winner = {value:.2f} (1 = red, -1 = black, 0 draw)") 89 | if turns <= 10: 90 | senv.render(state) 91 | if store: 92 | idx += 1 93 | sleep(random()) 94 | 95 | def start_game(self, idx, search_tree): 96 | pipes = self.cur_pipes.pop() 97 | 98 | if not self.config.play.share_mtcs_info_in_self_play or \ 99 | idx % self.config.play.reset_mtcs_info_per_game == 0: 100 | search_tree = defaultdict(VisitState) 101 | 102 | if random() > self.config.play.enable_resign_rate: 103 | enable_resign = True 104 | else: 105 | enable_resign = False 106 | 107 | self.player = CChessPlayer(self.config, search_tree=search_tree, pipes=pipes, 108 | enable_resign=enable_resign, debugging=False, use_history=self.use_history) 109 | 110 | state = senv.INIT_STATE 111 | history = [state] 112 | # policys = [] 113 | value = 0 114 | turns = 0 # even == red; odd == black 115 | game_over = False 116 | final_move = None 117 | no_eat_count = 0 118 | check = False 119 | no_act = [] 120 | increase_temp = False 121 | 122 | while not game_over: 123 | start_time = time() 124 | action, policy = self.player.action(state, turns, no_act, increase_temp=increase_temp) 125 | end_time = time() 126 | if action is None: 127 | logger.debug(f"{turns % 2} (0 = red; 1 = black) has resigned!") 128 | value = -1 129 | break 130 | # if self.config.opts.log_move: 131 | # logger.info(f"Process{self.pid} Playing: {turns % 2}, action: {action}, time: {(end_time - start_time):.1f}s") 132 | # logger.info(f"Process{self.pid} Playing: {turns % 2}, action: {action}, time: {(end_time - start_time):.1f}s") 133 | history.append(action) 134 | # policys.append(policy) 135 | try: 136 | state, no_eat = senv.new_step(state, action) 137 | except Exception as e: 138 | logger.error(f"{e}, no_act = {no_act}, policy = {policy}") 139 | game_over = True 140 | value = 0 141 | break 142 | turns += 1 143 | if no_eat: 144 | no_eat_count += 1 145 | else: 146 | no_eat_count = 0 147 | history.append(state) 148 | 149 | if no_eat_count >= 120 or turns / 2 >= self.config.play.max_game_length: 150 | game_over = True 151 | value = 0 152 | else: 153 | game_over, value, final_move, check = senv.done(state, need_check=True) 154 | if not game_over: 155 | if not senv.has_attack_chessman(state): 156 | logger.info(f"双方无进攻子力，作和。state = {state}") 157 | game_over = True 158 | value = 0 159 | increase_temp = False 160 | no_act = [] 161 | if not game_over and not check and state in history[:-1]: 162 | free_move = defaultdict(int) 163 | for i in range(len(history) - 1): 164 | if history[i] == state: 165 | if senv.will_check_or_catch(state, history[i+1]): 166 | no_act.append(history[i + 1]) 167 | elif not senv.be_catched(state, history[i+1]): 168 | increase_temp = True 169 | free_move[state] += 1 170 | if free_move[state] >= 3: 171 | # 作和棋处理 172 | game_over = True 173 | value = 0 174 | logger.info("闲着循环三次，作和棋处理") 175 | break 176 | 177 | if final_move: 178 | # policy = self.build_policy(final_move, False) 179 | history.append(final_move) 180 | # policys.append(policy) 181 | state = senv.step(state, final_move) 182 | turns += 1 183 | value = -value 184 | history.append(state) 185 | 186 | self.player.close() 187 | del search_tree 188 | del self.player 189 | gc.collect() 190 | if turns % 2 == 1: # balck turn 191 | value = -value 192 | 193 | v = value 194 | if turns < 10: 195 | if random() > 0.9: 196 | store = True 197 | else: 198 | store = False 199 | else: 200 | store = True 201 | 202 | if store: 203 | data = [history[0]] 204 | for i in range(turns): 205 | k = i * 2 206 | data.append([history[k + 1], value]) 207 | value = -value 208 | self.save_play_data(idx, data) 209 | 210 | self.cur_pipes.append(pipes) 211 | self.remove_play_data() 212 | return v, turns, state, store 213 | 214 | def save_play_data(self, idx, data): 215 | self.buffer += data 216 | 217 | if not idx % self.config.play_data.nb_game_in_file == 0: 218 | return 219 | 220 | rc = self.config.resource 221 | utc_dt = datetime.utcnow().replace(tzinfo=timezone.utc) 222 | bj_dt = utc_dt.astimezone(timezone(timedelta(hours=8))) 223 | game_id = bj_dt.strftime("%Y%m%d-%H%M%S.%f") 224 | filename = rc.play_data_filename_tmpl % game_id 225 | path = os.path.join(rc.play_data_dir, filename) 226 | logger.info(f"Process {self.pid} save play data to {path}") 227 | write_game_data_to_file(path, self.buffer) 228 | if self.config.internet.distributed: 229 | upload_worker = Thread(target=self.upload_play_data, args=(path, filename), name="upload_worker") 230 | upload_worker.daemon = True 231 | upload_worker.start() 232 | self.buffer = [] 233 | 234 | def upload_play_data(self, path, filename): 235 | digest = CChessModel.fetch_digest(self.config.resource.model_best_weight_path) 236 | data = {'digest': digest, 'username': self.config.internet.username, 'version': '2.4'} 237 | response = upload_file(self.config.internet.upload_url, path, filename, data, rm=False) 238 | if response is not None and response['status'] == 0: 239 | logger.info(f"Upload play data {filename} finished.") 240 | else: 241 | logger.error(f'Upload play data {filename} failed. {response.msg if response is not None else None}') 242 | 243 | def remove_play_data(self): 244 | files = get_game_data_filenames(self.config.resource) 245 | if len(files) < self.config.play_data.max_file_num: 246 | return 247 | try: 248 | for i in range(len(files) - self.config.play_data.max_file_num): 249 | os.remove(files[i]) 250 | except: 251 | pass 252 | 253 | def build_policy(self, action, flip): 254 | labels_n = len(ActionLabelsRed) 255 | move_lookup = {move: i for move, i in zip(ActionLabelsRed, range(labels_n))} 256 | policy = np.zeros(labels_n) 257 | 258 | policy[move_lookup[action]] = 1 259 | 260 | if flip: 261 | policy = flip_policy(policy) 262 | return list(policy) 263 | 264 | -------------------------------------------------------------------------------- /cchess_alphazero/worker/self_play_windows.py: -------------------------------------------------------------------------------- 1 | import os 2 | import gc 3 | import numpy as np 4 | from collections import deque 5 | from concurrent.futures import ProcessPoolExecutor 6 | from datetime import datetime 7 | from logging import getLogger 8 | from multiprocessing import Manager 9 | from threading import Thread 10 | from time import time 11 | from collections import defaultdict 12 | from threading import Lock 13 | from time import sleep 14 | from random import random 15 | 16 | import cchess_alphazero.environment.static_env as senv 17 | from cchess_alphazero.agent.model import CChessModel 18 | from cchess_alphazero.agent.player import CChessPlayer, VisitState 19 | from cchess_alphazero.agent.api import CChessModelAPI 20 | from cchess_alphazero.config import Config 21 | from cchess_alphazero.environment.env import CChessEnv 22 | from cchess_alphazero.environment.lookup_tables import Winner, ActionLabelsRed, flip_policy, flip_move 23 | from cchess_alphazero.lib.data_helper import get_game_data_filenames, write_game_data_to_file 24 | from cchess_alphazero.lib.model_helper import load_model_weight, save_as_best_model, load_best_model_weight_from_internet 25 | from cchess_alphazero.lib.tf_util import set_session_config 26 | from cchess_alphazero.lib.web_helper import upload_file 27 | 28 | logger = getLogger(__name__) 29 | 30 | job_done = Lock() 31 | thr_free = Lock() 32 | rst = None 33 | data = None 34 | futures =[] 35 | 36 | def start(config: Config): 37 | set_session_config(per_process_gpu_memory_fraction=1, allow_growth=True, device_list=config.opts.device_list) 38 | return SelfPlayWorker(config).start() 39 | 40 | class SelfPlayWorker: 41 | def __init__(self, config: Config): 42 | """ 43 | :param config: 44 | """ 45 | self.config = config 46 | self.current_model, self.use_history = self.load_model() 47 | self.m = Manager() 48 | self.cur_pipes = self.m.list([self.current_model.get_pipes() for _ in range(self.config.play.max_processes)]) 49 | 50 | def start(self): 51 | global job_done 52 | global thr_free 53 | global rst 54 | global data 55 | global futures 56 | 57 | self.buffer = [] 58 | need_to_renew_model = True 59 | job_done.acquire(True) 60 | logger.info(f"自我博弈开始，请耐心等待....") 61 | 62 | with ProcessPoolExecutor(max_workers=self.config.play.max_processes) as executor: 63 | game_idx = 0 64 | while True: 65 | game_idx += 1 66 | start_time = time() 67 | 68 | if len(futures) == 0: 69 | for i in range(self.config.play.max_processes): 70 | ff = executor.submit(self_play_buffer, self.config, self.cur_pipes, self.use_history) 71 | ff.add_done_callback(recall_fn) 72 | futures.append(ff) 73 | 74 | job_done.acquire(True) 75 | 76 | end_time = time() 77 | 78 | turns = rst[0] 79 | value = rst[1] 80 | logger.debug(f"对局完成：对局ID {game_idx} 耗时{(end_time - start_time):.1f} 秒, " 81 | f"{turns / 2}回合, 胜者 = {value:.2f} (1 = 红, -1 = 黑, 0 = 和)") 82 | self.buffer += data 83 | 84 | if (game_idx % self.config.play_data.nb_game_in_file) == 0: 85 | self.flush_buffer() 86 | self.remove_play_data(all=False) # remove old data 87 | ff = executor.submit(self_play_buffer, self.config, self.cur_pipes, self.use_history) 88 | ff.add_done_callback(recall_fn) 89 | futures.append(ff) # Keep it going 90 | thr_free.release() 91 | 92 | if len(data) > 0: 93 | self.flush_buffer() 94 | 95 | def load_model(self, config_file=None): 96 | use_history = False 97 | model = CChessModel(self.config) 98 | weight_path = self.config.resource.model_best_weight_path 99 | if not config_file: 100 | config_path = self.config.resource.model_best_config_path 101 | use_history = False 102 | else: 103 | config_path = os.path.join(self.config.resource.model_dir, config_file) 104 | try: 105 | if not load_model_weight(model, config_path, weight_path): 106 | model.build() 107 | save_as_best_model(model) 108 | use_history = True 109 | except Exception as e: 110 | logger.info(f"Exception {e}, 重新加载权重") 111 | return self.load_model(config_file='model_192x10_config.json') 112 | return model, use_history 113 | 114 | def flush_buffer(self): 115 | rc = self.config.resource 116 | game_id = datetime.now().strftime("%Y%m%d-%H%M%S.%f") 117 | filename = rc.play_data_filename_tmpl % game_id 118 | path = os.path.join(rc.play_data_dir, filename) 119 | logger.info("保存博弈数据到 %s" % (path)) 120 | write_game_data_to_file(path, self.buffer) 121 | if self.config.internet.distributed: 122 | upload_worker = Thread(target=self.upload_play_data, args=(path, filename)) 123 | upload_worker.start() 124 | self.buffer = [] 125 | 126 | def remove_play_data(self,all=False): 127 | files = get_game_data_filenames(self.config.resource) 128 | if (all): 129 | for path in files: 130 | os.remove(path) 131 | else: 132 | while len(files) > self.config.play_data.max_file_num: 133 | os.remove(files[0]) 134 | del files[0] 135 | 136 | def upload_play_data(self, path, filename): 137 | digest = CChessModel.fetch_digest(self.config.resource.model_best_weight_path) 138 | data = {'digest': digest, 'username': self.config.internet.username, 'version': '2.4'} 139 | response = upload_file(self.config.internet.upload_url, path, filename, data, rm=False) 140 | if response is not None and response['status'] == 0: 141 | logger.info(f"上传博弈数据 {filename} 成功.") 142 | else: 143 | logger.error(f'上传博弈数据 {filename} 失败. {response.msg if response is not None else None}') 144 | 145 | def recall_fn(future): 146 | global thr_free 147 | global job_done 148 | global rst 149 | global data 150 | global futures 151 | 152 | thr_free.acquire(True) 153 | rst, data = future.result() 154 | futures.remove(future) 155 | job_done.release() 156 | 157 | def self_play_buffer(config, cur, use_history=False) -> (tuple, list): 158 | pipe = cur.pop() # borrow 159 | 160 | if random() > config.play.enable_resign_rate: 161 | enable_resign = True 162 | else: 163 | enable_resign = False 164 | 165 | player = CChessPlayer(config, search_tree=defaultdict(VisitState), pipes=pipe, 166 | enable_resign=enable_resign, debugging=False, use_history=use_history) 167 | 168 | state = senv.INIT_STATE 169 | history = [state] 170 | # policys = [] 171 | value = 0 172 | turns = 0 173 | game_over = False 174 | final_move = None 175 | no_eat_count = 0 176 | check = False 177 | no_act = None 178 | increase_temp = False 179 | 180 | while not game_over: 181 | start_time = time() 182 | action, policy = player.action(state, turns, no_act, increase_temp=increase_temp) 183 | end_time = time() 184 | if action is None: 185 | print(f"{turns % 2} (0 = 红; 1 = 黑) 投降了!") 186 | value = -1 187 | break 188 | print(f"博弈中: 回合{turns / 2 + 1} {'红方走棋' if turns % 2 == 0 else '黑方走棋'}, 着法: {action}, 用时: {(end_time - start_time):.1f}s") 189 | # policys.append(policy) 190 | history.append(action) 191 | try: 192 | state, no_eat = senv.new_step(state, action) 193 | except Exception as e: 194 | logger.error(f"{e}, no_act = {no_act}, policy = {policy}") 195 | game_over = True 196 | value = 0 197 | break 198 | turns += 1 199 | if no_eat: 200 | no_eat_count += 1 201 | else: 202 | no_eat_count = 0 203 | history.append(state) 204 | 205 | if no_eat_count >= 120 or turns / 2 >= config.play.max_game_length: 206 | game_over = True 207 | value = 0 208 | else: 209 | game_over, value, final_move, check = senv.done(state, need_check=True) 210 | no_act = [] 211 | increase_temp = False 212 | if not game_over: 213 | if not senv.has_attack_chessman(state): 214 | logger.info(f"双方无进攻子力，作和。state = {state}") 215 | game_over = True 216 | value = 0 217 | if not game_over and not check and state in history[:-1]: 218 | free_move = defaultdict(int) 219 | for i in range(len(history) - 1): 220 | if history[i] == state: 221 | if senv.will_check_or_catch(state, history[i+1]): 222 | no_act.append(history[i + 1]) 223 | elif not senv.be_catched(state, history[i+1]): 224 | increase_temp = True 225 | free_move[state] += 1 226 | if free_move[state] >= 3: 227 | # 作和棋处理 228 | game_over = True 229 | value = 0 230 | logger.info("闲着循环三次，作和棋处理") 231 | break 232 | 233 | if final_move: 234 | # policy = build_policy(final_move, False) 235 | history.append(final_move) 236 | # policys.append(policy) 237 | state = senv.step(state, final_move) 238 | turns += 1 239 | value = -value 240 | history.append(state) 241 | 242 | player.close() 243 | del player 244 | gc.collect() 245 | 246 | if turns % 2 == 1: # balck turn 247 | value = -value 248 | 249 | v = value 250 | data = [history[0]] 251 | for i in range(turns): 252 | k = i * 2 253 | data.append([history[k + 1], value]) 254 | value = -value 255 | 256 | cur.append(pipe) 257 | return (turns, v), data 258 | 259 | def build_policy(action, flip): 260 | labels_n = len(ActionLabelsRed) 261 | move_lookup = {move: i for move, i in zip(ActionLabelsRed, range(labels_n))} 262 | policy = np.zeros(labels_n) 263 | 264 | policy[move_lookup[action]] = 1 265 | 266 | if flip: 267 | policy = flip_policy(policy) 268 | return list(policy) 269 | -------------------------------------------------------------------------------- /cchess_alphazero/worker/sl.py: -------------------------------------------------------------------------------- 1 | import os 2 | import numpy as np 3 | import pandas as pd 4 | 5 | from collections import deque 6 | from concurrent.futures import ProcessPoolExecutor 7 | from datetime import datetime 8 | from logging import getLogger 9 | from time import sleep 10 | from random import shuffle 11 | from time import time 12 | 13 | from cchess_alphazero.agent.model import CChessModel 14 | from cchess_alphazero.config import Config 15 | from cchess_alphazero.lib.data_helper import get_game_data_filenames, read_game_data_from_file 16 | from cchess_alphazero.lib.model_helper import load_sl_best_model_weight, save_as_sl_best_model 17 | from cchess_alphazero.environment.env import CChessEnv 18 | from cchess_alphazero.environment.lookup_tables import ActionLabelsRed, flip_policy, flip_move 19 | from cchess_alphazero.lib.tf_util import set_session_config 20 | 21 | from keras.optimizers import Adam 22 | from keras.callbacks import TensorBoard 23 | import keras.backend as K 24 | 25 | logger = getLogger(__name__) 26 | 27 | def start(config: Config): 28 | set_session_config(per_process_gpu_memory_fraction=1, allow_growth=True, device_list='0,1') 29 | return SupervisedWorker(config).start() 30 | 31 | class SupervisedWorker: 32 | def __init__(self, config:Config): 33 | self.config = config 34 | self.model = None 35 | self.loaded_data = deque(maxlen=self.config.trainer.dataset_size) 36 | self.dataset = deque(), deque(), deque() 37 | self.filenames = [] 38 | self.opt = None 39 | self.buffer = [] 40 | self.gameinfo = None 41 | self.moves = None 42 | self.config.opts.light = True 43 | 44 | def start(self): 45 | self.model = self.load_model() 46 | self.gameinfo = pd.read_csv(self.config.resource.sl_data_gameinfo) 47 | self.moves = pd.read_csv(self.config.resource.sl_data_move) 48 | self.training() 49 | 50 | def training(self): 51 | self.compile_model() 52 | total_steps = self.config.trainer.start_total_steps 53 | logger.info(f"Start training, game count = {len(self.gameinfo)}, step = {self.config.trainer.sl_game_step} games") 54 | 55 | for i in range(0, len(self.gameinfo), self.config.trainer.sl_game_step): 56 | games = self.gameinfo[i:i+self.config.trainer.sl_game_step] 57 | self.fill_queue(games) 58 | if len(self.dataset[0]) > self.config.trainer.batch_size: 59 | steps = self.train_epoch(self.config.trainer.epoch_to_checkpoint) 60 | total_steps += steps 61 | self.save_current_model() 62 | a, b, c = self.dataset 63 | a.clear() 64 | b.clear() 65 | c.clear() 66 | 67 | def train_epoch(self, epochs): 68 | tc = self.config.trainer 69 | state_ary, policy_ary, value_ary = self.collect_all_loaded_data() 70 | tensorboard_cb = TensorBoard(log_dir="./logs/tensorboard_sl/", batch_size=tc.batch_size, histogram_freq=1) 71 | self.model.model.fit(state_ary, [policy_ary, value_ary], 72 | batch_size=tc.batch_size, 73 | epochs=epochs, 74 | shuffle=True, 75 | validation_split=0.02, 76 | callbacks=[tensorboard_cb]) 77 | steps = (state_ary.shape[0] // tc.batch_size) * epochs 78 | return steps 79 | 80 | def compile_model(self): 81 | self.opt = Adam(lr=1e-2) 82 | losses = ['categorical_crossentropy', 'mean_squared_error'] # avoid overfit for supervised 83 | self.model.model.compile(optimizer=self.opt, loss=losses, loss_weights=self.config.trainer.loss_weights) 84 | 85 | def fill_queue(self, games): 86 | _tuple = self.generate_game_data(games) 87 | if _tuple is not None: 88 | for x, y in zip(self.dataset, _tuple): 89 | x.extend(y) 90 | 91 | def collect_all_loaded_data(self): 92 | state_ary, policy_ary, value_ary = self.dataset 93 | 94 | state_ary1 = np.asarray(state_ary, dtype=np.float32) 95 | policy_ary1 = np.asarray(policy_ary, dtype=np.float32) 96 | value_ary1 = np.asarray(value_ary, dtype=np.float32) 97 | return state_ary1, policy_ary1, value_ary1 98 | 99 | def load_model(self): 100 | model = CChessModel(self.config) 101 | if self.config.opts.new or not load_sl_best_model_weight(model): 102 | model.build() 103 | save_as_sl_best_model(model) 104 | return model 105 | 106 | def save_current_model(self): 107 | logger.debug("Save best sl model") 108 | save_as_sl_best_model(self.model) 109 | 110 | def generate_game_data(self, games): 111 | self.buffer = [] 112 | start_time = time() 113 | for idx, game in games.iterrows(): 114 | gid = game['gameID'] 115 | winner = game['winner'] 116 | move = self.moves[self.moves.gameID == gid] 117 | red = move[move.side == 'red'] 118 | black = move[move.side == 'black'] 119 | self.load_game(red, black, winner, idx) 120 | end_time = time() 121 | logger.debug(f"Loading {len(games)} games, time: {end_time - start_time}s") 122 | return self.convert_to_trainging_data() 123 | 124 | def load_game(self, red, black, winner, idx): 125 | env = CChessEnv(self.config).reset() 126 | red_moves = [] 127 | black_moves = [] 128 | turns = 1 129 | black_max_turn = black['turn'].max() 130 | red_max_turn = red['turn'].max() 131 | 132 | while turns < black_max_turn or turns < red_max_turn: 133 | if turns < red_max_turn: 134 | wxf_move = red[red.turn == turns]['move'].item() 135 | action = env.board.parse_WXF_move(wxf_move) 136 | try: 137 | red_moves.append([env.observation, self.build_policy(action, flip=False)]) 138 | except Exception as e: 139 | for i in range(10): 140 | logger.debug(f"{env.board.screen[i]}") 141 | logger.debug(f"{turns} {wxf_move} {action}") 142 | 143 | env.step(action) 144 | if turns < black_max_turn: 145 | wxf_move = black[black.turn == turns]['move'].item() 146 | action = env.board.parse_WXF_move(wxf_move) 147 | try: 148 | black_moves.append([env.observation, self.build_policy(action, flip=True)]) 149 | except Exception as e: 150 | for i in range(10): 151 | logger.debug(f"{env.board.screen[i]}") 152 | logger.debug(f"{turns} {wxf_move} {action}") 153 | 154 | env.step(action) 155 | turns += 1 156 | 157 | if winner == 'red': 158 | red_win = 1 159 | elif winner == 'black': 160 | red_win = -1 161 | else: 162 | red_win = 0 163 | 164 | for move in red_moves: 165 | move += [red_win] 166 | for move in black_moves: 167 | move += [-red_win] 168 | 169 | data = [] 170 | for i in range(len(red_moves)): 171 | data.append(red_moves[i]) 172 | if i < len(black_moves): 173 | data.append(black_moves[i]) 174 | self.buffer += data 175 | 176 | def build_policy(self, action, flip): 177 | labels_n = len(ActionLabelsRed) 178 | move_lookup = {move: i for move, i in zip(ActionLabelsRed, range(labels_n))} 179 | policy = np.zeros(labels_n) 180 | 181 | policy[move_lookup[action]] = 1 182 | 183 | if flip: 184 | policy = flip_policy(policy) 185 | return policy 186 | 187 | def convert_to_trainging_data(self): 188 | data = self.buffer 189 | state_list = [] 190 | policy_list = [] 191 | value_list = [] 192 | env = CChessEnv() 193 | 194 | for state_fen, policy, value in data: 195 | state_planes = env.fen_to_planes(state_fen) 196 | sl_value = value 197 | 198 | state_list.append(state_planes) 199 | policy_list.append(policy) 200 | value_list.append(sl_value) 201 | 202 | return np.asarray(state_list, dtype=np.float32), \ 203 | np.asarray(policy_list, dtype=np.float32), \ 204 | np.asarray(value_list, dtype=np.float32) 205 | 206 | 207 | 208 | -------------------------------------------------------------------------------- /cchess_alphazero/worker/sl_onegreen.py: -------------------------------------------------------------------------------- 1 | import os 2 | import numpy as np 3 | import json 4 | 5 | from collections import deque 6 | from concurrent.futures import ProcessPoolExecutor 7 | from datetime import datetime 8 | from logging import getLogger 9 | from time import sleep 10 | from random import shuffle 11 | from time import time 12 | 13 | import cchess_alphazero.environment.static_env as senv 14 | from cchess_alphazero.agent.model import CChessModel 15 | from cchess_alphazero.config import Config 16 | from cchess_alphazero.lib.data_helper import get_game_data_filenames, read_game_data_from_file 17 | from cchess_alphazero.lib.model_helper import load_sl_best_model_weight, save_as_sl_best_model 18 | from cchess_alphazero.environment.env import CChessEnv 19 | from cchess_alphazero.environment.lookup_tables import ActionLabelsRed, flip_policy, flip_move 20 | from cchess_alphazero.lib.tf_util import set_session_config 21 | from cchess_alphazero.environment.lookup_tables import Winner 22 | 23 | from keras.optimizers import Adam 24 | from keras.callbacks import TensorBoard 25 | import keras.backend as K 26 | 27 | logger = getLogger(__name__) 28 | 29 | def start(config: Config, skip): 30 | set_session_config(per_process_gpu_memory_fraction=1, allow_growth=True, device_list=config.opts.device_list) 31 | return SupervisedWorker(config).start(skip) 32 | 33 | class SupervisedWorker: 34 | def __init__(self, config:Config): 35 | self.config = config 36 | self.model = None 37 | self.loaded_data = deque(maxlen=self.config.trainer.dataset_size) 38 | self.dataset = deque(), deque(), deque() 39 | self.filenames = [] 40 | self.opt = None 41 | self.buffer = [] 42 | self.games = None 43 | 44 | def start(self, skip=0): 45 | self.model = self.load_model() 46 | with open(self.config.resource.sl_onegreen, 'r') as f: 47 | self.games = json.load(f) 48 | self.training(skip) 49 | 50 | def training(self, skip=0): 51 | self.compile_model() 52 | total_steps = self.config.trainer.start_total_steps 53 | logger.info(f"Start training, game count = {len(self.games)}, step = {self.config.trainer.sl_game_step} games, skip = {skip}") 54 | 55 | for i in range(skip, len(self.games), self.config.trainer.sl_game_step): 56 | games = self.games[i:i+self.config.trainer.sl_game_step] 57 | self.fill_queue(games) 58 | if len(self.dataset[0]) > self.config.trainer.batch_size: 59 | steps = self.train_epoch(self.config.trainer.epoch_to_checkpoint) 60 | total_steps += steps 61 | self.save_current_model() 62 | a, b, c = self.dataset 63 | a.clear() 64 | b.clear() 65 | c.clear() 66 | logger.debug(f"total steps = {total_steps}") 67 | 68 | def train_epoch(self, epochs): 69 | tc = self.config.trainer 70 | state_ary, policy_ary, value_ary = self.collect_all_loaded_data() 71 | tensorboard_cb = TensorBoard(log_dir="./logs/tensorboard_sl/", batch_size=tc.batch_size, histogram_freq=1) 72 | self.model.model.fit(state_ary, [policy_ary, value_ary], 73 | batch_size=tc.batch_size, 74 | epochs=epochs, 75 | shuffle=True, 76 | validation_split=0.02, 77 | callbacks=[tensorboard_cb]) 78 | steps = (state_ary.shape[0] // tc.batch_size) * epochs 79 | return steps 80 | 81 | def compile_model(self): 82 | self.opt = Adam(lr=0.003) 83 | losses = ['categorical_crossentropy', 'mean_squared_error'] 84 | self.model.model.compile(optimizer=self.opt, loss=losses, loss_weights=self.config.trainer.loss_weights) 85 | 86 | def fill_queue(self, games): 87 | _tuple = self.generate_game_data(games) 88 | if _tuple is not None: 89 | for x, y in zip(self.dataset, _tuple): 90 | x.extend(y) 91 | 92 | def collect_all_loaded_data(self): 93 | state_ary, policy_ary, value_ary = self.dataset 94 | 95 | state_ary1 = np.asarray(state_ary, dtype=np.float32) 96 | policy_ary1 = np.asarray(policy_ary, dtype=np.float32) 97 | value_ary1 = np.asarray(value_ary, dtype=np.float32) 98 | return state_ary1, policy_ary1, value_ary1 99 | 100 | def load_model(self): 101 | model = CChessModel(self.config) 102 | if self.config.opts.new or not load_sl_best_model_weight(model): 103 | model.build() 104 | save_as_sl_best_model(model) 105 | return model 106 | 107 | def save_current_model(self): 108 | logger.debug("Save best sl model") 109 | save_as_sl_best_model(self.model) 110 | 111 | def generate_game_data(self, games): 112 | self.buffer = [] 113 | start_time = time() 114 | idx = 0 115 | cnt = 0 116 | for game in games: 117 | init = game['init'] 118 | move_list = game['move_list'] 119 | winner = Winner.draw 120 | if game['result'] == '红胜' or '胜' in game['title']: 121 | winner = Winner.red 122 | elif game['result'] == '黑胜' or '负' in game['title']: 123 | winner = Winner.black 124 | else: 125 | winner = Winner.draw 126 | v = self.load_game(init, move_list, winner, idx, game['title'], game['url']) 127 | if v == 1 or v == -1: 128 | cnt += 1 129 | idx += 1 130 | end_time = time() 131 | logger.debug(f"Loading {len(games)} games, time: {end_time - start_time}s, end games = {cnt}") 132 | return self.convert_to_trainging_data() 133 | 134 | def load_game(self, init, move_list, winner, idx, title, url): 135 | turns = 0 136 | env = CChessEnv(self.config).reset(init) 137 | red_moves = [] 138 | black_moves = [] 139 | moves = [move_list[i:i+4] for i in range(len(move_list)) if i % 4 == 0] 140 | 141 | for move in moves: 142 | action = senv.parse_onegreen_move(move) 143 | try: 144 | if turns % 2 == 0: 145 | red_moves.append([env.observation, self.build_policy(action, flip=False)]) 146 | else: 147 | black_moves.append([env.observation, self.build_policy(action, flip=True)]) 148 | env.step(action) 149 | except: 150 | logger.error(f"Invalid Action: idx = {idx}, action = {action}, turns = {turns}, moves = {moves}, " 151 | f"winner = {winner}, init = {init}, title: {title}, url: {url}") 152 | return 153 | turns += 1 154 | 155 | if winner == Winner.red: 156 | red_win = 1 157 | elif winner == Winner.black: 158 | red_win = -1 159 | else: 160 | red_win = senv.evaluate(env.get_state()) 161 | if not env.red_to_move: 162 | red_win = -red_win 163 | 164 | for move in red_moves: 165 | move += [red_win] 166 | for move in black_moves: 167 | move += [-red_win] 168 | 169 | data = [] 170 | for i in range(len(red_moves)): 171 | data.append(red_moves[i]) 172 | if i < len(black_moves): 173 | data.append(black_moves[i]) 174 | self.buffer += data 175 | return red_win 176 | 177 | def build_policy(self, action, flip): 178 | labels_n = len(ActionLabelsRed) 179 | move_lookup = {move: i for move, i in zip(ActionLabelsRed, range(labels_n))} 180 | policy = np.zeros(labels_n) 181 | 182 | policy[move_lookup[action]] = 1 183 | 184 | if flip: 185 | policy = flip_policy(policy) 186 | return policy 187 | 188 | def convert_to_trainging_data(self): 189 | data = self.buffer 190 | state_list = [] 191 | policy_list = [] 192 | value_list = [] 193 | env = CChessEnv() 194 | 195 | for state_fen, policy, value in data: 196 | state_planes = env.fen_to_planes(state_fen) 197 | sl_value = value 198 | 199 | state_list.append(state_planes) 200 | policy_list.append(policy) 201 | value_list.append(sl_value) 202 | 203 | return np.asarray(state_list, dtype=np.float32), \ 204 | np.asarray(policy_list, dtype=np.float32), \ 205 | np.asarray(value_list, dtype=np.float32) 206 | 207 | 208 | 209 | -------------------------------------------------------------------------------- /colaboratory/eval.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | import multiprocessing as mp 4 | 5 | from logging import getLogger 6 | 7 | _PATH_ = os.path.dirname(os.path.dirname(__file__)) 8 | 9 | if _PATH_ not in sys.path: 10 | sys.path.append(_PATH_) 11 | 12 | from cchess_alphazero.lib.logger import setup_logger 13 | from cchess_alphazero.config import Config 14 | import cchess_alphazero.worker.compute_elo as evaluator 15 | 16 | def setup_parameters(config): 17 | num_cores = mp.cpu_count() 18 | max_processes = 2 19 | if len(sys.argv) > 1: 20 | max_processes = int(sys.argv[1]) 21 | if len(sys.argv) > 2: 22 | flag = sys.argv[2] 23 | if flag == 'new': 24 | config.internet.base_url = 'http://temp.52coding.com.cn' 25 | elif flag == 'new2': 26 | config.internet.base_url = 'http://temp3.52coding.com.cn' 27 | config.internet.upload_url = f'{config.internet.base_url}/api/upload_game_file/192x10' 28 | config.internet.upload_eval_url = f'{config.internet.base_url}/api/upload_eval_game_file' 29 | config.internet.get_latest_digest = f'{config.internet.base_url}/api/get_latest_digest/192x10' 30 | config.internet.get_evaluate_model_url = f'{config.internet.base_url}/api/query_for_evaluate' 31 | config.internet.update_elo_url = f'{config.internet.base_url}/api/add_eval_result/' 32 | search_threads = 20 33 | print(f"max_processes = {max_processes}, search_threads = {search_threads}") 34 | config.play.max_processes = max_processes 35 | config.play.search_threads = search_threads 36 | 37 | if __name__ == "__main__": 38 | sys.setrecursionlimit(10000) 39 | config_type = 'distribute' 40 | config = Config(config_type=config_type) 41 | config.opts.device_list = '0' 42 | config.opts.log_move = True 43 | config.resource.create_directories() 44 | setup_logger(config.resource.eval_log_path) 45 | config.eval.update_play_config(config.play) 46 | setup_parameters(config) 47 | # config.internet.download_base_url = 'http://alphazero-1251776088.cossh.myqcloud.com/model/' 48 | evaluator.start(config) 49 | -------------------------------------------------------------------------------- /colaboratory/run.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | import multiprocessing as mp 4 | 5 | from logging import getLogger 6 | 7 | _PATH_ = os.path.dirname(os.path.dirname(__file__)) 8 | 9 | if _PATH_ not in sys.path: 10 | sys.path.append(_PATH_) 11 | 12 | from cchess_alphazero.lib.logger import setup_logger 13 | from cchess_alphazero.config import Config, PlayWithHumanConfig 14 | from cchess_alphazero.worker import self_play 15 | 16 | def setup_parameters(config): 17 | if len(sys.argv) > 1: 18 | config.internet.username = sys.argv[1] 19 | print(f'用户名设置为：{config.internet.username}') 20 | num_cores = mp.cpu_count() 21 | max_processes = 2 22 | if len(sys.argv) > 2: 23 | max_processes = int(sys.argv[2]) 24 | if len(sys.argv) > 3: 25 | config.internet.base_url = sys.argv[3] 26 | config.internet.upload_url = f'{config.internet.base_url}/api/upload_game_file/192x10' 27 | config.internet.upload_eval_url = f'{config.internet.base_url}/api/upload_eval_game_file' 28 | config.internet.get_latest_digest = f'{config.internet.base_url}/api/get_latest_digest/192x10' 29 | # config.internet.get_evaluate_model_url = f'{config.internet.base_url}/api/query_for_evaluate' 30 | # config.internet.update_elo_url = f'{config.internet.base_url}/api/add_eval_result/' 31 | search_threads = 10 32 | print(f"max_processes = {max_processes}, search_threads = {search_threads}") 33 | config.play.max_processes = max_processes 34 | config.play.search_threads = search_threads 35 | 36 | if __name__ == "__main__": 37 | sys.setrecursionlimit(10000) 38 | config_type = 'distribute' 39 | config = Config(config_type=config_type) 40 | config.opts.device_list = '0' 41 | config.resource.create_directories() 42 | setup_logger(config.resource.main_log_path) 43 | config.internet.distributed = True 44 | config.opts.log_move = True 45 | setup_parameters(config) 46 | # config.internet.download_url = 'http://alphazero-1251776088.cossh.myqcloud.com/model/128x7/model_best_weight.h5' 47 | self_play.start(config) 48 | -------------------------------------------------------------------------------- /colaboratory/test.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | import multiprocessing as mp 4 | 5 | from logging import getLogger 6 | 7 | _PATH_ = os.path.dirname(os.path.dirname(__file__)) 8 | 9 | if _PATH_ not in sys.path: 10 | sys.path.append(_PATH_) 11 | 12 | from cchess_alphazero.lib.logger import setup_logger 13 | from cchess_alphazero.config import Config, PlayWithHumanConfig 14 | from cchess_alphazero.worker import self_play 15 | 16 | def setup_parameters(config): 17 | if len(sys.argv) > 1: 18 | config.internet.username = sys.argv[1] 19 | print(f'用户名设置为：{config.internet.username}') 20 | num_cores = mp.cpu_count() 21 | max_processes = 2 22 | if len(sys.argv) > 2: 23 | max_processes = int(sys.argv[2]) 24 | search_threads = 20 25 | print(f"max_processes = {max_processes}, search_threads = {search_threads}") 26 | config.play.max_processes = max_processes 27 | config.play.search_threads = search_threads 28 | 29 | if __name__ == "__main__": 30 | sys.setrecursionlimit(10000) 31 | config_type = 'distribute' 32 | config = Config(config_type=config_type) 33 | config.opts.device_list = '0' 34 | config.resource.create_directories() 35 | setup_logger(config.resource.main_log_path) 36 | config.internet.distributed = False 37 | config.opts.log_move = True 38 | config.opts.has_history = True 39 | setup_parameters(config) 40 | # config.internet.download_url = 'http://alphazero-1251776088.cossh.myqcloud.com/model/128x7/model_best_weight.h5' 41 | self_play.start(config) 42 | -------------------------------------------------------------------------------- /data/model/model_best_weight.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/data/model/model_best_weight.h5 -------------------------------------------------------------------------------- /elo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/elo.png -------------------------------------------------------------------------------- /freeze/evaluate.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | import multiprocessing as mp 4 | 5 | from logging import getLogger 6 | 7 | _PATH_ = os.path.dirname(os.path.dirname(__file__)) 8 | 9 | if _PATH_ not in sys.path: 10 | sys.path.append(_PATH_) 11 | 12 | from cchess_alphazero.lib.logger import setup_logger 13 | from cchess_alphazero.config import Config, PlayWithHumanConfig 14 | import cchess_alphazero.worker.compute_elo_windows as evaluate 15 | 16 | def setup_parameters(config): 17 | gpu = input(f"请输入GPU编号（0代表第一块，1代表第二块，以此类推...）：") 18 | config.opts.device_list = gpu 19 | num_cores = mp.cpu_count() 20 | max_processes = num_cores // 2 if num_cores < 20 else 10 21 | search_threads = 10 22 | max_processes = input(f"请输入运行进程数（推荐{max_processes}）：") 23 | max_processes = int(max_processes) 24 | print(f"max_processes = {max_processes}, search_threads = {search_threads}") 25 | config.play.max_processes = max_processes 26 | config.play.search_threads = search_threads 27 | 28 | 29 | if __name__ == "__main__": 30 | mp.freeze_support() 31 | sys.setrecursionlimit(10000) 32 | config_type = 'distribute' 33 | config = Config(config_type=config_type) 34 | config.resource.create_directories() 35 | setup_logger(config.resource.main_log_path) 36 | config.internet.distributed = True 37 | setup_parameters(config) 38 | evaluate.start(config) 39 | -------------------------------------------------------------------------------- /freeze/play_games.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | import multiprocessing as mp 4 | 5 | from logging import getLogger 6 | 7 | _PATH_ = os.path.dirname(os.path.dirname(__file__)) 8 | 9 | if _PATH_ not in sys.path: 10 | sys.path.append(_PATH_) 11 | 12 | 13 | from cchess_alphazero.lib.logger import setup_logger 14 | from cchess_alphazero.config import Config, PlayWithHumanConfig 15 | from cchess_alphazero.play_games import play 16 | 17 | 18 | def setup_parameters(config): 19 | num_cores = mp.cpu_count() 20 | search_threads = 10 if num_cores < 10 else 20 21 | print(f"search_threads = {search_threads}") 22 | config.play.search_threads = search_threads 23 | 24 | if __name__ == "__main__": 25 | mp.freeze_support() 26 | sys.setrecursionlimit(10000) 27 | config_type = 'distribute' 28 | 29 | config = Config(config_type=config_type) 30 | config.opts.device_list = '0' 31 | config.resource.create_directories() 32 | setup_logger(config.resource.play_log_path) 33 | config.opts.new = False 34 | config.opts.light = False 35 | pwhc = PlayWithHumanConfig() 36 | pwhc.update_play_config(config.play) 37 | config.opts.bg_style = 'WOOD' 38 | setup_parameters(config) 39 | simulation_num = input('请输入AI搜索次数（必须为整数）：') 40 | ai_move_first = input('AI执红？(Y/N)') 41 | ai_move_first = True if ai_move_first == 'Y' or ai_move_first == 'y' else False 42 | config.play.simulation_num_per_move = int(simulation_num) 43 | play.start(config, not ai_move_first) 44 | input('按任意键退出...') 45 | -------------------------------------------------------------------------------- /freeze/play_games.spec: -------------------------------------------------------------------------------- 1 | # -*- mode: python -*- 2 | 3 | block_cipher = None 4 | 5 | a = Analysis(['play_games.py'], 6 | pathex=['C:\\Users\\niuhe\\Desktop\\ChineseChess-AlphaZero-master'], 7 | binaries=[], 8 | datas=[ 9 | ('C:\\Users\\niuhe\\Desktop\\ChineseChess-AlphaZero-master\\cchess_alphazero\\play_games\\images\\WOOD.GIF', 'cchess_alphazero\\play_games\\images'), 10 | ('C:\\Users\\niuhe\\Desktop\\ChineseChess-AlphaZero-master\\cchess_alphazero\\play_games\\PingFang.ttc', 'cchess_alphazero\\play_games'), 11 | ('C:\\Users\\niuhe\\Desktop\\ChineseChess-AlphaZero-master\\cchess_alphazero\\play_games\\images\\WOOD\\*.GIF', 'cchess_alphazero\\play_games\\images\\WOOD') 12 | ], 13 | hiddenimports=[], 14 | hookspath=[], 15 | runtime_hooks=[], 16 | excludes=[], 17 | win_no_prefer_redirects=False, 18 | win_private_assemblies=False, 19 | cipher=block_cipher) 20 | pyz = PYZ(a.pure, a.zipped_data, 21 | cipher=block_cipher) 22 | exe = EXE(pyz, 23 | a.scripts, 24 | exclude_binaries=True, 25 | name='play_games', 26 | debug=False, 27 | strip=False, 28 | upx=True, 29 | console=True , resources=['cchess_alphazero\\\\play_games\\\\images']) 30 | coll = COLLECT(exe, 31 | a.binaries, 32 | a.zipfiles, 33 | a.datas, 34 | strip=False, 35 | upx=True, 36 | name='play_games') 37 | -------------------------------------------------------------------------------- /freeze/run_self_play.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | import multiprocessing as mp 4 | 5 | from logging import getLogger 6 | 7 | _PATH_ = os.path.dirname(os.path.dirname(__file__)) 8 | 9 | if _PATH_ not in sys.path: 10 | sys.path.append(_PATH_) 11 | 12 | from cchess_alphazero.lib.logger import setup_logger 13 | from cchess_alphazero.config import Config, PlayWithHumanConfig 14 | import cchess_alphazero.worker.self_play_windows as self_play 15 | 16 | def setup_parameters(config): 17 | username = input(f"请输入用户名：") 18 | config.internet.username = username 19 | gpu = input(f"请输入GPU编号（0代表第一块，1代表第二块，以此类推...）：") 20 | config.opts.device_list = gpu 21 | num_cores = mp.cpu_count() 22 | max_processes = num_cores // 2 if num_cores < 20 else 10 23 | search_threads = 10 24 | max_processes = input(f"请输入运行进程数（推荐{max_processes}）：") 25 | max_processes = int(max_processes) 26 | print(f"max_processes = {max_processes}, search_threads = {search_threads}") 27 | config.play.max_processes = max_processes 28 | config.play.search_threads = search_threads 29 | 30 | 31 | if __name__ == "__main__": 32 | mp.freeze_support() 33 | sys.setrecursionlimit(10000) 34 | config_type = 'distribute' 35 | config = Config(config_type=config_type) 36 | config.resource.create_directories() 37 | setup_logger(config.resource.main_log_path) 38 | config.internet.distributed = True 39 | setup_parameters(config) 40 | self_play.start(config) 41 | -------------------------------------------------------------------------------- /model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/model.png -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | h5py==2.7.1 2 | tensorflow-gpu==1.3.0 3 | tensorflow-tensorboard==0.1.8 4 | Keras==2.0.8 5 | numpy 6 | pandas 7 | pygame 8 | requests 9 | tqdm -------------------------------------------------------------------------------- /screenshots/board.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NeymarL/ChineseChess-AlphaZero/7f45b0cd7470f7f8e1da7331fc1a31374d6cc10f/screenshots/board.png --------------------------------------------------------------------------------