├── LICENSE ├── README.md ├── getMove.py ├── labels └── labels.txt ├── models └── network.model └── train.py /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2022 Simon Aydin 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # BackPolicy 2 | 3 | Policy network in TensorFlow to classify backgammon moves. The purpose of this network is to help a [reinforcement learning model](https://github.com/SimonG5/BackGS) learn to play backgammon. Using a policy network allows us to greatly reduce the branching factor. 4 | 5 | ## Training 6 | 7 | You can train the model by first extracting the datasets.zip file and then running the following command. 8 | 9 | ```bash 10 | python train.py 11 | ``` 12 | 13 | The network consists of 4 hidden layers with 128 neurons with an output layer consisting of 601 neurons representing each move on a backgammon board. 14 | 15 | ## Usage 16 | 17 | You can retrieve the 10 most interesting moves by running. 18 | 19 | ```bash 20 | python getMove.py 21 | ``` 22 | 23 | You will be asked to input a board which you can do following this syntax. 24 | 25 | ```bash 26 | 2,0,0,0,0,-5,0,-3,0,0,0,5,-5,0,0,0,3,0,5,0,0,0,0,-2,0,0,0,0,6,3,-1 27 | ``` 28 | 29 | The first 24 inputs are single digit values representing each point in this order. Where black checkers are negative values and white are positive. 30 | 31 | drawing 32 | 33 | The next 4 inputs represent the white bar, black bar, white home and black home in that order. 34 | Then comes 2 inputs being each dice. Lastly we have a value being either 1 or -1 where 1 represent white moving and black being black moving. 35 | 36 | The moves are returned in the standard [Backgammon notation](https://thegammonpress.com/backgammon-notation/). 37 | 38 | ```bash 39 | 1: 24/18 : 0.9781245 40 | ------------------------ 41 | 2: 8/2 : 0.008682405 42 | ------------------------ 43 | 3: Cannot/move : 0.007958977 44 | ------------------------ 45 | 4: 24/15 : 0.0024637433 46 | ------------------------ 47 | 5: 6/3 : 0.00073977234 48 | ------------------------ 49 | 6: 8/5 : 0.00060132524 50 | ------------------------ 51 | 7: 24/21 : 0.0004732638 52 | ------------------------ 53 | 8: 8/3 : 0.00044788906 54 | ------------------------ 55 | 9: 13/7 : 0.00035200707 56 | ------------------------ 57 | 10: 13/5 : 8.882125e-05 58 | ``` 59 | 60 | ## Datasets 61 | 62 | The datasets contain over 2 million different board positions with the corresponding correct move. All data points were scraped from games between high-level players. 63 | 64 | ## Result 65 | 66 | The network achieves a top 10 accuracy of 99% whilst having a regular accuracy of 60%. The top 10 accuracy is the more important metric in this scenario as we will branch over at least 10 moves in the reinforcement learning model. 67 | 68 | Feel free to tweak the hyper parameters to improve accuracy. 69 | 70 | ## Dependencies 71 | 72 | Tensorflow > 2.9.1 73 | -------------------------------------------------------------------------------- /getMove.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | import numpy as np 3 | 4 | count = 0 5 | outputMapping = {} 6 | 7 | with open("labels/labels.txt") as file: 8 | for line in file: 9 | outputMapping[count] = line.strip('\n') 10 | count += 1 11 | 12 | model = tf.keras.models.load_model('models/network.model') 13 | 14 | print("Please provide a board to classifice moves for") 15 | boardString = input() 16 | 17 | singluarInputs = boardString.split(",") 18 | input = np.zeros(31) 19 | 20 | for i in range(0, 31): 21 | input[i] = float(singluarInputs[i]) 22 | 23 | input = tf.keras.utils.normalize(input) 24 | predictions = model.predict([input]) 25 | 26 | topTenVals = np.sort(predictions[0])[-10:] 27 | topTen = np.argsort(predictions[0])[-10:] 28 | print("------------------------") 29 | 30 | for i in range(9, -1, -1): 31 | print(str(abs(i-10)) + ": " + 32 | str(outputMapping[topTen[i]]) + " : " + str(topTenVals[i])) 33 | print("------------------------") 34 | -------------------------------------------------------------------------------- /labels/labels.txt: -------------------------------------------------------------------------------- 1 | bar/1 2 | bar/2 3 | bar/3 4 | bar/4 5 | bar/5 6 | bar/6 7 | bar/7 8 | bar/8 9 | bar/9 10 | bar/10 11 | bar/11 12 | bar/12 13 | bar/13 14 | bar/14 15 | bar/15 16 | bar/16 17 | bar/17 18 | bar/18 19 | bar/19 20 | bar/20 21 | bar/21 22 | bar/22 23 | bar/23 24 | bar/24 25 | 1/off 26 | 2/off 27 | 3/off 28 | 4/off 29 | 5/off 30 | 6/off 31 | 7/off 32 | 8/off 33 | 9/off 34 | 10/off 35 | 11/off 36 | 12/off 37 | 13/off 38 | 14/off 39 | 15/off 40 | 16/off 41 | 17/off 42 | 18/off 43 | 19/off 44 | 20/off 45 | 21/off 46 | 22/off 47 | 23/off 48 | 24/off 49 | 1/2 50 | 1/3 51 | 1/4 52 | 1/5 53 | 1/6 54 | 1/7 55 | 1/8 56 | 1/9 57 | 1/10 58 | 1/11 59 | 1/12 60 | 1/13 61 | 1/14 62 | 1/15 63 | 1/16 64 | 1/17 65 | 1/18 66 | 1/19 67 | 1/20 68 | 1/21 69 | 1/22 70 | 1/23 71 | 1/24 72 | 2/1 73 | 2/3 74 | 2/4 75 | 2/5 76 | 2/6 77 | 2/7 78 | 2/8 79 | 2/9 80 | 2/10 81 | 2/11 82 | 2/12 83 | 2/13 84 | 2/14 85 | 2/15 86 | 2/16 87 | 2/17 88 | 2/18 89 | 2/19 90 | 2/20 91 | 2/21 92 | 2/22 93 | 2/23 94 | 2/24 95 | 3/2 96 | 3/1 97 | 3/4 98 | 3/5 99 | 3/6 100 | 3/7 101 | 3/8 102 | 3/9 103 | 3/10 104 | 3/11 105 | 3/12 106 | 3/13 107 | 3/14 108 | 3/15 109 | 3/16 110 | 3/17 111 | 3/18 112 | 3/19 113 | 3/20 114 | 3/21 115 | 3/22 116 | 3/23 117 | 3/24 118 | 4/3 119 | 4/2 120 | 4/1 121 | 4/5 122 | 4/6 123 | 4/7 124 | 4/8 125 | 4/9 126 | 4/10 127 | 4/11 128 | 4/12 129 | 4/13 130 | 4/14 131 | 4/15 132 | 4/16 133 | 4/17 134 | 4/18 135 | 4/19 136 | 4/20 137 | 4/21 138 | 4/22 139 | 4/23 140 | 4/24 141 | 5/4 142 | 5/3 143 | 5/2 144 | 5/1 145 | 5/6 146 | 5/7 147 | 5/8 148 | 5/9 149 | 5/10 150 | 5/11 151 | 5/12 152 | 5/13 153 | 5/14 154 | 5/15 155 | 5/16 156 | 5/17 157 | 5/18 158 | 5/19 159 | 5/20 160 | 5/21 161 | 5/22 162 | 5/23 163 | 5/24 164 | 6/5 165 | 6/4 166 | 6/3 167 | 6/2 168 | 6/1 169 | 6/7 170 | 6/8 171 | 6/9 172 | 6/10 173 | 6/11 174 | 6/12 175 | 6/13 176 | 6/14 177 | 6/15 178 | 6/16 179 | 6/17 180 | 6/18 181 | 6/19 182 | 6/20 183 | 6/21 184 | 6/22 185 | 6/23 186 | 6/24 187 | 7/6 188 | 7/5 189 | 7/4 190 | 7/3 191 | 7/2 192 | 7/1 193 | 7/8 194 | 7/9 195 | 7/10 196 | 7/11 197 | 7/12 198 | 7/13 199 | 7/14 200 | 7/15 201 | 7/16 202 | 7/17 203 | 7/18 204 | 7/19 205 | 7/20 206 | 7/21 207 | 7/22 208 | 7/23 209 | 7/24 210 | 8/7 211 | 8/6 212 | 8/5 213 | 8/4 214 | 8/3 215 | 8/2 216 | 8/1 217 | 8/9 218 | 8/10 219 | 8/11 220 | 8/12 221 | 8/13 222 | 8/14 223 | 8/15 224 | 8/16 225 | 8/17 226 | 8/18 227 | 8/19 228 | 8/20 229 | 8/21 230 | 8/22 231 | 8/23 232 | 8/24 233 | 9/8 234 | 9/7 235 | 9/6 236 | 9/5 237 | 9/4 238 | 9/3 239 | 9/2 240 | 9/1 241 | 9/10 242 | 9/11 243 | 9/12 244 | 9/13 245 | 9/14 246 | 9/15 247 | 9/16 248 | 9/17 249 | 9/18 250 | 9/19 251 | 9/20 252 | 9/21 253 | 9/22 254 | 9/23 255 | 9/24 256 | 10/9 257 | 10/8 258 | 10/7 259 | 10/6 260 | 10/5 261 | 10/4 262 | 10/3 263 | 10/2 264 | 10/1 265 | 10/11 266 | 10/12 267 | 10/13 268 | 10/14 269 | 10/15 270 | 10/16 271 | 10/17 272 | 10/18 273 | 10/19 274 | 10/20 275 | 10/21 276 | 10/22 277 | 10/23 278 | 10/24 279 | 11/10 280 | 11/9 281 | 11/8 282 | 11/7 283 | 11/6 284 | 11/5 285 | 11/4 286 | 11/3 287 | 11/2 288 | 11/1 289 | 11/12 290 | 11/13 291 | 11/14 292 | 11/15 293 | 11/16 294 | 11/17 295 | 11/18 296 | 11/19 297 | 11/20 298 | 11/21 299 | 11/22 300 | 11/23 301 | 11/24 302 | 12/11 303 | 12/10 304 | 12/9 305 | 12/8 306 | 12/7 307 | 12/6 308 | 12/5 309 | 12/4 310 | 12/3 311 | 12/2 312 | 12/1 313 | 12/13 314 | 12/14 315 | 12/15 316 | 12/16 317 | 12/17 318 | 12/18 319 | 12/19 320 | 12/20 321 | 12/21 322 | 12/22 323 | 12/23 324 | 12/24 325 | 13/12 326 | 13/11 327 | 13/10 328 | 13/9 329 | 13/8 330 | 13/7 331 | 13/6 332 | 13/5 333 | 13/4 334 | 13/3 335 | 13/2 336 | 13/1 337 | 13/14 338 | 13/15 339 | 13/16 340 | 13/17 341 | 13/18 342 | 13/19 343 | 13/20 344 | 13/21 345 | 13/22 346 | 13/23 347 | 13/24 348 | 14/13 349 | 14/12 350 | 14/11 351 | 14/10 352 | 14/9 353 | 14/8 354 | 14/7 355 | 14/6 356 | 14/5 357 | 14/4 358 | 14/3 359 | 14/2 360 | 14/1 361 | 14/15 362 | 14/16 363 | 14/17 364 | 14/18 365 | 14/19 366 | 14/20 367 | 14/21 368 | 14/22 369 | 14/23 370 | 14/24 371 | 15/14 372 | 15/13 373 | 15/12 374 | 15/11 375 | 15/10 376 | 15/9 377 | 15/8 378 | 15/7 379 | 15/6 380 | 15/5 381 | 15/4 382 | 15/3 383 | 15/2 384 | 15/1 385 | 15/16 386 | 15/17 387 | 15/18 388 | 15/19 389 | 15/20 390 | 15/21 391 | 15/22 392 | 15/23 393 | 15/24 394 | 16/15 395 | 16/14 396 | 16/13 397 | 16/12 398 | 16/11 399 | 16/10 400 | 16/9 401 | 16/8 402 | 16/7 403 | 16/6 404 | 16/5 405 | 16/4 406 | 16/3 407 | 16/2 408 | 16/1 409 | 16/17 410 | 16/18 411 | 16/19 412 | 16/20 413 | 16/21 414 | 16/22 415 | 16/23 416 | 16/24 417 | 17/16 418 | 17/15 419 | 17/14 420 | 17/13 421 | 17/12 422 | 17/11 423 | 17/10 424 | 17/9 425 | 17/8 426 | 17/7 427 | 17/6 428 | 17/5 429 | 17/4 430 | 17/3 431 | 17/2 432 | 17/1 433 | 17/18 434 | 17/19 435 | 17/20 436 | 17/21 437 | 17/22 438 | 17/23 439 | 17/24 440 | 18/17 441 | 18/16 442 | 18/15 443 | 18/14 444 | 18/13 445 | 18/12 446 | 18/11 447 | 18/10 448 | 18/9 449 | 18/8 450 | 18/7 451 | 18/6 452 | 18/5 453 | 18/4 454 | 18/3 455 | 18/2 456 | 18/1 457 | 18/19 458 | 18/20 459 | 18/21 460 | 18/22 461 | 18/23 462 | 18/24 463 | 19/18 464 | 19/17 465 | 19/16 466 | 19/15 467 | 19/14 468 | 19/13 469 | 19/12 470 | 19/11 471 | 19/10 472 | 19/9 473 | 19/8 474 | 19/7 475 | 19/6 476 | 19/5 477 | 19/4 478 | 19/3 479 | 19/2 480 | 19/1 481 | 19/20 482 | 19/21 483 | 19/22 484 | 19/23 485 | 19/24 486 | 20/19 487 | 20/18 488 | 20/17 489 | 20/16 490 | 20/15 491 | 20/14 492 | 20/13 493 | 20/12 494 | 20/11 495 | 20/10 496 | 20/9 497 | 20/8 498 | 20/7 499 | 20/6 500 | 20/5 501 | 20/4 502 | 20/3 503 | 20/2 504 | 20/1 505 | 20/21 506 | 20/22 507 | 20/23 508 | 20/24 509 | 21/20 510 | 21/19 511 | 21/18 512 | 21/17 513 | 21/16 514 | 21/15 515 | 21/14 516 | 21/13 517 | 21/12 518 | 21/11 519 | 21/10 520 | 21/9 521 | 21/8 522 | 21/7 523 | 21/6 524 | 21/5 525 | 21/4 526 | 21/3 527 | 21/2 528 | 21/1 529 | 21/22 530 | 21/23 531 | 21/24 532 | 22/21 533 | 22/20 534 | 22/19 535 | 22/18 536 | 22/17 537 | 22/16 538 | 22/15 539 | 22/14 540 | 22/13 541 | 22/12 542 | 22/11 543 | 22/10 544 | 22/9 545 | 22/8 546 | 22/7 547 | 22/6 548 | 22/5 549 | 22/4 550 | 22/3 551 | 22/2 552 | 22/1 553 | 22/23 554 | 22/24 555 | 23/22 556 | 23/21 557 | 23/20 558 | 23/19 559 | 23/18 560 | 23/17 561 | 23/16 562 | 23/15 563 | 23/14 564 | 23/13 565 | 23/12 566 | 23/11 567 | 23/10 568 | 23/9 569 | 23/8 570 | 23/7 571 | 23/6 572 | 23/5 573 | 23/4 574 | 23/3 575 | 23/2 576 | 23/1 577 | 23/24 578 | 24/23 579 | 24/22 580 | 24/21 581 | 24/20 582 | 24/19 583 | 24/18 584 | 24/17 585 | 24/16 586 | 24/15 587 | 24/14 588 | 24/13 589 | 24/12 590 | 24/11 591 | 24/10 592 | 24/9 593 | 24/8 594 | 24/7 595 | 24/6 596 | 24/5 597 | 24/4 598 | 24/3 599 | 24/2 600 | 24/1 601 | Cannot/move 602 | -------------------------------------------------------------------------------- /models/network.model: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/monkey531/BackPolicy/2c1fc1124b64b0ef6628ada78f4462d333f01531/models/network.model -------------------------------------------------------------------------------- /train.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | from numpy import genfromtxt 3 | 4 | trainingX = genfromtxt('datasets/trainingX.csv', delimiter=',') 5 | trainingY = genfromtxt('datasets/trainingY.csv') 6 | 7 | trainingX = tf.keras.utils.normalize(trainingX, axis=1) 8 | 9 | testX = genfromtxt('datasets/testX.csv', delimiter=',') 10 | testY = genfromtxt('datasets/testY.csv') 11 | 12 | testX = tf.keras.utils.normalize(testX, axis=1) 13 | 14 | model = tf.keras.models.Sequential() 15 | model.add(tf.keras.layers.Dense(128, input_dim=31, activation=tf.nn.relu)) 16 | model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu)) 17 | model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu)) 18 | model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu)) 19 | model.add(tf.keras.layers.Dense(601, activation=tf.nn.softmax)) 20 | 21 | model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=[ 22 | 'accuracy', tf.keras.metrics.SparseTopKCategoricalAccuracy(k=10)]) 23 | 24 | model.fit(trainingX, trainingY, epochs=10, 25 | batch_size=32, validation_split=0.15) 26 | 27 | model.evaluate(testX, testY) 28 | 29 | model.save('models/network.model') 30 | --------------------------------------------------------------------------------