2 |
3 | 
4 |
5 |
18 |
19 | # Table of contents
20 | - [Progress](#progress)
21 | - [Detection](#detection)
22 | - [Triggerbot](#triggerbot)
23 | - [Setup](#setup)
24 | - [For training](#for-training)
25 | - [For using](#for-using)
26 | - [Technical Details](#technical-details)
27 | - [TODO](#todo)
28 |
29 | # Progress
30 |
31 | ## Detection
32 |
33 | Here is an example video of the current model (as of April 7, 2023)! There are a couple mis-identifications, but overall I think it's really accurate.
34 |
35 |
4 |
5 |
6 | 
7 | 
8 | 
9 |
10 |
11 |
7 |
8 |
9 | 12 | Counter Strike Machine Learning Cheats is a project I created to learn image recognition machine learning. Using a modified CS2 ESP program to dump bounding box data into YOLO format, I am able to efficiently gather training data after just a little bit of cleanup! 13 |
14 |Inspired by Sequoia :)
15 | 16 |Disclaimer: This is created both as practice for myself and as an educational example of practical image-recognition and machine learning applications. It is not intended for use outside of private matches.
17 |
36 |
37 |
38 |
39 | ## Triggerbot
40 |
41 | Two examples of the triggerbot implementation (for whole bodies, not just heads yet). At no point in either of these clips is there a single real mouse press:
42 |
43 |
44 |
45 |
46 |
47 |
48 |
49 |
50 |
51 | # Setup
52 |
53 | Both training and usage require the installation of [Python 3](https://www.python.org/). Trust me, I *also* wish this were not the case. C++ compiled binaries one day I promise.
54 |
55 | This repo should always contain a `best.pt` model, which will always be updated to the best model I have trained so far. You can use this instead of training it yourself.
56 |
57 | ## For training
58 |
59 | 1. (optional) use a `venv`:
60 | ```sh
61 | python -m venv [venv name]
62 | .\[venv name]\Scripts\activate
63 | ```
64 | 2. Install `torch` with CUDA (if your GPU supports it) before installing everything else: https://pytorch.org/get-started/locally/
65 | 3. Install packages:
66 | ```sh
67 | pip install numpy ultralytics opencv-python
68 | ```
69 | 4. Set up folder structure:
70 | ```
71 | CSMLC/
72 | └── training/
73 | ├── all/
74 | │ ├── input/
75 | │ └── labels/
76 | ├── train/
77 | │ ├── images/
78 | │ └── labels/
79 | ├── test/
80 | │ ├── images/
81 | │ └── labels/
82 | └── val/
83 | ├── images/
84 | └── labels/
85 | ```
86 | The `all` folder is not required, you can split your data manually if you'd like, but if you want your folder splitting automated, you can put everything in the `all` folder and run `node split.js` (requires [NodeJS](https://nodejs.org/en), will be making a python version soon probably)
87 |
88 | 5. Train it!
89 | ```sh
90 | python training/train.py
91 | ```
92 |
93 | **Additional notes:**
94 | You can tweak anything training related in `custom.yaml` and `train.py`.
95 |
96 | ## For using
97 |
98 | 1. (optional) use a `venv`:
99 | ```sh
100 | python -m venv [venv name]
101 | .\[venv name]\Scripts\activate
102 | ```
103 | 2. Install `torch` with CUDA (if your GPU supports it) before installing everything else: https://pytorch.org/get-started/locally/
104 | 3. Install packages:
105 | ```sh
106 | pip install numpy ultralytics opencv-python dxcam pywin32
107 | ```
108 | 4. Run it!
109 | ```sh
110 | python training/screen.py
111 | ```
112 | This will run the script that opens a new window, which displays what the computer sees and shows the bounding boxes of where players are (see the [example video](#progress) for how that looks).
113 |
114 | # TODO
115 |
116 | - Trigger bot (train on heads only, perhaps)
117 | - Aim snapping
118 | - Natural-looking aim adjustment too?
119 |
120 | # Technical Details
121 |
122 | This project is built off the backbone of [YOLOv8](https://github.com/ultralytics/ultralytics), a super-fast and surprisingly accurate model for machine learning applications. A lot of the projects you will see that do something similar to this one are using slightly older versions (Sequoia uses v5, for example), so this one is currently the most state-of-the-art!
123 |
124 | To train the custom model, I gather data using a [CS2 ESP cheat](https://github.com/SpikeHD/cs2-data-dumper) that I modified (thank you to the original author of it!) to dump bounding box data directly to YOLO format. I then use my personal favorite labelling program, [OpenLabeling](https://github.com/Cartucho/OpenLabeling), to clean up the data (for example, getting rid of ESP boxes that are showing players through walls). A couple games of Deathmatch was enough to get it to it's current state.
125 |
126 | The YOLOv8 model is awesome. In fact, I'm sure you're curious about the training time on the above [example video](#progress):
127 |
128 | ```
129 | +-------+
130 | | specs |
131 | +-------+
132 | GPU: RTX 3070 8gb
133 | CPU: i7-9700k (not overclocked)
134 | RAM: 32GB
135 |
136 | +------------------+
137 | | training metrics |
138 | +------------------+
139 | Images: ~1.6k (about ~17gb of uncompressed BMPs)
140 | Epochs: 30
141 | Resolution: 928
142 | Batch: 8
143 | Approx. Time Taken to Train: 1h
144 | ```
145 | (I will try to remember to time the next training session properly)
146 |
147 | An hour to train?? Only ~1.6k images?? I don't know what is considered "good" or "bad" in the machine learning world yet, but I'd say that's pretty impressive for the result it gives.
148 |
149 |
--------------------------------------------------------------------------------
/training/__pycache__/aim.cpython-311.pyc:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/SpikeHD/CSMLC/7f8e2d057340cb3e60f55f5e6230fcfd40bcc54a/training/__pycache__/aim.cpython-311.pyc
--------------------------------------------------------------------------------
/training/__pycache__/trigger.cpython-311.pyc:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/SpikeHD/CSMLC/7f8e2d057340cb3e60f55f5e6230fcfd40bcc54a/training/__pycache__/trigger.cpython-311.pyc
--------------------------------------------------------------------------------
/training/aim.py:
--------------------------------------------------------------------------------
1 | import pydirectinput
2 | import numpy as np
3 |
4 | mouse_sens = 2.11
5 | max_sens = 8
6 |
7 | auto_fire_after_snap = False
8 |
9 | def aim_check(box_arr, listener):
10 | # This is similar to trigger bot, but this time we drag the mouse to towards the closest box
11 |
12 | # Get the center of the box
13 | box_center = (box_arr[0] + box_arr[2]) / 2, (box_arr[1] + box_arr[3]) / 2
14 |
15 | # Mouse pos is always the center of the screen
16 | mouse_pos = (1920 / 2, 1080 / 2)
17 |
18 | # Calculate the distance between the mouse and the box
19 | distance = (box_center[0] - mouse_pos[0], box_center[1] - mouse_pos[1])
20 |
21 | # Duration will depend on how far it is
22 | duration = np.linalg.norm(distance) / 1000
23 |
24 | # If too far, don't move the mouse
25 | if abs(distance[0]) > 80 and abs(distance[1]) > 80:
26 | return False
27 |
28 | # Multiply distances because mouse sensitivity
29 | distance = (distance[0] * (max_sens - (mouse_sens + 2)), distance[1] * (max_sens - (mouse_sens + 2)))
30 |
31 | # If the center is close enough, we don't need to move
32 | if abs(distance[0]) < 20 and abs(distance[1]) < 20:
33 | return False
34 |
35 | # Supress mouse events so we don't mess up the aiming
36 | listener.supress = True
37 |
38 | # Move the mouse towards the box
39 | pydirectinput.moveTo(int(distance[0] + mouse_pos[0]), int(distance[1] + mouse_pos[1]), duration=duration, disable_mouse_acceleration=False)
40 |
41 | if auto_fire_after_snap:
42 | pydirectinput.mouseDown(button='left')
43 | pydirectinput.mouseUp(button='left')
44 |
45 | # Unsupress mouse events
46 | listener.supress = False
47 |
48 | return True
--------------------------------------------------------------------------------
/training/best.pt:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/SpikeHD/CSMLC/7f8e2d057340cb3e60f55f5e6230fcfd40bcc54a/training/best.pt
--------------------------------------------------------------------------------
/training/cheat.py:
--------------------------------------------------------------------------------
1 | from ultralytics import YOLO
2 | import time
3 | import cv2
4 | import numpy as np
5 | import dxcam
6 | from ultralytics.yolo.utils.plotting import Annotator
7 | from pynput import mouse
8 |
9 | import trigger
10 | import aim
11 |
12 | # load best.pt
13 | model = YOLO('training/best.pt')
14 | model.to('cuda')
15 |
16 | camera = dxcam.create()
17 | frame = camera.grab() # full screen
18 |
19 | last_tb = time.time()
20 | last_aim = time.time()
21 |
22 | left_clicking = False
23 |
24 | def on_click(x, y, button, pressed):
25 | global left_clicking
26 | if button == mouse.Button.left:
27 | left_clicking = pressed
28 |
29 |
30 | listener = mouse.Listener(on_click=on_click)
31 | listener.start()
32 |
33 | while True:
34 | # read the screen
35 | frame = camera.grab()
36 |
37 | if type(frame) is np.ndarray:
38 | frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
39 |
40 | # predict the screen
41 | results = model.predict(frame, conf=0.25, verbose=False)
42 | closest_box = None
43 |
44 | # draw the bounding boxes
45 | for result in results:
46 | if hasattr(result.boxes, 'xyxy') and len(result.boxes.xyxy) <= 0:
47 | continue
48 |
49 | try:
50 | annotator = Annotator(np.ascontiguousarray(frame))
51 | boxes = result.boxes
52 |
53 | for box in boxes:
54 | b = box.xyxy[0] # get box coordinates in (top, left, bottom, right) format
55 | c = box.cls
56 | annotator.box_label(b, model.names[int(c)])
57 |
58 | if closest_box is None or b[1] < closest_box[1]:
59 | closest_box = b
60 | except:
61 | pass
62 |
63 | # If there is a close box, check to see if we can do stuff with it
64 | if closest_box is not None:
65 | box_arr = np.array(closest_box.cpu(), dtype=np.int32)
66 |
67 | # if time.time() - last_tb > 0.1:
68 | # # do triggerbot from trigger.py
69 | # if trigger.trigger_check(box_arr):
70 | # last_tb = time.time()
71 |
72 | if time.time() - last_aim > 0.2 and left_clicking:
73 | if aim.aim_check(box_arr, listener):
74 | last_aim = time.time()
75 |
76 | # Downscale to 1/4
77 | if type(frame) is np.ndarray:
78 | frame = cv2.resize(frame, (0, 0), fx=0.4, fy=0.4)
79 | # show the screen
80 | cv2.imshow('Screenshot', np.array(frame))
81 |
82 | # wait for a key press
83 | if cv2.waitKey(1) & 0xFF == ord('q'):
84 | break
85 |
--------------------------------------------------------------------------------
/training/custom.yaml:
--------------------------------------------------------------------------------
1 | path: .
2 | train: S:/CSMLC/training/train/images
3 | test: S:/CSMLC/training/test/images
4 | val: S:/CSMLC/training/val/images
5 |
6 | #Classes
7 | nc: 3
8 | names:
9 | 0: t
10 | 1: ct
11 | 2: planted_bomb
12 |
13 | #classes names
14 | #replace all class names list with your classes names
15 | names: ['t', 'ct', 'planted_bomb']
--------------------------------------------------------------------------------
/training/model_archives/best4.7.2023_1.pt:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/SpikeHD/CSMLC/7f8e2d057340cb3e60f55f5e6230fcfd40bcc54a/training/model_archives/best4.7.2023_1.pt
--------------------------------------------------------------------------------
/training/model_archives/best4.9.2023_1.pt:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/SpikeHD/CSMLC/7f8e2d057340cb3e60f55f5e6230fcfd40bcc54a/training/model_archives/best4.9.2023_1.pt
--------------------------------------------------------------------------------
/training/screen.py:
--------------------------------------------------------------------------------
1 | from ultralytics import YOLO
2 | import cv2
3 | import numpy as np
4 | import dxcam
5 | from ultralytics.yolo.utils.plotting import Annotator
6 |
7 | # load best.pt
8 | model = YOLO('./training/best.pt')
9 | model.to('cuda')
10 |
11 | camera = dxcam.create()
12 | frame = camera.grab() # full screen
13 |
14 | while True:
15 | # read the screen
16 | frame = camera.grab()
17 |
18 | if type(frame) is np.ndarray:
19 | frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
20 |
21 | # predict the screen
22 | results = model.predict(frame, conf=0.25)
23 |
24 | # draw the bounding boxes
25 | for result in results:
26 | if hasattr(result.boxes, 'xyxy') and len(result.boxes.xyxy) <= 0:
27 | continue
28 |
29 | annotator = Annotator(frame)
30 | boxes = result.boxes
31 |
32 | for box in boxes:
33 | b = box.xyxy[0] # get box coordinates in (top, left, bottom, right) format
34 | c = box.cls
35 | annotator.box_label(b, model.names[int(c)])
36 |
37 | # Downscale to 1/4
38 | if type(frame) is np.ndarray:
39 | frame = cv2.resize(frame, (0, 0), fx=0.4, fy=0.4)
40 |
41 | # show the screen
42 | cv2.imshow('Screenshot', np.array(frame))
43 |
44 | # wait for a key press
45 | if cv2.waitKey(1) & 0xFF == ord('q'):
46 | break
--------------------------------------------------------------------------------
/training/split.js:
--------------------------------------------------------------------------------
1 | const fs = require('fs')
2 |
3 | /**
4 | * This file will split up each entry in all/labels and all/input into test, train, and val
5 | * The split is 80/10/10
6 | */
7 |
8 | const labels = fs.readdirSync('./all/labels')
9 |
10 | // Label names corrospond to it's matching image name
11 | labels.forEach(label => {
12 | const lPath = `./all/labels/${label}`
13 | const iPath = `./all/input/${label.split('.')[0]}.bmp`
14 |
15 | // Move label to one of the folders
16 | const name = label.split('.')[0]
17 |
18 | const lExt = label.split('.')[1]
19 |
20 | const r = Math.random()
21 |
22 | if (!fs.existsSync(iPath)) return
23 |
24 | if (r < 0.8) {
25 | fs.copyFileSync(lPath, `./train/labels/${name}.${lExt}`)
26 | fs.copyFileSync(iPath, `./train/images/${name}.bmp`)
27 | }
28 |
29 | if (r >= 0.8 && r < 0.9) {
30 | fs.copyFileSync(lPath, `./val/labels/${name}.${lExt}`)
31 | fs.copyFileSync(iPath, `./val/images/${name}.bmp`)
32 | }
33 |
34 | if (r >= 0.9) {
35 | fs.copyFileSync(lPath, `./test/labels/${name}.${lExt}`)
36 | fs.copyFileSync(iPath, `./test/images/${name}.bmp`)
37 | }
38 | })
--------------------------------------------------------------------------------
/training/test.py:
--------------------------------------------------------------------------------
1 | from ultralytics import YOLO
2 |
3 | # Load the model.
4 | model = YOLO('../training/best.pt')
5 | model.to('cuda')
6 |
7 | if __name__ == '__main__':
8 | # Pick a random image from test/images/
9 | results = model.predict('training/test/images/00280.png', save=True)
10 |
--------------------------------------------------------------------------------
/training/train.py:
--------------------------------------------------------------------------------
1 | from ultralytics import YOLO
2 | import time
3 | import os
4 |
5 | # Count the images in test, train, and val folders
6 | test_images = len(os.listdir('S:/CSMLC/training/test/images'))
7 | train_images = len(os.listdir('S:/CSMLC/training/train/images'))
8 | val_images = len(os.listdir('S:/CSMLC/training/val/images'))
9 |
10 | # Load the model.
11 | model = YOLO('yolov8n.pt')
12 | model.to('cuda')
13 |
14 |
15 | if __name__ == '__main__':
16 | start = time.time()
17 |
18 | # Use the model
19 | results = model.train(
20 | data='S:/CSMLC/training/custom.yaml',
21 | imgsz=928,
22 | epochs=30,
23 | batch=8,
24 | name='yolov8n_custom'
25 | )
26 |
27 | end = time.time()
28 |
29 | # Print the number of images in each folder
30 | print(f'{test_images} images in the test folder.')
31 | print(f'{train_images} images in the train folder.')
32 | print(f'{val_images} images in the val folder.')
33 |
34 | print(f'Training took {end - start} seconds (or {round((end - start) / 60, 2)} minutes)')
--------------------------------------------------------------------------------
/training/trigger.py:
--------------------------------------------------------------------------------
1 | import numpy as np
2 | import win32api, win32con
3 |
4 | screen_size = win32api.GetSystemMetrics(0), win32api.GetSystemMetrics(1)
5 |
6 | def trigger_check(box_arr):
7 | print('trigger check')
8 |
9 | # the range the center can be between should increase as the box gets bigger
10 | x_offset = (box_arr[2] - box_arr[0]) / 2
11 | y_offset = (box_arr[3] - box_arr[1]) / 2
12 |
13 | print(x_offset)
14 | print(y_offset)
15 |
16 | # check if box is closes to center of screen
17 | x_range = (screen_size[0] / 2) - x_offset, (screen_size[0] / 2) + x_offset
18 | y_range = (screen_size[1] / 2) - y_offset, (screen_size[1] / 2) + y_offset
19 | box_center = (box_arr[0] + box_arr[2]) / 2, (box_arr[1] + box_arr[3]) / 2
20 |
21 | print(x_range, y_range)
22 | print(box_center)
23 |
24 | if box_center[0] < x_range[1] and box_center[0] > x_range[0] and box_center[1] < y_range[1] and box_center[1] > y_range[0]:
25 | print('bang!')
26 | win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN, 0, 0)
27 | win32api.mouse_event(win32con.MOUSEEVENTF_LEFTUP, 0, 0)
28 | return True
29 |
30 |
31 | def between(x, y, num):
32 | return x <= num <= y
33 |
--------------------------------------------------------------------------------