├── Hardware ├── PartsList.txt ├── README.md ├── TestPiCamera.py ├── TestServos.py ├── TestUSBCamera.sh ├── TestUltraSonic.py ├── UltraSonic.py ├── findDistance.py ├── testWheels.py └── testWheels_Individual_4WD.py ├── LICENSE ├── ObjectDetection ├── ObjectDetector.py ├── README.md ├── SaveJpgAsNumpyArray.py └── SavePiCameraAsNumpyArray.py ├── ObstacleAvoidance ├── ObstacleRL.py ├── ObstacleRLServos.py └── README.md ├── README.md ├── RobotBrain ├── BlackRobot_QLearning.py ├── BlackRobot_SARSA.py ├── BlackRobot_SARSA_Trace.py ├── ClearRobot.py ├── ClearRobot_SARSA_Trace.py ├── Current_ClearRobot.py ├── FindCats.py ├── FindDistance.py ├── MerlinRobot.jpg ├── MoveBlackRobot.py ├── MoveClearRobot.cpython-34.pyc ├── MoveClearRobot.py ├── README.md ├── SarsaPlots.py ├── output_graph.pb └── output_labels.txt ├── Software ├── LoadImagesPiCamera.py ├── README.md └── facedetector.py ├── catID ├── FindCats.py ├── README.md ├── findCats.py ├── label_cats.py ├── output_graph.pb ├── output_labels.txt ├── rename_cat_files.py ├── test1.jpg └── test2.jpg └── robots.jpg /Hardware/PartsList.txt: -------------------------------------------------------------------------------- 1 | 2 | 3 | Robot 1 (clear chassis) 4 | - Smart Video Car Kit from SunFounderDirecthttps://www.amazon.com/gp/product/B014KK89BW/ (I couldn't get this kit to work so I just used it for parts) 5 | - RaspberryPi 3 https://www.amazon.com/Raspberry-Model-1-2GHz-64-bit-quad-core/dp/B01CD5VC92/ 6 | - L298N Dual H Bridge Stepper Motor https://www.amazon.com/gp/product/B00AJGM37I 7 | - PCA9685 16 Channel, 12 Bit PWM Servo Driver https://www.amazon.com/gp/product/B014KTSMLA 8 | - Micro SD card 16 GB or larger 9 | - 370, 440 Ohm resistors, small bread board 10 | - Batteries, battery holder for wheel motor batteries 11 | - Battery Power Pack Expansion Board for Pi https://www.amazon.com/gp/product/B06Y2XBV8Q 12 | - misc wires https://www.amazon.com/gp/product/B01IB7UOFE 13 | - camera mounting bracket (https://www.adafruit.com/product/1434) 14 | - ultrasonic sensor mounting bracket ( https://www.amazon.com/gp/product/B01FDGU0GY/) 15 | 16 | 17 | Robot 2 (black chassis) 18 | - Actobatics Whippersnapper Rover http://www.frys.com/product/8458148?site=sr:SEARCH:MAIN_RSLT_PG 19 | - RaspberryPi 3 https://www.amazon.com/Raspberry-Model-1-2GHz-64-bit-quad-core/dp/B01CD5VC92/ 20 | - SainSmart L298N Dual H Bridge Stepper Motor https://www.amazon.com/gp/product/B00AJGM37I/ 21 | - Micro SD card 16 GB or larger 22 | - 370, 440 Ohm resistors, small bread board 23 | - Battery case and batteries 24 | - Battery Power Pack Expansion Board for Pi https://www.amazon.com/gp/product/B06Y2XBV8Q 25 | - RaspberryPi Camera Module V2 - 8 MP https://www.amazon.com/Raspberry-Pi-Camera-Module-Megapixel/dp/B01ER2SKFS/ 26 | - misc wires https://www.amazon.com/gp/product/B01IB7UOFE 27 | - camera mounting bracket 28 | - ultrasonic sensor mounting bracket 29 | -------------------------------------------------------------------------------- /Hardware/README.md: -------------------------------------------------------------------------------- 1 | 2 | # Hardware parts and test scripts 3 | 4 | Robot 1 (clear chassis) 5 | - Smart Video Car Kit from SunFounderDirecthttps://www.amazon.com/gp/product/B014KK89BW/ (I couldn't get this kit to work so I just used it for parts) 6 | - RaspberryPi 3 https://www.amazon.com/Raspberry-Model-1-2GHz-64-bit-quad-core/dp/B01CD5VC92/ 7 | - L298N Dual H Bridge Stepper Motor https://www.amazon.com/gp/product/B00AJGM37I 8 | - PCA9685 16 Channel, 12 Bit PWM Servo Driver https://www.amazon.com/gp/product/B014KTSMLA 9 | - Micro SD card 16 GB or larger 10 | - 370, 440 Ohm resistors, small bread board 11 | - Batteries, battery holder for wheel motor batteries 12 | - Battery Power Pack Expansion Board for Pi https://www.amazon.com/gp/product/B06Y2XBV8Q 13 | - misc wires https://www.amazon.com/gp/product/B01IB7UOFE 14 | - camera mounting bracket (https://www.adafruit.com/product/1434) 15 | - ultrasonic sensor mounting bracket ( https://www.amazon.com/gp/product/B01FDGU0GY/) 16 | 17 | 18 | Robot 2 (black chassis) 19 | - Actobatics Whippersnapper Rover http://www.frys.com/product/8458148?site=sr:SEARCH:MAIN_RSLT_PG 20 | - RaspberryPi 3 https://www.amazon.com/Raspberry-Model-1-2GHz-64-bit-quad-core/dp/B01CD5VC92/ 21 | - SainSmart L298N Dual H Bridge Stepper Motor https://www.amazon.com/gp/product/B00AJGM37I/ 22 | - Micro SD card 16 GB or larger 23 | - 370, 440 Ohm resistors, small bread board 24 | - Battery case and batteries 25 | - Battery Power Pack Expansion Board for Pi https://www.amazon.com/gp/product/B06Y2XBV8Q 26 | - RaspberryPi Camera Module V2 - 8 MP https://www.amazon.com/Raspberry-Pi-Camera-Module-Megapixel/dp/B01ER2SKFS/ 27 | - misc wires https://www.amazon.com/gp/product/B01IB7UOFE 28 | - camera mounting bracket 29 | - ultrasonic sensor mounting bracket 30 | 31 | 32 | 33 | Python script to test wheels: 34 | 35 | https://github.com/timestocome/RaspberryPi/blob/master/testWheels.py 36 | 37 | 38 | Source: 39 | 40 | http://deepaksinghviblog.blogspot.com/2014/08/raspberrypi-to-run-dc-motor-using-l298n.html 41 | 42 | 43 | 44 | 45 | ------- RaspberryPi camera ------- 46 | 47 | Turn on the camera in Raspberry Pi Configuration ( under preferences ) 48 | 49 | Python script to test camera video and image capture: 50 | 51 | https://github.com/timestocome/RaspberryPi/blob/master/TestPiCamera.py 52 | 53 | 54 | Source: 55 | 56 | https://www.raspberrypi.org/learning/getting-started-with-picamera/requirements/ 57 | 58 | 59 | 60 | ------- Ultra sonic sensor HC204 --------- 61 | 62 | Test script and notes to check SR204 is working 63 | 64 | https://github.com/timestocome/RaspberryPi/blob/master/test%20hardware%20scripts/TestUltraSonic.py 65 | 66 | 67 | ------- Handy things to know for debugging ------- 68 | 69 | Turn on i2c interface so you can connect and run accessories off your RaspberryPi 70 | 71 | https://learn.adafruit.com/adafruits-raspberry-pi-lesson-4-gpio-setup/configuring-i2c 72 | 73 | > sudo raspi-config 74 | -> i2c 75 | -> enable 76 | 77 | 78 | Check i2c connections and locations 79 | > sudo i2cdetect -y 1 80 | 81 | 82 | Check GPIU 83 | > gpio readall 84 | 85 | 86 | 87 | 88 | 89 | -------------------------------------------------------------------------------- /Hardware/TestPiCamera.py: -------------------------------------------------------------------------------- 1 | # test Raspberry Pi camera 2 | 3 | from picamera import PiCamera 4 | from time import sleep 5 | 6 | camera = PiCamera() 7 | 8 | # rotate camera view if needed 9 | camera.rotation = 180 10 | 11 | # live stream 12 | camera.start_preview() 13 | sleep(3) 14 | camera.stop_preview() 15 | 16 | 17 | # grab photo 18 | camera.capture('/home/pi/test.jpg') 19 | 20 | 21 | -------------------------------------------------------------------------------- /Hardware/TestServos.py: -------------------------------------------------------------------------------- 1 | 2 | # http://github.com/timestocome 3 | # Test servos SG90, TowerPro 4 | # Hooked up using PCA9865 5 | 6 | 7 | # java code but nice details and explainations 8 | # http://www.lediouris.net/RaspberryPI/servo/readme.html 9 | 10 | from time import sleep 11 | import Adafruit_PCA9685 12 | 13 | 14 | 15 | # init 16 | pwm = Adafruit_PCA9685.PCA9685() 17 | pwm.set_pwm_freq(60) 18 | 19 | # which servo is being sent the signal 20 | channel = 0 # using first of 16 channels 21 | 22 | 23 | 24 | # The SG90 has a min value of 150 and a max of 600 25 | # 375 is straight, 300-450 is about 90', 45' on each side of the center 26 | 27 | 28 | 29 | servo_min = 300 30 | servo_max = 450 31 | servo_center = (servo_min + servo_max) // 2 32 | 33 | 34 | 35 | position = servo_center 36 | pwm.set_pwm(channel, 0, position) 37 | print("? moved to ", position) 38 | 39 | 40 | 41 | for i in range(servo_min, servo_max, 5): 42 | print(i) 43 | pwm.set_pwm(channel, 0, i) 44 | sleep(0.1) 45 | 46 | 47 | # cleanup 48 | pwm.set_pwm(channel, 0, servo_center) 49 | pwm = None 50 | 51 | -------------------------------------------------------------------------------- /Hardware/TestUSBCamera.sh: -------------------------------------------------------------------------------- 1 | 2 | #!/bin/bash 3 | 4 | # date stamp 5 | timestamp=$(date +%Y_%m_%d_%H_%M) 6 | 7 | # grab image 8 | fswebcam --no-banner /home/pi/webcam/$timestamp.jpg 9 | 10 | -------------------------------------------------------------------------------- /Hardware/TestUltraSonic.py: -------------------------------------------------------------------------------- 1 | # http://github.com/timestocome 2 | 3 | 4 | # hook up and read ultra sonic SR204 sensors to Raspberry Pi 5 | # needs calibration still 6 | # I used 470 Ohm (R2) and 330 Ohm (R1) resisters 7 | # Vout / Vin = R2 / (R1 + R2) 8 | # Vout = 3.3 9 | # Vin = 5 10 | 11 | # photo of test wiring 12 | # https://github.com/timestocome/RaspberryPi/blob/master/photos_of_robots/Test%20SR204%20.jpg 13 | 14 | 15 | # source for starting project 16 | # https://electrosome.com/hc-sr04-ultrasonic-sensor-raspberry-pi/ 3/9 17 | 18 | 19 | 20 | 21 | 22 | import RPi.GPIO as gpio 23 | import time 24 | from time import sleep 25 | 26 | gpio.setmode(gpio.BOARD) 27 | 28 | trigger = 13 29 | echo = 11 30 | 31 | gpio.setup(trigger, gpio.OUT) # trigger 32 | gpio.setup(echo, gpio.IN) # echo 33 | 34 | 35 | pulse_start = 0. 36 | pulse_end = 0. 37 | 38 | # adjust sensor location 39 | distance_sensor_to_car_front = 4 * 2.54 # inches to cm 40 | 41 | 42 | 43 | while True: 44 | 45 | #print('init sensor....') 46 | gpio.output(trigger, False) 47 | time.sleep(0.5) 48 | 49 | #print('trigger...') 50 | gpio.output(trigger, True) 51 | time.sleep(0.00001) 52 | gpio.output(trigger, False) 53 | 54 | while gpio.input(echo) == 0: 55 | pulse_start = time.time() 56 | 57 | while gpio.input(echo) == 1: 58 | pulse_end = time.time() 59 | 60 | pulse_duration = pulse_end - pulse_start 61 | #print('duration... ', pulse_duration) 62 | 63 | 64 | distance = pulse_duration * 34300 / 2. # round trip so cut in half 65 | distance = round(distance, 2) 66 | 67 | if distance < 1000: 68 | print('Distance ', distance - distance_sensor_to_car_front) 69 | #else: 70 | # print('Out of range') 71 | 72 | 73 | 74 | 75 | gpio.cleanup() 76 | 77 | -------------------------------------------------------------------------------- /Hardware/UltraSonic.py: -------------------------------------------------------------------------------- 1 | 2 | 3 | import RPi.GPIO as gpio 4 | import time 5 | 6 | gpio.setmode(gpio.BOARD) 7 | 8 | print('test distance sensor') 9 | 10 | 11 | 12 | # set up and init 13 | trigger = 11 14 | echo = 13 15 | 16 | gpio.setup(trigger, gpio.OUT) 17 | gpio.setup(echo, gpio.IN) 18 | 19 | pulse_start = 0. 20 | pulse_end = 0. 21 | 22 | 23 | speed_of_sound = 343 * 100 24 | print("speed of sound in cm", speed_of_sound) 25 | 26 | car_length = 4 27 | 28 | # run forever 29 | 30 | while True: 31 | 32 | 33 | # clear trigger 34 | gpio.output(trigger, False) 35 | time.sleep(0.5) 36 | 37 | print('send pulse') 38 | # send pulse to trigger 39 | gpio.output(trigger, True) 40 | time.sleep(0.00001) 41 | gpio.output(trigger, False) 42 | 43 | # check echo for return signal 44 | while gpio.input(echo) == 0: 45 | pulse_start = time.time() 46 | 47 | while gpio.input(echo) == 1: 48 | pulse_end = time.time() 49 | 50 | pulse_duration = pulse_end - pulse_start 51 | distance = speed_of_sound / 2. * pulse_duration 52 | distance = round(distance, 2) 53 | distance /= 2.54 # inches 54 | 55 | # filter out things far away 56 | if distance < 300: 57 | print("distance (in) ", distance - car_length) 58 | 59 | 60 | 61 | 62 | gpio.cleanup() 63 | -------------------------------------------------------------------------------- /Hardware/findDistance.py: -------------------------------------------------------------------------------- 1 | # http://github.com/timestocome 2 | 3 | 4 | 5 | import RPi.GPIO as gpio 6 | import time 7 | 8 | 9 | class findDistance(object): 10 | 11 | 12 | def __init__(self): 13 | 14 | gpio.setmode(gpio.BOARD) 15 | 16 | # set up and init 17 | self.trigger = 11 18 | self.echo = 13 19 | 20 | gpio.setup(self.trigger, gpio.OUT) 21 | gpio.setup(self.echo, gpio.IN) 22 | 23 | self.pulse_start = 0. 24 | self.pulse_end = 0. 25 | self.speed_of_sound = 343 * 100 26 | self.car_length = 1 27 | 28 | 29 | 30 | def get_distance(self): 31 | 32 | # clear trigger 33 | gpio.output(self.trigger, False) 34 | time.sleep(0.5) 35 | 36 | # send pulse to trigger 37 | gpio.output(self.trigger, True) 38 | time.sleep(0.00001) 39 | gpio.output(self.trigger, False) 40 | 41 | 42 | # check echo for return signal 43 | while gpio.input(self.echo) == 0: 44 | pulse_start = time.time() 45 | 46 | while gpio.input(self.echo) == 1: 47 | pulse_end = time.time() 48 | 49 | pulse_duration = pulse_end - pulse_start 50 | distance = self.speed_of_sound / 2. * pulse_duration 51 | distance = round(distance, 2) 52 | distance /= 2.54 # inches 53 | 54 | # filter out things far away 55 | if distance > 100: 56 | distance = 100 57 | 58 | # filter out junk 59 | if distance < 1: 60 | disance = 1 61 | 62 | return distance 63 | 64 | 65 | def cleanup(self): 66 | gpio.cleanup() 67 | 68 | 69 | 70 | ################################################### 71 | # test class 72 | ''' 73 | test_distance = findDistance() 74 | distance = test_distance.get_distance() 75 | print(distance) 76 | test_distance.cleanup() 77 | ''' -------------------------------------------------------------------------------- /Hardware/testWheels.py: -------------------------------------------------------------------------------- 1 | 2 | 3 | # http://github.com/timestocome 4 | 5 | 6 | # pins 3, 5, 7, 24, 29, 31 7 | # gpio 0-8 are on by default on boot 8 | 9 | 10 | # more info 11 | # http://www.mcmanis.com/chuck/robotics/tutorial/h-bridge/ 12 | # http://www.bristolwatch.com/L298N/index.htm 13 | 14 | 15 | import RPi.GPIO as gpio 16 | from time import sleep 17 | 18 | gpio.setmode(gpio.BOARD) 19 | 20 | 21 | # 35, 36, 37, 38 22 | 23 | left_forward = 36 24 | right_forward = 35 25 | left_reverse = 38 26 | right_reverse = 37 27 | 28 | 29 | gpio.setup(left_forward, gpio.OUT) 30 | gpio.setup(right_forward, gpio.OUT) 31 | 32 | gpio.setup(left_reverse, gpio.OUT) 33 | gpio.setup(right_reverse, gpio.OUT) 34 | 35 | 36 | 37 | 38 | def go_forward(t): 39 | 40 | print('forward') 41 | 42 | gpio.output(right_forward, gpio.HIGH) 43 | gpio.output(left_forward, gpio.HIGH) 44 | 45 | sleep(t) 46 | gpio.output(right_forward, gpio.LOW) 47 | gpio.output(left_forward, gpio.LOW) 48 | 49 | 50 | def turn_left(t): 51 | 52 | print('left') 53 | gpio.output(right_forward, gpio.HIGH) 54 | 55 | sleep(t) 56 | gpio.output(right_forward, gpio.LOW) 57 | 58 | 59 | 60 | def turn_right(t): 61 | print('right') 62 | gpio.output(left_forward, gpio.HIGH) 63 | 64 | sleep(t) 65 | gpio.output(left_forward, gpio.LOW) 66 | 67 | 68 | 69 | def go_backward(t): 70 | 71 | print('reverse') 72 | 73 | gpio.output(right_reverse, gpio.HIGH) 74 | gpio.output(left_reverse, gpio.HIGH) 75 | 76 | sleep(t) 77 | gpio.output(right_reverse, gpio.LOW) 78 | gpio.output(left_reverse, gpio.LOW) 79 | 80 | 81 | 82 | def reverse_turn_right(t): 83 | print('reverse right') 84 | gpio.output(right_reverse, gpio.HIGH) 85 | 86 | sleep(t) 87 | gpio.output(right_reverse, gpio.LOW) 88 | 89 | 90 | 91 | def reverse_turn_left(t): 92 | print('reverse left') 93 | gpio.output(left_reverse, gpio.HIGH) 94 | 95 | sleep(t) 96 | gpio.output(left_reverse, gpio.LOW) 97 | 98 | 99 | 100 | def hard_right(t=1.): 101 | print('hard right') 102 | gpio.output(left_forward, gpio.HIGH) 103 | gpio.output(right_reverse, gpio.HIGH) 104 | 105 | sleep(t) 106 | gpio.output(left_forward, gpio.LOW) 107 | gpio.output(right_reverse, gpio.LOW) 108 | 109 | 110 | 111 | def hard_left(t=1.): 112 | print('hard left') 113 | gpio.output(right_forward, gpio.HIGH) 114 | gpio.output(left_reverse, gpio.HIGH) 115 | 116 | sleep(t) 117 | gpio.output(right_forward, gpio.LOW) 118 | gpio.output(left_reverse, gpio.LOW) 119 | 120 | 121 | def stop1(t=1): 122 | print('stop1') 123 | gpio.output(left_forward, gpio.HIGH) 124 | gpio.output(left_reverse, gpio.HIGH) 125 | 126 | sleep(t) 127 | gpio.output(left_forward, gpio.LOW) 128 | gpio.output(left_reverse, gpio.LOW) 129 | 130 | 131 | def stop2(t=1): 132 | print('stop2') 133 | gpio.output(right_forward, gpio.HIGH) 134 | gpio.output(right_reverse, gpio.HIGH) 135 | 136 | sleep(t) 137 | gpio.output(right_forward, gpio.LOW) 138 | gpio.output(right_reverse, gpio.LOW) 139 | 140 | 141 | 142 | # test 143 | 144 | ''' 145 | go_forward(2.) 146 | go_backward(2.) 147 | 148 | 149 | stop1(1) 150 | 151 | reverse_turn_right(2.) 152 | reverse_turn_left(2.) 153 | ''' 154 | turn_left(2.) 155 | turn_right(2.) 156 | 157 | stop2(1) 158 | ''' 159 | hard_left(2.) 160 | hard_right(2.) 161 | ''' 162 | 163 | 164 | # clean up 165 | gpio.cleanup() 166 | 167 | -------------------------------------------------------------------------------- /Hardware/testWheels_Individual_4WD.py: -------------------------------------------------------------------------------- 1 | 2 | 3 | import RPi.GPIO as gpio 4 | from time import sleep 5 | 6 | gpio.setmode(gpio.BOARD) 7 | 8 | # rear left 9 | gpio.setup(13, gpio.OUT) # forward 10 | gpio.setup(15, gpio.OUT) # reverse 11 | 12 | # rear right 13 | gpio.setup(31, gpio.OUT) # forward 14 | gpio.setup(29, gpio.OUT) # reverse 15 | 16 | # front right 17 | gpio.setup(40, gpio.OUT) # forward 18 | gpio.setup(7, gpio.OUT) # reverse 19 | 20 | # front left 21 | gpio.setup(38, gpio.OUT) # forward 22 | gpio.setup(36, gpio.OUT) # reverse 23 | 24 | 25 | 26 | 27 | def forward_front(t): 28 | 29 | gpio.output(40, gpio.HIGH) 30 | gpio.output(38, gpio.HIGH) 31 | 32 | sleep(t) 33 | gpio.output(40, gpio.LOW) 34 | gpio.output(38, gpio.LOW) 35 | 36 | 37 | def forward_rear(t): 38 | 39 | gpio.output(13, gpio.HIGH) 40 | gpio.output(31, gpio.HIGH) 41 | 42 | sleep(t) 43 | gpio.output(13, gpio.LOW) 44 | gpio.output(31, gpio.LOW) 45 | 46 | 47 | 48 | def forward_right(t): 49 | gpio.output(40, gpio.HIGH) 50 | gpio.output(31, gpio.HIGH) 51 | 52 | sleep(t) 53 | gpio.output(40, gpio.LOW) 54 | gpio.output(31, gpio.LOW) 55 | 56 | 57 | 58 | def forward_left(t): 59 | gpio.output(13, gpio.HIGH) 60 | gpio.output(38, gpio.HIGH) 61 | 62 | sleep(t) 63 | gpio.output(13, gpio.LOW) 64 | gpio.output(38, gpio.LOW) 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | def reverse_front(t): 74 | 75 | gpio.output(7, gpio.HIGH) 76 | gpio.output(36, gpio.HIGH) 77 | 78 | sleep(t) 79 | gpio.output(7, gpio.LOW) 80 | gpio.output(36, gpio.LOW) 81 | 82 | 83 | def reverse_rear(t): 84 | 85 | gpio.output(15, gpio.HIGH) 86 | gpio.output(29, gpio.HIGH) 87 | 88 | sleep(t) 89 | gpio.output(15, gpio.LOW) 90 | gpio.output(29, gpio.LOW) 91 | 92 | 93 | 94 | def reverse_right(t): 95 | gpio.output(29, gpio.HIGH) 96 | gpio.output(7, gpio.HIGH) 97 | 98 | sleep(t) 99 | gpio.output(29, gpio.LOW) 100 | gpio.output(7, gpio.LOW) 101 | 102 | 103 | 104 | def reverse_left(t): 105 | gpio.output(15, gpio.HIGH) 106 | gpio.output(36, gpio.HIGH) 107 | 108 | sleep(t) 109 | gpio.output(15, gpio.LOW) 110 | gpio.output(36, gpio.LOW) 111 | 112 | 113 | 114 | 115 | 116 | 117 | 118 | 119 | 120 | 121 | 122 | 123 | # test 124 | forward_right(2.0) 125 | forward_left(2.0) 126 | forward_rear(2.0) 127 | forward_front(2.0) 128 | 129 | reverse_right(2.0) 130 | reverse_left(2.0) 131 | reverse_rear(2.0) 132 | reverse_front(2.0) 133 | 134 | 135 | # clean up 136 | gpio.cleanup() 137 | 138 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Linda MacPhee-Cobb 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /ObjectDetection/ObjectDetector.py: -------------------------------------------------------------------------------- 1 | 2 | # http://github.com/timestocome 3 | 4 | # started here: 5 | # http://github.com/tensorflow/models/blob/master/research/object_detection 6 | # https://medium.com/towards-data-science/building-a-real-time-object-recognition-app-with-tensorflow-and-opencv-b7a2b4ebdc32 7 | 8 | 9 | 10 | import numpy as np 11 | import os 12 | import sys 13 | import tensorflow as tf 14 | 15 | from collections import defaultdict 16 | 17 | 18 | from utils import label_map_util 19 | 20 | 21 | # Path to frozen detection graph. This is the actual model that is used for the object detection. 22 | PATH_TO_CKPT = 'ssd_mobilenet_v1_coco_11_06_2017'+ '/frozen_inference_graph.pb' 23 | 24 | # List of the strings that is used to add correct label for each box. 25 | PATH_TO_LABELS = os.path.join('data', 'mscoco_label_map.pbtxt') 26 | 27 | NUM_CLASSES = 90 28 | 29 | 30 | 31 | # load saved model into memory 32 | detection_graph = tf.Graph() 33 | with detection_graph.as_default(): 34 | od_graph_def = tf.GraphDef() 35 | 36 | with tf.gfile.GFile(PATH_TO_CKPT, 'rb') as fid: 37 | serialized_graph = fid.read() 38 | od_graph_def.ParseFromString(serialized_graph) 39 | tf.import_graph_def(od_graph_def, name='') 40 | 41 | 42 | # load label map 43 | label_map = label_map_util.load_labelmap(PATH_TO_LABELS) 44 | categories = label_map_util.convert_label_map_to_categories(label_map, max_num_classes=NUM_CLASSES, use_display_name=True) 45 | category_index = label_map_util.create_category_index(categories) 46 | 47 | 48 | 49 | # helper code 50 | def load_image_into_numpy_array(image): 51 | 52 | (im_width, im_height) = image.size 53 | return np.array(image.getdata()).reshape((im_height, im_width, 3)).astype(np.uint8) 54 | 55 | 56 | 57 | with detection_graph.as_default(): 58 | 59 | sess = tf.Session(graph=detection_graph) 60 | 61 | # Definite input and output Tensors for detection_graph 62 | image_tensor = detection_graph.get_tensor_by_name('image_tensor:0') 63 | 64 | # Each box represents a part of the image where a particular object was detected. 65 | detection_boxes = detection_graph.get_tensor_by_name('detection_boxes:0') 66 | 67 | # Each score represent how level of confidence for each of the objects. 68 | detection_scores = detection_graph.get_tensor_by_name('detection_scores:0') 69 | detection_classes = detection_graph.get_tensor_by_name('detection_classes:0') 70 | num_detections = detection_graph.get_tensor_by_name('num_detections:0') 71 | 72 | 73 | # load an image from camera that has been stored as a numpy array 74 | image_np = np.load('image_cat.npy') 75 | 76 | 77 | # Expand dimensions since the model expects images to have shape: [1, None, None, 3] 78 | image_np_expanded = np.expand_dims(image_np, axis=0) 79 | 80 | # Actual detection. 81 | (boxes, scores, classes, num) = sess.run( 82 | [detection_boxes, detection_scores, detection_classes, num_detections], 83 | feed_dict={image_tensor: image_np_expanded}) 84 | 85 | # Print names and locations of first few items with confidence > 80% 86 | # to do - using this, + ultrasonic sensor locate cat position relative to robot 87 | for s in range(5): 88 | if scores[0, s] > .80: 89 | print(category_index.get(int(classes[0, s])).get('name'), boxes[0,s]) 90 | 91 | 92 | 93 | -------------------------------------------------------------------------------- /ObjectDetection/README.md: -------------------------------------------------------------------------------- 1 | 2 | # Object Detection on RaspberryPi using Google's TensorFlow Object Detection Model 3 | 4 | 5 | You'll need to download TensorFlow Models: 6 | 7 | https://github.com/tensorflow/models 8 | 9 | 10 | Follow the directions here to install, test the model: 11 | 12 | https://github.com/tensorflow/models/tree/master/research/object_detection 13 | 14 | 15 | Once it all works you can move the 'object_detection' directory out to your home directory 16 | - Re-run from one directory up from your object_detection directory 17 | - protoc object_detection/protos/*.proto --python_out=. 18 | 19 | 20 | Change line 22 in utils/label_map_util.py 21 | - from: ---> 22 from object_detection.protos import string_int_label_map_pb2 22 | - to: ---> 22 from protos import string_int_label_map_pb2 23 | 24 | 25 | 26 | 27 | To test your Raspberry Pi Images 28 | - Take an image using 29 | 30 | https://github.com/timestocome/RaspberryPi/blob/master/test%20hardware%20scripts/TestPiCamera.py 31 | 32 | 33 | 34 | Copy the image to object_detection/test_images/image3.jpg 35 | 36 | 37 | 38 | Run the Object Detector script ObjectDetector.py 39 | 40 | * line 73 TEST_IMAGE_PATHS will need to have the range set to (1,4) or as many images as you're testing 41 | -------------------------------------------------------------------------------- /ObjectDetection/SaveJpgAsNumpyArray.py: -------------------------------------------------------------------------------- 1 | 2 | 3 | # http://github.com/timestocome 4 | # 5 | # convert a jpg to a numpy array, save it 6 | # read it back in to be sure it saved correctly 7 | 8 | 9 | import numpy as np 10 | from PIL import Image 11 | import matplotlib.pyplot as plt 12 | 13 | image = Image.open('camera_images/image_cat.jpg') 14 | image_out = np.empty((240, 320, 3), dtype=np.uint8) 15 | np.save('image_cat.npy', np.array(image)) 16 | 17 | load_array = np.load('image_cat.npy') 18 | 19 | 20 | plt.imshow(load_array, interpolation='nearest') 21 | plt.show() 22 | -------------------------------------------------------------------------------- /ObjectDetection/SavePiCameraAsNumpyArray.py: -------------------------------------------------------------------------------- 1 | # http://github.com/timestocome 2 | 3 | 4 | # test Raspberry Pi camera 5 | # save image as numpy array 6 | # reload it to be sure it saved correctly 7 | 8 | 9 | 10 | 11 | from picamera import PiCamera 12 | from time import sleep 13 | import numpy as np 14 | 15 | 16 | 17 | camera = PiCamera() 18 | #camera.rotation = 180 19 | camera.resolution = (320, 240) 20 | 21 | camera.framerate = 10 22 | image_out = np.empty((240, 320, 3), dtype=np.uint8) 23 | camera.capture(image_out, 'rgb') 24 | np.save('image.npy', image_out) 25 | 26 | 27 | # test save 28 | from PIL import Image 29 | import matplotlib.pyplot as plt 30 | 31 | load_array = np.load('image.npy') 32 | plt.imshow(load_array, interpolation='nearest') 33 | plt.show() 34 | 35 | -------------------------------------------------------------------------------- /ObstacleAvoidance/ObstacleRL.py: -------------------------------------------------------------------------------- 1 | # http://github.com/timestocome 2 | 3 | 4 | # train a raspberry pi robot to wander the house while avoiding obstacles 5 | 6 | 7 | import tensorflow as tf 8 | import tensorflow.contrib.slim as slim 9 | import numpy as np 10 | import RPi.GPIO as gpio 11 | import time 12 | from time import sleep 13 | 14 | 15 | 16 | 17 | ############################################################################### 18 | # set up hardware 19 | ############################################################################### 20 | 21 | gpio.setmode(gpio.BOARD) # use pin numbers not gpio numbers 22 | 23 | 24 | # ultrasonic sensor 25 | trigger = 13 26 | echo = 11 27 | 28 | max_distance = 152 # sensor can read 400 cm we only need 5' 29 | 30 | gpio.setup(trigger, gpio.OUT) 31 | gpio.setup(echo, gpio.IN) 32 | 33 | 34 | # wheels ( 4 wheel motors ) 35 | reverse_left = 38 36 | reverse_right = 35 37 | forward_left = 40 38 | forward_right = 37 39 | 40 | gpio.setup(reverse_left, gpio.OUT) 41 | gpio.setup(forward_left, gpio.OUT) 42 | gpio.setup(forward_right, gpio.OUT) 43 | gpio.setup(reverse_right, gpio.OUT) 44 | 45 | wheel_pulse = 0.1 46 | 47 | ############################################################################## 48 | # load data from HC-SR204 UltraSonic distance sensor 49 | ############################################################################## 50 | 51 | # init 52 | pulse_start = 0. 53 | pulse_end = 0. 54 | distance_from_sensor_to_car_front = 2. * 2.54 55 | 56 | 57 | # flush sensor 58 | gpio.output(trigger, False) 59 | time.sleep(0.5) 60 | 61 | 62 | # distance to obstacle in path 63 | def get_state(sleep_time=wheel_pulse): 64 | 65 | # clear trigger sensor 66 | gpio.output(trigger, False) 67 | time.sleep(sleep_time) 68 | 69 | # send trigger pulse 70 | gpio.output(trigger, True) 71 | time.sleep(0.00001) 72 | gpio.output(trigger, False) 73 | 74 | while gpio.input(echo) == 0: 75 | pulse_start = time.time() 76 | 77 | while gpio.input(echo) == 1: 78 | pulse_end = time.time() 79 | 80 | 81 | pulse_duration = pulse_end - pulse_start 82 | distance = pulse_duration * 343 * 100 / 2. # speed of sound m/s * m to cm / round trip 83 | 84 | if distance > 2 and distance < 400: # sensor range 85 | distance = distance + distance_from_sensor_to_car_front 86 | 87 | # don't worry about things further 4' 88 | # this also reduces the size of the state machine 89 | if distance >= max_distance: 90 | distance = max_distance - 1 91 | 92 | return int(distance) 93 | 94 | 95 | 96 | 97 | 98 | ############################################################################## 99 | # perform action 100 | ############################################################################## 101 | actions = ['forward', 'reverse', 'turn_left', 'turn_right', 'hard_left', 'hard_right'] 102 | 103 | 104 | def forward(t=wheel_pulse): 105 | 106 | gpio.output(forward_right, gpio.HIGH) 107 | gpio.output(forward_left, gpio.HIGH) 108 | 109 | sleep(t) 110 | gpio.output(forward_right, gpio.LOW) 111 | gpio.output(forward_left, gpio.LOW) 112 | 113 | 114 | def turn_left(t=wheel_pulse): 115 | gpio.output(forward_right, gpio.HIGH) 116 | 117 | sleep(t) 118 | gpio.output(forward_right, gpio.LOW) 119 | 120 | 121 | 122 | def turn_right(t=wheel_pulse): 123 | gpio.output(forward_left, gpio.HIGH) 124 | 125 | sleep(t) 126 | gpio.output(forward_left, gpio.LOW) 127 | 128 | 129 | 130 | def reverse(t=wheel_pulse): 131 | 132 | gpio.output(reverse_left, gpio.HIGH) 133 | gpio.output(reverse_right, gpio.HIGH) 134 | 135 | sleep(t) 136 | gpio.output(reverse_left, gpio.LOW) 137 | gpio.output(reverse_right, gpio.LOW) 138 | 139 | 140 | 141 | def hard_right(t=wheel_pulse): 142 | gpio.output(forward_left, gpio.HIGH) 143 | gpio.output(reverse_right, gpio.HIGH) 144 | 145 | sleep(t) 146 | gpio.output(forward_left, gpio.LOW) 147 | gpio.output(reverse_right, gpio.LOW) 148 | 149 | 150 | 151 | def hard_left(t=wheel_pulse): 152 | gpio.output(forward_right, gpio.HIGH) 153 | gpio.output(reverse_left, gpio.HIGH) 154 | 155 | sleep(t) 156 | gpio.output(forward_right, gpio.LOW) 157 | gpio.output(reverse_left, gpio.LOW) 158 | 159 | 160 | 161 | ########################################################################## 162 | # cleanup 163 | ########################################################################## 164 | 165 | def cleanup(): 166 | 167 | gpio.cleanup() 168 | 169 | 170 | 171 | 172 | ########################################################################## 173 | # network 174 | ########################################################################## 175 | 176 | class world(): 177 | 178 | def __init__(self): 179 | 180 | self.state = get_state() 181 | self.states = np.zeros(max_distance + 1) 182 | self.num_states = len(self.states) 183 | self.num_actions = len(actions) 184 | 185 | print('state', self.state) 186 | print('num actions', self.num_actions) 187 | 188 | def move(self, action): 189 | 190 | state = get_state() 191 | reward = 0 192 | self.states[state] += 1 193 | 194 | # penatly for being too closes to an obstacle 195 | if state <= 2.54 * 5.: # buffer zone converted to cm 196 | reward = -3.0 197 | 198 | if action == 0: 199 | forward() 200 | reward = 1 201 | elif action == 1: 202 | reverse() 203 | reward = 1 204 | elif action == 2: 205 | turn_left() 206 | reward = 1 207 | elif action == 3: 208 | turn_right() 209 | reward = 1 210 | elif action == 4: 211 | hard_right() 212 | reward = 1 213 | elif action == 5: 214 | hard_left() 215 | reward = 1 216 | 217 | print("state %d, action %d, reward %d" % (state, action, reward)) 218 | 219 | return reward 220 | 221 | 222 | 223 | 224 | 225 | 226 | class agent(): 227 | 228 | def __init__(self, lr, s_size, a_size): 229 | 230 | self.state_in = tf.placeholder(shape=[1], dtype=tf.int32) 231 | state_in_OH = slim.one_hot_encoding(self.state_in, s_size) 232 | 233 | output = slim.fully_connected(state_in_OH, 234 | a_size, 235 | biases_initializer=None, 236 | activation_fn=tf.nn.sigmoid, 237 | weights_initializer=tf.ones_initializer()) 238 | self.output = tf.reshape(output, [-1]) 239 | 240 | self.chosen_action = tf.argmax(self.output, 0) 241 | self.reward_holder = tf.placeholder(shape=[1], dtype=tf.float32) 242 | self.action_holder = tf.placeholder(shape=[1], dtype=tf.int32) 243 | 244 | self.responsible_weight = tf.slice(self.output, self.action_holder, [1]) 245 | 246 | self.loss = -(tf.log(self.responsible_weight) * self.reward_holder) 247 | optimizer = tf.train.GradientDescentOptimizer(learning_rate=lr) 248 | self.update = optimizer.minimize(self.loss) 249 | 250 | 251 | 252 | tf.reset_default_graph() 253 | 254 | world = world() 255 | robot = agent(lr=0.001, s_size = world.num_states, a_size = world.num_actions) 256 | weights = tf.trainable_variables()[0] 257 | e = 0.9 # % time random action chosen, decreases over time 258 | 259 | init = tf.global_variables_initializer() 260 | 261 | total_episodes = 1000 + 1 # up this to loop forever once everything works 262 | total_reward = np.zeros([world.num_states, world.num_actions]) 263 | 264 | # numpy was resetting? idk heavily favoring 0, this is a hack around it 265 | random_actions = np.random.random_integers(0, world.num_actions-1, total_episodes) 266 | 267 | 268 | # used for debugging - remove in final version to speed things up 269 | distance = 0. # estimate how far robot has traveled 270 | choices = np.zeros(len(actions)) 271 | 272 | 273 | with tf.Session() as sess: 274 | sess.run(init) 275 | i = 0 276 | saver = tf.train.Saver() 277 | 278 | 279 | #saver.restore(sess, 'save/model.ckpt') 280 | 281 | 282 | while i < total_episodes: 283 | s = get_state() 284 | e *= 0.95 # reduce random searching over time 285 | e = max(e, .2) # keep epislon over 10% 286 | 287 | if np.random.rand(1) < e: 288 | action = random_actions[i] 289 | print('****** random action', action) 290 | else: 291 | action = sess.run(robot.chosen_action, feed_dict={robot.state_in:[s]}) 292 | 293 | reward = world.move(action) 294 | 295 | feed_dict = { robot.reward_holder: [reward], robot.action_holder: [action], robot.state_in: [s] } 296 | 297 | _, ww = sess.run([robot.update, weights], feed_dict = feed_dict) 298 | total_reward[s, action] += reward 299 | 300 | # used for debugging - comment out in final 301 | distance += reward 302 | choices[action] += 1 303 | 304 | if i % 1 == 0: 305 | print("Random tries: ", e) 306 | print("Distance: ", distance) 307 | print("Choices: ", choices) 308 | 309 | # save weights 310 | if i % 100 == 0: 311 | save_path = saver.save(sess, 'save/model.ckpt') 312 | 313 | i += 1 314 | 315 | 316 | cleanup() 317 | -------------------------------------------------------------------------------- /ObstacleAvoidance/ObstacleRLServos.py: -------------------------------------------------------------------------------- 1 | # http://github.com/timestocome 2 | 3 | 4 | # train a raspberry pi robot to wander the house while avoiding obstacles 5 | # this robot has a servo for steering ~90' range 6 | # rear wheel drive with separate controls for the wheels 7 | 8 | 9 | 10 | 11 | 12 | import tensorflow as tf 13 | import tensorflow.contrib.slim as slim 14 | import numpy as np 15 | 16 | import RPi.GPIO as gpio 17 | import Adafruit_PCA9685 18 | 19 | import time 20 | from time import sleep 21 | 22 | 23 | ############################################################################### 24 | # set up hardware 25 | ############################################################################### 26 | gpio.setmode(gpio.BOARD) # use pin numbers not gpio numbers 27 | 28 | # ultrasonic sensor ------------------------------------------------- 29 | trigger = 11 30 | echo = 13 31 | 32 | max_distance = 100 # sensor can read ~400 cm cap this to reduce state space 33 | 34 | gpio.setup(trigger, gpio.OUT) 35 | gpio.setup(echo, gpio.IN) 36 | 37 | 38 | # wheels ( 2wd, rear wheels) ----------------------------------------- 39 | reverse_left = 38 40 | reverse_right = 37 41 | forward_left = 26 42 | forward_right = 35 43 | 44 | gpio.setup(reverse_left, gpio.OUT) 45 | gpio.setup(forward_left, gpio.OUT) 46 | gpio.setup(forward_right, gpio.OUT) 47 | gpio.setup(reverse_right, gpio.OUT) 48 | 49 | wheel_pulse = 0.5 50 | 51 | 52 | # servo steering ------------------------------------------------- 53 | pwm = Adafruit_PCA9685.PCA9685() 54 | pwm.set_pwm_freq(60) 55 | 56 | # which servo is being sent the signal 57 | channel = 0 # using first of 16 channels 58 | 59 | servo_min = 300 60 | servo_max = 450 61 | servo_center = (servo_min + servo_max) // 2 62 | 63 | 64 | ############################################################################## 65 | # load data from HC-SR204 UltraSonic distance sensor 66 | # this is the input to the network 67 | ############################################################################## 68 | 69 | # init 70 | pulse_start = 0. 71 | pulse_end = 0. 72 | 73 | 74 | # flush sensor 75 | gpio.output(trigger, False) 76 | time.sleep(0.5) 77 | 78 | 79 | # distance to obstacle in path 80 | def get_state(sleep_time=wheel_pulse): 81 | 82 | # clear trigger sensor 83 | gpio.output(trigger, False) 84 | time.sleep(sleep_time) 85 | 86 | # send trigger pulse 87 | gpio.output(trigger, True) 88 | time.sleep(0.00001) 89 | gpio.output(trigger, False) 90 | 91 | while gpio.input(echo) == 0: 92 | pulse_start = time.time() 93 | 94 | while gpio.input(echo) == 1: 95 | pulse_end = time.time() 96 | 97 | 98 | pulse_duration = pulse_end - pulse_start 99 | distance = pulse_duration * 343 * 100 / 2. # speed of sound m/s * m to cm / round trip 100 | 101 | # check if beyond sensor range 400 cm or beyond where we care about 102 | if distance >= max_distance: 103 | distance = max_distance - 1 104 | 105 | # less than 2cm means bad data 106 | if distance <= 2: 107 | distance = 2 108 | 109 | return int(distance) 110 | 111 | 112 | 113 | 114 | 115 | ############################################################################## 116 | # actions robot can take 117 | # to reduce state space servo using discrete positions 118 | # and wheels only set to move forward or reverse 119 | # 120 | # wheels can also be coded to: 121 | # right reverse 122 | # left reverse 123 | # right forward 124 | # left forward 125 | # right forward, left reverse 126 | # right reverse, left forward 127 | # 128 | # servo can be positioned to any int between 300 and 450 129 | ############################################################################## 130 | actions = ['center_forward', 'hard_right_forward', 'right_forward', 'left_forward', 'hard_left_forward', 131 | 'center_reverse', 'hard_right_reverse', 'right_reverse', 'left_reverse', 'hard_left_reverse'] 132 | 133 | def center_forward(): 134 | pwm.set_pwm(0, 0, 375) 135 | forward() 136 | 137 | def hard_right_forward(): 138 | pwm.set_pwm(0, 0, 300) 139 | forward() 140 | 141 | def right_forward(): 142 | pwm.set_pwm(0, 0, 340) 143 | forward() 144 | 145 | def left_forward(): 146 | pwm.set_pwm(0, 0, 410) 147 | forward() 148 | 149 | def hard_left_forward(): 150 | pwm.set_pwm(0, 0, 450) 151 | forward() 152 | 153 | 154 | def center_reverse(): 155 | pwm.set_pwm(0, 0, 375) 156 | reverse() 157 | 158 | def hard_right_reverse(): 159 | pwm.set_pwm(0, 0, 300) 160 | reverse() 161 | 162 | def right_reverse(): 163 | pwm.set_pwm(0, 0, 340) 164 | reverse() 165 | 166 | def left_reverse(): 167 | pwm.set_pwm(0, 0, 410) 168 | reverse() 169 | 170 | def hard_left_reverse(): 171 | pwm.set_pwm(0, 0, 450) 172 | reverse() 173 | 174 | 175 | 176 | def forward(t=wheel_pulse): 177 | 178 | gpio.output(forward_right, gpio.HIGH) 179 | gpio.output(forward_left, gpio.HIGH) 180 | 181 | sleep(t) 182 | gpio.output(forward_right, gpio.LOW) 183 | gpio.output(forward_left, gpio.LOW) 184 | 185 | 186 | 187 | def reverse(t=wheel_pulse/2): 188 | 189 | gpio.output(reverse_left, gpio.HIGH) 190 | gpio.output(reverse_right, gpio.HIGH) 191 | 192 | sleep(t) 193 | gpio.output(reverse_left, gpio.LOW) 194 | gpio.output(reverse_right, gpio.LOW) 195 | 196 | 197 | 198 | 199 | 200 | ########################################################################## 201 | # cleanup 202 | ########################################################################## 203 | 204 | def cleanup(): 205 | 206 | gpio.cleanup() 207 | 208 | pwm.set_pwm(channel, 0, servo_center) 209 | pwm = None 210 | 211 | 212 | 213 | 214 | 215 | ########################################################################## 216 | # network 217 | ########################################################################## 218 | 219 | class world(): 220 | 221 | def __init__(self): 222 | 223 | self.state = get_state() 224 | self.states = np.zeros(max_distance + 1) 225 | self.num_states = len(self.states) 226 | self.num_actions = len(actions) 227 | 228 | print('state', self.state) 229 | print('num actions', self.num_actions) 230 | 231 | 232 | 233 | def move(self, action): 234 | 235 | state = get_state() 236 | reward = 0 237 | self.states[state] += 1 238 | 239 | 240 | 241 | # penalty for being too closes to an obstacle 242 | if state <= 5: # buffer zone in cm 243 | reward = -4.0 244 | 245 | if action == 0: 246 | center_forward() 247 | reward += 3 248 | elif action == 1: 249 | hard_right_forward() 250 | reward += 1 251 | elif action == 2: 252 | right_forward() 253 | reward += 2 254 | elif action == 3: 255 | left_forward() 256 | reward += 2 257 | elif action == 4: 258 | hard_left_forward() 259 | reward += 1 260 | elif action == 5: 261 | center_reverse() 262 | reward += 0 263 | elif action == 6: 264 | hard_right_reverse() 265 | reward += 0 266 | elif action == 7: 267 | right_reverse() 268 | reward += 0 269 | elif action == 8: 270 | left_reverse() 271 | reward += 0 272 | elif action == 9: 273 | hard_left_reverse() 274 | reward += 0 275 | 276 | 277 | 278 | print("state %d, action %d, reward %d" % (state, action, reward)) 279 | 280 | return reward 281 | 282 | 283 | 284 | 285 | 286 | 287 | class agent(): 288 | 289 | def __init__(self, lr, s_size, a_size): 290 | 291 | self.state_in = tf.placeholder(shape=[1], dtype=tf.int32) 292 | state_in_OH = slim.one_hot_encoding(self.state_in, s_size) 293 | 294 | output = slim.fully_connected(state_in_OH, 295 | a_size, 296 | biases_initializer=None, 297 | activation_fn=tf.nn.sigmoid, 298 | weights_initializer=tf.ones_initializer()) 299 | 300 | self.output = tf.reshape(output, [-1]) 301 | 302 | self.chosen_action = tf.argmax(self.output, 0) 303 | self.reward_holder = tf.placeholder(shape=[1], dtype=tf.float32) 304 | self.action_holder = tf.placeholder(shape=[1], dtype=tf.int32) 305 | 306 | self.responsible_weight = tf.slice(self.output, self.action_holder, [1]) 307 | 308 | self.loss = -(tf.log(self.responsible_weight) * self.reward_holder) 309 | optimizer = tf.train.GradientDescentOptimizer(learning_rate=lr) 310 | self.update = optimizer.minimize(self.loss) 311 | 312 | 313 | 314 | tf.reset_default_graph() 315 | 316 | world = world() 317 | robot = agent(lr=0.001, s_size = world.num_states, a_size = world.num_actions) 318 | weights = tf.trainable_variables()[0] 319 | e = 0.9 # % time random action chosen, decreases over time 320 | 321 | init = tf.global_variables_initializer() 322 | 323 | total_episodes = 1000 + 1 # up this to loop forever once everything works 324 | total_reward = np.zeros([world.num_states, world.num_actions]) 325 | 326 | # numpy was resetting? idk heavily favoring 0, this is a hack around it 327 | random_actions = np.random.random_integers(0, world.num_actions-1, total_episodes) 328 | 329 | 330 | # used for debugging - remove in final version to speed things up 331 | distance = 0. # estimate how far robot has traveled 332 | choices = np.zeros(len(actions)) 333 | 334 | 335 | with tf.Session() as sess: 336 | sess.run(init) 337 | i = 0 338 | saver = tf.train.Saver() 339 | 340 | # use to restore previous saved weights instead of random start after 341 | # everything works 342 | #saver.restore(sess, 'save/model.ckpt') 343 | 344 | 345 | while i < total_episodes: 346 | s = get_state() 347 | e *= 0.95 # reduce random searching over time 348 | e = max(e, .2) # keep epislon over 10%, higher for more random behavior 349 | 350 | if np.random.rand(1) < e: 351 | action = random_actions[i] 352 | else: 353 | action = sess.run(robot.chosen_action, feed_dict={robot.state_in:[s]}) 354 | 355 | reward = world.move(action) 356 | 357 | feed_dict = { robot.reward_holder: [reward], robot.action_holder: [action], robot.state_in: [s] } 358 | 359 | _, ww = sess.run([robot.update, weights], feed_dict = feed_dict) 360 | total_reward[s, action] += reward 361 | 362 | # used for debugging - comment out in final 363 | distance += reward 364 | choices[action] += 1 365 | 366 | if i % 1 == 0: 367 | print("Distance: ", distance) # forward distance covered 368 | print("Choices: ", choices) # make sure not locking onto one choice 369 | # end debugging statements 370 | 371 | 372 | 373 | # save weights 374 | if i % 100 == 0: 375 | save_path = saver.save(sess, 'save/model.ckpt') 376 | 377 | i += 1 378 | 379 | 380 | cleanup() 381 | -------------------------------------------------------------------------------- /ObstacleAvoidance/README.md: -------------------------------------------------------------------------------- 1 | 2 | Both robots learn to avoid obstacles using reinforcement learning in less than 1000 time steps 3 | 4 | 5 | Training tricks: 6 | 7 | Choose rewards that favor forward motion, heavily penalize getting close to objects 8 | 9 | Slowly reduce random choices over time 10 | 11 | Choose a very small state space 12 | --- the sensor can read 2cm - 400, reducing the far distance reduces the state machine and speeds up learning 13 | 14 | 15 | 16 | Movies of each robot learning to wander with out hitting things 17 | 18 | https://photos.app.goo.gl/OZ2WZesJWuyISXQD3 19 | 20 | 21 | 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Machine Learning on a Raspberry Pi Robot 2 | 3 | Project goal is to set up an autonomous wheeled robot that can move about the house and track pets 4 | 5 | This is where it all started, though it hasn't been nearly as easy as he makes it sound 6 | ( https://www.oreilly.com/learning/how-to-build-a-robot-that-sees-with-100-and-tensorflow ) 7 | 8 | 9 | 10 | 11 | 12 | Finished Parts: 13 | - robot can ID both the cats, if a cat in photo, from photos it takes 14 | - robot can tell how far it is from obstacles and learns to avoid them using reinforcement learning 15 | - robot uses simple QLearning RL algorithm to avoid obstacles and interact with cats 16 | - cat/no cat id + distance check + update RL values is running at a steady 2 FPS 17 | 18 | 19 | Run robot on boot with out logging on 20 | 21 | Add the following lines to the end of your .bashrc file 22 | 23 | * if you don't cd into the directory and just run python /home/pi/directoryOfCode it won't find your .pb model 24 | 25 | cd /home/pi/directoryOfCode 26 | 27 | python ClearRobot.py 28 | 29 | 30 | WIP: 31 | - put cats to work training robots 32 | - test other RL algorithms 33 | 34 | 35 | 36 | Photos of robots and movies 37 | https://photos.app.goo.gl/OZ2WZesJWuyISXQD3 38 | 39 | 40 | ![robots](https://github.com/timestocome/RaspberryPi-Robot/blob/master/robots.jpg) 41 | 42 | 43 | Robot with clear body has a servo to turn front wheels and rear wheel drive 44 | 45 | Robot with black body has 4 wheel drive and uses wheels to steer 46 | 47 | 48 | 49 | 50 | -------------------------------------------------------------------------------- /RobotBrain/BlackRobot_QLearning.py: -------------------------------------------------------------------------------- 1 | # http://github.com/timestocome 2 | 3 | 4 | # train a raspberry pi robot to wander the house while avoiding obstacles 5 | # and looking for cats 6 | 7 | # this robot uses wheels for steering 8 | # 4 wheel drive with separate controls each side 9 | 10 | 11 | import numpy as np 12 | from pathlib import Path 13 | 14 | 15 | 16 | from FindCats import FindCats 17 | from FindDistance import FindDistance 18 | from MoveBlackRobot import MoveRobot 19 | 20 | #import datetime 21 | 22 | 23 | 24 | 25 | # init 26 | cat_finder = FindCats() 27 | min_cat = cat_finder.min_cat 28 | merlin = cat_finder.merlin 29 | no_cat = cat_finder.no_cat 30 | 31 | distance_finder = FindDistance() 32 | max_distance = distance_finder.max_distance 33 | 34 | 35 | moveRobot = MoveRobot() 36 | actions = moveRobot.actions 37 | 38 | 39 | 40 | 41 | # robot environment 42 | def get_distance(): 43 | # returns distance 1-50 44 | distance = distance_finder.get_distance() 45 | return int(distance) 46 | 47 | 48 | def get_cat(): 49 | # returns 0 for Min, 1 for Merlin, 2 for no cat 50 | cat = cat_finder.is_cat() 51 | return np.argmax([cat[0][1], cat[1][1], cat[2][1]]) 52 | 53 | 54 | 55 | # robot actions 56 | def move(action, distance, cat): 57 | 58 | reward = 0.001 59 | min_distance = 12. 60 | 61 | # penalty for being too closes to an obstacle 62 | if distance <= min_distance: # buffer zone in cm 63 | reward -= min_distance / distance 64 | 65 | # reward for locating cat 66 | if cat == merlin: 67 | reward += 10 68 | if cat == min_cat: 69 | reward += 10 70 | 71 | 72 | # get reward for moving or robot will eventually park itself in middel of the room 73 | if action == 0: 74 | moveRobot.forward() 75 | reward += 3 # reward robot for covering distance 76 | elif action == 1: 77 | moveRobot.reverse() 78 | reward += 0.0 # discourage reverse, no sensors on back of robot 79 | elif action == 2: 80 | moveRobot.turn_left() 81 | reward += 1 82 | elif action == 3: 83 | moveRobot.turn_right() 84 | reward += 1 85 | elif action == 4: 86 | moveRobot.hard_left() 87 | reward += 1 88 | elif action == 5: 89 | moveRobot.hard_right() 90 | reward += 1 91 | 92 | 93 | #print("state %d %d, action %d, reward %d" % (distance, cat, action, reward)) 94 | 95 | return reward 96 | 97 | 98 | 99 | 100 | 101 | 102 | ############################################################################### 103 | # q learning happens here 104 | ############################################################################### 105 | 106 | n_distance_states = max_distance + 1 107 | n_cat_states = 3 108 | n_actions = len(actions) 109 | 110 | # training vars 111 | lr = 0.1 # learning rate 112 | gamma = 0.9 # memory (gamma^n_steps) 113 | n_loops = 500 # training loops to perform 114 | 115 | 116 | 117 | 118 | # new q-table 119 | def init_q_table(n_distance_states, n_cat_states, n_actions): 120 | 121 | table = np.zeros((n_distance_states, n_cat_states, n_actions)) 122 | return table 123 | 124 | 125 | 126 | # load saved q table 127 | def load_q_table(): 128 | 129 | t_1d = np.load('qTable.npy') 130 | table = t_1d.reshape(n_distance_states, n_cat_states, n_actions) 131 | 132 | return table 133 | 134 | 135 | 136 | 137 | def save_q_table(t): 138 | 139 | t_1d = t.reshape(n_distance_states * n_cat_states * n_actions, 1) 140 | np.save('qTable.npy', t_1d) 141 | 142 | 143 | 144 | def choose_action(d, c, q_table, epsilon): 145 | 146 | state_actions = q_table[d][c][:] 147 | 148 | # random move or no data recorded for this state yet 149 | if (np.random.uniform() < epsilon) or (np.sum(state_actions) == 0): 150 | action_chose = np.random.randint(n_actions) 151 | #epsilon *= .99 # lots of random searching when table is zero 152 | if epsilon > 0.1: epsilon *= 0.8 153 | else: 154 | action_chose = state_actions.argmax() 155 | 156 | return action_chose 157 | 158 | 159 | def rl(): 160 | 161 | # init new table if none found 162 | saved_table = Path('qTable.npy') 163 | if saved_table.is_file(): 164 | q_table = load_q_table() 165 | else: 166 | q_table = init_q_table(n_distance_states, n_cat_states, n_actions) 167 | 168 | 169 | epsilon = 1.0 # random choice % decreases over time 170 | 171 | n_steps = 0 172 | d = get_distance() 173 | c = get_cat() 174 | 175 | 176 | while n_steps < n_loops: 177 | #start = datetime.datetime.now() 178 | #print('step %d epsilon %lf' %(n_steps, epsilon)) 179 | 180 | # chose action and move robot 181 | a = choose_action(d, c, q_table, epsilon) 182 | reward = move(a, d, c) 183 | 184 | # update state 185 | d_next = get_distance() 186 | c_next = get_cat() 187 | 188 | # what robot thought would happen next 189 | q_predict = q_table[d][c][a] 190 | 191 | # what actually happened 192 | q_target = reward + gamma * q_table[d][c][a] 193 | 194 | # update q_table 195 | q_table[d][c][a] += lr * (q_target - q_predict) 196 | 197 | # wrap up 198 | d = d_next 199 | c = c_next 200 | 201 | n_steps += 1 202 | 203 | # save data 204 | if n_steps % 100 == 0: 205 | save_q_table(q_table) 206 | 207 | #print(datetime.datetime.now() - start) 208 | 209 | return q_table 210 | 211 | 212 | 213 | 214 | ############################################################################### 215 | # clean shut down of hardware 216 | ############################################################################### 217 | def cleanup(): 218 | 219 | cat_finder.cleanup() 220 | distance_finder.cleanup() 221 | moveRobot.cleanup() 222 | 223 | 224 | 225 | 226 | 227 | 228 | ############################################################################### 229 | # run code 230 | ############################################################################### 231 | 232 | q_table = rl() 233 | 234 | cleanup() 235 | 236 | 237 | ''' 238 | #q_table = load_q_table() 239 | print('--------------------------------') 240 | print('Final Q Table') 241 | 242 | for i in range(n_distance_states): 243 | for j in range(n_cat_states): 244 | print('distance %d, cat %d' %(i, j)) 245 | print('action values', q_table[i, j, :]) 246 | 247 | ''' 248 | 249 | #save_q_table() 250 | #load_q_table() 251 | -------------------------------------------------------------------------------- /RobotBrain/BlackRobot_SARSA.py: -------------------------------------------------------------------------------- 1 | # http://github.com/timestocome 2 | 3 | 4 | # train a raspberry pi robot to wander the house while avoiding obstacles 5 | # and looking for cats 6 | 7 | # this robot uses wheels for steering 8 | # 4 wheel drive with separate controls each side 9 | 10 | 11 | # change from off policy learning in first try 12 | # adapted from https://morvanzhou.github.io/tutorials/ 13 | 14 | 15 | import numpy as np 16 | from pathlib import Path 17 | 18 | 19 | 20 | from FindCats import FindCats 21 | from FindDistance import FindDistance 22 | from MoveBlackRobot import MoveRobot 23 | 24 | import datetime 25 | 26 | 27 | 28 | # init 29 | cat_finder = FindCats() 30 | min_cat = cat_finder.min_cat 31 | merlin = cat_finder.merlin 32 | no_cat = cat_finder.no_cat 33 | 34 | distance_finder = FindDistance() 35 | max_distance = distance_finder.max_distance 36 | 37 | 38 | moveRobot = MoveRobot() 39 | actions = moveRobot.actions 40 | 41 | qTable = 'qTable.npy' 42 | epsilon = 1.0 43 | 44 | 45 | # robot environment 46 | def get_distance(): 47 | # returns distance 1-50 48 | distance = distance_finder.get_distance() 49 | return int(distance) 50 | 51 | 52 | def get_cat(): 53 | # returns 0 for Min, 1 for Merlin, 2 for no cat 54 | cat = cat_finder.is_cat() 55 | return np.argmax([cat[0][1], cat[1][1], cat[2][1]]) 56 | 57 | 58 | 59 | # robot actions 60 | def move(action, distance, cat): 61 | 62 | reward = 0.001 63 | buffer_distance = 12. 64 | 65 | # penalty for being too closes to an obstacle 66 | if distance <=buffer_distance: # buffer zone in cm 67 | reward -= buffer_distance 68 | 69 | 70 | # reward for locating cat 71 | if cat == merlin: 72 | reward += 10 73 | if cat == min_cat: 74 | reward += 10 75 | 76 | 77 | # get reward for moving or robot will eventually park itself in middel of the room 78 | if action == 0: 79 | moveRobot.forward() 80 | reward += 3 # reward robot for covering distance 81 | elif action == 1: 82 | moveRobot.reverse() 83 | reward += 0.0 # discourage reverse, no sensors on back of robot 84 | elif action == 2: 85 | moveRobot.hard_left() 86 | reward += 1 87 | elif action == 3: 88 | moveRobot.hard_right() 89 | reward += 1 90 | ''' 91 | elif action == 4: 92 | moveRobot.turn_left() 93 | reward += 1 94 | elif action == 5: 95 | moveRobot.turn_right() 96 | reward += 1 97 | ''' 98 | 99 | 100 | #print("state %d %d, action %d, reward %d" % (distance, cat, action, reward)) 101 | 102 | return reward 103 | 104 | 105 | 106 | 107 | 108 | 109 | ############################################################################### 110 | # q learning happens here 111 | ############################################################################### 112 | 113 | n_distance_states = max_distance + 1 114 | n_cat_states = 3 115 | n_actions = len(actions) 116 | 117 | 118 | # training vars 119 | lr = 0.01 # learning rate 120 | gamma = 0.9 # memory (gamma^n_steps) 121 | n_loops = 500 # training loops to perform 122 | 123 | 124 | 125 | # init new table 126 | def init_q_table(n_distance_states, n_cat_states, n_actions): 127 | 128 | table = np.zeros((n_distance_states, n_cat_states, n_actions)) 129 | return table 130 | 131 | 132 | 133 | # load saved table from file 134 | def load_q_table(): 135 | 136 | t_1d = np.load(qTable) 137 | table = t_1d.reshape(n_distance_states, n_cat_states, n_actions) 138 | 139 | return table 140 | 141 | 142 | 143 | # write table to disk 144 | def save_q_table(t): 145 | 146 | t_1d = t.reshape(n_distance_states * n_cat_states * n_actions, 1) 147 | np.save(qTable, t_1d) 148 | 149 | 150 | 151 | def choose_action(d, c, q_table): 152 | 153 | global epsilon 154 | state_actions = q_table[d][c][:] 155 | 156 | # random move or no data recorded for this state yet 157 | if (np.random.uniform() < epsilon) or (np.sum(state_actions) == 0): 158 | 159 | action_chose = np.random.randint(n_actions) 160 | 161 | # decrease random moves over time to a minimum of 10% 162 | if epsilon > 0.1: 163 | epsilon *= 0.9 164 | 165 | else: 166 | action_chose = state_actions.argmax() 167 | 168 | return action_chose 169 | 170 | 171 | 172 | 173 | 174 | def rl(): 175 | 176 | # init new table if none found 177 | saved_table = Path(qTable) 178 | if saved_table.is_file(): 179 | q_table = load_q_table() 180 | else: 181 | q_table = init_q_table(n_distance_states, n_cat_states, n_actions) 182 | 183 | 184 | n_steps = 0 185 | 186 | 187 | # prime loop with first action 188 | d = get_distance() 189 | c = get_cat() 190 | a = choose_action(d, c, q_table) 191 | start_time = datetime.datetime.now() 192 | 193 | while n_steps < n_loops: 194 | 195 | # move robot and update state 196 | reward = move(a, d, c) 197 | d_next = get_distance() 198 | c_next = get_cat() 199 | 200 | 201 | # chose action based on next observation 202 | a_ = choose_action(d_next, c_next, q_table) 203 | 204 | # SARSA learning 205 | s_target = reward + gamma + q_table[d_next][c_next][a_] 206 | 207 | 208 | # what robot thought would happen next 209 | s_predict = q_table[d][c][a] 210 | 211 | 212 | # update q_table to reflect new knowledge 213 | q_table[d][c][a] += lr * (s_target - s_predict) 214 | 215 | 216 | # update state for next loop 217 | d = d_next 218 | c = c_next 219 | a = a_ 220 | 221 | n_steps += 1 222 | 223 | # save data every 100 steps incase of failure 224 | if n_steps % 100 == 0: 225 | save_q_table(q_table) 226 | print(datetime.datetime.now() - start_time) 227 | start_time = datetime.datetime.now() 228 | 229 | return q_table 230 | 231 | 232 | 233 | 234 | ############################################################################### 235 | # clean shut down of hardware 236 | ############################################################################### 237 | def cleanup(): 238 | 239 | cat_finder.cleanup() 240 | distance_finder.cleanup() 241 | moveRobot.cleanup() 242 | 243 | 244 | 245 | 246 | 247 | 248 | ############################################################################### 249 | # run code 250 | ############################################################################### 251 | 252 | q_table = rl() 253 | 254 | cleanup() 255 | 256 | 257 | ''' 258 | #q_table = load_q_table() 259 | print('--------------------------------') 260 | print('Final Q Table') 261 | 262 | for i in range(n_distance_states): 263 | for j in range(n_cat_states): 264 | print('distance %d, cat %d' %(i, j)) 265 | print('action values', s_table[i, j, :]) 266 | 267 | ''' 268 | 269 | 270 | # print actions by distance by cat 271 | z = np.zeros(n_distance_states) 272 | for i in range(n_distance_states): 273 | for j in range(n_cat_states): 274 | if j == 2: # no cat 275 | z[i] = np.argmax(q_table[i, j, :]) 276 | 277 | print('distance, action') 278 | for i in range(len(z)): 279 | a = int(z[i]) 280 | print(i, actions[a]) 281 | 282 | 283 | 284 | -------------------------------------------------------------------------------- /RobotBrain/BlackRobot_SARSA_Trace.py: -------------------------------------------------------------------------------- 1 | # http://github.com/timestocome 2 | 3 | 4 | # train a raspberry pi robot to wander the house while avoiding obstacles 5 | # and looking for cats 6 | 7 | # this robot uses wheels for steering 8 | # 4 wheel drive with separate controls each side 9 | 10 | 11 | # change from off policy learning in first try 12 | # adapted from https://morvanzhou.github.io/tutorials/ 13 | 14 | 15 | import numpy as np 16 | from pathlib import Path 17 | 18 | 19 | 20 | from FindCats import FindCats 21 | from FindDistance import FindDistance 22 | from MoveBlackRobot import MoveRobot 23 | 24 | 25 | import datetime 26 | 27 | 28 | # init 29 | cat_finder = FindCats() 30 | min_cat = cat_finder.min_cat 31 | merlin = cat_finder.merlin 32 | no_cat = cat_finder.no_cat 33 | 34 | distance_finder = FindDistance() 35 | max_distance = distance_finder.max_distance 36 | 37 | 38 | moveRobot = MoveRobot() 39 | actions = moveRobot.actions 40 | 41 | qTable = 'qTable2.npy' 42 | epsilon = 1.0 43 | 44 | 45 | # robot environment 46 | def get_distance(): 47 | # returns distance 1-50 48 | distance = distance_finder.get_distance() 49 | 50 | if distance < 1: distance = 1 51 | return int(distance) 52 | 53 | 54 | def get_cat(): 55 | # returns 0 for Min, 1 for Merlin, 2 for no cat 56 | cat = cat_finder.is_cat() 57 | return np.argmax([cat[0][1], cat[1][1], cat[2][1]]) 58 | 59 | 60 | 61 | # robot actions 62 | def move(action, distance, cat): 63 | 64 | reward = 0.001 65 | buffer_distance = 12. 66 | 67 | # penalty for being too closes to an obstacle 68 | if distance <= buffer_distance: # buffer zone in cm 69 | reward -= buffer_distance 70 | 71 | # reward for locating cat 72 | if cat == merlin: 73 | reward += 10 74 | if cat == min_cat: 75 | reward += 10 76 | 77 | # this robot is more cautious, increasing forward reward to offset that 78 | # get reward for moving or robot will eventually park itself in middle of the room 79 | if action == 0: 80 | moveRobot.forward() 81 | reward += 5 # reward robot for covering distance 82 | elif action == 1: 83 | moveRobot.reverse() 84 | reward += 0.0 # discourage reverse, no sensors on back of robot 85 | elif action == 2: 86 | moveRobot.hard_left() 87 | reward += 1 88 | elif action == 3: 89 | moveRobot.hard_right() 90 | reward += 1 91 | ''' 92 | # robot's wheels were slipping too much for these manouvers 93 | elif action == 4: 94 | moveRobot.turn_left() 95 | reward += 1 96 | elif action == 5: 97 | moveRobot.turn_right() 98 | reward += 1 99 | ''' 100 | 101 | #print("state %d %d, action %d, reward %d epsilon %lf" % (distance, cat, action, reward, epsilon)) 102 | 103 | return reward 104 | 105 | 106 | 107 | 108 | 109 | 110 | ############################################################################### 111 | # q learning happens here 112 | ############################################################################### 113 | 114 | n_distance_states = max_distance + 1 115 | n_cat_states = 3 116 | n_actions = len(actions) 117 | 118 | # training vars 119 | lr = 0.01 # learning rate 120 | gamma = 0.9 # memory (gamma^n_steps) 121 | n_loops = 500 # training loops to perform 122 | 123 | 124 | 125 | # init new table 126 | def init_q_table(n_distance_states, n_cat_states, n_actions): 127 | 128 | table = np.zeros((n_distance_states, n_cat_states, n_actions)) 129 | return table 130 | 131 | 132 | 133 | # load saved table from file 134 | def load_q_table(): 135 | 136 | t_1d = np.load(qTable) 137 | table = t_1d.reshape(n_distance_states, n_cat_states, n_actions) 138 | 139 | return table 140 | 141 | 142 | 143 | # write table to disk 144 | def save_q_table(t): 145 | 146 | t_1d = t.reshape(n_distance_states * n_cat_states * n_actions, 1) 147 | np.save(qTable, t_1d) 148 | 149 | 150 | 151 | def choose_action(d, c, q_table): 152 | 153 | global epsilon 154 | state_actions = q_table[d][c][:] 155 | 156 | # random move or no data recorded for this state yet 157 | if (np.random.uniform() < epsilon) or (np.sum(state_actions) == 0): 158 | 159 | action_chose = np.random.randint(n_actions) 160 | 161 | # decrease random moves over time to a minimum of 10% 162 | if epsilon > 0.1: epsilon *= 0.9 163 | 164 | else: 165 | action_chose = state_actions.argmax() 166 | 167 | return action_chose 168 | 169 | 170 | def rl(): 171 | 172 | # init new table if none found 173 | saved_table = Path(qTable) 174 | if saved_table.is_file(): 175 | q_table = load_q_table() 176 | else: 177 | q_table = init_q_table(n_distance_states, n_cat_states, n_actions) 178 | 179 | 180 | n_steps = 0 181 | 182 | 183 | # http://www-anw.cs.umass.edu/~barto/courses/cs687/Chapter%207.pdf pg 14+ 184 | # trace_decay of 0 == temporal difference learning 185 | # trace_decay of 1 == better monte carlo learning 186 | trace_decay = 0.9 # backward looking 187 | eligibility_trace = np.zeros((n_distance_states, n_cat_states, n_actions)) 188 | 189 | # prime loop with first action 190 | d = get_distance() 191 | c = get_cat() 192 | a = choose_action(d, c, q_table) 193 | start_time = datetime.datetime.now() 194 | 195 | 196 | while n_steps < n_loops: 197 | 198 | # move robot and update state 199 | reward = move(a, d, c) 200 | d_next = get_distance() 201 | c_next = get_cat() 202 | 203 | 204 | # chose action based on next observation 205 | a_next = choose_action(d_next, c_next, q_table) 206 | 207 | # SARSA learning 208 | s_target = reward + gamma + q_table[d_next][c_next][a_next] 209 | 210 | 211 | # what robot thought would happen next 212 | s_predict = q_table[d][c][a] 213 | 214 | 215 | # update eligibility trace 216 | eligibility_trace[d, c, a] += 1 217 | 218 | # update q_table to reflect new knowledge 219 | error = s_target - s_predict 220 | q_table[d][c][a] += lr * error * eligibility_trace[d][c][a] 221 | 222 | 223 | # decay eligibility trace 224 | eligibility_trace *= gamma * trace_decay 225 | 226 | # update state for next loop 227 | d = d_next 228 | c = c_next 229 | a = a_next 230 | 231 | n_steps += 1 232 | 233 | # save data every 100 steps incase of failure 234 | if n_steps % 100 == 0: 235 | save_q_table(q_table) 236 | print(datetime.datetime.now() - start_time) 237 | start_time = datetime.datetime.now() 238 | 239 | 240 | return q_table 241 | 242 | 243 | 244 | 245 | ############################################################################### 246 | # clean shut down of hardware 247 | ############################################################################### 248 | def cleanup(): 249 | 250 | cat_finder.cleanup() 251 | distance_finder.cleanup() 252 | moveRobot.cleanup() 253 | 254 | 255 | 256 | 257 | 258 | 259 | ############################################################################### 260 | # run code 261 | ############################################################################### 262 | 263 | q_table = rl() 264 | 265 | cleanup() 266 | 267 | 268 | ''' 269 | #q_table = load_q_table() 270 | print('--------------------------------') 271 | print('Final Q Table') 272 | 273 | for i in range(n_distance_states): 274 | for j in range(n_cat_states): 275 | print('distance %d, cat %d' %(i, j)) 276 | print('action values', s_table[i, j, :]) 277 | 278 | ''' 279 | 280 | 281 | 282 | # print actions by distance by cat 283 | z = np.zeros(n_distance_states) 284 | for i in range(n_distance_states): 285 | for j in range(n_cat_states): 286 | if j == 2: # no cat 287 | z[i] = np.argmax(q_table[i, j, :]) 288 | 289 | print('distance, action') 290 | for i in range(len(z)): 291 | a = int(z[i]) 292 | print(i, actions[a]) 293 | 294 | 295 | 296 | -------------------------------------------------------------------------------- /RobotBrain/ClearRobot.py: -------------------------------------------------------------------------------- 1 | # http://github.com/timestocome 2 | 3 | 4 | # train a raspberry pi robot to wander the house while avoiding obstacles 5 | # this robot has a servo for steering ~90' range 6 | # rear wheel drive with separate controls for the wheels 7 | 8 | 9 | 10 | import numpy as np 11 | import FindCats 12 | import FindDistance 13 | import MoveClearRobot 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | # init 22 | cat_finder = FindCats.FindCats() 23 | min_cat = cat_finder.min_cat 24 | merlin = cat_finder.merlin 25 | no_cat = cat_finder.no_cat 26 | 27 | distance_finder = FindDistance.FindDistance() 28 | max_distance = distance_finder.max_distance 29 | 30 | 31 | moveRobot = MoveClearRobot.MoveRobot() 32 | actions = moveRobot.actions 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | # robot environment 44 | def get_distance(): 45 | # returns distance 1-50 46 | distance = distance_finder.get_distance() 47 | return int(distance) 48 | 49 | 50 | def get_cat(): 51 | # returns 0 for Min, 1 for Merlin, 2 for no cat 52 | cat = cat_finder.is_cat() 53 | return np.argmax([cat[0][1], cat[1][1], cat[2][1]]) 54 | 55 | 56 | 57 | 58 | 59 | def move(action, distance, cat): 60 | 61 | reward = 0 62 | min_distance = 12. 63 | 64 | # penalty for being too closes to an obstacle 65 | if distance <= min_distance: # buffer zone in cm 66 | reward -= min_distance / distance 67 | 68 | 69 | # reward for locating cat 70 | if cat == merlin: 71 | reward += 10 72 | if cat == min_cat: 73 | reward += 10 74 | 75 | 76 | 77 | # otherwise get reward for moving 78 | if action == 0: 79 | moveRobot.center_forward() 80 | reward += 3 81 | elif action == 1: 82 | moveRobot.hard_right_forward() 83 | reward += 1 84 | elif action == 2: 85 | moveRobot.right_forward() 86 | reward += 2 87 | elif action == 3: 88 | moveRobot.left_forward() 89 | reward += 2 90 | elif action == 4: 91 | moveRobot.hard_left_forward() 92 | reward += 1 93 | elif action == 5: 94 | moveRobot.center_reverse() 95 | reward += 0 96 | elif action == 6: 97 | moveRobot.hard_right_reverse() 98 | reward += 0 99 | elif action == 7: 100 | moveRobot.right_reverse() 101 | reward += 0 102 | elif action == 8: 103 | moveRobot.left_reverse() 104 | reward += 0 105 | elif action == 9: 106 | moveRobot.hard_left_reverse() 107 | reward += 0 108 | 109 | 110 | 111 | print("distance %d, cat %d, action %d, reward %d" % (distance, cat, action, reward)) 112 | 113 | return reward 114 | 115 | 116 | 117 | 118 | def cleanup(): 119 | 120 | cat_finder.cleanup() 121 | distance_finder.cleanup() 122 | move_robot.cleanup() 123 | 124 | 125 | 126 | 127 | ############################################################################### 128 | # q learning happens here 129 | ############################################################################### 130 | 131 | n_distance_states = max_distance + 1 132 | n_cat_states = 3 133 | n_actions = len(actions) 134 | 135 | # training vars 136 | lr = 0.1 # learning rate 137 | gamma = 0.9 # memory (gamma^n_steps) 138 | n_loops = 50 # training loops to perform 139 | 140 | 141 | 142 | 143 | # new q-table 144 | def init_q_table(n_distance_states, n_cat_states, n_actions): 145 | 146 | table = np.zeros((n_distance_states, n_cat_states, n_actions)) 147 | return table 148 | 149 | 150 | 151 | # load saved q table 152 | def load_q_table(): 153 | 154 | t_1d = np.load('qTable.npy') 155 | table = t_1d.reshape(n_distance_states, n_cat_states, n_actions) 156 | 157 | return table 158 | 159 | 160 | 161 | 162 | def save_q_table(t): 163 | 164 | t_1d = t.reshape(n_distance_states * n_cat_states * n_actions, 1) 165 | np.save('qTable.npy', t_1d) 166 | 167 | 168 | 169 | def choose_action(d, c, q_table, epsilon): 170 | 171 | state_actions = q_table[d][c][:] 172 | 173 | # random move or no data recorded for this state yet 174 | if (np.random.uniform() < epsilon) or (np.sum(state_actions) == 0): 175 | action_chose = np.random.randint(n_actions) 176 | epsilon *= .99 177 | else: 178 | action_chose = state_actions.argmax() 179 | 180 | return action_chose 181 | 182 | 183 | 184 | def rl(): 185 | 186 | # init new table 187 | q_table = init_q_table(n_distance_states, n_cat_states, n_actions) 188 | 189 | # or continue using previous data 190 | #q_table = load_q_table() 191 | 192 | 193 | epsilon = 1.0 # random choice % decreases over time 194 | 195 | n_steps = 0 196 | d = get_distance() 197 | c = get_cat() 198 | 199 | 200 | while n_steps < n_loops: 201 | 202 | print('step %d epsilon %lf' %(n_steps, epsilon)) 203 | 204 | # chose action and move robot 205 | a = choose_action(d, c, q_table, epsilon) 206 | reward = move(action=a, distance=d, cat=c) 207 | 208 | # update state 209 | d_next = get_distance() 210 | c_next = get_cat() 211 | 212 | # what robot thought would happen next 213 | q_predict = q_table[d][c][a] 214 | 215 | # what actually happened 216 | q_target = reward + gamma * q_table[d][c][a] 217 | 218 | # update q_table 219 | q_table[d][c][a] += lr * (q_target - q_predict) 220 | 221 | # wrap up 222 | d = d_next 223 | c = c_next 224 | n_steps += 1 225 | 226 | # save data 227 | if n_steps % 10 == 0: 228 | save_q_table(q_table) 229 | 230 | return q_table 231 | 232 | 233 | 234 | 235 | ############################################################################### 236 | # clean shut down of hardware 237 | ############################################################################### 238 | def cleanup(): 239 | 240 | cat_finder.cleanup() 241 | distance_finder.cleanup() 242 | moveRobot.cleanup() 243 | 244 | 245 | 246 | 247 | 248 | 249 | ############################################################################### 250 | # run code 251 | ############################################################################### 252 | 253 | q_table = rl() 254 | 255 | cleanup() 256 | 257 | 258 | print('--------------------------------') 259 | print('Final Q Table') 260 | 261 | for i in range(n_distance_states): 262 | for j in range(n_cat_states): 263 | print('distance %d, cat %d' %(i, j)) 264 | print('action values', q_table[i, j, :]) 265 | 266 | 267 | 268 | save_q_table() 269 | load_q_table() 270 | 271 | 272 | -------------------------------------------------------------------------------- /RobotBrain/ClearRobot_SARSA_Trace.py: -------------------------------------------------------------------------------- 1 | # http://github.com/timestocome 2 | 3 | 4 | # train a raspberry pi robot to wander the house while avoiding obstacles 5 | # and looking for cats 6 | 7 | # this robot uses wheels for steering 8 | # 4 wheel drive with separate controls each side 9 | 10 | 11 | # change from off policy learning in first try 12 | # adapted from https://morvanzhou.github.io/tutorials/ 13 | 14 | 15 | import numpy as np 16 | from pathlib import Path 17 | 18 | 19 | 20 | from FindCats import FindCats 21 | from FindDistance import FindDistance 22 | from MoveClearRobot import MoveRobot 23 | 24 | 25 | import datetime 26 | 27 | 28 | # init 29 | cat_finder = FindCats() 30 | min_cat = cat_finder.min_cat 31 | merlin = cat_finder.merlin 32 | no_cat = cat_finder.no_cat 33 | 34 | distance_finder = FindDistance() 35 | max_distance = distance_finder.max_distance 36 | 37 | 38 | moveRobot = MoveRobot() 39 | actions = moveRobot.actions 40 | 41 | qTable = 'qTable2.npy' 42 | epsilon = 1.0 43 | 44 | 45 | # robot environment 46 | def get_distance(): 47 | # returns distance 1-50 48 | distance = distance_finder.get_distance() 49 | 50 | if distance < 1: distance = 1 51 | return int(distance) 52 | 53 | 54 | def get_cat(): 55 | # returns 0 for Min, 1 for Merlin, 2 for no cat 56 | cat = cat_finder.is_cat() 57 | return np.argmax([cat[0][1], cat[1][1], cat[2][1]]) 58 | 59 | 60 | 61 | # robot actions 62 | def move(action, distance, cat): 63 | 64 | reward = 0.001 65 | buffer_distance = 12. 66 | 67 | # penalty for being too closes to an obstacle 68 | if distance <= buffer_distance: # buffer zone in cm 69 | reward -= buffer_distance 70 | 71 | # reward for locating cat 72 | if cat == merlin: 73 | reward += 10 74 | if cat == min_cat: 75 | reward += 10 76 | 77 | 78 | #self.actions = ['center_forward', 'hard_right_forward', 'right_forward', 'left_forward', 'hard_left_forward', 79 | # 'center_reverse', 'hard_right_reverse', 'right_reverse', 'left_reverse', 'hard_left_reverse'] 80 | 81 | 82 | 83 | # this robot is more cautious, increasing forward reward to offset that 84 | # get reward for moving or robot will eventually park itself in middle of the room 85 | if action == 0: 86 | moveRobot.center_forward() 87 | reward += 5 # reward robot for covering distance 88 | elif action == 1: 89 | moveRobot.hard_right_forward() 90 | reward += 1 91 | elif action == 2: 92 | moveRobot.right_forward() 93 | reward += 2 94 | elif action == 3: 95 | moveRobot.left_forward() 96 | reward += 2 97 | elif action == 4: 98 | moveRobot.hard_left_forward() 99 | reward += 1 100 | elif action == 5: 101 | moveRobot.center_reverse() 102 | reward += 1 103 | elif action == 6: 104 | moveRobot.hard_right_reverse() 105 | reward += 1 106 | elif action == 7: 107 | moveRobot.right_reverse() 108 | reward += 1 109 | elif action == 8: 110 | moveRobot.left_reverse() 111 | reward == 1 112 | elif action == 9: 113 | moveRobot.hard_left_reverse() 114 | reward += 1 115 | 116 | 117 | #print("state %d %d, action %d, reward %d epsilon %lf" % (distance, cat, action, reward, epsilon)) 118 | 119 | return reward 120 | 121 | 122 | 123 | 124 | 125 | 126 | ############################################################################### 127 | # q learning happens here 128 | ############################################################################### 129 | 130 | n_distance_states = max_distance + 1 131 | n_cat_states = 3 132 | n_actions = len(actions) 133 | 134 | # training vars 135 | lr = 0.01 # learning rate 136 | gamma = 0.9 # memory (gamma^n_steps) 137 | n_loops = 500 # training loops to perform 138 | 139 | 140 | 141 | # init new table 142 | def init_q_table(n_distance_states, n_cat_states, n_actions): 143 | 144 | table = np.zeros((n_distance_states, n_cat_states, n_actions)) 145 | return table 146 | 147 | 148 | 149 | # load saved table from file 150 | def load_q_table(): 151 | 152 | t_1d = np.load(qTable) 153 | table = t_1d.reshape(n_distance_states, n_cat_states, n_actions) 154 | 155 | return table 156 | 157 | 158 | 159 | # write table to disk 160 | def save_q_table(t): 161 | 162 | t_1d = t.reshape(n_distance_states * n_cat_states * n_actions, 1) 163 | np.save(qTable, t_1d) 164 | 165 | 166 | 167 | def choose_action(d, c, q_table): 168 | 169 | global epsilon 170 | state_actions = q_table[d][c][:] 171 | 172 | # random move or no data recorded for this state yet 173 | if (np.random.uniform() < epsilon) or (np.sum(state_actions) == 0): 174 | 175 | action_chose = np.random.randint(n_actions) 176 | 177 | # decrease random moves over time to a minimum of 10% 178 | if epsilon > 0.1: epsilon *= 0.9 179 | 180 | else: 181 | action_chose = state_actions.argmax() 182 | 183 | return action_chose 184 | 185 | 186 | def rl(): 187 | 188 | # init new table if none found 189 | saved_table = Path(qTable) 190 | if saved_table.is_file(): 191 | q_table = load_q_table() 192 | else: 193 | q_table = init_q_table(n_distance_states, n_cat_states, n_actions) 194 | 195 | 196 | n_steps = 0 197 | 198 | 199 | # http://www-anw.cs.umass.edu/~barto/courses/cs687/Chapter%207.pdf pg 14+ 200 | # trace_decay of 0 == temporal difference learning 201 | # trace_decay of 1 == better monte carlo learning 202 | trace_decay = 0.9 # backward looking 203 | eligibility_trace = np.zeros((n_distance_states, n_cat_states, n_actions)) 204 | 205 | # prime loop with first action 206 | d = get_distance() 207 | c = get_cat() 208 | a = choose_action(d, c, q_table) 209 | start_time = datetime.datetime.now() 210 | 211 | 212 | while n_steps < n_loops: 213 | 214 | # move robot and update state 215 | reward = move(a, d, c) 216 | d_next = get_distance() 217 | c_next = get_cat() 218 | 219 | 220 | # chose action based on next observation 221 | a_next = choose_action(d_next, c_next, q_table) 222 | 223 | # SARSA learning 224 | s_target = reward + gamma + q_table[d_next][c_next][a_next] 225 | 226 | 227 | # what robot thought would happen next 228 | s_predict = q_table[d][c][a] 229 | 230 | 231 | # update eligibility trace 232 | eligibility_trace[d, c, a] += 1 233 | 234 | # update q_table to reflect new knowledge 235 | error = s_target - s_predict 236 | q_table[d][c][a] += lr * error * eligibility_trace[d][c][a] 237 | 238 | 239 | # decay eligibility trace 240 | eligibility_trace *= gamma * trace_decay 241 | 242 | # update state for next loop 243 | d = d_next 244 | c = c_next 245 | a = a_next 246 | 247 | n_steps += 1 248 | 249 | # save data every 100 steps incase of failure 250 | if n_steps % 100 == 0: 251 | save_q_table(q_table) 252 | print(datetime.datetime.now() - start_time) 253 | start_time = datetime.datetime.now() 254 | 255 | 256 | return q_table 257 | 258 | 259 | 260 | 261 | ############################################################################### 262 | # clean shut down of hardware 263 | ############################################################################### 264 | def cleanup(): 265 | 266 | cat_finder.cleanup() 267 | distance_finder.cleanup() 268 | moveRobot.cleanup() 269 | 270 | 271 | 272 | 273 | 274 | 275 | ############################################################################### 276 | # run code 277 | ############################################################################### 278 | 279 | q_table = rl() 280 | 281 | cleanup() 282 | 283 | 284 | ''' 285 | #q_table = load_q_table() 286 | print('--------------------------------') 287 | print('Final Q Table') 288 | 289 | for i in range(n_distance_states): 290 | for j in range(n_cat_states): 291 | print('distance %d, cat %d' %(i, j)) 292 | print('action values', s_table[i, j, :]) 293 | 294 | ''' 295 | 296 | 297 | 298 | # print actions by distance by cat 299 | z = np.zeros(n_distance_states) 300 | for i in range(n_distance_states): 301 | for j in range(n_cat_states): 302 | if j == 2: # no cat 303 | z[i] = np.argmax(q_table[i, j, :]) 304 | 305 | print('distance, action') 306 | for i in range(len(z)): 307 | a = int(z[i]) 308 | print(i, actions[a]) 309 | 310 | 311 | 312 | -------------------------------------------------------------------------------- /RobotBrain/Current_ClearRobot.py: -------------------------------------------------------------------------------- 1 | # http://github.com/timestocome 2 | 3 | 4 | # train a raspberry pi robot to wander the house while avoiding obstacles 5 | # and looking for cats 6 | # 2WD robot with a servo to control steering 7 | 8 | 9 | 10 | import numpy as np 11 | #from pathlib import Path 12 | import os.path 13 | 14 | from FindCats import FindCats 15 | from FindDistance import FindDistance 16 | from MoveClearRobot import MoveRobot 17 | 18 | import datetime 19 | 20 | 21 | 22 | # led light for cat 23 | import RPi.GPIO as gpio 24 | import time 25 | 26 | gpio.setmode(gpio.BOARD) 27 | gpio.setup(12, gpio.OUT) 28 | 29 | 30 | 31 | # test cat found led 32 | gpio.output(12, gpio.HIGH) 33 | time.sleep(5) 34 | gpio.output(12, gpio.LOW) 35 | 36 | 37 | 38 | 39 | ''' 40 | 41 | ################################################################################################# 42 | # utilities 43 | ################################################################################################# 44 | 45 | def save_state(s): 46 | np.save('state.npy', s) 47 | 48 | 49 | def load_state(): 50 | 51 | if os.path.isfile('state.npy'): 52 | 53 | state = np.load('state.npy') 54 | return state.reshape((n_cat, n_distance, n_actions)) 55 | 56 | else: 57 | return np.zeros((n_cat, n_distance, n_actions)) 58 | 59 | 60 | 61 | 62 | 63 | 64 | ################################################################################################ 65 | # init 66 | ################################################################################################ 67 | 68 | # init 69 | cat_finder = FindCats() 70 | min_cat = cat_finder.min_cat 71 | merlin = cat_finder.merlin 72 | no_cat = cat_finder.no_cat 73 | n_cat = 3 74 | 75 | 76 | distance_finder = FindDistance() 77 | max_distance = distance_finder.max_distance 78 | n_distance = max_distance + 1 79 | 80 | 81 | moveRobot = MoveRobot() 82 | actions = moveRobot.actions 83 | n_actions = len(actions) 84 | 85 | 86 | # possible states 87 | # rows min, merlin, no cat 88 | # columns are obstacle distance 0 - max distance +1 because arrays start at zero 89 | #state = np.zeros((n_cat, n_distance, n_actions)) 90 | 91 | # load saved state if it exists, else init to zeros 92 | state = load_state() 93 | 94 | # store moves here until we get a reward 95 | states_visited = [] 96 | 97 | 98 | 99 | # arrays are much easier and faster to work with than trees, so set up tree as arrays 100 | # tree needs each state, number of times visted and reward 101 | state_rewards = np.zeros((n_cat, n_distance)) 102 | action_rewards = np.zeros((n_cat, n_distance, n_actions)) 103 | 104 | 105 | # fill in rewards for various states 106 | state_rewards[min_cat,:] = 10 # found min 107 | state_rewards[merlin, :] = 10 # found merlin 108 | 109 | 110 | # too close to an obstacle 111 | buffer_distance = 12 # cm 112 | state_rewards[:, 0:buffer_distance] = -3 113 | 114 | 115 | # distance covered 116 | distance_traveled = 0 117 | 118 | 119 | 120 | # robot environment 121 | def get_distance(): 122 | # returns distance 1-50 123 | distance = distance_finder.get_distance() 124 | 125 | if distance < 1: distance = 1 126 | return int(distance) 127 | 128 | 129 | 130 | def get_cat(): 131 | # returns 0 for Min, 1 for Merlin, 2 for no cat 132 | cat = cat_finder.is_cat() 133 | 134 | found = np.argmax([cat[0][1], cat[1][1], cat[2][1]]) 135 | 136 | # turns on led when cat in sight, off otherwise 137 | if found == no_cat: 138 | gpio.output(12, gpio.LOW) 139 | else: 140 | gpio.output(12, gpio.HIGH) 141 | 142 | return found 143 | 144 | 145 | 146 | 147 | def update_values(r, dt, lr): 148 | 149 | # use number of steps required to get reward as lr 150 | # over many passes this will give actions closer to reward higher reward/penalty 151 | 152 | reward_per_state = (r + dt)/lr 153 | 154 | for i in range(len(states_visited)): 155 | c = states_visited[i][0] 156 | d = states_visited[i][1] 157 | a = states_visited[i][2] 158 | 159 | action_rewards[c][d][a] += reward_per_state 160 | #print(action_rewards[c][d][a]) 161 | 162 | 163 | 164 | 165 | 166 | 167 | # robot moves and reward is returned 168 | # robot receives reward for moving with out hitting obstacles 169 | # gets a large reward for catching either cat in its camera view 170 | # penatly for getting too close to obstacles 171 | # scale rewards from -1 to 1 172 | 173 | def move(action, distance, cat): 174 | 175 | global states_visited 176 | global distance_traveled 177 | 178 | reward = state_rewards[cat, distance] 179 | states_visited.append((cat, distance, action)) 180 | 181 | 182 | if reward != 0: 183 | update_values(reward, distance_traveled, len(states_visited)) 184 | states_visited = [] 185 | distance_traveled = 0 186 | 187 | #self.actions = ['center_forward', 'right_forward', 'left_forward', 188 | # 'center_reverse', 'right_reverse', 'left_reverse'] 189 | 190 | 191 | if action == 0: 192 | moveRobot.center_forward() 193 | elif action == 1: 194 | moveRobot.right_forward() 195 | elif action == 2: 196 | moveRobot.left_forward() 197 | elif action == 3: 198 | moveRobot.center_reverse() 199 | elif action == 4: 200 | moveRobot.right_reverse() 201 | elif action == 5: 202 | moveRobot.left_reverse() 203 | 204 | if action < 3: distance_traveled += 1 205 | else: distance_traveled -= 2 206 | # useful for adjusting rewards as needed 207 | #print("state %d %d, action %d, reward %d epsilon %lf" % (distance, cat, action, reward, epsilon)) 208 | 209 | 210 | return reward 211 | 212 | 213 | 214 | 215 | 216 | 217 | ############################################################################### 218 | # training 219 | ############################################################################### 220 | 221 | random_choice = .9 222 | for i in range(1000): 223 | #while(True): 224 | 225 | current_distance = get_distance() 226 | current_cat = get_cat() 227 | 228 | #print('distance %d cat %d' % (current_distance, current_cat)) 229 | 230 | action = np.random.randint(n_actions) 231 | 232 | if np.random.rand() > random_choice: 233 | action = np.argmax(action_rewards[current_cat, current_distance, :]) 234 | 235 | if random_choice > .1: 236 | random_choice *= .99 237 | 238 | move(action, current_distance, current_cat) 239 | 240 | if i%50 == 0: 241 | save_state(action_rewards) 242 | 243 | 244 | 245 | 246 | ############################################################################### 247 | # clean shut down of hardware 248 | ############################################################################### 249 | def cleanup(): 250 | 251 | cat_finder.cleanup() 252 | distance_finder.cleanup() 253 | moveRobot.cleanup() 254 | 255 | 256 | 257 | 258 | 259 | 260 | ############################################################################### 261 | # wrap things up 262 | ############################################################################### 263 | 264 | save_state(action_rewards) 265 | cleanup() 266 | 267 | 268 | # print actions by distance by cat 269 | z = np.zeros(n_distance) 270 | for i in range(n_distance-1): 271 | for j in range(n_cat-1): 272 | if j == 0: # no cat 273 | z[i] = np.argmax(action_rewards[i, j, :]) 274 | 275 | 276 | 277 | for i in range(len(z)): 278 | a = int(z[i]) 279 | print(i, actions[a]) 280 | 281 | ''' 282 | 283 | -------------------------------------------------------------------------------- /RobotBrain/FindCats.py: -------------------------------------------------------------------------------- 1 | # http://github.com/timestocome 2 | 3 | # take photo with PiCamera 4 | # determine if cat in photo 5 | 6 | # takes ~3 seconds to load 7 | # runs > 60fps, probably faster when not printing to screen 8 | 9 | 10 | 11 | import numpy as np 12 | import tensorflow as tf 13 | import picamera 14 | 15 | #import datetime 16 | 17 | class FindCats(object): 18 | 19 | def __init__(self): 20 | 21 | print('init cats') 22 | # set up constants 23 | self.height = 128 24 | self.width = 128 25 | self.input_mean = 0 26 | self.input_std = 255 27 | 28 | self.min_cat = 0 29 | self.merlin = 1 30 | self.no_cat = 2 31 | 32 | # setup graph 33 | # this is the saved graph that was trained on a desktop 34 | self.model_file = "output_graph.pb" 35 | self.label_file = "output_labels.txt" 36 | self.input_layer_name = "import/input" 37 | self.output_layer_name = "import/final_result" 38 | 39 | 40 | # setup camera 41 | camera = picamera.PiCamera() 42 | camera.resolution = (self.width, self.height) 43 | camera.vflip = True 44 | self.camera = camera 45 | 46 | 47 | 48 | # load saved, trained nn 49 | graph = tf.Graph() 50 | graph_def = tf.GraphDef() 51 | 52 | with open(self.model_file, "rb") as f: 53 | graph_def.ParseFromString(f.read()) 54 | 55 | with graph.as_default(): 56 | tf.import_graph_def(graph_def) 57 | 58 | self.graph = graph 59 | 60 | 61 | # load labels 62 | label = [] 63 | proto_as_ascii_lines = tf.gfile.GFile(self.label_file).readlines() 64 | 65 | for l in proto_as_ascii_lines: 66 | label.append(l.rstrip()) 67 | 68 | self.labels = label 69 | 70 | self.min_cat = 0 71 | self.merlin = 1 72 | self.no_cat = 2 73 | 74 | 75 | # create session 76 | self.sess = tf.Session(graph=self.graph) 77 | self.camera_sess = tf.Session() 78 | 79 | 80 | 81 | def capture_image(self): 82 | 83 | image = np.empty((self.width, self.height, 3), dtype=np.uint8) 84 | self.camera.capture(image, 'rgb') 85 | 86 | float_caster = image.astype(np.float32) 87 | dims_expander = float_caster.reshape(1, 128, 128, 3) 88 | result = dims_expander - self.input_mean 89 | result = result / self.input_std 90 | 91 | 92 | 93 | return result 94 | 95 | 96 | 97 | 98 | 99 | def is_cat(self): 100 | 101 | #now = datetime.datetime.now() 102 | 103 | # take photo 104 | t = self.capture_image() 105 | 106 | # see if Min, Merlin or no cat in photo 107 | input_operation = self.graph.get_operation_by_name(self.input_layer_name); 108 | output_operation = self.graph.get_operation_by_name(self.output_layer_name); 109 | 110 | results = self.sess.run(output_operation.outputs[0], {input_operation.outputs[0]: t}) 111 | results = np.squeeze(results) 112 | 113 | 114 | 115 | found = [] 116 | for i in range(3): 117 | found.append((self.labels[i], results[i])) 118 | 119 | #print(datetime.datetime.now() - now) 120 | return found 121 | 122 | 123 | 124 | 125 | def cleanup(self): 126 | self.sess.close() 127 | 128 | 129 | 130 | 131 | ################################################## 132 | # run graph in loop 133 | ################################################# 134 | 135 | # test 136 | ''' 137 | loop = FindCats() 138 | 139 | for i in range(100): 140 | found_cat = loop.is_cat() 141 | print(found_cat) 142 | 143 | 144 | loop.cleanup() 145 | ''' 146 | -------------------------------------------------------------------------------- /RobotBrain/FindDistance.py: -------------------------------------------------------------------------------- 1 | # http://github.com/timestocome 2 | 3 | 4 | 5 | import RPi.GPIO as gpio 6 | import time 7 | 8 | 9 | class FindDistance(object): 10 | 11 | 12 | def __init__(self): 13 | 14 | gpio.setmode(gpio.BOARD) 15 | 16 | # set up and init 17 | self.trigger = 11 18 | self.echo = 13 19 | 20 | gpio.setup(self.trigger, gpio.OUT) 21 | gpio.setup(self.echo, gpio.IN) 22 | 23 | self.pulse_start = 0. 24 | self.pulse_end = 0. 25 | self.speed_of_sound = 343 * 100 26 | self.max_distance = 100 27 | self.min_distance = 1 28 | 29 | 30 | def get_distance(self): 31 | 32 | # clear trigger 33 | gpio.output(self.trigger, False) 34 | time.sleep(0.1) 35 | 36 | # send pulse to trigger 37 | gpio.output(self.trigger, True) 38 | time.sleep(0.00001) 39 | gpio.output(self.trigger, False) 40 | 41 | 42 | # check echo for return signal 43 | while gpio.input(self.echo) == 0: 44 | self.pulse_start = time.time() 45 | 46 | while gpio.input(self.echo) == 1: 47 | self.pulse_end = time.time() 48 | 49 | pulse_duration = self.pulse_end - self.pulse_start 50 | distance = self.speed_of_sound / 2. * pulse_duration 51 | distance = round(distance, 2) 52 | distance /= 2.54 # inches 53 | 54 | # filter out things far away 55 | if distance > self.max_distance: 56 | distance = self.max_distance 57 | 58 | # filter out junk 59 | if distance < self.min_distance: 60 | disance = self.min_distance 61 | 62 | return distance 63 | 64 | 65 | def cleanup(self): 66 | gpio.cleanup() 67 | 68 | 69 | 70 | ################################################### 71 | # test class 72 | ''' 73 | test_distance = FindDistance() 74 | distance = test_distance.get_distance() 75 | print(distance) 76 | test_distance.cleanup() 77 | ''' -------------------------------------------------------------------------------- /RobotBrain/MerlinRobot.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/timestocome/RaspberryPi-Robot/b10d25cbfe2f7a60b82649503ea18213bdfd0f66/RobotBrain/MerlinRobot.jpg -------------------------------------------------------------------------------- /RobotBrain/MoveBlackRobot.py: -------------------------------------------------------------------------------- 1 | # http://github.com/timestocome 2 | 3 | # move 2wd robot using wheels to steer 4 | 5 | import RPi.GPIO as gpio 6 | import time 7 | from time import sleep 8 | 9 | 10 | 11 | 12 | class MoveRobot(object): 13 | 14 | 15 | def __init__(self): 16 | 17 | print('init robot gpio') 18 | # set up hardware 19 | gpio.setmode(gpio.BOARD) # use pin numbers not gpio numbers 20 | 21 | 22 | # wheels ( 4 wheel motors ) 23 | self.reverse_left = 38 24 | self.reverse_right = 35 25 | self.forward_left = 40 26 | self.forward_right = 37 27 | 28 | gpio.setup(self.reverse_left, gpio.OUT) 29 | gpio.setup(self.forward_left, gpio.OUT) 30 | gpio.setup(self.forward_right, gpio.OUT) 31 | gpio.setup(self.reverse_right, gpio.OUT) 32 | 33 | self.wheel_pulse = 0.2 34 | #self.actions = ['forward', 'reverse', 'turn_left', 'turn_right', 'hard_left', 'hard_right'] 35 | self.actions = ['forward', 'reverse', 'hard_left', 'hard_right'] 36 | 37 | 38 | def forward(self): 39 | 40 | gpio.output(self.forward_right, gpio.HIGH) 41 | gpio.output(self.forward_left, gpio.HIGH) 42 | 43 | sleep(self.wheel_pulse) 44 | gpio.output(self.forward_right, gpio.LOW) 45 | gpio.output(self.forward_left, gpio.LOW) 46 | 47 | ''' 48 | def turn_left(self): 49 | 50 | gpio.output(self.forward_right, gpio.HIGH) 51 | 52 | sleep(self.wheel_pulse) 53 | gpio.output(self.forward_right, gpio.LOW) 54 | 55 | 56 | 57 | def turn_right(self): 58 | gpio.output(self.forward_left, gpio.HIGH) 59 | 60 | sleep(self.wheel_pulse) 61 | gpio.output(self.forward_left, gpio.LOW) 62 | 63 | ''' 64 | 65 | def reverse(self): 66 | 67 | gpio.output(self.reverse_left, gpio.HIGH) 68 | gpio.output(self.reverse_right, gpio.HIGH) 69 | 70 | sleep(self.wheel_pulse) 71 | gpio.output(self.reverse_left, gpio.LOW) 72 | gpio.output(self.reverse_right, gpio.LOW) 73 | 74 | 75 | 76 | def hard_right(self): 77 | 78 | gpio.output(self.forward_left, gpio.HIGH) 79 | gpio.output(self.reverse_right, gpio.HIGH) 80 | 81 | sleep(self.wheel_pulse) 82 | gpio.output(self.forward_left, gpio.LOW) 83 | gpio.output(self.reverse_right, gpio.LOW) 84 | 85 | 86 | 87 | def hard_left(self): 88 | 89 | gpio.output(self.forward_right, gpio.HIGH) 90 | gpio.output(self.reverse_left, gpio.HIGH) 91 | 92 | sleep(self.wheel_pulse) 93 | gpio.output(self.forward_right, gpio.LOW) 94 | gpio.output(self.reverse_left, gpio.LOW) 95 | 96 | 97 | 98 | def cleanup(self): 99 | gpio.cleanup() 100 | 101 | 102 | 103 | ####################################################################### 104 | # test 105 | ''' 106 | move_robot = MoveRobot() 107 | move_robot.forward() 108 | move_robot.reverse() 109 | move_robot.turn_left() 110 | move_robot.turn_right() 111 | move_robot.hard_left() 112 | move_robot.hard_right() 113 | 114 | ''' 115 | 116 | 117 | -------------------------------------------------------------------------------- /RobotBrain/MoveClearRobot.cpython-34.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/timestocome/RaspberryPi-Robot/b10d25cbfe2f7a60b82649503ea18213bdfd0f66/RobotBrain/MoveClearRobot.cpython-34.pyc -------------------------------------------------------------------------------- /RobotBrain/MoveClearRobot.py: -------------------------------------------------------------------------------- 1 | # http://github.com/timestocome 2 | 3 | 4 | # movement functions for robot 5 | # this robot has a servo for steering ~90' range 6 | # rear wheel drive with separate controls for the wheels 7 | 8 | 9 | import RPi.GPIO as gpio 10 | import Adafruit_PCA9685 11 | 12 | import time 13 | from time import sleep 14 | 15 | 16 | 17 | 18 | class MoveRobot(object): 19 | 20 | 21 | def __init__(self): 22 | 23 | # set up hardware 24 | gpio.setmode(gpio.BOARD) # use pin numbers not gpio numbers 25 | 26 | # wheels ( 2wd, rear wheels) 27 | self.reverse_left = 38 28 | self.reverse_right = 37 29 | self.forward_left = 26 30 | self.forward_right = 35 31 | 32 | gpio.setup(self.reverse_left, gpio.OUT) 33 | gpio.setup(self.forward_left, gpio.OUT) 34 | gpio.setup(self.forward_right, gpio.OUT) 35 | gpio.setup(self.reverse_right, gpio.OUT) 36 | 37 | self.wheel_pulse = 0.5 38 | 39 | 40 | # servo steering 41 | self.pwm = Adafruit_PCA9685.PCA9685() 42 | self.pwm.set_pwm_freq(60) 43 | 44 | # which servo is being sent the signal 45 | self.channel = 0 # using first of 16 channels 46 | 47 | self.servo_min = 300 48 | self.servo_max = 450 49 | self.servo_center = (self.servo_min + self.servo_max) // 2 50 | 51 | 52 | # actions robot can take 53 | self.actions = ['center_forward', 'right_forward', 'left_forward', 54 | 'center_reverse','right_reverse', 'left_reverse'] 55 | 56 | 57 | def forward(self): 58 | 59 | gpio.output(self.forward_right, gpio.HIGH) 60 | gpio.output(self.forward_left, gpio.HIGH) 61 | 62 | sleep(self.wheel_pulse/2) 63 | gpio.output(self.forward_right, gpio.LOW) 64 | gpio.output(self.forward_left, gpio.LOW) 65 | 66 | self.forward = forward 67 | 68 | 69 | def reverse(self): 70 | 71 | gpio.output(self.reverse_left, gpio.HIGH) 72 | gpio.output(self.reverse_right, gpio.HIGH) 73 | 74 | sleep(self.wheel_pulse/2) 75 | gpio.output(self.reverse_left, gpio.LOW) 76 | gpio.output(self.reverse_right, gpio.LOW) 77 | 78 | self.reverse = reverse 79 | 80 | 81 | 82 | 83 | 84 | def center_forward(self): 85 | self.pwm.set_pwm(0, 0, 375) 86 | self.forward(self) 87 | 88 | 89 | def hard_right_forward(self): 90 | self.pwm.set_pwm(0, 0, 300) 91 | self.forward(self) 92 | 93 | 94 | def right_forward(self): 95 | self.pwm.set_pwm(0, 0, 340) 96 | self.forward(self) 97 | 98 | 99 | def left_forward(self): 100 | self.pwm.set_pwm(0, 0, 410) 101 | self.forward(self) 102 | 103 | 104 | def hard_left_forward(self): 105 | self.pwm.set_pwm(0, 0, 450) 106 | self.forward(self) 107 | 108 | 109 | def center_reverse(self): 110 | self.pwm.set_pwm(0, 0, 375) 111 | self.reverse(self) 112 | 113 | 114 | def hard_right_reverse(self): 115 | self.pwm.set_pwm(0, 0, 300) 116 | self.reverse(self) 117 | 118 | 119 | def right_reverse(self): 120 | self.pwm.set_pwm(0, 0, 340) 121 | self.reverse(self) 122 | 123 | 124 | def left_reverse(self): 125 | self.pwm.set_pwm(0, 0, 410) 126 | self.reverse(self) 127 | 128 | 129 | def hard_left_reverse(self): 130 | self.pwm.set_pwm(0, 0, 450) 131 | self.reverse(self) 132 | 133 | 134 | 135 | 136 | 137 | 138 | def cleanup(self): 139 | 140 | gpio.cleanup() 141 | 142 | self.pwm.set_pwm(0, 0, self.servo_center) 143 | self.pwm = None 144 | 145 | 146 | ################################################################## 147 | # test 148 | 149 | ''' 150 | move_robot = MoveRobot() 151 | 152 | 153 | move_robot.center_forward() 154 | move_robot.hard_right_forward() 155 | move_robot.right_forward() 156 | move_robot.left_forward() 157 | move_robot.hard_left_forward() 158 | move_robot.center_reverse() 159 | move_robot.hard_right_reverse() 160 | move_robot.right_reverse() 161 | move_robot.left_reverse() 162 | move_robot.hard_left_reverse() 163 | ''' 164 | 165 | -------------------------------------------------------------------------------- /RobotBrain/README.md: -------------------------------------------------------------------------------- 1 | # Main Robot Loop (WIP) 2 | 3 | This section will contain the reinforcement network that allows the robot to respond to the environment 4 | 5 | Current status: 6 | 7 | Main loop reads distance to nearest obstacle and checks to see if cat1 or cat2 or no cat is in view. 8 | Both robots are using a simplified QLearning Value algorithm to learn. 9 | 10 | If the files are reloaded each time the learning should improve with each run 11 | 12 | 13 | To do: 14 | - reducing the number of states so the robots will train faster 15 | - speed things up, ~~cat id is still very slow~~ fixed, now steady at ~2 FPS 16 | - explore other RL algorithms 17 | 18 | WIP: 19 | - The SARSA algorithm creates a timid robot: 20 | - Timid robot being trained to hunt cats and avoid obstacles using RL Sarsa prefers turns despite higher reward for forward motion. There are 4 turns / forward motion which also leads to favoring turns 21 | - Both robots loop: check for cat, check distance, chose action, move, collect reward, update qTable once per second 22 | 23 | 24 | - What the robot learned in 500 loops: 25 | - distance(cm) action if no cat 26 | - 1 right 27 | - 2 right 28 | - 14 hard_right 29 | - 15 left 30 | - 17 left 31 | - 24 hard_left 32 | - 25 right 33 | - 28 hard_left 34 | - 29 right 35 | - 30 left 36 | - 33 hard_left 37 | - 38 hard_right 38 | - 41 right 39 | - 42 hard_right 40 | - 44 right 41 | - 46 hard_right 42 | - 48 left 43 | - 49 right 44 | - 53 left 45 | - 54 hard_right 46 | - 55 hard_left 47 | - 56 hard_right 48 | - 61 left 49 | - 63 right 50 | - 67 hard_right 51 | - 68 left 52 | - 73 left 53 | - 74 hard_right 54 | - 75 right 55 | - 76 hard_right 56 | - 78 left 57 | - 80 hard_left 58 | - 83 hard_left 59 | - 87 right 60 | - 93 hard_left 61 | - 96 hard_right 62 | - 97 right 63 | 64 | Useful information: 65 | 66 | https://github.com/MorvanZhou/Reinforcement-learning-with-tensorflow 67 | 68 | https://sites.google.com/view/deep-rl-bootcamp/lectures 69 | 70 | 71 | 72 | The robot is going to have to be a quick learner to survive Merlin 73 | 74 | -------------------------------------------------------------------------------- /RobotBrain/SarsaPlots.py: -------------------------------------------------------------------------------- 1 | # http://github.com/timestocome 2 | 3 | 4 | # train a raspberry pi robot to wander the house while avoiding obstacles 5 | # and looking for cats 6 | 7 | # this robot uses wheels for steering 8 | # 4 wheel drive with separate controls each side 9 | 10 | 11 | # change from off policy learning in first try 12 | # adapted from https://morvanzhou.github.io/tutorials/ 13 | 14 | 15 | import numpy as np 16 | 17 | 18 | 19 | ############################################################################### 20 | # q learning happens here 21 | ############################################################################### 22 | actions = ['forward', 'reverse', 'turn_left', 'turn_right', 'hard_left', 'hard_right'] 23 | n_distance_states = 100 + 1 24 | n_cat_states = 3 25 | n_actions = 6 26 | 27 | qTable = 'qTable.npy' 28 | 29 | 30 | 31 | # load saved table from file 32 | def load_q_table(): 33 | 34 | t_1d = np.load(qTable) 35 | table = t_1d.reshape(n_distance_states, n_cat_states, n_actions) 36 | 37 | return table 38 | 39 | 40 | 41 | 42 | 43 | q_table = load_q_table() 44 | print('--------------------------------') 45 | print('Final Q Table') 46 | 47 | for i in range(n_distance_states): 48 | for j in range(n_cat_states): 49 | print('distance %d, cat %d' %(i, j)) 50 | print('action values', q_table[i, j, :]) 51 | 52 | 53 | 54 | 55 | # print actions by distance no cat 56 | z = np.zeros(n_distance_states) 57 | for i in range(n_distance_states): 58 | for j in range(n_cat_states): 59 | if j == 2: # no cat 60 | z[i] = np.argmax(q_table[i, j, :]) 61 | 62 | print('--------- distance/ action -------------') 63 | for i in range(len(z)): 64 | a = int(z[i]) 65 | print(i, actions[a]) -------------------------------------------------------------------------------- /RobotBrain/output_graph.pb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/timestocome/RaspberryPi-Robot/b10d25cbfe2f7a60b82649503ea18213bdfd0f66/RobotBrain/output_graph.pb -------------------------------------------------------------------------------- /RobotBrain/output_labels.txt: -------------------------------------------------------------------------------- 1 | min 2 | merlin 3 | no cat 4 | -------------------------------------------------------------------------------- /Software/LoadImagesPiCamera.py: -------------------------------------------------------------------------------- 1 | 2 | # https://github.com/timestocome/RaspberryPi 3 | 4 | # start grabbing video from RaspberryPi camera and saving to disk as numpy array 5 | # if i > 5 saves 7 per loop (0...6) 6 | 7 | 8 | # https://www.raspberrypi.org/learning/getting-started-with-picamera/worksheet/ 9 | # https://picamera.readthedocs.io/en/release-1.13/index.html 10 | 11 | from picamera import PiCamera 12 | from time import sleep 13 | import numpy as np 14 | 15 | 16 | 17 | # init 18 | camera = PiCamera() 19 | camera.rotation = 180 # rotate if image not upright 20 | 21 | # default is 1920 x 1080 drop it way down for TF 22 | height = 192 23 | width = 192 24 | camera.resolution = (width, height) 25 | camera.framerate = 1 26 | 27 | i = 0 28 | 29 | while(True): 30 | if i > 5: i = 0 31 | else: i+=1 32 | filename = '/home/pi/videoFeed/image%s.npy' % i 33 | 34 | output = np.empty((width, height, 3 ), dtype=np.uint8) 35 | camera.capture(output, 'rgb') 36 | np.save(filename, output) 37 | 38 | 39 | 40 | -------------------------------------------------------------------------------- /Software/README.md: -------------------------------------------------------------------------------- 1 | # RaspberryPi setup for machine learning robotics 2 | 3 | 4 | ### ----- Setup ----- 5 | 6 | RaspberryPi Software Guide (basic OS directions and images) 7 | 8 | https://www.raspberrypi.org/learning/software-guide/quickstart/ 9 | 10 | 11 | 12 | Download SD Formatter 13 | 14 | https://www.sdcard.org/ 15 | 16 | 17 | 18 | Download Raspbian 19 | 20 | https://www.raspberrypi.org/downloads/ 21 | 22 | 23 | Download Etcher.io 24 | 25 | https://etcher.io/ 26 | 27 | 28 | Insert SD card into your computer (16GB+) 29 | 30 | - If your computer won't read it ( My Surface and iMac wouldn't load some of the cards I tried) 31 | --- Stick the card in a camera and format it 32 | --- Then plug camera into computer and format with SD Formatter 33 | 34 | 35 | Use the SD Formatter to do a full format 36 | 37 | Use Etcher.io to flash the OS onto the card ( be sure to unzip Raspbian first ) 38 | 39 | 40 | 41 | 42 | ### ----- Install Emacs ----- 43 | Emacs is much easier to use than vi for editing system files 44 | 45 | > sudo apt-get install emacs 46 | 47 | 48 | 49 | ### ----- Setup the Pi Camera ----- 50 | 51 | Enable the camera 52 | > sudo raspi-config at a command prompt in the terminal 53 | - Select 5. Interfacing options 54 | - Select P1 Enable/Disable Camera 55 | - Select 'Finish' to save changes and reboot 56 | 57 | 58 | Python library and basics for accessing the camera 59 | 60 | https://www.raspberrypi.org/documentation/usage/camera/python/README.md 61 | 62 | 63 | ### ----- Install OpenCV ----- 64 | 65 | https://raspberrypi.stackexchange.com/questions/69169/how-to-install-opencv-on-raspberry-pi-3-in-raspbian-jessie 66 | 67 | 68 | Access and test OpenCV with Python 69 | 70 | http://www.pyimagesearch.com/2015/03/30/accessing-the-raspberry-pi-camera-with-opencv-and-python/ 71 | 72 | 73 | ### ----- Setup fixed IP and VNC ----- 74 | 75 | 76 | Set a fixed wireless ip on the RaspberryPi 77 | 78 | https://www.modmypi.com/blog/how-to-give-your-raspberry-pi-a-static-ip-address-update 79 | 80 | 81 | 82 | - edit /etc/dhcpcd.conf 83 | - add this to the bottom of the file, save it and reboot 84 | * use your router ip and ip address, you can find these by typing ifconfig at a command prompt 85 | 86 | ##### 87 | 88 | interface wlan0 89 | 90 | static ip_address=192.168.0.200 91 | 92 | static routers=192.168.0.1 93 | 94 | static domain_name_servers=192.168.0.1 95 | 96 | #### 97 | 98 | 99 | Set up a wireless VNC so you can access your RaspberriPi from your desktop computer 100 | 101 | https://www.realvnc.com/en/connect/docs/raspberry-pi.html#raspberry-pi-setup 102 | 103 | user: pi 104 | 105 | passwd: raspberry 106 | 107 | 108 | 109 | Set the VNC to run at boot every time you start your RaspberryPi 110 | 111 | > sudo systemctl enable vncserver-x11-serviced.service 112 | 113 | 114 | 115 | Edit /boot/config.txt on RaspberryPi to get a full screen resolution on your desktop 116 | 117 | https://www.raspberrypi.org/forums/viewtopic.php?t=200196 118 | ---- 119 | 120 | hdmi_force_hotplug=1 121 | hdmi_force_mode=1 122 | hdmi_drive=2 123 | hdmi_group=1 124 | hdmi_mode=16 125 | 126 | ---- 127 | 128 | 129 | ### ----- Install Mini Anaconda ----- 130 | 131 | https://www.continuum.io/blog/developer/anaconda-raspberry-pi 132 | 133 | https://stackoverflow.com/questions/39371772/how-to-install-anaconda-on-raspberry-pi-3-model-b 134 | 135 | 136 | ### ----- Install TensorFlow ----- 137 | 138 | - First check to be sure you are using the Python V3.4 or 3.5 139 | 140 | > python --version 141 | 142 | - Then follow the directions: 143 | 144 | sudo apt-get update && sudo apt-get upgrade 145 | sudo apt-get install python3-pip python3-dev 146 | pip3 install tensorflow 147 | 148 | 149 | 150 | - If you run into trouble check here: 151 | 152 | https://www.raspberrypi.org/magpi/tensorflow-ai-raspberry-pi/ 153 | 154 | 155 | 156 | - TensorFlow Object Detection model and installation instructions 157 | 158 | https://github.com/tensorflow/models/tree/master/research/object_detection 159 | 160 | 161 | 162 | ### ----- Backup and Restore SD Card ----- 163 | 164 | https://thepihut.com/blogs/raspberry-pi-tutorials/17789160-backing-up-and-restoring-your-raspberry-pis-sd-card 165 | 166 | Convert *.dmg to *.img 167 | 168 | http://www.instructables.com/id/Restore-DMG-to-SD-Card-MAC-OS-X/ 169 | 170 | 171 | ### ------- Transfer Files between Desktop and RaspberryPi ------- 172 | 173 | VNC lets you move files between your computer and the Pi 174 | 175 | https://www.realvnc.com/en/connect/docs/file-transfer.html 176 | 177 | https://www.realvnc.com/en/connect/docs/file-transfer.html#file-transfer-troubleshoot 178 | 179 | 180 | ### ------- Misc ------- 181 | 182 | none 183 | 184 | ### ----- Change default Python version ----- 185 | 186 | > cd /usr/bin 187 | 188 | > sudo rm python 189 | 190 | > sudo ln -s python2.7 python 191 | 192 | or ... 193 | 194 | > sudo ln -s python3.4 python 195 | 196 | 197 | 198 | 199 | -------------------------------------------------------------------------------- /Software/facedetector.py: -------------------------------------------------------------------------------- 1 | 2 | # http://github.com/timestocome 3 | 4 | # use the raspberri pi camera to stream a small image, slowly and check for faces. 5 | 6 | 7 | 8 | # adapted from 9 | # https://github.com/DeligenceTechnologies/Face-detection-using-OpenCV-With-Raspberry-PI/blob/master/face%20detection.py 10 | 11 | import io 12 | import picamera 13 | import cv2 14 | import numpy as np 15 | from time import sleep 16 | 17 | 18 | camera = picamera.PiCamera() 19 | camera.resolution = (320, 320) 20 | camera.framerate = 10 21 | sleep(2) 22 | 23 | while True: 24 | 25 | output = np.empty((320, 320, 3), dtype=np.uint8) 26 | camera.capture(output, 'bgr') 27 | 28 | 29 | 30 | #Now creates an OpenCV image 31 | #image = cv2.imdecode(output, 1) 32 | image = output.reshape((320, 320, 3)) 33 | 34 | #Load a cascade file for detecting faces 35 | face_cascade = cv2.CascadeClassifier('/usr/local/share/OpenCV/haarcascades/haarcascade_frontalface_alt.xml') 36 | 37 | #Convert to grayscale 38 | gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) 39 | 40 | #Look for faces in the image using the loaded cascade file 41 | faces = face_cascade.detectMultiScale(gray, 1.1, 5) 42 | 43 | print ("Found ", len(faces)) 44 | 45 | 46 | # save for later ....... 47 | #Draw a rectangle around every found face 48 | #for (x,y,w,h) in faces: 49 | # cv2.rectangle(image,(x,y),(x+w,y+h),(255,255,0),2) 50 | 51 | -------------------------------------------------------------------------------- /catID/FindCats.py: -------------------------------------------------------------------------------- 1 | # http://github.com/timestocome 2 | 3 | # take photo with PiCamera 4 | # determine if cat in photo 5 | 6 | # takes ~3 seconds to load 7 | # runs > 60fps, probably faster when not printing to screen 8 | 9 | 10 | 11 | import numpy as np 12 | import tensorflow as tf 13 | import picamera 14 | 15 | import datetime 16 | 17 | class FindCats(object): 18 | 19 | def __init__(self): 20 | 21 | print('init cats') 22 | # set up constants 23 | self.height = 128 24 | self.width = 128 25 | self.input_mean = 0 26 | self.input_std = 255 27 | 28 | self.min_cat = 0 29 | self.merlin = 1 30 | self.no_cat = 2 31 | 32 | # setup graph 33 | # this is the saved graph that was trained on a desktop 34 | self.model_file = "output_graph.pb" 35 | self.label_file = "output_labels.txt" 36 | self.input_layer_name = "import/input" 37 | self.output_layer_name = "import/final_result" 38 | 39 | 40 | # setup camera 41 | camera = picamera.PiCamera() 42 | camera.resolution = (self.width, self.height) 43 | camera.vflip = True 44 | self.camera = camera 45 | 46 | 47 | 48 | # load saved, trained nn 49 | graph = tf.Graph() 50 | graph_def = tf.GraphDef() 51 | 52 | with open(self.model_file, "rb") as f: 53 | graph_def.ParseFromString(f.read()) 54 | 55 | with graph.as_default(): 56 | tf.import_graph_def(graph_def) 57 | 58 | self.graph = graph 59 | 60 | 61 | # load labels 62 | label = [] 63 | proto_as_ascii_lines = tf.gfile.GFile(self.label_file).readlines() 64 | 65 | for l in proto_as_ascii_lines: 66 | label.append(l.rstrip()) 67 | 68 | self.labels = label 69 | 70 | self.min_cat = 0 71 | self.merlin = 1 72 | self.no_cat = 2 73 | 74 | 75 | # create session 76 | self.sess = tf.Session(graph=self.graph) 77 | self.camera_sess = tf.Session() 78 | 79 | 80 | 81 | def capture_image(self): 82 | 83 | image = np.empty((self.width, self.height, 3), dtype=np.uint8) 84 | self.camera.capture(image, 'rgb') 85 | 86 | float_caster = image.astype(np.float32) 87 | dims_expander = float_caster.reshape(1, 128, 128, 3) 88 | result = dims_expander - self.input_mean 89 | result = result / self.input_std 90 | 91 | 92 | 93 | return result 94 | 95 | 96 | 97 | 98 | 99 | def is_cat(self): 100 | 101 | now = datetime.datetime.now() 102 | 103 | # take photo 104 | t = self.capture_image() 105 | 106 | # see if Min, Merlin or no cat in photo 107 | input_operation = self.graph.get_operation_by_name(self.input_layer_name); 108 | output_operation = self.graph.get_operation_by_name(self.output_layer_name); 109 | 110 | results = self.sess.run(output_operation.outputs[0], {input_operation.outputs[0]: t}) 111 | results = np.squeeze(results) 112 | 113 | 114 | 115 | found = [] 116 | for i in range(3): 117 | found.append((self.labels[i], results[i])) 118 | 119 | 120 | print(datetime.datetime.now() - now) 121 | return found 122 | 123 | 124 | 125 | 126 | def cleanup(self): 127 | self.sess.close() 128 | 129 | 130 | 131 | 132 | ################################################## 133 | # run graph in loop 134 | ################################################# 135 | 136 | # test 137 | ''' 138 | loop = FindCats() 139 | 140 | for i in range(100): 141 | found_cat = loop.is_cat() 142 | print(found_cat) 143 | 144 | 145 | loop.cleanup() 146 | ''' 147 | -------------------------------------------------------------------------------- /catID/README.md: -------------------------------------------------------------------------------- 1 | # Cat id uses MobileNet to identify our two cats 2 | 3 | 4 | Google directions ( start here to be sure you understand how to use and load models): 5 | 6 | https://github.com/tensorflow/models/tree/master/research 7 | 8 | list of models: 9 | 10 | https://github.com/tensorflow/models/blob/master/research/slim/nets/mobilenet_v1.md 11 | 12 | directions to train a model on your images: 13 | 14 | https://hackernoon.com/creating-insanely-fast-image-classifiers-with-mobilenet-in-tensorflow-f030ce0a2991 15 | 16 | 17 | Training is best done on your desktop computer. Once it's complete copy the model, labels and label.py to 18 | the Raspberry Pi to run it. 19 | 20 | While training Tensorflow will put the saved model and label files in your /tmp dir 21 | 22 | 23 | On the Pi: 24 | - Tensorflow must be installed 25 | - output_graph.pb ( your model ) 26 | - output_labels.txt (your labels ) 27 | - label_cats.py ( Python script to load graph and 'test.jpg' to see if it is Min, Merlin, No cat ) 28 | 29 | 30 | -------------------------------------------------------------------------------- /catID/findCats.py: -------------------------------------------------------------------------------- 1 | # http://github.com/timestocome 2 | 3 | # take photo with PiCamera 4 | # determine if cat in photo 5 | 6 | # takes ~3 seconds to load 7 | # runs > 60fps, probably faster when not printing to screen 8 | 9 | 10 | 11 | import numpy as np 12 | import tensorflow as tf 13 | import picamera 14 | 15 | 16 | 17 | 18 | class findCats(object): 19 | 20 | def __init__(self): 21 | 22 | # set up constants 23 | self.height = 128 24 | self.width = 128 25 | self.input_mean = 0 26 | self.input_std = 255 27 | 28 | 29 | # setup graph 30 | # this is the saved graph that was trained on a desktop 31 | self.model_file = "output_graph.pb" 32 | self.label_file = "output_labels.txt" 33 | self.input_layer_name = "import/input" 34 | self.output_layer_name = "import/final_result" 35 | 36 | 37 | # setup camera 38 | camera = picamera.PiCamera() 39 | camera.resolution = (self.width, self.height) 40 | camera.vflip = True 41 | self.camera = camera 42 | 43 | 44 | 45 | # load saved, trained nn 46 | graph = tf.Graph() 47 | graph_def = tf.GraphDef() 48 | 49 | with open(self.model_file, "rb") as f: 50 | graph_def.ParseFromString(f.read()) 51 | 52 | with graph.as_default(): 53 | tf.import_graph_def(graph_def) 54 | 55 | self.graph = graph 56 | 57 | 58 | # load labels 59 | label = [] 60 | proto_as_ascii_lines = tf.gfile.GFile(self.label_file).readlines() 61 | 62 | for l in proto_as_ascii_lines: 63 | label.append(l.rstrip()) 64 | 65 | self.labels = label 66 | 67 | 68 | # create session 69 | self.sess = tf.Session(graph=self.graph) 70 | 71 | 72 | 73 | 74 | def capture_image(self): 75 | 76 | image = np.empty((self.width, self.height, 3), dtype=np.uint8) 77 | self.camera.capture(image, 'rgb') 78 | 79 | float_caster = tf.cast(image, tf.float32) 80 | dims_expander = tf.expand_dims(float_caster, 0); 81 | resized = tf.image.resize_bilinear(dims_expander, [self.height, self.width]) 82 | 83 | normalized = tf.divide(tf.subtract(resized, [self.input_mean]), [self.input_std]) 84 | sess = tf.Session() 85 | result = sess.run(normalized) 86 | 87 | return result 88 | 89 | 90 | 91 | def run_graph(self): 92 | 93 | # take photo 94 | t = self.capture_image() 95 | 96 | # see if Min, Merlin or no cat in photo 97 | input_operation = self.graph.get_operation_by_name(self.input_layer_name); 98 | output_operation = self.graph.get_operation_by_name(self.output_layer_name); 99 | 100 | results = self.sess.run(output_operation.outputs[0], {input_operation.outputs[0]: t}) 101 | results = np.squeeze(results) 102 | 103 | top_k = results.argsort()[-3:][::-1] 104 | 105 | # print results 106 | #for i in top_k: 107 | # print(self.labels[i], results[i]) 108 | 109 | found = [] 110 | for i in top_k: 111 | found.append((self.labels[i], results[i])) 112 | return found 113 | 114 | 115 | 116 | 117 | def cleanup(self): 118 | self.sess.close() 119 | 120 | 121 | 122 | 123 | ################################################## 124 | # run graph in loop 125 | ################################################# 126 | loop = findCats() 127 | 128 | while(True): 129 | found_cat = loop.run_graph() 130 | print(found_cat) 131 | print(found_cat[0][0], found_cat[0][1]) 132 | 133 | loop.cleanup() 134 | 135 | -------------------------------------------------------------------------------- /catID/label_cats.py: -------------------------------------------------------------------------------- 1 | # http://github.com/timestocome 2 | # cleaned up and streamlined to run on RaspberryPi 3 | 4 | 5 | # Copyright 2017 The TensorFlow Authors. All Rights Reserved. 6 | # 7 | # Licensed under the Apache License, Version 2.0 (the "License"); 8 | # you may not use this file except in compliance with the License. 9 | # You may obtain a copy of the License at 10 | # 11 | # http://www.apache.org/licenses/LICENSE-2.0 12 | # 13 | # Unless required by applicable law or agreed to in writing, software 14 | # distributed under the License is distributed on an "AS IS" BASIS, 15 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 | # See the License for the specific language governing permissions and 17 | # limitations under the License. 18 | # ============================================================================== 19 | 20 | import sys 21 | import numpy as np 22 | import tensorflow as tf 23 | 24 | 25 | 26 | def load_graph(model_file): 27 | 28 | graph = tf.Graph() 29 | graph_def = tf.GraphDef() 30 | 31 | with open(model_file, "rb") as f: 32 | graph_def.ParseFromString(f.read()) 33 | 34 | with graph.as_default(): 35 | tf.import_graph_def(graph_def) 36 | 37 | return graph 38 | 39 | 40 | def read_tensor_from_image_file(file_name='test.jpg', input_height=128, input_width=128, 41 | input_mean=0, input_std=255): 42 | 43 | 44 | input_name = "file_reader" 45 | output_name = "normalized" 46 | file_reader = tf.read_file(file_name, input_name) 47 | image_reader = tf.image.decode_jpeg(file_reader, channels = 3, name='jpeg_reader') 48 | float_caster = tf.cast(image_reader, tf.float32) 49 | dims_expander = tf.expand_dims(float_caster, 0); 50 | resized = tf.image.resize_bilinear(dims_expander, [input_height, input_width]) 51 | normalized = tf.divide(tf.subtract(resized, [input_mean]), [input_std]) 52 | sess = tf.Session() 53 | result = sess.run(normalized) 54 | 55 | return result 56 | 57 | 58 | 59 | def load_labels(label_file): 60 | 61 | label = [] 62 | proto_as_ascii_lines = tf.gfile.GFile(label_file).readlines() 63 | for l in proto_as_ascii_lines: 64 | label.append(l.rstrip()) 65 | 66 | return label 67 | 68 | 69 | 70 | if __name__ == "__main__": 71 | file_name = "test.jpg" 72 | model_file = "output_graph.pb" 73 | label_file = "output_labels.txt" 74 | input_height = 128 75 | input_width = 128 76 | input_mean = 0 77 | input_std = 255 78 | input_layer = "input" 79 | output_layer = "final_result" 80 | 81 | 82 | graph = load_graph(model_file) 83 | t = read_tensor_from_image_file(file_name, 84 | input_height=input_height, 85 | input_width=input_width, 86 | input_mean=input_mean, 87 | input_std=input_std) 88 | 89 | input_name = "import/" + input_layer 90 | output_name = "import/" + output_layer 91 | input_operation = graph.get_operation_by_name(input_name); 92 | output_operation = graph.get_operation_by_name(output_name); 93 | 94 | sess = tf.Session(graph=graph) 95 | 96 | results = sess.run(output_operation.outputs[0], 97 | {input_operation.outputs[0]: t}) 98 | 99 | results = np.squeeze(results) 100 | 101 | top_k = results.argsort()[-5:][::-1] 102 | labels = load_labels(label_file) 103 | 104 | for i in top_k: 105 | print(labels[i], results[i]) 106 | -------------------------------------------------------------------------------- /catID/output_graph.pb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/timestocome/RaspberryPi-Robot/b10d25cbfe2f7a60b82649503ea18213bdfd0f66/catID/output_graph.pb -------------------------------------------------------------------------------- /catID/output_labels.txt: -------------------------------------------------------------------------------- 1 | min 2 | merlin 3 | no cat 4 | -------------------------------------------------------------------------------- /catID/rename_cat_files.py: -------------------------------------------------------------------------------- 1 | # http://github.com/timestocome 2 | 3 | # clean up cat photos for use in training robots to 4 | # recognize each cat 5 | # 2 cats in household 6 | # 50 photos each, ~25 actual photos of each, ~25 similar looking cats from internet 7 | # crop each to 1000 x 1000 if needed (same size so have random amount of cat and background) 8 | # resize images to a max of 300 if needed, robot camera input is going to be set low 9 | # each cat is in a different dir, relabel files and combine 10 | # name each file with cat name to make labeling easier 11 | 12 | 13 | import os 14 | from os import rename 15 | from os.path import basename 16 | 17 | 18 | # get file list 19 | path = os.getcwd() 20 | 21 | merlin_files = os.listdir('merlin') 22 | min_files = os.listdir('min') 23 | 24 | 25 | os.chdir('merlin') 26 | i = 1 27 | for f in merlin_files: 28 | new_name = 'merlin_%d.jpg' % i 29 | old_name = f 30 | print(old_name, new_name) 31 | os.rename(old_name, new_name) 32 | i+= 1 33 | 34 | os.chdir(path) 35 | os.chdir('min') 36 | i = 1 37 | for f in min_files: 38 | new_name = 'min_%d.jpg' % i 39 | old_name = f 40 | print(old_name, new_name) 41 | os.rename(old_name, new_name) 42 | i+=1 -------------------------------------------------------------------------------- /catID/test1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/timestocome/RaspberryPi-Robot/b10d25cbfe2f7a60b82649503ea18213bdfd0f66/catID/test1.jpg -------------------------------------------------------------------------------- /catID/test2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/timestocome/RaspberryPi-Robot/b10d25cbfe2f7a60b82649503ea18213bdfd0f66/catID/test2.jpg -------------------------------------------------------------------------------- /robots.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/timestocome/RaspberryPi-Robot/b10d25cbfe2f7a60b82649503ea18213bdfd0f66/robots.jpg --------------------------------------------------------------------------------