├── README.md ├── ROS-glossary └── README.md └── docker-setup ├── Dockerfile ├── README.md ├── maps.txt ├── media ├── ipconfig.jpg ├── xlaunch-1.jpg └── xlaunch-2.jpg ├── ros-init.sh ├── run-simulator.sh └── set-display.sh /README.md: -------------------------------------------------------------------------------- 1 | # Crash Course on Autonomous Racing 2 | 3 | Here are some of the most essential topics that are recommended for getting a racing system ready, most of which are from [F1Tenth](https://f1tenth.org/learn.html). This is a good platform for introducing topics in autonomous racing. Some in-depth topics may not be possible to cover in this README, but can be freely expolored! 4 | 5 | **Recommended** to learn on-the-go, alongside system-specific learning/research. Every topic is highly useful for your benefit, but learn what you need for your own purpose. **Challenging**, but productive (and interesting) way to appreciate the design of autonomous systems, and to gain the most applicable skills. Especially good to learn if you are not in your final year of study, or want to learn outside of your studies. 6 | 7 | This document will get updated regularly, so you can star this repo, or fork it. Some extra content may be added in addition to just this README file. 8 | 9 | ### Before you start: 10 | 11 | - You can install Ubuntu 18.04 LTS as a dual-boot OS following the [Tutorial](https://ubuntu.com/tutorials/install-ubuntu-desktop#1-overview) (***BACKUP YOUR MAIN OS AND LOCAL DRIVES IN CASE OF DATA LOSS DUE TO INSTALLATION ERRORS, EVEN IF RARE***) - Most reliable way once you have the OS installed 12 | - You can run everything in a [Docker](https://www.docker.com/products/docker-desktop) container with Ubuntu 18.04 and [ros:melodic-robot-bionic](https://hub.docker.com/_/ros?tab=description) (reference to [Dockerfile](https://github.com/osrf/docker_images/blob/7ff09c2a75e902bc2bb25a1f1ae748ec4e9c7a4b/ros/melodic/ubuntu/bionic/robot/Dockerfile) and [Tutorial](http://wiki.ros.org/docker/Tutorials)) - You may need to give Docker [permissions](https://stackoverflow.com/questions/48957195/how-to-fix-docker-got-permission-denied-issue) after installation (using Docker engine in Ubuntu). Once you have Docker installed, try out our [quick-start guide](./docker-setup) for running the F1Tenth Simulator with Docker. 13 | - Ubuntu Virtual Machine (like VirtualBox) on your main OS is fine also, but may run slightly slower than dual-boot. Example guide [here](https://linuxhint.com/install_ubuntu_18-04_virtualbox/). 14 | - Guide to [ROS Melodic](http://wiki.ros.org/melodic/Installation/Ubuntu) installation (prerequisite for F1Tenth Sim is ros-melodic-desktop, but full version is recommended, in case you want to more yourself). 15 | - Get familiar with [Linear Algebra](http://cs229.stanford.edu/section/cs229-linalg.pdf) and [Probability](http://cs229.stanford.edu/section/cs229-prob.pdf) on-the-go, as it's very common in the theory the more in-depth you go 16 | - Check out the [AMZ team resources](https://github.com/AMZ-Driverless/fsd-resources) for extra stuff like example systems, datasets, and references! 17 | - ALSO: Start off from [F1Tenth Gym](https://github.com/f1tenth/f1tenth_gym) for easier setup and for trying out the labs! 18 | 19 | ### Stuck on anything? No worries!: 20 | 21 | - Ask away by creating an issue [here](https://github.com/FT-Autonomous/Autonomous_Crash_Course/issues), for asking for help with any aspect of the material (theory, setup, debugging). 22 | - Reach out to anyone and you'll get all the help possible! 23 | 24 | ### ROS Tips 25 | 26 | - This [cheat sheet](http://air.imag.fr/images/f/f7/ROScheatsheet.pdf) shows very comprehenisve command line commands for ROS, as well as common commands [here](https://subscription.packtpub.com/book/hardware_and_creative/9781782175193/1/ch01lvl1sec15/ros-commands-summary) 27 | 28 | ### Introduction to Autonomous Racing 29 | 30 | 1. [(F1Tenth) Lecture 1](https://f1tenth-coursekit.readthedocs.io/en/stable/lectures/ModuleA/lecture01.html) - Introduces F1Tenth platform, autonomous racing concepts, and simulator (Ignore UPenn-specific logistics) 31 | 2. [(F1Tenth) Tutorial 1](https://f1tenth-coursekit.readthedocs.io/en/stable/lectures/ModuleA/tutorial1.html) - Introduces the learning-based F1Tenth simulator (should run on most computers, backup main OS before making new Ubuntu installation). [Sim installation on Ubuntu and more](https://f1tenth.readthedocs.io/en/stable/going_forward/simulator/sim_info.html#adding-a-planning-node) 32 | 3. [(F1Tenth) Lab 1](https://f1tenth-coursekit.readthedocs.io/en/stable/assignments/labs/lab1.html) - Introduces the Robot Operating System (ROS) framework, good for learning main features (we will use ROS Melodic as the distro) ([Further tutorials on ROS in their wiki](http://wiki.ros.org/ROS/Tutorials)) 33 | 4. [(YouTube) The ethical dilemma of self-driving cars - Patrick Lin](https://www.youtube.com/watch?v=ixIoDYVfKA0) 34 | 5. [(YouTube) First Formula Student Germany (Driverless) 2017 Moments](https://www.youtube.com/watch?v=dZVfvsR8NM0) 35 | 6. [(Tool) Learn X in Y minutes](https://learnxinyminutes.com/) (for any language of your choice, we recommend Python, C/C++) 36 | 7. [(Tutorial) Using ROS with C++ and Python Nodes](http://wiki.ros.org/ROSNodeTutorialC%2B%2B) - Also includes tutorials for C++ and Python individually. This is for extra reference 37 | 38 | ### Core Concepts 39 | 40 | 1. [(F1Tenth) Lecture 2](https://f1tenth-coursekit.readthedocs.io/en/stable/lectures/ModuleA/lecture02.html) - Introduces safety systems, and AV sensors 41 | 2. [(F1Tenth) Tutorial 2](https://f1tenth-coursekit.readthedocs.io/en/stable/lectures/ModuleA/tutorial2.html) - Introduces the addition of system planners in F1Tenth Simulator 42 | 3. [(F1Tenth) Lab 2](https://f1tenth-coursekit.readthedocs.io/en/stable/assignments/labs/lab2.html) - Introduces the programming of an Automatic Braking System in F1Tenth Sim. Important safety feature of car 43 | 4. [(F1Tenth) Lecture 3](https://f1tenth-coursekit.readthedocs.io/en/stable/lectures/ModuleA/lecture03.html) - Introduces coordinates, reference frames, rigid body transformations, and tf2_ros 44 | 5. [(YouTube) Controlling Self Driving Cars](https://www.youtube.com/watch?v=4Y7zG48uHRo) - Brief introduction to controller design (PID) 45 | 6. [(F1Tenth) Lecture 4](https://f1tenth-coursekit.readthedocs.io/en/stable/lectures/ModuleA/lecture04.html) - Laplace Domain Dynamics & PID 46 | 7. [(F1Tenth) Lab 3](https://f1tenth-coursekit.readthedocs.io/en/stable/assignments/labs/lab3.html) - Wall Follower Controller Intro - Use a PID controller to race the car along the walls of the track! ([Brian Douglas on MATLAB](https://www.youtube.com/watch?v=wkfEZmsQqiA&list=PLn8PRpmsu08pQBgjxYFXSsODEF3Jqmm-y) has a useful intro to PIDs series, along with in-depth control systems topics on his own channel) 47 | 8. [(YouTube) AMZ Driverless presents Learning-Based Model Predictive Control](https://www.youtube.com/watch?v=bjlT-6KVQ7U) to overcome model inaccuracies! (quick 3 min research presentation) 48 | 49 | ### Reactive Racing Methods 50 | 51 | 1. [(F1Tenth) Lecture 6](https://f1tenth-coursekit.readthedocs.io/en/stable/lectures/ModuleB/lecture06.html) - Follow the Gap Method and variants (details, pros and cons of non-planning methods are introduced) 52 | 2. [(F1Tenth) Lab 4](https://f1tenth-coursekit.readthedocs.io/en/stable/assignments/labs/lab4.html) - Have a go at designing and tuning your own Follow the Gap method! 53 | 3. [(YouTube) Reactive racing success in head-to-head](https://www.youtube.com/watch?v=4uuvtkXtjwA) 54 | 55 | ### Mapping and Localization Intro 56 | 57 | 1. [(F1Tenth) Lecture 9 & 10](https://f1tenth-coursekit.readthedocs.io/en/stable/lectures/ModuleC/lecture09.html) - Details of the ideas, and some algorithms that can be used 58 | 2. [(F1Tenth) Lecture 11](https://f1tenth-coursekit.readthedocs.io/en/stable/lectures/ModuleC/lecture11.html) - Particle filter localization, and in-depth details of algorithm 59 | 3. [(F1Tenth) Lecture 12](https://f1tenth-coursekit.readthedocs.io/en/stable/lectures/ModuleC/lecture12.html) - Introduces modern SLAM using Google Cartographer as part of a mapping and localization system, key breakthroughs in efficiency, and guide on installing it 60 | 4. [(F1Tenth) Lab 5](https://f1tenth-coursekit.readthedocs.io/en/stable/assignments/labs/lab5.html) - Create your own Scan Matching system (group work is recommended from this lab onwards, provides a rigorous and effective way to learn about optimizing an algorithm like this, but challenging) 61 | 5. [(YouTube) AMZ Driverless on Redundant Perception and State Estimation](https://www.youtube.com/watch?v=ir_uqEYuT84) - How robust perception plays a key role in Localization 62 | 6. [(Book/Repo) Theory and implementation of Kalman/Bayesian Filters (useful for localization)](https://github.com/rlabbe/Kalman-and-Bayesian-Filters-in-Python) 63 | 64 | ### Planning Methods 65 | 66 | 1. [(F1Tenth) Lecture 13](https://f1tenth-coursekit.readthedocs.io/en/stable/lectures/ModuleD/lecture13.html) - Introduction to Pure Pursuit racing and waypoint following 67 | 2. [(F1Tenth) Lab 6](https://f1tenth-coursekit.readthedocs.io/en/stable/assignments/labs/lab6.html#) - Use Google Cartographer SLAM and Pure Pursuit planning to race your car! 68 | 3. [(F1Tenth) Lecture 14](https://f1tenth-coursekit.readthedocs.io/en/stable/lectures/ModuleD/lecture14.html) - Path Planning/Search methods, and RRT/RRT* for local planning 69 | 4. [(Tool) Uniform Grid Path Planner simulations by Lai Yan Kai](https://laiyankai.github.io/UniformPlanners/index.html) - Great for visualizing step-by-step calculations in any configuration for a few global planning algorithms 70 | 5. [(F1Tenth) Lab 7](https://f1tenth-coursekit.readthedocs.io/en/stable/assignments/labs/lab7.html) - Design your own motion planner system! Learn about different practical considerations of path planning, and RRT/RRT* 71 | 6. [(YouTube) MATLAB Intro to Autonomous Path Planning](https://www.youtube.com/watch?v=QR3U1dgc5RE) 72 | 7. [(F1Tenth) Lecture 20](https://f1tenth-coursekit.readthedocs.io/en/stable/lectures/ModuleF/lecture20.html) - Raceline Optimization (introduces methods for solving the quickest route for any track, or planning-based control problem) 73 | 74 | ### Control 75 | 76 | 1. [(YouTube) Intro to Fourier Transforms and Laplace Transforms](https://www.youtube.com/watch?v=1JnayXHhjlg&list=PLUMWjy5jgHK3jmgpXCQj3GRxM3u9BmO_v&index=1) - a baseline topic that you will see often in analyzing control problems, and a great, concise playlist for introducing the maths, and the purpose of these representations. More in-depth topics on the Brian Douglas channel 77 | 2. [(YouTube) Model Predictive Control (MPC) by MATLAB](https://www.youtube.com/watch?v=8U0xiOkDcmw&list=PLn8PRpmsu08ozoeoXgxPSBKLyd4YEHww8&index=1) - comprehensive introduction and guide on what MPC is about 78 | 3. [(F1Tenth) Lecture 21](https://f1tenth-coursekit.readthedocs.io/en/stable/lectures/ModuleF/lecture21.html) - MPC, alternative introduction, with further references for F1Tenth racing 79 | 80 | ### Vision 81 | 82 | 1. [(F1Tenth) Lecture 17](https://f1tenth-coursekit.readthedocs.io/en/stable/lectures/ModuleE/lecture17.html) - Introduction to vision model and pose estimation 83 | 2. [(F1Tenth) Lecture 18](https://f1tenth-coursekit.readthedocs.io/en/stable/lectures/ModuleE/lecture18.html) - Multi-view geometry, and YOLO object detection 84 | 3. [(YouTube) Stanford CS231n by Andrej Karpathy](https://www.youtube.com/watch?v=NfnWJUyUJYU&list=PLkt2uSq6rBVctENoVBg1TpCC7OQi31AlC&index=1) - rigorous coverage of computer vision, specifically Convolutional Neural Networks for Visual Recognition. Long, detailed lecture series. Pre-requisite to Deep Learning is [Machine Learning](http://cs229.stanford.edu/syllabus-autumn2018.html) 85 | 4. [PyTorch Tutorial](https://pytorch.org/tutorials/) - currently more common framework for Deep Learning researchers (also Lightning) 86 | 5. [Keras quick guide](https://keras.io/getting_started/intro_to_keras_for_researchers/) - built on TensorFlow framework, research work sources may also reference TF1.0, TF2.0, Keras (different API changes throughout TF versions, but should not be varying for core machine learning functionality, and similar to PyTorch in quite a few ways) 87 | 88 | ### Reinforcement Learning (RL) 89 | 90 | 1. [Introduction to Deep Learning](http://introtodeeplearning.com/) - Here's a link to the full MIT Deep Learning course. It's really easy to grasp and they have a bunch of interesting labs for you to work on. 91 | 2. [OpenAI Gym docs/tutorial](https://gym.openai.com/docs/) - This is the most widely used framework for RL experiment benchmarking 92 | 3. [Spinning Up by OpenAI](https://spinningup.openai.com/en/latest/) - Rigorous guide to RL and deep RL, useful for new researchers 93 | 4. [(YouTube) Solving Rubik's Cube with a single robot hand](https://www.youtube.com/watch?v=x4O8pojMF0w) - very challenging for even people to do one-handed 94 | 5. [(YouTube) David Silver's DeepMind intro to RL](https://www.youtube.com/watch?v=2pWv7GOvuf0&list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ&index=1) - lecture series that delivers holistic view on RL, very good for understanding but not the most practical for implementation of programs. 95 | 6. [Alternative RL course with Colab-ready notebooks](https://github.com/yandexdataschool/Practical_RL) 96 | 97 | -------------------------------------------------------------------------------- /ROS-glossary/README.md: -------------------------------------------------------------------------------- 1 | # ROS in a nutshell 2 | -------------------------------------------------------------------------------- /docker-setup/Dockerfile: -------------------------------------------------------------------------------- 1 | # pull container from https://hub.docker.com/_/ros, choosing the melodic-robot-bionic version 2 | FROM ros:melodic-robot-bionic 3 | 4 | # install dependencies for the F1Tenth simulator and wget 5 | RUN sudo apt-get update 6 | RUN sudo apt-get -y install ros-melodic-ackermann-msgs ros-melodic-tf2-geometry-msgs ros-melodic-interactive-markers ros-melodic-cv-bridge ros-melodic-image-transport ros-melodic-rviz ros-melodic-joy ros-melodic-map-server wget vim 7 | # switch to bash for running commands 8 | RUN rm /bin/sh && ln -s /bin/bash /bin/sh 9 | # initialise ROS, pull useful scripts from our repo and clone the F1Tenth repo into a new catkin workspace 10 | RUN source /ros_entrypoint.sh && \ 11 | source /opt/ros/melodic/setup.bash && \ 12 | wget https://raw.githubusercontent.com/FT-Autonomous/Autonomous_Crash_Course/main/docker-setup/run-simulator.sh && \ 13 | wget https://raw.githubusercontent.com/FT-Autonomous/Autonomous_Crash_Course/main/docker-setup/set-display.sh && \ 14 | wget https://raw.githubusercontent.com/FT-Autonomous/Autonomous_Crash_Course/main/docker-setup/ros-init.sh && \ 15 | wget https://raw.githubusercontent.com/FT-Autonomous/Autonomous_Crash_Course/main/docker-setup/maps.txt && \ 16 | mkdir -p /catkin_ws/src && \ 17 | cd catkin_ws/src && \ 18 | git clone https://github.com/f1tenth/f110_ros.git 19 | -------------------------------------------------------------------------------- /docker-setup/README.md: -------------------------------------------------------------------------------- 1 | ## Using ROS with Docker Guide 2 | 3 | ### Setting up the Docker Container 4 | 5 | 1. Clone this repository 6 | 7 | ``` 8 | git clone https://github.com/FT-Autonomous/Autonomous_Crash_Course.git 9 | ``` 10 | 11 | 2. Navigate to the docker-setup folder 12 | 13 | ``` 14 | cd Autonomous_Crash_Course/docker-setup 15 | ``` 16 | 17 | 3. Build the container 18 | 19 | The build might take a while to load the first time you run it. 20 | 21 | ``` 22 | docker build -t docker-ros . 23 | ``` 24 | 25 | 4. Run the container 26 | 27 | ``` 28 | docker run -it docker-ros 29 | ``` 30 | 31 | ### Running the F1Tenth Simulator 32 | 33 | When inside the running container, execute the run-simulator script. If you don't have display access set up yet it will say the Rviz process has died which is just the display. The simulator is still running. See below to set up display access. 34 | 35 | ``` 36 | source /run-simulator.sh 37 | ``` 38 | 39 | ### Enabling the Container to display GUIs on the host machine (Windows) 40 | 41 | Prerequisite: Chocolatey must be installed on your computer. 42 | 43 | Warning: Some firewalls may block access so it might be necessary to disable your firewall when running the simulator. 44 | 45 | Open Windows Powershell as Administrator and install the VcXsrv package 46 | 47 | ``` 48 | choco install xming 49 | ``` 50 | 51 | Open the XLaunch Application 52 | 53 | 54 | 55 | Follow the default settings for each step except the following 56 | 57 | 58 | 59 | Once the XLaunch setup is finished, find your local IP Address by entering an ipconfig command in your local terminal 60 | 61 | ``` 62 | ipconfig 63 | ``` 64 | 65 | Your local IP should be found similarly to where its shown below (ignore the white marks) 66 | 67 | 68 | 69 | Execute the setup-display script in the docker container passing your local IP Address as a parameter (ignore brackets below). 70 | 71 | ``` 72 | source /set-display.sh [Local IP Address] 73 | ``` 74 | 75 | Now when you run the simulator you should see visuals open in a new window. 76 | 77 | ### Enabling the Container to display GUIs on the host machine (Ubuntu, and other) 78 | 79 | First, install [Rocker](https://github.com/osrf/rocker), then after you have built your image (step 3), run the following command in your Terminal to run the Docker image: 80 | 81 | ``` 82 | rocker [optional: --nvidia or --devices /dev/dri/card0] --x11 docker-ros 83 | ``` 84 | -------------------------------------------------------------------------------- /docker-setup/maps.txt: -------------------------------------------------------------------------------- 1 | levine (default) 2 | levinelobby 3 | columbia 4 | mtl 5 | porto 6 | stata_basement 7 | torino 8 | -------------------------------------------------------------------------------- /docker-setup/media/ipconfig.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/FT-Autonomous/Autonomous_Crash_Course/cf6536d7986300b16977dc4579b133ed5f2991f3/docker-setup/media/ipconfig.jpg -------------------------------------------------------------------------------- /docker-setup/media/xlaunch-1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/FT-Autonomous/Autonomous_Crash_Course/cf6536d7986300b16977dc4579b133ed5f2991f3/docker-setup/media/xlaunch-1.jpg -------------------------------------------------------------------------------- /docker-setup/media/xlaunch-2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/FT-Autonomous/Autonomous_Crash_Course/cf6536d7986300b16977dc4579b133ed5f2991f3/docker-setup/media/xlaunch-2.jpg -------------------------------------------------------------------------------- /docker-setup/ros-init.sh: -------------------------------------------------------------------------------- 1 | source "/opt/ros/$ROS_DISTRO/setup.bash" 2 | -------------------------------------------------------------------------------- /docker-setup/run-simulator.sh: -------------------------------------------------------------------------------- 1 | cd /catkin_ws 2 | catkin_make 3 | source devel/setup.bash 4 | if [ $1 ] 5 | then 6 | roslaunch f110_simulator simulator.launch map:=/catkin_ws/src/f110_ros/f110_simulator/maps/$1.yaml 7 | else 8 | roslaunch f110_simulator simulator.launch 9 | fi 10 | -------------------------------------------------------------------------------- /docker-setup/set-display.sh: -------------------------------------------------------------------------------- 1 | export DISPLAY=$1:0.0 2 | --------------------------------------------------------------------------------