├── LICENSE ├── README.md ├── courses.md └── mathematical_rigor.md /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 PakistanAI 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # AI/ML Educational Guide 2 | An educational guide on how to get started in Machine Learning (ML) and Artificial Intelligence (AI) as documented by the [Pakistan.AI](https://www.facebook.com/PakistandotAI) group. Many such guides already exist on the Internet. However, the circumstances of Pakistan are different and hence require a customised guide. This guide assumes that readers have some exposure to programming and understand high school level math (Intermediate/Matriculation level Math across all path divisions). However, if you have no previous programming experience it is recommended that you go through this [Medium blog post](https://medium.com/@amandalmia18/guide-for-deep-learning-aspirants-with-focus-on-non-computer-science-students-87b1f7b3f4b9). At the very least you must do one programming course to continue on to the courses mentioned. We recommend Harvard's [CS50](https://www.edx.org/course/cs50s-introduction-computer-science-harvardx-cs50x). 3 | 4 | | Guide Contents | 5 | | ------------- | 6 | | [Courses](/courses.md) | 7 | | [A note on Mathematical Rigor](/mathematical_rigor.md) | 8 | 9 | ## Contributions 10 | This page is completeley maintained by volunteers! If you feel that this guide must include more information then simply start an issue, create a pull request or email at anasayubi7152@gmail.com (in case you are unfamiliar with github). 11 | -------------------------------------------------------------------------------- /courses.md: -------------------------------------------------------------------------------- 1 | # Courses 2 | 3 | | Contents | 4 | | ------------- | 5 | | [Artificial Intelligence](#general-artificial-intelligence) | 6 | | [Machine Learning](#machine-learning) | 7 | | [Statistics](#statistics) | 8 | | [Mathematics](#mathematics) | 9 | | [Deep Learning](#deep-learning) | 10 | 11 | ## Artificial Intelligence 12 | These courses present a holistic view of Artificial Intelligence as a whole. The courses mentioned here give an idea as to what AI is all about. 13 | 14 | #### [Artificial Intelligence Planning on Open.Ed by Austin Tate and Gerhard Wickler](https://open.ed.ac.uk/artificial-intelligence-planning/) 15 | Acces: Free 16 | Focuses on planning in AI and provides an overview of various planning techniques. Planning in AI involves modeling the real world including agents and actions, reasoning about the effects of actions and algorithms for efficiently finding a suitable sequence of actions (plan) to achieve a task. 17 | 18 | Advantages: 19 | - Great basic course that covers many planning problems and approaches 20 | - Short 21 | 22 | Disadvantages: 23 | - Not very intensive & lacks programming exercises 24 | 25 | #### [Intro to Artificial Intelligence on Udacity by Sebastian Thrun and Peter Norvig](https://eu.udacity.com/course/intro-to-artificial-intelligence--cs271) 26 | Access: Free 27 | This course provides an overview of many fields in AI including representation and planning, computer vision, robotics. 28 | 29 | Advantages: 30 | - Very intuitive and gentle introduction to each field with quick quizzes 31 | - Midterm and final assessments are present 32 | 33 | Disadvantages: 34 | - Old, not well maintained course. Some files necessary for completion are missing 35 | 36 | ## Machine Learning 37 | Most of the greatest feats of AI today (Speech recognition, [Google Search Engine](https://www.wired.com/2016/02/ai-is-changing-the-technology-behind-google-searches/) and etc) are possible due to a subdomain in AI called Machine Learning. This specific domain focuses on machines teaching themselves how to do a certain task (as defined by the programmer) based on input data i.e. the machine learns from the data hence the name "Machine Learning". Due to the mathematical nature of ML this guide involves mathematical courses as well. It is strongly recommended that students complete them for a better understanding of the subject. 38 | 39 | 40 | #### [Machine Learning on Coursera by Andrew Ng](https://www.coursera.org/learn/machine-learning) (highly recommended) 41 | Access: Audit / Financial aid (2-3 weeks approval time) / Payment 42 | This course is considered the best introduction to machine learning by the entire global community for its simplicity and concept-based learning. [Course notes](https://drive.google.com/open?id=1lmx-soWtL38ko7Oc3cz8ve2x6WIdT2es) contributed by [Anas Ayubi](https://www.linkedin.com/in/anasayubi/) and scanned by [Umair Hanif](https://www.linkedin.com/in/umairhanif00/) 43 | 44 | Advantages: 45 | - Andrew Ng dives into the algorithms used in machine learning by keeping mathematical involvement to a minimum whilst delivering all the intuition behind the mathematics used 46 | - Students are expected to code the machine learning algorithms themselves 47 | - Andrew Ng delivers practical advice on how to approach and solve machine learning problems throughout the course 48 | 49 | Disadvantages: 50 | - Delivering mathematical intuition becomes difficult due to the increased mathematical complexity in certain algorithms 51 | - Octave/MATLAB is used for programming assignments in the course (less recognised programming language within the machine learning community) 52 | 53 | #### [Machine Learning on Coursera by Carlos Guestrin and Emily Fox](https://www.coursera.org/learn/ml-foundations) 54 | Access: Audit / Financial aid (2-3 weeks approval time) / Payment 55 | This course can be considered as a less intensive and shorter substitute for [Andrew Ng's machine learning course](#andrew-ng-coursera). 56 | 57 | Advantages: 58 | - Algorithms are explained but students are not required to code them themselves - focus is mainly on learning how to use libraries 59 | - Python, a well used language in the data science community, is used throughout the assignments 60 | 61 | Disadvantages: 62 | - No practical advice on how to approach and solve machine learning problems is given throughout the course 63 | 64 | #### [Introduction to Statistical Learning and its applications in R (book)](https://www-bcf.usc.edu/~gareth/ISL/) 65 | Access: Free 66 | From the authors of what is considered by many as the mother of all books on machine learning 'Elements of Statistical Learning'. This book aims to be a simpler introduction to most of the contents of the aforementioned book, without much of the mathematical detail, along with how to implement the techniques taught in the R language. An absolute must-have for everyone wanting to get a solid grip on Machine Learning through the model-based, and also algorithmic approaches with labs and tons of exercises for practice. 67 | 68 | Advantages: 69 | - In-depth (compared to most of the famous courses out there) 70 | - Lots of exercises 71 | - [Video lectures](https://www.r-bloggers.com/in-depth-introduction-to-machine-learning-in-15-hours-of-expert-videos/) following the book by the authors themselves 72 | 73 | Disadvantages: 74 | - Rigorous 75 | 76 | ## Statistics 77 | 78 | #### [OpenIntro Statistics](https://www.openintro.org/) 79 | Access: Free 80 | A very down to earth introduction to Statistics and probability with an intuitive approach and real world examples. The book is accompanied by video lectures, most of which are by the author herself, Dr Mine Cetinkaya Rundell, and the site also has labs in R, which make it one of the most practically beneficial, and accessible courses on Statistics that we have come across. 81 | 82 | Advantages: 83 | - Good pedagogy, and no assumption of prior knowledge, yet pragmatic examples. 84 | - Plenty of exercises to help home in the concepts 85 | 86 | Disadvantages: 87 | - Although the book covers all topics usually taught in Statistics, but the treatment is still not too in-depth. One would need to move on to more advanced course or books on the subject 88 | 89 | #### [Statistics with R (specialization)](https://www.coursera.org/specializations/statistics) 90 | Access: Audit / Financial aid (2-3 weeks approval time for each individual course) / Payment 91 | Closely following the book mentioned above, this course adds to the above, the value of teaching you how to practice what you learn in R. The videos are mostly the same, also by Dr Mine. 92 | 93 | #### [Probability at Harvard by Joe Blitzstein](https://projects.iq.harvard.edu/stat110/home) 94 | Access: Free 95 | This course delivers an entire undergraduate experience of studying at Harvard and introduces a rigorous and intuition-based introduction of probability and how it is applied. 96 | 97 | Advantages: 98 | - Great insight is given into how probability exists in the real world and how it must be correctly applied to real life situations 99 | - Exercises are provided 100 | 101 | Disadvantages: 102 | - Course is considered challenging by most students 103 | - Difficult to follow the course if exercises are not attempted by students 104 | 105 | #### [Statistics and Probability by Khan Academy](https://www.khanacademy.org/math/statistics-probability) 106 | Access: Free 107 | For those who find [Probability at Harvard by Joe Blitzstein](#probability-at-harvard-by-joe-blitzstein) challenging Khan Academy's courses serve as a less rigorous alternative with an increased focus on intuition. 108 | 109 | Advantages: 110 | - Highly conceptual videos with focus on intuition 111 | - Exercises are provided 112 | 113 | Disadvantages: 114 | - Exercises are oversimplified and lack in real world applications - course does not deliver the skills required to apply concepts to real world applications 115 | 116 | ## Mathematics 117 | Although Andrew Ng, Carlos and Emily intentionally attempt to cover up the mathematics in order to not overwhelm beginners, students must not shy away from attempting to understand these topics in order to deepen their understanding regarding the subject. As the method of teaching in mathematics within Pakistan requires great improvement, **it is strongly recommended that the following courses are taken by all participants**. One more thing to note - our focus within mathematics courses is mainly towards rigorous university courses rather than "not so rigorous" MOOC courses. This is due to the fact MOOCs tend to simplify mathematical content which leads to limited applicability of that content to real life situations. We at [Pakistan.ai](https://www.facebook.com/PakistandotAI) can't emphasis how important it is to pick the correct mathematical courses hence **we dedicated an entire file to this topic**. Read more on this [here](/mathematical_rigor.md). 118 | 119 | #### [Linear Algebra at MIT by Gilbert Strang](https://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebra-fall-2011/index.htm) 120 | Access: Free 121 | This course delivers an entire undergraduate experience of studying at MIT and introduces a rigorous and intuition-based introduction of linear algebra and how it is applied. 122 | 123 | Advantages: 124 | - Great insight is given into how linear algebra is relevant and how it must be correctly applied to real life situations 125 | - Recitation videos are present - these videos walk you through a solution to a specific problem 126 | - Exercises are provided 127 | 128 | Disadvantages: 129 | - Course is considered challenging by most students 130 | - Difficult to follow the course if exercises are not attempted by students 131 | 132 | #### [Linear Algebra by Khan Academy](https://www.khanacademy.org/math/linear-algebra) 133 | Access: Free 134 | For those who find [Linear Algebra at MIT by Gilbert Strang](#linear-algebra-at-mit-by-gilbert-strang) challenging Khan Academy's courses serve as a less rigorous alternative with an increased focus on intuition. 135 | 136 | Advantages: 137 | - Highly conceptual videos with focus on intuition 138 | - Exercises are provided 139 | 140 | Disadvantages: 141 | - Exercises are oversimplified and lack in real world applications - course does not deliver the skills required to apply concepts to real world applications 142 | 143 | #### [Essence of linear algebra by 3Blue1Brown](https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab) 144 | Access: Free 145 | This Youtube playlist discusses the intuition of linear algebra from different perspectives and builds up understanding. It acts as great supplementary content to a course that a student is already currently doing in linear algebra. 146 | 147 | ## Deep Learning 148 | Deep Learning (DL) is recongnised as a subdomain of ML and a wide majority of all modern applications of AI lie within this specific domain. DL can be considered as an extension of Artificial Neural Networks (ANNs) where the architecture of the neural network is tweaked. The greatest benefit of DL is that it eliminates the need of feature engineering as required in ML with the unfortunate drawback that increased computing power and more data is required. Most of today's computer vision (CV) and natural language processing (NLP) is based entirely on DL. An understanding of ML is highly recommended before attempting DL courses. 149 | 150 | #### [Deep Learning Specialization on Coursera by Andrew Ng](https://www.coursera.org/specializations/deep-learning) 151 | Access: Audit / Financial aid (2-3 weeks approval time) / Payment 152 | This specialisation (a sequence of many courses) is considered as a logical extension from [Andrew Ng's machine learning course](#andrew-ng-coursera) and is recognised as the best introduction to deep learning by the entire global community for its simplicity and concept-based learning. This specialisation contains 5 courses. 153 | Note: You will have to apply for financial aid for each individual course 154 | 155 | Advantages: 156 | - Andrew Ng dives into the algorithms used in machine learning by keeping mathematical involvement to a minimum whilst delivering all the intuition behind the mathematics used 157 | - Students are expected to code the machine learning algorithms themselves 158 | - Andrew Ng delivers practical advice on how to approach and solve machine learning problems throughout the course 159 | - Python is used for programming assignments in the course (widely recognised programming language within the deep learning community) 160 | - The most relevant python libraries (within the deep learning community) are used in assignments 161 | - Computational power is provided to the participants of the course 162 | 163 | Disadvantages: 164 | - Delivering mathematical intuition becomes difficult due to the increased mathematical complexity in certain algorithms 165 | 166 | #### [Deep Learning with python by Francois Chollet (book)](https://www.amazon.com/Deep-Learning-Python-Francois-Chollet/dp/1617294438/ref=sr_1_1?ie=UTF8&qid=1530012174&sr=8-1&keywords=deep+learning+with+python) 167 | Access: Paid 168 | By the creator of the infamous keras library himself, this book is beyond doubt one of the best resources to dive into practical implementation of deep neural networks without the complexities, getting into which is not needed for the majority of practitioners. Succint, and brilliantly explained, this book is a must-have for all eyeing to get up and running with deep learning without falling into depths they cannot manage. *attempt at pun 169 | -------------------------------------------------------------------------------- /mathematical_rigor.md: -------------------------------------------------------------------------------- 1 | # Importance of Mathematical rigor in Machine Learning 2 | 3 | Within machine learning and all the fields related to it (data science, computer vision, NLP and etc) people usually come across 2 types of problems: 4 | 1) Problems where ready-made algorithms work 5 | 2) Problems where ready-made algorithms don't work 6 | Most companies that work in the ML domain work on problems of the second type. If this were not true then companies providing ML based services would not exist in the first place because then any developer could simply run a couple of functions (pfft... how difficult is it to copy a couple of functions? Just use Stack Overflow!) and get the required work done. So if you wanted to work at such a company, you would be required to implement algorithms on your own. At the bare minimum that requires a knowledge of statistics, linear algebra and calculus. If you have no idea about statistics chances are you won't even understand what's happening (not to mention actually implementing the actual formula) and if you have no knowledge of linear algebra chances are you'll write some code that will take days to execute as opposed to hours (linear algebra libraries allow you to write extremely efficient code that has the capacity to maximise the utiliztion of your processor along with any GPUs) 7 | 8 | > "No worries! I'll just work on the first type of problems - where I can possibly use ready-made solutions!" 9 | 10 | You've been presented with a problem that can be solved via ready-made algorithms. But hold on, should you use linear regression or neural nets? But there are so many type of neural nets, which one should you use? Technically speaking anomaly detection does the same thing as linear regression, so which one should you use? If you do choose anomaly detection with gaussian distribution then should you use the single-variate version or the multi-variate version? Is there any difference between the two? Oh, single variate assumes independence between variables. Hold on... what's independence? I also have all this data, how do I characterise it to fit the best possible machine learning model? You do that with distributions? What on earth is a distrubution?! 11 | All these questions are answered by having mathematical intuition of the algorithm. Guess how you get to understand that mathematical intuition!? 12 | 13 | > "Okay, thanks for killing my optimism off. I'll just take a Coursera course I guess" 14 | 15 | Stop! I guess you now understand that it's important for you to be able to apply what you learn or you're basically wasting your time! Most math courses on sites such as Coursera, Edx and etc. oversimplify the content and do not deliver the concepts required to apply your learning to real life situations. You're making the exact same mistake most Pakistani educational systems do by not learning to apply. Would you not want to know what the symbols mean in these mathematical courses? How to manipulate them correctly? Or what the intuition behind those symbols is? Hence we at [Pakistan.ai](https://www.facebook.com/PakistandotAI) started a guide to combat this issue by adding courses that are rigorous enough to teach you the concepts right. Find out more about these courses [here](/README.md#mathematics). 16 | 17 | > "Okay, I understand what you mean. How about I drop everything and go do a PhD in math now -_-" 18 | 19 | Machine learning does not require a PhD in mathematics, neither a masters and nor even a bachlors. But what it does require is that you understand the basic fundamentals of Probability and Statistics, Linear Algebra, and Calculus. Calculus is taught rather rigorously in Pakistan across all boards so this tends to be less of an issue. However Linear Algebra, and Probability and Statistics is not. 20 | Furthermore, there's another plus point to doing rigorous mathematical courses. Data Science and Machine learning are rather new fields (even abroad) that most employers think twice before hiring someone (especially in Pakistan) - it's difficult to tell whether a new employee within this domain can actually deliver value. However, individuals who are sharp and bright have a greater chance of getting over this barrier. Reason: employers know that such candidates will be capable of picking up things quickly and they will be able to make the correct critical decisions as to what needs to be done. Question is how does one become like that? 21 | **One reaches that potential by challenging himself/herself again and again and again.** You must go through the fire and then only you will come out as shimmering gold. Those difficult maths courses are the challenges you can take to show your client, or your employer or your investor that you are capable of taking high-tier courses, that you have the intellectual capability of overcoming intellectual challenges. Some might say that such gifts (picking things up quickly, having extraordinary critical thinking skills) are genetic. They are not. All modern psychological research points in this direction. 22 | 23 | > "I give up. You've convinced me." 24 | 25 | I'm glad I have. 26 | --------------------------------------------------------------------------------