11 | Reviewers: [Liu Yiwen](https://github.com/0blivious), [Tejas Bhuwania](https://github.com/Tejas2805) 12 | 13 |
7 |
8 | # Introduction to SQL
9 |
10 | **Author(s): [Amrut Prabhu](https://github.com/amrut-prabhu)**
11 |
12 | Reviewers: [Ronak Lakhotia](https://github.com/RonakLakhotia), [Rahul Rajesh](https://github.com/rrtheonlyone)
13 |
14 |
15 |
16 | * [What is SQL?](#what-is-sql)
17 | * [How Does SQL Work?](#how-does-sql-work)
18 | * [Why Learn SQL?](#why-learn-sql)
19 | * [Disadvantage](#disadvantage)
20 | * [How to Get Started With SQL?](#how-to-get-started-with-sql)
21 |
22 |
23 | ## What is SQL?
24 |
25 | A Relational Database Management System (RDBMS) is a popular database solution used by software applications to manage their data.
26 |
27 | **Structured Query Language** (**SQL**) is a programming language that is specifically designed for interacting with an RDBMS. Unlike other languages, SQL doesn't come as a standalone installation.
28 | Rather, RDBMSs %%like MySQL, Oracle, SQL Server and PostgreSQL%% come with an _implementation_ of SQL.
29 |
30 | ---
31 |
32 | ### How Does SQL Work?
33 |
34 | SQL uses **queries** to retrieve data. Here is an example of how an SQL query is used.
35 |
36 | Suppose we have the data table `Students` shown here:
37 |
38 | | ID | Name | Course | Faculty |
39 | | -- | ----- | ------ | ------- |
40 | | 1 | Alex | CS202 | CS |
41 | | 2 | Bob | MA303 | MA |
42 | | 3 | Cathy | CS202 | CS |
43 | | 4 | Daren | CS202 | CS |
44 | | 5 | Ellie | CS101 | CS |
45 | | 6 | Fred | MA303 | MA |
46 | | 7 | Gary | CS101 | CS |
47 | | 8 | Henry | CS404 | CS |
48 |
49 | We can use this SQL query to retrieve information from this table:
50 | ```sql
51 | SELECT Course, COUNT(*) num
52 | FROM Students
53 | WHERE Faculty = 'CS'
54 | GROUP BY Course
55 | HAVING COUNT(*) > 1
56 | ORDER BY num;
57 | ```
58 |
59 | This query first filters the entries in the `Students` table such that only entries that have `CS` as the faculty are considered.
60 | After that, it groups those entries into the 3 courses: `CS101`, `CS202` and `CS404`.
61 | Then, it removes courses that do not have more than 1 student, i.e., `CS404` is removed from consideration.
62 | Finally, it returns a list of courses with a count of the number of students, ordered in increasing order.
63 | So, the output of the query is:
64 |
65 | | Course | num |
66 | | ------ | --- |
67 | | CS101 | 2 |
68 | | CS202 | 3 |
69 |
70 | You can see how this simple query can prove to be extremely useful for getting this information when the table has a large number of rows. Queries can also be used to retrieve data that span across multiple tables. Apart from retrieving information, SQL can be used for creating, deleting and manipulating data with commands like `INSERT`, `DELETE` and `UPDATE` for entries, in addition to `CREATE`, `DROP` and `ALTER` for tables as a whole.
71 |
72 | For example, you can set up the structure of the table shown in Table 1 by executing the `CREATE TABLE` query shown here.
73 | It creates a new data table `Students`, with 4 fields (`ID`, `Name`, `Course` and `Faculty`) and specifies their data types.
74 |
75 | ```sql
76 | CREATE TABLE Students(ID int, Name varchar(255), Course varchar(255), Faculty varchar(255));
77 | ```
78 |
79 | We can also remove rows containing the `MA` faculty from Table 1 with this `DELETE` query.
80 |
81 | ```sql
82 | DELETE
83 | FROM Students
84 | WHERE Faculty='MA';
85 | ```
86 |
87 |
88 | You can experiment with this example on DB Fiddle (shown in Figure 1 below) by entering SQL queries and then running them.
89 |
90 |
91 |
92 | 
93 |
94 | _Figure 1. Using DB Fiddle to run queries in MySQL_
95 |
96 |
97 | **Online editors** like DB Fiddle can be used while learning SQL. However, to use SQL in application development, you need to consider other alternatives for running SQL queries:
98 |
99 | - **RDBMS software**: RDBMS installations usually come with a GUI (see Figure 2), but can also be accessed from the command line (see Figure 3).
100 |
101 |
102 | 
103 |
104 | _Figure 2. MySQL (an RDBMS) can be used in the MySQL Workbench GUI application_ ([source](https://www.mysql.com/products/workbench/))
105 |
106 | 
107 |
108 | _Figure 3. Logging in to the MySQL RDBMS and viewing tables from the command line_ ([source](https://www.researchgate.net/figure/The-MySQL-command-line-tool_fig5_328093393))
109 |
110 |
111 | - **APIs**: You can use APIs in programming languages or independent ones like [SQL API](https://carto.com/developers/sql-api/) (shown in Figure 4) to interact with databases. For example, the [SQLite3](https://www.pythoncentral.io/introduction-to-sqlite-in-python/) library can be used to interact with a database through Python.
112 |
113 |
114 | 
115 |
116 | _Figure 4. An example of how to use CARTO's SQL API, which uses a PostgreSQL database_ ([source](https://carto.com/developers/sql-api/))
117 |
118 |
119 | ---
120 |
121 | ## Why Learn SQL?
122 |
123 | The main reason why you should learn SQL is because of its widespread adoption. As SQL is not limited to any particular operating system or programming language, RDBMSs (which use SQL) are **widely used** by a lot of software applications %%(web, mobile etc.)%%. Hence, in order to utilize almost any relational database or RDBMS available today, you need to know the SQL language.
124 |
125 | ## Disadvantage
126 |
127 | The biggest issue in SQL is with regards to the syntax and features.
128 | Although SQL databases use established ANSI & ISO standards, some RDBMSs %%(like PostgreSQL, for example)%% add proprietary extensions to standard SQL.
129 | Due to this, the available feature set can vary according to what you're using.
130 | Furthermore, there may be minor changes in the syntaxes across RDBMSs.
131 | These factors can make SQL confusing and frustrating to use when switching across RDBMSs, since you would have to change your SQL queries.
132 |
133 | ---
134 |
135 | ## How to Get Started With SQL?
136 |
137 | As you saw in the example in the earlier section, SQL is really not that complex. It is easy to learn, even for beginners who do not have any prior experience with databases.
138 |
139 | To get started with SQL, you can either just use an online SQL playground or install an RDBMS. There are many free RDBMSs available like [PostgreSQL](https://www.postgresql.org/) and [MySQL](https://dev.mysql.com/downloads/).
140 | At the early stages of learning SQL, you can even use online editors like [DB Fiddle](https://www.db-fiddle.com/) %%(supports MySQL and SQLite)%% or [SqliteOnline](https://sqliteonline.com/) %%(supports SQLite)%% since they allow you to start learning without the hassle of setting up anything.
141 |
142 | Here are some recommended steps for learning SQL:
143 |
144 | 1. **Understand RDBMS**: Before jumping into the SQL language, refer to this book's [**DBMS** ]({{baseUrl}}/contents/data/databases/databases.html) chapter to understand the basic database concepts. More specifically, understand the basic concepts of RDBMS by going through Tutorials Point's [**SQL- RDBMS Concepts**](https://www.tutorialspoint.com/sql/sql-rdbms-concepts.htm) page. This is not _essential_, but will give you a better high-level understanding before diving into programming.
145 |
146 | 1. **Learn SQL Syntax**: These are some recommended resources that you can use to learn the SQL language.
147 | - The [**Guru99 SQL Tutorial for Beginners**](https://www.guru99.com/sql.html) is a good course for beginners. It is well organized and provides comprehensive information, which will ensure that you know the basics well.
148 |
149 | - The [**W3Schools SQL Tutorial**](https://www.w3schools.com/sql/default.asp) is a useful, interactive written tutorial in which you can run and modify SQL queries to see the examples in action.
150 |
151 | - In order to keep track of and remember the syntaxes that you're learning, you can refer to the [**W3Schools SQL Quick Reference Guide**](https://www.w3schools.com/sql/sql_ref_mysql.asp). This is particularly useful when you want to look up specific information quickly.
152 |
153 | 1. **Use SQL in practice**: Look into different RDBMSs and decide which one you want to work with. You can start by comparing the 3 mentioned in this [**Oracle vs MySQL vs SQL Server**](https://blog.udemy.com/oracle-vs-mysql-vs-sql-server/) post and work from there. Then, set up the corresponding RDBMS and integrate it with an application. This will give you good experience and exposure to how databases are used in practice.
154 |
155 | In the case of MySQL, a good resource for getting started with this is [**MySQLTutorial.org**](https://www.mysqltutorial.org/). The `Interfaces` section of the website contains multiple tutorials explaining how to integrate and work with MySQL from [Node.js](https://www.mysqltutorial.org/mysql-nodejs/), [Java](https://www.mysqltutorial.org/mysql-jdbc-tutorial/), [Python](https://www.mysqltutorial.org/python-mysql/) and more.
156 |
--------------------------------------------------------------------------------
/contents/data/sql/SQL_API.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/data/sql/SQL_API.png
--------------------------------------------------------------------------------
/contents/devops/DevOps.md:
--------------------------------------------------------------------------------
1 |
2 | title: DevOps
3 | pageNav: 3
4 |
5 |
6 |
2 | title: An Introduction to GPGPU
3 | pageNav: 3
4 |
5 |
6 |
2 | title: Introduction to iOS App Development
3 | pageNav: 3
4 |
5 |
6 |
2 | title: Java Synchronization
3 | pageNav: 3
4 |
5 |
6 |
2 | title: Libraries
3 | pageNav: 3
4 |
5 |
6 |
2 | title: Null Safety in Kotlin
3 | pageNav: 3
4 |
5 |
6 |
2 | title: Introduction to Kotlin
3 | pageNav: 3
4 |
5 |
6 |
2 | title: Performance Profiling
3 | pageNav: 3
4 |
5 |
6 |
2 | title: Profiling a Desktop Application In Visual Studio 2015
3 | pageNav: 3
4 |
5 |
6 |
2 | title: Web Performance
3 | pageNav: 3
4 |
5 |
6 |
2 | title: Workflow Automation with GitHub Actions
3 | pageNav: 3
4 |
5 |
6 |
93 |
94 | _Figure 1. Using DB Fiddle to run queries in MySQL_
95 |
103 |
104 | _Figure 2. MySQL (an RDBMS) can be used in the MySQL Workbench GUI application_ ([source](https://www.mysql.com/products/workbench/))
105 |
106 |
107 |
108 | _Figure 3. Logging in to the MySQL RDBMS and viewing tables from the command line_ ([source](https://www.researchgate.net/figure/The-MySQL-command-line-tool_fig5_328093393))
109 |
115 |
116 | _Figure 4. An example of how to use CARTO's SQL API, which uses a PostgreSQL database_ ([source](https://carto.com/developers/sql-api/))
117 |
7 |
8 | # DevOps
9 |
10 | Authors: [John Yong](https://github.com/whipermr5)
11 |
12 |
13 |
14 | * [What is DevOps?](#what-is-devops)
15 | * [Aspects of DevOps](#aspects-of-devops)
16 | * [Code Development & Review](#code-development-and-amp-review)
17 | * [Build & Test Automation](#build-and-amp-test-automation)
18 | * [Deployment & Infrastructure](#deployment-and-amp-infrastructure)
19 | * [Monitoring](#monitoring)
20 | * [Getting Started](#getting-started)
21 | * [Useful Links](#useful-links)
22 | * [Quick References](#quick-references)
23 | * [Further Reading](#further-reading)
24 |
25 |
26 | ## What is DevOps?
27 |
28 | > “DevOps is the practice of operations and development engineers participating together in the entire service lifecycle, from design through the development process to production support.” — [the agile admin](https://theagileadmin.com/what-is-devops/)
29 |
30 | The term **DevOps** comes from the amalgamation of two words - **Dev**elopment and **Op**eration**s**. While there are many definitions and opinions out there about what DevOps really is, and no one true answer, most agree that it is the idea of Developers and Operations working together to achieve a goal ([source](https://blog.xebialabs.com/2015/04/23/best-guide-to-getting-started-in-devops/)). This involves a **shift in mindset** away from the traditional view of Development and Operations as two distinct teams with few overlapping responsibilities, to a new mindset where developers and operations engineers communicate, collaborate and share the responsibility of developing and delivering software.
31 |
32 | Indeed, some see DevOps as a culture shift that involves more than just developers and operations teams, but also everyone involved in delivering value to end users, including testers and Quality Assurance, Project Management and Human Resources teams ([source](https://blog.nwcadence.com/devops-culture-not-team/)). However, this document will focus more on the development and operations side of DevOps, which is what budding software engineers like yourself need to know.
33 |
34 | > Tip: for a comprehensive discussion of what DevOps is and is not, refer to *the agile admin*'s article, [What Is DevOps?](https://theagileadmin.com/what-is-devops/)
35 |
36 | ## Aspects of DevOps
37 |
38 | While DevOps is predominantly an organisational culture and mindset, there are several aspects it covers that you, as a software engineering student, need to be familiar with before entering the industry. These are outlined in the DevOps toolchain ([source](https://en.wikipedia.org/wiki/DevOps#DevOps_toolchain)):
39 |
40 | 1. Code
41 | 2. Build
42 | 3. Test
43 | 4. Package
44 | 5. Release
45 | 6. Configure
46 | 7. Monitor
47 |
48 | These can be roughly summarised into four major aspects:
49 |
50 | ### Code Development & Review
51 |
52 | This includes using version control tools, following a process when developing, frequently merging code with a central branch, and going through a code review process.
53 |
54 | #### Relevant Tools
55 |
56 | Version Control Software (VCS)
57 |
58 | - [Git](https://git-scm.com/)
59 | - [Mercurial](https://www.mercurial-scm.org/)
60 |
61 | Web-based Hosted VCS
62 |
63 | - [GitHub](https://github.com/)
64 | - [GitLab](https://about.gitlab.com/)
65 | - [BitBucket](https://bitbucket.org/)
66 |
67 | ### Build & Test Automation
68 |
69 | Involves using continuous integration tools that automatically build and test your software upon every push to a code repository (Git, Mercurial, etc.), so you don't have to do it yourself. They can also be configured to run on specific branches or pull requests.
70 |
71 | #### Relevant Tools
72 |
73 | Local Build Tools
74 | > In addition to automating builds, these also manage dependencies
75 |
76 | - [Gradle](https://gradle.org/)
77 | - [Ant](https://ant.apache.org/)
78 | - [Maven](https://maven.apache.org/)
79 |
80 | Hosted CI Tools
81 | > These integrate with hosted VCS solutions like GitHub
82 |
83 | - [Travis CI](https://travis-ci.org/)
84 | - [Circle CI](https://circleci.com/)
85 | - [Jenkins](https://jenkins.io/) (self-hosted)
86 |
87 | ### Deployment & Infrastructure
88 |
89 | This is about getting your software to run in a live server environment so that it is publicly accessible to end users. It also involves packaging your app such that it can be deployed in different environments.
90 |
91 | #### Relevant Tools
92 |
93 | Infrastructure Management Tools
94 |
95 | - [Vagrant](https://www.vagrantup.com/) - portable virtual development environments
96 | - [Docker](https://www.docker.com/) - deployment of applications inside software containers
97 | - [Ansible](https://www.ansible.com/) - automated software provisioning, configuration management and deployment
98 | - [Puppet](https://puppet.com/) - configure infrastructure with declarative code
99 |
100 | Cloud Infrastructure
101 |
102 | - [DigitalOcean](https://www.digitalocean.com/) - cloud computing for developers
103 | - [Amazon Web Services](https://aws.amazon.com/) - scalable cloud computing services
104 |
105 | ### Monitoring
106 |
107 | After deployment, monitoring the system for any defects or potential defects is essential to minimise outages (infrastructure monitoring). This aspect also includes application performance management (APM) and log analysis.
108 |
109 | #### Relevant Tools
110 |
111 | - [Icinga](https://www.icinga.com/) - open source system and network monitoring [[install guide]](https://www.digitalocean.com/community/tutorials/how-to-use-icinga-to-monitor-your-servers-and-services-on-ubuntu-14-04)
112 | - [Monit](https://mmonit.com/monit/) - open source monitoring utility that also does auto recovery of services [[install guide]](https://www.digitalocean.com/community/tutorials/lemp-stack-monitoring-with-monit-on-ubuntu-14-04)
113 | - [ELK (Elasticsearch, Logstash, Kibana)](https://www.elastic.co/products) - analytics tools that work well together [[install guide]](https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-ubuntu-14-04)
114 |
115 | ## Getting Started
116 |
117 | As a software engineering student, the best way to get started is to gain some practical experience by trying out the various tools mentioned above. This is a suggested process you can try:
118 |
119 | 1. Contribute to an open source project on GitHub. Make several pull requests and get used to the flow of code review and frequent integration with the base branch.
120 | 2. Integrate build and test automation into an existing project that doesn't already use it (perhaps one of your past programming assignments). Write test cases for the project, put the project on GitHub, integrate Travis CI with it and get Travis to run the tests every time you push a commit.
121 | 3. Get familiar with the Ops side of things. Get a DigitalOcean droplet (use free credits from the GitHub education pack) and deploy a web app (any past project that you ran on localhost all the time). Try installing and configuring OpenVPN on the droplet (plus point - you get your own private VPN!)
122 | 4. Install a monitoring tool on the same droplet to notify you if the web app or the OpenVPN service goes down. Get it to email/ send you a Telegram message if this happens, and test it by manually stopping the web server/ OpenVPN.
123 | 5. Bonus: Try setting up automated deployment so your web app gets updated on the fly when code is pushed to the release branch.
124 |
125 | ## Useful Links
126 |
127 | ### Quick References
128 | - [What is DevOps?](https://aws.amazon.com/devops/what-is-devops/) - a concise article by Amazon Web Services
129 | - [The Essential DevOps Terms You Need to Know](https://blog.xebialabs.com/2016/03/21/essential-devops-terms/) - understand the most commonly used terms in the industry
130 |
131 | ### Further Reading
132 | - [Best Guide to Getting Started In DevOps](https://blog.xebialabs.com/2015/04/23/best-guide-to-getting-started-in-devops/) - recommends various places you can read up more on the topic
133 | - [9½ Simple Steps On How To Start With DevOps Today](https://devops.com/9%C2%BD-simple-steps-start-devops-today/) - includes practical suggestions like using a code static analysis tool
134 | - [A Pragmatic Guide to Getting Started with DevOps](https://www.ca.com/us/lpg/~/media/Files/eBooks/a-pragmatic-guide-to-getting-started-with-devops.pdf) - more of a management point of view but interesting
135 | - [The DevOps Handbook: How to Create World-Class Speed, Reliability, and Security in Technology Organizations](https://books.google.com.sg/books/about/The_Devops_Handbook.html?id=XrQcrgEACAAJ) - about managing tech organisations, written by a CTO
136 |
137 |
--------------------------------------------------------------------------------
/contents/docker/Container.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/docker/Container.png
--------------------------------------------------------------------------------
/contents/docker/DockerHub.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/docker/DockerHub.png
--------------------------------------------------------------------------------
/contents/docker/Docker_Architecture.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/docker/Docker_Architecture.png
--------------------------------------------------------------------------------
/contents/docker/Docker_Logo.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/docker/Docker_Logo.png
--------------------------------------------------------------------------------
/contents/docker/Docker_Microservices.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/docker/Docker_Microservices.jpg
--------------------------------------------------------------------------------
/contents/docker/Docker_Statistics.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/docker/Docker_Statistics.png
--------------------------------------------------------------------------------
/contents/docker/Docker_logo_011.0.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/docker/Docker_logo_011.0.png
--------------------------------------------------------------------------------
/contents/docker/VMvsContainer.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/docker/VMvsContainer.png
--------------------------------------------------------------------------------
/contents/docker/Virtual_Machine.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/docker/Virtual_Machine.png
--------------------------------------------------------------------------------
/contents/dotfiles/cd-autocomplete.gif:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/dotfiles/cd-autocomplete.gif
--------------------------------------------------------------------------------
/contents/dotfiles/default-prompt.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/dotfiles/default-prompt.png
--------------------------------------------------------------------------------
/contents/dotfiles/enhanced-prompt.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/dotfiles/enhanced-prompt.png
--------------------------------------------------------------------------------
/contents/dotfiles/git-flag-autocomplete.gif:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/dotfiles/git-flag-autocomplete.gif
--------------------------------------------------------------------------------
/contents/dotfiles/syntax-highlighting.gif:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/dotfiles/syntax-highlighting.gif
--------------------------------------------------------------------------------
/contents/elm/images/TEA.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/elm/images/TEA.png
--------------------------------------------------------------------------------
/contents/gpgpu/drivenet.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/gpgpu/drivenet.jpg
--------------------------------------------------------------------------------
/contents/gpgpu/gpgpu.md:
--------------------------------------------------------------------------------
1 |
7 |
8 | # An Introduction to GPGPU
9 |
10 | ###### Authors: [Pierce Anderson Fu](https://github.com/PierceAndy), [Nguyen Quoc Bao](https://github.com/bqnguyen94)
11 |
12 |
13 |
14 | * [§ 1. GPGPU](#1-gpgpu)
15 | * [§ 1.1 What is GPGPU?](#1-1-what-is-gpgpu)
16 | * [§ 1.2 Why Bother With Parallel Processing?](#1-2-why-bother-with-parallel-processing)
17 | * [§ 1.3 Aren't Multicore CPUs Enough?](#1-3-aren-t-multicore-cpus-enough)
18 | * [§ 1.4 What are the Challenges With GPGPU?](#1-4-what-are-the-challenges-with-gpgpu)
19 | * [§ 1.5 Implementations](#1-5-implementations)
20 | * [§ 1.6 GPGPU in Action](#1-6-gpgpu-in-action)
21 | * [§ 2. Further Readings](#2-further-readings)
22 | * [§ 3. References](#3-references)
23 |
24 |
25 | ## § 1. GPGPU
26 |
27 | ### § 1.1 What is GPGPU?
28 | GPGPU stands for General-purpose computing on graphics processing units. It is the use of a graphics processing unit (GPU), which typically handles computation only for computer graphics, to perform computation in applications traditionally handled by the central processing unit (CPU).[[1]](#footnote1)
29 |
30 | Simply put, it's a kind of parallel processing where we're trying to exploit the data-parallel hardware on GPUs to improve the throughput of our computers.
31 |
32 | ### § 1.2 Why Bother With Parallel Processing?
33 | Moore's law is the observation made by Gordon Moore that the density of transistors in an integrated circuit board doubles approximately every two years. It has long been co-opted by the semiconductor industry as a target, and consumers have taken this growth for granted.
34 |
35 | Because it suggests exponential growth, it is unsustainable and it cannot be expected to continue indefinitely. In the words of Moore himself, "It can't continue forever.".[[2]](#footnote2) There are hard physical limits to this scaling such as heat dissipation rate[[3]](#footnote3) and size of microprocessor features.[[4]](#footnote4)
36 |
37 | As software engineers, this means that free and regular performance gains can no longer be expected.[[5]](#footnote5) To fully exploit CPU throughput gains, we need to code differently.
38 |
39 | ### § 1.3 Aren't Multicore CPUs Enough?
40 | Between CPUs and GPUs, there are differences in **scale** and **architecture**.
41 | - In terms of **scale**, CPUs only have several cores while GPUs house up to thousands of cores.
42 | - In terms of **architecture**, CPUs are designed to handle sequential processing and branches effectively, while GPUs excel at performing simpler computations on large amounts of data.
43 |
44 | This means that CPUs and GPUs excel at different tasks. You'll typically want to utilize GPGPU on tasks that are data parallel and compute intensive (e.g. graphics, matrix operations).
45 |
46 | > ##### Definitions:
47 | >
48 | > *Data parallelism* refers to how a processor executes the same operation on different data elements simultaneously.
49 | >
50 | > *Compute intensive* refers to how the algorithm will have to process lots of data elements.
51 |
52 | ### § 1.4 What are the Challenges With GPGPU?
53 | Not all problems are inherently parallelizable.
54 |
55 | The SIMT (Single Instruction, Multiple Threads) architecture of GPUs means that they don't handle branches and inter-thread communication well.
56 |
57 | ### § 1.5 Implementations
58 |
59 | - CUDA: [Official website](https://www.nvidia.com/object/cuda_home_new.html)
60 | - OpenCL: [Official website](https://www.khronos.org/opencl/)
61 |
62 | ### § 1.6 GPGPU in Action
63 | The benefits of GPGPU are even more pronounced when it comes to embedded systems and Internet of Things (IoT) applications, where computing power is often compromised for physical space, weight, and power consumption. For instance, NVIDIA's Jetson TX2, a computation processor board that delivers the processing capability of the Pascal GPU architecture in a package the size of a business card[[6]](#footnote6). The Pascal architecture is used in many desktop computers, data centres, and supercomputers[[7]](#footnote7). Along with its small form factor, this makes the Jetson ideal for embedded systems that require intensive processing power.
64 |
65 | 
66 |
67 | *[NVIDIA Jetson TX2 embedded system-on-module with Thermal Transfer Plate (TTP)](https://devblogs.nvidia.com/jetson-tx2-delivers-twice-intelligence-edge/)*
68 |
69 | Packed with a NVIDIA Pascal GPU with 256 CUDA cores at maximally 1300 MHz[[6]](#footnote6), the TX2 is capable of intensive parallel computational tasks such as real-time vision processing or deploying deep neural networks, allowing mobile platforms to solve complex, real-world problems.
70 |
71 | 
72 |
73 | *[CES 2016: NVIDIA DRIVENet Demo - Visualizing a Self-Driving Future](https://www.youtube.com/watch?v=HJ58dbd5g8g)*
74 |
75 | Compatible with a range of components and large ecosystem products[[8]](#footnote8), the Jetson platform is now widely used for deploying vision and neural network processing onboard mobile platforms like self-driving cars, autonomous robots, drones, IoT, handheld medical devices, etc[[9]](#footnote9).
76 |
77 | ## § 2. Further Readings
78 | - [How concurrency is the next big change in software development since OO](http://www.gotw.ca/publications/concurrency-ddj.htm)
79 | - [Official CUDA C programming guide: What GPUs excel at processing, and why](https://docs.nvidia.com/cuda/cuda-c-programming-guide/#from-graphics-processing-to-general-purpose-parallel-computing)
80 | - [Official CUDA C programming guide: Architecture of NVIDIA GPUs](https://docs.nvidia.com/cuda/cuda-c-programming-guide/#simt-architecture)
81 | - [Lightning talk slides: An Introduction to GPGPU](https://github.com/nus-oss/lightningtalks/issues/10)
82 | - [se-edu's learning resource on CUDA](cuda.html)
83 |
84 | ## § 3. References
85 |
86 | [1]: https://en.wikipedia.org/wiki/General-purpose_computing_on_graphics_processing_units
87 | [2]: https://www.techworld.com/news/operating-systems/moores-law-is-dead-says-gordon-moore-3576581/
88 | [3]: https://theory.physics.lehigh.edu/rotkin/newdata/mypreprs/spie-09b.pdf
89 | [4]: https://arstechnica.com/gadgets/2016/07/itrs-roadmap-2021-moores-law/
90 | [5]: http://www.gotw.ca/publications/concurrency-ddj.htm
91 | [6]: https://www.nvidia.com/en-us/autonomous-machines/embedded-systems-dev-kits-modules/
92 | [7]: https://en.wikipedia.org/wiki/Tegra#Tegra_X1/
93 | [8]: https://elinux.org/Jetson_TX2#Ecosystem_Products
94 | [9]: https://developer.nvidia.com/embedded/learn/success-stories
95 | 96 |
97 |
--------------------------------------------------------------------------------
/contents/gpgpu/gpgpu_architecture.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/gpgpu/gpgpu_architecture.png
--------------------------------------------------------------------------------
/contents/gpgpu/jetson_tx2.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/gpgpu/jetson_tx2.jpg
--------------------------------------------------------------------------------
/contents/ios/IntroToIos.md:
--------------------------------------------------------------------------------
1 | 87 | [2]: https://www.techworld.com/news/operating-systems/moores-law-is-dead-says-gordon-moore-3576581/
88 | [3]: https://theory.physics.lehigh.edu/rotkin/newdata/mypreprs/spie-09b.pdf
89 | [4]: https://arstechnica.com/gadgets/2016/07/itrs-roadmap-2021-moores-law/
90 | [5]: http://www.gotw.ca/publications/concurrency-ddj.htm
91 | [6]: https://www.nvidia.com/en-us/autonomous-machines/embedded-systems-dev-kits-modules/
92 | [7]: https://en.wikipedia.org/wiki/Tegra#Tegra_X1/
93 | [8]: https://elinux.org/Jetson_TX2#Ecosystem_Products
94 | [9]: https://developer.nvidia.com/embedded/learn/success-stories
95 | 96 |
7 |
8 | # Introduction to iOS Development
9 |
10 | **Authors: [Bryan Lew](https://github.com/blewjy)**
11 |
12 | Reviewers: [Chester Sng](https://github.com/ChesterSng), [Jiang Chunhui](https://github.com/Adoby7), [Yu Pei, Henry](https://github.com/YuPeiHenry)
13 |
14 |
15 |
16 | * [What is iOS?](#what-is-ios)
17 | * [Why iOS?](#why-ios)
18 | * [Native iOS Applications vs. Cross-Platform Applications](#native-ios-applications-vs-cross-platform-applications)
19 | * [Getting Started With Native iOS Development](#getting-started-with-native-ios-development)
20 |
21 |
22 | ## What is iOS?
23 |
24 | iOS is the mobile operating system that runs on Apple's mobile devices, most notably the iPhone and the iPad. Applications that run on iOS can be downloaded officially from the App Store, and developers of iOS applications can submit their own applications to the App Store to share it with the rest of the world.
25 |
26 | ## Why iOS?
27 |
28 | [2 operating systems dominate the whole market share: iOS and Android](https://www.theverge.com/2017/2/16/14634656/android-ios-market-share-blackberry-2016). While more devices are running Android compared to iOS, given below are some areas in which iOS outshines Android:
29 |
30 | #### Better Compatibility and Standardisation
31 |
32 | iOS applications are specific to the Apple devices of the iPhone and iPad line, whereas Android runs across numerous different types of devices, including phones, tablets, watches, and more. The large variation of devices poses a compatibility problem from a developer's standpoint, because now applications have to be tested and working across all these different devices, and also regularly maintained this way. Each hardware device may possibly be running their own version of Android or will only support up to a particular version, thus Android developers will have to ensure that their application is also compatible across different versions of Android.
33 |
34 | With less devices to target, iOS developers have more control over their application and are able to customise the application to a greater extent, because there are less variations (in terms of device compatibility) to worry about. OS versioning in iOS is also more structured, with better backwards-compatability across their devices. Users of iOS devices (oldest supported is iPhone 5S, as of March 2019) are always immediately prompted whenever a new version of iOS is available for download. This means that as developers, we have to worry less about compatability issues, and focus more on the application itself.
35 |
36 | #### Lucrative Market
37 |
38 | While Android has higher market share of devices, iOS applications generate [nearly double](https://techcrunch.com/2018/07/16/apples-app-store-revenue-nearly-double-that-of-google-play-in-first-half-of-2018/) the revenue compared to Android, largely due to the fact that users of iOS devices are generally more likely to spend on apps.
39 |
40 | As an independent iOS developer, this means that your iOS application could bring in more revenue for you as compared to an Android one, assuming you choose to monetize it.
41 |
42 | More app revenue for iOS applications also means companies will look to tackle the iOS platform more so than Android, thus the demand for iOS developers will be high. This in turn results in [higher salary for iOS developers compared to their Android counterparts](https://www.fiercewireless.com/developer/ios-developers-earn-roughly-10k-more-than-android-counterparts-study-shows).
43 |
44 | #### Higher Quality Applications
45 |
46 | The Apple App Store [subjects the apps to higher quality control](https://developer.apple.com/app-store/review/) (as compared to Android Play Store) before they are allowed to be published. This means that applications on the App Store are more robust, and of higher quality in general.
47 |
48 | ## Native iOS Applications vs. Cross-Platform Applications
49 |
50 | Mobile applications that run on the iOS platform can be written both natively and using [cross-platform solutions](https://www.businessofapps.com/guide/cross-platform-mobile-app-development/). Native iOS applications are written using [Objective-C or Swift](https://android.jlelse.eu/objective-c-or-swift-which-technology-to-learn-for-ios-app-development-3c681d1a05ac) on the Xcode IDE that you can download if you are running a MacOS. Cross-platform solutions are tools that allow you to write code once and develop applications for more than 1 platform. Some examples include [React Native](https://facebook.github.io/react-native/), [Xamarin](https://visualstudio.microsoft.com/xamarin/), and [Ionic](https://ionicframework.com/). Both methods have their own pros and cons, and these factors not only affect the app developers, but also the users to some extent.
51 |
52 | The main attraction to developing mobile applications using cross-platform solutions is development time. Cross-platform solutions allow you to write code once, but push it out to more than one OS, usually iOS and Android being two of these platforms. This means that essentially, your development time is cut by half, because you are only writing code once for 2 separate applications. While this may sound like an attractive deal, there are a bunch of caveats to consider that can potentially be a deal breaker, [performance and compatability issues being the frontrunners](https://codeburst.io/native-vs-cross-platform-app-development-pros-and-cons-49f397bb38ac).
53 |
54 | If you are coming from a web development background, you might want to consider starting with React Native. React Native as a cross-platform mobile development framework will give you a mobile development environment that is very similar to a web development one (especially if you are familiar with ReactJS), using the same JavaScript structure as ReactJS, and also uses `props`, `state`, and all the standard React component lifecycle methods. It will allow you to learn about mobile development while on familiar ground.
55 |
56 | If you are a complete beginner to programming or do not have much software engineering experience, native iOS development with Swift might be a better choice to start with. Given that Swift is a statically typed language, more errors will be caught earlier, and you will be forced to be more structured and disciplined in your code. With Swift being a fast and readable language, it is not a bad language to start learning programming with or to pick up general software engineering skills with.
57 |
58 | ## Getting Started With Native iOS Development
59 |
60 | To get started with native iOS development, we have to first get some of the basic tools set up:
61 |
62 | - To use Xcode and write native iOS applications in Swift, you will need MacOS. If you are using a non-Mac operating system (Windows, Linux or others), you can install a MacOS Virtual Machine on your computer. This [blog post](https://medium.com/@twister.mr/installing-macos-to-virtualbox-1fcc5cf22801) is an excellent tutorial on how this can be done. If you already own a Macbook or an iMac, you are all set for this step.
63 | - Next, you should register for a free Apple Developer Account. Don't confuse this with the paid iOS Developer Program! Anyone can register for the Apple Developer Account for free. Go to [Apple's Developer Website](https://developer.apple.com/register/) to do this. It requires you to have an Apple ID, so you can use your existing one if you already have an Apple ID.
64 | - Once you have an Apple Developer Account, you can either directly download Xcode from the [website](https://developer.apple.com/xcode/), or search for Xcode on your Mac App Store. Xcode is the Integrated Development Environment (IDE) that provides you with everything you need to develop an iOS app from scratch. It also comes with the iPhone and iPad simulator that you will need to test your application.
65 |
66 | 
67 | ([Image](https://insights.dice.com/wp-content/uploads/2018/06/Xcode-Mac-iPad-Apple-Dice.png) from [Dice Insights](https://insights.dice.com/))
68 |
69 | Native iOS applications can be written in [Objective-C or Swift](https://android.jlelse.eu/objective-c-or-swift-which-technology-to-learn-for-ios-app-development-3c681d1a05ac). Swift is a relatively newer language, introduced only in 2014, while Objective-C is more of an old school programming language. If you are just starting out on iOS development, you should strongly consider using Swift. The main reason being that many of the documentations and help on the internet are written for iOS development in Swift, hence it will be easier to look for resources that target Swift instead of Objective-C. Furthermore, Apple mostly regards Swift as the main language for iOS programming now, and Objective-C support is now more of a backward-compatability.
70 |
71 | If you are new to Swift, you may want to read up on our Swift article: [Introduction to Swift]({{baseUrl}}/contents/swift/welcome-to-swift.html)
72 |
73 | If you are ready to begin developing your first iOS application, you will want to check out these iOS development tutorials and articles meant for first-timers:
74 | - [Codewithchris: How to make an iPhone app](https://codewithchris.com/how-to-make-an-iphone-app/)
75 | - [raywenderlich.com "How to build a simple iOS app"](https://www.raywenderlich.com/3114-ios-tutorial-how-to-create-a-simple-iphone-app-part-1-3)
76 |
77 | Some useful iOS development resources:
78 | - [Ray Wenderlich](https://www.raywenderlich.com/ios)
79 | - [Brian Advent](https://www.youtube.com/channel/UCysEngjfeIYapEER9K8aikw)
80 | - [Let's Build That App](https://www.youtube.com/channel/UCuP2vJ6kRutQBfRmdcI92mA)
81 |
82 |
83 |
--------------------------------------------------------------------------------
/contents/java/JavaSynchronization.md:
--------------------------------------------------------------------------------
1 |
7 |
8 | # Java Synchronization
9 |
10 | Authors: [Boxin](https://github.com/boxin-yang)
11 |
12 |
13 |
14 | * [Introduction](#introduction)
15 | * [Basic Syntax](#basic-syntax)
16 | * [Synchronized Statement](#synchronized-statement)
17 | * [Synchronized Method](#synchronized-method)
18 | * [More Features of Java Synchronization](#more-features-of-java-synchronization)
19 | * [wait() and notifyAll()](#wait-and-notifyall)
20 | * [Understand Which Object is Used as Monitor](#understand-which-object-is-used-as-monitor)
21 | * [Check Condition for wait() with While Loop](#check-condition-for-wait-with-while-loop)
22 | * [Breakpoint of notifyAll()](#breakpoint-of-notifyall)
23 | * [notify()](#notify)
24 | * [Related Areas](#related-areas)
25 |
26 |
27 | # Introduction
28 |
29 | Java Synchronization is the Java implementation of [Monitor][1]. The Monitor is used with [parallel threads][2] to ensure [mutual exclusion][3] in [critical section][4]. It is a mechanism to ensure [thread safety][5] when parallel threads are accessing shared data of an object.
30 |
31 | Java builds Monitor into [Java Object][6] and each object is itself a Monitor. Although Java has also implemented other mechanisms such as [Semaphore][7] in Java 5, Java Monitor is more widely used since it is supported in the first version of Java. In this tutorial, we will go through Java Synchronization in terms of basic syntax, useful features and best practice.
32 |
33 | # Basic Syntax
34 |
35 | Java Synchronization is done by using keyword **synchronized**. It is commonly used in two forms: *synchronized statement* and *synchronized method*. We will use example codes to illustrate how synchronized keyword is used.
36 |
37 | ## Synchronized Statement
38 |
39 | Let us look at the following code snippet to understand synchronized statement:
40 |
41 | ```
42 | synchronized(object) {
43 | // critical section
44 | }
45 | ```
46 |
47 | The **synchronized** keyword takes in an **object** as an argument and uses brackets to enclose the **critical section** that accesses object data. In the code above, the **object** argument is the object to be used as the Monitor. When **critical section** is executed, the **object Monitor** locks all data of the object and only the **critical section**in the synchronized block can access data of the **object**. When the execution of **critical section** finishes on a certain thread, the thread gives up the **object Monitor** lock and other threads can access the data of **object**. Therefore, as long as **critical section** is included in a synchronized block of a **object Monitor**, data of the object is guaranteed to be thread safe.
48 |
49 | ## Synchronized Method
50 |
51 | Another common way to use **synchronized** functionality is called **synchronized method**. The following code snippet is an example.
52 |
53 | ```
54 | public synchronized void foo() {
55 | // method code
56 | }
57 | ```
58 |
59 | In this example, **synchronized** is used as a method modifier to indicate that the method is synchronized. In the case of a synchronized method, all the content in the method is considered as **critical section** and the Monitor object used is dependent on the method. If the method is an instance method, then the Monitor object used is **this** instance. If the method is a class method, then the Monitor object used is a special class object initialized during run time for each class. The equivalence of the above code is:
60 |
61 | ```
62 | public void foo() {
63 | synchronized(this) {
64 | // method code
65 | }
66 | }
67 | ```
68 |
69 | # More Features of Java Synchronization
70 |
71 | Besides synchronized keyword to guard a critical section, another important feature of Java Synchronization is **wait()** and **notify()**.
72 |
73 | ## wait() and notifyAll()
74 | In a synchronized block, sometimes some conditions are required for execution. If such conditions are not met, wait() method can be used to give up the Monitor object lock and put the current thread into a * callback queue** once the conditions are met. Let us use the following code as an example.
75 |
76 | ```
77 | public synchronized void eatChicken() {
78 | while (!hasChicken) {
79 | wait();
80 | }
81 | eat();
82 | hasChicken = false;
83 | }
84 |
85 | public synchronized void cookChicken() {
86 | hasChicken = true;
87 | notifyAll();
88 | }
89 | ```
90 |
91 | In method eatChicken, the condition of hasChicken must be true before eat() can take place. However, instead of terminating the method, the program can call wait() and then this thread is put into a * callback queue**. When some other threads change the conditions, in this case, if cookChicken() method produces chicken and the condition for eatChicken() might be changed, cookChicken() can call notifyAll(). notifyALl() wakes up one by one all the threads in the * callback queue** associated with the Monitor object used to by coooChicken(). In this case, eatChicken() is woken up and eatChicken() gains Monitor object lock and starts execution from wait() onwards. This is basically how wait() and notifyAll() work in Java. However, here are some points to take notes to use them correctly.
92 |
93 | ### Understand Which Object is Used as Monitor
94 | As described from the very beginning, all the synchronized feature is associated with a Monitor object as a lock. This is also true for wait() and notifyAll(). The Monitor object associated with wait() and notifyAll() is the Monitor object associated with the synchronized block that calls wait() and notifyAll().
95 | When wait() is called, the current thread pull is put into the callback queue of the Monitor object of the synchronized block. When notifyAll() is called, only threads waiting in the Monitor object queue associated with notifyAll() will be notified. Therefore, in the above example of eatChicken() and cookChick(), the wait() and notifyAll() will only work if both eatChicken() and cookChicken() are associated with the same Monitor object.
96 |
97 | ### Check Condition for wait() With While Loop
98 | wait() is usually used to check condition(e.g. hasChicken in the method eatChicken()). It is a good practice to always use wait() inside a while loop to check for the condition. This is to avoid the situation in which thread is woken for an unintended purpose. The example below illustrates this situation:
99 | ```
100 | public synchronized void eatChicken() {
101 | if (!hasChicken) {
102 | wait();
103 | }
104 | eat();
105 | hasChicken = false;
106 | }
107 |
108 | public synchronized void drink() {
109 | if (!hasDrink) {
110 | wait();
111 | }
112 | drink();
113 | hasDrink = false;
114 | }
115 |
116 | public synchronized void cookChicken() {
117 | hasChicken = true;
118 | notifyAll();
119 | }
120 |
121 | public synchronized void prepareDrink() {
122 | hasDrink = true;
123 | notifyAll();
124 | }
125 | ```
126 |
127 | In this example, all four synchronized methods are associated with the same Monitor object. When both eatChicken() and drink() are called and are both put into the callback queue because current hasChicken is false and hasDrink is false, calling notifyAll() from cookChicken() will wake up both drink() and eatChicken() one after another. If while loop is not used to check for the condition, both drink() and eatChicken() will execute and the execution of drink() is not intended.
128 | However, instead if while loop is used to check for the condition, when drink() is woken, hasDrink will be checked again and drink() will call wait() again. One alternative solution is to make sure all wait() and notifyAll() associated with the same Monitor object are checking the same conditions. However, this is hard to maintain and verify.
129 |
130 | ### Breakpoint of notifyAll()
131 | One important point to take note when using notifyAll() is to understand what happens when notify all is called and how code gives up its Monitor object lock. Taking the following code snippet as an example:
132 |
133 | ```
134 | public synchronized void foo() {
135 | before();
136 | notifyAll();
137 | after();
138 | }
139 | ```
140 |
141 | When synchronized method foo() is called, it calls notifyAll() before after(). However, after notifyAll() is called, the method foo() does not give up Monitor object lock immediately. Instead, all codes in the synchronized block(in this case the method body) will be executed before the thread gives up the Monitor object lock and notifies all the threads in the callback queue.
142 |
143 | ## notify()
144 | Another feature of synchronized is notify(), which notifies only one thread in the callback queue. This method is less used than notifyAll() because notify() only wakes up one randomly chosen thread from callback queue and it is hard to assert that the thread woken up is the intended thread. However, there are special situations in which notify() is used and is preferred. When all threads that call wait() are checking for the same condition, it is no different to call anyone of the thread. notify() can be used in this situation and is better than nofityAll() because notifyAll() wake up all the threads and is, therefore, more costly. When one can be sure that all the threads in the callback queue are waiting for the same condition, one can use notify().
145 |
146 | # Related Areas
147 | Java Synchronization as demonstrated in this tutorial is easy to learn, and you are ready to use it after this tutorial. However, Java Synchronization is just a part of a bigger picture: Parallel programs in Java. To fully exploits the parallelism brought by modern hardware, you may also want to learn [Java Thread][2] and [Thread Safety][5]
148 |
149 |
150 | [1]: https://en.wikipedia.org/wiki/Monitor_(synchronization)
151 | [2]: to be updated when @ablyx finishes on java threads
152 | [3]: https://en.wikipedia.org/wiki/Mutual_exclusion
153 | [4]: https://en.wikipedia.org/wiki/Critical_section
154 | [5]: https://en.wikipedia.org/wiki/Thread_safety
155 | [6]: https://docs.oracle.com/javase/7/docs/api/java/lang/Object.html
156 | [7]: https://en.wikipedia.org/wiki/Semaphore_(programming)
157 |
158 |
159 |
--------------------------------------------------------------------------------
/contents/java/assertFailed.PNG:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/java/assertFailed.PNG
--------------------------------------------------------------------------------
/contents/java/assertPass.PNG:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/java/assertPass.PNG
--------------------------------------------------------------------------------
/contents/java/spring-overview.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/java/spring-overview.png
--------------------------------------------------------------------------------
/contents/java/thread_interference.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/java/thread_interference.png
--------------------------------------------------------------------------------
/contents/javascript/First_example.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/javascript/First_example.png
--------------------------------------------------------------------------------
/contents/javascript/Javascript-libraries.md:
--------------------------------------------------------------------------------
1 |
7 |
8 | # Libraries
9 |
10 | Authors: Li Kai
11 |
12 | As a beginner, there may be some hesistation in using libraries. The big advantage is the amount of benefit it brings to the team. Whether its the overhead of having to load the library, or having to learn a large library's syntax, not having to maintain more code makes the project more manageable, maintainable and that alone makes it worth its cost.
13 |
14 | ## Utility Libraries
15 |
16 | The limitation to Javascript is its limited utility library. That is why libraries like [lodash](https://lodash.com/) is such a popular module. There are a lot of functions being provided, which would help a lot in making shorter, more coherent syntax. Lodash reduces the need to write and maintain your own `_.assign` or `_.range` and even provides powerful functions such as `_.merge` and `_.groupBy`.
17 |
18 | ## Testing Libraries
19 |
20 | There are a large amount of Javascript testing libraries. The below is a non-exhaustive list.
21 |
22 | - [AVA](https://github.com/avajs/ava)
23 | - [Jasmine](https://jasmine.github.io/)
24 | - [Jest](https://facebook.github.io/jest/)
25 | - [Mocha](https://mochajs.org/)
26 | - [QUnitJs](https://qunitjs.com)
27 | - [Tape](https://github.com/substack/tape)
28 |
29 | So which one do you choose? Among the above, I have had the opportunity of write in all of the above except Tape.
30 |
31 | The general advice is to go for Mocha if you are new, due to its [large community](https://stateofjs.com/2016/testing/) and therefore overall amount of help available. When you are familiar with Mocha and understand its pitfalls (speed and organisation), you can explore other libraries such as Jest.
32 |
33 | Jest, due to its speed, support for React.js, support for asynchronous testing and helpful terminal outputs, seems to be the upcoming major player. The fact that it is being made by Facebook is icing to the cake.
34 |
35 | The odd test framework in the list above is QUnit, which is a test framework that runs on the browser instead of node.js like the others. QUnit has been around longer. Unless you are running JQuery and need to test for browser or UI related bugs, it is no longer a conventional choice.
36 |
37 |
38 |
--------------------------------------------------------------------------------
/contents/javascript/data.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/javascript/data.jpg
--------------------------------------------------------------------------------
/contents/javascript/dependency.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/javascript/dependency.png
--------------------------------------------------------------------------------
/contents/javascript/file.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/javascript/file.png
--------------------------------------------------------------------------------
/contents/javascript/html.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/javascript/html.png
--------------------------------------------------------------------------------
/contents/javascript/javascript-framework-react-images/ReactVirtualDOM.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/javascript/javascript-framework-react-images/ReactVirtualDOM.png
--------------------------------------------------------------------------------
/contents/javascript/javascript-framework-redux-images/3basiccomponents.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/javascript/javascript-framework-redux-images/3basiccomponents.png
--------------------------------------------------------------------------------
/contents/javascript/javascript-framework-vue-images/vue-lifecycle-diagram.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/javascript/javascript-framework-vue-images/vue-lifecycle-diagram.png
--------------------------------------------------------------------------------
/contents/kotlin/NullSafety.md:
--------------------------------------------------------------------------------
1 |
7 |
8 | # Null Safety in Kotlin
9 |
10 | Author: [Pan Haozhe](https://github.com/Haozhe321)
11 |
12 |
13 |
14 | * [What is Null Safety?](#what-is-null-safety)
15 | * [NullPointerException](#nullpointerexception)
16 | * [How Does Kotlin Handle Null Safety?](#how-does-kotlin-handle-null-safety)
17 | * [Nullable and Non-Nullable Type](#nullable-and-non-nullable-type)
18 | * [Safety Operators in Kotlin](#safety-operators-in-kotlin)
19 | * [Safe Call Operator](#safe-call-operator)
20 | * [Elvis Operators](#elvis-operators)
21 | * [Not-Null Assertion Operator](#not-null-assertion-operator)
22 | * [Summary](#summary)
23 | * [Learning Resources](#learning-resources)
24 |
25 |
26 | >“I call it my billion-dollar mistake. It was the invention of the null reference…My goal was to ensure that all use of references should be absolutely safe, with **checking performed automatically by the compiler**. But I couldn't resist the temptation to put in a null reference, simply because it was so easy to implement.”
27 | -Tony Hoare
28 |
29 | This document explains Kotlin's null safety feature. For an overview of Kotlin, see [here](kotlin.html).
30 |
31 | # What is Null Safety?
32 | _Null Safety_ (or _void safety_) is the guarantee that no object reference will have a `null` value.
33 | In object-oriented languages, access to objects is achieved through references. A typical function call is of the form `object.func()`; `object` denotes a reference to a certain object, and `func` denotes a function call. At execution time, the reference to `object` can be `void`, leading to run-time exceptions (In the case of Java, a NullPointerException) and often abnormal termination of the program.
34 |
35 | ## NullPointerException
36 | When developing Android applications in Java, [NullPointerException (NPE)](https://docs.oracle.com/javase/9/docs/api/java/lang/NullPointerException.html) was a big problem. In fact, About [one third of app crashes can be attributed to NPE](https://image.slidesharecdn.com/droidcon-bugsense-130408170720-phpapp01/95/droid-con-bugsense-16-638.jpg?cb=1365440918). To see how it happens, let's take a look at the piece of Java code below:
37 |
38 | ```java
39 | String a = null;
40 | if(a.length > 5) {
41 | //do something
42 | }
43 | ```
44 | When the above code is run, an NPE will be thrown on line 2 because a `null` object has no methods. To prevent an object from taking on a `null` value, programmers typically resort to doing additional checks like this:
45 | ```java
46 | String a = null;
47 | if(a != null && a.length > 5) {
48 | //do something
49 | }
50 | ```
51 | And of course, that's all fine, until we want to do something more complex. Say Bob belongs to a department, and we want to get the name of the department manager. That will look like this:
52 | ```java
53 | String managerName = bob.department.manager.name;
54 | ```
55 |
56 | Because each variable can be `null`, to prevent the NPE we put the code in the following code block:
57 | ```java
58 | if(bob != null) {
59 | Department department = bob.department;
60 | if(department != null) {
61 | Employee manager = department.manager;
62 | if(manager != null) {
63 | String name = manager.name;
64 | if(name != null) {
65 | //do something
66 | }
67 | }
68 | }
69 | }
70 | ```
71 | The deep-nested `if` statement reduces readability of our code.
72 |
73 | The other way is to use [Java Optionals](https://www.oracle.com/technetwork/articles/java/java8-optional-2175753.html).
74 | For the first example above, we can do
75 | ``` java
76 | a.ifPresent(this::doSomething);
77 | ```
78 |
79 | And for the deeply-nested `if` statements, the verbosity can be reduced with
80 | ``` java
81 | bob.map(Person::getDepartment)
82 | .map(Person::getManager)
83 | .map(Person::getName)
84 | .ifPresent(Person::doSomething);
85 | ```
86 | `map()` is a method in Java Optionals class that applies the function inside the parentheses to the object that is calling it. If the object is not present, it will return an empty Optional.
87 |
88 | Let's see how Kotlin deals with this issue while maintaining a simple and readable syntax.
89 |
90 | # How Does Kotlin Handle Null Safety?
91 | ## Nullable and Non-Nullable Type
92 | In Kotlin, a type can be _nullable_ or _non-nullable_, determined by the presence of a `?`. For example, an object of type `String` is non-nullable, while an object of type `String?` is nullable.
93 |
94 | As the compiler catches `null` assignments to non-nullable objects, the following would result in compilation error.
95 | ```Kotlin
96 | var firstString: String = "hello world"
97 | firstString = null //compilation error
98 | ```
99 | In comparison, the following assignment to a nullable type is allowed
100 | ```Kotlin
101 | var secondString: String? = "hello world"
102 | secondString = null //okay
103 | ```
104 |
105 | In the first case, we can safely call `firstString.length` without having to worry about a NPE because `firstString` can never be `null`.
106 | In the second case, `secondString` can potentially be `null`, so `secondString.length` will result in a compilation error as the compiler see the danger of such statement and blocks it early.
107 |
108 |
109 | ## Safety Operators in Kotlin
110 | Although non-nullable type is a strong feature in Kotlin, the [interoperability](https://kotlinlang.org/docs/reference/java-interop.html) with Java means that we have to use variables as nullable type sometimes. In the previous section, we seem to have hit an obstacle as the compiler blocks the call to `secondString.length`. In this section we look at some ways of overcoming this problem.
111 | ### Safe Call Operator
112 | Represented by `?.`, the safe call operator is used in this way
113 | ```kotlin
114 | secondString?.length
115 | ```
116 | This returns the length of `secondString` if `secondString` is not `null`, and `null` otherwise.
117 |
118 | Now we can chain like this
119 | ```Kotlin
120 | bob?.department?.manager?.name
121 | ```
122 | This chain will return `null` if any of the variables inside the chain is `null`.
123 |
124 |
125 | ### Elvis Operators
126 | Represented by `?:`, the Elvis operator is used in this way
127 | ```kotlin
128 | val length = secondString?.length ?: -1
129 | ```
130 | If the expression to the left of `?:` is not null, the Elvis operator (`?.`) will return it as it is; else it will return a default value supplied (-1 in this case).
131 |
132 | We also notice the use of safe call operator together with Elvis operator in the same statement.
133 |
134 | But the Elvis operator is more powerful than this. `return` and `throw` statements are legitimate default values on the right side of the Elvis operator. So you can define your own error message to aid debugging. For example:
135 | ```kotlin
136 | fun myFunc(node: Node): String? {
137 | val parent = node.getParent() ?: return null
138 | val name = node.getName() ?: throw IllegalArgumentException("Name expected")
139 | // ...
140 | }
141 | ```
142 | Doing so like this can help programmers to check for function arguments before carrying on with the required computation.
143 |
144 | At this point you may ask, "What if I still want my NPE?"
145 |
146 |
147 | ### Not-Null Assertion Operator
148 | Represented by `!!`, the not-null assertion operator is used in this way
149 | ```kotlin
150 | val stringLength = secondString!!.length
151 | ```
152 | The operator converts any value to a non-nullable type and throws an exception if the value is null. In the above example, `stringLength` will be assigned the length of `secondString` if `secondString` is not `null`; if secondString is `null`, a NPE is thrown. Kotlin tries to reduce the number of NPEs thrown as it is a run-time exception that is difficult to debug, in addition to creating so many app crashes. Hence NPEs in Kotlin are explicitly asked for.
153 |
154 |
155 | # Summary
156 | 1. Kotlin increases null safety of programs because some of the work required to ensure null safety is offloaded from the programmer to the compiler, which is less error prone.
157 | 2. Null Safety is enforced by the Kotlin language. This is better compared to Java Optionals which is a Class and not a language construct like Kotlin's null-safety system.
158 | 3. If you expect your object to **not** take on a `null` value, make it a non-nullable type!
159 | 4. Even if you make your object a nullable type, Kotlin handles it better than Java because it can help to prevent NPE. An generic NPE is hard to debug; in Kotlin a descriptive message could be given to make debugging easier (with the help of Elvis operator).
160 |
161 |
162 | # Learning Resources
163 | * [Null Safety in Kotlin](https://kotlinlang.org/docs/reference/null-safety.html)
164 | * [Comprehensive Guide to Null Safety in Kotlin](https://www.baeldung.com/kotlin-null-safety)
165 | * [The Billion Dollar Mistake](https://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare)
166 |
167 |
168 |
--------------------------------------------------------------------------------
/contents/kotlin/kotlin.md:
--------------------------------------------------------------------------------
1 |
7 |
8 | # Introduction to Kotlin
9 |
10 | Author: [Shradheya Thakre](https://github.com/tshradheya)
11 |
12 |
13 |
14 | * [Overview](#overview)
15 | * [Why Kotlin?](#why-kotlin)
16 | * [Concise](#concise)
17 | * [Safe](#safe)
18 | * [Interoperable](#interoperable)
19 | * [Tool Friendly](#tool-friendly)
20 | * [Kotlin for Android Apps](#kotlin-for-android-apps)
21 | * [Advantages of Shifting to Kotlin](#advantages-of-shifting-to-kotlin)
22 | * [Drawbacks of Shifting to Kotlin](#drawbacks-of-shifting-to-kotlin)
23 | * [Resources to Learn Kotlin](#resources-to-learn-kotlin)
24 |
25 |
26 | # Overview
27 |
28 | Kotlin is a general purpose, open source, statically typed programming language for the JVM and Android that combines object-oriented and functional programming features.
29 |
30 | # Why Kotlin?
31 |
32 | ### Concise
33 |
34 | Highly reduces the amount of boilerplate code.
35 | - A Java object with getters, setters, equals(), hashCode(), toString() and copy() can be created in a single line in Kotlin
36 | ``` kotlin
37 | data class Customer(val name: String, val email: String, val company: String)
38 | ```
39 | - Assigning values based on ranges is much simpler than Java. Below are extracts of code with similar logic in Java and Kotlin:
40 |
41 | **Kotlin**
42 | ``` kotlin
43 | val quartile: Int
44 | quartile = when (playPercentage) {
45 | in 0..24 -> 0
46 | in 25..49 -> 1
47 | ...
48 | }
49 | ```
50 |
51 | **Java**
52 | ``` java
53 | int quartile;
54 | if(playPercentage >= 0 && playPercentage <= 24) {
55 | quartile = 0;
56 | } else if(playPercentage >= 25 && playPercentage <= 49) {
57 | quartile = 1;
58 | }
59 | ...
60 | ```
61 |
62 | ### Safe
63 |
64 | Kotlin protects you from mistakenly operating on [nullable](https://kotlinlang.org/docs/reference/null-safety.html) types
65 | - Get compilation error when you mistakenly try operating on nullable types
66 | ``` kotlin
67 | val name: String? = null // Nullable type
68 | println(name.length()) // Compilation error
69 | ```
70 |
71 | ### Interoperable
72 |
73 | - Target either the JVM or JavaScript. Write code in Kotlin and decide where you want to deploy
74 | ``` kotlin
75 | import kotlin.browser.window
76 | fun onLoad() {
77 | window.document.body!!.innerHTML += "
Hello, Kotlin!" 78 | } 79 | ``` 80 | - You can literally continue work on your old Java projects using Kotlin. All your favorite Java frameworks are still available, and whatever framework you’ll write in Kotlin can be easily adopted by other Java developers 81 | 82 | ### Tool Friendly 83 | 84 | Hyperbolically, a programming language is only as good as what its tools can provide. This is why the advantage of using Kotlin is the built-in language support from IntelliJ. Any Java IDE for e.g. IntelliJ, Eclipse and Android Studio, can be used to write and compile Kotlin code. It also contains the aforementioned Java-to-Kotlin converter and code generators for Java and JavaScript from Kotlin code. 85 | 86 | # Kotlin for Android Apps 87 | 88 | While Java is one of the world's most widely used programming languages and is pretty much the official language of Android development, there are many reasons why Java might not always be the best option for your Android projects. 89 | 90 | The biggest issue is that Java isn’t a modern language, and although Java 8 was a huge step forward for the platform, introducing lots of features that developers had been waiting for (including lambda functions), at the time of writing Android only supports a subset of Java 8 features. 91 | 92 | ### Advantages of Shifting to Kotlin 93 | 94 | - Interchangeability With Java 95 | - Easy learning curve 96 | - Combine the best of functional and procedural programming 97 | - First-class Android Studio support 98 | - More concise code 99 | 100 | ### Drawbacks of Shifting to Kotlin 101 | 102 | - Extra runtime size due to increase in size of `.apk` 103 | - Initial readability of code may be hindered for core Java developers 104 | - Smaller community support 105 | 106 | # Resources to Learn Kotlin 107 | 108 | - [Official Resources by Android](https://developer.android.com/kotlin/resources.html) 109 | - [Kotlin Documentation](https://kotlinlang.org/docs/reference/) 110 | - [Cheatsheet for shifiting from Java to Kotlin](https://github.com/MindorksOpenSource/from-java-to-kotlin) 111 | 112 |
113 |
--------------------------------------------------------------------------------
/contents/naturalLanguage/constituencyParse.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/naturalLanguage/constituencyParse.png
--------------------------------------------------------------------------------
/contents/naturalLanguage/dbsChatbot.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/naturalLanguage/dbsChatbot.png
--------------------------------------------------------------------------------
/contents/naturalLanguage/openIE.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/naturalLanguage/openIE.png
--------------------------------------------------------------------------------
/contents/naturalLanguage/partsOfSpeech.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/naturalLanguage/partsOfSpeech.png
--------------------------------------------------------------------------------
/contents/naturalLanguage/word2vec.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/naturalLanguage/word2vec.png
--------------------------------------------------------------------------------
/contents/naturalLanguage/wordNet.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/naturalLanguage/wordNet.png
--------------------------------------------------------------------------------
/contents/node/eventloop.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/node/eventloop.png
--------------------------------------------------------------------------------
/contents/performance/PerformanceProfiling.md:
--------------------------------------------------------------------------------
1 | Hello, Kotlin!" 78 | } 79 | ``` 80 | - You can literally continue work on your old Java projects using Kotlin. All your favorite Java frameworks are still available, and whatever framework you’ll write in Kotlin can be easily adopted by other Java developers 81 | 82 | ### Tool Friendly 83 | 84 | Hyperbolically, a programming language is only as good as what its tools can provide. This is why the advantage of using Kotlin is the built-in language support from IntelliJ. Any Java IDE for e.g. IntelliJ, Eclipse and Android Studio, can be used to write and compile Kotlin code. It also contains the aforementioned Java-to-Kotlin converter and code generators for Java and JavaScript from Kotlin code. 85 | 86 | # Kotlin for Android Apps 87 | 88 | While Java is one of the world's most widely used programming languages and is pretty much the official language of Android development, there are many reasons why Java might not always be the best option for your Android projects. 89 | 90 | The biggest issue is that Java isn’t a modern language, and although Java 8 was a huge step forward for the platform, introducing lots of features that developers had been waiting for (including lambda functions), at the time of writing Android only supports a subset of Java 8 features. 91 | 92 | ### Advantages of Shifting to Kotlin 93 | 94 | - Interchangeability With Java 95 | - Easy learning curve 96 | - Combine the best of functional and procedural programming 97 | - First-class Android Studio support 98 | - More concise code 99 | 100 | ### Drawbacks of Shifting to Kotlin 101 | 102 | - Extra runtime size due to increase in size of `.apk` 103 | - Initial readability of code may be hindered for core Java developers 104 | - Smaller community support 105 | 106 | # Resources to Learn Kotlin 107 | 108 | - [Official Resources by Android](https://developer.android.com/kotlin/resources.html) 109 | - [Kotlin Documentation](https://kotlinlang.org/docs/reference/) 110 | - [Cheatsheet for shifiting from Java to Kotlin](https://github.com/MindorksOpenSource/from-java-to-kotlin) 111 | 112 |
7 |
8 | # Performance Profiling
9 |
10 | Author: [Ong Heng Le](https://github.com/initialshl)
11 |
12 |
13 |
14 | * [Overview](#overview)
15 | * [Performance Profiling For The First Time](#performance-profiling-for-the-first-time)
16 | * [Advanced Topics](#advanced-topics)
17 | * [Sampling Interval](#sampling-interval)
18 | * [Instrumentation Overhead](#instrumentation-overhead)
19 | * [Profiling Other Types Of Data](#profiling-other-types-of-data)
20 | * [Resources](#resources)
21 |
22 |
23 | ## Overview
24 |
25 | A performance profiler is a tool which collects data (such as function timings) about
26 | your program to identify the areas with performance issues, commonly used in code
27 | optimization. There are two common types of performance profiling methods.
28 |
29 | * **CPU sampling**: Collects *samples* at fixed intervals, which provides an overview of your program's performance
30 | * **Instrumentation**: Collects detailed *elapsed times* for your program's function calls
31 |
32 | Learn [which profiling method to use for your program](https://blogs.msdn.microsoft.com/ejarvi/2005/04/07/the-choice-between-sampling-and-instrumentation/)
33 | when choosing between CPU sampling and instrumentation.
34 |
35 | ## Performance Profiling for the First Time
36 |
37 | This article provides resources to help you identify the performance issues as you
38 | perform profiling on your program for the first time. The general steps in profiling a
39 | program for the first time are:
40 |
41 | 1. Run a performance profiling session
42 | 1. View the performance report
43 | 1. Trace down to the problem
44 | 1. Identify areas for improvements upwards
45 |
46 | Most profilers are similar in their functionalities, user interface, and use of
47 | technical terms. You may adapt the tutorials in this article to your preferred profiling
48 | tools on your own project.
49 |
50 | * [Profiling a Desktop Application In Visual Studio 2015](ProfilingDesktopAppVS2015.html)
51 |
52 | ## Advanced Topics
53 |
54 | ### Sampling Interval
55 |
56 | CPU sampling captures data at fixed intervals, usually based on CPU cycles or time. Many
57 | profilers offer the option to **set a custom interval or perform sampling based on events**,
58 | such as page faults or system calls, in the sampling settings.
59 |
60 | * For Visual Studio 2015: [How to: Choose Sampling Events](https://msdn.microsoft.com/en-us/library/ms182376.aspx)
61 | * For YourKit Java Profiler: [YourKit Java Profiler Help - CPU sampling settings](https://www.yourkit.com/docs/java/help/sampling_settings.jsp) 62 | 63 | ### Instrumentation Overhead 64 | 65 | Instrumentation profiling incurs a substantial overhead, which is an increase in file size 66 | and execution time of the program. This makes it unsuitable for large projects. In such 67 | cases, it is recommended to **limit instrumentation to a specific portion of your project**. 68 | 69 | * For Visual Studio 2015: [How to: Limit Instrumentation to Specific Functions](https://msdn.microsoft.com/en-us/library/cc470663.aspx) 70 | or [Specific DLLs](https://msdn.microsoft.com/en-us/library/bb385752.aspx) 71 | 72 | To reduce instrumentation overhead, some profilers may exclude *small functions*, which 73 | are short functions that do not make any function calls, and treat the time as being 74 | spent in their calling functions. This behavior is usually enabled by default, which may 75 | be undesirable when you want to **examine *small functions* carefully**. Most profilers 76 | offer the option to change this behavior. 77 | 78 | * For Visual Studio 2015: [How to: Exclude or Include Short Functions from Instrumentation](https://msdn.microsoft.com/en-us/library/bb514150.aspx) 79 | 80 | ## Profiling Other Types of Data 81 | 82 | Other than collecting performance statistics and timing data, profilers are also able 83 | to collect other information such as memory allocation and GPU usage. 84 | 85 | For Visual Studio 2015: 86 | 87 | 1. List of [Profiling Tools](https://msdn.microsoft.com/en-us/library/mt210448.aspx) (A [table](https://msdn.microsoft.com/en-us/library/mt210448.aspx#Anchor_10) which shows the most suitable tool for each project) 88 | 1. [How to: Collect Performance Data for a Web Site](https://msdn.microsoft.com/en-us/library/2s0xxa1d.aspx) (ASP.NET and JavaScript) 89 | 1. [Collecting .NET Memory Allocation and Lifetime Data](https://msdn.microsoft.com/en-us/library/dd264934.aspx) 90 | 1. [Collecting Thread and Process Concurrency Data](https://msdn.microsoft.com/en-us/library/dd265004.aspx) 91 | 1. [Collecting tier interaction data](https://msdn.microsoft.com/en-us/library/dd465169.aspx) 92 | 93 | ## Resources 94 | 95 | 1. What is a software profiler?: [Profiling Overview](https://msdn.microsoft.com/en-us/library/bb384493(v=vs.110).aspx) 96 | 1. Common performance profiling methods: [Understanding Performance Collection Methods](https://msdn.microsoft.com/en-us/library/dd264994.aspx) 97 | 1. Learn the best practices in profiling: [Advanced Profiling: Theory in Practice with NetBeans IDE](https://netbeans.org/community/magazine/html/04/profiler.html) 98 | 1. Why do profilers exclude small functions from instrumentation by default?: [Excluding Small Functions From Instrumentation](https://blogs.msdn.microsoft.com/profiler/2008/07/08/excluding-small-functions-from-instrumentation/) 99 |
100 |
--------------------------------------------------------------------------------
/contents/performance/ProfilingDesktopAppVS2015.md:
--------------------------------------------------------------------------------
1 | 61 | * For YourKit Java Profiler: [YourKit Java Profiler Help - CPU sampling settings](https://www.yourkit.com/docs/java/help/sampling_settings.jsp) 62 | 63 | ### Instrumentation Overhead 64 | 65 | Instrumentation profiling incurs a substantial overhead, which is an increase in file size 66 | and execution time of the program. This makes it unsuitable for large projects. In such 67 | cases, it is recommended to **limit instrumentation to a specific portion of your project**. 68 | 69 | * For Visual Studio 2015: [How to: Limit Instrumentation to Specific Functions](https://msdn.microsoft.com/en-us/library/cc470663.aspx) 70 | or [Specific DLLs](https://msdn.microsoft.com/en-us/library/bb385752.aspx) 71 | 72 | To reduce instrumentation overhead, some profilers may exclude *small functions*, which 73 | are short functions that do not make any function calls, and treat the time as being 74 | spent in their calling functions. This behavior is usually enabled by default, which may 75 | be undesirable when you want to **examine *small functions* carefully**. Most profilers 76 | offer the option to change this behavior. 77 | 78 | * For Visual Studio 2015: [How to: Exclude or Include Short Functions from Instrumentation](https://msdn.microsoft.com/en-us/library/bb514150.aspx) 79 | 80 | ## Profiling Other Types of Data 81 | 82 | Other than collecting performance statistics and timing data, profilers are also able 83 | to collect other information such as memory allocation and GPU usage. 84 | 85 | For Visual Studio 2015: 86 | 87 | 1. List of [Profiling Tools](https://msdn.microsoft.com/en-us/library/mt210448.aspx) (A [table](https://msdn.microsoft.com/en-us/library/mt210448.aspx#Anchor_10) which shows the most suitable tool for each project) 88 | 1. [How to: Collect Performance Data for a Web Site](https://msdn.microsoft.com/en-us/library/2s0xxa1d.aspx) (ASP.NET and JavaScript) 89 | 1. [Collecting .NET Memory Allocation and Lifetime Data](https://msdn.microsoft.com/en-us/library/dd264934.aspx) 90 | 1. [Collecting Thread and Process Concurrency Data](https://msdn.microsoft.com/en-us/library/dd265004.aspx) 91 | 1. [Collecting tier interaction data](https://msdn.microsoft.com/en-us/library/dd465169.aspx) 92 | 93 | ## Resources 94 | 95 | 1. What is a software profiler?: [Profiling Overview](https://msdn.microsoft.com/en-us/library/bb384493(v=vs.110).aspx) 96 | 1. Common performance profiling methods: [Understanding Performance Collection Methods](https://msdn.microsoft.com/en-us/library/dd264994.aspx) 97 | 1. Learn the best practices in profiling: [Advanced Profiling: Theory in Practice with NetBeans IDE](https://netbeans.org/community/magazine/html/04/profiler.html) 98 | 1. Why do profilers exclude small functions from instrumentation by default?: [Excluding Small Functions From Instrumentation](https://blogs.msdn.microsoft.com/profiler/2008/07/08/excluding-small-functions-from-instrumentation/) 99 |
7 |
8 | # Profiling a Desktop Application in Visual Studio 2015
9 |
10 | Author: [Ong Heng Le](https://github.com/initialshl)
11 |
12 | ## Overview
13 |
14 | This tutorial demonstrates how to get an overview of your desktop application's performance
15 | using the profiler included in [Visual Studio 2015](https://www.visualstudio.com/downloads/).
16 |
17 | ## 1. Run a Performance Profiling Session
18 |
19 | To start a performance profiling session, follow this guide on
20 | [Creating and running a performance session](https://msdn.microsoft.com/en-us/library/ms182372.aspx#Anchor_0).
21 |
22 | ## 2. View the performance report
23 |
24 | The profiler will generate the performance report after the profiling session, which you
25 | can explore on your own. *Inclusive and exclusive data* provides meaningful information
26 | about the execution time of each function.
27 |
28 | * **Inclusive samples**: Collected during execution of the function itself and all functions it calls
29 | * **Exclusive samples**: Collected during execution of the function itself only
30 |
31 | > The Visual Studio Profiler Team Blog has a [good explanation on inclusive and exclusive data values](https://blogs.msdn.microsoft.com/profiler/2004/06/09/what-are-exclusive-and-inclusive/).
32 |
33 | ## 3. Trace Down to the Problem
34 |
35 | You can identify the performance issues using the performance report. The `Summary` view
36 | shows these two useful information analyzed by the profiler.
37 |
38 | * **Functions With Most Individual Work**: The functions which took up the most execution time *(exclusive samples)*
39 | * **Hot Path**: The branch of the call tree which took up the most execution time *(inclusive samples)*
40 |
41 | To locate performance issues quickly, the **Functions With Most Individual Work** provides
42 | a list of functions which are usually candidates for optimization.
43 |
44 | If you would like to trace down to the problem more carefully, the **Hot Path** is a good
45 | starting point. This will familiarize you about the most expensive execution path taken
46 | by your program.
47 |
48 | > You can follow this [walkthrough](https://msdn.microsoft.com/en-us/library/ms182398.aspx)
49 | to experience how to identify performance problems and optimize your code.
50 |
51 | ## 4. Identify Areas for Improvements Upwards
52 |
53 | You have identified the problem, and may now want to optimize the code in the
54 | function body. But before that, here's a final tip: It is sometimes easier
55 | (and better) to optimize by
56 | **reducing the number of calls to that function in its calling functions.**
57 |
58 | ## Resources
59 |
60 | 1. The general steps in profiling your program: [Beginners Guide to Performance Profiling](https://msdn.microsoft.com/en-us/library/ms182372.aspx)
61 | 1. Read my chapter on performance profiling for more advanced topics: [Performance Profiling](PerformanceProfiling.html)
62 |
63 |
64 |
--------------------------------------------------------------------------------
/contents/performance/WebPerformance.md:
--------------------------------------------------------------------------------
1 |
7 |
8 | # Web Performance
9 |
10 | **Author(s): [Marvin Chin](https://github.com/marvinchin)**
11 |
12 | Reviewers: [Lu Yang Kenneth](https://github.com/luyangkenneth), [Monika Manuela Hengki](https://www.github.com/monmanuela)
13 |
14 |
15 |
16 | * [What is Web Performance](#what-is-web-performance)
17 | * [Why Web Performance Matters](#why-web-performance-matters)
18 | * [Key Ideas in Improving Web Performance](#key-ideas-in-improving-web-performance)
19 | * [Measuring Web Performance](#measuring-web-performance)
20 | * [Additional Resources](#additional-resources)
21 |
22 |
23 | ## What is Web Performance?
24 |
25 | Web Performance is a broad term that refers to how performant a web application *feels* to its users. This includes many aspects such as how long the site takes to load, how quickly the site becomes interactive, and how responsive it feels when the user interacts with the site.
26 |
27 | As web applications grow more complex, it becomes increasingly important for web developers to be aware of the factors that affect performance, and consider how performance can impact the user experience.
28 |
29 | ## Why Web Performance Matters?
30 |
31 | The performance of your web application has a direct impact on its ability to attract and retain users:
32 | - Sites which take more than 3 seconds to load are more likely to be abandoned by users [[source](https://developer.akamai.com/blog/2016/09/14/mobile-load-time-user-abandonment)].
33 | - Conversely, reducing page load times have been shown to have significant impact on improving user engagement [[source](https://medium.com/carousell-insider/how-we-made-carousells-mobile-web-experience-3x-faster-bbb3be93e006)].
34 | - In addition, search engines have begun to use page load times as a factor in determining search rankings [[source](https://webmasters.googleblog.com/2018/01/using-page-speed-in-mobile-search.html)].
35 | - Improving web performance is also essential in making your web application accessible to users from emerging markets, where low-end devices and limited bandwidth are the norm [[source](https://building.calibreapp.com/beyond-the-bubble-real-world-performance-9c991dcd5342)].
36 |
37 | In order to deliver a positive user experience, web developers must ensure that their applications meet acceptable performance standards.
38 |
39 | ## Key Ideas in Improving Web Performance
40 | There are many factors which affect web performance. Here, we give an overview of some of the key ideas that can be used to improve the performance of your web application.
41 |
42 | **Idea 1: Reduce Javascript Payloads**
43 |
44 | Loading and executing Javascript is often the slowest part of the page load process [[source](https://medium.com/@addyosmani/the-cost-of-javascript-in-2018-7d8950fbb5d4)]. Reducing the amount of Javascript that needs to be loaded thus significantly reduces the time taken for your site to load.
45 |
46 | Here are some steps that web developers can take to reduce the amount of Javascript that clients need to load:
47 | - Use code splitting to load only the Javascript required for the page being accessed [[source](https://developers.google.com/web/fundamentals/performance/optimizing-javascript/code-splitting/)].
48 | - Remove unused code (often parts of libraries that are not used in the application) with tree shaking [[source](https://developers.google.com/web/fundamentals/performance/optimizing-javascript/tree-shaking/)].
49 | - Ensure that the delivered Javascript is minified [[source](https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/javascript-startup-optimization/)].
50 | - Create performance budgets that specify the maximum size of Javascript payloads, and enforce them to ensure that the size of your Javascript payloads remain under control [[source](https://web.dev/fast/incorporate-performance-budgets-into-your-build-tools)].
51 |
52 | **Idea 2: Optimize Images**
53 |
54 | Images form a significant portion of the resources loaded on web applications [[source](https://httparchive.org/reports/page-weight)]. To improve performance and speed up the site for users, web developers should thus try to reduce the amount of bandwidth used to load images.
55 |
56 | Images can be optimized for the web in the following ways:
57 | - Use the correct file format that provides the appropriate balance between quality, detail, and file size. For example, you should use JPEG for photographs that contain lots of colors, PNG for logos and icons which have less colors but contain sharper details, and GIF for animated images [[source](https://medium.com/beginners-guide-to-mobile-web-development/web-image-formats-googles-webp-17e2fe5fc53e)].
58 | - Compress your images to reduce the amount of data that needs to be loaded [[source](https://www.html5rocks.com/en/tutorials/speed/img-compression/)]. Be careful to strike a balance between image size and quality for optimal user experience.
59 | - Deliver appropriately sized images based on the resolution of the client device [[source](https://developer.mozilla.org/en-US/docs/Learn/HTML/Multimedia_and_embedding/Responsive_images)]. This prevents smaller (often mobile or lower-end) devices from wasting bandwidth loading large images that cannot be displayed at their full resolution on the device.
60 |
61 | **Idea 3: Use Progressive Rendering**
62 |
63 | The sooner the user sees content being displayed on the page, the faster they perceives the site to be. Progressive rendering achieves this by avoiding rendering the entire page all at once, but instead ordering the loading of content in a manner that allows *some* parts of the page to be rendered as quickly as possible.
64 |
65 | Here are several ways how progressive loading can be effectively applied:
66 | - Load only the most critical content as part of the initial render, and fetch remaining resources asynchronously after the render [[source](https://developers.google.com/web/fundamentals/performance/critical-rendering-path/)]. Code splitting can also be helpful here to reduce the amount of Javascript that needs to be loaded for the initial render [[source](https://developers.google.com/web/fundamentals/performance/optimizing-javascript/code-splitting/)].
67 | - Use lazy loading to defer requesting non-critical resources until they are needed [[source](https://developers.google.com/web/fundamentals/performance/lazy-loading-guidance/images-and-video/)].
68 | - Show content placeholders while fetching resources to indicate loading progress to the user [[source](https://medium.com/@praveencnaik/content-placeholder-the-new-design-trend-for-audience-involvement-e2ab533d7304)].
69 |
70 | ## Measuring Web Performance
71 |
72 | Improving web performance is a continuous, ongoing process. The ability to measure and track performance is necessary for developers to monitor the impact of their changes over time.
73 |
74 | Here are some ways that you can measure the performance of your web applications:
75 | - Test your site under various network conditions with [Chrome DevTools](https://developers.google.com/web/tools/chrome-devtools/network/network-conditions). This is helpful to get a feel for how the site feels for users on slower devices during the development process.
76 | - Audit web performance using [Lighthouse](https://developers.google.com/web/tools/lighthouse/). This provides many useful metrics for understanding the performance of your site and identifying performance bottlenecks under controlled conditions.
77 | - Use [WebPageTest](https://www.webpagetest.org/) to test the performance of your website from multiple locations around the world at real consumer connection speeds. This provides real-world performance indicators that is especially useful if the target audience of your site resides in a different region.
78 | - Collect user-centric performance metrics to help you to monitor how your application actually performs in a real-world scenario, and how your site performance relates to other user engagement metrics [[source](https://developers.google.com/web/fundamentals/performance/user-centric-performance-metrics)].
79 |
80 |
81 | ## Additional Resources
82 | - [Google's Web Performance Fundamentals](https://developers.google.com/web/fundamentals/performance/why-performance-matters/) is a comprehensive resource that explains many factors that affect web performance.
83 | - [Kinsta's Beginner Guide to Website Speed Optimization](https://kinsta.com/learn/page-speed/) is a guided resource for improving website speeds.
84 | - [Twitter Lite Case Studey](https://medium.com/@paularmstrong/twitter-lite-and-high-performance-react-progressive-web-apps-at-scale-d28a00e780a3) is a real-world study that highlights how web performance principles can be applied to great effect.
85 |
86 |
87 |
--------------------------------------------------------------------------------
/contents/projectManagement/githubActions.md:
--------------------------------------------------------------------------------
1 |
7 |
8 | # GitHub Actions
9 |
10 | Author: [Alfred Yip](https://github.com/alyip98)
11 |
12 | Reviewers: [James Pang](https://github.com/jamessspanggg), [Tan Yuanhong](https://github.com/le0tan), [Tejas Bhuwania](https://github.com/Tejas2805), [Daryl Tan](https://github.com/openorclose)
13 |
14 |
15 |
16 |
17 | * [What is GitHub Actions](#what-is-github-actions)
18 | * [Why GitHub Actions](#why-github-actions)
19 | * [Easy to Use](#1-easy-to-use)
20 | * [Reusable Actions](#2-reusable-actions)
21 | * [Cross OS Support](#3-cross-os-support)
22 | * [Docker Support](#4-docker-support)
23 | * [Powerful](#5-powerful)
24 | * [Affordable](#6-affordable)
25 | * [Concepts](#concepts)
26 | * [Set up a Simple Workflow for a Node.js App](#set-up-a-simple-workflow-for-a-node-js-app)
27 | * [Where to Go From Here](#where-to-go-from-here)
28 |
29 |
30 | ## What is GitHub Actions
31 |
32 |
33 | GitHub Actions is a **workflow automation** solution that is tightly **integrated** into the GitHub ecosystem.
34 | It is commonly used to provide end-to-end continuous integration and deployment (CI/CD) directly in a repository.
35 | Beyond that, users also have the potential to automate more than just CI/CD with GitHub Actions through GitHub's webhooks API,
36 | including workflows such as labelling issues and tracking bundle size.
37 |
38 | ---
39 |
40 | ## Why GitHub Actions
41 |
42 |
43 | ### 1. Easy to Use
44 | Setting up a workflow in a repository is easy and hassle-free. [Starter workflows](https://github.com/actions/starter-workflows) for popular languages are available and the built-in editor offers code completion and documentation at a glance.
45 | 
46 |
47 | ### 2. Reusable Actions
48 | Actions are small, reusable units in a workflow. When setting up a workflow, we can reuse actions written by the community. These actions are available at the marketplace.
49 |
50 |
51 | 
52 |
53 |
54 |
55 | ### 3. Cross OS Support
56 | GitHub Actions gives you the option to [run workflows](https://help.github.com/en/actions/reference/virtual-environments-for-github-hosted-runners#supported-runners-and-hardware-resources) on Ubuntu, Windows and macOS.
57 |
58 | ### 4. Docker Support
59 | GitHub Actions provides the ability to run docker images for projects that use a dockerized environment.
60 |
61 | ### 5. Powerful
62 | Integration with the GitHub ecosystem allows GitHub Actions to do more than just build/test/deploy. You can automate everything related to your GitHub project, including bug triaging, labeling, automated dependency updating etc.
63 | Workflows can be triggered by events that happen on GitHub and will have access to the context of the issue/PR/commit that triggered it.
64 |
65 | ### 6. Affordable
66 | Like many of its competitors, GitHub Actions provides **free unlimited runner time for public repositories**.
67 |
68 | For private repositories, the runner time you get with a free plan is limited, but still ample for hobby projects.
69 |
70 | ---
71 |
72 | ## Concepts
73 |
74 |
75 | * **Workflow**
76 | A workflow describes a sequence of jobs to be run when a trigger event occurs.
77 | Each job consists of a series of steps, which may be actions or shell commands.
78 | Each workflow is defined as a `.yml` file in the `.github/workflows` directory of a repository.
79 |
80 | * **Action**
81 | An action is a piece of code that performs an atomic task.
82 | For example, [checking out a repository](https://github.com/actions/checkout), or [caching dependencies](https://github.com/actions/cache).
83 |
84 | * **Triggers**
85 | Workflows can be triggered by events on the GitHub repository, ranging from opening pull requests or issues, making commits to scheduled events, or even from an external event by calling GitHub's REST API endpoint.
86 |
87 | ---
88 |
89 | ## Set Up a Simple Workflow for a Node.js App
90 |
93 |
94 | GitHub has made it extremely easy to set up a simple workflow without even leaving the browser.
95 |
96 | 
97 |
98 | 1. Head to the Actions tab on your repository
99 | 1. You can choose to set up a workflow from a template or from scratch. We will use the Node.js template in this example
100 | 1. The online editor is powerful and easy to use, boasting features such as inline documentation and autocomplete
101 | 
102 | 1. The template workflow will
103 | 1. Begin when a commit is pushed or a pull request is opened to `master`
104 | 1. Checkout the current repo (using `actions/checkout`)
105 | 1. Install dependencies `npm ci`
106 | 1. Build `npm run build --if-present`
107 | 1. Run tests `npm run test`
108 | 1. Report the status of the workflow via a status badge
109 | 
110 | 1. Since we set the workflow up on the `master` branch, when we commit the newly-created workflow file, we can see GitHub Actions in action by navigating to the Actions tab
111 | 
112 |
113 | ---
114 |
115 | ## Where to Go From Here
116 |
119 | The official [documentation](https://help.github.com/en/actions) is a good resource to learn more about GitHub Actions.
120 | The [starter workflows](https://github.com/actions/starter-workflows) repository contains good examples of
121 | CI workflows for popular languages as well as automation workflows for GitHub.
122 | Here is a list of [awesome actions](https://github.com/sdras/awesome-actions) that you can use when writing your workflow.
123 |
124 |
125 | A powerful extension to GitHub Actions is to use the [github-script](https://github.com/actions/github-script) action
126 | to run JavaScript within the workflow using GitHub's [Octokit](https://octokit.github.io/rest.js/v17/) API.
127 | The Octokit API allows you to create workflows that span across GitHub's ecosystem of services.
128 |
129 | Here is a compiled list of popular repositories that use GitHub Actions that you can use as a reference. Do note that this is far from a comprehensive list and is more of a sample of how organizations are leveraging GitHub Actions in their development and production workflow.
130 |
131 |
132 |
133 | Org/Project | Workflows | Notes
134 | --- | --- | ---
135 | Bootstrap | [test](https://github.com/twbs/bootstrap/blob/master/.github/workflows/test.yml) | Comprehensive set of tests including custom tests and coveralls
136 | Oh My Zsh | [test](https://github.com/ohmyzsh/ohmyzsh/blob/master/.github/workflows/main.yml) | Workflow uses mainly shell commands
137 | React Native | [needs-attention](https://github.com/facebook/react-native/blob/master/.github/workflows/needs-attention.yml), [process-label-actions](https://github.com/facebook/react-native/blob/master/.github/workflows/process-label-actions.yml) | Automatically apply labels to newly created issues and adds comments to issues based on labels
138 | Node.js | [workflows](https://github.com/nodejs/node/tree/master/.github/workflows) | Linters, tests and build jobs
139 | Chart.js | [CI](https://github.com/chartjs/Chart.js/blob/master/.github/workflows/ci.yml) | Conditional workflow for windows and ubuntu
140 | Next.js | [workflows](https://github.com/zeit/next.js/tree/canary/.github/workflows) | Complex yet well structured build/test/deploy workflow. Automatically reports bundle size statistics in pull requests and on releases. Scheduled tests that run every 12 hours.
141 |
142 |
143 |
--------------------------------------------------------------------------------
/contents/projectManagement/images/actions-tab.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/projectManagement/images/actions-tab.png
--------------------------------------------------------------------------------
/contents/projectManagement/images/editor.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/projectManagement/images/editor.png
--------------------------------------------------------------------------------
/contents/projectManagement/images/effectiveSteps.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/projectManagement/images/effectiveSteps.jpg
--------------------------------------------------------------------------------
/contents/projectManagement/images/mantra.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/projectManagement/images/mantra.jpg
--------------------------------------------------------------------------------
/contents/projectManagement/images/marketplace.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/projectManagement/images/marketplace.png
--------------------------------------------------------------------------------
/contents/projectManagement/images/progress.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/projectManagement/images/progress.png
--------------------------------------------------------------------------------
/contents/projectManagement/images/projectManagement-agileDev-iterations.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/projectManagement/images/projectManagement-agileDev-iterations.jpg
--------------------------------------------------------------------------------
/contents/projectManagement/images/status.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/projectManagement/images/status.png
--------------------------------------------------------------------------------
/contents/projectManagement/images/tddSteps.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/projectManagement/images/tddSteps.jpg
--------------------------------------------------------------------------------
/contents/ruby/Ruby.md:
--------------------------------------------------------------------------------
1 |
2 | title: Introduction to Ruby
3 | pageNav: 3
4 |
5 |
6 |
2 | title: Security - https
3 | pageNav: 3
4 |
5 |
6 |
2 | title: Cross Site Request Forgery (CSRF)
3 | pageNav: 3
4 |
5 |
6 |
2 | title: ESLint
3 | pageNav: 3
4 |
5 |
6 |
2 | title: FindBugs
3 | pageNav: 3
4 |
5 |
6 |
2 | title: PMD
3 | pageNav: 3
4 |
5 |
6 |
2 | title: CheckStyle
3 | pageNav: 3
4 |
5 |
6 |
2 | title: Static Analysis
3 | pageNav: 3
4 |
5 |
6 |
2 | title: Introduction to ZeroMQ
3 | pageNav: 3
4 |
5 |
6 |
7 |
8 | # Introduction to Ruby
9 |
10 | Authors: [Wilson Kurniawan](https://github.com/wkurniawan07)
11 |
12 |
13 |
14 | * [Ruby Overview](#ruby-overview)
15 | * [Getting Started](#getting-started)
16 | * [Everything is an Object](#everything-is-an-object)
17 | * [Readability is King](#readability-is-king)
18 | * [Functional Programming is Encouraged](#functional-programming-is-encouraged)
19 | * [Object-Oriented Programming is Possible, Too](#object-oriented-programming-is-possible-too)
20 | * [Advanced Topics](#advanced-topics)
21 | * [Ruby Frameworks and DevOps](#ruby-frameworks-and-devops)
22 |
23 |
24 | ## Ruby Overview
25 |
26 | **Ruby** is a **dynamic**, **reflective**, **object-oriented**, **general-purpose** programming-language, designed and developed by Yukihiro "Matz" Matsumoto in 1990s.
27 | It is widely used across many fields of computer science and software engineering, and its most prevalent use is in web development.
28 |
29 | Ruby boasts high programmer productivity with its concise, elegant, and human-readable syntax, which is widely agreed to outweigh the loss in execution speed (as compared to compiled languages such as C and Java) unless the latter is critical.
30 |
31 | > Often people, especially computer engineers, focus on the machines. They think, "By doing this, the machine will run faster. By doing this, the machine will run more effectively. By doing this, the machine will something something something."
32 | > They are focusing on machines. But in fact, we need to focus on humans, on how humans care about doing programming or operating the application of the machines. We are the masters. They are the slaves.
33 | >
34 | > *Yukihiro Matsumoto, creator of Ruby language*
35 |
36 | Popular websites running on Ruby: GitHub, Airbnb, Twitter, Shopify.
37 |
38 | ## Getting Started
39 |
40 | [The official website of Ruby](https://www.ruby-lang.org/en/) will have everything you need from release notes to downloads to documentations.
41 |
42 | To install Ruby, go to the download page of the above website and follow the instructions.
43 | If you're a Mac user, you don't need to do anything; Macs come with Ruby pre-installed!
44 |
45 | For beginners and intermediate users, [tutorialspoint](https://www.tutorialspoint.com/ruby/index.htm) has great introductory Ruby tutorials, and [rubylearning.com](http://rubylearning.com/satishtalim/tutorial.html) is another great place.
46 | We will explore some of the more prominent language features of Ruby.
47 |
48 | ### Everything is an Object
49 |
50 | Ruby has no concept of "primitives". Anything that can be assigned to a variable is an object. Even numbers and boolean values `true` and `false` are objects.
51 |
52 | ```rb
53 | 5.times do
54 | puts "Hello world!"
55 | end
56 |
57 | # Hello world!
58 | # Hello world!
59 | # Hello world!
60 | # Hello world!
61 | # Hello world!
62 | ```
63 |
64 | As such, it becomes very easy to introduce new functions to those "primitives":
65 |
66 | ```rb
67 | class Fixnum # integers in Ruby belong to this class
68 | def next()
69 | return self + 1
70 | end
71 | end
72 |
73 | 7.next()
74 | # => 8
75 | ```
76 |
77 | ### Readability is King
78 |
79 | > Ruby is simple in appearance, but is very complex inside, just like our human body.
80 | >
81 | > *Yukihiro Matsumoto*
82 |
83 | Some languages attempt to make their codes as close to pseudocode as possible, but Ruby takes it to another level: Ruby code flows almost as smoothly as an English literary text.
84 |
85 | ```rb
86 | for num in 0..5
87 | puts "I am counting #{num}!"
88 | end
89 |
90 | unless number.even?
91 | puts "Number is odd!"
92 | end
93 |
94 | ["chocolate", "strawberry", "vanilla"].each { |flavour|
95 | make_ice_cream(flavour)
96 | }
97 | ```
98 |
99 | *No, you're not reading an English poem. You're reading Ruby!*
100 |
101 | ### Functional Programming is Encouraged
102 |
103 | For those familiar with [higher-order functions](https://www.cse.unsw.edu.au/~en1000/haskell/hof.html), Ruby supports *map*, *fold-left*, and *filter*.
104 |
105 | ```rb
106 | # Map
107 | [1, 2, 3, 4, 5].map { |n| n * n }
108 | # => [1, 4, 9, 16, 25]
109 |
110 | # Fold-left
111 | [1, 2, 3, 4, 5].inject(0) { |sum, n| sum + n }
112 | # => 15
113 |
114 | # Filter
115 | [1, 2, 3, 4, 5].select { |n| n.even? }
116 | # => [2, 4]
117 | ```
118 |
119 | Joel McCracken makes an [excellent short presentation](https://joelmccracken.github.io/functional-programming-in-ruby/#/) on how functions are treated in Ruby.
120 |
121 | ### Object-Oriented Programming is Possible, Too
122 |
123 | ```rb
124 | class Circle
125 | def initialize(r)
126 | @radius = r
127 | end
128 |
129 | def get_radius
130 | return @radius
131 | end
132 |
133 | def get_area
134 | return Math::PI * @radius * @radius
135 | end
136 | end
137 |
138 | c = Circle.new(5)
139 | c.get_area
140 | # => 78.53981633974483
141 | ```
142 |
143 | More on this in [Object-oriented Ruby tutorial](https://www.tutorialspoint.com/ruby/ruby_object_oriented.htm).
144 |
145 | ## Advanced Topics
146 |
147 | - [Blocks in Ruby](https://www.tutorialspoint.com/ruby/ruby_blocks.htm) - ever imagine that a method invocation can be another method's parameter?
148 | - [Modules and mixins in Ruby](https://www.tutorialspoint.com/ruby/ruby_modules.htm) - provides namespacing, and makes multiple inheritance (or a variant of it, technically) possible.
149 | - [Threads and fibers in Ruby](http://pltconfusion.com/concurrency_primitives_and_abstractions_in_ruby/) - dealing with concurrencies.
150 | - [Metaprogramming in Ruby](https://www.toptal.com/ruby/ruby-metaprogramming-cooler-than-it-sounds) - when your Ruby code *writes* Ruby code at runtime.
151 | - [Domain-Specific Languages (DSLs) in Ruby](https://www.leighhalliday.com/creating-ruby-dsl) - Ruby's amazing support for blocks and metaprogramming makes it a first choice for many developers to write a DSL.
152 | - [Ruby style guide](https://github.com/bbatsov/ruby-style-guide) - as agreed by the community at large.
153 | - [21 Ruby tricks](http://www.rubyinside.com/21-ruby-tricks-902.html) - making good use of Ruby's language features.
154 | - [Ruby best practices](http://www.reedbushey.com/119Ruby%20Best%20Practices.pdf) - a book written by a Ruby expert with foreword provided by Yukihiro Matsumoto himself.
155 |
156 | ## Ruby Frameworks and DevOps
157 |
158 | As with most other languages, tools and frameworks exist for serious Ruby developers and project managers to assist many of their tasks.
159 |
160 | - **Frameworks:** [Ruby on Rails](https://rubyonrails.org) is by far the most popular web application framework for Ruby. Another popular framework is [Sinatra](http://www.sinatrarb.com).
161 | - **Libraries:** Ruby libraries come in form of *gems*. The complete registry can be found on [this website](https://rubygems.org).
162 | - **IDE:** [Aptana Studio](http://www.aptana.com/products/studio3.html) is the favourite IDE for Rails developers. Other alternatives are [RubyMine](https://www.jetbrains.com/ruby/) (commercial) and Eclipse with [RDT plugin](https://sourceforge.net/projects/rubyeclipse/).
163 | - **Task Automation:** [Rake](http://docs.seattlerb.org/rake/) is the most commonly used automation tool.
164 | - **Static Analysis:** [RuboCop](https://batsov.com/rubocop/) is the sole leading static analysis tool for Ruby language.
165 |
166 | [This repository](https://github.com/markets/awesome-ruby) lists down a large collection of Ruby libraries, tools, frameworks, and software.
167 |
168 |
169 |
--------------------------------------------------------------------------------
/contents/ruby/mvc.jpeg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/ruby/mvc.jpeg
--------------------------------------------------------------------------------
/contents/security/Https.md:
--------------------------------------------------------------------------------
1 |
7 |
8 | # Security - https
9 |
10 | Author: [Boxin][1]
11 |
12 |
13 |
14 | * [Overview](#overview)
15 | * [Why Do We Need HTTPS?](#why-do-we-need-https)
16 | * [Why Is HTTPS Secure?](#why-is-https-secure)
17 | * [How to Set up HTTPS?](#how-to-set-up-https)
18 | * [Misuse of HTTPS](#misuse-of-https)
19 | * [Conclusion](#conclusion)
20 |
21 |
22 | ## Overview
23 |
24 | HTTPS is the end-to-end encryption on data on top of HTTP to prevent network sniffing (eavesdropping data packets). In this tutorial, we will cover four pointers to have a better understanding of https. They are:
25 |
26 | - [Why do we need HTTPS?](#why-do-we-need-https)
27 | - [Why is HTTPS secure?](#why-is-https-secure)
28 | - [How to set up HTTPS?](#how-to-set-up-https)
29 | - [Misuse of HTTPS](#misuse-of-https)
30 |
31 | ### Why Do We Need HTTPS?
32 |
33 | The web application usually runs over IP network, which is vulnerable to network sniffing. The old HTTP transmits data packets in plain text and if the network is sniffed, the sniffer can see confidential information in the data packets such as the password or [session tokens][2]. Here are some examples on how a plain text could be sniffed.
34 |
35 | - On public free wifi, a Wifi adapter in monitor mode would be able to capture all the ongoing packets to and from the wifi access point, regardless of its destination. If the traffic is transmitted over HTTP, the data sent over wifi is in plain text and the session token and password can be stolen. One famous example would be [Firesheep][3], a Firefox plugin to sniff session token used in websites such as Facebook. This has made Facebook [change its default protocol from HTTP to HTTPS][4]
36 |
37 | - Our network packets usually travel through switches and routers around the globe to reach the destination. Any one of them, if compromised, could expose our network traffic to the sniffer. [Network tap][5] is an example of a device used to sniff network traffic.
38 |
39 | - Our Internet architecture relies on [DNS][6] for domain name and IP mapping and [ARP][7] for MAC address and IP mapping. None of the above is built with security in mind. Common attacks such as [DNS cache poisoning][8] or [ARP poisoning][9]
40 | could redirect your traffic for monitoring.
41 |
42 | All in all, the Internet architecture that we rely on for network transmission is very vulnerable to network sniffing. If we were to use HTTP, which transmits packets in plain text, no confidentiality could be guaranteed for our web application. Therefore, we need to use HTTPS as an end to end encryption to secure our network packets.
43 |
44 | ### Why Is HTTPS Secure?
45 |
46 | As aforementioned, our network is not secure, so how could HTTPS help? HTTPS is built on top of HTTP with the addition of [SSL][10] to encrypt the plain text message. The purpose of this encryption is to make sure only client and the server could decrypt the message with required keys, and sniffer cannot decrypt packets even though they may sniff packets.
47 |
48 | There are mainly 3 encryption algorithms used in HTTPS, namely RSA, Diffie Hellman and Elliptic Curve Algorithm. They are more thoroughly explained in [Introduction to Cryptography][11] section. These algorithms prevent sniffers from decrypting packets without knowing the keys used because the best attack algorithms known at the moment run in [sub-exponential][12] time. Therefore, the attack is believed to be computationally infeasible when the keysize is large enough (e.g. 2048 bits for RSA), though it is not mathematically proven.
49 |
50 | Thus, by using https, we can be sure that even though our network packets are transmitted over an insecure network, sniffers cannot understand the content of our encrypted packets.
51 |
52 | Besides providing secure network traffic, HTTPS also provides server validation through Certificate Authority (CA) architecture. A detailed explanation on CA is [here][13] . In short, CA works by issuing the server a digital certificate that can only be produced by CA. When the server sends its digital certificate to the client, client browsers verify the digital certificate with CA to check whether the server is indeed the intended server. To obtain such digital certificate, the server needs to apply to CA and CA will verify the server before issuing the digital certificate.
53 |
54 | ### How to Set up HTTPS?
55 | In order set up HTTPS on your server, you would need to have:
56 |
57 | 1. A dedicated IP address for your server.
58 | 2. Obtain a digital certificate from a CA.
59 |
60 | For example, one of the CA is [digicert][14] and they provide SSL [installation guide][15] for different platforms. Usually, the SSL setup could be found on the CA that you obtained a digital signature from.
61 |
62 | However, most CAs are not free of charge. One free initiative to provide free domain validation certificates is [Lets Encrypt][16].
63 |
64 | ## Misuse of HTTPS
65 |
66 | Up to this moment, it seems that nothing could go wrong with HTTPS. However, in real life, there could be weakness on the implementation of HTTPS.
67 |
68 | #### SHA-1 Collision Attack
69 | As aforementioned, Certificate Authority (CA) signs a digital certificate provided by the server to prove the identity of the server. However, in the real-world implementation, the CA does not sign the digital certificate directly, rather CA signs the fixed length [hashed digest][17] of the digital certificate for efficiency. This introduces the possibility of two different digital certificates with the same hashed digest. Thus, if the attackers manage to forge a fake digital certificate with the same hashed digest of another valid digital certificate, the browser would trust the attackers server and all the servers signed by the attackers. This loophole occurs with SHA-1 hashing algorithm and SHA-1 is no longer used in HTTPS after 2016. In 2017, Google has announced an algorithm to forge a duplicated SHA-1 hash and the report is [here][18]. A more detailed explanation of this problem is found [here][19].
70 |
71 | #### Weak Diffie-Hellman Attack
72 | During the [cipher suite negotiation][20] process of HTTPS, client and server send each other in plain text the HTTPS standard they support and the most secure standard is chosen to be used. However, this can be exploited for HTTPS downgrade by a [man in the middle attack][21]. The man in the middle could send forged cipher suite negotiation to both server and client to indicate the maximum security supported is only 512 bits Diffie Hellman and trick server and client to encrypt with 512 bits Diffie Hellman. At the moment, without knowing the keys, 512 bits Diffie Hellman algorithm could be decrypted with sufficient resources. A more detailed description could be found [here][22]
73 |
74 | #### Session Hijacking on Partially Protected Websites
75 | Some websites use HTTPS only on the login page to encrypt the login credentials of users. However, this is vulnerable to [session hijacking][23] attack. Most websites use cookies and session tokens to maintain a stateful connection with its users, and the session token is embedded into each packet to authenticate the identity of users. Without using HTTPS in all the websites in the domain, the packets containing session token is transmitted in plaintext and sniffers can easily obtain the session token to impersonate the victim. A demonstration of this attack on Qoo10.com user can be found [here][24]
76 |
77 | ## Conclusion
78 | HTTPS provides security to a web application. If the web application requires secure network traffic (e.g. online banking), HTTPS should be implemented. However, servers usually need to pay for the digital certificate used by HTTPS. Also, the additional layer of encryption and decryption adds overhead to network traffic (though the impact is not significant). If secure network traffic is not required (e.g. University home page), HTTPS may not be used.
79 |
80 | [1]: https://github.com/boxin-yang
81 | [2]: https://searchsoftwarequality.techtarget.com/definition/session-ID
82 | [3]: https://github.com/codebutler/firesheep
83 | [4]: https://www.facebook.com/notes/facebook-engineering/secure-browsing-by-default/10151590414803920/
84 | [5]: https://searchnetworking.techtarget.com/definition/Network-tap
85 | [6]: https://en.wikipedia.org/wiki/Domain_Name_System
86 | [7]: https://en.wikipedia.org/wiki/Address_Resolution_Protocol
87 | [8]: https://en.wikipedia.org/wiki/DNS_spoofing
88 | [9]: https://en.wikipedia.org/wiki/ARP_spoofing
89 | [10]: https://www.digicert.com/ssl.htm
90 | [11]: ../security/cryptography.md
91 | [12]: https://en.wikipedia.org/wiki/Time_complexity#Sub-exponential_time
92 | [13]: https://www.globalsign.com/en-sg/ssl-information-center/what-are-certification-authorities-trust-hierarchies/
93 | [14]: https://www.digicert.com
94 | [15]: https://www.digicert.com/ssl-certificate-installation.htm
95 | [16]: https://letsencrypt.org/
96 | [17]: https://en.wikipedia.org/wiki/Cryptographic_hash_function
97 | [18]: https://security.googleblog.com/2017/02/announcing-first-sha1-collision.html?m=1
98 | [19]: https://www.sott.net/article/275524-Why-HTTPS-and-SSL-are-not-as-secure-as-you-think
99 | [20]: https://en.wikipedia.org/wiki/Cipher_suite
100 | [21]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack
101 | [22]: https://weakdh.org/
102 | [23]: https://en.wikipedia.org/wiki/Session_hijacking
103 | [24]: https://www.youtube.com/watch?v=BjTwNzoMUuk
104 |
105 |
--------------------------------------------------------------------------------
/contents/security/crossSiteRequestForgery/crossSiteRequestForgery.md:
--------------------------------------------------------------------------------
1 |
7 |
8 | # Cross Site Request Forgery (CSRF)
9 |
10 | Author: [Tran Tien Dat](https://github.com/tran-tien-dat)
11 |
12 |
13 |
14 | * [Flow of a CSRF attack](#flow-of-a-csrf-attack)
15 | * [Step 1: The Victim Logs in to a Vulnerable Web Service](#step-1-the-victim-logs-in-to-a-vulnerable-web-service)
16 | * [Step 2: The Victim Visits an Untrusted Website Controlled by the Attacker](#step-2-the-victim-visits-an-untrusted-website-controlled-by-the-attacker)
17 | * [Step 3: The Untrusted Website Makes Requests to the Vulnerable Web Service on the Victim's Behalf](#step-3-the-untrusted-website-makes-requests-to-the-vulnerable-web-service-on-the-victim-s-behalf)
18 | * [Conditions for a Successful CSRF Attack](#conditions-for-a-successful-csrf-attack)
19 | * [Defense Against CSRF Attacks](#defense-against-csrf-attacks)
20 | * [References](#references)
21 |
22 |
23 | Cross-Site Request Forgery (CSRF) is a dangerous type of attack that has affected major sites like [Gmail](http://archive.oreilly.com/pub/post/gmail_exploit_contact_list_hij.html) and [Netflix](https://blog.jeremiahgrossman.com/2006/10/more-on-netflixs-csrf-advisory.html) in the past. This article attempts to give an easy-to-digest introduction to the attack and how to protect your website from it.
24 |
25 | ## Flow of a CSRF Attack
26 |
27 | A CSRF attack tricks the victim to perform actions that they do not intend to do on a web application in which they're currently authenticated. It generally consists of 3 steps:
28 | 1. The victim logs in to a vulnerable web service
29 | 2. The victim visits an untrusted website controlled by the attacker
30 | 3. The untrusted website makes requests to the vulnerable web service on the victim's behalf
31 |
32 | These steps are best explained through an example. Suppose that Alice is a customer of the banking website `www.example-bank.com`. Alice wants to transfer money to her friend Bob (because she does not like the fact that he keeps paying for her meals when they go out together).
33 |
34 | ### Step 1: The Victim Logs in to a Vulnerable Web Service
35 |
36 | Alice first logs in to the website at `www.example-bank.com/login`, providing her username and password. She then clicks on a hyperlink on the site to go to the URL `www.example-bank.com/transfer` to perform her transaction. Note that Alice does not have to enter her username and password again, but the bank still knows that the request is made by her. This is done through a mechanism of the HTTP protocol called a [HTTP Cookie](https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies). A *cookie* is a small piece of information that is sent along with all HTTP requests to a particular website. In our case, after receiving Alice's credentials for logging in, the bank instructs Alice's browser to store a *cookie* `auth=1abcd2ek3292fsa390sdf` and **send this cookie together with every subsequent request** to `www.example-bank.com`. Thus, once the web server sees this cookie in the request, it knows that this is an authenticated request from Alice. Many websites use this cookie mechanism so that users only have to log in once.
37 |
38 | At `www.example-bank.com/transfer`, Alice fills out the following HTML form to transfer money to Bob:
39 |
40 | ```HTML
41 |
47 | ```
48 |
49 | As Alice clicks the submit button, the browser will send the following request to `www.example-bank.com`:
50 |
51 | ```
52 | POST https://www.example-bank.com/transfer HTTP/1.1
53 | Cookie: auth=1abcd2ek3292fsa390sdf
54 |
55 | receiver-name=Bob&receiver-account-no=123456&amount=100
56 | ```
57 |
58 | The bank's server verifies that the cookie `auth=1abcd2ek3292fsa390sdf` is valid and is associated with Alice. It then proceeds to transfer $100 from Alice's account to account number 123456, which belongs to Bob.
59 |
60 | ### Step 2: The Victim Visits an Untrusted Website Controlled by the Attacker
61 |
62 | Alice then receives a chat message from her not-so-trustworthy friend, Eve:
63 |
64 | ```
65 | Eve: OMG Alice! This site is selling the shoes you have always
66 | wanted for half the price: www.i-am-not-evil.com/shoes
67 | ```
68 |
69 | Excited by this piece of news, Alice clicks on the hyperlink to check out `www.i-am-not-evil.com`.
70 |
71 | ### Step 3: The Untrusted Website Makes Requests to the Vulnerable Web Service on the Victim's Behalf
72 |
73 | While browsing `www.i-am-not-evil.com`, Alice clicks on the `View more pictures` button on the website to see more pictures of her favorite pair of shoes. Unbeknownst to her, that button has the following HTML code:
74 |
75 | ```HTML
76 |
82 | ```
83 |
84 | So, when she clicks that button, the browser actually sends the following request to `https://www.example-bank.com`:
85 |
86 | ```
87 | POST https://www.example-bank.com/transfer HTTP/1.1
88 | Cookie: auth=1abcd2ek3292fsa390sdf
89 |
90 | receiver-name=Eve&receiver-account-no=987654&amount=100000
91 | ```
92 |
93 | Since Alice is still logged in to `www.example-bank.com`, the browser automatically attaches the cookie `auth=1abcd2ek3292fsa390sdf` to any requests made to `www.example-bank.com`. Hence, the bank's server considers this as an authenticated request from Alice and proceeds to transfer $100,000 from Alice's account to account number 987654, which belongs to the attacker!
94 |
95 | ## Conditions for a Successful CSRF Attack
96 |
97 | From the example, we can see that a CSRF attack works by forging a valid request which **inherits the identity and privileges of the victim** on the vulnerable web server. Such inheritance of privileges is possible because the browser attaches any credentials associated with a website to all requests made to the site. Therefore, if the user is currently authenticated to the site, the site will have no way to distinguish between the forged request sent by the victim and a legitimate request sent by the victim. In short, there are 3 conditions necessary for the attack:
98 |
99 | 1. The target website is vulnerable to a CSRF attack (i.e. it cannot distinguish between forged and legitimate requests).
100 | 2. The victim is authenticated to the target website.
101 | 3. The victim is tricked to visit an untrusted website by social engineering.
102 |
103 | *Note: [Social engineering](https://en.wikipedia.org/wiki/Social_engineering_%28security%29) refers to psychological manipulation of people into performing actions. In our example, Eve shares a link with a description that matches Alice's interest, which increases the chance that Alice will follow the link. Thus, this action is a form of social engineering.*
104 |
105 | ## Defense Against CSRF Attacks
106 |
107 | Both users and websites can take actions to defend themselves against CSRF attacks. As the user, one can reduce the likelihood of being attacked by logging out of sensitive services like banking immediately after carrying out transactions. This will negate condition number 2. Condition number 3 can be mitigated if the user is extremely careful and never visits websites they do not know. However, users generally do not follow this and just click on interesting links they encounter on social media. Hence, this threat of social engineering could never be fully eliminated.
108 |
109 | As for the web server, it can eliminate condition number 1 by adding a secret token as a hidden field to all forms that it serves. This token should be random and cannot be guessed by the attacker. Back to our example, the bank website should use a form like the following:
110 |
111 | ```HTML
112 |
119 | ```
120 |
121 | So a valid request by Alice made from the bank's own website will also carry this secret token `csrf-token=abcd23ksk3l2faad2kdkl`. Since the token is added by the bank's website on its webpages, forged requests from other websites will not know the token. Thus, when the bank's server receives a request, it can distinguish between a legitimate and a forged request by checking that the CSRF token is present and valid.
122 |
123 | ## References
124 |
125 | - https://owasp.org/www-community/attacks/csrf (A more technical description of the attack)
126 | - https://owasp.org/www-project-cheat-sheets/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html (In-depth discussion of the various defense approaches, including those that do not work)
127 | - https://www.cgisecurity.com/csrf-faq.html (Short FAQs about CSRF)
128 | - https://docs.djangoproject.com/en/2.0/ref/csrf/ (CSRF Protection in Django)
129 |
130 |
131 |
--------------------------------------------------------------------------------
/contents/security/crossSiteScripting/images/StatisticGraphic.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/security/crossSiteScripting/images/StatisticGraphic.png
--------------------------------------------------------------------------------
/contents/security/passwordStorage/P.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/security/passwordStorage/P.png
--------------------------------------------------------------------------------
/contents/security/passwordStorage/W.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/security/passwordStorage/W.png
--------------------------------------------------------------------------------
/contents/security/passwordStorage/cheese.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/security/passwordStorage/cheese.png
--------------------------------------------------------------------------------
/contents/security/passwordStorage/pigpen.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/security/passwordStorage/pigpen.png
--------------------------------------------------------------------------------
/contents/shells/check-file.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/shells/check-file.png
--------------------------------------------------------------------------------
/contents/shells/check-permission.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/shells/check-permission.png
--------------------------------------------------------------------------------
/contents/staticAnalysis/ESLint.md:
--------------------------------------------------------------------------------
1 |
7 |
8 | # ESLint
9 |
10 | Author: [Nicholas Chua](https://github.com/nus-cs3281/2018/blob/master/students/nicholasChua/nicholasChua-Resume.md)
11 |
12 |
13 |
14 | * [Overview](#overview)
15 | * [Rules](#rules)
16 | * [Setting Up](#setting-up)
17 | * [Usage](#usage)
18 | * [Plugins](#plugins)
19 | * [Resources](#resources)
20 |
21 |
22 | ## Overview
23 |
24 | [ESLint](https://eslint.org/docs/about/) is an open source JavaScript linter.
25 |
26 | Code linting is a type of static analysis that helps find problematic patterns or code that does not follow a certain style guideline. JavaScript, being a loosely-typed, interpreted language, is prone to developer errors and therefore linting is especially important when working with JavaScript.
27 |
28 | ESLint is shipped with a wide range of [built-in rules](https://eslint.org/docs/rules/) that can be configured on installation. It also offers flexibility by allowing importing your own custom rules via [plugins](https://eslint.org/docs/developer-guide/working-with-plugins).
29 |
30 | ## Rules
31 | ESLint comes shipped with [built-in rules](https://eslint.org/docs/rules/) that can be [configured](#setting-up) to suit your needs.
32 |
33 | An example of a rule would be [`brace-style`](https://eslint.org/docs/rules/brace-style). It enforces a specific placement of braces relative to the control statement and body. Brace style is based on preference or organization standards, and ESLint has options to enforce the brace style of your choice. They include:
34 |
35 | [One true brace style (1TBS)](https://en.wikipedia.org/wiki/Indentation_style#Variant:_1TBS_(OTBS))
36 | ```js
37 | if (foo) {
38 | bar();
39 | } else {
40 | baz();
41 | }
42 | ```
43 |
44 | [Stroustrup](https://en.wikipedia.org/wiki/Indentation_style#Variant:_Stroustrup)
45 | ```js
46 | if (foo) {
47 | bar();
48 | }
49 | else {
50 | baz();
51 | }
52 | ```
53 |
54 | [Allman](https://en.wikipedia.org/wiki/Indentation_style#Allman_style)
55 | ```js
56 | if (foo)
57 | {
58 | bar();
59 | }
60 | else
61 | {
62 | baz();
63 | }
64 | ```
65 |
66 | An additional option `allowSingleLine` can be enabled to allow opening and closing braces for a block to be on the same line:
67 | ```js
68 | if (foo) { bar(); }
69 | else { baz(); }
70 | ```
71 |
72 | If you are unsure about the options of the rule you want to change, you can refer to the comprehensive [rule documentation](https://eslint.org/docs/rules/).
73 |
74 | ## Setting up
75 |
76 | After [installing with npm](https://eslint.org/docs/user-guide/getting-started#installation-and-usage), ESLint can be configured in [two ways](https://eslint.org/docs/user-guide/configuring#configuring-rules):
77 |
78 | **Single configuration file**. In this example, the rule is configured in `.eslintrc.json` and enforces all braces in the **project** to follow Allman style:
79 | ```json
80 | {
81 | "rules": {
82 | "brace-style": ["error", "allman"]
83 | }
84 | }
85 | ```
86 |
87 | **Configuration comments** inside your file. In this example, the comments enforces all braces in the **file** to follow Allman style:
88 | ```js
89 | /* eslint brace-style: ["error", "allman"] */
90 | if (foo === bar) { // should be on the next line
91 | return;
92 | }
93 | ```
94 | Linting output:
95 | ```shell
96 | 1:10 error Opening curly brace appears on the same line as controlling statement brace-style
97 | ```
98 |
99 | 100 | **NOTE**: That specifying the rule option as "error" will result in the exit code of 1 when the rule is violated. If you wish for ESLint to alert you but not fail the linting, you may set it to "warn": 101 | 102 | ```js 103 | /* eslint brace-style: ["warn", "allman"] */ 104 | if (foo === bar) { // warning displayed but does not affect exit code 105 | return; 106 | } 107 | ``` 108 | Linting output: 109 | ```shell 110 | 1:10 warning Opening curly brace appears on the same line as controlling statement brace-style 111 | ``` 112 |
113 | 114 | If you are not sure what rules to add to your project, you can import [configurations](https://eslint.org/docs/developer-guide/shareable-configs) published by more experienced developers or organizations. 115 | 116 | Here are some popular style guide's configurations: 117 | * [Airbnb](https://www.npmjs.com/package/eslint-config-airbnb) 118 | * [Standard](https://www.npmjs.com/package/eslint-config-standard) 119 | * [Google](https://www.npmjs.com/package/eslint-config-google) 120 | 121 | At the same time, ESLint allows you to disable rules for [specific files and directories](https://eslint.org/docs/user-guide/configuring#ignoring-files-and-directories) or for [specific lines](https://eslint.org/docs/user-guide/configuring#disabling-rules-with-inline-comments). 122 | 123 | ## Usage 124 | Once configured, you can lint your files with [terminal commands](https://eslint.org/docs/user-guide/command-line-interface) and you can add [options](https://eslint.org/docs/user-guide/command-line-interface#options) to the command based on your needs. A useful example would be the `--fix` flag which can be used to fix all fixable errors in your code. 125 | ``` 126 | $ eslint --fix file.js 127 | ``` 128 |
129 | 130 | ESLint can be easily integrated with your build tool of choice. Here are some [plugins](https://eslint.org/docs/user-guide/integrations#build-tools) for some popular build tools. ESLint can also work with Continuous Integration tools like Travis, AppVeyor and CircleCI. Configuration is explained in this [example using Travis](https://medium.com/jason-feng/travis-ci-guide-bdc03b3dbc9d). You can also learn how to automate ESLint to [run every time you commit](https://medium.com/the-node-js-collection/why-and-how-to-use-eslint-in-your-project-742d0bc61ed7). 131 | 132 |
133 | 134 | ESLint also has [editor integrations](https://eslint.org/docs/user-guide/integrations#editors) such as error highlighting in many popular editors like Atom and WebStorm. 135 | 136 | [ESLint Package for Atom](https://atom.io/packages/linter-eslint) 137 | 138 |  139 | 140 | ## Plugins 141 | Custom rules can be [imported](https://eslint.org/docs/user-guide/configuring#using-the-configuration-from-a-plugin) via Plugins. You can [learn how to write your own](https://eslint.org/docs/developer-guide/working-with-plugins) or [use existing ones](https://www.npmjs.com/browse/keyword/eslintplugin) that are already published to npm. 142 | 143 | A popular plugin would be [eslint-plugin-lodash](https://github.com/wix/eslint-plugin-lodash) which is meant to enforce good practices and styling with the utility library [Lodash](https://lodash.com/). You can checkout other plugins [here](https://github.com/dustinspecker/awesome-eslint#plugins). 144 | 145 | ## Resources 146 | * [ESLint](https://eslint.org/) 147 | * [ESLint Github](https://github.com/eslint/eslint): You can contribute to the project but remember to read their [guidelines](https://eslint.org/docs/developer-guide/contributing/) first 148 | * [npm Developer Guide](https://docs.npmjs.com/misc/developers): If you're writing your own custom rules/configs, you need to know how to publish them on npm. 149 | * [Awesome ESLint](https://github.com/dustinspecker/awesome-eslint): repository for ESLint configs, plugins and other useful links. 150 | 151 |
152 |
--------------------------------------------------------------------------------
/contents/staticAnalysis/FindBugs.md:
--------------------------------------------------------------------------------
1 | 99 | 100 | **NOTE**: That specifying the rule option as "error" will result in the exit code of 1 when the rule is violated. If you wish for ESLint to alert you but not fail the linting, you may set it to "warn": 101 | 102 | ```js 103 | /* eslint brace-style: ["warn", "allman"] */ 104 | if (foo === bar) { // warning displayed but does not affect exit code 105 | return; 106 | } 107 | ``` 108 | Linting output: 109 | ```shell 110 | 1:10 warning Opening curly brace appears on the same line as controlling statement brace-style 111 | ``` 112 |
113 | 114 | If you are not sure what rules to add to your project, you can import [configurations](https://eslint.org/docs/developer-guide/shareable-configs) published by more experienced developers or organizations. 115 | 116 | Here are some popular style guide's configurations: 117 | * [Airbnb](https://www.npmjs.com/package/eslint-config-airbnb) 118 | * [Standard](https://www.npmjs.com/package/eslint-config-standard) 119 | * [Google](https://www.npmjs.com/package/eslint-config-google) 120 | 121 | At the same time, ESLint allows you to disable rules for [specific files and directories](https://eslint.org/docs/user-guide/configuring#ignoring-files-and-directories) or for [specific lines](https://eslint.org/docs/user-guide/configuring#disabling-rules-with-inline-comments). 122 | 123 | ## Usage 124 | Once configured, you can lint your files with [terminal commands](https://eslint.org/docs/user-guide/command-line-interface) and you can add [options](https://eslint.org/docs/user-guide/command-line-interface#options) to the command based on your needs. A useful example would be the `--fix` flag which can be used to fix all fixable errors in your code. 125 | ``` 126 | $ eslint --fix file.js 127 | ``` 128 |
129 | 130 | ESLint can be easily integrated with your build tool of choice. Here are some [plugins](https://eslint.org/docs/user-guide/integrations#build-tools) for some popular build tools. ESLint can also work with Continuous Integration tools like Travis, AppVeyor and CircleCI. Configuration is explained in this [example using Travis](https://medium.com/jason-feng/travis-ci-guide-bdc03b3dbc9d). You can also learn how to automate ESLint to [run every time you commit](https://medium.com/the-node-js-collection/why-and-how-to-use-eslint-in-your-project-742d0bc61ed7). 131 | 132 |
133 | 134 | ESLint also has [editor integrations](https://eslint.org/docs/user-guide/integrations#editors) such as error highlighting in many popular editors like Atom and WebStorm. 135 | 136 | [ESLint Package for Atom](https://atom.io/packages/linter-eslint) 137 | 138 |  139 | 140 | ## Plugins 141 | Custom rules can be [imported](https://eslint.org/docs/user-guide/configuring#using-the-configuration-from-a-plugin) via Plugins. You can [learn how to write your own](https://eslint.org/docs/developer-guide/working-with-plugins) or [use existing ones](https://www.npmjs.com/browse/keyword/eslintplugin) that are already published to npm. 142 | 143 | A popular plugin would be [eslint-plugin-lodash](https://github.com/wix/eslint-plugin-lodash) which is meant to enforce good practices and styling with the utility library [Lodash](https://lodash.com/). You can checkout other plugins [here](https://github.com/dustinspecker/awesome-eslint#plugins). 144 | 145 | ## Resources 146 | * [ESLint](https://eslint.org/) 147 | * [ESLint Github](https://github.com/eslint/eslint): You can contribute to the project but remember to read their [guidelines](https://eslint.org/docs/developer-guide/contributing/) first 148 | * [npm Developer Guide](https://docs.npmjs.com/misc/developers): If you're writing your own custom rules/configs, you need to know how to publish them on npm. 149 | * [Awesome ESLint](https://github.com/dustinspecker/awesome-eslint): repository for ESLint configs, plugins and other useful links. 150 | 151 |
7 |
8 | # FindBugs
9 |
10 | Author: [Xiao Pu](https://nus-oss.github.io/cs3281-website/students/AY1617S2/xiaoPu/xiaoPu-Resume.html), [Shradheya Thakre](https://github.com/tshradheya)
11 |
12 |
13 |
14 | * [Overview](#overview)
15 | * [Features](#features)
16 | * [Examples of Bugs That Can Be Found Using FindBugs](#examples-of-bugs-that-can-be-found-using-findbugs)
17 | * [Incorrectly Overriding Methods](#incorrectly-overriding-methods)
18 | * [Find Hash-Equals Mismatch](#find-hash-equals-mismatch)
19 | * [Return Value of Method Ignored](#return-value-of-method-ignored)
20 | * [Null Pointer Dereference](#null-pointer-dereference)
21 | * [How to Use It?](#how-to-use-it)
22 | * [Configuration](#configuration)
23 | * [Suppress Warnings](#suppress-warnings)
24 | * [Running](#running)
25 | * [SpotBugs - The Successor of Findbugs](#spotbugs-the-successor-of-findbugs)
26 | * [Advanced Topics](#advanced-topics)
27 | * [Resources](#resources)
28 |
29 |
30 | ## Overview
31 |
32 | FindBugs is a static analysis tool to find bugs in **Java** programs. It looks for 'bug patterns' in the code and signals possible violations. Potential errors are classified in four ranks:
33 | 1. scariest
34 | 1. scary
35 | 1. troubling
36 | 1. of concern
37 |
38 | This is a hint to the developer about their possible impact or severity. For example, the bug 'null pointer dereference' has the pattern — A program declares a non-nullable variable but assigns `null` to the variable somewhere and uses it later.
39 |
40 | ## Features
41 |
42 | The "bug patterns" can be divided into nine groups:
43 | 1. Bad practice
44 | 1. Correctness
45 | 1. Experimental
46 | 1. Internationalization
47 | 1. Malicious code vulnerability
48 | 1. Multithreaded correctness
49 | 1. Performance
50 | 1. Security
51 | 1. Dodgy code
52 |
53 | Refer to [FindBugs official documentation](http://findbugs.sourceforge.net/bugDescriptions.html) for a comprehensive list of bugs and the explanation of each bug.
54 |
55 | FindBugs analyses bytecode in compiled Java `.class` file and checks multiple files at the same time. This is unlike [CheckStyle](checkStyle.html) or [PMD](PMD.html) which can only check files one by one and analyse Java source code, allowing FindBugs to spot errors that would have been missed by CheckStyle and PMD. For example, one of the bug patterns in FindBugs is `RCN: Redundant nullcheck of value known to be non-null`. FindBugs will analyse all the assignments to a particular variable in the code base and then check whether the `nullcheck` for the variable is redundant or not.
56 |
57 | ## Examples of Bugs That Can Be Found Using FindBugs
58 |
59 | ### Incorrectly Overriding Methods
60 |
61 | Consider the following code:
62 |
63 | ``` java
64 | class Foo {
65 | //... data members ...
66 | //... methods ...
67 |
68 | //Overriding equals method - the wrong way
69 | public boolean equals(Foo foo) {
70 | //... logic ...
71 | }
72 | }
73 | ```
74 |
75 | In the above code, if `foo.equals()` method is called, the `equals()` method of `Object` class rather than `Foo` class will be called. This is due to the way the Java code resolves overloaded methods at compile-time. FindBugs warns the developer of possible cases when a class defines a co-variant version of the `equals()` or `compareTo()` method.
76 |
77 | ### Find Hash-Equals Mismatch
78 |
79 | The hashCode() and equals() method are called by many `Collection` based classes like - List, Maps, Sets, etc. FindBugs helps in finding problems when a class **overrides the `equals()` but not the `hashCode()` method or vice-versa**. Overriding only one of the `equals()` or `hashCode()` method can cause methods of Collection based classes to fail and hence FindBugs helps in reporting these errors at an early stage
80 |
81 | ### Return Value of Method Ignored
82 |
83 | FindBugs helps in finding places where your code has ignored the return value of method when it shouldn't have been.
84 |
85 | ``` java
86 | 1 String s = "bob";
87 | 2 s.replace('b', 'p');
88 | 3 boolean isCorrect = s.equals("pop"); //isCorrect is `false`
89 | ```
90 |
91 | In the above examples, one would assume that the variable `isCorrect` is assigned `true` because the `line 2` replaces `b` with `p`. However since strings are immutable, the `replace()` function actually returns a new string with updated value rather than updating the string the method is called on.
92 |
93 | Hence, `line 2` should be `String newString = s.replace('b', 'p'); //newString ="pop"`
94 |
95 | ### Null Pointer Dereference
96 |
97 | FindBugs looks for cases where a code path will or could cause a null pointer exception.
98 |
99 | ``` java
100 | 1 Person person = aMap.get("bob");
101 | 2 if (person != null) {
102 | 3 person.updateAccessTime();
103 | 4 }
104 | 5 String name = person.getName();
105 | ```
106 |
107 | In the above example, the `aMap` may or may not contain "bob", so FindBugs will report *possible* `NullPointerException` at `line 5`
108 |
109 | ## How to Use It?
110 |
111 | ### Configuration
112 |
113 | You can tell FindBugs which bug patterns to exclude and include by using [FilterFiles](http://findbugs.sourceforge.net/manual/filter.html). By default, if no filter files are provided, FindBugs will run all checks.
114 |
115 | ### Suppress Warnings
116 |
117 | You can use filter files with `exclude` option in FindBugs as discussed above to suppress warnings.
118 |
119 | In addition, you can also use `SuppressWarnings` [annotation](http://findbugs.sourceforge.net/manual/annotations.html) to filter out unwanted violations.
120 |
121 | ### Running
122 |
123 | There are several ways to run FindBugs.
124 |
125 | GUI:
126 |
127 | - [Using the FindBugs GUI](http://findbugs.sourceforge.net/manual/gui.html)
128 |
129 | Command Line:
130 |
131 | - [Command Line](http://findbugs.sourceforge.net/manual/running.html)
132 |
133 | Build Automation Tools:
134 |
135 | - [Ant Task](http://findbugs.sourceforge.net/manual/anttask.html)
136 | - [FindBugs Maven Plugin](https://gleclaire.github.io/findbugs-maven-plugin/)
137 | - [Gradle FindBugs](https://docs.gradle.org/current/userguide/findbugs_plugin.html)
138 |
139 | IDE Integration:
140 |
141 | - [Eclipse Integration](http://findbugs.sourceforge.net/manual/eclipse.html)
142 | - [IntelliJ Integration](https://plugins.jetbrains.com/plugin/3847-findbugs-idea)
143 | - [NetBeans Integration](https://netbeans.org/kb/docs/java/code-inspect.html)
144 |
145 | ## SpotBugs - The Successor of Findbugs
146 |
147 | On November 2016, FindBugs was [declared dead](https://mailman.cs.umd.edu/pipermail/findbugs-discuss/2016-November/004321.html) and [SpotBugs](https://spotbugs.github.io/) was [declared as its successor](https://mailman.cs.umd.edu/pipermail/findbugs-discuss/2017-September/004383.html) in September 2017.
148 |
149 | The current projects using `FindBugs` can make a shift to `SpotBugs` by following the [migration manual](https://spotbugs.readthedocs.io/en/latest/migration.html)
150 |
151 | ## Advanced Topics
152 |
153 | - [Data mining of bugs with FindBugs](http://findbugs.sourceforge.net/manual/datamining.html): The data for each analysis will be collected and you can use these statistics for data mining.
154 | - [Configure Analysis Properties](http://findbugs.sourceforge.net/manual/analysisprops.html#analysisproptable): You can define properties to configure options for some checks. For example, you can define the assertion methods in your project so that null pointer dereference bug detector will not raise violations if assertion methods are used.
155 | - [Use annotations](http://findbugs.sourceforge.net/manual/annotations.html): You can use annotations to indicate your intents so that FindBugs can issue warnings more appropriately.
156 | - [Writing custom detectors](https://www.ibm.com/developerworks/library/j-findbug2/): You can follow the tutorial step by step to write your customised detector.
157 |
158 | ## Resources
159 |
160 | - [FindBugs Official Website](http://findbugs.sourceforge.net): Official website of FindBugs. You can find more documentations here.
161 | - [An Evaluation of FindBugs](https://www.cs.cmu.edu/~aldrich/courses/654/tools/Sandcastle-FindBugs-2009.pdf): Analysis of FindBugs in 2009's version, some content may be outdated. Useful for understanding the benefits and drawbacks.
162 | - [Improve the quality of your code](https://www.ibm.com/developerworks/library/j-findbug1/): Some examples showing the bugs reported by FindBugs. You can get a rough idea of how FindBugs will help you in your project.
163 |
164 |
165 |
--------------------------------------------------------------------------------
/contents/staticAnalysis/PMD.md:
--------------------------------------------------------------------------------
1 |
7 |
8 | # PMD
9 |
10 | Author: [Xiao Pu](https://nus-oss.github.io/cs3281-website/students/AY1617S2/xiaoPu/xiaoPu-Resume.html)
11 |
12 |
13 |
14 | * [Overview](#overview)
15 | * [Features](#features)
16 | * [Limitation](#limitation)
17 | * [How to Use It?](#how-to-use-it)
18 | * [Download](#download)
19 | * [Configuration](#configuration)
20 | * [Suppress Warnings](#suppress-warnings)
21 | * [Command Line Usage](#command-line-usage)
22 | * [Integration With Build Automation Tools](#integration-with-build-automation-tools)
23 | * [Integration With IDEs](#integration-with-ides)
24 | * [Available Rulesets](#available-rulesets)
25 | * [How Does It Work?](#how-does-it-work)
26 | * [Customization](#customization)
27 | * [Resources](#resources)
28 |
29 |
30 | ## Overview
31 |
32 | PMD is a static analyser for Java, JavaScript, Salesforce.com Apex, PL/SQL, Apache Velocity, XML, XSL. The copy/paste-detector([CPD](http://pmd.sourceforge.net/pmd-4.3.0/cpd.html)), which helps to find duplicated code, is also included as an add-on in PMD.
33 |
34 | *Note that the links in this chapter is based on [PMD version 5.5.3](https://pmd.github.io/pmd-5.5.3/). You can check the latest version and documentation at its [GitHub Page](https://pmd.github.io/).*
35 |
36 | ## Features
37 | Rules in PMD represent patterns in code. PMD is supposed to check these rules(patterns) and signal violations to programmers. For example, the `OverrideBothEqualsAndHashcode` rule requires programmers to override both `public boolean Object.equals(Object other)`, and `public int Object.hashCode()`, or override neither.
38 |
39 | PMD supports checking rules for the following languages.
40 |
41 | - [Java](https://pmd.github.io/pmd-5.5.3/pmd-java/index.html) (includes [JSP](https://pmd.github.io/pmd-5.5.3/pmd-java/index.html) with [limitations](https://pmd.github.io/pmd-5.5.3/pmd-jsp/index.html))
42 | - [JavaScript](https://pmd.github.io/pmd-5.5.3/pmd-javascript/index.html)
43 | - [Apex](https://pmd.github.io/pmd-5.5.3/pmd-apex/index.html)
44 | - [PL/SQL](https://pmd.github.io/pmd-5.5.3/pmd-plsql/index.html)
45 | - [Velocity](https://pmd.github.io/pmd-5.4.1/pmd-vm/index.html)
46 | - [XML and XSL](https://pmd.github.io/pmd-5.5.3/pmd-xml/index.html)
47 |
48 | PMD doesn't support checking rules for the following languages. Only Copy/Paste Detector ([CPD](http://pmd.sourceforge.net/pmd-4.3.0/cpd.html)) is supported for them.
49 |
50 | - [C++](https://pmd.github.io/pmd-5.5.3/pmd-cs/index.html)
51 | - [C#](https://pmd.github.io/pmd-5.5.3/pmd-cpp/index.html)
52 | - [Fortran](https://pmd.github.io/pmd-5.5.3/pmd-fortran/index.html)
53 | - [Go](https://pmd.github.io/pmd-5.5.3/pmd-go/index.html)
54 | - [Groovy](https://pmd.github.io/pmd-5.5.3/pmd-groovy/index.html)
55 | - [Matlab](https://pmd.github.io/pmd-5.5.3/pmd-matlab/index.html)
56 | - [Objective-C](https://pmd.github.io/pmd-5.5.3/pmd-objectivec/index.html)
57 | - [Perl](https://pmd.github.io/pmd-5.5.3/pmd-perl/index.html) (only very basic CPD support)
58 | - [PHP](https://pmd.github.io/pmd-5.5.3/pmd-php/index.html)
59 | - [Python](https://pmd.github.io/pmd-5.5.3/pmd-python/index.html)
60 | - [Ruby](https://pmd.github.io/pmd-5.5.3/pmd-ruby/index.html)
61 | - [Swift](https://pmd.github.io/pmd-5.5.3/pmd-swift/index.html)
62 | - [Scala](https://pmd.github.io/pmd-5.5.3/pmd-scala/index.html)
63 |
64 | ## Limitation
65 | Limitations are almost the same as [CheckStyle](CheckStyle.html).
66 |
67 | - The examined code has to be compilable. The reason is described in [How does it work](#how-does-it-work) section.
68 | - Files will be examined one by one, which means you cannot check multiple files at the same time.
69 | - For example, you cannot determine the full inheritance hierarchy of a class as you need to examine the parent class while checking the child class.
70 |
71 | ## How to Use It?
72 |
73 | ### Download
74 | PMD can be run on both Windows and Linux/Unix operating system with the help of [Java JRE](https://www.oracle.com/technetwork/java/javase/overview/index.html) 1.7 or higher. Refer to [How to install PMD and CPD](https://pmd.github.io/pmd-5.5.3/usage/installing.html) for more details.
75 |
76 | ### Configuration
77 | You can configure PMD to only include the rules that your want (see [How to make a new rule set](https://pmd.github.io/pmd-5.5.3/customizing/howtomakearuleset.html)).
78 |
79 | ### Suppress Warnings
80 | PMD supports suppressing warnings in four ways:
81 |
82 | - Annotations
83 | - Comments
84 | - Violation Suppress Regex
85 | - Violation Suppress XPath
86 |
87 | The details are described [here](https://pmd.github.io/pmd-5.5.3/usage/suppressing.html).
88 |
89 | ### Command Line Usage
90 | PMD can be launched by using command line with various arguments. For details, please refer to [Running PMD via command line](https://pmd.github.io/pmd-5.5.3/usage/running.html).
91 |
92 | ### Integration With Build Automation Tools
93 | - [Ant task usage](https://pmd.github.io/pmd-5.5.3/usage/ant-task.html)
94 | - [The PMD Plugin in Gradle](https://docs.gradle.org/current/userguide/pmd_plugin.html)
95 | - [Maven 1 PMD plugin](https://pmd.github.io/pmd-5.5.3/usage/maven-plugin.html)
96 | - [Maven 2 PMD plugin](https://pmd.github.io/pmd-5.5.3/usage/mvn-plugin.html)
97 |
98 | ### Integration With IDEs
99 | PMD can be integrated with most of IDEs, inlcuding BlueJ, CodeGuide, Eclipse, eclipse-pmd, Emacs, Gel, IntelliJ IDEA, IntelliJ IDEA - QAPlug, JBuilder, JCreator, JDeveloper, JEdit, Maven, Maven 2, NetBeans, TextPad, WebLOgic Workshop 8.1.x. For instruction to integrate with those IDEs, please refer to [PMD integrations](https://pmd.github.io/pmd-5.5.3/usage/integrations.html).
100 |
101 | ## Available Rulesets
102 | There is not pre-defined rules set. You need to [define your rule sets](https://pmd.github.io/pmd-5.5.3/customizing/howtomakearuleset.html) by yourself.
103 |
104 | PMD has organised rules into different categories. For example, the rules for Java has been categorised into 26 sections, which will help you quickly find the rules that you want.
105 |
106 | You may refer to the [Features](#features) section to view the rules according to your languages and refer to [Configuration](#configuration) section to configure your rulesets.
107 |
108 | ## How Does it Work?
109 | PMD use [JavaCC](https://javacc.org) to parse your code to a [AST(Abstract Syntax Tree)](https://en.wikipedia.org/wiki/Abstract_syntax_tree) and visited it recursively ([more details](https://pmd.github.io/pmd-5.4.1/customizing/howitworks.html)). Thus, one of the requirements for PMD to work is that the code must be in valid Java syntax. You can view the syntax tree by using [bin/designer.bat](https://pmd.github.io/pmd-5.5.3/customizing/howtowritearule.html)
110 |
111 | ## Customization
112 | - [How to write a PMD rule](https://pmd.github.io/pmd-5.5.3/customizing/howtowritearule.html) (A quicker way to write rule sets [Using XPath in PMD](https://pmd.github.io/pmd-5.4.1/customizing/xpathruletutorial.html), [XPath Rule tutorial](https://pmd.github.io/pmd-5.4.1/customizing/xpathruletutorial.html))
113 | - [Add supports for new language](https://pmd.github.io/pmd-5.4.1/customizing/new-language.html) ([Add supports for CPD](https://pmd.github.io/pmd-5.4.1/customizing/cpd-parser-howto.html))
114 |
115 | ## Resources
116 | - [PMD](https://pmd.github.io/): PMD official website. You can download the latest version, view online documentation there.
117 | - [PMD Github](https://github.com/pmd/pmd): PMD GitHub page. You can contribute to the project or report bugs there.
118 | - [StackOverflow PMD](https://stackoverflow.com/questions/tagged/pmd): Question/Answer forum in StackOverflow for PMD. You can ask question related to the using of PMD.
119 |
120 |
--------------------------------------------------------------------------------
/contents/staticAnalysis/checkStyle.md:
--------------------------------------------------------------------------------
1 |
7 |
8 | # CheckStyle
9 |
10 | Author: [Xiao Pu](https://nus-oss.github.io/cs3281-website/students/AY1617S2/xiaoPu/xiaoPu-Resume.html)
11 |
12 |
13 |
14 | * [Overview](#overview)
15 | * [Features](#features)
16 | * [Limitations](#limitations)
17 | * [How to Use It?](#how-to-use-it)
18 | * [Configuration](#configuration)
19 | * [Suppress Warnings](#suppress-warnings)
20 | * [Running](#running)
21 | * [Available Configurations](#available-configurations)
22 | * [How Does It Work?](#how-does-it-work)
23 | * [Customisation](#customisation)
24 | * [Resources](#resources)
25 |
26 |
27 | ## Overview
28 |
29 | CheckStyle is a static analyser for **Java**. It can be used to assist developers in [static analysis](intro.html) process.
30 |
31 | ## Features
32 | CheckStyle will examine code based on different rules(or checks).
33 |
34 | According to the [checks list](http://checkstyle.sourceforge.net/checks.html) provided by CheckStyle, the checks(rules) can be divided into 14 sections.
35 |
36 | - Annotations
37 | - Block Checks
38 | - Class Design
39 | - Coding
40 | - Headers
41 | - Imports
42 | - Javadoc Comments
43 | - Metrics
44 | - Miscellaneous
45 | - Modifiers
46 | - Naming Conventions
47 | - Regexp
48 | - Size Violations
49 | - Whitespace
50 |
51 | ## Limitations
52 | As described [here](http://checkstyle.sourceforge.net/writingchecks.html#Limitations), there are several limitations in CheckStyle.
53 |
54 | - The code must be written in ASCII characters only.
55 | - The examined code has to be compilable. The reason is described in [How does it work](#how-does-it-work) section.
56 | - Files will be examined one by one, which means you cannot check multiple files at the same time.
57 | - For example, you cannot determine the full inheritance hierarchy of a class as you need to examine the parent class while checking the child class.
58 |
59 | ## How to Use It?
60 |
61 | ### Configuration
62 | CheckStyle uses a [configuration file](http://checkstyle.sourceforge.net/config.html) to know all the rules that it is supposed to check.
63 |
64 | ### Suppress Warnings
65 | CheckStyle supports suppressing warnings in four ways:
66 |
67 | - [Annotations](http://checkstyle.sourceforge.net/config_filters.html#SuppressWarningsFilter)
68 | - [Comments](http://checkstyle.sourceforge.net/config_filters.html#SuppressionCommentFilter)
69 | - [File Filter](http//checkstyle.sourceforge.net/config_filefilters.html#BeforeExecutionExclusionFileFilter)
70 | - [Configuration File](http://checkstyle.sourceforge.net/config_filters.html#SuppressionFilter)
71 |
72 | ### Running
73 | There are several ways to run CheckStyle.
74 |
75 | Command Line:
76 |
77 | - [Command Line](http://checkstyle.sourceforge.net/cmdline.html)
78 |
79 | Build Automation Tools:
80 |
81 | - [Ant Task](http://checkstyle.sourceforge.net/anttask.html)
82 | - [Maven Checkstyle](https://maven.apache.org/plugins/maven-checkstyle-plugin/)
83 | - [Gradle Checkstyle](https://docs.gradle.org/current/userguide/checkstyle_plugin.html)
84 |
85 | IDE Integration:
86 |
87 | - [Eclipse Integration](http://eclipse-cs.sourceforge.net/#!/)
88 | - [IntelliJ Integration](https://plugins.jetbrains.com/idea/plugin/1065-checkstyle-idea)
89 |
90 | ## Available Configurations
91 | There are two widely used configurations: [Sun Code Conversions](https://www.oracle.com/technetwork/java/javase/documentation/codeconvtoc-136057.html) and [Google Java Style](http://checkstyle.sourceforge.net/reports/google-java-style.html). Some common rules are already included in these configurations.
92 |
93 | ## How Does It Work?
94 | CheckStyle will use [ANTLR](https://www.antlr.org) to parse your code into a [AST(Abstract Syntax Tree)](https://en.wikipedia.org/wiki/Abstract_syntax_tree) and visit it in a [DFS(Depth First Search)](https://en.wikipedia.org/wiki/Depth-first_search) pattern to check violations. Thus, it is necessary to make the code compilable in order for the ANTLR to work. You can view the syntax tree using [CheckStyle Grammar Tree Viewer](http://checkstyle.sourceforge.net/writingchecks.html#The_Checkstyle_SDK_Gui)
95 |
96 | ## Customisation
97 | - [Writing Checks](http://checkstyle.sourceforge.net/writingchecks.html) (I want to write my own check for Java code.)
98 | - [Writing Javadoc Checks](http://checkstyle.sourceforge.net/writingjavadocchecks.html) (I want to enforce new rules for writing Javadoc header comment.)
99 | - [Writing Filters](http://checkstyle.sourceforge.net/writingfilters.html) (I will do something when violations are found.)
100 | - [Writing File Filters](http://checkstyle.sourceforge.net/writingfilefilters.html) (I want to check the rules against specific files.)
101 | - [Writing Listeners](http://checkstyle.sourceforge.net/writinglisteners.html) (I want different notifications (verbose printer, sending emails, etc) when violations are thrown.)
102 |
103 | ## Resources
104 | - [CheckStyle](http://checkstyle.sourceforge.net/): CheckStyle official website. You can download the latest version, view online documentation there.
105 | - [CheckStyle Github](https://github.com/checkstyle/checkstyle): CheckStyle GitHub page. You can contribute to the project or report bugs there.
106 | - [StackOverflow CheckStyle](https://stackoverflow.com/questions/tagged/checkstyle): Question/Answer forum in StackOverflow for CheckStyle. You can ask question related to the using of CheckStyle.
107 |
108 |
--------------------------------------------------------------------------------
/contents/staticAnalysis/intro.md:
--------------------------------------------------------------------------------
1 |
7 |
8 | # Introduction to Static Analysis
9 |
10 | Author: [Xiao Pu](https://nus-oss.github.io/cs3281-website/students/AY1617S2/xiaoPu/xiaoPu-Resume.html)
11 |
12 |
13 |
14 | * [Overview](#overview)
15 | * [Why Static Analysis?](#why-static-analysis)
16 | * [Analyse Thoroughly](#analyse-thoroughly)
17 | * [Find Potential Bugs Early](#find-potential-bugs-early)
18 | * [Enforce a Standard in Project](#enforce-a-standard-in-project)
19 | * [Improve Code Quality](#improve-code-quality)
20 | * [Limitation of Static Analysis](#limitation-of-static-analysis)
21 | * [False Positives](#false-positives)
22 | * [Cannot Catch Error Introduced in Runtime Environment](#cannot-catch-error-introduced-in-runtime-environment)
23 | * [How to Do Static Analysis (Static Analysis Tools)](#how-to-do-static-analysis-static-analysis-tools)
24 | * [References](#references)
25 |
26 |
27 | ## Overview
28 |
29 | Static analysis is the process of analysing computer programme **without** executing the code. This practice is often used to ensure that codes follow certain structures or standards (e.g coding standards).
30 |
31 | It is possible to do static analysis manually, but there are automated tools(static analysers) that can assist developers in this process.
32 |
33 | ## Why Static Analysis?
34 |
35 | ### Analyse Thoroughly
36 | In some situations, it is impossible to achieve 100% test coverage. There will be some sections in the code that are not covered by test cases, which may result in bugs ([see](#find-potential-bugs-early) how static analysis will help you find bugs). In static analysis, all the related files/codes will be analysed.
37 |
38 | ### Find Potential Bugs Early
39 | Static analysis can find bugs before the execution. For example, some programmers may forget to add `break` statement in `switch` statement.
40 |
41 | ``` java
42 | switch(colour) {
43 | case 'blue':
44 | value = 1;
45 | case 'green':
46 | value = 2;
47 | }
48 | ```
49 | Static analysis tools will automatically alert the programmers about the potential problems/bugs.
50 |
51 | ### Enforce a Standard in Project
52 | Many projects enforce certain coding standards. For example, some project require the following format for `if` statement.
53 |
54 | ``` java
55 | if (condition) {
56 | // true
57 | } else {
58 | // false
59 | }
60 | ```
61 |
62 | While others enforce the following standard:
63 |
64 | ``` java
65 | if (condition) {
66 | // true
67 | }
68 | else {
69 | // false
70 | }
71 | ```
72 | Such standards can be configured in static analysis tools and the tools will help you enforce the standards.
73 |
74 | ### Improve Code Quality
75 | Static analysis will pick up common pitfalls in coding and suggest changes to help you improve your code quality. For example, for the following Java code:
76 |
77 | ``` java
78 | if (isConditionTrue()) {
79 | return true;
80 | } else {
81 | return false;
82 | }
83 | ```
84 | Majority of static analysis tools will point out that this can be simplified to:
85 |
86 | ``` java
87 | return isConditionTrue();
88 | ```
89 |
90 | ## Limitation of Static Analysis
91 |
92 | ### False Positives
93 | Since static analysis tools only recognize patterns, there might be false positives introduced.
94 |
95 | ``` java
96 | try {
97 | // logic part
98 | } catch (Throwable t) {
99 | // alert user
100 | }
101 | ```
102 | For example, the above code will catch any `Throwable` object and alert users that a fatal error has occurred in the system. In many static analysis tools, catching `Throwable` is regarded as a bad practice and thus the tools will prompt the error to developers. However, in this case, we want to provide a friendly alert for system crash instead of showing an ugly stack track. It is acceptable to catch `Throwable` and thus the violation detected by static analysis tools is a false positive.
103 |
104 | **Solution**: Many static analysis tools provide ways to suppress the warnings. For example, in [PMD](PMD.html) (a static analysis tool), `@SuppressWarnings` annotation can be used. In this case,
105 |
106 | ``` java
107 | @SuppressWarnings("PMD.AvoidCatchingThrowable") // used as fallback
108 | ```
109 | is the correct way to suppress warnings.
110 |
111 | ### Cannot Catch Error Introduced in Runtime Environment
112 | Since static analysis is done without executing the programme. Some vulnerabilities that are introduced in the runtime cannot be caught. Thus, you should **not** merely depend on static analysis tools to find bugs. Comprehensive test cases are also needed to verify the functionalities in logic, UI and etc.
113 |
114 | ## How to Do Static Analysis (Static Analysis Tools)
115 | There are several static analysis tools that can be used to assist the process.
116 |
117 | - [List of tools for static code analysis - Wikipedia](https://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis): You can find static analysis tools that supports the language you use.
118 | - [Codacy - Automated code reviews & code analytics](https://www.codacy.com/): A code reviews tool that has integrated different static analysis tools. It can show data or statistics reported by different static analysis tools for each commit.
119 |
120 | Here, we will introduce several well-known ones in detail. You can click the hyperlinks to look through them. We organised them by languages.
121 |
122 | - Java
123 | - [CheckStyle](checkStyle.html)
124 | - [PMD](PMD.html)
125 | - [FindBugs](FindBugs.html)
126 |
127 | - JavaScript
128 | - [eslint](ESLint.html)
129 |
130 | ## References
131 |
132 | - [Why Static Code Analysis is Important?](https://javarevisited.blogspot.sg/2014/02/why-static-code-analysis-is-important.html)
133 |
134 |
135 |
--------------------------------------------------------------------------------
/contents/testing/images/tddSteps.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/testing/images/tddSteps.jpg
--------------------------------------------------------------------------------
/contents/testing/integration-test/big-bang-integration.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/testing/integration-test/big-bang-integration.png
--------------------------------------------------------------------------------
/contents/testing/integration-test/sample-architecture.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/testing/integration-test/sample-architecture.png
--------------------------------------------------------------------------------
/contents/testing/security-testing/Sec-Test-Fig-1.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/testing/security-testing/Sec-Test-Fig-1.png
--------------------------------------------------------------------------------
/contents/testing/security-testing/Sec-Test-Fig-2.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/testing/security-testing/Sec-Test-Fig-2.png
--------------------------------------------------------------------------------
/contents/uix/accessibility/alt-tag.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/uix/accessibility/alt-tag.png
--------------------------------------------------------------------------------
/contents/uix/accessibility/bg-new.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/uix/accessibility/bg-new.png
--------------------------------------------------------------------------------
/contents/uix/accessibility/bg-old.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/uix/accessibility/bg-old.png
--------------------------------------------------------------------------------
/contents/uix/accessibility/fb-achromatopsia.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/uix/accessibility/fb-achromatopsia.png
--------------------------------------------------------------------------------
/contents/uix/accessibility/fb-deuteranopia.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/uix/accessibility/fb-deuteranopia.png
--------------------------------------------------------------------------------
/contents/uix/accessibility/fb-more-indicators.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/uix/accessibility/fb-more-indicators.png
--------------------------------------------------------------------------------
/contents/uix/accessibility/fb-normal.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/uix/accessibility/fb-normal.png
--------------------------------------------------------------------------------
/contents/uix/accessibility/lh-audit.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/uix/accessibility/lh-audit.png
--------------------------------------------------------------------------------
/contents/uix/accessibility/tc-blurred.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/uix/accessibility/tc-blurred.png
--------------------------------------------------------------------------------
/contents/uix/accessibility/tc-normal.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/uix/accessibility/tc-normal.png
--------------------------------------------------------------------------------
/contents/uix/accessibility/website_protanopia.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/uix/accessibility/website_protanopia.png
--------------------------------------------------------------------------------
/contents/uix/accessibility/website_trace.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/contents/uix/accessibility/website_trace.png
--------------------------------------------------------------------------------
/contents/zeromq/zeromq.md:
--------------------------------------------------------------------------------
1 |
7 |
8 | # Introduction to ZeroMQ
9 |
10 | **Author: [Tan Yuanhong](https://github.com/le0tan)**
11 |
12 | Reviewers: [Tiu Wee Han](https://github.com/tiuweehan), [Tejas Bhuwania](https://github.com/Tejas2805)
13 |
14 |
15 |
16 | * [What is ZeroMQ?](#what-is-zeromq)
17 | * [What is Asynchronous Messaging and Message Queue?](#what-is-asynchronous-messaging-and-message-queue)
18 | * [Why ZeroMQ?](#why-zeromq)
19 | * [Ease of Use](#ease-of-use)
20 | * [Multi-Transport](#multi-transport)
21 | * [High Performance](#high-performance)
22 | * [Getting Started](#getting-started)
23 |
24 |
25 | ## What is ZeroMQ?
26 |
27 | [ZeroMQ](https://zeromq.org/) is a high-performance asynchronous messaging library. It provides a message queue (MQ) for all processes communicating using the library, without a dedicated message broker - thus the *zero* in the name of *ZeroMQ*.
28 |
29 | ### What is **Asynchronous Communication** and **Message Queue**?
30 |
31 | **Asynchronous** (or async, in short) **communication** means the two (or more than two) ends of the communication don't need to be occupied at the same time to send messages. You may consider making a phone-call synchronous and sending an email async. And **message queue** is one of the async communication protocols that puts the messages in a queue, stores them temporarily until the recipient retrieves them.
32 |
33 | Notice the word **"protocol"** used in the description above - communication protocol remains the same despite the actual medium of transport. It doesn't matter if your message is sent and received within the process between threads, or between processes, or between different physical machines across the network. It merely specifies how communicators are supposed to encode/send and decode/receive their messages.
34 |
35 | ## Why ZeroMQ?
36 |
37 | **Essentially**: By adopting ZeroMQ in your concurrent application, you're less likely to have (or at least have less) performance bottleneck on inter-process/thread/machine communication, while still being able to write communication logic in familiar socket APIs.
38 |
39 | ### Ease of Use
40 |
41 | Most operating systems provides system calls or equivalent support for message queues (e.g. [POSIX mq](http://man7.org/linux/man-pages/man7/mq_overview.7.html), [Win32 winmsg](https://docs.microsoft.com/en-us/windows/win32/winmsg/about-messages-and-message-queues)). However, the way they operate is far from high-level and making it directly interact with underlying OS APIs could hurt the scalability of your application as you may, in the future, find yourself in need of distributing the workload to different machines.
42 |
43 | ZeroMQ looks like an embeddable networking library (read: easy to use) but acts as a concurrency framework (read: high performance). Using Python's built-in socket library as an example, if we are to write a simple TCP server in Python , it looks like this:
44 |
45 | ```python
46 | import socket
47 |
48 | s = socket.socket() # Create the socket
49 | s.bind(('', 12345)) # Bind socket to a port
50 | s.listen(5) # Listen on the socket
51 |
52 | # Communicate using the socket
53 | while True:
54 | c, addr = s.accept() # Receive input
55 | c.send('Connected. Now closing connection.') # Send response
56 | c.close()
57 | ```
58 |
59 | Implementing the same functionality in ZeroMQ, it looks like this:
60 |
61 | ```python
62 | import time
63 | import zmq
64 |
65 | context = zmq.Context()
66 | socket = context.socket(zmq.REP) # Create the socket
67 | socket.bind("tcp://*:12345") # Bind socket to a port
68 |
69 | # Communicate using the socket
70 | while True:
71 | message = socket.recv() # Receive input
72 | socket.send(b"Connected. Now closing connection.") # Send response
73 | ```
74 |
75 | The similarity between the two is rather obvious - as long as you have experience with socket programming, using ZeroMQ to make communication between your concurrent processes possible and efficient should be no problem at all.
76 |
77 | ### Multi-Transport
78 |
79 | Unlike system calls, ZeroMQ supports various transports like in-process, inter-process, TCP, and multicast. And you don't need to change your code to switch underlying transport as ZeroMQ abstracted the details away, leaving you a reusable messaging layer. All you need to do is to update the definition (i.e. `context.socket(...)` and `socket.bind(...)`) of the socket and you're ready to scale your application from multi-core concurrency to a cluster of machines.
80 |
81 | ### High Performance
82 |
83 | Unlike networking libraries , ZeroMQ performs well enough to "be the fabric for clustered products". To give you a rough idea of how lightweight ZeroMQ is, the C++ source code for the ZeroMQ core engine [libzmq](https://github.com/zeromq/libzmq/tree/master/src) is only 1.5 megabytes. Moreover, in the [benchmark](http://wiki.zeromq.org/area:results) for ZeroMQ, the latency is measured in microseconds and the throughput is measured in million messages per second. Even CERN [approved](http://cds.cern.ch/record/1391410/files/CERN-ATS-2011-196.pdf?version=1) the performance of ZeroMQ in the setting of operating accelerators.
84 |
85 | ## Getting Started
86 |
87 | Developing software for concurrent operation is by no means entry-level content. Luckily, the creator(s) of ZeroMQ has written a guide explaining not only the basic use cases of ZeroMQ, but also the ideas behind their design choices and probably most importantly, why they feel necessary to develop a library like this.
88 |
89 | [ZeroMQ - The Guide](http://zguide.zeromq.org/)
90 |
91 | If you simply want to get started soon and understand the details as you adopt the library, you can download the library in your favorite language [here](https://zeromq.org/download/), then follow the corresponding examples written in the language of your choice.
92 |
93 | [ZeroMQ - Get Started](https://zeromq.org/get-started/)
94 |
95 | ## Acknowledgements
96 |
97 | In addition to the links mentioned above, here are the links used in the writing of this article:
98 |
99 | - [Socket Programming in Python](https://www.tutorialspoint.com/python/python_networking.htm)
100 | - [Build Algorithmic Trading Strategies with Python & ZeroMQ: Part 1](https://www.youtube.com/watch?v=4MxjFTQHTfw&t)
101 | - [Why ZeroMQ?](https://www.youtube.com/watch?v=_JCBphyciAs)
--------------------------------------------------------------------------------
/contributing.md:
--------------------------------------------------------------------------------
1 |
2 | title: "Contributing to this Project"
3 | layout: "landing.md"
4 |
5 |
6 |
2 | title: "A Student's guide to Software Engineering Tools & Techniques"
3 | layout: "landing.md"
4 |
5 |
6 |
7 |
8 | # Contributing to this Project
9 |
10 | We welcome contributions to this project.
11 |
12 | ## How to Contribute
13 |
14 | You can report errors, send suggestions, ask questions. Here is how you can contact us.
15 |
16 | You can also add new topics or update existing content by following these steps:
17 | * [Fork this repo](https://github.com/se-edu/learningresources/fork) and clone it to your Computer.
18 | * Install [MarkBind](https://markbind.github.io) in your Computer.
19 | * Post in [our issue tracker](https://github.com/nus-oss/learningresources/issues) an outline of your proposed contribution. Wait for our response.
20 | * After receiving the go-ahead from us, send your proposed contribution as a PR.
21 |
22 |
23 |
25 |
26 | ## Guidelines for Writing Content
27 |
28 | * **Do include a table of contents.** A table of contents tells the reader what to expect, such as how long the article is and what content is covered. It also serves as a reference that makes it easier to jump around from section to section. In our case, we only show the table of contents for mobile users and for printing as Markbind already provides a navigation panel on the right side of every article. Feel free to refer to this [pull request](https://github.com/se-edu/learningresources/pull/185) to see how to implement this behavior.
29 | * **Do use Title Case for headings.** Using title case for headings gives articles a more polished, professional look that is more pleasing to the eye. If you are unfamiliar with the rules of title casing, you may refer to the following [website](https://apastyle.apa.org/style-grammar-guidelines/capitalization/title-case) or use online title case converters such as the one found [here](https://titlecase.com/).
30 | * **It is not a normal textbook. It’s a collection of articles**, each containing a **_curated_** list of **_tried-and-tested_** learning resources, **_organized_** in a meaningful sequence, with **_commentary_**.
31 | * The article can be on any technical topic useful for SE students.
32 | * Don’t try to follow the structure of normal textbooks.
33 | * There’s no need to use a very formal tone that you see in some text books. Instead, prefer an informal tone you would use in a blog post or an stackoverflow question.
34 | * The article should point to the _best_ (in your opinion) external learning resources for the topic. That means it should not include resources that you did not try yourself.
35 | * ****The suggested structure**** for an Article
36 | * **What is X?**: Explain how X fits into the big picture of SE. Describe it relative to topics the reader is likely to know.
37 | * **Why X?**: Motivate the reader to learn about X. Describe benefits of X to make the reader interested in X. Try to give a balanced view of by mentioning also WHY NOT X i.e., mention both advantages and disadvantages.
38 | * **How does X work?** How is X being used? This is a simple high-level overview of the tool to give the reader some concrete sense of X (as opposed to limiting to an entirely abstract description). It's useful to give concrete examples such as code examples. ==Do not try to 'teach' how to use the X== (assuming your in the style of a tutorial. If the tool is worth learning, there must be good tutorials about it already.
39 | * **How to Get Started with X?**: Provide a learning path for the reader. Try to give one good learning path rather than many random resources.
40 | * **Where to Go from Here?**: Give more resources. Instead of listing a lot of links, provide a brief summary of what value each resource can provide the reader.
41 | * **Do not assume a lot of prior knowledge on the part of the reader.** The less prior knowledge you assume, the wider your reader base becomes. The article is aimed at typical SE students, but not necessarily SE students in your school. Just because it is covered in a core module in your school does not mean it will be known to every reader.
42 | * **Do not re-invent the wheel by writing a lot of original content.** Instead, give a brief high-level view and point to other existing resources that you recommend the reader to use.
43 | original source.
44 | * Note that ==the reader should still get a complete and useful picture== even if she does not refer any of the given links. i.e., make the content self-contained and grounded/concrete instead of too abstract to be of much use. To that end, you may repeat/adapt content snippets from other resources (instead of simply giving a link) with proper attribution.
45 | * If you reuse assets from other sources %%(e.g., diagrams)%%, remember to ==cite the original source==.
46 | * **Stay an _independent_ observer**. When writing about a tool/technique that has competing alternatives, the article will have more credibility if you write it from the point of view of an independent observer. For example, avoid unsubstantiated marketing claims e.g., _X is the best tool for doing Y_. Instead, you can cite quotes by other credible sources or from the tool/technique itself. See the two examples below:
47 | > Tool X claims to have the fastest performance [[source]()].
48 |
49 | > [Tool X's website]() has the following to say about its performance:
50 | >> Tool X has the best performance of its class. ...
51 | * **Prefer visuals rather than long paragraphs.**
52 | * **Some of the existing articles might not follow the above guidelines** (the guidelines emerged over time). Feel free to revise existing content to fit the guidelines.
53 |
54 |
55 |
56 | A fairly decent example of applying the above guidelines can be found in the article [_Introduction to Go_]({{ baseUrl }}/contents/go/Go.html).
57 |
58 |
59 |
60 |
61 |
62 |
--------------------------------------------------------------------------------
/css/main.css:
--------------------------------------------------------------------------------
1 | mark {
2 | background-color: #ff0;
3 | border-radius: 5px;
4 | padding-top: 0;
5 | padding-bottom: 0;
6 | }
7 |
8 | .indented {
9 | padding-left: 20px;
10 | }
11 |
12 | .theme-card img {
13 | width: 100%;
14 | }
15 |
16 | /* Scrollbar */
17 |
18 | .slim-scroll::-webkit-scrollbar {
19 | width: 5px;
20 | }
21 |
22 | .slim-scroll::-webkit-scrollbar-thumb {
23 | background: #808080;
24 | border-radius: 20px;
25 | }
26 |
27 | .slim-scroll::-webkit-scrollbar-track {
28 | background: transparent;
29 | border-radius: 20px;
30 | }
31 |
32 | .slim-scroll-blue::-webkit-scrollbar {
33 | width: 5px;
34 | }
35 |
36 | .slim-scroll-blue::-webkit-scrollbar-thumb {
37 | background: #00b0ef;
38 | border-radius: 20px;
39 | }
40 |
41 | .slim-scroll-blue::-webkit-scrollbar-track {
42 | background: transparent;
43 | border-radius: 20px;
44 | }
45 |
46 | /* Layout containers */
47 |
48 | #flex-body {
49 | display: flex;
50 | flex: 1;
51 | align-items: start;
52 | }
53 |
54 | #content-wrapper {
55 | flex: 1;
56 | margin: 0 auto;
57 | min-width: 0;
58 | max-width: 1000px;
59 | padding: 0.8rem 20px 0 20px;
60 | transition: 0.4s;
61 | -webkit-transition: 0.4s;
62 | }
63 |
64 | #site-nav,
65 | #page-nav {
66 | position: sticky;
67 | top: 0;
68 | flex: 0 0 auto;
69 | padding-top: 1rem;
70 | max-width: 300px;
71 | width: 300px;
72 | }
73 |
74 | #site-nav {
75 | display: flex;
76 | flex-direction: column;
77 | border-right: 1px solid lightgrey;
78 | padding-bottom: 20px;
79 | z-index: 999;
80 | max-height: 100vh;
81 | }
82 |
83 | .site-nav-top {
84 | margin: 0.8rem 0;
85 | padding: 0 12px 12px 12px;
86 | }
87 |
88 | .nav-component {
89 | overflow-y: auto;
90 | }
91 |
92 | #page-nav {
93 | display: block;
94 | border-left: 1px solid lightgrey;
95 | max-height: 90vh;
96 | }
97 |
98 | #page-nav .nav-component {
99 | max-height: 90vh;
100 | }
101 |
102 | @media screen and (max-width: 1299.98px) {
103 | #page-nav {
104 | display: none;
105 | }
106 | }
107 |
108 | /* Bootstrap medium(md) responsive breakpoint */
109 | @media screen and (max-width: 991.98px) {
110 | #site-nav {
111 | display: none;
112 | }
113 |
114 | footer {
115 | margin-left: 60px;
116 | }
117 | }
118 |
119 | /* Bootstrap small(sm) responsive breakpoint */
120 | @media (max-width: 767.98px) {
121 | #content-wrapper {
122 | padding: 0 10px;
123 | }
124 | }
125 |
126 | /* Bootstrap extra small(xs) responsive breakpoint */
127 | @media screen and (max-width: 575.98px) {
128 | #site-nav {
129 | display: none;
130 | }
131 |
132 | footer {
133 | margin-left: 0;
134 | }
135 | }
136 |
137 | /* Hide site navigation when printing */
138 | @media print {
139 | #site-nav {
140 | display: none;
141 | }
142 |
143 | #page-nav {
144 | display: none;
145 | }
146 | }
147 |
148 | /* ----------- site-specific styles ------------------- */
149 |
150 | .website-content {
151 | margin: 12px auto 40px auto;
152 | max-width: 1000px;
153 | overflow: hidden;
154 | width:90%;
155 | }
156 |
157 | .indented-less {
158 | padding-left: 10px;
159 | }
160 |
161 | .indented {
162 | padding-left: 20px;
163 | }
164 |
165 | .indented-level2 {
166 | padding-left: 40px;
167 | }
168 |
169 | .indented-level3 {
170 | padding-left: 60px;
171 | }
172 |
--------------------------------------------------------------------------------
/favicon.ico:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/se-edu/learningresources/20e849f7ad3f2d2019fd356535b07a75ac9b35a1/favicon.ico
--------------------------------------------------------------------------------
/index.md:
--------------------------------------------------------------------------------
1 |
7 |
8 | This [SE-EDU](https://se-edu.github.io) sub-project is **a collection of articles that introduce various tools and techniques that may be useful for Software Engineering students**.
9 |
10 | Here is the list of articles:
11 |
12 |
13 |
14 | {% include "_markbind/layouts/toc.md" %}
15 |
16 |
17 |
18 |
--------------------------------------------------------------------------------
/site.json:
--------------------------------------------------------------------------------
1 | {
2 | "baseUrl": "/learningresources",
3 | "titlePrefix": "SE-EDU/LearningResources",
4 | "style": {
5 | "codeTheme": "light"
6 | },
7 | "ignore": [
8 | "_markbind/logs/*",
9 | "_site/*",
10 | "site.json",
11 | "*.md"
12 | ],
13 | "pages": [
14 | {
15 | "glob": "**/*.md"
16 | }
17 | ],
18 | "deploy": {
19 | "message": "Site Update."
20 | }
21 | }
22 |
--------------------------------------------------------------------------------