├── CONTRIBUTING.md ├── README.md ├── building-web-apps-together ├── 01-title-intro.md ├── 02-code-quality.md ├── 03-open-source.md ├── 04-documentation.md ├── 05-respect.md ├── 06-conclusion.md ├── README.md ├── contributors.md └── img │ └── device-lab.jpg ├── humans.txt ├── img └── Hippocratic_oath.jpg ├── preface.md ├── web-apps-for-everyone ├── 01-title-intro.md ├── 02-progressive-enhancement.md ├── 03-accessibility.md ├── 04-inclusive-forms.md ├── 05-conclusion.md ├── README.md ├── contributors.md └── img │ ├── browser-chart.png │ ├── facebook-gender.png │ ├── facebook-signup.png │ ├── google-gender-options.png │ ├── google-gender-other.png │ ├── link-perception.png │ ├── mojibake-tweets.png │ ├── mojibake-tweets.sketch │ ├── name-fields.png │ ├── name-fields.sketch │ ├── name-what.png │ ├── name-what.sketch │ ├── network-connectivity-chrome.png │ ├── placeholder-field.png │ ├── placeholder-field.sketch │ ├── progressive-enhancement-gradient.png │ ├── russian-post.jpg │ └── twitter-no-gender.png ├── web-apps-privacy-security ├── 01-title-intro.md ├── 02-privacy.md ├── 03-https.md ├── 04-security.md ├── 05-data-export.md ├── 06-conclusion.md ├── README.md ├── contributors.md └── img │ ├── OAuth-access-request.png │ ├── OAuth-signin.png │ ├── U2F-USB-Token.jpg │ ├── atlantic.png │ ├── cookie-tracking.png │ ├── http.png │ ├── https-url-example.png │ ├── ico-cookie.png │ ├── medium-dnt.png │ ├── no-https-url-example.png │ └── securityheaders.png └── web-apps-that-work-everywhere ├── 01-title-intro.md ├── 02-url.md ├── 03-responsive-design.md ├── 04-web-performance.md ├── 05-offline.md ├── 06-conclusion.md ├── README.md ├── contributors.md └── img ├── average-page-weight.png ├── filmstrip.png ├── first-website.png ├── load-time.png ├── network-tab.png ├── st-506.jpg ├── sw-network.png ├── sw-sources.png ├── sw-throttle.png ├── throttle.png └── waterfall.png /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing 2 | 3 | I welcome and love pull requests from everyone. By participating in this project, you agree to abide by the [code of conduct]. 4 | 5 | [code of conduct]: http://todogroup.org/opencodeofconduct/#EthicalWeb/adamdscott@protonmail.com 6 | 7 | Fork, clone the repo, and make your change: 8 | 9 | ``` 10 | git clone git@github.com:your-username/ethical-web-dev.git 11 | ``` 12 | 13 | If you'd like to be added as a contributor to this project, update the [humans.txt][humans] file. 14 | 15 | [humans]: https://github.com/ascott1/ethicalweb.org/blob/master/humans.txt 16 | 17 | Push to your fork and [submit a pull request][pr]. 18 | 19 | [pr]: https://github.com/ascott1/ethicalweb.org/compare/ 20 | 21 | At this point you're waiting on me. I try to comment on pull requests 22 | within three days (and, typically, one business day). I may suggest 23 | some changes, improvements, or alternatives. 24 | 25 | By contributing to this repo you acknowledge that: 26 | 27 | 1. Your contribution may be published in the final O'Reilly version of the title, for which you will receive credit in the book but no additional compensation. 28 | 2. Your contribution may also be published on the ethicalweb.org site. 29 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Ethical Web Development 2 | 3 | As web developers, we are responsible for shaping the experiences of user's online lives. By making choices that are ethical and user-centered, we create a better web for everyone. 4 | 5 | In Tim Berners-Lee's [The World Wide Web: A very short personal history](https://www.w3.org/People/Berners-Lee/ShortHistory.html), he states: 6 | 7 | > The dream behind the Web is of a common information space in which we communicate by sharing information. Its universality is essential: the fact that a hypertext link can point to anything, be it personal, local or global, be it draft or highly polished. There was a second part of the dream, too, dependent on the Web being so generally used that it became a realistic mirror (or in fact the primary embodiment) of the ways in which we work and play and socialize. That was that once the state of our interactions was on line, we could then use computers to help us analyse it, make sense of what we are doing, where we individually fit in, and how we can better work together. 8 | 9 | We have the opportunity to drive that vision of a universal web for everyone forward. 10 | 11 | _Ethical Web Development_ will be a series of short digital books that explore the ethics of practical development topics. The books will be released throughout 2016 and early 2017 as free downloads from [O'Reilly](http://www.oreilly.com/). 12 | 13 | ## [Building Web Apps that Work For Everyone](web-apps-for-everyone/README.md) 14 | 15 | - Progressive enhancement 16 | - Web accessibility 17 | - Building inclusive forms 18 | 19 | ## [Building Web Apps that Work Everywhere](web-apps-that-work-everywhere/README.md) 20 | 21 | - Responsive design 22 | - Web Performance 23 | - Deep links 24 | - Offline 25 | - Building offline-first apps 26 | 27 | ## [Building Web Apps that Respect A User's Privacy and Security](web-apps-privacy-security/README.md) 28 | 29 | - User privacy and `Do Not Track` 30 | - HTTPS 31 | - Web application security 32 | - User data 33 | 34 | ## Building Web Apps Together 35 | 36 | - Coding standards 37 | - Linting 38 | - Testing 39 | - Continuous Integration 40 | - Open Source 41 | - Contributing to open source 42 | - Using an open source license 43 | - Consuming open source code 44 | - Treating other developers with respect 45 | - Offering, following, and enforcing a code of conduct for open source projects 46 | 47 | ## Charities 48 | 49 | For each title I will be donating 20% of the proceeds to a charity that promotes and encourages the open web. 50 | 51 | - Building Web Apps that Work For Everyone: [W3C](https://www.w3.org/support/) 52 | - Building Web Apps that Work Everywhere: [World Wide Web Foundation](http://webfoundation.org/) 53 | - Building Web Apps that Respect A User's Privacy and Security: [Electronic Frontier Foundation](https://www.eff.org/) 54 | - Building Web Apps Together: [Mozilla](https://www.mozilla.org/en-US/) and [Girl Develop It](https://www.girldevelopit.com/) 55 | 56 | ## Contributing 57 | 58 | I really appreciate feedback and contributions to this project. 59 | 60 | For feedback, please [create an issue](https://github.com/ascott1/ethical-web-dev/issues) or you can email me directly at [adamdscott@protonmail.com](mailto:adamdscott@protonmail.com). 61 | 62 | If you are interested in contributing, please read the [CONTRIBUTING.md](https://github.com/ascott1/ethical-web-dev/blob/master/CONTRIBUTING.md) file for more details. 63 | 64 | ### Code of Conduct 65 | 66 | This project adheres to the [Contributor Covenant 1.4][code-of-conduct]. By participating, you are expected to honor this code. Please report unacceptable behavior to adamdscott@protonmail.com. 67 | [code-of-conduct]: http://contributor-covenant.org/version/1/4/ 68 | -------------------------------------------------------------------------------- /building-web-apps-together/01-title-intro.md: -------------------------------------------------------------------------------- 1 | # Introduction 2 | 3 | I learned to be a web developer by working alone. It started when I was a high school student in the mid-1990's and had the idea for a website. I spent a lot of free time, in a spare bedroom at my parents' house, copy and pasting HTML snippets that I found by clicking `View > Source` in my web browser. In college, I majored in Multimedia Art, which required two programming classes. I took the classes, working on the required projects alone in my dorm room. After college, I took a break from the web to manage a bookstore and spend time teaching, but a few years later my love of web development was rekindled. When this happened, I re-entered the world of development through independent learning with books and tutorials, eventually doing independent contracting. It wasn't until several years into my career that I partnered with another developer on a project. 4 | 5 | Now, my experience of solitude may be a bit extreme, but I imagine that many of you reading this also got your start working alone. Development is typically first learned as a deep-thinking and independent activity, yet in the workforce it is a highly social endeavor. As professional web developers, we often partner with other developers to build complex systems. To do this, we review and critique one another's code, come to common understandings, make compromises, write documentation, and interact with open source in ways that we were unable to do as solo learners. 6 | 7 | ## Our Responsibility 8 | 9 | The intent of this title is to provide guidelines for how web developers can better work together. The four topics we will explore are: 10 | 11 | - Writing quality code 12 | - Contributing to open source 13 | - Improving our documentation practices 14 | - Recognizing the need for inclusion and collegial respect 15 | 16 | If we define ethics as "making choices that have the most positive effect for the largest number of people," placing an emphasis on positive collaboration is critical. Empathetic development practices lead to the creation of empathetic products. 17 | -------------------------------------------------------------------------------- /building-web-apps-together/03-open-source.md: -------------------------------------------------------------------------------- 1 | # Open Source 2 | 3 | > A knotty puzzle may hold a scientist up for a century, when it may be that a colleague has the solution already and is not even aware of the puzzle that it might solve.” 4 | 5 | ― Isaac Asimov, The Robots of Dawn 6 | 7 | I didn't quite "get" open source software the first time I came across it knowingly. I was a young technology-focused teacher, just scraping by and my interest in technology exceeded my expendable income. This meant that at first, open source software was, in my eyes, a stop gap to support my interests while avoiding both buying expensive software or illegally pirating it. Gradually, a funny thing happened: I stopped seeing open source tools as lesser options and came to embrace both the software and the surrounding community. This grew into a feeling of awe as I came to greatly respect the amount of volunteer, long-distance, asynchronous, collaboration that went into building fully fledged software. 8 | 9 | By the time I discovered open source, it was far from new. Source code has been freely distributed with some software since the days of early computing. In the 1980's and early 1990's, the culture of free software grew alongside the early Internet. Linus ToTorvalds's open development of Linux, Richard Stallman's Free Software Foundation, the transition of Netscape to Mozilla, and the Java programming language were all foundational elements in the development of open source. Today, open source is bigger than ever. The prevalence of Git and the popularity of [GitHub.com](https://github.com/) has created a massive social hub for open source. 10 | 11 | Open source software is a hugely important part of technology culture. The text editing tool that I am using to write this is open source, the web browser that I am using to research this book is open source, and the operating system running on my machine is built upon the open source Unix OS. WordPress, the open source blogging software, is estimated to run over 25% of all sites on the web. Not only are large projects open source, but many of the sites, tools, and applications that we rely on are built using smaller open source libraries. 12 | 13 | As web developers, it is inevitable that we interact with open source in some way. For many developers that may be solely as consumers of open source, but many of us may maintain our own projects or contribute to others with code, bug reports, or support requests. When we do so, we are taking part in a large social web of developers. When our contributions are inclusive and empathetic we help to create a better development culture. 14 | 15 | ## Being an Open Source Maintainer 16 | 17 | Being an open source maintainer often starts out simply: a developer has an idea for a project or abstracts a piece of a larger project and releases the source code. More often than not, this may be the end of the work. The code is out in the wild for anyone to use, but sometimes that code may is something that other developers find useful. When we release open source code, we are now open source maintainers. 18 | 19 | As a project grows in popularity an open source maintainer also becomes a shepherd of the culture that surrounds the project. Sometimes that culture only consists of a handful of users with infrequent interactions and other times it could become something that feels like it needs near constant care and attention. 20 | 21 | If an open source maintainer is a cultural shepherd, what are the ethical responsibilities of that role? Open source maintainers have four primary concerns: 22 | 23 | - Be kind, welcoming, and friendly 24 | - Include and enforce a code of conduct 25 | - Include a License 26 | - Avoid burnout 27 | 28 | ### Be Welcoming and Friendly 29 | 30 | Perhaps the most important thing we can do as open source maintainers is to be welcoming and friendly to those using our project. The primary ways that we interact with users are through documentation and our responses to issues. These are both text-based mediums, which means that the reader does not have any of the physical cues of conversation that aid in understanding our tone. 31 | 32 | We'll be covering documentation in the next chapter, but in brief when writing documentation for our project we should aim to be as inviting as possible. Users of our project may come from a range of experience, skill-levels, and cultural/language backgrounds. Great documentation doesn't make assumptions about what the reader may already know or understand. It's ok to provide quick instructions for experienced users who just want to dive in, but also take care in providing explicit instructions or linking to useful resources to help less those with less experience understand the codebase as well as the problem the code is attempting to solve. Doing so will both set the tone for the project itself and aid in minimizing needless issues being created. 33 | 34 | Issues and bug fix requests are perhaps the most challenging part of being an open source maintainer, particularly as a project grows in popularity. Issue responses are where we interact with the community most directly. Consider these two common types of issue interactions: 35 | 36 | #### Interaction #1: The Frustrated User 37 | 38 | _**User**: This software is totally broken since the last update. This needs to be fixed asap._ 39 | 40 | When we come across these types of issues and requests it can be very frustrating. This is our work and is something that we likely take pride in. Not only that, but we've shared it publicly and freely, while working on it in our precious spare time. That means time away from family, friends, and other things that we love outside of programming. As such, it can feel upsetting or even infuriating when someone is overly critical or makes demands of our projects. The most important thing we can do when we come across these types of issues is to react calmly. On the Internet this is often referred to as "don't feed the trolls." There is nothing wrong with closing an issue with a simple "I'm sorry this does not meet your needs." 41 | 42 | We can also take a proactive approach towards mitigating these types of issue creators by including templates for issues and pull requests in our projects. [GitHub](https://github.com/blog/2111-issue-and-pull-request-templates) offers the ability to include templates in your project that will create an outline for every new issue and pull request that is created. Good templates will provide a framework for contributors to create issues and contribute to the project, potentially alleviating the lack of context that these often contain. The developer Steve Mao has created a helpful set of [example templates](https://github.com/stevemao/github-issue-templates) and a collection of example issue and pull request templates has been collected by the project [Awesome GitHub Templates](https://github.com/devspace/awesome-github-templates). 43 | 44 | #### Interaction #2: The Curt Maintainer 45 | 46 | _**User**: I'm having trouble with X in Y environment. I've tried reinstalling with the latest patch but haven't had any luck. Any ideas? 47 | 48 | **Maintainer**: This is a known issue. Update the dependencies._ 49 | 50 | The above may not look too bad. Likely, as a maintainer we're doing this work in our spare time while doing our best to respond to as many issues as possible. However, to a user, a response such as the above may come across as dismissive or even hostile. If they are filing an issue, it is likely that they are either trying to help or have come across a bug that impacts what they are working on. A short response from us can make them less likely to help in the future or serve to compound their frustration. Let's consider how else we could respond, but first let's take a quick detour into communication theory. In his book [Nonviolent Communication: A Language of Life](http://www.nonviolentcommunication.com/) Dr. Marshall B. Rosenberg lays out a process for positive communication. The four key steps to this process are: 51 | 52 | 1. Observations - Objectively describing the situation. 53 | 2. Feelings - Saying how you feel about what you've observed. 54 | 3. Needs - The needs that are not being met due to these feelings. 55 | 4. Requests - Making clear request for action. 56 | 57 | All of the above are done without assigning and form of blame. By adapting our response, based on Dr. Rosenberg's model, we can improve our issue response: 58 | 59 | First make an observation by re-stating the the problem and acknowledging the user's feelings: _"Hi @user, I'm sorry that you are having trouble with X in the Y environment, that can be really frustrating."_ 60 | 61 | Next, describe how the users needs may be met: _"I've come across this issue before and it can usually be resolved by updating the dependencies." The best way to do that is to remove the dependency directory and reinstall. Here are the commands for doing that..._ 62 | 63 | Lastly, make a request for further action or follow-up from the user: _"_Please let me know if this has resolved your issue._" 64 | 65 | Putting this all together, our response may look something like this: 66 | 67 | _**Maintainer**: Hi @user, I'm sorry that you are having trouble with X in the Y environment, that can be really frustrating. I've come across this issue before and it can usually be resolved by updating the dependencies. The best way to do that is to remove the dependency directory and reinstall. Here are the commands for doing that:_ 68 | 69 | ``` 70 | rm -rf node_modules 71 | npm install 72 | ``` 73 | 74 | _Please let me know if this has resolved your issue._ 75 | 76 | In this response we acknowledged the user's problem by restating it, addressed the user needs with a concrete recommendation fix and a simple description if they are unfamiliar, and made a clear request for them to verify the fix. Though this response is longer, it would have only taken an extra minute to type out and goes a long way towards building a more welcoming community. 77 | 78 | 79 | **Note:** 80 | _Sometimes as an open source maintainer we just don't have time to respond to issues in a timely manner. We're busy at work, on vacation, or have a lot happening in our personal lives. This is ok (and good!). We shouldn't feel a persistent obligation in these scenarios. Instead, when you get a chance simply respond to issues with an acknowledgement such as "I'm sorry for the slow response. I've been traveling If this is still an issue do you mind responding and I will see what needs to be done to address it. Thank you!"_ 81 | 82 | #### Using Bots 83 | 84 | For very active open source projects, I love the bot recommendation made by Felix Krause in his article [Scaling open source communities](https://krausefx.com/blog/scaling-open-source-communities). In the article he demonstrates how the [fastlane](https://github.com/fastlane) project uses bots to auto check that new issues contain all of the required information as well as for closing stale issues. This both reduces the burden of response on the maintainer and reduces friction with the community by clearly and consistently following the same guidelines for all issues. 85 | 86 | ### Open Source Codes of Conduct 87 | 88 | Perhaps one of the most important documents that can be included in a project is a code of conduct. A good code of conduct will establish clear expectations for positive interaction among the community as well as outcomes for those who fail to meet those expectations. The goal of the code of conduct is to provide guidelines for an open and inclusive community around our projects. 89 | 90 | In her article [Codes Of Conduct 101 + FAQ](https://www.ashedryden.com/blog/codes-of-conduct-101-faq), programmer, advocate, and consultant Ashe Dryden describes a successful Code of Conduct as needing these four parts: 91 | 92 | - A statement of unacceptable behavior 93 | - Information on how the policy will be enforced 94 | - Information on how and whom to make an incident report to 95 | - Training and reference materials for organizers, staff, and volunteers on how to respond to incident reports 96 | 97 | For establishing a statement of unacceptable behavior and including information on how the policy will be enforced, I recommend using or at least basing your policy on the [Contributor Covenant](http://contributor-covenant.org/). The Contributor Covenant is a Creative Commons licensed resource that includes a pledge of inclusion, standards of behavior, examples of both model and unacceptable behavior, responsibilities of the maintainers, information on how to report incidents, and enforcement information. To include the Contributors Covenant in your project, I recommend adding a `code_of_conduct.md` file the to the project files along with a link in the project README that states: 98 | 99 | ``` 100 | ## Code of Conduct 101 | 102 | This project adheres to the [Contributor Covenant 1.4][code-of-conduct.md]. By participating, you are expected to honor this code. Please report unacceptable behavior to user@example.com. 103 | ``` 104 | 105 | Including the code of conduct is only the first step. We must also be prepared and informed to take action should an incident occur. For these incidents, the Django Project has a very useful [enforcement manual](https://www.djangoproject.com/conduct/enforcement-manual/). For large or active projects, I recommend adapting this manual to meet the needs of your project and publicly sharing the commitment to enforcement. For a project with multiple maintainers, it is critical that the entire team is well-versed in both the code of conduct as well as the enforcement actions that may be taken. 106 | 107 | ### Open Source Licenses 108 | 109 | Some people in the open source community feel _very_ strongly about a particular license. I am not here to indoctrinate you into using a particular license, but simply to encourage you to use _a_ license. The Open Source Initiative maintains a [canonical list](https://opensource.org/licenses/alphabetical) of licenses approved by the organization, the most popular of which are: 110 | 111 | - Apache License 2.0 112 | - BSD 3-Clause "New" or "Revised" license 113 | - BSD 2-Clause "Simplified" or "FreeBSD" license 114 | - GNU General Public License (GPL) 115 | - GNU Library or "Lesser" General Public License (LGPL) 116 | - MIT license 117 | - Mozilla Public License 2.0 118 | - Common Development and Distribution License 119 | - Eclipse Public License 120 | 121 | The benefit of applying a license is that it allows you to create the terms by which your project can be used. The MIT license, for example, allows users to use your code in any way they choose as long as they provide some form of attribution. By contrast, the GPL requires that any derivative work license itself under the same terms, which effectively prohibits closed-source projects form using code licensed under the GPL. By choosing a license for your code, you are indicating how it may be used and credited. Doing so also opens our projects up to being used (or not) by organizations who may be unable to do so with unlicensed code. 122 | 123 | The simplest way to include a license in your project is to include the license text or a declaration of which license is being used at the bottom of the project's README. You may also include the full text of the license in a `LICENSE` at the top-level of the project. If you are unsure of which license to apply to your projects, I recommend the guide at [choosealicense.com](https://choosealicense.com). The site, created by GitHub, guides users to popular licenses based on specific needs and concerns. 124 | 125 | ### Avoiding Maintainer Burnout 126 | 127 | If all of the above sounds exhausting, that's because it often is. Being an open source maintainer is a lot of work and is often done as an extra-curricular hobby outside of work hours. This means that open source development is relegated to those who are privileged with free-time, which may not be an ongoing possibility as a developer's life changes. Due to the demands, time commitments, and, at times, guilt that comes along with being an open source maintainer, many eventually experience burnout. 128 | 129 | If this all feels familiar, I'd encourage you to read the perspective of developers who have battled with open source burnout: 130 | 131 | 132 | - [Maintainer's Guide to Staying Positive](https://github.com/jonschlinkert/maintainers-guide-to-staying-positive) by Jon Schlinkert 133 | - [What it feels like to be an open-source maintainer](https://nolanlawson.com/2017/03/05/what-it-feels-like-to-be-an-open-source-maintainer/) by Nolan Lawson 134 | - [How to Avoid Burnout Managing an Open Source Project](https://thenewstack.io/darker-side-open-source/) by Dawn Foster 135 | - [Dear JavaScript](https://medium.com/@thejameskyle/dear-javascript-7e14ffcae36c) by James Kyle 136 | 137 | If you have been contributing to open source and feel burnt out, I'd offer these three pieces of advice: 138 | 139 | 1. **Take care of your health and well-being.** Things like sleep, nourishment, and time with friends and family are important. Though the speed of change in web development can cause us to feel like we must be constantly involved, being a well-rounded and healthy human will help us to also be better developers. 140 | 2. **It's OK to ignore things.** As issues and pull requests pile up, we may become overwhelmed or even guilty by a lack of action. Nearly every time I've spoken to the author of a popular open source project, they mention a feeling of guilt for not addressing a bug or responding to outstanding issues. Though it may be difficult to do, it's important to let go of the nagging feeling that these create and accept that it's impossible for one person to respond to every need of a project. 141 | 3. **Don't be afraid to deprecate a project.** When a project has completely lost your interest or attention, it may be better to deprecate the project or hand it over to other maintainers. The easiest way to do this is to place a large **DEPRECATED** notice in the project description and README. You can also offer to transfer ownership of the project if someone else is interested in taking it over. This signifies to users that you will no longer be responding to issues or contributing new features and bug-fixes. Doing so is not admitting defeat, but rather allowing ourselves to move on to different things. 142 | 143 | ## Contributing to Someone Else's Project 144 | 145 | There are a number of ways in which we may contribute to open source projects without being a project maintainer. These may include contributing code, filing bug reports, improving documentation, and requesting support. As soon as we perform any of these actions, we are a part of the community that surrounds the project. Similar to his resource for open source maintainers, developer Jon Schlinkert has written a guide for [Idiomatic Contributing](https://github.com/jonschlinkert/idiomatic-contributing), which is a useful reference for anyone looking to be involved in an open source community. 146 | 147 | In general, contributors to open source should seek to: 148 | 149 | - Always follow a project's code of conduct. 150 | - Familiarize themselves with the goals and scope of the project. 151 | - Check for existing issues or bug reports before filing a new one. 152 | - Be kind to the project maintainers and other community members. 153 | - Follow the coding conventions that the project already has in place. 154 | - Write tests for any new features or code changes. If a test suite is not in place, describe how the maintainer can test your changes. 155 | - Be patient and recognize the right of the maintainers to be slow to respond. 156 | - Be accepting of rejection when a project chooses not to merge code or implement a recommendation. 157 | 158 | Participating in an open source community can be a wonderful experience. By following the guidelines above as well as being aware of the challenges placed upon open source maintainers, can help us be positive community contributors. 159 | 160 | ## Further Reading 161 | 162 | - [The Cathedral & the Bazaar](http://shop.oreilly.com/product/9780596001087.do) by Eric S. Raymond 163 | - [Open Source Guides](https://opensource.guide/) 164 | - [Producing Open Source Software](http://producingoss.com/) by Karl Fogel 165 | - [Scaling Open Source Communities](https://krausefx.com/blog/scaling-open-source-communities) by Felix Krause 166 | - [A template for creating open source contributor guidelines](https://opensource.com/life/16/3/contributor-guidelines-template-and-tips) by Safia Abdalla 167 | -------------------------------------------------------------------------------- /building-web-apps-together/04-documentation.md: -------------------------------------------------------------------------------- 1 | # Documentation 2 | 3 | > Writing is hard work. A clear sentence is no accident. 4 | 5 | - William Zinsser, author of _On Writing Well_ 6 | 7 | I'm going to begin this chapter by describing two common scenarios on the life of a developer. 8 | 9 | In our first scenario, meet Harlow. Today is Harlow's first day on a new project. The team has a well-established codebase, a great working environment, and a robust test suite. As Harlow sits down at her desk, she's excited to get up to speed with the team. After the morning stand-up meeting she's pointed to the project's documentation for installation with a slight grimace from her colleague Riley. He mentions that the docs "might be a little out of date, but should hopefully be enough to get you going." Harlow then spends the rest of the day following the documentation until she gets stuck at which point she is forced to dig through code or ask colleagues for guidance. What might have taken a few minutes becomes a day-long exercise in frustration, tampering Harlow's initial excitement. 10 | 11 | In our second scenario, meet Harrison. He's working on a web app and finds a library that, at first glance, seems incredibly useful for his project. As he attempts to integrate it with his codebase he discovers that parts of the API seem to be glossed over in the documentation or even undocumented. In the end, he walks away from the project in favor of another solution. 12 | 13 | Those these scenarios may be slightly exaggerated, but I'm reasonably certain that many of us can relate. These problems were not primarily caused by low quality code, but rather by poor documentation. 14 | 15 | If useful documentation is so important to the success of projects and developer well-being, why don't all projects have it? The answer, I believe, is that like good code, good documentation is difficult and time consuming to write. The goal of this chapter is to convince you that writing good documentation is deserving of your valuable attention as well as provide you with concrete guidelines for doing so. 16 | 17 | ## The Eight Rules of Good Documentation 18 | 19 | In my eyes there are eight rules that we can follow to produce good documentation: 20 | 21 | 1. Write documentation that is inviting and clear 22 | 2. Write documentation that is comprehensive, detailing all aspects of the project 23 | 3. Write documentation that is skimmable 24 | 4. Write documentation that offers examples of how to use the software 25 | 5. Write documentation that has repetition, when useful 26 | 6. Write documentation that is up-to-date 27 | 7. Write documentation that is easy to contribute to 28 | 8. Write documentation that is easy to find 29 | 30 | The most important rule of good documentation is for it to **be as inviting as possible**. This means that we should aim to write it in the clearest terms possible without skipping over any steps. We should avoid making assumptions about what our users may know. Sometimes this can seem to be overkill and we may be tempted to say something like "every X developer knows about Y," but we each bring our own background and set of experiences to a project. Though this may result in more verbose documentation, it is ultimately simpler as there is less guesswork involved for developers with all levels of experience. 31 | 32 | Our documentation should aim to **be comprehensive**. This means that all aspects of the project are documented. Undocumented features or exceptions can lead to frustration and become a time-suck as users and other developers are forced to read through code to find the answers they need. Fully documenting all features takes away this kind of ambiguity. 33 | 34 | When we write documentation that is **skimmable** we help users find the content they need quickly. Making documentation skimmable can be accomplished by using clear headings, bulleted lists, and links. For large project documentation a table of contents or clear navigation will help users to skip straight to what they need, rather than scrolling through a single long document. 35 | 36 | Documentation that features **examples** allows users to see how they might use the code themselves. Aim to provide examples of the most common use-cases for the project, while letting the comprehensive documentation detail every possibility. 37 | 38 | It is perfectly acceptable to **include some repetition** in our documentation, which the [Write the Docs project](http://www.writethedocs.org/guide/writing/docs-principles/#arid) terms ARID (accepts (some) repetition in documentation). Doing so acknowledges that users may not read the full docs or that some information is relevant in multiple places in the documentation. While good code may be DRY, good writing aims to be clear and sometimes this means repeating ourselves. The Write the Docs project calls out the the difference between writing that is ARID, DRY, and WET in this way: 39 | 40 | > The pursuit of minimizing repetition remains valiant! ARID does not mean [WET](https://en.wikipedia.org/wiki/Don't_repeat_yourself#DRY_vs_WET_solutions), hence the word choice. It means: try to keep things as DRY as possible but also recognize that you’ll inevitably need some amount of “moisture” to produce documentation. 41 | 42 | Effective documentation is **kept up-to-date**. This is surprisingly challenging. We may begin our project with the best of intentions and great documentation, but as our software evolves and we are quickly iterating it can be easy to fall out step. If you are working as part of an agile development team, I recommend adding documentation to your team's "definition of done." For independent projects, try to treat documentation as an important final step. 43 | 44 | Documentation that is **easy to contribute to** is also easy to keep up-to-date. The simplest way to make documentation easy to contribute to is to treat it as code, storing it as text in source control. The site and book [Docs Like Code](http://docslikecode.com/about/) advocates for treating our docs like our code by using source control, automating builds, and applying software development tools and techniques to our documentation practices. 45 | 46 | Documentation is only as helpful as it is **easy to find.** Keeping an updated README file and linking to more extensive documentation at the top of the README when necessary helps to keep discoverability simple. 47 | 48 | I hope that these guidelines are useful as you draft your project's documentation. Sometimes it is helpful to remember that documentation isn't just for other developers, but often for our future selves as well. When we return to a project after a number of months, we will appreciate the work we put into clear and up-to-date documentation. 49 | 50 | ## The Importance of the README 51 | 52 | For many developers, a project's README will be their introduction to a piece of software. For small projects, the README may be its only form of documentation. Successful READMEs are both informative and inviting, giving the reader a solid understanding of the project and providing the confidence needed to use the software. 53 | 54 | Successful READMEs contain a combination of several key elements. Most importantly the README should be up to date and provide a useful getting started guide, written in a warm and welcoming tone. Here are the key components of a good README: 55 | 56 | - A brief (1 paragraph or less) description of the project 57 | - A list of the key features of the software 58 | - A getting started guide that details the installation steps, guidelines for usage, and instructions for building and deploying the software, if applicable 59 | - Information on where to go if the user needs help, such as the issue tracker, IRC channel, or a core developer's email address 60 | - Guidelines for how to contribute to the project, including how to file a pull request, coding standards, and how to run the test suite 61 | - A code of conduct for the project that demonstrates inclusion and outlines clear enforcement actions 62 | - A License for the project 63 | 64 | Most importantly, a project's README needs to be kept up to date. Few things are more frustrating than discovering that a feature documented in the README has changed or been removed. Marc Esher, a software development manager at the Consumer Financial Protection Bureau, wrote about holding a [README Refresh Day](https://cfpb.github.io/articles/readme-refresh-day/) with the development team. If you work on a team, doing events such as these regularly can be a great way to both improve the contents of your READMEs and to explore software built by other teams. 65 | 66 | ### README Template 67 | 68 | Here's a README template that can be used for your own projects. Feel free to adapt it to your project's needs. 69 | 70 | ``` 71 | # Project Title 72 | 73 | Provide a brief (1 paragraph or less), meaningful description of the project 74 | and what it does. If the project has a UI, include a screenshot as well. 75 | 76 | If more comprehensive documentation exists, link to it here. 77 | 78 | ## Features 79 | 80 | Describe the core features of the project (what does it do?) in the form of a 81 | bulleted list: 82 | 83 | - Feature #1 84 | - Feature #2 85 | - Feature #3 86 | 87 | ## Getting Started 88 | 89 | Provide installation instructions, general usage guidance, API examples, and 90 | build and deployment information. Assume as little prior knowledge as possible, 91 | describing everything in clear and coherent steps. Avoid words such as "just" 92 | and "simple," which can be off putting to users who do not understand the 93 | instructions. 94 | 95 | ### Installation/Dependencies 96 | 97 | How does a user get up and running with your project? What dependencies does 98 | the project have? Aim to describe these in clear and simple steps. Provide 99 | external links. 100 | 101 | ### Usage 102 | 103 | Provide clear examples of how the project may be used. For large projects with 104 | external documentation, provide a few examples and link to the full docs here. 105 | 106 | ### Build/Deployment 107 | 108 | If the user will be building or deploying the project, add any useful guidance. 109 | 110 | ## Getting Help 111 | 112 | What should users do and expect when they encounter bugs or get stuck using 113 | your project? Set expectations for support, link to the issue tracker and 114 | roadmap, if applicable. 115 | 116 | Where should users go if they have a question? (Stack Overflow, Gitter, IRC, 117 | mailing list, etc.) 118 | 119 | If desired, you may also provide links to core contributor email addresses. 120 | 121 | ## Contributing Guidelines 122 | 123 | Include instructions for setting up the development environment, code 124 | standards, running tests, and submitting pull requests. Aim to be inclusive and 125 | welcoming. It may be useful to link to a seperate CONTRIBUTING.md file. See 126 | this example from the Hoodie project as an exemplar: 127 | https://github.com/hoodiehq/hoodie/blob/master/CONTRIBUTING.md 128 | 129 | ## Code of Conduct 130 | 131 | Open Source projects should follow a code of conduct. Provide a link to the 132 | Code of Conduct for your project. I recommend using the Contributor Covenant: 133 | http://contributor-covenant.org/ 134 | 135 | ## License 136 | 137 | Include a license for your project. If you need help choosing a license, use 138 | this guide: https://choosealicense.com/ 139 | ``` 140 | 141 | I've open sourced this template and provided guidelines for use at https://github.com/ascott1/readme-template. 142 | 143 | ## Project Documentation 144 | 145 | Larger codebases will require documentation that extends beyond a README. In these instances a good README (or READMEs if the project is split into several smaller codebases) is still an incredibly valuable starting point. It is also helpful to keep the documentation as near to the code as possible. This may mean generating a static website from plain text or markdown files and storing it in the same code repository or using the repository wiki provided by a source control hosting provider. 146 | 147 | One other important aspect of large project documentation is to use it as a means for capturing decision making whenever possible. As developers we are constantly making tradeoffs and decisions that may have impacts on the functionality or use of the codebase. When we capture these ideas, future developers are able to understand the thinking behind this decision. All too often, when decision making is not captured, developers will attempt to re-architect something until they hit the same wall. This "oh, I see why they did it that way" epiphany and wasted energy can be avoided when we accurately capture the thinking behind our decisions. 148 | 149 | ## When Developers Are Your Customers 150 | 151 | For some organizations, developers are our customers. We may build products that handle authentication, messaging, deployment jobs, site availability monitoring, etc. For these types of products, documentation needs to always be treated as a first class citizen, as it is the way our customers learn about our product. 152 | 153 | The messaging service Twilio is a great example of this. The [Twilio docs](https://www.twilio.com/docs/) are exemplary, covering a wide range of languages, providing quick start guides, offering interactive tutorials, and a full API reference. The Twilio organization recognizes the value of these docs, employing a team of developers to work on their documentation and the associated tooling full-time. 154 | 155 | ## Wrapping Up 156 | 157 | By focusing on writing good documentation, we are showing empathy for the developers who use our project. Doing so helps to create a project environment that is inclusive, welcoming, and collaborative. 158 | 159 | ## Further Reading 160 | 161 | - [Write the Docs' Documentation Guide](http://www.writethedocs.org/guide/) 162 | - [Docs Like Code](http://docslikecode.com/) by Anne Gentle 163 | - [Writing great documentation](https://byrslf.co/writing-great-documentation-44d90367115a) by Taylor Singletary 164 | - [Beautiful Docs](https://github.com/PharkMillups/beautiful-docs) 165 | -------------------------------------------------------------------------------- /building-web-apps-together/05-respect.md: -------------------------------------------------------------------------------- 1 | # Treating Other Developers with Respect 2 | 3 | This will be the shortest chapter of the series, but is likely the most important. It is critical that we treat our colleagues with respect. The primary way we do this is by working to create an inclusive and safe environment for everyone regardless of gender, race, sexual orientation, age, or cultural background. Whether it is online, at work, at a meetup, or attending a conference, be kind and respectful to those around you. In this chapter, we'll briefly touch upon unconscious bias and creating inclusive environments. 4 | 5 | ## Unconscious Bias 6 | 7 | In the 1970's the typical orchestra was comprised of over 90% male musicians. Today orchestras are more likely made up of one third female musicians, with likelihood that a woman musician will be selected increasing by 30-60%. What changed to spur this increase in female classical musicians? It was the introduction of blind auditions. Today's selection process has musicians audition without being seen by the judging committee. Early on, this was proven to require further modification as women began removing their shoes, so that judges would not hear the tell-tale sound the shoes made when the musician approached the chair. This is an example of unconscious bias. The judges largely assumed, presumably without intention or awareness of their preference, that men were better musicians. When the chance for this bias was removed, more women were selected. 8 | 9 | Unconscious biases are stereotypes that we hold without being explicitly aware of them. They are often the result of quick judgements, which are "a feature of evolution," according to Harvard psychology professor [Mahzarin Banaji](https://www.nytimes.com/2015/05/07/opinion/nicholas-kristof-our-biased-brains.html). For early humans it was useful to quickly categorize things, such as predators or potential threats. Today, we retain this skill, and while in many life situations bias serves as a decision making shortcut, it can also have unintended negative consequences, particularly when it applies to how we think about other people. 10 | 11 | The fact that these are implicit biases, means that we are making judgements that we are unaware of. Because of this, the first step is to gain awareness of the unconscious biases that we may contain. Harvard's Project Implicit has created a series of [implicit association tests](https://implicit.harvard.edu/implicit/takeatest.html), which allow users to perform a series of online actions that may help to reveal hidden biases. 12 | 13 | Recently, Google has undertaken an intensive unconscious bias training effort with employees and offers the training materials as part of their [ReWork project](https://rework.withgoogle.com/guides/unbiasing-raise-awareness/steps/introduction/). These materials include video information, links to scientific studies, and the full training materials used by Google. Other large tech companies have undertake similar efforts, with Microsoft offering a full [online training module](https://www.microsoft.com/en-us/diversity/training) and Facebook providing a series of [video lecture modules](https://managingbias.fb.com/). 14 | 15 | By becoming more aware of how we may be influenced by our unconscious biases, we are able to improve how we collaborate with our colleagues and peers. 16 | 17 | 18 | ## Creating Welcoming Environments 19 | 20 | Just as we discussed creating inclusive spaces for open source projects, we should seek to do the same for physical spaces. Codes of Conduct should also be applied to events such as meetup groups, hackathons, and conferences. If you work for a small company, ensure they also have clear conduct policies in place. Here are a few resources for in-person event codes of conduct: 21 | 22 | - [Ada Intiative's HOWTO design a code of conduct for your community](https://adainitiative.org/2014/02/18/howto-design-a-code-of-conduct-for-your-community/) 23 | - [Hack Code of Conduct](https://hackcodeofconduct.org/) 24 | - [Geek Feminism's Conference anti-harassment/Policy](http://geekfeminism.wikia.com/wiki/Conference_anti-harassment/Policy) 25 | - [Citizen Code of Conduct](http://citizencodeofconduct.org/) 26 | 27 | Community Manager, Sarah Jane Morris has put together a [guide for hosting inclusive events](https://github.com/SarahJaneMorris/inclusive-events-guide/blob/master/guide.md) which serves as a useful template for creating a positive atmosphere. Though some of the resources are bay area specific, it offers a solid template for creating a welcoming event culture. If we are involved in organizing an event, we may also want to consider alternatives to the typical happy hour or alcohol-centric venues. In her article [Alcohol and Inclusivity: Planning Tech Events with Non-Alcoholic Options](https://modelviewculture.com/pieces/alcohol-and-inclusivity-planning-tech-events-with-non-alcoholic-options), Community Manager Kara Sowles suggests five tips for including non-alcoholic drinks at events: 28 | 29 | 1. Provide an equal number and quality of alcoholic and non-alcoholic drink options. 30 | 2. Display alcoholic and non-alcoholic drinks together at the event. 31 | 3. Advertise alcoholic and non-alcoholic drinks equally before the event. 32 | 4. If listed cocktails are being served, list an equal number of non-alcoholic mocktails. 33 | 5. Have water freely available, in clear sight, and easy to obtain. 34 | 35 | Taking these types of considerations is not limited to events. In a [profile](http://www.bizjournals.com/portland/blog/techflash/2016/02/how-one-portland-startup-baked-diversity-into-its.html) of the Portland, Oregon startup Paresh Patel the CEO of PayRange discussed the company's decision to re-address some of the typical startup office perks: 36 | 37 | > “We don’t have kegs in our office, those can make some people uncomfortable. We don’t have pool tables. We don’t have Friday happy hour. Instead, we have family nights out for dinner, vegetarian microwaves, summer picnics where employees can have fun with their kids.” 38 | 39 | These are just a examples of how more welcoming environments are created. By focusing on the needs and well-being of everyone, we are able to build stronger communities. 40 | 41 | ## Further Reading 42 | 43 | We must all do our part to create an inclusive work culture. Wether we work remotely or are co-located, we all spend a lot of time with colleagues. For many of us, web development extends beyond being a day job into our social lives and hobbies, where we should also seek to be open and welcoming to all. The guides and articles below are incredibly useful tools for building positive environments. 44 | 45 | - [Project Include](http://projectinclude.org/) 46 | - [Hire More Women in Tech](https://www.hiremorewomenintech.com/) 47 | - [Ally Skills Workshop](https://frameshiftconsulting.com/ally-skills-workshop/) 48 | - [How Diversity Makes Us Smarter](https://www.scientificamerican.com/article/how-diversity-makes-us-smarter/) by By Katherine W. Phillips 49 | - [Model View Culture](https://modelviewculture.com/) 50 | -------------------------------------------------------------------------------- /building-web-apps-together/06-conclusion.md: -------------------------------------------------------------------------------- 1 | # In conclusion 2 | 3 | Thank you for taking the time to read “Build Web Apps Together.” I hope that through reading this title you see value in building an inclusive developer environment that focuses on quality and takes advantage of the open source community. These encompass a small portion of the work we can do as web developers to ensure that the web is an open and accessible space for all users. My hope is that you now feel empowered and excited to build applications in this way. 4 | 5 | If throughout your reading you have come across things that are missing or could be improved, I would encourage you to contribute back to the book. This title is available as open source and contributions can be made by: 6 | 7 | - Contributing directly to the GitHub repository with a pull request . 8 | - Creating an issue in the book’s GitHub repository . 9 | - Reaching out to me through [email](mailto:adamdscott@protonmail.com) or [Twitter](https://twitter.com/adamdscott). 10 | 11 | Twenty percent of the proceeds from each Ethical Web Development title will be donated to organizations whose work has a positive impact on the issues described. For this title, I will be donating to the [Mozilla](https://www.mozilla.org/en-US/) and [Girl Develop It](https://www.girldevelopit.com/). Mozilla is a non-profit organization which provides software, documentation, and education for the open web. Girl Develop It offers classes and community support for women of diverse backgrounds who are interested in learning web and software development. 12 | 13 | If you've enjoyed this title, I hope that you'll join me in supporting these two great organizations. 14 | 15 | This title is the first in a series of digital reports I am authoring on the subject of ethical web development. Future titles in the series will cover building web applications that work everywhere, building web applications that respect a user's privacy and security, and working with development peers. You can learn more about the series at [ethicalweb.org](https://ethicalweb.org). 16 | -------------------------------------------------------------------------------- /building-web-apps-together/README.md: -------------------------------------------------------------------------------- 1 | # Building Web Apps Together 2 | 3 | - [Preface](https://github.com/ascott1/ethical-web-dev/blob/master/preface.md) 4 | - [Introduction](01-title-intro.md) 5 | - [Code Quality](02-code-quality.md) 6 | - [Open Source](03-open-source.md) 7 | - [Documentation](04-documentation.md) 8 | - [Treating Other Developers with Respect](05-respect.md) 9 | - [Conclusion](06-conclusion.md) 10 | - [Contributors](contributors.md) 11 | -------------------------------------------------------------------------------- /building-web-apps-together/contributors.md: -------------------------------------------------------------------------------- 1 | # Contributors 2 | 3 | ## About the Author 4 | 5 | Adam D. Scott is a developer and educator based in Connecticut. He currently works as a senior front-end development fellow at the Consumer Financial Protection Bureau, where he focuses on building open source tools. Additionally, he has worked in education for over a decade, teaching and writing curriculum on a range of technical topics. His first book, WordPress for Education, was published in 2012. His video course, Introduction to Modern Front-End Development, was published by O'Reilly in 2015. 6 | 7 | ## Technical Reviewers 8 | 9 | 10 | ## Editor 11 | 12 | Meg Foley 13 | 14 | ## Other Contributions 15 | 16 | The following people have graciously contributed feedback and improvements. 17 | 18 | - 19 | 20 | Contributions and suggestions have also been made to the ethicalweb.org site and the core principles of ethical web development. Those contributions are stored at 21 | -------------------------------------------------------------------------------- /building-web-apps-together/img/device-lab.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/building-web-apps-together/img/device-lab.jpg -------------------------------------------------------------------------------- /humans.txt: -------------------------------------------------------------------------------- 1 | Ethical Web Development is a project by: 2 | 3 | Adam Scott 4 | - adamscott.website 5 | - twitter.com/adamdscott 6 | 7 | Contributors to the ethicalweb.org site can be found at https://ethicalweb.org/humans.txt 8 | 9 | Additional contributions and feedback have been provided by: 10 | 11 | Chris Contolini 12 | - https://contolini.com 13 | Andy Chosak -------------------------------------------------------------------------------- /img/Hippocratic_oath.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/img/Hippocratic_oath.jpg -------------------------------------------------------------------------------- /preface.md: -------------------------------------------------------------------------------- 1 | # Preface 2 | 3 | As web developers, we are responsible for shaping the experiences of user's online lives. By making choices that are ethical and user-centered, we create a better web for everyone. The Ethical Web Development series aims to take a look at the ethical issues of web development. 4 | 5 | With this in mind, I’ve attempted to divide the ethical issues of web development into four core principles. 6 | 7 | 1. Web applications should work for everyone. 8 | 2. Web applications should work everywhere. 9 | 3. Web applications should respect a user's privacy and security. 10 | 4. Web developers should be considerate of their peers. 11 | 12 | The first three are all about making ethical decisions for the users of our sites and applications. When we build web applications we are making decisions for others, often unknowingly to those users. 13 | 14 | The fourth principle concerns how we interact with others in our industry. Though the media often presents the image of a lone hacker toiling away in a damp and dusty basement, the work we do is quite social and relies on a vast web of connected dependencies on the work of others. 15 | 16 | ## What are ethics? 17 | 18 | If we’re going to be discussing the ethics of web development, let’s first come to a common understanding of how we apply the term “ethics.” The study of ethics falls into four categories: 19 | 20 | - **Meta-ethics**, an attempt to understand the underlying questions of ethics and morality. 21 | - **Descriptive ethics**, the study and research of people’s beliefs. 22 | - **Normative ethics**, the study of ethical action and creation of standards of right and wrong. 23 | - **Applied ethics**, the analysis of ethical issues, such as business ethics, environmental ethics, and social morality. 24 | 25 | For our purposes we will be doing our best to determine a normative set of standards of ethics as applied to web development and then take an applied ethics approach. 26 | 27 | Within normative ethical theory there is the idea of Consequentialism, which argues that the ethical value of an action is based on the result of the action. In short, the consequences of doing something become the standard of right or wrong. 28 | 29 | One form of Consequentialism, called Utilitarianism, states that an action is right if it leads to the most happiness, or well-being, for the greatest number of people. This utilitarian approach is the framework I’ve chosen to use as we explore the ethics of web development. 30 | 31 | Whew! We fell down a deep dark hole of philosophical terminology, but I think it all boils down to this: 32 | 33 | **Make choices that have the most positive effect for the largest number of people.** 34 | 35 | 36 | ### Professional Ethics 37 | 38 | Many professions have a standard expectation of behavior. These may be legally mandated or a social norm, but often take the form of a code of ethics that details conventions, standards, and expectations of those who practice the profession. The idea of a professional code ethics can be traced back to the Hippocratic Oath, an oath taken by medical professionals first written during the fifth century BC. Today, medical schools continue to administer the Hippocratic or similar professional oath. 39 | 40 | ![A fragment of the Hippocratic Oath from the 3rd-century](img/Hippocratic_oath.jpg) 41 | 42 | In the book, [“Thinking Like an Engineer,”](http://ethics.iit.edu/publication/md_te.html) Michael Davis describes the idea of a code of conduct for professionals: 43 | 44 | > “… prescribes how professionals are to pursue their common ideal so that each may do the best she can at a minimal cost to herself and those she cares about… The code is to protect each professional from certain pressures (for example, the pressure to cut corners to save money) by making it reasonably likely (and more likely then otherwise) that most other members of the profession will not take advantage of her good conduct. A code is a solution to a coordination problem.” 45 | 46 | My hope is that this work helps to inspire a code of ethics for web development, guiding our work in a way that is professional and inclusive. 47 | 48 | The approaches I’ve laid out are merely my take on how web development can provide the greatest happiness for the greatest number of people. These approaches are likely to evolve as technology changes and may be unique for many development situations. I invite you to read my practical application of these ideas and hope that you apply them in some fashion to your own work. 49 | 50 | This series is a work in progress and I invite you to contribute. To learn more, visit ethicalweb.org. 51 | 52 | ## Intended Audience 53 | 54 | This title, and others in the Ethical Web Development series, is intended for web developers and web development team decision makers who are interested in exploring the ethical boundaries of web development. I assume a basic understanding of fundamental web development topics such as HTML, CSS, JavaScript, and HTTP. Despite this assumption, I‘ve done my best to describe these topics in a way that is approachable and understandable. 55 | -------------------------------------------------------------------------------- /web-apps-for-everyone/01-title-intro.md: -------------------------------------------------------------------------------- 1 | # Introduction 2 | 3 | In 1998 Tim Berners-Lee, the creator of the Web, published [The World Wide Web: A very short personal history](https://www.w3.org/People/Berners-Lee/ShortHistory.html). In this brief essay he states: 4 | 5 | > The dream behind the Web is of a common information space in which we communicate by sharing information. Its universality is essential: the fact that a hypertext link can point to anything, be it personal, local or global, be it draft or highly polished. 6 | 7 | To Berners-Lee the universality of the Web is exactly that which allowed the Web to grow from a single server at his desk to a global communication network with over 3 billions users worldwide[^1]. While built upon the technologies established in those early days of the Web, today's Web has grown beyond the concept of hyperlinked documents. Today we build rich graphical interfaces that encompass anything from a text document to a real-time video application while also providing for many their primary means for personal interaction, information, and fundamental social services. 8 | 9 | With the rise of information and immersive applications on the Web, we have created a global network that is heavily relied upon by society. Pausing to think about this, it is a beautiful thing and, true to Berners-Lee’s vision, there remains little barrier to entry to publishing a site or application. However, as web developers it is a professional and social responsibility to ensure that our sites and applications work for as many people as possible. 10 | 11 | I have often been tempted to regard browser or device testing casually in favor of using the latest and greatest tools and browser features. Learning to use these new tools is one of the things that make web development so enjoyable, but we must temper this desire with the ability to build sites that work for as many users as possible. We should avoid shutting out users or denying them our services due to technical constraints. When we do this, we are taking an elite position, potentially shutting out the poor, disabled, and elderly. Imagine a storefront who didn’t allow customers to enter if their shoes and clothes were too old. As a society we would find that offensive and the shopkeeper would likely be publicly disgraced on the evening news. However, we often put banners on our site that say, “This site only supports X browser or newer,” when a visitor accesses it with an older browser. Or worse, the site will silently fail, akin to the shopkeeper completely ignoring a customer. 12 | 13 | ## It Just Works 14 | 15 | My wife and I began dating in 2003 and within a year or so I became her family’s default “computer expert.” In 2005 I helped my father in-law, Marty, pick out a new computer for himself. To him this was akin to an appliance purchase and we picked out a sturdy desktop computer, which has been in continuous use since then. We’ve made some upgrades to the RAM and I’ve done my best to point him to using an evergreen browser that automatically updates, but those no longer update on his aged XP system. When I asked him why he doesn’t upgrade, he just shrugs and says “it still works.” For him, the existence of the Web browser is enough. He assumes that by typing in a URL, that the browser and machine connecting shouldn’t make a difference. 16 | 17 | When my grandfather passed away my grandmother, Kathy, wanted to learn to use a computer and connect the Web. Her primary device is an inexpensive and outdated Android tablet that connects to the Web through a wireless connection from the rural area where she lives. She uses it to check Facebook, read the news, read books, and play solitaire. 18 | 19 | As developer we want to assume that people like Marty and Kathy are edge cases. Looking at the top browsers currently in use, when grouped together, device specific browsers, outdated “evergreen” browser versions, and uncommon open source browsers occupy the second largest percentage of market share[^2]. Though each of these browsers and versions may only show up in our analytics as a fraction of a percent, when grouped together they become a powerful representation of the market. 20 | 21 | ![Chart of browser market share](img/browser-chart.png) 22 | 23 | Though the users of these browsers may not be the target demographic for our applications and services, by not making accommodations for them we are denying them the opportunity to participate. 24 | 25 | [^2]: The site StatCounter provides these metrics http://gs.statcounter.com/#all-browser_version_partially_combined-ww-yearly-2015-2016-bar. I’ve made the full list available as a csv at https://gist.github.com/ascott1/1f9b8fdc7529e4dd7823. 26 | 27 | ## A Responsibility 28 | 29 | As web developers, we are gatekeepers to the vast troves of information and interaction across the Web. With this comes a responsibility to ensure that the Web is an open and inclusive space for all. The following chapters attempt to lay a groundwork for inclusive web development through: 30 | 31 | - **Progressive enhancement.** By building progressively we can ensure that all users have access to a base experience, regardless of technology or network conditions. 32 | - **Accessibility.** By building accessible user interfaces we ensure that everyone has equal access to our applications regardless of disability or age. 33 | - **Inclusive forms.** Forms allow users to interact directly with the Web, making it a two way form of communication. By creating Web forms that are inclusive and usable, we demonstrate our dedication to inclusion. 34 | 35 | 36 | 37 | [^1]: http://www.internetlivestats.com/internet-users/ 38 | -------------------------------------------------------------------------------- /web-apps-for-everyone/02-progressive-enhancement.md: -------------------------------------------------------------------------------- 1 | # Progressive Enhancement 2 | 3 | Progressive enhancement is a term that often insights intense debate. For many, progressive enhancement can be summed up as “make your site work without JavaScript.” While, developing a site that works without JavaScript often does fall under the web of progressive enhancement, it can define a much more nuanced experience. 4 | 5 | In Aaron Gustafson’s seminal A List Apart Article, [Understanding Progressive Enhancement](http://alistapart.com/article/understandingprogressiveenhancement), he describes progressive enhancement as a peanut M&M. The peanut being the core experience, which is essential to the user. The chocolate encompasses the features and design that take us beyond the naked peanut experience and add some much loved flavor. Finally, the candy shell, though not necessarily needed, provides added features, such as not melting in your hand. Oftentimes this example is translated to HTML as the peanut, CSS as the chocolate and JavaScript as the candy shell. 6 | 7 | In today’s web application landscape it may be over simplified to consider progressive enhancement as simply “works without JavaScript.” In fact, many of the rich interactions and immersive experiences that have come to define the modern web certainly require JavaScript. For progressive enhancement to be considered an ethical issue in web development, we must tie it back to user needs. Progressive enhancement is about defining what a user needs to get from your website and ensuring that it is always delivered to them, in a way that will work regardless of network conditions, device, or browser. 8 | 9 | I prefer [Jeremy Keith’s take](https://youtu.be/-yIbKaA3wCo) that progressive enhancement is a “process” rather than a specific technique or set of technologies. By Keith’s definition this process looks like: 10 | 11 | 1. Identify the core functionality 12 | 2. Make that functionality available using the simplest technology 13 | 3. Enhance! 14 | 15 | As a developer it is our job to define what is the core functionality of our application and what is an enhancement. This allows us to develop a baseline to build from, yet the baseline for any given project may be different. 16 | 17 | In his 2012 article, [Stumbling on the Escalator](https://www.christianheilmann.com/2012/02/16/stumbling-on-the-escalator/), Christian Heilmann appropriated a Mitch Helberg comedy bit about escalators for progressive enhancement: 18 | 19 | > An escalator can never break – it can only become stairs. You would never see an “Escalator Temporarily Out Of Order” sign, just “Escalator Temporarily Stairs. Sorry for the convenience. We apologize for the fact that you can still get up there.” 20 | 21 | As a person who has spent a lot of time in the Washington DC metro system, I can really appreciate this analogy. Fortunately, when an escalator is out I am not trapped underground, but instead can huff up the now stairs to the street. 22 | 23 | Oftentimes, when beginning a project, I am presented with a set of business requirements or a beautiful design. From these, it can be easy to see the end goal, but skip the baseline experience. If, in the case of the escalator, my requirement was to “build a transportation system that will allow Metro riders to travel from the terminal to the street,” my initial reaction may be to create an elevator. You can imagine how this might become problematic. 24 | 25 | Developing web apps works in much the same way. If we only consider the end goal, we run the risk of leaving our users stranded. By focusing on and providing a solid baseline for our users, we set ourselves up for success in many other aspects of ethical web development, such as accessibility and performance. 26 | 27 | ## Defining core functionality 28 | 29 | If progressive enhancement is the process of defining a core functionality and building up from there, how do we define that initial baseline? The goal is to consider the bare minimum that a user requires to use our application. Once we have defined this, we can layer on additional style and functionality. For some applications this may be a completely JavaScript free version of the experience, while for others it may be a less fully featured version, and still others it may be providing some server rendered content on the initial page load only. 30 | 31 | The key is to think of progressive enhancement not as a binary option, but instead as a range, determining what is the best decision for users. In this way, progressive enhancement is a gradient rather than an either/or option. Our responsibility is to decide where on this gradient our particular application falls. 32 | 33 | ![gradient image representing progressive enhancement](img/progressive-enhancement-gradient.png) 34 | 35 | I’d encourage you to take a few minutes and consider what the core functionality might look like for a few different types of websites and applications. Identify the primary goal of the site and determine the minimum amount of technology needed to implement it. To take it a step further, write some markup or pseudo code explaining how you might implement those baseline and features. 36 | 37 | - News website 38 | - Social network (write text posts and read the feed of friends) 39 | - Image sharing website 40 | - Web chat application 41 | - Video chat application 42 | 43 | When working on your own applications, try to perform the same exercise. First, determine the core functionality for your users and build the application from there. This programatic approach also pairs well with the [Agile](http://www.agilemanifesto.org/) approach to software development, where the goal is to deliver working software at the end of each development sprint. If we first deliver a core experience, we can iteratively build upon that experience while continuing to deliver value. 44 | 45 | ## Progressive Enhancement Is Still Relevant 46 | 47 | Some may question how relevant progressive enhancement is today, when a small percentage of users browse the web with JavaScript disabled[^1]. This places the focus too heavily on progressive enhancement as a JavaScript free version of a site. In fact, some types of applications, such as video chat, absolutely require some form of JavaScript to work in the browser. The goal of progressive enhancement is to provide the absolute minimum for a working product and ensure that it is delivered to user's browser. 48 | 49 | Ideally, this working minimum product is simply HTML without any external resources such as images, video CSS, or JavaScript. When a user's browser requests our site we can be certain that they will receive HTML (or nothing at all). By creating a working version of our application, even with a minimal experience, using the smallest number of assets, we can be sure that the user is able to access our content in some form. 50 | 51 | The Digital Services team at Gov.uk [provides a number of scenarios](https://www.gov.uk/service-manual/making-software/progressive-enhancement.html#it-isnt-about-javascript-off) where asset requests may fail: 52 | 53 | > - temporary network errors 54 | > - DNS lookup failures 55 | > - server that the resource is found on could be overloaded or down, and fail to respond in time or at all 56 | > - large institutions (eg banks and financial institutions, some government departments) having corporate firewalls that block, remove or alter content from the Internet 57 | > - mobile network providers resampling images and altering content so that load times faster and reduce bandwidth consumed 58 | > - antivirus and personal firewall software that will alter and/or block content 59 | 60 | Additionally, in the blog post [How many people are missing out on JavaScript enhancement?](https://gds.blog.gov.uk/2013/10/21/how-many-people-are-missing-out-on-javascript-enhancement/), they add: 61 | 62 | > - existing JavaScript errors in the browser (ie from browser add-ons, toolbars etc) 63 | > - page being left between requesting the base image and the script/noscript image 64 | > - browsers that pre-load pages they incorrectly predict you will visit 65 | 66 | While those of us developing for the web often have nice hardware and speedy web connections, that may not be true for many of our potential users. Those in developing or rural areas may have limited or outdated devices and slow connections. In 2015 the Facebook development team began participating in [2G Tuesdays](http://www.businessinsider.com/facebook-2g-tuesdays-to-slow-employee-internet-speeds-down-2015-10), which allows them to experience their applications as though they are being served over these slower networks. I would encourage you to do the same. 67 | 68 | Today's browser development tools [allow us to mimic network conditions](https://developers.google.com/web/tools/chrome-devtools/profile/network-performance/network-conditions), experiencing what it is like for these users to access our sites. We will explore the topic of web performance in greater detail in the an upcoming title in this series. 69 | 70 | ![screenshot of the network connectivity tools in Google Chrome](img/network-connectivity-chrome.png) 71 | 72 | Though you may have never used, tested an application with, or even heard of it, the [Opera Mini](http://www.opera.com/mobile) browser currently has over 300 million users worldwide[^2]. The browser is designed to greatly decrease mobile bandwidth usage by routing pages through Opera's servers and optimizing them. To do this, Opera Mini only supports a subset of modern web standards. Here are a few of the things that are unsupported by Opera Mini: 73 | 74 | - Web fonts (which also means no icon fonts) 75 | - HTML5 structural elements and form features 76 | - Text decoration styling 77 | - Video and audio elements 78 | 79 | The site [WTF Opera Mini](http://wtfoperamini.com/) collects the full set of modern web standards that are not supported in the browser. As you can imagine, without a progressive enhancement strategy, our sites may be completely inaccessible for all 300 million+ Opera Mini users. 80 | 81 | If developing an application that is exclusively for users who are likely in an urban area with strong Internet speeds and nice hardware, it may feel as if we are exempt from concerning ourselves with connection issues. Recently, developer [Jake Archibald](https://jakearchibald.com/) coined the termed Lie-Fi. This is a connection where our mobile device seems to be connected to wifi, but sites are slow to load as they feebly connect to our struggling signal. 82 | 83 | In addition to the conditions described above, there may be external factors at play. In 2014 the UK's Sky broadband accidentally briefly [blocked the jQuery CDN](http://www.thinkbroadband.com/news/6261-sky-parental-controls-break-jquery-website.html), presumably leaving many users perplexed with broken websites. More recently the ability to compose and publish a Tweet [became unavailable in the Twitter web client](http://molily.de/javascript-failure/). This was caused by a regular expression which was being served by a CDN without the proper character encoding. Though the JavaScript likely worked in all local and quality assurance testing, once it was available on the web it disabled one of the site’s most critical features. 84 | 85 | ### Run your own experiment 86 | 87 | The gov.uk's digital services team was curious to see how many users were missing out on JavaScript resources when accessing their site. To test this, they [ran an experiment](https://gds.blog.gov.uk/2013/10/21/how-many-people-are-missing-out-on-javascript-enhancement/) by adding three images to a page: 88 | 89 | - An image that all browsers should request 90 | - An image that would only be requested via JavaScript 91 | - An image that only browsers with JavaScript disabled would request 92 | 93 | The results of this experiment are really fascinating. Though only a fraction of a percentage of users requested the JavaScript disabled image, those that failed to load the image requested via JavaScript were significantly higher. 94 | 95 | If possible, I'd encourage you and your teams to conduct a similar experiment. This allows us to base the decision to support (or not support) Javascript-disabled users with data, rather than assumptions or world averages. 96 | 97 | To run this experiment on your own site, first create three empty gif files named `base-js.gif`, `with-js.gif`, `without-js.gif`. Then you can use the following snippet in your HTML, which as been adapted from gov.uk’s experiment: 98 | 99 | ``` 100 | 102 | 115 | 119 | ``` 120 | 121 | This will place the tiny gifs off screen, appending the `with-js.gif` file to a `wrapper` element when a user's JavaScript executes. Users without JavaScript will have the `without-js.gif` image, while all users will receive `base-js.gif`. 122 | 123 | [^1]: In 2010 Yahoo conducted what is considered the [definitive study of JavaScript usage](https://developer.yahoo.com/blogs/ydn/many-users-javascript-disabled-14121.html) finding that the percentage of users with JavaScript disabled ranged from 0.26% to 2.06%, depending on the country of origin. Sadly, these statistics are woefully out of date. In 2013 the UK's Digital Services team did a [similar study](https://gds.blog.gov.uk/2013/10/21/how-many-people-are-missing-out-on-javascript-enhancement/) and found that 1.1% of their users were not receiving JavaScript. The German site [darwe.de](http://darwe.de) analyzes JavaScript enablement in real time and shows a [much larger percentage](http://www.darw.de/statistik/statistik-js.php) of users with JavaScript disabled visiting their site. 124 | 125 | [^2]: [The unknown browser with 300 million users that’s breaking your site](http://thenextweb.com/dd/2015/12/24/the-unknown-browser-with-300-million-users-thats-breaking-your-site/) 126 | 127 | ## How can we approach progressive enhancement today? 128 | 129 | Recently, I was talking with my friend and colleague [Scott Cranfill](http://www.scottcranfill.com) about a progressive enhancement strategy for a project he was working on. This project was mostly static content, but also included an auto loan calculator. When discussing how he might approach this from a progressive enhancement angle, he mentioned that he thought he default markup should simple include the formula that the calculator uses. Once the page's assets load, a fully functional dynamic calculator will display. This means that nearly every user will only see and interact with the calculator, but should something go wrong, a user will still be presented with something that is potentially useful. I loved this pragmatic approach. It wasn't about “making it work without JavaScript,” but instead about making it work for everyone. 130 | 131 | In the world of modern JavaScript driven web applications, there are still several practical approaches we can take to build progressively enhanced sites. These approaches can be simple or leverage exciting web technology buzz words such as Isomorphic JavaScript or Progressive Web Applications. Since progressive enhancement is not a one sized fits all approach, you may want to evaluate this options and choose the one that best works for your project. 132 | 133 | Let's take a look at a few of these options and how they may be used to build the best possible experience for a user. 134 | 135 | Perhaps the simplest and most progressive is to completely avoid a JavaScript dependent first page render. By rendering all of the **necessary** content on the server, we can ensure that a user receives a useable page, even if only our HTML makes it to their browser. The key here is to focus on what is necessary. There may be additional JavaScript required functionality, but if it isn't necessary we can allow it to quietly fail in the background or present the user with different content. 136 | 137 | With this (or any approach), if you choose to serve a library from a CDN, I would recommend following the [HTML5 Boilerplate's](https://html5boilerplate.com/) lead to provide a local fallback as well. This allows us to leverage the benefits of the CDN while ensuring that the user has the opportunity to downloads the scripts should there be an issue with the CDN, such as unexpected down time or being blocked by an ISP or third party browser add-on. 138 | 139 | ```html 140 | 141 | 142 | ``` 143 | 144 | Another option, or one that may paired with the previous, is to sniff out outdated browsers and avoid serving JavaScript to those browsers. We can continue to serve our core content and functionality to those browsers (it was progressively enhanced after all!), but offer a significantly simpler experience. 145 | 146 | To sniff out older browsers, we can use a technique demonstrated by Jake Archibald from [his talk](https://youtu.be/EVEiIlJSx_Y) at Nordic.js in 2015. This checks for the availability of the Page Visibility JavaScript API, which is available in only present in modern browsers [^3]. If Page Visibility is unavailable, the code exits without attempting to execute. 147 | 148 | ```javascript 149 | (function() { 150 | if (!('visibilityState' in document)) { 151 | return false; 152 | } 153 | 154 | // rest of your code 155 | }()); 156 | ``` 157 | 158 | 161 | 162 | [^3]: http://caniuse.com/#feat=pagevisibility 163 | 164 | For JavaScript dependent applications we could render the landing page as HTML on the server while prefetching the JavaScript for the rest of the application. 165 | 166 | ``` 167 | 168 | ``` 169 | 170 | This approach gives our user's the opportunity to download and cache the application's JavaScript, without impacting the performance or requirement on a mostly static page. Soon browsers will begin implementing the [Preload specification](https://w3c.github.io/preload/), which will be similar to Prefetch, but enable additional browser features. 171 | 172 | In action preload looks similar to prefetch: 173 | 174 | ``` 175 | 176 | ``` 177 | 178 | ### Prefetch and Preload resources 179 | 180 | - [Prefetching, preloading, prebrowsing](https://css-tricks.com/prefetching-preloading-prebrowsing/) 181 | - [HTML5 Prefetch](https://medium.com/@luisvieira_gmr/html5-prefetch-1e54f6dda15d) 182 | - [Preload: What Is It Good For?](https://www.smashingmagazine.com/2016/02/preload-what-is-it-good-for/) 183 | 184 | You may be thinking, “but I want to build *modern* JavaScript web applications.” Certainly these techniques feel out of sync with the approaches of some of the popular JavaScript frameworks, but recently we've seen the most popular web application approaches trend back towards a progressive enhancement model. 185 | 186 | Isomorphic or Universal JavaScript is a technique that allows a developer to pair server and client side JavaScript into a write once, run everywhere approach. This technique means that the initial application will render on the server, using Node.js and then run in the browser. When building a progressively enhanced Isomorphic app we can start by building our server rendered version of the applications and layer on the Isomorphic approach. 187 | 188 | A similar approach was taken by the team behind the recent [Google+ redesign](https://developers.google.com/web/showcase/case-study/googleplus): 189 | 190 | > With server-side rendering we make sure that the user can begin reading as soon as the HTML is loaded, and no JavaScript needs to run in order to update the contents of the page. Once the page is loaded and the user clicks on a link, we do not want to perform a full round-trip to render everything again. This is where client-side rendering becomes important — we just need to fetch the data and the templates, and render the new page on the client. This involves lots of tradeoffs; so we used a framework that makes server-side and client-side rendering easy without the downside of having to implement everything twice — on the server and on the client. 191 | 192 | 199 | 200 | If a fully Isomorphic JavaScript approach is overkill for an application, Henrik Joreteg has coined the term [Lazymorphic applications](https://blog.andyet.com/2015/05/18/lazymorphic-apps-bringing-back-static-web/). A Lazymorphic app is simply one where the developer pre-renders as much of the application as possible as static files at build-time. Using this approach we can choose what we render, making something useful for the user while withholding JavaScript dependent features. 201 | 202 | Lastly, the term [Progressive Web Apps](https://developers.google.com/web/progressive-web-apps) has recently taken hold. Rather than specific technology, this term has come to encompass several interrelated techniques and approaches to web development. This is an approach that pairs nicely with all of those listed above. 203 | 204 | 205 | In his article [Progressive Web Apps: Escaping Tabs Without Losing Our Soul](https://infrequently.org/2015/06/progressive-apps-escaping-tabs-without-losing-our-soul/), Alex Russell described progressive web applications in this way: 206 | 207 | > - Responsive: to fit any form factor 208 | > - Connectivity independent: Progressively-enhanced with Service Workers to let them work offline 209 | > - App-like-interactions: Adopt a Shell + Content application model to create appy navigations & interactions 210 | > - Fresh: Transparently always up-to-date thanks to the Service Worker update process 211 | > - Safe: Served via TLS (a Service Worker requirement) to prevent snooping 212 | > - Discoverable: Are identifiable as “applications” thanks to W3C Manifests and Service Worker registration scope allowing search engines to find them 213 | > - Re-engageable: Can access the re-engagement UIs of the OS; e.g. Push Notifications 214 | > - Installable: to the home screen through browser-provided prompts, allowing users to “keep” apps they find most useful without the hassle of an app store 215 | > - Linkable: meaning they’re zero-friction, zero-install, and easy to share. The social power of URLs matters. 216 | 217 | The Progressive Web Application approach described above is well aligned to an ethical web application experience by focusing on delivering an application experience that works for every user. 218 | 219 | 228 | 229 | 230 | ## In Summary 231 | 232 | There are a variety of techniques and approaches that allows us to build progressively enhanced modern web sites and applications. I've outlined a few of them above. By beginning with the core functionality, we are able to ensure that our application works for the maximum number of people. This provides us with a baseline to provide working software for all users in a range of situations. 233 | 234 | From an ethical standpoint, progressive enhancement provides several benefits to our users. By following a progressive enhancement process, we can be sure that we are building our applications in a way that allows them to be available for as many users as possible, regardless of device, connection, or browser. 235 | 236 | ### Additional resources 237 | 238 | - [Understanding Progressive Enhancement](http://alistapart.com/article/understandingprogressiveenhancement) 239 | - [Progressive enhancement: How to create pages that work regardless of browser capability](https://www.gov.uk/service-manual/making-software/progressive-enhancement.html) 240 | - [Cutting the Mustard](http://responsivenews.co.uk/post/18948466399/cutting-the-mustard) 241 | - [Progressive enhancement is still important](https://jakearchibald.com/2013/progressive-enhancement-still-important/) 242 | - [Stop Breaking the Web](https://ponyfoo.com/articles/stop-breaking-the-web) 243 | -------------------------------------------------------------------------------- /web-apps-for-everyone/04-inclusive-forms.md: -------------------------------------------------------------------------------- 1 | # Developing inclusive forms 2 | 3 | Forms allow users to interact directly with a site. They are often the thing that differentiates a web site from a web application. 4 | 5 | ## What’s in a name? 6 | 7 | In Dale Carnegie’s influential 1936 self-help book, *How To Win Friends and Influence People*, he states “a person's name is, to that person, the sweetest and most important sound in any language.” Names are a core part of our personal identities. We often identify with them, turn at the sound of them said across the room, and intuitively appreciate when a person we have just met understands our names. 8 | 9 | Unfortunately, as web developers, it is possible to make assumptions about names that lead to their incorrect handling. When working with names, we should be prepared for a variety of characters, spacing, and unique international formats. 10 | 11 | In his article [Falsehoods Programmers Believe About Names](http://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/), Patrick McKenzie lists out 40 common misconceptions. Among them, the assumptions that: 12 | 13 | > - People have exactly one canonical full name. 14 | > - People’s names fit within a certain defined amount of space. 15 | > - People’s names are written in any single character set. 16 | > - People have last names, family names, or anything else which is shared by folks recognized as their relatives. 17 | > - My system will never have to deal with names from China, Japan, Korea, Ireland, the United Kingdom, the United States, Spain, Mexico, Brazil, Peru, Russia, Sweden, Botswana, South Africa, Trinidad, Haiti, France, or the Klingon Empire, all of which have “weird” naming schemes in common use. 18 | 19 | The full list is well worth a read, as it succinctly points out many potential missteps. 20 | 21 | In her article, [Hello, My Name is ](http://alistapart.com/article/hello-my-name-is-error), Aimee Gonzalez-Cameron shares her story of taking the GRE, an exam administered for admission to Graduate School in the United States. One of the first instructions in registering for the exam was as follows: 22 | 23 | > Important: The name you use when you register for a GRE test must exactly match (excluding accents, apostrophes and spaces) the name on the identification (ID) documents that you will present on the day of your GRE test. If it does not, you may be prohibited from taking the test or your test scores may be canceled after you take the test. For example, a last name of Fernandez de Córdova would be entered as FernandezdeCordova. 24 | 25 | As she points out, “Students shouldn’t stress about instructions or worry that their answers will be thrown out because they can’t complete the first step correctly.” The lack of a technical system that properly handles a common American surname format is both culturally insensitive and requires extra instruction for correct handling. 26 | 27 | Perhaps relatable from the perspective of many developers is the case of Christopher Null. Without reading further, you may already be shaking your head at the heartache that a last name of “Null” may cause when dealing with web forms. In his article, [Hello, I’m Mr. Null. My Name Makes Me Invisible to Computers](http://www.wired.com/2015/11/null/), he details his experience using the web with the last name of Null. Since “null” is used to represent an empty string in the majority of programming languages, it is sometimes used to check for blank form fields. Because of this, many form fields will assume the field is blank, report an error, or crash, forcing him to use a different last name. 28 | 29 | As developers, we can take a more inclusive strategy to working with names, treating these not as edge cases, but instead by expecting a wide variety of potential inputs. 30 | 31 | ### International names 32 | 33 | Names come in many different formats around the world, however it is easy to apply our own cultural biases when designing systems that deal with names. As an American, for instance, my bias is to consider names in the format of a first name followed by a surname. Based on that format I make several, potentially false assumptions, about things such as familial relationship. However, there are many different ways that a name can be constructed even with a single country or culture. Let’s look at a few of these structures to see how they may challenge our assumptions. 34 | 35 | 36 | #### Multiple names 37 | 38 | Many names may be longer than the given name, family name format. In many Spanish and Portuguese speaking countries it is common to compose a name of one or two given names and two or three family names consisting of the mother’s surname followed by the father’s surname. In some cases, the conjunction de (“of”) may be added between the maternal and paternal surnames, or sometimes surnames may reflect geographic origin. 39 | 40 | Arabic names are traditionally much longer than given and family names, often having specific meaning. This description from [Wikipedia](https://en.wikipedia.org/wiki/Arabic_name) highlights the false assumptions that a non-Arabic speaking person may make about the traditional Arabic name Abdul Rahman bin Omar al-Ahmad: 41 | 42 | > With “Abdul”: Arabic names may be written “Abdul (something),” but “Abdul” means “servant of the” and is not, by itself, a name. Thus for example, to address Abdul Rahman bin Omar al-Ahmad by his given name, one says “Abdul Rahman,” not merely “Abdul”. If he introduces himself as “Abdul Rahman” (which means “the servant of the Merciful”), one does not say “Mr. Rahman” (as “Rahman” is not a family name but part of his (theophoric) personal name); instead it would be Mr. al-Ahmad, the latter being the family name. 43 | 44 | 45 | #### Name order 46 | 47 | Names do not always appear in the format of a given name followed by a family name, meaning that a typical form field of “First name” followed by “Last name,” may not produce the intended results. As an example, Chinese names place the surname before the personal name. 48 | 49 | Rather than a family surname, Icelandic names follow a patronymic (and, occasionally, matronymic) naming format. For example if an Icelandic man named Birgir has a son named Jón, Jón’s full name would be Jón Birgisson (“Birgir’s son”). If Jón then had a daughter named Sigrún, Sigrún would be named Sigrún Jónsdóttir (“Jon’s daughter”). Because of this, a list of Icelandic names would be expected to be sorted by given name rather than family name. 50 | 51 | #### Characters 52 | 53 | Names from many regions may consist of characters outside of the latin alphabet. There are those that may not make use of the latin alphabet in written form such as Arabic, Cyrillic, or Japanese. Though many of these languages also have Romanized versions, such as the Japanese name Yamada Tarō (山田太郎). There are also accented characters such as ó, ü, and ñ. Names may also contain a mix of ß. Names may contain non letter characters such as apostrophes, such as the Irish name Francis O'Neill, which forms may attempt to strip during validation as unacceptable characters. 54 | 55 | #### Further reading 56 | 57 | These are only a few examples of how names may differ around the world. Additionally, they assume that a person’s name is derived from a single culture, ignoring the possibility that name attributes from multiple cultures may be applied to a person’s name. The W3C’s article [Personal names around the world](https://www.w3.org/International/questions/qa-personal-names) dives into greater detail and links to several additional Wikipedia articles discussing naming formats. 58 | 59 | ### Mojibake 60 | 61 | [Mojibake](https://en.wikipedia.org/wiki/Mojibake) is a term used to describe the garbled set of characters that are produced through an improper use of character encoding. Mojibake is typically caused by text that lacks proper (or any) Unicode encoding. When a user’s name contains special characters it may not be uncommon for them to often see mojibake versions of their name. A quick Twitter image search results for mojibake reveals many encoding issues across the web, though it is likely that the majority go undocumented or are documented without knowing the term. 62 | 63 | In his talk, [Hello, my name is __________.](http://patch.codes/talks/hello-my-name-is/), developer Nova Patch surfaced several examples of Mojibake affecting users of web services. Perhaps the most well documented and consistent mojibake mangling of a name belongs to Nóirín Plunkett, who shared several instances of their mojibaked named on Twitter[^1]. 64 | 65 | ![image OF Nóirín’s tweets displaying mojibake](img/mojibake-tweets.png) 66 | 67 | 70 | 71 | Perhaps one of the more impressive mojibake instances, was of a Russian postal worker who hand [corrected a package’s mojibake](http://text-mode.tumblr.com/post/31409503070/russian-postmen-fix-an-error-caused-by-an). This illustrates how common encoding problems can be when working with Cyrillic languages. In fact, there is even a Russian specific term for mojibake, krakozyabrı. 72 | 73 | ![image of hand decoded mojibake](img/russian-post.jpg) 74 | 75 | (Image via http://text-mode.tumblr.com/) 76 | 77 | [^1]: Here are the Tweets referenced in the image: 78 | - https://twitter.com/noirinp/status/77745010547769344 79 | - https://twitter.com/noirinp/status/151004631818977281 80 | - https://twitter.com/noirinp/status/162264316203114498 81 | - https://twitter.com/noirinp/status/394893223145271296 82 | - https://twitter.com/noirinp/status/600750084410642432 83 | 84 | ### What are we to do? 85 | 86 | Now that we’ve taken a quick look at the importance and value of names, we can consider how we can best implement name inclusive fields in our forms. We can do this by considering the format of the field itself and the way we handle the character encoding of the field. 87 | 88 | #### Input format 89 | 90 | If possible, create name fields that are a single text input. Allow the input field to take in long names as well as accepting special characters and spaces. If possible, avoid limiting the length of the field in your database as well, so that the individual’s name is never truncated when it is returned to them. 91 | 92 | ![When possible use a single input for name fields](img/name-fields.png) 93 | 94 | If you plan to address the user through the web interface, email or other means, it may be worth adding an additional field that asks “What should we call you?” This allows the user to enter the name they most associate themselves with. 95 | 96 | ![If you will address the person add a “what should we call you” field](img/name-what.png) 97 | 98 | #### Character Encoding 99 | 100 | As we’ve seen with Mojibake, character encoding can present its own unique set of challenges. To avoid the accidental mangling of names, we should permit punctuation (such as hyphens and apostrophes), allow spaces, and avoid changing character encoding formats between systems, such as form to database. I’ll save a complete explanation of character encoding for developers who are much smarter than I, but as a rule of thumb use “UTF-8” encoding both on the front-end and database. 101 | 102 | In HTML, simply add the character set meta tag specifying UTF-8 to the `` of the page: 103 | 104 | ``` 105 | 106 | ``` 107 | 108 | 109 | 110 | ### Further Reading 111 | 112 | - [Hello, my name is __________.](http://patch.codes/talks/hello-my-name-is/) 113 | - [Falsehoods Programmers Believe About Names](http://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/) 114 | - [Personal names around the world](https://www.w3.org/International/questions/qa-personal-names) 115 | - [Hello, My Name is ](http://alistapart.com/article/hello-my-name-is-error) 116 | - [Personal Histories](http://www.sarawb.com/2015/01/13/personal-histories/) 117 | - [The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets](http://www.joelonsoftware.com/articles/Unicode.html) 118 | - [No Such Thing as Plain Text](https://www.cqse.eu/en/blog/no-such-thing-as-plain-text/) 119 | - [UTF-8 Everywhere](http://utf8everywhere.org/) 120 | - [Multilingual form encoding](https://www.w3.org/International/questions/qa-forms-utf-8) 121 | 122 | 123 | ## Inclusive gender 124 | 125 | For many, gender is not simply the binary sex of either male or female as determined at birth. The advocacy group, [GLAAD](http://www.glaad.org/) defines transgender as: 126 | 127 | > An umbrella term (adj.) for people whose gender identity and/or gender expression differs from the sex they were assigned at birth. The term may include but is not limited to: transsexuals, cross-dressers and other gender-variant people. 128 | 129 | The [most cited study](http://williamsinstitute.law.ucla.edu/wp-content/uploads/Gates-How-Many-People-LGBT-Apr-2011.pdf) on transgender population numbers in the United States places the transgender population at 0.3% or roughly 700,000 adults in the United States. As the the author of the report and the FiveThirtyEight article, [Why We Don’t Know The Size Of The Transgender Population](https://fivethirtyeight.com/features/why-we-dont-know-the-size-of-the-transgender-population/) these numbers may be inaccurate, tending towards low, due to the lack of non-binary gender options on official forms such as the census as well as a reluctance to provide the information when asked. 130 | 131 | To be as inclusive as possible, we can build systems that accept and respect non-binary gender options. When including gender in a form, my recommendation is to: 132 | 133 | 1. Provide male and female options 134 | 2. Provide a “custom” text input. If data collection is important, you may provide autocomplete suggestions, but still allow custom inputs. 135 | 3. Offer a “prefer not to say” option 136 | 137 | Both Facebook and Google follow patterns similar to those described above. 138 | 139 | Google offers the choices of Male, Female, Decline to State, and Custom in a select menu. 140 | 141 | ![Google offers four gender choices in a select menu](img/google-gender-options.png) 142 | 143 | If a user chooses “Custom,” they are presented with a text input box and a choice of pronoun to be addressed by. 144 | 145 | ![A selection of other displays a text input and choice of pronoun](img/google-gender-other.png) 146 | 147 | By contrast, Facebook requires a binary gender choice during account creation. 148 | 149 | ![Facebook’s sign up form presents users with a binary gender choice](img/facebook-signup.png) 150 | 151 | However, once a person has created a Facebook account they are able to select a more inclusive gender. Facebook’s pattern offers three choices: Male, Female, and Custom. When a user selects “Custom” they are given a text input box with auto-complete suggestions as well as a selection of pronouns to be addressed by. 152 | 153 | ![Facebook allows a user to select a custom gender, offers auto-complete gender suggestions, and provides a choice of pronouns](img/facebook-gender.png) 154 | 155 | 156 | 157 | ### What about titles? 158 | 159 | Forms may often include a title field, with gendered choices such as Mr., Ms., and Mrs. Not requiring these fields or providing a text input option gives users the most control over this option. By doing so, we allow those who prefer not to use a title to do so as well as those with a non-binary gender to not be forced into using a gendered titled. 160 | 161 | ### Further Reading 162 | - [How to ask about gender](http://www.yoomee.com/how-to-ask-about-gender) 163 | - [Think Outside The Box recommendations for forms](http://www.lgbt.cusu.cam.ac.uk/campaigns/think/forms/) 164 | - [Sex and Gender](http://www.formulate.com.au/blog/sex-and-gender) 165 | 166 | ## In summary 167 | 168 | When we ask users to complete a form with personal information we are asking about their personal identity. By considering name formats, internationalization, and gender we provide online spaces that are welcoming and inclusive to all. 169 | -------------------------------------------------------------------------------- /web-apps-for-everyone/05-conclusion.md: -------------------------------------------------------------------------------- 1 | # In conclusion 2 | 3 | Thank you for taking the time to read “Building Web Apps for Everyone.” I hope that through reading this title you see value in using progressive enhancement, considering accessibility, and building inclusive web forms. These encompass a small portion of the work we can do as web developers to ensure that the web is an open and inclusive space for all users. My hope is that you now feel empowered and excited to build applications in this way. 4 | 5 | If throughout your reading you have come across things that are missing or could be improved, I would encourage you to contribute back to the book. This title is available as open source and contributions can be made by: 6 | 7 | - Contributing directly to the GitHub repository with a pull request . 8 | - Creating an issue in the book’s GitHub repository . 9 | - Reaching out to me through [email](mailto:adamdscott@protonmail.com) or [Twitter](https://twitter.com/adamdscott). 10 | 11 | Twenty percent of the proceeds from each Ethical Web Development title will be donated to an organization whose work has a positive impact on the issues described. For this title, I will be donating to the World Wide Web Consortium (W3C). The W3C works to ensure “that the Web remains open, accessible and interoperable for everyone around the world” and authored much of the content that was used to research this title. 12 | 13 | If you are interested in supporting the W3C’s work please consider making a donation at [w3.org/support](https://www.w3.org/support/). 14 | 15 | This title is the first in a series of digital reports I am authoring on the subject of ethical web development. Future titles in the series will cover building web applications that work everywhere, building web applications that respect a user's privacy and security, and working with development peers. You can learn more about the series at [ethicalweb.org](https://ethicalweb.org). 16 | -------------------------------------------------------------------------------- /web-apps-for-everyone/README.md: -------------------------------------------------------------------------------- 1 | # Building Web Applications for Everyone 2 | 3 | - [Preface](https://github.com/ascott1/ethical-web-dev/blob/master/preface.md) 4 | - [Introduction](01-title-intro.md) 5 | - [Progressive Enhancement](02-progressive-enhancement.md) 6 | - [Accessibility](03-accessibility.md) 7 | - [Inclusive Forms](04-inclusive-forms.md) 8 | - [Conclusion](05-conclusion.md) 9 | - [Contributors](contributors.md) 10 | -------------------------------------------------------------------------------- /web-apps-for-everyone/contributors.md: -------------------------------------------------------------------------------- 1 | # Contributors 2 | 3 | ## About the Author 4 | 5 | Adam D. Scott is a developer and educator based in Connecticut. He currently works as a senior front-end development fellow at the Consumer Financial Protection Bureau, where he focuses on building open source tools. Additionally, he has worked in education for over a decade, teaching and writing curriculum on a range of technical topics. His first book, WordPress for Education, was published in 2012. His video course, Introduction to Modern Front-End Development, was published by O'Reilly in 2015. 6 | 7 | ## Technical Reviewers 8 | 9 | Chris Contolini is an open source software developer. He is currently a Senior Technology Fellow at the Consumer Financial Protection Bureau where he focuses on ChatOps and front-end web development. He lives and works from a ten-foot box truck retrofitted with solar panels, running water, and broadband Internet access. He works mostly from national forests and has been known to frequent the Bay Area and Portland, OR. 10 | 11 | ## Editor 12 | 13 | Meg Foley 14 | 15 | ## Other Contributions 16 | 17 | The following people have graciously contributed feedback and improvements. 18 | 19 | - Andy Chosak 20 | - Shashank Khandelwal 21 | 22 | Contributions and suggestions have also been made to the ethicalweb.org site and the core principles of ethical web development. Those contributions are stored at 23 | -------------------------------------------------------------------------------- /web-apps-for-everyone/img/browser-chart.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-for-everyone/img/browser-chart.png -------------------------------------------------------------------------------- /web-apps-for-everyone/img/facebook-gender.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-for-everyone/img/facebook-gender.png -------------------------------------------------------------------------------- /web-apps-for-everyone/img/facebook-signup.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-for-everyone/img/facebook-signup.png -------------------------------------------------------------------------------- /web-apps-for-everyone/img/google-gender-options.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-for-everyone/img/google-gender-options.png -------------------------------------------------------------------------------- /web-apps-for-everyone/img/google-gender-other.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-for-everyone/img/google-gender-other.png -------------------------------------------------------------------------------- /web-apps-for-everyone/img/link-perception.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-for-everyone/img/link-perception.png -------------------------------------------------------------------------------- /web-apps-for-everyone/img/mojibake-tweets.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-for-everyone/img/mojibake-tweets.png -------------------------------------------------------------------------------- /web-apps-for-everyone/img/mojibake-tweets.sketch: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-for-everyone/img/mojibake-tweets.sketch -------------------------------------------------------------------------------- /web-apps-for-everyone/img/name-fields.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-for-everyone/img/name-fields.png -------------------------------------------------------------------------------- /web-apps-for-everyone/img/name-fields.sketch: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-for-everyone/img/name-fields.sketch -------------------------------------------------------------------------------- /web-apps-for-everyone/img/name-what.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-for-everyone/img/name-what.png -------------------------------------------------------------------------------- /web-apps-for-everyone/img/name-what.sketch: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-for-everyone/img/name-what.sketch -------------------------------------------------------------------------------- /web-apps-for-everyone/img/network-connectivity-chrome.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-for-everyone/img/network-connectivity-chrome.png -------------------------------------------------------------------------------- /web-apps-for-everyone/img/placeholder-field.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-for-everyone/img/placeholder-field.png -------------------------------------------------------------------------------- /web-apps-for-everyone/img/placeholder-field.sketch: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-for-everyone/img/placeholder-field.sketch -------------------------------------------------------------------------------- /web-apps-for-everyone/img/progressive-enhancement-gradient.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-for-everyone/img/progressive-enhancement-gradient.png -------------------------------------------------------------------------------- /web-apps-for-everyone/img/russian-post.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-for-everyone/img/russian-post.jpg -------------------------------------------------------------------------------- /web-apps-for-everyone/img/twitter-no-gender.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-for-everyone/img/twitter-no-gender.png -------------------------------------------------------------------------------- /web-apps-privacy-security/01-title-intro.md: -------------------------------------------------------------------------------- 1 | # Introduction 2 | 3 | > All human beings have three lives: public, private, and secret. 4 | 5 | ― Gabriel Garcí­a Márquez, Gabriel García Márquez: a Life 6 | 7 | > If only the “controversial” stuff is private, then privacy is itself suspicious. Thus, privacy should be on by default. 8 | 9 | — [Tim Bray](https://www.tbray.org/ongoing/When/201x/2012/12/02/HTTPS) 10 | 11 | We live more and more of our lives digitally. We consistently create and entrust significant portions of our social, health, financial, and work data with web services. We then link that data together by connecting accounts and permitting the services that we use to track the other sites we visit. Our use of search engines can predict patterns and insights into our health and personalities. In 2016 John Paparrizos MSc, Ryen W. White PhD, and Eric Horvitz MD, PhD published [a study](http://jop.ascopubs.org/content/early/2016/06/02/JOP.2015.010504.abstract) where they were able to use anonymized Bing search queries to predict Pancreatic cancer. 12 | 13 | In the article [With Great Data Comes Great Responsibility](https://medium.com/@jazzpazz/with-great-data-comes-great-responsibility-72d3e1c94e27#.twfg85eus) Pascal Raabe (Paz) eloquently describes the way our digital data represents our lives: 14 | 15 | > We’re now producing more data on a daily basis than through all of history. The digital traces we’re leaving behind with every click, every tweet and even every step that we make create a time machine for ourselves. These traces of our existence form the photo album of a lifetime. We don’t have to rely on memory alone but can turn to technology to augment our biological memories and virtually remember everything. 16 | 17 | In the light of how much data we produce, The security of our data has become a major point of concern among many people. Web surveillance, corporate tracking, and data leaks have even become common leading news stories. In a [2016 Pew Research survey](http://www.pewresearch.org/fact-tank/2016/01/20/the-state-of-privacy-in-america/) on the state of privacy in America, it was found that few Americans are confident in the security or privacy of our data. 18 | 19 | > Americans express a consistent lack of confidence about the security of everyday communication channels and the organizations that control them – particularly when it comes to the use of online tools. And they exhibited a deep lack of faith in organizations of all kinds, public or private, in protecting the personal information they collect. Only tiny minorities say they are “very confident” that the records maintained by these organizations will remain private and secure. 20 | 21 | In 2016 the writer Walter Kirn wrote about the state of modern surveillance for the Atlantic magazine in an article titled [If You’re Not Paranoid, You’re Crazy](http://www.theatlantic.com/magazine/archive/2015/11/if-youre-not-paranoid-youre-crazy/407833/). The online version of the article, hosted on the Atlantic's website, contains at least 17 detected user trackers[^1] (upper right in the image below). Even when we are discussing tracking, we are creating data that is being tracked. 22 | 23 | ![img/atlantic.png](img/atlantic.png) 24 | 25 | [^1]: As detected by the Privacy Badger browser plugin 26 | 27 | ## Our Responsibility 28 | 29 | As web developers we are the first line of defense in protecting our user's data and privacy. In this title, we will explore some ways in which we can work to maintain the privacy and security of our user's data. The four main concepts we'll cover are: 30 | 31 | 1. Respecting user privacy settings 32 | 2. Encrypting user connections with our sites 33 | 3. Working to ensure the security of our user's information 34 | 4. Providing a means for users to export their data 35 | 36 | If we define ethics as "making choices that have the most positive effect for the largest number of people," putting in place strong security protections and placing privacy and data control in the hands of our users can be considered the ethical approach. By taking extra care to respect our user's privacy and security, we are showing greater commitment to their needs and desires. 37 | -------------------------------------------------------------------------------- /web-apps-privacy-security/02-privacy.md: -------------------------------------------------------------------------------- 1 | # Respecting user privacy 2 | 3 | This has happened to all of us, one evening we're shopping for something mundane like new bed sheets by reading reviews and browsing a few online retailers and the next time we open one of our favorite websites up pops an ad for bed linens. What's going on here? Even for those of us that spend our days (and nights) developing for the web this can be confounding. How does the site have access to our shopping habits? And just how much do these sites know about us? 4 | 5 | This feeling of helplessness is not uncommon. According to the [Pew Research Center](http://www.pewresearch.org/fact-tank/2016/01/20/the-state-of-privacy-in-america/), 91% of American adults "agree or strongly agree that consumers have lost control of how personal information is collected and used by companies." Many users may be comfortable giving away information in exchange for products and services, but more often than not the depth and breadth of that information is unclear to the user. Meanwhile, advertising networks and social media sites have bits of code that are spread across the web, tracking users between sites. 6 | 7 | As web developers how can we work to maintain the privacy or our users? In this chapter we'll look at how web tracking works and ways in which we can hand greater privacy controls back to our users. 8 | 9 | 10 | ## How Users are Tracked 11 | 12 | As users browse the web, they are being tracked and, as web developers, we are often enabling and supporting that surveillance. This isn't a case of tin-foil hat paranoia as we introduce the code of ad networks to support our work, add social media share buttons that allow users to easily share our site's content, or use analytics software to help us better understand user experience. These sites track user behavior with the intention of providing them with a more unique experience. While this may seem harmless or well intended, this is typically done without the knowledge or permission of the end-user. 13 | 14 | The simplest way that web tracking works is that a user visits a site which installs a cookie from a third-party. When we visit another site with the same third-party tracker, the tracker is notified. This allows the third-party to build a unique user profile. 15 | 16 | ![Diagram of how cookie tracking works](img/cookie-tracking.png) 17 | 18 | The intention of this tracking is typically to provide more targeted services, advertising, or products. However, the things we buy, the news we read, the politics we support, and our religious beliefs are often embedded into our browsing history. To many, without explicit permission this knowledge feels intrusive. 19 | 20 | 21 | ## What does your browser know about you? 22 | 23 | Those aware of user use tracking may take a few steps to beat trackers at their own game. Ad blockers such as [uBlock Origin](https://github.com/gorhill/uBlock/) block advertisements as well as third-party advertising trackers. Other browser extensions such as [Privacy Badger](https://www.eff.org/privacybadger) and [Ghostery](https://www.ghostery.com/) attempt to block all third-party beacons from any source. However, even with tools like these, sites may be able to track users based on the unique footprint their browser leaves behind. In fact, the irony of using these tools according to the W3C slide deck [Is preventing browser fingerprinting a lost cause?](https://www.w3.org/wiki/images/7/7d/Is_preventing_browser_fingerprinting_a_lost_cause.pdf), is that "fine grained settings or incomplete tools used by a limited population can make users of these settings and tools easier to track." 24 | 25 | Browser's can easily detect the user's IP address, user agent, location, browser plugins, hardware, and even battery level. Web developer Robin Linus developed the site [What every browser knows about you](http://webkay.robinlinus.com/) to show off the level of detail available to developers and site owners. Additionally, the tools [Am I Unique?](https://amiunique.org/) and [Panopticlick](https://panopticlick.eff.org) offer quick overviews of how unique your browser fingerprint is. 26 | 27 | 32 | 33 | 34 | ## Do Not Track 35 | 36 | With this information about the ways in which users can be tracked, how can we, as web developers, advocate for our user's privacy? My belief is that the first step is to advocate for the respect of the [Do Not Track](https://www.w3.org/TR/tracking-compliance/)(DNT) browser setting. Do Not Track is a browser setting that allows users to specify a preference to not be tracked by the sites they visit. When a user has enabled the Do Not Track setting in their browser, the browser responds with the HTTP header field `DNT`. 37 | 38 | According to the [Electronic Frontier Foundation](https://www.eff.org/pages/understanding-effs-do-not-track-policy-universal-opt-out-tracking), Do Not Track boils down to sites agreeing not to collect personally identifiable information through methods such as cookies and fingerprinting as well as agreeing not to retain individual user browser data beyond 10 days. The noted exceptions to this policy are when a site is legally responsible for maintaining this information, the information is needed to complete a transaction, or if a user has given explicit consent. 39 | 40 | With Do Not Track enabled, browsers send an HTTP header response with a `DNT` value of `1`. The following is a sample header response, which includes a `DNT` value: 41 | 42 | ``` 43 | Host: "www.example.com" 44 | Accept: "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" 45 | Accept-Language: "en-US,en;q=0.5" 46 | Accept-Encoding: "gzip, deflate, br" 47 | DNT: "1" 48 | ``` 49 | 50 | Do Not Track does not automatically disable tracking in a user's browser. Instead, as developers, we are then responsible for appropriately handling this user request in our applications. 51 | 52 | 53 | 59 | 60 | ### Detecting Do Not Track 61 | 62 | We can easily detect and respond to Do Not Track on the client side of our applications in client-side JavaScript by using the `navigator.doNotTrack` property. This will return a value of `1` for any user who has enabled Do Not Track while returning `0` for a user who has opted in to tracking and `unspecified` for users who have not enabled the setting. 63 | 64 | For example we could detect for the Do Not Track setting and avoid setting a cookie in a user's browser: 65 | 66 | ``` 67 | // store user do not track setting s a variables 68 | var dnt = navigator.doNotTrack; 69 | 70 | if (dnt !==1) { 71 | // set cookie only if DNT not enabled 72 | document.cookie = 'example'; 73 | } 74 | ``` 75 | 76 | The site [DoNotTrack.us](http://donottrack.us/), created and maintained by Stanford and Princeton researchers Jonathan Mayer and Arvind Narayanan, helpfully offers web server configurations and templates for web application frameworks in ASP, Java, Perl, PHP, and Django. 77 | 78 | Here is the recommended code when working with the Django framework, which offers a good example for any framework or language: 79 | 80 | ``` 81 | DoNotTrackHeader = "DNT" 82 | DoNotTrackValue = "1" 83 | 84 | pyHeader = "HTTP_" + DoNotTrackHeader.replace("-", "_").upper() 85 | 86 | # request is an HttpRequest 87 | if (pyHeader in request.META) and (request.META[pyHeader] == DoNotTrackValue): 88 | # Do Not Track is enabled 89 | else: 90 | # Do Not Track is not enabled 91 | ``` 92 | 93 | Since [DoNotTrack.us](http://donottrack.us/) does not offer a Node.JS example of detecting Do Not Track, here is a simple HTTP Server that will check for the DNT header response from a user's browser: 94 | 95 | ``` 96 | var http = require('http'); 97 | 98 | http.createServer(function (req, res) { 99 | 100 | var dnt = req.headers.dnt === '1' || false; 101 | 102 | if (dnt) { 103 | // Do Not Track is enabled 104 | } else {; 105 | // Do Not Track is not enabled 106 | } 107 | 108 | res.end(); 109 | }).listen(3000); 110 | ``` 111 | 112 | Additionally the npm package [tinfoilhat](https://www.npmjs.com/package/tinfoilhat) offers an interface for detecting the Do Not Track setting in Node and executing a callback based on the user's setting. 113 | 114 | Based on these examples, we can see that detecting a user's Do Not Track setting is relatively straightforward. Once we have taken this important first step, how do we handle Do Not Track requests? 115 | 116 | ### Respecting Do Not Track 117 | 118 | The Mozilla Developer Network helpfully offers [DNT case studies](https://developer.mozilla.org/en-US/docs/Web/Security/Do_not_track_field_guide/Case_studies) and the site DoNotTrack.us provides [The Do Not Track Cookbook](http://donottrack.us/cookbook/), which explore a number of Do Not Track usage scenarios. The examples include practical applications of Do Not Track for advertising companies, technology providers, media companies, and software companies. 119 | 120 | 121 | ### Sites that Respect Do Not Track 122 | 123 | Some well known social sites have taken the lead on implementing Do Not Track. [Twitter supports Do Not Track](https://support.twitter.com/articles/20169453?lang=en) by disabling tailored suggestions and tailored ads when a user has the setting enabled. However, it's worth noting that Twitter does not disable analytic tracking or third-party advertising tracking that uses Twitter data across the web. [Pinterest supports Do Not Track](https://help.pinterest.com/en/articles/we-support-do-not-track) and according to the site's [Privacy Policy](https://help.pinterest.com/en/articles/personalization-and-data), a user with Do Not Track enabled is opted out of Pinterest's personalization feature, which tracks users around the web in order to provide further customization of Pinterest content. 124 | 125 | Medium.com has a clear and effective [Do Not Track Policy](https://medium.com/policy/how-we-handle-do-not-track-requests-on-medium-f2b4b4fb7c5e). When a user with Do Not Track enabled is logged in, they are presented with this message: 126 | 127 | > You have Do Not Track enabled, or are browsing privately. Medium respects your request for privacy: to read in stealth mode, stay logged out. While you are signed in, we collect some information about your interactions with the site in order to personalize your experience, offer suggested reading, and connect you with your network. More details can be found here. 128 | 129 | Medium also states that they do not track users across other websites around the web. This policy is clear and consistent, providing a strong example of how a successful site can respect a user's Do Not Track setting. 130 | 131 | The site DoNotTrack.us offers a [list of companies honoring Do Not Track](http://donottrack.us/implementations), including advertising companies, analytics services, data providers, and more. Unfortunately this list appears to be incomplete and outdated, but offers a good jumping off point for exploring exemplars across a range of industries. 132 | 133 | ## Web Analytics 134 | 135 | One of the biggest challenges of handling user privacy is determining best practices for web analytics. By definition, the goal of web analytics is to track users, though the aim is typically to better understand how our sites are used so that we can continually adapt and improve them to user needs. 136 | 137 | To improve user privacy, when using analytics, we should ensure that our analytics provider anonymizes our users, limits tracking cookies to our domain, and that it does not share user information with third parties. The [US Government's digital analytics program](https://analytics.usa.gov/#explanation) has taken this approach, through ensuring that Google Analytics does not track individuals, share information with third parties, and anonymizes all user IP addresses. 138 | 139 | As an additional example, the analytics provider [Piwik](https://piwik.org) actively seeks to [maintain user privacy](http://piwik.org/blog/2014/01/data-privacy-day-january-28th/) while working with user analytics through: 140 | 141 | - Providing an analytics opt-out mechanism 142 | - Deleting logs older than a few months 143 | - Anonymizing IP addresses 144 | - Respecting Do Not Track 145 | - Setting a short expiration date for cookies 146 | 147 | These provide a good baseline for how we should aim to handle analytics on our site with any provider. By taking this extra care with user information, we may continue to use analytics to provide greater insights into use of our sites while maintaining user privacy. 148 | 149 | ## De-identification 150 | 151 | Though avoiding the tracking of users completely is preferred, there may be instances where this choice is outside of our control as web developers. In these cases, we may be able to guide the decision to de-identify collected user data, ensuring that the privacy of our users remains intact. The goal of any de-identification is to ensure that any collected data cannot be used to identify the person who created the data in any way. 152 | 153 | However, de-identification is not without its limitations, as de-identified data sets can be paired with other data sets to identify an individual. In the paper [No silver bullet: De-identification still doesn't work](http://randomwalker.info/publications/no-silver-bullet-de-identification.pdf) Arvind Narayanan and Edward W. Felten explore the limits of de-identification. Cryptographic techniques such as [differential privacy](https://en.wikipedia.org/wiki/Differential_privacy) can be used as another layer to help to limit the identification of individual users within collected datasets. 154 | 155 | ## User Consent and Awareness 156 | 157 | In 2011 the European Union passed legislation requiring user consent before using tracking technology. Specifically the [privacy directive specifies](http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:32002L0058:EN:HTML): 158 | 159 | > Member States shall ensure that the use of electronic communications networks to store information or to gain access to information stored in the terminal equipment of a subscriber or user is only allowed on condition that the subscriber or user concerned is provided with clear and comprehensive information in accordance with Directive 95/46/EC, inter alia about the purposes of the processing, and is offered the right to refuse such processing by the data controller. 160 | 161 | This means that any site using cookies, web beacons, or similar technology must inform the user and receive explicit permission before tracking the user. If you live in Europe or have visited a European website, you are likely familiar with the common request to track banner. This law is not without [controversy](http://celso.io/2016/01/31/cookies.html) as many feel that these banners are ignored, viewed as a nuisance, or otherwise not taken seriously. 162 | 163 | In the UK, the guidance has been to simply inform users that they are being tracked, providing no option to opt-out. For example, the site [ico.org.uk](https://ico.org.uk), the "UK’s independent authority set up to uphold information rights in the public interest, promoting openness by public bodies and data privacy for individuals," opts users in, but clicking the "Information and Settings" link provides information about browser settings and disabling cookies on the site. 164 | 165 | ![Image of ico.org.uk's cookie alert](img/ico-cookie.png) 166 | 167 | Though based in the United States, the site Medium.com alerts users, with DNT enabled, how their information will be used and expresses consent when a user is logging in. For a private experience, Medium recommends remaining logged out and commits to not tracking those users. 168 | 169 | ![Medium.com's login screen with DNT enabled](img/medium-dnt.png) 170 | 171 | ### Creating a Do Not Track Policy 172 | 173 | While there is value in informing users, I believe that the best way to provide privacy controls to users is by respecting the "Do Not Track" browser setting. This allows users to set a privacy preference once and forget it, rather than maintaining individual settings across the web. Since there is no absolute definition of what Do Not Track encompasses, to effectively implement it you will likely need to develop a DNT policy for your site or application. 174 | 175 | The Electronic Frontier Foundation(EFF) provides a [sample Do Not Track policy](https://www.eff.org/dnt-policy). This document serves as a solid foundation for any site's Do Not Track policy and can be used verbatim or adapted for an organization's needs. EFF also provides a set of [frequently asked questions](https://www.eff.org/dnt-policy#faq) and a [human readable summary](https://www.eff.org/pages/understanding-effs-do-not-track-policy-universal-opt-out-tracking) of the policy. 176 | 177 | By committing to a Do Not Track policy we are able to ensure that we comply with the tracking preferences of our users. 178 | 179 | ## Further Reading 180 | 181 | - [The Emerging Ethical Standards for Studying Corporate Data](http://www.recode.net/2016/6/14/11923286/facebook-emotional-contagion-controversy-data-research-review-policy-ethics) by Jules Polonetsky and Dennis Hirsch 182 | - [Do Not Track Is No Threat to Ad Supported Business](https://cyberlaw.stanford.edu/blog/2011/01/do-not-track-no-threat-ad-supported-businesses) by Jonathan Mayer 183 | - [Electronic Frontier Foundation's Guide to Do Not Track](https://www.eff.org/issues/do-not-track) 184 | - [Mozilla: The Do Not Track Field Guide](https://developer.mozilla.org/en-US/docs/Web/Security/Do_not_track_field_guide) 185 | - [W3C: Tracking Compliance and Scope](https://www.w3.org/TR/2015/WD-tracking-compliance-20150714/) 186 | - [W3C: Header Field Definitions](https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html) 187 | -------------------------------------------------------------------------------- /web-apps-privacy-security/03-https.md: -------------------------------------------------------------------------------- 1 | # Encrypting User Connections with HTTPS 2 | 3 | "S is for secure" may sound like a line from a children's TV show, but when appended to HTTP that's exactly what it means. HTTPS was first developed for use in [Netscape Navigator](https://books.google.com/books?id=FLvsis4_QhEC&pg=PA344&hl=en#v=onepage&q&f=false) in 1994 and became an important indicator of security for e-commerce and banking sites on the developing web. As we move an ever increasing amount of personal data and information across the web, ensuring user privacy and the authenticity of information become increasingly important. 4 | 5 | Over a standard HTTP connection, users are open to advertising injection, content changes, and additional tracking that isn't possible over HTTPS. This is both bad for users and takes away control from site owners. Because of this, there has been a movement towards building HTTPS-only sites. Despite this, [less than 11%](https://trends.builtwith.com/ssl/SSL-by-Default) of the top million websites currently use HTTPS by default. 6 | 7 | In this chapter, we'll explore how HTTPS works, investigate the benefits of HTTPS-only sites, and look at how we can enable HTTPS for our sites today. 8 | 9 | ## How HTTPS Works 10 | 11 | At the most basic level, the HTTP request and response cycle is when a web connected computer requests a specific resource through a URL and a server responds with that resource, such as an HTML page. 12 | 13 | ![HTTP request/response cycle](img/http.png) 14 | (Icons by [unlimicon](https://thenounproject.com/unlimicon/)) 15 | 16 | When this information is requested, not only are the files sent over the wire, but so user information, such as the the user's IP address, location, browser information, system information, and more. More importantly, all of this information is sent as unencrypted plain text over the public internet, meaning that any network sitting between the user's browser and the server has access to that information. This means that when I request a website, like the graphic above, what I'm really saying is "Hello, I'm user 192.00.000.001 in the United States using Mozilla Firefox 48.0.1 on an Intel Macintosh 10.11.6 and would like the `/page.html` resource from http://ethicalweb.org." The server, in turn responds by returning the unencrypted resource to the user's browser. 17 | 18 | HTTPS works similarly to HTTP, but adds a layer of SSL(Secure Sockets Layer)/ TLS(Transport Layer Security) encryption. This means that requests and responses are made over a secure encrypted connection. These requests only include the user's IP address and the domain of the requested resource. In this instance my request would appear as "Hello, I'm user 192.00.000.001 and would like a resource from https://ethicalweb.org." The server would then respond with an encrypted version of the resource. 19 | 20 | --- 21 | 22 | **ASIDE** 23 | TLS is the updated and more secure version of SSL. Throughout the remainder of the chapter, I will refer to SSL/TLS simply as TLS, though some external references may use SSL as the catch-all term. Confusing? Yup! This represents one of the many reasons that HTTPS can seem intimidating. 24 | 25 | --- 26 | 27 | The United States Government's [https-only standard](https://https.cio.gov/faq/#what-does-https-do?) helpfully demonstrates the difference between these two requests. The standard unencrypted HTTP request includes a number of headers about the client and request: 28 | 29 | ![Standard HTTP headers](https://https.cio.gov/assets/images/with-http-headers.png) 30 | 31 | By contrast the encrypted HTTPS request limits this information: 32 | 33 | ![HTTPS headers](https://https.cio.gov/assets/images/with-https-headers.png) 34 | 35 | ### How the TLS Connection Works 36 | 37 | Let's take a closer look at how the TLS connection works. To provide an encrypted connection, a site must obtain a TLS certificate. TLS certificates are used to verify the authenticity of the domain, relay information about the certificate itself, and contain a public key that will be exchanged with the user's browser. 38 | 39 | The steps of the process are much like purchasing a car (only a lot faster!): 40 | 41 | 1. Greet one another 42 | 2. Exchange the certificate 43 | 3. Exchange the keys 44 | 45 | First, the user's client says hello by reaching out to the server and requesting the https resource. This request contains all of the information about the user's connection that the server will need, such as the supported TLS version. In our car metaphor, in this step we've walked in to the dealership and asked to buy a car, we state the type of car we'd like to buy, and offered up our trade-in vehicle. 46 | 47 | The next step is to exchange the certificate. After the initial client request, the server will respond with a TLS certificate. This certificate has been either self-signed or issued by a trusted certificate authority and contains information such as name of the domain it is attached to, the name of the certificate owner, dates that the certificate is valid, and a public key. In our car purchase metaphor, this is the deed to the car. With this information, we're able to verify that the seller actually owns the car we're purchasing. 48 | 49 | Lastly, the browser and server exchange keys for data encryption and decryption. Along with the certificate, the server sends along a public key. In response, the browser sends the server an encrypted request for the specific URL/assets it is trying to access. The web server then decrypts this information and returns an encrypted version to the client, which then decrypts it locally. In our car purchasing metaphor, we are now handing over the keys to our trade-in, obtaining the key for our new vehicle, and driving away! 50 | 51 | All of this happens seamlessly and instantly to a user, but this process adds a the important layer of encrypted protection that HTTPS provides. 52 | 53 | --- 54 | 55 | **ASIDE** 56 | The keys used in this exchanged are use a symmetric key algorithm, agreed upon between the client and server during the initial connection. Symmetric keys work by using the same key to encrypt and decrypt. To make this process secure, this key is transmitted from the client to server using an asymmetric algorithm (a public/private key exchange), using the server's public key which is contained in the TLS certificate. It's like a double-decker encryption sandwich, ensuring that the information remains secure while traveling between the user and server. 57 | 58 | --- 59 | 60 | ## Why Use HTTPS 61 | 62 | As we looked at what HTTPS is and how it works, we can begin to see some of the value it provides to our users as well as ourselves as site owners and maintainers. Specifically HTTPS is important for: 63 | 64 | - User privacy and security 65 | - Site authenticity & integrity 66 | - Browser deprecated HTTP 67 | - Potential search ranking improvements 68 | 69 | Let's take a closer look at each of these. 70 | 71 | ### User Privacy and Security 72 | 73 | In the previous chapter we looked at the value we can provide by respecting a user's privacy with Do Not Track. However, many users are simply unaware of these types of features. One way that we can aid the privacy of all users is by using HTTPS on our sites. This provides our users with private, encrypted connections to our sites. HTTPS prevents monitoring of sites on public networks as well as preventing passive attackers from eavesdropping on a user's web traffic. 74 | 75 | ### Site Authenticity 76 | 77 | HTTPS aids in verifying the authenticity of a site and its content. When a site is served over HTTPS a user can feel confident that they are visiting the site they intended and that its content is what the site owner had intended for the user. 78 | 79 | When describing the decision to move to HTTPS, popular news website BuzzFeed detailed the [authenticity benefits of HTTPS](https://www.buzzfeed.com/jasonreich/buzzfeed-and-https): 80 | 81 | > Verification is a lesser known, but equally important benefit of HTTPS. It helps prevent what is called a Man-in-the-Middle attack, or MITM attack. An MITM attack via your browser can change the content of any non-HTTPS website you’re visiting without you knowing. This means an attacker can [modify news stories](http://newstweek.com/2011-01-07-device-distorts-news-on-wireless-neworks) to change or remove info, or they can change the contact details on a BuzzFeed contributor’s author page so you see a fake account the attacker controls. 82 | 83 | 84 | ### Browser Deprecated HTTP 85 | 86 | Currently, browsers display an indication whenever a site is being served securely using HTTPS. This appears as a green padlock next to the site's URL: 87 | 88 | ![](img/https-url-example.png) 89 | 90 | However, there is a lack of indicator for browsers that are not using HTTPS: 91 | 92 | ![](img/no-https-url-example.png) 93 | 94 | Recently the Chromium team pointed out that "people do not generally perceive the absence of a warning sign" and consequently suggested that browsers instead [mark HTTP as insecure](https://www.chromium.org/Home/chromium-security/marking-http-as-non-secure), alerting users that sites served over HTTP. 95 | 96 | #### New Browser Features 97 | 98 | The second way that browsers are deprecating HTTP is by making new browser APIs available only to sites server over HTTPS. These include offline capabilities with service workers (covered in [Building Web Apps that Work Everywhere](http://www.oreilly.com/web-platform/free/building-web-apps-that-work-everywhere.csp)), the ability to access user camera and audio with [getUserMedia](https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia), and user location information with the [geolocation](https://developer.mozilla.org/en-US/docs/Web/API/Geolocation/Using_geolocation) API. Looking at the types of information these APIs will have access to, I'm thankful that browser vendors have decided that they should only be accessed over a secure connection. As and added benefit, as we develop forward-thinking applications, HTTPS will quickly become a requirement. 99 | 100 | ### Improved Search Rankings 101 | 102 | In 2014 Google announced that the search engine would begin to prioritize sites using HTTPS in search results. According to the [blog post announcement](https://security.googleblog.com/2014/08/https-as-ranking-signal_6.html): 103 | 104 | > [O]ver the past few months we’ve been running tests taking into account whether sites use secure, encrypted connections as a signal in our search ranking algorithms. We’ve seen positive results, so we’re starting to use HTTPS as a ranking signal. For now it's only a very lightweight signal—affecting fewer than 1% of global queries, and carrying less weight than other signals such as high-quality content—while we give webmasters time to switch to HTTPS. But over time, we may decide to strengthen it, because we’d like to encourage all website owners to switch from HTTP to HTTPS to keep everyone safe on the web. 105 | 106 | If non-technical colleagues or clients are not yet convinced on the need for HTTPS everywhere, the potential for improved search rankings may serve as an additional selling point. 107 | 108 | ## Implementing HTTPS 109 | 110 | Now that we have looked at how HTTPS works and explored why we should use it, let's take a look at implementing HTTPS for our own sites. 111 | 112 | ### Let's Encrypt 113 | 114 | Perhaps one of the most exciting changes in HTTPS over the past few years is the creation of [Let's Encrypt](https://letsencrypt.org/). Let’s Encrypt is a free, automated, and open certificate authority (CA) created by the [Internet Security Research Group (ISRG)](https://letsencrypt.org/isrg/). The stated objective of Let's Encrypt is "to make it possible to set up an HTTPS server and have it automatically obtain a browser-trusted certificate, without any human intervention." 115 | 116 | Though Let's Encrypt provides an open certificate authority, the actual implementation can be challenging. Thankfully many community clients have been created to simplify the implementation process. The most useful, and the one recommended by the Let's Encrypt team, is [Certbot](https://certbot.eff.org/). Developed by the [Electronic Frontier Foundation](https://www.eff.org/), Certbot works by automatically fetching and deploying Let's Encrypt generated TLS certificates to our server. 117 | 118 | The excellent Certbot documentation allows us to select a specific server and operating system and provides instructions based on these conditions. Let's look at how we would implement Certbot on an Apache server running on Ubuntu 16.04. 119 | 120 | A version of Certbot is packaged for 16.04, meaning from our server we can run `apt-get` to install it: 121 | 122 | ``` 123 | $ sudo apt-get install python-letsencrypt-apache 124 | ``` 125 | 126 | Let's Encrypt ships with a beta Apache plugin that will automate obtaining and installing the certificate. To do so, simply run: 127 | 128 | ``` 129 | $ letsencrypt --apache 130 | ``` 131 | 132 | And that's it! With those few simple commands we will have installed a TLS certificate for our server. To find guidelines for installation for your specific server configuration, visit [certbot.eff.org](https://certbot.eff.org). 133 | 134 | #### Renewal 135 | 136 | Let's Encrypt certificates are valid for 90 days, meaning they will need to be renewed on a regular basis. To do that we could log in to our server every 90 days and run: 137 | 138 | ``` 139 | $ letsencrypt renew 140 | ``` 141 | 142 | However, this manual process seems like it has a high likelihood of failure (what if we're on vacation, ill, or simply just forget?!). Instead Certbot recommends running a cron job that will test for renewal on a daily basis. First let's test the renewal process: 143 | 144 | ``` 145 | $ letsencrypt renew --dry-run 146 | ``` 147 | 148 | Once we've verified that this works, we can create the cron job. We'll create a job that runs the renew script twice daily at 5:17am and 5:17pm (certbot requests that the jobs run at a random minute within the hour). 149 | 150 | First open the `crontab`: 151 | 152 | ``` 153 | $ crontab -e 154 | ``` 155 | 156 | Then add the following to the file: 157 | 158 | ``` 159 | 17 05, 17 17 * * * letsencrypt renew 160 | ``` 161 | 162 | With this our Let's Encrypt issued certificate will automatically renew when needed. 163 | 164 | 165 | ### Other Certificate Options 166 | 167 | Though Let's Encrypt is a fantastic and recommended option, it may not be the right one for you or your organization. If you are using Amazon Web Services, they now offer [free TLS certificates](https://aws.amazon.com/certificate-manager) that are very easy to set up and deploy. I have used this service and it is a great and simple option. Another option, [SSLMate](https://sslmate.com/), works similarly to Let's Encrypt by automating certificates, but is not free. 168 | 169 | For some it may also be preferable to go the traditional route of purchasing the certificate from a Certificate Authority (CA) and uploading it to the server. Common TLS CA's are [Verisign](https://www.verisign.com/), [Thawte](https://www.thawte.com/), and [RapidSSL](https://www.rapidssl.com/). 170 | 171 | When implementing TLS on your server, Mozilla provides an [Configuration Generator](https://mozilla.github.io/server-side-tls/ssl-config-generator/). This easily outputs the configuration needed for popular servers such as Apache, Nginx, and Lighttpd with a variety of TLS certificate types. Once configured, SSL Labs provides a [SSL Server Test](https://www.ssllabs.com/ssltest/), which allows us analyze the TLS configuration of our server. 172 | 173 | ## Other Considerations 174 | 175 | Once we have implemented HTTPS, there are a few site-wide changes to take into consideration: 176 | 177 | - Redirecting HTTP to HTTPS 178 | - Enabling HTTP Strict Transport Security 179 | - Preventing mixed content and using relative URLs 180 | - Using secure cookies 181 | 182 | ## Redirect HTTP to HTTPS 183 | 184 | If we add HTTPS to an existing site, it may be worth redirecting all HTTP requests to HTTPS. This will ensure that all existing external links are served over a secure connection. 185 | 186 | Following our previous Let's Encrypt example, we could redirect all links with Apache by adding the following to our Virtual Host: 187 | 188 | ``` 189 | ServerName www.example.com 190 | Redirect "/" "https://www.example.com/" 191 | ``` 192 | 193 | 194 | ## HTTP Strict Transport Security 195 | 196 | When forwarding HTTP to HTTPS, the user is initially opening a request with the unencrypted version of our site before being redirected. This does open users up to a man in the middle attack. To prevent this from happening on future visits, we can pair the forward with HTTP Strict Transport Security, to ensure that users only access the site over HTTPS. 197 | 198 | HTTP Strict Transport Security (HSTS) is a browser feature that allows a site to request that it only be served over HTTPS on future visits. This works by a server providing a `Strict-Transport-Security` along with a max-age. Once receiving this header, the browser will only request pages from that domain over HTTPS. 199 | 200 | Here is the HSTS header, along with an expiration of on year, and instructions to include sub domains: 201 | 202 | ``` 203 | Strict-Transport-Security: max-age=31536000; includeSubDomains 204 | ``` 205 | 206 | To set the HSTS header in Apache, we would add the following to our Virtual Host: 207 | 208 | ``` 209 | Header always set Strict-Transport-Security "max-age=63072000; includeSubdomains; preload" 210 | ``` 211 | 212 | 213 | ## Mixed Content and Relative URLs 214 | 215 | Mixed content occurs when a site is served over a secure HTTPS connection, but contains links to resources such as images, CSS, or JavaScript that is served over HTTP. When this occurs, browsers display an error message to users, warning them that the site contains insecure content. 216 | 217 | This often happens in error or may occur when a site is converted to HTTPS and has lingering absolute links. To avoid this situation, convert links beginning with `http://` to `https://` or use relative URLs when linking to local files. 218 | 219 | 220 | ### Secure cookies 221 | 222 | When sending cookies form our server application over an HTTPS connection, we should enable the `secure` flag. Using the `secure` flag will ensure that the cookie request only be sent over an encrypted connection (HTTPS). 223 | 224 | For example, when setting a cookie using the popular Node.js web framework Express, `secure` is an added cookie parameter: 225 | 226 | ``` 227 | res.cookie('user', 'adam', { secure: true }); 228 | ``` 229 | 230 | In the Django framework, it is a matter of setting the `SESSION_COOKIE_SECURE` and `CSRF_COOKIE_SECURE` settings to `True`. 231 | 232 | ## Conclusion 233 | 234 | HTTPS provides numerous benefits to both site owners and users and helps to make the web more secure. Whatever the method you choose to implement HTTPS for your sites, you are taking important steps to improve the security and privacy of your users. 235 | 236 | ## Further Reading 237 | 238 | - [The United States Government's HTTPS-Only Standard](https://https.cio.gov/) 239 | - [GOV.UK's Using HTTPS](https://www.gov.uk/service-manual/technology/using-https) 240 | - [How Does HTTPS Actually Work?](http://robertheaton.com/2014/03/27/how-does-https-actually-work/) by Rob Heaton 241 | - [Is TLS Fast Yet?](https://istlsfastyet.com/) 242 | - [Securing the Web](https://www.w3.org/2001/tag/doc/web-https), W3C report 243 | - [Encrypting data in transit](https://developers.google.com/web/fundamentals/security/encrypt-in-transit/), Google Developers resource 244 | - [We're Deprecating HTTP And It's Going To Be Okay](https://konklone.com/post/were-deprecating-http-and-its-going-to-be-okay) by Eric Mill 245 | -------------------------------------------------------------------------------- /web-apps-privacy-security/05-data-export.md: -------------------------------------------------------------------------------- 1 | # Preserving User Data 2 | 3 | Now that we've put a lot of effort into securing and ensuring our user's data is private, we will also want to consider our user's ownership and access to their data. As users pour their personal and work lives into the applications we build, the data this creates can become a reflection of their lives. Our applications may store photos, documents, journals, nots, private reflections, user locations, food preferences, family relationships, meeting information, and connections between all of these things. While this information can be incredibly powerful in continuing to build and improve our applications, our users have a personal investment in the data they have created and shared with us. 4 | 5 | In 2009 the site GeoCities was shuttered. GeoCities was as an early free web hosting platform and was considered an important piece of early web history. Though Yahoo, who had acquired GeoCities in 1999, provided guidance for how to preserver their sites elsewhere, many of the sites were no longer actively maintained, ensuring that they would be lost forever. In light of this, several projects such as the [Internet Archive: GeoCities](https://archive.org/web/geocities.php), [Archive Team](http://www.archiveteam.org/index.php?title=GeoCities_Project), [ReoCities](http://reocities.com/), and [OoCities](http://www.oocities.org/) undertook Herculean efforts to archive or mirror the original GeoCities content. 6 | 7 | In 2011 the social check-in service Gowalla [announced](http://blog.gowalla.com/post/13782997303/gowalla-going-to-facebook) that the service would be shutting down. Gowalla was an early competitor with Facebook and had a passionate and enthusiastic user base. In a blog post, Gowalla founder, Josh Williams stated that "[w]e plan to provide an easy way to export your Passport data, your Stamp and Pin data (along with your legacy Item data), and your photos as well." Unfortunately, despite the best of intentions of the Gowalla team, the ability to export data was not added before the service was fully shut down, causing all Gowalla user data to be lost. 8 | 9 | These are just two of many interesting examples of site closures or significant feature changes that can cause user data to be lost. As developers, we are entrusted with user data. By providing users a means to export their data, we are able to give them more control over how and where it is used. 10 | 11 | 12 | ## Data Ownership 13 | 14 | Why owns the data generated within our application? Though it may be easiest to say "the user," this can become an increasingly complicated question when we consider things such as collaborative documents, online discussions, and shared calendars, who have may have an initial creator but ultimately have multiple maintainers. What about the sites themselves? Sometimes a Terms of Service may insist on ownership or exclusive rights to a user's created content. As part of Facebook's Terms of Service, the company enforces exclusive rights to any content created within or posted to the site: 15 | 16 | > For content that is covered by intellectual property rights, like photos and videos (IP content), you specifically give us the following permission, subject to your privacy and application settings: you grant us a non-exclusive, transferable, sub-licensable, royalty-free, worldwide license to use any IP content that you post on or in connection with Facebook (IP License). 17 | 18 | In doing this, we take the power away from the user and assert ownership over the content they have created. Though there is a business case for this, it comes at a potential cost to our users. The creator of the Web, Tim Berners-Lee, has spoken out in favor of user-owned data, stating that “the data that [firms] have about you isn’t valuable to them as it is to you.” 19 | 20 | If we take this perspective, we should aim to open user data to our users and provide a means to export it from our site in an open format. 21 | 22 | In his article, [Rights to Your Data and Your Own Uber ‘God’ View](https://medium.com/@milesgrimshaw/rights-to-our-data-and-your-own-uber-god-view-dc8416a3751#.91g92jhdd), Miles Grimshaw suggests adapting a Creative Commons style license for personal data, which would be adopted by services collecting this data. 23 | 24 | You are free to: 25 | 26 | **Download**  —  free access to your raw data in standard file formats 27 | 28 | **Share**  —  copy and redistribute the data in any medium or format 29 | 30 | **Adapt ** —  remix, transform, and build upon the data 31 | 32 | Under the following terms: 33 | 34 | **Attribution** —  You must provide a sign-up link to the application 35 | 36 | The, since acquired, start-up Kifi had a forward thinking approach to user data, stating in a [blog post](https://medium.com/kifi-engineering/exporting-user-data-71a060bdb774#.1w2hbekka) that: 37 | 38 | > Any service that manages your data has an implicit contract with users: you give us your data and we’ll organize it, but it’s still your data; we are just stewards for it. At Kifi, one way we try to fulfill our end of this contract is by making sure users can export their data for offline use (or so they can import it into another service). 39 | 40 | These ideas are not limited to start-ups or small services. In 2012 Twitter introduced the ability to [download an archive of your Tweets](https://blog.twitter.com/2012/your-twitter-archive), giving users both permenant access to their Twitter content as well as the potential ability to import it into another service. Google also provides the ability to [download an archive](https://support.google.com/accounts/answer/3024190?hl=en) of the data created with any of its services including the ability to easily store the archive in common file sharing applications such as DropBox, Google Drive, and Microsoft OneDrive. 41 | 42 | By giving our users ownership and access to their data, we can be better stewards of that information. This aids us in creating long-lasting user content and opens up the potential for users to adapt and use their data in novel and interesting ways. Most importantly, by providing access to user data we are able to give ownership of the data our users create directly to the user. 43 | 44 | ## Deleting User Data 45 | 46 | An inevitable reality is that some users will want to stop using the services we build. In many cases, these users may simply allow their accounts to decay, but other users will explicitly seek to delete their accounts and associated information. When a user does delete their account, we should also delete it from our databases, rather than simply hide the user's content within our site or application. Doing so will be more in line with user expectations and ensures that user data is removed in the case of a data breach. 47 | 48 | ## Archiving and Graceful Shutdown 49 | 50 | At the beginning of the chapter, we looked at a few web application shut downs and how the loss of their data impacted users. According to the United States Small Business Administration, nearly [40% of small business fail after three years](http://www.bls.gov/bdm/us_age_naics_00_table7.txt). In the world of tech start-ups, that number is significantly higher, as reportedly [9 out of 10 start-ups fail](http://www.forbes.com/sites/neilpatel/2015/01/16/90-of-startups-will-fail-heres-what-you-need-to-know-about-the-10/#7751c49955e1). This also fails to take into account, web applications that are acquired or owned and closed by large companies. 51 | 52 | The group Archive Team works to catalog and preserve digital history, but also keeps a [Deathwatch](http://archiveteam.org/index.php?title=Deathwatch) of sites risking shutdown and advice for individuals on [backing up our data](http://archiveteam.org/index.php?title=Introduction). Though this is a wonderful project, we cannot assume that users will back-up their data. When our services are closing down we can do so gracefully. The music streaming service Rdio closed its doors in 2015, but in doing so offered a [farewell](https://web.archive.org/web/20160109002117/http://www.rdio.com/farewell/), which included the ability for users to download CSV files of things such as their playlists and saved music to be imported into another service. As the site [Hi.co](http://hi.co/) shuttered, it's founder Craig Mod committed to keeping the archive on the web for the next ten years, making individual contributions exportable, and producing five nick-plated books of the site to be preserved. In [an article about the shutdown](https://medium.com/@craigmod/archiving-our-online-communities-e5868eab4d9a#.c2tg22g46), Mod wrote: 53 | 54 | > At the same time we understand the moral duty we took on in creating Hi.co — in opening it up to submissions and user generated content. There was an implicit pact: You give us your stories about place, and we’ll give you a place to put your stories. This was not an ephemeral pact. 55 | 56 | Though we may not choose to nickel-plate our own service's contents, providing exports will ensure that users are able to preserve their data if they choose to do so. 57 | 58 | ## Further Reading 59 | 60 | - [With Great Data Comes Great Responsibility](https://medium.com/@jazzpazz/with-great-data-comes-great-responsibility-72d3e1c94e27#.ls4gmwey7) by Pascal Raabe 61 | - [Archiving a Website for Ten Thousand Years](http://www.theatlantic.com/technology/archive/2016/05/archiving-a-website-for-ten-thousand-years/482385/) by Glenn Fleishman 62 | - [Preserving Digital History](http://chnm.gmu.edu/digitalhistory/preserving/index.php) by Daniel J. Cohen and Roy Rosenzweig 63 | -------------------------------------------------------------------------------- /web-apps-privacy-security/06-conclusion.md: -------------------------------------------------------------------------------- 1 | # In conclusion 2 | 3 | Thank you for taking the time to read Building Web Apps That Respect a User's Privacy and Security. In this title we've explored the value of respecting user privacy, using HTTPS, following security best practices, and permitting users to export their data. My hope is that you now feel empowered and excited to build applications in this way. 4 | 5 | If throughout your reading you have come across things that are missing or could be improved, I would encourage you to contribute back to the book. This title is available as open source and contributions can be made by: 6 | 7 | - Contributing directly to the GitHub repository with a pull request . 8 | - Creating an issue in the book’s GitHub repository . 9 | - Reaching out to me through [email](mailto:adamdscott@protonmail.com) or [Twitter](https://twitter.com/adamdscott). 10 | 11 | Twenty percent of the proceeds from each Ethical Web Development title will be donated to an organization whose work has a positive impact on the issues described. For this title, I will be donating to the Electronic Frontier Foundation (EFF). The EFF "champions user privacy, free expression, and innovation through impact litigation, policy analysis, grassroots activism, and technology development." The work and research of the EFF was instrumental to the research involved in this title. 12 | 13 | If you are interested in supporting the EFF's work please consider getting involved at [eff.org](https://www.eff.org). 14 | 15 | This title is the first in a series of digital reports I am authoring on the subject of ethical web development. Future titles in the series will cover building web applications that work everywhere, building web applications that respect a user's privacy and security, and working with development peers. You can learn more about the series at [ethicalweb.org](https://ethicalweb.org). 16 | -------------------------------------------------------------------------------- /web-apps-privacy-security/README.md: -------------------------------------------------------------------------------- 1 | # Building Web Apps that Respect A User's Privacy and Security 2 | 3 | - [Preface](https://github.com/ascott1/ethical-web-dev/blob/master/preface.md) 4 | - [Introduction](01-title-intro.md) 5 | - [Respecting user privacy](02-privacy.md) 6 | - [Encrypting user connections with https](03-https.md) 7 | - [Securing user data](04-security.md) 8 | - [Allowing user data export](05-data-export.md) 9 | - [Conclusion](06-conclusion.md) 10 | - [Contributors](contributors.md) 11 | -------------------------------------------------------------------------------- /web-apps-privacy-security/contributors.md: -------------------------------------------------------------------------------- 1 | # Contributors 2 | 3 | ## About the Author 4 | 5 | Adam D. Scott is a developer and educator based in Connecticut. He currently works as a the development lead at the Consumer Financial Protection Bureau, where he leads a team of open source developers. Additionally, he has worked in education for over a decade, teaching and writing curriculum on a range of technical topics. His first book, WordPress for Education, was published in 2012. His video course, Introduction to Modern Front-End Development, was published by O'Reilly in 2015. This is the third title in a series on the ethics of web development published by O'Reilly. 6 | 7 | ## Technical Reviewer 8 | 9 | Judith Myerson is a systems architect and engineer. Her areas of interest include enterprise-wide systems, database technologies, network & system administration, security, operating systems, programming, desktop environments, software engineering, web development, and project management. 10 | 11 | ## Other Contributions 12 | 13 | The following people have graciously contributed feedback and improvements. 14 | 15 | - Meg Foley, Editor. 16 | - [Eric Mill](https://konklone.com/) for a thoughtful review and feedback of the HTTPS chapter. 17 | - Jonathan Crane for typo fixes. 18 | 19 | Contributions and suggestions have also been made to the ethicalweb.org site and the core principles of ethical web development. Those contributions are stored at 20 | -------------------------------------------------------------------------------- /web-apps-privacy-security/img/OAuth-access-request.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-privacy-security/img/OAuth-access-request.png -------------------------------------------------------------------------------- /web-apps-privacy-security/img/OAuth-signin.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-privacy-security/img/OAuth-signin.png -------------------------------------------------------------------------------- /web-apps-privacy-security/img/U2F-USB-Token.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-privacy-security/img/U2F-USB-Token.jpg -------------------------------------------------------------------------------- /web-apps-privacy-security/img/atlantic.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-privacy-security/img/atlantic.png -------------------------------------------------------------------------------- /web-apps-privacy-security/img/cookie-tracking.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-privacy-security/img/cookie-tracking.png -------------------------------------------------------------------------------- /web-apps-privacy-security/img/http.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-privacy-security/img/http.png -------------------------------------------------------------------------------- /web-apps-privacy-security/img/https-url-example.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-privacy-security/img/https-url-example.png -------------------------------------------------------------------------------- /web-apps-privacy-security/img/ico-cookie.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-privacy-security/img/ico-cookie.png -------------------------------------------------------------------------------- /web-apps-privacy-security/img/medium-dnt.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-privacy-security/img/medium-dnt.png -------------------------------------------------------------------------------- /web-apps-privacy-security/img/no-https-url-example.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-privacy-security/img/no-https-url-example.png -------------------------------------------------------------------------------- /web-apps-privacy-security/img/securityheaders.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-privacy-security/img/securityheaders.png -------------------------------------------------------------------------------- /web-apps-that-work-everywhere/01-title-intro.md: -------------------------------------------------------------------------------- 1 | # Introduction 2 | 3 | In 2007, at the 3GSM conference in Barcelona, Tim Berners-Lee, the creator of the Web, gave a [keynote address on the mobile web](https://www.w3.org/2007/Talks/0222-3gsm-tbl/text). In this talk, which happened 6 months prior to the release of the original iPhone, Berners-Lee states: 4 | 5 | > The Web is designed, in turn, to be **universal**: to include anything and anyone. This universality includes an independence of hardware device and operating system… and clearly this includes the mobile platform. It also has to allow links between data from any form of life, academic, commercial, private or government. It can't censor: it must allow scribbled ideas and learned journals, and leave it to others to distinguish these. It has to be independent of language and of culture. It has to provide as good an access as it can for people with disabilities. 6 | 7 | This idea of universality has become even more critical in our increasingly diverse world of web access. By design, the Web works across platforms and devices, easily connecting rich documents with one another and providing access to users around the world. Despite this universal default, as web developers, it is our responsibility to build a web that is accessible to all. But before we look at how of building an everywhere Web, let’s consider why. 8 | 9 | In the United States, where I live, nearly 1 in 5 adults own a smartphone, but either do not have access to high-speed internet at home or have limited access other than their cell phone[^1]. Additionally, mobile devices are heavily depended upon for access to a wide range of social and cultural services. According to the [Pew Internet Study](http://www.pewinternet.org/2015/04/01/us-smartphone-use-in-2015/), smartphone users report that in the past year: 10 | 11 | - 62% have used their phone to look up information about a health condition. 12 | - 57% have used their phone to do online banking. 13 | - 44% have used their phone to look up real estate listings or other information about a place to live. 14 | - 43% to look up information about a job. 15 | - 40% to look up government services or information. 16 | - 30% to take a class or get educational content. 17 | - 18% to submit a job application. 18 | 19 | [^1]: http://www.pewinternet.org/2015/04/01/us-smartphone-use-in-2015/ 20 | 21 | Meanwhile, smartphone ownership in emerging and developing nations has dramatically increased over recent years, rising to a median of 37% in 2015[^2] with worldwide 3G coverage reaching 69%[^3].This rise in access can come at a cost, as fixed-broadband is 3 times more expensive and mobile data is twice as expensive in developing countries than in developed countries[^3].Worldwide Internet speeds can vary wildly as well ranging from an average of nearly 40 Mbit/s in Korea to 0.09 Mbit/s in Zambia[^3]. 22 | 23 | **It’s predicted that by 2020 there will be 7.8 billion mobile-connected devices, exceeding the world’s population**[^4]. 24 | 25 | [^2]: http://www.pewglobal.org/2016/02/22/smartphone-ownership-and-internet-usage-continues-to-climb-in-emerging-economies 26 | [^3]: http://www.itu.int/en/ITU-D/Statistics/Documents/facts/ICTFactsFigures2015.pdf 27 | [^4]:https://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/mobile-white-paper-c11-520862.html 28 | 29 | In his talk [Small, Faster Websites](https://bocoup.com/weblog/smaller-faster-websites), Mat "Wilto" Marquis describes the challenge of building for an everywhere web in this way: 30 | 31 | > Building massive, resource-heavy sites means excluding millions of users that have only ever known the web by way of feature phones or slightly better—users paying for every kilobyte they consume; users that already have to keep tabs on which sites they need to avoid day-to-day because of the cost of visiting them. I don’t mean some nebulous hand-wavy “bandwidth cost,” either—I mean actual economic cost. 32 | 33 | Despite the challenges of building for a diverse, multi-device Web served over a variety of connection speeds, we can make user-centered choices that enable greater access to our sites and data. We can do this through: 34 | 35 | - Exposing permanent, human readable, deep links 36 | - Building sites that are responsive to a range of viewport sizes 37 | - Valuing the performance of our site’s and the bandwidth they consume 38 | - Leveraging off-line first capabilities that support varying network conditions 39 | 40 | Through this report we’ll explore these topics and take a practical approach to putting them into practice. -------------------------------------------------------------------------------- /web-apps-that-work-everywhere/02-url.md: -------------------------------------------------------------------------------- 1 | # URL Design 2 | 3 | The humble hyperlink one of the most powerful aspects of the web. This ability to connect to any resource on the web with through a URL is what makes the everywhere web possible. As developers we should aim to expose permanent, human readable, deep links to our users. 4 | 5 | In 1996 the creator of the web, Tim Berners-Lee, drafted [Universal Resource Identifiers -- Axioms of Web Architecture](https://www.w3.org/DesignIssues/Axioms.html). This document consists of several axioms of URL design, many technical in nature, but the first (and arguably most important) is “universality.” By Berners-Lee’s definition, “any resource anywhere can be given a URI” and “any resource of significance *should* be given a URI” (emphasis mine). By conforming to these expectations of the web we make it easier for our users to share and interact with the web. 6 | 7 | ## Permanence 8 | 9 | > What makes a cool URI? 10 | > A cool URI is one which does not change. 11 | > What sorts of URI change? 12 | > *URIs don't change: people change them.* 13 | 14 | — [THE W3C](https://www.w3.org/Provider/Style/URI.html) 15 | 16 | One of the beautiful things about developing for the web is the ability to evolve our applications over time, immediately deploying updates to every user. With this ability, however, we often introduce states of fluctuation as we change server configurations, undergo content redesigns, and adapt to new technologies and frameworks. In the paper [Perma: Scoping and Addressing the Problem of Link and Reference Rot in Legal Citations](http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2329161) the authors point out that “more than 70% of the URLs within the Harvard Law Review and other journals, and 50% of the URLs found within United States Supreme Court opinions, do not link to the originally cited information.” This is often referred to as “link rot,” where once valid URLs no longer return the originally linked resource. The prevalence of link rot is something that online archiving tools such as the Internet Archive’s [Wayback Machine](https://archive.org/web/) and [permalink.cc](https://perma.cc/) attempt to combat. For his part, Tim Berners-Lee has wrote about the idea of [persistent domains](https://www.w3.org/DesignIssues/PersistentDomains) 16 years ago, but this idea has, thus far, failed to become a reality. 17 | 18 | As developers, we should avoid arbitrarily changing URLs for our applications as much as possible. If significant changes to content require a URL change, we should always forward the previous URL to the new page. When creating permanent URLs the first step is to ensure that technology does not dictate the URL. Often sites display language filetypes at the end of a URL such as `.php` or `.asp`. This doesn’t accommodate for future iterations of an application that may be built upon a different technology stack. By remaining technology independent in URL design we take the first step towards more permanent URLs. 19 | 20 | The importance or persistent URLs is that they help to preserve the web. When URLs persist, outside links remain active, user bookmarks remain relevant, and information remains consistent. By focusing on good URL design we can help to ensure the permanence of URLs across the web. 21 | 22 | ## Shareable URLs 23 | 24 | Commenting on an early draft of the [Principles for Ethical Web Development](https://ethicalweb.org/), [Dean Marano](https://github.com/deanmarano) raised the important issue of creating shareable URLs. 25 | 26 | > One thing that for me is very important when building apps is the ability to share a URL - either with myself or with others - easily. By leveraging this built in feature of the web, it makes it much easier to share, bookmark, and be a good web citizen. 27 | 28 | This ability to link and share is a key advantage that web development has over other forms of application development. A few ways that we can aid this practice in our applications, is to give our users the ability to link out to content that is within our applications, without requiring a login when possible, ensuring that URLs are updated when doing client-side routing, and by avoiding non-standard URL formats such as hash-bang URLs (`http://example.com/#!/foo/`). 29 | 30 | ## URL Design 31 | 32 | Simply Providing URLs is the first step, but as Jakob Nielsen has described them [URLs are a form of user interface](https://css-tricks.com/guidelines-for-uri-design/). Even in the era of search engines, a [study from Microsoft Research](http://research.microsoft.com/apps/pubs/default.aspx?id=70395) to determine if URLs were observed by average users revealed that users spent 24% of their gaze time looking at the URLs in search results. With this in mind, how can we design URLs that are effective and usable? 33 | 34 | ### Keep URLs simple 35 | 36 | Effective URLs are simple, short, and human-friendly. This makes them easier to type and remember for users. 37 | 38 | WordPress is the most popular content manager for the web and powers over 25%[^1] percent of websites. Unfortunately, until relatively recently[^2], the default WordPress [permalink structure](https://codex.wordpress.org/Introduction_to_Blogging#Pretty_Permalinks) produced URLs such as: 39 | 40 | ``` 41 | /index.php?p=423 42 | ``` 43 | 44 | 45 | To a user this URL format is seemingly random and arbitrary. Fortunately, WordPress allowed users to create “pretty” permalink structure, and as of 2015, now does this by default. The pretty permalink structure can be descriptive and clean, such as: 46 | 47 | ``` 48 | /posts/effective-altruism/ 49 | ``` 50 | 51 | WordPress core contributor Eric Lewis [reportedly commented on the change](https://wptavern.com/wordpress-4-2-will-automatically-enable-pretty-permalinks-for-new-sites-on-installation) saying that “Delivering pretty permalinks by default seems in line with a bunch of core philosophies – great out-of-the-box, design for the majority, simplicity, clean, lean and mean.” I agree with Eric, this is a great change, beneficial to users across the web, and a great example of how much more legible a well designed link can be. 52 | 53 | By creating link structures that are simple and human readable, we are able to provide our users with a clear description of a URLs content. 54 | 55 | [^1]: https://w3techs.com/technologies/history_overview/content_management/all/y 56 | [^2]:https://core.trac.wordpress.org/changeset/31089 57 | 58 | ### Make URLs Meaningful and Consistent 59 | 60 | URLs should be both meaningful and consistent throughout a site. Meaningful URLs clearly represent a resource and accurately describe its contents with the title and, when useful, keywords. A website that holds a blog may put blog posts within a `/blog/` URL structure such as `/blog/url-design` and `/blog/ethical-web`. These URLs make the intent of the resource clear and are understandable to the user. URLs should also be consistent, using recognizable patterns. If when logged into an application my profile’s URL is `https://example.com/user/adamscott`, I would expect to find another user’s profile with the same URL structure of `/user/username`. 61 | 62 | ### Make URLs “Hackable” 63 | 64 | URLs should be “hackable” up the tree of the URL in a way that allows users to visualize the site structure. For example, if a URL is `https://example.com/artist/albums/album-name/` changing the URL to `https://example.com/artist/albums/` would return a page displaying the artist’s albums and `https://example.com/artist/` an artist page. Doing this makes our URLs more meaningful and predictable for our users, while allowing them to navigate down the tree and share URLs through only the logical URL structure. 65 | 66 | As [Peter Bryant](http://blog.2partsmagic.com/restful-uri-design/) describes this: 67 | 68 | > If your URLs are meaningful they may also be predictable. If your users understand them and can predict what a url for a given resource is then may be able to go ‘straight there’ without having to find a hyperlink on a page. 69 | 70 | By providing users with a hackable URL tree, we enable them to have an immediate understanding of how our site structure works. 71 | 72 | ### API URL Design 73 | 74 | Often, when designing URLs we are not limited to designing them for end-users. APIs provide a URL interface for both internal and external developers when interacting with our services. To make our API URLs more user friendly we can aim to focus on URL permanence and comprehension. 75 | 76 | Much like HTML URLs, when design API URLs we should focus on permanence. As technology and services change, it is likely that our API will evolve. When exposing a public API, it is common practice to host our API on a subdomain named, API. This allows us to run our API in its own environment while tying it to our top level domain. 77 | 78 | ``` 79 | https://api.example.com 80 | ``` 81 | 82 | Perhaps one of the most important things we can do for permanence is to always include an API version in the URL. This allows us to adopt changes to our application’s technology and features while continuing to return results for past versions of the API. 83 | 84 | ``` 85 | https://api.example.com/v1/ 86 | ``` 87 | 88 | Use nouns to identify resources. This describes what the resource returns rather than what they do. This means that we should favor identifiers such as `users` over verb-driven identifiers like `getUsers` or `list-users`. 89 | 90 | ``` 91 | https://api.example.com/v1/users/ 92 | ``` 93 | 94 | Similar to page URLs, APIs should work up and down the URL tree, making them “hackable.” If `/users/` returns a list of users `/users/username` should return the results for a specific username. 95 | 96 | ``` 97 | https://api.example.com/v1/users/ 98 | https://api.example.com/v1/users/psinger/ 99 | ``` 100 | 101 | Lastly, our API should filter advanced results by query parameters. This allows us to keep the base of our URLs reasonably simple and clean while allowing for advanced filters and sorting requirements. 102 | 103 | ``` 104 | /users/psinger/friends?sort=date 105 | ``` 106 | 107 | As API developers, the URL is our interface. By considering their permanence the design URLs we are building more useful and sustainable APIs. 108 | 109 | — 110 | 111 | Through purposeful design of our URLs, we can create URLs for our applications that are both easy to navigate and share by our users. Focusing on URL permanence, simplicity, and predictability aids in building an everywhere web that simplifies access for everyone. 112 | 113 | ## Further Reading 114 | 115 | - [Tim Berners-Lee’s Axioms](https://www.w3.org/DesignIssues/Axioms.html) 116 | - [Persistent Domains - Strawman ideas on Web Architecture](https://www.w3.org/DesignIssues/PersistentDomains.html) 117 | - [Guidelines for URI Design](https://css-tricks.com/guidelines-for-uri-design/) 118 | - [Lessons from a 40 Year Old](http://a.wholelottanothing.org/2012/03/my-webstock-talk.html) 119 | - [URL as UI](https://www.nngroup.com/articles/url-as-ui/) 120 | - [REST-ful URI Design](http://blog.2partsmagic.com/restful-uri-design/) 121 | - [Best Practices for Designing a Pragmatic RESTful API](http://www.vinaysahni.com/best-practices-for-a-pragmatic-restful-api) 122 | -------------------------------------------------------------------------------- /web-apps-that-work-everywhere/03-responsive-design.md: -------------------------------------------------------------------------------- 1 | # Responsive Design 2 | 3 | For more than a decade of the web’s existence, we could safely assume that each user of our site would be accessing it through a computer screen. Despite this, early websites were, by default, adaptive to a variety of screen sizes. The web’s first site, Tim Berners-Lee’s [World Wide Web](http://info.cern.ch/hypertext/WWW/TheProject.html), works beautifully at a range of screen sizes.[^1] 4 | 5 | ![Screenshot of the first website with a narrow viewport](img/first-website.png) 6 | 7 | Despite this, we spent time researching and considering the typical browser width and assumed that our users would be perched in front of a reasonably large screen, with a dedicated keyboard. With the evolution of mobile devices, those assumptions have changed. Users may access our sites quickly, on the go, from a wide range of screen sizes. With the diversity of devices and screens, we can no longer safely make assumptions about the screen size of our users. 8 | 9 | The initial reaction to the rise of smartphones was to create dedicated mobile versions of our sites. This often sat at a `m.`[footnote: such as http://m.example.com] subdomain and provided a mobile optimized experience. At first, this seemed like a great solution as it allowed users to access our services in a format that was streamlined for their device. For developers, this also meant maintaining multiple codebases. For users, this often meant dealing with a limited subset of functionality when using a mobile device. 10 | 11 | Today, the number of devices that connect to the web is rapidly expanding. Users may access our applications from a desktop computer, a mobile phone, a tablet, a reading device, a watch, a video game system, or in their car. The site [Global Stat Counter](http://gs.statcounter.com/#resolution-ww-monthly-201505-201605) reports that 119 different screen resolutions have accessed the web over the past year. 12 | 13 | In 2010, Ethan Marcotte coined the term [Responsive Design](http://alistapart.com/article/responsive-web-design), to describe the practice of building web sites that adapt to a range of screen sizes. By building responsively, we can develop a single codebase that acclimates to the screen size of the device being used by the user. This allows us to make fewer assumptions while delivering a site that works in any context. 14 | 15 | Responsive design consists of three core elements: 16 | 17 | - **Fluid grids** allow the layout of the screen to condense and expand to fill the screen size, rather than providing a strict width. 18 | - **Flexible media** means that our images and videos are also not limited by a pre-determined width, but an adapt with the content of the page. 19 | - **Media queries** are a CSS technique that allow developers to apply different CSS rules in varying contexts. 20 | 21 | By combining these three browser capabilities, we are able to develop sites for a wide range of browser sizes. When we build responsively, we are ensuring that our sites are delivered to our users in a way that works well in the context that they are accessing our site. 22 | 23 | [^1]: Though Berners-Lee’s first website adapts to any browser width, it still scales on most mobile browsers due to browser behavior. As we will discuss later in the chapter, adding a viewport meta tag to our HTML prevents this from happening. 24 | 25 | ## Process 26 | 27 | > Responsive design is not about “designing for mobile.” But it’s not about “designing for the desktop,” either. Rather, it’s about adopting a more flexible, device-agnostic approach to designing for the web. 28 | 29 | — [Ethan Marcotte](http://unstoppablerobotninja.com/entry/toffee-nosed/) 30 | 31 | The process of responsive design can be broken down into four steps. 32 | 33 | 1. Instruct the browser viewport to adapt to the screen size. 34 | 2. Set flexible media elements that can adapt to the width of the container. 35 | 3. Develop a [device agnostic](http://trentwalton.com/2014/03/10/device-agnostic/) baseline experience. 36 | 4. Use CSS3 media queries to enhance the experience at a variety of screen sizes (often termed “break points”). 37 | 38 | Let’s tease this process apart by creating a very simple responsive page. 39 | 40 | By default, mobile browsers will render the page at a desktop screen width. This means that users will need to pinch and zoom to be able to read and access our content. To tell the browser to scale, we can add a meta viewport tag to the `` of the HTML document. 41 | 42 | ``` 43 | 44 | ``` 45 | 46 | The most basic approach to responsive media to scale our images and other media elements to the width of their parent container. In our CSS file we apply a `max-width: 100%` to media objects to ensure that they never overflow beyond the container width. In chapter 4, we will explore how to serve various image sizes depending on browser context. 47 | 48 | ``` 49 | img, 50 | obj, 51 | video { 52 | max-width: 100%; 53 | height: auto; 54 | } 55 | ``` 56 | 57 | 58 | With the baseline of a scaled browser viewport and flexible media, we can begin developing the core experience. The core experience can encompass things such as typography, color, and base styles that should appear in all browsers. By doing so, we ensure that every user is served a site that will work well in their browser regardless of capability. Originally, this approach was termed mobile first, but I’ve come to favor Trent Walton’s description of [device agnosticism](http://trentwalton.com/2014/03/10/device-agnostic/). By taking this approach, we are developing in a future friendly way that is prepared for devices of all sizes[^2]. 59 | 60 | With our baseline styles in place, we can begin adding styles based on browser width. To do this, we use CSS media queries, which allow us to apply specific styles to given browser widths. These can and should be based on the ideal conditions of our application content. For the purpose of responsive design, we’ll focus on `max-width` and `min-width` media queries. 61 | 62 | A `max-width` media query allows us to define styles that will only appear up until a certain breakpoint. 63 | 64 | ``` 65 | @media (max-width: 600px) { 66 | /* Smaller device/browser styles go here */ 67 | } 68 | ``` 69 | 70 | In contrast, `min-width` media queries allow us to set styles that are only applied at larger browser sizes. 71 | 72 | ``` 73 | @media (min-width: 601px) { 74 | /* Larger device/browser styles go here */ 75 | } 76 | ``` 77 | 78 | In the end, we may wind up with a style sheet that is structured with base styles followed by media queries defining styles for various browser sizes, often termed breakpoints. 79 | 80 | ``` 81 | /* Base Styles */ 82 | 83 | @media (max-width: 600px) { 84 | /* Smaller device/browser styles */ 85 | } 86 | 87 | @media (min-width: 601px) { 88 | /* Large device/browser styles */ 89 | } 90 | 91 | @media (min-width: 850px) { 92 | /* Larger device/browser styles */ 93 | } 94 | 95 | @media (min-width: 1100px) { 96 | /* Largest device/browser styles */ 97 | } 98 | 99 | ``` 100 | 101 | By using breakpoints, we can define styles based on the context of the user’s browser, adapting the content of our site to better meet their needs. 102 | 103 | [^2]: Brad Frost demonstrating a project from 2013 on an Apple Watch https://youtu.be/BzckCgE5glI 104 | 105 | ### Note: CSS Frameworks 106 | 107 | If you are not a fan of writing CSS (and who could blame you?), you may opt to use a CSS framework such as [Bootstrap](https://getbootstrap.com/) or [Foundation](http://foundation.zurb.com/). These and many other UI frameworks are responsive by default and can be useful for rapid prototyping and quick interface development. 108 | 109 | ## Considerations 110 | 111 | When developing a responsive design there are a number of conditions a developer should take into account. Primarily, we should avoid making assumptions about user context and build for non-ideal conditions. 112 | 113 | Some of the key considerations for developing responsive designs: 114 | 115 | - Provide users with large click areas for links and buttons. 116 | - Ensure that site navigation is accessible and easy to understand. 117 | - Make forms as simple as possible and autofill form content when possible. 118 | - Focus on the content of the application and set breakpoints accordingly, rather than by common device sizes. 119 | 120 | 121 | ## Further Reading 122 | 123 | - [Responsive Design](http://alistapart.com/article/responsive-web-design) 124 | - [This is Responsive](https://bradfrost.github.io/this-is-responsive/index.html) 125 | - [Responsive Web Design, 2nd Edition](http://shop.oreilly.com/product/9781937557188.do) -------------------------------------------------------------------------------- /web-apps-that-work-everywhere/05-offline.md: -------------------------------------------------------------------------------- 1 | # Offline 2 | 3 | > We live in a disconnected & battery powered world, but our technology and best practices are a leftover from the always connected & steadily powered past. 4 | 5 | — [offlinefirst.org](http://offlinefirst.org/) 6 | 7 | As discussed in the previous chapter, good web performance benefits all of our users, especially those on slow connections. But often users are accessing our sites in variable network conditions. A person may pick up their phone and begin browsing through our site at home over wifi, but open the browser again offline on the subway, only to be presented with dreaded offline error messages. 8 | 9 | Even more infuriating are the times where we appear to be connected to the network, but assets are failing to load. This experience is something that developer [Jake Archibald](https://jakearchibald.com/) has termed Lie-Fi. Everything seems like it should be working, but are slow to load as they feebly connect to our struggling signal. 10 | 11 | There are a number of potential reasons, besides a poor signal that a user may experience poor network conditions, such as: 12 | 13 | - An overloaded cellular network 14 | - Problems with the website’s server 15 | - A misconfigured proxy 16 | - Being nearby a previously accessed wifi network 17 | 18 | Creating offline experiences for our users can provide us the ability to brand and give better error messaging to our users on a poor connection, provide limited functionality of our sites to offline users, or even create seamless offline experiences. As a bonus, offline web applications work blazingly fast, providing a benefit to users on all types of connections. In this chapter, we’ll look at two technologies that make offline experiences possible, Service Workers and in-browser databases. 19 | 20 | 21 | ## Service Workers 22 | 23 | [Service Workers](https://www.w3.org/TR/service-workers/) are a script that runs separately from the page, which provide us with a way to make our sites to work offline, run faster, and add capabilities for background features. With the limits of connectivity, Service Workers provide us with a means to build offline-first capable applications, which will load content for our users, after an initial site visit, regardless of network conditions. Best of all, Service Workers are truly a progressive enhancement, layering on an additional feature to supporting browsers without changing the functionality of our site for users of non-supporting browsers. 24 | 25 | Service Workers present us with many possibilities for how we handle user connectivity. For our purposes, let’s build a simple static site example that will cache all of our site’s static assets. If you are interested in following along, you can download a Service Worker free version of this example at [https://github.com/ascott1/sw-demo/archive/no-sw.zip](https://github.com/ascott1/sw-demo/archive/no-sw.zip). 26 | 27 | 38 | 39 | 40 | The first step of working with a Service Worker is registering the script that will contain our Service Worker code. Let’s begin by adding that code to our HTML pages. At the bottom of the page, just before the closing `` tag let’s add the script registration: 41 | 42 | ``` 43 | 44 | 55 | ``` 56 | 57 | This script checks for service worker support and if the support is available points the browser to a service worker script (in our case `service-worker.js`). For debugging purposes we’re also catching errors and logging the error to the console. 58 | 59 | Now that we have our script registration, let’s write our service worker. To begin create a `service-worker.js` file and place it in the root of the directory. Let’s start by specifying a version of our cache and listing the files we would like the service worker to cache. In our case we’ll cache our two HTML pages, a CSS file, a JS file, and an image. 60 | 61 | ``` 62 | var cacheVersion = 'v1'; 63 | 64 | filesToCache = [ 65 | '/', 66 | '/index.html', 67 | '/about.html', 68 | '/css/main.css', 69 | '/js/main.js', 70 | '/img/gear.png' 71 | ] 72 | ``` 73 | 74 | If we make changes to our site, we would need to increment the cacheVersion value or risk users being served content from our cache. 75 | 76 | Now we can set up two event listeners in our service worker, `install` and `fetch`. The `install` service worker provides the browser with instructions for installing our cached files, while `fetch` provides the browser with guidelines for handling fetch events by providing the browser with either our cached files or those received over the network. 77 | 78 | ```js 79 | self.addEventListener('install', function (e) { 80 | e.waitUntil(caches.open(cacheVersion) 81 | .then(function (cache) { 82 | return cache.addAll(filesToCache) 83 | .then(function () { 84 | return self.skipWaiting(); 85 | }); 86 | })); 87 | }); 88 | 89 | self.addEventListener('fetch', function (event) { 90 | event.respondWith(caches.match(event.request) 91 | .then(function (res) { 92 | return res || fetch(event.request); 93 | })); 94 | }); 95 | ``` 96 | 97 | You can the full version of our service-worker.js file at [https://github.com/ascott1/sw-demo/blob/gh-pages/service-worker.js](https://github.com/ascott1/sw-demo/blob/gh-pages/service-worker.js). 98 | 99 | With these additions our simple static site is ready to work offline. To see it in action, visit the demo page at [https://ascott1.github.io/sw-demo/](https://ascott1.github.io/sw-demo/). 100 | 101 | To test the offline capability of our site, we can visit the Network panel, change the Throttling setting to Offline, and reload our page. 102 | 103 | ![img/sw-throttle.png](img/sw-throttle.png) 104 | 105 | Despite being offline, our site and site assets continue to load and are navigable. This example is simple, loading a two page static site and with minimal error handling. To dive into how these concepts can be applied to production ready sites and applications, see the further reading section at the end of this chapter. 106 | 107 | [^1]: https://jakearchibald.github.io/isserviceworkerready/ 108 | 109 | ### Service Worker Tools 110 | 111 | Managing our site’s Service Worker by hand can become unwieldy, thankfully the Google Chrome team has developed two incredibly useful tools for incorporating Service Workers into our development process. 112 | 113 | [sw-precache](https://github.com/GoogleChrome/sw-precache) is a Node.js module that generates service workers for precaching static resources, similar to our demo. sw-precache even handles the versioning and cache busting, making it much simpler than managing a service worker by hand. Helpfully, they also provide sample [Gulp](https://github.com/GoogleChrome/sw-precache/blob/master/demo/gulpfile.js) and [Grunt](https://github.com/GoogleChrome/sw-precache/blob/master/demo/Gruntfile.js) configurations. The module can also be used standalone from the command line or as part of a [package.json script](https://github.com/ascott1/ethicalweb.org/blob/master/package.json#L28). 114 | 115 | Here is a sample Gulp configuration for sw-precache that would cache all of our HTML, CSS, JS, and image files. 116 | 117 | ``` 118 | var swPrecache = require('sw-precache'); 119 | 120 | gulp.task('generate-service-worker', function(callback) { 121 | swPrecache.write('service-worker.js'), { 122 | staticFileGlobs: [ 123 | rootDir + '/**/*.{ 124 | html, 125 | css 126 | js, 127 | png, 128 | jpg, 129 | gif, 130 | svg 131 | }' 132 | ] 133 | }, callback); 134 | }); 135 | ``` 136 | 137 | [sw-toolbox](https://github.com/GoogleChrome/sw-toolbox) is a script that can be imported into a Service Working, providing an API for helpers such as a variety of caching strategies, Express-style and Regex routing, and cache age. The full API is available at [https://googlechrome.github.io/sw-toolbox/docs/master/tutorial-api](https://googlechrome.github.io/sw-toolbox/docs/master/tutorial-api). 138 | 139 | ## In-Browser Databases 140 | 141 | In-browser databases provide us with a way to store persistent data directly in a user’s browser. This allows us to store user data locally or to sync data from a database for offline use. This is similar to how a native mobile application might handle user data, storing user files locally and periodically syncing with a server when a device is connected to the network. 142 | 143 | The standard for in browser storage is [IndexedDB](https://www.w3.org/TR/IndexedDB/), a hierarchical key/value database for in browser use with good browser support[^1]. Let’s look at how we might add an IndexedDB database to a site. 144 | 145 | The first step when working with IndexedDB is to create and open a database. 146 | 147 | ``` 148 | var indexedDB = window.indexedDB; 149 | var open = indexedDB.open('ShuttleDatabase', 1); 150 | ``` 151 | 152 | Next we will create the schema for our database, by adding the object stores we will need for our database as part of the `on upgradeneeded`method: 153 | 154 | ``` 155 | open.onupgradeneeded = function() { 156 | var db = open.result; 157 | var store = db.createObjectStore('Missions', {keyPath: "id"}); 158 | }; 159 | ``` 160 | 161 | Then we can create event handlers for both successful creation or to handle errors. 162 | 163 | ``` 164 | open.onerror = function(event) { 165 | // error handler 166 | console.log( 167 | 'Houston, we have problem: ' + event.target.errorCode 168 | ); 169 | }; 170 | 171 | open.onsuccess = function(event) { 172 | // success 173 | console.log('We have liftoff!'); 174 | }; 175 | ``` 176 | 177 | Now let’s start a new database transaction and add some data to our database. 178 | 179 | ``` 180 | open.onsuccess = function() { 181 | var db = open.result; 182 | var transaction = db.transaction('Missions', 'readwrite'); 183 | var objectStore = transaction.objectStore('Missions'); 184 | 185 | // our data 186 | objectStore.put({ 187 | id: "STS-41-D", 188 | shuttle: "Discovery", 189 | crew: 6, 190 | launchDate: new Date(1984, 07, 30, 12, 41, 50) // 30 August 1984 12:41:50 191 | }); 192 | objectStore.put({ 193 | id: "STS-51-J", 194 | shuttle: "Atlantis", 195 | crew: 5, 196 | launchDate: new Date(1985, 09, 03, 15, 15, 30) // 3 October 1985 15:15:30 197 | }); 198 | } 199 | ``` 200 | 201 | We can then query that data inside of our `onsuccess` handler. 202 | 203 | ``` 204 | var getDiscovery = objectStore.get('STS-41-D'); 205 | var getAtlantis = objectStore.get('STS-51-J'); 206 | 207 | getColumbia.onsuccess = function() { 208 | console.log(getDiscovery.result.shuttle); 209 | }; 210 | 211 | getChallenger.onsuccess = function() { 212 | console.log(getAtlantis.result.launchDate); 213 | }; 214 | ``` 215 | 216 | Lastly we need to close the database transaction once we are done. 217 | 218 | ``` 219 | transaction.oncomplete = function() { 220 | db.close(); 221 | }; 222 | ``` 223 | 224 | Putting it all together, it would look like this: 225 | 226 | ```javascript 227 | var indexedDB = window.indexedDB; 228 | 229 | // open or create the database 230 | var open = indexedDB.open('ShuttlesDatabase', 1); 231 | 232 | // open or create the schema 233 | open.onupgradeneeded = function() { 234 | var db = open.result; 235 | var store = db.createObjectStore('Missions', {keyPath: "id"}); 236 | }; 237 | 238 | // handle errors 239 | open.onerror = function(event) { 240 | console.log( 241 | 'Houston, we have problem: ' + event.target.errorCode 242 | ); 243 | }; 244 | 245 | open.onsuccess = function() { 246 | // begin the transaction 247 | var db = open.result; 248 | var transaction = db.transaction('Missions', 'readwrite'); 249 | var objectStore = transaction.objectStore('Missions'); 250 | 251 | // add data 252 | objectStore.put({ 253 | id: "STS-41-D", 254 | shuttle: "Discovery", 255 | crew: 6, 256 | launchDate: new Date(1984, 07, 30, 12, 41, 50) // 30 August 1984 12:41:50 257 | }); 258 | objectStore.put({ 259 | id: "STS-51-J", 260 | shuttle: "Atlantis", 261 | crew: 5, 262 | launchDate: new Date(1985, 09, 03, 15, 15, 30) // 3 October 1985 15:15:30 263 | }); 264 | 265 | // query our data 266 | var getDiscovery = objectStore.get('STS-41-D'); 267 | var getAtlantis = objectStore.get('STS-51-J'); 268 | 269 | getColumbia.onsuccess = function() { 270 | console.log(getDiscovery.result.shuttle); 271 | }; 272 | 273 | getChallenger.onsuccess = function() { 274 | console.log(getAtlantis.result.launchDate); 275 | }; 276 | 277 | // close the db when the transaction is done 278 | transaction.oncomplete = function() { 279 | db.close(); 280 | }; 281 | } 282 | ``` 283 | 284 | IndexedDB is an exciting technology, but the API leaves a little to be desired. [localForage](https://github.com/mozilla/localForage) is a library from Mozilla that creates an asynchronous API (using either Promises or Node-style callbacks) for in-browser databases. It also expands the browser capability of offline storage by supporting IndexedDB and WebSQL with a localStorage fallback. Through these additions, localForage simplifies the code needed to create, add data to, and retrieve data from our database. Here’s a version of the above code that would add our data to localForage and log the results. 285 | 286 | ``` 287 | // our data 288 | var shuttles = [ 289 | { 290 | id: "STS-41-D", 291 | shuttle: "Discovery", 292 | crew: 6, 293 | launchDate: new Date(1984, 07, 30, 12, 41, 50) // 30 August 1984 12:41:50 294 | }, 295 | { 296 | id: "STS-51-J", 297 | shuttle: "Atlantis", 298 | crew: 5, 299 | launchDate: new Date(1985, 09, 03, 15, 15, 30) // 3 October 1985 15:15:30 300 | } 301 | ]; 302 | 303 | // store the data 304 | localforage.setItem('shuttles', shuttles); 305 | 306 | // retrieve the data 307 | localforage.getItem('shuttles').then(function(value) { 308 | console.log(value); 309 | }).catch(function(err) { 310 | console.log('Houston, we have a problem'); 311 | }); 312 | 313 | ``` 314 | 315 | 316 | Though our in-browser database may make it simpler for users to access our applications in a disconnected state, it is likely that we will not want the data to live only in the browser. To handle this, we will most likely want to sync user data when the user is online. We can do this with IndexedDB and our database of choice. Another attractive option is [PouchDB](https://pouchdb.com/), which is a JavaScript implementation of [Apache CouchDB](https://couchdb.apache.org/). PouchDB provides a local database API and makes it easy to sync the local database with any remote instance of CouchDB. 317 | 318 | Using an in-browser database may not be ideal for all applications, but it expands the suite of solutions for building applications that are responsive in a wide variety of network conditions. By considering these solutions, we give our users the opportunity to connect with our application’s data offline. 319 | 320 | [^1]: http://caniuse.com/#feat=indexeddb 321 | 322 | ## Additional libraries and tools 323 | 324 | The libraries and tools covered in this chapter are just a small fraction of those available to us for developing offline capable applications. Below is a list of a few other useful tools that are worth your investigation. 325 | 326 | - [Hoodie](http://hood.ie/) 327 | - [remoteStorage](https://remotestorage.io/) 328 | - [Kinto](http://www.kinto-storage.org/) 329 | - [IndexedDB Promised](https://github.com/jakearchibald/indexeddb-promised) 330 | - [Webpack offline-plugin](https://github.com/NekR/offline-plugin) 331 | - [UpUp](https://www.talater.com/upup/) 332 | - [Offline.js](http://github.hubspot.com/offline/docs/welcome/) 333 | - [Hyperboot](http://hyperboot.org/) 334 | 335 | 336 | ## Conclusion 337 | 338 | The site [Offline States](http://offlinestat.es/) collects screenshots of mobile applications in a disconnected state, providing good inspiration for how to handle (or not handle) disconnected user states. 339 | 340 | ## Further Reading 341 | 342 | - [Offline First](http://offlinefirst.org/) 343 | - [The Offline Cookbook](https://jakearchibald.com/2014/offline-cookbook/) 344 | - [Designing Offline-First Web Apps](http://alistapart.com/article/offline-first) 345 | - [Offline-First Awesome List](https://github.com/pazguille/offline-first) 346 | - [Service Worker Cookbook](https://serviceworke.rs/) 347 | - [Service Workers Explained](https://github.com/slightlyoff/ServiceWorker/blob/master/explainer.md) 348 | - [Your first offline web app](https://developers.google.com/web/fundamentals/getting-started/your-first-offline-web-app) 349 | - [Using IndexedDB](https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Using_IndexedDB) 350 | - [MDN: Working Offline](https://developer.mozilla.org/en-US/Apps/Fundamentals/Offline) 351 | - [Service Workers in Production](https://developers.google.com/web/showcase/2015/service-workers-iowa) -------------------------------------------------------------------------------- /web-apps-that-work-everywhere/06-conclusion.md: -------------------------------------------------------------------------------- 1 | # In conclusion 2 | 3 | Thank you for taking the time to read “Building Web Apps that Work Everywhere.” 4 | 5 | If throughout your reading you have come across things that are missing or could be improved, I would encourage you to contribute back to the book. This title is available as open source and contributions can be made by: 6 | 7 | - Contributing directly to the GitHub repository with a pull request . 8 | - Creating an issue in the book’s GitHub repository . 9 | - Reaching out to me through [email](mailto:adamdscott@protonmail.com) or [Twitter](https://twitter.com/adamdscott). 10 | 11 | 20% of the proceeds, from each Ethical Web Development title will be donated to an organization whose work has a positive impact on the issues described. For this title, I will be donating to the World Wide Web Foundation. The World Wide Web Foundation was established in 2009 by Web inventor Sir Tim Berners-Lee to advance the open Web as a public good and a basic right. 12 | 13 | If you are interested in supporting the World Wide Web Foundation’s work please consider making a donation at [webfoundation.org/get-involved](http://webfoundation.org/get-involved/). 14 | 15 | This title is part of a series of digital reports I am authoring on the subject of ethical web development. Other titles in the series will cover building web applications that work for everyone, building web applications that respect a user's privacy and security, and working with development peers. You can learn more about the series at . 16 | -------------------------------------------------------------------------------- /web-apps-that-work-everywhere/README.md: -------------------------------------------------------------------------------- 1 | # Building Web Applications that Work Everywhere 2 | 3 | - [Preface](https://github.com/ascott1/ethical-web-dev/blob/master/preface.md) 4 | - [Introduction](01-title-intro.md) 5 | - [URLs](02-url.md) 6 | - [Responsive Design](03-responsive-design.md) 7 | - [Web Performance](04-web-performance.md) 8 | - [Offline](05-offline.md) 9 | - [Conclusion](06-conclusion.md) 10 | - [Contributors](contributors.md) 11 | -------------------------------------------------------------------------------- /web-apps-that-work-everywhere/contributors.md: -------------------------------------------------------------------------------- 1 | # Contributors 2 | 3 | ## About the Author 4 | 5 | Adam D. Scott is a developer and educator based in Connecticut. He currently works as a senior front-end development fellow at the Consumer Financial Protection Bureau, where he focuses on building open source tools. Additionally, he has worked in education for over a decade, teaching and writing curriculum on a range of technical topics. His first book, WordPress for Education, was published in 2012. His video course, Introduction to Modern Front-End Development, was published by O'Reilly in 2015. 6 | 7 | ## Technical Reviewer 8 | 9 | TBD 10 | 11 | ## Editor 12 | 13 | Meg Foley 14 | 15 | ## Other Contributions 16 | 17 | The following people have graciously contributed feedback and improvements. 18 | 19 | - TBD 20 | 21 | Contributions and suggestions have also been made to the ethicalweb.org site and the core principles of ethical web development. Those contributions are stored at 22 | -------------------------------------------------------------------------------- /web-apps-that-work-everywhere/img/average-page-weight.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-that-work-everywhere/img/average-page-weight.png -------------------------------------------------------------------------------- /web-apps-that-work-everywhere/img/filmstrip.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-that-work-everywhere/img/filmstrip.png -------------------------------------------------------------------------------- /web-apps-that-work-everywhere/img/first-website.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-that-work-everywhere/img/first-website.png -------------------------------------------------------------------------------- /web-apps-that-work-everywhere/img/load-time.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-that-work-everywhere/img/load-time.png -------------------------------------------------------------------------------- /web-apps-that-work-everywhere/img/network-tab.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-that-work-everywhere/img/network-tab.png -------------------------------------------------------------------------------- /web-apps-that-work-everywhere/img/st-506.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-that-work-everywhere/img/st-506.jpg -------------------------------------------------------------------------------- /web-apps-that-work-everywhere/img/sw-network.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-that-work-everywhere/img/sw-network.png -------------------------------------------------------------------------------- /web-apps-that-work-everywhere/img/sw-sources.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-that-work-everywhere/img/sw-sources.png -------------------------------------------------------------------------------- /web-apps-that-work-everywhere/img/sw-throttle.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-that-work-everywhere/img/sw-throttle.png -------------------------------------------------------------------------------- /web-apps-that-work-everywhere/img/throttle.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-that-work-everywhere/img/throttle.png -------------------------------------------------------------------------------- /web-apps-that-work-everywhere/img/waterfall.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ascott1/ethical-web-dev/2b34ea22ab96d11477b817b0f36ab16ba7dc2190/web-apps-that-work-everywhere/img/waterfall.png --------------------------------------------------------------------------------