├── _data ├── favicons.yml ├── footer.yml ├── header.yml ├── usa_identifier.yml ├── theme.yml └── navigation.yml ├── .ruby-version ├── .bundler-version ├── assets ├── js │ ├── main.js │ ├── application.js │ └── private-eye.js └── images │ ├── drill-small-wikimedia.png │ ├── screenshot-fake-epg-hacked.png │ ├── rerun-workflow-circleci-screenshot.png │ ├── angle-arrow-down-white.svg │ ├── angle-arrow-up-white.svg │ ├── Slack_Mark.svg │ └── gsa-logo-blue.svg ├── favicon.ico ├── _includes ├── components │ ├── tag-suggestion.html │ ├── tag-default.html │ ├── tag-requirement.html │ ├── tag-caution.html │ ├── tag-standard.html │ └── header.html ├── categorylinks.html └── footer.html ├── _pages ├── sharepoint │ ├── power_bi.png │ ├── text_box.png │ ├── client_side.png │ ├── power_app.png │ ├── dev_workflow.png │ ├── dev_architecture.png │ ├── embedded_excel.png │ └── sharepoint_list.png ├── javascript │ └── atom-eslint-example.png ├── architecture-reviews │ ├── micro-purchase │ │ ├── roles.png │ │ └── service.png │ ├── data-act-pilot │ │ └── data-act-diagram.png │ └── data-act-pilot.md ├── incident-reports │ ├── C2 │ │ └── c2-outage-report-2016-08-10.pdf │ └── cloud-gov.md ├── markdown.md ├── index.md ├── example-workflows.md ├── resources.md ├── ruby.md ├── tests │ └── tests.md ├── license.md ├── frontend.md ├── people.md ├── architecture-reviews.md ├── books-we-have-read.md ├── security │ ├── output-encoding.md │ ├── dependency-remediation.md │ ├── cloud-services.md │ └── content-security-policy.md ├── about.md ├── browser-testing.md ├── incident-reports.md ├── workflow.md ├── integrations.md ├── development-environments.md ├── laptop-setup.md ├── continuous-deployment.md ├── security.md ├── python.md ├── nodejs.md ├── language-selection.md ├── datastore-selection.md ├── ruby │ └── rubocop.yml ├── web-architecture.md ├── release-strategies.md ├── code-review.md └── project-setup.md ├── .pa11yci ├── docker-compose.yml ├── Dockerfile ├── .gitignore ├── _sass ├── _uswds-theme-settings.scss ├── _usa_identifier.scss └── _uswds-theme-custom-styles.scss ├── 404.html ├── Gemfile ├── doc └── bpdr │ ├── 0001-record-best-practices-decisions.md │ ├── 0002-on-call-recs.md │ └── 0003-set-criteria-for-static-site-framework.md ├── package.json ├── _config.yml ├── .circleci └── config.yml ├── LICENSE.md ├── CONTRIBUTING.md ├── .github └── workflows │ └── pa11y.yml ├── Rakefile ├── Gemfile.lock └── README.md /_data/favicons.yml: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /.ruby-version: -------------------------------------------------------------------------------- 1 | 2.7.2 2 | -------------------------------------------------------------------------------- /.bundler-version: -------------------------------------------------------------------------------- 1 | 2.1.4 2 | -------------------------------------------------------------------------------- /assets/js/main.js: -------------------------------------------------------------------------------- 1 | --- 2 | --- 3 | -------------------------------------------------------------------------------- /_data/footer.yml: -------------------------------------------------------------------------------- 1 | # Explicitly disable footer; only using anchor 2 | type: 3 | -------------------------------------------------------------------------------- /favicon.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/18F/development-guide/HEAD/favicon.ico -------------------------------------------------------------------------------- /_includes/components/tag-suggestion.html: -------------------------------------------------------------------------------- 1 | Suggestion 2 | -------------------------------------------------------------------------------- /_data/header.yml: -------------------------------------------------------------------------------- 1 | usa_banner: true 2 | type: extended 3 | primary: 4 | links: primary 5 | -------------------------------------------------------------------------------- /_includes/components/tag-default.html: -------------------------------------------------------------------------------- 1 | Default 2 | -------------------------------------------------------------------------------- /_pages/sharepoint/power_bi.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/18F/development-guide/HEAD/_pages/sharepoint/power_bi.png -------------------------------------------------------------------------------- /_pages/sharepoint/text_box.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/18F/development-guide/HEAD/_pages/sharepoint/text_box.png -------------------------------------------------------------------------------- /_includes/components/tag-requirement.html: -------------------------------------------------------------------------------- 1 | Requirement 2 | -------------------------------------------------------------------------------- /_pages/sharepoint/client_side.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/18F/development-guide/HEAD/_pages/sharepoint/client_side.png -------------------------------------------------------------------------------- /_pages/sharepoint/power_app.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/18F/development-guide/HEAD/_pages/sharepoint/power_app.png -------------------------------------------------------------------------------- /.pa11yci: -------------------------------------------------------------------------------- 1 | { 2 | "defaults": { 3 | "concurrency": 4, 4 | "standard": "WCAG2AA", 5 | "runners": ["axe"] 6 | } 7 | } -------------------------------------------------------------------------------- /_includes/components/tag-caution.html: -------------------------------------------------------------------------------- 1 | Caution 2 | -------------------------------------------------------------------------------- /_pages/sharepoint/dev_workflow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/18F/development-guide/HEAD/_pages/sharepoint/dev_workflow.png -------------------------------------------------------------------------------- /_includes/components/tag-standard.html: -------------------------------------------------------------------------------- 1 | Standard 2 | -------------------------------------------------------------------------------- /_pages/sharepoint/dev_architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/18F/development-guide/HEAD/_pages/sharepoint/dev_architecture.png -------------------------------------------------------------------------------- /_pages/sharepoint/embedded_excel.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/18F/development-guide/HEAD/_pages/sharepoint/embedded_excel.png -------------------------------------------------------------------------------- /_pages/sharepoint/sharepoint_list.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/18F/development-guide/HEAD/_pages/sharepoint/sharepoint_list.png -------------------------------------------------------------------------------- /assets/images/drill-small-wikimedia.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/18F/development-guide/HEAD/assets/images/drill-small-wikimedia.png -------------------------------------------------------------------------------- /_pages/javascript/atom-eslint-example.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/18F/development-guide/HEAD/_pages/javascript/atom-eslint-example.png -------------------------------------------------------------------------------- /assets/images/screenshot-fake-epg-hacked.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/18F/development-guide/HEAD/assets/images/screenshot-fake-epg-hacked.png -------------------------------------------------------------------------------- /_pages/architecture-reviews/micro-purchase/roles.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/18F/development-guide/HEAD/_pages/architecture-reviews/micro-purchase/roles.png -------------------------------------------------------------------------------- /assets/images/rerun-workflow-circleci-screenshot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/18F/development-guide/HEAD/assets/images/rerun-workflow-circleci-screenshot.png -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: "3" 2 | services: 3 | web: 4 | build: . 5 | ports: 6 | - "4000:4000" 7 | volumes: 8 | - .:/usr/src/app 9 | -------------------------------------------------------------------------------- /_pages/architecture-reviews/micro-purchase/service.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/18F/development-guide/HEAD/_pages/architecture-reviews/micro-purchase/service.png -------------------------------------------------------------------------------- /_pages/incident-reports/C2/c2-outage-report-2016-08-10.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/18F/development-guide/HEAD/_pages/incident-reports/C2/c2-outage-report-2016-08-10.pdf -------------------------------------------------------------------------------- /_pages/architecture-reviews/data-act-pilot/data-act-diagram.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/18F/development-guide/HEAD/_pages/architecture-reviews/data-act-pilot/data-act-diagram.png -------------------------------------------------------------------------------- /_pages/incident-reports/cloud-gov.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: 'Cloud.gov post mortems' 3 | --- 4 | 5 | Cloud.gov post mortems (including incident response) are hosted at 6 | https://github.com/18F/cg-postmortems 7 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ruby:2.7.2 2 | 3 | WORKDIR /usr/src/app 4 | 5 | COPY Gemfile Gemfile.lock ./ 6 | RUN bundle install 7 | 8 | ENV LC_ALL=C.UTF-8 9 | 10 | CMD bundle exec jekyll serve --host 0.0.0.0 --incremental 11 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.gem 2 | node_modules/ 3 | .DS_Store 4 | 5 | # Environment normalization: 6 | /.bundle/ 7 | /vendor/bundle 8 | /lib/bundler/man/ 9 | 10 | # unless supporting rvm < 1.11.0 or doing something fancy, ignore this: 11 | .rvmrc 12 | 13 | # Jekyll stuff 14 | .jekyll-cache 15 | .jekyll-metadata 16 | .jekyll-cache 17 | _site 18 | .sass-cache 19 | -------------------------------------------------------------------------------- /_sass/_uswds-theme-settings.scss: -------------------------------------------------------------------------------- 1 | // Just put any changed settings in here! 2 | 3 | $theme-show-notifications: false; 4 | $theme-font-type-sans: "helvetica"; 5 | $theme-font-role-heading: "sans"; 6 | 7 | $theme-color-primary-dark: "blue-80v"; 8 | $theme-color-primary-darker: "blue-90"; 9 | $theme-type-scale-sm: 6; 10 | $theme-type-scale-md: 7; 11 | -------------------------------------------------------------------------------- /_pages/markdown.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Markdown Guide 3 | sidenav: languages 4 | sticky_sidenav: true 5 | --- 6 | 7 | ["Markdown is a text-to-HTML conversion tool for web writers. Markdown allows you to write using an easy-to-read, easy-to-write plain text format."](https://daringfireball.net/projects/markdown/) 8 | 9 | Use [Prettier](https://prettier.io/) for automatic code formatting. 10 | -------------------------------------------------------------------------------- /_includes/categorylinks.html: -------------------------------------------------------------------------------- 1 |
2 | {% for link in include.links %} 3 | {% if forloop.first == true %} 4 |

{{ link.text }}

5 | Overview
6 | {% else %} 7 | {{ link.text }}
8 | {% endif %} 9 | {% endfor %} 10 |
11 | -------------------------------------------------------------------------------- /404.html: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | permalink: /404.html 4 | --- 5 | 6 | 19 | 20 |
21 |

404

22 | 23 |

Page not found :(

24 |

The requested page could not be found.

25 |
26 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | source 'https://rubygems.org' 2 | 3 | ruby '2.7.2' 4 | 5 | gem 'jekyll' 6 | gem 'mini_racer' 7 | 8 | # If you put them in a jekyll_plugins group they’ll automatically be required into Jekyll 9 | # One of 3 ways to load plug-ins, don't need to do both 10 | group :jekyll_plugins do 11 | gem 'uswds-jekyll', '~> 5.3' 12 | gem 'jekyll-redirect-from' 13 | gem 'jekyll-sitemap' 14 | gem 'jemoji', '>= 0.12.0' 15 | gem 'jekyll-last-modified-at' 16 | end 17 | 18 | group :development, :test do 19 | gem 'html-proofer' 20 | gem 'rake' 21 | end 22 | -------------------------------------------------------------------------------- /assets/images/angle-arrow-down-white.svg: -------------------------------------------------------------------------------- 1 | angle-arrow-down-white -------------------------------------------------------------------------------- /assets/images/angle-arrow-up-white.svg: -------------------------------------------------------------------------------- 1 | angle-arrow-up-white -------------------------------------------------------------------------------- /doc/bpdr/0001-record-best-practices-decisions.md: -------------------------------------------------------------------------------- 1 | 1. Record best practices decisions 2 | Date: 2022-03-10 3 | 4 | Status 5 | Accepted 6 | 7 | Context 8 | As we make best practices decisions, it would be good to have a record 9 | of those decisions so that people can understand the historical context 10 | and review/revisit whether the circumstances of those decisions are still 11 | appropriate. 12 | 13 | Decision 14 | We will use Best Practices Decision Records (BPDR), modified from Architecture Decision 15 | Records (ADR) [as described by Michael Nygard](https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions). 16 | 17 | Consequences 18 | Best practices decisions are available in a consistent format to help explain the 19 | reasons and motivations behind creating a best practice. -------------------------------------------------------------------------------- /_pages/index.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Guidelines and Best Practices 3 | permalink: / 4 | sticky_sidenav: true 5 | --- 6 | 7 | This guide is where the TTS Engineering Practices Guild collects its best practices and resources for software development at TTS, as well as on our partner engagements. Our focus is cloud-native digital services and our recommendations in this guide reflect the needs of that domain. 8 | 9 |
10 |
11 | {% include categorylinks.html links=site.data.navigation.about %} 12 | {% include categorylinks.html links=site.data.navigation.approach %} 13 |
14 |
15 | {% include categorylinks.html links=site.data.navigation.tools %} 16 | {% include categorylinks.html links=site.data.navigation.languages %} 17 |
18 |
19 | {% include categorylinks.html links=site.data.navigation.security %} 20 |
21 |
22 | -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "development-guide", 3 | "version": "1.0.0", 4 | "description": "This repo will contain the guidelines and best practices for the engineering chapter at 18F. The site is a living document.", 5 | "main": "index.js", 6 | "private": true, 7 | "author": "18F", 8 | "license": "CC0-1.0", 9 | "scripts": { 10 | "start-detached": "bundle exec jekyll serve --detach", 11 | "pa11y-ci:home": "pa11y-ci http://localhost:4000", 12 | "pa11y-ci:sitemap": "pa11y-ci --sitemap https://engineering.18f.gov/sitemap.xml --sitemap-find https://engineering.18f.gov --sitemap-replace http://localhost:4000 --sitemap-exclude \"/*.pdf\"" 13 | }, 14 | "repository": { 15 | "type": "git", 16 | "url": "git+https://github.com/18F/development-guide.git" 17 | }, 18 | "bugs": { 19 | "url": "https://github.com/18F/development-guide/issues" 20 | }, 21 | "devDependencies": { 22 | "pa11y-ci": "^2.4.0" 23 | } 24 | } 25 | -------------------------------------------------------------------------------- /_config.yml: -------------------------------------------------------------------------------- 1 | theme: uswds-jekyll 2 | exclude: 3 | - CONTRIBUTING.md 4 | - Dockerfile 5 | - docker-compose.yml 6 | - Gemfile 7 | - Gemfile.lock 8 | - LICENSE.md 9 | - README.md 10 | - node_modules 11 | 12 | permalink: pretty 13 | 14 | sass: 15 | style: :compressed 16 | 17 | collections: 18 | pages: 19 | output: true 20 | permalink: /:path/ 21 | 22 | defaults: 23 | - scope: 24 | path: "" 25 | values: 26 | layout: post 27 | 28 | scripts: 29 | - /assets/uswds/js/uswds.min.js 30 | - /assets/js/private-eye.js 31 | - /assets/js/application.js 32 | 33 | title: TTS Engineering Practices Guide 34 | description: A set of guidelines and best practices for an awesome engineering team 35 | url: "https://engineering.18f.gov" 36 | search_site_handle: engineering.18f.gov 37 | 38 | google_analytics_ua: UA-48605964-19 39 | dap_agency: GSA 40 | dap_subagency: TTS 41 | 42 | github_info: 43 | organization: 18F 44 | repository: development-guide 45 | default_branch: main 46 | -------------------------------------------------------------------------------- /.circleci/config.yml: -------------------------------------------------------------------------------- 1 | version: 2.1 2 | jobs: 3 | build: 4 | docker: 5 | - image: circleci/ruby:2.7.2 6 | environment: 7 | # fix encoding 8 | - LANG: C.UTF-8 9 | steps: 10 | - checkout 11 | 12 | - restore_cache: 13 | keys: 14 | # Find a cache corresponding to this specific package-lock.json checksum 15 | # when this file is changed, this key will fail 16 | - v1-gem-dependencies-{{ checksum "Gemfile.lock" }} 17 | # Find the most recently generated cache used from any branch 18 | - v1-gem-dependencies- 19 | 20 | - run: 21 | name: Install dependencies 22 | command: bundle install --jobs=4 --retry=3 --path vendor/bundle 23 | 24 | - save_cache: 25 | key: v1-gem-dependencies-{{ checksum "Gemfile.lock" }} 26 | paths: 27 | - vendor/bundle 28 | 29 | - run: 30 | name: Build and Test site 31 | command: bundle exec rake ci_test 32 | -------------------------------------------------------------------------------- /_sass/_usa_identifier.scss: -------------------------------------------------------------------------------- 1 | // Identifier component - - - - - - - - - - - - - - - 2 | // Using the USWDS Design Tokens 3 | 4 | // fix footer to the bottom on short pages 5 | @include at-media("desktop") { 6 | body { 7 | display: flex; 8 | flex-direction: column; 9 | min-height: 100vh; 10 | } 11 | } 12 | 13 | .usa-identifier { 14 | margin-top: auto; 15 | } 16 | 17 | .component-identifier-meta{ 18 | @include u-bg('white'); 19 | @include u-text('black'); 20 | .page-meta{ 21 | @include u-padding-y(1); 22 | @include u-display('block'); 23 | @include at-media('tablet') { 24 | @include u-display('flex'); 25 | @include u-font('sans', 'sm'); 26 | @include u-flex('align-center'); 27 | @include u-flex('justify-end'); 28 | } 29 | p{ 30 | @include u-margin-y('05'); 31 | @include u-flex('align-center'); 32 | @include u-line-height('sans', 2); 33 | @include u-font('sans', 'xs'); 34 | @include at-media('tablet') { 35 | @include u-font('sans', 'sm'); 36 | } 37 | } 38 | } 39 | } 40 | -------------------------------------------------------------------------------- /doc/bpdr/0002-on-call-recs.md: -------------------------------------------------------------------------------- 1 | 1. On-call recommendations 2 | Date: 2022-03-10 3 | 4 | Status 5 | Accepted 6 | 7 | Context 8 | 18F recently built an emergent website that needed on-call support from non-team members. 9 | Because of the urgency of the situation, the on-call needs were left to the last minute. 10 | 11 | Meanwhile, 18F is moving towards being more product-focused, and as such, we may need 12 | to support products in the long- or short-term going forward. 13 | 14 | Decision 15 | [On-call recommendations](https://engineering.18f.gov/on-call/) were created by Davida Marion, 16 | under consultation with Christa Hartsock and Logan McDonald. Recommendations were 17 | [reviewed by](https://github.com/18F/development-guide/pull/298): 18 | Jessica Dussault, Laura Gerhardt, Alex Soble, Michael Sullivan, Neil Martinsen-Burrell, 19 | Ryan Ahern, Sarah Withee, and Sven Aas. 20 | 21 | Consequences 22 | Engineers have a reviewed set of recommendations to take into account while working on builds 23 | that seem like they may have on-call needs. These recommendations can be inserted into the 24 | staffing phase, the development phase, and the support phase. -------------------------------------------------------------------------------- /_pages/example-workflows.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Example Workflows 3 | sidenav: approach 4 | sticky_sidenav: true 5 | --- 6 | Here we've collected descriptions of team processes (particularly around 7 | submitting code) that various projects have used in the past. Please consider 8 | using one as an example until a more formal template is provided. 9 | 10 | * [ATF eRegs](https://github.com/18F/atf-eregs/blob/master/CONTRIBUTING.md) 11 | * [cloud.gov](https://github.com/18F/cg-product/blob/master/DeliveryProcess.md) 12 | * [cloud.gov Dashboard](https://github.com/18F/cg-dashboard/blob/master/CONTRIBUTING.md) 13 | * [Communicart (C2)](https://github.com/18F/C2/blob/master/doc/team_practices.md) 14 | * [Data to Decisions (D2D)](https://docs.google.com/document/d/1N9DBZj3zooA2nK00-7_WOSEr1DOsLx5aTtxaaSFBdoM/edit#) 15 | * [eManifest](https://github.com/18F/e-manifest/blob/master/CONTRIBUTING.md#development-practices) 16 | * [eRegs Notice & Comment](https://github.com/eregs/notice-and-comment/blob/master/CONTRIBUTING.md) 17 | * [FEC's API](https://github.com/18F/openFEC/blob/develop/CONTRIBUTING.md) 18 | * [FEC's Legal Resources](https://github.com/18F/fec-eregs/blob/master/CONTRIBUTING.md) 19 | -------------------------------------------------------------------------------- /assets/js/application.js: -------------------------------------------------------------------------------- 1 | document.addEventListener('DOMContentLoaded', function() { 2 | PrivateEye({ 3 | defaultMessage: "This link is private to GSA.", 4 | ignoreUrls: [ 5 | '18f.slack.com', 6 | 'anywhere.gsa.gov', 7 | 'bookit.gsa.gov', 8 | 'calendar.gsa.gov', 9 | 'connect.gsa.gov', 10 | 'docs.google.com', 11 | 'drive.google.com', 12 | 'ea.gsa.gov', 13 | 'email.gsa.gov', 14 | 'eopf.opm.gov', 15 | 'gcims.gsa.gov', 16 | 'github.com/18F/Accessibility_Reviews', 17 | 'github.com/18F/blog-drafts', 18 | 'github.com/18F/codereviews', 19 | 'github.com/18F/DevOps', 20 | 'github.com/18F/Infrastructure', 21 | 'github.com/18F/security-incidents', 22 | 'github.com/18F/staffing', 23 | 'github.com/18F/team-api.18f.gov', 24 | 'github.com/18F/writing-lab', 25 | 'gkey.gsa.gov', 26 | 'gsa-tts.slack.com', 27 | 'gsa.my.salesforce.com', 28 | 'gsaolu.gsa.gov', 29 | 'hrlinks.gsa.gov', 30 | 'hrprod.hr.gsa.gov', 31 | 'insite.gsa.gov', 32 | 'mail.gsa.gov', 33 | 'meet.gsa.gov', 34 | 'sign.gsa.gov', 35 | 'tock.18f.gov' 36 | ] 37 | }); 38 | }, false ); 39 | -------------------------------------------------------------------------------- /_pages/resources.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Resources 3 | sidenav: about 4 | sticky_sidenav: true 5 | subnav: 6 | - text: Related links 7 | href: "#related-links" 8 | --- 9 | 10 | This guide evolved from work spearheaded by the 18F Engineering Chapter and the 18F Front-End Guild, with additional content adapted from the [Consumer Financial Protection Bureau (CFPB)](https://github.com/cfpb/development/). 11 | 12 | ## Related links 13 | 14 | - [Launching software at TTS](https://handbook.tts.gsa.gov/#launching-software) 15 | - [18F De-risking guide](https://derisking-guide.18f.gov/) 16 | - [Agile Principles and 18F Practices](https://agile.18f.gov/) 17 | - [18F guide to Consulting Engineering](https://handbook.tts.gsa.gov/18f/projects-partners/consulting-engineering-guide/) 18 | - [18F Engineering Field Guide](https://docs.google.com/document/d/1ipYaMx2oyuI4RUMdiDgIG1Q4BiiWZn6cVXp6-RcMFjI) 19 | - [U.S. Web Design System](https://designsystem.digital.gov/) 20 | - [Cloud.gov Production-ready guide](https://cloud.gov/docs/deployment/production-ready/) 21 | - [Login.gov Identity Playbook](https://login.gov/playbook/) 22 | - [Digital.gov Guides and Resources](https://digital.gov/resources/) 23 | - [GSA Tech Guides](https://tech.gsa.gov/guides/opensource/) 24 | -------------------------------------------------------------------------------- /_pages/ruby.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Ruby Guide 3 | sidenav: languages 4 | sticky_sidenav: true 5 | --- 6 | Help us make this section better by 7 | [submitting an issue](https://github.com/18F/development-guide) or joining us 8 | in the [#ruby](https://18f.slack.com/messages/ruby/) channel! 9 | 10 | A guide for writing and maintaining Ruby and Rails applications 11 | 12 | ## Style Guide 13 | 14 | Follow the [Ruby Style Guide](https://github.com/bbatsov/ruby-style-guide) and 15 | enforce it via static analysis tools such as [Code Climate] and [Rubocop]. You 16 | can copy the [recommended Rubocop configuration](rubocop.yml) in your Ruby 17 | project and make any changes based on your team's preferences. 18 | 19 | Note that the Rubocop configuration linked above only includes settings that 20 | differ from the default configuration. We tend to agree with most of the 21 | default settings. 22 | 23 | Whenever a `Style` Rubocop setting provides multiple options, at least one 24 | option must be chosen. A cop that supports different styles must never be 25 | disabled outright. The point is to pick one style and use it consistently. 26 | 27 | ## Testing 28 | 29 | ### Validating HTML output 30 | 31 | We use HTMLProofer for testing rendered HTML automatically. Please see 32 | the [test page] for more details. 33 | 34 | [Code Climate]: https://codeclimate.com 35 | [Rubocop]: https://github.com/bbatsov/rubocop 36 | [test page]: /tests#html-output-ruby 37 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | # License 2 | 3 | As a work of the [United States government](https://www.usa.gov/), this project is in the public domain within the United States of America. 4 | 5 | Additionally, we waive copyright and related rights in the work worldwide through the CC0 1.0 Universal public domain dedication. 6 | 7 | ## CC0 1.0 Universal Summary 8 | 9 | This is a human-readable summary of the [Legal Code (read the full text)](https://creativecommons.org/publicdomain/zero/1.0/legalcode). 10 | 11 | ### No Copyright 12 | 13 | The person who associated a work with this deed has dedicated the work to the public domain by waiving all of their rights to the work worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law. 14 | 15 | You can copy, modify, distribute, and perform the work, even for commercial purposes, all without asking permission. 16 | 17 | ### Other Information 18 | 19 | In no way are the patent or trademark rights of any person affected by CC0, nor are the rights that other persons may have in the work or in how the work is used, such as publicity or privacy rights. 20 | 21 | Unless expressly stated otherwise, the person who associated a work with this deed makes no warranties about the work, and disclaims liability for all uses of the work, to the fullest extent permitted by applicable law. When using or citing the work, you should not imply endorsement by the author or the affirmer. 22 | -------------------------------------------------------------------------------- /_pages/tests/tests.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Automated Testing 3 | sidenav: tools 4 | permalink: /tests/ 5 | sticky_sidenav: true 6 | --- 7 | 8 | There are many different types of automated testing, which all have 9 | separate roles to play. Ideally, automated testing can be run locally 10 | as well as part of a continuous integration workflow. 11 | ## Validating HTML output 12 | 13 | ### Ruby {#html-output-ruby} 14 | 15 | [HTMLProofer](https://github.com/gjtorikian/html-proofer) is the most common 16 | tool in use for ruby projects at TTS for validating HTML output. It is used in our 17 | [guides](https://18f.gsa.gov/guides/) to ensure that internal links are not 18 | broken, but it can also be used for a broad range of image, link, and script 19 | tests. 20 | 21 | HTMLProofer can be run on the command line directly, as a 22 | [Rake task](https://github.com/18F/isildurs-bane/blob/699502eeb374bf3414c1336290cb622e9a0f8847/Rakefile) 23 | or as part of a [CI action](https://github.com/18F/handbook/blob/cf5a76af5a1463496cd7eb1a14fdc7a422aa5ae6/.circleci/config.yml#L58-L60). 24 | 25 | **Warning**: 26 | Some government websites are very sensitive to crawlers, and so testing external government links 27 | can cause HTMLProofer to fail with an opaque error like `1 No Error`. If you are testing external 28 | links in a CI/CD pipeline, you may regularly have to manually check errors to ensure that they are 29 | valid. When a link is actually broken, HTMLProofer will display a diagnostic message about where 30 | the bad link originated. -------------------------------------------------------------------------------- /_pages/license.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: License 3 | sidenav: about 4 | sticky_sidenav: true 5 | subnav: 6 | - text: No copyright 7 | href: "#no-copyright" 8 | - text: Other information 9 | href: "#other-information" 10 | --- 11 | As a work of the [United States government](https://www.usa.gov/), this project is in the public domain within the United States of America. 12 | 13 | Additionally, we waive copyright and related rights in the work worldwide through the [CC0 1.0 Universal public domain dedication](https://creativecommons.org/publicdomain/zero/1.0/legalcode). 14 | 15 | ## No copyright 16 | 17 | The person who associated a work with this deed has dedicated the work to the public domain by waiving all of their rights to the work worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law. 18 | 19 | You can copy, modify, distribute, and perform the work, even for commercial purposes, all without asking permission. 20 | 21 | ## Other information 22 | 23 | In no way are the patent or trademark rights of any person affected by CC0, nor are the rights that other persons may have in the work or in how the work is used, such as publicity or privacy rights. 24 | 25 | Unless expressly stated otherwise, the person who associated a work with this deed makes no warranties about the work, and disclaims liability for all uses of the work, to the fullest extent permitted by applicable law. When using or citing the work, you should not imply endorsement by the author or the affirmer. 26 | -------------------------------------------------------------------------------- /_pages/frontend.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Front-End Disciplines 3 | sidenav: approach 4 | sticky_sidenav: true 5 | --- 6 | 7 | ## What is front end? 8 | 9 | The Front-End Guild did a series of exercises to determine the 10 | fundamental differences between the front-end design and front-end 11 | developer roles at 18F. Using [some of our own research 12 | methods](https://methods.18f.gov), the Front-End Guild came up with 13 | the following recommendations on knowing the difference between the 14 | two disciplines: 15 | 16 | **Front-end designers** design, write, and implement the 17 | presentational code base for websites and applications. They should 18 | have a clear understanding of design fundamentals and systems, such 19 | as interface style guides, responsive design, grid systems, front-end 20 | frameworks, and accessibility best practices. Front-end designers 21 | should feel comfortable creating and implementing design systems 22 | using semantic HTML5, CSS/Sass and be able to assist in debugging 23 | this aspect of the code base. 24 | 25 | **Front-end developers** architect, write, and implement the 26 | functional code base for websites and applications. They should have 27 | a clear understanding of client-side render and response, such as 28 | HTTP methods, API consumption, the browser loading/rendering 29 | pipeline, and accessibility best practices. Front-end developers 30 | should feel comfortable developing and implementing client-side 31 | interactions and frameworks using semantic HTML5 and JavaScript, and 32 | should be able to help with debugging, testing, and performance 33 | optimization of the code base. 34 | 35 | ## Related topics 36 | * [CSS]({{site.baseurl}}/css) 37 | * [JavaScript]({{site.baseurl}}/javascript) 38 | * [Security]({{site.baseurl}}/security) -------------------------------------------------------------------------------- /assets/images/Slack_Mark.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 5 | 11 | 12 | 13 | 14 | 16 | 17 | 18 | 19 | 21 | 22 | 23 | 24 | 26 | 27 | 28 | 29 | 31 | 32 | 33 | 34 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Welcome! 2 | 3 | We're so glad you're thinking about contributing to a [open source project of the U.S. government](https://code.gov/)! If you're unsure about anything, just ask -- or submit the issue or pull request anyway. The worst that can happen is you'll be politely asked to change something. We love all friendly contributions. 4 | 5 | We encourage you to read this project's CONTRIBUTING policy (you are here), its [LICENSE](LICENSE.md), and its [README](README.md). 6 | 7 | ## Policies 8 | 9 | We want to ensure a welcoming environment for all of our projects. Our staff follow the [TTS Code of Conduct](https://18f.gsa.gov/code-of-conduct/) and all contributors should do the same. 10 | 11 | We adhere to the [18F Open Source Policy](https://github.com/18f/open-source-policy). If you have any questions, just [shoot us an email](mailto:18f@gsa.gov). 12 | 13 | As part of a U.S. government agency, the General Services Administration (GSA)’s Technology Transformation Services (TTS) takes seriously our responsibility to protect the public’s information, including financial and personal information, from unwarranted disclosure. For more information about security and vulnerability disclosure for our projects, please read our [18F Vulnerability Disclosure Policy](https://18f.gsa.gov/vulnerability-disclosure-policy/). 14 | 15 | ## Public domain 16 | 17 | This project is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the [CC0 1.0 Universal public domain dedication](https://creativecommons.org/publicdomain/zero/1.0/). 18 | 19 | All contributions to this project will be released under the CC0 dedication. By submitting a pull request or issue, you are agreeing to comply with this waiver of copyright interest. 20 | -------------------------------------------------------------------------------- /_data/usa_identifier.yml: -------------------------------------------------------------------------------- 1 | site_name: TTS Engineering Practices Guide 2 | site_email: 18F@gsa.gov 3 | site_url: https://engineering.18f.gov 4 | site_about: https://engineering.18f.gov 5 | agency: U.S. General Services Administration 6 | agency_acronym: GSA 7 | agency_logo: gsa-logo-blue.svg 8 | agency_url: https://www.gsa.gov 9 | agency_about_url: https://www.gsa.gov/about 10 | org_primary: Technology Transformation Services 11 | org_primary_acronym: TTS 12 | org_primary_url: https://www.gsa.gov/tts/ 13 | org_primary_email: tts-info@gsa.gov 14 | org_primary_about: https://www.gsa.gov/tts/ 15 | org_primary_bio: "As part of GSA’s Technology Transformation Services (TTS), we apply modern methodologies and technologies to improve the public’s experience with government. We help agencies make their services more accessible, efficient, and effective with modern applications, platforms, processes, personnel, and software solutions." 16 | org_secondary: Technology Transformation Services 17 | org_secondary_acronym: TTS 18 | org_secondary_logo: 18f-logo-blue.svg 19 | org_secondary_url: https://18f.gsa.gov 20 | org_secondary_email: 18F@gsa.gov 21 | org_secondary_about: https://18f.gsa.gov/about/ 22 | org_secondary_bio: "TTS Solutions is a portfolio of products and services that help agencies improve delivery of information and services to the public." 23 | foia_request_url: "https://www.gsa.gov/reference/freedom-of-information-act-foia" 24 | fraud_waste_abuse_url: "https://www.gsaig.gov/" 25 | no_fear_act_url: "https://www.gsa.gov/about-us/organization/office-of-civil-rights/notification-and-federal-employee-antidiscrimination-and-retaliation-act-of-2002" 26 | budget_performance_url: "https://www.gsa.gov/reference/reports/budget-performance" 27 | accessibility_url: "https://www.gsa.gov/website-information/accessibility-aids" 28 | usagov_contact_url: "https://www.usa.gov/contact" 29 | edit_page: 30 | - text: "Edit this page" 31 | privacy_policy_url: "https://www.gsa.gov/website-information/website-policies" 32 | -------------------------------------------------------------------------------- /_pages/people.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Feedback Guide 3 | sidenav: approach 4 | sticky_sidenav: true 5 | --- 6 | Here are some attributes of giving feedback in a highly constructive way that we have learned and used over the years. 7 | 8 | ## Timely 9 | Ideally feedback, positive or negative, is given as close to the event as possible. Regular [1:1 meetings](https://handbook.18f.gov/one-on-ones/) are a great venue for delivering feedback. 10 | 11 | ## Evidence 12 | Gather evidence for the feedback instead of relying on vague reports or hunches. Have concrete examples, ideally more than one for negative feedback. 13 | 14 | ## Behaviors 15 | Describe behaviors (is late to meetings) rather than traits or emotions (doesn't care about coworkers). You'll never know anyone else's thoughts, feelings, or motivations. You can observe behaviors in an objective and factual manner. This can be a hard habit change if you've built up years of giving feedback in the other way, but keep at it, it is worth switching. 16 | 17 | ## Results 18 | When you describe a behavior, also say the result. This goes for positive and negative feedback. "When you review pull requests right away, you really help the velocity of the whole team." "When you are regularly late for meetings, the effectiveness of the rest of the meeting is reduced because the meeting leader has to repeat information or wait for you." 19 | 20 | ## Future Focus 21 | Don't dwell on the past, but focus on the future. The future can be altered, the past will never change. 22 | 23 | ## Top Performers 24 | Giving constructive feedback to a top performer is not nitpicking, it is actually some of the highest leverage work we can do. Most folks (not everyone of course) are eager to hear ways they can keep growing. We don't help folks by having no ideas for them. 25 | 26 | ## End of Year Assessment Guides 27 | 28 | TTS, as a part of GSA, has a mature [performance management and recognition system](https://insite.gsa.gov/portal/content/500278). 29 | This includes an end-of-year performance assessment. 30 | 31 | * [Historical Engineering Assessment Guide]({{site.baseurl}}/people/assessment) 32 | -------------------------------------------------------------------------------- /.github/workflows/pa11y.yml: -------------------------------------------------------------------------------- 1 | # Pulled from Daniel Mundra's blog post https://accessibility.civicactions.com/posts/automated-accessibility-testing-leveraging-github-actions-and-pa11y-ci-with-axe 2 | name: pa11y tests 3 | 4 | on: [pull_request] 5 | 6 | jobs: 7 | build: 8 | name: Building site and running pa11y-ci tests 9 | runs-on: ubuntu-latest 10 | 11 | steps: 12 | - name: Checkout source. 13 | uses: actions/checkout@v2 14 | 15 | - name: Install jekyll site dependencies. 16 | uses: ruby/setup-ruby@v1 17 | with: 18 | ruby-version: 2.7.2 19 | bundler-cache: true 20 | 21 | - name: Install pa11y-ci dependencies. 22 | run: npm install 23 | 24 | - name: Start up jekyll server. 25 | run: npm run start-detached 26 | 27 | - name: Run pa11y-ci. 28 | run: npm run pa11y-ci:sitemap 2>&1 | tee pa11y_output.txt 29 | 30 | - name: Read pa11y_output file. 31 | id: pa11y_output 32 | uses: juliangruber/read-file-action@v1 33 | with: 34 | path: ./pa11y_output.txt 35 | 36 | - name: Comment on pull request. 37 | uses: thollander/actions-comment-pull-request@main 38 | with: 39 | GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} 40 | message: '
Pa11y testing results 41 | 42 | 43 | ```${{ steps.pa11y_output.outputs.content }}``` 44 | 45 |
' 46 | 47 | # A defect in pa11y is causing intermitent page crashes [1]. Until it is 48 | # fixed, if a page crashes during a scan, do not mark the pa11y scan as 49 | # failed. 50 | # [1] https://github.com/pa11y/pa11y-ci/issues/128 51 | - name: Check for pa11y failures. 52 | if: "${{ contains(steps.pa11y_output.outputs.content, 'errno 2') && !contains(steps.pa11y_output.outputs.content, 'UnhandledPromiseRejectionWarning: Error: Page crashed!') }}" 53 | run: | 54 | echo "::error::The site is failing accessibility tests. Please review the comment in the pull request or the pa11y-ci step in the workflow for details." 55 | exit 1 56 | -------------------------------------------------------------------------------- /Rakefile: -------------------------------------------------------------------------------- 1 | desc 'Build the site for testing' 2 | task :test_build do 3 | # https://github.com/jekyll/jekyll/issues/4122#issuecomment-159439360 4 | sh 'jekyll build -d _test_site/' 5 | end 6 | 7 | 8 | require 'html-proofer' 9 | 10 | # Keep in sync with the `ignoreUrls` in `./assets/js/application.js`. 11 | BASE_PROOFER_OPTS = { 12 | url_ignore: [ 13 | %r{https://18f.slack.com}i, 14 | %r{https://anywhere.gsa.gov}i, 15 | %r{https://bookit.gsa.gov}i, 16 | %r{https://calendar.gsa.gov}i, 17 | %r{https://connect.gsa.gov}i, 18 | %r{https://docs.google.com}i, 19 | %r{https://drive.google.com}i, 20 | %r{https://ea.gsa.gov}i, 21 | %r{https://email.gsa.gov}i, 22 | %r{https://eopf.opm.gov}i, 23 | %r{https://gcims.gsa.gov}i, 24 | %r{https://github.com/18F/Accessibility_Reviews}i, 25 | %r{https://github.com/18F/blog-drafts}i, 26 | %r{https://github.com/18F/codereviews}i, 27 | %r{https://github.com/18F/DevOps}i, 28 | %r{https://github.com/18F/Infrastructure}i, 29 | %r{https://github.com/18F/security-incidents}i, 30 | %r{https://github.com/18F/staffing}i, 31 | %r{https://github.com/18F/team-api.18f.gov}i, 32 | %r{https://github.com/18F/writing-lab}i, 33 | %r{https://gkey.gsa.gov}i, 34 | %r{https://gsa-tts.slack.com}i, 35 | %r{https://gsa.my.salesforce.com}i, 36 | %r{https://gsaolu.gsa.gov}i, 37 | %r{https://hrlinks.gsa.gov}i, 38 | %r{https://hrprod.hr.gsa.gov}i, 39 | %r{https://insite.gsa.gov}i, 40 | %r{https://mail.gsa.gov}i, 41 | %r{https://meet.gsa.gov}i, 42 | %r{https://sign.gsa.gov}i, 43 | %r{https://tock.18f.gov}i, 44 | # https://github.com/gjtorikian/html-proofer/issues/118 45 | '#' 46 | ] 47 | } 48 | 49 | desc 'Build and test the site, checking local URLs only' 50 | task ci_test: [:test_build] do 51 | HTMLProofer.check_directory('./_test_site', BASE_PROOFER_OPTS.merge( 52 | disable_external: true 53 | )).run 54 | end 55 | 56 | desc 'Build and test the site, checking all URLs' 57 | task test: [:test_build] do 58 | HTMLProofer.check_directory('./_test_site', BASE_PROOFER_OPTS).run 59 | end 60 | -------------------------------------------------------------------------------- /_pages/architecture-reviews.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Architecture Reviews 3 | sidenav: approach 4 | sticky_sidenav: true 5 | --- 6 | 7 | Maintainable technology projects require handoffs between developers, and with new teammates comes fresh perspectives. Building a [transparent and remote-friendly 8 | workplace](https://18f.gsa.gov/2015/10/15/best-practices-for-distributed-teams/) is a great start to assist in knowledge transfer, as well as keeping projects as simple and obvious as possible and documenting key decisions. 9 | 10 | ## Simplicity 11 | 12 | We’ve done two projects exploring different aspects of simplicity — first, the [DATA Act Pilot: Simplicity is Key]({{site.baseurl}}/architecture-reviews/data-act-pilot) (2016) project explored the ideas of: 13 | 14 | - Building for a least common denominator (CSVs) gave the project reach (more users could participate) and reduced code complexity. 15 | - Pulling out validation rules into a separate, easy-to-modify format made the product flexible and simple to maintain. 16 | 17 | The second explores the idea of simplifying acquisitions in [Micro-purchase: Do one thing well]({{site.baseurl}}/architecture-reviews/micro-purchase) (2016) by using code boundaries in projects to define lines between micro-purchases of developer time. 18 | 19 | ## Documenting key decisions 20 | 21 | Some 18F projects have found success using [Architecture Decision Records](https://adr.github.io/) to capture key decisions and the context to which they were made, with the goal of allowing future project developers to know if a decision should be revisited or not. The decision records are typically stored in the repository alongside the code, using [this template](https://github.com/joelparkerhenderson/architecture-decision-record/blob/main/templates/decision-record-template-by-michael-nygard/index.md). For example: 22 | 23 | - [18F/piipan](https://github.com/18F/piipan/tree/main/docs/adr) 24 | - [HHS/Head-Start-TTADP](https://github.com/HHS/Head-Start-TTADP/tree/main/docs/adr) 25 | - [HHS/TANF-app](https://github.com/HHS/TANF-app/tree/main/docs/Architecture%20Decision%20Record) 26 | - [transcom/mymove](https://github.com/transcom/mymove/tree/master/docs/adr) 27 | -------------------------------------------------------------------------------- /_pages/books-we-have-read.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Books we've read 3 | sidenav: tools 4 | sticky_sidenav: true 5 | --- 6 | 7 | Engineers at TTS like to read! Here are a few books and other publications that folks have mentioned that they have read in #dev and elsewhere. 8 | 9 | - *Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations* by Nicole Forsgren, Jez Humble, and Gene Kim 10 | - *Being Wrong: Adventures in the Margin of Error* by Kathryn Schulz 11 | - *Debugging Teams: Better Productivity through Collaboration* by Brian W. Fitzpatrick 12 | - *The Design of Everyday Things* by Don Norman 13 | - *Domain-Driven Design Distilled* by Vaughn Vernon 14 | - *Domain-Driven Design: Tackling Complexity in the Heart of Software* by Eric Evans 15 | - *The Effective Engineer* by Edmond Lau and Bret Taylor 16 | - *Fifty Quick Ideas to Improve Your User Stories* by Gojko Adzic and David Evans 17 | - *The Field Guide to Understanding 'Human Error'* by Sidney Dekker 18 | - *Inviting Disaster: Lessons From the Edge of Technology* by James R. Chiles 19 | - *It Doesn't Have to Be Crazy at Work* by Jason Fried and David Heinemeier Hansson 20 | - *Kill It with Fire: Manage Aging Computer Systems (and Future Proof Modern Ones)* by Marianne Bellotti 21 | - *The Manager's Path: A Guide for Tech Leaders Navigating Growth and Change* by Camille Fournier 22 | - *Monolith to Microservices: Evolutionary Patterns to Transform Your Monolith* by Sam Newman 23 | - *The Mythical Man-Month: Essays on Software Engineering* by Frederick Brooks Jr. 24 | - *[Prototyping for tiny fingers](http://fpl.cs.depaul.edu/jriely/360/extras/prototyping-for-tiny-fingers.pdf)* by Marc Rettig 25 | - *[Reflections on Trusting Trust ](https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf)* by Ken Thompson 26 | - *[Shape Up: Stop Running in Circles 27 | and Ship Work that Matters](https://basecamp.com/shapeup/webbook)* by Ryan Singer 28 | - *[Site Reliability Engineering: How Google Runs Production Systems](https://sre.google/sre-book/table-of-contents/)* by Betsy Beyer, Chris Jones, Jennifer Petoff, and Niall Richard Murphy 29 | - *Systems Performance: Enterprise and the Cloud* by Brendan Gregg 30 | - *Threat Modeling: Designing for Security* by Adam Shostack 31 | - *[Twenty Ways to Split Stories](https://xp123.com/articles/twenty-ways-to-split-stories/)* by Bill Wake 32 | - *A Whack on the Side of the Head: How You Can Be More Creative* by Roger von Oech 33 | - *[You Reap What You Code](https://ferd.ca/you-reap-what-you-code.html)* by Fred Hebert 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | -------------------------------------------------------------------------------- /_pages/security/output-encoding.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Output Encoding 3 | sidenav: security 4 | sticky_sidenav: true 5 | --- 6 | 7 | ## What is cross site scripting (XSS)? 8 | 9 | Cross site scripting, or XSS, is a form of attack on a web application which involves executing code on a user's browser. Output encoding is a defense against XSS attacks. 10 | 11 | To get a more extensive understanding of XSS, see [excess xss](https://excess-xss.com/). 12 | 13 | ## How to correctly encode output 14 | 15 | Protecting from XSS attacks requires developers to consider how data is being displayed on a page. If the data could come from a user's input in any way (including through the site's URL), then correct encoding of the output has to be considered. 16 | 17 | Since most web applications at TTS are built through JavaScript or backend frameworks, this guide will go over output encoding issues by those frameworks in addition to plain JavaScript. 18 | 19 | ### Vanilla JavaScript 20 | 21 | When writing plain JavaScript, developers have to consider where data is coming from whenever it's being output in the web application. While data that comes from a backend database usually needs output encoding, code also could need output encoding when extracting data from the current page's url (which an attacker could modify and send to a user). 22 | 23 | The [excess xss prevention](https://excess-xss.com/#xss-prevention) section has up-to-date information on how to prevent XSS attacks in all the possible ways they could happen. 24 | 25 | ### React 26 | 27 | By default React DOM escapes all output. This means that output in JSX components will usually be safe, which is great news. This is discussed in [React's documentation](https://reactjs.org/docs/introducing-jsx.html#jsx-prevents-injection-attacks). 28 | 29 | There are some cases where output may not be correctly escaped in React components. Here are some of those cases: 30 | 31 | - Using the `dangerouslySetInnerHTML` prop (it's named this for a reason) 32 | - Passing state from the server, JSON stringifying it without serializing it 33 | - Using user supplied `href` values 34 | - Incorrect usages of `eval` 35 | 36 | This [dailyjs article](https://medium.com/dailyjs/exploiting-script-injection-flaws-in-reactjs-883fb1fe36c1) discusses these various problems and how to avoid them. 37 | 38 | ### Angular 39 | 40 | Angular also does a good job of escaping output by default. In general, earlier versions of Angular 1 had more security vulnerabilities, so a safe bet is to ensure the project is on the most recent version of Angular. To learn more about potential vulnerabilities, the [Angular site](https://docs.angularjs.org/guide/security) provides detailed information. 41 | -------------------------------------------------------------------------------- /assets/images/gsa-logo-blue.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | gsa-logo-blue 4 | 5 | 6 | 7 | 8 | 9 | 10 | -------------------------------------------------------------------------------- /_data/theme.yml: -------------------------------------------------------------------------------- 1 | ## Color configurations 2 | ## 'primary' - Defines content link color, primary button bg color, return to top link color 3 | ## 'primary darker' - Defines content link hover color, primary button hover bg color, return to top link hover color 4 | ## 'base' - Defines header site title, active top nav link color, left nav link color, link color in primary and main footer and text in usa-banner 5 | ## 'visited' - Defines visited link color 6 | # 7 | colors: 8 | primary: '#046B99' 9 | primary-darker: '#1C304A' 10 | # base: 'orange' 11 | # visited: 'red' 12 | # 13 | # # USA banner colors 14 | # usa-banner: 15 | # usa-banner-bg: 'LightBlue' 16 | # usa-banner-text: 'Black' 17 | # usa-banner-link: 'Gray' 18 | # usa-banner-link-hover: 'Gray' 19 | # 20 | # # Header colors 21 | header: 22 | header-bg: 'white' 23 | # header-title: '#1C304A' 24 | # header-link: 'DarkSlateGray' 25 | # header-link-hover: 'Indigo' 26 | # 27 | # # Alt section colors 28 | # alt-section: 29 | # bg-color: '#e6e3d0' 30 | # header-color: 'DarkSlateGray' 31 | # text-color: 'Black' 32 | # link-color: 'red' 33 | # link-hover-color: 'orange' 34 | # 35 | # # Hero colors 36 | # hero: 37 | # hero-bg: 'white' 38 | # hero-header: 'DarkSlateGray' 39 | # hero-header-alt: 'DarkSlateGray' 40 | # hero-text: 'Gray' 41 | # hero-link: 'DodgerBlue' 42 | # hero-button-bg: 'DodgerBlue' 43 | # hero-button-text: 'white' 44 | # 45 | # # Top navigation colors 46 | # top-navigation: 47 | # top-nav-bg: 'DarkSlateGray' 48 | # top-nav-link: 'white' 49 | # top-nav-link-hover: 'black' 50 | # top-nav-link-hover-bg: '#6eb7b7' 51 | # top-nav-link-current: 'black' 52 | # top-nav-dropdown-bg: '#6eb7b7' 53 | # top-nav-dropdown-link: 'orange' 54 | # top-nav-dropdown-link-hover-bg: 'DarkSlateGray' 55 | # 56 | # # Side navigation colors 57 | # side-navigation: 58 | # side-nav-bg: 'white' 59 | # side-nav-link: 'black' 60 | # side-nav-link-hover: 'white' 61 | # side-nav-link-hover-bg: 'DarkSlateGray' 62 | # side-nav-link-current: 'DarkSlateGray' 63 | # 64 | # # Footer colors 65 | # footer: 66 | # primary-footer-bg: 'Gainsboro' 67 | # primary-footer-link: 'black' 68 | # footer-bg: 'DarkSlateGray' 69 | # footer-text: 'white' 70 | # footer-link: 'LightBlue' 71 | # 72 | # 73 | # # Font configurations 74 | # # 'sans' - Defines navigation and body text 75 | # # 'serif' - Defines headings 76 | # # 'normal' - Defines the default font weight 77 | # # 'bold' - Defines the bold font weight 78 | # 79 | fonts: 80 | sans: >- 81 | "Helvetica Neue", "Helvetica", "Arial", sans-serif 82 | serif: >- 83 | "Helvetica Neue", "Helvetica", "Arial", sans-serif 84 | # normal: 300 85 | # bold: 500 86 | -------------------------------------------------------------------------------- /_pages/about.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: About this guide 3 | sidenav: about 4 | sticky_sidenav: true 5 | subnav: 6 | - text: How we classify best practices 7 | href: "#how-we-classify-best-practices" 8 | --- 9 | 10 | [Technology Transformation Services (TTS)](https://www.gsa.gov/about-us/organization/federal-acquisition-service/technology-transformation-services) — which includes [18F](https://18f.gsa.gov/), [Centers of Excellence (CoE)](https://coe.gsa.gov/), [Presidential Innovation Fellows (PIF)](https://presidentialinnovationfellows.gov/), and [TTS Solutions](https://www.gsa.gov/about-us/organization/federal-acquisition-service/technology-transformation-services/tts-solutions) — promote best practices across specialty areas through guilds. 11 | 12 | Getting new practices into the guide is pretty light on process. Feel free to raise a topic in Slack or at a guild meeting and drive to some consensus. Once you've done that, document your findings, submit a PR, and ask in #dev for a quick review. If you think a proposal might be controversial after getting some consensus prior, please post the draft PR to #dev (and elsewhere if you don’t think target audience is in that channel) and solicit feedback. 13 | 14 | ## How we classify best practices 15 | 16 | These documents are structured by topic; under topics we have classified we indicate "Requirement", 17 | "Standard", "Default", "Suggestion", and "Caution". 18 | 19 | If a classification is not present on a topic or a reference to a tool or practice, it should be presumed 20 | to be a {%include components/tag-suggestion.html %} and the decision is left at your discretion. If you are unsure, ask in #dev, as the topic or tool may be a good candidate for classification. 21 | 22 | {%include components/tag-requirement.html %} indicates practices that _must_ be done for 23 | regulatory, legal, compliance, or other reasons. 24 | 25 | {%include components/tag-standard.html %} signifies practices that have a strong consensus across TTS; they 26 | should generally be followed to ease the ATO process and make on-boarding 27 | simpler. 28 | 29 | {%include components/tag-default.html %} practices are safe selections that tend to be used by a large number of our 30 | projects; you may find yourself with a better or more tailored solution, 31 | however. 32 | 33 | {%include components/tag-suggestion.html %} indicates examples that have worked well on a project or two; 34 | they're not widely used enough to be defaults, but are worth considering. 35 | 36 | {%include components/tag-caution.html %} marks approaches that have significant pitfalls or should not be used for 37 | security/compliance reasons. 38 | 39 | If a specific classification is not present on a topic or reference to a tool or practice, it should be presumed 40 | to be a {%include components/tag-suggestion.html %}. 41 | -------------------------------------------------------------------------------- /_data/navigation.yml: -------------------------------------------------------------------------------- 1 | primary: 2 | - text: About this guide 3 | href: /about 4 | name: about 5 | - text: Our approach 6 | href: /workflow/ 7 | name: approach 8 | - text: Tools 9 | href: /integrations/ 10 | name: tools 11 | - text: Languages & Runtimes 12 | href: /language-selection/ 13 | name: languages 14 | - text: Security 15 | href: /security/ 16 | name: security 17 | 18 | about: 19 | - text: About this guide 20 | href: /about 21 | - text: License 22 | href: /license/ 23 | - text: Resources 24 | href: /resources/ 25 | 26 | approach: 27 | - text: Our approach 28 | href: /workflow/ 29 | - text: Feedback 30 | href: /people/ 31 | - text: Code Review 32 | href: /code-review/ 33 | - text: Development Environments 34 | href: /development-environments/ 35 | - text: On-Call 36 | href: /on-call/ 37 | - text: Incident Reports 38 | href: /incident-reports/ 39 | - text: Architecture Reviews 40 | href: /architecture-reviews/ 41 | - text: Front-End Disciplines 42 | href: /frontend/ 43 | - text: APIs 44 | href: /apis/ 45 | - text: Release Strategies 46 | href: /release-strategies/ 47 | - text: Example Workflows 48 | href: /example-workflows/ 49 | 50 | tools: 51 | - text: Tools 52 | href: /integrations/ 53 | - text: Laptop Setup 54 | href: /laptop-setup/ 55 | - text: Project Setup 56 | href: /project-setup/ 57 | - text: Docker for Development 58 | href: /docker/ 59 | - text: Browser Testing 60 | href: /browser-testing/ 61 | - text: Automated Testing 62 | href: /tests/ 63 | - text: Accessibility Scanning 64 | href: /accessibility-scanning/ 65 | - text: Continuous Deployment 66 | href: /continuous-deployment/ 67 | - text: Datastore Selection 68 | href: /datastore-selection/ 69 | - text: Choosing a Web App Architecture 70 | href: /web-architecture/ 71 | - text: SharePoint Primer 72 | href: /sharepoint/ 73 | - text: Books we've read 74 | href: /books-we-have-read/ 75 | 76 | languages: 77 | - text: Languages & Runtimes 78 | href: /language-selection/ 79 | - text: JavaScript 80 | href: /javascript/ 81 | - text: Markdown 82 | href: /markdown/ 83 | - text: Node.js 84 | href: /nodejs/ 85 | - text: Python 86 | href: /python/ 87 | - text: Ruby 88 | href: /ruby/ 89 | - text: CSS 90 | href: /css/ 91 | 92 | # TODO: figure out how to nest links 93 | security: 94 | - text: Security 95 | href: /security/ 96 | - text: Content Security Policy (CSP) 97 | href: /security/content-security-policy/ 98 | - text: Output Encoding 99 | href: /security/output-encoding/ 100 | - text: Vulnerable dependency remediation 101 | href: /security/dependency-remediation/ 102 | - text: Cloud Services 103 | href: /security/cloud-services/ 104 | - text: Incident Response Drills 105 | href: /security/incident-response-drills/ 106 | -------------------------------------------------------------------------------- /_pages/browser-testing.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Browser Testing 3 | sidenav: tools 4 | sticky_sidenav: true 5 | --- 6 | 7 | Regular cross-browser testing helps ensures your web site or application is accessible and functional to a broad audience, especially as Internet Explorer 11 is pervasive in federal government but TTS developers use Chrome on macOS almost exclusively. 8 | 9 | Using a responsive web design and a standards-based approach will result in wide compatibility across modern browsers, but at the edges your product can still trip up on browser-specific quirks. For example, compare the [list of web technologies supported in Internet Explorer 11 versus Chrome](https://caniuse.com/#compare=ie+11,chrome+86). [Progressive Web Application](https://web.dev/progressive-web-apps/)-related features and behaviors also vary widely across devices and browsers. 10 | 11 | A few options are available at TTS that allow testing on multiple browser platforms. An automated testing solution is ideal, but even manual cross-browser testing remains valuable. 12 | 13 | ## Windows 14 | 15 | To test with Windows-native browsers like Internet Explorer or Microsoft Edge: 16 | 17 | - [Citrix Workspace](https://handbook.tts.gsa.gov/vmware-horizon/) 18 | * Pros: 19 | * No extra installation necessary 20 | * Cons: 21 | * You can’t pick the browser versions; the Standard Desktop image includes Internet Explorer 11 and Microsoft Edge (EdgeHTML layout engine) 22 | * You can’t access localhost 23 | * Manual testing only 24 | - Virtual machine (VM) 25 | * [Request access to VirtualBox](https://gsa.servicenowservices.com/sp/?id=sc_cat_item&sys_id=1bfdfdca78d3a400ce3ddff91a64940b). 26 | * Pros: 27 | * [VMs are available for IE8+ at no cost from Microsoft](https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/) 28 | * You can [access localhost](https://medium.com/@urubuz/accessing-localhost-in-mac-from-windows-vm-in-virtualbox-312a3de6fedb) 29 | * Can support local, automated testing through [Selenium](https://www.selenium.dev/) 30 | * Cons: 31 | * Takes a bit of setup 32 | 33 | ## Android 34 | 35 | For Google Chrome for Android: 36 | 37 | - [Android Studio](https://developer.android.com/studio) with [AVD Manager](https://developer.android.com/studio/run/managing-avds) 38 | * The GSA IT Helpdesk will need to configure policy on your macOS workstation in order for the Android emulator to run. 39 | 40 | ## iOS 41 | 42 | For Safari on iOS: 43 | 44 | - [Xcode](https://developer.apple.com/xcode/) with Simulator 45 | * If you are using the latest version of macOS currently released by Apple, Xcode is available in the Mac App Store. Otherwise, find the version of Xcode that maps to your macOS version on [Apple Developer](https://developer.apple.com/download/). 46 | 47 | ## Chrome-family browsers and Firefox 48 | 49 | - [Cypress](https://www.cypress.io/features) 50 | * The command-line test runner and local GUI is [approved for use](https://handbook.tts.gsa.gov/software/search/#cypress). However, the [Cypress Dashboard SaaS](https://www.cypress.io/dashboard/) is not. 51 | -------------------------------------------------------------------------------- /assets/js/private-eye.js: -------------------------------------------------------------------------------- 1 | // https://github.com/18F/private-eye version 2.0.0 2 | (function() { 3 | 'use strict'; 4 | 5 | var STYLES = 'a.private-link::after { content: "\\1F512"; font-size: 0.75em; vertical-align: top; }'; 6 | var STYLES_ID = '_privateEye-styles'; 7 | 8 | var DEFAULT_OPTIONS = { 9 | defaultMessage: 'This is a link to a private site, which may or may not be accessible to you.', 10 | wrapper: '' 11 | }; 12 | 13 | var isString = function(str) { return !!str && typeof str === 'string'; }; 14 | var isArray = function(arr) { return !!arr && arr.length; }; 15 | 16 | var optionValidators = { 17 | defaultMessage: isString, 18 | wrapper: isString, 19 | ignoreUrls: isArray, 20 | }; 21 | 22 | function setStyles() { 23 | var styles = document.createElement('style'); 24 | styles.innerHTML = STYLES; 25 | styles.id = STYLES_ID; 26 | document.body.appendChild(styles); 27 | } 28 | 29 | function getOptions(opts) { 30 | var newObj = {}; 31 | 32 | for (var prop in DEFAULT_OPTIONS) { 33 | newObj[prop] = DEFAULT_OPTIONS[prop]; 34 | } 35 | 36 | for (var prop in opts) { 37 | var val = opts[prop]; 38 | 39 | if (optionValidators[prop](val)) { 40 | newObj[prop] = val; 41 | } 42 | } 43 | 44 | return newObj; 45 | } 46 | 47 | var PrivateEye = function(opts) { 48 | // The old docs recommend calling this as a function. This is here to detect 49 | // those cases and make sure backward compatibility stays intact now that the 50 | // new syntax is preferred. 51 | if (!(this instanceof PrivateEye)) { 52 | return new PrivateEye(opts); 53 | } 54 | 55 | // Don't add the styles to the page more than once. 56 | if (!document.getElementById(STYLES_ID)) { 57 | setStyles(); 58 | } 59 | 60 | this.opts = getOptions(opts); 61 | 62 | this.checkLinks(); 63 | }; 64 | 65 | PrivateEye.prototype.checkLinks = function() { 66 | var self = this; 67 | 68 | this.opts.ignoreUrls.forEach(function(url) { 69 | var hrefValue; 70 | var titleValue; 71 | 72 | // If the `url` is an Object, then parse the properties `message` & `url` 73 | if (url === Object(url)) { 74 | titleValue = url.message; 75 | hrefValue = url.url; 76 | } else { 77 | hrefValue = url; 78 | titleValue = self.opts.defaultMessage; 79 | } 80 | 81 | var wrapper = self.opts.wrapper.length ? self.opts.wrapper + ' ' : ''; 82 | var selector = wrapper + 'a'; 83 | var anchors = document.querySelectorAll(selector); 84 | 85 | Array.prototype.forEach.call(anchors, function(anchor) { 86 | var anchorHref = anchor.href.toLowerCase().trim(); 87 | 88 | if (anchorHref.indexOf(hrefValue.toLowerCase()) !== -1) { 89 | anchor.className += ' private-link'; 90 | 91 | // Only replace the anchor's title if it is empty 92 | if (!anchor.title) { 93 | anchor.title = titleValue; 94 | } 95 | } 96 | }); 97 | }); 98 | } 99 | 100 | if (typeof module === 'object' && typeof module.exports === 'object') { 101 | module.exports = PrivateEye; 102 | } else { 103 | window.PrivateEye = PrivateEye; 104 | } 105 | })(); 106 | -------------------------------------------------------------------------------- /_pages/security/dependency-remediation.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Remediating vulnerable dependencies 3 | sidenav: security 4 | sticky_sidenav: true 5 | --- 6 | 7 | Your application should have dependency scanning to ensure that the 8 | libraries your code relies on do not have vulnerabilities within them. 9 | For more on how to set up vulnerability scanning see the 10 | [Before You Ship 11 | guide](https://before-you-ship.18f.gov/security/static-analysis/#dependency-analysis)’s 12 | suggestions. 13 | 14 | In operating a system with a dependency scan you’ll find that 15 | vulnerabilities do pop up in your dependencies, and this is a guide on 16 | how to remediate those vulnerabilities to keep your application secure. 17 | 18 | The following suggestions are in the order in which you would try these 19 | strategies. 20 | 21 | ## Apply a patch 22 | 23 | Oftentimes when your dependency scanner finds a vulnerability, it will 24 | provide a suggested patch to remediate the vulnerability. If the scanner 25 | doesn’t provide a patch a quick search of the package’s documentation 26 | may also reveal a patch that you can use. In this case, create a new 27 | branch via git version control and ensure that your tests pass; that the 28 | application still runs as expected; and finally, run your code with your 29 | continuous integration suite. If there are unexpected behaviors or 30 | failing tests, you may have to refactor your code to incorporate the 31 | patch. Once you’ve applied the patch and refactored it, you can submit a 32 | pull request to fix your application. 33 | 34 | ## Use selective resolutions 35 | 36 | In some cases, your dependency scanner will not be able to provide an 37 | immediate patch but it will indicate a package version in which the 38 | vulnerability is fixed. In some package managers such as \`yarn\` you 39 | can pin your dependency to the fixed version by using a 40 | “[selective 41 | resolution](https://classic.yarnpkg.com/en/docs/selective-version-resolutions/)”. 42 | This will bump up versions that are children of top level dependencies. 43 | If you are using \`npm\` you can install 44 | \`[https://www.npmjs.com/package/npm-force-resolutions](https://www.npmjs.com/package/npm-force-resolutions)\` 45 | to draw the same behavior. 46 | 47 | ## Check if it is a false positive 48 | 49 | If there is no existing patch or version update that will remediate the 50 | vulnerabilities, you may want to investigate the offending code and see 51 | if your use of that library would even trigger that part of the code. If 52 | it does not, the vulnerability may be a false positive. If you can 53 | confirm this, you should document this — ideally in your dependency 54 | scanners configuration (ex. .snyk) or somewhere within your code 55 | repository. 56 | 57 | ## Pull upstream 58 | 59 | In cases where the maintainer of a package has not yet resolved the 60 | security vulnerability and you are able to patch it yourself, fork the 61 | original package to fix it locally and then create a pull request in the 62 | package repository. Not only does this help fix the security concern, 63 | but also promotes good open-source culture. Because this requires the 64 | maintainer of the package to review and accept your pull request, this 65 | strategy may take longer to complete. If your pull request is accepted, 66 | be sure to update your dependency file back to the main distribution of 67 | the dependency. 68 | 69 | ## Accept the risk 70 | 71 | If the risk is low, take a look at where the vulnerability resides in 72 | your dependency tree. If the vulnerability is associated with a 73 | development dependency rather than with public code, the risk associated 74 | with the vulnerability might be acceptable (in the short term). 75 | -------------------------------------------------------------------------------- /_pages/incident-reports.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Incident Reports 3 | sidenav: approach 4 | sticky_sidenav: true 5 | redirect_from: 6 | /incident-reports/cloud-gov/ 7 | --- 8 | 9 | Though we fully expect to write dependable applications, every project will 10 | experience service disruptions and other significant failings. In all cases, 11 | we want to learn from our mistakes both within our projects and more broadly. 12 | Incident reports, which detail the events and how they were resolved, are an 13 | excellent mechanism for sharing this information. 14 | 15 | Note that this document won't discuss what to do during a security incident, 16 | if cloud.gov is having issues, what to report to your client, etc. For those, 17 | see: 18 | * https://github.com/18F/security-incidents 19 | * https://cloudgov.statuspage.io/ 20 | * cloud-gov-support@gsa.gov 21 | * Slack: #infrastructure, #incident-response, #cloud-gov-support 22 | 23 | ## Key components 24 | At the high level, we want to follow Mark Imbriaco's [formula for writing a great post-mortem report](https://www.digitalocean.com/blog/inside-digitalocean-mark-imbriaco/#:~:text=Departing%20From%20GitHub): 25 | 1. Apologize for what happened. 26 | 1. Demonstrate you understand what happened. 27 | 1. Explain what you will do to reduce the likelihood of it happening again. 28 | 29 | Before any deep analysis we should write a **timeline**, beginning at the time 30 | the incident was discovered and ending at the point the incident was declared 31 | over. Events in this timeline include deploys, configuration changes, key 32 | moments of discovery, client communications, and anything else that'd be 33 | relevant to understanding the incident. If further analysis discovers that 34 | certain events caused the incident, those events should also be added 35 | 36 | Analyze the **factors** that contributed to the incident. Here it's important 37 | to emphasize the [Retrospective Prime 38 | Directive](https://retrospectivewiki.org/index.php?title=The_Prime_Directive); 39 | paraphrased: everyone did their best; there should be no judgment of 40 | individuals. If lucky, we will discover a single **root cause**, but often we 41 | will find a sort-of comedy of errors or serious of unfortunate events that 42 | collectively led to the incident. 43 | 44 | Propose, discuss, and prioritize **preventative measures**. This is the key 45 | outcome for the project team: we want to avoid these types of problems in the 46 | future. 47 | 48 | Define a single place to put these artifacts and be consistent. It doesn't 49 | matter if it's GitHub issues, Google Docs, a wiki, etc. so long as it's kept 50 | together and easy to reference by both the team and interested stakeholders. 51 | Don't make folks search for the information. 52 | 53 | ## Examples 54 | * [C2](./C2/c2-outage-report-2016-08-10.pdf) 55 | 56 | ## Additional resources 57 | * John Allspaw's [introduction](https://codeascraft.com/2012/05/22/blameless-postmortems/) 58 | * This [great * presentation](http://www.slideshare.net/danmil30/how-to-run-a-postmortem-with-humans-not-robots-velocity-2013) by Dan Milstein 59 | * Mark Imbriaco's [formula for writing a great post-mortem report](https://www.digitalocean.com/blog/inside-digitalocean-mark-imbriaco/#:~:text=Departing%20From%20GitHub), 60 | and [an example with some commentary](https://medium.com/@faruque/post-mortem-communication-789f396c7dd6#.t1u4ziduf) 61 | * @peculiaire's [template for retrospective meetings](https://github.com/peculiaire/incident-lifecycle/blob/master/retrotemplate.md) 62 | (adapted from what Heroku uses) 63 | * Allspaw's article on [Infinite How's](https://www.oreilly.com/ideas/the-infinite-hows) 64 | - an alternative to Five Whys (that arguably works better) 65 | * @danluu's [collection of post-mortems](https://github.com/danluu/post-mortems) 66 | -------------------------------------------------------------------------------- /doc/bpdr/0003-set-criteria-for-static-site-framework.md: -------------------------------------------------------------------------------- 1 | # 3. Set Criteria for Recommended Static Site Generator 2 | 3 | ## Status 4 | Accepted 5 | 6 | ## Context 7 | TTS needs to build a lot of server-rendered websites, both for its own purposes and for other government agencies. 8 | 9 | Jekyll has been the popular go-to framework for this work, for a few reasons: 10 | - Many 18F employees were Rubyists 11 | - For a while, it had strong community and institutional support 12 | - Github uses it as the engine for Github Pages 13 | - Federalist (now Cloud.gov Pages) supports it and USWDS built multiple themes for it, allowing teams to stand up a new government website very quickly 14 | - Markdown pages were relatively easier to edit by a wider variety of TTS employees than other frameworks 15 | 16 | In mid-2021 some core maintainers said Jekyll was [in hiatus](https://www.theregister.com/2021/09/14/future_of_jekyll_project_engine/) and recommended other frameworks to switch to. This hasn't been stated on the official website, and the [Jekyll Github repo](https://github.com/jekyll/jekyll) remains active, so Jekyll's future seems unclear. 17 | 18 | In addition, a number of TTS employees are unsatisfied with Jekyll because: 19 | - build times have become slower relative to other frameworks 20 | - It’s difficult to support asset rendering (including building USWDS, a common component of our sites) in a way that supports far-futures cache headers 21 | - Building locally in development is difficult to impossible without admin rights, hindering handoff to folks with less computer privileges 22 | 23 | For these reasons, we want teams to avoid standing up new Jekyll sites, but we don't yet have consensus on what the alternative should be. 24 | 25 | While we don't know what the single alternative to Jekyll should be, we agree that TTS should recommend one alternative over multiple, since a single alternative will be easier for TTS staff to support. 26 | 27 | In order to get consensus on the alternative, we should first agree on a criteria for what we need and want in a static site generator. 28 | 29 | ## Decision 30 | 31 | Any static site generator used by TTS needs to: 32 | - be written in a language that many TTS'ers can support 33 | - have sufficient open-source community support and longevity 34 | - be supported by Cloud.gov Pages 35 | - be supported by USWDS 36 | - allow containerization through Docker 37 | - be able to be used for both prototyping and production 38 | - have build times that don't impede development 39 | - not make a11y hard 40 | - have multilingual support 41 | - enable people without specialized engineering skills to edit static content 42 | 43 | It would be nice to have: 44 | - built-in support for common compiles-to-web languages, like SASS, ESNext, and TypeScript 45 | - the ability to reuse/share layouts or templates across sites 46 | 47 | Current candidates (in no particular order): 48 | - [11ty](https://www.11ty.dev/) 49 | - [Hugo](https://gohugo.io/) 50 | - [Next](https://nextjs.org/) 51 | - [Gatsby](https://www.gatsbyjs.com/) 52 | - [Bridgetown](https://www.bridgetownrb.com/) 53 | - Build our own 54 | - [Middleman](https://middlemanapp.com/) 55 | 56 | ## Consequences 57 | 58 | ### Risks of staying with Jekyll 59 | - Degredation of framework and dependencies as community support wanes 60 | - If Jekyll's period of support officially ends, we will be forced to migrate at that point, or risk running deprecated software 61 | 62 | ## Risks of moving from Jekyll 63 | - We will still need to maintain our many Jekyll apps, for a time 64 | - We will need to decide whether these apps should be deprecated or moved to a new platform 65 | - Moving to a new platform will take a lot of work and support 66 | 67 | ## Next Steps 68 | The Guild will make a checklist based on these criteria, share it out with teams using these alternatives, and decide what to do next based on the feedback we get. 69 | -------------------------------------------------------------------------------- /Gemfile.lock: -------------------------------------------------------------------------------- 1 | GEM 2 | remote: https://rubygems.org/ 3 | specs: 4 | activesupport (7.0.7.2) 5 | concurrent-ruby (~> 1.0, >= 1.0.2) 6 | i18n (>= 1.6, < 2) 7 | minitest (>= 5.1) 8 | tzinfo (~> 2.0) 9 | addressable (2.8.0) 10 | public_suffix (>= 2.0.2, < 5.0) 11 | autoprefixer-rails (9.8.6.5) 12 | execjs 13 | colorator (1.1.0) 14 | concurrent-ruby (1.2.2) 15 | em-websocket (0.5.2) 16 | eventmachine (>= 0.12.9) 17 | http_parser.rb (~> 0.6.0) 18 | ethon (0.12.0) 19 | ffi (>= 1.3.0) 20 | eventmachine (1.2.7) 21 | execjs (2.7.0) 22 | ffi (1.14.2) 23 | forwardable-extended (2.6.0) 24 | gemoji (3.0.1) 25 | html-pipeline (2.14.2) 26 | activesupport (>= 2) 27 | nokogiri (>= 1.4) 28 | html-proofer (3.18.8) 29 | addressable (~> 2.3) 30 | mercenary (~> 0.3) 31 | nokogumbo (~> 2.0) 32 | parallel (~> 1.3) 33 | rainbow (~> 3.0) 34 | typhoeus (~> 1.3) 35 | yell (~> 2.0) 36 | http_parser.rb (0.6.0) 37 | i18n (1.14.1) 38 | concurrent-ruby (~> 1.0) 39 | jekyll (4.2.0) 40 | addressable (~> 2.4) 41 | colorator (~> 1.0) 42 | em-websocket (~> 0.5) 43 | i18n (~> 1.0) 44 | jekyll-sass-converter (~> 2.0) 45 | jekyll-watch (~> 2.0) 46 | kramdown (~> 2.3) 47 | kramdown-parser-gfm (~> 1.0) 48 | liquid (~> 4.0) 49 | mercenary (~> 0.4.0) 50 | pathutil (~> 0.9) 51 | rouge (~> 3.0) 52 | safe_yaml (~> 1.0) 53 | terminal-table (~> 2.0) 54 | jekyll-autoprefixer (1.0.2) 55 | autoprefixer-rails (~> 9.3) 56 | jekyll-last-modified-at (1.3.0) 57 | jekyll (>= 3.7, < 5.0) 58 | posix-spawn (~> 0.3.9) 59 | jekyll-redirect-from (0.16.0) 60 | jekyll (>= 3.3, < 5.0) 61 | jekyll-sass-converter (2.1.0) 62 | sassc (> 2.0.1, < 3.0) 63 | jekyll-sitemap (1.4.0) 64 | jekyll (>= 3.7, < 5.0) 65 | jekyll-watch (2.2.1) 66 | listen (~> 3.0) 67 | jemoji (0.12.0) 68 | gemoji (~> 3.0) 69 | html-pipeline (~> 2.2) 70 | jekyll (>= 3.0, < 5.0) 71 | kramdown (2.3.1) 72 | rexml 73 | kramdown-parser-gfm (1.1.0) 74 | kramdown (~> 2.0) 75 | libv8 (8.4.255.0) 76 | liquid (4.0.3) 77 | listen (3.4.1) 78 | rb-fsevent (~> 0.10, >= 0.10.3) 79 | rb-inotify (~> 0.9, >= 0.9.10) 80 | mercenary (0.4.0) 81 | mini_portile2 (2.8.1) 82 | mini_racer (0.3.1) 83 | libv8 (~> 8.4.255) 84 | minitest (5.19.0) 85 | nokogiri (1.14.3) 86 | mini_portile2 (~> 2.8.0) 87 | racc (~> 1.4) 88 | nokogumbo (2.0.4) 89 | nokogiri (~> 1.8, >= 1.8.4) 90 | parallel (1.20.1) 91 | pathutil (0.16.2) 92 | forwardable-extended (~> 2.6) 93 | posix-spawn (0.3.15) 94 | public_suffix (4.0.6) 95 | racc (1.6.2) 96 | rainbow (3.0.0) 97 | rake (13.0.3) 98 | rb-fsevent (0.10.4) 99 | rb-inotify (0.10.1) 100 | ffi (~> 1.0) 101 | rexml (3.2.5) 102 | rouge (3.26.0) 103 | safe_yaml (1.0.5) 104 | sassc (2.4.0) 105 | ffi (~> 1.9) 106 | terminal-table (2.0.0) 107 | unicode-display_width (~> 1.1, >= 1.1.1) 108 | typhoeus (1.4.0) 109 | ethon (>= 0.9.0) 110 | tzinfo (2.0.6) 111 | concurrent-ruby (~> 1.0) 112 | unicode-display_width (1.7.0) 113 | uswds-jekyll (5.3.0) 114 | jekyll (>= 4.0, < 5) 115 | jekyll-autoprefixer 116 | mini_racer 117 | yell (2.2.2) 118 | 119 | PLATFORMS 120 | ruby 121 | 122 | DEPENDENCIES 123 | html-proofer 124 | jekyll 125 | jekyll-last-modified-at 126 | jekyll-redirect-from 127 | jekyll-sitemap 128 | jemoji (>= 0.12.0) 129 | mini_racer 130 | rake 131 | uswds-jekyll (~> 5.3) 132 | 133 | RUBY VERSION 134 | ruby 2.7.2p137 135 | 136 | BUNDLED WITH 137 | 2.1.4 138 | -------------------------------------------------------------------------------- /_pages/workflow.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Our approach 3 | sidenav: approach 4 | sticky_sidenav: true 5 | --- 6 | 7 | Project teams may vary, but across TTS engineering we aim for consistency 8 | around deployments, git etiquette, and similar workflow conventions. 9 | 10 | ## Continuous Integration & Deployment 11 | 12 | ### All Projects {%include components/tag-standard.html %} 13 | 14 | - Ensure that your project is running **automated tests** in CI. Successful 15 | test completion should be a requirement for deployment. 16 | - Generally, **CI should perform deployments**. This ensures the deployments 17 | are repeatable and don’t rely on individual development environments. See 18 | our [documentation on continuous deployment]({{site.baseurl}}/continuous-deployment) for 19 | details on how to set this up. 20 | - Deployments should be **zero-downtime**, achievable through tools like 21 | [Cloud Foundry's rolling deployment process](https://docs.cloudfoundry.org/devguide/deploy-apps/rolling-deploy.html). 22 | - In addition to deployments after code change, we generally need to 23 | (automatically) **re-deploy daily** to ensure the running containers haven’t 24 | been tampered with (an ATO compliance requirement). See CircleCI’s 25 | [“schedule” 26 | docs](https://circleci.com/docs/2.0/configuration-reference/#schedule) for 27 | details. 28 | 29 | ## Git & GitHub {%include components/tag-standard.html %} 30 | 31 | Git is our version control system of choice and 32 | GitHub is our current repository platform, but how to use these tools can be spelled out 33 | in a bit more detail. Note that we are looking to consolidate this with our existing 34 | documentation on [code review]({{site.baseurl}}/code-review) and [example 35 | workflows]({{site.baseurl}}/example-workflows). 36 | 37 | ### Security {%include components/tag-requirement.html %} 38 | 39 | _For best practices on protecting sensitive information, check out the [TTS 40 | Handbook](https://handbook.tts.gsa.gov/sensitive-information/#protecting-tts-systems)._ 41 | 42 | Enable 43 | [**two-factor authentication**](https://help.github.com/articles/about-two-factor-authentication/) for 44 | your GitHub account. This is required for all TTS employees. 45 | 46 | As part of the ATO process, we require any branches which 47 | trigger automated deployment be [**protected**](https://help.github.com/articles/about-protected-branches/) 48 | by passing CI and peer review. 49 | 50 | ### Other considerations 51 | 52 | #### {%include components/tag-standard.html %} 53 | * Default to **public** for new repositories. See our 54 | [guidelines](https://github.com/18F/open-source-policy/blob/master/practice.md) 55 | about open source for more detail. 56 | * Enforce reviewer approval of pull requests against the main branch. 57 | * We prefer **branches** over forks to ease internal collaboration. *If your project has many outside contributors, consider forks instead.* 58 | * Keep your repository **clean**; delete merged branches and avoid committing 59 | files specific to your dev environment (e.g. `.DS_Store`). 60 | * Consider [**signing commits** with a GPG 61 | key](https://help.github.com/articles/signing-commits-with-gpg/) 62 | 63 | 64 | #### {%include components/tag-suggestion.html %} 65 | * When in doubt, use feature branches and [**gitflow**](http://nvie.com/posts/a-successful-git-branching-model/) as your branch naming scheme. 66 | * Follow [this 67 | guidance](http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html) about **good commit messages**. 68 | * Consider using [Allstar](https://github.com/ossf/allstar) to enforce consistent branch protection rules or to require commit signing on all repositories in your organization. For example, see the [Allstar configuration](https://github.com/cloud-gov/.allstar) that the cloud.gov team is using. 69 | 70 | ## Code style 71 | 72 | {%include components/tag-standard.html %} Use an opinionated automated code formatter whenever possible. This saves teams from wasting time arguing about code style, and makes it easy to comply. Specific suggestions in [the pages for each language]({{site.baseurl}}/language-selection/). 73 | -------------------------------------------------------------------------------- /_pages/integrations.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Tools 3 | sidenav: tools 4 | sticky_sidenav: true 5 | --- 6 | 7 | Our software is built on the shoulders of giants. We use several third parties 8 | to perform continuous monitoring, host our apps, etc. 9 | 10 | ## Standards 11 | 12 | While we don’t formally recommend third-party services, a handful have been 13 | [approved](https://handbook.tts.gsa.gov/software/#get-access-to-software-we-already-have), procured, and provide services which 14 | are essential for attaining an [Authority-to-Operate (ATO)](https://atos.open-control.org). To get started quickly 15 | and to ease the ATO burden, use these systems; pain awaits your project 16 | otherwise. 17 | 18 | | Purpose | Tool | 19 | | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | 20 | | Back end Performance Monitoring | [New Relic APM](https://newrelic.com/products/application-monitoring) | 21 | | Continuous Integration | [CircleCI](https://circleci.com) and/or [GitHub Actions](https://github.com/features/actions) | 22 | | Dependency Analysis | [Snyk](https://snyk.io) and/or [GitHub](https://docs.github.com/en/github/managing-security-vulnerabilities/managing-vulnerabilities-in-your-projects-dependencies) | 23 | | Front end Performance Monitoring | [Google Analytics](https://developers.google.com/analytics/devguides/collection/analyticsjs/user-timings) (via [DAP][dap])
Note: New Relic is not approved. | 24 | | Infrastructure as a Service (IaaS) | [_See Before You Ship_](https://before-you-ship.18f.gov/infrastructure/#infrastructure-as-a-service-iaas) | 25 | | Platform as a Service (PaaS) | [cloud.gov](https://cloud.gov) | 26 | | Source Control | [GitHub](https://handbook.tts.gsa.gov/github/) | 27 | | Static Site Hosting | [Federalist](https://before-you-ship.18f.gov/infrastructure/federalist/) | 28 | | Uptime Monitoring | [New Relic Synthetics](https://newrelic.com/products/synthetics) | 29 | | User Analytics | [Digital Analytics Program (DAP)][dap] | 30 | 31 | [dap]: https://digital.gov/guides/dap/ 32 | 33 | ## Defaults 34 | 35 | We also track a second set of services that aren’t as essential as the above, 36 | but which we provide for consistency and shared knowledge across projects. 37 | These are good defaults should their need arise; you should generally think 38 | twice before building these tools yourself. 39 | 40 | | Purpose | Tool | 41 | | ---------------------- | -------------------------------------------------------- | 42 | | API Analytics | [api.data.gov](https://api.data.gov/about/) | 43 | | API Throttling | [api.data.gov](https://api.data.gov/about/) | 44 | | Site Search | [search.gov](https://search.gov/) | 45 | | Static Code Analysis | [Code Climate Quality](https://codeclimate.com/quality/) | 46 | | Test Coverage Tracking | [Code Climate Quality](https://codeclimate.com/quality/) | 47 | -------------------------------------------------------------------------------- /_pages/development-environments.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Development Environments 3 | sidenav: approach 4 | sticky_sidenav: true 5 | --- 6 | 7 | Development environments should be designed so as as to be _Easy to set up_, _Well-documented_, and _Reproducible_. 8 | 9 | These principles help keep your colleagues happy, save you time from supporting developer ramp-up, make your repo much more likely to receive open source contributions, and will make it easier to find and fix bugs. 10 | 11 | We elaborate on each principle below. In a related guide section, we show how [Docker is a good supporting tool](../docker/) to achieve these aims. 12 | 13 | ## Principles 14 | 15 | **Easy to set up** 16 | 17 | Dependencies should be easy to install, and a new developer should be able to clone the application and run its "hello world" equivalent with just a few commands. Seed data creation, database schema migrations, and configuration should be automated. Configuration that requires the developer to obtain keys from an external source (such as signing up for an API) should be kept to a minimum. If possible, try to use a mocked version of any external services by default. Many of the good practices for testing (e.g. don't rely on external services) can and should be applied to the development environment. 18 | 19 | **Well-documented** 20 | 21 | At minimum, there should be a README.md file describing what the software does, how to run its "hello world" equivalent, how to run tests, and all external dependencies (like APIs it talks to). If the software is easy to set up, the documentation need not be very long, which is easier to maintain and to keep accurate. 22 | 23 | Moreover, while it's great to use code comments or other documentation tools, often the best documentation is the code itself--that is, if the code is easy to comprehend and contextualize, there might not be a pressing need for extraneous explanation of each and every function. 24 | 25 | Make your continuous integration configuration as concise and readable as possible. 26 | 27 | Instead of: 28 | 29 | ```yml 30 | before_install: 31 | - . $HOME/.nvm/nvm.sh 32 | - nvm install stable 33 | - nvm use stable 34 | - npm install 35 | # Install PhantomJS 2.1.1 manually 36 | - "export PHANTOMJS_VERSION=2.1.1" 37 | - "phantomjs --version" 38 | - "export PATH=$PWD/travis_phantomjs/phantomjs-$PHANTOMJS_VERSION-linux-x86_64/bin:$PATH" 39 | - "phantomjs --version" 40 | - "if [ $(phantomjs --version) != '$PHANTOMJS_VERSION' ]; then rm -rf $PWD/travis_phantomjs; mkdir -p $PWD/travis_phantomjs; fi" 41 | - "if [ $(phantomjs --version) != '$PHANTOMJS_VERSION' ]; then wget https://github.com/Medium/phantomjs/releases/download/v$PHANTOMJS_VERSION/phantomjs-$PHANTOMJS_VERSION-linux-x86_64.tar.bz2 -O $PWD/travis_phantomjs/phantomjs-$PHANTOMJS_VERSION-linux-x86_64.tar.bz2; fi" 42 | - "if [ $(phantomjs --version) != '$PHANTOMJS_VERSION' ]; then tar -xvf $PWD/travis_phantomjs/phantomjs-$PHANTOMJS_VERSION-linux-x86_64.tar.bz2 -C $PWD/travis_phantomjs; fi" 43 | - "phantomjs --version" 44 | before_script: 45 | - cp config/application.yml.example config/application.yml 46 | - cp certs/saml.crt.example certs/saml.crt 47 | - cp keys/saml.key.enc.example keys/saml.key.enc 48 | - bin/rake db:setup --trace 49 | - bin/rake assets:precompile 50 | ``` 51 | 52 | Refactor both blocks into shell scripts: 53 | 54 | ```yml 55 | before_install: 56 | - ./script/install_dependencies.sh 57 | - ./script/install_phantomjs.sh 58 | 59 | before_script: 60 | - ./script/setup_config.sh 61 | - bin/rake db:setup --trace 62 | - bin/rake assets:precompile 63 | ``` 64 | 65 | Now each set of scripts are much easier run in other environments: locally, in a different CI environment, etc. Moreover, think of the YAML keys as annotations for the scripts. 66 | 67 | **Reproducible** 68 | 69 | A good development environment should be reproducible across different computers, platforms, and environments. Reproducibility helps ensure that bugs are not idiosyncratic to any one person's bespoke computing environment--rather they are intrinsic to the repository itself such that time can be spent debugging the repository code and not the environment on a person's computer. Pinning dependencies such as language runtimes and databases to specific versions is a great way to help achieve this. 70 | -------------------------------------------------------------------------------- /_pages/laptop-setup.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Laptop Setup 3 | sidenav: tools 4 | sticky_sidenav: true 5 | --- 6 | 7 | While you are welcome to customize your laptop, here are some tools that have worked for others on the engineering team! 8 | 9 | * [ChromeDriver] for headless website testing 10 | * [chruby] for managing [Ruby] versions. 11 | * [Cloud Foundry CLI] for command line access to 18F's Cloud Foundry-based application platform 12 | * [Caulking] checks for many common types of API tokens and other [sensitive information](https://handbook.tts.gsa.gov/sensitive-information/) before you commit 13 | * [GitHub Desktop] for setting up your SSH keys automatically 14 | * [Homebrew] for managing operating system libraries 15 | * [Homebrew Cask] for quickly installing Mac apps from the command line 16 | * [Homebrew Services] so you can easily stop, start, and restart services 17 | * [gh] (official) or [hub] (unofficial) GitHub-maintained CLI tools for interacting with the GitHub API 18 | * [nvm] for managing Node.js versions if you do not have [Node.js] already installed (Includes latest [Node.js] and [NPM], for running apps and installing JavaScript packages) 19 | * [pyenv] for managing Python versions if you do not have [Python] already installed (includes the latest 3.x [Python]) 20 | * [ruby-install] for installing different versions of Ruby 21 | * [Slack] for communicating with your team 22 | * [The Silver Searcher] for finding things in files 23 | * [Virtualenv] for creating isolated Python environments (via [pyenv] if it is installed) 24 | * [Virtualenvwrapper] for extending Virtualenv (via [pyenv] if it is installed) 25 | * [Zsh] as your shell 26 | 27 | [ChromeDriver]: http://chromedriver.chromium.org/ 28 | [chruby]: https://github.com/postmodern/chruby 29 | [Cloud Foundry CLI]: https://github.com/cloudfoundry/cli 30 | [Caulking]: https://github.com/cloud-gov/caulking 31 | [Github Desktop]: https://desktop.github.com/ 32 | [Homebrew]: http://brew.sh/ 33 | [Homebrew Cask]: https://github.com/Homebrew/homebrew-cask 34 | [Homebrew Services]: https://github.com/Homebrew/homebrew-services 35 | [gh]: https://cli.github.com/ 36 | [hub]: https://github.com/github/hub 37 | [Node.js]: http://nodejs.org/ 38 | [NPM]: https://www.npmjs.org/ 39 | [nvm]: https://github.com/nvm-sh/nvm 40 | [pyenv]: https://github.com/pyenv/pyenv 41 | [Python]: https://www.python.org/ 42 | [Ruby]: https://www.ruby-lang.org/en/ 43 | [ruby-install]: https://github.com/postmodern/ruby-install 44 | [Slack]: https://slack.com/ 45 | [The Silver Searcher]: https://github.com/ggreer/the_silver_searcher 46 | [Virtualenv]: https://virtualenv.pypa.io/en/latest/ 47 | [Virtualenvwrapper]: http://virtualenvwrapper.readthedocs.org/en/latest/# 48 | [Zsh]: http://www.zsh.org/ 49 | 50 | ## Licensed Software 51 | 52 | * [Docker] for all your containerization needs 53 | 54 | The docker daemon and docker CLI tools are available on a free license. 55 | 56 | [Docker Desktop] requires a paid license. Ask in [#admins-docker](https://gsa-tts.slack.com/app_redirect?channel=admins-docker) in Slack to get a license. 57 | 58 | [Docker]: https://www.docker.com/ 59 | [Docker Desktop]: https://www.docker.com/products/docker-desktop/ 60 | 61 | ## Other customizations 62 | 63 | * [Atom] - GitHub's open source text editor 64 | * [Exuberant Ctags] for indexing files for vim tab completion 65 | * [Firefox] for testing your website on a browser other than Chrome 66 | * [iTerm2] - an awesome replacement for the OS X Terminal 67 | * [reattach-to-user-namespace] to allow copy and paste from Tmux 68 | * [Spectacle] - automatic window manipulation 69 | * [Sublime Text 3] for coding all the things 70 | * [Tmux] for saving project state and switching between projects 71 | * [Vim] for those who prefer the command line 72 | * [VSCode] - Microsoft's open source text editor 73 | 74 | [Atom]: https://atom.io/ 75 | [Exuberant Ctags]: http://ctags.sourceforge.net/ 76 | [Firefox]: https://www.mozilla.org/en-US/firefox/new/ 77 | [iTerm2]: http://iterm2.com/ 78 | [reattach-to-user-namespace]: https://github.com/ChrisJohnsen/tmux-MacOSX-pasteboard 79 | [Spectacle]: https://www.spectacleapp.com/ 80 | [Sublime Text 3]: http://www.sublimetext.com/3 81 | [Tmux]: https://tmux.github.io/ 82 | [Vim]: http://www.vim.org/ 83 | [VSCode]: https://code.visualstudio.com/ 84 | 85 | These suggestions were culled from the deprecated (as of October 2020) [laptop script]. 86 | If you are looking for references on how to build personal dotfiles, that repo may be of use. 87 | 88 | [laptop script]: https://github.com/18F/laptop 89 | -------------------------------------------------------------------------------- /_pages/continuous-deployment.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Continuous Deployment 3 | sidenav: tools 4 | sticky_sidenav: true 5 | --- 6 | 7 | ## Pre-requisites 8 | 9 | This guide assumes that you already have: 10 | - [a GitHub account](https://handbook.18f.gov/github/) and a GitHub repository, typically under the [18F GitHub organization](https://github.com/18F) 11 | - [a CircleCI account](https://circleci.com/signup/) (Log In with GitHub) 12 | - [a cloud.gov account](https://cloud.gov/docs/getting-started/accounts/?) 13 | 14 | ## 1. Getting deployer credentials 15 | 16 | Use the [instructions on Cloud.gov](https://cloud.gov/docs/apps/continuous-deployment/#provisioning-deployment-credentials) to create a deployer account for your app. Your deployer credentials will regularly expire, so please make sure to update them periodically. 17 | 18 | 19 | ## 2. Configure the continuous integration service 20 | 21 | ### Circle CI 22 | 23 | Add your cloud.gov deployer service credentials as environment variables to CircleCI. Save them as the CF_USERNAME and CF_PASSWORD. 24 | 25 | **Update `.circleci/config.yml`** 26 | 27 | ```yml 28 | version: 2.1 29 | 30 | jobs: 31 | test: 32 | # Your test configuration goes here. 33 | # Ruby - https://circleci.com/docs/2.0/language-ruby/ 34 | # Python - https://circleci.com/docs/2.0/language-python/ 35 | # JS - https://circleci.com/docs/2.0/language-javascript/ 36 | 37 | deploy: 38 | docker: 39 | # This image has the latest cf-cli as well as zero downtime plugins, if needed. 40 | - image: governmentpaas/cf-cli:latest 41 | 42 | steps: 43 | - checkout 44 | - run: 45 | name: deploy to cloud.gov 46 | command: | 47 | # Set $CF_USERNAME and $CF_PASSWORD in CircleCI settings. 48 | # $CF_ORG, $CF_SPACE, and $APP_NAME can also be set in CircleCI settings or hardcoded here. 49 | cf api https://api.fr.cloud.gov 50 | cf auth "$CF_USERNAME" "$CF_PASSWORD" 51 | cf target -o "$CF_ORG" -s "$CF_SPACE" 52 | cf zero-downtime-push "$APP_NAME" -p '.' -f path/to/manifest.yml 53 | 54 | workflows: 55 | test: 56 | jobs: 57 | - test: 58 | filters: 59 | branches: 60 | ignore: master 61 | deploy: 62 | jobs: 63 | - test: 64 | filters: 65 | branches: 66 | only: master 67 | - deploy: 68 | requires: 69 | - test 70 | filters: 71 | branches: 72 | only: master 73 | ``` 74 | 75 | Done! 76 | 77 | 78 | ## Add manifests 79 | Cloud.gov (and Cloud Foundry) use manifest files to specify how an app should be built on cloud.gov. You will now add two separate files, a `manifest.yml` for your production app and a `manifest-staging.yml` for your development application. 80 | 81 | Generally your production application will have multiple instances while your staging will only have one. Manifests can be short and sweet, or extensive. See the full [cloud foundry documentation on manifests](https://docs.cloudfoundry.org/devguide/deploy-apps/manifest.html#minimal-manifest). 82 | 83 | For an example manifest and manifest-staging see here: 84 | [Acquisitions Manifest](https://github.com/18F/acquisitions.18f.gov/blob/develop/manifest.yml) 85 | [Acquisitions Manifest-Staging](https://github.com/18F/acquisitions.18f.gov/blob/develop/manifest-staging.yml) 86 | 87 | 88 | ## Zero Downtime Deploy Options 89 | - {%include components/tag-standard.html %} The native Cloud Foundry [rolling app deployments](https://docs.cloudfoundry.org/devguide/deploy-apps/rolling-deploy.html#deploy) CLI is preferred over other options. 90 | - {%include components/tag-caution.html %} [Autopilot](https://github.com/contraband/autopilot) is a plugin historically used by a lot of TTS projects and used in both of the above examples. **It is now unmaintained and archived and does not support buildpacks.** We recommend moving to the official CF deployment commands. 91 | - If you are using autopilot and your application successfully deploys to cloud.gov but does not start, which may happen for an application that does not have an adequate test suite, you may have to go into the cf target space and manually delete the "APP_NAME-venerable" application in order to make use of `autopilot` again. 92 | - [`blue-green-deploy`](https://github.com/bluemixgaragelondon/cf-blue-green-deploy) another plugin similar to autopilot. 93 | - An official CircleCI / Cloud Foundry Orb is also available at 94 | -------------------------------------------------------------------------------- /_pages/security.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Security 3 | sidenav: security 4 | sticky_sidenav: true 5 | --- 6 | 7 | Security is everybody's responsibility at TTS but if you're not used to thinking about security in your day-to-day job, 8 | that's understandable. We come from a range of different backgrounds, some of which involved security more than others. 9 | 10 | There are practices that we as developers should adhere to as much as possible when building websites. We aim to give everyone 11 | a framework to think about security, and ways to approach it with our partners. 12 | 13 | This is an ever-expanding list; if an important issue is unrepresented, please feel free to open up a PR with your expertise 14 | or add an issue! 15 | 16 | ## FISMA 17 | Every system you'll work on at TTS has a FISMA level of impact. 18 | The [Federal Information Security Modernization Act](https://www.cisa.gov/federal-information-security-modernization-act) 19 | (FISMA) was introduced to ensure that all government systems have a framework to handle confidential and sensitive information 20 | in a secure way. 21 | 22 | The impact level of a system is determined by how adverse the impact would be if the confidentiality, integrity and/or 23 | availability of system was compromised. 24 | 25 | ``` 26 | The potential impact is LOW if the loss of confidentiality, integrity, or availability could be expected to have a limited adverse effect on organizational operations, organizational assets, or individuals. 27 | ... 28 | The potential impact is MODERATE if the loss of confidentiality, integrity, or availability could be expected to have a serious adverse effect on organizational operations, organizational assets, or individuals. 29 | ... 30 | The potential impact is HIGH if the loss of confidentiality, integrity, or availability could be expected to have a severe or catastrophic adverse effect on organizational operations, organizational assets, or individual 31 | ``` 32 | [Standards for Security Categorization of Federal Information and Information Systems](https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.199.pdf) 33 | 34 | ## When Should I Be Thinking About Security? 35 | 36 | The short answer: Always! 37 | 38 | Some longer answers: 39 | 40 | ### When starting work on a new system 41 | 42 | When starting work on a brand new system, it can feel like everything is going to go perfectly! But it's important to begin 43 | building keeping in mind that things can (and likely will!) go wrong in unexpected ways. 44 | 45 | Make sure you, or the system owners, have a way of knowing when something goes wrong. Start by asking some difficult questions: 46 | 47 | * How will we know if the system is hacked? 48 | * How will we know if there is a data leak? 49 | * What will happen if there is a data breach? 50 | * What is our escalation policy when things go wrong? 51 | 52 | If the project requires an [ATO](https://atos.open-control.org/) (or has one already), some of these topics may be explicitly covered in that process. Even if an ATO is not required, these are still important questions to ask. 53 | 54 | ### When starting work on an existing system 55 | 56 | Ideally, every vendor would be employing security best practices! But sometimes we'll start working on a project, and encounter a security 57 | flaw or potential breach that needs to be addressed ASAP. While those flaws need to be escalated, we should be thinking about how to 58 | communicate those kinds of issues without potentially alienating important relationships. 59 | 60 | *Before* getting access to a project's code base, it's a good idea to ask about a preferred escalation policy. 61 | 62 | * If I see a security problem, what is the best way to communicate that? 63 | * If it's an issue that needs to be addressed immediately, how should we work together to escalate? 64 | 65 | Premptively asking these questions can help keep focus on the security issues at hand if you immediately see problems when you get code 66 | access -- and having an answer to those questions is important for every project! 67 | 68 | After you get access, it's a good idea to ensure that there are also answers to the questions listed in the New System section 69 | 70 | ### Data Flow 71 | 72 | Security is paramount when thinking about how data flows into, through, and out of your system. 73 | 74 | When the boundaries of different systems or domains bump up against each other, there are opportunities for security breaches. 75 | These "boundaries" can be in forms such as: 76 | 77 | * third party API 78 | * installed agent 79 | * form that posts input to your database 80 | * downloading/uploading to any part of the cloud 81 | * webhook integration 82 | 83 | Thinking about the edges of your system, how they're exposed, and to whom will help you make better decisions about security. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | **⚠️ This guide has moved to the [consolidated 18F guides repository](https://github.com/18F/guides).** 2 | 3 | # Technology Transformation Services (TTS) Engineering Practices 4 | 5 | [TTS](https://www.gsa.gov/about-us/organization/federal-acquisition-service/technology-transformation-services) offices and programs — including [18F](https://18f.gsa.gov), [Centers of Excellence](https://coe.gsa.gov) (CoE), [Presidential Innovation Fellows](https://presidentialinnovationfellows.gov) (PIF), and [Solutions](https://www.gsa.gov/about-us/organization/federal-acquisition-service/technology-transformation-services/tts-solutions) — promote team best practices across specialty areas through guilds. These guilds support their members in whatever way deemed most appropriate by those members themselves. 6 | 7 | This repo is where the TTS Engineering Practices Guild keeps its guide to best practices and resources for software development. Documentation is written using [Markdown](https://kramdown.gettalong.org/quickref.html) syntax and published as HTML using the [Jekyll](https://jekyllrb.com) static site generator. 8 | 9 | ## Quicklinks 10 | 11 | - Published guide: [engineering.18f.gov](https://engineering.18f.gov) 12 | - Content: [_pages](_pages) 13 | - [CONTRIBUTING.md](CONTRIBUTING.md) on how to build this guide locally and submitting PRs/issues. 14 | 15 | ## Our mission 16 | We believe that government-developed software products should be functional, maintainable, and thoughtfully designed. Our guild helps TTS promote the adoption and advancement of software engineering best practices. In this way, TTS can lead by example while providing effective services that help our partners and customers fulfill their missions. 17 | To achieve our vision, the Engineering Practices Guild works to: 18 | - Support the continuous learning necessary for successful software engineering work. 19 | - Provide TTS developers with easy-to-understand, actionable guidance around software engineering best practices. 20 | - Promote a central knowledge base of shared tools, common patterns, tutorials, and exemplary source code repositories, to help build technical capacity at our partner agencies so that they might better govern their software development efforts. 21 | - Create a robust and supportive internal environment so that we can, in turn, bolster healthy external communities related to our work. 22 | 23 | ## How to track what we're doing, and how you can be involved! 24 | 25 | We use issues in this repo to track work. If you'd like to suggest a new topic or flag an issue, please [file an issue](https://github.com/18F/development-guide/issues/new/). 26 | 27 | The software development industry is ever-changing, and our guide is a living document. Please suggest edits or changes via pull request. 28 | 29 | Getting new practices into the guide is pretty light on process. Feel free to raise a topic in Slack or at a guild meeting and drive to some consensus. Once you've done that, document your findings, submit a PR, and ask in #dev for a quick review. If you think a proposal might be controversial after getting some consensus prior, please post the draft PR to #dev (and elsewhere if you don’t think target audience is in that channel) and solicit feedback. 30 | 31 | ## Development 32 | 33 | To run the site locally, we recommend using: 34 | - [`git`](https://git-scm.com) 35 | - `docker` and `docker-compose` (included in [Docker Desktop](https://www.docker.com/products/docker-desktop)) 36 | 37 | 1. Clone the repository: 38 | 39 | ```sh 40 | git clone https://github.com/18F/development-guide.git 41 | ``` 42 | 1. From within the repository directory, run: 43 | 44 | ```sh 45 | docker-compose up --build 46 | ``` 47 | 48 | 1. Open http://localhost:4000 49 | 50 | 51 | To check if the links referenced in the site content are valid, run: 52 | 53 | ```sh 54 | docker-compose run web bundle exec rake test 55 | ``` 56 | 57 | To only check internal links, run: 58 | ```sh 59 | docker-compose run web bundle exec rake ci_test 60 | ``` 61 | 62 | Note that the automated CircleCI integration process will only check internal links, as many websites will generate spurious errors. 63 | 64 | ## Public domain 65 | 66 | This project is in the worldwide [public domain](LICENSE.md). As stated in [CONTRIBUTING](CONTRIBUTING.md): 67 | 68 | > This project is in the public domain within the United States, and copyright 69 | > and related rights in the work worldwide are waived through the [CC0 1.0 70 | > Universal public domain 71 | > dedication](https://creativecommons.org/publicdomain/zero/1.0/). 72 | > 73 | > All contributions to this project will be released under the CC0 74 | >dedication. By submitting a pull request, you are agreeing to comply 75 | >with this waiver of copyright interest. 76 | -------------------------------------------------------------------------------- /_sass/_uswds-theme-custom-styles.scss: -------------------------------------------------------------------------------- 1 | 2 | //// 18F styles 3 | 4 | // Color 5 | $color-light: color("cyan-10v"); // was #b3efff 6 | $color-bright: color("cyan-30v"); // was #00cfff 7 | $color-medium: color("primary"); 8 | $color-dark: color("primary-dark"); // was #1C304A 9 | $color-base: color("primary-dark"); // was #1F2E4A 10 | $color-black: color("black"); 11 | $color-inverse: color("white"); 12 | 13 | $color-bright-hover: color("accent-cool-dark"); 14 | $color-medium-hover: color("blue-60"); 15 | 16 | $color-gray-lightest: color("base-lightest"); // Adding bc used in our $border-light in this file. 17 | $color-gray-hover: color("gray-2"); 18 | $color-gray-divider: color("gray-20"); 19 | 20 | $color-gray-border: color("base-dark"); 21 | 22 | 23 | // Typography 24 | .usa-prose{ 25 | h1,h2,h3,h4,h5,h6 { 26 | color: $color-dark; 27 | } 28 | 29 | p, li { 30 | color: $color-base; 31 | } 32 | } 33 | .usa-list li, 34 | .usa-prose>ul li, 35 | .usa-prose>ol li { 36 | margin-bottom: units(0.5); 37 | } 38 | .usa-prose>ul li>ul, 39 | .usa-prose>ul li>ol, 40 | .usa-prose>ol li>ul, 41 | .usa-prose>ol li>ol { 42 | margin-top: units(1.5); 43 | } 44 | 45 | // Links 46 | a { 47 | color: $color-medium; 48 | } 49 | 50 | a:hover { 51 | color: $color-medium-hover; 52 | } 53 | 54 | // Header 55 | .usa-nav .usa-current::after, 56 | .usa-header--extended .usa-current:hover::after, 57 | .usa-nav__primary>.usa-nav__primary-item > .usa-current:hover::after { 58 | background-color: $color-bright; 59 | height: units(0.5); 60 | } 61 | 62 | .usa-nav__primary>.usa-nav__primary-item > .usa-current, 63 | .usa-nav__primary>.usa-nav__primary-item > a:hover { 64 | color: $color-black; 65 | } 66 | 67 | .usa-nav__primary>.usa-nav__primary-item > a:hover { 68 | background-color: $color-gray-lightest; 69 | } 70 | 71 | .usa-nav__primary>.usa-nav__primary-item > a:hover::after { 72 | height: 0; 73 | } 74 | 75 | .usa-nav__secondary-links { 76 | display: none; 77 | } 78 | 79 | // usa-sidenav 80 | 81 | .usa-sidenav a { 82 | color: $color-base; 83 | } 84 | 85 | .usa-sidenav > .usa-sidenav__item { 86 | padding-left: 0; 87 | } 88 | 89 | .usa-sidenav .usa-current { 90 | color: $color-dark; 91 | } 92 | 93 | .usa-sidenav .usa-current::after { 94 | background-color: $color-bright; 95 | border-radius: 0; 96 | width: units(1); 97 | top: 0; 98 | bottom: 0; 99 | } 100 | 101 | .usa-sidenav a:hover { 102 | color: inherit; 103 | background-color: $color-gray-lightest; 104 | } 105 | 106 | //// Customization 107 | 108 | // Set max-width on container to contain lines 109 | .usa-layout-docs__main { 110 | max-width: units("tablet"); 111 | } 112 | 113 | // Set line-height to specify distance between lines of text 114 | .usa-prose > h1, 115 | .usa-prose > h2, 116 | .usa-prose > h3, 117 | .usa-prose > h4, 118 | .usa-prose > h5, 119 | .usa-prose > h6 { 120 | @include u-line-height("heading", 3); 121 | } 122 | 123 | // Set property values for code block 124 | pre { 125 | white-space: pre-wrap; 126 | } 127 | 128 | // Show Slack icon before Slack Links 129 | a[href^="https://gsa-tts.slack.com"]::before { 130 | content: ""; 131 | $size: units(4); 132 | width: $size; 133 | height: $size; 134 | display: inline-block; 135 | vertical-align: middle; 136 | margin: units(-2px) units(-0.5) 0; 137 | background-size: 100%; 138 | background-repeat: no-repeat; 139 | background-image: url("../../assets/images/Slack_Mark.svg"); 140 | } 141 | .usa-prose>ul.list-item--margin-bottom-extra>li, 142 | .usa-prose>ol.list-item--margin-bottom-extra>li { 143 | @include u-margin-bottom(1.5); 144 | } 145 | 146 | @include at-media("desktop") { 147 | .usa-header--extended .usa-logo { 148 | max-width: none; 149 | } 150 | 151 | .usa-nav__primary>.usa-nav__primary-item { 152 | @include u-font("ui", "xs"); 153 | } 154 | 155 | .usa-header--extended .usa-nav__primary-item>.usa-current::after { 156 | background-color: $color-bright; 157 | } 158 | 159 | .usa-header--extended .usa-nav__primary-item>.usa-nav__link:not(.usa-current):hover::after { 160 | display: none; 161 | } 162 | } 163 | 164 | .usa-layout-docs__sidenav { 165 | overflow: visible; 166 | order: 0; 167 | margin-bottom: units(3); 168 | padding-top: 0; 169 | position: relative; 170 | } 171 | 172 | .usa-header .usa-logo a { 173 | display: inline-block; 174 | } 175 | 176 | @include at-media("desktop") { 177 | .usa-layout-docs__sidenav { 178 | @include grid-col(3); 179 | position: sticky; 180 | } 181 | 182 | .usa-layout-docs__main { 183 | @include u-flex("fill"); 184 | } 185 | } 186 | 187 | 188 | //// Footer (usa-anchor styles) 189 | @import '_usa_identifier'; 190 | -------------------------------------------------------------------------------- /_includes/footer.html: -------------------------------------------------------------------------------- 1 | {% assign identifier = site.data.usa_identifier %} 2 | 3 | 91 | -------------------------------------------------------------------------------- /_pages/python.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Python Development Guide 3 | sidenav: languages 4 | sticky_sidenav: true 5 | --- 6 | 7 | This is a **WORK IN PROGRESS**. Help us make it better by [submitting an 8 | issue](https://github.com/18F/development-guide) or joining us in the 9 | [#python](https://gsa-tts.slack.com/messages/C02ES0C3R) channel! 10 | 11 | This document is structured by topic; under each, we include “Standards”, 12 | “Defaults”, and “Suggestions”. 13 | 14 | **Standards** are practices that have a strong consensus across TTS; they 15 | should generally be followed to ease the ATO process and make on-boarding 16 | simpler. 17 | 18 | **Defaults** are safe selections that tend to be used by a large number of our 19 | projects; you may find yourself with a better or more tailored solution, 20 | however. 21 | 22 | **Suggestions** contain examples that have worked well on a project or two; 23 | they're not widely used enough to be defaults, but are worth considering. 24 | 25 | ## Versions 26 | 27 | We've **standardized** on Python 3.x over 2.x. All new projects should begin 28 | their life in 3.x, though legacy libraries can continue to support 2.x (in 29 | addition to 3.x) through [tox](https://tox.readthedocs.io/en/latest/). When 30 | starting a Python project, select the [latest Python 31 | release](https://github.com/cloudfoundry/python-buildpack/releases) available 32 | on cloud.gov and incrementally update as new releases are issued. 33 | 34 | When using [Django], we **default** to starting with the most recent [Long Term 35 | Support](https://www.djangoproject.com/download/#supported-versions) release. 36 | This will give your project the most runway of support. If a second LTS becomes 37 | available while building the project, upgrade at the earliest convenience. Devs 38 | that follow will thank you. 39 | 40 | Otherwise, our **standard** practice is to use the latest release of our 41 | libraries when first installing. Security updates (as indicated by GitHub or 42 | Snyk) should be applied ASAP, but all libs should be updated at some routine 43 | interval (e.g. quarterly). 44 | 45 | Finally, in an effort to ensure our deployments are repeatable, our code 46 | **standards** require all dependencies (including dependencies' dependencies) 47 | be pinned to specific versions. This should also apply to the development 48 | environment (e.g. linters, testing tools, etc.) **Suggestions** for 49 | implementing that include 50 | * [pip-tools](https://github.com/jazzband/pip-tools)'s `pip-sync` 51 | * [pipenv](https://github.com/pypa/pipenv)'s `Pipfile.lock` 52 | * [vendoring 53 | dependencies](https://docs.cloudfoundry.org/buildpacks/python/index.html#vendoring) 54 | (though this is only a partial solution) 55 | 56 | ## Style 57 | 58 | Our **standard** tool for ensuring consistency across Python code bases is 59 | [flake8](http://flake8.pycqa.org/en/latest/). Its **default** settings are a 60 | good first step, as is using its [integration with 61 | isort](https://pypi.python.org/pypi/flake8-isort) for import order. We 62 | **suggest** investigating flake8's [plugin 63 | ecosystem](https://pypi.python.org/pypi?%3Aaction=search&term=flake8&submit=search) 64 | for more functionality. 65 | 66 | Use [Black](https://black.readthedocs.io/en/stable/) for automatic code formatting. 67 | 68 | Using Code Climate to measure complexity scores (by way of 69 | [radon](https://pypi.python.org/pypi/radon)) is also a reasonable **default** 70 | to ensure you see potentially confounding functions and classes. 71 | 72 | ## Libraries 73 | 74 | The Python ecosystem is large and full of alternative solutions to similar 75 | problems. Here we document a few common use cases and the libraries we 76 | recommend when trying to solve them. 77 | 78 | | Purpose | Library | Conviction | 79 | | --- | --- | --- | 80 | | Test Runner | [py.test](https://docs.pytest.org/en/latest/) | Standard | 81 | | Web framework | [Django] | Default | 82 | | ORM | [Django] | Default | 83 | | API | [Django Rest Framework](http://www.django-rest-framework.org/) | Default | 84 | | HTTP Client | [Requests](http://docs.python-requests.org/en/master/) | Default | 85 | | Task Queue | [Celery](https://github.com/celery/celery) | Suggestion | 86 | 87 | ## Type support 88 | 89 | Python 3.5 and beyond have had partial support for static type hints. Static 90 | typing can both make code authors' intent clearer and reduce the number of 91 | bugs through static analysis. It's also notorious for slowing down the pace of 92 | prototyping and requiring a great deal of boiler-plate. 93 | 94 | Given this state, we believe it's reasonable to **default** to using type 95 | annotations when they make your intent clearer (i.e. as a form of 96 | documentation). We **suggest** using a static analysis tool (such as 97 | [mypy](http://mypy.readthedocs.io/en/latest/)) to catch logic bugs, but only 98 | where it's practical. Consider a white-list of files to run against. 99 | 100 | [Django]: https://www.djangoproject.com/ 101 | -------------------------------------------------------------------------------- /_pages/security/cloud-services.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Cloud Services 3 | sidenav: security 4 | sticky_sidenav: true 5 | --- 6 | A common practice is storing files in the cloud; places like [Amazon Simple Storage Service](https://docs.aws.amazon.com/s3/index.html) 7 | (Amazon S3), [Google Cloud](https://cloud.google.com/storage/docs/introduction), or 8 | [Azure Storage](https://docs.microsoft.com/en-us/rest/api/storageservices/delegate-access-with-shared-access-signature). 9 | These services give developer an easy way to store data in the cloud, and download it when needed. But this 10 | introduces security considerations. 11 | 12 | When thinking about how to upload or download data in your application, there are always tradeoffs to think about -- often 13 | processes that are easier to use are less secure; likewise a workflow that is more complex is often more secure. 14 | 15 | These tradeoffs get more significant depending on the FISMA level of your system. 16 | 17 | ## Presigned URLs 18 | A common method of allowing users to transfer data without credentials is to use [presigned urls](https://docs.aws.amazon.com/AmazonS3/latest/userguide/ShareObjectPreSignedURL.html). 19 | (Azure refers to these as [shared access signatures](https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-storage-sharedaccesssignature-permissions), but they are a similar concept.) 20 | 21 | There are some differences between service providers; the below details are specific to S3 as that has 22 | been our most common use case and is supported by [cloud.gov](https://cloud.gov/docs/services/s3/). 23 | 24 | ### Some Things to know about S3 presigned URLs 25 | * They can be reused until they expire 26 | * Default expiration time is 15 minutes 27 | * They can be used by *anyone* 28 | * There is no default file size limit on uploads 29 | * Uploads use PUT by default 30 | 31 | A presigned URL is an *easily-shareable* URL that is generated with an authenticated user's security credentials. They 32 | are created with specific actions attached to them, as well as an expiration date and time; the URL will remain valid 33 | until that expiration moment. 34 | 35 | A shareable URL that bypasses security authentication is very convenient to use! However, the tradeoff is that anyone 36 | with access to that URL can use it. If a user for some reason decided to post a presigned upload URL to the internet, 37 | anyone could use that URL to upload data to your bucket until the expiration time was passed. 38 | 39 | This introduces a "user error" or "insider threat" security vulnerability to your system. Even if your users are good actors, 40 | some attack vectors could be: 41 | * A bad actor scanning spaces of URLs to find publicly-available files. 42 | * Anything with access to the client would have access to the URLs and the accompanying actions - this could include an installed untrustworthy browser extension. 43 | 44 | ### FISMA Low: 45 | Be cautious but proceed with presigned URLs if you feel it is the right choice for your system. 46 | 47 | We recommend taking [mitigation steps](#mitigation-steps) to secure your system. 48 | 49 | ### FISMA Medium: 50 | Really consider the tradeoffs. What kind of adverse impact might happen if a bad actor gets hold of a presigned URL to your system? 51 | 52 | * Can they access PII? 53 | * Can they upload junk data or other harmful information? 54 | 55 | If you've thought things through and it's the best or only option, proceed but definitely take [mitigation steps](#mitigation-steps) 56 | to secure your system. 57 | 58 | ### FISMA High: 59 | {%include components/tag-caution.html %} We do not recommend using presigned URLs for this kind of system. The only real barrier 60 | between a bad actor and user data is the obscurity/randomness of the URL and the expiration duration. 61 | 62 | ### Mitigation Steps 63 | **All Actions** 64 | * Generate expiration times that are *very* short lived -- think seconds rather than minutes. 65 | * Don't log unencrypted presigned URLs 66 | 67 | **Upload Actions** 68 | * Use a [POST action](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-UsingHTTPPOST.html) rather than PUT 69 | * Construct a [POST policy](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-HTTPPOSTConstructPolicy.html) to 70 | * limit file size as appropriate to your use case 71 | * limit file type as appropriate to your use case 72 | * Scan for viruses 73 | 74 | ## Proxying the file download 75 | 76 | This option is less "convenient" in that there is no easily-shareable URL that is generated. However, if your 77 | system has a higher FISMA impact level, or if you don't need to generate a shareable URL, this is often a more 78 | secure option. 79 | 80 | This will be more specific to your server, but the basic steps are: 81 | 82 | * Create an endpoint in your server that the client can call 83 | * Check authentication and authorization when the endpoint is hit 84 | * Gracefully capture the issue and respond with an error if the client is not authorized for this action 85 | * If the client is authenticated and authorized, construct an authenticated GET to the cloud provider 86 | * Serve the file to the client as a response to the endpoint -------------------------------------------------------------------------------- /_pages/nodejs.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Node.js 3 | sidenav: languages 4 | sticky_sidenav: true 5 | --- 6 | 7 | ## Related topics 8 | * [JavaScript]({{site.baseurl}}/javascript) 9 | 10 | ## Introduction 11 | 12 | This is a **WORK IN PROGRESS**. Help us make it better by [submitting an 13 | issue](https://github.com/18F/development-guide) or joining us in the 14 | [#javascript](https://gsa-tts.slack.com/messages/C032KSPPQ) channel! 15 | 16 | This document is structured by topic; under each, we include “Standards”, 17 | “Defaults”, and “Suggestions”. 18 | 19 | **Standards** are practices that have a strong consensus across TTS; they 20 | should generally be followed to ease the ATO process and make on-boarding 21 | simpler. 22 | 23 | **Defaults** are safe selections that tend to be used by a large number of our 24 | projects; you may find yourself with a better or more tailored solution, 25 | however. 26 | 27 | **Suggestions** contain examples that have worked well on a project or two; 28 | they're not widely used enough to be defaults, but are worth considering. 29 | 30 | 31 | ## Versions 32 | 33 | Node.js supports certain versions as Long-term Support (LTS) versions. We have 34 | **standardized** to using the [current active 35 | LTS](https://github.com/nodejs/LTS#lts-schedule1) version, currently v8.x. 36 | 37 | 38 | JavaScript is an evolving language defined by 39 | [ECMAScript](https://en.wikipedia.org/wiki/ECMAScript). Modern browsers 40 | support ES5, but when writing code on the server, you don't have to write 41 | browser-compatible code. With `--harmony`, Node.js [supports many ES6 and ES7 42 | features](http://node.green/). Similarly, on both the front end _and_ back end, 43 | you can use webpack, browserify, babel, etc. to transpile your code. This 44 | allows you to fill in the feature gaps that are not yet supported by Node.js. 45 | This also means your browser code and server code use the same ECMAScript 46 | features. Given these capabilities, we recommend **default**ing to using a 47 | transpiler and targeting the same ES version for back end and front end code. 48 | Be sure to always polyfill missing features in the browser. We **suggest** 49 | using ES6 as your source language. 50 | 51 | For libraries, our **standard** practice is to use the latest release when 52 | first installing. Security updates (as indicated by GitHub or Snyk) should be 53 | applied ASAP, but all libs should be updated at some routine interval (e.g. 54 | quarterly). 55 | 56 | Finally, in an effort to ensure our deployments are repeatable, our code 57 | **standards** require all dependencies (including dependencies' dependencies) 58 | be pinned to specific versions. This should also apply to the development 59 | environment (e.g. linters, testing tools, etc.) **Suggestions** for 60 | implementing that include 61 | * npm's [package-lock.json](https://docs.npmjs.com/files/package-lock.json) 62 | (npm >= 5) 63 | * npm's [npm-shrinkwrap.json](https://docs.npmjs.com/files/shrinkwrap.json) 64 | (npm < 5) 65 | * yarn's [yarn.lock](https://yarnpkg.com/lang/en/docs/yarn-lock/) 66 | * [vendoring 67 | dependencies](http://docs.cloudfoundry.org/buildpacks/node/index.html#vendoring) 68 | (though this is only a partial solution) 69 | 70 | ## Style 71 | 72 | Our **standard** tool for ensuring consistency across Node code bases is 73 | [eslint](http://eslint.org/), using the [AirBnB style 74 | guide](https://github.com/airbnb/javascript) as a **default** configuration. 75 | See the [Front End Guide](https://frontend.18f.gov/javascript/style/) for 76 | installation instructions. 77 | 78 | ## Libraries 79 | 80 | Frameworks provide a common baseline for your application. ATO requires 81 | configurations that most frameworks provide out of the box, or make it very easy 82 | to enable. Here, we document a few common use cases and the libraries we 83 | recommend when trying to solve them. 84 | 85 | | Purpose | Library | Conviction | 86 | | --- | --- | --- | 87 | | Test Runner | [Mocha](https://mochajs.org/) | Default | 88 | | Test Spies, Stubs, etc. | [Sinon](https://www.npmjs.com/package/sinon) | Default | 89 | | Test assertions | [Chai](https://www.npmjs.com/package/chai) | Default | 90 | | Web framework | [Express](https://expressjs.com) | Default | 91 | | Authentication | [Passport](http://www.passportjs.org/) | Default | 92 | | Security Headers | [Helmet](https://www.npmjs.com/package/helmet) | Default | 93 | | Embedded JSON | [htmlescape](https://www.npmjs.com/package/htmlescape) | Default | 94 | | ORM | [sequelize](https://www.npmjs.com/package/sequelize) | Suggestion | 95 | | Sessions | [cookie-session](https://www.npmjs.com/package/cookie-session) | Suggestion | 96 | | CSRF | [csurf](https://www.npmjs.com/package/csurf) | Suggestion | 97 | 98 | ## Scaffolding your application 99 | 100 | Scaffolding tools can help take care of some of boilerplate of initializing a 101 | project. Some scaffolds are generator tools, others are kits intended to be 102 | forked. There are many available on GitHub. You might also choose one based on 103 | the front end frameworks you want to use and then add the server-side pieces 104 | yourself. You also don't have to use one at all. That said, don't spend too 105 | much time finding the perfect generator. Once your application has been 106 | scaffolded, you're not going to scaffold it again. 107 | 108 | We **suggest** using one of the following scaffolds: 109 | 110 | - [express-generator](https://www.npmjs.com/package/express-generator) 111 | - [express-babel](https://github.com/vmasto/express-babel) 112 | 113 | ## Type support 114 | 115 | Static typing can both make code authors' intent clearer and reduce the number 116 | of bugs through static analysis. It's also notorious for slowing down the pace 117 | of prototyping and requiring a great deal of boiler-plate. Some TTS 118 | projects are using [TypeScript](https://www.typescriptlang.org/) 119 | to add static typing, while a handful are testing out 120 | [Flow](https://flow.org/). 121 | 122 | At this time, we **suggest** considering a static type-checker, and using 123 | TypeScript if it'd aid your project's workflow. 124 | -------------------------------------------------------------------------------- /_pages/language-selection.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Languages & Runtimes 3 | sidenav: languages 4 | sticky_sidenav: true 5 | --- 6 | 7 | Many factors should influence how TTS engineers make technology selections 8 | for their projects. Here, we discuss recommendations on how to select the 9 | _language_ used in your projects. This document doesn't offer hard-and-fast 10 | rules, focusing instead on several language aspects 11 | which should be balanced against each other. Also, we explicitly note that 12 | language selection is ultimately the decision (and responsibility) of the 13 | project team. The guidance below is meant to _assist_ your team make this 14 | decision. 15 | 16 | This is a living document. After looking at [existing 17 | tickets](https://github.com/18F/development-guide/issues), [start an 18 | issue](https://github.com/18F/development-guide/issues/new) or pull request in 19 | GitHub to suggest changes to these standards. 20 | 21 | ## Our strongest languages 22 | 23 | TTS has historically worked in three **primary** languages: JavaScript, 24 | Python, and Ruby. While we certainly have a healthy smattering of other odds 25 | and ends, these three are our primary strength as 26 | indicated by data from past projects, engineer skill sets, and billable hours. 27 | These languages are generally a safe bet as we know we can support them and be 28 | productive with them. 29 | 30 | Looking at this same data and applying preferences from the guiding factors 31 | below, we also see a **second tier** of familiarity around Go, Java, and PHP. 32 | These languages may be good fits depending on the project at hand. In 33 | particular, domain preference and client familiarity may 34 | be major reasons to select one of these languages for your project. We feel 35 | relatively comfortable supporting these languages, but if all other factors 36 | are equal, we encourage selecting from our first tier. 37 | 38 | ## Frequently-used frameworks 39 | 40 | The following are used widely in TTS: 41 | 42 | | Purpose | Tool | 43 | | ------------------------------------ | ------------------------------------------------------------------------------------------------------------ | 44 | | CSS framework | [_More info_]({{site.baseurl}}/css/#frameworks) | 45 | | Infrastructure/configuration as code | [Terraform](https://www.terraform.io/) | 46 | | Static site generator | [Jekyll](https://jekyllrb.com/) (with the [uswds-jekyll](https://github.com/18F/uswds-jekyll) theme) or Hugo | 47 | 48 | ## Project scope 49 | 50 | Perhaps the most important factor to weigh when 51 | considering languages is the estimated project scope. If we anticipate a 52 | large, long-standing project which will be handed off to our agency partners, 53 | we should be conservative in our language selection. These projects warrant 54 | our most standard approach, which generally translates to the selection of one 55 | of our primary languages. On the other hand, if writing a one-off script or 56 | small internal project, we have significantly more latitude to try 57 | experimental languages. 58 | 59 | Let's consider some examples for reference. As a 60 | reminder, in almost all situations, selecting one of our primary languages is 61 | a viable option. 62 | 63 | * Scrape GitHub statistics one-shot - R would be a fine solution. Due to the 64 | small scope of the project, we get much more value 65 | from R than we lose due to risk of unmaintainability, etc. 66 | * Integration tests for an internal app - Here, we've 67 | ratcheted up risk: a team will need to maintain these 68 | tests in the future. However, if the team were so inclined, using something 69 | like Perl might still be a viable option due to the limited impact of these 70 | tests. 71 | * Agency-centric open data API - Given the public-facing 72 | project scope and assuming other factors are equal, we should write this in 73 | one of our primary languages. We can be confident we'll do the best job, 74 | most efficiently with this approach. 75 | 76 | ## Factors 77 | 78 | Languages are not all equal. We next consider a 79 | handful of factors to weigh when selecting a language for a project. In no 80 | particular order: 81 | 82 | * Open source - TTS is a strong proponent of open source; we want our language 83 | selection to reflect that. We want open APIs, open 84 | source binaries, and community participation. 85 | * Domain preference - certain problem domains emphasize particular languages. 86 | For example, the Cloud Foundry community has a preference for Go, while 87 | SOAP-heavy specs are more friendly in Java, and Rails 88 | projects promote CoffeeScript. We want to use the tools appropriate for the 89 | job. 90 | * Team familiarity - The more TTS engineers who are 91 | comfortable with a language, the safer it is to use. 92 | We want our project to be accessible to many, both within TTS and without. 93 | * Stability - bleeding edge languages are more risky. If the standard API is 94 | changing every few months, we aren't going to be able to maintain the 95 | project. Try to fathom how this bodes when we're 96 | handing this project off. 97 | * Active ecosystem - some languages are on the rise while others are falling 98 | or never gained traction. We will be leaning on the 99 | community for support and potentially to hire contractors and vendors. Less 100 | active communities make this more difficult. 101 | * Inclusive communities - TTS promotes welcoming cultures and we want to see 102 | that reflected in our language selection. We want to reward language leaders 103 | who promote diverse opinions and have open standards processes. We see 104 | languages led by cabals as risks. 105 | * Library support - we don't want to reinvent the wheel 106 | (or the database adapter). Languages with a thriving library ecosystem will 107 | be easier and most cost effective to work with. 108 | * Hand-off considerations - is our agency partner comfortable with a 109 | particular framework? If they already know how to deploy 110 | XYZ apps (and hire XYZ developers), it behooves us to 111 | write an XYZ app. 112 | 113 | ## Other impact 114 | 115 | By setting forth the above, we can imagine potential 116 | impact on our hiring, project selection, and larger processes. The progression 117 | is logical and we may want to use this document as evidence when deciding how 118 | to iterate those systems. Concrete details are outside the scope of this 119 | document, however, and we anticipate (and proclaim) no immediate changes. 120 | -------------------------------------------------------------------------------- /_pages/datastore-selection.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Datastore Selection 3 | sidenav: tools 4 | sticky_sidenav: true 5 | --- 6 | 7 | We're fortunate to have dozens of battle-tested datastores available to us, 8 | filling many different niches and general use cases. Each has its own 9 | strengths, weaknesses, configuration, backup system, security profile, and 10 | cognitive overhead. In an effort to make this selection simpler, TTS 11 | engineering **defaults** to using Postgres for the majority of our 12 | applications. This also allows us to collectively learn best practices around 13 | security configurations, indexing strategies, and so forth, particularly 14 | between large, open data projects. 15 | 16 | ## Our use case 17 | 18 | Though we build many different types of systems, we have enough commonality 19 | across projects that we can note some generalities. 20 | 21 | 1. We have significantly more reads than writes. 22 | 2. We almost always have structured data when designing schemas. Very few 23 | projects need flexible/unstructured models. 24 | 3. Maintainability is more important than "interesting" or "cutting-edge". 25 | We're building software to hand over to agency partners; choosing tools 26 | that are easy to develop with *and* easy for long-term maintenance is 27 | critical to that success. 28 | 4. Similarly, stability is more important than performance. Our projects often 29 | waffle between active development and maintenance mode; we need to prepare 30 | for the latter. We don't know who will be available to fix a downed 31 | database. 32 | 33 | Of course, performance, writes, etc. are still important factors to a 34 | well-rounded application, but the above generalities give us a direction to 35 | lean (think: the [agile manifesto](http://agilemanifesto.org/)). 36 | 37 | This general use case negates a large number of the benefits of NoSQL and 38 | other niche solutions. On the other hand, we frequently need more flexibility 39 | than RedShift, VoltDB, and other highly scalable datastores allow. 40 | 41 | ## Postgres serves our needs 42 | 43 | We've used Postgres in some form on the majority of our projects and found 44 | that it covers almost all (if not all) of our needs. When starting a new 45 | project (that requires a datastore), we use Postgres; as the project grows, we 46 | first attempt to build solutions *within* Postgres, pushing it and researching 47 | features we need. Only if we hit a significant wall do we reach for another 48 | datastore. 49 | 50 | ### Text search 51 | 52 | A frequent reason to integrate Elastic, Solr, etc. is a need for text search 53 | (specifics, often, undefined). Luckily, Postgres has this functionality built 54 | in (either directly or by enabling additional modules). Let's cover four of 55 | the more interesting types of text search: 56 | 57 | 1. *Full text search*, search over large blobs of text, is provided by the 58 | `tsearch` module. This is capable of word stemming (e.g. ensuring "book" 59 | will be found if searching for "books"), skipping stop words (e.g. "am", 60 | "is", "the"), weighting fields, returning results with highlighted matches, 61 | creating a dictionary of synonyms, and more. The necessary vectorization 62 | can also be precomputed and indexed. Collectively, this account for 90+% of 63 | our need for a separate search index. 64 | 2. *Trigrams* break the search term and potential results into three-letter, 65 | overlapping strings and count the number of matches. This type of search 66 | works really well for smaller text fields, like searching over city names. 67 | This is provided in Postgres by the `pg_trgm` module, and can also be 68 | precomputed and indexed. 69 | 3. *Edit distance* calculates the number of "edits" (character 70 | inserts/deletes/shifts) between a search term and text corpus, making it 71 | very useful for finding misspelled words. This is provided by the 72 | `fuzzystrmatch` module. 73 | 4. *Soundex* convert characters into phonemes (sounds), allowing for search 74 | over homophones like "Kris" and "Chris". This is excellent option for 75 | certain types of word search such as name matching, and also provided by 76 | the `fuzzystrmatch` module. The necessary transforms can be precomputed and 77 | indexed. 78 | 79 | Better still, these features are either already integrated into our existing 80 | ORMs (as with Django) or a quick module install away (including ActiveRecord, 81 | Sequelize, and Sqlalchemy). 82 | 83 | ### Unstructured data 84 | 85 | Another common argument for using Mongo, Elastic, or Couch is the need to 86 | store data without a predefined structure (notably, arbitrary JSON). While 87 | this use case doesn't come up frequently for us, Postgres has us covered when 88 | it does. **JSON** is a first class data type in Postgres, meaning we can 89 | store arbitrary structures, query arbitrarily nested values, index the 90 | results, add constraints, etc. Postgres doesn't have map-reduce-style 91 | functions, but can perform all of the SQL aggregations we're familiar with 92 | over those JSON structures, which satisfies the majority of this need. 93 | 94 | Additionally, Postgres has an older data type called **HStore**, which maps 95 | arbitrary string keys to string values. This provides the same query, 96 | indexing, etc. options as JSON, but can't be nested and only allows string 97 | values. HStore's been around much longer however, so may have more support in 98 | your ORMs (though JSON is supported by most, at this point). Postgres also 99 | supports **XML** as a native data type for the few eXist-db hold outs. 100 | 101 | ### And more 102 | 103 | If you reach for Mongo for GIS support, take a look at PostGIS, which has many 104 | more features. If you're using a document store to store hierarchical data, 105 | consider using the [nested set 106 | model](https://en.wikipedia.org/wiki/Nested_set_model) in Postgres. Heck, it's 107 | pretty easy to replace GridFS, though storing files in the database is 108 | generally a bad idea. The Foreign Data Wrapper also gives a very powerful 109 | avenue for including data from arbitrary sources. 110 | 111 | As the pattern indicates, for almost all of the situations we might want to 112 | reach for another datastore, Postgres has a viable (often superior) solution. 113 | 114 | ## Not a requirement 115 | 116 | Postgres is our *default* datastore, not a *requirement*. There are certainly 117 | use cases where it isn't the right choice, and we don't take a dogmatic 118 | stance. If you have a Wordpress project, you should probably use MySQL; their 119 | support for other databases is cursory, at best. Few task queue libraries work 120 | well with Postgres; using Redis makes a lot of sense here. Memcached is still 121 | a workhorse for page caching. Be pragmatic. Push Postgres first, but be 122 | prepared to support alternatives if the need arises. 123 | -------------------------------------------------------------------------------- /_includes/components/header.html: -------------------------------------------------------------------------------- 1 | {% if header %} 2 |
3 | {% if header.type == 'basic' %} 4 |