├── APIs.md ├── DATA_INFRA.md ├── EM.md ├── ENG_PRACTICES.md └── README.md /APIs.md: -------------------------------------------------------------------------------- 1 | | internal | external | 2 | | -------- | -------- | 3 | | image charts | https://www.image-charts.com/ | 4 | -------------------------------------------------------------------------------- /DATA_INFRA.md: -------------------------------------------------------------------------------- 1 | ### Reading 2 | - [Machine Learning: The High Interest Credit Card of Technical Debt](https://ai.google/research/pubs/pub43146) 3 | - [The Care & Feeding of Data Scientists](https://oreilly-ds-report.s3.amazonaws.com/Care_and_Feeding_of_Data_Scientists.pdf) 4 | - [Designing Data-Intensive Applications](https://www.amazon.com/Designing-Data-Intensive-Applications-Reliable-Maintainable/dp/1449373321) 5 | - [Storytelling with Data: A Data Visualization Guide for Business Professionals](https://www.amazon.com/Storytelling-Data-Visualization-Business-Professionals/dp/1119002257) 6 | 7 | ### Distributed Frameworks 8 | - spark 9 | - https://github.com/ray-project/ray 10 | - https://dask.org/ 11 | 12 | ### Warehouse 13 | - redshift 14 | - snowflake 15 | - bigquery 16 | - vertica 17 | 18 | ### Data Lake 19 | - databricks delta 20 | - https://github.com/apache/iceberg 21 | - https://github.com/apache/hudi 22 | - https://www.dremio.com/ 23 | 24 | ### Change data capture / streaming 25 | - https://www.attunity.com/ 26 | - https://streamsets.com/ 27 | - https://github.com/debezium/debezium 28 | 29 | ### data versioning 30 | - https://projectnessie.org/ 31 | 32 | ### OLAP 33 | - https://pinot.apache.org/ 34 | - https://kylin.apache.org/ 35 | 36 | ### Streaming 37 | - kafka 38 | - flink 39 | - beam 40 | 41 | ### real-time/document-search 42 | - elasticsearch 43 | - rockset 44 | 45 | ### Notebooks / exploration 46 | - jupyter 47 | - https://zeppelin.apache.org/ 48 | - https://github.com/ironmussa/Optimus 49 | - https://colab.research.google.com/ 50 | - https://www.saturncloud.io/ 51 | - https://rmarkdown.rstudio.com/ 52 | - https://www.querybook.org/ 53 | - https://deepnote.com/ 54 | - https://cnvrg.io/ 55 | - dominodatalab 56 | 57 | ### Labeling 58 | - https://www.snorkel.org/ 59 | - https://www.figure-eight.com/ 60 | - https://scale.ai 61 | - https://www.labelbox.com 62 | 63 | ### Notebook management 64 | - https://github.com/nteract/papermill 65 | 66 | ### Orchestration 67 | - airflow 68 | - luigi 69 | - oozie 70 | - https://github.com/dagster-io/dagster 71 | - https://github.com/PrefectHQ/prefect 72 | - https://www.getdbt.com/product/ 73 | 74 | ### Dataflow/transformation managers 75 | - streamset 76 | - nifi 77 | 78 | ### unstructured/semi-structured data prep 79 | - https://www.trifacta.com/ 80 | 81 | ### Solutions 82 | - snowplow 83 | - https://www.prefect.io/ 84 | - https://www.ascend.io/ 85 | - https://www.datmo.com/ 86 | - TFX-OSS 87 | 88 | ### data quality monitoring 89 | - https://www.anomalo.com/ 90 | - https://github.com/awslabs/deequ 91 | 92 | ### anomaly detection as a service 93 | - https://www.anodot.com/ 94 | 95 | ### event data 96 | - segment 97 | - Rudderstack 98 | - Metarouter 99 | - snowplow 100 | 101 | ### A/B testing 102 | - https://www.optimizely.com/ 103 | - https://www.abtasty.com/ 104 | - https://github.com/YahooArchive/mendel 105 | 106 | ### Behavioral analytics 107 | - https://www.fullstory.com/ 108 | - https://www.hotjar.com/ 109 | 110 | ### Feature flagging 111 | - https://launchdarkly.com/ 112 | - https://www.split.io/ 113 | - https://github.com/checkr/flagr 114 | 115 | ### Authorization 116 | - https://www.osohq.com/ 117 | ### Catalogue / metadata 118 | - https://github.com/airbnb/knowledge-repo 119 | - https://github.com/lyft/amundsen 120 | - https://eng.uber.com/databook/ 121 | - https://medium.com/netflix-techblog/metacat-making-big-data-discoverable-and-meaningful-at-netflix-56fb36a53520 122 | - https://cloud.google.com/data-catalog/ 123 | - https://github.com/linkedin/datahub 124 | - [airbnb: democratizing data](https://medium.com/airbnb-engineering/democratizing-data-at-airbnb-852d76c51770), [airbnb: scaling knowledge](https://medium.com/airbnb-engineering/scaling-knowledge-at-airbnb-875d73eff091) 125 | 126 | ### BI 127 | - [superset](https://superset.incubator.apache.org/) 128 | - looker 129 | - periscope 130 | 131 | ### etl / ingestion-as-a-service 132 | - fivetran 133 | - singer.io 134 | - talend 135 | - https://github.com/airbytehq/airbyte 136 | - https://github.com/linkedin/brooklin 137 | 138 | ### reverse-etl: piping data around for salesOps/marketingOps/CRM etc 139 | - https://tray.io/ 140 | - https://www.snaplogic.com/ 141 | - https://www.polytomic.com/ 142 | - https://www.getcensus.com/ 143 | 144 | ### Sales analytics 145 | - https://www.chorus.ai/ 146 | - https://www.gong.io/ 147 | 148 | ### timeseries forecasting 149 | - https://github.com/linkedin/greykite 150 | 151 | ### Neat utils 152 | - query CSVs w/ SQL: [cq](https://github.com/jolmg/cq) 153 | - jq 154 | 155 | ### Generating fake data 156 | - https://github.com/lk-geimfari/mimesis 157 | 158 | ### privacy / compliance 159 | - https://www.datagrail.io/ 160 | - immuta 161 | - privacera 162 | - ethyca 163 | - osano 164 | -------------------------------------------------------------------------------- /EM.md: -------------------------------------------------------------------------------- 1 | ## Engineering Management 2 | 3 | ### People 4 | 5 | #### understanding self/team 6 | - [Myers-Briggs](https://en.wikipedia.org/wiki/Myers%E2%80%93Briggs_Type_Indicator) 7 | - https://www.jocrf.org/ 8 | - [StrengthsFinder](https://www.amazon.com/StrengthsFinder-2-0-Tom-Rath/dp/159562015X) 9 | - DISC assessment 10 | - [Adjective Check List](https://en.wikipedia.org/wiki/Adjective_Check_List) 11 | - [Enneagram](https://www.integrative9.com/) 12 | - [Standout](https://www.tmbc.com/standout-2-0-assessment/) 13 | 14 | - [the muppet theory](https://www.theatlantic.com/culture/archive/2012/06/muppet-theory-explains-humanity/327133/) 15 | - [Neil on Software](https://neilonsoftware.com/highly-effective-software-developers/) 16 | 17 | #### career convo 18 | - https://www.radicalcandor.com/blog/problem-career-conversations/ 19 | - https://firstround.com/review/three-powerful-conversations-managers-must-have-to-develop-their-people/ 20 | - https://www.cultureamp.com/blog/letting-go-of-our-best-employees/ 21 | - https://hbr.org/amp/2013/06/tours-of-duty-the-new-employer-employee-compact 22 | - ikigai 23 | - https://staffeng.com/ 24 | - https://www.amazon.jobs/en/landing_pages/pe-community-tenets 25 | - https://www.progression.fyi/ 26 | 27 | #### team health 28 | - https://www.atlassian.com/team-playbook/health-monitor 29 | 30 | #### retention 31 | - stay interview 32 | 33 | #### communication 34 | - [how to say no](https://www.starterstory.com/how-to-say-no) 35 | 36 | #### on-demand mentors/coaching 37 | - (technical) https://www.codementor.io/ 38 | - (EM-specific) https://www.platohq.com/ 39 | - https://www.workbravely.com/ 40 | - https://www.betterup.com/ 41 | - (exec) https://athenaalliance.com/ 42 | 43 | #### hiring 44 | - https://stripe.com/guides/atlas/scaling-eng 45 | - [[CloserIQ] Sales Interview Grading Template : Sample](https://docs.google.com/document/d/1iPw2p90HbEciKpt84JSVwefFnYtkN_W3X9SVV6FtvCg/edit) 46 | - [Google’s sample structured interview grading rubric](https://docs.google.com/spreadsheets/d/17U5XuQgM1v52niQU-LBrk2j4qaQhcnziuUBCYTA2fMo/pubhtml) 47 | - https://medium.engineering/engineering-interviews-grading-rubric-8b409bec021f 48 | - https://hbr.org/2016/02/a-scorecard-for-making-better-hiring-decisions 49 | 50 | ##### hiring distributed teams / contract eng 51 | - https://remotive.io/remote-companies 52 | - https://terminal.io/ 53 | - https://pilot.co/ 54 | - https://arc.dev/ 55 | - https://worca.io/ 56 | - https://turing.com/ 57 | - https://www.letsdeel.com/ 58 | - https://www.boundlesshq.com/ 59 | - https://meet.jobs/ 60 | - https://github.com/lukasz-madon/awesome-remote-job 61 | - https://www.10xmanagement.com/ 62 | - https://www.facetdev.com/ 63 | - https://www.toptal.com/ 64 | - https://x-team.com/ 65 | - https://symphony.is/ 66 | 67 | ##### hiring/assessment/interviewing-as-a-service 68 | - https://karat.com/ 69 | - https://hired.com/ 70 | - https://interviewing.io/ 71 | - https://codesignal.com/ 72 | - https://www.qualified.io/ 73 | 74 | ##### sourcing 75 | - https://www.traitly.com/ 76 | - https://www.dover.io/ 77 | 78 | ##### hiring from referrals 79 | - https://www.teamable.com/ 80 | 81 | ##### d&i 82 | - https://gaingels.com/ 83 | - https://www.devcolor.org/ 84 | - http://www.code2040.org/ 85 | - https://www.colorstack.org/ 86 | - https://www.womenwhocode.com/ 87 | - https://www.nextplaycommunity.com/ 88 | - https://techqueria.org/ 89 | - https://girlgeek.io/ 90 | - https://www.diversifytech.co/ 91 | - https://afrotech.com/ 92 | - https://www.devinclusiongiveaways.com/ 93 | - https://iamremarkable.withgoogle.com/ 94 | 95 | ##### recruiting 96 | - https://www.bincsearch.com/ 97 | - security focused: https://coderedpartners.com/ 98 | 99 | ##### executive recruiting 100 | - https://rivierapartners.com/ 101 | - https://www.daversapartners.com/ 102 | 103 | ##### layoff lists 104 | - https://layoffs.fyi/ 105 | - https://www.layoffs.tech/ 106 | - https://www.onesoftlanding.com/ 107 | - https://candor.co/layoff-list/ 108 | 109 | ##### leveling 110 | - https://dropbox.github.io/dbx-career-framework/overview.html 111 | - https://staffeng.com/ 112 | - https://levels.fyi/ 113 | 114 | 115 | ### Process 116 | 117 | #### 1:1s 118 | - https://medium.com/@mrabkin/the-art-of-the-awkward-1-1-f4e1dcbd1c5c 119 | - https://medium.com/@mrabkin/awkward-1-1s-the-art-of-getting-honest-feedback-2843078b2880 120 | 121 | #### 1:1/checkin facilitation/nudge engines 122 | - https://www.range.co/ 123 | - https://knowyourteam.com/ 124 | - https://www.readymetrics.com/ 125 | - https://humu.com/ 126 | 127 | #### meetings / facilitation 128 | - http://www.debonogroup.com/six_thinking_hats.php 129 | - [gCal speedy meetings setting](https://www.bettercloud.com/monitor/the-academy/efficient-effective-meetings-google-calendar/) 130 | - https://www.nytimes.com/2015/02/08/opinion/sunday/sheryl-sandberg-and-adam-grant-on-women-doing-office-housework.html 131 | - https://courses.lumenlearning.com/introductiontocommunication/chapter/groups-roles/ 132 | 133 | #### frameworks 134 | - situational leadership (kenblanchard) 135 | - https://firstround.com/review/Keith-Rabois-on-the-role-of-a-COO-how-to-hire-and-why-transparency-matters/ 136 | 137 | #### decision making 138 | - https://firstround.com/review/how-this-head-of-engineering-boosted-transparency-at-instagram/ 139 | - https://www.atlassian.com/team-playbook/plays/daci 140 | - https://en.wikipedia.org/wiki/Time_management#The_Eisenhower_Method 141 | 142 | 143 | ### Resources 144 | 145 | #### Books 146 | - first 90 days 147 | - managing humans 148 | - high output management 149 | - hard thing about hard things 150 | - measure what matters (aka the okr book) 151 | - sapien 152 | - an elegant puzzle 153 | - The Five Dysfunctions of a Team: A Leadership Fable 154 | - the Advantage 155 | - the phoenix project 156 | - radical candor 157 | - Peopleware 158 | - What Got You Here Won't Get You There 159 | - An Elegant Puzzle: Systems of Engineering Management 160 | - High Growth Handbook 161 | - The Manager's Path 162 | - Dare to Lead 163 | - The Messy Middle 164 | - https://medium.com/range/leadership-reading-list-951c5ccb4370 165 | - Fierce Conversations 166 | - [The Culture Map](https://www.amazon.com/Culture-Map-Breaking-Invisible-Boundaries/dp/1610392507) 167 | 168 | #### Reading 169 | - http://randsinrepose.com/ 170 | - https://lethain.com 171 | - https://medium.com/the-year-of-the-looking-glass 172 | - https://rework.withgoogle.com/ 173 | - https://rework.withgoogle.com/guides/managers-identify-what-makes-a-great-manager/steps/learn-about-googles-manager-research/ 174 | - https://rework.withgoogle.com/print/guides/5721312655835136/ 175 | - High Output Management [TLDR](https://medium.com/@iantien/top-takeaways-from-andy-grove-s-high-output-management-2e0ecfb1ea63) and [Forward by Horowitz](https://a16z.com/2015/11/13/high-output-management/) 176 | - [RACI](http://firstround.com/review/how-this-head-of-engineering-boosted-transparency-at-instagram/) and [this](https://en.wikipedia.org/wiki/Responsibility_assignment_matrix) 177 | - [Manager READMEs](https://hackernoon.com/12-manager-readmes-from-silicon-valleys-top-tech-companies-26588a660afe) 178 | - https://www.manager-tools.com/ 179 | - [6 Must Reads for Managers Giving Feedback](http://firstround.com/review/our-6-must-reads-for-managers-to-give-feedback-that-helps-people-grow/) 180 | - [Failed #SquadGoals - Spotify doesn’t use “the Spotify model” and neither should you.](https://www.jeremiahlee.com/posts/failed-squad-goals/) 181 | -------------------------------------------------------------------------------- /ENG_PRACTICES.md: -------------------------------------------------------------------------------- 1 | also https://github.com/google/eng-practices 2 | 3 | ### Misc Resources 4 | 5 | email lists 6 | - https://maleallies.com/ 7 | - https://hello.cultureamp.com/subscribe-to-the-people-geekly 8 | - https://andrewchen.co/ 9 | 10 | 11 | #### secops, privacy 12 | - https://www.hackerone.com/blog/security-vendors-startups-lob-cant-live-without 13 | - https://12factor.net/ 14 | - https://cheatsheetseries.owasp.org/index.html 15 | - https://sudo.pagerduty.com/ 16 | - [vendor sec assessment](https://blogs.dropbox.com/tech/2019/03/towards-better-vendor-security-assessments/) 17 | - https://haveibeenpwned.com/ 18 | - https://developer.github.com/partnerships/token-scanning/ 19 | - compliance vendors: 20 | - coalfire, a-lign, armanino, ncc group 21 | - pentest vendors: 22 | - https://www.includesecurity.com/ 23 | - https://cobalt.io/ 24 | - https://cure53.de/ 25 | - https://application.security/ 26 | - https://snyk.io/ 27 | - https://doppler.com/ 28 | - https://github.com/rawdigits/go-flashpaper 29 | - https://darknetdiaries.com/ 30 | 31 | #### compliance 32 | - https://github.com/pivotal/LicenseFinder 33 | - https://github.com/AlDanial/cloc 34 | 35 | #### devops 36 | - [google SRE book](https://landing.google.com/sre/book/index.html) 37 | - getting infra-as-code off the ground (terraform): https://gruntwork.io/ 38 | - https://github.com/etsy/DebriefingFacilitationGuide 39 | - https://how.complexsystems.fail/ 40 | 41 | #### refactoring 42 | - https://refactoring.guru/ 43 | 44 | #### code quality / style 45 | - https://prettier.io/ 46 | - https://github.com/google/code-prettify 47 | 48 | #### doing code review 49 | - https://google.github.io/eng-practices/review/reviewer/ 50 | 51 | #### development process 52 | - https://trunkbaseddevelopment.com/ 53 | 54 | #### papers 55 | - https://blog.acolyer.org/ 56 | 57 | #### fraud/spam related services: 58 | - https://pro.whitepages.com/ (vet employer/identity) 59 | - https://www.lexisnexis.com/en-us/gateway.page (vet employer/identity) 60 | - https://risk.lexisnexis.com/products/threatmetrix 61 | - https://emailage.com/ (vet e-mail legitimacy) 62 | 63 | industry-collaborative risk-data-exchange project: 64 | - https://www.zerofox.com/ (commercial company) 65 | - https://developers.facebook.com/products/threat-exchange (by fb) 66 | - arkose labs, perimeterX, Shape, Cequence 67 | 68 | ### Employment Law Compliance 69 | - https://www.lawroom.com/ 70 | - https://www.lifetheatre.com/training-topics/ 71 | 72 | #### when all else fails, call: 73 | - https://layeraleph.com/ (mikeyd@) 74 | 75 | #### design/marketing/uiux 76 | - https://fuzzco.com/ 77 | - http://omstudio.co/ 78 | 79 | #### well being 80 | - https://www.workonwellbeing.com/ 81 | 82 | #### advisor/founder agreement 83 | - https://fi.co/fast 84 | 85 | ### other random stuff 86 | - https://asciinema.org 87 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | A handy lookup table of similar technology and services to help ex-googlers survive the *real* world :) 2 | pull-requests are very welcomed. __Please do not list any confidential projects!__ 3 | 4 | For a working example of (some) of these technologies integrated together, see: 5 | https://github.com/google/startup-os 6 | 7 | See also: [System Design Primer](https://github.com/donnemartin/system-design-primer), [The Hadoop Ecosystem Table](https://hadoopecosystemtable.github.io/), [Map AWS services to GCP products](https://cloud.google.com/free/docs/map-aws-google-cloud-platform), [Database of Databases](https://dbdb.io/), [Google Open Source Glossary](https://opensource.google/docs/glossary/) 8 | 9 | ## Technology 10 | 11 | ### Core Technology 12 | 13 | | Google Internal | Google External | Open Source / Real-World | 14 | | --------------- | ---------------------------------------- | ---------------------------------------- | 15 | | MapReduce | | [Apache Hadoop](https://github.com/apache/hadoop), [Spark](https://github.com/apache/spark) | 16 | | [Flume](https://research.google/pubs/flumejava-easy-efficient-data-parallel-pipelines/) | [DataFlow](https://cloud.google.com/dataflow) | [Apache Beam](https://beam.apache.org/) | 17 | | Protocol Buffer | [Protobuf](https://github.com/google/protobuf), [FlatBuffers](https://google.github.io/flatbuffers/) | [Cap'n Proto](https://capnproto.org/), [Thrift](https://github.com/apache/thrift), [Avro](https://github.com/apache/avro), [Amazon Ion](https://amzn.github.io/ion-docs/), [CBOR](https://cbor.io/), [kryo](https://github.com/EsotericSoftware/kryo) | 18 | | Stubby | [gRPC](https://github.com/grpc/grpc) | [Thrift](https://github.com/apache/thrift), [Bolt](https://boltprotocol.org/) | 19 | | Chubby | | [Apache Zookeeper](https://github.com/apache/zookeeper), [etcd](https://github.com/coreos/etcd), [HashiCorp Consul](https://github.com/hashicorp/consul) | 20 | | Goops / PubSub | [Cloud Pub/Sub](https://cloud.google.com/pubsub) | [Apache Kafka](https://github.com/apache/kafka), [Apache Pulsar](https://github.com/apache/incubator-pulsar), [Facebook LogDevice](https://github.com/facebookincubator/LogDevice) | 21 | | `//base` | | [abseil](https://github.com/abseil) | 22 | 23 | ### Infrastructure 24 | 25 | | Google Internal | Google External | Open Source / Real-World | 26 | | -------------------- | ---------------------------------------- | ---------------------------------------- | 27 | | Borg | [Kubernetes](https://kubernetes.io/) | [Apache Mesos](https://github.com/apache/mesos), [Apache Aurora](https://aurora.apache.org/), [HashiCorp Nomad](https://github.com/hashicorp/nomad) | 28 | | GSLB | [Cloud Load Balancing](https://cloud.google.com/load-balancing) - Internal | AWS ELB, [Istio](https://istio.io/), [linkerd](https://linkerd.io/) 29 | | OnePlatform | [API Gateway](https://cloud.google.com/api-gateway) | [Swagger](https://swagger.io/) | 30 | | GFE, Maglev, uberproxy | [Cloud Load Balancing](https://cloud.google.com/load-balancing) - HTTPS / External | [envoy](https://www.envoyproxy.io/), AWS ALB, [HAProxy](https://www.haproxy.org/), [nginx](https://www.nginx.com/), [F5](https://f5.com/products/big-ip) | 31 | | uberproxy (sso proxy) | [Identity-Aware Proxy](https://cloud.google.com/iap) | [buzzfeed-sso](https://github.com/buzzfeed/sso), [Pomerium](https://pomerium.io/), [Envoy Gateway](https://gateway.envoyproxy.io/docs/tasks/security/oidc/) | 32 | | Zanzibar | [Zanzibar Research Paper](https://research.google/pubs/pub48190/) | [SpiceDB](https://github.com/authzed/spicedb)/[authzed](https://authzed.com/), [Ory Keto](https://www.ory.sh/keto/docs/), [topaz](https://github.com/aserto-dev/topaz), [Opal](https://opal.dev/), [(iam)Keycloak](https://www.keycloak.org/), [Warrant](https://github.com/warrant-dev/warrant), [OpenFGA](https://openfga.dev/) | 33 | | data center hardware | [open compute](https://www.opencompute.org/) | | 34 | | Jupiter, Starblaze | | | 35 | | B4, Stargate, TE | | | 36 | | USPS, Andromeda | | | 37 | | ESDN | | [Faucet](https://github.com/faucetsdn/faucet) 38 | | [broccoli man](https://www.youtube.com/watch?v=3t6L-FlfeaI) | || 39 | 40 | 41 | ### Storage 42 | 43 | | Google Internal | Google External | Open Source / Real-World | 44 | | ---------------------------------------- | ---------------------------------------- | ---------------------------------------- | 45 | | GFS/Colossus | | [HDFS](https://hadoop.apache.org/docs/r1.2.1/hdfs_design.html), [Ceph](https://ceph.com), [GlusterFS](https://www.gluster.org), [MooseFS](https://moosefs.com/products/#moosefs) | 46 | | Bigtable | [Cloud Bigtable](https://cloud.google.com/bigtable) | [PrestoDB](https://prestodb.io/), [Cassandra](https://github.com/apache/cassandra), [HBase](https://github.com/apache/hbase), [Accumulo](https://github.com/apache/accumulo), [DynamoDB](https://aws.amazon.com/dynamodb), [ScyllaDB](https://www.scylladb.com/) | 47 | | [Spanner](https://research.google.com/archive/spanner.html) | [Cloud Spanner](https://cloud.google.com/spanner/) | [Vitess](https://vitess.io), [CockroachDB](https://github.com/cockroachdb/cockroach), [TiDB](https://github.com/pingcap/tidb) | 48 | | ColumnIO / [Capacitor](https://cloud.google.com/blog/products/bigquery/inside-capacitor-bigquerys-next-generation-columnar-storage-format) | [BigQuery storage](https://cloud.google.com/bigquery/docs/storage_overview) | [Apache Parquet](https://parquet.apache.org), [ORC](https://orc.apache.org/docs/), [Apache Iceberg](https://iceberg.apache.org/) | 49 | | sstable | [levelDB](https://github.com/google/leveldb) | [RocksDB](https://rocksdb.org), [pebble](https://github.com/cockroachdb/pebble) | 50 | | zippy | [Snappy](https://github.com/google/snappy) | [lz4](https://github.com/lz4/lz4) | 51 | | RecordIO | [Riegeli](https://github.com/google/riegeli), [TFRecords](https://github.com/tensorflow/docs/blob/r1.10/site/en/api_guides/python/python_io.md#tfrecords-format-details), & in [OR-Tools](https://github.com/google/or-tools/blob/stable/ortools/base/recordio.h), [szl](https://github.com/google/szl/blob/master/src/utilities/recordio.cc) | [stuffed-record-stream](https://github.com/backtrace-labs/stuffed-record-stream) | 52 | 53 | 54 | ### Services 55 | | Google Internal | Google External | Open Source | SaaS | 56 | | ---------------------------------------- | ---------------------------------------- | ---------------------------------------- | ---------------------------------------- | 57 | | Dremel | [BigQuery](https://cloud.google.com/bigquery/) | [Apache Drill](https://github.com/apache/drill), [Presto](https://prestodb.io), Spark(sort-of), | [AWS Athena](https://aws.amazon.com/athena/), [Redshift Spectrum](https://aws.amazon.com/redshift/spectrum/), [Snowflake](https://www.snowflake.com) | 58 | | Dremel UI | | [Redash](https://github.com/getredash/redash), [Metabase](https://github.com/metabase/metabase), [Apache Superset](https://github.com/apache/incubator-superset) | | 59 | | Search (Mustang, Alexandria) | | [Elasticsearch](https://github.com/elastic/elasticsearch), [OpenSearch](https://opensearch.org/), Solr, Lucene | [algolia](https://www.algolia.com/) | 60 | | pubsub | [pubsub](https://cloud.google.com/pubsub/docs/overview) | [NATS.io](https://nats.io), [RabbitMQ](https://github.com/rabbitmq), [PubNub](https://www.pubnub.com/) | AWS SQS/SNS, [AWS AppSync](https://aws.amazon.com/appsync) | 61 | | [MillWheel](https://ai.google/research/pubs/pub41378) | [Cloud Dataflow](https://cloud.google.com/dataflow/) | [Apache Flink](https://flink.apache.org/), beam | | 62 | | Colab | [Colaboratory](https://colab.research.google.com/) | [Jupyter](https://jupyter.org) | [Observable](https://observablehq.com/) | 63 | | PLX | [Google Data Studio](https://datastudio.google.com/) | | [Mode](https://mode.com) | 64 | | Monarch | [paper](https://research.google/pubs/pub50652/) | | | 65 | | Napa | [paper](https://research.google/pubs/pub50617/) | | | 66 | | MakerSuite| [MakerSuite](https://makersuite.google.com) | | | 67 | 68 | 69 | ### DevOps 70 | | Google Internal | Google External | Real-World | 71 | | ------------------------------- | ---------------------------------------- | ---------------------------------------- | 72 | | Assimilator | | [Dominator](https://github.com/Cloud-Foundations/Dominator) | 73 | | Blaze | [Bazel](https://bazel.build) | [Buck](https://buckbuild.com/), [Pants](https://www.pantsbuild.org/), [please.build](https://please.build/), [Blade](https://github.com/chen3feng/blade-build), [recc](https://gitlab.com/bloomberg/recc), [EngFlow](https://www.engflow.com/), [BuildBuddy](https://www.buildbuddy.io/), [flare.build](https://flare.build) | 74 | | Oncall | | [PagerDuty](https://pagerduty.com), [OpsGenie](https://www.opsgenie.com/), [VictorOps](https://victorops.com/) | 75 | | varz/borgmon/monarch | [Cloud Monitoring](https://cloud.google.com/monitoring) | [Datadog](https://www.datadoghq.com/), [Prometheus](https://prometheus.io), [M3](https://m3db.io/), [librato](https://www.librato.com), [newrelic](https://newrelic.com), skylight, scout, [Scotty](https://github.com/Cloud-Foundations/scotty)/[tricorder](https://github.com/Cloud-Foundations/tricorder), [netdata](https://github.com/netdata/netdata), [bosun](https://bosun.org/), also [this](https://vimeo.com/173610242) and [this](https://prometheus.io/docs/introduction/comparison/) | 76 | | Viceroy | Cloud Monitoring | [Grafana](https://grafana.com/) | 77 | | Exception/Error Tracking (thirdeye) | | Sentry.io, Raygun.io, [Rollbar](https://rollbar.com), Honeybadger, Airbrake, OverOps, [ELK stack](https://www.elastic.co/what-is/elk-stack) | 78 | | styleguides | [google styleguides](https://github.com/google/styleguide) | [PEP-8](https://www.python.org/dev/peps/pep-0008/), [HoundCI auto-style-reviewer](https://houndci.com/) | 79 | | Sponge | | [EngFlow](https://www.engflow.com/) | 80 | | Blaze/Forge/TAP/BuildCop | [Cloud Build](https://cloud.google.com/cloud-build/) | [buildkite](https://buildkite.com/), [CircleCI](https://circleci.com), [Drone](https://drone.io/), [EngFlow](https://www.engflow.com/), [github](https://github.blog/2019-08-08-github-actions-now-supports-ci-cd/), [gitlabCI](https://about.gitlab.com/product/continuous-integration/), [jenkins](https://jenkins.io/), [travis](https://travis-ci.org/) | 81 | | Forge/ObjFS | | [EngFlow](https://www.engflow.com/), [flare.build](https://flare.build) | 82 | | Sandman(test env)/Guitar | | | 83 | | Sisyphus / Rapid | | [Spinnaker](https://www.spinnaker.io/), [lambdaCD](https://www.lambda.cd), screwdriver.cd, [CodeShip](https://codeship.com), [shipit-engine](https://github.com/Shopify/shipit-engine), [GoCD](https://www.gocd.org), [AWS CodeDeploy](https://aws.amazon.com/codedeploy/), [Capistrano](https://www.capistranorb.com), [Fabric](https://www.fabfile.org), [ConcourseCI](https://concourse.ci/), [samson](https://github.com/zendesk/samson) | 84 | | MPM | | [Docker](https://www.docker.com/), [OCI](https://opencontainers.org/) | 85 | | borgcfg / gcl, [Prodspec + Annealing](https://www.usenix.org/publications/loginonline/prodspec-and-annealing-intent-based-actuation-google-production) | [Jsonnet](https://jsonnet.org/), [Cue](https://cuelang.org/) | [AWS Cloudformation](https://aws.amazon.com/cloudformation/), Puppet, Chef, Salt, Ansible, [Terraform](https://www.terraform.io), [kubecfg](https://github.com/kubecfg/kubecfg), [pulumi](https://github.com/pulumi/pulumi), [Nix](https://nix.dev/), [Pkl](https://pkl-lang.org/) | 86 | | logging, analog | [StackDriver](https://cloud.google.com/stackdriver/) | [logstash](https://github.com/elastic/logstash), [fluentd](https://github.com/fluent/fluentd), [PaperTrail](https://www.papertrail.com/), [cernan](https://github.com/postmates/cernan), [loki](https://grafana.com/oss/loki/) | 87 | | CodeSearch, Grimoire | [Zoekt](https://github.com/google/zoekt) [kythe](https://github.com/kythe/kythe) [Code Search](https://developers.google.com/code-search/user/getting-started) (for Google open source code only, with separate UI for Android and Chromium. Go CLI [source](https://github.com/google/codesearch).) | [Sourcegraph](https://sourcegraph.com), [OpenGrok](https://github.com/OpenGrok/OpenGrok/), [livegrep](https://github.com/livegrep/livegrep) | 88 | | Critique, Gerrit, Mondrian etc. | [Gerrit](https://www.gerritcodereview.com/) | [Reviewable](https://reviewable.io) , [Phabricator](https://www.phacility.com/phabricator/) | 89 | | cider | | [Eclipse Che](https://www.eclipse.org/che/), [Cloud9](https://c9.io/), [gitpod.io](https://gitpod.io), [Coder](https://coder.com/), [Code-Server (VSCode in a Tab)](https://github.com/cdr/code-server), [DevZero](https://www.devzero.io/) | 90 | | buganizer | [Google Issue Tracker](https://issuetracker.google.com/) | [JIRA](https://www.atlassian.com/software/jira), [bugzilla](https://www.bugzilla.org/), github issues, [Linear](https://linear.app/) | 91 | | Bugjuggler | | [SnoozeThis](https://www.snoozethis.com/) | 92 | | ToTT | [Google Test Blog](https://testing.googleblog.com/) | [Increment](https://increment.com/) | 93 | | Copybara / MOE | [Copybara](https://github.com/google/copybara), [MOE](https://github.com/google/MOE) | | 94 | | workflow/dependency management | | [Luigi](https://github.com/spotify/luigi), [Airflow](https://github.com/apache/airflow), [digdag](https://github.com/treasure-data/digdag), [Pachyderm](https://github.com/pachyderm/pachyderm), [Dask](https://github.com/dask/dask) | 95 | | ErrorProne | [ErrorProne](https://errorprone.info/) | [SpotBugs](https://spotbugs.github.io/), [FindBugs](https://findbugs.sourceforge.net/) | 96 | | [Dapper](https://ai.google/research/pubs/pub36356) | [stackdriver trace](https://cloud.google.com/trace/) | [zipkin](https://github.com/openzipkin/zipkin), [OpenTelemetry](https://opentelemetry.io/), [jaeger](https://www.jaegertracing.io/), [LightStep](https://lightstep.com), [Honeycomb](https://www.honeycomb.io/trace/) | 97 | | C++ Tips of the Week | [Abseil C++ Tips of the Week](https://abseil.io/tips/) | | 98 | | [DiRT](https://cloud.google.com/blog/products/management-tools/shrinking-the-time-to-mitigate-production-incidents) | | [ChaosMonkey](https://github.com/Netflix/chaosmonkey), [aws fis](https://aws.amazon.com/fis/) | 99 | | [Rosie](https://cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/fulltext) | | [microplane](https://github.com/Clever/microplane), [silver-platter](https://github.com/jelmer/silver-platter) | 100 | | API Improvements Proposals | [AIP](https://google.aip.dev/) | | 101 | | g4 {fix, submit} | | [Trunk.io](https://trunk.io), [Graphite](https://graphite.dev/) | 102 | | probers | | [cloudprober](https://github.com/cloudprober/cloudprober) | 103 | | GWP | [paper](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/36575.pdf) | [Parca](https://www.parca.dev), [Polar Signals](https://www.polarsignals.com) | 104 | 105 | ### Security 106 | | Google Internal | Google External | Open Source | 107 | | -------------------------------- | --------------- | ---------------------------------------- | 108 | | prodaccess/LOAS | | [Keymaster](https://github.com/Cloud-Foundations/keymaster) | 109 | | prod secrets/identity management | | [chamber](https://github.com/segmentio/chamber), [knox](https://github.com/pinterest/knox), [SPIFFE](https://spiffe.io/) | 110 | 111 | ## IT / Operations / Misc 112 | | Google Internal | Real-World | 113 | | ---------------------------------------- | ---------------------------------------- | 114 | | software engineering at google [1](https://arxiv.org/ftp/arxiv/papers/1702/1702.01715.pdf), [2](https://abseil.io/resources/swe-book) | [Software Engineering at Google: Lessons Learned from Programming Over Time](https://www.amazon.com/Software-Engineering-Google-Lessons-Programming/dp/1492082791) | 115 | | [SRE @ google](https://sre.google/sre-book/table-of-contents/) | | 116 | | valentine | [Vault](https://www.vaultproject.io/), [1Password](https://support.1password.com/create-share-vaults/), [Lastpass](https://lastpass.com), [pass](https://www.passwordstore.org/), [keeper](https://www.keepersecurity.com/) | 117 | | OWNERS files in repo | [github CODEOWNERS](https://github.com/blog/2392-introducing-code-owners) | 118 | | snippets | [Khan/snippets](https://github.com/Khan/snippets) | 119 | | SnipIt | [recordit](https://recordit.co/), [CloudApp](https://www.getcloudapp.com/), [dropbox screenshots](https://help.dropbox.com/installs-integrations/photos/screenshots), [Snippyly](https://snippyly.com/) | 120 | | gpaste | [pastee](https://paste.ee/), [pastebin](https://pastebin.com/) | 121 | | stuff (SaaS IT management) | [productiv](https://productiv.com/), [intello](https://www.intello.io/), [zylo](https://zylo.com/) | 122 | | stuff (Device Management) | [jamf](https://www.jamf.com/) | 123 | | device security monitoring | [Red Canary](https://redcanary.com/) | 124 | | beyondcorp | [beyondcorp](https://www.beyondcorp.com/) | 125 | | [go/ links](https://medium.com/@golinks/the-full-history-of-go-links-and-the-golink-system-cbc6d2c8bb3) | [golinks](https://www.golinks.io/), [go](https://github.com/kellegous/go), [Goat](https://goatcodes.com/), [trotto](https://github.com/trotto/go-links), [go-shorten](https://github.com/thomasdesr/go-shorten) | 126 | | google3 philosophy | [innersource](https://resources.github.com/whitepapers/introduction-to-innersource/), [monorepo](https://cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/fulltext), [YouTube talk](https://www.youtube.com/watch?v=W71BTkUbdqE) | 127 | | doing code review | [code review](https://google.github.io/eng-practices/review/reviewer/) | 128 | | safely sharing 1-time secrets | [sendsecure.ly](https://sendsecure.ly), [croc](https://github.com/schollz/croc), [onetimesecret](https://github.com/onetimesecret/onetimesecret), [privatebin](https://privatebin.info/) | 129 | | messaging | [mattermost](https://github.com/mattermost/mattermost-server), [Slack](https://slack.com), gchat | 130 | | tech talks | [TechTalks @ Google](https://www.youtube.com/user/GoogleTechTalks/videos) | 131 | | g3doc | [writethedocs](https://www.writethedocs.org/), [docs-as-code](https://idratherbewriting.com/learnapidoc/pubapis_docs_as_code.html) | 132 | | GUTS | [spoke](https://www.atspoke.com/), [freshservice](https://freshservice.com/) | 133 | | MOMA | [glean](https://www.glean.com/) | 134 | 135 | ## PeopleOps / Culture 136 | | Google Internal | Real-World | 137 | | ---------------------------------------- | ---------------------------------------- | 138 | | [OKR](https://en.wikipedia.org/wiki/OKR) | [CultureAmp OKR](https://www.cultureamp.com/okr-solution-b), [Lattice Goals](https://lattice.com/goals), [Ally](https://www.ally.io/), [workboard](https://www.workboard.com/) | 139 | | HRIS/ERP | [Namely](https://namely.com), [BambooHR](https://www.bamboohr.com/), [Workday](https://workday.com), [Paylocity](https://www.paylocity.com/) | 140 | | [peer bonus](https://www.forbes.com/sites/jurgenappelo/2015/07/08/the-peer-to-peer-bonus-system/#33c47ef84329) | [bonus.ly](https://bonus.ly/) | 141 | | kudos | [heytaco](https://www.heytaco.chat/), [slack ++ bot](https://github.com/tdmalone/working-plusplus/) | 142 | | perks | [fond](https://fond.co) | 143 | | talks | [Talks @ Google](https://www.youtube.com/c/talksatgoogle/videos) | 144 | | dory | [slido](https://www.sli.do/), [dory(app)](https://dory.app/) | 145 | | edge, people ops training | [LifeLabs](https://www.lifelabslearning.com/) | 146 | | googlegeist | [Culture Amp](https://www.cultureamp.com/), [humu](https://www.humu.com/), [tinypulse](https://www.tinypulse.com/), [peakon](https://peakon.com/) | 147 | | Meng | [Search Inside Yourself](https://www.amazon.com/Search-Inside-Yourself-Unexpected-Achieving/dp/0062116932) | 148 | | Lazlo | [Work Rules](https://www.amazon.com/Work-Rules-Insights-Inside-Transform/dp/1455554790/) | 149 | | Claire Stapleton | [Tech Support - existential advice for the modern tech worker](https://techsupport.substack.com/) | 150 | | books about google | [How Google Works](https://www.amazon.com/How-Google-Works-Eric-Schmidt/dp/1455582328), [In The Plex](https://www.amazon.com/Plex-Google-Thinks-Works-Shapes/dp/1416596585), [Software Engineering at Google](https://www.amazon.com/Software-Engineering-Google-Lessons-Programming/dp/1492082791) | 151 | 152 | 153 | also check out [xoogler.co](https://xoogler.co/), which organizes events, slack channels etc 154 | 155 | *disclaimer: I'm not affiliated with any of the technologies/products mentioned above.* 156 | 157 | *disclaimer: I left Google a number of years ago so some of the naming might be dated.* 158 | --------------------------------------------------------------------------------