├── .gitignore ├── .readthedocs.yaml ├── LICENSE ├── Makefile ├── README.md ├── conf.py ├── deploying-grr-clients ├── how-to-check-if-client-talks-back.md ├── index.md ├── life-of-grr-client.md ├── on-linux.md ├── on-mac-os-x.md ├── on-windows.md ├── overview.md ├── protection.md └── troubleshooting.md ├── developing-grr ├── codelabs │ ├── assets │ │ ├── client-id-on-terminal.png │ │ ├── dummy-flow-result.png │ │ ├── how-to-add-a-client-action.png │ │ ├── how-to-add-a-flow.png │ │ └── start-dummy-flow.png │ ├── how-to-add-a-client-action.md │ ├── how-to-add-a-flow.md │ └── index.md ├── contributing-docs.md ├── contributing.md ├── implementation-details │ ├── authorizating-and-auditing.md │ ├── client-path-spec.md │ ├── foreman.md │ ├── grr-messages.md │ └── index.md ├── index.md ├── running-tests.md ├── setting-up-compose-dev-env.md ├── setting-up-dev-env.md └── workstation │ ├── Dockerfile │ ├── index.md │ ├── settings.json │ └── workstation_startup_d │ ├── 011_mount_filestore.sh │ ├── 200_install-demo.sh │ └── 250_default-ide-color-theme.sh ├── faq.md ├── images ├── client-approval-dialog.png ├── client-approval-review-page.png ├── client-installers.png ├── client_view.png ├── collect_from_client.png ├── collected_from_client.png ├── create_hunt_from_flow.png ├── cron_job_viewer.png ├── distributed_fleetspeak.png ├── download.png ├── downloaded_file.png ├── fleetspeak_database.png ├── flow_details.png ├── flow_results.png ├── grr_certs_keys.png ├── grr_changes_applied.png ├── grr_claat.png ├── grr_client_details.png ├── grr_clients.png ├── grr_code_explore.png ├── grr_code_menu.png ├── grr_code_oss.png ├── grr_code_terminal.png ├── grr_codelab_client.png ├── grr_codelab_flow.png ├── grr_components.png ├── grr_components_up.png ├── grr_compose_up.png ├── grr_database.png ├── grr_debug_mode.png ├── grr_dev_ready.png ├── grr_development.png ├── grr_flow_success.png ├── grr_logo_real_sm.png ├── grr_run_flow.png ├── grr_ws_config.png ├── hexview.png ├── hostname_multiple_versions.png ├── how_to_check_if_client_talks_back.png ├── hunt_rules.png ├── icons │ ├── copy.png │ ├── editdelete.png │ ├── modify.png │ ├── new.png │ ├── pause_button.png │ ├── play_button.png │ ├── play_force_button.png │ ├── robot.png │ └── stop.png ├── manage_flows.png ├── rapid-hunting-eligible.png ├── refresh_buttons.png ├── sandbox_fleetspeak.png ├── settings_view.png ├── signed_url_impersonation.png ├── signed_url_overview.png ├── signed_url_sa_key.png ├── signed_url_workload_identity.png ├── stat_info.png ├── timeline.png └── vfs_full.png ├── index.md ├── installing-and-running-grr ├── advanced-usecases │ ├── index.md │ ├── sandbox.md │ ├── sandbox │ │ ├── config │ │ │ ├── fleetspeak-admin │ │ │ │ ├── components.textproto │ │ │ │ └── services.textproto │ │ │ ├── fleetspeak-frontend │ │ │ │ ├── components.textproto │ │ │ │ └── services.textproto │ │ │ ├── grr-client │ │ │ │ ├── client.yaml │ │ │ │ ├── communicator.txt │ │ │ │ ├── config.textproto │ │ │ │ └── textservices │ │ │ │ │ └── grr.service │ │ │ └── grr-server │ │ │ │ └── server.local.yaml │ │ ├── containers │ │ │ └── envoy │ │ │ │ └── Dockerfile │ │ ├── createConfig.sh │ │ ├── docker-compose.yaml │ │ └── envoy-https-https.yaml │ └── scaling.md ├── from-released-pip.md ├── from-source.md ├── index.md ├── overview.md ├── securing-access.md ├── troubleshooting.md ├── via-docker-compose.md └── via-docker.md ├── investigating-with-grr ├── artifacts │ ├── collecting.md │ ├── defining.md │ ├── index.md │ └── overview.md ├── automation-with-api.md ├── client-server-communication.md ├── collect-large-file.md ├── cron-jobs.md ├── flows │ ├── index.md │ ├── literal-and-regex-matching.md │ ├── specifying-file-paths.md │ ├── specifying-windows-registry-paths.md │ ├── starting.md │ ├── what-are-flows.md │ └── working-with-results.md ├── glossary.md ├── hunts │ ├── controlling-hunts.md │ ├── index.md │ ├── limits.md │ ├── rapid-hunts.md │ ├── rules.md │ ├── starting.md │ ├── troubleshooting.md │ ├── what-are-hunts.md │ └── working-with-results.md ├── index.md ├── interactive-scripting.md ├── osquery.md ├── output-plugins.md ├── overview.md ├── pushing-code.md ├── searching-for-client.md ├── security-considerations.md └── vfs │ ├── index.md │ ├── navigation.md │ ├── timelines.md │ └── virtual-file-system.md ├── maintaining-and-tuning ├── approval-based-workflow.md ├── building-custom-client-templates.md ├── configuration │ ├── contexts.md │ ├── file-organization.md │ ├── filtering.md │ ├── index.md │ └── parameter-expansion.md ├── email-configuration.md ├── grr-datastore.md ├── index.md ├── key-management │ ├── index.md │ ├── rotating.md │ └── which-keys-and-how.md ├── low-level-maintenance-with-console.md ├── monitoring.md ├── repacking-clients.md ├── scaling.md ├── screenshots │ └── add_notification_channel.png ├── troubleshooting.md └── user-management │ ├── authentication.md │ ├── index.md │ ├── limiting-access-with-routers.md │ ├── running-behind-apache.md │ └── running-behind-nginx.md ├── publications.md ├── quickstart.md ├── release-notes.md ├── requirements.txt └── what-is-grr.md /.gitignore: -------------------------------------------------------------------------------- 1 | _build/** 2 | .*/** 3 | 4 | *.pem 5 | -------------------------------------------------------------------------------- /.readthedocs.yaml: -------------------------------------------------------------------------------- 1 | # .readthedocs.yaml 2 | # Read the Docs configuration file 3 | # See https://docs.readthedocs.io/en/stable/config-file/v2.html for details 4 | 5 | # Required 6 | version: 2 7 | 8 | # Set the version of Python and other tools you might need 9 | build: 10 | os: ubuntu-22.04 11 | tools: 12 | python: "3.9" 13 | 14 | # Build documentation in the docs/ directory with Sphinx 15 | sphinx: 16 | configuration: conf.py 17 | 18 | # We recommend specifying your dependencies to enable reproducible builds: 19 | # https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html 20 | python: 21 | install: 22 | - requirements: requirements.txt 23 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | # Minimal makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | SPHINXPROJ = GRR 8 | SOURCEDIR = . 9 | BUILDDIR = _build 10 | 11 | # Put it first so that "make" without argument is like "make help". 12 | help: 13 | @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) 14 | 15 | .PHONY: help Makefile 16 | 17 | # Catch-all target: route all unknown targets to Sphinx using the new 18 | # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). 19 | %: Makefile 20 | @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | GRR Documentation 2 | ================================ 3 | 4 | [![Documentation Status](https://readthedocs.org/projects/grr-doc/badge/?version=latest)](https://readthedocs.org/projects/grr-doc/builds/) 5 | 6 | This repository contains GRR documentation sources. Rendered documentation is hosted on [grr-doc.readthedocs.io](http://grr-doc.readthedocs.io). 7 | 8 | See [this document](http://grr-doc.readthedocs.io/en/v3.2.1/developing-grr/contributing-docs.html) if you're interesting in fixing/contributing to the docs. 9 | -------------------------------------------------------------------------------- /conf.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # GRR documentation build configuration file, created by 4 | # sphinx-quickstart on Wed Nov 22 17:54:03 2017. 5 | # 6 | # This file is execfile()d with the current directory set to its 7 | # containing dir. 8 | # 9 | # Note that not all possible configuration values are present in this 10 | # autogenerated file. 11 | # 12 | # All configuration values have a default; values that are commented out 13 | # serve to show the default. 14 | 15 | # If extensions (or modules to document with autodoc) are in another directory, 16 | # add these directories to sys.path here. If the directory is relative to the 17 | # documentation root, use os.path.abspath to make it absolute, like shown here. 18 | # 19 | # import os 20 | # import sys 21 | # sys.path.insert(0, os.path.abspath('.')) 22 | 23 | 24 | # Add any Sphinx extension module names here, as strings. They can be 25 | # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom 26 | # ones. 27 | extensions = [ 28 | 'myst_parser', 29 | ] 30 | 31 | # Add any paths that contain templates here, relative to this directory. 32 | templates_path = ['_templates'] 33 | 34 | # The master toctree document. 35 | master_doc = 'index' 36 | 37 | # General information about the project. 38 | project = u'GRR' 39 | copyright = u'2023, GRR team' 40 | author = u'GRR team' 41 | 42 | # The version info for the project you're documenting, acts as replacement for 43 | # |version| and |release|, also used in various other places throughout the 44 | # built documents. 45 | # 46 | # The short X.Y version. 47 | version = u'' 48 | # The full version, including alpha/beta/rc tags. 49 | release = u'' 50 | 51 | # The language for content autogenerated by Sphinx. Refer to documentation 52 | # for a list of supported languages. 53 | # 54 | # This is also used if you do content translation via gettext catalogs. 55 | # Usually you set "language" from the command line for these cases. 56 | language = None 57 | 58 | # List of patterns, relative to source directory, that match files and 59 | # directories to ignore when looking for source files. 60 | # This patterns also effect to html_static_path and html_extra_path 61 | exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] 62 | 63 | # The name of the Pygments (syntax highlighting) style to use. 64 | pygments_style = 'sphinx' 65 | 66 | # If true, `todo` and `todoList` produce output, else they produce nothing. 67 | todo_include_todos = False 68 | 69 | # The theme to use for HTML and HTML Help pages. See the documentation for 70 | # a list of builtin themes. 71 | # 72 | html_theme = 'sphinx_rtd_theme' 73 | html_static_path = [] 74 | html_theme_options = { 75 | # Toc options 76 | 'collapse_navigation': True, 77 | 'sticky_navigation': True, 78 | 'navigation_depth': 4, 79 | 'includehidden': True, 80 | 'titles_only': True, 81 | } 82 | 83 | # Output file base name for HTML help builder. 84 | htmlhelp_basename = 'GRRdoc' 85 | 86 | 87 | # Configure sphinx to replace predefined version variables. 88 | from docutils import nodes, transforms 89 | 90 | class ProcessLink(transforms.Transform): 91 | 92 | default_priority = 1000 93 | 94 | text_replacements = { 95 | "__GRR_VERSION__": "3.4.7.1", 96 | "__GRR_DEB_VERSION__": "3.4.7-1", 97 | "__FLEETSPEAK_PIP_VERSION__": "0.1.13" 98 | } 99 | 100 | def find_replace(self, node): 101 | if isinstance(node, nodes.Text): 102 | for k, v in self.text_replacements.items(): 103 | if k in node.astext(): 104 | repl = nodes.Text(node.replace(k, v)) 105 | node.parent.replace(node, repl) 106 | 107 | return node 108 | 109 | def traverse(self, node): 110 | """Traverse the document tree rooted at node. 111 | node : docutil node 112 | current root node to traverse 113 | """ 114 | self.find_replace(node) 115 | 116 | for c in node.children: 117 | self.traverse(c) 118 | 119 | def apply(self): 120 | self.current_level = 0 121 | self.traverse(self.document) 122 | 123 | def setup(app): 124 | app.add_transform(ProcessLink) 125 | -------------------------------------------------------------------------------- /deploying-grr-clients/how-to-check-if-client-talks-back.md: -------------------------------------------------------------------------------- 1 | # How to check if a deployed client talks back to the GRR server 2 | 3 | If a client installation is successful, the client should appear in the web UI search within a 4 | few seconds. 5 | 6 | After install, hit Enter in the search box in the top left corner of the 7 | UI to see all of your clients that have enrolled with the server. 8 | 9 | ![Screenshot](../images/how_to_check_if_client_talks_back.png) 10 | 11 | If you 12 | don’t see the clients, follow the [troubleshooting 13 | steps](troubleshooting.md). 14 | -------------------------------------------------------------------------------- /deploying-grr-clients/index.md: -------------------------------------------------------------------------------- 1 | # Deploying GRR clients 2 | 3 | ```{toctree} 4 | --- 5 | maxdepth: 1 6 | --- 7 | Overview 8 | How to check if a deployed client talks back to GRR server 9 | On Windows 10 | On macOS 11 | On Linux 12 | Life of a GRR client (what happens after deployment) 13 | Client Protection 14 | Troubleshooting ("I don't see my clients") 15 | ``` 16 | 17 | -------------------------------------------------------------------------------- /deploying-grr-clients/on-linux.md: -------------------------------------------------------------------------------- 1 | # Installing GRR clients on Linux 2 | 3 | For Linux you will see a deb and rpms, install the appropriate one. 4 | For testing purposes you can run the client on the same machine as 5 | the server if you like. 6 | 7 | The process depends on your environment, if you have a 8 | mechanism such as puppet, then building as a Deb package and deploying 9 | that way makes the most sense. Alternatively you can deploy using ssh: 10 | 11 | scp client_version.deb host:/tmp/ 12 | ssh host sudo dpkg -i /tmp/client_version.deb 13 | 14 | 15 | ## Uninstalling on Linux 16 | 17 | This is a quick manual on how to remove the GRR client completely from a machine. 18 | 19 | On Linux the standard system packaging (deb, pkg) is used by default. 20 | Use the standard uninstall mechanisms for uninstalling. 21 | 22 | ```docker 23 | dpkg -r grr 24 | ``` 25 | 26 | This might leave some config files lying around, if a complete purge is necessary, the list of files to delete is: 27 | 28 | ```docker 29 | /usr/lib/grr/* 30 | /etc/grr.local.yaml 31 | /etc/init/grr.conf 32 | ``` 33 | 34 | The GRR service can be stopped using 35 | 36 | ```docker 37 | sudo service grr stop 38 | ``` 39 | -------------------------------------------------------------------------------- /deploying-grr-clients/on-mac-os-x.md: -------------------------------------------------------------------------------- 1 | # Installing GRR clients on Mac OS X 2 | 3 | For OSX you will see a pkg file, install the pkg. It will add a 4 | launchd item and start it. 5 | 6 | See [Linux instructions](on-linux.md). They apply also to OSX. 7 | 8 | On OSX you can also use the Uninstall GRR flow. 9 | 10 | ## Uninstalling on Mac OS X 11 | 12 | This is a quick manual on how to remove the GRR client completely from a machine. 13 | 14 | On OSX, pkg uninstall is not supported. The files to delete are: 15 | 16 | ```docker 17 | /usr/local/lib/grr/* 18 | /etc/grr.local.yaml 19 | /Library/LaunchDaemons/com.google.code.grr.plist 20 | ``` 21 | 22 | The service can be stopped using 23 | 24 | ```docker 25 | sudo launchctl unload /Library/LaunchDaemons/com.google.corp.grr.plist 26 | ``` 27 | -------------------------------------------------------------------------------- /deploying-grr-clients/overview.md: -------------------------------------------------------------------------------- 1 | # Overview 2 | 3 | This document describes getting clients up and running for 4 | the first time. 5 | 6 | ## Getting started 7 | 8 | Once we’ve got the GRR server installed and running we’ll 9 | want to start deploying some clients. 10 | 11 | To do so we’ll need to: 12 | 13 | 1. Download the specific client version we need to install on the 14 | system. 15 | 16 | 2. Decide on a deployment method. 17 | 18 | 3. Perform the deployment and verify the results. 19 | 20 | ## Requirements 21 | 22 | - A number of machines (or VMs) to talk to the server. OSX, Windows and Linux 23 | clients are supported. Client and server can run on the same host for 24 | testing purposes. 25 | 26 | ## Installing the Clients 27 | 28 | ### Downloading clients 29 | 30 | If your server install went successfully, the clients should have been uploaded 31 | to the server with working configurations, and should be available in 32 | the Admin UI. 33 | 34 | Click on "Binaries" in the left menu. The files should be in the 35 | executables directory, under installers. 36 | 37 | ![Screenshot](../images/client-installers.png) 38 | 39 | If your server configuration has changed your clients will need to be 40 | repacked with an updated config. For details see the server documentation. 41 | 42 | Installation steps differ significantly depending on the operating system so we split 43 | it into separate sections below. 44 | 45 | Run the client on the target machine as administrator: [Windows instructions](on-windows.md), [OSX instructions](on-mac-os-x.md), [Linux instructions](on-linux.md). 46 | 47 | See [How to check if a deployed client talks back to the GRR server](how-to-check-if-client-talks-back.md). 48 | 49 | ## Uninstalling GRR 50 | 51 | A quick manual on how to remove the GRR client completely from a machine is included in the platform-specific docs: [Windows instructions](on-windows.md#uninstalling-grr), [OSX instructions](on-mac-os-x.md#uninstalling-grr), [Linux instructions](on-linux.md#uninstalling-grr) 52 | 53 | ## Notes 54 | 55 | ### Deploying at scale 56 | 57 | There shouldn’t be any special considerations for deploying GRR clients 58 | at scale. If the server can’t handle the load, the clients should 59 | happily back off and wait their turn. However, we recommend a staged 60 | rollout if possible. 61 | 62 | ### Client and Server Version Compatibility and Numbering 63 | 64 | We try hard to avoid breaking backwards compatibility for clients since 65 | upgrading can be painful, but occasionally we need to make changes that 66 | require a new client version to work. As a general rule you want to 67 | upgrade the server first, then upgrade the clients fairly soon after. 68 | 69 | Matching major/minor versions of client and server should work well 70 | together. i.e. Clients 3.1.0.0 and 3.1.6.2 should work well with servers 71 | 3.1.0.0 and 3.1.9.7 because they are all 3.1 series. We introduced this 72 | approach for the 3.1.0.0 release. 73 | 74 | For older servers and clients, matching the last digit provided similar 75 | guarantees. i.e. client 3.0.0.7 was released with server 0.3.0-7 and 76 | should work well together. 77 | -------------------------------------------------------------------------------- /deploying-grr-clients/protection.md: -------------------------------------------------------------------------------- 1 | # GRR Client Protection 2 | 3 | The open source agent does not contain protection against being disabled 4 | by administrator/root on the machine. E.g. on Windows, if an attacker 5 | stops the service, the agent will stop and will no longer be reachable. 6 | Currently, it is up to the deployer of GRR to provide more protection 7 | for the service. 8 | 9 | ## Obfuscation 10 | 11 | If every deployment in the world is running from the same location and 12 | the same code, e.g. `c:\program files\grr\grr.exe`, it becomes a pretty 13 | obvious thing for an attacker to look for and disable. Luckily the 14 | attacker has the same problem an investigator has in finding malware on 15 | a system, and we can use the same techniques to protect the client. One 16 | of the key benefits of having an open architecture is that customization 17 | of the client and server is easy, and completely within your control. 18 | 19 | For a test, or low security deployment, using the defaults open source 20 | agents is fine. However, in a secure environment we strongly recommend 21 | using some form of obfuscation. 22 | 23 | This can come in many forms, but to give some examples: 24 | 25 | - Changing service, and binary names 26 | 27 | - Changing registry keys 28 | 29 | - Obfuscating the underlying python code 30 | 31 | - Using a packer to obfuscate the resulting binary 32 | 33 | - Implementing advanced protective or obfuscation functionality such 34 | as those used in rootkits 35 | 36 | - Implementing watchers to monitor for failure of the client 37 | 38 | GRR does not include any obfuscation mechanisms by default. But we 39 | attempt to make this relatively easy by controlling the build process 40 | through the configuration file. 41 | 42 | ## Enrollment 43 | 44 | In the default setup, clients can register to the GRR server with no 45 | prior knowledge. This means that anyone who has a copy of the GRR agent, 46 | and knows the address of your GRR server can register their client to 47 | your deployment. This significantly eases deployment, and is generally 48 | considered low risk as the client has no control or trust on the server. 49 | 50 | However, it does introduce some risk, in particular: 51 | 52 | - If there are flows or hunts you deploy to the entire fleet, a 53 | malicious client may receive them. These could give away information 54 | about what you are searching for. 55 | 56 | - Clients are allowed to send some limited messages to the server 57 | without prompting, these are called Well Known flows. By default 58 | these can be used to send log messages, or errors. A malicious 59 | client using these could fill up logs and disk space. 60 | 61 | - If you have custom Well Known Flows that perform interesting 62 | actions. You need to be aware that untrusted clients can call them. 63 | Most often this could result in a DoS condition, e.g. through a 64 | client sending multiple install failure or client crash messages. 65 | 66 | In many environments this risk is unwarranted, so we suggest 67 | implementing further authorization in the Enrollment Flow using some 68 | information that only your client knows, to authenticate it before 69 | allowing it to become a registered client. 70 | 71 | **Note** that this does not give someone the ability to overwrite data from 72 | another client, as client name collisions are protected. 73 | -------------------------------------------------------------------------------- /developing-grr/codelabs/assets/client-id-on-terminal.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/developing-grr/codelabs/assets/client-id-on-terminal.png -------------------------------------------------------------------------------- /developing-grr/codelabs/assets/dummy-flow-result.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/developing-grr/codelabs/assets/dummy-flow-result.png -------------------------------------------------------------------------------- /developing-grr/codelabs/assets/how-to-add-a-client-action.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/developing-grr/codelabs/assets/how-to-add-a-client-action.png -------------------------------------------------------------------------------- /developing-grr/codelabs/assets/how-to-add-a-flow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/developing-grr/codelabs/assets/how-to-add-a-flow.png -------------------------------------------------------------------------------- /developing-grr/codelabs/assets/start-dummy-flow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/developing-grr/codelabs/assets/start-dummy-flow.png -------------------------------------------------------------------------------- /developing-grr/codelabs/index.md: -------------------------------------------------------------------------------- 1 | # GRR Codelabs 2 | 3 | [Codelabs](https://github.com/googlecodelabs/tools) are interactive instructional tutorials, which can be authored in markdown syntax (or Google Docs) using some simple formatting conventions. 4 | This doc contains the documentation and instructions you’ll need for building and publishing the GRR codelabs (and creating your own in case you choose to do so). 5 | 6 | If you're interested in authoring codelabs, create a markdown file (or a Google Doc) following the [Codelab Formatting Guide](FORMAT-GUIDE.md) and see the online [documentation page](https://github.com/googlecodelabs/tools/blob/main/claat/README.md) on how to use the [```claat```](https://github.com/googlecodelabs/tools/blob/main/claat/README.md) command line tool. 7 | 8 | Also, consider joining the [codelab-authors Google Group](https://groups.google.com/forum/#!forum/codelab-authors), which connects you with other authors and provides updates on new releases. 9 | 10 | ## Available GRR Codelabs 11 | 12 | The currently available Codelabs are the following: 13 | - [How to add a Client Action](./how-to-add-a-client-action.md), and 14 | - [How to add a Flow](./how-to-add-a-flow.md) 15 | 16 | ## Building the GRR Codelabs 17 | 18 | Run the ```claat``` command line tool to transform the markdown content in the Codelabs into one of the supported output formats. The default supported format is HTML. The ```claat``` tool also supports additional formats, see the [```claat``` documentation](https://github.com/googlecodelabs/tools/blob/main/claat/README.md) for more details. 19 | 20 | ``` 21 | claat export how-to-add-a-client-action.md 22 | # The output should look like the below: 23 | # ok how-to-add-a-client-action 24 | 25 | claat export how-to-add-a-flow.md 26 | # The output should look like the below: 27 | # ok how-to-add-a-flow 28 | ``` 29 | 30 | This will create the subdirectories with a static HTML page for each of the Codelabs. 31 | 32 | ``` 33 | ls -la how-to-add-a-client-action 34 | # The output should look like the below: 35 | # codelab.json 36 | # img 37 | # index.html 38 | 39 | ls -la how-to-add-a-flow 40 | # The output should look like the below: 41 | # codelab.json 42 | # img 43 | # index.html 44 | ``` 45 | 46 | If you're interested in altering the Codelabs then follow the instructions in the [Codelab Formatting Guide](https://github.com/googlecodelabs/tools/blob/main/FORMAT-GUIDE.md). 47 | 48 | ## Serving and accessing the GRR Codelabs 49 | 50 | You can serve the exported Codelabs on any webserver. 51 | 52 | However, for your convenience the ```claat``` tool also ships with a simple webserver to get you started. 53 | Use the command below to serve the GRR Codelabs. 54 | 55 | ``` 56 | claat serve 57 | # The output should look like the below: 58 | # Serving codelabs on localhost:9090, opening browser tab now... 59 | ``` 60 | 61 | You can now access the Codelabs by pointing your browser to [http://localhost:9090](http://localhost:9090) 62 | 63 | Pointing your browser to [http://localhost:9090/how-to-add-a-client-action/index.html](http://localhost:9090/how-to-add-a-client-action/index.html) should load the Client Action codelab depicted below. 64 | 65 | ![How to add a client action](./assets/how-to-add-a-client-action.png "How to add a client action") 66 | 67 | When pointing your browser to [http://localhost:9090/how-to-add-a-flow/index.html](http://localhost:9090/how-to-add-a-flow/index.html) you should see the Flow codelab depicted below. 68 | 69 | ![How to add a flow](./assets/how-to-add-a-flow.png "How to add a flow") 70 | -------------------------------------------------------------------------------- /developing-grr/contributing-docs.md: -------------------------------------------------------------------------------- 1 | # Contributing Documentation 2 | 3 | GRR's documentation is stored on [GitHub](https://github.com/google/grr-doc) and is hosted on [readthedocs](https://grr-doc.readthedocs.io). 4 | 5 | ## Documentation repository structure 6 | 7 | *master* branch of [github.com/google/grr-doc](https://github.com/google/grr-doc) contains the HEAD version of GRR documentation. Effectively, it has the documentation that will be included into the next GRR release. 8 | 9 | Every time a release is made, we create a new documentation branch. I.e. we have such branches as *v3.2.0*, *v3.2.1*, etc. 10 | 11 | >**NOTE:** whenever a change in *master* branch is done, it should be applied to the latest release branch (unless the change is not backwards-compatible). Reasoning: latest release branch is what GRR users see on readthedocs. 12 | 13 | ## Contributing small documentation changes 14 | 15 | If a suggested change doesn't include any new files, the easiest way to submit it is to edit the docs directly on GitHub. GitHub will then create a pull request for you, and then one of GRR team members can review the pull request and merge it into the master branch and latest release branch. 16 | 17 | ## Setting up editing environment with Sphinx 18 | 19 | Readthedocs.io uses Sphinx to render the documentation. In order to have a live preview of the changes done to a local grr-doc clone, one should do the following : 20 | 21 | ```bash 22 | mkdir ~/tmp/grr-doc 23 | cd ~/tmp/grr-doc 24 | 25 | virtualenv . 26 | source bin/activate 27 | 28 | git clone https://github.com/google/grr-doc 29 | 30 | pip install -r grr-doc/requirements.txt 31 | 32 | sphinx-autobuild grr-doc out 33 | ``` 34 | 35 | The code above should start the Sphinx documentation server at http://127.0.0.1:8000/. The server will monitor documentation file for changes and update instantly. 36 | 37 | ## Using Visual Studio Code as an editing tool 38 | 39 | GRR documentation should be written in Markdown, so you can write it using any text editor you prefer. However, Visual Studio Code has a builtin support for Markdown and makes editing it really easy: you have a preview window in a separate pane that is updated as you type. 40 | 41 | To give VSCode a try, do the following: 42 | 1. Download and install it from [https://code.visualstudio.com/download](https://code.visualstudio.com/download). 43 | 1. Launch VSCode. Click on File->Open Folder. Point the dialog to checked-out grr-doc sources. 44 | 1. Click on any of the Markdown files. Then press Ctrl-K V (this means press Ctrl + K, then separately, after Ctrl + K is not pressed anymore, press V). 45 | 1. Now if you edit the Markdown file the preview pane is going to be updated automatically. 46 | -------------------------------------------------------------------------------- /developing-grr/contributing.md: -------------------------------------------------------------------------------- 1 | # Contributing Code 2 | 3 | GRR is a somewhat complex piece of code. While this complexity is 4 | necessary to provide the scale and performance characteristics we need, 5 | it makes it more difficult to get involved and understand the code base. 6 | So just be aware that making significant changes in the core will be 7 | difficult if you don’t have a software engineering background, or solid 8 | experience with python that includes use of things like pdb and 9 | profiling. 10 | 11 | That said, we want your contribution\! You don’t need to be a veteran 12 | pythonista to contribute artifacts or parsers. But whatever your 13 | experience, we strongly recommend starting somewhere simple before 14 | embarking on core changes and reading our documentation. In particular 15 | we recommend these as good starting points: 16 | 17 | - Build a standalone console script to perform the actions you want. A 18 | standalone console script won’t benefit from being able to be run as 19 | a Collector or a Hunt, but it will get you familiar with the API, 20 | and an experienced developer can convert it into a fully fledged 21 | flow afterwards. 22 | 23 | - Add to Artifacts or an existing flow. Many flows could do with work 24 | to detect more things or support additional platforms. 25 | 26 | - Add a new parser to parse a new filetype, e.g. if you have a 27 | different Anti-virus or HIDS log you want to parse. 28 | 29 | ## Contributor License Agreement 30 | 31 | GRR is an opensource project released under the [Apache 32 | License](https://github.com/google/grr/blob/master/LICENSE) and you should feel 33 | free to use it in any way compatible with this. However, in order to accept 34 | changes into the GRR mainline repository we must ask that keep a signed a 35 | [Google Contributor License Agreement](https://cla.developers.google.com/clas) 36 | on file. 37 | 38 | ## Style 39 | 40 | The [Chromium](http://www.chromium.org/developers/contributing-code) and 41 | [Plaso](http://plaso.kiddaland.net/developer/style-guide) projects have 42 | some good general advice about code contributions that is worth reading. 43 | In particular, make sure you’re communicating via the dev list before 44 | you get too far into a feature or bug, it’s possible we’re writing 45 | something similar or have already fixed the bug. 46 | 47 | Code needs to conform to the [Google Python Style 48 | Guide](http://google.github.io/styleguide/pyguide.html). 49 | Note that despite what the guide says, we use two-space indentation, and 50 | MixedCase instead of lower\_underscore for function names since this is 51 | the internal standard. Two-space indentation also applies to CSS. 52 | 53 | ## Setup 54 | 55 | We use the github [fork and pull review 56 | process](https://help.github.com/articles/using-pull-requests) to review 57 | all contributions. 58 | 59 | First, fork the [GRR repository](https://github.com/google/grr) by 60 | following [the github 61 | instructions](https://help.github.com/articles/fork-a-repo). 62 | 63 | Now that you have a github.com/your-username/grr repository: 64 | 65 | # Make a new branch for the bug/feature 66 | $ git checkout -b my_shiny_feature 67 | 68 | # Make your changes, add any new files 69 | $ git add newmodule.py newmodule_test.py 70 | 71 | When you’re ready for review, [sync your branch with 72 | upstream](https://help.github.com/articles/syncing-a-fork): 73 | 74 | $ git fetch upstream 75 | $ git merge upstream/master 76 | 77 | # Fix any conflicts and commit your changes 78 | $ git commit -a 79 | $ git push origin HEAD 80 | 81 | Use the GitHub Web UI to [create and send the pull 82 | request](https://help.github.com/articles/using-pull-requests). We’ll 83 | review the change. 84 | 85 | # Make review changes 86 | $ git commit -a 87 | $ git push origin HEAD 88 | 89 | Once we’re done with review we’ll commit the pull request. 90 | -------------------------------------------------------------------------------- /developing-grr/implementation-details/authorizating-and-auditing.md: -------------------------------------------------------------------------------- 1 | # Authorization and Auditing 2 | 3 | GRR contains an audit framework that logs all requests coming through the GRR 4 | API (the admin UI also uses said API as the backend so all requests through the 5 | UI are logged too). The username that is logged is generated by the chosen 6 | WebAuthManager (in practice often the BasicWebAuthManager which uses basic http 7 | authentication). 8 | 9 | ## Reading the Audit Log 10 | 11 | The audit log is currently not surfaced in any UI. The following code snippet 12 | illustrated hot to read the audit log through the GRR console: 13 | 14 | ```python 15 | end_time = rdfvalue.RDFDatetime.Now() 16 | start_time = end_time - rdfvalue.Duration("52w") 17 | 18 | result = data_store.REL_DB.ReadAPIAuditEntries(min_timestamp=start_time, 19 | max_timestamp=end_time) 20 | 21 | print result[0] 22 | 23 | message APIAuditEntry { 24 | http_request_path : u'/api/clients/C.071d6e8cc079bf11/flows/33D85D32?' 25 | response_code : OK 26 | router_method_name : u'GetFlow' 27 | timestamp : RDFDatetime: 28 | 2019-05-13 14:54:19 29 | username : u'amoser' 30 | } 31 | 32 | ``` 33 | -------------------------------------------------------------------------------- /developing-grr/implementation-details/foreman.md: -------------------------------------------------------------------------------- 1 | # Foreman 2 | 3 | The Foreman is a client scheduling service. At a regular intervals 4 | (defaults to every 30 minutes) the client will report in asking if there 5 | are Foreman actions for it. At the time of this check in, the Foreman 6 | will be queried to decide if there are any jobs that match the host, if 7 | there are, appropriate flows will be created for the client. This 8 | mechanism is generally used by Hunts to schedule flows on a large number 9 | of clients. 10 | 11 | The foreman maintains a list of rules, if the rule matches a client when 12 | it checks in, the specified flow will execute on the client. The rules 13 | work against client attributes allowing for things like "All XP Machines" 14 | or "All OSX machines installed after 01.01.2011". For more information 15 | see the section about [hunt rules](../../../investigating-with-grr/hunts/rules/). 16 | 17 | The foreman check-in request is a special request made by the client. When the 18 | server sees this request it does the following: 19 | 20 | 1. Determines how long since this client did a Foreman check-in. 21 | 22 | 2. Determines the set of rules that are non-expired and haven’t 23 | previously been checked by the client. 24 | 25 | 3. Matches those rules against the client’s attributes to determine if 26 | there is a match. 27 | 28 | 4. If there is a match, run the associated flow. 29 | 30 | The reason for the separate Foreman check-in request is that the rule 31 | matching can be expensive when you have a lot of clients, so having 32 | these less frequent saves a lot of processing. 33 | -------------------------------------------------------------------------------- /developing-grr/implementation-details/grr-messages.md: -------------------------------------------------------------------------------- 1 | # GRR Messages 2 | 3 | On the wire, the client and server interchange messages. We term the 4 | messages sent from server to the client *requests*, while messages sent 5 | from the client to the server are *responses*. Requests sent to the 6 | client ask the client to perform some action, for example 7 | *ListDirectory*. We term these actions the *client action*. A single 8 | request may elicit multiple responses. 9 | 10 | For example, the request message *ListDirectory* will elicit a response 11 | for each file in the directory (potentially thousands). Requests and 12 | responses are tracked using an incremental *request\_id* and 13 | *response\_id*. 14 | 15 | In order to indicate when all responses have been sent for a particular 16 | request, the client sends a special STATUS message as the last response. 17 | In the event of errors, the message contains details of the error 18 | (including backtrace). If the action completed successfully, an OK 19 | status is returned. 20 | 21 | Messages are encoded as GrrMessage protobufs: 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 |
Important GRR Message protobuf fields.

session_id

A unique integer for all packets generated by this flow.

name

The name of the Action to be called on the client (See below).

args

A serialized protobuf which will be interpreted by the Action.

request_id

An incrementing number of this request (see below)

response_id

An incrementing number of the response (see below)

52 | 53 | ![Typical Message Request/Response Sequence.](../../images/messages.png 54 | "fig:") 55 | 56 | Figure 2 illustrates a typical sequence of messages. Request 1 was sent 57 | from the server to the client, and elicited 3 responses, in addition to 58 | a status message. 59 | 60 | When the server sends the client messages, the messages are tagged in 61 | the data store with a lease time. If the client does not reply for these 62 | requests within the lease time, the requests become available for lease 63 | again. This is designed for the case of the client rebooting or losing 64 | connectivity part way through running the action. In this case, the 65 | request is re-transmitted and the action is run again. 66 | -------------------------------------------------------------------------------- /developing-grr/implementation-details/index.md: -------------------------------------------------------------------------------- 1 | # Implementation details 2 | 3 | ```{toctree} 4 | --- 5 | maxdepth: 1 6 | --- 7 | GRR messages 8 | Client Path Specification 9 | Foreman 10 | Authorization and Auditing 11 | ``` 12 | -------------------------------------------------------------------------------- /developing-grr/index.md: -------------------------------------------------------------------------------- 1 | # Developing GRR 2 | 3 | ```{toctree} 4 | --- 5 | maxdepth: 1 6 | --- 7 | Setting up development environment 8 | Setting up a Docker Compose development environment 9 | Running the tests 10 | Contributing code 11 | Contributing documentation 12 | Implementation details 13 | Codelabs 14 | GRR Workstation 15 | ``` 16 | -------------------------------------------------------------------------------- /developing-grr/running-tests.md: -------------------------------------------------------------------------------- 1 | # Running the tests 2 | 3 | For running tests GRR uses the [pytest](https://pytest.org) framework. 4 | 5 | ### Prerequisites 6 | 7 | Make sure you have correctly set-up the 8 | [development environment](setting-up-dev-env.md), especially the part about 9 | installing the `grr-response-test` package. 10 | 11 | This setup is sufficient for running most of the tests. However, GRR also has 12 | a UI component written in [Angular](https://angular.io/). For testing that part 13 | we use [Selenium](http://seleniumhq.org) and 14 | [ChromeDriver](https://sites.google.com/a/chromium.org/chromedriver/) which need 15 | to be installed first. You can skip this part if you do not need to execute UI 16 | tests but we strongly recommend to run them as well. 17 | 18 | On Debian-based distros simply install `chromium-driver` package: 19 | 20 | ```bash 21 | sudo apt install chromium-driver 22 | ``` 23 | 24 | If there is no `chromium-driver` available in your repositories you may try 25 | installing Chromium browser and then downloading latest `chromium-driver` binary 26 | from [the official website](https://sites.google.com/a/chromium.org/chromedriver/downloads). 27 | After downloading unpack it somewhere and add it to your `$PATH` or just move it 28 | to `/usr/bin`. 29 | 30 | ### Running the whole test suite 31 | 32 | To run all the tests, navigate to the root GRR directory and simply execute: 33 | 34 | ```bash 35 | pytest 36 | ``` 37 | 38 | This will automatically discover and execute all test cases. 39 | 40 | ### Running tests in parallel 41 | 42 | To use pytest to run tests in parallel, install the pytest-xdist plugin 43 | 44 | ```docker 45 | pip install pytest-xdist 46 | ``` 47 | 48 | and run 49 | 50 | ```docker 51 | pytest -n 52 | ``` 53 | 54 | ### Running the tests selectively 55 | 56 | Running all the tests is reasonable when you want to test everything before 57 | publishing your code, but it is not feasible during development. Usually you 58 | just want to execute only tests in a particular directory, like so: 59 | 60 | ```bash 61 | pytest grr/server/aff4_objects 62 | ``` 63 | 64 | Or just a particular file: 65 | 66 | ```bash 67 | pytest grr/server/aff4_objects/filestore_test.py 68 | ``` 69 | 70 | Or just a particular test class: 71 | 72 | ```bash 73 | pytest grr/server/aff4_objects/filestore_test.py::HashFileStoreTest 74 | ``` 75 | 76 | Or even just a single test method: 77 | 78 | ```docker 79 | pytest grr/server/aff4_objects/filestore_test.py::HashFileStoreTest::testListHashes 80 | ``` 81 | 82 | ### Ignoring test cases 83 | 84 | Some kind of tests are particularly slow to run. For example, all UI tests are 85 | based on running a real web browser instance and simulating its actions which is 86 | painfully sluggish and can be very annoying if we want to test non-UI code. 87 | 88 | In this case we can skip all tests in particular directory using the `--ignore` 89 | flag, e.g.: 90 | 91 | ```bash 92 | pytest --ignore=grr/gui/selenium_tests 93 | ``` 94 | 95 | This will run all the tests except the Selenium ones. 96 | 97 | ### Benchmarks 98 | 99 | Benchmarks are not really testing the code correctness so there is no point in 100 | running them every time we want to publish our code. This is why the test suite 101 | will not run them by default. However, sometimes they can be useful for testing 102 | the performance and sanity of our system. 103 | 104 | In order to run the test suite including the benchmarks, pass the `--benchmark` 105 | option to the test runner: 106 | 107 | ```bash 108 | pytest --benchmark 109 | ``` 110 | 111 | ### Debugging 112 | 113 | If our tests are failing and we need to fix our code the 114 | [Python Debugger](https://docs.python.org/3/library/pdb.html) can come in handy. 115 | 116 | If you run pytest with `--pdb` flag then upon a failure the program execution 117 | will be halted and you will be dropped into the PDB shell where you can 118 | investigate the problem in the environment it occurred. 119 | 120 | If you set breakpoints in your code manually using `pdb.set_trace()` you will 121 | notice a weird behaviour when running your tests. This is because pytest 122 | intercepts the standard input and output writes, breaking the PDB shell. To deal 123 | with this behaviour simply run tests with `-s` flag - it will prevent pytest 124 | from doing that. 125 | 126 | ### More information 127 | 128 | The functionalities outlined in this guide are just a tip of the pytest 129 | capabilities. For more information consult pytest's man page, check 130 | `pytest --help` or visit the [pytest homepage](https://pytest.org). 131 | -------------------------------------------------------------------------------- /developing-grr/workstation/Dockerfile: -------------------------------------------------------------------------------- 1 | # Copyright 2024 Google LLC 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | FROM europe-west1-docker.pkg.dev/cloud-workstations-images/predefined/code-oss:latest 15 | 16 | ENV DEBIAN_FRONTEND=noninteractive 17 | 18 | RUN wget https://github.com/googlecodelabs/tools/releases/download/v2.2.6/claat-linux-amd64 && \ 19 | mv claat-linux-amd64 /usr/local/bin/claat && \ 20 | chmod 755 /usr/local/bin/claat 21 | 22 | RUN mkdir -p /var/src && \ 23 | cd /var/src && \ 24 | git clone https://github.com/google/grr.git && \ 25 | git config --global --add safe.directory /var/src/grr && \ 26 | git clone https://github.com/google/grr-doc.git && \ 27 | git config --global --add safe.directory /var/src/grr-doc 28 | 29 | COPY ./workstation_startup_d/011_mount_filestore.sh /etc/workstation-startup.d/ 30 | RUN chmod +x /etc/workstation-startup.d/011_mount_filestore.sh 31 | 32 | COPY ./workstation_startup_d/200_install-demo.sh /etc/workstation-startup.d/ 33 | RUN chmod +x /etc/workstation-startup.d/200_install-demo.sh 34 | 35 | COPY ./settings.json /demo/ 36 | COPY ./workstation_startup_d/250_default-ide-color-theme.sh /etc/workstation-startup.d/ 37 | RUN chmod +x /etc/workstation-startup.d/250_default-ide-color-theme.sh 38 | -------------------------------------------------------------------------------- /developing-grr/workstation/index.md: -------------------------------------------------------------------------------- 1 | # Creating a GRR Workstation - Code OSS based development environment 2 | 3 | ## Introduction 4 | 5 | This document covers how you can create a GRR (Cloud) Workstation [container image](https://cloud.google.com/workstations/docs/customize-container-images) with a Code OSS based development environment that you can run either as a [Google Cloud Workstation](https://cloud.google.com/workstations/docs/overview) or locally on your computer in case you have Docker installed. 6 | 7 | Follow the instructions below to: 8 | 1. Build the container image, 9 | 2. Run the container image as a Cloud Workstation on Google Cloud, or 10 | 3. Run the container image on your computer locally. 11 | 12 | ## 1. Building the GRR Workstation container image 13 | 14 | The GRR Workstation consist of a container image that bundles the Code OSS development environment, the GRR source code repo, the codelabs and some useful tooling that you can leverage when working on the GRR source code. 15 | Refer to the online documentation for more background about [Google Cloud Workstations](https://cloud.google.com/workstations/docs/overview) and how to build a [customised container image](https://cloud.google.com/workstations/docs/customize-container-images). 16 | 17 | The ```Dockerfile``` in this directory contains all the instructions to build your own container image. 18 | 19 | To build the GRR Workstation container image rund the command below: 20 | 21 | ``` 22 | docker build -t grr-station . 23 | ``` 24 | 25 | Note 26 | * The Cloud Workstation setup takes advantage of mounting a [Cloud Firestore](https://cloud.google.com/filestore/docs/overview) shared (NFS) directory that can be used for sharing files between workstations (i.e. for convenient collaboration between team members). 27 | * The setup works fine without that shared Cloud Filestore directory in case you do not require that feature you need to do nothing extra. 28 | * However, in case you want to take advantage of the shared directory then follow the instructions in the [Cloud Filestore online documentation](https://cloud.google.com/filestore/docs/creating-instances) to get a Filestore created that can be mounted into the Cloud Workstations. 29 | * Make sure you set the following two environment variables when you [create your workstation configuration](https://cloud.google.com/workstations/docs/create-configuration). 30 | * ```FILESTORE_INSTANCE_IP: xxx.xxx.xxx.xxx``` 31 | * ```FILESTORE_SHARE_NAME: workstations``` 32 | 33 | ## 2. Run the GRR Workstation on Google Cloud 34 | 35 | Refer to the [Google Cloud online documentation](https://cloud.google.com/workstations/docs/quickstart-set-up-workstations-console) for how to set up workstation clusters, workstation configuration, and workstation. Check the prerequisites [before you begin](https://cloud.google.com/workstations/docs/quickstart-set-up-workstations-console#before-you-begin), and then follow these steps: 36 | 37 | 1. [Create a workstation cluster](https://cloud.google.com/workstations/docs/quickstart-set-up-workstations-console#create_a_workstation_cluster) 38 | 2. [Create a workstation configuration](https://cloud.google.com/workstations/docs/quickstart-set-up-workstations-console#create_a_workstation_configuration), make sure you select your ```grr-station``` container image when you create your configuration. 39 | 40 | ![grr_ws_config](../../images/grr_ws_config.png) 41 | 42 | 3. [Create and launch a workstation](https://cloud.google.com/workstations/docs/quickstart-set-up-workstations-console#create_and_launch_a_workstation) 43 | 44 | ## 3. Run the GRR Workstation locally 45 | 46 | In case you would like to run the GRR Workstation on your computer locally you can do so with the command below. 47 | 48 | Please note that your mileage might vary depending on the spec of your machine. 49 | 50 | ``` 51 | docker run --rm -d --env='DOCKER_OPTS=' --volume=/var/lib/docker --privileged \ 52 | --name grr-station -p 8080:80 -p 8000:8000 -p9090:9090 grr-station 53 | ``` 54 | 55 | You can now connect to the 56 | - GRR Workstation by pointing your browser to [http://localhost:8080](http://localhost:8080) 57 | - GRR itself by pointing your browser to [http://localhost:8000](http://localhost:8000) 58 | - Codelabs by pointing your browser to [http://localhost:9090](http://localhost:9090) 59 | -------------------------------------------------------------------------------- /developing-grr/workstation/settings.json: -------------------------------------------------------------------------------- 1 | { 2 | "window.autoDetectColorScheme": true, 3 | "workbench.colorTheme": "Code OSS Dark" 4 | } 5 | -------------------------------------------------------------------------------- /developing-grr/workstation/workstation_startup_d/011_mount_filestore.sh: -------------------------------------------------------------------------------- 1 | # Copyright 2024 Google LLC 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | #!/bin/bash 15 | 16 | # https://cloud.google.com/workstations/docs/mount-filestore-nfs-instances 17 | 18 | if [ -z ${FILESTORE_INSTANCE_IP} ]; then 19 | echo "The FILESTORE_INSTANCE_IP environment variable is empty" 20 | exit 21 | fi 22 | 23 | if [ -z ${FILESTORE_SHARE_NAME} ]; then 24 | echo "The FILESTORE_SHARE_NAME environment variable is empty" 25 | exit 26 | fi 27 | 28 | sudo rpcbind 29 | sudo mkdir -p /home/user/shared_cache 30 | sudo mount -o rw,intr ${FILESTORE_INSTANCE_IP}:/${FILESTORE_SHARE_NAME} /home/user/shared_cache 31 | sudo chmod go+rw /home/user/shared_cache 32 | -------------------------------------------------------------------------------- /developing-grr/workstation/workstation_startup_d/200_install-demo.sh: -------------------------------------------------------------------------------- 1 | # Copyright 2024 Google LLC 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | #!/bin/bash 15 | # shellcheck disable=SC1091 16 | export USER=user 17 | BASE=$(pwd) 18 | export BASE 19 | export LC_ALL=C.UTF-8 20 | echo "-----------------" 21 | echo "Waiting for Docker to start" 22 | while (! docker stats --no-stream ); do 23 | # Docker takes a few seconds to initialize 24 | echo "Waiting for Docker to launch..." 25 | sleep 1 26 | done 27 | echo "-----------------" 28 | echo "Synchronising repos..." 29 | if [ ! -f /home/user/src/grr/README.md ]; then 30 | cp -R /var/src /home/user/src 31 | chown -R user:user /home/user/src 32 | git config --global --add safe.directory /home/user/src/grr 33 | git config --global --add safe.directory /home/user/src/grr-doc 34 | fi 35 | echo "-----------------" 36 | echo "Pulling grr..." 37 | cd /home/user/src/grr/ 38 | git pull 39 | echo "-----------------" 40 | echo "Pulling grr-doc..." 41 | cd /home/user/src/grr-doc/ 42 | git pull 43 | echo "-----------------" 44 | echo "Starting claat..." 45 | cd /home/user/src/grr-doc/developing-grr/codelabs 46 | claat export how-to-add-a-client-action.md 47 | claat export how-to-add-a-flow.md 48 | claat serve -addr 0.0.0.0:9090 & 49 | echo "-----------------" 50 | cd "$BASE" || exit 51 | echo "demo installation done" 52 | echo "-----------------" 53 | -------------------------------------------------------------------------------- /developing-grr/workstation/workstation_startup_d/250_default-ide-color-theme.sh: -------------------------------------------------------------------------------- 1 | # Copyright 2024 Google LLC 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | #!/bin/bash 15 | cd /home/user/.codeoss-cloudworkstations/data/Machine/ 16 | if [ -f settings.json ]; then 17 | cat <<< $(jq '. += {"window.autoDetectColorScheme": true}' settings.json) > settings.json 18 | cat <<< $(jq '. += {"workbench.colorTheme": "Default Dark Modern"}' settings.json) > settings.json 19 | else 20 | cp /demo/settings.json . 21 | chown user:user settings.json 22 | fi 23 | cd - 24 | -------------------------------------------------------------------------------- /images/client-approval-dialog.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/client-approval-dialog.png -------------------------------------------------------------------------------- /images/client-approval-review-page.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/client-approval-review-page.png -------------------------------------------------------------------------------- /images/client-installers.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/client-installers.png -------------------------------------------------------------------------------- /images/client_view.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/client_view.png -------------------------------------------------------------------------------- /images/collect_from_client.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/collect_from_client.png -------------------------------------------------------------------------------- /images/collected_from_client.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/collected_from_client.png -------------------------------------------------------------------------------- /images/create_hunt_from_flow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/create_hunt_from_flow.png -------------------------------------------------------------------------------- /images/cron_job_viewer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/cron_job_viewer.png -------------------------------------------------------------------------------- /images/distributed_fleetspeak.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/distributed_fleetspeak.png -------------------------------------------------------------------------------- /images/download.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/download.png -------------------------------------------------------------------------------- /images/downloaded_file.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/downloaded_file.png -------------------------------------------------------------------------------- /images/fleetspeak_database.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/fleetspeak_database.png -------------------------------------------------------------------------------- /images/flow_details.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/flow_details.png -------------------------------------------------------------------------------- /images/flow_results.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/flow_results.png -------------------------------------------------------------------------------- /images/grr_certs_keys.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_certs_keys.png -------------------------------------------------------------------------------- /images/grr_changes_applied.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_changes_applied.png -------------------------------------------------------------------------------- /images/grr_claat.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_claat.png -------------------------------------------------------------------------------- /images/grr_client_details.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_client_details.png -------------------------------------------------------------------------------- /images/grr_clients.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_clients.png -------------------------------------------------------------------------------- /images/grr_code_explore.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_code_explore.png -------------------------------------------------------------------------------- /images/grr_code_menu.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_code_menu.png -------------------------------------------------------------------------------- /images/grr_code_oss.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_code_oss.png -------------------------------------------------------------------------------- /images/grr_code_terminal.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_code_terminal.png -------------------------------------------------------------------------------- /images/grr_codelab_client.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_codelab_client.png -------------------------------------------------------------------------------- /images/grr_codelab_flow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_codelab_flow.png -------------------------------------------------------------------------------- /images/grr_components.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_components.png -------------------------------------------------------------------------------- /images/grr_components_up.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_components_up.png -------------------------------------------------------------------------------- /images/grr_compose_up.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_compose_up.png -------------------------------------------------------------------------------- /images/grr_database.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_database.png -------------------------------------------------------------------------------- /images/grr_debug_mode.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_debug_mode.png -------------------------------------------------------------------------------- /images/grr_dev_ready.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_dev_ready.png -------------------------------------------------------------------------------- /images/grr_development.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_development.png -------------------------------------------------------------------------------- /images/grr_flow_success.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_flow_success.png -------------------------------------------------------------------------------- /images/grr_logo_real_sm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_logo_real_sm.png -------------------------------------------------------------------------------- /images/grr_run_flow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_run_flow.png -------------------------------------------------------------------------------- /images/grr_ws_config.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/grr_ws_config.png -------------------------------------------------------------------------------- /images/hexview.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/hexview.png -------------------------------------------------------------------------------- /images/hostname_multiple_versions.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/hostname_multiple_versions.png -------------------------------------------------------------------------------- /images/how_to_check_if_client_talks_back.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/how_to_check_if_client_talks_back.png -------------------------------------------------------------------------------- /images/hunt_rules.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/hunt_rules.png -------------------------------------------------------------------------------- /images/icons/copy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/icons/copy.png -------------------------------------------------------------------------------- /images/icons/editdelete.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/icons/editdelete.png -------------------------------------------------------------------------------- /images/icons/modify.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/icons/modify.png -------------------------------------------------------------------------------- /images/icons/new.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/icons/new.png -------------------------------------------------------------------------------- /images/icons/pause_button.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/icons/pause_button.png -------------------------------------------------------------------------------- /images/icons/play_button.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/icons/play_button.png -------------------------------------------------------------------------------- /images/icons/play_force_button.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/icons/play_force_button.png -------------------------------------------------------------------------------- /images/icons/robot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/icons/robot.png -------------------------------------------------------------------------------- /images/icons/stop.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/icons/stop.png -------------------------------------------------------------------------------- /images/manage_flows.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/manage_flows.png -------------------------------------------------------------------------------- /images/rapid-hunting-eligible.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/rapid-hunting-eligible.png -------------------------------------------------------------------------------- /images/refresh_buttons.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/refresh_buttons.png -------------------------------------------------------------------------------- /images/sandbox_fleetspeak.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/sandbox_fleetspeak.png -------------------------------------------------------------------------------- /images/settings_view.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/settings_view.png -------------------------------------------------------------------------------- /images/signed_url_impersonation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/signed_url_impersonation.png -------------------------------------------------------------------------------- /images/signed_url_overview.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/signed_url_overview.png -------------------------------------------------------------------------------- /images/signed_url_sa_key.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/signed_url_sa_key.png -------------------------------------------------------------------------------- /images/signed_url_workload_identity.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/signed_url_workload_identity.png -------------------------------------------------------------------------------- /images/stat_info.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/stat_info.png -------------------------------------------------------------------------------- /images/timeline.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/timeline.png -------------------------------------------------------------------------------- /images/vfs_full.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/images/vfs_full.png -------------------------------------------------------------------------------- /index.md: -------------------------------------------------------------------------------- 1 | ![GRR logo](images/grr_logo_real_sm.png) 2 | 3 | GRR Rapid Response is an incident response framework focused on remote live forensics. 4 | 5 | It consists of a python client (agent) that is installed on target systems, and python server infrastructure that can manage and talk to clients. 6 | 7 | The goal of GRR is to support forensics and investigations in a fast, scalable manner to allow analysts to quickly triage attacks and perform analysis remotely. 8 | 9 | ## GRR on GitHub 10 | 11 | GRR is open source (Apache License 2.0) and is developed on GitHub: [github.com/google/grr](https://github.com/google/grr) 12 | 13 | ## Contacts 14 | 15 | * GitHub issues: [github.com/google/grr/issues](https://github.com/google/grr/issues) 16 | * GRR Users mailing list: [grr-users](https://groups.google.com/forum/#!forum/grr-users) 17 | * Follow us [on twitter](https://twitter.com/grrresponse) for announcements of GRR user meetups. We use a [gitter chat room](https://gitter.im/google/grr) during meetups. 18 | 19 | ## Table of contents 20 | 21 | ```{toctree} 22 | --- 23 | maxdepth: 2 24 | --- 25 | What is GRR? 26 | Quickstart 27 | Installing and Running GRR 28 | Deploying GRR clients 29 | Investigating with GRR 30 | Maintaining and tuning GRR deployment 31 | GRR and Fleetspeak setup 32 | Developing GRR 33 | Release Notes 34 | F.A.Q 35 | Publications 36 | ``` 37 | 38 | -------------------------------------------------------------------------------- /installing-and-running-grr/advanced-usecases/index.md: -------------------------------------------------------------------------------- 1 | # Advanced Usecases 2 | 3 | ```{toctree} 4 | --- 5 | maxdepth: 1 6 | --- 7 | Scaling a fleetspeak setup 8 | Running GRR and Fleetspeak with Envoy 9 | ``` 10 | -------------------------------------------------------------------------------- /installing-and-running-grr/advanced-usecases/sandbox.md: -------------------------------------------------------------------------------- 1 | # Running GRR and Fleetspeak with Envoy 2 | 3 | The following diagram shows the high level architecture of the sandbox environment. 4 | 5 | ![sandbox fleetspeak](../../images/sandbox_fleetspeak.png) 6 | 7 | This sandbox uses [Docker Compose](https://docs.docker.com/compose/) to demonstrate how to run [GRR](https://github.com/google/grr) and [Fleetspeak](https://github.com/google/fleetspeak) with [Envoy](https://www.envoyproxy.io/) leveraging Fleetspeak in the HTTPS header frontend mode. 8 | This frontend mode lends itself to showcase an example of Fleetspeak's more involved frontend modes. 9 | Fleetspeak offers several other frontend modes to run GRR/Fleetspeak in varying contexts. You can find more information about them in the [Fleetspeak repository](https://github.com/google/fleetspeak/docs/sandboxes.md). 10 | 11 | The sandbox also features the sample configurations that can be used to learn about Fleetspeak's frontend mode options and how to model your own configurations. 12 | 13 | Before you begin you will need to setup the sandbox environment. 14 | It is important that you follow all the steps in the [setup instructions](#setup-instructions) section before running the sandbox. 15 | 16 | ## Setup the sandbox environment 17 | - [Follow the "Install * " instructions for GRR](../../installing-and-running-grr/via-docker-compose.md) 18 | - [Clone the GRR Doc repository](#clone-the-grr-doc-repository) 19 | - [Create Configurations](#create-configurations) 20 | 21 | ### Clone the GRR Doc repository 22 | If you have not cloned the GRR Doc repository already, clone it with: 23 | 24 | ``` 25 | git clone https://github.com/google/grr 26 | ``` 27 | 28 | ### Create configurations 29 | 30 | ``` 31 | cd grr-doc/installing-and-running-grr/advanced-usecases/sandbox 32 | ./createConfig.sh 33 | ``` 34 | 35 | ## Run the GRR Fleetspeak sandbox demo 36 | ``` 37 | docker compose up --build -d 38 | ``` 39 | When you run the sandbox for the first time Docker will have to build the containers before running them. 40 | This will take a couple of minutes so you might want to grab a cup of coffee now. 41 | The next runs will be a lot faster. 42 | 43 | ## Connect to the GRR UI 44 | Once the containers are up and running you can connect to the GRR UI. 45 | You will have to create a ```demo``` user first. The script will ask you for a password. We suggest you use ```demo``` for simplicity. 46 | 47 | You can create the ```demo``` user by executing the command listed in the box below. 48 | ``` 49 | docker exec -it sandbox-grr-admin-1 grr_config_updater add_user demo --secondary_configs /config/grr-server/server.local.yaml 50 | ``` 51 | With the ```demo``` user in place you can now point your browser to the GRR UI: [http://localhost:8000](http://localhost:8000) 52 | 53 | ## Stop the GRR Fleetspeak sandbox demo 54 | ``` 55 | docker compose down 56 | ``` 57 | -------------------------------------------------------------------------------- /installing-and-running-grr/advanced-usecases/sandbox/config/fleetspeak-admin/components.textproto: -------------------------------------------------------------------------------- 1 | mysql_data_source_name: "fleetspeak-user:fleetspeak-password@tcp(mysql-fleetspeak:3306)/fleetspeak" 2 | admin_config: < 3 | listen_address: "0.0.0.0:9091" 4 | > 5 | notification_use_http_notifier: true 6 | -------------------------------------------------------------------------------- /installing-and-running-grr/advanced-usecases/sandbox/config/fleetspeak-admin/services.textproto: -------------------------------------------------------------------------------- 1 | services { 2 | name: "GRR" 3 | factory: "GRPC" 4 | config { 5 | [type.googleapis.com/fleetspeak.grpcservice.Config] { 6 | target: "grr-frontend:11111" 7 | insecure: true 8 | } 9 | } 10 | } 11 | broadcast_poll_time { 12 | seconds: 1 13 | } 14 | -------------------------------------------------------------------------------- /installing-and-running-grr/advanced-usecases/sandbox/config/fleetspeak-frontend/components.textproto: -------------------------------------------------------------------------------- 1 | mysql_data_source_name: "fleetspeak-user:fleetspeak-password@tcp(mysql-fleetspeak:3306)/fleetspeak" 2 | https_config: < 3 | listen_address: "0.0.0.0:9090" 4 | certificates: "FLEETSPEAK_CERT" 5 | key: "FLEETSPEAK_KEY" 6 | frontend_config: < 7 | https_header_checksum_config: < 8 | client_certificate_header: "x-client-certificate" 9 | client_certificate_checksum_header: "x-client-cert-hash" 10 | > 11 | > 12 | > 13 | health_check_config: < 14 | listen_address: "0.0.0.0:8080" 15 | > 16 | notification_listen_address: "0.0.0.0:12000" 17 | notification_public_address: "fleetspeak-frontend:12000" 18 | -------------------------------------------------------------------------------- /installing-and-running-grr/advanced-usecases/sandbox/config/fleetspeak-frontend/services.textproto: -------------------------------------------------------------------------------- 1 | services { 2 | name: "GRR" 3 | factory: "GRPC" 4 | config { 5 | [type.googleapis.com/fleetspeak.grpcservice.Config] { 6 | target: "grr-frontend:11111" 7 | insecure: true 8 | } 9 | } 10 | } 11 | broadcast_poll_time { 12 | seconds: 1 13 | } 14 | -------------------------------------------------------------------------------- /installing-and-running-grr/advanced-usecases/sandbox/config/grr-client/client.yaml: -------------------------------------------------------------------------------- 1 | Client.fleetspeak_enabled: true 2 | ClientBuilder.fleetspeak_bundled: true 3 | Client.foreman_check_frequency: 30 4 | 5 | Logging.verbose: true 6 | Logging.engines: file,stderr 7 | Logging.path: / 8 | Logging.filename: /grr-client.log 9 | 10 | Client.executable_signing_public_key: "%(/config/public-key.pem|file)" 11 | Target:Linux: 12 | ClientBuilder.fleetspeak_client_config: /config/grr-client/config.textproto 13 | Target:Windows: 14 | ClientBuilder.fleetspeak_client_config: /config/grr-client/config.textproto 15 | Target:Darwin: 16 | ClientBuilder.fleetspeak_client_config: /config/grr-client/config.textproto 17 | -------------------------------------------------------------------------------- /installing-and-running-grr/advanced-usecases/sandbox/config/grr-client/communicator.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/installing-and-running-grr/advanced-usecases/sandbox/config/grr-client/communicator.txt -------------------------------------------------------------------------------- /installing-and-running-grr/advanced-usecases/sandbox/config/grr-client/config.textproto: -------------------------------------------------------------------------------- 1 | client_certificate_header: "x-client-certificate" 2 | trusted_certs: "TRUSTED_FLEETSPEAK_CERT" 3 | server: "fleetspeak-server:10000" 4 | client_label: "" 5 | filesystem_handler: < 6 | configuration_directory: "/config/grr-client/" 7 | state_file: "/fleetspeak.state" 8 | > 9 | streaming: true 10 | -------------------------------------------------------------------------------- /installing-and-running-grr/advanced-usecases/sandbox/config/grr-client/textservices/grr.service: -------------------------------------------------------------------------------- 1 | name: "GRR" 2 | factory: "Daemon" 3 | config: { 4 | [type.googleapis.com/fleetspeak.daemonservice.Config]: { 5 | argv: "python" 6 | argv: "-m" 7 | argv: "grr_response_client.client" 8 | argv: "--secondary_configs" 9 | argv: "/config/grr-client/client.yaml" 10 | } 11 | } 12 | -------------------------------------------------------------------------------- /installing-and-running-grr/advanced-usecases/sandbox/config/grr-server/server.local.yaml: -------------------------------------------------------------------------------- 1 | Database.implementation: MysqlDB 2 | Blobstore.implementation: DbBlobStore 3 | 4 | API.DefaultRouter: ApiCallRouterWithoutChecks 5 | 6 | Server.initialized: true 7 | Server.fleetspeak_enabled: true 8 | Server.fleetspeak_server: fleetspeak-admin:9091 9 | FleetspeakFrontend Context: 10 | Server.fleetspeak_message_listen_address: 0.0.0.0:11111 11 | 12 | Mysql.host: mysql-grr 13 | Mysql.port: 3306 14 | Mysql.database: grr 15 | Mysql.username: grr-user 16 | Mysql.password: grr-password 17 | Mysql.database_name: grr 18 | Mysql.database_username: grr-user 19 | Mysql.database_password: grr-password 20 | Mysql.flow_processing_threads_max: 20 21 | 22 | AdminUI.url: http://localhost:8000 23 | AdminUI.bind: 0.0.0.0 24 | AdminUI.headless: true 25 | AdminUI.use_precompiled_js: true 26 | AdminUI.csrf_secret_key: random_passphrase_123____PLEASE_REPLACE___ 27 | 28 | Logging.domain: localhost 29 | Logging.verbose: true 30 | Logging.engines: file,stderr 31 | Logging.path: /grr 32 | Logging.filename: /grr/grr-server.log 33 | Monitoring.alert_email: grr-monitoring@localhost 34 | Monitoring.emergency_access_email: grr-emergency@localhost 35 | 36 | Client.executable_signing_public_key: | 37 | %(/config/public-key.pem|file) 38 | 39 | PrivateKeys.executable_signing_private_key: | 40 | %(/config/private-key.pem|file) 41 | 42 | 43 | 44 | -------------------------------------------------------------------------------- /installing-and-running-grr/advanced-usecases/sandbox/containers/envoy/Dockerfile: -------------------------------------------------------------------------------- 1 | ARG ENVOY_IMAGE="${ENVOY_IMAGE:-envoyproxy/envoy}" 2 | ARG ENVOY_VARIANT="${ENVOY_VARIANT:-dev}" 3 | 4 | 5 | FROM ${ENVOY_IMAGE}:${ENVOY_VARIANT} as envoy-base 6 | ARG ENVOY_CONFIG=envoy.yaml 7 | ENV ENVOY_CONFIG="$ENVOY_CONFIG" 8 | ENV DEBIAN_FRONTEND=noninteractive 9 | RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \ 10 | --mount=type=cache,target=/var/lib/apt/lists,sharing=locked \ 11 | rm -f /etc/apt/apt.conf.d/docker-clean \ 12 | && echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' | tee /etc/apt/apt.conf.d/keep-cache \ 13 | && apt-get -qq update -y \ 14 | && apt-get -qq install --no-install-recommends -y curl 15 | COPY --chmod=777 "$ENVOY_CONFIG" /etc/envoy.yaml 16 | COPY --chmod=644 fleetspeak-cert.pem /etc/cert.pem 17 | COPY --chmod=644 fleetspeak-key.pem /etc/key.pem 18 | CMD ["/usr/local/bin/envoy", "-c", "/etc/envoy.yaml"] 19 | 20 | FROM envoy-base as envoy-admin 21 | ARG ENVOY_ADMIN_PORT=10001 22 | ENV ENVOY_ADMIN_PORT="$ENVOY_ADMIN_PORT" 23 | HEALTHCHECK \ 24 | --interval=1s \ 25 | --timeout=1s \ 26 | --start-period=1s \ 27 | --retries=3 \ 28 | CMD curl -s "localhost:${ENVOY_ADMIN_PORT}/stats?filter=server.state" | grep 0 \ 29 | && curl -s "localhost:${ENVOY_ADMIN_PORT}/stats?filter=listener_manager.workers_started" | grep 1 30 | 31 | FROM envoy-base as envoy-fault-injection 32 | RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \ 33 | --mount=type=cache,target=/var/lib/apt/lists,sharing=locked \ 34 | apt-get -qq update -y \ 35 | && apt-get -qq install --no-install-recommends -y tree 36 | COPY enable_delay_fault_injection.sh disable_delay_fault_injection.sh enable_abort_fault_injection.sh disable_abort_fault_injection.sh send_request.sh / 37 | 38 | 39 | FROM envoy-base as envoy-jaeger-native 40 | # 41 | # for discussion on jaeger binary compatibility, and the source of the file, see here: 42 | # https://github.com/envoyproxy/envoy/issues/11382#issuecomment-638012072 43 | # 44 | RUN echo "4a7d17d4724ee890490bcd6cfdedb12a02316a3d33214348d30979abd201f1ca /usr/local/lib/libjaegertracing_plugin.so" > /tmp/checksum \ 45 | && curl -Ls https://github.com/envoyproxy/misc/releases/download/jaegertracing-plugin/jaegertracing-plugin-centos.tar.gz \ 46 | | tar zxf - -C /usr/local/lib \ 47 | && mv /usr/local/lib/libjaegertracing.so.0.4.2 /usr/local/lib/libjaegertracing_plugin.so \ 48 | && sha256sum -c /tmp/checksum \ 49 | && rm /tmp/checksum 50 | 51 | 52 | FROM envoy-base as envoy-load-balancing 53 | RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \ 54 | --mount=type=cache,target=/var/lib/apt/lists,sharing=locked \ 55 | apt-get -qq update -y \ 56 | && apt-get -qq install --no-install-recommends -y python3 57 | COPY ./client.py /client.py 58 | EXPOSE 8001 59 | 60 | 61 | FROM envoy-base as envoy-double-proxy-base 62 | COPY --chmod=777 ./certs/ca.crt /certs/cacert.pem 63 | 64 | 65 | FROM envoy-double-proxy-base as envoy-double-proxy-frontend 66 | COPY --chmod=777 ./certs/postgres-frontend.example.com.crt /certs/clientcert.pem 67 | COPY --chmod=777 ./certs/example.com.key /certs/clientkey.pem 68 | 69 | 70 | FROM envoy-double-proxy-base as envoy-double-proxy-backend 71 | COPY --chmod=777 ./certs/postgres-backend.example.com.crt /certs/servercert.pem 72 | COPY --chmod=777 ./certs/example.com.key /certs/serverkey.pem 73 | 74 | 75 | FROM envoy-base as envoy-certs 76 | COPY --chmod=777 ./certs /certs 77 | 78 | 79 | FROM envoy-base as envoy-lua 80 | ADD --chmod=777 ./lib/mylibrary.lua /lib/mylibrary.lua 81 | 82 | 83 | FROM envoy-base as envoy-go 84 | ENV GODEBUG=cgocheck=0 85 | COPY --chmod=777 ./lib/simple.so /lib/simple.so 86 | 87 | 88 | FROM envoy-base as envoy-ext_authz 89 | COPY --chmod=777 ./config /etc/envoy-config 90 | COPY --chmod=777 ./run_envoy.sh /run_envoy.sh 91 | CMD ["/bin/sh", "/run_envoy.sh"] 92 | 93 | 94 | FROM envoy-base as envoy-dynamic-fs 95 | COPY --chmod=777 ./configs /var/lib/envoy 96 | 97 | 98 | FROM envoy-base 99 | -------------------------------------------------------------------------------- /installing-and-running-grr/advanced-usecases/sandbox/createConfig.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | openssl ecparam -list_curves 6 | 7 | # Generate key pair .pem files, which is linked in the GRR client and 8 | # server configs (client.yaml, server.local.yaml). 9 | openssl genrsa -out config/private-key.pem 10 | openssl rsa -in config/private-key.pem -pubout -out config/public-key.pem 11 | 12 | # Create a CA/trusted private key and cert for Fleetspeak. 13 | openssl ecparam -name prime256v1 -genkey -noout \ 14 | -out fleetspeak-ca-key.pem 15 | openssl req -new -x509 -days 365 -subj "/CN=Fleetspeak CA"\ 16 | -key fleetspeak-ca-key.pem \ 17 | -out fleetspeak-ca-cert.pem \ 18 | 19 | # Create keys for CA signed key and cert for fleetspeak. Resulting files are also 20 | # copied in the envoy container, see containers/envoy/Dockerfile). 21 | openssl ecparam -name prime256v1 -genkey -noout \ 22 | -out fleetspeak-key.pem 23 | openssl req -new -x509 -days 365 -subj "/CN=Fleetspeak CA" -addext "subjectAltName = DNS:fleetspeak-server" \ 24 | -key fleetspeak-key.pem \ 25 | -out fleetspeak-cert.pem \ 26 | -CA fleetspeak-ca-cert.pem \ 27 | -CAkey fleetspeak-ca-key.pem 28 | 29 | # Replace placeholders in fleetspeak and grr-client textproto files. 30 | TRUSTED_FLEETSPEAK_CERT=$(sed ':a;N;$!ba;s/\n/\\\\n/g' fleetspeak-ca-cert.pem) 31 | FLEETSPEAK_KEY=$(sed ':a;N;$!ba;s/\n/\\\\n/g' fleetspeak-key.pem) 32 | FLEETSPEAK_CERT=$(sed ':a;N;$!ba;s/\n/\\\\n/g' fleetspeak-cert.pem) 33 | 34 | sed -i 's@FLEETSPEAK_CERT@'"$FLEETSPEAK_CERT"'@' ./config/fleetspeak-frontend/components.textproto 35 | sed -i 's@FLEETSPEAK_KEY@'"$FLEETSPEAK_KEY"'@' ./config/fleetspeak-frontend/components.textproto 36 | sed -i 's@TRUSTED_FLEETSPEAK_CERT@'"$TRUSTED_FLEETSPEAK_CERT"'@' ./config/grr-client/config.textproto 37 | -------------------------------------------------------------------------------- /installing-and-running-grr/advanced-usecases/sandbox/docker-compose.yaml: -------------------------------------------------------------------------------- 1 | services: 2 | 3 | mysql-fleetspeak: 4 | image: mysql:8.2 5 | restart: always 6 | hostname: mysql-fleetspeak 7 | environment: 8 | MYSQL_DATABASE: 'fleetspeak' 9 | MYSQL_USER: 'fleetspeak-user' 10 | MYSQL_PASSWORD: 'fleetspeak-password' 11 | MYSQL_ROOT_PASSWORD: 'password' 12 | ports: 13 | - '3306:3306' 14 | expose: 15 | - '3306' 16 | healthcheck: 17 | test: ["CMD", "mysqladmin", "ping", "-h", "localhost"] 18 | timeout: 5s 19 | retries: 10 20 | 21 | mysql-grr: 22 | image: mysql:8.2 23 | restart: always 24 | command: --log-bin-trust-function-creators=ON 25 | hostname: mysql-grr 26 | environment: 27 | MYSQL_DATABASE: 'grr' 28 | MYSQL_USER: 'grr-user' 29 | MYSQL_PASSWORD: 'grr-password' 30 | MYSQL_ROOT_PASSWORD: 'password' 31 | ports: 32 | - '3307:3306' 33 | expose: 34 | - '3306' 35 | healthcheck: 36 | test: ["CMD", "mysqladmin", "ping", "-h", "localhost"] 37 | timeout: 5s 38 | retries: 10 39 | 40 | front-envoy: 41 | build: 42 | context: . 43 | dockerfile: ./containers/envoy/Dockerfile 44 | args: 45 | ENVOY_CONFIG: ./envoy-https-https.yaml 46 | hostname: fleetspeak-server 47 | ports: 48 | - "10000:10000" 49 | 50 | fleetspeak-admin: 51 | image: ghcr.io/google/fleetspeak:latest 52 | hostname: fleetspeak-admin 53 | depends_on: 54 | mysql-fleetspeak: 55 | condition: service_healthy 56 | entrypoint: [ 57 | "server", 58 | "-components_config", 59 | "/config/fleetspeak-admin/components.textproto", 60 | "-services_config", 61 | "/config/fleetspeak-admin/services.textproto", 62 | "-alsologtostderr" 63 | ] 64 | volumes: 65 | - "./config:/config" 66 | ports: 67 | - '9091:9091' 68 | expose: 69 | - '9091' 70 | 71 | fleetspeak-frontend: 72 | image: ghcr.io/google/fleetspeak:latest 73 | hostname: fleetspeak-frontend 74 | depends_on: 75 | mysql-fleetspeak: 76 | condition: service_healthy 77 | entrypoint: [ 78 | "server", 79 | "-components_config", 80 | "/config/fleetspeak-frontend/components.textproto", 81 | "-services_config", 82 | "/config/fleetspeak-frontend/services.textproto", 83 | "-alsologtostderr" 84 | ] 85 | volumes: 86 | - "./config:/config" 87 | ports: 88 | - '9090:9090' 89 | - '8080:8080' 90 | - '12000:12000' 91 | expose: 92 | - '9090' 93 | - '8080' 94 | - '12000' 95 | healthcheck: 96 | test: ["CMD", "curl", "http://localhost:8080"] 97 | timeout: 5s 98 | retries: 10 99 | 100 | grr-client: 101 | image: ghcr.io/google/grr:latest 102 | hostname: grr-client 103 | depends_on: 104 | front-envoy: 105 | condition: 106 | service_started 107 | fleetspeak-frontend: 108 | condition: 109 | service_healthy 110 | entrypoint: [ 111 | "/bin/bash", 112 | "-c", 113 | "grr_client_build repack_multiple --templates /client_templates/*/*.zip --repack_configs /config/grr-client/client.yaml --output_dir /installers && dpkg -i /installers/client/*.deb && fleetspeak-client -config /config/grr-client/config.textproto" 114 | ] 115 | volumes: 116 | - "./config/:/config/" 117 | 118 | grr-admin: 119 | image: ghcr.io/google/grr:latest 120 | hostname: grr-admin 121 | depends_on: 122 | mysql-grr: 123 | condition: service_healthy 124 | command: 125 | - -component 126 | - admin_ui 127 | - -config 128 | - /config/grr-server/server.local.yaml 129 | - --verbose 130 | volumes: 131 | - "${PWD}/config/:/config/" 132 | ports: 133 | - '8000:8000' 134 | expose: 135 | - '8000' 136 | 137 | grr-frontend: 138 | image: ghcr.io/google/grr:latest 139 | hostname: grr-frontend 140 | depends_on: 141 | mysql-grr: 142 | condition: service_healthy 143 | command: 144 | - -component 145 | - frontend 146 | - -config 147 | - /configs/server.local.yaml 148 | - --verbose 149 | volumes: 150 | - "${PWD}/config/grr-server/:/configs/" 151 | ports: 152 | - '11111:11111' 153 | expose: 154 | - '11111' 155 | 156 | grr-worker: 157 | image: ghcr.io/google/grr:latest 158 | hostname: grr-worker 159 | depends_on: 160 | mysql-grr: 161 | condition: service_healthy 162 | command: 163 | - -component 164 | - worker 165 | - -config 166 | - /configs/server.local.yaml 167 | - --verbose 168 | volumes: 169 | - "${PWD}/config/grr-server/:/configs/" 170 | -------------------------------------------------------------------------------- /installing-and-running-grr/advanced-usecases/scaling.md: -------------------------------------------------------------------------------- 1 | # Scaling a fleetspeak setup 2 | 3 | The following diagram shows a standard, distributed fleetspeak setup: 4 | 5 | 6 | 7 | ![distributed fleetspeak](../../images/distributed_fleetspeak.png) 8 | 9 | The fleetspeak server setup consists of 2 components: 10 | 11 | - **Fleetspeak Server**: This component terminates the streaming HTTPS 12 | connections from the fleetspeak clients. Each fleetspeak client is 13 | connected to a particular server instance at a time. 14 | - **Fleetspeak Admin Server**: This component provides an administrative GRPC 15 | interface as well as an interface for the GRR server to send messages to 16 | clients. 17 | 18 | Both components are included in the single binary `/usr/bin/fleetspeak-server` 19 | and can be enabled respectively in the config file 20 | `/etc/fleetspeak-server/server.components.config`. The binary is automatically 21 | started by the systemd unit `fleetspeak-server`. 22 | 23 | In the legacy GRR protocol, clients would periodically poll the server for 24 | messages, leading to delays in message delivery. Fleetspeak on the other hand 25 | uses persistent, streaming connections and thus can deliver messages without a 26 | significant delay. For this mechanism to work, a simple, HTTP based 27 | notification protocol between the server an admin server is necessary: if the 28 | admin server submits a message for a particular client, it notifies the 29 | respective server so that the server immediately delivers the message to the 30 | client. 31 | 32 | To run a fleetspeak in a distributed setup: 33 | 34 | - Install the GRR release deb on all machines. 35 | 36 | - Take the files `/etc/fleetspeak-server/server.services.config` and 37 | `/etc/fleetspeak-server/server.components.config` from a working 38 | installation, adapt them and distribute them to the target machines. 39 | 40 | ## Running the Fleetspeak Server component 41 | 42 | To run a server component, keep the `https_config` section and remove the 43 | `admin_config` section in the file 44 | `/etc/fleetspeak-server/server.components.config`. 45 | 46 | Add 2 more config options: 47 | 48 | - `notification_listen_address`: the IP address and port on which the server 49 | listens for notifications from the admin server. This port is not meant 50 | to be publicly accessible, but will need to be reachable from other servers. 51 | 52 | - `notification_public_address` the IP address and port advertised to other servers. 53 | 54 | The final config should look like: 55 | 56 | ``` 57 | mysql_data_source_name: "[...]" 58 | https_config: < 59 | listen_address: "192.168.1.83:4443" 60 | certificates: "-----BEGIN CERTIFICATE-----\n [...]" 61 | key: "-----BEGIN EC PRIVATE KEY----- [...]" 62 | > 63 | notification_listen_address: "192.168.1.83:10000" 64 | notification_public_address: "192.168.1.83:10000" 65 | ``` 66 | 67 | Next, configure a TCP/IP load balancer balancing traffic to the port specified 68 | in `https_config.listen_address` on the respective server instances. 69 | 70 | In case that your load balancer removes the client's IP address (i.e. 71 | fleetspeak sees connections originating from the load balancer's IP), it's 72 | recommended to use the [PROXY 73 | protocol](https://www.haproxy.org/download/2.3/doc/proxy-protocol.txt). In this 74 | case, add one more option in the file 75 | `/etc/fleetspeak-server/server.components.config`: 76 | 77 | ``` 78 | proxy_protocol: true 79 | ``` 80 | 81 | To make GRR/fleetspeak clients use the distributed fleetspeak setup, run 82 | `sudo grr_config_updater initialize` to repack the GRR clients. If the load balancer 83 | listens on `1.2.3.4:4443`, answer the fleetspeak related questions as follows: 84 | 85 | ``` 86 | Use Fleetspeak (next generation communication framework)? [yN]: [N]: y 87 | Please enter your hostname e.g. grr.example.com [grr-server]: 1.2.3.4 88 | Fleetspeak public HTTPS port [4443]: 4443 89 | ``` 90 | 91 | ## Running the Fleetspeak Admin Server component 92 | 93 | To run the admin server component, keep the `admin_config` section and remove 94 | the `https_config` section in the file 95 | `/etc/fleetspeak-server/server.components.config`. 96 | 97 | Add one more config option: 98 | 99 | - `notification_use_http_notifier`: enables usage of the HTTP based notification protocol. 100 | 101 | The final config should like: 102 | 103 | ``` 104 | mysql_data_source_name: "[...]" 105 | admin_config: < 106 | listen_address: "192.168.1.83:4444" 107 | > 108 | notification_use_http_notifier:true 109 | ``` 110 | 111 | Next, configure a TCP/IP load balancer balancing traffic to the port specified 112 | in `admin_config.listen_address` on the respective admin server instances. 113 | 114 | To make the GRR server use the distributed admin server setup, edit the config 115 | file `/etc/grr/server.local.yaml`. If the load balancer listens on 116 | `1.2.3.4:4444`, set the following config option: 117 | 118 | ``` 119 | Server.fleetspeak_server: 1.2.3.4:4444 120 | ``` 121 | -------------------------------------------------------------------------------- /installing-and-running-grr/from-released-pip.md: -------------------------------------------------------------------------------- 1 | # Installing from released PIP packages 2 | 3 | If the templates included in release server debs are not 4 | compatible with the platform you would like to run them on, 5 | you have the option of installing GRR from PIP on your target platform, then 6 | building your own. 7 | 8 | First, install the prerequisites: 9 | 10 | * Ubuntu: 11 | 12 | ```bash 13 | apt install -y debhelper dpkg-dev python-dev python-pip rpm zip 14 | ``` 15 | 16 | * Centos: 17 | 18 | ```bash 19 | yum install -y epel-release python-devel wget which libffi-devel \ 20 | openssl-devel zip git gcc gcc-c++ redhat-rpm-config 21 | 22 | yum install -y python-pip 23 | ``` 24 | 25 | Next, upgrade pip and install virtualenv: 26 | 27 | ```bash 28 | sudo pip install --upgrade pip virtualenv 29 | ``` 30 | 31 | Next, create a virtualenv and install the GRR client-builder package: 32 | 33 | ```bash 34 | virtualenv GRR_ENV 35 | 36 | source GRR_ENV/bin/activate 37 | 38 | pip install grr-response-client-builder 39 | ``` 40 | 41 | Once that is done, you can build a template for your platform with: 42 | 43 | ```bash 44 | grr_client_build build --output mytemplates 45 | ``` 46 | 47 | and repack it with: 48 | 49 | ```bash 50 | grr_client_build repack --template mytemplates/*.zip --output_dir mytemplates 51 | ``` 52 | 53 | If you would like to experiment with GRR's server components, you will need 54 | to first install the `grr-response-server` pip package. Administrative 55 | commands such as as `grr_server` and `grr_config_updater` will be added to 56 | the virtualenv. You can then launch the server components as follows: 57 | 58 | ```bash 59 | # To run fleetspeak server. 60 | grr_server --component fleetspeak_server --verbose 61 | # To run AdminUI. 62 | grr_server --component admin_ui --verbose 63 | # To run the worker. 64 | grr_server --component worker --verbose 65 | # To run the frontend. 66 | grr_server --component frontend --verbose 67 | ``` 68 | 69 | Note that GRR requires Python 3.6+, so for platforms with older default Python 70 | versions (e.g Centos 6), you need to build a newer version of Python from source 71 | and use that for creating the virtualenv. 72 | -------------------------------------------------------------------------------- /installing-and-running-grr/from-source.md: -------------------------------------------------------------------------------- 1 | # Installing from source 2 | 3 | Here's how to install GRR from source (from github HEAD). This method is useful when you intend to do some development or when you cannot or don't want to run Docker or install from pip packages. 4 | 5 | For the instructions below, we assume that Ubuntu 22 is used. 6 | 7 | ## Prerequisites 8 | 9 | First, install the prerequisites: 10 | 11 | ```bash 12 | sudo apt-get update 13 | sudo apt install -y fakeroot debhelper libffi-dev libssl-dev python3-dev \ 14 | python3-pip python3-venv wget openjdk-8-jdk zip git devscripts dh-systemd \ 15 | dh-virtualenv libc6-i386 lib32z1 asciidoc libmysqlclient-dev 16 | 17 | # Go (version 1.13 or newer) is needed to build Fleetspeak. If you already 18 | # have Go installed on your system, you can skip this part. 19 | # We use amd64 version here, you might need to download another Go 20 | # distribution matching your architecture (see https://go.dev/dl/). 21 | wget https://go.dev/dl/go1.22.2.linux-amd64.tar.gz 22 | sudo bash -c "rm -rf /usr/local/go && tar -C /usr/local -xzf go1.22.2.linux-amd64.tar.gz" 23 | export PATH=$PATH:/usr/local/go/bin 24 | ``` 25 | 26 | 27 | ## Fleetpseak 28 | 29 | [Fleetspeak][fleetspeak] is a communication middleware (effectively, a client and a server) 30 | used by GRR to communicate with endpoints. 31 | 32 | Follow the [guide][fleetspeak-guide] on how to install, configure services and run the Fleetspeak server. 33 | 34 | [fleetspeak]: https://github.com/google/fleetspeak 35 | [fleetspeak-guide]: https://github.com/google/fleetspeak/blob/master/docs/guide.md 36 | Create and activate a Python virtualenv at $HOME/INSTALL: 37 | 38 | 39 | ## Building GRR 40 | 41 | Next, you need to build GRR. Download the GitHub repo and cd into its directory: 42 | 43 | ```bash 44 | git clone https://github.com/google/grr.git 45 | cd grr 46 | ``` 47 | 48 | ### If your goal is development 49 | 50 | Run the following script to install GRR into the virtualenv: 51 | 52 | ```bash 53 | ./travis/install.sh 54 | ``` 55 | 56 | Please check [Setting up your own MySQL database](installing-and-running-grr/via-docker-compose.html#setting-up-your-own-mysql-database) for 57 | instructions on how to configure MySQL server. 58 | 59 | You should now be able to run GRR commands from inside the virtualenv, 60 | particularly, you can trigger GRR server configuration. 61 | 62 | ```bash 63 | source $HOME/INSTALL/bin/activate 64 | grr_config_updater initialize # Initialize GRR's configuration 65 | ``` 66 | 67 | Then, to run individual GRR components (for a Fleetspeak-enabled configuration): 68 | 69 | ```bash 70 | # To run fleetspeak server. 71 | grr_server --component fleetspeak_server --verbose 72 | # To run AdminUI. 73 | grr_server --component admin_ui --verbose 74 | # To run the worker. 75 | grr_server --component worker --verbose 76 | # To run the frontend. 77 | grr_server --component frontend --verbose 78 | ``` 79 | 80 | ### If your goal is to build a custom GRR client template 81 | 82 | In this case you don't need a full blown GRR server setup. Run the following: 83 | 84 | ```bash 85 | ./travis/install_client_builder.sh 86 | ``` 87 | 88 | This will install necessary PIP packages into your virtualenv to run the 89 | `grr_client_build` command. See 90 | [Building custom client templates](../maintaining-and-tuning/building-custom-client-templates.md) 91 | for how to use it. 92 | -------------------------------------------------------------------------------- /installing-and-running-grr/index.md: -------------------------------------------------------------------------------- 1 | # Installing and Running GRR 2 | 3 | ```{toctree} 4 | --- 5 | maxdepth: 1 6 | --- 7 | Overview 8 | Running GRR via Docker Compose (recommended) 9 | Running the GRR Docker image 10 | Securing access to GRR server (important) 11 | Installing from released PIP packages 12 | Installing from source 13 | Troubleshooting ("GRR server doesn't seem to run") 14 | Advanced Usecases 15 | ``` 16 | 17 | -------------------------------------------------------------------------------- /installing-and-running-grr/overview.md: -------------------------------------------------------------------------------- 1 | # Overview 2 | 3 | Depending on your use case, there are multiple ways of installing the 4 | GRR server components. Most users will want to 5 | [run GRR via the Docker Compose stack](via-docker-compose.md). 6 | 7 | Alternatively, GRR can be [installed from PIP](from-released-pip.md). 8 | 9 | You can install also GRR from source as described 10 | [here](from-source.md). You should then be able to modify GRR code and build 11 | your own server/client installers. 12 | 13 | 14 | ## GRR components 15 | 16 | ![grr_components](../images/grr_components.png) 17 | 18 | GRR consists of a client and a few server components. 19 | 20 | Whenever GRR server is installed using one of the methods mentioned above, all the server components are installed. This is a simple way to setup smaller GRR server installations: each component will run as a separate server process on the same machine. However, when scaling GRR server to run on multiple machines, each component can exist on its own machine in the data center and run separately. 21 | 22 | GRR components are: 23 | 24 | #### Client 25 | 26 | The GRR client **is not a server component** but it comes bundled with GRR server. GRR client is deployed on corporate assets using the usual mechanism for software distribution and updates (e.g. SMS, apt). The client consists of 2 processes: the fleetspeak client responsible for implementig the communication protocol based on a streaming HTTPS connection. The GRR client process sends and receives GRR messages from the server and implements the business logic. 27 | 28 | #### GRR Datastore 29 | The data store acts both as a central storage component for data, and as a communication mechanism for all GRR server components. 30 | 31 | #### Fleetspeak server 32 | 33 | This component terminates the streaming HTTPS connections from the fleetspeak clients and implements the communication protocol. It receives messages from clients and delivers queued messages to the clients. 34 | 35 | #### Fleetspeak admin server 36 | 37 | This component provides an interface for the GRR server to send messages to clients. 38 | 39 | #### Fleetspeak database 40 | 41 | The fleetspeak system uses a MySQL database for queing messages for clients. 42 | 43 | #### GRR Front End Servers 44 | The front end servers' main task is to receive GRR messages from the fleetspeak server, un-bundle the contained messages and queue these on the data store. The front end also fetches any messages queued for the client and sends them to the client. 45 | 46 | #### GRR Worker 47 | In order to remain scalable, the front end does not do any processing of data, preferring to offload processing to special worker components. The number of workers can be tuned in response to increased workload. Workers typically check queues in the data stores for responses from the client, process those and re-queue new requests for the clients (See Flows and Queues). 48 | 49 | #### GRR Web UI 50 | GRR Web UI is the central application which enables the incident responder or forensic analyst to interact with the system. It allows for analysis tasks to be queued for the clients, and results of previous stored analysis to be examined. It also acts as an API endpoint: GRR API can be used for automation and integration with other systems. 51 | 52 | -------------------------------------------------------------------------------- /installing-and-running-grr/securing-access.md: -------------------------------------------------------------------------------- 1 | # Securing access to GRR server (important!) 2 | 3 | GRR is a powerful system. However, with power comes responsibility and potential security risks. 4 | 5 | If you're running GRR for anything besides simple demo purposes, it's extremely important to properly secure access to GRR server infrastructure. Because: 6 | 7 | 1. Anybody who has root access to your GRR server effectively becomes root on all systems running GRR client talking to your GRR server. 8 | 9 | 1. Anybody who has direct write access to GRR datastore effectively becomes root on all systems running GRR client talking to your GRR server. 10 | 11 | Consequently, it's important to secure your GRR infrastructure. Please follow a security checklist below. 12 | 13 | ## GRR Security Checklist 14 | 15 | 1. Generate new CA/server keys on initial install. Back up these keys somewhere securely (see [Key Management](../maintaining-and-tuning/key-management/which-keys-and-how.md)). 16 | 17 | 1. Maximally restrict SSH access (or any other kind of direct access) to GRR server machines. 18 | 19 | 1. Make sure GRR web UI is not exposed to the Internet and is protected. 20 | 21 | For a high security environment: 22 | 23 | 1. Make sure GRR's web UI is served through an Apache or Nginx proxy via HTTPS. If you're using any kind of internal authentication/authorization system, limit access to GRR web UI when configuring Apache or Nginx. See [user authentication](../maintaining-and-tuning/user-management/authentication.md) documentation. 24 | 25 | 1. If there're more than just a few people working with GRR, turn on [GRR approval-based access control](../maintaining-and-tuning/approval-based-workflow.md) 26 | 27 | 1. Regenerate code signing key with passphrases for additional security. 28 | 29 | 1. Run the http server serving clients on a separate machine to the workers. 30 | 31 | 1. Ensure the database server is using strong passwords and is well protected. 32 | 33 | 1. Produce obfuscated clients (repack the clients with a different *Client.name* setting) 34 | -------------------------------------------------------------------------------- /installing-and-running-grr/troubleshooting.md: -------------------------------------------------------------------------------- 1 | # Troubleshooting ("GRR server doesn't seem to run") 2 | 3 | This page describes common issues encountered when installing the GRR 4 | server components. 5 | 6 | ## OperationalError: Access denied; you need (at least one of) the SUPER privilege(s) for this operation 7 | 8 | MySQL server used by GRR has `max_allowed_packet` setting that's too low. It has to be at 9 | least 20971520. If the setting is lower, GRR tries to execute 10 | `SET GLOBAL max_allowed_packet=20971520;` statement and fails if it doesn't have enough 11 | privileges. To mitigate the issue, change the MySQL server's `max_allowed_packet` setting 12 | (by executing the statement above by a user with enough privileges, for example). 13 | 14 | ## ImportError: cannot import name jobs_pb2 or similar 15 | 16 | If you see "ImportError: cannot import name jobs_pb2" or a similar error for any 17 | other _pb2 file, you need to regenerate the protobuf files. Just run 18 | 19 | ```bash 20 | python setup.py build 21 | sudo python setup.py install 22 | ``` 23 | 24 | ## The upstart/init.d scripts show no output 25 | 26 | When I run an init.d script e.g. "/etc/init.d/grr-http-server start" it does not 27 | show me any output. 28 | 29 | Make sure that the "START" parameter in the corresponding default file, 30 | e.g. "/etc/default/grr-http-server", has been changed to "yes". 31 | 32 | ## I cannot start any/some of the GRR services using the init.d scripts 33 | 34 | When I run an init.d script e.g. "/etc/init.d/grr-http-server start" it 35 | indicates it started the service although when I check with 36 | "/etc/init.d/grr-http-server status" it says it is not running. 37 | 38 | You can troubleshoot by running the services in the foreground, e.g. to run the 39 | HTTP Front-end server in the foreground: 40 | 41 | ```bash 42 | sudo grr_server --start_http_server --verbose 43 | ``` 44 | 45 | ## Any/some of the GRR services are not running correctly 46 | 47 | Check if the logs contain an indication of what is going wrong. 48 | 49 | Troubleshoot by running the services in the foreground, e.g. to run the UI in 50 | the foreground: 51 | 52 | ```bash 53 | sudo grr_server --verbose --start_ui 54 | ``` 55 | 56 | ## Cannot open libtsk3.so.3 57 | 58 | error while loading shared libraries: libtsk3.so.3: cannot open shared object 59 | file: No such file or directory 60 | 61 | The libtsk3 library cannot be found in the ld cache. Check if the path to 62 | libtsk3.so.3 is in /etc/ld.so.conf (or equivalent) and update the cache: 63 | 64 | ```bash 65 | sudo ldconfig 66 | ``` 67 | 68 | ## Protobuf 69 | Travis jobs for GRR's github repository use 70 | [this](https://github.com/google/grr/blob/master/travis/install_protobuf.sh) 71 | script to install protobuf. Older versions of protobuf may not be compatible 72 | with the GRR version you are trying to install. 73 | 74 | ## System-wide Python packages 75 | To avoid version conflicts with the GRR server's Python environment (such 76 | as [this one](https://github.com/google/grr/issues/704)), we 77 | recommend using virtualenvs wherever possible on machines where GRR 78 | server components are to be installed. 79 | -------------------------------------------------------------------------------- /investigating-with-grr/artifacts/collecting.md: -------------------------------------------------------------------------------- 1 | # Collecting Artifacts 2 | 3 | Artifacts can be collected using the artifact collector 4 | [flow](../flows/starting.md). Multiple artifacts can be collected at once. 5 | 6 | Using artifacts in hunts is particularly powerful as an artifact like 7 | `JavaCacheFiles` can be scheduled in a single hunt across all three operating 8 | systems, and the artifact itself determines the correct paths to be downloaded 9 | for each operating system. 10 | -------------------------------------------------------------------------------- /investigating-with-grr/artifacts/defining.md: -------------------------------------------------------------------------------- 1 | # Defining Artifacts 2 | 3 | ## Basics 4 | 5 | GRR artifacts are defined in YAML. The main artifact repository is hosted on 6 | [GitHub][artifact-repository]. You can browse [existing artifact definitions] 7 | [artifact-samples] or read the exhaustive [syntax overview][artifact-syntax]. 8 | 9 | ## Knowledgebase 10 | 11 | We use a standard set of machine information collected from the host for 12 | variable interpolation. This collection of data is called the [*knowledgebase*] 13 | [artifact-knowledgebase] and is referenced with a `%%variable%%` syntax. 14 | 15 | The artifact defines where the data lives. Once it is retrieved by GRR a 16 | [parser][artifact-parsers] can optionally be applied to turn the collected 17 | information into a more useful format, such as parsing a browser history file 18 | to produce URLs. 19 | 20 | ## Uploading new definitions 21 | 22 | New artifacts should be added to the [forensic artifacts repository] 23 | [artifact-repository]. 24 | 25 | The changes can be imported into GRR by running `make` in the `grr/artifacts` 26 | directory. This will delete the existing artifacts, checkout the latest version 27 | of the artifact repository and add all of the YAML definitions into GRR’s 28 | directory. Running `python setup.py build` will have the same effect. The new 29 | artifacts will be available once the server is restarted. 30 | 31 | Artifacts can also be uploaded via the Artifacts GUI and used immediately 32 | without the need for a restart. 33 | 34 | ### Local definitions 35 | 36 | Artifacts that are specific to your environment or need to remain private can be 37 | added to the `grr/artifacts/local` directory. This directory will remain 38 | untouched when you update the main artifacts repository. You can also use this 39 | directory to test new artifacts before they are added to the main public 40 | repository. 41 | 42 | ### Flow templates 43 | 44 | We currently support using the artifact format to call GRR-specific 45 | functionality, such as invoking a GRR client action or listing processes. Such 46 | "artifacts" are called *flow templates* and since they are GRR-specific they 47 | remain in the GRR repository in the `grr/artifacts/flow_templates` directory. 48 | 49 | This is a temporary working name. We intend to rework this functionality into a 50 | more general, powerful and configurable way to call GRR from YAML. 51 | 52 | [artifact-repository]: https://github.com/ForensicArtifacts/artifacts 53 | [artifact-samples]: https://github.com/ForensicArtifacts/artifacts/tree/master/data 54 | [artifact-syntax]: https://github.com/ForensicArtifacts/artifacts/blob/master/docs/Artifacts%20definition%20format%20and%20style%20guide.asciidoc 55 | [artifact-knowledgebase]: https://github.com/google/grr/blob/master/grr/proto/knowledge_base.proto 56 | [artifact-parsers]: https://github.com/google/grr/tree/master/grr/parsers 57 | -------------------------------------------------------------------------------- /investigating-with-grr/artifacts/index.md: -------------------------------------------------------------------------------- 1 | # Artifacts 2 | 3 | ```{toctree} 4 | --- 5 | maxdepth: 1 6 | --- 7 | Overview 8 | Collecting 9 | Defining 10 | ``` 11 | -------------------------------------------------------------------------------- /investigating-with-grr/artifacts/overview.md: -------------------------------------------------------------------------------- 1 | # Artifacts 2 | 3 | ## Outline 4 | 5 | During a security investigation responders need to quickly retrieve common 6 | pieces of information that include items such as logs, configured services, cron 7 | jobs, patch state, user accounts, and much more. These pieces of information are 8 | known as forensic artifacts, and their location and format vary drastically 9 | across systems. 10 | 11 | We have built a framework to describe forensic artifacts that allows them to be 12 | collected and customised quickly using GRR. This collection was initially 13 | contained inside the GRR repository, but we have now moved it out to [a separate 14 | repository][artifact-repository] to make access simple for other tools. 15 | 16 | ## Goals 17 | 18 | The goals of the GRR artifacts implementation are: 19 | 20 | - Describe artifacts with enough precision that they can be collected 21 | automatically without user input. 22 | 23 | - Cover modern versions of Mac, Windows, and Linux and common software 24 | products of interest for forensics. 25 | 26 | - Provide a standard variable interpolation scheme that allows artifacts to 27 | simply specify concepts like "all user home directories", `%TEMP%`, 28 | `%SYSTEMROOT%` etc. 29 | 30 | - Allow grouping across operating systems and products e.g. "Chrome 31 | Web History" artifact knows where the web history is for Chrome on Mac, 32 | Windows and Linux. 33 | 34 | - Allow grouping of artifacts into high level concepts like *persistence 35 | mechanisms*, and investigation specific meta-artifacts. 36 | 37 | - To create simple, shareable, non-GRR-specific human-readable definitions 38 | that allow people unfamiliar with the system to create new artifacts. i.e. 39 | not XML or a domain specific language. 40 | 41 | - The ability to write new artifacts, upload them to GRR and be able to 42 | collect them immediately. 43 | 44 | [artifact-repository]: https://github.com/ForensicArtifacts/artifacts 45 | -------------------------------------------------------------------------------- /investigating-with-grr/automation-with-api.md: -------------------------------------------------------------------------------- 1 | # Automation using the GRR API 2 | 3 | GRR AdminUI server also serves as an API endpoint. We provide the *grr_api_client* Python library which can be used for easy GRR automation. See the [API client documentation](https://github.com/google/grr/tree/master/api_client/python) for more details. 4 | 5 | There's also an [API client library in PowerShell](https://github.com/swisscom/PowerGRR) working on Windows, Linux and macOS for GRR automation and scripting, developed by Swisscom. 6 | -------------------------------------------------------------------------------- /investigating-with-grr/client-server-communication.md: -------------------------------------------------------------------------------- 1 | # Client-Server Communication 2 | 3 | When a Flow needs to request information from a client, it queues up a message 4 | for the client. GRR clients poll the GRR server approximately every 10 minutes, 5 | and it will receive the message and begin responding to the request at the next 6 | poll. 7 | 8 | After a client performs some work, it will normally enter 'fast-poll' mode in 9 | which it polls much more rapidly. Therefore when an analyst requests data from a 10 | machine, it might initially take some minutes to respond but additional requests 11 | will be noticed more quickly. 12 | 13 | ## Protocol 14 | 15 | The client poll is an HTTP request. It passes a signed and encrypted payload and 16 | expects the same from the GRR server. The client signs using its client 17 | key. This key is created on the client when first run, and the GRR ID is 18 | actually just a fingerprint of this key. 19 | 20 | This means that no configuration is required by the client to establish an 21 | identity, but that clients cannot eavesdrop on or impersonate other clients. 22 | -------------------------------------------------------------------------------- /investigating-with-grr/cron-jobs.md: -------------------------------------------------------------------------------- 1 | # Cron Jobs in GRR 2 | 3 | Click on "Cron Jobs" in the left menu to open a list of GRR cron jobs. GRR has a few system cron jobs that run periodically. These jobs perform periodical cleanup and maintenance tasks. 4 | 5 | You can also use GRR's Cron Jobs UI to schedule periodical hunts. 6 | 7 | ![Cron Jobs](../images/cron_job_viewer.png) 8 | 9 | -------------------------------------------------------------------------------- /investigating-with-grr/flows/index.md: -------------------------------------------------------------------------------- 1 | # Flows 2 | 3 | ```{toctree} 4 | --- 5 | maxdepth: 1 6 | --- 7 | What are Flows and how to use them 8 | Starting Flows 9 | Specifying File Paths 10 | Literal and Regex Matching 11 | Specifying Windows Registry paths 12 | Working with Flow results 13 | ``` 14 | -------------------------------------------------------------------------------- /investigating-with-grr/flows/literal-and-regex-matching.md: -------------------------------------------------------------------------------- 1 | # Grep Syntax 2 | 3 | A number of GRR flows (such as the File Finder) accept 4 | Grep specifications, which are a powerful way to search file and memory 5 | contents. There are two types of grep syntax: literal and regex. 6 | 7 | ## Literal Matches 8 | 9 | Use this when you have a simple string to match, or want to match a byte 10 | string. Here’s a simple string example (note no quotes required): 11 | ```docker 12 | allyourbase 13 | ``` 14 | 15 | And a byte string example: 16 | ```docker 17 | MZ\x90\x00\x03\x00\x00\x00\x04\x00\x00\x00 18 | ``` 19 | 20 | To minimise the potential for errors we recommend using python to create 21 | byte strings for you where possible, e.g. the above byte string was 22 | created in ipython like this: 23 | 24 | ``` 25 | In [1]: content = open("test.exe","rb").read(12) 26 | 27 | In [2]: content 28 | Out[2]: 'MZ\x90\x00\x03\x00\x00\x00\x04\x00\x00\x00' 29 | ``` 30 | 31 | ## Regex Matches 32 | 33 | Use this when you need more complex matching. The format is a regular 34 | python regex (see ) with the 35 | following switches applied automatically: 36 | 37 | ```docker 38 | re.IGNORECASE | re.DOTALL | re.MULTILINE 39 | ``` 40 | 41 | An example regex is below. The entire match is reported, () groups are 42 | not broken out separately. Also note that 10 bytes before and after will 43 | be added to any matches by default - use the Advanced menu to change 44 | this behavior: 45 | 46 | ``` 47 | Accepted [^ ]+ for [^ ]+ from [0-9.]+ port [0-9]+ ssh 48 | ``` -------------------------------------------------------------------------------- /investigating-with-grr/flows/specifying-file-paths.md: -------------------------------------------------------------------------------- 1 | # Specifying File Paths 2 | 3 | Providing file names to flows is a core part of GRR, and many flows have 4 | been consolidated into the File Finder flow, which uses a 5 | glob+interpolation syntax. 6 | 7 | ## File Path Examples 8 | 9 | All executables or dlls in each user’s download directory: 10 | 11 | ```docker 12 | %%users.homedir%%\Downloads\*.{exe,dll} 13 | ``` 14 | 15 | All .evtx files found up to three directories under 16 | "C:\Windows\System32\winevt": 17 | 18 | ```docker 19 | %%environ_systemroot%%\System32\winevt\**.evtx 20 | ``` 21 | 22 | "findme.txt" files in user homedirs, up to 10 directories deep: 23 | 24 | ```docker 25 | %%users.homedir%%/**10/findme.txt 26 | ``` 27 | 28 | ## File Paths: backslash or forward slash? 29 | 30 | Either forward "/home/me" or backslash "C:\\Users\\me" path 31 | specifications are allowed for any target OS. They will be converted to 32 | a common format internally. We recommend using whatever is normal for 33 | the target OS: (backslash for Windows, fwdslash for OS X and Linux). 34 | 35 | ## File Path Interpolation 36 | 37 | GRR supports path interpolation from values in the artifact Knowledge 38 | Base. Interpolated values are enclosed with %%, and may expand to 39 | multiple elements. e.g. 40 | 41 | ``` 42 | %%users.homedir%% 43 | ``` 44 | 45 | Might expand to the following paths on Windows: 46 | 47 | ```docker 48 | C:\Users\alice 49 | C:\Users\bob 50 | C:\Users\eve 51 | ``` 52 | 53 | and on OS X: 54 | 55 | ```docker 56 | /Users/alice 57 | /Users/bob 58 | /Users/eve 59 | ``` 60 | 61 | and on Linux: 62 | 63 | ```docker 64 | /home/alice 65 | /usr/local/home/bob 66 | /home/local/eve 67 | ``` 68 | 69 | A full list of possible interpolation values can be found by typing %% 70 | in the gui. The canonical reference is the 71 | [proto/knowledge\_base.proto](https://github.com/google/grr/blob/master/grr/proto/grr_response_proto/knowledge_base.proto) file, which also contains docstrings for each type. 72 | 73 | ## Path Globbing 74 | 75 | Curly braces work similarly to bash, e.g: 76 | 77 | ```docker 78 | {one,two}.{txt,doc} 79 | ``` 80 | 81 | Will match: 82 | ```docker 83 | one.txt 84 | two.txt 85 | one.doc 86 | two.doc 87 | ``` 88 | 89 | 90 | Recursive searching of a directory is performed with `**`. The default 91 | search depth is 3 directories. So: 92 | 93 | ```docker 94 | /root/**/*.doc 95 | ``` 96 | 97 | Will match: 98 | 99 | ```docker 100 | /root/blah.doc 101 | /root/1/something.doc 102 | /root/1/2/other.doc 103 | /root/1/2/3/another.doc 104 | ``` 105 | 106 | More depth can be specified by adding a number to the `**`, e.g. this 107 | performs the same search 10 levels deep: 108 | 109 | ```docker 110 | /root/**10/*.doc 111 | ``` 112 | 113 | It can also be combined with interpolation. 114 | 115 | The following statement: 116 | 117 | ``` 118 | %%environ_systemdrive%%\**5\FOOBAR 119 | ``` 120 | 121 | Will match: 122 | 123 | ``` 124 | C:/1/FOOBAR 125 | C:/1/2/FOOBAR 126 | C:/1/2/3/FOOBAR 127 | ``` 128 | 129 | > **IMPORTANT:** Due to a [regression](https://github.com/google/grr/issues/915), the recursive glob `**` behaves differently in ClientFileFinder. This will be fixed in a future version. 130 | > 131 | > **Note:** FileFinder transfers all data to the server and does the matching server side. This might lead to terrible performance when used with deep recursive directory searches. For a faster alternative that has the drawback of leaking the path you are searching for to the potentially compromised client, use the *ClientFileFinder* flow which does the matching right on the client. 132 | -------------------------------------------------------------------------------- /investigating-with-grr/flows/specifying-windows-registry-paths.md: -------------------------------------------------------------------------------- 1 | # Specifying Windows Registry Paths 2 | 3 | When specifying registry paths, GRR uses the following hive names (these 4 | can also be found by looking at the registry folder under "Browse 5 | Virtual Filesystem"): 6 | 7 | HKEY_CLASSES_ROOT 8 | HKEY_CURRENT_CONFIG 9 | HKEY_CURRENT_USER 10 | HKEY_DYN_DATA 11 | HKEY_LOCAL_MACHINE 12 | HKEY_PERFORMANCE_DATA 13 | HKEY_USERS 14 | 15 | The Registry Finder flow uses the same path globbing 16 | and interpolation system as described in [Specifying File Paths](specifying-file-paths.md). 17 | Examples: 18 | 19 | ```docker 20 | HKEY_USERS\%%users.sid%%\Software\Microsoft\Windows\CurrentVersion\Run\* 21 | HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\RunOnce\* 22 | ``` 23 | 24 | RegistryFinder will retrieve the values of any keys specified and report 25 | them in the registry data field. Default values will be retrieved and 26 | reported in registry data of the parent key. E.g. for this registry 27 | structure: 28 | 29 | ```docker 30 | HKEY_LOCAL_MACHINE\Software\test: 31 | (Default) = "defaultdata" 32 | subkey = "subkeydata" 33 | ``` 34 | 35 | Collecting this: 36 | 37 | ```docker 38 | HKEY_LOCAL_MACHINE\Software\test\* 39 | ``` 40 | 41 | Will give results like: 42 | ```docker 43 | Path: /HKEY_LOCAL_MACHINE/SOFTWARE/test 44 | Registry data: defaultdata 45 | 46 | Path: /HKEY_LOCAL_MACHINE/SOFTWARE/test/subkey 47 | Registry data: subkeydata 48 | ``` -------------------------------------------------------------------------------- /investigating-with-grr/flows/starting.md: -------------------------------------------------------------------------------- 1 | # Starting Flows 2 | 3 | To start a new Flow simply click on the *Start new flows* option on the 4 | left panel while having a client selected. The main panel will populate with the holy trinity of panels. The tree view shows all the Flows organized by category. 5 | 6 | For example, in order to start a *FileFinder* flow, 7 | expand the *FileSystem* category and select the corresponding item. 8 | The flow view will populate with a form with all the user-configurable 9 | parameters for this flow. What’s more, because each parameter has a 10 | well-defined type, GRR shows you widgets to select a value for each 11 | of them. 12 | 13 | The FileFinder flow accepts a range parameters: 14 | 15 | 1. *Paths*. This is a list of textual paths that you want to look at. 16 | 17 | 2. *Pathtype*. Which VFS handler you want to use for the path. 18 | Available options are: 19 | 20 | - **OS**. Uses the OS "open" facility. These are the most 21 | straightforward for a first user. Examples of *os* paths are 22 | `C:/Windows` on Windows or `/etc/init.d/` on Linux/OSX. 23 | 24 | - **TSK**. Use Sleuthkit. Because Sleuthkit is invoked a path to 25 | the device is needed along the actual directory path. Examples 26 | of *tsk* paths are 27 | `\\?\Volume{19b4a721-6e90-12d3-fa01-806e6f6e6963}\Windows` for 28 | Windows or `/dev/sda1/init.d/` on Linux (But GRR is smart enough to figure out what you want if you use `C:\Windows` or `/init.d/` instead even though there is some guessing involved). 29 | 30 | - **REGISTRY**. Windows-related. You can open the live Windows 31 | registry as if it was a virtual filesystem. So you can specify 32 | a path such as `HKEY_LOCAL_MACHINE/Select/Current`. 33 | 34 | - **MEMORY** and **TMPFILE** are internal and should not be used in most cases. 35 | 36 | 3. *Condition*. The *FileFinder* can filter files based on condition like file size or file contents. The different conditions should be self explanatory. Multiple conditions can be stacked, the file will only be processed if it fulfills them all. 37 | 38 | 4. *Action*. Once a file passes all the conditions, the action decides what should be done with it. Options are **STAT**, **HASH** and **DOWNLOAD**. Stat basically just indicates if a file exists, this is mostly used to list directories (path `C:\Windows\*` and action STAT). Hash returns a list of hashes of the file and Download collects the file from the client and stores it on the server. 39 | 40 | For this example, a good set of arguments would be a directory listing, something like path `C:\Windows\*` or `/tmp/*` and action **STAT**. Once you’ve filled in each required field, click on *Launch* and if all 41 | parameters validated, the Flow will run. Now you can go to the *Manage 42 | launched flows* view to find it running or track it. 43 | 44 | > **Important** 45 | > Not all flows might be available on every platform. When trying to run 46 | > a flow that’s not available in the given platform an error will show 47 | > up. 48 | 49 | ### Available flows ### 50 | 51 | The easiest ways to see the current flows is to check in the AdminUI under StartFlow. These have useful documentation. 52 | 53 | Note that by default only BASIC flows are shown in the Admin UI. By clicking the settings (gear icon) in the top right, you can enable ADVANCED flows. With this set you will see many of the underlying flows which are sometimes useful, but require a deeper understanding of GRR. -------------------------------------------------------------------------------- /investigating-with-grr/flows/what-are-flows.md: -------------------------------------------------------------------------------- 1 | # GRR Flows 2 | 3 | When designing GRR, one of the main goals was achieving great 4 | scalability. One of the main resource hogs with the client-server model 5 | is that while a client is active all resources that might have been 6 | needed on the server side to communicate with it and do processing are 7 | held (think temporary buffers, sockets, file descriptors...​). Even when 8 | the client itself is doing operations that take time such as heavy 9 | computations or waiting on I/O, resources are held on the server. 10 | 11 | When trying to deal with thousands of clients at the same time, this 12 | would translates into the server hoarding many unneeded resources. 13 | 14 | To solve the resource hogging problem, Flows were created. Flows are the 15 | server-side code entities that call client actions. These calls are done 16 | asynchronously. That is, they are requested and their results become 17 | available later on. Flows are like a state machine, where transition 18 | between states happens when the results of client actions return to the 19 | server. So here’s what happens when the GRR server launches a typical 20 | Flow. 21 | 22 | 1. The GRR server executes the initial Flow state. 23 | 24 | 2. This state asks for one or more client actions to be performed on 25 | the client. 26 | 27 | 3. The server clears all the resources this Flow has requested and 28 | waits for responses from the client to come back. 29 | 30 | 4. When responses are received, the server fetches all the needed 31 | resources again and runs the Flow state where it expects these 32 | responses. If more client actions are requested by this state it 33 | goes back to step 2. Otherwise... 34 | 35 | 5. The results of this Flow are stored and the flow state is updated. 36 | 37 | Flows have a second very interesting property. For flows that make use 38 | of some of the most primitive client actions, because all of the logic 39 | is encapsulated on the server side and the client doesn’t have any state 40 | at all, they naturally survive reboots while processing is taking place. 41 | 42 | In the GRR UI, while having selected a client in the GUI, clicking on the *Manage launched flows* link on the left panel takes you to a view 43 | that shows all the Flows that have been requested on this client. 44 | 45 | ![Launched flows view](../../images/manage_flows.png "Manage Launched FLows View") 46 | 47 | The table view shows the current state of the flow, what’s the flow identifier (*Path*), the name of the Flow launched, the date when it was launched, when it was last active and who created it. 48 | 49 | As you can see, a couple of Flows have been launched in the shown example: 50 | 51 | 1. *CAEnroler*. This is the first flow ever to launch for any client. 52 | It is the enroling Flow which gets the client set up server side. 53 | 54 | 2. *Interrogate*. After enroling, a client sends some information about 55 | the machine it’s running in such as the hostname, MAC address or 56 | users available on the system. This is the flow that fetches this 57 | information and if you remember the *Host Information* option, most 58 | information is contained there. 59 | 60 | This flow has also a few subflows shown in the table - flows can call other flows to collect more information from the client. 61 | 62 | 3. *ListDirectory*. A Flow that lists the contents of a directory. This 63 | is what happened when the refresh button was pressed on the GUI. 64 | 65 | 4. *FileFinder*. This flow is used to download and list files on the client machine. 66 | 67 | Let’s see the *ListDirectory* flow in detail. You can click on any flow 68 | to get detailed information. 69 | 70 | ![ListDirectory flow details](../../images/flow_details.png 71 | "Details for the ListDirectory flow.") 72 | 73 | There’s a lot of information here. Some bits worth mentioning are: 74 | 75 | - Flow *state*, which tells us whether it finished correctly (oddly named **TERMINATED**) or not (**ERROR**), or if it’s still running (**RUNNING**). 76 | - *Arguments* show the arguments this flow was started with. 77 | - In *Context* there is information about resource usage of the flow and metadata about how many messages were sent back and forth to the client. 78 | - *State Data* shows internal server side state of this flow. This information is mostly useful for debugging. 79 | 80 | The other tabs shown on the flow details view are 81 | 82 | - *Requests*. Show what pending requests this flow is waiting for while it's running. Mostly for debugging. 83 | - *Results*. The results returned by this flow. More information in [Working with Results](working-with-results.md). 84 | - *Log*. The log for this flow. 85 | - *Api*. This tab shows how the flow can be started automatically using the GRR API. See the [Chapter on Automation](../automation-with-api.md). -------------------------------------------------------------------------------- /investigating-with-grr/flows/working-with-results.md: -------------------------------------------------------------------------------- 1 | # Working with Flow results 2 | 3 | All flows in GRR produce a list of (0 or more) results. For example in the image below, the results for a FileFinder flow are shown. The type of the items in the list is *FileFinderResult* which are essentially Stat entries that also reference collected files in case the file finder was also downloading the files from the client to the GRR server. 4 | 5 | ![Flow Results](../../images/flow_results.png "Results for a FileFinder flow") 6 | 7 | GRR only provides rudimentary analysis functionality for results. Results can be filtered using the filter input on the right which does a full text match for results (including all metadata fields) but in many cases it's better to export the results and use external analysis tools instead. 8 | 9 | GRR offers to download the results in three different formats (the `Download As:` dropdown on the right: 10 | 11 | - CSV 12 | - YAML 13 | - SQlite 14 | 15 | **NOTE** All those formats will flatten the data, i.e., you might lose some information. In most cases this does not make a difference but it's good to be aware that this happens. 16 | 17 | ## Files ## 18 | 19 | In the case where the results contain items that reference files (StatEntry for example), another option will be shown that allows the download of the referenced files as a ZIP or TAR archive. Additionally, GRR shows an export command that can be pasted into a GRR API shell and will download the same ZIP file programmatically using the GRR API. 20 | 21 | 22 | -------------------------------------------------------------------------------- /investigating-with-grr/glossary.md: -------------------------------------------------------------------------------- 1 | # Glossary 2 | 3 | ## AFF4 4 | 5 | AFF4 was the data model used for storage in GRR that has since been 6 | deprecated. You can read about the usage in the [GRR paper][grr-paper] linked 7 | above and there is additional documentation on [Forensics Wiki][fw-aff4]. 8 | 9 | ## Agent 10 | 11 | A platform-specific program that is installed on machines that one might want to 12 | investigate. It communicates with the GRR server and can perform client actions 13 | at the server's request. 14 | 15 | ## Client 16 | 17 | A system that has an agent installed. Also used to refer to the specific 18 | instance of an agent running in that system. 19 | 20 | ## Client Action 21 | 22 | A client action is an action that a client can perform on behalf of the server. 23 | It is the base unit of work on the client. Client actions are initiated by the 24 | server through flows. Example client actions are `ListDirectory`, 25 | `EnumerateFilesystems`, `Uninstall`. 26 | 27 | ## Collection 28 | 29 | A collection is a logical set of objects stored in the AFF4 database. Generally 30 | these are a list of URNs containing a grouping of data such as artifacts or 31 | events from a client. 32 | 33 | ## Datastore 34 | 35 | The backend is where all AFF4 and scheduler data is stored. It is provided as an 36 | abstraction to allow for replacement of the datastore without significant 37 | rewrite. The datastore supports read, write, querying and filtering. 38 | 39 | ## Flow 40 | 41 | A logical collection of server or client actions which achieve a given 42 | objective. A flow is the core unit of work in the GRR server. For example a 43 | `BrowserHistory` flow contains all the logic to download, extract and display 44 | browser history from a client. Flows can call other flows to get their job done. 45 | E.g. A `CollectBrowserHistory` flow might call `ListDirectory` and `GetFile` to 46 | do it's work. A flow is implemented as a class that inherits from `GRRFlow`. 47 | 48 | ## Frontend server 49 | 50 | Server-side component that sends and receives messages back and forth from 51 | clients. 52 | 53 | ## Hunt 54 | 55 | A hunt is a mechanism for managing the execution of a flow on a large number of 56 | machines. A hunt is normally used when you are searching for a specific piece of 57 | data across a fleet of machines. Hunts allow for monitoring and reporting of 58 | status. 59 | 60 | ## Message 61 | 62 | Transfer unit in GRR that transports information from a flow to a client and 63 | vice versa. 64 | 65 | ## Worker 66 | 67 | Once receiving a message from a client a worker will wake up the flow that 68 | requested its results and execute it. 69 | 70 | [grr-paper]: https://storage.googleapis.com/docs.grr-response.com/scalable_datastore.pdf 71 | [fw-aff4]: http://www.forensicswiki.org/wiki/AFF4 72 | -------------------------------------------------------------------------------- /investigating-with-grr/hunts/controlling-hunts.md: -------------------------------------------------------------------------------- 1 | # Hunt Controls 2 | 3 | ### Create a new hunt 4 | 5 | ![new](../../images/icons/new.png) 6 | 7 | Use this button to create a new hunt. 8 | 9 | ### Start a Hunt 10 | 11 | ![play button](../../images/icons/play_button.png) 12 | 13 | Use this button to start a newly created hunt. New hunts are created in 14 | the PAUSED state, so you’ll need to do this to run them. Hunts that 15 | reach their client limit will also be set to PAUSED, use this button to 16 | restart them after you have removed the client limit (see modify below). 17 | 18 | ### Stop a Hunt 19 | 20 | ![stop button](../../images/icons/stop.png) 21 | 22 | Stopping a hunt will prevent new clients from being scheduled and 23 | interrupt in-progress flows the next time they change state. This is a 24 | hard stop, so in-progress results will be lost, but results already 25 | reported are unaffected. Once a hunt is stopped, there is no way to start it again. 26 | 27 | 28 | ### Modify a Hunt 29 | 30 | ![modify](../../images/icons/modify.png) 31 | 32 | The modify button allows you to change the hunt client limit and the 33 | hunt expiry time. Typically you use this to remove (set to 0) or increase a client limit to let the hunt run on more machines. Modifying an existing hunt doesn’t require re-approval. Hunts can only be modified in the PAUSED state. 34 | 35 | ### Copy a Hunt 36 | 37 | ![modify](../../images/icons/copy.png) 38 | 39 | The copy button creates a new hunt with the same set of parameters as the currently selected one. If you find a mistake in a scheduled hunt and have to stop it, this button can be used to quickly reschedule a fixed version of the hunt. 40 | 41 | ### Delete a Hunt 42 | 43 | ![editdelete](../../images/icons/editdelete.png) 44 | 45 | Use this to remove an unwanted hunt. For accountability reasons hunts 46 | can only be deleted if they haven’t run on any clients. 47 | 48 | ### Show automated hunts 49 | 50 | ![robot](../../images/icons/robot.png) 51 | 52 | Use this button to display all hunts, including those created by 53 | cronjobs. These are hidden by default to reduce UI clutter. 54 | -------------------------------------------------------------------------------- /investigating-with-grr/hunts/index.md: -------------------------------------------------------------------------------- 1 | # Hunts 2 | 3 | ```{toctree} 4 | --- 5 | maxdepth: 1 6 | --- 7 | What are "hunts" and how to use them 8 | Starting Hunts 9 | Hunt Rules 10 | Hunt Limits, DOs and DONTs 11 | Hunt Controls 12 | Working with Hunt results 13 | Rapid hunts 14 | Troubleshooting ("Why is my hunt doing nothing?") 15 | ``` 16 | -------------------------------------------------------------------------------- /investigating-with-grr/hunts/limits.md: -------------------------------------------------------------------------------- 1 | # Hunt Limits, DOs and DONTs 2 | 3 | ## Creating Hunts ## 4 | 5 | As already outlined in the section about [creating hunts](starting.md), hunts might negatively impact both the client machines they run on and also the GRR system by causing extensive load. In order to prevent this, we recommend to always start hunts either by 6 | 7 | - copying an existing and tested flow or 8 | - copying an existing hunt, applying only minor modifications. 9 | 10 | We realize that sometimes unexpected problems can still arise and therefore GRR applies some limits to hunts. 11 | 12 | ## Default Hunt Limits 13 | 14 | There are two sets of limits in place for hunts. GRR enforces both, limits on individual clients and limits on the average resource usage for the hunt. 15 | 16 | ### Individual Client Limits 17 | 18 | Analoguous to what happens with flows, GRR enforces limits on the resource usage for each client participating in a hunt. Since the impact of a hunt is potentially much larger than for a single flow, the limits for hunts are lower than the flow limits. The current defaults are: 19 | 20 | - 600 cpu seconds per client 21 | - 100 MB of network traffic per client 22 | 23 | ### Limits on Average Resource Usage 24 | 25 | Once a hunt has processed 1000 clients, the average resource usage is also checked and enforced by GRR. The defaults for those limits are: 26 | 27 | - 1000 results on average per client 28 | - 60 cpu seconds on average per client 29 | - 10 MB of network traffic on average per client 30 | 31 | All the limits in this section can be overridden in the advanced settings when scheduling hunts. Use at your own risk. 32 | -------------------------------------------------------------------------------- /investigating-with-grr/hunts/rapid-hunts.md: -------------------------------------------------------------------------------- 1 | # Rapid hunts 2 | 3 | Rapid hunting is a GRR UI feature that automatically sets *client rate* to 0 for certain types of hunts, thus making such hunts much faster. 4 | 5 | ## Enabling rapid hunts 6 | 7 | Add the following to GRR server configuration file: 8 | 9 | ```yaml 10 | AdminUI.rapid_hunts_enabled: True 11 | ``` 12 | 13 | ## Rapid hunts UI 14 | 15 | When rapid hunts are enabled, hunts creation wizard shows an eligibility note on its *Review* page. 16 | 17 | ![Rapid hunt screenshot](../../images/rapid-hunting-eligible.png "Rapid hunt screenshot") 18 | 19 | If a hunt is deemed rapid hunting eligible, its *client rate* is set to 0 automatically. User can still override it to a non-0 value, if needed. 20 | 21 | There's also an additional eligibility note on a *Hunt Overview* page. 22 | 23 | ## Rapid hunting eligibility 24 | 25 | A hunt should meet following conditions to be considered rapid-hunting eligible: 26 | 27 | 1. It should be a *FileFinder* or *ClientFileFinder* hunt. 28 | 1. Action should be *STAT* or *HASH* (but not *DOWNLOAD*). 29 | 1. Each glob expression can have **at most** one wildcard in it. For example, `/foo/bar/*` is ok, but `/foo/*/bar/*` is not. 30 | 31 | NOTE: if approvals are turned on, rapid hunts would still need to be approved. 32 | 33 | -------------------------------------------------------------------------------- /investigating-with-grr/hunts/rules.md: -------------------------------------------------------------------------------- 1 | # Hunt Rules # 2 | 3 | Hunt rules are used to define a subset of the clients to run a hunt on. The most common rules are 4 | - Limiting the hunt to a single operating system and 5 | - Limiting the hunt to clients that have a certain label attached 6 | 7 | but GRR offers also regex and integer matching on all client attributes so more sophisiticated hunt rules can be created. 8 | 9 | The rule section of the New Hunt Wizard can be seen in the following image: 10 | 11 | ![Rules Section](../../images/hunt_rules.png "Rules Section") 12 | 13 | GRR allows to specify more than one rule for hunts. By default, those rules are evaluated using a logical AND, meaning a client will only run the hunt if it satisfies all the conditions specified by every rule. This behavior can be changed with the dropdown on top, changing 14 | 15 | - Match all (logical AND) 16 | 17 | to 18 | 19 | - Match any (logical OR) 20 | -------------------------------------------------------------------------------- /investigating-with-grr/hunts/starting.md: -------------------------------------------------------------------------------- 1 | ## Creating a Hunt 2 | 3 | ### Manual Hunt Creation ### 4 | 5 | You can create a new Hunt in the Hunts section of the UI. To 6 | create a Hunt: 7 | 8 | 1. Click the + button 9 | 10 | 2. Select a flow you want to run and fill out all parameters as you would do for a flow running on a single client. 11 | 12 | 3. The second page of the Hunt Wizard lets you set hunt parameters. They are: 13 | 14 | - *Hunt Description* A friendly name for this hunt to show in the UI. 15 | - *Client Limit* A limit on the number of clients this hunt should run on. This can be changed later so it makes sense to start the hunt on a few clients first (we like 100) to see how it goes and only later remove the limit. 16 | - *Crash Limit* If more clients than indicated in this parameter return an error, the hunt is automatically paused. 17 | - *Expiry Time* Hunts are never really done since new clients might appear at any time and they should run the hunt too. This time indicates the lifetime of a hunt after which it is considered done. 18 | - *Client Rate* Number of clients to schedule the hunt on per minute. The default of 20 we have found to be safe, avoiding overloading the server for intensive hunts with lots of message passing (e.g. multiple flows, lots of results). A value of 0 disables rate limiting and clients will be scheduled as fast as possible. Use this with care: light hunts you need to run quickly. 19 | - Some more advanced options are available behind the advanced link but they should not need adjustment in most cases. 20 | 21 | 3. Set any output plugins, such as receiving an email for each result. 22 | 23 | 5. Set Hunt Rules. Most commonly this is used to select a specific platform target, e.g. Windows. See the [Rules section](rules.md). 24 | 25 | 6. Click Run 26 | 27 | Unless approvals are required, you can start the hunt immediately. 28 | 29 | ### Creating a Hunt from a Flow ### 30 | 31 | Manual hunt creation is a bit dangerous because mistakes happen and they can propagate to many machines in short time. A better way of creating a hunt is to copy an existing and tested flow. In order to do so, create a flow on a client with the parameters you want to hunt for and make sure that it works. Once you are convinced that this flow is what you want to run on the fleet, use the "Create Hunt from Flow" button in the Manage launched flows view: 32 | 33 | ![Hunt for a flow button](../../images/create_hunt_from_flow.png "Create hunt from flow button") 34 | 35 | This will copy all flow parameters to a new hunt and you can continue to set all Hunt parameters starting from Step 3 above. 36 | -------------------------------------------------------------------------------- /investigating-with-grr/hunts/troubleshooting.md: -------------------------------------------------------------------------------- 1 | # Troubleshooting ("Why is my hunt doing nothing?") 2 | 3 | - There are caches involved in the frontend server, you may need to 4 | wait a couple of minutes before the first client picks up the flow. 5 | 6 | - Clients only check if there is hunt work to do when doing a foreman 7 | check. The frequency of these checks are specified in the 8 | `Client.foreman_check_frequency` parameter. This defaults to 9 | every 30 minutes. 10 | 11 | - Even when a client issues a foreman check, the flows may not 12 | immediately start. Instead, the process is asynchronous, so the 13 | check tells the server to check its hunt rules to see if there are 14 | things for the client to do. If there are, it schedules them, but 15 | the client may not do its regular poll and pick up that flow until 16 | `Client.poll_max period` (10 minutes by default). 17 | 18 | - When you run a hunt you can specify a "Client Rate" as specified 19 | in [creating hunts](starting.md). If this is set low (but not 0), you can expect a slow hunt. 20 | 21 | - When running a hunt under high server load, clients seem appear 22 | complete in batches. This results in the completion graph appearing 23 | "stepped". The clients are finishing normally, but their results are 24 | being processed and logged in batches by the Hunt. When the system 25 | is under load, this hunt processing takes some time to complete 26 | resulting in the *steps*. 27 | -------------------------------------------------------------------------------- /investigating-with-grr/hunts/what-are-hunts.md: -------------------------------------------------------------------------------- 1 | # What are "Hunts" and how to use them 2 | 3 | Hunting is one of the key features of GRR. Anything you can do on a 4 | single client, should be able to be done on thousands of clients just as 5 | easily. 6 | 7 | A hunt specifies a Flow, the Flow parameters, and a set of rules for 8 | which machines to run the Flow on. -------------------------------------------------------------------------------- /investigating-with-grr/hunts/working-with-results.md: -------------------------------------------------------------------------------- 1 | # Working with Hunt results 2 | 3 | Hunt results in GRR work almost the same as Flow results. The only difference is that each item is annotated with the client id of the client that produced it. Everything else (filtering, exporting, generating archives with collected files) works exactly the same as with flow results as described in [the flow results section](../flows/working-with-results.md). -------------------------------------------------------------------------------- /investigating-with-grr/index.md: -------------------------------------------------------------------------------- 1 | # Investigating with GRR 2 | 3 | ```{toctree} 4 | --- 5 | maxdepth: 1 6 | --- 7 | Overview 8 | Client-Server communication 9 | Security considerations 10 | Searching for a client 11 | Flows 12 | Hunts 13 | Virtual File System 14 | Artifacts 15 | Cron Jobs in GRR 16 | Osquery 17 | Collect Large File 18 | Automation with GRR API 19 | Interactive scripting 20 | Output Plugins 21 | Emergency Code and Binary Pushes 22 | Glossary 23 | ``` 24 | -------------------------------------------------------------------------------- /investigating-with-grr/interactive-scripting.md: -------------------------------------------------------------------------------- 1 | # Interactive scripting 2 | 3 | GRR Colab is an alternative GRR API that allows you to work with GRR 4 | interactively. It is represented as IPython extension and can be used with 5 | [IPython][ipython] itself, [Jupyter][jupyter] or [Colaboratory][colab]. 6 | 7 | ## Extension API 8 | 9 | The GRR Colab offers two APIs. 10 | 11 | *Python API* allows to work with multiple clients simultaneously since each 12 | client is represented as a usual Python object. All functions return Protobuf 13 | objects for non-trivial values. In order to start a flow you just call a single 14 | method. Python API does not have a state so you have to always specify absolute 15 | paths. 16 | 17 | *Magic API* has a state and that's why it allows to work with a single client 18 | only since you have to select a client first. The magic commands have Bash-like 19 | syntax and return Pandas dataframes for non-trivial values. The state allows 20 | you to work with relative paths also. 21 | 22 | More details about both APIs you can find [here][jupyter-notebook] in the 23 | Jupyter notebook demo. 24 | 25 | ## Enabling GRR IPython extension 26 | 27 | First, install `grr-colab` Python package. On successful installation you 28 | should be able to import `grr_colab` from Python (or IPython) terminal. Python 29 | API should be already available for you. 30 | 31 | In order to enable Magic API `grr_colab.ipython_extension` must be loaded. You 32 | may do this in two ways: 33 | 34 | * If you want the extension to be loaded on the IPython kernel start, open 35 | IPython config file and add `grr_colab.ipython_extension` to 36 | `c.InteractiveShellApp.extensions` list. Then Magic API should be available 37 | right after creating new Jupyter notebook or starting new IPython session. 38 | 39 | * If you want to enable the extension for the current session only, run 40 | `%load_ext grr_colab.ipython_extension` command in you Jupyter notebook or 41 | IPython terminal. 42 | 43 | [jupyter]: https://jupyter.org/ 44 | [ipython]: https://ipython.org/ 45 | [colab]: https://colab.research.google.com/ 46 | [jupyter-notebook]: https://nbviewer.jupyter.org/github/google/grr/blob/master/colab/examples/demo.ipynb 47 | -------------------------------------------------------------------------------- /investigating-with-grr/osquery.md: -------------------------------------------------------------------------------- 1 | osquery 2 | ======= 3 | 4 | [osquery](https://osquery.io/) is a utility created by Facebook that exposes system information through an SQL API. By writing SQL queries one can list processes, devices, network connections, users or browse directory structure. All that data is well-structured and described by the associated [schema](https://osquery.io/schema/). By mixing that with capabilities of SQL we get a really powerful tool for analysts. 5 | 6 | Setup 7 | ----- 8 | 9 | The osquery executable has to be available on the target client. Usually, it is as easy as downloading [a package](https://osquery.io/downloads/official/) from the official website for appropriate platform and putting it somewhere where GRR can access it, e.g. `/opt/osquery/`. Then, the location of the main osquery executable has to be specified in the `Osquery.path` option in the GRR configuration: 10 | 11 | ```yaml 12 | (...) 13 | 14 | Platform:Linux: 15 | (...) 16 | 17 | Osquery.path: '/opt/osquery/3.3/usr/bin/osqueryi' 18 | 19 | (...) 20 | 21 | Platform:Windows: 22 | (...) 23 | 24 | Osquery.path: 'C:\\Program Files\\osquery\\osqueryd.exe' 25 | 26 | (...) 27 | ``` 28 | 29 | Note that it does not really matter whether you use `osqueryi` or `osqueryd`, both should work just fine. 30 | 31 | *Make sure you repack/redeploy GRR clients after the Osquery.path configuration setting is updated.* 32 | 33 | 34 | Investigating 35 | ------------- 36 | 37 | To run a query on the desired client, simply start a new flow under "Collectors > Osquery". The flow itself is very simple with "query" field being the important one. For example, to run a simple query that lists all the processes, you can use the following: 38 | 39 | ``` 40 | SELECT * FROM processes; 41 | ``` 42 | 43 | After a moment, the client should yield an SQL table with all the desired results. 44 | 45 | Of course, it is possible to use all SQL features, such as joins: 46 | 47 | ``` 48 | SELECT cmdline, total_size FROM osquery_info JOIN processes USING (pid); 49 | ``` 50 | 51 | Refer to the osquery [schema](https://osquery.io/schema/) documentation to discover what tables and columns are available. 52 | 53 | #### Options 54 | 55 | ##### Timeout 56 | 57 | The timeout field can be used to specify a time after which the query is killed if it is not able to produce any results. Usually, queries should return immediately and the default timeout should be more than enough. However, it is possible to write more sophisticated queries (e.g. ones that search through the file system) where it might make sense to increase it. 58 | 59 | #### Ignore `stderr` errors 60 | 61 | By default, GRR uses `stderr` output produced by osquery to determine whether the execution was successful or not (e.g. due to syntax errors in the specified query). However, occasionally osquery can also produce warning messages that will cause the entire flow to fail. In such cases, it might be desired to simply suppress them. 62 | 63 | Note that this option should be used with caution and not be ticked if not needed—otherwise errors in the query might go unnoticed. 64 | -------------------------------------------------------------------------------- /investigating-with-grr/output-plugins.md: -------------------------------------------------------------------------------- 1 | # Output Plugins 2 | This document outlines how to send flow and hunt results from GRR to other systems and formats for further analysis. 3 | 4 | Every GRR installation supports 5 | [many Output Plugins](https://github.com/google/grr/tree/master/grr/server/grr_response_server/output_plugins) by default 6 | to save flow/hunt results in different formats or send the results to other systems. 7 | 8 | ## Splunk 9 | 10 | GRR can send flow results as Events to Splunk's HTTP Event Collector. To set this up, add a new HTTP Event Collector in 11 | Splunk's Data Input Settings. Events automatically get `grr` as default source and 12 | `grr_flow_result` as default sourcetype unless you override this in Splunk's or GRR's configuration. 13 | 14 | Configure GRR to send data to your Splunk instance by specifying the following values in your server configuration YAML file. 15 | `url` refers to the absolute API URL of your Splunk installation, including scheme and port. `token` is the generated access 16 | token from the Splunk HEC settings. More configuration options, including how to deal with Splunk Cloud's self-signed 17 | certificates are found in 18 | [config/output_plugins.py](https://github.com/google/grr/blob/master/grr/core/grr_response_core/config/output_plugins.py). 19 | ```yaml 20 | Splunk.url: https://input-prd-p-123456788901.cloud.splunk.com:8088 21 | Splunk.token: 97e96c19-9bf1-4618-a079-37c567b577dc 22 | ``` 23 | 24 | After restarting your server, you can send flow and hunt results to Splunk, by enabling `SplunkOutputPlugin` when launching a 25 | flow/hunt. Here's an example event that will be sent to Splunk, where the analyst set `incident-123` as annotation for the 26 | flow. 27 | 28 | ```js 29 | { 30 | "annotations": ["incident-123"], 31 | "client": { 32 | "hostname": "max.example.com", 33 | "os": "Linux", 34 | "usernames": "max", 35 | // ... 36 | }, 37 | "flow": { 38 | "name": "ClientFileFinder", 39 | "creator": "sherlock", 40 | "flowId": "23B01B77", 41 | // ... 42 | }, 43 | "resultType": "FileFinderResult", 44 | "result": { 45 | "statEntry": { 46 | "stMode": "33261", 47 | "stSize": "1444", 48 | "pathspec": { 49 | "path": "/home/max/Downloads/rickmortyseason6earlyaccess.sh", 50 | // ... 51 | }, 52 | // ... 53 | } 54 | } 55 | } 56 | ``` 57 | 58 | ## BigQuery 59 | Our blog contains a 60 | [post explaining how to setup and use BigQuery](http://grr-response.blogspot.com/2015/11/using-bigquery-to-analyze-data.html). 61 | 62 | 63 | -------------------------------------------------------------------------------- /investigating-with-grr/overview.md: -------------------------------------------------------------------------------- 1 | 2 | # Overview 3 | 4 | Once set up and configured, the GRR user interface is a web 5 | interface which allows the analyst to search for connected client (agent) 6 | machines, examine what data has been collected from the machines and issue 7 | requests to collect additional data. 8 | 9 | The GRR server also provides access to this functionality through a JSON API. 10 | Client libraries to support scripting from python or go are provided. 11 | 12 | ## Starting Points 13 | 14 | Depending on the type of client data that the analyst is interested in, there 15 | are several places that they might start. 16 | 17 | ### Virtual File System 18 | 19 | The [virtual file system](vfs/virtual-file-system/) shows the files, directories, 20 | and registry entries which have already been collect from a client. It shows 21 | when the entry was collected, and provides some buttons to collect additional 22 | buttons of this sort. 23 | 24 | This is a natural starting point for ad-hock examination of an individual 25 | machine. 26 | 27 | ### Flows 28 | 29 | A [Flow](flows/what-are-flows.md) performs one or more operations on a client 30 | machine, in order to collect or check for data. For example, the data collection 31 | buttons shown by the [virtual file system](vfs/virtual-file-system/) start flows to 32 | collect specific files and directories. However, flows can do many other 33 | things - from searching a directory for files containing a particular substring, 34 | to recording the current network configuration. The administrative interface 35 | shows for each client the flows which have been launched against it also the 36 | flow's status and any results returned. 37 | 38 | When an analyst would like to collect a specific bit of information about a 39 | machine, they will need to directly or indirectly run a flow. 40 | 41 | ### Hunts 42 | 43 | A [Hunt](hunts/what-are-hunts.md) is a mechanism to run a Flow on a number of 44 | clients. For example, this makes it possible to check if any Windows machine in 45 | the fleet has a file with a particular name in a particular location. 46 | 47 | ### Artifacts 48 | An [Artifact](artifacts/overview.md) is a way to collect and name a group of 49 | files or other data that an analyst might want to collect as a unit. For 50 | example, an artifact might try to collect all the common linux persistence 51 | mechanisms. 52 | -------------------------------------------------------------------------------- /investigating-with-grr/pushing-code.md: -------------------------------------------------------------------------------- 1 | # Emergency Pushing of Code and Binaries 2 | 3 | GRR offers functionality to deploy custom code or binaries to the 4 | whole fleet quickly. It goes without saying that this functinality 5 | can lead to problems and should therefore be used with care - we 6 | recommend using this as a last resort means in case things have gone 7 | terribly bad and not relying on this functionality for normal 8 | operations. 9 | 10 | Binaries and python code can be pushed from the server to the 11 | clients to enable new functionality. 12 | The code that is pushed from the server must be signed by the 13 | corresponding private key for `Client.executable_signing_public_key` for 14 | python and binaries. These signatures will 15 | be checked by the client to ensure they match before the code is used. 16 | 17 | What is actually sent to the client is the code or binary wrapped in a 18 | protobuf which will contain a hash, a signature and some other 19 | configuration data. 20 | 21 | To sign code requires use of config\_updater utility. In a secure 22 | environment the signing may occur on a different box from the server, 23 | but the examples below show the basic example. 24 | 25 | ## Deploying Arbitrary Python Code. 26 | 27 | To execute an arbitrary python blob, you need to create a file with 28 | python code that has the following attributes: 29 | 30 | - Code in the file must work when executed by exec() in the context of 31 | a running GRR client. 32 | 33 | - Any return data that you want sent back to the server can be 34 | stored encoded as a string in a variable called 35 | "magic\_return\_str". Alternatively, GRR collects all output from 36 | print statements and puts them into the magic\_return\_string. 37 | 38 | E.g. as a simple example. The following code modifies the clients 39 | poll\_max setting and pings test.com. 40 | 41 | ``` python 42 | import commands 43 | status, output = commands.getstatusoutput("ping -c 3 test.com") 44 | config_lib.CONFIG.Set("Client.poll_max", 100) 45 | config_lib.CONFIG.Write() 46 | magic_return_str = "poll_max successfully set. ping output %s" % output 47 | ``` 48 | 49 | This file then needs to be signed and converted into the protobuf format 50 | required, and then needs to be uploaded to the data store. You can do 51 | this using the following command line. 52 | 53 | ``` docker 54 | grr_config_updater upload_python --file=myfile.py --platform=windows 55 | ``` 56 | 57 | The uploaded files live by convention in python\_hacks\ and are 58 | viewable in the "Binaries" section of the Admin UI. 59 | 60 | The ExecutePythonHack Flow is provided for executing the file on a 61 | client. This is visible in the Admin UI under Administrative flows if 62 | Advanced Mode is enabled. 63 | 64 | **Note**: Specifying arguments to a PythonHack is possible as well 65 | through the py\_args argument, this can be useful for making the hack 66 | more generic. 67 | 68 | ## Deploying Executables. 69 | 70 | The GRR Agent provides an ExecuteBinaryCommand Client Action which 71 | allows us to send a binary and set of command line arguments to be 72 | executed. Again, the binary must be signed using the executable 73 | signing key (config option PrivateKeys.executable\_signing\_private\_key). 74 | 75 | To sign an exe for execution use the config updater 76 | script. 77 | 78 | ``` docker 79 | db@host:$ grr_config_updater upload_exe --file=/tmp/bazinga.exe --platform=windows 80 | Using configuration 81 | Uploaded successfully to /config/executables/windows/installers/bazinga.exe 82 | db@host:$ 83 | ``` 84 | 85 | This file can then be executed with the LaunchBinary flow which is in 86 | the Administrative flows if Advanced Mode is enabled. 87 | -------------------------------------------------------------------------------- /investigating-with-grr/searching-for-client.md: -------------------------------------------------------------------------------- 1 | # Searching For A Client 2 | 3 | In order to start interfacing with a client, we first need to search for 4 | it in the GUI. The GRR search bar is located at the top of the GUI and 5 | allows you to search clients based on: 6 | 7 | - **Hostname:** "host:myhost-name" 8 | 9 | - **Fully Qualified Domain Name (FQDN):** 10 | "fqdn:myhost-name.organization.com", also prefixes of components, 11 | e.g. "fqdn:myhost-name.organization" 12 | 13 | - **MAC address:** "mac:eeffaabbccdd". 14 | 15 | - **IP address:** "ip:10.10.10.10", also prefixes of bytes "ip:10.10". 16 | Note that IP address is only collected during interrogate, which by 17 | default is run once per week. 18 | 19 | - **User:** "user:john" 20 | 21 | - **Label:** "label:testmachines". Finds hosts with a particular GRR 22 | label. 23 | 24 | - **Time of Last Data Update:** Time ranges can be given using 25 | "start\_date:" and "end\_date:" prefixes. The data is interpreted as 26 | a human readable timestamp. Examples: start\_date:2015, 27 | end\_date:2018-01-01. 28 | 29 | All of these keywords also work without the type specifier, though with 30 | less precision. For example "johnsmith" is both a user name and a 31 | hostname name, it will match both. 32 | 33 | Furthermore there are additional keywords such as OS and OS version. So 34 | "Windows" will find all windows machines and "6.1.7601SP1" will match 35 | Windows 7 machines with SP1 installed, "6.1.7601" will match those 36 | without a service pack. 37 | 38 | **By default, the search index only considers clients that have checked 39 | in during the last six months.** To override this behavior, use an 40 | explicit "start\_date:" directive as specified above. 41 | 42 | ## Interpreting Client Search Results 43 | 44 | Searching returns a list of clients with the following information about 45 | each one: 46 | 47 | - **Online**: An icon indicating whether the host is online or not. 48 | Green means online; yellow, offline for some time; red, offline for 49 | a long time. 50 | 51 | - **Subject**: The client IDentifier. This is how GRR refers 52 | internally to the system. 53 | 54 | - **Host**: The name of the host as the operating system sees it. 55 | 56 | - **Version**: The operating system version. 57 | 58 | - **MAC**: A list of MAC addresses of the system. 59 | 60 | - **Usernames**: A list of user accounts the operating system knows 61 | about (usually users local to the system or that have logged in). 62 | 63 | - **First Seen**: The time when the client first talked to the server. 64 | 65 | - **OS install time**: The timestamp for the operating system install. 66 | 67 | - **Labels**: Any labels applied to this client. 68 | 69 | - **Last Checkin**: The last time the client communicated with the 70 | server. 71 | 72 | Once you’ve found the client you were looking for, click on it and both 73 | the left panel and main panel will change to reflect you’re now working 74 | with a client. 75 | -------------------------------------------------------------------------------- /investigating-with-grr/security-considerations.md: -------------------------------------------------------------------------------- 1 | # Security Considerations 2 | 3 | As described in [Client-Server Communication](client-server-communication.md), 4 | communication between the GRR client and server is secure against eavesdropping 5 | and impersonation. Furthermore, the server's record of flows run and files 6 | downloaded is not changed after the fact. It is intended to provide a secure 7 | archive of data that has been gathered from machines. 8 | 9 | ## Root Privileges 10 | 11 | The GRR client agent normally runs as root, and by design is capable of reading any 12 | data on the system. Furthermore some flows can use significant resources on 13 | client (e.g. searching a large directory recursively) and in fact the GRR agent 14 | does have the ability to download and execute server provided code, though the 15 | code does need to be signed. 16 | 17 | For these reasons, access to the GRR server should be considered tantamount to 18 | have root access to all GRR client machines. While we provide auditing and 19 | authorization support to mitigate the risks of this, the risks inherient in this 20 | should be understood when configuring and using GRR. 21 | -------------------------------------------------------------------------------- /investigating-with-grr/vfs/index.md: -------------------------------------------------------------------------------- 1 | # Virtual File System 2 | 3 | ```{toctree} 4 | --- 5 | maxdepth: 1 6 | --- 7 | Intro 8 | Navigating the VFS 9 | Timelines 10 | ``` 11 | -------------------------------------------------------------------------------- /investigating-with-grr/vfs/navigation.md: -------------------------------------------------------------------------------- 1 | # Navigating the VFS # 2 | 3 | Using the tree navigator, an analyst can look at all the files / registry values *that GRR has already downloaded from the client*. This is important to note, the view only shows what the server has stored for the client machine, not the current state. Sometimes this means that directories or Registry keys show up empty but not because there are no keys / values but just because the corresponding information has never been collected. 4 | 5 | In order to populate the VFS, there are two buttons that trigger collection of more information: 6 | 7 | ![Refresh](../../images/refresh_buttons.png "Refresh Buttons") 8 | 9 | The left button triggers a refresh of the current branch of the VFS with data collected from the client - similar to listing a directory. GRR automatically figures out which kind of data is currently displayed (OS based files, TSK based files, Registry information) and schedules the corresponding collection action. 10 | 11 | The button marked with an R schedules a recursive refresh. 12 | 13 | > **Note** that recursive refrehses with a large depth might impact the client and GRR system. 14 | 15 | ## Looking at Files ## 16 | 17 | In the file detail view, GRR always shows Stat information about the file: 18 | 19 | ![Stat Information](../../images/stat_info.png "Stat Information") 20 | 21 | Note that there is a version dropdown on top of this page and all data is timestamped. Whenever GRR collects information from a client machine, it just creates a new version of the stored object in the database, old information is never overwritten. This is why it's always possible to go back in the GRR DB and look at previously collected data - both metadata and contents - by simply selecting an earlier version. 22 | 23 | Often, GRR knows about the file but has not collected any contents yet. In the download tab there is an option to collect the file from the client: 24 | 25 | ![Collect From Client](../../images/collect_from_client.png "Collect From Client") 26 | 27 | Again, GRR will collect the file the correct way (OS or TSK) automatically. Once the file is available, the same download view will have options to download the file to the analysts machine. Basic text and hex views are also provided. 28 | 29 | ![Collected From Client](../../images/collected_from_client.png "Collected From Client") 30 | 31 | ![Hex View](../../images/hexview.png "Hex View") 32 | 33 | In the tree view, files with actual contents stored in GRR will have a small downloaded icon next to the size information. In this screenshot, bunzip2 is the only file with data collected: 34 | 35 | ![Downloaded File](../../images/downloaded_file.png "Downloaded File") 36 | 37 | ### Bulk Downloading of Files ### 38 | 39 | GRR offers the option to conviniently download all files collected from a client for offline analysis. The button 40 | 41 | ![File Download Button](../../images/download.png "Download Button") 42 | 43 | can be used to 44 | 45 | - Download all files with data in the current directory and below or 46 | - Download all files with data ever collected from that client 47 | 48 | as a zip archive. The drop down choses between the two options. 49 | -------------------------------------------------------------------------------- /investigating-with-grr/vfs/timelines.md: -------------------------------------------------------------------------------- 1 | # Creating Timelines # 2 | 3 | The GRR VFS can be switched to a timeline view where all files *GRR knows about* are shown sorted by their various timestamps: 4 | 5 | ![Timeline](../../images/timeline.png "Timeline") 6 | 7 | Use the button on the top right in the VFS view to switch between Timeline and File List View. The drop down menu can be used to download the generated timeline as CSV. 8 | 9 | Note that 10 | 11 | - the timeline is always generated by recursively descending into directories starting from the currently selected one. 12 | - the timeline only contains files that GRR has downloaded metadata before. This timelining mechanism is fully server side, if you haven't collected the metadata yet, the recursive refresh button will do this for you. 13 | 14 | ## TimelineFlow ## 15 | 16 | The `TimelineFlow` provides a one-click solution for retrieving a snapshot (stat information for each file) of the entire filesystem. Once collected, the snapshot can be downloaded in [Sleuthkit's Body format](http://wiki.sleuthkit.org/index.php?title=Body_file) for further analysis. 17 | 18 | To use the `TimelineFlow`: 19 | 20 | 1. Start a `TimelineFlow` (can be found in `Collectors > Timeline`) 21 | 22 | 1. Once completed, run the GRR API shell to download the results: 23 | 24 | ```python 25 | from grr_response_proto.api import timeline_pb2 26 | grrapi.Client('C.0123456789abcdef').Flow('12345678') \ 27 | .GetCollectedTimeline(fmt=timeline_pb2.ApiGetCollectedTimelineArgs.BODY) \ 28 | .WriteToFile('/tmp/out.body') 29 | ``` 30 | -------------------------------------------------------------------------------- /investigating-with-grr/vfs/virtual-file-system.md: -------------------------------------------------------------------------------- 1 | # Virtual File System 2 | 3 | Every time GRR collects forensics information (files, Registry data, ...) from a client, it stores it server side in the data store. This information is presented to the analyst in a unified view we call the Virtual Filesystem view (menu item *Browse Virtual Filesystem*). 4 | 5 | A screenshot is shown below ([larger Image](https://github.com/google/grr-doc/raw/markdown-new/docs/images/vfs_full.png)). 6 | 7 | ![VFS view](../../images/vfs_full.png "VFS View") 8 | 9 | ### VFS Tree ### 10 | 11 | On the left there is a tree navigator. The most important top level items are: 12 | 13 | - *fs* This gives a view of the client's filesystem. 14 | - *fs/os* Under fs/os, GRR maps all the files it collected on a client *using the OS apis*, i.e., standard file manipulation system calls. Under fs/os, GRR shows everything that is mounted under / for Linux and Mac clients, for Windows clients, under fs/os there are entries like C: for each disk present on the client system. 15 | - *fs/tsk* GRR also uses The Sleuth Kit (TSK) to open disks and images in raw mode and parsing the file systems on them. Every file collected in that way will be presented under fs/tsk. The first level under fs/tsk will be a raw disk, for example sda1 for Linux or \\\\?\Volume{5f62dc05-4cfc-58fd-b82c-579426ac01b8} for Windows. 16 | - *registry* Only present for Windows machines, this branch gives a view into the live registry on the client machine. The first level after /registry are hives, HKEY_LOCAL_MACHINE or HKEY_USERS for example. 17 | 18 | Note how in the screenshot, the folders under /fs/tsk and /fs/os actually reference the same folder on disk ("/"). However, there is an additional folder startup-KoGcRS that is only visible when using TSK, it gets filtered by the operating system API. 19 | 20 | -------------------------------------------------------------------------------- /maintaining-and-tuning/approval-based-workflow.md: -------------------------------------------------------------------------------- 1 | # Approval-based workflow 2 | 3 | GRR has support for an approval-based access control system. To turn it on, set the *API.DefaultRouter* configuration option in the server configuration (see [API routers documentation](../maintaining-and-tuning/user-management/limiting-access-with-routers.md) for more details): 4 | 5 | ``` yaml 6 | API.DefaultRouter: ApiCallRouterWithApprovalChecks 7 | ``` 8 | 9 | Approval-based workflow places certain restrictions on what users can do without getting an explicit approval from another user. 10 | 11 | ## Actions that **don't require** an approval 12 | 13 | 1. Searching for clients. 14 | 1. Getting information about a client host (including the history). 15 | 1. Looking at the list of hunts. 16 | 1. Looking at hunt results, logs and other related data. 17 | 1. Looking at cron jobs. 18 | 19 | 20 | ## Actions that **do require** an approval 21 | 22 | 1. Browsing client's Virtual File System. 23 | 1. Inspecting client's flows. 24 | 1. Starting new flows on a client. 25 | 1. Starting a new hunt (**NOTE**: this is a special case - starting a hunt requires an approval from an [admin user](user-management/authentication.md)). 26 | 1. Downloading files collected by a hunt. 27 | 1. Starting a new cron job. 28 | 29 | ## Requesting an approval 30 | 31 | Whenever you try to perform an action that requires an approval in the GRR web UI, you'll see a popup dialog similar to this one: 32 | 33 | ![GRR approval dialog](../images/client-approval-dialog.png) 34 | 35 | You need to fill-in a comma-separated list of approvers. Each approver will get a notification about a pending approval request in their GRR web UI. 36 | 37 | You can also select *CC* option to send an email to a predefined email address. This helps if you have multiple possible approvers and they're all subscribed to the same mailing list. *CC* option will only work if you have a proper [Email Configuration](email-configuration) in your GRR server config. 38 | 39 | For every approval you also need to enter a reason you request one. This is done for auditing purposes. Also, when you access a client that you have an approval for, you'll see the reason displayed in GRR web UI's top left corner. 40 | 41 | ## Granting an approval 42 | 43 | Whenever somebody requests an approval, you'll get a notification in the GRR web UI: the notification button in the top right corner will turn red and display a number of pending notifications. 44 | 45 | When you click on a *Please grant access to <...>* notification, you'll see an approval request review page similar to this one: 46 | 47 | ![GRR approval review page](../images/client-approval-review-page.png) 48 | 49 | You can see the approval request reason on the review page and also some details about the object that the requestor is trying to access (a client, a hunt or a cron job). If you find the approval request justified, you click on the "Approve" button. The requestor will receive a notification then stating that the access was granted. 50 | -------------------------------------------------------------------------------- /maintaining-and-tuning/building-custom-client-templates.md: -------------------------------------------------------------------------------- 1 | # Building custom client templates 2 | 3 | ## Building the templates 4 | 5 | Note: templates have to be built on a target platform (i.e. if you need a 6 | Windows 10 GRR client template, you have to build on Windows 10). 7 | 8 | After 9 | [installing the GRR server components](../installing-and-running-grr/overview.md), 10 | you can build client templates with 11 | 12 | ```bash 13 | grr_client_build build --output mytemplates 14 | ``` 15 | 16 | If you can't install prebuilt Debian or PIP packages, you'd need to install 17 | GRR from source. You might choose to install the full server or just 18 | the packages needed to build the templates. Please see 19 | [Installing from source](../installing-and-running-grr/from-source.md) for details. 20 | 21 | 22 | ## Repacking the templates 23 | 24 | The templates are generic GRR client builds that have to be repacked to be used 25 | with a particular GRR server deployment. Repacking is a process that injects 26 | server-specific configuration (like the frontend address, labels, etc). 27 | 28 | While templates have to be built on the client target platform, repacking 29 | has to be done on your GRR server, as GRR server contains the configuration 30 | (i.e., crypto credentials for the clients to use). 31 | 32 | ```bash 33 | grr_client_build repack --template mytemplates/*.zip --output_dir mytemplates 34 | ``` 35 | 36 | ## Samples 37 | 38 | We have fully automated client builds for Linux, OSX and Windows. The scripts 39 | we use are mostly in the [Travis](https://github.com/google/grr/tree/master/travis) 40 | and [Appveyor](https://github.com/google/grr/tree/master/appveyor) 41 | directories on GitHub. Duplicating what we do there might give you custom clients 42 | with little effort. 43 | -------------------------------------------------------------------------------- /maintaining-and-tuning/configuration/contexts.md: -------------------------------------------------------------------------------- 1 | # Configuration Contexts 2 | 3 | The next important concept to understand is that of configuration 4 | contexts. GRR consists of many components, which share most of their 5 | configuration. Often, however, we want to specify slight differences 6 | between each component. If we were to write a separate configuration 7 | file say for building each client version (Currently, 3 operating systems, and 2 8 | architectures) as well as for each component (e.g. worker, web UI, 9 | frontend etc), we would end up with an unmaintainable mess of many 10 | configuration files. It would make more sense to put most of the common 11 | configuration parameters in the same file, and only specify the 12 | different ones, depending on the component which is running. 13 | 14 | GRR introduces the concept of running context. The context is a list of 15 | attributes about the currently running component. When the code accesses 16 | the configuration system, the configuration system considers the 17 | currently running context and returns the appropriate value of the 18 | required parameter. 19 | 20 | For example, consider the parameter `Logging.filename` which specifies the 21 | location of the log file. In a full GRR installation, we wish the GRR Admin 22 | UI to log to `/var/log/grr/adminui.log` while the frontend should log to 23 | `/var/log/grr/frontend.log`. We can therefore specify in the YAML config 24 | file: 25 | 26 | ``` yaml 27 | Logging.filename: /var/log/grr/grr_server.log 28 | 29 | AdminUI Context: 30 | Logging.filename: /var/log/grr/adminui.log 31 | 32 | Frontend Context: 33 | Logging.filename: /var/log/grr/httpserver.log 34 | 35 | Client Context: 36 | Logging.filename: /var/log/grr/grr_client.log 37 | 38 | Platform:Windows: 39 | Logging.filename: c:\\windows\\system32\\grr.log 40 | ``` 41 | 42 | When the *AdminUI* program starts up, it populates its context with 43 | `AdminUI Context`. This forces the configuration system to return the 44 | value specific to this context. Note that other components, such as the 45 | worker will set a context of *Worker Context*, which will not match any 46 | of the above overrides. Therefore the worker will simply log to the 47 | default of `/var/log/grr/grr_server.log`. 48 | 49 | Note that it is possible to have multiple contexts defined at the same 50 | time. So for example, GRR client running under windows will have the 51 | contexts, `Client Context` and `Platform:Windows`, while a GRR client 52 | running under Linux will have the context `Client Context` and 53 | `Platform:Linux`. When several possibilities can apply, the option which 54 | matches the most context parameters will be selected. So in the above 55 | example, linux and osx clients will have a context like `Client Context` 56 | and will select `/var/log/grr/grr_client.log`. On the other hand, a 57 | windows client will also match the `Platform:Windows` context, and will 58 | therefore select `c:\windows\system32\grr.log`. 59 | 60 | The following is a non-exhaustive list of available contexts: 61 | 62 | - *AdminUI Context*, *Worker Context*, *Frontend Context* etc. Are 63 | defined when running one of the main programs. 64 | 65 | - *Test Context* is defined when running unit tests. 66 | 67 | - *Arch:i386* and *Arch:amd64* are set when running a 32 or 64 bit 68 | GRR client binary. 69 | 70 | - *Installer Context* is defined when the windows installer is active 71 | (i.e. during installation) 72 | 73 | - *Platform:Windows*, *Platform:Linux*, *Platform:Darwin* are set when 74 | running GRR client under these platforms. 75 | 76 | The full list can be seen [here](https://github.com/google/grr/blob/master/grr/core/grr_response_core/config/contexts.py). 77 | -------------------------------------------------------------------------------- /maintaining-and-tuning/configuration/file-organization.md: -------------------------------------------------------------------------------- 1 | # Configuration file organization 2 | 3 | When GRR is installed on a system, a system wide, distribution specific, configuration file is also installed (by default `/etc/grr/grr_server.yaml`). This specifies the basic configuration of the GRR service (i.e. various distribution specific locations). However, the configuration typically needs to be updated with site specific parameters (for example new crypto keys, the `Client.control_urls` setting to allow the client to connect to the server, etc.) 4 | 5 | In order to avoid overwriting the user customized configuration files when GRR is updated in future, we write site specific configuration files to a location specified by the `Config.writeback` config option (by default `/etc/grr/server.local.yaml`). This local file only contains the parameters which are changed from the defaults, or the system configuration file. 6 | 7 | > **Note** 8 | > 9 | > The system configuration file is never modified, all updated configurations are always written to the writeback location. Do not edit the system configuration file as changes in this file will be lost when GRR is upgraded. 10 | 11 | You can see all available configuration options and their current values in the "Settings" pane in GRR's web UI. 12 | 13 | ![Settings View](../../images/settings_view.png) 14 | -------------------------------------------------------------------------------- /maintaining-and-tuning/configuration/filtering.md: -------------------------------------------------------------------------------- 1 | # Filtering 2 | 3 | The configuration system may be extended to provide additional 4 | functionality accessible from the configuration file. For example 5 | consider the following configuration file: 6 | 7 | ``` yaml 8 | Logging.path: "%(HOME|env)/grr.log" 9 | ``` 10 | 11 | This expansion sequence uses the *env* filter which expands a value 12 | from the environment. In this case the environment variable *HOME* 13 | will be expanded into the `Logging.path` parameter to place the log 14 | file in the user’s home directory. 15 | 16 | There are a number of additional interesting filters. The *file* filter 17 | allows including other files from the filesystem into the configuration 18 | file. For example, some people prefer to keep their certificates in 19 | separate files rather than paste them into the config file: 20 | 21 | ``` yaml 22 | CA.certificate: "%(/etc/certificates/ca.pem|file)" 23 | ``` 24 | 25 | It is even possible to nest expansion sequences. For example this 26 | retrieves the client’s private key from a location which depends on the 27 | client name: 28 | 29 | ``` yaml 30 | Client.private_key: "%(/etc/grr/client/%(Client.name)/private_key.pem|file)" 31 | ``` 32 | -------------------------------------------------------------------------------- /maintaining-and-tuning/configuration/index.md: -------------------------------------------------------------------------------- 1 | # Changing GRR server configuration 2 | 3 | ```{toctree} 4 | --- 5 | maxdepth: 1 6 | --- 7 | Configuration file organization 8 | Configuration contexts 9 | Parameter Expansion 10 | Filtering 11 | ``` 12 | 13 | -------------------------------------------------------------------------------- /maintaining-and-tuning/configuration/parameter-expansion.md: -------------------------------------------------------------------------------- 1 | # Parameter Expansion 2 | 3 | The GRR configuration file format allows for expansion of configuration 4 | parameters inside other parameters. For example consider the following 5 | configuration file: 6 | 7 | ```docker 8 | Client.name: GRR 9 | 10 | Nanny.service_name: "%(Client.name)service.exe" 11 | Nanny.service_key_hive: HKEY_LOCAL_MACHINE 12 | Nanny.service_key: Software\\%(Client.name) 13 | Nanny.child_command_line: | 14 | %(child_binary) --config "%(Client.install_path)\\%(Client.binary_name).yaml" 15 | ``` 16 | 17 | The expansion sequence `%(parameter_name)` substitutes or expands the 18 | parameter into the string. This allows us to build values automatically 19 | based on other values. For example above, the `Nanny.service_name` will 20 | be `GRRservice.exe` and the `service_key` will be set to `Software\GRR` 21 | 22 | Expansion is very useful, but sometimes it gets in the way. For example, 23 | if we need to pass literal % escape sequences. Consider the 24 | Logging.format parameter which is actually a python format 25 | string: 26 | 27 | ```docker 28 | Logging.format: \%(levelname\)s \%(module\)s:\%(lineno\)s] \%(message\)s 29 | Logging.format: %{%(levelname)s %(module)s:%(lineno)s] %(message)s} 30 | ``` 31 | 32 | - This form escapes GRR’s special escaping sequence by preceding both 33 | opening and closing sequences with the backslash character. 34 | 35 | - This variation uses the literal expansion sequence *%{}* to declare 36 | the entire line as a literal string and prevent expansion. 37 | -------------------------------------------------------------------------------- /maintaining-and-tuning/email-configuration.md: -------------------------------------------------------------------------------- 1 | # Email configuration 2 | 3 | This section assumes you have already installed an MTA, such as [Postfix](http://www.postfix.org/) or [nullmailer](http://untroubled.org/nullmailer/). After you have successfully tested your mail transfer agent, please proceed to the steps outlined below. 4 | 5 | To configure GRR to send emails for reports or other purposes: 6 | 7 | Ensure email settings are correct by running back through the configuration script if needed (or by checking `/etc/grr/server.local.yaml`): 8 | 9 | ``` bash 10 | grr_config_updater initialize 11 | ``` 12 | 13 | Edit ```/etc/grr/server.local.yaml``` to include the following at the end of the file: 14 | 15 | ``` yaml 16 | Worker.smtp_server: 17 | Worker.smtp_port: 18 | ``` 19 | 20 | and, if needed, 21 | 22 | ``` yaml 23 | Worker.smtp_starttls: True 24 | Worker.smtp_user: 25 | Worker.smtp_password: 26 | ``` 27 | 28 | After configuration is complete, restart the GRR worker(s). You can test this configuration by running *OnlineNotification" flow on any of the online clients (in the "Administrative" category). This flow sends an email to a given email address as soon as the client goes online. If the client is already online, the email will be sent next time it checks in (normally within 10 minutes). 29 | 30 | ## Custom Email addresses 31 | 32 | When GRR sends an Email to a GRR user, the destination Email address defaults to `@`. Starting with version [3.4.2.0](https://grr-doc.readthedocs.io/en/latest/release-notes.html#july-06-2020), GRR can alternatively use custom Email addresses. In this mode, GRR will store a separate Email address for each user in the data store. To enable this mode, set the following option in the GRR config: 33 | 34 | 35 | ```yaml 36 | Email.enable_custom_email_address: true 37 | ``` 38 | 39 | The value of the custom Email addresses can be set in one of the following ways: 40 | 41 | 1. When using `RemoteUserWebAuthManager` for authentication, the Email address can be passed to GRR in the `X-Remote-Extra-Email` HTTP header. (The actual of the header is configurable using the `AdminUI.remote_email_header` option). 42 | 43 | 1. Using the GRR root API: 44 | 45 | ```bash 46 | grr_api_shell_raw_access --exec_code "grrapi.root.CreateGrrUser(username='foo', email='foo@bar.org')" 47 | ``` 48 | 49 | ```bash 50 | grr_api_shell_raw_access --exec_code "grrapi.root.GrrUser('foo').Get().Modify(email='foo2@bar.org')" 51 | ``` 52 | -------------------------------------------------------------------------------- /maintaining-and-tuning/grr-datastore.md: -------------------------------------------------------------------------------- 1 | # GRR datastore 2 | 3 | GRR needs a database backend in order to run. Previously, GRR supported 4 | SQLite and MySQL databases, but support for SQLite was dropped after 5 | GRR [v3.2.3.2](https://grr-doc.readthedocs.io/en/v3.2.3/release-notes.html) was 6 | released. MySQL is now GRR's only supported database backend. 7 | 8 | For users that were using the obsoleted *SQLiteDataStore*, please note that 9 | GRR does not have functionality for migrating data to the MySQL datastore. No 10 | need to fret though - a client re-deployment will not be necessary. Existing 11 | clients will still be able to talk to the GRR server when starting off with a 12 | new database. Client re-enrollment and interrogation should happen 13 | automatically. 14 | 15 | ## AFF4 deprecation 16 | 17 | Starting from the version **3.3.0** GRR uses a new datastore format by default - **REL_DB**. REL_DB is backwards-incompatible with the now-deprecated AFF4 datastore format (even though they both use MySQL as a backend). 18 | 19 | Use of AFF4-based deployments is now discouraged. REL_DB is expected to be much more stable and performant. 20 | 21 | ## Switching an AFF4-based deployment to use REL_DB 22 | 23 | To check which datastore implementation you're using: go to "Setting" in the GRR AdminUI and check the "Database" section. If `Database.aff4_enabled` is set to `0` and `Database.enabled` is set to `1`, you're running REL_DB. 24 | 25 | ***Note (IMPORTANT!):*** *There's no established data migration process to migrate the data from AFF4 data storage to REL_DB. You'll lose all the historical data when you switch an existing deployment to use REL_DB. ***However, the clients, will continue to work without issues.*** They will show up in the new datastore first time they check in (normally every 10 minutes).* 26 | 27 | To switch an existing GRR deployment to REL_DB. 28 | 29 | 1. Backup your MySQL database and GRR server configuration files *(this is a good idea for any kind of GRR upgrade, ideally you should also have regular backups set up)*. 30 | 31 | 1. Upgrade GRR to a version greater or equal than 3.3.0.0. 32 | 33 | 1. Run the following to update the GRR server configuration file: 34 | 35 | ```bash 36 | sudo grr_config_updater switch_datastore 37 | # Add at least one admin user to the empty REL_DB datastore. 38 | sudo grr_config_updater add_user --admin admin 39 | ``` 40 | 41 | 4. Answer the questions asked by the tool. 42 | 43 | 5. Restart GRR server. 44 | 45 | ``` 46 | sudo service grr-server restart 47 | ``` 48 | 49 | At this point GRR will run with an empty REL_DB datastore. Your clients will still be able to talk to the server and will appear in the AdminUI as soon as they check in (normally within 10 minutes). 50 | 51 | ## Switching back to AFF4-based deployment 52 | 53 | If for some reason you had an issue with switching to REL_DB and would like to go back to using AFF4 datastore at your GRR deployment, edit your configuration file at `/etc/grr/server.local.yaml` and make sure you have the settings below set correctly (or simply restore the file to its backed up version): 54 | 55 | ```yaml 56 | Database.aff4_enabled: True 57 | Database.enabled: False 58 | # Ignore the following line if you had custom 59 | # Blobstore.implementation setting before. 60 | Blobstore.implementation: MemoryStreamBlobStore 61 | ``` 62 | -------------------------------------------------------------------------------- /maintaining-and-tuning/index.md: -------------------------------------------------------------------------------- 1 | # Maintaining and tuning GRR deployment 2 | 3 | ```{toctree} 4 | --- 5 | maxdepth: 1 6 | --- 7 | Changing GRR server configuration 8 | Key management 9 | User management 10 | Email configuration 11 | Monitoring 12 | Approval-based access control 13 | Repacking GRR clients 14 | Low-level maintenance with grr_console 15 | GRR datastore 16 | Scaling GRR 17 | Building custom client templates 18 | Troubleshooting 19 | ``` 20 | -------------------------------------------------------------------------------- /maintaining-and-tuning/key-management/index.md: -------------------------------------------------------------------------------- 1 | # Key management 2 | 3 | ```{toctree} 4 | --- 5 | maxdepth: 1 6 | --- 7 | Which keys are used in GRR and how 8 | Rotating the keys 9 | ``` 10 | -------------------------------------------------------------------------------- /maintaining-and-tuning/key-management/rotating.md: -------------------------------------------------------------------------------- 1 | # Rotating the keys 2 | 3 | ## Rotating Client Keys ## 4 | 5 | The keys used by the fleetspeak client to talk to the server do not have to be / cannot be rotated since the client id depends directly on the private key. Just removing the fleetspeak state file results in the client generating a new key pair but the client will also get a new client id. 6 | 7 | Client keys used for signing executables and drivers can be changed in the config as described in the [documentation about keys](which-keys-and-how.md) but 8 | 9 | - Clients need to be repacked and redeployed so they can use the new keys. 10 | - Old clients on the fleet will not be able to use the new keys anymore so there will be some issues during transition. 11 | 12 | 13 | ## Rotating Server Keys ## 14 | 15 | Issuing a new server key can be done using the 16 | ```docker 17 | grr_config_updater rotate_server_key 18 | ``` 19 | command. This will update the server configuration with a new private key that is signed by the same CA. Existing clients will not lose connectivity by changing those keys even though old clients might need to be restarted before they will accept the new server credentials. 20 | 21 | **Note** Changing the server key will invoke GRRs certificate revocation process. All clients that have seen the new server certificate will from that point on refuse to accept the previous key so it's not possible to revert to the old key. 22 | 23 | -------------------------------------------------------------------------------- /maintaining-and-tuning/low-level-maintenance-with-console.md: -------------------------------------------------------------------------------- 1 | # Low-level maintenance with grr_console 2 | 3 | GRR is shipped with a *grr_console* binary. This is effectively an IPython shell that gets initialized with a few basic GRR objects and offers you direct access to internal GRR classes. 4 | 5 | In cases when there's no proper tool or UI or API for performing a certain task, grr_console may be useful. However, given that it exposes undocumented GRR internals and gives you complete unrestricted and unaudited access to GRR datastore, it's important to remember that: 6 | 7 | 1. It's very easy to shoot yourself in the foot and make the system unusable by running a wrong piece of code in *grr_console*. 8 | 1. GRR is a project that's being actively developed. Class names, import paths, interfaces are changing all the time. Even if you have a snippet of code that works in *grr_console* today, it may stop working in a month. 9 | 10 | ## Console Snippets (might break any time) 11 | 12 | ### Delete a client from the database 13 | 14 | ```python 15 | In [1]: print data_store.REL_DB.ReadClientSnapshot(u"C.83558528f50f993e").knowledge_base.fqdn 16 | example.host.com 17 | 18 | In [2]: data_store.REL_DB.DeleteClient(u"C.83558528f50f993e") 19 | 20 | ``` 21 | 22 | ### Delete all signed binaries from the database 23 | 24 | ```python 25 | In [1]: ids = data_store.REL_DB.ReadIDsForAllSignedBinaries() 26 | 27 | In [2]: for i in ids: 28 | ...: data_store.REL_DB.DeleteSignedBinaryReferences(i) 29 | ...: 30 | 31 | ``` 32 | -------------------------------------------------------------------------------- /maintaining-and-tuning/scaling.md: -------------------------------------------------------------------------------- 1 | ## Scaling GRR within a single server 2 | 3 | For 3.1.0 and later, use a [systemd drop-in 4 | override](http://askubuntu.com/questions/659267/how-do-i-override-or-configure-systemd-services) 5 | to control how many copies of each component you run on each machine. 6 | This can initially be done using: 7 | 8 | ```docker 9 | sudo systemctl edit grr-server 10 | ``` 11 | 12 | which creates "/etc/systemd/system/grr-server.service.d/override.conf". 13 | You’ll want to turn this into a template file and control via puppet or 14 | similar. An example override that just runs 3 workers looks like: 15 | 16 | [Service] 17 | ExecReload= 18 | ExecReload=/bin/systemctl --no-block reload grr-server@worker.service grr-server@worker2.service grr-server@worker3.service 19 | ExecStart= 20 | ExecStart=/bin/systemctl --no-block start grr-server@worker.service grr-server@worker2.service grr-server@worker3.service 21 | ExecStop= 22 | ExecStop=/bin/systemctl --no-block stop grr-server@worker.service grr-server@worker2.service grr-server@worker3.service 23 | 24 | When starting multiple copies of the UI and the web frontend you also 25 | need to tell GRR which ports it should be using. So if you want 10 http 26 | frontends on a machine you would configure your systemd drop-in to start 27 | 10 copies and then set Frontend.port\_max so that you have a range of 10 28 | ports from Frontend.bind\_port. (I.E. set Frontend.bind\_port to 8080 29 | and Frontend.port\_max to 8089) You can then configure your load 30 | balancer to distribute across that port range. AdminUI.port\_max works 31 | the same way for the UI. 32 | 33 | ## Large Scale Deployment 34 | 35 | The [GRR server components](implementation.md#grr-component-overview) 36 | should be distributed across multiple machines in any deployment where 37 | you expect to have more than a few hundred clients, or even smaller 38 | deployments if you plan on doing intensive hunting. The performance 39 | needs of the various components are discussed 40 | below, and some real-world example 41 | deployment configurations are [described in the FAQ](../faq.md). 42 | 43 | You should install the GRR package on all machines and use configuration 44 | management (chef, puppet etc.) to: 45 | 46 | - Distribute the same grr-server.yaml to each machine 47 | 48 | - Control how many of each component to run on each machine (see next 49 | section for details) 50 | 51 | For scaling the fleetspeak communication framework, see [the instructions in 52 | the fleetspeak section](../fleetspeak/scaling.md). 53 | 54 | 55 | ## Component Performance Needs 56 | 57 | - **Worker**: you will probably want to run more than one worker. In a 58 | large deployment where you are running numerous hunts it makes sense 59 | to run 20+ workers. As long as the datastore scales, the more 60 | workers you have the faster things get done. We previously had a 61 | config setting that forked worker processes off, but this turned out 62 | to play badly with datastore connection pools, the stats store, and 63 | monitoring ports so it was removed. 64 | 65 | - **HTTP frontend**: The frontend http server can be a significant 66 | bottleneck. By default we ship with a simple http server, but this 67 | is single process, written in python which means it may have thread 68 | lock issues. To get better performance you will need to run multiple 69 | instances of the HTTP frontend behind a reverse HTTP proxy (i.e. Apache 70 | or Nginx). Assuming your datastore handles it, these should scale 71 | linearly. 72 | 73 | - **Web UI**: The admin UI component is usually under light load, but 74 | you can run as many as you want for redundancy (you'll need to run them 75 | behind Apache or Nginx to load-balance the traffic). The more concurrent 76 | GRR users you have, the more instances you need. This is also the 77 | API server, so if you intend to use the API heavily run more. 78 | -------------------------------------------------------------------------------- /maintaining-and-tuning/screenshots/add_notification_channel.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/google/grr-doc/071f6d050597c5f35f42731f1d8461f71f28dd07/maintaining-and-tuning/screenshots/add_notification_channel.png -------------------------------------------------------------------------------- /maintaining-and-tuning/troubleshooting.md: -------------------------------------------------------------------------------- 1 | # Troubleshooting 2 | Please check the output of your system's service logs and/or GRR logging directory (default: 3 | `/usr/share/grr-server/lib/python2.7/site-packages/grr_response_core/var/log/`) for more information regarding failures. 4 | 5 | ## The upstart/init.d scripts show no output 6 | 7 | When I run an init.d script e.g. "/etc/init.d/grr-http-server start" (deprecated) it 8 | does not show me any output. 9 | 10 | Make sure that the "START" parameter in the corresponding default file, 11 | e.g. "/etc/default/grr-http-server", has been changed to "yes". 12 | 13 | ## I cannot start any/some of the GRR services using the service scripts 14 | 15 | When I run an init.d script e.g. "/etc/init.d/grr-http-server start" (deprecated) it 16 | indicates it started the service although when I check with 17 | "/etc/init.d/grr-http-server status" it says it is not running. 18 | 19 | You can troubleshoot by running the services in the foreground, e.g. to 20 | run the HTTP Front-end server in the foreground: 21 | 22 | ``` bash 23 | sudo grr_server --component [frontend|adminui|worker] --verbose 24 | ``` 25 | 26 | ## Any/some of the GRR services are not running correctly 27 | 28 | Check if the logs contain an indication of what is going wrong. 29 | 30 | Troubleshoot by running the services in the foreground, e.g. to run the 31 | UI in the foreground: 32 | 33 | ``` bash 34 | sudo grr_server --component [frontend|adminui|worker] --verbose 35 | ``` 36 | ## deprecated 37 | 38 | 39 | ## Cannot open libtsk3.so.3 40 | 41 | error while loading shared libraries: libtsk3.so.3: cannot open shared 42 | object file: No such file or directory 43 | 44 | The libtsk3 library cannot be found in the ld cache. Check if the path 45 | to libtsk3.so.3 is in /etc/ld.so.conf (or equivalent) and update the 46 | cache: 47 | 48 | ``` bash 49 | sudo ldconfig 50 | ``` 51 | -------------------------------------------------------------------------------- /maintaining-and-tuning/user-management/authentication.md: -------------------------------------------------------------------------------- 1 | # Users in GRR 2 | 3 | ## Concept 4 | 5 | GRR has a concept of users of the system. The GUI supports basic authentication and this verfication of user identity is used in all auditing functions (so for example GRR can properly record which user accessed which client, and who executed flows on clients). **There is no logout button. To change users you will need to invalidate your session.** 6 | 7 | A GRR user will be created as an admin by default. This is only important if the [approval-based workflow](../approval-based-workflow.md) is turned on, since only "admin" users can approve hunts. We are aware of a bug that all users are created with admin permissions, however, the approval system is the authority for access across GRR, when enabled. 8 | 9 | To add the user joe as an admin: 10 | 11 | ``` 12 | db@host:~$ sudo grr_config_updater add_user joe 13 | Using configuration 14 | Please enter password for user 'joe': 15 | Updating user joe 16 | 17 | Username: joe 18 | Password: set 19 | ``` 20 | 21 | To list all users: 22 | 23 | ``` 24 | db@host:~$ sudo grr_config_updater show_user 25 | Using configuration 26 | 27 | Username: test 28 | Is Admin: True 29 | ``` 30 | 31 | To update a user password: 32 | 33 | ``` 34 | db@host:~$ sudo grr_config_updater update_user joe --password 35 | Using configuration 36 | Updating user joe 37 | 38 | Username: joe 39 | Is Admin: True 40 | ``` 41 | Available commands are (--admin is currently not functional in v3.3.0.0): 42 | 43 | ``` 44 | usage: grr_config_updater update_user [-h] [--helpfull] [--password] [--admin] 45 | username 46 | 47 | positional arguments: 48 | username Username to update. 49 | 50 | optional arguments: 51 | -h, --help show this help message and exit 52 | --helpfull show full help message and exit 53 | --password Reset the password for this user (will prompt for password). 54 | --admin Make the user an admin, if they aren't already. 55 | ``` 56 | ## Authentication 57 | 58 | The AdminUI uses HTTP Basic Auth authentication by default, based on the passwords within 59 | the user objects stored in the data store, but we **don't expect you to use this 60 | in production** (see [Securing Access](../../installing-and-running-grr/securing-access.md) for more details). 61 | 62 | There is so much diversity and customization in enterprise 63 | authentication schemes that there isn't a good way to provide a solution that 64 | works for a majority of users. Large enterprises most likely already have internal webapps 65 | that use authentication, this is just one more. Most people have found the 66 | easiest approach is to sit Apache (or similar) in front of the GRR Admin UI as 67 | a reverse proxy and use an existing SSO plugin that already works for that 68 | platform. Alternatively, with more work you can handle auth inside GRR by 69 | writing a Webauth Manager (`AdminUI.webauth_manager` config option) that uses an 70 | SSO or SAML based authentication mechanism. 71 | -------------------------------------------------------------------------------- /maintaining-and-tuning/user-management/index.md: -------------------------------------------------------------------------------- 1 | # User management 2 | 3 | ```{toctree} 4 | --- 5 | maxdepth: 1 6 | --- 7 | User authentication 8 | Running GRR UI behind Apache 9 | Running GRR UI behind Nginx 10 | Limiting access to GRR UI/API with API routers 11 | ``` 12 | -------------------------------------------------------------------------------- /maintaining-and-tuning/user-management/limiting-access-with-routers.md: -------------------------------------------------------------------------------- 1 | # Limiting access to GRR UI/API with API routers 2 | 3 | GRR has a notion of *API call routers*. Every API request that GRR server handles is processed in a following fashion: 4 | 5 | 1. A username of a user making the request is matched against rules in the router configuration file (as defined in the `API.RouterACLConfigFile` config option). 6 | - If router configuration file is not set, a default router is used. Default router is specified in the `API.DefaultRouter` configuration option. 7 | - If one of the rules in the `API.RouterACLConfigFile` matches, then a router from the matching rule is used. 8 | 2. The API router is used to process the request. 9 | 10 | GRR has a few predefined API routers: 11 | 12 | - *DisabledApiCallRouter* - a router that will return an error for all possible requests. 13 | - *ApiCallRouterWithoutChecks* *(default)* - a router that performs no access checks and just gives blanket access to the system. 14 | - *ApiCallRouterWithApprovalChecks* - a router that enables GRR approvals-based workflow for better auditing. See [Approval-based workflow](../approval-based-workflow.md) for more details. 15 | - *ApiCallRobotRouter* - a router that provides limited access to the system that is suitable for automation. Normally used together with *ApiCallRouterWithApprovalChecks* so that scripts and robots can perform actions without getting approvals. 16 | 17 | ## Various configuration scenarios 18 | 19 | ### All GRR users should have unrestricted access *(default)* 20 | 21 | In GRR server configuration file: 22 | 23 | ``` yaml 24 | API.DefaultRouter: ApiCallRouterWithoutChecks 25 | ``` 26 | 27 | ### All GRR users should follow audit-based workflow 28 | 29 | In GRR server configuration file: 30 | 31 | ``` yaml 32 | API.DefaultRouter: ApiCallRouterWithApprovalChecks 33 | ``` 34 | 35 | ### GRR users should follow audit-based workflow, but user "john" should have blanket access. 36 | 37 | In GRR server configuration file: 38 | 39 | ``` yaml 40 | API.RouterACLConfigFile: /etc/grr_api_acls.yaml 41 | API.DefaultRouter: ApiCallRouterWithApprovalChecks 42 | ``` 43 | 44 | In `/etc/grr_api_acls.yaml`: 45 | 46 | ``` yaml 47 | router: "ApiCallRouterWithoutChecks" 48 | users: 49 | - "john" 50 | ``` 51 | 52 | **NOTE**: for example, you can set up user 'john' as a robot user, so that his credentials are used by scripts talking to the GRR API. 53 | 54 | 55 | Sample api_acls.yaml: 56 | 57 | ``` 58 | router: "ApiCallRobotRouter" 59 | users: 60 | - "serviceAccount1" 61 | 62 | router: "ApiCallRouterWithApprovalChecks" 63 | users: 64 | - "test" 65 | - "test2" 66 | 67 | router: "ApiCallRouterWithoutChecks" 68 | users: 69 | - "adminuser1" 70 | - "adminuser2" 71 | 72 | ``` 73 | -------------------------------------------------------------------------------- /maintaining-and-tuning/user-management/running-behind-apache.md: -------------------------------------------------------------------------------- 1 | # Running GRR UI behind Apache 2 | 3 | Running apache as a reverse proxy in front of the GRR admin UI is a good 4 | way to provide SSL protection for the UI traffic and also integrate with 5 | corporate single sign on (if available), for authentication. 6 | 7 | To set this up: 8 | 9 | - Buy an SSL certificate, or generate a self-signed one if you’re only 10 | testing. 11 | 12 | - Place the public key into “/etc/ssl/certs/“ and ensure it’s world 13 | readable 14 | 15 | ```docker 16 | chmod 644 /etc/ssl/certs/grr_ssl_certificate_filename.crt 17 | ``` 18 | 19 | - Place the private key into “/etc/ssl/private” and ensure it is **NOT** 20 | world readable 21 | 22 | ```docker 23 | chmod 400 /etc/ssl/private/grr_ssl_certificate_filename.key 24 | ``` 25 | 26 | - Install apache2 and required modules 27 | 28 | ```docker 29 | apt-get install apache2 30 | a2enmod proxy 31 | a2enmod ssl 32 | a2enmod proxy_http 33 | ``` 34 | 35 | - Disable any default apache files currently enabled (probably 36 | 000-default.conf, but check for others that may interfere with GRR) 37 | 38 | ```docker 39 | a2dissite 000-default 40 | ``` 41 | 42 | - Redirect port 80 HTTP to 443 HTTPS 43 | 44 | - Create the file "/etc/apache2/sites-available/redirect.conf" and copy 45 | the text below into it. 46 | 47 | ```docker 48 | 49 | Redirect "/" "https://" 50 | 51 | ``` 52 | 53 | - Create the file "/etc/apache2/sites-available/grr\_reverse\_proxy.conf" 54 | and copy the text below into it. 55 | 56 | ```docker 57 | 58 | SSLEngine On 59 | SSLCertificateFile /etc/ssl/certs/grr_ssl_certificate_filename.crt 60 | SSLCertificateKeyFile /etc/ssl/private/grr_ssl_certificate_filename.key 61 | ProxyPass / http://127.0.0.1:8000/ 62 | ProxyPassReverse / http://127.0.0.1:8000/ 63 | 64 | ``` 65 | 66 | - Enable the new apache files 67 | 68 | ```docker 69 | a2ensite redirect.conf 70 | a2ensite grr_reverse_proxy.conf 71 | ``` 72 | 73 | - Restart apache 74 | 75 | ```docker 76 | service apache2 restart 77 | ``` 78 | 79 | **NOTE**: This reverse proxy will only proxy the AdminUI. It will have 80 | no impact on the agent communications on port 8080. It is advised to 81 | restrict access to the AdminUI at the network level. 82 | -------------------------------------------------------------------------------- /publications.md: -------------------------------------------------------------------------------- 1 | # Publications 2 | 3 | ## Papers 4 | 5 | [A scalable file based data store for forensic 6 | analysis](https://storage.googleapis.com/docs.grr-response.com/scalable_datastore.pdf), 7 | by Flavio Cruz, Andreas Moser, Michael Cohen. DFRWS 2015 Europe. 8 | 9 | [Hunting in the enterprise: Forensic triage and incident 10 | response](https://storage.googleapis.com/docs.grr-response.com/MoserCohenHunting.pdf), 11 | by Moser, Andreas, and Michael I. Cohen. Digital Investigation, 2013. 12 | 13 | [Distributed forensics and incident response in the 14 | enterprise](https://storage.googleapis.com/docs.grr-response.com/distributed_forensics.pdf), 15 | by M.I. Cohen, D. Bilby, G. Caronni. Digital Investigation, 2011. 16 | 17 | ## Presentations 18 | 19 | [GRR Meetup: GRR Users Meetup: Fleetspeak, API client, Go rewrite](https://storage.googleapis.com/docs.grr-response.com/GRR%20Meetup-%20Mar%202017.pdf) @ Mar 2017 GRR Meetup by A. Moser, M. Bushkov, B. Galehouse, M. Lakomy. [Video](https://youtu.be/SIvf7-Lzp2M). 20 | 21 | [GRR Meetup: 3.1.0 Release](https://storage.googleapis.com/docs.grr-response.com/GRR%20Meetup-%20Server-Client%20Release%20Apr%202016.pdf) @ Apr 2016 GRR Meetup by G. Castle and M. Cohen. [Video](https://youtu.be/SIvf7-Lzp2M). 22 | 23 | [GRR Meetup: API Edition](https://storage.googleapis.com/docs.grr-response.com/GRR%20Meetup-%20API%20Edition%20Nov%202015.pdf) @ Nov 2015 GRR Meetup by G. Castle and M. Bushkov. Covers API basics and discusses where we are headed with API dev. 24 | 25 | [GRR Hunting Best Practices](https://storage.googleapis.com/docs.grr-response.com/GRR%20Hunting%20for%20meetup%20Oct%202015.pdf) @ Oct 2015 GRR Meetup by G. Castle. 26 | 27 | [Tactical use of GRR Artifacts, Fuse, Plaso and Timesketch](https://storage.googleapis.com/docs.grr-response.com/ACSC%202015-%20Defending%20the%20Gibson%20in%202015.pdf) by D. Bilby [Demo Video (youtube)](https://www.youtube.com/watch?v=JciAp0uB7AY). 28 | 29 | "[Intro to GRR](https://storage.googleapis.com/docs.grr-response.com/OpenNSM_GRR_Presentation_Feb_2015.pdf)" for the Open Source Network Security Monitoring Group @ UIUC Feb 2015. [Video](https://www.youtube.com/watch?v=ZAXtCMspgY0) by G. Castle 30 | 31 | "[GRR: Find all the badness, collect all the 32 | things](https://storage.googleapis.com/docs.grr-response.com/GRR_Blackhat_2014_Greg_Castle.pdf)" 33 | @ Blackhat 2014 [Video](http://www.youtube.com/watch?v=DudGrSv26NY) by G. 34 | Castle. Also presented [a 20 min version of the same 35 | talk](https://youtu.be/eLehQ3QmdEA) for [Hacker 36 | Hotshots](https://www.concise-courses.com/past/). 37 | 38 | [OSDFC 2012 GRR Overview](https://storage.googleapis.com/docs.grr-response.com/GRR%20Rapid%20Response%20-%20OSFC%202012.pdf), by D. Bilby 39 | 40 | ## Workshops 41 | 42 | GRR Workshop at CERN, February 2018 [slides](https://storage.googleapis.com/docs.grr-response.com/GRR%20Workshop%20%40%20CERN%202018.pdf) by M. Bushkov, B. Galehouse, M. Lakomy, and A. Moser 43 | 44 | ICDF2C Prague 2017 Workshop [slides](https://storage.googleapis.com/docs.grr-response.com/GRR%20Workshop%20%40%20ICDF2C%20Prague%202017.pdf) by M. Bushkov and B. Galehouse 45 | 46 | DFRWS US 2015 Workshop 47 | [slides](https://storage.googleapis.com/docs.grr-response.com/GRR_Workshop_DFRWS_US_2015.pdf) 48 | and [scripts to setup your own workshop 49 | infrastructure](https://github.com/destijl/grr-workshop-setup) by G. Castle. 50 | 51 | DFRWS EU 2015 Workshop [workshop slides](https://storage.googleapis.com/docs.grr-response.com/GRR%20DFRWS%20EU%20Workshop%202015%20Export.pdf) by A. Moser 52 | 53 | OSDF 2013 [workshop presentation](https://storage.googleapis.com/docs.grr-response.com/GRR%20OSDF%20Workshop%202013.pdf) by D. Bilby. 54 | 55 | ## Podcasts 56 | 57 | GRR Rapid Response on [Down the Security Rabbithole](http://podcast.wh1t3rabbit.net/dtr-episode-98-grr-grr-rapid-response-0) Jun 2014 by G. Castle. 58 | 59 | ## Whitepapers 60 | 61 | [GRR Forensic Artifacts White Paper](https://storage.googleapis.com/docs.grr-response.com/GRR_Artifacts_Whitepaper_Blackhat2014_Greg_Castle.pdf), by G. Castle 62 | 63 | ## External Presentations 64 | 65 | These are presentations about GRR by people outside the core GRR team. 66 | 67 | "[Human Hunting](https://github.com/pidydx/slides/blob/master/HumanHunting.pdf)" @BSidesSF 2015 by Sean Gillespie. Covers how Yahoo is using GRR. [Video](https://www.youtube.com/watch?v=4qCvx3SnAm4). 68 | 69 | "[GRR Rapid Response: Remote Live Forensics for Incident Response](http://kernsec.org/files/lss2015/GRR.pdf)" @ [Linux Security Summit](http://kernsec.org/wiki/index.php/Linux_Security_Summit_2015) Aug 2015 by Sean Gillespie. 70 | -------------------------------------------------------------------------------- /quickstart.md: -------------------------------------------------------------------------------- 1 | # Quickstart (have GRR running in 5 minutes) 2 | 3 | This page describes how to get GRR clients and server components up and 4 | running for the first time. If you have Docker installed, and just want to 5 | experiment with a container that already has GRR set up, you can follow 6 | the instructions given [here](installing-and-running-grr/via-docker.md). It takes 7 | ~2 minutes to download the image and initialize the server. Or to quickly start 8 | all components and have a full demo setup running follow the instructions 9 | [here](installing-and-running-grr/via-docker-compose.md) 10 | 11 | On successful start of the Docker Compose stack, you should have an admin 12 | interface running on port 8000 by default. One Docker container runs the 13 | GRR client which can already be used. 14 | 15 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | alabaster==0.7.12 2 | attrs==21.2.0 3 | Babel==2.9.1 4 | certifi==2021.10.8 5 | charset-normalizer==2.0.7 6 | docutils==0.17.1 7 | idna==3.3 8 | imagesize==1.3.0 9 | Jinja2==3.0.3 10 | markdown-it-py==1.1.0 11 | MarkupSafe==2.0.1 12 | mdit-py-plugins==0.2.8 13 | myst-parser==0.15.2 14 | packaging==21.3 15 | Pygments==2.10.0 16 | pyparsing==3.0.6 17 | pytz==2021.3 18 | PyYAML==6.0 19 | requests==2.26.0 20 | snowballstemmer==2.2.0 21 | Sphinx==4.3.0 22 | sphinx-autobuild==2021.3.14 23 | sphinx-rtd-theme==1.0.0 24 | sphinxcontrib-applehelp==1.0.2 25 | sphinxcontrib-devhelp==1.0.2 26 | sphinxcontrib-htmlhelp==2.0.0 27 | sphinxcontrib-jsmath==1.0.1 28 | sphinxcontrib-qthelp==1.0.3 29 | sphinxcontrib-serializinghtml==1.1.5 30 | urllib3==1.26.7 31 | -------------------------------------------------------------------------------- /what-is-grr.md: -------------------------------------------------------------------------------- 1 | # What is GRR? 2 | 3 | GRR Rapid Response is an incident response framework focused on remote live forensics. 4 | 5 | The goal of GRR is to support forensics and investigations in a fast, scalable manner to allow analysts to quickly triage attacks and perform analysis remotely. 6 | 7 | GRR consists of 2 parts: client and server. 8 | 9 | **GRR client** is deployed on systems that one might want to investigate. On every such system, once deployed, GRR client periodically polls GRR frontend servers for work. "Work" means running a specific action: downloading file, listing a directory, etc. 10 | 11 | **GRR server** infrastructure consists of several components (frontends, workers, UI servers, fleetspeak) and provides web-based graphical user interface and an API endpoint that allows analysts to schedule actions on clients and view and process collected data. 12 | 13 | ## Remote forensics at scale 14 | 15 | GRR was built to run at scale so that analysts are capable of effectively collecting and processing data from large numbers of machines. GRR was built with following scenarios in mind: 16 | 17 | * Joe saw something weird, check his machine *(p.s. Joe is on holiday in Cambodia and on 3G)* 18 | * Forensically acquire 25 machines for analysis *(p.s. they're in 5 continents and none are Windows)* 19 | * Tell me if this machine is compromised *(while you're at it, check 100,000 of them - i.e. "hunt" across the fleet)* 20 | 21 | 22 | ## GRR client features 23 | 24 | * Cross-platform support for Linux, OS X and Windows clients. 25 | * Live remote memory analysis using YARA library. 26 | * Powerful search and download capabilities for files and the Windows registry. 27 | * OS-level and raw file system access, using the SleuthKit (TSK). 28 | * Secure communication infrastructure designed for Internet deployment. 29 | * Detailed monitoring of client CPU, memory, IO usage and self-imposed limits. 30 | 31 | ## GRR server features 32 | 33 | * Fully fledged response capabilities handling most incident response and 34 | forensics tasks. 35 | * Enterprise hunting (searching across a fleet of machines) support. 36 | * Fast and simple collection of hundreds of digital forensic artifacts. 37 | * AngularJS Web UI and RESTful JSON API with client libraries in Python, PowerShell and Go. 38 | * Powerful data export features supporting variety of formats and output plugins. 39 | * Fully scalable back-end capable of handling large deployments. 40 | * Automated scheduling for recurring tasks. 41 | * Asynchronous design allowing future task scheduling for clients, designed to work with a large fleet of laptops. 42 | --------------------------------------------------------------------------------