├── README.md
├── screenshots
    ├── .DS_Store
    ├── fzf-dbt_demo.gif
    ├── fzf-dbt_model_plus_view.png
    └── fzf-dbt_other_view.png
└── src
    └── fzf_dbt.sh


/README.md:
--------------------------------------------------------------------------------
  1 | # fzf-dbt - An interactive search for dbt models
  2 | 
  3 | Adds interactive model search and selection to dbt-cli.
  4 | 
  5 | Instead of having to remember your model names, you can simply type `dbt run -m **<TAB>` and an interactive search will appear.
  6 | 
  7 | It works by integrating the command line fuzzy finder [fzf](https://github.com/junegunn/fzf) with [dbt](https://github.com/dbt-labs/dbt-core).
  8 | 
  9 | 1. [Demo](#demo)
 10 |     1. [Interactive search and model selection](#interactive-search-and-model-selection)
 11 |     2. [Model code preview](#model-code-preview)
 12 |     3. [Tag and package selection](#tag-and-package-selection)
 13 |     4. [Selection modifiers `+` and `@` are also supported](#selection-modifiers--and--are-also-supported)
 14 | 2. [Installation](#installation)
 15 |     1. [Install fzf](#install-fzf)
 16 |     2. [Install other dependencies](#install-other-dependencies)
 17 |     3. [Install fzf-dbt](#install-fzf-dbt)
 18 | 3. [Usage](#usage)
 19 | 4. [Configuration](#configuration)
 20 |     1. [Preview command](#preview-command)
 21 |     2. [Window Height](#window-height)
 22 | 5. [Limitations](#limitations)
 23 | 6. [Related Projects](#related-projects)
 24 | 
 25 | ## Demo
 26 | 
 27 | ![fzf-dbt - demo video](screenshots/fzf-dbt_demo.gif)
 28 | 
 29 | ### Interactive search and model selection
 30 | 
 31 | You can narrow down the available models by typing. The search is "fuzzy". So if you want to select the model `netsuite_customers_source`, you can simply type `custso` and it will appear as the top match.
 32 | 
 33 | You can select the model by pressing `enter` and it will be added to your command line. Alternatively, you can also press `tab` or `shift-tab` to select multiple models at once.
 34 | 
 35 | ### Model code preview
 36 | 
 37 | While browsing the models, you can also see a preview of the model code on the right. You can even use [bat](https://github.com/sharkdp/bat) to get code syntax highlighting.
 38 | 
 39 | The preview content can also be scrolled with your trackpad or mouse wheel.
 40 | 
 41 | ### Tag and package selection
 42 | 
 43 | ![fzf-dbt - other view](screenshots/fzf-dbt_other_view.png)
 44 | 
 45 | In the default view fzf-dbt shows you all your dbt models. But you can also press `.` to switch to the tag and package selection view.
 46 | 
 47 | While in the tag and package selection view, the preview will show you all the models that would be included with the tag or package.
 48 | 
 49 | To switch back to the model view, press `,`.
 50 | 
 51 | ### Selection modifiers `+` and `@` are also supported
 52 | 
 53 | Simply press `<` (`shift and ,`) to switch to the advanced model selection view that contains all models including modifiers, such as `2+stg_orders`.
 54 | 
 55 | You can switch back to the other views by pressing `,` or `.`.
 56 | 
 57 | ![fzf-dbt - other view](screenshots/fzf-dbt_model_plus_view.png)
 58 | 
 59 | ## Installation
 60 | 
 61 | Please keep in mind that currently fzf-dbt only work with zsh. So if you are using bash, you'll need to switch to zsh.
 62 | 
 63 | ### Install fzf
 64 | 
 65 | First you need to install [fzf](https://github.com/junegunn/fzf) itself.
 66 | 
 67 | ```shell
 68 | # Download the fzf source
 69 | git clone --depth 1 https://github.com/junegunn/fzf.git ~/.fzf
 70 | 
 71 | # Run the fzf install script
 72 | ~/.fzf/install --key-bindings --completion --update-rc
 73 | ```
 74 | 
 75 | Then simply open a new shell in a new terminal window or tab.
 76 | 
 77 | You can test whether fzf was installed successfully by typing `ssh **<TAB>`. An fzf shell with a list of previously connected SSH hosts should appear.
 78 | 
 79 | You are good to go if fzf appears. If not, [please consult fzf's own documentation here](https://github.com/junegunn/fzf).
 80 | 
 81 | ### Install other dependencies
 82 | 
 83 | You will also need to install the command line JSON processor [jq](https://stedolan.github.io/jq/). It's used to parse dbt's manifest json.
 84 | 
 85 | ```shell
 86 | # macOS
 87 | brew install jq
 88 | 
 89 | # Ubuntu and debian
 90 | sudo apt-get install jq
 91 | ```
 92 | 
 93 | For other systems, please consult the [jq's documentation](https://stedolan.github.io/jq/download/).
 94 | 
 95 | Optionally, you can also install [bat](https://github.com/sharkdp/bat), which is a `cat` clone that supports code syntax highlighting.
 96 | 
 97 | ```shell
 98 | # macOS
 99 | brew install bat
100 | 
101 | # Ubuntu and debian
102 | # Download and install the deb file from https://github.com/sharkdp/bat/releases
103 | ```
104 | 
105 | ### Install fzf-dbt
106 | 
107 | ```shell
108 | 
109 | # Copy and paste the following command to add fzf-dbt to your `.zshrc`
110 | 
111 | cat <<EOF >> ~/.zshrc
112 | FZF_DBT_PATH=~/.fzf-dbt/fzf-dbt.sh
113 | if [[ ! -f $FZF_DBT_PATH ]]; then
114 |     FZF_DBT_DIR=$(dirname $FZF_DBT_PATH)
115 |     print -P "%F{green}Installing fzf-dbt into $FZF_DBT_DIR%f"
116 |     mkdir -p $FZF_DBT_DIR
117 |     command curl -L https://raw.githubusercontent.com/Infused-Insight/fzf-dbt/main/src/fzf_dbt.sh > $FZF_DBT_PATH && \
118 |         print -P "%F{green}Installation successful.%f" || \
119 |         print -P "%F{red}The download has failed.%f"
120 | fi
121 | 
122 | export FZF_DBT_PREVIEW_CMD="cat {}"
123 | export FZF_DBT_HEIGHT=80%
124 | source $FZF_DBT_PATH
125 | EOF
126 | ```
127 | 
128 | After that simply start a new shell in your terminal. Zsh will download and activate fzf-dbt.
129 | 
130 | ## Usage
131 | 
132 | **Activation**
133 | 
134 | You can activate fzf-dbt by typing `**<TAB>` while entering a dbt command, such as `dbt run -m **<TAB>`.
135 | 
136 | It's also not limited to the `-m` parameter and works everywhere like `--select`, `--exclude` and so on.
137 | 
138 | **Search**
139 | 
140 | To search, simply start typing and fzf will show you the models that match your input.
141 | 
142 | **Model Selection**
143 | 
144 | To select a single model, navigate to it with the `up`/`down` keyboard keys and press `enter`.
145 | 
146 | Alternatively, you can also use `tab` or `shift+tab` to select multiple models before sending all of them to the command line with `enter`.
147 | 
148 | **View switching**
149 | 
150 | In addition to that, you can also switch different views by pressing `,`, `.` and `<`. This allows you to select tags, package paths and models with `+` and `@` modifiers.
151 | 
152 | Please refer to the [demo section](#demo) above for more information and a demo video.
153 | 
154 | ## Configuration
155 | 
156 | ### Preview command
157 | 
158 | You can adjust the command that is used to show a preview of the dbt models.
159 | 
160 | To do this, simply set the `FZF_DBT_PREVIEW_CMD` environment variable. The `{}` placeholder will be replaced with the file path of the model.
161 | 
162 | It's recommended to use [bat](https://github.com/sharkdp/bat), which is a `cat` clone that supports code syntax highlighting.
163 | 
164 | Edit your `.zshrc` and adjust the `FZF_DBT_PREVIEW_CMD` variable to something like...
165 | 
166 | ```shell
167 | export FZF_DBT_PREVIEW_CMD='bat --theme OneHalfLight --color=always --style=numbers {}'
168 | ```
169 | 
170 | ### Window Height
171 | 
172 | You can also adjust the height of the window. By default, it is 80% of your terminal window's height.
173 | 
174 | But you can adjust it with...
175 | 
176 | ```shell
177 | export FZF_DBT_HEIGHT=40%
178 | 
179 | ```
180 | 
181 | ## Limitations
182 | 
183 | In order to be fast, fzf-dbt parses dbt's `manifest.json` file. But this file is only updated when you run the `compile`, `run`, `test` or another dbt command that compiles the project.
184 | 
185 | Therefore if you add a models, but don't run any of these commands, fzf-dbt won't be able to find those models.
186 | 
187 | ## Related Projects
188 | 
189 | * [dbt-completion](https://github.com/dbt-labs/dbt-completion.bash)
190 |   * Bash and zsh tab-completion scripts.
191 |   * In addition to allowing you to tab through the models, they also provide completions for all other parameters.
192 |   * So you can for example type `dbt run --<TAB><TAB>` and see which other options the `run` command supports.
193 | * [fzf-tab](https://github.com/Aloxaf/fzf-tab)
194 |   * Forwards all your shell completions into fzf.
195 |   * Amazing when combined with dbt-completions.
196 |   * You can type `dbt run --<TAB><TAB>`, fzf will appear and you can interactivly search for the option you want to add to your command.
197 | 


--------------------------------------------------------------------------------
/screenshots/.DS_Store:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/Infused-Insight/fzf-dbt/e39a6cdc1d3f9beff7ccb5969eec1b45b3615e9e/screenshots/.DS_Store


--------------------------------------------------------------------------------
/screenshots/fzf-dbt_demo.gif:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/Infused-Insight/fzf-dbt/e39a6cdc1d3f9beff7ccb5969eec1b45b3615e9e/screenshots/fzf-dbt_demo.gif


--------------------------------------------------------------------------------
/screenshots/fzf-dbt_model_plus_view.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/Infused-Insight/fzf-dbt/e39a6cdc1d3f9beff7ccb5969eec1b45b3615e9e/screenshots/fzf-dbt_model_plus_view.png


--------------------------------------------------------------------------------
/screenshots/fzf-dbt_other_view.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/Infused-Insight/fzf-dbt/e39a6cdc1d3f9beff7ccb5969eec1b45b3615e9e/screenshots/fzf-dbt_other_view.png


--------------------------------------------------------------------------------
/src/fzf_dbt.sh:
--------------------------------------------------------------------------------
  1 | #!/bin/zsh
  2 | 
  3 | FZF_DBT_PATH=$0
  4 | 
  5 | # A jq filter that simplifies the dbt manifest json and only keeps
  6 | # the keys we need.
  7 | JQ_DBT_MODEL_FILTER='
  8 |     .nodes |
  9 |     to_entries |
 10 |     map({
 11 |         model: .value.name,
 12 |         resource_type: .value.resource_type,
 13 |         file_path: (.value.root_path + "/" + .value.original_file_path),
 14 |         package_path: (.value.fqn[:-1] | join(".") ), tags: .value.tags
 15 |     }) |
 16 |     .[] |
 17 |     select(.resource_type == "model")
 18 | '
 19 | 
 20 | 
 21 | # Walk up the filesystem until we find a dbt_project.yml file,
 22 | # then return the path which contains it (if found).
 23 | # Taken from the _dbt zsh completion script:
 24 | # https://github.com/dbt-labs/dbt-completion.bash/blob/master/_dbt
 25 | _dbt_fzf_get_project_root() {
 26 |   slashes=${PWD//[^\/]/}
 27 |   directory="$PWD"
 28 |   for (( n=${#slashes}; n>0; --n ))
 29 |   do
 30 |     test -e "$directory/dbt_project.yml" && echo "$directory" && return
 31 |     directory="$directory/.."
 32 |   done
 33 | }
 34 | 
 35 | 
 36 | # Prints path of the dbt manifest.json
 37 | _dbt_fzf_get_manifest_path() {
 38 |     local project_dir=$(_dbt_fzf_get_project_root)
 39 |     if [ -z "$project_dir" ]
 40 |     then
 41 |         return
 42 |     fi
 43 | 
 44 |     echo "${project_dir}/target/manifest.json"
 45 | }
 46 | 
 47 | 
 48 | # Prints a list of all dbt models
 49 | _dbt_fzf_get_model_list() {
 50 |     local manifest_path=$(_dbt_fzf_get_manifest_path)
 51 | 
 52 |     if [ -z "$manifest_path" ]
 53 |     then
 54 |         echo "No dbt project at the current path."
 55 |         return
 56 |     fi
 57 | 
 58 |     jq -r "$JQ_DBT_MODEL_FILTER | .model" $manifest_path | sort
 59 | }
 60 | 
 61 | # Prints a list of all dbt tags
 62 | _dbt_fzf_get_tag_list() {
 63 |     local manifest_path=$(_dbt_fzf_get_manifest_path)
 64 | 
 65 |     if [ -z "$manifest_path" ]
 66 |     then
 67 |         echo "No dbt project at the current path."
 68 |         return
 69 |     fi
 70 | 
 71 |     jq -r "$JQ_DBT_MODEL_FILTER | (\"tag:\" + .tags[])" $manifest_path \
 72 |     | sort | uniq
 73 | }
 74 | 
 75 | # Prints a list of all dbt package paths
 76 | _dbt_fzf_get_package_paths() {
 77 |     local manifest_path=$(_dbt_fzf_get_manifest_path)
 78 | 
 79 |     if [ -z "$manifest_path" ]
 80 |     then
 81 |         echo "No dbt project at the current path."
 82 |         return
 83 |     fi
 84 | 
 85 |     local package_list=$(
 86 |         jq -r "$JQ_DBT_MODEL_FILTER | .package_path" $manifest_path \
 87 |         | sort | uniq
 88 |     )
 89 | 
 90 |     echo $package_list | awk '{
 91 |         parts_len=split($0, parts, ".");
 92 |         for(part = 1; part <= parts_len; part++) {
 93 |             path = "";
 94 |             for(i = 1; i<= part; i++) {
 95 |                 printf("%s", parts[i]);
 96 | 
 97 |                 if(i == part) {
 98 |                     printf("\n");
 99 |                 } else {
100 |                     printf(".");
101 |                 }
102 |             }
103 |         }
104 |     };' \
105 |     | sort | uniq
106 | }
107 | 
108 | # Returns the file path of a model
109 | _dbt_fzf_get_path_for_model() {
110 |     local model_name=$1
111 | 
112 |     if [ -z "$model_name" ]
113 |     then
114 |         echo "No model name specified in first arg."
115 |         return
116 |     fi
117 | 
118 |     local manifest_path=$(_dbt_fzf_get_manifest_path)
119 | 
120 |     if [ -z "$manifest_path" ]
121 |     then
122 |         echo "No dbt project at the current path."
123 |         return
124 |     fi
125 | 
126 | 
127 |     local model_path=$(
128 |         jq \
129 |             "
130 |                 $JQ_DBT_MODEL_FILTER | 
131 |                 select(.model == \"$model_name\") |
132 |                 .file_path
133 |             " \
134 |             $manifest_path
135 |     )
136 | 
137 |     echo $model_path
138 | }
139 | 
140 | # Prints all models that have the tag that is supplied in the first argument
141 | _dbt_fzf_get_models_for_tag() {
142 |     local tag_name=$1
143 | 
144 |     if [ -z "$tag_name" ]
145 |     then
146 |         echo "No tag name specified in first arg."
147 |         return
148 |     fi
149 | 
150 |     # Remove "tag:" prefix if present
151 |     if [[ $tag_name == tag:* ]]
152 |     then
153 |         tag_name=$(echo $tag_name | cut -c 5-)
154 |     fi
155 | 
156 |     local manifest_path=$(_dbt_fzf_get_manifest_path)
157 |     if [ -z "$manifest_path" ]
158 |     then
159 |         echo "No dbt project at the current path."
160 |         return
161 |     fi
162 | 
163 |     jq \
164 |         -r \
165 |         "
166 |             $JQ_DBT_MODEL_FILTER |
167 |             select(.tags[] | contains (\"$tag_name\")) |
168 |             .model
169 |         " \
170 |         $manifest_path \
171 |     | sort 
172 | }
173 | 
174 | # Prints all models that are within the package path that is supplied
175 | # in the first argument.
176 | _dbt_fzf_get_models_for_package_path() {
177 |     local package_path=$1
178 | 
179 |     if [ -z "$package_path" ]
180 |     then
181 |         echo "No package path specified in first arg."
182 |         return
183 |     fi
184 | 
185 |     local manifest_path=$(_dbt_fzf_get_manifest_path)
186 |     if [ -z "$manifest_path" ]
187 |     then
188 |         echo "No dbt project at the current path."
189 |         return
190 |     fi
191 | 
192 |     jq \
193 |         -r \
194 |         "
195 |             $JQ_DBT_MODEL_FILTER |
196 |             select(.package_path|startswith(\"$package_path\")) |
197 |             .model
198 |         " \
199 |         $manifest_path \
200 |     | sort 
201 | }
202 | 
203 | # Tries to find a list of models for the selection.
204 | # First tries to check if the selection is a tag and looks for
205 | # all models that have that tag.
206 | # Then tries to treat the selection as a package path and looks for
207 | # all models within that package path.
208 | _dbt_fzf_get_models_for_selection() {
209 |     local fzf_selection=$1
210 | 
211 |     local selection_models=""
212 | 
213 |     # If the selection starts with "tag:", try to get all models
214 |     # that have that tag applied
215 |     if [[ $fzf_selection == tag:* ]]
216 |     then
217 |         selection_models=$(
218 |             _dbt_fzf_get_models_for_tag $fzf_selection
219 |         )
220 |     fi
221 | 
222 |     # If we didn't find models using the tag search, try the 
223 |     # package path search
224 |     if [ -z "$selection_models" ]
225 |     then
226 |         selection_models=$(
227 |             _dbt_fzf_get_models_for_package_path $fzf_selection
228 |         )
229 |     fi
230 | 
231 |     # If we still couldn't find any models, then try to use `dbt ls`
232 |     # to get the models for the selections.
233 |     # This is slower, but allows us to find models even for selectors 
234 |     # that use "+"" and "@" modifiers.
235 |     if [ -z "$selection_models" ]
236 |     then
237 |         selection_models=$(
238 |             dbt ls -m "$fzf_selection" \
239 |             | awk '{ n=split($1, part, "."); print part[n] }'
240 |         )
241 |     fi
242 | 
243 |     echo $selection_models
244 | }
245 | 
246 | # These functions return selection lists for fzf.
247 | # The first line of these methods is used as the header in fzf.
248 | _dbt_fzf_show_models() {
249 |     echo "\033[0;31m[models - ,]\033[0m   [selectors - .]"
250 |     _dbt_fzf_get_model_list
251 | }
252 | 
253 | _dbt_fzf_show_models_plus() {
254 |     echo "\033[0;31m[models+ - <]\033[0m   [selectors - .]"
255 |     local model_list=$(_dbt_fzf_get_model_list)
256 |     
257 |     (
258 |         echo $model_list
259 |         echo $model_list | awk '{print "+" $0;}'
260 |         echo $model_list | awk '{print $0 "+";}'
261 |         echo $model_list | awk '{print "+" $0 "+";}'
262 |         echo $model_list | awk '{print "@" $0;}'
263 | 
264 |         for i in {1..3}
265 |         do
266 |             echo $model_list | awk -v i="$i" '{print i "+" $0;}'
267 |             echo $model_list | awk -v i="$i" '{print $0 "+" i;}'
268 |         done
269 |     ) | sort
270 | }
271 | 
272 | _dbt_fzf_show_selectors() {
273 |     echo "[models - ,]   \033[0;31m[selectors - .]\033[0m"
274 |     (
275 |         _dbt_fzf_get_tag_list
276 |         _dbt_fzf_get_package_paths
277 |     )
278 | }
279 | 
280 | # This is the actual function called to launch fzf when you type **<tab>
281 | _fzf_complete_dbt() {
282 |     local height=${FZF_DBT_HEIGHT-80%}
283 |     _fzf_complete \
284 |         --multi \
285 |         --reverse \
286 |         --prompt="dbt> " \
287 |         --bind=",:reload( source $FZF_DBT_PATH; _dbt_fzf_show_models )" \
288 |         --bind="<:reload( source $FZF_DBT_PATH; _dbt_fzf_show_models_plus )" \
289 |         --bind=".:reload( source $FZF_DBT_PATH; _dbt_fzf_show_selectors )" \
290 |         --header-lines=1 \
291 |         --preview "source $FZF_DBT_PATH;  _dbt_fzf_preview {}" \
292 |         --height $height \
293 |         -- "$@" \
294 |         < <( _dbt_fzf_show_models )
295 | }
296 | 
297 | 
298 | # This function generates the preview command inside fzt.
299 | #
300 | # You can adjust the command that will be used to output the model
301 | # code by setting the environment variable `FZF_DBT_PREVIEW_CMD`.
302 | #
303 | # You can use `{}` to specify where the file path of the model will be
304 | # inserted.
305 | #
306 | # For example, you can use `bat` to get a syntax highlighted preview:
307 | # `export FZF_DBT_PREVIEW_CMD='bat --theme OneHalfLight --color=always
308 | # --style=numbers {}'`
309 | 
310 | _dbt_fzf_preview() {
311 |     local fzf_selection=$1
312 |     local preview_cmd=${FZF_DBT_PREVIEW_CMD-"cat {}"}
313 |     local manifest_path=$(_dbt_fzf_get_manifest_path)
314 | 
315 |     if [ -z "$fzf_selection" ]
316 |     then
317 |         echo "No dbt fzf_selection specified for preview."
318 |         return
319 |     fi
320 | 
321 |     if [ -z "$manifest_path" ]
322 |     then
323 |         echo "No dbt project at the current path."
324 |         return
325 |     fi
326 | 
327 |     local final_preview_cmd=""
328 | 
329 |     # Try to get a model path for the selection
330 |     local model_path=$(_dbt_fzf_get_path_for_model $fzf_selection)
331 | 
332 |     # If we found a model path, then show the model content as the preview
333 |     if [ ! -z "$model_path" ]
334 |     then
335 |         final_preview_cmd=${preview_cmd//"{}"/$model_path}
336 |     fi
337 | 
338 |     # If no model path was found, try to get a list of models for
339 |     # the fzf selection and display it
340 |     if [ -z "$final_preview_cmd" ]
341 |     then
342 | 
343 |         local selection_models=$(
344 |             _dbt_fzf_get_models_for_selection $fzf_selection
345 |         )
346 | 
347 |         # Add line numbers before models
348 |         local selection_models_numbered=$(
349 |             echo $selection_models \
350 |             | awk '{ printf( "\t%002d) %s\n", NR, $0 ) }'
351 |         )
352 | 
353 |         # Count models
354 |         local model_count=$(echo "$selection_models" | wc -l | xargs)
355 | 
356 |         # Generate a preview command that shows all models within selection
357 |         final_preview_cmd="
358 |             echo 'Selection \"$fzf_selection\" contains $model_count models...\n\n$selection_models_numbered'
359 |         "
360 |     fi
361 | 
362 |     # If no preview cmd could be generated, show error message
363 |     if [ -z "$final_preview_cmd" ]
364 |     then
365 |         final_preview_cmd=(
366 |             "echo 'Could not generate a preview for selection \"$fzf_selection\" '"
367 |         )
368 |     fi
369 | 
370 |     # Run the preview command
371 |     zsh -c "$final_preview_cmd"
372 | }
373 | 


--------------------------------------------------------------------------------