├── .github └── FUNDING.yml ├── CHANGELOG.md ├── README.md ├── docs └── images │ ├── data-notebook-extension.png │ ├── data-notebook-output-rendering.gif │ ├── data-notebook-prql-data-table-output.gif │ ├── data-notebook-sql-query.gif │ ├── duckdb-pro-tools-code-intellisense.gif │ ├── duckdb-pro-tools-db-tree-view.gif │ ├── duckdb-pro-tools-v1.3.0-docs.gif │ ├── duckdb-pro-tools-views.gif │ ├── duckdb-tools-extensions.png │ ├── duckdb-vscode-extensions.png │ ├── markdow-sql-magic-line-select.png │ ├── markdown-sql-bookmarks.gif │ ├── markdown-sql-bookmarks.png │ ├── markdown-sql-code-highlight.png │ ├── markdown-sql-extracted-sql-code.png │ ├── markdown-sql-in-quarto.gif │ ├── markdown-sql-in-rmd.gif │ ├── markdown-sql-jupysql-intro.png │ ├── markdown-sql-jupysql.gif │ ├── markdown-sql-pro-tools-contributions.png │ ├── markdown-sql-pro-tools-copy-code-lens.png │ ├── markdown-sql-pro-tools-notebook.png │ ├── markdown-sql-pro-tools-v1.1.0.gif │ ├── markdown-sql-pro-tools.gif │ ├── markdown-sql-pro-tools.png │ ├── markdown-sql-selections.png │ ├── markdown-sql-tools.png │ ├── pg-sql-tools-commands-1.png │ ├── pg-sql-tools-commands-2.png │ ├── pg-sql-tools-extension.png │ ├── pg-sql-tools-features.png │ ├── pg-sql-tools-install-from-vsix.png │ ├── pg-sql-tools.png │ ├── pro-data-tools.png │ ├── prql-code-lens-pro-tools.gif │ ├── prql-code-lens.gif │ ├── prql-code-lens.png │ ├── prql-notebook.png │ ├── prql-open-with-data-notebook.gif │ ├── prql-pro-tools-v1.2.0-docs.gif │ ├── prql-pro-tools.png │ ├── random-fractals-data-tools.png │ ├── sql-code-blocks-on-github.png │ ├── sql-tools-drivers.png │ ├── sqlite-commands.png │ ├── sqlite-functions.png │ ├── sqlite-indexes.png │ ├── sqlite-modules.png │ ├── sqlite-pro-tools.png │ ├── sqlite-settings.png │ ├── sqlite-table-columns.png │ ├── sqlite-tools-extension.png │ └── sqlite-tools.png ├── duckdb-tools.md ├── markdown-sql-tools.md ├── prql-tools.md └── sqlite-tools.md /.github/FUNDING.yml: -------------------------------------------------------------------------------- 1 | github: [RandomFractals] 2 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # Changelog 2 | 3 | Combined [Pro Data Tools](https://github.com/RandomFractals/pro-data-tools#pro-data-tools) changelog for all the VS Code extensions and versions released to our [Pro sponsors](https://github.com/sponsors/RandomFractals) on GitHub. 4 | 5 | ## DuckDB SQL Tools v1.6.0 - [2024-07-14] 6 | 7 | - Update DuckDB SQL Tools Preview to support DuckDB v0.10 8 | - Update DuckDB SQL Tools Preview docs with DuckDB v0.10 support info 9 | - Package and publish DuckDB SQL Tools Preview v1.6.0 with DuckDB v0.10.2 support 10 | 11 | ## DuckDB Pro Tools v1.6.0 - [2024-07-12] 12 | 13 | - Update DuckDB Pro SQL Tools to support DuckDB 0.10 version 14 | - Update DuckDB Pro SQL Tools docs with DuckDB v0.10 support info 15 | - Package and publish DuckDB Pro Tools v1.6.0 16 | 17 | ## DuckDB SQL Tools v1.5.0 - [2024-07-12] 18 | 19 | - Convert `BigInt` values and arrays to `Number` in DuckDB `TableData` query results for display in SQL Tools records view 20 | - Update DuckDB version support message in DuckDB SQL Tools Preview 21 | - Update DuckDB SQL Tools Preview `LICENSE.txt` 22 | - Package and publish DuckDB SQL Tools Preview v1.5.0 with `BigInt` support 23 | 24 | ## DuckDB Pro Tools v1.5.0 - [2024-07-09] 25 | 26 | - Update DuckDB Pro Tools to use DuckDB v0.9.2 NodeJS libraries 27 | - Update DuckDB Pro Tools `README.md` with new DuckDB v0.9.2 support info 28 | - Update DuckDB Pro Tools with DuckDB v0.9.2 support info in Pro Data Tools documentation repository 29 | - Link supported advanced DuckDB views sections in the DuckDB Pro Tools intro 30 | - Update Sponsor links in DuckDB Pro Tools README.md 31 | - Update Future Pro Data Taools Releases section and remove old Twitter links from DuckDB Pro Tools README.md 32 | - Cast `bigint` values to `Number` in DuckDB query `TableData` results 33 | - Remove VS Code Memory Limit section from DuckDB Pro Tools doc 34 | - Set Copyright © 2024 in `LICENSE.txt` 35 | - Package and publish DuckDB Pro Tools v1.5.0 with DuckDB v0.9.2 support 36 | 37 | ## SQLite Pro Tools v1.1.0 - [2024-07-04] 38 | 39 | - Add `chinook.db` with typed columns 40 | - Update year to 2024 in the license text file 41 | - Update SQLite3 Node JS library to the latest v5.1.7 version 42 | - Package and publish SQLite Pro Tools v1.1.0 43 | 44 | ## DuckDB SQL Tools v1.4.0 - [2023-12-01] 45 | 46 | - Update DuckDB SQL Tools to support the latest DuckDB 0.9.2 version 47 | - Remove VS Code Memory Limit section from DuckDB SQL Tools doc 48 | - Add new SQLite Pro Tools info to the Pro Data Tools sections in DuckDB SQL Tools `README.md` 49 | - Package and publish DuckDB SQL Tools v1.4.0 with DuckDB v0.9.2 support 50 | 51 | ## SQLite Pro Tools v1.0.0 - [2023-11-27] 52 | 53 | - Create new SQLite Tools VS Code extension codebase from SQL Tools and drivers monorepo 54 | - Update SQLite3 Node JS library to the latest v5.1.6 version 55 | - Create custom SQLite Pro Tools extension and database connection display icons 56 | - Add column data type icons display for tables and views in SQL Tools Connection Explorer 57 | - Display SQLite database Indexes in SQL Tools Connection Explorer 58 | - Display SQLite Modules in SQL Tools Connection Explorer 59 | - Add Random Fractals Inc. license file 60 | - Replace Tables and Views folder icons with built-in VS Code windows and layers icons in SQL Tools Connection Explorer 61 | - Add SQLite Settings/Compile Options display to SQL Tools Connection Explorer 62 | - Create `/data` folder with demo SQLite database to use for testing 63 | - Create `SQLite Tools:` shortcut commands to list all databases, tables, views, indexes, foreign keys, triggers, configuration options, etc. 64 | - Refine SQLite database schema queries in `queries.ts` 65 | - Add Car Company demo database for testing 66 | - Add `SQLite Tools: View DB Stats` shortcut command 67 | - Display index table and column name in `View All Indexes` SQLite Tools command and SQL query 68 | - Add `SQLite Tools: View All Functions` shortcut command 69 | - Add `SQLite Tools: View Module List` shortcut command 70 | - Add `SQLite Tools: View Pragma List` shortcut command 71 | - Display SQLite Functions list in SQL Tools Connection Explorer tree view 72 | - Document SQLite Pro Tools features, views, configuration settings, installation, demo data and commands in `README.md` 73 | - Create SQLite Pro Tools `CHANGELOG.md` for the first private beta version extension package release 74 | - Package and publish new SQLite Pro Tools extension 75 | 76 | ## DuckDB SQL Tools v1.3.0 - [2023-10-27] 77 | 78 | - Update DuckDB SQL Tools and demo DuckDB configs to use new DuckDB v0.9.1 NodeJS libraries 79 | - Update DuckDB SQL Tools `chinook-duckdb.ipynb` and generated demo `chinook.duckdb` to use DuckDB v0.9.1 80 | - Update DuckDB SQL Tools documentation with new DuckDB v0.9.1 support info 81 | - Package and publish DuckDB SQL Tools v1.3.0 with DuckDB v0.9.1 support 82 | 83 | ## PRQL Pro Tools v1.3.0 - [2023-10-14] 84 | 85 | - Update PRQL JS library to v0.9.5 86 | - Update PRQL query examples to work with `prql-js` v0.9.5 87 | - Package and publish PRQL Pro Tools v1.3.0 release 88 | 89 | ## DuckDB SQL Tools v1.2.0 - [2023-10-06] 90 | 91 | - Update free public DuckDB SQL Tools Preview extension to support DuckDB v0.8.1 92 | - Update SQL Tools drivers link and screenshot in docs 93 | - Add DuckDB Pro Tools View Commands demo gif to DuckDB SQL Tools documentation 94 | - Update PRQL Pro Tools links in DuckDB SQL Tools documentation 95 | - Update new Markdown SQL Pro Tools info in DuckDB SQL Tools documentation 96 | - Add new Data Notebook Pro Tools info to DuckDB SQL Tools documentation 97 | - Update Random Fractals Data Tools and new Pro Data Tools sections in DuckDB SQL Tools documentation 98 | - Package and publish DuckDB SQL Tools v1.2.0 release with DuckDB v0.8.1 support 99 | 100 | ## Markdown SQL Pro Tools v1.3.0 - [2023-08-14] 101 | 102 | - Skip adding top level `Select All`, `Extract All`, and `Bookmark All` SQL code lenses to open markdown documents without SQL code blocks 103 | 104 | ## Markdown SQL Pro Tools v1.2.0 - [2023-08-03] 105 | 106 | - Support SQL magics (`%%sql`) in markdown SQL code fences and parsing of SQL code blocks for SQL code lenses 107 | - Add SQL code lenses to SQL code blocks and string literals in Observable markdown (`.omd`) and Observable JS (`.ojs`) documents 108 | - Add SQL code lenses to SQL code blocks and cells with SQL magics in `.qmd` Quarto markdown documents 109 | - Add SQL code lenses to SQL code blocks in `.Rmd` markdown documents 110 | - Add new top level `Extract All` SQL code lens to markdown documents 111 | - Add SQL code lenses to single line SQL magic with `%sql` and `select` statements 112 | - Add new top level `Select All` SQL code lens to the supported `.*md` markdown documents with SQL code blocks 113 | - Skip SQL code blocks with only `--` comment lines and no SQL statements when adding SQL code lenses to open SQL document 114 | - Highlight SQL code blocks in markdown document using SQL Tools query code decorator styles 115 | - Create Observable DuckDB Data Tables example with SQL string literals to demo SQL code lenses in Observable JS, markdown and Quarto markdown documents 116 | - Refactor SQL code blocks and string literals extraction and SQL code lenses construction for supported documents 117 | - Add new top level `Bookmark All` SQL code lens to markdown documents 118 | - Document, package and publish Markdown SQL Pro Tools v1.2.0 release 119 | 120 | ## Markdown SQL Pro Tools v1.1.0 - [2023-07-07] 121 | 122 | - Rename `Execute SQL statement(s)` code lens title to `Execute` 123 | - Rename `Execute All SQL statement(s)` code lens title to `Execute All` 124 | - Use `notebook-execute-all` and `notebook-execute` VS Code icons for the `Execute All` and `Execute` SQL code lens icons 125 | - Don't add top-level `Execute All` SQL statement(s) code lens when no SQL code blocks are detected in the open `.md` markdown document 126 | - Add `Execute`, `Select`, `Copy`, and `Create` SQL code lenses to individual SQL statements in standard `.sql` documents 127 | - Add `Copy` SQL to Clipboard code lens to SQL code blocks in `.md` markdown documents 128 | - Add `Create` SQL code lens to SQL code blocks in `.md` markdown documents to open selected SQL code in a new virtual SQL document VS Code editor 129 | - Rename `Select SQL code` code lens title to `Select` with `Select SQL code ...` in the code lens tooltip 130 | - Use `Markdown SQL Tools` category for all custom SQL code lens commands 131 | - Refactor SQL code lenses provider and SQL code blocks matching for markdown documents 132 | - Add new `Features` section to `README.md` 133 | - Add new `Copy` SQL to clipboard and `Create` new SQL document code lenses info and example to `README.md` 134 | - Update VS Code SQL and Data Tools sections in `README.md` 135 | - Update Markdown SQL Pro Tools section in new public `pro-data-tools` documentation repository 136 | - Update `Data Notebook` extension info and demo gifs in `README.md` 137 | - Create new Markdow SQL Pro Tools demo gif and update extension intro section in `README.md` 138 | - Package and publish Markdown SQL Pro Tools v1.1.0 release 139 | 140 | ## DuckDB Pro Tools v1.4.0 - [2023-06-27] 141 | 142 | DuckDB Pro Tools v1.4.0 updates DuckDB NodeJS libraries, create demo DuckDB Jupyter notebooks, and demo `.duckdb` files in the `/data` folder to [DuckDB v0.8.1](https://github.com/duckdb/duckdb/releases/tag/v0.8.1). 143 | 144 | - Update DuckDB Pro Tools to use DuckDB v0.8.1 NodeJS libraries 145 | - Update all `.duckdb` files and create DuckDB Jupyter notebooks in DuckDB Pro Tools `/data` folder to use DuckDB v0.8.1 146 | - Add DuckDB version info and detailed error logging to DuckDB Pro Tools connection error message display 147 | - Update DuckDB Pro Tools `README.md` with DuckDB v0.8.1 support info 148 | - Add VS Code SQL and Data Tools sections to DuckDB Pro Tools `README.md` 149 | - Package and publish DuckDB Pro Tools v1.4.0 with DuckDB v0.8.1 support 150 | 151 | ## DuckDB SQL Tools v1.1.0 - [2023-06-26] 152 | 153 | - Update DuckDB Sql Tools extension to use DuckDB v0.7.1 154 | - Update DuckDB Sql Tools demo `chinook.duckdb` file to use DuckDB v0.7.1 storage format 155 | - Add DuckDB version info, detailed error logging, and DuckDB Pro Tools info to the DuckDB Sql Tools driver and DuckDB connection error message display 156 | - Update DuckDB SQL Tools `README.md` with new DuckDB v0.7.1 support info 157 | - Add VS Code SQL and Pro Data Tools info to the public DuckDB Sql Tools extension `README.md` 158 | - Package and publish DuckDB SQL Tools v1.1.0 with DuckDB v0.7.1 support 159 | 160 | ## PRQL Pro Tools v1.2.0 - [2023-06-23] 161 | 162 | - Update PRQL Pro Tools to use `prql-js` v0.8.1, DuckDB Pro Tools v1.3.0, and DuckDB v0.8.0 163 | 164 | ## DuckDB Pro Tools v1.3.0 - [2023-06-22] 165 | 166 | DuckDB Pro Tools v1.3.0 updates DuckDB NodeJS libraries, create demo DuckDB Jupyter notebooks, and demo `.duckdb` files in the `/data` folder to [DuckDB v0.8.0](https://github.com/duckdb/duckdb/releases/tag/v0.8.0). 167 | 168 | - Update DuckDB Pro Tools to use DuckDB v0.8.0 NodeJS libraries 169 | - Update all `.duckdb` files and create DuckDB Jupyter notebooks in DuckDB Pro Tools `/data` folder to use DuckDB v0.8.0 170 | - Update DuckDB Pro Tools `README.md` with DuckDB v0.8.0 support info 171 | - Package and publish DuckDB Pro Tools v1.3.0 with DuckDB v0.8.0 support 172 | 173 | ## Markdown SQL Pro Tools v1.0.0 - [2023-06-20] 174 | 175 | - Scaffold `Markdown Sql Pro Tools` extension code base from PRQL Code Lens in PRQL Pro Tools VS Code extension 176 | - Create custom Markdown SQL Pro Tools extension icon 177 | - Enhance SQL Code Lens provider and SQL Tools extension plugin to support SQL code blocks in markdown documents 178 | - Create Markdown Sql code selection command 179 | - Add `Execute SQL statement(s)` code lens to the SQL code blocks in markdown documents 180 | - Document, package, and release Markdown SQL Pro Tools v1.0.0 181 | 182 | ## DuckDB Pro Tools v1.2.0 - [2023-05-17] 183 | 184 | DuckDB Pro Tools v1.2.0 adds over 30 new DuckDB views and metadata shortcut commands to the VSCode Command Palette under `DuckDB Tools` commands category, and features new `prompt.sql` query examples created with the free [Hugging Face Code Autocomplete](https://marketplace.visualstudio.com/items?itemName=HuggingFace.huggingface-vscode) extension. 185 | 186 | - Add DuckDB metadata functions shortcut commands to DuckDB Pro Tools 187 | - Move DuckDBDatabase namespace and interfaces to new duckdb.ts 188 | - Move DuckDBPro tools driver name and extension id/name constants to constants.ts 189 | - Add AI generated query examples for the employees database demo data in DuckDB Pro Tools 190 | - Package and publish DuckDB Pro Tools v1.2.0 release 191 | 192 | ## DuckDB Pro Tools v1.1.0 - [2023-05-01] 193 | 194 | DuckdDB Pro Tools v1.1.0 release with extended DB objects tree view display, improved SQL intellisense, and alpha implementation of read/write DB access mode and connections handling. 195 | 196 | - Create Enhanced DB tree view with more DB objects for the DuckDB Pro Tools extension 197 | - Add more chinook demo db sql queries to run in a data notebook 198 | - Open DuckDBPro instances in read write access mode by default 199 | - Add DuckDB Pro Tools v1.1.0 images and feature demo gifs to the public DuckDB Sql Tools repository 200 | - Document and publish DuckDB Pro Tools v1.1.0 with extended DuckDB instance tree view objects and results display 201 | 202 | ## DuckDB Pro Tools v1.0.3 - [2023-03-23] 203 | 204 | First private DuckDB Pro Tools beta release. 205 | 206 | - Add more info about database storage and versions to DuckDB Storage section in docs 207 | - Create new Employees DuckDB with parquet files to run PRQL queries 208 | - Add PRQL and SQL query examples from PRQL book 209 | - Create new duckdb-pro-tools extension package 210 | - Add Run PRQL on active DuckDB connection to DuckDB Pro Tools extension 211 | - Updated chinook.duckdb SQL queries generated from PRQL to use the latest PRQL library version (0.5.0) 212 | - Move atrist-track-prices.prql and generated sql to data/chinook/sqlite data and queries folder 213 | - Link DuckDB Sql Tools home page to public preview/docs repo 214 | - Use new PRQL extension settings in this project config 215 | - Update chinook.duckdb demo data to DuckDB v0.7.1 storage format in DuckDB Pro Tools extension package 216 | - Use duckdb.js library v0.7.1 in the Pro version of DuckDB Tools 217 | - Move new PRQL Run Query Code Lens to new PRQL Tools repository 218 | - Remove PRQL Code Lens code from DuckDB Pro Tools extension 219 | - Set DuckDB Pro SQLTools driver name to DuckDB Pro 220 | - Add employees.duckdb config to workspace settings using new DuckDBPro driver to test it 221 | - Add a black dot to the DuckDB Pro extension and DB state icons 222 | - Add TPC-DS /data folder with DuckDB schema SQL script, load data SQL, and TPC-DS SQL queries and data 223 | - Rename TPC-DS load.sql to load-csv.sql and add load-parquet.sql script 224 | - Add /gbif data folder with sample SQL and PRQL queries for AWS hosted public datasets 225 | - Update all create DuckDB notebooks and .duckdb files in /data folder to use DuckDB v0.7.1 storage format 226 | - Create DuckDB SVG icons for the DuckDB Pro Tools version 227 | - Create new DuckDB Pro Tools documentation 228 | - Use new DuckDB Pro svg icons for conenctions display instead of png 229 | - Move Chicago crimes create table and query examples to new data/chicago-crimes folder 230 | - Add new chicago-crimes.duckdb in-memory db pro config to data folder 231 | - Change GBIF demo duckdb config to DuckDBPro instance 232 | - Rename GBIF data/sql folder to data/gbif-observations 233 | - Cleanup DuckDB and DuckDBPro db configs for DuckDBPro Tools release 234 | - Install and load DuckDB JSON extension for all open DuckDB connections 235 | - Add remote JSON data loading examples 236 | - Remove the old playlists demo data and config 237 | - Add DuckDB Pro Tools docs/images and demo gifs 238 | - Create tpc-h-duckdb-gen.ipynb Jupyter notebook and test DuckDB Pro Tools with TPC-H 1, 10, and 100 DB scale factors 239 | - Add Chicago Traffic Estimates Jupyter notebook and Restbook to test DuckDB HTTPFS and JSON data loading from URL with redirects 240 | - Add DuckDB Pro Tools images to the public DuckDB Sql Tools repository for images to load on Extension Info page in VSCode 241 | 242 | ## PRQL Pro Tools v1.1.0 - [2023-03-15] 243 | 244 | - Update `prql-js` to v0.6.1 to execute PRQL loops and use other new PRQL language and compiler features 245 | - Add GBIF sample PRQL query run gif to docs 246 | 247 | ## PRQL Pro Tools v1.0.0 - [2023-03-03] 248 | 249 | - Move new PRQL Run Query Code Lens from DuckDB Tools monorepo to this new PRQL Tools monorepo 250 | - Create custom PRQL Run extension icons for PRQL Tools 251 | - Document PRQL Code Lens extension package 252 | - Package and publish the first PRQL Code Lens extension private beta v. release 253 | - Add `/data` folder with our `chinook.duckdb` instance from DuckDB Sql Tools for testing and sample runs 254 | - Create new /scripts/prql folder with .prql files for testing and sample PRQL Query runs 255 | - Add PRQL vscode extension dependency for PRQL documents grammar support 256 | - Use prql.target setting to get SQL for the PRQL Run Query code lens 257 | - Create PRQL Pro Tools README.md with a summary of PRQL Pro Tools extension pack 258 | 259 | ## DuckDB SQL Tools v1.0.2 - [2023-01-28] 260 | 261 | Documentation Update Release. 262 | 263 | - Refine DuckDB SQL Tools extension intro, features list, and other doc sections in README.md 264 | 265 | ## DuckDB SQL Tools v1.0.1 - [2023-01-27] 266 | 267 | Minor extension packaging update for the extension gallery banner background color display in VSCode marketplace. 268 | 269 | - Add gallery banner color to extension package manifest 270 | 271 | ## DuckDB SQL Tools v1.0.0 - [2023-01-27] 272 | 273 | Initial Public Preview Release. 274 | 275 | - Create DuckDB tools monorepo structure 276 | - Create duckdb-sql-tools vscode extension package 277 | - Create DuckDB Sql Tools driver base code 278 | - Create DuckDB sql tools icons 279 | - Add DuckDB SQL tools demo gif for docs 280 | - Create DuckDB sql tools driver using NodeJS libraries 281 | - Add PRQL query examples for the demo db and data files in data folder 282 | - Add chinook demo db data files in different formats 283 | - Create DuckDB SQL Tools preview version README.md 284 | - Add Random Fractals Inc. license file 285 | - Allow to create readwrite `:memory:` database connections 286 | - Add httpfs data loading example 287 | - Add sample duckdb sql query examples 288 | - Add better chinook db create sql script with primary keys and indexes 289 | - Add chinook duckdb Jupyter notebook and create new chinook.duckdb file using duckdb v0.6.1 290 | - Add VSCode Memory Limit info to Limitations section in vscode extension README.md 291 | - Add additional DuckDB keywords, pragmas, and metadata functions to SQL keywords 292 | - Add demo data section to vscode extension README.md 293 | - Create CHANGELOG.md for the DuckDB SQL Tools extension v1.0.0 release 294 | - Package and publish DuckDB Sql Tools v1.0.0 Preview release 295 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Random Fractals Inc. Data Tools 2 | 3 | [Random Fractals](https://github.com/RandomFractals) Inc. [**Data Tools**](https://marketplace.visualstudio.com/publishers/RandomFractalsInc) 🛠️ is a collection of public [data visualization extensions](https://marketplace.visualstudio.com/search?term=dataViz&target=VSCode&category=All%20categories&sortBy=Relevance), [data viewers](https://marketplace.visualstudio.com/search?term=data%20viewer&target=VSCode&category=All%20categories&sortBy=Relevance), VS Code [Notebook renderers](https://marketplace.visualstudio.com/search?term=notebook%20renderer&target=VSCode&category=All%20categories&sortBy=Relevance), and [code snippets](https://marketplace.visualstudio.com/search?term=Random%20Fractals%20code%20snippets&target=VSCode&category=All%20categories&sortBy=Relevance) for devs and data scientists using VS Code IDE, published under our [Random Fractals Inc.](https://marketplace.visualstudio.com/publishers/RandomFractalsInc) ☂️ org. 4 | 5 | ![Random Fractals Data Tools](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/random-fractals-data-tools.png?raw=true) 6 | 7 | # Pro Data Tools 8 | 9 | **Pro Data Tools** is our new Premium set of SQL and [PRQL](https://prql-lang.org/) Code Lenses and Data Viewers created for our monthly [**Pro sponsors**](https://github.com/sponsors/RandomFractals/sponsorships?tier_id=18884) on GitHub to enhance SQL development and SQL statements execution runtime workflow in VS Code IDE, connected to the different database management systems via [SQL Tools extension](https://marketplace.visualstudio.com/items?itemName=mtxr.sqltools), database plugins, and [SQL Tools drivers](https://marketplace.visualstudio.com/search?term=tag%3Asqltools-driver&target=VSCode&category=All%20categories&sortBy=Rating). 10 | 11 | ![Pro Data Tools](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/pro-data-tools.png?raw=true) 12 | 13 | ![SQLite Pro Tools VS Code Extension Info](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/sqlite-tools-extension.png?raw=true) 14 | 15 | ## DuckDB Pro Tools 16 | 17 | Our [**Pro Data Tools**](https://vis.social/tags/ProDataTools) 🛠️ include [DuckDB Pro Tools](https://github.com/RandomFractals/pro-data-tools/blob/main/duckdb-tools.md#duckdb-pro-tools) extension that adds advanced [DuckDB](https://duckdb.org/) connection features and support to VSCode IDE, and provides database schemas display, DuckDB extensions and settings views, information schema and catalog views, SQL query API and user interfaces integrated with the popular SQL Tools extension, SQL query editor, SQL language server, and VS Code data processing tools. 18 | 19 | ![DuckDB Pro Tools Views](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/duckdb-pro-tools-views.gif?raw=true) 20 | 21 | You can read more about the free and public [DuckDB SQL Tools](https://github.com/RandomFractals/pro-data-tools/blob/main/duckdb-tools.md#duckdb-sql-tools) *Preview* VS Code extension and new Premium [DuckDB Pro Tools](https://github.com/RandomFractals/pro-data-tools/blob/main/duckdb-tools.md#duckdb-pro-tools) extension capabilites and features in our new [DuckDB Tools](https://randomfractals.github.io/pro-data-tools/duckdb-tools) documentation page. 22 | 23 | ## SQLite Pro Tools 24 | 25 | **Pro Data Tools** VS Code extensions also include [**SQLite Pro Tools**](https://github.com/RandomFractals/pro-data-tools/blob/main/sqlite-tools.md). **SQLite Pro Tools** provide over a dozen of common SQLite database shortcut [Commands](https://github.com/RandomFractals/pro-data-tools/blob/main/sqlite-tools.md#commands), display table column data type and primary key indicators, SQLite database [Indexes](https://github.com/RandomFractals/pro-data-tools/blob/main/sqlite-tools.md#indexes), [Modules](https://github.com/RandomFractals/pro-data-tools/blob/main/sqlite-tools.md#modules), [Configuration Settings](https://github.com/RandomFractals/pro-data-tools/blob/main/sqlite-tools.md#configuration-settings) and [Functions](https://github.com/RandomFractals/pro-data-tools/blob/main/sqlite-tools.md#functions). 26 | 27 | ![SQLite Pro Tools Views](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/sqlite-tools.png?raw=true) 28 | 29 | ## PRQL Pro Tools 30 | 31 | Our [**Pro sponsors**](https://github.com/sponsors/RandomFractals/sponsorships?tier_id=18884) on GitHub also get access to the Premium [PRQL Pro Tools](https://github.com/RandomFractals/pro-data-tools/blob/main/prql-tools.md) VS Code extension. **PRQL Pro Tools** extension comes with a custom [PRQL Code Lens](https://github.com/RandomFractals/pro-data-tools/blob/main/prql-tools.md#prql-code-lens) SQL Tools plugin that allows you to run [PRQL](https://prql-lang.org/) queries for the supported PRQL target [SQL dialects](https://prql-lang.org/book/project/target.html#dialects) **directly** using any of the supported SQL Tools extension drivers for the different database management systems. 32 | 33 | ![PRQL Pro Tools Code Lens](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/prql-code-lens-pro-tools.gif?raw=true) 34 | 35 | ## Markdown SQL Pro Tools 36 | 37 | [**Markdown SQL Pro Tools**](https://github.com/RandomFractals/pro-data-tools/blob/main/markdown-sql-tools.md) extension adds custom SQL [Code Lenses](https://code.visualstudio.com/api/language-extensions/programmatic-language-features#codelens-show-actionable-context-information-within-source-code) to SQL code blocks in standard SQL and various markdown documents in VS Code IDE. 38 | 39 | The provided **SQL Code Lenses** include top-level SQL and markdown document `Execute All`, `Select All`, `Extract All`, `Bookmark All` SQL code lenses, `Execute`, `Select`, and `Copy` SQL code block lenses, and `Create` New SQL document from SQL code block lens. 40 | 41 | Our **SQL Code Lenses** integrate with the popular [SQL Tools](https://vscode-sqltools.mteixeira.dev/en/home/) Connections manager, supported [SQL Tools drivers](https://marketplace.visualstudio.com/search?term=tag%3Asqltools-driver&target=VSCode&category=All%20categories&sortBy=Rating), and query results viewer to execute SQL **directly** on any of the supported SQL Tools database management systems and view results. 42 | 43 | ![Markdown SQL Pro Tools](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/markdown-sql-pro-tools-v1.1.0.gif?raw=true) 44 | 45 | **Markdodown SQL Pro Tools** extension is available under the new [**Markdown Sql Pro Tools**](https://github.com/sponsors/RandomFractals/sponsorships?tier_id=295482) monthly sponsor subscription on GitHub and also includes access to the [**DuckDB Pro**](https://randomfractals.github.io/pro-data-tools/#duckdb-pro-tools) SQL Tools plugin and [**PRQL Code Lens**](https://github.com/RandomFractals/pro-data-tools/blob/main/prql-tools.md#prql-code-lens) extension from our Premium [**Pro Data Tools**](https://randomfractals.github.io/pro-data-tools/#pro-data-tools) extensions pack. 46 | 47 | ## Data Notebook Pro Tools 48 | 49 | The new **Data Notebook Pro Tools** extension from our Premium **Pro Data Tools** extensions pack will be released later this year for our [**Pro sponsors**](https://github.com/sponsors/RandomFractals/sponsorships?tier_id=18884) on GitHub. 50 | 51 | **Data Notebook Pro** extension integrates SQL Tools connections, supported database management systems, our [**Data Table Renderers**](https://marketplace.visualstudio.com/items?itemName=RandomFractalsInc.vscode-data-table) and VS Code Jupyter Notebook user interface to view and run SQL queries from `.sql` files and `.md` markdown documents with SQL code blocks using native VS Code Notebook View. 52 | 53 | ![Data Notebook Extension](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/data-notebook-extension.png?raw=true) 54 | 55 | ### Data Notebook SQL Query 56 | 57 | Simple example of running SQL query with the new **Data Notebook Pro Tools** extension: 58 | 59 | ![Data Notebook SQL Query](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/data-notebook-sql-query.gif?raw=true) 60 | 61 | ### Data Notebook Output Rendering 62 | 63 | Using different [**Data Table Renderers**](https://marketplace.visualstudio.com/items?itemName=RandomFractalsInc.vscode-data-table) in the **Data Notebook** created from SQL document with multiple SQL queries: 64 | 65 | ![Data Notebook Output Rendering](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/data-notebook-output-rendering.gif?raw=true) 66 | 67 | ## Future Pro Data Tools Releases 68 | 69 | We plan to publish new **Pro Data Tools** updates and releases monthly. The best way to keep up to date on the new features and enhancements added to **DuckDB Pro Tools** extension, [PRQL Pro Tools](https://github.com/RandomFractals/pro-data-tools/blob/main/prql-tools.md), recently released [Markdown SQL Pro Tools](https://github.com/RandomFractals/pro-data-tools/tree/main#markdown-sql-pro-tools), and the upcoming [Data Notebook Pro](https://vis.social/tags/DataNotebookPro) extension is to follow us on LinkedIn [@TarasNovak](https://www.linkedin.com/in/tarasnovak/recent-activity/all/), or follow [Data Samurai](https://vis.social/@dataSamurai) on Vis Social Mastodon. We share new [Pro Data Tools](https://vis.social/tags/ProDataTools) related demos and content there frequently. 70 | 71 | ## Pro Data Tools Changelog 72 | 73 | We also created new combined **Pro Data Tools** [changelog](https://randomfractals.github.io/pro-data-tools/CHANGELOG) to capture all the premium VS Code extensions and versions with detailed features list released to our [Pro sponsors](https://github.com/sponsors/RandomFractals/sponsorships?tier_id=18884) on GitHub this year. 74 | 75 | 📜 [https://github.com/RandomFractals/pro-data-tools/blob/main/CHANGELOG.md](https://github.com/RandomFractals/pro-data-tools/blob/main/CHANGELOG.md) 76 | 77 | # Feedback 78 | 79 | Please use our new [Pro Data Tools GitHub Discussions](https://github.com/RandomFractals/pro-data-tools/discussions) portal to submit your feedback, share examples of how you are using our public VS Code [data viz extensions](https://marketplace.visualstudio.com/publishers/RandomFractalsInc) and new **Pro Data Tools**, or request new trivial and premium features. Our goal with these generic Data Tools 🛠️ and SQL Tools extensions is to make local and remote data more accessible and easier to use and visualize for all the devs and data scientists out there using VS Code IDE, VSCodium and Azure Data Studio. 80 | 81 | # Support 82 | 83 | Become a [Fan or a Pro Sponsor](https://github.com/sponsors/RandomFractals) on GitHub to support our dev work on these Data Tools and other [Random Fractals, Inc. code and data viz extensions](https://marketplace.visualstudio.com/publishers/RandomFractalsInc) if you find them useful, educational, or enhancing your daily dataViz dev code workflows and exploratory data analysis experience. 84 | 85 | 💖 [https://github.com/sponsors/RandomFractals](https://github.com/sponsors/RandomFractals) 86 | -------------------------------------------------------------------------------- /docs/images/data-notebook-extension.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/data-notebook-extension.png -------------------------------------------------------------------------------- /docs/images/data-notebook-output-rendering.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/data-notebook-output-rendering.gif -------------------------------------------------------------------------------- /docs/images/data-notebook-prql-data-table-output.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/data-notebook-prql-data-table-output.gif -------------------------------------------------------------------------------- /docs/images/data-notebook-sql-query.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/data-notebook-sql-query.gif -------------------------------------------------------------------------------- /docs/images/duckdb-pro-tools-code-intellisense.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/duckdb-pro-tools-code-intellisense.gif -------------------------------------------------------------------------------- /docs/images/duckdb-pro-tools-db-tree-view.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/duckdb-pro-tools-db-tree-view.gif -------------------------------------------------------------------------------- /docs/images/duckdb-pro-tools-v1.3.0-docs.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/duckdb-pro-tools-v1.3.0-docs.gif -------------------------------------------------------------------------------- /docs/images/duckdb-pro-tools-views.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/duckdb-pro-tools-views.gif -------------------------------------------------------------------------------- /docs/images/duckdb-tools-extensions.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/duckdb-tools-extensions.png -------------------------------------------------------------------------------- /docs/images/duckdb-vscode-extensions.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/duckdb-vscode-extensions.png -------------------------------------------------------------------------------- /docs/images/markdow-sql-magic-line-select.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/markdow-sql-magic-line-select.png -------------------------------------------------------------------------------- /docs/images/markdown-sql-bookmarks.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/markdown-sql-bookmarks.gif -------------------------------------------------------------------------------- /docs/images/markdown-sql-bookmarks.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/markdown-sql-bookmarks.png -------------------------------------------------------------------------------- /docs/images/markdown-sql-code-highlight.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/markdown-sql-code-highlight.png -------------------------------------------------------------------------------- /docs/images/markdown-sql-extracted-sql-code.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/markdown-sql-extracted-sql-code.png -------------------------------------------------------------------------------- /docs/images/markdown-sql-in-quarto.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/markdown-sql-in-quarto.gif -------------------------------------------------------------------------------- /docs/images/markdown-sql-in-rmd.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/markdown-sql-in-rmd.gif -------------------------------------------------------------------------------- /docs/images/markdown-sql-jupysql-intro.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/markdown-sql-jupysql-intro.png -------------------------------------------------------------------------------- /docs/images/markdown-sql-jupysql.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/markdown-sql-jupysql.gif -------------------------------------------------------------------------------- /docs/images/markdown-sql-pro-tools-contributions.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/markdown-sql-pro-tools-contributions.png -------------------------------------------------------------------------------- /docs/images/markdown-sql-pro-tools-copy-code-lens.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/markdown-sql-pro-tools-copy-code-lens.png -------------------------------------------------------------------------------- /docs/images/markdown-sql-pro-tools-notebook.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/markdown-sql-pro-tools-notebook.png -------------------------------------------------------------------------------- /docs/images/markdown-sql-pro-tools-v1.1.0.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/markdown-sql-pro-tools-v1.1.0.gif -------------------------------------------------------------------------------- /docs/images/markdown-sql-pro-tools.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/markdown-sql-pro-tools.gif -------------------------------------------------------------------------------- /docs/images/markdown-sql-pro-tools.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/markdown-sql-pro-tools.png -------------------------------------------------------------------------------- /docs/images/markdown-sql-selections.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/markdown-sql-selections.png -------------------------------------------------------------------------------- /docs/images/markdown-sql-tools.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/markdown-sql-tools.png -------------------------------------------------------------------------------- /docs/images/pg-sql-tools-commands-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/pg-sql-tools-commands-1.png -------------------------------------------------------------------------------- /docs/images/pg-sql-tools-commands-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/pg-sql-tools-commands-2.png -------------------------------------------------------------------------------- /docs/images/pg-sql-tools-extension.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/pg-sql-tools-extension.png -------------------------------------------------------------------------------- /docs/images/pg-sql-tools-features.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/pg-sql-tools-features.png -------------------------------------------------------------------------------- /docs/images/pg-sql-tools-install-from-vsix.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/pg-sql-tools-install-from-vsix.png -------------------------------------------------------------------------------- /docs/images/pg-sql-tools.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/pg-sql-tools.png -------------------------------------------------------------------------------- /docs/images/pro-data-tools.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/pro-data-tools.png -------------------------------------------------------------------------------- /docs/images/prql-code-lens-pro-tools.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/prql-code-lens-pro-tools.gif -------------------------------------------------------------------------------- /docs/images/prql-code-lens.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/prql-code-lens.gif -------------------------------------------------------------------------------- /docs/images/prql-code-lens.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/prql-code-lens.png -------------------------------------------------------------------------------- /docs/images/prql-notebook.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/prql-notebook.png -------------------------------------------------------------------------------- /docs/images/prql-open-with-data-notebook.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/prql-open-with-data-notebook.gif -------------------------------------------------------------------------------- /docs/images/prql-pro-tools-v1.2.0-docs.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/prql-pro-tools-v1.2.0-docs.gif -------------------------------------------------------------------------------- /docs/images/prql-pro-tools.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/prql-pro-tools.png -------------------------------------------------------------------------------- /docs/images/random-fractals-data-tools.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/random-fractals-data-tools.png -------------------------------------------------------------------------------- /docs/images/sql-code-blocks-on-github.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/sql-code-blocks-on-github.png -------------------------------------------------------------------------------- /docs/images/sql-tools-drivers.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/sql-tools-drivers.png -------------------------------------------------------------------------------- /docs/images/sqlite-commands.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/sqlite-commands.png -------------------------------------------------------------------------------- /docs/images/sqlite-functions.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/sqlite-functions.png -------------------------------------------------------------------------------- /docs/images/sqlite-indexes.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/sqlite-indexes.png -------------------------------------------------------------------------------- /docs/images/sqlite-modules.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/sqlite-modules.png -------------------------------------------------------------------------------- /docs/images/sqlite-pro-tools.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/sqlite-pro-tools.png -------------------------------------------------------------------------------- /docs/images/sqlite-settings.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/sqlite-settings.png -------------------------------------------------------------------------------- /docs/images/sqlite-table-columns.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/sqlite-table-columns.png -------------------------------------------------------------------------------- /docs/images/sqlite-tools-extension.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/sqlite-tools-extension.png -------------------------------------------------------------------------------- /docs/images/sqlite-tools.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RandomFractals/pro-data-tools/e648ee1ee0700f110d87ee4603d5b2aa4ab66ef2/docs/images/sqlite-tools.png -------------------------------------------------------------------------------- /duckdb-tools.md: -------------------------------------------------------------------------------- 1 | # DuckDB Tools 2 | 3 | [DuckDB](https://duckdb.org/docs/) is an in-process SQL [OLAP](https://en.wikipedia.org/wiki/Online_analytical_processing) database management system that uses vectorized data engine and parallel query processing optimized for analytics. DuckDB provides extensive [SQL support](https://duckdb.org/docs/sql/introduction) with direct [data loading](https://duckdb.org/docs/data/overview#csv-loading) and query capabilities for local and remote data sources in `Parquet`, `CSV`, and `JSON` data formats. 4 | 5 | [Random Fractals, Inc.](https://marketplace.visualstudio.com/publishers/RandomFractalsInc) created two DuckDB Visual Studio Code IDE extensions that allow you to view and query local data, DuckDB files, and remote data sources: [DuckDB Pro Tools](https://randomfractals.github.io/pro-data-tools/#duckdb-pro-tools) and [DuckDB SQL Tools](https://marketplace.visualstudio.com/items?itemName=RandomFractalsInc.duckdb-sql-tools). 6 | 7 | ![DuckDB VS Code Extensions](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/duckdb-vscode-extensions.png?raw=true) 8 | 9 | # DuckDB SQL Tools 10 | 11 | The free and public [DuckDB SQL Tools](https://marketplace.visualstudio.com/items?itemName=RandomFractalsInc.duckdb-sql-tools) *Preview* VS Code extension adds basic DuckDB SQL support and provides DuckDB connection management, SQL query API and user interfaces for the popular [SQL Tools](https://vscode-sqltools.mteixeira.dev/en/home/) extension, SQL query editor, SQL language server, and data processing tools. 12 | 13 | ![DuckDB SQL Tools](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-sql-tools.gif?raw=true) 14 | 15 | ## DuckDB SQL Tools *Preview* Features 16 | 17 | [DuckDB SQL Tools](https://marketplace.visualstudio.com/items?itemName=RandomFractalsInc.duckdb-sql-tools) *Preview* extension v1.5.0 adds the following basic DuckDB SQL capabilities to VS Code IDE for working with [DuckDB v0.9](https://duckdb.org/2023/09/26/announcing-duckdb-090.html) instances: 18 | 19 | - **Connect** to a local DuckDB [v0.9.2](https://github.com/duckdb/duckdb/releases/tag/v0.9.2) file instance 20 | - **Create** new in-memory DuckDB instance 21 | - **View** DuckDB v0.9.2 tables, columns, and views 22 | - **Run** SQL queries on open DuckDB v0.9.2 connections 23 | - **Attach** SQLite database files to in-memory DuckDB instances 24 | - **Query** remote `CSV` and `Parquet` data files with [DuckDB HTTPFS](https://duckdb.org/docs/extensions/httpfs.html) extension 25 | - **Create** in-memory DuckDB tables from remote data sources and query results 26 | - **Manage** DuckDB connections in SQL Tools Connection Explorer 27 | - **Autocomplete** SQL keywords, table names, column names, and view names in SQL editor for an active DuckDB connection 28 | - **Save** named SQL query Bookmarks 29 | - **Use** SQL Query History 30 | - **Export** SQL query results in `CSV` and `JSON` data formats 31 | 32 | [SQL Tools documentation](https://vscode-sqltools.mteixeira.dev/en/home/) contains more information about the other generic SQL Tools extension features. 33 | 34 | ## DuckDB SQL Tools Installation 35 | 36 | Install [DuckDB SQL Tools](https://github.com/RandomFractals/duckdb-sql-tools) *Preview* extension from [VS Code marketplace](https://marketplace.visualstudio.com/items?itemName=RandomFractalsInc.duckdb-sql-tools) or directly in VS Code IDE from Extensions tab (`ctrl+shift+x`) by searching for `DuckDB`. 37 | 38 | ![DuckDB SQL Tools VS Code Extension Info](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-sql-tools-extension-info.png?raw=true) 39 | 40 | **DuckDB SQL Tools** [Installation](https://github.com/RandomFractals/duckdb-sql-tools#installation) documentation has instructions on how to install this *Preview* extension in [VSCodium](https://vscodium.com/) and other VS Code based IDEs like [Azure Data Studio](https://github.com/Microsoft/azuredatastudio). 41 | 42 | **DuckDB SQL Tools** documentation also lists all the [Limitations](https://github.com/RandomFractals/duckdb-sql-tools#limitations) and restrictions of this free public DuckDB VS Code extension *Preview* version, and recommended SQL Tools extension [Configuration](https://github.com/RandomFractals/duckdb-sql-tools#configuration) settings to use with our DuckDB VS Code Tools. 43 | 44 | # Pro Data Tools 45 | 46 | [**Pro Data Tools**](https://randomfractals.github.io/pro-data-tools/#pro-data-tools) is our new Visual Studio Code extensions pack with premium SQL query and data view features created for the monthly [**Pro sponsors**](https://github.com/sponsors/RandomFractals/sponsorships?tier_id=18884) of our extensions on GitHub. 47 | 48 | Our **Pro Data Tools** for devs and data scientists using VS Code IDE include **DuckDB Pro Tools**, [PRQL Pro Tools](https://randomfractals.github.io/pro-data-tools/#prql-pro-tools) and new [Markdown SQL Pro Tools](https://randomfractals.github.io/pro-data-tools/#markdown-sql-pro-tools). 49 | 50 | ![Pro Data Tools](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/pro-data-tools.png?raw=true) 51 | 52 | **Pro Data Tools** enhance SQL development experience, documentation and notebooks SQL code preview, SQL query execution and results rendering capablities for the different database management systems supported by the SQL Tools extension and plugins. 53 | 54 | # DuckDB Pro Tools 55 | 56 | **DuckDB Pro Tools** extension adds advanced DuckDB read/write connection features and support to VS Code IDE, and provides full [Database Schemas](https://randomfractals.github.io/pro-data-tools/duckdb-tools#duckdb-objects-tree-view) display, [DuckDB Extensions](https://randomfractals.github.io/pro-data-tools/duckdb-tools#extensions), [Views](https://randomfractals.github.io/pro-data-tools/duckdb-tools#duckdb-views), [Functions](https://randomfractals.github.io/pro-data-tools/duckdb-tools#functions), [Types](https://randomfractals.github.io/pro-data-tools/duckdb-tools#duckdb-types), [Indexes](https://randomfractals.github.io/pro-data-tools/duckdb-tools#indexes), [Sequences](https://randomfractals.github.io/pro-data-tools/duckdb-tools#sequences), [SQL Keywords](https://randomfractals.github.io/pro-data-tools/duckdb-tools#duckdb-sql-keywords) and [Configuration Settings](https://randomfractals.github.io/pro-data-tools/duckdb-tools#configuration-settings) views, information schema and [PG catalog](https://randomfractals.github.io/pro-data-tools/duckdb-tools#pg-catalog) views, SQL query API and user interfaces integrated with the popular [SQL Tools](https://vscode-sqltools.mteixeira.dev/en/home/) extension, SQL query editor, SQL language server, and VS Code data processing tools. 57 | 58 | ![DuckDB Pro Tools Views](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-pro-tools-views.gif?raw=true) 59 | 60 | ## DuckDB Pro Tools Features 61 | 62 | The latest version of [DuckDB Pro Tools](https://github.com/RandomFractals/pro-data-tools/blob/main/duckdb-tools.md#duckdb-pro-tools) extension adds the following capabilities to VS Code IDE for working with [DuckDB v0.10](https://duckdb.org/2024/06/03/announcing-duckdb-100.html) instances: 63 | 64 | - **Connect** to a local DuckDB [v0.10.2](https://github.com/duckdb/duckdb-node/releases/tag/v0.10.2) instance 65 | - **Create** new in-memory DuckDB instance 66 | - **Import** local and remote `CSV`, `JSON` and `Parquet` data files into in-memory DuckDB instance for exploratory data analysis (EDA) 67 | - **View** DuckDB v0.10.2 databases, schemas, tables, columns, views, indexes, sequences, extensions, settings, functions, types and keywords in SQL Tools Connections Explorer 68 | - **Run** SQL queries on active DuckDB connections 69 | - **Attach** SQLite database files to in-memory DuckDB instances to run analytical queries 70 | - **Query** remote `CSV`, `Parquet`, and `JSON` data files with [DuckDB HTTPFS](https://duckdb.org/docs/extensions/httpfs.html) extension and new [DuckDB JSON](https://duckdb.org/2023/03/03/json.html) extension 71 | - **Create** in-memory DuckDB tables from remote data sources and query results 72 | - **Manage** DuckDB 0.10.2 connections in SQL Tools Connections Explorer 73 | - **Auto-complete** SQL keywords, DuckDB instance schemas, table names, column names, and view names for active DuckDB connections in VS Code SQL editor 74 | - **Save** named SQL query Bookmarks 75 | - **Use** SQL Query History 76 | - **Export** DuckDB query results in `CSV` and `JSON` data formats 77 | - **Use** [PRQL Code Lens](https://github.com/RandomFractals/prql-pro-tools#prql-code-lens) from our new [PRQL Pro Tools](https://randomfractals.github.io/pro-data-tools/#prql-pro-tools) collection to generate and run SQL queries on active DuckDB connection 78 | - **Explore** new `employees.duckdb` demo data, PRQL and SQL sample queries 79 | - **Run** sample `chicago-crimes` and `gbif-observations` [PRQL](https://prql-lang.org/) and SQL queries on GitHub and AWS S3 hosted `parquet` data files 80 | - **Use** over 30 built-in `DuckDB Tools` views and metadata shortcut commands from VS Code `Command Palette...` 81 | 82 | ## DuckDB Pro Tools Installation 83 | 84 | Become a [Pro Sponsor](https://github.com/sponsors/RandomFractals/sponsorships?tier_id=18884) of our new **Pro Data Tools** work on GitHub to access **DuckDB Pro Tools**, [PRQL Pro Tools](https://randomfractals.github.io/pro-data-tools/#prql-pro-tools) and our new [Markdown SQL Pro Tools](https://randomfractals.github.io/pro-data-tools/#markdown-sql-pro-tools) releases and extension `.vsix` installation packages. 85 | 86 | All of our GitHub **Pro** sponsors get invited as collaborators to our private [Pro Data Tools](https://randomfractals.github.io/pro-data-tools/#pro-data-tools) repositories, and will be notified about the upcoming updates, new releases, additions, and new [Data Notebook Pro Tools](https://randomfractals.github.io/pro-data-tools/#data-notebook-pro-tools) extension we'll be releasing later this year. 87 | 88 | To install **DuckDB Pro Tools**, download the latest `duckdb-pro-tools-x.x.x.vsix` from the attached **Assets** in repository [releases](https://github.com/RandomFractals/duckdb-pro-tools/releases). 89 | 90 | Use VS Code IDE [Install from VSIX](https://code.visualstudio.com/docs/editor/extension-marketplace#_install-from-a-vsix) feature to install downloaded **DuckDB Pro Tools** extension package in VS Code or any other `VSIX`-compatible IDE. **DuckDB Pro Tools** work in [VSCodium](https://vscodium.com/) and the latest version of [Azure Data Studio](https://github.com/Microsoft/azuredatastudio) IDE. 91 | 92 | ![DuckDB Pro Tools VS Code Extension Info](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-tools-extensions.png?raw=true) 93 | 94 | The other [DuckDB Sql Tools](https://marketplace.visualstudio.com/items?itemName=RandomFractalsInc.duckdb-sql-tools) extension you see in the **Installed** DuckDB extensions image above is our Free Trial public version of **DuckDB Tools** that only supports limited set of features, demo DuckDB files and sample queries to run. 95 | 96 | ## DuckDB Upgrade 97 | 98 | If you already have the free public DuckDB SQL Tools installed, and configured new **DuckDBPro** connection to use with **DuckDB Pro Tools** extension, you'll be prompted to update `duckdb-async` library to v0.10.2 to use the latest DuckDB version and features. 99 | 100 | You might need to restart VS Code after duckdb library update for the new DuckDB 0.10.2 connection and queries to work. Below is a quick demo of that DuckDB library update process. 101 | 102 | ![DuckDB Pro Tools Libraries Update](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-pro-tools-upgrade.gif?raw=true) 103 | 104 | **Note**: **DuckDB Pro Tools** use [DuckDB Node.js API](https://github.com/duckdb/duckdb-node) and require a local [Node.js installation](https://nodejs.org/en/download) to query DuckDB instances. 105 | 106 | Download and install **Node.js** from the official [node.js downloads](https://nodejs.org/en/download) page. Node.js is used as a local web server to host local data and DuckDB files. Node.js will install **npm** tool we use to install `duckdb-async` library to establish DuckDB connections. 107 | 108 | We use Node.js DuckDB library instead of the limited DuckDB WASM browser JS library other similar DuckDB data tools use to enable faster data imports and loading via multi-threaded [`node-gyp`](https://github.com/nodejs/node-gyp) DuckDB native API interface. 109 | 110 | # DuckDB Objects Tree View 111 | 112 | **DuckDB Pro Tools** v1.1.0 and higher now display the full catalog of DuckDb instance objects to view and query from SQL Tools Connections Explorer. The extended DuckDB objects tree view includes main DuckDB instance, `system`, and `temp` databases with the corresponding `main`, [`information_schema`](https://duckdb.org/docs/sql/information_schema) and [`pg_catalog`](https://www.postgresql.org/docs/current/catalogs-overview.html) schemas. 113 | 114 | ![DuckDB Pro Tools DB Tree View](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-pro-tools-db-tree-view.gif?raw=true) 115 | 116 | ## PG Catalog 117 | 118 | DuckDB provides many standard PostgreSQL [`pg_catalog`](https://www.postgresql.org/docs/current/catalogs-overview.html) functions and views for compatibility with PostgreSQL database and SQL syntax. You can explore those internal DuckDB views and function signatures via `pg_catalag` schema in the DuckDB Objects tree view. 119 | 120 | ![DuckDB Pro Tools PG Catalog](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-pro-tools-pg-catalog-functions.png?raw=true) 121 | 122 | ## DuckDB Objects and Views 123 | 124 | **DuckDB Pro Tools** v1.1.0 and higher DB Objects tree view also shows the full list of DuckDB [indexes](https://duckdb.org/docs/sql/indexes), [sequences](https://duckdb.org/docs/sql/statements/create_sequence), [`information_schema`](https://duckdb.org/docs/sql/information_schema#database-catalog-and-schema) and other custom `duckdb` [metadata functions](https://duckdb.org/docs/sql/duckdb_table_functions) and internal views. 125 | 126 | ![DuckDB Pro Tools Views](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-pro-tools-views.gif?raw=true) 127 | 128 | ### DuckDB Views 129 | 130 | ![DuckDB Pro Tools Views](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-pro-tools-internal-views.png?raw=true) 131 | 132 | ### Indexes 133 | 134 | ![DuckDB Pro Tools Indexes](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-pro-tools-indexes-view.png?raw=true) 135 | 136 | ### Sequences 137 | 138 | ![DuckDB Pro Tools Sequences](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-pro-tools-sequences-view.png?raw=true) 139 | 140 | ### Extensions 141 | 142 | ![DuckDB Pro Tools Extensions](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-pro-tools-extensions-tree-view.png?raw=true) 143 | 144 | ### Functions 145 | 146 | ![DuckDB Pro Tools DB Functions](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-pro-tools-functions-tree-view.png?raw=true) 147 | 148 | ### DuckDB Types 149 | 150 | ![DuckDB Pro Tools DB Types](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-pro-tools-types-tree-view.png?raw=true) 151 | 152 | ### DuckDB SQL Keywords 153 | 154 | ![DuckDB Pro Tools SQL Keywords](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-pro-tools-keywords-tree-view.png?raw=true) 155 | 156 | ## Configuration Settings 157 | 158 | New DuckDB [configuration settings](https://duckdb.org/docs/sql/configuration) display with setting type icons and current values in the DuckDB Objects tree view item description. 159 | 160 | ![DuckDB Configuration Settings Tree View](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-pro-tools-settings-tree-view.png?raw=true) 161 | 162 | ## Table and View Columns 163 | 164 | DuckDB configuration settings, database schema types, table and view columns now show basic data type icons for strings, numeric, date and timestamp columns and setting types. 165 | 166 | ![DuckDB Table and View Columns](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-pro-tools-columns-view.png?raw=true) 167 | 168 | Additionally, **DuckDB Pro Tools** v1.1.0 and higher display primary columns in tables, and `not null` column value constraints. 169 | 170 | # DuckDB View Commands 171 | 172 | **DuckDB Pro Tools** v1.2.0 added over 30 built-in DuckDB views and metadata shortcut commands to the Command Palette you can access via `ctrl/cmd+shift+p` shortcut in VS Code or from `View -> Commnad Palette...` menu by typing `DuckDB Tools` in the `>` command prompt: 173 | 174 | ![DuckDB Tools View Commands](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-pro-tools-view-commands.gif?raw=true) 175 | 176 | # DuckDB Extensions 177 | 178 | DuckDB egnine itself provides a number of [Extensions](https://duckdb.org/docs/extensions/overview) you can install and load to work with local and remote `CSV` and `Parquet` data files over [HTTPFS](https://duckdb.org/docs/extensions/httpfs), enable [Full Text Search](https://duckdb.org/docs/extensions/full_text_search), attach [SQLite](https://sqlite.org/about.html) database with [SQLite Scanner](https://duckdb.org/docs/extensions/sqlite_scanner), or attach [PostgreSQL](https://www.postgresql.org/about/) database instance with DuckDB [Postgres Scanner](https://duckdb.org/docs/extensions/postgres_scanner). 179 | 180 | You can check the list of core and installed DuckDB extensions by running the following SQL query on an open/active DuckDB connection: 181 | 182 | ```sql 183 | select * from duckdb_extensions(); 184 | ``` 185 | 186 | ![DuckDB Pro Tools DuckDB Extensions](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-pro-tools-duckdb-extensions.png?raw=true) 187 | 188 | ## DuckDB JSON 189 | 190 | [DuckDB v0.7](https://duckdb.org/2023/02/13/announcing-duckdb-070.html) and higher DuckDB `v0.8.1` and `v0.9.2` include new and improved [`JSON` extension](https://duckdb.org/2023/03/03/json.html) capabilities to work with JSON documents. **DuckDB Pro Tools** install and load DuckDB JSON extension for all active DuckDB conenctions in order to enable quick JSON data imports and other JSON related features without extra SQL setup. 191 | 192 | Quick DuckDB JSON extension query example and steps to parse and load remote `JSON` data document from a public Github repository with **DuckDB Pro Tools** in VS Code: 193 | 194 | ⥱ Create new in-memory DuckDBPro config. 195 | 196 | ⥱ Use JSON data http://raw.githubusercontent.com URL in your SELECT statement. 197 | 198 | ⥱ Optionally use `CREATE TABLE ... AS` SQL statement to load retrieved `JSON` data into new DuckDB table and run multiple queries on that data loaded into memory. 199 | 200 | ![DuckDB Pro Tools JSON Data Load](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-pro-tools-json-data-load.gif?raw=true) 201 | 202 | **Note**: the US airports demo data in example above is from our VS Code [Data Table Renderers](https://github.com/RandomFractals/vscode-data-table/tree/main/data) extension we will use later in the upcoming [Data Notebook Pro](https://github.com/RandomFractals/pro-data-tools#data-notebook-pro-tools) extension to query and display tabular data results from many different data sources, including DuckDB. New **Data Notebook Pro** extension will be released in Q4 of 2024. 203 | 204 | Quick Data Notebook extension info preview with a simple Data Connections side panel and a few commands already implemented for that custom VS Code Notebook extension we plan to release with SQL and PRQL queries support. 205 | 206 | ![Data Notebook Pro Extension Info](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/data-notebook-extension.png?raw=true) 207 | 208 | ## DuckDB HTTPFS 209 | 210 | **DuckDB Pro Tools** extension also installs and loads [HTTPFS](https://duckdb.org/docs/extensions/httpfs) DuckDB extension by default for all the open database instances. You can add other [DuckDB Extensions](https://duckdb.org/docs/extensions/overview) to an open database instance by running [`INSTALL` and `LOAD`](https://duckdb.org/docs/extensions/overview#remote-installation) extension SQL statements. 211 | 212 | Example of loading trimmed down Chicago crimes data reported in 2022 into an in-memory DuckDB instance from a `.parquet` data file hosted in our [Chicago Crimes](https://github.com/RandomFractals/chicago-crimes) data and analytical tools demo Github repository: 213 | 214 | ![DuckDB Pro Tools HTTPFS](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-sql-tools-httpfs-data-load.gif?raw=true) 215 | 216 | This example uses implicitly loaded DuckDB HTTPFS extension to query reported Chicago crimes parquet data file with over 210K recorded crime reports, creates a `CrimeReports` table in a new DuckDB `:memory:` instance from remote parquet data file, and queries imported data. 217 | 218 | ## DuckDB SQLite Scanner 219 | 220 | [SQLite](https://sqlite.org/about.html) database users can use **DuckDB Pro Tools** VS Code extension and DuckDB [SQLite Scanner](https://duckdb.org/docs/extensions/sqlite_scanner) extension to add data from SQLite database to in-memory DuckDB instance. 221 | 222 | Run the following SQL statements to add SQLite Scanner DuckDB extension to an open database connection: 223 | 224 | ```sql 225 | INSTALL sqlite; 226 | LOAD sqlite; 227 | ``` 228 | 229 | With the loaded SQLite Scanner DuckDB extension you can attach SQLite database file to a DuckDB database instance. Attached SQLite database tables will show up as views in DuckDB instance. 230 | 231 | Run the following SQL `CALL` function to attach SQLite database instance: 232 | 233 | ```sql 234 | CALL sqlite_attach('E:\\projects\\data\\tools\\duckdb-tools\\data\\chinook\\sqlite\\chinook.sqlite'); 235 | ``` 236 | 237 | ![DuckDB Pro Tools SQLite Scanner](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-sql-tools-sqlite-scanner.gif?raw=true) 238 | 239 | ## DuckDB File References 240 | 241 | **Note**: **DuckDB Pro Tools** extension uses [DuckDB NodeJS Client API](https://duckdb.org/docs/api/nodejs/overview). In order to work with local data files, you need to specify full path to your local database or data files in SQL statements that reference local file paths. 242 | 243 | Future versions of this extension will simplify local file path references by deducing absolute file path from the local DuckDB file connection string or open VS Code project workspace folder path, and replace relative database or data file references with the corresponding absolute path in an open VS Code project workspace. 244 | 245 | Also, note in the `sqlite_attach()` SQL function call statement above we are escaping `\` file path delimiters on Windows OS by using `\\` characters sequence. 246 | 247 | # Demo Data 248 | 249 | **DuckDB Pro Tools** extension documentation repository contains sample [/data](https://github.com/RandomFractals/duckdb-pro-tools/tree/main/data/chinook/duckdb) folder with the popular [`chinook.duckdb`](https://github.com/RandomFractals/duckdb-pro-tools/blob/main/data/chinook/duckdb/chinook.duckdb) and [`chinook.sqlite`](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/data/chinook/sqlite/chinook.sqlite) database files you can experiment with. 250 | 251 | This private beta repository also contains new SQL scripts, create demo DuckDB files Jupyter notebooks, `schema.sql` and `load.sql` DuckDB scripts with demo `CSV` and `Parquet` data files, and new PRQL and SQL query examples for [Chicago Crimes](https://github.com/RandomFractals/duckdb-pro-tools/tree/main/data/chicago-crimes), [Employees](https://github.com/RandomFractals/duckdb-pro-tools/tree/main/data/employees), [GBIF Observations](https://github.com/RandomFractals/duckdb-pro-tools/tree/main/data/gbif-observations), [TPC-DS](https://github.com/RandomFractals/duckdb-pro-tools/tree/main/data/tpc-ds) and [TPC-H](https://github.com/RandomFractals/duckdb-pro-tools/tree/main/data/tpc-h) benchmarks, and some public [Transporation](https://github.com/RandomFractals/duckdb-pro-tools/tree/main/data/transportation) datasets query examples our Pro sponsors on github can use to get started with **DuckDB Pro Tools** in VS Code. 252 | 253 | ![DuckDB Pro Tools Demo Data](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-pro-tools-data.png?raw=true) 254 | 255 | ## SQL Query Examples 256 | 257 | The demo [`/data`](https://github.com/RandomFractals/duckdb-pro-tools/tree/main/data) folder contains sample `.sql` files with SQL and PRQL query examples you can try. You can also easily generate new SQL queries using recently released [Hugging Face Code Autocomplete](https://marketplace.visualstudio.com/items?itemName=HuggingFace.huggingface-vscode) VS Code extension. 258 | 259 | For example, SQL queries in [employees/prompt.sql](https://github.com/RandomFractals/duckdb-pro-tools/blob/main/data/employees/prompt.sql) were generated with HF Code Autocomplete. 260 | 261 | ![DuckDB SQL Code Autocomplete](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-pro-tools-prompt-queries-gen.gif?raw=true) 262 | 263 | ![DuckDB Prompt Queries Run](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-pro-tools-employees-prompt-queries.gif?raw=true) 264 | 265 | We'll add more demo data, sample DuckDB files, PRQL and SQL query examples in future **DuckDB Pro Tools** releases. 266 | 267 | # Limitations 268 | 269 | While **DuckDB Pro Tools** offer more features and examples, and work with the latest version of DuckDB, there are still some limitations in the initial version of this extension which we plan to address in the upcoming monthly updates and releases. 270 | 271 | ## DuckDB Storage 272 | 273 | The latest version of **DuckDB Pro Tools** extension supports local DuckDB instances created with [DuckDB v0.10.2](https://github.com/duckdb/duckdb/releases) engine. Database instances and files created with prior versions of DuckDB are not supported as they use different compression and [storage](https://duckdb.org/internals/storage) formats and the structure of `.duckdb` file has been changing as DuckDB engine is evolving. 274 | 275 | You can still use our free limited capabilities public [DuckDB SQL Tools](https://github.com/RandomFractals/duckdb-sql-tools) extension to load and query older DuckDB v0.6.1 data files. Switching between older DuckDB Sql Tools and this **DuckDB Pro Tools** version and DuckDB file configs will prompt you to upgrade or downgrade the corresponding DuckDB Node.js library accordingly, but you can keep both extensions and switch the extension you want to use depending on the DuckDB files version you are working with. 276 | 277 | Otherwise, Use [DuckDB CLI](https://duckdb.org/docs/api/cli.html) to export data from the older database file versions and create new `.duckdb` file using the latest DuckDB storage implemenation. 278 | 279 | Read [Announcing DuckDB 0.6.0](https://duckdb.org/2022/11/14/announcing-duckdb-060.html), [Announcing DuckDB 0.7.0](https://duckdb.org/2023/02/13/announcing-duckdb-070.html), and [Announcing DuckDB 0.8.0](https://duckdb.org/2023/05/17/announcing-duckdb-080.html) blog posts for more information about DuckDB storage improvements, new features, and other changes. 280 | 281 | ## DuckDB Read/Write 282 | 283 | **DuckDB Pro Tools** extension v1.1.0 and above now open `.duckdb` database files in `read-write` file access mode. 284 | 285 | **NOTE**: current DuckDB write access mode and connection handling in **DuckDB Pro Tools** v1.1.0 and latest is very alpha and some restrictions still apply: 286 | 287 | - DuckDB v0.10.2 still supports only **one** active `write` connection, or multipple `readers` and no `writers` in its latest implementation. 288 | 289 | - DuckDB Pro Tools will create a `read-only` connection if opening `.duckdb` file in `read-write` access mode fails. 290 | 291 | - Additionally, we now add [`PRAGMA enable_checkpoint_on_shutdown`](https://duckdb.org/docs/sql/pragmas#enable_checkpoint_on_shutdown-disable_checkpoint_on_shutdown) to all open DuckDB connections to run [`checkpoint`](https://duckdb.org/docs/sql/statements/checkpoint) when all DuckDB connecitons are closed and the parent NodeJS process releases a lock on the `.wal` write ahead log file in order to synchronize all writes and database updates to the main `.duckdb` file. 292 | 293 | **Important**: we recommend you don't close open DuckDB connections via SQL Tools Connections close context menu and let VS Code, NodeJS, and DuckDB processes handle data sync from `.wal` files on shutdown. 294 | 295 | **DuckDB Pro Tools** let you open multiple DuckDB connections while keeping only one DuckDB connection as current or active when you select that connection in SQL Tools Connections Explorer view. If you close an active DuckDB connection and get an error while trying to reconnect, we suggest you **restart VS Code** for DuckDB to release `.wal` file lock and grant new write or read-only database file access. 296 | 297 | We are working on a new custom DuckDB Connection Pooling interface that will handle DuckDB connections better, and will prompt you to close another write connection open by another process or a program, when the DuckDB `.wal` file is detected or locked by another process for the local DuckDB instance you attempt to update. 298 | 299 | ## In-Memory DuckDB 300 | 301 | **DuckDB Pro Tools** extension lets you create in-memory database instances by specifying `:memory:` keyword in the Database File field of the new DuckDB connection in SQL Tools Connection Assistant as demonstrated in DuckDB HTTPFS and SQLite Scanner extension usage examples above. 302 | 303 | You can leverage writable `:memory:` DuckDB instances in **DuckDB Pro Tools** to a great extent considering they allow you to load and query remote data files and export data as needed, as demonstrated in `JSON`, `HTTPFS` and other examples above. 304 | 305 | In-Memory DuckDB instances function similar to [In-Memory SQLite Databases](https://www.sqlite.org/inmemorydb.html). You can use [DuckDB Import Data](https://duckdb.org/docs/data/overview) and [Atach](https://duckdb.org/docs/sql/statements/attach) features available via DuckDB SQL statements, and use [Export Database](https://duckdb.org/docs/sql/statements/export) SQL statements to export created in-memory DuckDB instances. 306 | 307 | # Configuration 308 | 309 | [SQL Tools](https://marketplace.visualstudio.com/items?itemName=mtxr.sqltools) extension provides many configuration [Settings](https://vscode-sqltools.mteixeira.dev/en/settings/) users can toggle to change database connection and tree view display options, sql formatting, and results display. 310 | 311 | The following [SQL Tools Settings](https://vscode-sqltools.mteixeira.dev/en/settings/) were used while creating and testing **DuckDB Pro Tools** extension and are recommended for working with DuckDB instances efficiently. We suggest you set these preferences in User Settings in VS Code by navigating to `File -> Preferences -> Settings -> User -> Extensions -> SQLTools`, or adding them to your global VS Code `settings.json` config file using the JSON code snippet below: 312 | 313 | ```json 314 | { 315 | ... 316 | "sqltools.useNodeRuntime": true, 317 | "sqltools.disableNodeDetectNotifications": true, 318 | "sqltools.autoOpenSessionFiles": false, 319 | "sqltools.results.limit": 10000, 320 | "sqltools.results.location": "current", 321 | } 322 | ``` 323 | 324 | | Setting | Description | 325 | | --- | --- | 326 | | ```"sqltools.useNodeRuntime": true``` | Enable Node runtime in order to use [DuckDB NodeJS API](https://github.com/duckdb/duckdb/tree/master/tools/nodejs) this database SQL tools extension depends on. | 327 | | ```"sqltools.disableNodeDetectNotifications": true``` | Disable Node runtime detection notifications after initial SQLTools extension installation to prevent Node runtime information message display on every new VS Code session start. | 328 | | ```"sqltools.autoOpenSessionFiles": false``` | Prevent auto open of new session SQL editor instance after connecting to the databaase instance. | 329 | | ```"sqltools.results.limit": 10000``` | Maximum number of records to return in results. SQL Tools defaults to displaying only 50 records in query results view. Changing this limit setting to 1000 or 10000 will show more data rows to inspect in result views. | 330 | | ```"sqltools.results.location": "current"``` | Defines the editor group to use for result table views. SQL Tools displays all results in the next editor group to show results on the side next to the active SQL query editor. Changing this setting to `current` will display results in the same editor group and display more result columns. | 331 | 332 | # Feedback 333 | 334 | Please use our [DuckDB SQL Tools GitHub Discussions](https://github.com/RandomFractals/duckdb-sql-tools/discussions) portal to submit your feedback, share examples of how you are using our DuckDB VS Code extensions, or request new trivial and premium features. Our goal with these DuckDB SQL and Pro Data Tools extensions is to make DuckDB more accessible and easier to use in VS Code IDE. 335 | 336 | # Support 337 | 338 | Become a [Fan or a Pro Sponsor](https://github.com/sponsors/RandomFractals) on GitHub to support our dev work on these DuckDB Tools and other [Random Fractals, Inc. code and data viz extensions](https://marketplace.visualstudio.com/publishers/RandomFractalsInc) if you find them useful, educational, or enhancing your daily dataViz dev code workflows and exploratory data analysis experience. 339 | 340 | 💖 [https://github.com/sponsors/RandomFractals](https://github.com/sponsors/RandomFractals) 341 | -------------------------------------------------------------------------------- /markdown-sql-tools.md: -------------------------------------------------------------------------------- 1 | ## Markdown SQL Pro Tools 2 | 3 | **Markdown SQL Pro Tools** extension adds custom SQL [Code Lenses](https://code.visualstudio.com/api/language-extensions/programmatic-language-features#codelens-show-actionable-context-information-within-source-code) to SQL code blocks in standard SQL and various markdown documents in VS Code IDE. 4 | 5 | The provided **SQL Code Lenses** include top-level SQL and markdown document `Execute All`, `Select All`, `Extract All`, `Bookmark All` SQL code lenses, `Execute`, `Select`, and `Copy` SQL code block lenses, and `Create` New SQL document from SQL code block lens. 6 | 7 | Our **SQL Code Lenses** integrate with the popular [SQL Tools](https://vscode-sqltools.mteixeira.dev/en/home/) Connections manager, supported [SQL Tools drivers](https://marketplace.visualstudio.com/search?term=tag%3Asqltools-driver&target=VSCode&category=All%20categories&sortBy=Rating), and query results viewer to execute SQL **directly** on any of the supported SQL Tools database management systems and view results. 8 | 9 | ![Markdown SQL Pro Tools](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/markdown-sql-pro-tools-v1.1.0.gif?raw=true) 10 | 11 | ## Markdown SQL Features 12 | 13 | **Markdown SQL Pro Tools** add the following top-level **SQL Code Lenses** to SQL and markdown documents open in VS Code IDE: 14 | 15 | - `Execute All` SQL code lens to execute all SQL statements in the open SQL or markdown document 16 | - `Select All` SQL code lens to select all SQL code blocks in the open markdown document 17 | - `Extract All` SQL code lens to extract all SQL code blocks from the open markdown document 18 | - `Bookmark All` SQL code lens to bookmark all SQL code blocks in the open markdown document 19 | 20 | This extension also adds the following SQL Code Lenses to individual SQL code blocks in the open markdown documents and SQL statements in SQL documents: 21 | 22 | - `Execute` SQL code lens to run SQL code using active SQL Tools database connection 23 | - `Select` SQL code lens to select SQL code block in the open SQL or markdown document editor 24 | - `Copy` SQL code lens to copy SQL code block to the clipboard 25 | - `Create` SQL code lens to create new SQL document from SQL code block in the open text editor 26 | 27 | ### Supported Markdown Documents and SQL Highlights 28 | 29 | **Markdown SQL Pro Tools** detect and highlight SQL code blocks in stanadard `.md` markdown documents, `.Rmd` [R markdown](https://rmarkdown.rstudio.com/) documents, `.qmd` [Quarto](https://quarto.org/) publishing documents, `.ojs` and `.omd` [Observable JS](https://marketplace.visualstudio.com/items?itemName=GordonSmith.observable-js) and markdown documents. 30 | 31 | The supported SQL code blocks include [code fences](https://www.markdownguide.org/extended-syntax/#fenced-code-blocks) with `sql` tag, `%%sql` magic code blocks, and `%sql` magic code lines with `SELECT` statements. 32 | 33 | ![Markdown SQL Code Highlights](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/markdown-sql-code-highlight.png?raw=true) 34 | 35 | You can use new top-level `Select All` SQL code lens to select all detected SQL code blocks in the open markdown document. 36 | 37 | ![Markdown SQL Code Selections](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/markdown-sql-selections.png?raw=true) 38 | 39 | ### Markdown SQL Code Bookmarks 40 | 41 | Users of the popular [Bookmarks](https://marketplace.visualstudio.com/items?itemName=alefragnani.Bookmarks) VS Code extension can use new top-level `Bookmark All` SQL code lens to toggle SQL code bookmarks in the open markdown documents. 42 | 43 | ![Markdown SQL Code Bookmarks](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/markdown-sql-bookmarks.png?raw=true) 44 | 45 | Using **Markdown SQL Pro Tools** with new **SQL Bookmarks**, SQLite and our **DuckDB Pro** SQL Tools in VS Code IDE on [Ploomber's JupySQL](https://github.com/ploomber/jupysql/blob/master/doc/quick-start.md) Jupyter book markdown docs: 46 | 47 | ![Using Markdown SQL Code Bookmarks](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/markdown-sql-bookmarks.gif?raw=true) 48 | 49 | **Note**: Install [Bookmarks](https://marketplace.visualstudio.com/items?itemName=alefragnani.Bookmarks) extension separately to see new `Bookmark All` SQL code lens enabled in the supported markdown documents. 50 | 51 | ### Interactive SQL in Quarto Docs 52 | 53 | The **Markdown SQL Pro Tools** v1.2.0 and above add **SQL Code Lenses** to SQL code blocks in [Quarto](https://quarto.org/) markdown docs in VS Code IDE. 54 | 55 | Quick demo of setting up local HR SQLite database with SQL Tools VS Code extension, rendering Quarto markdown document with [Quarto](https://marketplace.visualstudio.com/items?itemName=quarto.quarto) extension, and using our new SQL code lenses to Execute SQL queries from standard `.md` and `.qmd` markdown documents code: 56 | 57 | ![Interactive SQL in Quarto Docs](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/markdown-sql-in-quarto.gif?raw=true) 58 | 59 | You can install and use [Interactive SQL](https://github.com/shafayetShafee/interactive-sql) Quarto extension to experiment with it in VS Code with Quarto markdown docs. 60 | 61 | ### Interactive SQL in R Markdown 62 | 63 | You can also use new **SQL Code Lenses** in [R Markdown](https://rmarkdown.rstudio.com/) documents in VS Code IDE. 64 | 65 | [SQL in RMD](https://github.com/sciencificity/sql-in-rmd/tree/main) GitHub repository provides a great example of using SQL code chunks in R markdown documents. 66 | 67 | Setting up demo SQLite database connection and using our **SQL Code Lenses** to run SQL queries from R markdown document: 68 | 69 | ![Interactive SQL in R Markdown](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/markdown-sql-in-rmd.gif?raw=true) 70 | 71 | ## Markdown SQL Feature Contributions 72 | 73 | **Markdown SQL Pro Tools** contribute the following Commands and Activation Events to VS Code IDE for working with SQL code in markdown and SQL documents. 74 | 75 | ![Markdown SQL Pro Tools Feature Contributions](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/markdown-sql-pro-tools-contributions.png?raw=true) 76 | 77 | **Markdodown SQL Pro Tools** extension is available under the new [**Markdown Sql Pro Tools**](https://github.com/sponsors/RandomFractals/sponsorships?tier_id=295482) monthly sponsor subscription on GitHub and also includes access to the [**DuckDB Pro**](https://randomfractals.github.io/pro-data-tools/#duckdb-pro-tools) SQL Tools plugin and [**PRQL Code Lens**](https://github.com/RandomFractals/pro-data-tools/blob/main/prql-tools.md#prql-code-lens) extension from our Premium [**Pro Data Tools**](https://randomfractals.github.io/pro-data-tools/#pro-data-tools) extensions pack. 78 | -------------------------------------------------------------------------------- /prql-tools.md: -------------------------------------------------------------------------------- 1 | # PRQL Pro Tools 2 | 3 | **PRQL Pro Tools** is a collection of custom [PRQL](https://prql-lang.org/) extensions integrated with the popular [SQL Tools](https://vscode-sqltools.mteixeira.dev/en/home/) database management system and plugins to use in VS Code IDE. 4 | 5 | ![PRQL Code Lens](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/prql-code-lens.gif?raw=true) 6 | 7 | ## PRQL 8 | 9 | [PRQL](https://prql-lang.org/) is a modern language for transforming data. PRQL is database agnostic and compiles to many SQL dialects for the target database management system as described in [PRQL Book](https://prql-lang.org/book). 10 | 11 | ![PRQL Showcase](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/prql-showcase.png?raw=true) 12 | 13 | PRQL tools include [PRQL VS Code](https://marketplace.visualstudio.com/items?itemName=PRQL-lang.prql-vscode) extension that adds PRQL language support, SQL Preview and other PRQL related settings and features to work with it in VS Code IDE. 14 | 15 | ![PRQL VS Code Extension](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/prql-vscode-extension.png?raw=true) 16 | 17 | ## SQL Tools 18 | 19 | [SQL Tools](https://marketplace.visualstudio.com/items?itemName=mtxr.sqltools) provide connections to many commonly used databases in VS Code. SQL Tools come with many built-in database drivers and an extensive list of community [drivers](https://marketplace.visualstudio.com/search?term=tag%3Asqltools-driver&target=VSCode&category=All%20categories&sortBy=Rating) users can use to connect to the database management system of their choosing, view database objects in SQL Tools Connection Explorer, and run SQL queries. 20 | 21 | ![SQL Tools Drivers](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/sql-tools-drivers.png?raw=true) 22 | 23 | Most of the SQL Tools database drivers use the target database management system browser JavaScript API to connect to the local database server or remote database instances via SQL Tools Connection Assistant user interface: 24 | 25 | ![SQL Tools Connection Assistant](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/sqltools-connection-assistant.png?raw=true) 26 | 27 | Some of the SQL Tools extensions, such as [SQLite](https://vscode-sqltools.mteixeira.dev/en/drivers/sq-lite/) and [DuckDB Pro Tools](https://github.com/RandomFractals/pro-data-tools/blob/main/duckdb-tools.md#duckdb-pro-tools) require [NodeJS](https://nodejs.org/en/download/) installation and `sqltools.useNodeRuntime` setting enabled for tighter inegration of the native database api and access to the local file system to load and view file-based database instances using NodeJS server for the database files and local data files hosting runtime. 28 | 29 | ![DuckDB Pro Tools Connection](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-sql-tools-connection.png?raw=true) 30 | 31 | ## PRQL Pro Tools Extension Pack 32 | 33 | **PRQL Pro Tools** is a private beta collection of custom PRQL VS Code extensions for our monthly [**Pro**](https://github.com/sponsors/RandomFractals/sponsorships?tier_id=18884) supporters on github, and can be viewed as an extension pack to enhance PRQL development and runtime workflow connected to the different database management system via SQL Tools extension and database plugins. 34 | 35 | ## PRQL Code Lens 36 | 37 | The initial private beta release of **PRQL Pro Tools** comes with **PRQL Code Lens** extension that allows you to run PRQL queries for the supported PRQL target [SQL dialects](https://prql-lang.org/book/project/target.html#dialects) **directly** against any of the supported SQL Tools extensions and database management systems. 38 | 39 | ![PRQL Code Lens](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/prql-code-lens.png?raw=true) 40 | 41 | ### Supported Databases 42 | 43 | You can explore [SQL Tools drivers](https://marketplace.visualstudio.com/search?term=tag%3Asqltools-driver&target=VSCode&category=All%20categories&sortBy=Rating) in VS Code Marketplace and install the corresponding database drivers depending on target database management system you intend to use with **PRQL Code Lens** and [SQL Tools](https://marketplace.visualstudio.com/items?itemName=mtxr.sqltools). 44 | 45 | We recommend you install [SQLite](https://vscode-sqltools.mteixeira.dev/en/drivers/sq-lite/) or [DuckDB Pro Tools](https://github.com/RandomFractals/pro-data-tools/blob/main/duckdb-tools.md#duckdb-pro-tools) to get started. 46 | 47 | ### Configuration 48 | 49 | **PRQL Code Lens** uses the corresponding [PRQL Settings](https://github.com/prql/prql-vscode#prql-settings) and `prql.target` for the target database management system when creating SQL query to run at runtime. 50 | 51 | See [SQL Tools Settings](https://vscode-sqltools.mteixeira.dev/en/settings/) documentation and specific SQL Tools driver settings to manage SQL query results display and other database specific settings. For example, [DuckDB Pro Tools](https://github.com/RandomFractals/pro-data-tools/blob/main/duckdb-tools.md#duckdb-pro-tools) list some of them in the [Configuration](https://github.com/RandomFractals/pro-data-tools/blob/main/duckdb-tools.md#configuration) section. 52 | 53 | ### Demo Data 54 | 55 | **DuckDB SQL Tools** repository has [sample `/data`](https://github.com/RandomFractals/duckdb-sql-tools/tree/main/data) folder you can use to download SQLite and DuckDB demo `chinook` database files and sample PRQL queries you can run. 56 | 57 | ![Chinook DB Data and Queries](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/duckdb-sql-tools-data.png?raw=true) 58 | 59 | **PRQL Pro Tools** repository also has `/data` folder with sample `employees` database data files, schema and load SQL scripts, `create-employees-duckdb.ipynb` Jupyter notebook, and some PRQL query samples you can download, load and run with **PRQL Code Lens**. More `chinook` and `employees` database PRQL sample queries will be added later. 60 | 61 | ## PRQL Query Examples 62 | 63 | Base [PRQL VScode](https://github.com/PRQL/prql-vscode) extension has a few new PRQL [examples](https://github.com/PRQL/prql-vscode/tree/main/examples) that demo new [PRQL compiler v0.6](https://github.com/PRQL/prql/releases) features such as `loop`, `case`, `func` and using variables with `let`. 64 | 65 | We also recently added a couple of [GBIF](https://github.com/gbif/occurrence/blob/master/aws-public-data.md) dataset [PRQL query examples](https://github.com/RandomFractals/duckdb-sql-tools/tree/main/data/gbif) to demo loading and querying remote public datasets hosted by AWS with **PRQL Code Lens** extension and [DuckDB SQL Tools](https://github.com/RandomFractals/duckdb-sql-tools): 66 | 67 | ![GBIF PRQL DuckDB Pro Tools Examples](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/gbif-top-10-duck-species-observations-prql.gif?raw=true) 68 | 69 | ## PRQL Notebook 70 | 71 | Future versions of **PRQL Pro Tools** will include custom **PRQL Notebook** extension integrated with SQL Tools extension, supported database management systems, our [Data Table Renderers](https://marketplace.visualstudio.com/items?itemName=RandomFractalsInc.vscode-data-table) and VS Code Notebooks user interface to view and run PRQL queries from `.prql` file using Notebook UI: 72 | 73 | ![PRQL Notebook](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/prql-notebook.png?raw=true) 74 | 75 | 76 | ## PRQL Terminal 77 | 78 | We also plan to release a custom version of **PRQL Terminal** extension as part of the **PRQL Pro Tools** extension pack for our monthly [**Pro**](https://github.com/sponsors/RandomFractals/sponsorships?sponsor=RandomFractals&tier_id=18884) supporters later this year. 79 | 80 | **PRQL Terminal** will have a walk-through to install `prqlc` CLI tools and will integrate it with the built-in VS Code File Manager and Terminal panel to run PRQL queries from the command line with VS Code commands, shortcuts, and `.prql` file context menus for the CLI fans. 81 | 82 | ![PRQL Terminal](https://github.com/RandomFractals/duckdb-sql-tools/blob/main/docs/images/prql-terminal.png?raw=true) 83 | 84 | ## Installation and Support 85 | 86 | You can get access to **PRQL Pro Tools** by becoming a [**Pro** sponsor](https://github.com/sponsors/RandomFractals/sponsorships?tier_id=18884) of our **Pro Data Tools** work on GitHub. 87 | 88 | All Pro sponsors get invited as collaborators to our private [Pro Data Tools](https://github.com/RandomFractals/pro-data-tools/tree/main#pro-data-tools) repositories, and will be notified about the upcoming updates, new releases, additions, and new [Data Notebook Pro](https://github.com/RandomFractals/pro-data-tools/tree/main#data-notebook-pro-tools) extension we'll be releasing later this year. 89 | 90 | Help us reach our goal of getting 20 Pro monthly sponsors on GitHub. 🙏 91 | 92 | 💖 https://github.com/sponsors/RandomFractals 93 | -------------------------------------------------------------------------------- /sqlite-tools.md: -------------------------------------------------------------------------------- 1 | # SQLite Pro Tools 2 | 3 | **SQLite Pro Tools** extension adds advanced [SQLite](https://www.sqlite.org/) database connection features and support to [VS Code](https://code.visualstudio.com/) IDE. 4 | 5 | ![SQLite Pro Tools Views](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/sqlite-tools.png?raw=true) 6 | 7 | # Features 8 | 9 | [SQLite](https://www.sqlite.org/about.html) is an in-process, self-contained, serverless, zero-configuration, transactional SQL database engine. SQLite is an embedded SQL database engine. Unlike most other SQL databases, SQLite does not have a separate server process. SQLite reads and writes directly to ordinary disk files. A complete SQL database with multiple tables, indices, triggers, and views, is contained in a single disk file. 10 | 11 | [SQLite Pro Tools](https://github.com/RandomFractals/sqlite-pro-tools) extension adds the following capabilities to VS Code IDE for working with SQLite database files: 12 | 13 | - **Connect** to a local [SQLite](https://www.sqlite.org/index.html) database instance 14 | - **Create** new in-memory SQLite database instance 15 | - **View** SQLite databases, tables, views, columns, indexes, modules, settings, and functions in SQL Tools Connections Explorer 16 | - **Run** SQL queries on active SQLite database connections 17 | - **Manage** SQLite database connections in SQL Tools Connections Explorer 18 | - **Auto-complete** SQL keywords, SQLite instance table names, column names, and view names for active SQLite database connections in VS Code SQL editor 19 | - **Save** named SQL query Bookmarks 20 | - **Use** SQL Query History 21 | - **Export** SQLite database query results in `CSV` and `JSON` data formats 22 | - **Use** `SQLite Tools` views and metadata shortcut commands from VS Code `Command Palette...` 23 | 24 | See [SQL Tools documentation](https://vscode-sqltools.mteixeira.dev/en/home/) for a comprehensive list of SQL Tools extension features contributed to VS Code IDE. 25 | 26 | # Installation 27 | 28 | Become a [Pro Sponsor](https://github.com/sponsors/RandomFractals/sponsorships?tier_id=18884) of our new **Pro Data Tools** work on GitHub to access **SQLite Pro Tools**, [DuckDB Pro Tools](https://github.com/RandomFractals/pro-data-tools/tree/main#duckdb-pro-tools), [PRQL Pro Tools](https://randomfractals.github.io/pro-data-tools/#prql-pro-tools) and our new [Markdown SQL Pro Tools](https://randomfractals.github.io/pro-data-tools/#markdown-sql-pro-tools) releases and extension `.vsix` installation packages. 29 | 30 | All of our GitHub **Pro** sponsors get invited as collaborators to our private [Pro Data Tools](https://randomfractals.github.io/pro-data-tools/#pro-data-tools) repositories, and will be notified about the upcoming updates, new releases, additions, and new [Data Notebook Pro Tools](https://randomfractals.github.io/pro-data-tools/#data-notebook-pro-tools) extension we'll be releasing later this year. 31 | 32 | To install **SQLite Pro Tools**, download the latest `sqlite-pro-tools-x.x.x.vsix` from the attached **Assets** in this repository [releases](https://github.com/RandomFractals/sqlite-pro-tools/releases). 33 | 34 | Use VS Code IDE [Install from VSIX](https://code.visualstudio.com/docs/editor/extension-marketplace#_install-from-a-vsix) feature to install downloaded **SQLite Pro Tools** extension package in VS Code or any other `VSIX`-compatible IDE. **SQLite Pro Tools** work in [VSCodium](https://vscodium.com/) too. 35 | 36 | ![SQLite Pro Tools VS Code Extension Info](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/sqlite-tools-extension.png?raw=true) 37 | 38 | **Note**: **SQLite Pro Tools** use [Node JS SQLite3 API](https://github.com/TryGhost/node-sqlite3) and require a local [Node.js installation](https://nodejs.org/en/download) to load and query SQLite database files. 39 | 40 | Download and install **Node.js** from the official [node.js downloads](https://nodejs.org/en/download) page. 41 | 42 | ## Table and View Columns 43 | 44 | **SQLite Pro Tools** display basic data type icons for text, numeric, and date table and view columns. 45 | 46 | ![SQLite Table and View Columns](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/sqlite-table-columns.png?raw=true) 47 | 48 | **SQLite Pro Tools** also display primary key column indicators and `not null` column value constraints. 49 | 50 | ### Indexes 51 | 52 | ![SQLite Indexes](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/sqlite-indexes.png?raw=true) 53 | 54 | ### Modules 55 | 56 | ![SQLite Modules](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/sqlite-modules.png?raw=true) 57 | 58 | ## Configuration Settings 59 | 60 | SQLite configuration settings / [compile options](https://www.sqlite.org/compile.html): 61 | 62 | ![SQLite Settings](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/sqlite-settings.png?raw=true) 63 | 64 | ### Functions 65 | 66 | ![SQLite Functions](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/sqlite-functions.png?raw=true) 67 | 68 | # Commands 69 | 70 | **SQLite Pro Tools** provide many standard SQLite database and metadata shortcut commands you can access via `ctrl/cmd+shift+p` shortcut in VS Code or from `View -> Commnad Palette...` menu by typing `SQLite Tools` in the `>` command prompt: 71 | 72 | ![SQLite Tools Commands](https://github.com/RandomFractals/pro-data-tools/blob/main/docs/images/sqlite-commands.png?raw=true) 73 | 74 | # Demo Data 75 | 76 | **SQLite Pro Tools** extension documentation repository contains sample `/data` folder with the popular `chinook` database files and [`car-company`](https://github.com/dtaivpp/car_company_database) SQLite database files and scripts you can experiment with. 77 | 78 | # In-Memory SQLite DB 79 | 80 | **SQLite Pro Tools** extension lets you create in-memory database instances by specifying `:memory:` keyword in the Database File field of the new SQLite connection in SQL Tools Connection Assistant. Consult [In-Memory SQLite Databases](https://www.sqlite.org/inmemorydb.html) documentation for more info. 81 | 82 | # Configuration 83 | 84 | [SQL Tools](https://marketplace.visualstudio.com/items?itemName=mtxr.sqltools) extension provides many configuration [Settings](https://vscode-sqltools.mteixeira.dev/en/settings/) users can toggle to change database connection and tree view display options, sql formatting, and results display. 85 | 86 | The following [SQL Tools Settings](https://vscode-sqltools.mteixeira.dev/en/settings/) were used while creating and testing this **SQLite Pro Tools** extension and are recommended for working with DuckDB instances efficiently. We suggest you set these preferences in User Settings in VS Code by navigating to `File -> Preferences -> Settings -> User -> Extensions -> SQLTools`, or adding them to your global VS Code `settings.json` config file using the JSON code snippet below: 87 | 88 | ```json 89 | { 90 | ... 91 | "sqltools.useNodeRuntime": true, 92 | "sqltools.disableNodeDetectNotifications": true, 93 | "sqltools.autoOpenSessionFiles": false, 94 | "sqltools.results.limit": 10000, 95 | "sqltools.results.location": "current", 96 | } 97 | ``` 98 | 99 | | Setting | Description | 100 | | --- | --- | 101 | | ```"sqltools.useNodeRuntime": true``` | Enable Node runtime in order to use [SQLite3 NodeJS API](https://github.com/TryGhost/node-sqlite3) this database SQL tools extension depends on. | 102 | | ```"sqltools.disableNodeDetectNotifications": true``` | Disable Node runtime detection notifications after initial SQLTools extension installation to prevent Node runtime information message display on every new VS Code session start. | 103 | | ```"sqltools.autoOpenSessionFiles": false``` | Prevent auto open of new session SQL editor instance after connecting to the databaase instance. | 104 | | ```"sqltools.results.limit": 10000``` | Maximum number of records to return in results. SQL Tools defaults to displaying only 50 records in query results view. Changing this limit setting to 1000 or 10000 will show more data rows to inspect in result views. | 105 | | ```"sqltools.results.location": "current"``` | Defines the editor group to use for result table views. SQL Tools displays all results in the next editor group to show results on the side next to the active SQL query editor. Changing this setting to `current` will display results in the same editor group and display more result columns. | 106 | 107 | # Support 108 | 109 | Become a [Fan or a Pro Sponsor](https://github.com/sponsors/RandomFractals) on GitHub to support our dev work on **SQLite Pro Tools**, new [**Pro Data Tools**](https://github.com/RandomFractals/pro-data-tools/tree/main#pro-data-tools) and other free and public [Random Fractals, Inc. code and data viz extensions](https://marketplace.visualstudio.com/publishers/RandomFractalsInc). 110 | 111 | Help us reach our goal of getting at least 10 **Pro** sponsors on GitHub. 🙏 112 | 113 | 💖 https://github.com/sponsors/RandomFractals 114 | --------------------------------------------------------------------------------