├── CHANGELOG.md ├── mkdocs.yml ├── oracle ├── README.md ├── oracle_historical_ash_info.sql └── oracle_current_ash_info.sql ├── requirements.txt ├── mssql ├── README.md ├── mssql_detect_tde_db_encryption.sql └── mssql_memory_usage.sql ├── requirements-docs.txt ├── site ├── img │ ├── grid.png │ └── favicon.ico ├── webfonts │ ├── fa-brands-400.ttf │ ├── fa-regular-400.ttf │ ├── fa-solid-900.ttf │ ├── fa-solid-900.woff2 │ ├── fa-brands-400.woff2 │ ├── fa-regular-400.woff2 │ ├── fa-v4compatibility.ttf │ └── fa-v4compatibility.woff2 ├── sitemap.xml ├── css │ ├── solid.min.css │ └── v4-font-face.min.css ├── js │ ├── darkmode.js │ └── base.js ├── search │ ├── main.js │ └── worker.js ├── 404.html └── developer_guide │ ├── releases │ └── index.html │ ├── developer_setup │ └── index.html │ └── commands │ └── index.html ├── .bumpversion.cfg ├── docs ├── developer_guide │ ├── developer_setup.md │ ├── releases.md │ ├── workflows.md │ └── commands.md ├── .gitkeep ├── index.md └── code_of_conduct.md ├── .gitignore ├── README.md ├── postgres ├── postgres_non_default_params.sql ├── postgres_queries_running_in_parallel.sql ├── postgres_top_10_table_seq_scans.sql ├── postgres_active_parallel_workers.sql ├── postgres_table_cache_hit_rate.sql ├── postgres_average_row_size.sql ├── postgres_table_index_used_pct.sql ├── postgres_top_10_long_running_queries.sql ├── postgres_vacuum_progress_basic.sql ├── postgres_determine_oldest_xmin_age.sql ├── postgres_index_cache_ratio.sql ├── postgres_transaction_age_by_table.sql ├── postgres_database_user_settings.sql ├── postgres_indexes_supporting_pk_constraints.sql ├── postgres_waits_by_individual_pid.sql ├── postgres_monitor_waits.sql ├── postgres_list_pg_buffercache_for_all_objects.sql ├── postgres_list_pg_buffercache_blocks_for_object.sql ├── postgres_blocking_lock_wait_events.sql ├── postgres_active_connections.sql ├── postgres_top_10_objects_using_pg_buffercache.sql ├── postgres_index_object_size.sql ├── postgres_ranked_waits.sql ├── postgres_determine_oldest_xmin_by_pid.sql ├── postgres_idle_connection_times.sql ├── postgres_wal_generation_over_known_time.sql ├── postgres_table_reloptions.sql ├── postgres_check_table_reloptions.sql ├── postgres_index_information_details.sql ├── postgres_duplicate_indexes.sql ├── postgres_table_constraint_info.sql ├── postgres_table_object_size.sql ├── postgres_index_usage_stats.sql ├── postgres_vacuum_progress_detailed.sql ├── postgres_top_10_sql_by_execution.sql ├── postgres_blocking_lock_sql.sql ├── postgres_top_10_sql_by_total_exec_time.sql ├── postgres_top_10_sql_by_mean_exec_time.sql ├── postgres_google_db_advisor_example.sql ├── postgres_table_cache_hit_vs_disk_hit_ratio.sql ├── postgres_unindexed_foreign_keys.sql ├── postgres_index_build_status.sql ├── postgres_blocking_lock_pid_tree.sql ├── postgres_report_high_water_mark_wasted_space.sql ├── postgres_find_unused_indexes.sql ├── postgres_table_stats_status.sql ├── postgres_object_sizes.sql ├── postgres_index_bloat_status.sql ├── postgres_table_and_index_bloat_stats.sql ├── postgres_table_bloat_check.sql ├── README.md └── postgres_vacuum_activity_estimate_stats.sql ├── .github └── workflows │ ├── ci.yaml │ └── release.yaml ├── CODE_OF_CONDUCT.md └── Makefile /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | * Initial Release -------------------------------------------------------------------------------- /mkdocs.yml: -------------------------------------------------------------------------------- 1 | site_name: My Docs 2 | -------------------------------------------------------------------------------- /oracle/README.md: -------------------------------------------------------------------------------- 1 | # Oracle SQL Scripts - Shorthand Commands for sqlplus 2 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | bump2version 2 | ruff 3 | sqlfluff 4 | pre-commit 5 | m2r -------------------------------------------------------------------------------- /mssql/README.md: -------------------------------------------------------------------------------- 1 | # SQL Server SQL Scripts - Shorthand Commands for sqlcmd 2 | 3 | -------------------------------------------------------------------------------- /requirements-docs.txt: -------------------------------------------------------------------------------- 1 | mkdocs 2 | mkdocs-material 3 | mkdocstrings 4 | mkdocstrings-python -------------------------------------------------------------------------------- /site/img/grid.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shane-borden/sqlScripts/HEAD/site/img/grid.png -------------------------------------------------------------------------------- /site/img/favicon.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shane-borden/sqlScripts/HEAD/site/img/favicon.ico -------------------------------------------------------------------------------- /site/webfonts/fa-brands-400.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shane-borden/sqlScripts/HEAD/site/webfonts/fa-brands-400.ttf -------------------------------------------------------------------------------- /site/webfonts/fa-regular-400.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shane-borden/sqlScripts/HEAD/site/webfonts/fa-regular-400.ttf -------------------------------------------------------------------------------- /site/webfonts/fa-solid-900.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shane-borden/sqlScripts/HEAD/site/webfonts/fa-solid-900.ttf -------------------------------------------------------------------------------- /site/webfonts/fa-solid-900.woff2: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shane-borden/sqlScripts/HEAD/site/webfonts/fa-solid-900.woff2 -------------------------------------------------------------------------------- /site/webfonts/fa-brands-400.woff2: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shane-borden/sqlScripts/HEAD/site/webfonts/fa-brands-400.woff2 -------------------------------------------------------------------------------- /site/webfonts/fa-regular-400.woff2: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shane-borden/sqlScripts/HEAD/site/webfonts/fa-regular-400.woff2 -------------------------------------------------------------------------------- /site/sitemap.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | -------------------------------------------------------------------------------- /site/webfonts/fa-v4compatibility.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shane-borden/sqlScripts/HEAD/site/webfonts/fa-v4compatibility.ttf -------------------------------------------------------------------------------- /site/webfonts/fa-v4compatibility.woff2: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shane-borden/sqlScripts/HEAD/site/webfonts/fa-v4compatibility.woff2 -------------------------------------------------------------------------------- /.bumpversion.cfg: -------------------------------------------------------------------------------- 1 | [bumpversion] 2 | current_version = 0.1.2 3 | commit = False 4 | tag = False 5 | 6 | [bumpversion:file:README.md] 7 | search = Version {current_version} 8 | replace = Version {new_version} 9 | -------------------------------------------------------------------------------- /docs/developer_guide/developer_setup.md: -------------------------------------------------------------------------------- 1 | # Developer Setup 2 | 3 | To begin local development, clone the [shane-borden/sqlScripts](https://github.com/shane-borden/sqlScripts) repository and use one of the following methods to build it. All commands should be executed from inside of the project home folder. 4 | 5 | ## Configure environment 6 | 7 | ```bash 8 | make install 9 | ``` 10 | -------------------------------------------------------------------------------- /docs/developer_guide/releases.md: -------------------------------------------------------------------------------- 1 | # Releases 2 | 3 | A release should consist of the following two steps from a tested, linted, and up to date copy of the _main_ branch: 4 | 5 | 1. `make pre-release increment={major/minor/patch}` - Commit the version number bump and create a new tag locally. The version number follows semantic versioning standards (major.minor.patch) and the tag is the version number prepended with a 'v'. 6 | 7 | 2. `git push --follow-tags` - Update the _main_ branch with only the changes from the version bump. Publish the new tag and kick off the release workflow. 8 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | 2 | # C extensions 3 | *.so 4 | 5 | # Django stuff: 6 | *.log 7 | local_settings.py 8 | db.sqlite3 9 | db.sqlite3-journal 10 | 11 | # Distribution / packaging 12 | dist/ 13 | 14 | # Environments 15 | .env* 16 | !.env.example 17 | .venv 18 | env/ 19 | venv/ 20 | ENV/ 21 | env.bak/ 22 | venv.bak/ 23 | .venv 24 | service_account.json 25 | 26 | # osx 27 | .ds_store 28 | .DS_Store 29 | 30 | # vscode 31 | # .vscode 32 | 33 | # App specifics 34 | .env 35 | README.txt 36 | mssql/README.txt 37 | oracle/README.txt 38 | postgres/README.txt 39 | site/index.html 40 | site/sitemap.xml.gz -------------------------------------------------------------------------------- /site/css/solid.min.css: -------------------------------------------------------------------------------- 1 | /*! 2 | * Font Awesome Free 6.5.1 by @fontawesome - https://fontawesome.com 3 | * License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) 4 | * Copyright 2023 Fonticons, Inc. 5 | */ 6 | :host,:root{--fa-style-family-classic:"Font Awesome 6 Free";--fa-font-solid:normal 900 1em/1 "Font Awesome 6 Free"}@font-face{font-family:"Font Awesome 6 Free";font-style:normal;font-weight:900;font-display:block;src:url(../webfonts/fa-solid-900.woff2) format("woff2"),url(../webfonts/fa-solid-900.ttf) format("truetype")}.fa-solid,.fas{font-weight:900} -------------------------------------------------------------------------------- /docs/developer_guide/workflows.md: -------------------------------------------------------------------------------- 1 | # Workflows 2 | 3 | ## Release 4 | 5 | - Linting and testing steps must pass before the release steps can begin. 6 | - Documentation is automatically published to the `gh-pages` branch and hosted on github pages. 7 | - All github release tags, docker image tags, and PyPI version numbers are in agreement with one another and follow semantic versioning standards. 8 | - Builds collections packages and attaches to release 9 | 10 | ## Build and Publish Docs 11 | 12 | - Build the documentation, publish to the `gh-pages` branch, and release to github pages. 13 | - Runs only on a manual trigger in the github actions tab. 14 | -------------------------------------------------------------------------------- /docs/.gitkeep: -------------------------------------------------------------------------------- 1 | // Copyright 2024 shaneborden 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // https://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # DBMS SQL Scripts for Postgres / Oracle / SQLServer 2 | by Shane Borden 3 | 4 | Version 0.1.2 5 | 6 | These scripts have been collected and modified to help diagnose and expose metadata about various DBMS Products 7 | 8 | This SQL Script Toolkit is a "free to use" script (Covered by the Apache 2.0 License) 9 | that collects diagnostic Information from the target DBMS System. 10 | 11 | Some of the scripts contained in this collection have been modified from other scripts available in the public domain. Every effort has been made to credit original authors where practical. 12 | 13 | ## Steps 14 | 15 | 1. Connect to your target dbms 16 | 2. Execute the SQL File against the target and use the resultant information as necessary! -------------------------------------------------------------------------------- /postgres/postgres_non_default_params.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT name, source, setting 16 | FROM pg_settings 17 | WHERE source != 'default' 18 | AND source != 'override' 19 | ORDER by 2, 1; -------------------------------------------------------------------------------- /docs/developer_guide/commands.md: -------------------------------------------------------------------------------- 1 | # Commands 2 | 3 | - `make install` - Creates a new virtual environment for development. 4 | 5 | - `make clean` - Remove all build, testing, and static documentation files. 6 | 7 | - `make build` - Build a folder containing a set of the latest database collection scripts. 8 | 9 | - `make gen-docs` - Generate HTML documentation. 10 | 11 | - `make docs` - Generate HTML documentation and serve it to the browser. 12 | 13 | - `make pre-release increment={major/minor/patch}` - Bump the version and create a release tag. Should only be run from the _main_ branch. Passes the increment value to bump2version to create a new version number dynamically. The new version number will be added to _\_\_version\_\_.py_ and _pyproject.toml_ and a new commit will be logged. The tag will be created from the new commit. 14 | -------------------------------------------------------------------------------- /mssql/mssql_detect_tde_db_encryption.sql: -------------------------------------------------------------------------------- 1 | SELECT 2 | d.name AS DatabaseName, 3 | d.is_encrypted, 4 | dek.encryption_state, 5 | CASE dek.encryption_state 6 | WHEN 0 THEN 'No Database Encryption Key Present, Not Encrypted' 7 | WHEN 1 THEN 'Unencrypted' 8 | WHEN 2 THEN 'Encryption in Progress' 9 | WHEN 3 THEN 'Encrypted' 10 | WHEN 4 THEN 'Key Change in Progress' 11 | WHEN 5 THEN 'Decryption in Progress' 12 | WHEN 6 THEN 'Protection Change in Progress' 13 | ELSE 'Not Encrypted or No Key' -- Or NULL if no entry in dek 14 | END AS EncryptionStatusDescription, 15 | dek.percent_complete, 16 | dek.key_algorithm, 17 | dek.key_length 18 | FROM 19 | sys.databases d 20 | LEFT JOIN 21 | sys.dm_database_encryption_keys dek ON d.database_id = dek.database_id 22 | WHERE d.is_encrypted <> 0; -------------------------------------------------------------------------------- /postgres/postgres_queries_running_in_parallel.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT query, leader_pid, 16 | array_agg(pid) filter(WHERE leader_pid != pid) AS members 17 | FROM pg_stat_activity 18 | WHERE leader_pid IS NOT NULL 19 | GROUP BY query, leader_pid; -------------------------------------------------------------------------------- /docs/index.md: -------------------------------------------------------------------------------- 1 | # Database Performance Scripts Collection 2 | 3 | ## Central collection of sql scripts that can help diagnose problems and expose metadata about various DBMS Products. 4 | 5 | * Collect performance and other metadata for your current PostgreSQL, Oracle and SQL Server environments. 6 | 7 | ## Quick Start 8 | 9 | Download the collection scripts from the latest release. Instructions for execution are included in the bundled README. 10 | * [Oracle](https://github.com/shane-borden/sqlScripts/releases/latest/download/sql-scripts-oracle.zip) 11 | * [MSSQL](https://github.com/shane-borden/sqlScripts/releases/latest/download/sql-scripts-sqlserver.zip) 12 | * [Postgres](https://github.com/shane-borden/sqlScripts/releases/latest/download/sql-scripts-postgres.zip) 13 | 14 | Instructions for execution are included in the README bundled with the collection scripts. 15 | -------------------------------------------------------------------------------- /postgres/postgres_top_10_table_seq_scans.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT 16 | relid, 17 | relname, 18 | seq_scan, 19 | pg_size_pretty(pg_relation_size(relid)) 20 | FROM 21 | pg_stat_user_tables 22 | ORDER BY 23 | seq_scan DESC 24 | LIMIT 10; 25 | -------------------------------------------------------------------------------- /postgres/postgres_active_parallel_workers.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT 16 | current_setting('max_parallel_workers')::integer AS max_workers, 17 | count(*) AS active_workers 18 | FROM 19 | pg_stat_activity 20 | WHERE 21 | backend_type = 'parallel worker'; -------------------------------------------------------------------------------- /postgres/postgres_table_cache_hit_rate.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT 16 | sum(heap_blks_read) AS heap_read, 17 | sum(heap_blks_hit) AS heap_hit, 18 | (sum(heap_blks_hit) - sum(heap_blks_read)) / sum(heap_blks_hit) AS ratio 19 | FROM 20 | pg_statio_user_tables; -------------------------------------------------------------------------------- /postgres/postgres_average_row_size.sql: -------------------------------------------------------------------------------- 1 | WITH table_size AS ( 2 | SELECT 3 | c.relname AS table_name, 4 | pg_table_size(c.oid) AS main_table_size, 5 | pg_total_relation_size(c.oid) AS total_size, 6 | pg_total_relation_size(c.oid) - pg_table_size(c.oid) AS toast_size 7 | FROM pg_class c 8 | WHERE c.relname in ('packaged_events','media') 9 | AND c.relkind = 'r' 10 | ), 11 | row_count AS ( 12 | SELECT 13 | relname AS table_name, 14 | n_live_tup AS row_count 15 | FROM pg_stat_user_tables 16 | WHERE relname in ('packaged_events','media') 17 | ) 18 | SELECT 19 | ts.table_name, 20 | ts.main_table_size, 21 | ts.toast_size, 22 | ts.total_size, 23 | COALESCE(rc.row_count, 1) AS row_count, 24 | (ts.total_size / NULLIF(rc.row_count, 0)) AS avg_row_size 25 | FROM table_size ts 26 | LEFT JOIN row_count rc ON ts.table_name = rc.table_name; 27 | -------------------------------------------------------------------------------- /.github/workflows/ci.yaml: -------------------------------------------------------------------------------- 1 | # Copyright 2024 shaneborden 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # https://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | name: ci 16 | on: 17 | pull_request: 18 | push: 19 | branches: 20 | - main 21 | jobs: 22 | build: 23 | runs-on: ubuntu-latest 24 | 25 | steps: 26 | - uses: actions/checkout@v4 27 | - name: Building Collection scripts 28 | run: make build -------------------------------------------------------------------------------- /postgres/postgres_table_index_used_pct.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT 16 | relname, 17 | 100 * idx_scan / (seq_scan + idx_scan) percent_of_times_index_used, 18 | n_live_tup rows_in_table 19 | FROM 20 | pg_stat_user_tables 21 | WHERE (seq_scan + idx_scan) > 0 22 | ORDER BY 23 | n_live_tup DESC; -------------------------------------------------------------------------------- /postgres/postgres_top_10_long_running_queries.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT pid, age(backend_xid) AS age_in_xids, 16 | now () - xact_start AS xact_age, 17 | now () - query_start AS query_age, 18 | state, 19 | query 20 | FROM pg_stat_activity 21 | WHERE state != 'idle' 22 | ORDER BY 2 DESC 23 | LIMIT 10; -------------------------------------------------------------------------------- /postgres/postgres_vacuum_progress_basic.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | /* Basic Vacuum Progress */ 16 | SELECT 17 | pid, 18 | datname, 19 | relid::regclass relname, 20 | phase, 21 | heap_blks_total, 22 | heap_blks_scanned, 23 | heap_blks_vacuumed, 24 | index_vacuum_count, 25 | max_dead_tuples, 26 | num_dead_tuples 27 | FROM 28 | pg_stat_progress_vacuum; 29 | -------------------------------------------------------------------------------- /postgres/postgres_determine_oldest_xmin_age.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT 16 | (SELECT max(age(backend_xmin)) FROM pg_stat_activity) as oldest_running_xact, 17 | (SELECT max(age(transaction)) FROM pg_prepared_xacts) as oldest_prepared_xact, 18 | (SELECT max(age(xmin)) FROM pg_replication_slots) as oldest_replication_slot, 19 | (SELECT max(age(backend_xmin)) FROM pg_stat_replication) as oldest_replica_xact 20 | ; -------------------------------------------------------------------------------- /postgres/postgres_index_cache_ratio.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT 16 | relname, 17 | indexrelname, 18 | sum(idx_blks_read) as idx_read, 19 | sum(idx_blks_hit) as idx_hit, 20 | ROUND((sum(idx_blks_hit) - sum(idx_blks_read)) / sum(idx_blks_hit),4) as ratio 21 | FROM pg_statio_user_indexes 22 | WHERE (idx_blks_read > 0 and idx_blks_hit > 0) AND relname like '%' AND indexrelname like '%' 23 | GROUP BY 24 | relname, 25 | indexrelname; -------------------------------------------------------------------------------- /postgres/postgres_transaction_age_by_table.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT 16 | c.oid::regclass AS table_name, 17 | greatest (age(c.relfrozenxid), age(t.relfrozenxid)) AS "TXID age", 18 | (greatest (age(c.relfrozenxid), age(t.relfrozenxid))::numeric / 1000000000 * 100)::numeric(4, 2) AS "% WRAPAROUND RISK" 19 | FROM 20 | pg_class c 21 | LEFT JOIN pg_class t ON c.reltoastrelid = t.oid 22 | WHERE 23 | c.relkind IN ('r', 'm') 24 | ORDER BY 25 | 2 DESC; -------------------------------------------------------------------------------- /postgres/postgres_database_user_settings.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2025 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | /* find options like search_path assigned to users or databases */ 16 | SELECT coalesce(role.rolname, 'database wide') as role, 17 | coalesce(db.datname, 'cluster wide') as database, 18 | setconfig as what_changed 19 | FROM pg_db_role_setting role_setting 20 | LEFT JOIN pg_roles role ON role.oid = role_setting.setrole 21 | LEFT JOIN pg_database db ON db.oid = role_setting.setdatabase; 22 | 23 | /* or from psql */ 24 | \drds -------------------------------------------------------------------------------- /mssql/mssql_memory_usage.sql: -------------------------------------------------------------------------------- 1 | /* Copyright 2024 shaneborden 2 | // 3 | // Licensed under the Apache License, Version 2.0 (the "License"); 4 | // you may not use this file except in compliance with the License. 5 | // You may obtain a copy of the License at 6 | // 7 | // https://www.apache.org/licenses/LICENSE-2.0 8 | // 9 | // Unless required by applicable law or agreed to in writing, software 10 | // distributed under the License is distributed on an "AS IS" BASIS, 11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | // See the License for the specific language governing permissions and 13 | // limitations under the License. 14 | 15 | https://learn.microsoft.com/en-us/sql/relational-databases/performance-monitor/monitor-memory-usage?view=sql-server-ver16 16 | 17 | */ 18 | 19 | SELECT 20 | (physical_memory_in_use_kb/1024) AS Memory_used_by_Sqlserver_MB, 21 | (locked_page_allocations_kb/1024) AS Locked_pages_used_by_Sqlserver_MB, 22 | (total_virtual_address_space_kb/1024) AS Total_VAS_in_MB, 23 | process_physical_memory_low, 24 | process_virtual_memory_low 25 | FROM sys.dm_os_process_memory; -------------------------------------------------------------------------------- /postgres/postgres_indexes_supporting_pk_constraints.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | \prompt 'Enter schema_name: ' vSchemaName 16 | SELECT 17 | n.nspname schema_name, 18 | c.conname constraint_name, 19 | c.contype constraint_type, 20 | i.relname index_name, 21 | t.relname table_name 22 | FROM 23 | pg_constraint c 24 | JOIN pg_namespace n ON (c.connamespace = n.oid 25 | AND n.nspname = :'vSchemaName') 26 | JOIN pg_class i ON (c.conindid = i.oid) 27 | JOIN pg_class t ON (c.conrelid = t.oid) 28 | WHERE 29 | c.contype = 'p'; 30 | -------------------------------------------------------------------------------- /postgres/postgres_waits_by_individual_pid.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | \prompt 'Enter PID to check: ' pidvar 16 | SELECT 17 | pid, 18 | usename, 19 | application_name, 20 | TO_CHAR(backend_start, 'YYYY-MM-DD HH24:MI') AS backend_start, 21 | TO_CHAR(xact_start, 'YYYY-MM-DD HH24:MI:SS') AS xact_start, 22 | TO_CHAR(query_start, 'YYYY-MM-DD HH24:MI:SS') AS query_start, 23 | TO_CHAR(state_change, 'YYYY-MM-DD HH24:MI') AS state_change, 24 | wait_event_type, 25 | wait_event, 26 | state, 27 | query_id, 28 | query 29 | FROM 30 | pg_stat_activity 31 | WHERE 32 | pid = :'pidvar'; 33 | 34 | \watch 1 -------------------------------------------------------------------------------- /postgres/postgres_monitor_waits.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | \prompt 'Enter PID to check: ' pidvar 16 | select 17 | pid, 18 | usename, 19 | application_name, 20 | TO_CHAR( 21 | backend_start, 'YYYY-MM-DD HH24:MI') as backend_start, 22 | TO_CHAR( 23 | xact_start, 'YYYY-MM-DD HH24:MI:SS') as xact_start, 24 | TO_CHAR( 25 | query_start, 'YYYY-MM-DD HH24:MI:SS') as query_start, 26 | TO_CHAR( 27 | state_change, 'YYYY-MM-DD HH24:MI') as state_change, 28 | wait_event_type, 29 | wait_event, 30 | state, 31 | query_id, 32 | query 33 | from 34 | pg_stat_activity 35 | where 36 | pid = :'pidvar'; 37 | 38 | \watch 1 39 | -------------------------------------------------------------------------------- /postgres/postgres_list_pg_buffercache_for_all_objects.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT 16 | n.nspname, 17 | c.relname, 18 | count(*) AS buffers 19 | FROM 20 | pg_buffercache b 21 | JOIN pg_class c ON b.relfilenode = pg_relation_filenode(c.oid) 22 | AND b.reldatabase IN (0, ( 23 | SELECT 24 | oid 25 | FROM pg_database 26 | WHERE 27 | datname = current_database())) 28 | JOIN pg_namespace n ON n.oid = c.relnamespace 29 | WHERE 30 | n.nspname = 'public' 31 | GROUP BY 32 | n.nspname, 33 | c.relname 34 | ORDER BY 35 | 3 DESC; 36 | -------------------------------------------------------------------------------- /postgres/postgres_list_pg_buffercache_blocks_for_object.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | \prompt 'Enter relation name: ' vrelname 16 | 17 | SELECT 18 | n.nspname, 19 | c.relname, 20 | c.relfilenode, 21 | c.oid, 22 | b.relblocknumber 23 | FROM 24 | pg_buffercache b 25 | JOIN pg_class c ON b.relfilenode = pg_relation_filenode(c.oid) 26 | AND b.reldatabase IN (0, ( 27 | SELECT 28 | oid 29 | FROM pg_database 30 | WHERE 31 | datname = current_database())) 32 | JOIN pg_namespace n ON n.oid = c.relnamespace 33 | WHERE c.relname = :'vrelname' 34 | ORDER BY 35 | 5; 36 | -------------------------------------------------------------------------------- /postgres/postgres_blocking_lock_wait_events.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | select 16 | round(EXTRACT(EPOCH FROM (clock_timestamp() - query_start))::numeric, 5) as query_age , 17 | round(EXTRACT(EPOCH FROM (clock_timestamp() - xact_start))::numeric, 5) as xact_age, 18 | pid, 19 | pg_blocking_pids(PID), 20 | wait_event, 21 | substr(query , 1, 80) 22 | query , 23 | * 24 | --, pg_terminate_backend(PID) 25 | from pg_stat_activity a, 26 | (select unnest( string_to_array(replace(replace(pg_blocking_pids(PID)::text,'{',''),'}',''), ',')) as bpids, 27 | lower(query) 28 | from pg_stat_activity 29 | --where lower(query) like '%drop%' or lower(query) like '%alter%' 30 | ) b 31 | where 32 | PID = b.bpids::integer ; -------------------------------------------------------------------------------- /postgres/postgres_active_connections.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT 16 | pid, 17 | usename, 18 | state, 19 | client_addr, 20 | application_name, 21 | CURRENT_TIMESTAMP - state_change time_in_idle, 22 | rank() OVER (PARTITION BY client_addr ORDER BY backend_start DESC) AS rank 23 | FROM 24 | pg_stat_activity 25 | WHERE 26 | -- Exclude the thread owned connection (ie no auto-kill) 27 | pid <> pg_backend_pid() 28 | AND 29 | -- Exclude known applications connections 30 | application_name !~ '(?:pgAdmin.+)' 31 | AND 32 | -- Include connections to the same database the thread is connected to 33 | datname = current_database() 34 | AND 35 | -- Include inactive connections only 36 | state NOT IN ('idle', 'disabled'); -------------------------------------------------------------------------------- /postgres/postgres_top_10_objects_using_pg_buffercache.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT 16 | n.nspname, 17 | c.relname, 18 | c.relfilenode, 19 | c.oid, 20 | min(b.relblocknumber), 21 | max(b.relblocknumber), 22 | count(*) AS buffers 23 | FROM 24 | pg_buffercache b 25 | JOIN pg_class c ON b.relfilenode = pg_relation_filenode(c.oid) 26 | AND b.reldatabase IN (0, ( 27 | SELECT 28 | oid 29 | FROM pg_database 30 | WHERE 31 | datname = current_database())) 32 | JOIN pg_namespace n ON n.oid = c.relnamespace 33 | GROUP BY 34 | n.nspname, 35 | c.relname, 36 | c.relfilenode, 37 | c.oid 38 | ORDER BY 39 | 7 DESC 40 | LIMIT 10; 41 | -------------------------------------------------------------------------------- /postgres/postgres_index_object_size.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2025 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT 16 | n.nspname AS schemaname, 17 | pgi.tablename AS tablename, 18 | c.oid::regclass::text AS index_name, 19 | pi.inhparent::regclass::text AS top_index_name, 20 | pg_total_relation_size(c.oid) AS size, 21 | pg_size_pretty(pg_total_relation_size(c.oid)) AS pretty_size 22 | FROM 23 | pg_class c 24 | JOIN pg_namespace n ON c.relnamespace = n.oid 25 | JOIN pg_indexes pgi ON pgi.indexname = c.oid::regclass::text 26 | AND pgi.schemaname = n.nspname 27 | LEFT JOIN pg_inherits pi ON c.oid = pi.inhrelid 28 | WHERE 29 | c.relkind IN ('i') 30 | AND (n.nspname NOT IN ('pg_toast') 31 | AND n.nspname LIKE '%') 32 | AND (c.oid::regclass::text LIKE '%' 33 | AND pi.inhparent::regclass::text LIKE '%') 34 | ORDER BY 35 | 4 NULLS LAST; -------------------------------------------------------------------------------- /postgres/postgres_ranked_waits.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | /* Rank Wait Events */ 16 | with 17 | waits AS ( 18 | SELECT wait_event, 19 | rank() OVER (ORDER BY count(wait_event) DESC) rn 20 | FROM pg_stat_activity 21 | WHERE wait_event IS NOT NULL 22 | GROUP BY 23 | wait_event 24 | order by count(wait_event) asc 25 | ), 26 | total AS ( 27 | SELECT SUM(rn) total_waits FROM waits 28 | ) 29 | SELECT DISTINCT 30 | h.wait_event, 31 | h.rn, 32 | ROUND(100 * h.rn / t.total_waits, 1) percent 33 | FROM waits h, 34 | total t 35 | WHERE h.rn >= t.total_waits / 1000 AND rn <= 14 36 | UNION ALL 37 | SELECT 'Others', 38 | COALESCE(SUM(h.rn), 0) rn, 39 | COALESCE(ROUND(100 * SUM(h.rn) / AVG(t.total_waits), 1), 0) percent 40 | FROM waits h, 41 | total t 42 | WHERE h.rn < t.total_waits / 1000 OR rn > 14 43 | ORDER BY 2 DESC NULLS LAST; -------------------------------------------------------------------------------- /.github/workflows/release.yaml: -------------------------------------------------------------------------------- 1 | name: release 2 | on: 3 | workflow_dispatch: 4 | 5 | jobs: 6 | build_release: 7 | name: build_release 8 | runs-on: ubuntu-latest 9 | steps: 10 | - name: Checkout repo 11 | uses: actions/checkout@v4 12 | - name: Pull current code version 13 | run: | 14 | echo "release_version=Release $(grep -m 1 current_version .bumpversion.cfg | tr -s ' ' | tr -d '"' | tr -d "'" | cut -d' ' -f3)" >> $GITHUB_ENV 15 | id: version 16 | - name: Generate release tag 17 | run: | 18 | BUMPVERSION="$(grep -m 1 current_version .bumpversion.cfg | tr -s ' ' | tr -d '"' | tr -d "'" | cut -d' ' -f3)" 19 | echo "release_tag=v$BUMPVERSION" >> $GITHUB_ENV 20 | id: tag 21 | - name: Build SQL Scripts Assets 22 | run: make build 23 | - name: Create release 24 | id: create_release 25 | uses: softprops/action-gh-release@v2 26 | with: 27 | draft: false 28 | prerelease: true 29 | tag_name: ${{ env.RELEASE_TAG }} 30 | name: ${{ env.RELEASE_VERSION }} 31 | body_path: CHANGELOG.md 32 | files: | 33 | ./dist/sql-scripts-oracle.zip 34 | ./dist/sql-scripts-sqlserver.zip 35 | ./dist/sql-scripts-postgres.zip 36 | env: 37 | GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} 38 | RELEASE_TAG: ${{ env.release_tag }} 39 | RELEASE_VERSION: ${{ env.release_version }} -------------------------------------------------------------------------------- /postgres/postgres_determine_oldest_xmin_by_pid.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | /* Determine which pids map to the oldest xmin age */ 16 | SELECT 17 | pid, 18 | usename, 19 | datname, 20 | client_addr, 21 | state, 22 | now() - backend_start connection_total_time, 23 | now() - state_change time_in_state, 24 | now() - xact_start time_in_xact, 25 | now() - query_start time_in_query, 26 | wait_event, 27 | left (query, 40), 28 | max(age(backend_xmin)) 29 | FROM 30 | pg_stat_activity 31 | WHERE 32 | 1=1 33 | AND backend_xmin IS NOT NULL 34 | GROUP BY 35 | pid, 36 | usename, 37 | datname, 38 | client_addr, 39 | state, 40 | now() - backend_start, 41 | now() - state_change, 42 | now() - xact_start, 43 | now() - query_start, 44 | wait_event, 45 | left (query, 40) 46 | ORDER BY 47 | max(age(backend_xmin)) DESC; -------------------------------------------------------------------------------- /postgres/postgres_idle_connection_times.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT 16 | pid,usename,client_addr,current_timestamp - state_change time_in_idle, 17 | rank() over (partition by client_addr order by backend_start DESC) as rank 18 | FROM 19 | pg_stat_activity 20 | WHERE 21 | -- Exclude the thread owned connection (ie no auto-kill) 22 | pid <> pg_backend_pid( ) 23 | AND 24 | -- Exclude known applications connections 25 | application_name !~ '(?:psql)|(?:pgAdmin.+)' 26 | AND 27 | -- Include connections to the same database the thread is connected to 28 | datname = current_database() 29 | AND 30 | -- Include inactive connections only 31 | state in ('idle', 'idle in transaction', 'idle in transaction (aborted)', 'disabled') 32 | AND 33 | -- Include old connections (found with the state_change field) 34 | current_timestamp - state_change > interval '5 minutes'; -------------------------------------------------------------------------------- /postgres/postgres_wal_generation_over_known_time.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | \prompt 'Enter sleep time: ' vSleepTime 16 | 17 | SELECT 18 | *, 19 | pg_current_wal_lsn () 20 | FROM 21 | pg_stat_database 22 | WHERE 23 | datname = current_database() \gset 24 | 25 | select pg_sleep(:'vSleepTime'); 26 | 27 | SELECT 28 | blks_hit - :blks_hit "blk hit", 29 | blks_read - :blks_read "blk read", 30 | tup_inserted - :tup_inserted "ins", 31 | tup_updated - :tup_updated "upd", 32 | tup_deleted - :tup_deleted "del", 33 | tup_returned - :tup_returned "tup ret", 34 | tup_fetched - :tup_fetched "tup fch", 35 | xact_commit - :xact_commit "commit", 36 | xact_rollback - :xact_rollback "rbk", 37 | pg_size_pretty(pg_wal_lsn_diff (pg_current_wal_lsn (), :'pg_current_wal_lsn')) "WAL", 38 | pg_size_pretty(temp_bytes - :temp_bytes) "temp" 39 | FROM 40 | pg_stat_database 41 | WHERE 42 | datname = current_database(); 43 | -------------------------------------------------------------------------------- /postgres/postgres_table_reloptions.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT 16 | oid, 17 | s.schemaname, 18 | oid::regclass table_name, 19 | substr(unnest(reloptions), 1, strpos(unnest(reloptions), '=') - 1) option, 20 | substr(unnest(reloptions), 1 + strpos(unnest(reloptions), '=')) value 21 | FROM 22 | pg_class c 23 | JOIN pg_stat_all_tables s ON (s.relid = c.oid) 24 | WHERE 25 | reloptions IS NOT NULL 26 | AND (s.schemaname, 27 | s.relname) IN ( 28 | SELECT 29 | t.table_schema, 30 | t.table_name 31 | FROM 32 | information_schema.tables t 33 | JOIN pg_catalog.pg_class c ON (t.table_name = c.relname) 34 | JOIN pg_catalog.pg_user u ON (c.relowner = u.usesysid) 35 | WHERE 36 | t.table_schema LIKE '%' 37 | AND u.usename LIKE '%' 38 | AND t.table_name LIKE '%' 39 | AND t.table_schema NOT IN ('information_schema', 'pg_catalog')); -------------------------------------------------------------------------------- /postgres/postgres_check_table_reloptions.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT 16 | oid, 17 | s.schemaname, 18 | oid::regclass table_name, 19 | substr(unnest(reloptions), 1, strpos(unnest(reloptions), '=') - 1) option, 20 | substr(unnest(reloptions), 1 + strpos(unnest(reloptions), '=')) value 21 | FROM 22 | pg_class c 23 | JOIN pg_stat_all_tables s ON (s.relid = c.oid) 24 | WHERE 25 | reloptions IS NOT NULL 26 | AND (s.schemaname,s.relname) IN ( 27 | SELECT 28 | t.table_schema, 29 | t.table_name 30 | FROM 31 | information_schema.tables t 32 | JOIN pg_catalog.pg_class c ON (t.table_name = c.relname) 33 | JOIN pg_catalog.pg_user u ON (c.relowner = u.usesysid) 34 | WHERE 35 | t.table_schema LIKE '%' 36 | AND u.usename LIKE '%' 37 | AND t.table_name LIKE '%' 38 | AND t.table_schema NOT IN ('information_schema', 'pg_catalog') 39 | ); 40 | -------------------------------------------------------------------------------- /postgres/postgres_index_information_details.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | \prompt 'Enter schema_name: ' vSchemaName 16 | SELECT 17 | pg_class.relname, 18 | pg_size_pretty(pg_class.reltuples::bigint) AS rows_in_bytes, 19 | pg_class.reltuples AS num_rows, 20 | COUNT(*) AS total_indexes, 21 | COUNT(*) FILTER ( WHERE indisunique) AS unique_indexes, 22 | COUNT(*) FILTER ( WHERE indnatts = 1 ) AS single_column_indexes, 23 | COUNT(*) FILTER ( WHERE indnatts IS DISTINCT FROM 1 ) AS multi_column_indexes 24 | FROM 25 | pg_namespace 26 | LEFT JOIN pg_class ON pg_namespace.oid = pg_class.relnamespace 27 | LEFT JOIN pg_index ON pg_class.oid = pg_index.indrelid 28 | WHERE 29 | pg_namespace.nspname = :'vSchemaName' AND 30 | pg_class.relkind = 'r' 31 | GROUP BY pg_class.relname, pg_class.reltuples 32 | ORDER BY pg.namespace.nspname, pg_class.reltuples DESC; 33 | -------------------------------------------------------------------------------- /postgres/postgres_duplicate_indexes.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2025 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT 16 | c.relname AS relname, 17 | pg_size_pretty(sum(pg_relation_size(idx))::bigint) AS size, 18 | (array_agg(idx))[1] AS idx1, 19 | (array_agg(idx))[2] AS idx2, 20 | (array_agg(idx))[3] AS idx3, 21 | (array_agg(idx))[4] AS idx4, 22 | (array_agg(idx))[5] AS idx5, 23 | (array_agg(idx))[6] AS idx6, 24 | (array_agg(idx))[7] AS idx7, 25 | (array_agg(idx))[8] AS idx8, 26 | (array_agg(idx))[9] AS idx9, 27 | (array_agg(idx))[10] AS idx10 28 | FROM ( 29 | SELECT 30 | indrelid, 31 | indexrelid::regclass AS idx, 32 | (indrelid::text || E'\n' || indclass::text || E'\n' || indkey::text || E'\n' || coalesce(indexprs::text, '') || E'\n' || coalesce(indpred::text, '')) AS key 33 | FROM 34 | pg_index) sub 35 | JOIN pg_class c ON (c.oid = sub.indrelid) 36 | GROUP BY 37 | relname, 38 | key 39 | HAVING 40 | count(*) > 1 41 | ORDER BY 42 | sum(pg_relation_size(idx)) DESC; 43 | -------------------------------------------------------------------------------- /postgres/postgres_table_constraint_info.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT 16 | tc.table_schema, 17 | tc.table_name, 18 | pgc.contype, 19 | string_agg(col.column_name, ', ') AS columns, 20 | tc.constraint_name, 21 | tc.enforced, 22 | pgc.convalidated 23 | FROM 24 | information_schema.table_constraints tc 25 | JOIN pg_namespace nsp ON nsp.nspname = tc.constraint_schema 26 | JOIN pg_constraint pgc ON pgc.conname = tc.constraint_name 27 | AND pgc.connamespace = nsp.oid 28 | AND pgc.contype IN ('c', 'p', 'f') 29 | JOIN information_schema.columns col ON col.table_schema = tc.table_schema 30 | AND col.table_name = tc.table_name 31 | AND col.ordinal_position = ANY (pgc.conkey) 32 | WHERE 33 | tc.constraint_schema NOT IN ('pg_catalog', 'information_schema') 34 | GROUP BY 35 | tc.table_schema, 36 | tc.table_name, 37 | tc.constraint_name, 38 | pgc.contype, 39 | tc.enforced, 40 | pgc.convalidated 41 | ORDER BY 42 | tc.table_schema, 43 | tc.table_name; -------------------------------------------------------------------------------- /postgres/postgres_table_object_size.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2025 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT 16 | *, 17 | pg_size_pretty(table_bytes) AS table, 18 | pg_size_pretty(toast_bytes) AS toast, 19 | pg_size_pretty(index_bytes) AS index, 20 | pg_size_pretty(total_bytes) AS total 21 | FROM ( 22 | SELECT 23 | *, total_bytes - index_bytes - COALESCE(toast_bytes, 0) AS table_bytes 24 | FROM ( 25 | SELECT 26 | c.oid, 27 | n.nspname AS table_schema, 28 | c.relname AS table_name, 29 | c.reltuples AS row_estimate, 30 | pct.relname AS toast_table_name, 31 | pg_total_relation_size(c.oid) AS total_bytes, 32 | pg_indexes_size(c.oid) AS index_bytes, 33 | pg_total_relation_size(c.reltoastrelid) AS toast_bytes 34 | FROM 35 | pg_class c 36 | JOIN pg_class pct ON (c.reltoastrelid = pct.oid) 37 | LEFT JOIN pg_namespace n ON n.oid = c.relnamespace 38 | WHERE c.relkind = 'r' 39 | ) a 40 | ) a 41 | WHERE table_schema = 'public' 42 | AND table_name like '%' 43 | AND total_bytes > 0 44 | ORDER BY total_bytes DESC; 45 | -------------------------------------------------------------------------------- /postgres/postgres_index_usage_stats.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | \prompt 'Enter schema_name: ' vSchemaName 16 | SELECT 17 | t.schemaname, 18 | t.tablename, 19 | c.reltuples::bigint AS num_rows, 20 | pg_size_pretty(pg_relation_size(c.oid)) AS table_size, 21 | psai.indexrelname AS index_name, 22 | pg_size_pretty(pg_relation_size(i.indexrelid)) AS index_size, 23 | CASE WHEN i.indisunique THEN 'Y' ELSE 'N' END AS "unique", 24 | psai.idx_scan AS number_of_scans, 25 | psai.idx_tup_read AS tuples_read, 26 | psai.idx_tup_fetch AS tuples_fetched 27 | FROM 28 | pg_tables t 29 | LEFT JOIN pg_class c ON t.tablename = c.relname 30 | LEFT JOIN pg_index i ON c.oid = i.indrelid 31 | LEFT JOIN pg_stat_all_indexes psai ON i.indexrelid = psai.indexrelid 32 | WHERE 33 | t.schemaname NOT IN ('pg_catalog', 'information_schema') 34 | AND t.schemaname IN (:'vSchemaName') 35 | ORDER BY 1, 2; 36 | -------------------------------------------------------------------------------- /site/css/v4-font-face.min.css: -------------------------------------------------------------------------------- 1 | /*! 2 | * Font Awesome Free 6.5.1 by @fontawesome - https://fontawesome.com 3 | * License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) 4 | * Copyright 2023 Fonticons, Inc. 5 | */ 6 | @font-face{font-family:"FontAwesome";font-display:block;src:url(../webfonts/fa-solid-900.woff2) format("woff2"),url(../webfonts/fa-solid-900.ttf) format("truetype")}@font-face{font-family:"FontAwesome";font-display:block;src:url(../webfonts/fa-brands-400.woff2) format("woff2"),url(../webfonts/fa-brands-400.ttf) format("truetype")}@font-face{font-family:"FontAwesome";font-display:block;src:url(../webfonts/fa-regular-400.woff2) format("woff2"),url(../webfonts/fa-regular-400.ttf) format("truetype");unicode-range:u+f003,u+f006,u+f014,u+f016-f017,u+f01a-f01b,u+f01d,u+f022,u+f03e,u+f044,u+f046,u+f05c-f05d,u+f06e,u+f070,u+f087-f088,u+f08a,u+f094,u+f096-f097,u+f09d,u+f0a0,u+f0a2,u+f0a4-f0a7,u+f0c5,u+f0c7,u+f0e5-f0e6,u+f0eb,u+f0f6-f0f8,u+f10c,u+f114-f115,u+f118-f11a,u+f11c-f11d,u+f133,u+f147,u+f14e,u+f150-f152,u+f185-f186,u+f18e,u+f190-f192,u+f196,u+f1c1-f1c9,u+f1d9,u+f1db,u+f1e3,u+f1ea,u+f1f7,u+f1f9,u+f20a,u+f247-f248,u+f24a,u+f24d,u+f255-f25b,u+f25d,u+f271-f274,u+f278,u+f27b,u+f28c,u+f28e,u+f29c,u+f2b5,u+f2b7,u+f2ba,u+f2bc,u+f2be,u+f2c0-f2c1,u+f2c3,u+f2d0,u+f2d2,u+f2d4,u+f2dc}@font-face{font-family:"FontAwesome";font-display:block;src:url(../webfonts/fa-v4compatibility.woff2) format("woff2"),url(../webfonts/fa-v4compatibility.ttf) format("truetype");unicode-range:u+f041,u+f047,u+f065-f066,u+f07d-f07e,u+f080,u+f08b,u+f08e,u+f090,u+f09a,u+f0ac,u+f0ae,u+f0b2,u+f0d0,u+f0d6,u+f0e4,u+f0ec,u+f10a-f10b,u+f123,u+f13e,u+f148-f149,u+f14c,u+f156,u+f15e,u+f160-f161,u+f163,u+f175-f178,u+f195,u+f1f8,u+f219,u+f27a} -------------------------------------------------------------------------------- /postgres/postgres_vacuum_progress_detailed.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | /* Detailed Vacuum Progress */ 16 | SELECT 17 | p.pid, 18 | now() - a.xact_start AS duration, 19 | coalesce(wait_event_type || '.' || wait_event, 'f') AS waiting, 20 | CASE WHEN a.query ~* '^autovacuum.*to prevent wraparound' THEN 21 | 'wraparound' 22 | WHEN a.query ~* '^vacuum' THEN 23 | 'user' 24 | ELSE 25 | 'regular' 26 | END AS mode, 27 | p.datname AS database, 28 | p.relid::regclass AS table, 29 | p.phase, 30 | pg_size_pretty(p.heap_blks_total * current_setting('block_size')::int) AS table_size, 31 | pg_size_pretty(pg_total_relation_size(relid)) AS total_size, 32 | pg_size_pretty(p.heap_blks_scanned * current_setting('block_size')::int) AS scanned, 33 | pg_size_pretty(p.heap_blks_vacuumed * current_setting('block_size')::int) AS vacuumed, 34 | round(100.0 * p.heap_blks_scanned / p.heap_blks_total, 1) AS scanned_pct, 35 | round(100.0 * p.heap_blks_vacuumed / p.heap_blks_total, 1) AS vacuumed_pct, 36 | p.index_vacuum_count, 37 | round(100.0 * p.num_dead_tuples / p.max_dead_tuples, 1) AS dead_pct 38 | FROM 39 | pg_stat_progress_vacuum p 40 | JOIN pg_stat_activity a USING (pid) 41 | ORDER BY 42 | now() - a.xact_start DESC; 43 | -------------------------------------------------------------------------------- /postgres/postgres_top_10_sql_by_execution.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | WITH 16 | hist AS ( 17 | SELECT queryid::text, 18 | SUBSTRING(query from 1 for 100) query, 19 | ROW_NUMBER () OVER (ORDER BY calls DESC) rn, 20 | calls 21 | FROM pg_stat_statements 22 | WHERE queryid IS NOT NULL 23 | AND query::text not like '%pg_%' 24 | AND query::text not like '%g_%' 25 | AND query::text not like '%heartbeat%' 26 | AND query::text not like '%SELECT $1%' 27 | AND query::text not like '%google_%' 28 | AND query::text not like 'SELECT txid_current%' 29 | AND query::text not like 'CREATE TEMPORARY TABLE%' 30 | AND query::text not like 'EXPLAIN%' 31 | AND query::text not like 'vacuum%' 32 | AND query::text not like 'analyze%' 33 | GROUP BY 34 | queryid, 35 | SUBSTRING(query from 1 for 100), 36 | calls 37 | ), 38 | total AS ( 39 | SELECT SUM(calls) calls FROM hist 40 | ) 41 | SELECT DISTINCT 42 | h.queryid::text, 43 | h.calls, 44 | ROUND(100 * h.calls / t.calls, 1) percent, 45 | h.query 46 | FROM hist h, 47 | total t 48 | WHERE h.calls >= t.calls / 1000 AND rn <= 14 49 | UNION ALL 50 | SELECT 'Others', 51 | COALESCE(SUM(h.calls), 0) calls, 52 | COALESCE(ROUND(100 * SUM(h.calls) / AVG(t.calls), 1), 0) percent, 53 | NULL sql_text 54 | FROM hist h, 55 | total t 56 | WHERE h.calls < t.calls / 1000 OR rn > 14 57 | ORDER BY 2 DESC NULLS LAST; 58 | -------------------------------------------------------------------------------- /postgres/postgres_blocking_lock_sql.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT 16 | blocked_locks.pid AS blocked_pid, 17 | blocked_activity.usename AS blocked_user, 18 | blocking_locks.pid AS blocking_pid, 19 | blocking_activity.usename AS blocking_user, 20 | blocked_activity.query AS blocked_statement, 21 | blocked_activity.wait_event AS blocked_wait_event, 22 | blocking_activity.wait_event AS blocking_wait_event, 23 | blocking_activity.query AS current_statement_in_blocking_process 24 | FROM 25 | pg_catalog.pg_locks blocked_locks 26 | JOIN pg_catalog.pg_stat_activity blocked_activity ON blocked_activity.pid = blocked_locks.pid 27 | JOIN pg_catalog.pg_locks blocking_locks ON blocking_locks.locktype = blocked_locks.locktype 28 | AND blocking_locks.database IS NOT DISTINCT FROM blocked_locks.database 29 | AND blocking_locks.relation IS NOT DISTINCT FROM blocked_locks.relation 30 | AND blocking_locks.page IS NOT DISTINCT FROM blocked_locks.page 31 | AND blocking_locks.tuple IS NOT DISTINCT FROM blocked_locks.tuple 32 | AND blocking_locks.virtualxid IS NOT DISTINCT FROM blocked_locks.virtualxid 33 | AND blocking_locks.transactionid IS NOT DISTINCT FROM blocked_locks.transactionid 34 | AND blocking_locks.classid IS NOT DISTINCT FROM blocked_locks.classid 35 | AND blocking_locks.objid IS NOT DISTINCT FROM blocked_locks.objid 36 | AND blocking_locks.objsubid IS NOT DISTINCT FROM blocked_locks.objsubid 37 | AND blocking_locks.pid != blocked_locks.pid 38 | JOIN pg_catalog.pg_stat_activity blocking_activity ON blocking_activity.pid = blocking_locks.pid 39 | WHERE 40 | NOT blocked_locks.granted; -------------------------------------------------------------------------------- /postgres/postgres_top_10_sql_by_total_exec_time.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | /* Top SQL by Total Exec Time */ 16 | WITH 17 | hist AS ( 18 | SELECT queryid::text, 19 | SUBSTRING(query from 1 for 100) query, 20 | ROW_NUMBER () OVER (ORDER BY total_exec_time::numeric DESC) rn, 21 | SUM(total_exec_time::numeric) total_exec_time 22 | FROM pg_stat_statements 23 | WHERE queryid IS NOT NULL 24 | AND query::text not like '%pg_%' 25 | AND query::text not like '%g_%' 26 | AND query::text not like '%heartbeat%' 27 | AND query::text not like '%SELECT $1%' 28 | AND query::text not like '%google_%' 29 | AND query::text not like 'SELECT txid_current%' 30 | AND query::text not like 'CREATE TEMPORARY TABLE%' 31 | AND query::text not like 'EXPLAIN%' 32 | AND query::text not like 'vacuum%' 33 | AND query::text not like 'analyze%' 34 | GROUP BY 35 | queryid, 36 | SUBSTRING(query from 1 for 100), 37 | total_exec_time::numeric 38 | ), 39 | total AS ( 40 | SELECT SUM(total_exec_time::numeric) total_exec_time FROM hist 41 | ) 42 | SELECT DISTINCT 43 | h.queryid::text, 44 | ROUND(h.total_exec_time::numeric,3) total_exec_time, 45 | ROUND(100 * h.total_exec_time / t.total_exec_time, 1) percent, 46 | h.query 47 | FROM hist h, 48 | total t 49 | WHERE h.total_exec_time >= t.total_exec_time / 1000 AND rn <= 14 50 | UNION ALL 51 | SELECT 'Others', 52 | ROUND(COALESCE(SUM(h.total_exec_time::numeric), 0), 3) total_exec_time, 53 | COALESCE(ROUND(100 * SUM(h.total_exec_time) / AVG(t.total_exec_time), 1), 0) percent, 54 | NULL sql_text 55 | FROM hist h, 56 | total t 57 | WHERE h.total_exec_time < t.total_exec_time / 1000 OR rn > 14 58 | ORDER BY 3 DESC NULLS LAST; 59 | -------------------------------------------------------------------------------- /postgres/postgres_top_10_sql_by_mean_exec_time.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | /* Top SQL by Mean Exec Time */ 16 | WITH 17 | hist AS ( 18 | SELECT queryid::text, 19 | SUBSTRING(query from 1 for 100) query, 20 | ROW_NUMBER () OVER (ORDER BY mean_exec_time::numeric DESC) rn, 21 | SUM(mean_exec_time::numeric) mean_exec_time 22 | FROM pg_stat_statements 23 | WHERE queryid IS NOT NULL 24 | AND query::text not like '%pg_%' 25 | AND query::text not like '%g_%' 26 | AND query::text not like '%heartbeat%' 27 | AND query::text not like '%SELECT $1%' 28 | AND query::text not like '%google_%' 29 | AND query::text not like 'SELECT txid_current%' 30 | AND query::text not like 'CREATE TEMPORARY TABLE%' 31 | AND query::text not like 'EXPLAIN%' 32 | AND query::text not like 'DROP EXTENSION%' 33 | AND query::text not like 'vacuum%' 34 | AND query::text not like 'analyze%' 35 | AND query::text not like 'COPY%' 36 | AND query::text not like 'FETCH FORWARD%' 37 | AND query::text not like '%shared_buffers_active%' 38 | GROUP BY 39 | queryid, 40 | SUBSTRING(query from 1 for 100), 41 | mean_exec_time::numeric 42 | ), 43 | total AS ( 44 | SELECT SUM(mean_exec_time::numeric) mean_exec_time FROM hist 45 | ) 46 | SELECT DISTINCT 47 | h.queryid::text, 48 | ROUND(h.mean_exec_time::numeric,3) mean_exec_time, 49 | ROUND(100 * h.mean_exec_time / t.mean_exec_time, 1) percent, 50 | h.query 51 | FROM hist h, 52 | total t 53 | WHERE h.mean_exec_time >= t.mean_exec_time / 1000 AND rn <= 14 54 | UNION ALL 55 | SELECT 'Others', 56 | ROUND(COALESCE(SUM(h.mean_exec_time), 0), 3) mean_exec_time, 57 | COALESCE(ROUND(100 * SUM(h.mean_exec_time) / AVG(t.mean_exec_time), 1), 0) percent, 58 | NULL sql_text 59 | FROM hist h, 60 | total t 61 | WHERE h.mean_exec_time < t.mean_exec_time / 1000 OR rn > 14 62 | ORDER BY 3 DESC NULLS LAST; 63 | -------------------------------------------------------------------------------- /postgres/postgres_google_db_advisor_example.sql: -------------------------------------------------------------------------------- 1 | /* Set AlloyDB Flag to have advisor run 1x per hour */ 2 | google_db_advisor.auto_advisor_schedule = 'EVERY 1 HOURS' 3 | 4 | /* Check location of extensions */ 5 | (postgres@10.3.1.17:5432) [postgres] > \dx hypopg 6 | List of installed extensions 7 | Name | Version | Schema | Description 8 | --------+---------+--------+------------- 9 | hypopg | 1.3.2 | public | 10 | (postgres@10.3.1.17:5432) [postgres] > \dx google_db_advisor 11 | List of installed extensions 12 | Name | Version | Schema | Description 13 | -------------------+---------+--------+------------- 14 | google_db_advisor | 1.0 | public | 15 | 16 | CREATE TABLE public.index_advisor_test ( 17 | id int, 18 | value numeric, 19 | product_id int, 20 | effective_date timestamp(3) 21 | ); 22 | INSERT INTO public.index_advisor_test VALUES ( 23 | generate_series(0,100000000), 24 | random()*1000, 25 | random()*100, 26 | current_timestamp(3)); 27 | 28 | CREATE SCHEMA idx_advisor; 29 | 30 | CREATE TABLE idx_advisor.index_advisor_test ( 31 | id int, 32 | value numeric, 33 | product_id int, 34 | effective_date timestamp(3) 35 | ); 36 | INSERT INTO idx_advisor.index_advisor_test VALUES ( 37 | generate_series(0,100000000), 38 | random()*1000, 39 | random()*100, 40 | current_timestamp(3)); 41 | 42 | ANALYZE VERBOSE public.index_advisor_test; 43 | ANALYZE VERBOSE idx_advisor.index_advisor_test; 44 | 45 | /* Run a query that could possibly benefit from an index: */ 46 | 47 | SET enable_ultra_fast_cache_explain_output TO ON; 48 | EXPLAIN (analyze, verbose, columnar_engine, costs, settings, buffers, wal, timing, summary, format text) 49 | select /* INDEX ADVISOR TEST */ * from public.index_advisor_test where id = 500533; 50 | 51 | EXPLAIN (analyze, verbose, columnar_engine, costs, settings, buffers, wal, timing, summary, format text) 52 | select /* INDEX ADVISOR TEST */ * from idx_advisor.index_advisor_test where id = 500533; 53 | 54 | 55 | /* Check index advisor las run */ 56 | SELECT DISTINCT recommended_indexes, query 57 | FROM google_db_advisor_workload_report r, google_db_advisor_workload_statements s 58 | WHERE r.query_id = s.query_id; 59 | 60 | SELECT 61 | r.recommended_indexes, 62 | s.query 63 | FROM 64 | google_db_advisor_workload_report r, 65 | google_db_advisor_workload_statements s 66 | WHERE 67 | r.query_id = s.query_id 68 | AND r.db_id = s.db_id 69 | AND r.user_id = s.user_id 70 | AND length(r.recommended_indexes) > 0; 71 | 72 | select * from google_db_advisor_workload_report_detail; 73 | 74 | 75 | 76 | -------------------------------------------------------------------------------- /postgres/postgres_table_cache_hit_vs_disk_hit_ratio.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | WITH all_tables AS ( 16 | SELECT 17 | * 18 | FROM ( 19 | SELECT 20 | 'all'::text AS table_name, 21 | sum((coalesce(heap_blks_read, 0) + coalesce(idx_blks_read, 0) + coalesce(toast_blks_read, 0) + coalesce(tidx_blks_read, 0))) AS from_disk, 22 | sum((coalesce(heap_blks_hit, 0) + coalesce(idx_blks_hit, 0) + coalesce(toast_blks_hit, 0) + coalesce(tidx_blks_hit, 0))) AS from_cache 23 | FROM 24 | pg_statio_all_tables --> change to pg_statio_USER_tables if you want to check only user tables (excluding postgres's own tables) 25 | ) a 26 | WHERE (from_disk + from_cache) > 0 -- discard tables without hits 27 | ), 28 | tables AS ( 29 | SELECT 30 | * 31 | FROM ( 32 | SELECT 33 | relname AS table_name, 34 | ((coalesce(heap_blks_read, 0) + coalesce(idx_blks_read, 0) + coalesce(toast_blks_read, 0) + coalesce(tidx_blks_read, 0))) AS from_disk, 35 | ((coalesce(heap_blks_hit, 0) + coalesce(idx_blks_hit, 0) + coalesce(toast_blks_hit, 0) + coalesce(tidx_blks_hit, 0))) AS from_cache 36 | FROM 37 | pg_statio_all_tables --> change to pg_statio_USER_tables if you want to check only user tables (excluding postgres's own tables) 38 | ) a 39 | WHERE (from_disk + from_cache) > 0 -- discard tables without hits 40 | ) 41 | SELECT 42 | table_name AS "table name", 43 | from_disk AS "disk hits", 44 | round((from_disk::numeric / (from_disk + from_cache)::numeric) * 100.0, 2) AS "% disk hits", 45 | round((from_cache::numeric / (from_disk + from_cache)::numeric) * 100.0, 2) AS "% cache hits", 46 | (from_disk + from_cache) AS "total hits" 47 | FROM ( 48 | SELECT 49 | * 50 | FROM 51 | all_tables 52 | UNION ALL 53 | SELECT 54 | * 55 | FROM 56 | tables) a 57 | ORDER BY 58 | ( 59 | CASE WHEN table_name = 'all' THEN 60 | 0 61 | ELSE 62 | 1 63 | END), 64 | from_disk DESC; -------------------------------------------------------------------------------- /postgres/postgres_unindexed_foreign_keys.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | WITH y AS ( 16 | SELECT 17 | pg_catalog.format('%I.%I', n1.nspname, c1.relname) AS referencing_tbl, 18 | pg_catalog.quote_ident(a1.attname) AS referencing_column, 19 | t.conname AS existing_fk_on_referencing_tbl, 20 | pg_catalog.format('%I.%I', n2.nspname, c2.relname) AS referenced_tbl, 21 | pg_catalog.quote_ident(a2.attname) AS referenced_column, 22 | pg_relation_size(pg_catalog.format('%I.%I', n1.nspname, c1.relname)) AS referencing_tbl_bytes, 23 | pg_relation_size(pg_catalog.format('%I.%I', n2.nspname, c2.relname)) AS referenced_tbl_bytes, 24 | pg_catalog.format($$CREATE INDEX %I_idx ON %I.%I(%I);$$, t.conname, n1.nspname, c1.relname, a1.attname) AS suggestion 25 | FROM 26 | pg_catalog.pg_constraint t 27 | JOIN pg_catalog.pg_attribute a1 ON a1.attrelid = t.conrelid 28 | AND a1.attnum = t.conkey[1] 29 | JOIN pg_catalog.pg_class c1 ON c1.oid = t.conrelid 30 | JOIN pg_catalog.pg_namespace n1 ON n1.oid = c1.relnamespace 31 | JOIN pg_catalog.pg_class c2 ON c2.oid = t.confrelid 32 | JOIN pg_catalog.pg_namespace n2 ON n2.oid = c2.relnamespace 33 | JOIN pg_catalog.pg_attribute a2 ON a2.attrelid = t.confrelid 34 | AND a2.attnum = t.confkey[1] 35 | WHERE 36 | t.contype = 'f' 37 | AND NOT EXISTS ( 38 | SELECT 39 | 1 40 | FROM 41 | pg_catalog.pg_index i 42 | WHERE 43 | i.indrelid = t.conrelid 44 | AND i.indkey[0] = t.conkey[1])) 45 | SELECT 46 | referencing_tbl, 47 | referencing_column, 48 | existing_fk_on_referencing_tbl, 49 | referenced_tbl, 50 | referenced_column, 51 | pg_size_pretty(referencing_tbl_bytes) AS referencing_tbl_size, 52 | pg_size_pretty(referenced_tbl_bytes) AS referenced_tbl_size, 53 | suggestion 54 | FROM 55 | y 56 | ORDER BY 57 | referencing_tbl_bytes DESC, 58 | referenced_tbl_bytes DESC, 59 | referencing_tbl, 60 | referenced_tbl, 61 | referencing_column, 62 | referenced_column; 63 | -------------------------------------------------------------------------------- /site/js/darkmode.js: -------------------------------------------------------------------------------- 1 | function setColorMode(mode) { 2 | // Switch between light/dark theme. `mode` is a string value of either 'dark' or 'light'. 3 | var hljs_light = document.getElementById('hljs-light'), 4 | hljs_dark = document.getElementById('hljs-dark'); 5 | document.documentElement.setAttribute('data-bs-theme', mode); 6 | if (mode == 'dark') { 7 | hljs_light.disabled = true; 8 | hljs_dark.disabled = false; 9 | } else { 10 | hljs_dark.disabled = true; 11 | hljs_light.disabled = false; 12 | } 13 | } 14 | 15 | function updateModeToggle(mode) { 16 | // Update icon and toggle checkmarks of color mode selector. 17 | var menu = document.getElementById('theme-menu'); 18 | document.querySelectorAll('[data-bs-theme-value]') 19 | .forEach(function(toggle) { 20 | if (mode == toggle.getAttribute('data-bs-theme-value')) { 21 | toggle.setAttribute('aria-pressed', 'true'); 22 | toggle.lastElementChild.classList.remove('d-none'); 23 | menu.firstElementChild.setAttribute('class', toggle.firstElementChild.getAttribute('class')); 24 | } else { 25 | toggle.setAttribute('aria-pressed', 'false'); 26 | toggle.lastElementChild.classList.add('d-none'); 27 | } 28 | }); 29 | } 30 | 31 | function onSystemColorSchemeChange(event) { 32 | // Update site color mode to match system color mode. 33 | setColorMode(event.matches ? 'dark' : 'light'); 34 | } 35 | 36 | var mql = window.matchMedia('(prefers-color-scheme: dark)'), 37 | defaultMode = document.documentElement.getAttribute('data-bs-theme'), 38 | storedMode = localStorage.getItem('mkdocs-colormode'); 39 | if (storedMode && storedMode != 'auto') { 40 | setColorMode(storedMode); 41 | updateModeToggle(storedMode); 42 | } else if (storedMode == 'auto' || defaultMode == 'auto') { 43 | setColorMode(mql.matches ? 'dark' : 'light'); 44 | updateModeToggle('auto'); 45 | mql.addEventListener('change', onSystemColorSchemeChange); 46 | } else { 47 | setColorMode(defaultMode); 48 | updateModeToggle(defaultMode); 49 | } 50 | 51 | document.querySelectorAll('[data-bs-theme-value]') 52 | .forEach(function(toggle) { 53 | toggle.addEventListener('click', function (e) { 54 | var mode = e.currentTarget.getAttribute('data-bs-theme-value'); 55 | localStorage.setItem('mkdocs-colormode', mode); 56 | if (mode == 'auto') { 57 | setColorMode(mql.matches ? 'dark' : 'light'); 58 | mql.addEventListener('change', onSystemColorSchemeChange); 59 | } else { 60 | setColorMode(mode); 61 | mql.removeEventListener('change', onSystemColorSchemeChange); 62 | } 63 | updateModeToggle(mode); 64 | }); 65 | }); 66 | -------------------------------------------------------------------------------- /postgres/postgres_index_build_status.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | -- Basis of script from https://wiki.postgresql.org/wiki/Index_Progress 16 | 17 | select 18 | now(), 19 | query_start as started_at, 20 | now() - query_start as query_duration, 21 | format('[%s] %s', a.pid, a.query) as pid_and_query, 22 | index_relid::regclass as index_name, 23 | relid::regclass as table_name, 24 | (pg_size_pretty(pg_relation_size(relid))) as table_size, 25 | nullif(wait_event_type, '') || ': ' || wait_event as wait_type_and_event, 26 | phase, 27 | format( 28 | '%s (%s of %s)', 29 | coalesce((round(100 * blocks_done::numeric / nullif(blocks_total, 0), 2))::text || '%', 'N/A'), 30 | coalesce(blocks_done::text, '?'), 31 | coalesce(blocks_total::text, '?') 32 | ) as blocks_progress, 33 | format( 34 | '%s (%s of %s)', 35 | coalesce((round(100 * tuples_done::numeric / nullif(tuples_total, 0), 2))::text || '%', 'N/A'), 36 | coalesce(tuples_done::text, '?'), 37 | coalesce(tuples_total::text, '?') 38 | ) as tuples_progress, 39 | current_locker_pid, 40 | (select nullif(left(query, 150), '') || '...' from pg_stat_activity a where a.pid = current_locker_pid) as current_locker_query, 41 | format( 42 | '%s (%s of %s)', 43 | coalesce((round(100 * lockers_done::numeric / nullif(lockers_total, 0), 2))::text || '%', 'N/A'), 44 | coalesce(lockers_done::text, '?'), 45 | coalesce(lockers_total::text, '?') 46 | ) as lockers_progress, 47 | format( 48 | '%s (%s of %s)', 49 | coalesce((round(100 * partitions_done::numeric / nullif(partitions_total, 0), 2))::text || '%', 'N/A'), 50 | coalesce(partitions_done::text, '?'), 51 | coalesce(partitions_total::text, '?') 52 | ) as partitions_progress, 53 | ( 54 | select 55 | format( 56 | '%s (%s of %s)', 57 | coalesce((round(100 * n_dead_tup::numeric / nullif(reltuples::numeric, 0), 2))::text || '%', 'N/A'), 58 | coalesce(n_dead_tup::text, '?'), 59 | coalesce(reltuples::int8::text, '?') 60 | ) 61 | from pg_stat_all_tables t, pg_class tc 62 | where t.relid = p.relid and tc.oid = p.relid 63 | ) as table_dead_tuples 64 | from pg_stat_progress_create_index p 65 | left join pg_stat_activity a on a.pid = p.pid 66 | order by p.index_relid; -------------------------------------------------------------------------------- /postgres/postgres_blocking_lock_pid_tree.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | WITH RECURSIVE activity AS ( 16 | SELECT 17 | pg_blocking_pids (pid) blocked_by, 18 | *, 19 | age(clock_timestamp(), xact_start)::interval(0) AS tx_age, 20 | -- "pg_locks.waitstart" – PG14+ only; for older versions: age(clock_timestamp(), state_change) as wait_age 21 | age(clock_timestamp(), ( 22 | SELECT 23 | max(l.waitstart) 24 | FROM pg_locks l 25 | WHERE 26 | a.pid = l.pid))::interval(0) AS wait_age 27 | FROM 28 | pg_stat_activity a 29 | WHERE 30 | state IS DISTINCT FROM 'idle' 31 | ), 32 | blockers AS ( 33 | SELECT 34 | array_agg(DISTINCT c ORDER BY c) AS pids 35 | FROM ( 36 | SELECT 37 | unnest(blocked_by) 38 | FROM 39 | activity) AS dt (c) 40 | ), 41 | tree AS ( 42 | SELECT 43 | activity.*, 44 | 1 AS level, 45 | activity.pid AS top_blocker_pid, 46 | ARRAY[activity.pid] AS path, 47 | ARRAY[activity.pid]::int[] AS all_blockers_above 48 | FROM 49 | activity, 50 | blockers 51 | WHERE 52 | ARRAY[pid] <@ blockers.pids 53 | AND blocked_by = '{}'::int[] 54 | UNION ALL 55 | SELECT 56 | activity.*, 57 | tree.level + 1 AS level, 58 | tree.top_blocker_pid, 59 | path || ARRAY[activity.pid] AS path, 60 | tree.all_blockers_above || array_agg(activity.pid) OVER () AS all_blockers_above 61 | FROM 62 | activity, 63 | tree 64 | WHERE 65 | NOT ARRAY[activity.pid] <@ tree.all_blockers_above 66 | AND activity.blocked_by <> '{}'::int[] 67 | AND activity.blocked_by <@ tree.all_blockers_above 68 | ) 69 | SELECT 70 | pid, 71 | blocked_by, 72 | CASE WHEN wait_event_type <> 'Lock' THEN 73 | replace(state, 'idle in transaction', 'idletx') 74 | ELSE 75 | 'waiting' 76 | END AS state, 77 | wait_event_type || ':' || wait_event AS wait, 78 | wait_age, 79 | tx_age, 80 | to_char(age(backend_xid), 'FM999,999,999,990') AS xid_age, 81 | to_char(2147483647 - age(backend_xmin), 'FM999,999,999,990') AS xmin_ttf, 82 | datname, 83 | usename, 84 | ( 85 | SELECT 86 | count(DISTINCT t1.pid) 87 | FROM 88 | tree t1 89 | WHERE 90 | ARRAY[tree.pid] <@ t1.path 91 | AND t1.pid <> tree.pid) AS blkd, 92 | format('%s %s%s', lpad('[' || pid::text || ']', 9, ' '), repeat('.', level -1) || CASE WHEN level > 1 THEN 93 | ' ' 94 | END, 95 | LEFT (query, 1000)) AS query 96 | FROM 97 | tree 98 | ORDER BY 99 | top_blocker_pid, 100 | level, 101 | pid; -------------------------------------------------------------------------------- /postgres/postgres_report_high_water_mark_wasted_space.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | CREATE EXTENSION pg_freespacemap; 16 | 17 | CREATE OR REPLACE FUNCTION show_empty_pages(p_table_name TEXT) 18 | RETURNS VOID AS $$ 19 | DECLARE 20 | -- Core processing variables 21 | table_oid_regclass REGCLASS; 22 | block_size BIGINT; 23 | fsm_granularity BIGINT; 24 | max_fsm_free_space BIGINT; 25 | total_pages BIGINT; 26 | high_water_mark BIGINT := 0; 27 | 28 | -- Variables for the final summary 29 | first_empty_block BIGINT; 30 | free_pages_at_end BIGINT; 31 | free_space_at_end TEXT; 32 | BEGIN 33 | -- Setup 34 | table_oid_regclass := p_table_name::regclass; 35 | block_size := current_setting('block_size')::bigint; 36 | SELECT relpages INTO total_pages FROM pg_class WHERE oid = table_oid_regclass; 37 | fsm_granularity := block_size / 256; 38 | max_fsm_free_space := floor((block_size - 24) / fsm_granularity) * fsm_granularity; 39 | 40 | -------------------------------------------------------------------------------- 41 | -- PASS 1: FIND THE HIGH-WATER MARK (last page with data) 42 | -------------------------------------------------------------------------------- 43 | FOR i IN REVERSE (total_pages - 1)..0 LOOP 44 | IF pg_freespace(table_oid_regclass, i) < max_fsm_free_space THEN 45 | high_water_mark := i; 46 | EXIT; 47 | END IF; 48 | END LOOP; 49 | 50 | -------------------------------------------------------------------------------- 51 | -- FINAL STEP: CALCULATE AND RAISE THE SUMMARY NOTICE 52 | -------------------------------------------------------------------------------- 53 | first_empty_block := high_water_mark + 1; 54 | free_pages_at_end := total_pages - first_empty_block; 55 | IF free_pages_at_end < 0 THEN 56 | free_pages_at_end := 0; 57 | END IF; 58 | free_space_at_end := pg_size_pretty(free_pages_at_end * block_size); 59 | 60 | RAISE NOTICE '-------------------------------------------------------------'; 61 | RAISE NOTICE 'Summary for table: %', p_table_name; 62 | RAISE NOTICE '-------------------------------------------------------------'; 63 | RAISE NOTICE 'The High Water Mark (HWM) is at page: %', total_pages; 64 | IF total_pages <> first_empty_block THEN 65 | RAISE NOTICE 'First potentially empty page is at: %', first_empty_block; 66 | RAISE NOTICE 'Total Pages in Table: %', total_pages; 67 | RAISE NOTICE 'Number of potentially truncatable pages at the end: %', free_pages_at_end; 68 | RAISE NOTICE 'Amount of free space at the end of the table: %', free_space_at_end; 69 | ELSE 70 | RAISE NOTICE 'There are no empty pages to truncate'; 71 | END IF; 72 | RAISE NOTICE '-------------------------------------------------------------'; 73 | END; 74 | $$ LANGUAGE plpgsql; 75 | -------------------------------------------------------------------------------- /postgres/postgres_find_unused_indexes.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2025 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | 16 | WITH table_scans as ( 17 | SELECT relid, 18 | tables.idx_scan + tables.seq_scan as all_scans, 19 | ( tables.n_tup_ins + tables.n_tup_upd + tables.n_tup_del ) as writes, 20 | pg_relation_size(relid) as table_size 21 | FROM pg_stat_all_tables as tables 22 | WHERE schemaname not in ('pg_toast','pg_catalog','partman') 23 | ), 24 | all_writes as ( 25 | SELECT sum(writes) as total_writes 26 | FROM table_scans 27 | ), 28 | indexes as ( 29 | SELECT idx_stat.relid, idx_stat.indexrelid, 30 | idx_stat.schemaname, idx_stat.relname as tablename, 31 | idx_stat.indexrelname as indexname, 32 | idx_stat.idx_scan, 33 | pg_relation_size(idx_stat.indexrelid) as index_bytes, 34 | indexdef ~* 'USING btree' AS idx_is_btree 35 | FROM pg_stat_user_indexes as idx_stat 36 | JOIN pg_index 37 | USING (indexrelid) 38 | JOIN pg_indexes as indexes 39 | ON idx_stat.schemaname = indexes.schemaname 40 | AND idx_stat.relname = indexes.tablename 41 | AND idx_stat.indexrelname = indexes.indexname 42 | WHERE pg_index.indisunique = FALSE 43 | ), 44 | index_ratios AS ( 45 | SELECT schemaname, tablename, indexname, 46 | idx_scan, all_scans, 47 | round(( CASE WHEN all_scans = 0 THEN 0.0::NUMERIC 48 | ELSE idx_scan::NUMERIC/all_scans * 100 END),2) as index_scan_pct, 49 | writes, 50 | round((CASE WHEN writes = 0 THEN idx_scan::NUMERIC ELSE idx_scan::NUMERIC/writes END),2) 51 | as scans_per_write, 52 | pg_size_pretty(index_bytes) as index_size_pretty, 53 | pg_size_pretty(table_size) as table_size, 54 | idx_is_btree, 55 | index_bytes as index_size_bytes 56 | FROM indexes 57 | JOIN table_scans 58 | USING (relid) 59 | ), 60 | index_groups AS ( 61 | SELECT 'Never Used Indexes' as reason, *, 1 as grp 62 | FROM index_ratios 63 | WHERE 64 | idx_scan = 0 65 | and idx_is_btree 66 | UNION ALL 67 | SELECT 'Low Scans, High Writes' as reason, *, 2 as grp 68 | FROM index_ratios 69 | WHERE 70 | scans_per_write <= 1 71 | and index_scan_pct < 10 72 | and idx_scan > 0 73 | and writes > 100 74 | and idx_is_btree 75 | UNION ALL 76 | SELECT 'Seldom Used Large Indexes' as reason, *, 3 as grp 77 | FROM index_ratios 78 | WHERE 79 | index_scan_pct < 5 80 | and scans_per_write > 1 81 | and idx_scan > 0 82 | and idx_is_btree 83 | and index_size_bytes > 100000000 84 | UNION ALL 85 | SELECT 'High-Write Large Non-Btree' as reason, index_ratios.*, 4 as grp 86 | FROM index_ratios, all_writes 87 | WHERE 88 | ( writes::NUMERIC / ( total_writes + 1 ) ) > 0.02 89 | AND NOT idx_is_btree 90 | AND index_size_bytes > 100000000 91 | ORDER BY grp, index_size_bytes DESC ) 92 | SELECT reason, schemaname, tablename, indexname, 93 | index_scan_pct, scans_per_write, index_size_pretty,index_size_bytes, table_size 94 | FROM index_groups 95 | WHERE tablename like '%' 96 | ORDER BY reason, index_size_bytes,table_size; 97 | -------------------------------------------------------------------------------- /site/search/main.js: -------------------------------------------------------------------------------- 1 | function getSearchTermFromLocation() { 2 | var sPageURL = window.location.search.substring(1); 3 | var sURLVariables = sPageURL.split('&'); 4 | for (var i = 0; i < sURLVariables.length; i++) { 5 | var sParameterName = sURLVariables[i].split('='); 6 | if (sParameterName[0] == 'q') { 7 | return decodeURIComponent(sParameterName[1].replace(/\+/g, '%20')); 8 | } 9 | } 10 | } 11 | 12 | function joinUrl (base, path) { 13 | if (path.substring(0, 1) === "/") { 14 | // path starts with `/`. Thus it is absolute. 15 | return path; 16 | } 17 | if (base.substring(base.length-1) === "/") { 18 | // base ends with `/` 19 | return base + path; 20 | } 21 | return base + "/" + path; 22 | } 23 | 24 | function escapeHtml (value) { 25 | return value.replace(/&/g, '&') 26 | .replace(/"/g, '"') 27 | .replace(//g, '>'); 29 | } 30 | 31 | function formatResult (location, title, summary) { 32 | return '

'+ escapeHtml(title) + '

' + escapeHtml(summary) +'

'; 33 | } 34 | 35 | function displayResults (results) { 36 | var search_results = document.getElementById("mkdocs-search-results"); 37 | while (search_results.firstChild) { 38 | search_results.removeChild(search_results.firstChild); 39 | } 40 | if (results.length > 0){ 41 | for (var i=0; i < results.length; i++){ 42 | var result = results[i]; 43 | var html = formatResult(result.location, result.title, result.summary); 44 | search_results.insertAdjacentHTML('beforeend', html); 45 | } 46 | } else { 47 | var noResultsText = search_results.getAttribute('data-no-results-text'); 48 | if (!noResultsText) { 49 | noResultsText = "No results found"; 50 | } 51 | search_results.insertAdjacentHTML('beforeend', '

' + noResultsText + '

'); 52 | } 53 | } 54 | 55 | function doSearch () { 56 | var query = document.getElementById('mkdocs-search-query').value; 57 | if (query.length > min_search_length) { 58 | if (!window.Worker) { 59 | displayResults(search(query)); 60 | } else { 61 | searchWorker.postMessage({query: query}); 62 | } 63 | } else { 64 | // Clear results for short queries 65 | displayResults([]); 66 | } 67 | } 68 | 69 | function initSearch () { 70 | var search_input = document.getElementById('mkdocs-search-query'); 71 | if (search_input) { 72 | search_input.addEventListener("keyup", doSearch); 73 | } 74 | var term = getSearchTermFromLocation(); 75 | if (term) { 76 | search_input.value = term; 77 | doSearch(); 78 | } 79 | } 80 | 81 | function onWorkerMessage (e) { 82 | if (e.data.allowSearch) { 83 | initSearch(); 84 | } else if (e.data.results) { 85 | var results = e.data.results; 86 | displayResults(results); 87 | } else if (e.data.config) { 88 | min_search_length = e.data.config.min_search_length-1; 89 | } 90 | } 91 | 92 | if (!window.Worker) { 93 | console.log('Web Worker API not supported'); 94 | // load index in main thread 95 | $.getScript(joinUrl(base_url, "search/worker.js")).done(function () { 96 | console.log('Loaded worker'); 97 | init(); 98 | window.postMessage = function (msg) { 99 | onWorkerMessage({data: msg}); 100 | }; 101 | }).fail(function (jqxhr, settings, exception) { 102 | console.error('Could not load worker.js'); 103 | }); 104 | } else { 105 | // Wrap search in a web worker 106 | var searchWorker = new Worker(joinUrl(base_url, "search/worker.js")); 107 | searchWorker.postMessage({init: true}); 108 | searchWorker.onmessage = onWorkerMessage; 109 | } 110 | -------------------------------------------------------------------------------- /site/search/worker.js: -------------------------------------------------------------------------------- 1 | var base_path = 'function' === typeof importScripts ? '.' : '/search/'; 2 | var allowSearch = false; 3 | var index; 4 | var documents = {}; 5 | var lang = ['en']; 6 | var data; 7 | 8 | function getScript(script, callback) { 9 | console.log('Loading script: ' + script); 10 | $.getScript(base_path + script).done(function () { 11 | callback(); 12 | }).fail(function (jqxhr, settings, exception) { 13 | console.log('Error: ' + exception); 14 | }); 15 | } 16 | 17 | function getScriptsInOrder(scripts, callback) { 18 | if (scripts.length === 0) { 19 | callback(); 20 | return; 21 | } 22 | getScript(scripts[0], function() { 23 | getScriptsInOrder(scripts.slice(1), callback); 24 | }); 25 | } 26 | 27 | function loadScripts(urls, callback) { 28 | if( 'function' === typeof importScripts ) { 29 | importScripts.apply(null, urls); 30 | callback(); 31 | } else { 32 | getScriptsInOrder(urls, callback); 33 | } 34 | } 35 | 36 | function onJSONLoaded () { 37 | data = JSON.parse(this.responseText); 38 | var scriptsToLoad = ['lunr.js']; 39 | if (data.config && data.config.lang && data.config.lang.length) { 40 | lang = data.config.lang; 41 | } 42 | if (lang.length > 1 || lang[0] !== "en") { 43 | scriptsToLoad.push('lunr.stemmer.support.js'); 44 | if (lang.length > 1) { 45 | scriptsToLoad.push('lunr.multi.js'); 46 | } 47 | if (lang.includes("ja") || lang.includes("jp")) { 48 | scriptsToLoad.push('tinyseg.js'); 49 | } 50 | for (var i=0; i < lang.length; i++) { 51 | if (lang[i] != 'en') { 52 | scriptsToLoad.push(['lunr', lang[i], 'js'].join('.')); 53 | } 54 | } 55 | } 56 | loadScripts(scriptsToLoad, onScriptsLoaded); 57 | } 58 | 59 | function onScriptsLoaded () { 60 | console.log('All search scripts loaded, building Lunr index...'); 61 | if (data.config && data.config.separator && data.config.separator.length) { 62 | lunr.tokenizer.separator = new RegExp(data.config.separator); 63 | } 64 | 65 | if (data.index) { 66 | index = lunr.Index.load(data.index); 67 | data.docs.forEach(function (doc) { 68 | documents[doc.location] = doc; 69 | }); 70 | console.log('Lunr pre-built index loaded, search ready'); 71 | } else { 72 | index = lunr(function () { 73 | if (lang.length === 1 && lang[0] !== "en" && lunr[lang[0]]) { 74 | this.use(lunr[lang[0]]); 75 | } else if (lang.length > 1) { 76 | this.use(lunr.multiLanguage.apply(null, lang)); // spread operator not supported in all browsers: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_operator#Browser_compatibility 77 | } 78 | this.field('title'); 79 | this.field('text'); 80 | this.ref('location'); 81 | 82 | for (var i=0; i < data.docs.length; i++) { 83 | var doc = data.docs[i]; 84 | this.add(doc); 85 | documents[doc.location] = doc; 86 | } 87 | }); 88 | console.log('Lunr index built, search ready'); 89 | } 90 | allowSearch = true; 91 | postMessage({config: data.config}); 92 | postMessage({allowSearch: allowSearch}); 93 | } 94 | 95 | function init () { 96 | var oReq = new XMLHttpRequest(); 97 | oReq.addEventListener("load", onJSONLoaded); 98 | var index_path = base_path + '/search_index.json'; 99 | if( 'function' === typeof importScripts ){ 100 | index_path = 'search_index.json'; 101 | } 102 | oReq.open("GET", index_path); 103 | oReq.send(); 104 | } 105 | 106 | function search (query) { 107 | if (!allowSearch) { 108 | console.error('Assets for search still loading'); 109 | return; 110 | } 111 | 112 | var resultDocuments = []; 113 | var results = index.search(query); 114 | for (var i=0; i < results.length; i++){ 115 | var result = results[i]; 116 | doc = documents[result.ref]; 117 | doc.summary = doc.text.substring(0, 200); 118 | resultDocuments.push(doc); 119 | } 120 | return resultDocuments; 121 | } 122 | 123 | if( 'function' === typeof importScripts ) { 124 | onmessage = function (e) { 125 | if (e.data.init) { 126 | init(); 127 | } else if (e.data.query) { 128 | postMessage({ results: search(e.data.query) }); 129 | } else { 130 | console.error("Worker - Unrecognized message: " + e); 131 | } 132 | }; 133 | } 134 | -------------------------------------------------------------------------------- /postgres/postgres_table_stats_status.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | WITH table_stats AS ( 16 | SELECT 17 | schemaname, 18 | tablename, 19 | count(1) AS stats_count 20 | FROM 21 | pg_stats 22 | WHERE 23 | schemaname LIKE '%' 24 | GROUP BY 25 | schemaname, 26 | tablename 27 | ) 28 | SELECT 29 | pt.schemaname AS schema_name, 30 | pt.tablename AS table_name, 31 | CASE WHEN pi.inhparent::text IS NULL THEN 32 | NULL 33 | ELSE 34 | pi.inhparent::regclass 35 | END AS top_table_name, 36 | pc.reltuples::numeric AS est_num_live_rows, 37 | ps.n_live_tup AS rows_in_stats, 38 | ps.n_tup_ins AS tot_num_inserts, 39 | ps.n_tup_upd AS tot_num_updates, 40 | ps.n_tup_del AS tot_num_deletes, 41 | ps.n_mod_since_analyze AS modified_rows, 42 | CASE WHEN pc.reltuples::numeric < 0 THEN 43 | 100.00 44 | WHEN (ps.n_tup_ins + ps.n_tup_upd + ps.n_tup_del) = 0 THEN 45 | 0.00 46 | WHEN ps.n_mod_since_analyze::numeric > 0 47 | AND pc.reltuples::numeric > 0 THEN 48 | CASE WHEN (ROUND((ps.n_mod_since_analyze / pc.reltuples)::numeric, 2) * 100) > 100 THEN 49 | 100 50 | ELSE 51 | ROUND((ps.n_mod_since_analyze / pc.reltuples)::numeric, 2) * 100 52 | END 53 | WHEN ps.n_mod_since_analyze::numeric > 0 54 | AND pc.reltuples::numeric = 0 THEN 55 | ROUND((ps.n_mod_since_analyze / 1)::numeric, 2) * 100 56 | WHEN pc.reltuples = 0 57 | AND ps.n_live_tup = 0 58 | AND ps.n_mod_since_analyze = 0 THEN 59 | 0.00 60 | ELSE 61 | ROUND((ps.n_live_tup / pc.reltuples)::numeric - 1, 2) 62 | END AS percent_stale, 63 | CASE WHEN ps.last_analyze IS NULL 64 | AND ps.last_autoanalyze IS NOT NULL THEN 65 | 'Auto - ' || TO_CHAR(ps.last_autoanalyze, 'DD-MON-YY HH24:MI') 66 | WHEN ps.last_analyze IS NULL 67 | AND ts.stats_count IS NULL 68 | AND ps.last_autoanalyze IS NULL THEN 69 | 'No Stats Available' 70 | WHEN ps.last_analyze IS NULL 71 | AND ts.stats_count > 0 72 | AND ps.last_autoanalyze IS NULL THEN 73 | 'pg_stats - Status Unknown' 74 | WHEN ps.last_analyze IS NULL 75 | AND pc.reltuples > 0 76 | AND ps.last_autoanalyze IS NULL THEN 77 | 'pg_class - Status Unknown' 78 | WHEN ps.last_analyze IS NOT NULL 79 | AND ps.last_autoanalyze IS NOT NULL 80 | AND (ps.last_autoanalyze > ps.last_analyze) THEN 81 | 'Auto - ' || TO_CHAR(ps.last_autoanalyze, 'DD-MON-YY HH24:MI') 82 | ELSE 83 | 'Manual - ' || TO_CHAR(ps.last_analyze, 'DD-MON-YY HH24:MI') 84 | END AS last_analyzed, 85 | CASE WHEN ps.last_vacuum IS NULL 86 | AND ps.last_autovacuum IS NOT NULL THEN 87 | 'Auto - ' || TO_CHAR(ps.last_autovacuum, 'DD-MON-YY HH24:MI') 88 | WHEN ps.last_vacuum IS NOT NULL 89 | AND ps.last_autovacuum IS NULL THEN 90 | 'Manual - ' || TO_CHAR(ps.last_vacuum, 'DD-MON-YY HH24:MI') 91 | WHEN ps.last_vacuum > ps.last_autovacuum THEN 92 | 'Manual - ' || TO_CHAR(ps.last_vacuum, 'DD-MON-YY HH24:MI') 93 | ELSE 94 | 'Auto - ' || TO_CHAR(ps.last_autovacuum, 'DD-MON-YY HH24:MI') 95 | END AS vacuum_status 96 | FROM 97 | pg_tables pt 98 | JOIN pg_stat_all_tables ps ON ps.schemaname = pt.schemaname 99 | AND ps.relname = pt.tablename 100 | JOIN pg_class pc ON ps.relid = pc.oid 101 | LEFT JOIN pg_catalog.pg_inherits pi ON ps.relid = pi.inhrelid 102 | LEFT JOIN table_stats ts ON ts.schemaname = pt.schemaname 103 | AND ts.tablename = pt.tablename 104 | WHERE 105 | pt.tablename LIKE '%' 106 | ORDER BY 107 | 3 NULLS LAST, 108 | 1, 109 | 2; -------------------------------------------------------------------------------- /docs/code_of_conduct.md: -------------------------------------------------------------------------------- 1 | # Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as 6 | contributors and maintainers pledge to making participation in our project and 7 | our community a harassment-free experience for everyone, regardless of age, body 8 | size, disability, ethnicity, gender identity and expression, level of 9 | experience, education, socio-economic status, nationality, personal appearance, 10 | race, religion, or sexual identity and orientation. 11 | 12 | ## Our Standards 13 | 14 | Examples of behavior that contributes to creating a positive environment 15 | include: 16 | 17 | * Using welcoming and inclusive language 18 | * Being respectful of differing viewpoints and experiences 19 | * Gracefully accepting constructive criticism 20 | * Focusing on what is best for the community 21 | * Showing empathy towards other community members 22 | 23 | Examples of unacceptable behavior by participants include: 24 | 25 | * The use of sexualized language or imagery and unwelcome sexual attention or 26 | advances 27 | * Trolling, insulting/derogatory comments, and personal or political attacks 28 | * Public or private harassment 29 | * Publishing others' private information, such as a physical or electronic 30 | address, without explicit permission 31 | * Other conduct which could reasonably be considered inappropriate in a 32 | professional setting 33 | 34 | ## Our Responsibilities 35 | 36 | Project maintainers are responsible for clarifying the standards of acceptable 37 | behavior and are expected to take appropriate and fair corrective action in 38 | response to any instances of unacceptable behavior. 39 | 40 | Project maintainers have the right and responsibility to remove, edit, or reject 41 | comments, commits, code, wiki edits, issues, and other contributions that are 42 | not aligned to this Code of Conduct, or to ban temporarily or permanently any 43 | contributor for other behaviors that they deem inappropriate, threatening, 44 | offensive, or harmful. 45 | 46 | ## Scope 47 | 48 | This Code of Conduct applies both within project spaces and in public spaces 49 | when an individual is representing the project or its community. Examples of 50 | representing a project or community include using an official project e-mail 51 | address, posting via an official social media account, or acting as an appointed 52 | representative at an online or offline event. Representation of a project may be 53 | further defined and clarified by project maintainers. 54 | 55 | This Code of Conduct also applies outside the project spaces when the Project 56 | Steward has a reasonable belief that an individual's behavior may have a 57 | negative impact on the project or its community. 58 | 59 | ## Conflict Resolution 60 | 61 | We do not believe that all conflict is bad; healthy debate and disagreement 62 | often yield positive results. However, it is never okay to be disrespectful or 63 | to engage in behavior that violates the project’s code of conduct. 64 | 65 | If you see someone violating the code of conduct, you are encouraged to address 66 | the behavior directly with those involved. Many issues can be resolved quickly 67 | and easily, and this gives people more control over the outcome of their 68 | dispute. If you are unable to resolve the matter for any reason, or if the 69 | behavior is threatening or harassing, report it. We are dedicated to providing 70 | an environment where participants feel welcome and safe. 71 | 72 | Reports should be directed to Cody Fincher & Warren Puziewicz , the 73 | Project Steward(s) for Database Migration Assessment framework. It is the Project Steward’s duty to 74 | receive and address reported violations of the code of conduct. They will then 75 | work with a committee consisting of representatives from the Open Source 76 | Programs Office and the Google Open Source Strategy team. If for any reason you 77 | are uncomfortable reaching out to the Project Steward, please email 78 | opensource@google.com. 79 | 80 | We will investigate every complaint, but you may not receive a direct response. 81 | We will use our discretion in determining when and how to follow up on reported 82 | incidents, which may range from not taking action to permanent expulsion from 83 | the project and project-sponsored spaces. We will notify the accused of the 84 | report and provide them an opportunity to discuss it before any action is taken. 85 | The identity of the reporter will be omitted from the details of the report 86 | supplied to the accused. In potentially harmful situations, such as ongoing 87 | harassment or threats to anyone's safety, we may take action without notice. 88 | 89 | ## Attribution 90 | 91 | This Code of Conduct is adapted from the Contributor Covenant, version 1.4, 92 | available at 93 | 94 | -------------------------------------------------------------------------------- /postgres/postgres_object_sizes.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2025 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | /* Table Sizes */ 16 | SELECT 17 | n.nspname as schemaname, 18 | c.oid::regclass::text AS table_name, 19 | pi.inhparent::regclass::text AS top_table_name, 20 | pg_total_relation_size(c.oid) as size, 21 | pg_size_pretty(pg_total_relation_size(c.oid)) as pretty_size 22 | FROM pg_class c 23 | JOIN pg_namespace n on c.relnamespace = n.oid 24 | LEFT JOIN pg_inherits pi on c.oid = pi.inhrelid 25 | WHERE c.relkind IN ('r', 't', 'm') 26 | AND (n.nspname NOT IN('pg_toast') AND n.nspname LIKE '%') 27 | AND (c.oid::regclass::text LIKE '%' and pi.inhparent::regclass::text LIKE '%') 28 | ORDER BY 2 NULLS LAST; 29 | 30 | SELECT 31 | *, 32 | pg_size_pretty(table_bytes) AS table, 33 | pg_size_pretty(toast_bytes) AS toast, 34 | pg_size_pretty(index_bytes) AS index, 35 | pg_size_pretty(total_bytes) AS total 36 | FROM ( 37 | SELECT 38 | *, total_bytes - index_bytes - COALESCE(toast_bytes, 0) AS table_bytes 39 | FROM ( 40 | SELECT 41 | c.oid, 42 | n.nspname AS table_schema, 43 | c.relname AS table_name, 44 | c.reltuples AS row_estimate, 45 | pct.relname AS toast_table_name, 46 | pg_total_relation_size(c.oid) AS total_bytes, 47 | pg_indexes_size(c.oid) AS index_bytes, 48 | pg_total_relation_size(c.reltoastrelid) AS toast_bytes 49 | FROM 50 | pg_class c 51 | JOIN pg_class pct ON (c.reltoastrelid = pct.oid) 52 | LEFT JOIN pg_namespace n ON n.oid = c.relnamespace 53 | WHERE c.relkind = 'r' 54 | ) a 55 | ) a 56 | WHERE table_schema like '%' 57 | AND table_name like '%' 58 | AND total_bytes > 0 59 | ORDER BY table_name DESC; 60 | 61 | /* Index Sizes */ 62 | SELECT 63 | n.nspname as schemaname, 64 | pgi.tablename as tablename, 65 | c.oid::regclass::text AS index_name, 66 | pi.inhparent::regclass::text AS top_index_name, 67 | pg_total_relation_size(c.oid) as size, 68 | pg_size_pretty(pg_total_relation_size(c.oid)) as pretty_size 69 | FROM pg_class c 70 | JOIN pg_namespace n on c.relnamespace = n.oid 71 | JOIN pg_indexes pgi on pgi.indexname = c.oid::regclass::text and pgi.schemaname = n.nspname 72 | LEFT JOIN pg_inherits pi on c.oid = pi.inhrelid 73 | WHERE c.relkind IN ('i') 74 | AND (n.nspname NOT IN('pg_toast') AND n.nspname LIKE '%') 75 | AND (c.oid::regclass::text LIKE '%' and pi.inhparent::regclass::text LIKE '%') 76 | ORDER BY 4 NULLS LAST; 77 | 78 | /* Temp Table Size */ 79 | SELECT 80 | n.nspname as SchemaName 81 | ,c.relname as RelationName 82 | ,CASE c.relkind 83 | WHEN 'r' THEN 'table' 84 | WHEN 'v' THEN 'view' 85 | WHEN 'i' THEN 'index' 86 | WHEN 'S' THEN 'sequence' 87 | WHEN 's' THEN 'special' 88 | END as RelationType 89 | ,pg_catalog.pg_get_userbyid(c.relowner) as RelationOwner 90 | ,pg_size_pretty(pg_relation_size(n.nspname ||'.'|| c.relname)) as RelationSize 91 | FROM pg_catalog.pg_class c 92 | LEFT JOIN pg_catalog.pg_namespace n 93 | ON n.oid = c.relnamespace 94 | WHERE c.relkind IN ('r','s') 95 | AND (n.nspname !~ '^pg_toast' and nspname like 'pg_temp%') 96 | ORDER BY pg_relation_size(n.nspname ||'.'|| c.relname) DESC; 97 | 98 | /* String Columns + live tuples + table size*/ 99 | SELECT 100 | c.table_catalog, 101 | c.table_schema, 102 | c.table_name, 103 | c.column_name, 104 | c.data_type, 105 | c.character_maximum_length, 106 | st.n_live_tup, 107 | pg_size_pretty(pg_total_relation_size(pc.oid)) as pretty_size 108 | FROM 109 | information_schema.columns c 110 | JOIN pg_stat_all_tables st ON (st.schemaname = c.table_schema AND st.relname = c.table_name) 111 | JOIN pg_class pc ON (pc.relname = c.table_name) 112 | JOIN pg_namespace n ON (pc.relnamespace = n.oid AND n.nspname = c.table_schema) 113 | WHERE 114 | table_schema NOT IN ('dbms_alert', 'dbms_assert', 'dbms_output', 'dbms_pipe', 'dbms_random', 'dbms_utility', 'information_schema', 'oracle', 'pg_catalog', 'pg_toast', 'plunit', 'plvchr', 'plvdate', 'plvlex', 'plvstr', 'plvsubst', 'utl_file') 115 | AND data_type NOT IN ('ARRAY', 'anyarray', 'bigint', 'boolean', 'bytea', 'double precision', 'inet', 'integer', 'interval', 'numeric', 'oid', 'pg_dependencies', 'pg_lsn', 'pg_ndistinct', 'pg_node_tree', 'real', 'regclass', 'regproc', 'regtype', 'smallint', 'timestamp with time zone', 'timestamp without time zone', 'xid') 116 | and c.table_catalog = current_database() 117 | ORDER BY 118 | 1, 119 | 2, 120 | 3; -------------------------------------------------------------------------------- /postgres/postgres_index_bloat_status.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | WITH btree_index_atts AS ( 16 | SELECT nspname, 17 | indexclass.relname as index_name, 18 | indexclass.reltuples, 19 | indexclass.relpages, 20 | indrelid, 21 | indexclass.relam, 22 | tableclass.relname as tablename, 23 | indexrelid as index_oid, 24 | pg_attribute.attname, 25 | pg_attribute.attnum 26 | FROM pg_index 27 | JOIN pg_class AS indexclass ON pg_index.indexrelid = indexclass.oid 28 | JOIN pg_class AS tableclass ON pg_index.indrelid = tableclass.oid 29 | JOIN pg_namespace ON pg_namespace.oid = indexclass.relnamespace 30 | JOIN pg_am ON indexclass.relam = pg_am.oid 31 | JOIN pg_attribute ON pg_attribute.attrelid = tableclass.oid 32 | AND pg_attribute.attnum = ANY(indkey) 33 | WHERE pg_am.amname = 'btree' and indexclass.relpages > 0 34 | AND nspname NOT IN ('pg_catalog','information_schema') 35 | AND tableclass.relname like '%' 36 | ), 37 | index_item_sizes AS ( 38 | SELECT 39 | ind_atts.nspname, ind_atts.index_name, 40 | ind_atts.reltuples, ind_atts.relpages, ind_atts.relam, 41 | indrelid AS table_oid, index_oid, 42 | current_setting('block_size')::numeric AS bs, 43 | 8 AS maxalign, 44 | 24 AS pagehdr, 45 | CASE WHEN max(coalesce(pg_stats.null_frac,0)) = 0 46 | THEN 2 47 | ELSE 6 48 | END AS index_tuple_hdr, 49 | sum( (1-coalesce(pg_stats.null_frac, 0)) * coalesce(pg_stats.avg_width, 1024) ) AS nulldatawidth 50 | FROM btree_index_atts AS ind_atts 51 | left JOIN pg_stats ON pg_stats.schemaname = ind_atts.nspname 52 | -- stats for regular index columns 53 | AND ( (pg_stats.tablename = ind_atts.tablename AND pg_stats.attname = ind_atts.attname) 54 | -- stats for functional indexes 55 | OR (pg_stats.tablename = ind_atts.index_name AND pg_stats.attname = ind_atts.attname)) 56 | WHERE ind_atts.attnum > 0 57 | GROUP BY 1, 2, 3, 4, 5, 6, 7, 8, 9 58 | ), 59 | index_aligned_est AS ( 60 | SELECT maxalign, bs, nspname, index_name, reltuples, 61 | relpages, relam, table_oid, index_oid, 62 | coalesce ( 63 | ceil ( 64 | reltuples * ( 6 65 | + maxalign 66 | - CASE 67 | WHEN index_tuple_hdr%maxalign = 0 THEN maxalign 68 | ELSE index_tuple_hdr%maxalign 69 | END 70 | + nulldatawidth 71 | + maxalign 72 | - CASE /* Add padding to the data to align on MAXALIGN */ 73 | WHEN nulldatawidth::integer%maxalign = 0 THEN maxalign 74 | ELSE nulldatawidth::integer%maxalign 75 | END 76 | )::numeric 77 | / ( bs - pagehdr::NUMERIC ) 78 | +1 ) 79 | , 0 ) 80 | as expected 81 | FROM index_item_sizes 82 | ), 83 | raw_bloat AS ( 84 | SELECT current_database() as dbname, nspname, pg_class.relname AS table_name, index_name, 85 | bs*(index_aligned_est.relpages)::bigint AS totalbytes, expected, 86 | CASE 87 | WHEN index_aligned_est.relpages <= expected 88 | THEN 0 89 | ELSE bs*(index_aligned_est.relpages-expected)::bigint 90 | END AS wastedbytes, 91 | CASE 92 | WHEN index_aligned_est.relpages <= expected 93 | THEN 0 94 | ELSE bs*(index_aligned_est.relpages-expected)::bigint * 100 / (bs*(index_aligned_est.relpages)::bigint) 95 | END AS realbloat, 96 | pg_relation_size(index_aligned_est.table_oid) as table_bytes, 97 | stat.idx_scan as index_scans 98 | FROM index_aligned_est 99 | JOIN pg_class ON pg_class.oid=index_aligned_est.table_oid 100 | JOIN pg_stat_user_indexes AS stat ON index_aligned_est.index_oid = stat.indexrelid 101 | ), 102 | format_bloat AS ( 103 | SELECT dbname as database_name, nspname as schema_name, table_name, index_name, 104 | round(realbloat) as bloat_pct, round(wastedbytes/(1024^2)::NUMERIC) as bloat_mb, 105 | round(totalbytes/(1024^2)::NUMERIC,3) as index_mb, 106 | round(table_bytes/(1024^2)::NUMERIC,3) as table_mb, 107 | index_scans 108 | FROM raw_bloat 109 | ) 110 | -- final query outputting the bloated indexes 111 | -- change the where and order by to change 112 | -- what shows up as bloated 113 | SELECT * 114 | FROM format_bloat 115 | --WHERE ( bloat_pct > 50 and bloat_mb > 10 ) 116 | --WHERE index_name like '%_user_id_idx' 117 | ORDER BY bloat_pct DESC NULLS LAST; -------------------------------------------------------------------------------- /postgres/postgres_table_and_index_bloat_stats.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT 16 | current_database(), 17 | schemaname, 18 | tablename, 19 | /*reltuples::bigint, relpages::bigint, otta,*/ 20 | ROUND(( 21 | CASE WHEN otta = 0 THEN 22 | 0.0 23 | ELSE 24 | sml.relpages::float / otta 25 | END)::numeric, 1) AS tbloat, 26 | CASE WHEN relpages < otta THEN 27 | 0 28 | ELSE 29 | bs * (sml.relpages - otta)::bigint 30 | END AS wastedbytes, 31 | iname, 32 | /*ituples::bigint, ipages::bigint, iotta,*/ 33 | ROUND(( 34 | CASE WHEN iotta = 0 35 | OR ipages = 0 THEN 36 | 0.0 37 | ELSE 38 | ipages::float / iotta 39 | END)::numeric, 1) AS ibloat, 40 | CASE WHEN ipages < iotta THEN 41 | 0 42 | ELSE 43 | bs * (ipages - iotta) 44 | END AS wastedibytes 45 | FROM ( 46 | SELECT 47 | schemaname, 48 | tablename, 49 | cc.reltuples, 50 | cc.relpages, 51 | bs, 52 | CEIL((cc.reltuples * ((datahdr + ma - ( 53 | CASE WHEN datahdr % ma = 0 THEN 54 | ma 55 | ELSE 56 | datahdr % ma 57 | END)) + nullhdr2 + 4)) / (bs - 20::float)) AS otta, 58 | COALESCE(c2.relname, '?') AS iname, 59 | COALESCE(c2.reltuples, 0) AS ituples, 60 | COALESCE(c2.relpages, 0) AS ipages, 61 | COALESCE(CEIL((c2.reltuples * (datahdr - 12)) / (bs - 20::float)), 0) AS iotta -- very rough approximation, assumes all cols 62 | FROM ( 63 | SELECT 64 | ma, 65 | bs, 66 | schemaname, 67 | tablename, 68 | (datawidth + (hdr + ma - ( 69 | CASE WHEN hdr % ma = 0 THEN 70 | ma 71 | ELSE 72 | hdr % ma 73 | END)))::numeric AS datahdr, 74 | (maxfracsum * (nullhdr + ma - ( 75 | CASE WHEN nullhdr % ma = 0 THEN 76 | ma 77 | ELSE 78 | nullhdr % ma 79 | END))) AS nullhdr2 80 | FROM ( 81 | SELECT 82 | schemaname, 83 | tablename, 84 | hdr, 85 | ma, 86 | bs, 87 | SUM((1 - null_frac) * avg_width) AS datawidth, 88 | MAX(null_frac) AS maxfracsum, 89 | hdr + ( 90 | SELECT 91 | 1 + count(*) / 8 92 | FROM 93 | pg_stats s2 94 | WHERE 95 | null_frac <> 0 96 | AND s2.schemaname = s.schemaname 97 | AND s2.tablename = s.tablename) AS nullhdr 98 | FROM 99 | pg_stats s, 100 | ( 101 | SELECT 102 | ( 103 | SELECT 104 | current_setting('block_size')::numeric) AS bs, 105 | CASE WHEN substring(v, 12, 3) IN ('8.0', '8.1', '8.2') THEN 106 | 27 107 | ELSE 108 | 23 109 | END AS hdr, 110 | CASE WHEN v ~ 'mingw32' THEN 111 | 8 112 | ELSE 113 | 4 114 | END AS ma 115 | FROM ( 116 | SELECT 117 | version() AS v) AS foo) AS constants 118 | GROUP BY 119 | 1, 120 | 2, 121 | 3, 122 | 4, 123 | 5) AS foo) AS rs 124 | JOIN pg_class cc ON cc.relname = rs.tablename 125 | JOIN pg_namespace nn ON cc.relnamespace = nn.oid 126 | AND nn.nspname = rs.schemaname 127 | AND nn.nspname <> 'information_schema' 128 | LEFT JOIN pg_index i ON indrelid = cc.oid 129 | LEFT JOIN pg_class c2 ON c2.oid = i.indexrelid) AS sml 130 | WHERE 131 | ROUND(( 132 | CASE WHEN otta = 0 THEN 133 | 0.0 134 | ELSE 135 | sml.relpages::float / otta 136 | END)::numeric, 1) > 0 137 | ORDER BY 138 | 2,4,3 DESC; 139 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | 16 | 17 | # Code of Conduct 18 | 19 | ## Our Pledge 20 | 21 | In the interest of fostering an open and welcoming environment, we as 22 | contributors and maintainers pledge to making participation in our project and 23 | our community a harassment-free experience for everyone, regardless of age, body 24 | size, disability, ethnicity, gender identity and expression, level of 25 | experience, education, socio-economic status, nationality, personal appearance, 26 | race, religion, or sexual identity and orientation. 27 | 28 | ## Our Standards 29 | 30 | Examples of behavior that contributes to creating a positive environment 31 | include: 32 | 33 | * Using welcoming and inclusive language 34 | * Being respectful of differing viewpoints and experiences 35 | * Gracefully accepting constructive criticism 36 | * Focusing on what is best for the community 37 | * Showing empathy towards other community members 38 | 39 | Examples of unacceptable behavior by participants include: 40 | 41 | * The use of sexualized language or imagery and unwelcome sexual attention or 42 | advances 43 | * Trolling, insulting/derogatory comments, and personal or political attacks 44 | * Public or private harassment 45 | * Publishing others' private information, such as a physical or electronic 46 | address, without explicit permission 47 | * Other conduct which could reasonably be considered inappropriate in a 48 | professional setting 49 | 50 | ## Our Responsibilities 51 | 52 | Project maintainers are responsible for clarifying the standards of acceptable 53 | behavior and are expected to take appropriate and fair corrective action in 54 | response to any instances of unacceptable behavior. 55 | 56 | Project maintainers have the right and responsibility to remove, edit, or reject 57 | comments, commits, code, wiki edits, issues, and other contributions that are 58 | not aligned to this Code of Conduct, or to ban temporarily or permanently any 59 | contributor for other behaviors that they deem inappropriate, threatening, 60 | offensive, or harmful. 61 | 62 | ## Scope 63 | 64 | This Code of Conduct applies both within project spaces and in public spaces 65 | when an individual is representing the project or its community. Examples of 66 | representing a project or community include using an official project e-mail 67 | address, posting via an official social media account, or acting as an appointed 68 | representative at an online or offline event. Representation of a project may be 69 | further defined and clarified by project maintainers. 70 | 71 | This Code of Conduct also applies outside the project spaces when the Project 72 | Steward has a reasonable belief that an individual's behavior may have a 73 | negative impact on the project or its community. 74 | 75 | ## Conflict Resolution 76 | 77 | We do not believe that all conflict is bad; healthy debate and disagreement 78 | often yield positive results. However, it is never okay to be disrespectful or 79 | to engage in behavior that violates the project’s code of conduct. 80 | 81 | If you see someone violating the code of conduct, you are encouraged to address 82 | the behavior directly with those involved. Many issues can be resolved quickly 83 | and easily, and this gives people more control over the outcome of their 84 | dispute. If you are unable to resolve the matter for any reason, or if the 85 | behavior is threatening or harassing, report it. We are dedicated to providing 86 | an environment where participants feel welcome and safe. 87 | 88 | Reports should be directed to *[PROJECT STEWARD NAME(s) AND EMAIL(s)]*, the 89 | Project Steward(s) for *[PROJECT NAME]*. It is the Project Steward’s duty to 90 | receive and address reported violations of the code of conduct. They will then 91 | work with a committee consisting of representatives from the Open Source 92 | Programs Office and the Google Open Source Strategy team. If for any reason you 93 | are uncomfortable reaching out to the Project Steward, please email 94 | opensource@google.com. 95 | 96 | We will investigate every complaint, but you may not receive a direct response. 97 | We will use our discretion in determining when and how to follow up on reported 98 | incidents, which may range from not taking action to permanent expulsion from 99 | the project and project-sponsored spaces. We will notify the accused of the 100 | report and provide them an opportunity to discuss it before any action is taken. 101 | The identity of the reporter will be omitted from the details of the report 102 | supplied to the accused. In potentially harmful situations, such as ongoing 103 | harassment or threats to anyone's safety, we may take action without notice. 104 | 105 | ## Attribution 106 | 107 | This Code of Conduct is adapted from the Contributor Covenant, version 1.4, 108 | available at 109 | https://www.contributor-covenant.org/version/1/4/code-of-conduct.html 110 | 111 | Note: A version of this file is also available in the 112 | [New Project repo](https://github.com/google/new-project/blob/master/docs/code-of-conduct.md). -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | .DEFAULT_GOAL:=help 2 | .ONESHELL: 3 | VENV_EXISTS=$(shell python3 -c "if __import__('pathlib').Path('.venv/bin/activate').exists(): print('yes')") 4 | VERSION := $(shell grep -m 1 current_version .bumpversion.cfg | tr -s ' ' | tr -d '"' | tr -d "'" | cut -d' ' -f3) 5 | BUILD_DIR=dist 6 | COLLECTOR_PACKAGE=sql-scripts 7 | BASE_DIR=$(shell pwd) 8 | 9 | .EXPORT_ALL_VARIABLES: 10 | 11 | ifndef VERBOSE 12 | .SILENT: 13 | endif 14 | 15 | REPO_INFO ?= $(shell git config --get remote.origin.url) 16 | COMMIT_SHA ?= git-$(shell git rev-parse --short HEAD) 17 | 18 | help: ## Display this help 19 | @awk 'BEGIN {FS = ":.*##"; printf "\nUsage:\n make \033[36m\033[0m\n"} /^[a-zA-Z0-9_-]+:.*?##/ { printf " \033[36m%-15s\033[0m %s\n", $$1, $$2 } /^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) } ' $(MAKEFILE_LIST) 20 | 21 | .PHONY: install 22 | install: ## Install the project in dev mode. 23 | @if [ "$(VENV_EXISTS)" ]; then source .venv/bin/activate; fi 24 | @if [ ! "$(VENV_EXISTS)" ]; then python3 -m venv .venv && source .venv/bin/activate; fi 25 | .venv/bin/pip install -U wheel setuptools cython pip mypy sqlfluff && .venv/bin/pip install -U -r requirements.txt -r requirements-docs.txt 26 | @echo "=> Build environment installed successfully. ** If you want to re-install or update, 'make install'" 27 | 28 | .PHONY: clean 29 | clean: ## Cleanup temporary build artifacts 30 | @echo "${INFO} Cleaning working directory... 🧹" 31 | @rm -rf .pytest_cache .ruff_cache .hypothesis build/ dist/ .eggs/ .coverage coverage.xml coverage.json htmlcov/ .pytest_cache tests/.pytest_cache tests/**/.pytest_cache .mypy_cache .unasyncd_cache/ .auto_pytabs_cache node_modules >/dev/null 2>&1 32 | @find . -name '*.egg-info' -exec rm -rf {} + >/dev/null 2>&1 33 | @find . -type f -name '*.egg' -exec rm -f {} + >/dev/null 2>&1 34 | @find . -name '*.pyc' -exec rm -f {} + >/dev/null 2>&1 35 | @find . -name '*.pyo' -exec rm -f {} + >/dev/null 2>&1 36 | @find . -name '*~' -exec rm -f {} + >/dev/null 2>&1 37 | @find . -name '__pycache__' -exec rm -rf {} + >/dev/null 2>&1 38 | @find . -name '.ipynb_checkpoints' -exec rm -rf {} + >/dev/null 2>&1 39 | @echo "${OK} Working directory cleaned" 40 | 41 | .PHONY: clean-sqlscripts 42 | clean-sqlscripts: 43 | @echo "=> Cleaning previous build artifacts for sql scripts..." 44 | @rm -Rf $(BUILD_DIR)/collector/* 45 | 46 | .PHONY: build-sqlscripts 47 | build-sqlscripts: clean-sqlscripts ## Build the collector SQL scripts. 48 | @rm -rf ./$(BUILD_DIR)/collector 49 | python3 -c "import m2r; python_text = m2r.convert(open('README.md').read()); f = open('README.txt', 'w'); f.write(python_text); f.close()" 50 | @echo "=> Building SQL Helper Scripts for Oracle version $(VERSION)..." 51 | python3 -c "import m2r; python_text = m2r.convert(open('oracle/README.md').read()); f = open('oracle/README.txt', 'w'); f.write(python_text); f.close()" 52 | @mkdir -p $(BUILD_DIR)/oracle 53 | @cp ./oracle/*.sql $(BUILD_DIR)/oracle 54 | @cp LICENSE $(BUILD_DIR)/oracle 55 | echo "SQL Helper Scripts for Oracle Database version $(VERSION) ($(COMMIT_SHA))" > $(BUILD_DIR)/oracle/VERSION.txt 56 | 57 | @echo "=> Building SQL Helper Scripts for Microsoft SQL Server version $(VERSION)..." 58 | python3 -c "import m2r; python_text = m2r.convert(open('mssql/README.md').read()); f = open('mssql/README.txt', 'w'); f.write(python_text); f.close()" 59 | @mkdir -p $(BUILD_DIR)/sqlserver 60 | @cp mssql/*.sql $(BUILD_DIR)/sqlserver 61 | @cp mssql/README.txt $(BUILD_DIR)/sqlserver 62 | @cp LICENSE $(BUILD_DIR)/sqlserver 63 | @echo "SQL Helper Scripts for Microsoft SQL Server version $(VERSION) ($(COMMIT_SHA))" > $(BUILD_DIR)/sqlserver/VERSION.txt 64 | 65 | @echo "=> Building SQL Helper Scripts for Postgresql version $(VERSION)..." 66 | python3 -c "import m2r; python_text = m2r.convert(open('postgres/README.md').read()); f = open('postgres/README.txt', 'w'); f.write(python_text); f.close()" 67 | @mkdir -p $(BUILD_DIR)/postgres 68 | @cp postgres/*.sql $(BUILD_DIR)/postgres 69 | @cp postgres/.psqlrc $(BUILD_DIR)/postgres 70 | @cp postgres/README.txt $(BUILD_DIR)/postgres 71 | @cp postgres/README.md $(BUILD_DIR)/postgres 72 | @cp LICENSE $(BUILD_DIR)/postgres 73 | @echo "SQL Helper Scripts for Postgres version $(VERSION) ($(COMMIT_SHA))" > $(BUILD_DIR)/postgres/VERSION.txt 74 | 75 | @make package-sqlscripts 76 | 77 | .PHONY: package-sqlscripts 78 | package-sqlscripts: 79 | @rm -f ./$(BUILD_DIR)/$(COLLECTOR_PACKAGE)*.bz2 80 | @rm -f ./$(BUILD_DIR)/$(COLLECTOR_PACKAGE)*.zip 81 | 82 | @echo "=> Packaging SQL Scripts for Oracle..." 83 | @echo "Zipping files in ./$(BUILD_DIR)/oracle" 84 | @cd $(BASE_DIR)/$(BUILD_DIR)/oracle; zip -r $(BASE_DIR)/$(BUILD_DIR)/$(COLLECTOR_PACKAGE)-oracle.zip * 85 | 86 | @echo "=> Packaging SQL Scripts for Microsoft SQL Server..." 87 | @echo "Zipping files in ./$(BUILD_DIR)/sqlserver" 88 | @cd $(BASE_DIR)/$(BUILD_DIR)/sqlserver; zip -r $(BASE_DIR)/$(BUILD_DIR)/$(COLLECTOR_PACKAGE)-sqlserver.zip * 89 | 90 | @echo "=> Packaging SQL Scripts for Postgres..." 91 | @echo "Zipping files in ./$(BUILD_DIR)/postgres" 92 | @cd $(BASE_DIR)/$(BUILD_DIR)/postgres; zip -r $(BASE_DIR)/$(BUILD_DIR)/$(COLLECTOR_PACKAGE)-postgres.zip * 93 | 94 | .PHONY: build 95 | build: build-sqlscripts ## Build and package the collectors 96 | 97 | 98 | ############### 99 | # docs # 100 | ############### 101 | .PHONY: doc-privs 102 | doc-privs: ## Extract the list of privileges required from code and create the documentation 103 | cat > docs/user_guide/oracle/permissions.md <> docs/user_guide/oracle/permissions.md 116 | 117 | .PHONY: gen-docs 118 | gen-docs: ## generate HTML documentation 119 | ./.venv/bin/mkdocs build 120 | 121 | .PHONY: docs 122 | docs: ## generate HTML documentation and serve it to the browser 123 | ./.venv/bin/mkdocs build 124 | ./.venv/bin/mkdocs serve 125 | 126 | .PHONY: pre-release 127 | pre-release: ## bump the version and create the release tag 128 | make gen-docs 129 | make clean 130 | .venv/bin/bump2version $(increment) 131 | head .bumpversion.cfg | grep ^current_version 132 | make build 133 | -------------------------------------------------------------------------------- /postgres/postgres_table_bloat_check.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2025 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | -- While this script has been slightly modified, 16 | -- original credit for this script is credited to PGExperts / Citus Data 17 | 18 | WITH constants AS ( 19 | -- define some constants for sizes of things 20 | -- for reference down the query and easy maintenance 21 | SELECT current_setting('block_size')::numeric AS bs, 23 AS hdr, 8 AS ma 22 | ), 23 | no_stats AS ( 24 | -- screen out table who have attributes 25 | -- which dont have stats, such as JSON 26 | SELECT table_schema, table_name, 27 | n_live_tup::numeric as est_rows, 28 | pg_table_size(relid)::numeric as table_size 29 | FROM information_schema.columns 30 | JOIN pg_stat_user_tables as psut 31 | ON table_schema = psut.schemaname 32 | AND table_name = psut.relname 33 | LEFT OUTER JOIN pg_stats 34 | ON table_schema = pg_stats.schemaname 35 | AND table_name = pg_stats.tablename 36 | AND column_name = attname 37 | WHERE attname IS NULL 38 | AND table_schema NOT IN ('pg_catalog', 'information_schema') 39 | GROUP BY table_schema, table_name, relid, n_live_tup 40 | ), 41 | null_headers AS ( 42 | -- calculate null header sizes 43 | -- omitting tables which dont have complete stats 44 | -- and attributes which aren't visible 45 | SELECT 46 | hdr+1+(sum(case when null_frac <> 0 THEN 1 else 0 END)/8) as nullhdr, 47 | SUM((1-null_frac)*avg_width) as datawidth, 48 | MAX(null_frac) as maxfracsum, 49 | schemaname, 50 | tablename, 51 | hdr, ma, bs 52 | FROM pg_stats CROSS JOIN constants 53 | LEFT OUTER JOIN no_stats 54 | ON schemaname = no_stats.table_schema 55 | AND tablename = no_stats.table_name 56 | WHERE schemaname NOT IN ('pg_catalog', 'information_schema') 57 | AND no_stats.table_name IS NULL 58 | AND EXISTS ( SELECT 1 59 | FROM information_schema.columns 60 | WHERE schemaname = columns.table_schema 61 | AND tablename = columns.table_name ) 62 | GROUP BY schemaname, tablename, hdr, ma, bs 63 | ), 64 | data_headers AS ( 65 | -- estimate header and row size 66 | SELECT 67 | ma, bs, hdr, schemaname, tablename, 68 | (datawidth+(hdr+ma-(case when hdr%ma=0 THEN ma ELSE hdr%ma END)))::numeric AS datahdr, 69 | (maxfracsum*(nullhdr+ma-(case when nullhdr%ma=0 THEN ma ELSE nullhdr%ma END))) AS nullhdr2 70 | FROM null_headers 71 | ), 72 | table_estimates AS ( 73 | -- make estimates of how large the table should be 74 | -- based on row and page size 75 | SELECT schemaname, tablename, bs, 76 | reltuples::numeric as est_rows, relpages * bs as table_bytes, 77 | CEIL((reltuples* 78 | (datahdr + nullhdr2 + 4 + ma - 79 | (CASE WHEN datahdr%ma=0 80 | THEN ma ELSE datahdr%ma END) 81 | )/(bs-20))) * bs AS expected_bytes, 82 | reltoastrelid 83 | FROM data_headers 84 | JOIN pg_class ON tablename = relname 85 | JOIN pg_namespace ON relnamespace = pg_namespace.oid 86 | AND schemaname = nspname 87 | WHERE pg_class.relkind = 'r' 88 | ), 89 | estimates_with_toast AS ( 90 | -- add in estimated TOAST table sizes 91 | -- estimate based on 4 toast tuples per page because we dont have 92 | -- anything better. also append the no_data tables 93 | SELECT schemaname, tablename, 94 | TRUE as can_estimate, 95 | est_rows, 96 | table_bytes + ( coalesce(toast.relpages, 0) * bs ) as table_bytes, 97 | expected_bytes + ( ceil( coalesce(toast.reltuples, 0) / 4 ) * bs ) as expected_bytes 98 | FROM table_estimates LEFT OUTER JOIN pg_class as toast 99 | ON table_estimates.reltoastrelid = toast.oid 100 | AND toast.relkind = 't' 101 | ), 102 | table_estimates_plus AS ( 103 | -- add some extra metadata to the table data 104 | -- and calculations to be reused 105 | -- including whether we cant estimate it 106 | -- or whether we think it might be compressed 107 | SELECT current_database() as databasename, 108 | schemaname, tablename, can_estimate, 109 | est_rows, 110 | CASE WHEN table_bytes > 0 111 | THEN table_bytes::NUMERIC 112 | ELSE NULL::NUMERIC END 113 | AS table_bytes, 114 | CASE WHEN expected_bytes > 0 115 | THEN expected_bytes::NUMERIC 116 | ELSE NULL::NUMERIC END 117 | AS expected_bytes, 118 | CASE WHEN expected_bytes > 0 AND table_bytes > 0 119 | AND expected_bytes <= table_bytes 120 | THEN (table_bytes - expected_bytes)::NUMERIC 121 | ELSE 0::NUMERIC END AS bloat_bytes 122 | FROM estimates_with_toast 123 | UNION ALL 124 | SELECT current_database() as databasename, 125 | table_schema, table_name, FALSE, 126 | est_rows, table_size, 127 | NULL::NUMERIC, NULL::NUMERIC 128 | FROM no_stats 129 | ), 130 | bloat_data AS ( 131 | -- do final math calculations and formatting 132 | select current_database() as databasename, 133 | schemaname, tablename, can_estimate, 134 | table_bytes, round(table_bytes/(1024^2)::NUMERIC,3) as table_mb, 135 | expected_bytes, round(expected_bytes/(1024^2)::NUMERIC,3) as expected_mb, 136 | round(bloat_bytes*100/table_bytes) as pct_bloat, 137 | round(bloat_bytes/(1024::NUMERIC^2),2) as mb_bloat, 138 | table_bytes, expected_bytes, est_rows 139 | FROM table_estimates_plus 140 | ) 141 | -- filter output for bloated tables 142 | SELECT databasename, schemaname, tablename, 143 | can_estimate, 144 | est_rows, 145 | pct_bloat, mb_bloat, 146 | table_mb 147 | FROM bloat_data 148 | -- this where clause defines which tables actually appear 149 | -- in the bloat chart 150 | -- example below filters for tables which are either 50% 151 | -- bloated and more than 20mb in size, or more than 25% 152 | -- bloated and more than 4GB in size 153 | --WHERE ( pct_bloat >= 50 AND mb_bloat >= 10 ) 154 | -- OR ( pct_bloat >= 25 AND mb_bloat >= 1000 ) 155 | -- WHERE tablename = '%' 156 | ORDER BY mb_bloat DESC NULLS LAST; 157 | -------------------------------------------------------------------------------- /postgres/README.md: -------------------------------------------------------------------------------- 1 | # Postgres SQL Scripts - Shorthand Commands for PSQL 2 | 3 | [//]: # (Original Layout for this Readme.md was from https://gist.github.com/apolloclark) 4 | 5 | ## Use of .psqlrc 6 | The .psqlrc file allows you to configure personalized settings, add shortcuts and add custom commands. It is stored in the users home directory and will automatically be read upon launch of `psql`. 7 | 8 | An example of a .psqlrc file can be found here: 9 | - Sample .psqlrc file for example .psqlrc content. 10 | 11 | ##### Ignore .psqlrc 12 | ```sql 13 | psql --username= --dbname= --host= --port= --no-psqlrc 14 | ``` 15 | 16 |
17 | 18 | ## General psql navigation 19 | ##### Connect 20 | Password can be exported at the command line by using: 21 | ```bash 22 | export PGPASSWORD= 23 | ``` 24 | General connections can be made using: 25 | ```sql 26 | psql -U -d -h -p 27 | 28 | psql --username= --dbname= --host= --port= 29 | 30 | psql -U -d -h -f 31 | 32 | psql --username= --dbname= --host= --file= 33 | ``` 34 | 35 | ##### Disconnect 36 | ```sql 37 | \q 38 | \! 39 | ``` 40 | 41 | ##### Clear scrollback 42 | ```sql 43 | (CTRL + L) 44 | ``` 45 | 46 | ##### Set Resultset to "Unaligned" 47 | ```sql 48 | \a 49 | ``` 50 | 51 | ##### Set Resultset to "Extended" 52 | ```sql 53 | \x 54 | ``` 55 |
56 | 57 | ## Information about the connected system 58 | 59 | ##### Server version 60 | ``` 61 | SHOW SERVER_VERSION; 62 | ``` 63 | 64 | ##### Show connection information (SSL USED? PSQL VERSION?) 65 | ```sql 66 | \conninfo 67 | ``` 68 | 69 | ##### Show environment variables 70 | ```sql 71 | SHOW ALL; 72 | ``` 73 | 74 | ##### List all roles in the instance 75 | ```sql 76 | SELECT rolname FROM pg_roles; 77 | ``` 78 | 79 | ##### Show currently connected user 80 | ```sql 81 | SELECT current_user; 82 | ``` 83 | 84 | ##### Show current user permissions 85 | ```sql 86 | \du 87 | \du+ 88 | ``` 89 | 90 | ##### Show current user's session settings 91 | ```sql 92 | \drds 93 | ``` 94 | 95 | ##### show current database 96 | ```sql 97 | SELECT current_database(); 98 | ``` 99 | 100 | ##### show all tables in database 101 | ```sql 102 | \dt 103 | \dt+ 104 | ``` 105 | 106 | ##### list functions 107 | ```sql 108 | \df 109 | \df+ 110 | ``` 111 |
112 | 113 | ## Databases 114 | 115 | ##### list databases with size information 116 | ```sql 117 | \l 118 | \l+ 119 | ``` 120 | 121 | ##### Connect to database 122 | ```sql 123 | \c 124 | ``` 125 | 126 | ##### Connect to database as different user 127 | ```sql 128 | \c 129 | ``` 130 | 131 |
132 | 133 | ## Users 134 | 135 | ##### create user 136 | http://www.postgresql.org/docs/current/static/sql-createuser.html 137 | ```sql 138 | CREATE USER WITH PASSWORD ''; 139 | ``` 140 | 141 | ##### drop user 142 | http://www.postgresql.org/docs/current/static/sql-dropuser.html 143 | ```sql 144 | DROP USER IF EXISTS ; 145 | ``` 146 | 147 | ##### alter user password 148 | http://www.postgresql.org/docs/current/static/sql-alterrole.html 149 | ```sql 150 | ALTER ROLE WITH PASSWORD ''; 151 | ``` 152 |
153 | 154 | ## Permissions 155 | 156 | ##### Grant all permissions on database 157 | http://www.postgresql.org/docs/current/static/sql-grant.html 158 | ```sql 159 | GRANT ALL PRIVILEGES ON DATABASE TO ; 160 | ``` 161 | 162 | ##### Grant connect permission on database 163 | ```sql 164 | GRANT CONNECT ON DATABASE TO ; 165 | ``` 166 | 167 | ##### Grant individual permissions on schema 168 | ```sql 169 | GRANT USAGE ON SCHEMA public TO ; 170 | ``` 171 | 172 | ##### Grant permissions to functions 173 | ```sql 174 | GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA public TO ; 175 | ``` 176 | 177 | ##### Grant permissions to select, update, insert, delete, on all tables in a schema 178 | ```sql 179 | GRANT SELECT, UPDATE, INSERT ON ALL TABLES IN SCHEMA public TO ; 180 | ``` 181 | 182 | ##### Grant all permissions on an individual table 183 | ```sql 184 | GRANT SELECT, UPDATE, INSERT ON TO ; 185 | ``` 186 | 187 | ##### Grant a single permission on an individual table 188 | ```sql 189 | GRANT SELECT ON ALL TABLES IN SCHEMA public TO ; 190 | ``` 191 |
192 | 193 | ## Schema 194 | 195 | ##### List schemas 196 | ```sql 197 | \dn 198 | 199 | \dn+ 200 | 201 | SELECT schema_name FROM information_schema.schemata; 202 | 203 | SELECT nspname FROM pg_catalog.pg_namespace; 204 | ``` 205 | 206 | ##### Create a schema 207 | http://www.postgresql.org/docs/current/static/sql-createschema.html 208 | ```sql 209 | CREATE SCHEMA IF NOT EXISTS ; 210 | ``` 211 | 212 | ##### Drop a schema 213 | http://www.postgresql.org/docs/current/static/sql-dropschema.html 214 | ```sql 215 | DROP SCHEMA IF EXISTS CASCADE; 216 | ``` 217 |
218 | 219 | ## Tables 220 | 221 | ##### List all tables, in current db (limited by your search_path) 222 | ```sql 223 | \dt 224 | \dt+ 225 | 226 | SELECT table_schema,table_name FROM information_schema.tables ORDER BY table_schema,table_name; 227 | ``` 228 | 229 | ##### List tables, globally (not limited by search_path) 230 | ```sql 231 | \dt *.* 232 | 233 | SELECT * FROM pg_catalog.pg_tables 234 | ``` 235 | 236 | ##### List table schema 237 | ```sql 238 | \d 239 | \d+ 240 | 241 | SELECT column_name, data_type, character_maximum_length 242 | FROM INFORMATION_SCHEMA.COLUMNS 243 | WHERE table_name = ''; 244 | ``` 245 | 246 |
247 | 248 | ## Columns 249 | 250 | ##### Add a column 251 | ```sql 252 | ALTER TABLE IF EXISTS 253 | ADD []; 254 | ``` 255 | 256 | ##### Change a column datatype 257 | ```sql 258 | ALTER TABLE IF EXISTS 259 | ALTER TYPE []; 260 | ``` 261 | 262 | ##### Delete a column (beware of table locks) 263 | ```sql 264 | ALTER TABLE IF EXISTS 265 | DROP ; 266 | ``` 267 |
268 | 269 | ## Scripting 270 | 271 | ##### Export table into CSV file 272 | http://www.postgresql.org/docs/current/static/sql-copy.html 273 | ```sql 274 | \copy TO '' CSV 275 | ``` 276 | 277 | ##### Export table, only specific columns, to CSV file 278 | ```sql 279 | \copy (,,) TO '' CSV 280 | ``` 281 | 282 | ##### Import CSV file into table 283 | http://www.postgresql.org/docs/current/static/sql-copy.html 284 | ```sql 285 | \copy FROM '' CSV 286 | ``` 287 | 288 | ##### Import CSV file into table, only specific columns 289 | ```sql 290 | \copy (,,) FROM '' CSV 291 | ``` 292 |
293 | 294 | 295 | 296 | -------------------------------------------------------------------------------- /postgres/postgres_vacuum_activity_estimate_stats.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | /* Vaccum Stats Script */ 16 | 17 | WITH tbl_reloptions AS ( 18 | SELECT 19 | oid, 20 | oid::regclass table_name, 21 | substr(unnest(reloptions), 1, strpos(unnest(reloptions), '=') -1) option, 22 | substr(unnest(reloptions), 1 + strpos(unnest(reloptions), '=')) value 23 | FROM 24 | pg_class c 25 | WHERE reloptions is NOT null) 26 | SELECT 27 | --to_char(now(), 'YYYY-MM-DD HH:MI'), 28 | current_database() ||'.'|| s.schemaname ||'.'|| s.relname as relname, 29 | n_live_tup live_tup, 30 | n_dead_tup dead_dup, 31 | n_tup_hot_upd hot_upd, 32 | n_mod_since_analyze mod_since_stats, 33 | n_ins_since_vacuum ins_since_vac, 34 | case 35 | when avacinsscalefactor.value is not null and avacinsthresh.value is not null 36 | then ROUND(((n_live_tup * avacinsscalefactor.value::numeric) + avacinsthresh.value::numeric),0) 37 | when avacinsscalefactor.value is null and avacinsthresh.value is not null 38 | then ROUND(((n_live_tup * (select setting::numeric from pg_settings where name = 'autovacuum_vacuum_insert_scale_factor')) + avacinsthresh.value::numeric),0) 39 | when avacinsscalefactor.value is not null and avacinsthresh.value is null 40 | then ROUND(((n_live_tup * avacinsscalefactor.value::numeric) + (select setting::numeric from pg_settings where name = 'autovacuum_vacuum_insert_threshold')),0) 41 | else ROUND(((n_live_tup * (select setting::numeric from pg_settings where name = 'autovacuum_vacuum_insert_scale_factor')) + (select setting::numeric from pg_settings where name = 'autovacuum_vacuum_insert_threshold')),0) 42 | end as ins_for_vac, 43 | case 44 | when avacscalefactor.value is not null and avacthresh.value is not null 45 | then ROUND(((n_live_tup * avacscalefactor.value::numeric) + avacthresh.value::numeric),0) 46 | when avacscalefactor.value is null and avacthresh.value is not null 47 | then ROUND(((n_live_tup * (select setting::numeric from pg_settings where name = 'autovacuum_vacuum_scale_factor')) + avacthresh.value::numeric),0) 48 | when avacscalefactor.value is not null and avacthresh.value is null 49 | then ROUND(((n_live_tup * avacscalefactor.value::numeric) + (select setting::numeric from pg_settings where name = 'autovacuum_vacuum_threshold')),0) 50 | else ROUND(((n_live_tup * (select setting::numeric from pg_settings where name = 'autovacuum_vacuum_scale_factor')) + (select setting::numeric from pg_settings where name = 'autovacuum_vacuum_threshold')),0) 51 | end as mods_for_vac, 52 | case 53 | when avacanalyzescalefactor.value is not null and avacanalyzethresh.value is not null 54 | then ROUND(((n_live_tup * avacanalyzescalefactor.value::numeric) + avacanalyzethresh.value::numeric),0) 55 | when avacanalyzescalefactor.value is null and avacanalyzethresh.value is not null 56 | then ROUND(((n_live_tup * (select setting::numeric from pg_settings where name = 'autovacuum_analyze_scale_factor')) + avacanalyzethresh.value::numeric),0) 57 | when avacanalyzescalefactor.value is not null and avacanalyzethresh.value is null 58 | then ROUND(((n_live_tup * avacanalyzescalefactor.value::numeric) + (select setting::numeric from pg_settings where name = 'autovacuum_analyze_threshold')),0) 59 | else ROUND(((n_live_tup * (select setting::numeric from pg_settings where name = 'autovacuum_analyze_scale_factor')) + (select setting::numeric from pg_settings where name = 'autovacuum_analyze_threshold')),0) 60 | end as mods_for_stats, 61 | case 62 | when avacfreezeage is not null 63 | then ROUND((greatest(age(c.relfrozenxid),age(t.relfrozenxid))::numeric / avacfreezeage.value::numeric * 100),2) 64 | else ROUND((greatest(age(c.relfrozenxid),age(t.relfrozenxid))::numeric / (select setting::numeric from pg_settings where name = 'autovacuum_freeze_max_age') * 100),2) 65 | end as avac_pct_frz, 66 | greatest(age(c.relfrozenxid),age(t.relfrozenxid)) max_txid_age, 67 | to_char(last_vacuum, 'YYYY-MM-DD HH24:MI') last_vac, 68 | to_char(last_analyze, 'YYYY-MM-DD HH24:MI') last_stats, 69 | to_char(last_autovacuum, 'YYYY-MM-DD HH24:MI') last_avac, 70 | to_char(last_autoanalyze, 'YYYY-MM-DD HH24:MI') last_astats, 71 | vacuum_count vac_cnt, 72 | analyze_count stats_cnt, 73 | autovacuum_count avac_cnt, 74 | autoanalyze_count astats_cnt, 75 | c.reloptions, 76 | case 77 | when avacenabled.value is not null 78 | then avacenabled.value::text 79 | when (select setting::text from pg_settings where name = 'autovacuum') = 'on' 80 | then 'true' 81 | else 'false' 82 | end as autovac_enabled 83 | FROM 84 | pg_stat_all_tables s 85 | JOIN pg_class c ON (s.relid = c.oid) 86 | LEFT JOIN pg_class t ON c.reltoastrelid = t.oid 87 | LEFT JOIN tbl_reloptions avacinsscalefactor on (s.relid = avacinsscalefactor.oid and avacinsscalefactor.option = 'autovacuum_vacuum_insert_scale_factor') 88 | LEFT JOIN tbl_reloptions avacinsthresh on (s.relid = avacinsthresh.oid and avacinsthresh.option = 'autovacuum_vacuum_insert_threshold') 89 | LEFT JOIN tbl_reloptions avacscalefactor on (s.relid = avacscalefactor.oid and avacscalefactor.option = 'autovacuum_vacuum_scale_factor') 90 | LEFT JOIN tbl_reloptions avacthresh on (s.relid = avacthresh.oid and avacthresh.option = 'autovacuum_vacuum_threshold') 91 | LEFT JOIN tbl_reloptions avacanalyzescalefactor on (s.relid = avacanalyzescalefactor.oid and avacanalyzescalefactor.option = 'autovacuum_analyze_scale_factor') 92 | LEFT JOIN tbl_reloptions avacanalyzethresh on (s.relid = avacanalyzethresh.oid and avacanalyzethresh.option = 'autovacuum_analyze_threshold') 93 | LEFT JOIN tbl_reloptions avacfreezeage on (s.relid = avacfreezeage.oid and avacfreezeage.option = 'autovacuum_freeze_max_age') 94 | LEFT JOIN tbl_reloptions avacenabled on (s.relid = avacenabled.oid and avacenabled.option = 'autovacuum_enabled') 95 | WHERE 96 | s.relname IN ( 97 | SELECT 98 | t.table_name 99 | FROM 100 | information_schema.tables t 101 | JOIN pg_catalog.pg_class c ON (t.table_name = c.relname) 102 | LEFT JOIN pg_catalog.pg_user u ON (c.relowner = u.usesysid) 103 | WHERE 104 | t.table_schema like '%' 105 | AND (u.usename like '%' OR u.usename is null) 106 | AND t.table_name like '%' 107 | AND t.table_schema not in ('information_schema','pg_catalog') 108 | AND t.table_type not in ('VIEW') 109 | AND t.table_catalog = current_database()) 110 | --AND n_dead_tup >= 0 111 | --AND n_live_tup > 0 112 | ORDER BY 3; 113 | -------------------------------------------------------------------------------- /site/404.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | My Docs 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 76 | 77 |
78 |
79 | 80 |
81 |
82 |

404

83 |

Page not found

84 |
85 |
86 | 87 | 88 |
89 |
90 | 91 |
92 |
93 |

Documentation built with MkDocs.

94 |
95 | 96 | 100 | 101 | 102 | 103 | 163 | 164 | 165 | 166 | -------------------------------------------------------------------------------- /oracle/oracle_historical_ash_info.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | SELECT 16 | * 17 | FROM 18 | ( 19 | SELECT 20 | b.* 21 | FROM 22 | ( 23 | SELECT 24 | a.*, 25 | round(avg(exec_et_secs) over( 26 | PARTITION BY sql_id 27 | ),2) AS avg_exec, 28 | MIN(exec_et_secs) OVER( 29 | PARTITION BY sql_id 30 | ) AS min_exec, 31 | MAX(exec_et_secs) OVER( 32 | PARTITION BY sql_id 33 | ) AS max_exec, 34 | round(STDDEV(exec_et_secs) OVER( 35 | PARTITION BY sql_id 36 | ),2) AS stddev, 37 | count(distinct sql_exec_id) over(partition by sql_id) as exec_count, 38 | count(distinct SQL_PLAN_HASH_VALUE) over (partition by sql_id) as plan_hash_count, 39 | round(avg(exec_et_secs) OVER (partition by sql_plan_hash_value),2) as avg_plan_exec_et, 40 | min(sample_time) over(partition by sql_id) as first_exec, 41 | max(sample_time) over(partition by sql_id) as last_exec 42 | FROM 43 | ( 44 | SELECT 45 | ash.sample_time, 46 | TO_CHAR(ash.sample_time, 'YYYY-MM-DD HH24:MI:SS') AS sample_time_char, 47 | ash.sql_exec_id, 48 | ash.sql_id, 49 | ash.sql_child_number, 50 | ash.program, 51 | ash.module, 52 | ash.user_id, 53 | du.username, 54 | TO_CHAR(ash.sql_exec_start, 'YYYY-MM-DD HH24:MI:SS') AS sql_exec_start, 55 | substr(TO_CHAR(ash.sample_time - ash.sql_exec_start, 'HH:MM:SS.FF'), - 12) AS sample_et, 56 | MAX( (EXTRACT(HOUR FROM ash.sample_time - ash.sql_exec_start) * 3600) + 57 | (EXTRACT(MINUTE FROM ash.sample_time - ash.sql_exec_start) * 60) + 58 | EXTRACT(SECOND FROM ash.sample_time - ash.sql_exec_start)) OVER( 59 | PARTITION BY ash.instance_number, ash.sql_exec_id, ash.sql_id, ash.session_id 60 | ) AS exec_et_secs, 61 | ash.sample_time - lag(ash.sample_time) over( partition by ash.INSTANCE_NUMBER, session_id, session_serial# order by sample_time) as et_between_samples, 62 | event, 63 | wait_class, 64 | seq#, 65 | top_level_sql_id, 66 | sql_plan_hash_value, 67 | sql_plan_line_id, 68 | sql_plan_operation, 69 | sql_plan_options, 70 | o.object_name, 71 | MAX(ash.sample_time) OVER( 72 | PARTITION BY ash.instance_number, ash.sql_exec_id, ash.sql_id,ash.session_id 73 | ) AS max_sample, 74 | client_id, 75 | machine, 76 | port, 77 | session_id, 78 | session_serial#, 79 | ash.instance_number as inst_id, 80 | blocking_session, 81 | blocking_inst_id, 82 | blocking_session_status, 83 | in_connection_mgmt, 84 | in_parse, 85 | in_hard_parse, 86 | in_sql_execution, 87 | in_plsql_execution, 88 | in_plsql_rpc, 89 | in_plsql_compilation, 90 | in_java_execution, 91 | in_bind, 92 | in_cursor_close, 93 | in_sequence_load, 94 | substr(sq.sql_text, 1, 175) AS the_sql, 95 | substr(tsql.sql_text, 1, 500) AS top_sql, 96 | sq.EXACT_MATCHING_SIGNATURE, 97 | sq.FORCE_MATCHING_SIGNATURE , 98 | blocking_hangchain_info, 99 | current_obj#, 100 | current_file#, 101 | current_block#, 102 | current_row#, 103 | p1text, 104 | p1, 105 | p2text, 106 | p2, 107 | p3text, 108 | p3 109 | FROM 110 | dba_hist_active_sess_history ash 111 | LEFT OUTER JOIN gv$sql sq ON ash.sql_id = sq.sql_id 112 | AND ash.instance_number = sq.inst_id 113 | AND ash.sql_child_number = sq.child_number 114 | LEFT OUTER JOIN ( 115 | SELECT DISTINCT 116 | sql_id, 117 | to_char(dbms_lob.substr(sql_text,70,1)) as sql_text 118 | FROM 119 | dba_hist_sqltext 120 | ) tsql ON ash.top_level_sql_id = tsql.sql_id 121 | LEFT OUTER JOIN dba_users du on du.user_id = ash.user_id 122 | LEFT OUTER JOIN dba_objects o on o.object_id = ash.current_obj# 123 | where ash.sample_time BETWEEN NVL(TO_DATE(:starttm, 'YYYY-MM-DD HH24:MI'), (SELECT MAX(sample_time)-(1/24) FROM dba_hist_active_sess_history)) AND NVL(TO_DATE(:endtm, 'YYYY-MM-DD HH24:MI'), (SELECT MAX(sample_time) FROM dba_hist_active_sess_history)) 124 | --and ash.module like 'smsclientengine%' 125 | and tsql.sql_id in ('2r6qwm2aaaz0c') 126 | --and ash.sql_id in('cpqu9tntjv392','cnjccdbsr30gw','57ctd87hnrhp3','1x9vq586h508p') 127 | --and upper(sq.sql_text) like 'WITH BILL_WINDOW_CYCLES AS%' 128 | --and machine like 'gtlassoa%' 129 | --and du.username='DWREP' 130 | ) a 131 | ) b 132 | ) 133 | /*ASHDATA*/ 134 | where nvl(the_sql,'x') not like 'SELECT /* ash_recent_sessions */%' 135 | --and nvl(event,'x')='enq: TX - row lock contention' 136 | --and nvl(event,'x')!='ges generic event' 137 | --and nvl(module,'x') != 'DBMS_SCHEDULER' 138 | --and (the_sql like '%SUPER%' ) 139 | --and module='SUPER' 140 | --and module like 'SUPER%' 141 | --and sql_id ='9zk77arubrvwm' --dw87m1uqwdwp0 142 | --and sql_id like '0ws0gj%' 143 | --and the_sql like '%SUPER%' 144 | --and sql_id in ('64x3rsjh1n6yg', '13nu1znn8cc1t','6ywzgz3nnpykd') 145 | --and machine = 'SUPER' 146 | --and upper(the_sql) like '%CI_ADJ%' 147 | --and session_id= 8839 148 | --and exec_et_secs >60 149 | --and nvl(event,'x') not in ('library cache lock', 'cursor: pin S wait on X') 150 | order by 1 ,2,3,4; -------------------------------------------------------------------------------- /oracle/oracle_current_ash_info.sql: -------------------------------------------------------------------------------- 1 | -- Copyright 2024 shaneborden 2 | -- 3 | -- Licensed under the Apache License, Version 2.0 (the "License"); 4 | -- you may not use this file except in compliance with the License. 5 | -- You may obtain a copy of the License at 6 | -- 7 | -- https://www.apache.org/licenses/LICENSE-2.0 8 | -- 9 | -- Unless required by applicable law or agreed to in writing, software 10 | -- distributed under the License is distributed on an "AS IS" BASIS, 11 | -- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | -- See the License for the specific language governing permissions and 13 | -- limitations under the License. 14 | 15 | set lines 180 16 | set pages 1000 17 | alter session set nls_date_format = 'YYYY-MM-DD HH24:MI:SS'; 18 | alter session set nls_timestamp_format = 'YYYY-MM-DD HH24:MI:SS.FF'; 19 | -- Show activity for recent sessions. Uses gv$active_session_history, so only recent data available. 20 | -- Requires license for Tuning and Diagnostics 21 | -- Defaults to activity within the last 2 minutes if start_time and end_time parameters are NULL 22 | -- Parameter format is 'YYYY-MM-DD HH24:MI' 23 | SELECT /* ash_recent_sessions */ 24 | * 25 | FROM 26 | ( 27 | SELECT 28 | b.* 29 | FROM 30 | ( 31 | SELECT 32 | a.*, 33 | round(avg(exec_et_secs) over( 34 | PARTITION BY sql_id 35 | ),2) AS avg_exec, 36 | MIN(exec_et_secs) OVER( 37 | PARTITION BY sql_id 38 | ) AS min_exec, 39 | MAX(exec_et_secs) OVER( 40 | PARTITION BY sql_id 41 | ) AS max_exec, 42 | round(STDDEV(exec_et_secs) OVER( 43 | PARTITION BY sql_id 44 | ),2) AS stddev, 45 | count(distinct sql_exec_id) over(partition by inst_id, session_id, SESSION_SERIAL#, sql_id) as exec_count, 46 | count(distinct SQL_PLAN_HASH_VALUE) over (partition by sql_id) as plan_hash_count, 47 | round(avg(exec_et_secs) OVER (partition by sql_plan_hash_value),2) as avg_plan_exec_et, 48 | min(sample_time) over(partition by sql_id) as first_exec, 49 | max(sample_time) over(partition by sql_id) as last_exec 50 | FROM 51 | ( 52 | SELECT 53 | ash.sample_time, 54 | TO_CHAR(ash.sample_time, 'YYYY-MM-DD HH24:MI:SS') AS sample_time_char, 55 | ash.sql_exec_id, 56 | ash.sql_id, 57 | ash.sql_child_number, 58 | ash.program, 59 | ash.module, 60 | ash.action, 61 | ash.session_type, 62 | ash.user_id, 63 | du.username, 64 | ash.client_id, 65 | gs.resource_consumer_group, 66 | TO_CHAR(ash.sql_exec_start, 'YYYY-MM-DD HH24:MI:SS') AS sql_exec_start, 67 | substr(TO_CHAR(ash.sample_time - ash.sql_exec_start, 'HH:MM:SS.FF'), - 12) AS sample_et, 68 | MAX( (EXTRACT(HOUR FROM ash.sample_time - ash.sql_exec_start) * 3600) + 69 | (EXTRACT(MINUTE FROM ash.sample_time - ash.sql_exec_start) * 60) + 70 | EXTRACT(SECOND FROM ash.sample_time - ash.sql_exec_start)) OVER( 71 | PARTITION BY ash.inst_id, ash.sql_exec_id, ash.sql_id, ash.session_id 72 | ) AS exec_et_secs, 73 | -- Use ET_Between_Samples to help identify when a connection has been idle for a while. 74 | -- This is the elapsed time between samples for the connection. 75 | ash.sample_time - lag(ash.sample_time) over( partition by ash.inst_id, ash.session_id, ash.session_serial# order by ash.sample_time) as et_between_samples, 76 | tm_delta_time, 77 | tm_delta_cpu_time, 78 | tm_delta_db_time, 79 | (tm_delta_cpu_time/10000)+tm_delta_db_time as tot_time, 80 | tm_delta_time - tm_delta_db_time as diftime, 81 | ash.event, 82 | ash.wait_class, 83 | ash.inst_id, 84 | ash.session_id, 85 | ash.SESSION_SERIAL#, 86 | ash.ecid, 87 | ash.seq#, 88 | top_level_sql_id, 89 | sql_plan_hash_value, 90 | sql_plan_line_id, 91 | sql_plan_operation, 92 | sql_plan_options, 93 | o.owner, 94 | o.object_name, 95 | o.subobject_name, 96 | o.object_type, 97 | MAX(ash.sample_time) OVER( 98 | PARTITION BY ash.inst_id, ash.sql_exec_id, ash.sql_id, ash.session_id 99 | ) AS max_sample, 100 | --client_id, 101 | ash.machine, 102 | ash.port, 103 | ash.blocking_session, 104 | ash.blocking_inst_id, 105 | ash.blocking_session_status, 106 | in_connection_mgmt, 107 | in_parse, 108 | in_hard_parse, 109 | in_sql_execution, 110 | in_plsql_execution, 111 | in_plsql_rpc, 112 | in_plsql_compilation, 113 | in_java_execution, 114 | in_bind, 115 | in_cursor_close, 116 | in_sequence_load, 117 | sq.sql_text as the_sql, 118 | tsql.sql_text as top_sql, 119 | sq.EXACT_MATCHING_SIGNATURE, 120 | sq.FORCE_MATCHING_SIGNATURE , 121 | ash.blocking_hangchain_info, 122 | ash.current_obj#, 123 | ash.current_file#, 124 | ash.current_block#, 125 | ash.current_row#, 126 | ash.p1text, 127 | ash.p1, 128 | ash.p2text, 129 | ash.p2, 130 | ash.p3text, 131 | ash.p3 132 | FROM 133 | gv$active_session_history ash 134 | LEFT OUTER JOIN gv$sql sq ON ash.sql_id = sq.sql_id 135 | AND ash.inst_id = sq.inst_id 136 | AND ash.sql_child_number = sq.child_number 137 | LEFT OUTER JOIN ( 138 | SELECT DISTINCT 139 | sql_id, 140 | dbms_lob.substr(sql_text,75,1) as sql_text 141 | FROM 142 | gv$sql 143 | ) tsql ON ash.top_level_sql_id = tsql.sql_id 144 | LEFT OUTER JOIN dba_users du on du.user_id = ash.user_id 145 | LEFT OUTER JOIN dba_objects o on o.object_id = ash.current_obj# 146 | LEFT OUTER JOIN gv$session gs on gs.inst_id = ash.inst_id and gs.sid = ash.session_id and gs.serial# = ash.session_serial# 147 | where ash.sample_time BETWEEN nvl(TO_DATE(:starttm, 'YYYY-MM-DD HH24:MI'), sysdate-(1/24/30)) AND nvl(TO_DATE(:endtm, 'YYYY-MM-DD HH24:MI'), sysdate) 148 | and nvl(ash.event, 'x') != 'ges generic event' 149 | /* Add filters here */ 150 | and ash.sql_id in ( '2r6qwm2aaaz0c' ) 151 | -- and ash.machine like 'EXA%' 152 | -- and ash.sql_plan_hash_value = 534356389 153 | -- and upper(sq.sql_text) like 'UPDATE%' 154 | -- and du.username = 'CA12345' 155 | -- and ash.module like 'SUPER%' 156 | -- and ash.client_id = 'SUPER' 157 | -- and nvl(event,'x') = 'enq: TX - row lock contention' 158 | ) a 159 | ) b 160 | ) 161 | where nvl(the_sql,'x') not like 'SELECT /* ash_recent_sessions */%' 162 | --and exec_et_secs >3600 163 | --and module !='GoldenGate' 164 | order by sample_time desc 165 | / -------------------------------------------------------------------------------- /site/js/base.js: -------------------------------------------------------------------------------- 1 | function getSearchTerm() { 2 | var sPageURL = window.location.search.substring(1); 3 | var sURLVariables = sPageURL.split('&'); 4 | for (var i = 0; i < sURLVariables.length; i++) { 5 | var sParameterName = sURLVariables[i].split('='); 6 | if (sParameterName[0] == 'q') { 7 | return sParameterName[1]; 8 | } 9 | } 10 | } 11 | 12 | function applyTopPadding() { 13 | // Update various absolute positions to match where the main container 14 | // starts. This is necessary for handling multi-line nav headers, since 15 | // that pushes the main container down. 16 | var container = document.querySelector('body > .container'); 17 | var offset = container.offsetTop; 18 | 19 | document.documentElement.style.scrollPaddingTop = offset + 'px'; 20 | document.querySelectorAll('.bs-sidebar.affix').forEach(function(sidebar) { 21 | sidebar.style.top = offset + 'px'; 22 | }); 23 | } 24 | 25 | document.addEventListener("DOMContentLoaded", function () { 26 | var search_term = getSearchTerm(); 27 | var search_modal = new bootstrap.Modal(document.getElementById('mkdocs_search_modal')); 28 | var keyboard_modal = new bootstrap.Modal(document.getElementById('mkdocs_keyboard_modal')); 29 | 30 | if (search_term) { 31 | search_modal.show(); 32 | } 33 | 34 | // make sure search input gets autofocus every time modal opens. 35 | document.getElementById('mkdocs_search_modal').addEventListener('shown.bs.modal', function() { 36 | document.getElementById('mkdocs-search-query').focus(); 37 | }); 38 | 39 | // Close search modal when result is selected 40 | // The links get added later so listen to parent 41 | document.getElementById('mkdocs-search-results').addEventListener('click', function(e) { 42 | if (e.target.tagName === 'A') { 43 | search_modal.hide(); 44 | } 45 | }); 46 | 47 | // Populate keyboard modal with proper Keys 48 | document.querySelector('.help.shortcut kbd').innerHTML = keyCodes[shortcuts.help]; 49 | document.querySelector('.prev.shortcut kbd').innerHTML = keyCodes[shortcuts.previous]; 50 | document.querySelector('.next.shortcut kbd').innerHTML = keyCodes[shortcuts.next]; 51 | document.querySelector('.search.shortcut kbd').innerHTML = keyCodes[shortcuts.search]; 52 | 53 | // Keyboard navigation 54 | document.addEventListener("keydown", function(e) { 55 | if (e.target.tagName === 'INPUT' || e.target.tagName === 'TEXTAREA') return true; 56 | var key = e.which || e.keyCode || window.event && window.event.keyCode; 57 | var page; 58 | switch (key) { 59 | case shortcuts.next: 60 | page = document.querySelector('.navbar a[rel="next"]'); 61 | break; 62 | case shortcuts.previous: 63 | page = document.querySelector('.navbar a[rel="prev"]'); 64 | break; 65 | case shortcuts.search: 66 | e.preventDefault(); 67 | keyboard_modal.hide(); 68 | search_modal.show(); 69 | document.getElementById('mkdocs-search-query').focus(); 70 | break; 71 | case shortcuts.help: 72 | search_modal.hide(); 73 | keyboard_modal.show(); 74 | break; 75 | default: break; 76 | } 77 | if (page && page.hasAttribute('href')) { 78 | keyboard_modal.hide(); 79 | window.location.href = page.getAttribute('href'); 80 | } 81 | }); 82 | 83 | document.querySelectorAll('table').forEach(function(table) { 84 | table.classList.add('table', 'table-striped', 'table-hover'); 85 | }); 86 | 87 | function showInnerDropdown(item) { 88 | var popup = item.nextElementSibling; 89 | popup.classList.add('show'); 90 | item.classList.add('open'); 91 | 92 | // First, close any sibling dropdowns. 93 | var container = item.parentElement.parentElement; 94 | container.querySelectorAll(':scope > .dropdown-submenu > a').forEach(function(el) { 95 | if (el !== item) { 96 | hideInnerDropdown(el); 97 | } 98 | }); 99 | 100 | var popupMargin = 10; 101 | var maxBottom = window.innerHeight - popupMargin; 102 | var bounds = item.getBoundingClientRect(); 103 | 104 | popup.style.left = bounds.right + 'px'; 105 | if (bounds.top + popup.clientHeight > maxBottom && 106 | bounds.top > window.innerHeight / 2) { 107 | popup.style.top = (bounds.bottom - popup.clientHeight) + 'px'; 108 | popup.style.maxHeight = (bounds.bottom - popupMargin) + 'px'; 109 | } else { 110 | popup.style.top = bounds.top + 'px'; 111 | popup.style.maxHeight = (maxBottom - bounds.top) + 'px'; 112 | } 113 | } 114 | 115 | function hideInnerDropdown(item) { 116 | var popup = item.nextElementSibling; 117 | popup.classList.remove('show'); 118 | item.classList.remove('open'); 119 | 120 | popup.scrollTop = 0; 121 | var menu = popup.querySelector('.dropdown-menu'); 122 | if (menu) { 123 | menu.scrollTop = 0; 124 | } 125 | var dropdown = popup.querySelector('.dropdown-submenu > a'); 126 | if (dropdown) { 127 | dropdown.classList.remove('open'); 128 | } 129 | } 130 | 131 | document.querySelectorAll('.dropdown-submenu > a').forEach(function(item) { 132 | item.addEventListener('click', function(e) { 133 | if (item.nextElementSibling.classList.contains('show')) { 134 | hideInnerDropdown(item); 135 | } else { 136 | showInnerDropdown(item); 137 | } 138 | 139 | e.stopPropagation(); 140 | e.preventDefault(); 141 | }); 142 | }); 143 | 144 | document.querySelectorAll('.dropdown-menu').forEach(function(menu) { 145 | menu.parentElement.addEventListener('hide.bs.dropdown', function() { 146 | menu.scrollTop = 0; 147 | var dropdown = menu.querySelector('.dropdown-submenu > a'); 148 | if (dropdown) { 149 | dropdown.classList.remove('open'); 150 | } 151 | menu.querySelectorAll('.dropdown-menu .dropdown-menu').forEach(function(submenu) { 152 | submenu.classList.remove('show'); 153 | }); 154 | }); 155 | }); 156 | 157 | applyTopPadding(); 158 | }); 159 | 160 | window.addEventListener('resize', applyTopPadding); 161 | 162 | var scrollSpy = new bootstrap.ScrollSpy(document.body, { 163 | target: '.bs-sidebar' 164 | }); 165 | 166 | /* Prevent disabled links from causing a page reload */ 167 | document.querySelectorAll("li.disabled a").forEach(function(item) { 168 | item.addEventListener("click", function(event) { 169 | event.preventDefault(); 170 | }); 171 | }); 172 | 173 | // See https://www.cambiaresearch.com/articles/15/javascript-char-codes-key-codes 174 | // We only list common keys below. Obscure keys are omitted and their use is discouraged. 175 | var keyCodes = { 176 | 8: 'backspace', 177 | 9: 'tab', 178 | 13: 'enter', 179 | 16: 'shift', 180 | 17: 'ctrl', 181 | 18: 'alt', 182 | 19: 'pause/break', 183 | 20: 'caps lock', 184 | 27: 'escape', 185 | 32: 'spacebar', 186 | 33: 'page up', 187 | 34: 'page down', 188 | 35: 'end', 189 | 36: 'home', 190 | 37: '←', 191 | 38: '↑', 192 | 39: '→', 193 | 40: '↓', 194 | 45: 'insert', 195 | 46: 'delete', 196 | 48: '0', 197 | 49: '1', 198 | 50: '2', 199 | 51: '3', 200 | 52: '4', 201 | 53: '5', 202 | 54: '6', 203 | 55: '7', 204 | 56: '8', 205 | 57: '9', 206 | 65: 'a', 207 | 66: 'b', 208 | 67: 'c', 209 | 68: 'd', 210 | 69: 'e', 211 | 70: 'f', 212 | 71: 'g', 213 | 72: 'h', 214 | 73: 'i', 215 | 74: 'j', 216 | 75: 'k', 217 | 76: 'l', 218 | 77: 'm', 219 | 78: 'n', 220 | 79: 'o', 221 | 80: 'p', 222 | 81: 'q', 223 | 82: 'r', 224 | 83: 's', 225 | 84: 't', 226 | 85: 'u', 227 | 86: 'v', 228 | 87: 'w', 229 | 88: 'x', 230 | 89: 'y', 231 | 90: 'z', 232 | 91: 'Left Windows Key / Left ⌘', 233 | 92: 'Right Windows Key', 234 | 93: 'Windows Menu / Right ⌘', 235 | 96: 'numpad 0', 236 | 97: 'numpad 1', 237 | 98: 'numpad 2', 238 | 99: 'numpad 3', 239 | 100: 'numpad 4', 240 | 101: 'numpad 5', 241 | 102: 'numpad 6', 242 | 103: 'numpad 7', 243 | 104: 'numpad 8', 244 | 105: 'numpad 9', 245 | 106: 'multiply', 246 | 107: 'add', 247 | 109: 'subtract', 248 | 110: 'decimal point', 249 | 111: 'divide', 250 | 112: 'f1', 251 | 113: 'f2', 252 | 114: 'f3', 253 | 115: 'f4', 254 | 116: 'f5', 255 | 117: 'f6', 256 | 118: 'f7', 257 | 119: 'f8', 258 | 120: 'f9', 259 | 121: 'f10', 260 | 122: 'f11', 261 | 123: 'f12', 262 | 124: 'f13', 263 | 125: 'f14', 264 | 126: 'f15', 265 | 127: 'f16', 266 | 128: 'f17', 267 | 129: 'f18', 268 | 130: 'f19', 269 | 131: 'f20', 270 | 132: 'f21', 271 | 133: 'f22', 272 | 134: 'f23', 273 | 135: 'f24', 274 | 144: 'num lock', 275 | 145: 'scroll lock', 276 | 186: ';', 277 | 187: '=', 278 | 188: ',', 279 | 189: '‐', 280 | 190: '.', 281 | 191: '?', 282 | 192: '`', 283 | 219: '[', 284 | 220: '\', 285 | 221: ']', 286 | 222: ''', 287 | }; 288 | -------------------------------------------------------------------------------- /site/developer_guide/releases/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | Releases - My Docs 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 86 | 87 |
88 |
89 |
107 |
108 | 109 |

Releases

110 |

A release should consist of the following two steps from a tested, linted, and up to date copy of the main branch:

111 |
    112 |
  1. 113 |

    make pre-release increment={major/minor/patch} - Commit the version number bump and create a new tag locally. The version number follows semantic versioning standards (major.minor.patch) and the tag is the version number prepended with a 'v'.

    114 |
  2. 115 |
  3. 116 |

    git push --follow-tags - Update the main branch with only the changes from the version bump. Publish the new tag and kick off the release workflow.

    117 |
  4. 118 |
119 |
120 |
121 | 122 |
123 |
124 |

Documentation built with MkDocs.

125 |
126 | 127 | 131 | 132 | 133 | 134 | 194 | 195 | 196 | 197 | -------------------------------------------------------------------------------- /site/developer_guide/developer_setup/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | Developer Setup - My Docs 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 86 | 87 |
88 |
89 |
111 |
112 | 113 |

Developer Setup

114 |

To begin local development, clone the shane-borden/sqlScripts repository and use one of the following methods to build it. All commands should be executed from inside of the project home folder.

115 |

Configure environment

116 |
make install
117 | 
118 |
119 |
120 | 121 |
122 |
123 |

Documentation built with MkDocs.

124 |
125 | 126 | 130 | 131 | 132 | 133 | 193 | 194 | 195 | 196 | -------------------------------------------------------------------------------- /site/developer_guide/commands/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | Commands - My Docs 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 86 | 87 |
88 |
89 |
107 |
108 | 109 |

Commands

110 |
    111 |
  • 112 |

    make install - Creates a new virtual environment for development.

    113 |
  • 114 |
  • 115 |

    make clean - Remove all build, testing, and static documentation files.

    116 |
  • 117 |
  • 118 |

    make build - Build a folder containing a set of the latest database collection scripts.

    119 |
  • 120 |
  • 121 |

    make gen-docs - Generate HTML documentation.

    122 |
  • 123 |
  • 124 |

    make docs - Generate HTML documentation and serve it to the browser.

    125 |
  • 126 |
  • 127 |

    make pre-release increment={major/minor/patch} - Bump the version and create a release tag. Should only be run from the main branch. Passes the increment value to bump2version to create a new version number dynamically. The new version number will be added to __version__.py and pyproject.toml and a new commit will be logged. The tag will be created from the new commit.

    128 |
  • 129 |
130 |
131 |
132 | 133 |
134 |
135 |

Documentation built with MkDocs.

136 |
137 | 138 | 142 | 143 | 144 | 145 | 205 | 206 | 207 | 208 | --------------------------------------------------------------------------------