├── website ├── static │ ├── .nojekyll │ └── img │ │ ├── favicon.ico │ │ ├── journey_api.png │ │ ├── journey_tum.png │ │ ├── journey_hyper.png │ │ ├── hyper-social-card.png │ │ ├── journey_hyper_tableau.png │ │ ├── journey_hyper_tableau_salesforce.png │ │ ├── devicon-windows.svg │ │ ├── devicon-macos.svg │ │ ├── hyper-logo.svg │ │ ├── tableau-logo.svg │ │ └── hyper-social-card.svg ├── babel.config.js ├── .prettierrc ├── docs │ ├── guides │ │ ├── index.md │ │ ├── hyper_file │ │ │ ├── read.md │ │ │ ├── publish.md │ │ │ ├── insert_csv.md │ │ │ ├── optimize.md │ │ │ ├── create_update.md │ │ │ └── geodata.md │ │ ├── pandas_integration.md │ │ └── sql_commands.md │ ├── sql.md │ ├── hyper-api │ │ ├── index.md │ │ ├── connection.md │ │ └── hyper_process.md │ ├── index.md │ ├── faq.md │ ├── installation.md │ └── releases.md ├── src │ ├── components │ │ └── DownloadPicker │ │ │ ├── styles.module.css │ │ │ └── index.tsx │ ├── theme │ │ ├── prism-sql-template.js │ │ └── prism-include-languages.js │ ├── os_detection.ts │ ├── pages │ │ ├── index.module.css │ │ ├── index.tsx │ │ └── journey.tsx │ ├── css │ │ └── custom.css │ └── config.ts ├── tsconfig.json ├── _analytics.js ├── download_links.ts ├── sidebars.js ├── package.json └── docusaurus.config.js ├── CODEOWNERS ├── .gitignore ├── SECURITY.md ├── .github └── workflows │ ├── proof.yaml │ └── deploy.yaml ├── README.md ├── LICENSE.txt └── CODE_OF_CONDUCT.md /website/static/.nojekyll: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /website/static/img/favicon.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-db/HEAD/website/static/img/favicon.ico -------------------------------------------------------------------------------- /website/static/img/journey_api.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-db/HEAD/website/static/img/journey_api.png -------------------------------------------------------------------------------- /website/static/img/journey_tum.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-db/HEAD/website/static/img/journey_tum.png -------------------------------------------------------------------------------- /website/babel.config.js: -------------------------------------------------------------------------------- 1 | module.exports = { 2 | presets: [require.resolve('@docusaurus/core/lib/babel/preset')], 3 | }; 4 | -------------------------------------------------------------------------------- /website/static/img/journey_hyper.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-db/HEAD/website/static/img/journey_hyper.png -------------------------------------------------------------------------------- /website/static/img/hyper-social-card.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-db/HEAD/website/static/img/hyper-social-card.png -------------------------------------------------------------------------------- /website/.prettierrc: -------------------------------------------------------------------------------- 1 | trailingComma: "all" 2 | tabWidth: 4 3 | semi: true 4 | singleQuote: true 5 | arrowParens: "avoid" 6 | printWidth: 120 -------------------------------------------------------------------------------- /website/static/img/journey_hyper_tableau.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-db/HEAD/website/static/img/journey_hyper_tableau.png -------------------------------------------------------------------------------- /website/docs/guides/index.md: -------------------------------------------------------------------------------- 1 | # Guides 2 | 3 | ```mdx-code-block 4 | import DocCardList from '@theme/DocCardList'; 5 | 6 | 7 | ``` -------------------------------------------------------------------------------- /website/src/components/DownloadPicker/styles.module.css: -------------------------------------------------------------------------------- 1 | .svgicon { 2 | display: inline-block; 3 | height: 1.5em; 4 | margin: 0 0.2em; 5 | } 6 | -------------------------------------------------------------------------------- /CODEOWNERS: -------------------------------------------------------------------------------- 1 | # Comment line immediately above ownership line is reserved for related other information. Please be careful while editing. 2 | #ECCN:Open Source 3 | -------------------------------------------------------------------------------- /website/static/img/journey_hyper_tableau_salesforce.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-db/HEAD/website/static/img/journey_hyper_tableau_salesforce.png -------------------------------------------------------------------------------- /website/tsconfig.json: -------------------------------------------------------------------------------- 1 | { 2 | // This file is not used in compilation. It is here just for a nice editor experience. 3 | "extends": "@tsconfig/docusaurus/tsconfig.json", 4 | "compilerOptions": { 5 | "baseUrl": "." 6 | } 7 | } 8 | -------------------------------------------------------------------------------- /website/src/theme/prism-sql-template.js: -------------------------------------------------------------------------------- 1 | Prism.languages.sql_template = { 2 | comment: /--.*/, 3 | variable: [/<[^>\s]+>/, { pattern: /(\s|\[|\[, ])\.\.\.(?=\]|\))/, lookbehind: true }], 4 | keyword: /\b[A-Z_]+\b/, 5 | string: /\'[^\']*\'/, 6 | punctuation: /[;[\]{}]/, 7 | }; 8 | -------------------------------------------------------------------------------- /website/static/img/devicon-windows.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Dependencies 2 | website/node_modules 3 | 4 | # Production 5 | website/build 6 | 7 | # Generated files 8 | website/.docusaurus 9 | website/.cache-loader 10 | 11 | # Misc 12 | .DS_Store 13 | .env.local 14 | .env.development.local 15 | .env.test.local 16 | .env.production.local 17 | 18 | npm-debug.log* 19 | yarn-debug.log* 20 | yarn-error.log* 21 | 22 | .vscode 23 | -------------------------------------------------------------------------------- /SECURITY.md: -------------------------------------------------------------------------------- 1 | ## Security 2 | 3 | Please report any security issue to [security@salesforce.com](mailto:security@salesforce.com) 4 | as soon as it is discovered. This library limits its runtime dependencies in 5 | order to reduce the total cost of ownership as much as can be, but all consumers 6 | should remain vigilant and have their security stakeholders review all third-party 7 | products (3PP) like this one and their dependencies. -------------------------------------------------------------------------------- /website/src/os_detection.ts: -------------------------------------------------------------------------------- 1 | import ExecutionEnvironment from '@docusaurus/ExecutionEnvironment'; 2 | 3 | export function detectOS() { 4 | if (!ExecutionEnvironment.canUseDOM) { 5 | return undefined; 6 | } 7 | let os = navigator.userAgent; 8 | if (os.search('Windows') !== -1) { 9 | return 'windows'; 10 | } else if (os.search('Mac') !== -1) { 11 | return 'macos'; 12 | } else if (os.search('Linux') !== -1 && os.search('X11') !== -1) { 13 | return 'linux'; 14 | } 15 | return undefined; 16 | } 17 | -------------------------------------------------------------------------------- /website/src/pages/index.module.css: -------------------------------------------------------------------------------- 1 | /** 2 | * CSS files with the .module.css suffix will be treated as CSS modules 3 | * and scoped locally. 4 | */ 5 | 6 | .heroBanner { 7 | padding: 4rem 0; 8 | text-align: center; 9 | position: relative; 10 | overflow: hidden; 11 | } 12 | 13 | @media screen and (max-width: 996px) { 14 | .heroBanner { 15 | padding: 2rem; 16 | } 17 | } 18 | 19 | .heroLogo { 20 | width: 8rem; 21 | margin: 0.5rem; 22 | } 23 | 24 | .buttons { 25 | display: flex; 26 | align-items: center; 27 | justify-content: center; 28 | } 29 | -------------------------------------------------------------------------------- /website/_analytics.js: -------------------------------------------------------------------------------- 1 | import ExecutionEnvironment from '@docusaurus/ExecutionEnvironment'; 2 | 3 | if (ExecutionEnvironment.canUseDOM) { 4 | 5 | // OptanonConsentNoticeStart 6 | 7 | const script = document.createElement('script'); 8 | 9 | script.src = 'https://cdn.cookielaw.org/scripttemplates/otSDKStub.js'; 10 | script.type = 'text/javascript'; 11 | script.setAttribute('data-domain-script', '019321c1-4b7e-7313-9372-ddbd938f50ea') 12 | script.innerHTML="function OptanonWrapper() { }"; 13 | 14 | document.getElementsByTagName('head')[0].appendChild(script); 15 | 16 | // OptanonConsentNoticeStart --> 17 | 18 | } -------------------------------------------------------------------------------- /website/docs/sql.md: -------------------------------------------------------------------------------- 1 | # SQL Reference 2 | 3 | Hyper uses the **Salesforce Data Cloud SQL dialect** which is documented as part of the Data Cloud Developer documentation. 4 | Please be aware that Data-Cloud-specific functions are not available in Hyper API. 5 | Those functions are marked accordingly in the "Applies To" headers of the individual documentation pages. 6 | 7 | 8 | ```mdx-code-block 9 | import Link from '@docusaurus/Link'; 10 | 11 | 12 | 13 | 14 | ``` -------------------------------------------------------------------------------- /website/static/img/devicon-macos.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | -------------------------------------------------------------------------------- /website/docs/hyper-api/index.md: -------------------------------------------------------------------------------- 1 | # Client-side API 2 | 3 | Hyper is a full-fledged, relational SQL database system. 4 | Tableau Hyper API bundles the Hyper database server (`hyperd`) with a client-side library to interact with `hyperd`. 5 | The API provides utilities for: 6 | 7 | * Starting a `hyperd` process, connecting to it and sending SQL queries (`HyperProcess` and `Connection`) 8 | * Inserting data into tables (the `Inserter` class) 9 | * Meta data queries (`connection.catalog`) 10 | * SQL escaping (`escape_string_literal` etc.) 11 | 12 | This section describes the high-level abstractions provided by Hyper API. 13 | The provided examples use Python, but the underlying concepts apply for all client languages. 14 | You can find the API details for your specific client language in the corresponding API reference. -------------------------------------------------------------------------------- /website/download_links.ts: -------------------------------------------------------------------------------- 1 | import { config } from './src/config'; 2 | import { spawn } from 'child_process'; 3 | 4 | function downloadFile(targetPath: string, url: string) { 5 | const cmd = `curl --output ${targetPath} ${url}` 6 | console.log(`Downloading ${url} to ${targetPath}`); 7 | console.log(cmd) 8 | spawn(cmd, [], { shell: true, stdio: 'inherit' }) 9 | } 10 | 11 | async function downloadArtifact(artifactName: string) { 12 | const url = config.download[artifactName]; 13 | if(!url) throw `Unknown artifact ${artifactName}` 14 | const fileSuffix = url.split(".").slice(-1); 15 | downloadFile(artifactName + "." + fileSuffix, url) 16 | } 17 | 18 | async function doDownloads() { 19 | const args = process.argv.slice(2); 20 | for (const v of args) { 21 | await downloadArtifact(v) 22 | }; 23 | } 24 | 25 | doDownloads().catch((e) => { 26 | console.log(e); 27 | process.exit(1); 28 | }) 29 | -------------------------------------------------------------------------------- /website/src/theme/prism-include-languages.js: -------------------------------------------------------------------------------- 1 | import siteConfig from '@generated/docusaurus.config'; 2 | export default function prismIncludeLanguages(PrismObject) { 3 | const { 4 | themeConfig: { prism }, 5 | } = siteConfig; 6 | const { additionalLanguages } = prism; 7 | // Prism components work on the Prism instance on the window, while prism- 8 | // react-renderer uses its own Prism instance. We temporarily mount the 9 | // instance onto window, import components to enhance it, then remove it to 10 | // avoid polluting global namespace. 11 | // You can mutate PrismObject: registering plugins, deleting languages... As 12 | // long as you don't re-assign it 13 | globalThis.Prism = PrismObject; 14 | additionalLanguages.forEach(lang => { 15 | // eslint-disable-next-line global-require, import/no-dynamic-require 16 | require(`prismjs/components/prism-${lang}`); 17 | }); 18 | require('@site/src/theme/prism-sql-template'); 19 | delete globalThis.Prism; 20 | } 21 | -------------------------------------------------------------------------------- /.github/workflows/proof.yaml: -------------------------------------------------------------------------------- 1 | name: Proof commit 2 | 3 | on: [push, pull_request] 4 | 5 | permissions: 6 | contents: read 7 | 8 | concurrency: 9 | group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }} 10 | cancel-in-progress: true 11 | 12 | jobs: 13 | build: 14 | runs-on: ubuntu-latest 15 | steps: 16 | - name: Checkout 17 | uses: actions/checkout@v4 18 | 19 | - name: Setup node 20 | uses: actions/setup-node@v4 21 | with: 22 | node-version: '20.8' 23 | cache: yarn 24 | cache-dependency-path: 'website/yarn.lock' 25 | 26 | - name: Install dependencies 27 | working-directory: website 28 | run: yarn install --frozen-lockfile 29 | 30 | - name: Lint 31 | working-directory: website 32 | run: yarn run lint:check 33 | 34 | - name: Setup Pages 35 | id: setup-pages 36 | uses: actions/configure-pages@v3 37 | 38 | - name: Build webpage 39 | working-directory: website 40 | run: yarn build 41 | env: 42 | GITHUB_ORIGIN: ${{ steps.setup-pages.outputs.origin }} 43 | GITHUB_BASE_PATH: ${{ steps.setup-pages.outputs.base_path }} 44 | 45 | - name: Upload webpage artifact 46 | uses: actions/upload-pages-artifact@v3 47 | with: 48 | path: 'website/build' 49 | -------------------------------------------------------------------------------- /website/src/pages/index.tsx: -------------------------------------------------------------------------------- 1 | import React from 'react'; 2 | import clsx from 'clsx'; 3 | import Link from '@docusaurus/Link'; 4 | import useDocusaurusContext from '@docusaurus/useDocusaurusContext'; 5 | import useBaseUrl from '@docusaurus/useBaseUrl'; 6 | import Layout from '@theme/Layout'; 7 | 8 | import styles from './index.module.css'; 9 | 10 | function HomepageHeader() { 11 | const { siteConfig } = useDocusaurusContext(); 12 | const logoSrc = useBaseUrl('/img/hyper-logo.svg'); 13 | return ( 14 |
15 |
16 | 17 |

{siteConfig.title}

18 |

{siteConfig.tagline}

19 |
20 | 21 | Get Started with Hyper 22 | 23 |
24 |
25 |
26 | ); 27 | } 28 | 29 | export default function Home(): JSX.Element { 30 | const { siteConfig } = useDocusaurusContext(); 31 | return ( 32 | 33 | 34 | 35 | ); 36 | } 37 | -------------------------------------------------------------------------------- /website/sidebars.js: -------------------------------------------------------------------------------- 1 | // @ts-check 2 | 3 | /** @type {import('@docusaurus/plugin-content-docs').SidebarsConfig} */ 4 | const sidebars = { 5 | docs: [ 6 | "index", 7 | "releases", 8 | 'installation', 9 | { 10 | type: 'category', 11 | label: 'Guides', 12 | link: { type: 'doc', id: 'guides/index' }, 13 | items: [ 14 | 'guides/sql_commands', 15 | 'guides/pandas_integration', 16 | 'guides/hyper_file/create_update', 17 | 'guides/hyper_file/read', 18 | 'guides/hyper_file/insert_csv', 19 | 'guides/hyper_file/geodata', 20 | 'guides/hyper_file/optimize', 21 | 'guides/hyper_file/publish', 22 | ], 23 | }, 24 | { 25 | type: 'category', 26 | label: 'Client-side API', 27 | link: { type: 'doc', id: 'hyper-api/index' }, 28 | items: [ 29 | 'hyper-api/hyper_process', 30 | 'hyper-api/connection', 31 | { type: 'link', label: "Examples", href: "https://github.com/tableau/hyper-api-samples" }, 32 | { type: 'link', label: "Python Reference", href: "pathname:///lang_docs/py/index.html" }, 33 | { type: 'link', label: "C++ Reference", href: "pathname:///lang_docs/cxx/index.html" }, 34 | { type: 'link', label: "Java Reference", href: "pathname:///lang_docs/java/overview-summary.html" }, 35 | ], 36 | }, 37 | 'sql', 38 | "faq", 39 | ], 40 | }; 41 | 42 | module.exports = sidebars; 43 | -------------------------------------------------------------------------------- /website/static/img/hyper-logo.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | -------------------------------------------------------------------------------- /website/package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "hyper-db-docs", 3 | "version": "0.0.0", 4 | "private": true, 5 | "scripts": { 6 | "docusaurus": "docusaurus", 7 | "start": "docusaurus start", 8 | "build": "docusaurus build", 9 | "swizzle": "docusaurus swizzle", 10 | "deploy": "docusaurus deploy", 11 | "clear": "docusaurus clear", 12 | "serve": "docusaurus serve", 13 | "write-translations": "docusaurus write-translations", 14 | "write-heading-ids": "docusaurus write-heading-ids", 15 | "typecheck": "tsc", 16 | "lint": "prettier --write src", 17 | "lint:check": "prettier --check src" 18 | }, 19 | "dependencies": { 20 | "@docusaurus/core": "^2.4.3", 21 | "@docusaurus/plugin-google-tag-manager": "^2.4.3", 22 | "@docusaurus/preset-classic": "^2.4.3", 23 | "@mdx-js/react": "^1.6.22", 24 | "clsx": "^2.0.0", 25 | "prism-react-renderer": "^1.3.5", 26 | "react": "^17.0.2", 27 | "react-dom": "^17.0.2", 28 | "react-vertical-timeline-component": "^3.6.0", 29 | "remark-deflist": "^1.0.0" 30 | }, 31 | "devDependencies": { 32 | "@docusaurus/module-type-aliases": "^2.4.3", 33 | "@tsconfig/docusaurus": "^1.0.5", 34 | "prettier": "^3.0.3", 35 | "ts-node": "^10.9.1", 36 | "typescript": "^5.2.2" 37 | }, 38 | "browserslist": { 39 | "production": [ 40 | ">0.5%", 41 | "not dead", 42 | "not op_mini all" 43 | ], 44 | "development": [ 45 | "last 1 chrome version", 46 | "last 1 firefox version", 47 | "last 1 safari version" 48 | ] 49 | }, 50 | "engines": { 51 | "node": ">=20.8" 52 | } 53 | } 54 | -------------------------------------------------------------------------------- /website/docs/guides/hyper_file/read.md: -------------------------------------------------------------------------------- 1 | # Read Data from Hyper Files 2 | 3 | To read data from a `.hyper` file, open a `Connection` to the file and 4 | then use the [SELECT](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/select.html) command to retrieve data from the file. 5 | 6 | ```python 7 | from tableauhyperapi import HyperProcess, Connection, Telemetry, CreateMode, Inserter 8 | 9 | with HyperProcess(Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 10 | with Connection(hyper.endpoint, 'TrivialExample.hyper', CreateMode.NONE) as connection: 11 | # Read all data from the table. 12 | # But be careful with this! It might be a lot of data and Python 13 | # might not be able to cope with that much data... 14 | my_data = connection.execute_list_query(""" 15 | SELECT * FROM "Extract"."Extract" 16 | """) 17 | print(my_data) 18 | 19 | # Maybe we are only interested in a more specific part of data? 20 | # If so, use the power of SQL to only retrieve the data you are 21 | # actually interested in! 22 | max_in_table = connection.execute_scalar_query(""" 23 | SELECT MAX(value) FROM "Extract"."Extract 24 | """) 25 | print(max_in_table) 26 | ``` 27 | 28 | In general, you can send arbitrarily complex queries against the data in a Hyper file. 29 | More information on the available SQL commands can be found in the [SQL reference](../../sql). 30 | For more information on how to issue SQL queries from Python (or your preferred language) and how to make programatically craft SQL queries, see the [Executing SQL Commands](../sql_commands) guide. -------------------------------------------------------------------------------- /website/static/img/tableau-logo.svg: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /website/src/css/custom.css: -------------------------------------------------------------------------------- 1 | /** 2 | * Any CSS included here will be global. The classic template 3 | * bundles Infima by default. Infima is a CSS framework designed to 4 | * work well for content-centric websites. 5 | */ 6 | 7 | /* You can override the default Infima variables here. */ 8 | :root { 9 | --ifm-color-primary: #2b407b; 10 | --ifm-color-primary-dark: #273a6f; 11 | --ifm-color-primary-darker: #253669; 12 | --ifm-color-primary-darkest: #1e2d56; 13 | --ifm-color-primary-light: #2f4687; 14 | --ifm-color-primary-lighter: #314a8d; 15 | --ifm-color-primary-lightest: #3853a0; 16 | --ifm-code-font-size: 95%; 17 | --docusaurus-highlighted-code-line-bg: rgba(0, 0, 0, 0.1); 18 | } 19 | 20 | /* For readability concerns, you should choose a lighter palette in dark mode. */ 21 | [data-theme='dark'] { 22 | --ifm-color-primary: #25c2a0; 23 | --ifm-color-primary-dark: #21af90; 24 | --ifm-color-primary-darker: #1fa588; 25 | --ifm-color-primary-darkest: #1a8870; 26 | --ifm-color-primary-light: #29d5b0; 27 | --ifm-color-primary-lighter: #32d8b4; 28 | --ifm-color-primary-lightest: #4fddbf; 29 | --docusaurus-highlighted-code-line-bg: rgba(0, 0, 0, 0.3); 30 | } 31 | 32 | /* Make it easier to see links within our documentation */ 33 | .theme-doc-markdown a { 34 | text-decoration: underline; 35 | } 36 | .theme-doc-markdown a.card { 37 | text-decoration: none; 38 | } 39 | 40 | .vertical-timeline-custom .vertical-timeline-element-content { 41 | box-shadow: 0; 42 | background: #2b407b21; 43 | } 44 | 45 | .vertical-timeline-custom .vertical-timeline-element-icon { 46 | box-shadow: 47 | 0 0 0 3px var(--ifm-color-primary), 48 | inset 0 2px 0 #00000014, 49 | 0 3px 0 4px #0000000d; 50 | background-color: #ffffff; 51 | } 52 | 53 | .vertical-timeline-custom .vertical-timeline-element-icon img { 54 | border-radius: 50%; 55 | } 56 | -------------------------------------------------------------------------------- /website/src/config.ts: -------------------------------------------------------------------------------- 1 | const version_long = '0.0.24081.rd9c4b45a'; 2 | const version_short = version_long.substr(0, version_long.lastIndexOf('.')); 3 | 4 | const downloadBaseUrl = 'https://downloads.tableau.com/tssoftware/'; 5 | 6 | export const config = { 7 | download: { 8 | windows_py: `${downloadBaseUrl}/tableauhyperapi-${version_short}-py3-none-win_amd64.whl`, 9 | windows_cxx: `${downloadBaseUrl}/tableauhyperapi-cxx-windows-x86_64-release-main.${version_long}.zip`, 10 | windows_java: `${downloadBaseUrl}/tableauhyperapi-java-windows-x86_64-release-main.${version_long}.zip`, 11 | 12 | macos_py: `${downloadBaseUrl}/tableauhyperapi-${version_short}-py3-none-macosx_10_11_x86_64.whl`, 13 | macos_cxx: `${downloadBaseUrl}/tableauhyperapi-cxx-macos-x86_64-release-main.${version_long}.zip`, 14 | macos_java: `${downloadBaseUrl}/tableauhyperapi-java-macos-x86_64-release-main.${version_long}.zip`, 15 | 16 | macos_arm_py: `${downloadBaseUrl}/tableauhyperapi-${version_short}-py3-none-macosx_13_0_arm64.whl`, 17 | macos_arm_cxx: `${downloadBaseUrl}/tableauhyperapi-cxx-macos-arm64-release-main.${version_long}.zip`, 18 | macos_arm_java: `${downloadBaseUrl}/tableauhyperapi-java-macos-arm64-release-main.${version_long}.zip`, 19 | 20 | linux_py: `${downloadBaseUrl}/tableauhyperapi-${version_short}-py3-none-manylinux2014_x86_64.whl`, 21 | linux_cxx: `${downloadBaseUrl}/tableauhyperapi-cxx-linux-x86_64-release-main.${version_long}.zip`, 22 | linux_java: `${downloadBaseUrl}/tableauhyperapi-java-linux-x86_64-release-main.${version_long}.zip`, 23 | 24 | docs_py: `${downloadBaseUrl}/tableauhyperapi-py-docs-main.${version_long}.zip`, 25 | docs_java: `${downloadBaseUrl}/tableauhyperapi-java-docs-main.${version_long}.zip`, 26 | docs_cxx: `${downloadBaseUrl}/tableauhyperapi-cxx-docs-main.${version_long}.zip`, 27 | }, 28 | version_long: version_long, 29 | version_short: version_short, 30 | }; 31 | -------------------------------------------------------------------------------- /website/docs/index.md: -------------------------------------------------------------------------------- 1 | # Tableau Hyper API 2 | 3 | Hyper is Tableau's blazingly fast SQL engine powering Tableau's real time analytics and interactive exploration and Tableau Prep's ETL transformations. 4 | With Hyper API, you can directly leverage the full speed of Hyper to: 5 | 6 | * ingest data into `.hyper` files which you can then analyze with Tableau Desktop or upload to Tableau Cloud 7 | * update data in existing Hyper files, ensuring that your Tableau analysis is based on the freshest, most up-to-date data 8 | * open existing files, and read data from those files 9 | * load data directly from CSV files, much faster, and without having to write custom code to do so 10 | * read widespread data formats such as CSVs, Parquet and Iceberg files 11 | * use the power of SQL, e.g. to transform and clean up your data before visualizing it with Tableau 12 | 13 | Those capabilities can be used, e.g., to: 14 | 15 | - Make data accessible to Tableau which is stored in data sources not 16 | currently natively supported by Tableau 17 | - Automate custom extract, transform and load (ETL) processes 18 | - Implement rolling window updates or custom incremental updates 19 | 20 | The Hyper API gives you the tools for interacting with local `.hyper` extract files. 21 | You can then publish the those Hyper files to Tableau Cloud or Tableau Server 22 | using the 23 | [Tableau Server REST API](https://onlinehelp.tableau.com/current/api/rest_api/en-us/help.htm) 24 | and the 25 | [Tableau Server Client](https://tableau.github.io/server-client-python/#) library. 26 | Furthermore, you can open `.hyper` files directly in Tableau Desktop and Tableau Prep. 27 | 28 | ## Runs anywhere 29 | 30 | The Hyper API libraries are available for the most commonly used languages, i.e.: 31 | 32 | - Python 33 | - C++ 34 | - Java 35 | 36 | and is available for Windows, macOS and Linux. 37 | It can be used both in the cloud as well as on small consumer-grade laptops. 38 | 39 | More details on the supported platforms and hardware requirements can be found in the [Installation instructions](installation). 40 | -------------------------------------------------------------------------------- /website/docs/faq.md: -------------------------------------------------------------------------------- 1 | # Frequently Asked Questions 2 | 3 | ## Who is behind Hyper? 4 | 5 | Hyper and Hyper API are developed by the Hyper team at Salesforce and powers the Tableau analytical experience. 6 | Our team has a strong research background because Hyper originally started as a research project, and many of the original founders of Hyper are still with the project. 7 | 8 | ## How is Hyper related to HyPer? 9 | 10 | Hyper started as a research project at [Technical University of Munich](https://db.in.tum.de/). Back then, the project was still called "HyPer", the "P" was lower-cased soon after the acquisition in 2016. 11 | You can still find find links to all the research paper on [HyPer's website](https://hyper-db.de/). 12 | The code base of Hyper is an evolution of the research project HyPer and many of the foundational papers of HyPer are still applicable. 13 | For more information on Hyper's journey, see [Our Journey](/journey) 14 | 15 | ## Why does HyperAPI only have `0.0.*` versions? 16 | 17 | No worries, our `0.0.*` versioning scheme does not imply that Hyper is still in a pre-alpha version. In fact, Hyper has been battle-proven at Tableau for many years. 18 | 19 | Our versioning scheme comes from the Tableau release numbers, where e.g. 2023.1.2 is the second patch version of the first release in the year 2023. With Hyper API, we are releasing new versions more frequently than the Tableau releases. Since there are no directly corresponding Tableau releases to our Hyper releases, we just decided to use `0.0.*` versions for Hyper API. 20 | 21 | ## How can I get help with using Hyper? {#getting-help} 22 | 23 | If you need help with using Hyper in the context of Tableau products, please use the official [Tableau support channels](https://www.tableau.com/support). That way it will be ensured that your support request gets the adequate attention. 24 | If you are having problems with Hyper API or Hyper itself, you can furthermore reach us by [opening an issue on Github](https://github.com/tableau/hyper-db/issues/new) or you can get in touch with us on [Hyper's Slack channel](https://join.slack.com/t/tableau-datadev/shared_invite/zt-1q4rrimsh-lHHKzrhid1MR4aMOkrnAFQ). 25 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Hyper's website 2 | 3 | You found the source code backing Hyper's website hosted at https://tableau.github.io/hyper-db 🙂 4 | 5 | If you want to read our documentation or any other content from our webpage, I highly recommend you do so on https://tableau.github.io/hyper-db instead of browsing the source code here. 6 | If, however, you want to contribute to our documentation, e.g. by documenting new use cases, fixing typos, writing a blog post, then you found exactly the right spot. 7 | Contributions are very welcome, you don't need to be a Salesforce employee. 8 | Just open a pull request on this Github repository. 9 | For larger contributions, however, you might want to create a [Github issue](https://github.com/tableau/hyper-db/issues) first, to discuss your idea. 10 | 11 | ## Building 12 | 13 | Hyper's website is built using [Docusaurus 2](https://docusaurus.io/), a modern static website generator. 14 | 15 | ### Installation 16 | 17 | After installing [yarn](https://yarnpkg.com/) and [nodejs](https://nodejs.org/en/download/package-manager), run 18 | 19 | ``` 20 | $ yarn 21 | ``` 22 | 23 | from the `website` subfolder within this repository. 24 | 25 | ### Local Development 26 | 27 | ``` 28 | $ yarn start 29 | ``` 30 | 31 | This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server. 32 | 33 | ### Creating a Pull Request 34 | 35 | To get your changes merged, create a Pull Request. 36 | Github Actions are configured to automatically test for changes. 37 | For simple typo or wording fixes, you don't need a peer review - feel free to just go ahead and merge. 38 | For actual new content, we encourage to get at least one peer review. 39 | 40 | Documentation changes which apply right away to the already available Hyper version should be merged directly to `main`. 41 | If your documentation only applies to the upcoming Hyper version, merge your changes into the `upcoming` branch instead. 42 | 43 | ### Deployment 44 | 45 | As soon as your pull request to https://github.com/tableau/hyper-db is merged to `main`, the new web page will be deployed automatically within a couple of minutes. 46 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 Salesforce, Inc. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | 23 | 24 | ---- 25 | 26 | The SQL documentation (in the folder `website/docs/sql`) is based on and includes content from the PostgreSQL documentation. PostgreSQL is licensed under the PostgreSQL License which can be found below: 27 | 28 | PostgreSQL is released under the PostgreSQL License, a liberal Open Source license, similar to the BSD or MIT licenses. 29 | 30 | PostgreSQL Database Management System 31 | (formerly known as Postgres, then as Postgres95) 32 | 33 | Portions Copyright © 1996-2023, The PostgreSQL Global Development Group 34 | 35 | Portions Copyright © 1994, The Regents of the University of California 36 | 37 | Permission to use, copy, modify, and distribute this software and its documentation for any purpose, without fee, and without a written agreement is hereby granted, provided that the above copyright notice and this paragraph and the following two paragraphs appear in all copies. 38 | 39 | IN NO EVENT SHALL THE UNIVERSITY OF CALIFORNIA BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF THE UNIVERSITY OF CALIFORNIA HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 40 | 41 | THE UNIVERSITY OF CALIFORNIA SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS ON AN "AS IS" BASIS, AND THE UNIVERSITY OF CALIFORNIA HAS NO OBLIGATIONS TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -------------------------------------------------------------------------------- /website/docs/guides/hyper_file/publish.md: -------------------------------------------------------------------------------- 1 | # Publish to Tableau Cloud / Server 2 | 3 | When you are ready to make your Hyper file available to other users, you can publish it to Tableau Server or Tableau Cloud. 4 | There are multiple options, depending on whether your file contains a single table or more than one table. 5 | 6 | * **Option 1:** Use Tableau Desktop to publish the file as part of either a workbook (a `.twbx` file with an embedded data source) or a separate packaged data source (a `.tdsx` file). For details, see [Publish a Data Source](https://help.tableau.com/current/pro/desktop/en-us/publish_datasources.htm). 7 | 8 | * **Option 2:** Use either _tabcmd_, the _Tableau REST API_, or the _Tableau Server Client_ library for Python to programmatically publish the `.hyper` file as is, without packaging it into a data source. Starting with Tableau Server 2021.4, the data model for the data source is generated from the foreign keys in the .hyper file. In particular, a relationship between two tables is generated whenever they are connected with a foreign key. Tableau Server versions 2021.3 and earlier do not support this and thus have the restriction that the file must contain exactly one schema with one table. Additionally, Tableau Server versions 2019.4 and older require both the schema and the table to be named “Extract”; in other words, the qualified name of the single table must be `"Extract"."Extract"` (note the double quotes, since the capitalization matters). This restriction has been removed in Tableau Server 2020.1. See [Publishing a single-table Hyper file directly to Tableau Cloud/Server](https://github.com/tableau/hyper-api-samples/tree/main/Community-Supported/publish-hyper) and [Publishing a multi-table Hyper file to Tableau Cloud/Server ](https://github.com/tableau/hyper-api-samples/tree/main/Community-Supported/publish-multi-table-hyper-legacy) for examples with the Hyper API and the Tableau Server Client library. 9 | 10 | * **Option 3:** Manually assemble your own packaged data source by packaging the Hyper file and a corresponding data source (`.tds`) into a single zipped file and changing the file extension from `.zip` to `.tdsx`. The `.tds` file is essentially an XML file that describes the properties (connections, joins, et cetera) of the Hyper extract. Note, however, that Tableau provides no support for this method and no programmatic way to create these files. That said, it is fairly easy to modify an existing saved data source for your purposes. For example, you could use Tableau Desktop to save the `.hyper` file; then, from the _Data_ menu, select the extract, choose _Add to Saved Data Sources_, and edit the saved `.tds` file as appropriate. For an (unsupported) example that programmatically swaps out the file in an existing packaged data source, see [Publishing a Multi-Table Hyper File to Tableau Cloud, Tableau Server](https://github.com/tableau/hyper-api-samples/tree/main/Community-Supported/publish-multi-table-hyper). 11 | 12 | As noted, Options 1 and 3 will also work if your Hyper file contains more than one table, but Option 2 will not. 13 | -------------------------------------------------------------------------------- /.github/workflows/deploy.yaml: -------------------------------------------------------------------------------- 1 | name: Deploy webpage 2 | 3 | # We combine both `main` and `upcoming` into a single webpage 4 | on: 5 | push: 6 | branches: 7 | - 'main' 8 | - 'upcoming' 9 | 10 | # Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages 11 | permissions: 12 | contents: read 13 | pages: write 14 | id-token: write 15 | 16 | concurrency: 17 | group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }} 18 | cancel-in-progress: true 19 | 20 | jobs: 21 | build: 22 | runs-on: ubuntu-latest 23 | environment: 24 | name: github-pages 25 | url: ${{ steps.deployment.outputs.page_url }} 26 | steps: 27 | - uses: actions/configure-pages@v3 28 | id: setup-pages 29 | 30 | - name: Checkout main 31 | uses: actions/checkout@v4 32 | with: 33 | ref: main 34 | path: main 35 | 36 | - name: Checkout upcoming 37 | uses: actions/checkout@v4 38 | with: 39 | ref: upcoming 40 | path: upcoming 41 | 42 | - name: Setup node 43 | uses: actions/setup-node@v4 44 | with: 45 | node-version: '20.8' 46 | cache: yarn 47 | cache-dependency-path: | 48 | main/website/yarn.lock 49 | upcoming/website/yarn.lock 50 | 51 | - name: Install dependencies (main) 52 | working-directory: main/website 53 | run: yarn install --frozen-lockfile 54 | 55 | - name: Install dependencies (upcoming) 56 | working-directory: upcoming/website 57 | run: yarn install --frozen-lockfile 58 | 59 | - name: Download language docs 60 | working-directory: main/website 61 | run: yarn ts-node download_links.ts docs_py docs_cxx docs_java 62 | 63 | - name: Build webpage for `main` branch 64 | working-directory: main/website 65 | run: yarn build 66 | env: 67 | GITHUB_ORIGIN: ${{ steps.setup-pages.outputs.origin }} 68 | GITHUB_BASE_PATH: ${{ steps.setup-pages.outputs.base_path }} 69 | 70 | - name: Build webpage for `upcoming` branch 71 | working-directory: upcoming/website 72 | run: yarn build 73 | env: 74 | GITHUB_ORIGIN: ${{ steps.setup-pages.outputs.origin }} 75 | GITHUB_BASE_PATH: ${{ steps.setup-pages.outputs.base_path }}/upcoming 76 | IS_UPCOMING: '1' 77 | 78 | - name: Combine `main`, `upcoming` and language docs 79 | run: | 80 | mv main/website/build combined 81 | mv upcoming/website/build combined/upcoming 82 | mkdir combined/lang_docs 83 | unzip main/website/docs_py.zip -d combined/lang_docs 84 | mv combined/lang_docs/tableauhyperapi-py-docs-* combined/lang_docs/py 85 | unzip main/website/docs_cxx.zip -d combined/lang_docs 86 | mv combined/lang_docs/tableauhyperapi-cxx-docs-* combined/lang_docs/cxx 87 | unzip main/website/docs_java.zip -d combined/lang_docs 88 | mv combined/lang_docs/tableauhyperapi-java-docs-* combined/lang_docs/java 89 | 90 | - name: Upload webpage artifact 91 | uses: actions/upload-pages-artifact@v3 92 | with: 93 | path: 'combined' 94 | 95 | - name: Deploy to GitHub Pages 96 | id: deployment 97 | uses: actions/deploy-pages@v4 98 | -------------------------------------------------------------------------------- /website/src/components/DownloadPicker/index.tsx: -------------------------------------------------------------------------------- 1 | import React from 'react'; 2 | import { config } from '@site/src/config'; 3 | import { detectOS } from '@site/src/os_detection'; 4 | import Tabs from '@theme/Tabs'; 5 | import TabItem from '@theme/TabItem'; 6 | import styles from './styles.module.css'; 7 | import LinuxIcon from '@site/static/img/devicon-linux.svg'; 8 | import WindowsIcon from '@site/static/img/devicon-windows.svg'; 9 | import MacosIcon from '@site/static/img/devicon-macos.svg'; 10 | 11 | export function DownloadPicker() { 12 | return ( 13 | 14 | 18 | Windows 19 | 20 | } 21 | > 22 | 33 | 34 | 38 | macOS 39 | 40 | } 41 | > 42 | 62 | 63 | 67 | Linux 68 | 69 | } 70 | > 71 | 82 | 83 | 84 | ); 85 | } 86 | -------------------------------------------------------------------------------- /website/docs/guides/pandas_integration.md: -------------------------------------------------------------------------------- 1 | # Using Hyper with Pandas 2 | 3 | If you want to use Hyper API together with pandas, the [pantab](https://pantab.readthedocs.io/en/latest/index.html) library has you covered. 4 | pantab allows you to read and write pandas dataframes from/to Hyper. 5 | pantab internally wraps Hyper API and provides convenience functions for integration with pandas on top of it. 6 | To install pantab, simply run `pip install pantab`. 7 | 8 | On this page, we provide a couple of usage examples for pantab. 9 | For more information on pantab, see the [pantab examples](https://pantab.readthedocs.io/en/latest/examples.html) and the [pantab API reference](https://pantab.readthedocs.io/en/latest/api.html). 10 | 11 | ## Loading Data through Pandas 12 | 13 | Pandas can read data from a plethora of other database systems (PostgreSQL, MS-SQL, Google Big Query, ...) out-of-the-box and can read many common file formats (such as Excel, HTML tables, ...). 14 | Through `pantab`, we can leverage this support to ingest data into a Hyper file, making the data accessible for analysis in Tableau. 15 | 16 | E.g., loading data from a PostgreSQL database into a Hyper file needs only a couple of lines: 17 | 18 | ```python 19 | import pandas as pd 20 | import pantab 21 | 22 | animals_df = pd.read_sql('SELECT * FROM animals', 'postgres:///my_db_name') 23 | pantab.frame_to_hyper(animals_df, "animals.hyper", table="animals") 24 | ``` 25 | 26 | The `frame_to_hyper` function writes a data frame into the specified Hyper file under a given table name. 27 | It internally handles all interactions with Hyper API (spawing a `HyperProcess`, creating a `Connection` to it, sending the data from pandas to Hyper, ...). 28 | 29 | Of course, we can also ingest data from arbitrary other sources supported by pantab. 30 | To read data from an Excel file, we would use: 31 | 32 | ```python 33 | import pandas as pd 34 | import pantab 35 | 36 | animals_df = pd.read_excel('animals.xlsx') 37 | pantab.frame_to_hyper(animals_df, "animals.hyper", table="animals") 38 | ``` 39 | 40 | ## Querying Parquet files From Pandas 41 | 42 | To kickstart your creativity on potential use cases, let's use Hyper to run an analytical query on a Parquet file - and read the result back to a pandas frame using `pantab`. 43 | 44 | You can send SQL queries to Hyper and get the result as a pandas dataframe using `pantab.frame_from_hyper_query`. 45 | Combined with Hyper's capabilities to [query external file formats](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/external-files.html), you can use this, e.g., to directly run queries on your Parquet files, Iceberg tables or Parquet files. 46 | The following example demonstrates this on the Parquet file `orders_10rows.parquet` which you can [download here](https://github.com/tableau/hyper-api-samples/raw/main/Community-Supported/parquet-to-hyper/orders_10rows.parquet). 47 | 48 | ```python 49 | import pandas as pd 50 | import pantab 51 | from tableauhyperapi import HyperProcess, Telemetry, Connection 52 | 53 | with HyperProcess(Telemetry.DO_NOT_SEND_USAGE_DATA_TO_TABLEAU) as hyper: 54 | with Connection(hyper.endpoint) as connection: 55 | results_df = pantab.frame_from_hyper_query(connection, """ 56 | SELECT 57 | YEAR(o_orderdate), 58 | CAST(SUM(o_totalprice) AS double precision) 59 | FROM external('orders_10rows.parquet') 60 | GROUP BY YEAR(o_orderdate) 61 | """) 62 | # The query result is now in a pandas data frame. 63 | # We have the whole power of the pandas ecosystem at our 64 | # fingertips - and we only print it... how boring... 65 | print(results_df) 66 | ``` 67 | 68 | :::note Missing Numerics Support in pantab 69 | pantab currently does not yet support reading `NUMERIC` types. 70 | The query shown above works around this issue by using a `CAST`. 71 | ::: 72 | -------------------------------------------------------------------------------- /website/docs/guides/hyper_file/insert_csv.md: -------------------------------------------------------------------------------- 1 | # Insert Data Directly from CSV Files 2 | 3 | Comma-separated values (CSV) are a popular file format to import and export tabular data from programs. Hyper is able to directly load data into a Hyper table. Using the [COPY FROM](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/copy-from.html) command, you can copy the data much faster than you could by iteratively adding the data one row at a time. 4 | 5 | ```python 6 | from pathlib import Path 7 | from tableauhyperapi import HyperProcess, Telemetry, Connection, CreateMode, \ 8 | TableDefinition, NOT_NULLABLE, SqlType, escape_string_literal 9 | 10 | customer_table = TableDefinition( 11 | # Since the table name is not prefixed with an explicit schema name, the table will reside in the default "public" namespace. 12 | table_name="Customer", 13 | columns=[ 14 | TableDefinition.Column("Customer ID", SqlType.text(), NOT_NULLABLE), 15 | TableDefinition.Column("Customer Name", SqlType.text(), NOT_NULLABLE), 16 | TableDefinition.Column("Loyalty Reward Points", SqlType.big_int(), NOT_NULLABLE), 17 | TableDefinition.Column("Segment", SqlType.text(), NOT_NULLABLE) 18 | ] 19 | ) 20 | 21 | path_to_database = Path("customer.hyper") 22 | 23 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 24 | with Connection(endpoint=hyper.endpoint, 25 | database=path_to_database, 26 | create_mode=CreateMode.CREATE_AND_REPLACE, 27 | parameters=connection_parameters) as connection: 28 | connection.catalog.create_table(customer_table) 29 | 30 | # You can find the sample CSV file in https://github.com/tableau/hyper-api-samples/tree/main/Tableau-Supported/Python/data 31 | path_to_csv = str(Path(__file__).parent / "data" / "customers.csv") 32 | 33 | # Load all rows into the "Customers" table from the CSV file. 34 | # 35 | # You might have to adjust the COPY parameters to the format of your specific csv file. 36 | # The example assumes that your columns are separated with the ',' character 37 | # and that NULL values are encoded via the string 'NULL'. 38 | # Also be aware that the `header` option is used in this example: 39 | # It treats the first line of the csv file as a header and does not import it. 40 | added_row_count = connection.execute_command(f""" 41 | COPY {customer_table.table_name} from {escape_string_literal(path_to_csv)} 42 | WITH (format => 'csv', NULL => 'NULL', delimiter => ',', header => true) 43 | """) 44 | 45 | print(f"{added_row_count} rows were loaded from the CSV file.") 46 | ``` 47 | 48 | 1. Start the `HyperProcess` and connect to the `.hyper` file. 49 | 50 | Just like other extract file operations, anytime you want to work with a `.hyper` extract file, you need to start the `HyperProcess` and then open a connection to create a new file `.hyper`, or to connect to an existing file. 51 | 52 | 2. Define and create the table to contain the CSV data. 53 | 54 | Before you can copy the data from the CSV file, you need to define the table (columns and data types) and create the table in the connection catalog. Note that the schema *must match* the schema in the CSV file. 55 | 56 | 3. Issue the `COPY FROM` command. 57 | 58 | The [COPY FROM](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/copy-from.html) 59 | command instructs Hyper to directly insert data from an external file into a table. 60 | The `COPY` command's `WITH` clause specifies additional details about the file format: In this case, the CSV file uses a comma as the delimiter and has a header row. 61 | 62 | To construct the SQL command and correctly escape the file path, we use `escape_string_literal`. 63 | We then use the `execute_command` method to execute the SQL command. 64 | See [Execute SQL commands](../sql_commands.md) for more information on how to construct and execute SQL commands. 65 | 66 | 4. Check the return value. 67 | 68 | The `execute_command` method returns the number of rows that were added to the table. You can check this value or log or report it in some fashion. If you are adding multiple tables to the `.hyper` file, you could use the same connection to execute additional COPY commands. -------------------------------------------------------------------------------- /website/docs/guides/hyper_file/optimize.md: -------------------------------------------------------------------------------- 1 | # Optimize Hyper File Storage 2 | 3 | Hyper files (`.hyper`) store data in a proprietary, optimized database format. 4 | Like any file system or database that supports write, update, and delete operations, with heavy active use, the data can become fragmented over time resulting in larger files and increased query latencies. 5 | To improve the performance of your `.hyper` extract files, you can follow these guidelines. 6 | 7 | ## Use the latest Hyper file format 8 | 9 | By default, Hyper will always use the database file format version 0 for maximum compatibility. Version 0 is guaranteed to be always supported by every version of Hyper, and you should be able to open these files in very old versions of Hyper or Tableau if you need to. 10 | 11 | However, if you know that you'll only be using currently supported versions of Hyper or Tableau to open your Hyper files, it is recommended to use the latest database file format version. 12 | New file format versions often include relevant performance improvements (for example, later versions improve compression drastically on files with deleted rows). 13 | You can find a list of database file format versions and improvements introduced in these versions [here](../../hyper-api/hyper_process#default_database_version). 14 | 15 | To use a specific database file format version, you'll need to use the `default_database_version` setting. For example, to use format version 2, set `default_database_version=2`. 16 | 17 | Using the latest available database file format version should lessen the need to manually defragment or otherwise optimize your file for size or performance. 18 | However, for extremely fragmented files you might still benefit from manually optimizing your file. 19 | 20 | ## Rewrite your Hyper file in an optimized format 21 | 22 | If you have a Hyper file that has become fragmented or is still using an older file version where you want to take advantage of new version features, you can 23 | update your existing Hyper databases by checking the version and updating the version prior to performing other operations on them. For instance the Python script 24 | below does this while maintaining a backup of the old Hyper file. 25 | 26 | ```python 27 | import os 28 | from tableauhyperapi import HyperProcess, Connection, Telemetry, CreateMode 29 | 30 | TARGET_DATABASE_VERSION = "2" 31 | 32 | with HyperProcess(Telemetry.SEND_USAGE_DATA_TO_TABLEAU, 33 | parameters = {"default_database_version": TARGET_DATABASE_VERSION}) as hyper: 34 | should_update_version = False 35 | with Connection(hyper.endpoint, 'existing.hyper', CreateMode.CREATE_IF_NOT_EXISTS) as connection: 36 | # check the current version of the extract 37 | 38 | version = connection.execute_scalar_query("SELECT database_version from pg_catalog.hyper_database") 39 | if version < TARGET_DATABASE_VERSION: 40 | print(f'found version {version}, upgrading to version {TARGET_DATABASE_VERSION}') 41 | should_update_version = True 42 | 43 | if should_update_version: 44 | with Connection(hyper.endpoint) as connection: 45 | connection.execute_command(f""" 46 | CREATE DATABASE "updatedversion.hyper" WITH VERSION {TARGET_DATABASE_VERSION} FROM "existing.hyper" 47 | """) 48 | # make a backup of the existing hyper file - will overwrite any existing file 49 | os.replace("existing.hyper", "existing.bak.hyper") 50 | 51 | # rename the new file to match old database name 52 | os.replace("updatedversion.hyper", "existing.hyper") 53 | with Connection(hyper.endpoint, 'existing.hyper', CreateMode.CREATE_IF_NOT_EXISTS) as connection: 54 | # perform normal operations on connection 55 | ... 56 | ``` 57 | 58 | ## Guidelines for avoiding fragmentation 59 | 60 | The level of file compression in a `.hyper` file depends both on the characteristics of the contained data but also on the insertion/deletion patterns that you use. If you expect to repeatedly delete, insert, or update rows of data, there are patterns that are more likely to achieve optimal file compression, and others that are more likely to result in file fragmentation. 61 | 62 | You can optimally compress a `.hyper` file if: 63 | 64 | * The deletion precedes the insertion and the file is closed or unloaded in between operations. 65 | 66 | * The deletion affects a continuous section of data. 67 | 68 | The `.hyper` file can become fragmented if you delete noncontinuous ranges of rows. Fragmentation can make it harder to re-use the space where the data was deleted, and can complicate how the `.hyper` file is compressed when it is written to disk. Fragmentation really only becomes an issue when you have large amounts of data. 69 | 70 | In general, it is better to order your data in such a way that deletion and insertion patterns cover the largest possible continuous ranges. If you feel that your `.hyper` file is too fragmented, you can use a Python script that is available on the Hyper API Samples project on GitHub to defragment your file. See [Rewrite your Hyper file in an optimized format](#rewrite-your-hyper-file-in-an-optimized-format). 71 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Salesforce Open Source Community Code of Conduct 2 | 3 | ## About the Code of Conduct 4 | 5 | Equality is a core value at Salesforce. We believe a diverse and inclusive 6 | community fosters innovation and creativity, and are committed to building a 7 | culture where everyone feels included. 8 | 9 | Salesforce open-source projects are committed to providing a friendly, safe, and 10 | welcoming environment for all, regardless of gender identity and expression, 11 | sexual orientation, disability, physical appearance, body size, ethnicity, nationality, 12 | race, age, religion, level of experience, education, socioeconomic status, or 13 | other similar personal characteristics. 14 | 15 | The goal of this code of conduct is to specify a baseline standard of behavior so 16 | that people with different social values and communication styles can work 17 | together effectively, productively, and respectfully in our open source community. 18 | It also establishes a mechanism for reporting issues and resolving conflicts. 19 | 20 | All questions and reports of abusive, harassing, or otherwise unacceptable behavior 21 | in a Salesforce open-source project may be reported by contacting the Salesforce 22 | Open Source Conduct Committee at ossconduct@salesforce.com. 23 | 24 | ## Our Pledge 25 | 26 | In the interest of fostering an open and welcoming environment, we as 27 | contributors and maintainers pledge to making participation in our project and 28 | our community a harassment-free experience for everyone, regardless of gender 29 | identity and expression, sexual orientation, disability, physical appearance, 30 | body size, ethnicity, nationality, race, age, religion, level of experience, education, 31 | socioeconomic status, or other similar personal characteristics. 32 | 33 | ## Our Standards 34 | 35 | Examples of behavior that contributes to creating a positive environment 36 | include: 37 | 38 | * Using welcoming and inclusive language 39 | * Being respectful of differing viewpoints and experiences 40 | * Gracefully accepting constructive criticism 41 | * Focusing on what is best for the community 42 | * Showing empathy toward other community members 43 | 44 | Examples of unacceptable behavior by participants include: 45 | 46 | * The use of sexualized language or imagery and unwelcome sexual attention or 47 | advances 48 | * Personal attacks, insulting/derogatory comments, or trolling 49 | * Public or private harassment 50 | * Publishing, or threatening to publish, others' private information—such as 51 | a physical or electronic address—without explicit permission 52 | * Other conduct which could reasonably be considered inappropriate in a 53 | professional setting 54 | * Advocating for or encouraging any of the above behaviors 55 | 56 | ## Our Responsibilities 57 | 58 | Project maintainers are responsible for clarifying the standards of acceptable 59 | behavior and are expected to take appropriate and fair corrective action in 60 | response to any instances of unacceptable behavior. 61 | 62 | Project maintainers have the right and responsibility to remove, edit, or 63 | reject comments, commits, code, wiki edits, issues, and other contributions 64 | that are not aligned with this Code of Conduct, or to ban temporarily or 65 | permanently any contributor for other behaviors that they deem inappropriate, 66 | threatening, offensive, or harmful. 67 | 68 | ## Scope 69 | 70 | This Code of Conduct applies both within project spaces and in public spaces 71 | when an individual is representing the project or its community. Examples of 72 | representing a project or community include using an official project email 73 | address, posting via an official social media account, or acting as an appointed 74 | representative at an online or offline event. Representation of a project may be 75 | further defined and clarified by project maintainers. 76 | 77 | ## Enforcement 78 | 79 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 80 | reported by contacting the Salesforce Open Source Conduct Committee 81 | at ossconduct@salesforce.com. All complaints will be reviewed and investigated 82 | and will result in a response that is deemed necessary and appropriate to the 83 | circumstances. The committee is obligated to maintain confidentiality with 84 | regard to the reporter of an incident. Further details of specific enforcement 85 | policies may be posted separately. 86 | 87 | Project maintainers who do not follow or enforce the Code of Conduct in good 88 | faith may face temporary or permanent repercussions as determined by other 89 | members of the project's leadership and the Salesforce Open Source Conduct 90 | Committee. 91 | 92 | ## Attribution 93 | 94 | This Code of Conduct is adapted from the [Contributor Covenant][contributor-covenant-home], 95 | version 1.4, available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html. 96 | It includes adaptions and additions from [Go Community Code of Conduct][golang-coc], 97 | [CNCF Code of Conduct][cncf-coc], and [Microsoft Open Source Code of Conduct][microsoft-coc]. 98 | 99 | This Code of Conduct is licensed under the [Creative Commons Attribution 3.0 License][cc-by-3-us]. 100 | 101 | [contributor-covenant-home]: https://www.contributor-covenant.org (https://www.contributor-covenant.org/) 102 | [golang-coc]: https://golang.org/conduct 103 | [cncf-coc]: https://github.com/cncf/foundation/blob/master/code-of-conduct.md 104 | [microsoft-coc]: https://opensource.microsoft.com/codeofconduct/ 105 | [cc-by-3-us]: https://creativecommons.org/licenses/by/3.0/us/ -------------------------------------------------------------------------------- /website/docs/guides/hyper_file/create_update.md: -------------------------------------------------------------------------------- 1 | # Create and Update Hyper Files 2 | 3 | Hyper stores its tables in `.hyper` files. Using Hyper API, you can create `.hyper` files and then insert, delete, update, and read data from those files. 4 | You can then use `.hyper` files as data sources in Tableau, across the whole family of Tableau offerings (Tableau Desktop, Tableau Cloud, Tableau Prep, ...). 5 | Thereby, putting your data into a `.hyper` file is a good way to prepare your data for analyzing it with Tableau. 6 | 7 | This guide will outline the basic steps for creating and updating data within a `.hyper` file. 8 | 9 | :::tip Use Pandas to Load Your Data 10 | While this guide shows you the most versatile way to load your data, using the [integration with the pandas data framework](../pandas_integration.md#loading-data-through-pandas) might get you to your goal more quickly. 11 | ::: 12 | 13 | ## Creating a Hyper File 14 | 15 | The following script creates a simple Hyper file with a single table: 16 | 17 | ```python 18 | from tableauhyperapi import HyperProcess, Connection, Telemetry, CreateMode, \ 19 | TableDefinition, TableName, SqlType, Inserter 20 | 21 | with HyperProcess(Telemetry.SEND_USAGE_DATA_TO_TABLEAU, 22 | parameters = {"default_database_version": "2"}) as hyper: 23 | with Connection(hyper.endpoint, 'TrivialExample.hyper', CreateMode.CREATE_AND_REPLACE) as connection: 24 | # Create an `Extract` table inside an `Extract` schema 25 | connection.catalog.create_schema('Extract') 26 | example_table = TableDefinition(TableName('Extract','Extract'), [ 27 | TableDefinition.Column('rowID', SqlType.big_int()), 28 | TableDefinition.Column('value', SqlType.big_int()), 29 | ]) 30 | connection.catalog.create_table(example_table) 31 | 32 | # Insert data using the `Inserter` class 33 | with Inserter(connection, example_table) as inserter: 34 | for i in range (1, 101): 35 | inserter.add_row( 36 | [ i, i ] 37 | ) 38 | inserter.execute() 39 | ``` 40 | 41 | The script consist of 3 high-level steps: 42 | 43 | 1. Start a Hyper process. The [`HyperProcess`](../../hyper-api/hyper_process) 44 | 2. Create a connection to the `.hyper` file. Since we create the [`Connection`](../../hyper-api/connection) class with the `CreateMode.CREATE_AND_REPLACE`, the `.hyper` file will be automatically created if it does not exist yet, and will be overwritten if a file with that name already exists. 45 | 3. Defining the table. In this case, we are using the Python utilities `TableDefinition` and `catalog.create_table`. We could have also used a [CREATE TABLE](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/create-table.html) SQL command. 46 | 4. Insert the data. In the example, we use the `Inserter` utility to provide the data from Python. You can also use [INSERT](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/insert.html) or [COPY](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/copy-from.html) statements or any other means to load data into the table. E.g., you can thereby directly load your table from a CSV file. 47 | 48 | #### File Format Versions 49 | 50 | By default, Hyper will use the initial default file format version 0. 51 | We recommend to use a more recent file format version, as shown in the example. 52 | In general, all reasonably up-to-date versions of Tableau should be able to read file format version 2, but if you are still using (very) outdated Tableau products, you might want to use file format 0 instead 53 | To learn more about the available versions and product support, see the [Hyper Process Settings](../../hyper-api/hyper_process#default_database_version). 54 | 55 | ## Update an existing extract file 56 | 57 | The workflow for updating an existing extract is similar to the workflow for the basic creation of an extract. 58 | 59 | 1. Start the `HyperProcess`. 60 | 2. Connect to the database (`.hyper` file) using the `Connection` object. 61 | 3. Append, insert, or update data in the table(s). 62 | 63 | The main difference when connecting is that you use `CreateMode.NONE` instead of `CreateMode.CREATE_AND_REPLACE`. 64 | By using `CreateMode.NONE`, Hyper will connect to a pre-existing file instead of recreating a new, empty file. 65 | Since the default for `CreateMode` is `NONE`, you can also just leave this parameter out completely. 66 | 67 | You can then use SQL commands ([INSERT](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/insert.html), [UPDATE](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/update.html), [DELETE](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/delete.html), [COPY](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/copy-from.html) or the `Inserter` utility class to change the data in the table. 68 | You could also create new tables or drop existing tables. 69 | 70 | The following example removes rows with a `value < 50` and appends two new row to an existing table within an extract file: 71 | 72 | ```python 73 | from tableauhyperapi import HyperProcess, Connection, Telemetry, CreateMode, Inserter 74 | 75 | with HyperProcess(Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 76 | with Connection(hyper.endpoint, 'TrivialExample.hyper', CreateMode.NONE) as connection: 77 | # Delete every row where `value < 50` 78 | connection.execute_command(""" 79 | DELETE FROM "Extract"."Extract" WHERE value < 50 80 | """) 81 | 82 | # Insert two new rows 83 | with Inserter(connection, TableName('Extract','Extract')) as inserter: 84 | inserter.add_row([101, 101]) 85 | inserter.add_row([102, 102]) 86 | inserter.execute() 87 | ``` 88 | -------------------------------------------------------------------------------- /website/docs/hyper-api/connection.md: -------------------------------------------------------------------------------- 1 | # The `Connection` 2 | 3 | To connect to a running `HyperProcess`, use the `Connection` class. 4 | The `Connection` class then provides you means to interact with Hyper by sending SQL commands or by using some other utilities like the `catalog` or the `Inserter` class. 5 | 6 | ```python 7 | from tableauhyperapi import HyperProcess, Telemetry, Connection 8 | 9 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 10 | with Connection(endpoint=hyper.endpoint) as connection: 11 | print(connection.execute_scalar_query("SELECT 1+3")) 12 | ``` 13 | 14 | The `Connection` can be created from an endpoint string. 15 | The endpoint string specifies how to connect to Hyper (which protocol to use, which port to use, ...) and can be obtained from the `HyperProcess.endpoint` property. 16 | Multiple connections can use the same `HyperProcess` instance, and Hyper is able to serve multiple SQL queries in parallel. 17 | However, each `Connection` must only ever be used on a single thread. 18 | `Connection` objects are not thread-safe. 19 | 20 | Just like you should shutdown the `HyperProcess`, you should always close the connection when your application is finished interacting with the `.hyper` file. 21 | If you create the connection using a `with` statement (in Python; `using` in C#, a RAII scope in C++, or `try-with-resources`in Java), the connection will be closed at the end of the `with`. 22 | Alternatively, we could also call `connection.close()` to explicitly close the connection. 23 | 24 | ## Sending SQL queries {#sending-sql-commands} 25 | 26 | The Connection class provides methods for executing SQL statements and queries. 27 | For more details, see [Executing SQL Commands](../guides/sql_commands.md) 28 | 29 | ## Connecting to Hyper files 30 | 31 | By default, a `Connection` is connected against no Hyper file at all. 32 | This means that commands such as `CREATE TABLE` will fail, because there is no corresponding `.hyper` file into which Hyper could store that table. 33 | Those database-less connections are still useful to, e.g., interact with external Parquet files, though. 34 | 35 | To connect with a Hyper file, pass its file path as the `database` parameter to the `Connection` constructor. 36 | 37 | ```python 38 | from tableauhyperapi import HyperProcess, Telemetry, Connection, CreateMode 39 | 40 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 41 | with Connection(hyper.endpoint, 'TrivialExample.hyper', CreateMode.NONE) as connection: 42 | print(connection.execute_scalar_query('SELECT COUNT(*) FROM "my_table"')) 43 | ``` 44 | 45 | :::tip Hyper file extension 46 | 47 | While Hyper itself allows database files with arbitrary file extensions, we highly 48 | recommend using the `.hyper` file extension. This is also the only file extension 49 | under which Hyper files are supported in Tableau products. 50 | 51 | ::: 52 | 53 | In addition, the `CreateMode` specifies how Hyper should react if the given Hyper file does not exist: 54 | 55 | Mode | Behavior 56 | ---- | ---- 57 | `NONE` | The database file will not be created. It is expected that the database already exists. An error will be raised if it does not exist 58 | `CREATE` | The database will be created. It is expected that the database does not exist. An error will be raised if the file already exists; a pre-existing file will not be overwritten. 59 | `CREATE_AND_REPLACE` | Create an empty database. If the database file already exists, replace it by a new, empty database. 60 | `CREATE_IF_NOT_EXISTS` | If the database file already exists, connect to it. Otherwise create a new, empty database and connect to it. 61 | 62 | :::caution Single HyperProcess per file 63 | 64 | A `.hyper` file can only be opened by one process at a time. 65 | That is, while your application is connected to the `.hyper` file, it has exclusive access: no other instance of Hyper can connect to the file. 66 | For example, you can't have a `.hyper` file opened in Tableau and at the same time use the Hyper API to read from or write to the same file. 67 | You can, however, open multiple connections to the same `.hyper` file at the same time by reusing the same `HyperProcess` instance. 68 | 69 | ::: 70 | 71 | ## Connection settings 72 | 73 | Connection settings only apply to a single connection. 74 | Other connections to the same Hyper process are not affected. 75 | They can be set during connection startup. 76 | With the Hyper API, they can be passed to the `Connection` constructor. 77 | 78 | 79 | ### Date and Time Settings {#datetimesettings} 80 | 81 | These settings control how date and time data is handled in a Hyper 82 | connection. 83 | 84 | #### lc_time 85 | 86 | Controls the Locale setting that is used for dates. A Locale controls 87 | which cultural preferences the application should apply. For example, 88 | the literal `Januar 1. 2002` can be converted to a date with the German 89 | locale `de` but not with the English locale `en_US`. 90 | 91 | Allowed values start with a [two-letter 92 | ISO-639](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) language 93 | code and an optional [two-letter 94 | ISO-3166](https://en.wikipedia.org/wiki/List_of_ISO_3166_country_codes) 95 | country code. If a country code is used, an underscore has to be used to 96 | separate it from the language code. Some examples are: `en_US` (English: 97 | United States), `en_GB` (English: Great Britain), `de` (German), `de_AT` 98 | (German: Austria). 99 | 100 | :::note 101 | This setting has no influence on the order of day, month, and year 102 | inside a date literal. This is controlled by the 103 | [date_style](#date_style) setting. 104 | ::: 105 | 106 | Default value: `en_US` 107 | 108 | #### date_style 109 | 110 | Controls how date strings are interpreted. `Y`, `M` and `D` stand for 111 | Year, Month, and Day respectively. 112 | 113 | For example, the string "01/02/2000" could be interpreted as "2nd of 114 | January 2000" or "1st of February 2000". The first possibility is chosen 115 | with the `MDY` date style while the second is chosen with the `DMY` date 116 | style. 117 | 118 | This setting also affects date parsing from CSV files. 119 | 120 | Default value: `MDY` 121 | -------------------------------------------------------------------------------- /website/docs/guides/sql_commands.md: -------------------------------------------------------------------------------- 1 | # Execute SQL Commands 2 | 3 | Hyper API is a wrapper around Hyper, a full-fledged SQL database. 4 | As such, you can use SQL queries and commands to interact with Hyper: 5 | 6 | ```python 7 | from tableauhyperapi import HyperProcess, Telemetry, Connection 8 | 9 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 10 | with Connection(endpoint=hyper.endpoint) as connection: 11 | connection.execute_command(""" 12 | CREATE TEMPORARY TABLE animals(name, legs) AS VALUES 13 | ('dog', 4), 14 | ('cat', 4), 15 | ('bird', 2), 16 | ('kangaroo', 2), 17 | ('centipede', 100) 18 | """) 19 | 20 | with connection.execute_query("SELECT name FROM animals") as results: 21 | for row in results: 22 | print(row) 23 | 24 | bipeds = connection.execute_list_query( 25 | "SELECT name FROM animals WHERE legs = 2") 26 | print(bipeds) 27 | 28 | max_legs = connection.execute_scalar_query( 29 | "SELECT MAX(legs) FROM animals") 30 | print("max legs: ", max_legs) 31 | ``` 32 | 33 | First, we need to start a Hyper process and connect to it. 34 | The [`HyperProcess`](../hyper-api/hyper_process.md) and the `Connection` class handle those two aspects for us. 35 | Using the connection object, we can then send SQL commands and SQL queries to Hyper. 36 | 37 | ### Functions for executing SQL queries {#query-functions} 38 | 39 | The Connection class provides methods for executing SQL statements and queries. 40 | There is one method for SQL commands, and three methods for queries which differ primarily in their return value: 41 | 42 | | Method | Returns | 43 | | ---- | ---- | 44 | `execute_command` | The count of affected rows, if available. None if not. | 45 | `execute_query` | A Hyper API `result` object. You can iterate over the returned rows using this object. | 46 | `execute_list_query` | A list of rows. (Python only) | 47 | `execute_scalar_query` | The value from one row, one column. | 48 | 49 | `execute_command` is meant to be used for SQL commands like 50 | [CREATE TABLE](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/create-table.html), 51 | [COPY FROM](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/copy-from.html), 52 | [INSERT](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/insert.html), 53 | [UPDATE](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/update.html), 54 | [DELETE](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/delete.html) etc., all of which don't produce any 55 | result tuples but instead are executed because we are interested in their 56 | side effects. `execute_command` returns the number of rows affected by the 57 | command. 58 | The different `execute_query` variants are meant for SQL queries, 59 | i.e. SQL statements that return actual tuples. 60 | 61 | `execute_query` is the most versatile among the query functions: 62 | Through the returned `result` object, you can access both the result rows 63 | as well as metadata about the query result, such as the names and 64 | data types of the returned columns. 65 | To access the result rows, simply iterate over the result object using 66 | a `for` loop. 67 | Make sure to close the `result` object when it is no longer needed. 68 | In Python, this can be ensured by using a `with` block. 69 | 70 | `execute_list_query` is a convenience method which reads the complete 71 | result of a query into a Python list. 72 | 73 | `execute_scalar_query` is a convenience method meant for queries which 74 | return exactly one row consisting of a single column, and returns 75 | that single result value. 76 | 77 | ## Constructing Hyper SQL statements {#escapings} 78 | 79 | Using Hyper SQL you can, e.g., insert, update, and delete data from tables, import data from 80 | Parquet files or pose arbitrarily complex analytical queries. 81 | For a reference documentation of the supported commands, see our [SQL reference](/docs/sql). 82 | 83 | Because the SQL statements are passed to the Hyper API as strings, you need to ensure that 84 | identifiers and string values are properly encoded. 85 | As long as you are using hardcoding your query strings, this won't be a problem. 86 | You can simply make sure that your query is correctly written: 87 | 88 | ```python 89 | max_legs = connection.execute_scalar_query("SELECT MAX(legs) FROM animals") 90 | ``` 91 | 92 | You can use the full power of Python (or your preferred client language) to 93 | construct your SQL strings. E.g., in Python, formatted string literals 94 | (or "f-strings") are a very useful tool to build SQL commands: 95 | 96 | ```python 97 | # Create 10 tables, just for fun... 98 | for id in range(0, 10): 99 | connection.execute_command(f""" 100 | CREATE TEMPORARY TABLE test_table_{id}(my_column int); 101 | """) 102 | ``` 103 | 104 | However, you must be careful to correctly escape your names and strings. 105 | E.g., the following code will not work 106 | 107 | ```python 108 | table_name = "Guest Names" 109 | value = "Francisco Eduardo" 110 | # This will send an invalid command to Hyper 111 | connection.execute_command(f"INSERT INTO {table_name} VALUES({value})") 112 | ``` 113 | 114 | because the SQL command sent to Hyper would be 115 | 116 | ``` 117 | INSERT INTO Guest Names VALUES(Francisco Eduardo) 118 | ``` 119 | 120 | which is incorrect. The correct query would be 121 | 122 | ``` 123 | INSERT INTO "Guest Names" VALUES('Francisco Eduardo') 124 | ``` 125 | 126 | The table name must be in double quotes and the string constant in single quotes. 127 | 128 | Escaping for identifiers and strings is documented in [General Syntax](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/syntax.html). 129 | Instead of reimplementing those escaping rules by yourself, you can use the `escape_name` 130 | and `escape_string_literal` functions to correctly format identifiers and strings in 131 | your SQL statements. 132 | Use `escape_name` for identifiers, such as column or table names. 133 | Use `escape_string_literal` when you need to use quoted string values. 134 | Furthermore, the utility classes `TableName` etc. are automatically escaped correctly when 135 | formatted as a string. 136 | 137 | For example, if you have a table named `Customers` you could use `{TableName('Customers')}` in the SQL statement. 138 | 139 | ```python 140 | table = TableName("Customers") 141 | for name in ["Dennis Kane", "Dorothe Hagen"]: 142 | row_count = connection.execute_command(f""" 143 | DELETE FROM {table} 144 | WHERE "Name" = {escape_string_literal(name)} 145 | """) 146 | ``` 147 | -------------------------------------------------------------------------------- /website/docusaurus.config.js: -------------------------------------------------------------------------------- 1 | // @ts-check 2 | // Note: type annotations allow type checking and IDEs autocompletion 3 | 4 | const lightCodeTheme = require('prism-react-renderer/themes/github'); 5 | 6 | const getConfig = async () => { 7 | const remarkDefList = (await import("remark-deflist")).default; 8 | 9 | const isUpcomingVersion = !!process.env.IS_UPCOMING 10 | const isInofficial = process.env.GITHUB_ORIGIN !== 'https://tableau.github.io'; 11 | 12 | var title = 'Hyper'; 13 | if (isInofficial) title = 'Inofficial Hyper'; 14 | else if (isUpcomingVersion) title = 'Pre-Release Hyper'; 15 | 16 | /** @type {import('@docusaurus/types').Config} */ 17 | return { 18 | title: title, 19 | tagline: 'The SQL Database for Interactive Analytics of the Freshest State of Data.', 20 | favicon: 'img/favicon.ico', 21 | 22 | // Pick up URL and baseUrl from config parameters set by the Github action runner 23 | url: process.env.GITHUB_ORIGIN ?? 'http://localhost/', 24 | baseUrl: (process.env.GITHUB_BASE_PATH ?? '') + '/', 25 | 26 | // We don't want to have the preview version or clones of the Github repository 27 | // indexed by search engines 28 | noIndex: isUpcomingVersion || isInofficial, 29 | 30 | // We want all issues to be reported as build errors 31 | onBrokenLinks: 'throw', 32 | onBrokenMarkdownLinks: 'throw', 33 | 34 | // Even if you don't use internalization, you can use this field to set useful 35 | // metadata like html lang. For example, if your site is Chinese, you may want 36 | // to replace "en" with "zh-Hans". 37 | i18n: { 38 | defaultLocale: 'en', 39 | locales: ['en'], 40 | }, 41 | 42 | presets: [ 43 | [ 44 | 'classic', 45 | /** @type {import('@docusaurus/preset-classic').Options} */ 46 | { 47 | docs: { 48 | sidebarPath: require.resolve('./sidebars.js'), 49 | // Please change this to your repo. 50 | // Remove this to remove the "edit this page" links. 51 | editUrl: 52 | 'https://github.dev/tableau/hyper-db/blob/main/website/', 53 | remarkPlugins: [remarkDefList], 54 | }, 55 | theme: { 56 | customCss: require.resolve('./src/css/custom.css'), 57 | }, 58 | }, 59 | ], 60 | ], 61 | 62 | themeConfig: 63 | /** @type {import('@docusaurus/preset-classic').ThemeConfig} */ 64 | ({ 65 | /* We intentionally don't have a social card, as we are not 66 | * happy with the design, yet */ 67 | // image: 'img/hyper-social-card.png', 68 | navbar: { 69 | title: 'Hyper', 70 | logo: { 71 | alt: 'Hyper', 72 | src: 'img/hyper-logo.svg', 73 | }, 74 | items: [ 75 | { 76 | to: '/journey', 77 | label: 'Our Journey', 78 | }, 79 | { 80 | to: '/docs', 81 | position: 'left', 82 | label: 'Documentation', 83 | }, 84 | ], 85 | }, 86 | colorMode: { 87 | defaultMode: 'light', 88 | disableSwitch: true, 89 | }, 90 | footer: { 91 | style: 'dark', 92 | links: [ 93 | { 94 | title: 'Docs', 95 | items: [ 96 | { 97 | label: 'Releases', 98 | to: '/docs/releases', 99 | }, 100 | { 101 | label: 'Installation', 102 | to: '/docs/installation', 103 | }, 104 | { 105 | label: 'Guides', 106 | to: '/docs/guides', 107 | }, 108 | { 109 | label: 'SQL Reference', 110 | to: '/docs/sql', 111 | }, 112 | ], 113 | }, 114 | { 115 | title: 'More', 116 | items: [ 117 | { 118 | label: 'GitHub', 119 | href: 'https://github.com/tableau/hyper-db', 120 | }, 121 | { 122 | label: 'Slack', 123 | href: 'https://join.slack.com/t/tableau-datadev/shared_invite/zt-1q4rrimsh-lHHKzrhid1MR4aMOkrnAFQ', 124 | }, 125 | { 126 | label: 'Getting Help', 127 | href: '/docs/faq#getting-help', 128 | }, 129 | ], 130 | }, 131 | { 132 | title: 'Legal', 133 | items: [ 134 | { 135 | label: 'Legal', 136 | href: 'https://www.salesforce.com/company/legal/', 137 | }, 138 | { 139 | label: 'Terms of Service', 140 | href: 'https://www.salesforce.com/company/legal/sfdc-website-terms-of-service/', 141 | }, 142 | { 143 | label: 'Privacy', 144 | href: 'https://www.salesforce.com/company/privacy/' 145 | }, 146 | { 147 | label: 'Responsible Disclosure', 148 | href: 'https://www.salesforce.com/company/legal/disclosure/', 149 | }, 150 | { 151 | label: 'Trust', 152 | href: 'https://trust.salesforce.com/', 153 | }, 154 | { 155 | html: `Cookie Preferences`, 156 | } 157 | ] 158 | }, 159 | ], 160 | copyright: `Copyright © ${new Date().getFullYear()} Salesforce, Inc. Built with Docusaurus.`, 161 | }, 162 | prism: { 163 | theme: lightCodeTheme, 164 | }, 165 | announcementBar: isUpcomingVersion ? { 166 | content: 167 | 'You are browsing a preview of the documentation for the upcoming Hyper version.', 168 | backgroundColor: '#a00', 169 | textColor: '#fff', 170 | isCloseable: false, 171 | } : undefined, 172 | }), 173 | 174 | plugins: [ 175 | [ 176 | '@docusaurus/plugin-google-tag-manager', 177 | { 178 | containerId: 'GTM-BVCN', 179 | }, 180 | ], 181 | [ 182 | '@docusaurus/plugin-google-gtag', 183 | { 184 | trackingID: '469571326', 185 | anonymizeIP: true, 186 | }, 187 | ], 188 | ], 189 | 190 | clientModules: [ 191 | require.resolve('./_analytics.js'), 192 | ], 193 | }; 194 | } 195 | 196 | module.exports = getConfig; 197 | -------------------------------------------------------------------------------- /website/docs/guides/hyper_file/geodata.md: -------------------------------------------------------------------------------- 1 | # Add Geospatial Data to a Hyper File 2 | 3 | Tableau supports geospatial data (`tableau.tabgeography`) in `.hyper` files. 4 | This guide describes how you can use the Hyper API to add geospatial data to Hyper files. 5 | 6 | The Hyper API does not directly accept Well-known text (WKT) for geospatial data. 7 | Instead you need to use a `CAST("column_as_text" AS TABLEAU.TABGEOGRAPHY)` expression in the inserter to provide the geospatial data as `text` strings. 8 | Hyper API's inserter pushes the `CAST("column_as_text" AS TABLEAU.TABGEOGRAPHY)` expression down to Hyper where the `text` strings are converted to geospatial data. 9 | 10 | ## Overview of inserting geospatial data into a hyper file 11 | 12 | The basic process for adding geospatial data involves defining your inputs to Hyper APIs inserter and specifying how to convert the text strings to the `tableau.tabgeography` type using Hyper SQL expressions. Hyper APIs inserter pushes the expression down to Hyper to convert a text string to geospatial data on the fly during insertion. 13 | 14 | When you add the text strings into the Hyper file, the text must be in the Well-known text (WKT) format for geospatial data. The WKT format is defined by the Open GIS Consortium, Inc, in the [*OpenGIS Simple Features Specification For SQL*](https://www.opengeospatial.org/standards/sfa). The types include **Point**, **MultiPoint**, **LineString**, **MultiLineString**, **Polygon**, and **MultiPolygon**. 15 | 16 | ## Create tables for text and geospatial data 17 | 18 | 1. Define and create a table in the `.hyper` file with a `SqlType.tabgeography()` column. This is the table that you will use in Tableau. For example, the following Python code snippet creates a table to hold location data. The table is called `Extract` and is in the `Extract` namespace (or schema) similar to Hyper files created by Tableau. 19 | 20 | ```python 21 | connection.catalog.create_schema('Extract') 22 | geo_table = TableDefinition(TableName('Extract','Extract'), [ 23 | TableDefinition.Column('Name', SqlType.text(), nullability=NOT_NULLABLE), 24 | TableDefinition.Column('Location', SqlType.tabgeography(), nullability=NOT_NULLABLE), 25 | ]) 26 | connection.catalog.create_table(geo_table) 27 | ``` 28 | 29 | 2. Define your inputs to the inserter as a List of `TableDefinition.Column`. This definition will be similar to the TableDefinition of the `Extract` table created before except that the columns with `SqlType.tabgeography()` will be specified as `SqlType.text()` type 30 | 31 | ```python 32 | # Inserter definition contains the column definition for the values that are inserted 33 | # The data input has two text values Name and Location_as_text 34 | inserter_definition = [ 35 | TableDefinition.Column(name='Name', type=SqlType.text(), nullability=NOT_NULLABLE), 36 | TableDefinition.Column(name='Location_as_text', type=SqlType.text(), nullability=NOT_NULLABLE)] 37 | ``` 38 | 39 | 3. Specify the conversion of `SqlType.text()` to `SqlType.tabgeography()` using `CAST` expression in `Inserter.ColumnMapping`. Specify all columns into which data is inserter in `Inserter.ColumnMapping` list. For columns that do not require any transformations provide only the names 40 | 41 | ```python 42 | column_mappings = [ 43 | 'Name', 44 | Inserter.ColumnMapping('Location', f'CAST({escape_name("Location_as_text")} AS TABLEAU.TABGEOGRAPHY)') 45 | ] 46 | ``` 47 | 48 | ## Insert the geospatial data as text (WKT) into the text table 49 | 50 | When you add the text data to the Hyper file, the text must be in the Well-known text (WKT) format for geospatial data, such as Point, Polygon, etc. For example, to specify location data, you would use `point(Longitude Latitude)`. 51 | 52 | The following Python code example inserts two rows of data with location information into a table that is defined to hold geospatial data. 53 | 54 | ```python 55 | data_to_insert = [ 56 | [ 'Seattle', "point(-122.338083 47.647528)" ], 57 | [ 'Munich' , "point(11.584329 48.139257)" ] 58 | ] 59 | 60 | with Inserter(connection, geo_table, column_mappings, inserter_definition = inserter_definition) as inserter: 61 | inserter.add_rows(rows=data_to_insert) 62 | inserter.execute() 63 | ``` 64 | 65 | Note if you have WKT data in a comma-separated value (CSV) file, you can use the [COPY](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/copy-from.html) command to insert the data from a CSV file. The command automatically converts the WKT strings to the `tableau.tabgeography` data type. For more information, see the [Example code using copy from CSV](#example-code-using-copy-from-csv) and the Help topic [Insert Data Directly from CSV Files](./insert_csv) and the CSV sample on GitHub, [hyper-api-samples](https://github.com/tableau/hyper-api-samples). 66 | 67 | ## Example code using the Inserter 68 | 69 | The following example Python code illustrates how you can create a `.hyper` file that contains location (`tableau.tabgeography`) information by using expressions in the Inserter. 70 | 71 | ```python 72 | from tableauhyperapi import Connection, HyperProcess, SqlType, TableDefinition, \ 73 | escape_string_literal, escape_name, NOT_NULLABLE, Telemetry, Inserter, CreateMode, TableName 74 | 75 | 76 | with HyperProcess(Telemetry.SEND_USAGE_DATA_TO_TABLEAU, 'myapp' ) as hyper: 77 | with Connection(hyper.endpoint, 'GeospatialExample.hyper', CreateMode.CREATE_AND_REPLACE) as connection: 78 | # Create a table with a `tableau.tabgeography` column 79 | connection.catalog.create_schema('Extract') 80 | geo_table = TableDefinition(TableName('Extract','Extract'), [ 81 | TableDefinition.Column('Name', SqlType.text(), nullability=NOT_NULLABLE), 82 | TableDefinition.Column('Location', SqlType.tabgeography(), nullability=NOT_NULLABLE), 83 | ]) 84 | print("The geo_table is defined.") 85 | connection.catalog.create_table(geo_table) 86 | 87 | # Inserter definition contains the column definition for the values that are inserted 88 | # The data input has two text values Name and Location_as_text 89 | inserter_definition = [ 90 | TableDefinition.Column(name='Name', type=SqlType.text(), nullability=NOT_NULLABLE), 91 | TableDefinition.Column(name='Location_as_text', type=SqlType.text(), nullability=NOT_NULLABLE)] 92 | 93 | # Column 'Name' is inserted into "Extract"."Extract" as-is. 94 | # Column 'Location' in "Extract"."Extract" of `tableau.tabgeography` type is computed from Column 'Location_as_text' of `text` type 95 | # using the expression 'CAST("Location_as_text") AS TABLEAU.TABGEOGRAPHY'. 96 | # Inserter.ColumnMapping is used for mapping the CAST expression to Column 'Location'. 97 | column_mappings = [ 98 | 'Name', 99 | Inserter.ColumnMapping('Location', f'CAST({escape_name("Location_as_text")} AS TABLEAU.TABGEOGRAPHY)') 100 | ] 101 | 102 | # Format the data as Well-known text (WKT) 103 | data_to_insert = [ 104 | [ 'Seattle', "point(-122.338083 47.647528)" ], 105 | [ 'Munich' , "point(11.584329 48.139257)" ] 106 | ] 107 | 108 | # Insert data into "Extract"."Extract" table with CAST expression. 109 | with Inserter(connection, geo_table, column_mappings, inserter_definition = inserter_definition) as inserter: 110 | inserter.add_rows(rows=data_to_insert) 111 | inserter.execute() 112 | print("The data was added to the table.") 113 | ``` 114 | 115 | ## Example code using copy from CSV 116 | 117 | When you copy the text data from a CSV file to the Hyper file, the text data is converted to geospatial data. Just as with the Inserter, the data must be in the Well-known text (WKT) format for geospatial data, such as Point, Polygon, etc. For example, to specify location data, you would use `point(Longitude Latitude)`. 118 | 119 | The following Python code example copies two rows of data from a CSV file into a table that is defined to hold geospatial data. The location data is in a CSV file (`locations.csv`) that looks like the following: 120 | 121 | ```csv title=locations.csv 122 | Name, Location 123 | Seattle, point(-122.338083 47.647528) 124 | Munich , point(11.584329 48.139257) 125 | ``` 126 | 127 | ```python title=import_geo_csv.py 128 | from tableauhyperapi import Connection, HyperProcess, SqlType, TableDefinition, \ 129 | escape_string_literal, escape_name, NOT_NULLABLE, Telemetry, Inserter, CreateMode, TableName 130 | 131 | # CSV file that contains location data in Well-known text (WKT) format 132 | path_to_csv = "locations.csv" 133 | 134 | with HyperProcess(Telemetry.SEND_USAGE_DATA_TO_TABLEAU, 'myapp' ) as hyper: 135 | with Connection(hyper.endpoint, 'GeospatialFromCSVExample.hyper', CreateMode.CREATE_AND_REPLACE) as connection: 136 | # Create a table with a `tableau.tabgeography` column 137 | connection.catalog.create_schema('Extract') 138 | geo_table = TableDefinition(TableName('Extract','Extract'), [ 139 | TableDefinition.Column('Name', SqlType.text(), nullability=NOT_NULLABLE), 140 | TableDefinition.Column('Location', SqlType.tabgeography(), nullability=NOT_NULLABLE)]) 141 | connection.catalog.create_table(geo_table) 142 | 143 | # Load all rows into the geo_table from the CSV file. 144 | # `execute_command` executes a SQL statement and returns the impacted row count. 145 | count_in_geo_table = connection.execute_command( 146 | command=f"COPY {geo_table.table_name} from {escape_string_literal(path_to_csv)} with " 147 | f"(format csv, NULL 'NULL', delimiter ',', header)") 148 | print(f"The number of rows in table {geo_table.table_name} is {count_in_geo_table}.") 149 | ``` 150 | -------------------------------------------------------------------------------- /website/docs/installation.md: -------------------------------------------------------------------------------- 1 | --- 2 | description: Advanced installation instructions 3 | --- 4 | 5 | # Installation 6 | 7 | 8 | ```mdx-code-block 9 | import Tabs from '@theme/Tabs'; 10 | import TabItem from '@theme/TabItem'; 11 | import { Link } from "react-router-dom"; 12 | ``` 13 | 14 | Hyper API is available for Python, C++ and Java supporting Windows, Mac and Linux each. 15 | Depending on the language, the installation can be complex. 16 | This page contains the detailed requirements and installation instructions for all languages. 17 | 18 | ## License 19 | 20 | The Hyper API packages are released under the Apache 2.0 License. 21 | The exact license text can be found inside the packages, after unzipping. 22 | The documentation is licensed under the MIT License. 23 | The [source code of the documentation](https://github.com/tableau/hyper-db) can be found on GitHub. 24 | 25 | 26 | ## Hardware requirements 27 | 28 | The Hyper API only supports 64-bit platforms. It has the following minimum hardware requirements: 29 | 30 | - Intel Nehalem, Apple Silicon or AMD Bulldozer processor or newer 31 | - 2 GB memory 32 | - 1.5 GB minimum free disk space 33 | 34 | 35 | ## Supported platforms 36 | 37 | - macOS 10.13 or newer (for Intel) 38 | - macOS 13.0 or newer (for Apple Silicon) 39 | - Microsoft Windows 8 or newer (64-bit) 40 | - Microsoft Windows Server 2016, 2012, 2012 R2, 2008 R2, 2019, 2022 41 | - Ubuntu 18.04 LTS and newer 42 | - Amazon Linux 2023, Red Hat Enterprise Linux (RHEL) 8.3+ and 9.x+, Oracle Linux 8.3+ and 9.x+, CentOS Stream 8.3+ and 9.x 43 | 44 | ## Language-specific Requirements 45 | 46 | ```mdx-code-block 47 | 48 | 49 | ``` 50 | 51 | * **[Python 3.8](https://www.python.org/downloads/)** (or newer) is required. 52 | - Install the 64-bit version (for example, for Windows, the download is listed as `Windows x86-64`). 53 | - On Windows, we recommend to select the option **Add Python 3.x to PATH** during installation. 54 | 55 | ```mdx-code-block 56 | 57 | 58 | ``` 59 | 60 | * The C++ headers use C++11, so a **standard-compliant compiler with thorough C++11 support** is required. Our internal testing is done with the newest Clang. 61 | 62 | :::tip 63 | Using at least C++17 is recommended, as the API uses C++17 classes, such as `std::optional` and `std::string_view`. If the headers are compiled with a standard older than C++17, interface-compatible classes of the API will be used instead. 64 | ::: 65 | 66 | * For the examples, a CMake project file is provided. To build it, **[CMake](https://cmake.org/download/)** must be installed on your computer and available in your PATH. 67 | * On Windows, you also need Visual Studio 2015 (or newer). 68 | 69 | ```mdx-code-block 70 | 71 | 72 | ``` 73 | 74 | * **[JDK 8](https://www.oracle.com/java/technologies/downloads/)** (or newer) is required. 75 | 76 | ```mdx-code-block 77 | 78 | 79 | ``` 80 | 81 | ## Instructions 82 | 83 | ```mdx-code-block 84 | 85 | 86 | ``` 87 | 88 | :::note 89 | The following instructions assume that you have set up a virtual environment for Python. For more information on creating virtual environments, see [venv - Creation of virtual environments](https://docs.python.org/3/library/venv.html) in the Python Standard Library. 90 | ::: 91 | 92 | 1. Open a terminal and navigate to the `venv` directory. 93 | 94 | 1. Install the **[tableauhyperapi](https://pypi.org/project/tableauhyperapi/)** module using **pip**. 95 | 96 | ``` 97 | pip install tableauhyperapi 98 | ``` 99 | 100 | If you previously installed the `tableauhyperapi`, you can upgrade to the latest version using the following command. 101 | 102 | ``` 103 | pip install --upgrade tableauhyperapi 104 | ``` 105 | 106 | :::warning 107 | Linux installations require `pip` version 19.3 or newer. Note that `pip` versions 20.0 and 20.1 are not working because of issues with `pip`. 108 | ::: 109 | 110 | Alternatively, you can [download](/docs/releases#download) the Python **Hyper API wheel package (`.whl` file)** for your operating system. Use `pip` to install the `.whl` file you downloaded. 111 | * On Windows: `Scripts\pip install [*path_to_whl_file*]` 112 | * On Linux/macOS: `bin/pip install [*path_to_whl_file*]` 113 | 114 | 1. Try out the examples: 115 | - Download the [samples from Github](https://github.com/tableau/hyper-api-samples). 116 | - Run the Python examples (for example, try `insert_data_into_single_table.py`). 117 | - On Windows: 118 | `[venv_directory]\Scripts\python hyper-api-samples\Tableau-Supported\Python\insert_data_into_single_table.py` 119 | - On Linux/macOS: 120 | `[venv_directory]/bin/python hyper-api-samples/Tableau-Supported/Python/insert_data_into_single_table.py` 121 | 122 | ```mdx-code-block 123 | 124 | 125 | ``` 126 | 127 | 1. [Download](/docs/releases#download) the C++ Hyper API package file (`.zip` file) for your operating system. 128 | 129 | 1. Unzip the Hyper API package file to a convenient location. 130 | 131 | 1. To build and run the examples on the command line: 132 | - Open a terminal. On Windows, open the **VS2019 x64 Native Tools Command Prompt**. 133 | - Navigate into the `examples` directory of the extracted C++ Hyper API package. 134 | - Configure the project with `cmake`: 135 | - On Windows: `cmake -G "Visual Studio 16 2019 Win64" .` 136 | - On Linux/macOS: `cmake .` 137 | - Build the examples: 138 | - On Windows: `cmake --build . --config Debug` 139 | or `cmake --build . --config Release` 140 | - On Linux/macOS: `cmake --build .` 141 | - Run the examples 142 | - On Windows: `ctest --verbose -C Debug` 143 | or `ctest --verbose -C Release` 144 | - On Linux/macOS: `ctest --verbose` 145 | 146 | 1. To build and run the examples in Visual Studio, go to **File > Open > Folder** and select the `examples` directory of the `tableauhyperapi-cxx` package. 147 | 148 | 1. If you want to build your own applications, you need to install the Hyper API library on your computer. To do this: 149 | - On Windows: Append the extracted bin directory to your system `PATH` variable. 150 | - On Linux: Append the extracted lib directory to your system `LD_LIBRARY_PATH` variable. 151 | - On macOS: You need to include the lib directory as `RPATH` for every binary. 152 | 153 | ```mdx-code-block 154 | 155 | 156 | ``` 157 | 158 | 1. [Download](/docs/releases#download) the Java Hyper API package file (`.zip` file) for your operating system. 159 | 160 | 1. Unzip the Hyper API package file to a convenient location. 161 | 162 | 1. To build and run the examples on the command line: 163 | - Open a terminal and navigate into the `examples` directory of the extracted Java Hyper API package. 164 | - Build the examples: `gradle build` 165 | - Run the examples: `gradle run` 166 | 167 | 1. You can import the example project into IntelliJ or Eclipse: 168 | - IntelliJ: 169 | - Import the Hyper API into IntelliJ by opening the `build.gradle` file from the unzipped Hyper API package. 170 | - In the **Gradle** view, choose an example and select **Tasks > application > run**. 171 | - Eclipse: 172 | - You need to have a recent version of Eclipse installed to use Gradle. 173 | - Import the Hyper API into Eclipse as an existing Gradle project (**File > Import > Gradle > Existing Gradle Project**). 174 | - Select the directory of the unzipped Hyper API package as the "Project root directory". 175 | - Select **Override workspace settings** in the "Import Options" dialog and use at least Gradle version 5.5. 176 | - In the **Gradle Tasks** view, choose an example and select **application > run**. 177 | 178 | 1. You can create new projects that use the Hyper API in IntelliJ or Eclipse: 179 | - Create a new Java project. Make sure the JDK is set to Java 8, that is, **JavaSDK-1.8**. 180 | - Add the following libraries from the `lib` directory of the unzipped Hyper API package as external JARs: 181 | `tableauhyperapi.jar` 182 | `tableauhyperapi-windows.jar` (Windows), 183 | `tableauhyperapi-linux.jar` (Linux), 184 | `tableauhyperapi-darwin.jar` (macOS) 185 | `jna-5.6.0.jar` 186 | - Copy the `hyper` folder from the `lib` directory of the unzipped Hyper API package next to the JARs, as in the example project. 187 | - To enable proper Javadoc, add the `hapi-javadoc.jar` from the lib directory as the Javadoc archive. 188 | 189 | 190 | ### Java Native Access Library 191 | 192 | If your security requirements require you to run Java applications with the system property `jna.nounpack` set to `true`, which disables unpacking from a `.jar` file, you need to obtain the native Hyper API library in another way. While you could extract the library from the `.jar` file in the Java Hyper API package, the easiest way is to download and unzip the C++ Hyper API package for your platform, as described in the following steps: 193 | 194 | 1. [Download](/docs/releases#download) `hyperapi-cxx` package for your platform. 195 | 1. Unzip the package and place the native Hyper API library in a directory or folder accessible by the Java application. 196 | * For Windows, the native library (`tableauhyperapi.dll`) file is in the `bin` directory of the `.zip` file. 197 | * For Linux, the library (`libtableauhyperapi.so`) is in the `lib` directory of the `.zip` file. 198 | * For macOS, the library (`libtableauhyperapi.dylib`) is in the `lib` folder of the `.zip` file. 199 | 1. Set system property `jna.library.path` with value set to the absolute path of the folder or directory that contains the native library file for your platform (from step 2). 200 | 201 | ```mdx-code-block 202 | 203 | 204 | ``` 205 | -------------------------------------------------------------------------------- /website/src/pages/journey.tsx: -------------------------------------------------------------------------------- 1 | import React from 'react'; 2 | import { VerticalTimeline, VerticalTimelineElement } from 'react-vertical-timeline-component'; 3 | import useBaseUrl from '@docusaurus/useBaseUrl'; 4 | import Layout from '@theme/Layout'; 5 | import 'react-vertical-timeline-component/style.min.css'; 6 | 7 | import clsx from 'clsx'; 8 | import styles from './index.module.css'; 9 | 10 | export function HyperTimeline() { 11 | return ( 12 | 13 | } date="2008"> 14 |

Hyper Started as a Research Project

15 |

16 | In 2008 and under the guidance of professors Alfons Kemper and Thomas Neumann, the journey of Hyper 17 | started as a research project at the Technical University of Munich 18 | . Strictly speaking, Hyper was then written HyPer with a capital P as an acronym for " 19 | Hybrid High Performance". As part of the Tableau acquisition 20 | in 2016, HyPer became Hyper. Legend has it that{' '} 21 | 22 | a popular 90's techno song called Hyper Hyper 23 | {' '} 24 | had influenced the name finding, but that is in the realm of urban myths. 25 |

26 | 27 | } date="over the next years"> 28 |

Hyper's Success in Academia

29 |

30 | 31 | More than 50 peer-reviewed articles about Hyper 32 | {' '} 33 | have been published in journals as well as in the proceedings of various database conferences and 34 | workshops. Several of these publications and demos have received awards, including awards at the 35 | prestigious IEEE ICDE, ACM SIGMOD, and VLDB conferences. We are especially honored that two of our 36 | foundational publications have received the test of time award, one of the highest distinctions in 37 | the database research community: 38 |

39 |
    40 |
  • 41 |

    42 | The first academic publication on the Hyper database system has been published in the 43 | proceedings of the IEEE International Conference on Data Engineering 2011 (" 44 | 45 | HyPer: A hybrid OLTP&OLAP main memory database system based on virtual memory 46 | snapshots 47 | 48 | ", ICDE 2011) and has received the{' '} 49 | 50 | IEEE ICDE 10-year influential paper award 51 | {' '} 52 | in 2021. 53 |

    54 |
    • 55 |
  • 56 |

    57 | " 58 | 59 | Efficiently Compiling Efficient Query Plans for Modern Hardware 60 | 61 | ", published in the proceedings of the 37th International Conference on Very Large Data 62 | Bases 2011 (VLDB 2011), has received the VLDB 2021 Test of Time Award. This publication 63 | describes the foundation of Hyper's compiling query execution engine and was the first 64 | to introduce data-centric code generation and compilation for query processing on modern 65 | hardware. The then-novel technique that Thomas Neumann had pioneered in Hyper has been 66 | adopted by many of the leading database engines over the past decade. Alongside vectorized 67 | query processing, query compilation is one of the two state-of-the-art processing paradigms 68 | for fast query processing. 69 |

    70 |
    • 71 |
72 | 73 | } date="2015"> 74 |

Hyper Startup

75 |

76 | In 2015, the six co-founders Dr. Jan Finis, Prof. Alfons Kemper, Ph.D., Prof. Dr. Viktor Leis, Dr. 77 | Tobias Muehlbauer, Prof. Dr. Thomas Neumann, and Dr. Wolf Roediger founded the Hyper spinoff startup 78 | with the mission to develop a database that disrupts the way people manage and analyze data on 79 | modern hardware. 80 |

81 | 82 | } date="2016"> 83 |

Tableau acquires Hyper

84 |

85 | In March 2016, Hyper was{' '} 86 | 87 | acquired by Tableau 88 | 89 | . 90 |

91 | 92 | } date="2018"> 93 |

Hyper Powers Tableau

94 |

95 | In January 2018 and after 18 months of integration,{' '} 96 | 97 | Tableau shipped version 10.5 of its products, all powered by Hyper as its new data engine 98 | 99 | . With Hyper's ability to slice and dice massive volumes of data in seconds, Tableau customers 100 | got up to 5X faster query speeds and up to 3X faster extract creation speeds compared to the 101 | previous Tableau data engine. This allowed faster insights for even larger data sets, giving 102 | organizations the ability to scale their analysis to more people. Later in 2018,{' '} 103 | 104 | Tableau launched its novel Prep product powered by Hyper as its data processing engine 105 | 106 | . Prep delivered new data preparation capabilities that brought a direct and visual experience. Data 107 | prep was made simple and the integration with the Tableau analytical workflow allowed people to get 108 | insights from their cleaned and enriched data sets even faster. 109 |

110 | 111 | } date="2019"> 112 |

Launch of Hyper API

113 |

114 | In 2019, Tableau launched Hyper API. The API allows customers and partners to 115 | automate their integrations with Tableau extracts, including the creation of new extract files as 116 | well as inserting, deleting, updating, and reading data from existing extracts. Hyper API was the 117 | first solution to expose Hyper's SQL capabilities to Tableau's end customers and partners. 118 |

119 | 120 | } 122 | date="2021 until today" 123 | > 124 |

Hyper at Salesforce

125 |

Hyper is the Salesforce Database for interactive analytics of the freshest state of data.

126 | 127 | 128 | ); 129 | } 130 | 131 | export default function OurJourney(): JSX.Element { 132 | return ( 133 | <> 134 | 138 |
139 |
140 |
141 |

Hyper's journey

142 |

143 | from a research project towards an industry-hardened database 144 |

145 |
146 |
147 | 148 |
149 | 150 | 151 | ); 152 | } 153 | -------------------------------------------------------------------------------- /website/docs/hyper-api/hyper_process.md: -------------------------------------------------------------------------------- 1 | # The `HyperProcess` 2 | 3 | Hyper itself is a full-fledged, standalone database server (`hyperd`). 4 | Hyper API comes bundled with `hyperd` together with a utility class (`HyperProcess`) which allows you to spawn the Hyper database server locally on your machine. 5 | 6 | To spawn a Hyper process, use: 7 | 8 | ```python 9 | from tableauhyperapi import HyperProcess, Telemetry 10 | 11 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 12 | print(hyper.endpoint) 13 | ``` 14 | 15 | This starts up a local Hyper database server, and then prints the connection string (`endpoint`). 16 | This connection string describes the used protocol (TCP, domain sockets, ...) and the corresponding information like port numbers. 17 | The `Connection` class can then be used to connect against this endpoint. 18 | While the `HyperProcess` is running, you can create and connect to as many `.hyper` files as you want. 19 | 20 | After you no longer need a Hyper database server, you should shutdown the `HyperProcess`. 21 | If you call the `HyperProcess` in a `with` statement (Python), `using` statement (C#), scope (C++), or `try-with-resources` statement (Java), the `hyperd` process will safely shutdown automatically at the end of the `with` statement. 22 | 23 | ## Performance best practices {#performance} 24 | 25 | Compared to other database systems, Hyper starts up very fast (in the order of 100 milliseconds). 26 | Still, starting up and shutting down the server takes time. 27 | Hence, you should keep the process running and only close or shutdown the `HyperProcess` when your application is finished. 28 | E.g., when updating multiple tables inside a `.hyper` file, do not restart the `HyperProcess` for every table, but instead use the same process for updating all of your tables. 29 | 30 | Furthermore, you should only have one instance of Hyper running at any given time. 31 | Hyper internally monitors its memory assumption, and makes sure that it only uses up to 80% of your system's RAM memory, such that your overall system stays responsive. 32 | If multiple Hyper processes are running at the same time, they might overload the system, and Hyper's internal resource management mechanisms will not be able to counteract this. 33 | 34 | ## Telemetry Data {#telemetry} 35 | 36 | The `HyperProcess` can be instructed to send telemetry on Hyper API usage to Tableau. 37 | To send usage data, set `telemetry` to `Telemetry.SEND_USAGE_DATA_TO_TABLEAU` when you start the process. 38 | To opt out, set `telemetry` to `Telemetry.DO_NOT_SEND_USAGE_DATA_TO_TABLEAU`. 39 | 40 | To help us improve Hyper and justify further investments into Hyper API, you can share usage data with us. 41 | Tableau collects data that helps us learn how our products are being used so we can improve existing features and develop new ones. 42 | All usage data is collected and handled according to the [Tableau Privacy Policy](https://tableau.com/privacy). 43 | 44 | ## Locating the `hyperd` binary 45 | 46 | To spawn the `hyperd` executable, `HyperProcess` must be able to locate this binary, first. 47 | By default, `HyperProcess` is able to automatically find the `hyperd` executable bundled inside Hyper API. 48 | However, if you are rebundling Hyper API, this logic might fail. 49 | In those cases, you can use the `hyper_path` parameter to explicitly specify the location of the folder (!) containing the `hyperd` binary (not the path to the binary itself!). 50 | 51 | ```python 52 | from tableauhyperapi import HyperProcess, Telemetry 53 | 54 | HyperProcess(telemetry=Telemetry.DO_NOT_SEND_USAGE_DATA_TO_TABLEAU, 55 | hyper_path="/home/avogelsgesang/development/hyper/build/bin") as hyper: 56 | print(hyper.endpoint) 57 | ``` 58 | 59 | :::note For internal prototyping 60 | 61 | Using the `hyper_path`, you can also instruct Hyper API to interact with a different version of `hyperd`. 62 | Thereby, you can use Hyper API to quickly script a benchmark or more extensive test cases for your 63 | new feature or performance improvement. 64 | 65 | ::: 66 | 67 | ## Process Settings {#process-settings} 68 | 69 | The behavior of the Hyper process can be customized using a couple of settings. 70 | They influence all connections to Hyper. 71 | Those settings can be set during startup of the process through the `parameters` argument of the `HyperProcess`: 72 | 73 | ```python 74 | process_parameters = {"default_database_version": "2"} 75 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU, 76 | parameters=process_parameters) as hyper: 77 | print(hyper.endpoint) 78 | ``` 79 | 80 | ### Connectivity Settings {#connectivitysettings} 81 | 82 | These settings control how Hyper communicates with its clients. 83 | 84 | #### domain_socket_dir 85 | 86 | Specifies the directory that Hyper uses for domain sockets. It only has 87 | an effect if Hyper uses domain sockets (using domain sockets is the 88 | default behavior, see [use_tcp_port](#use_tcp_port)). 89 | 90 | Default value: `/tmp` 91 | 92 | :::note 93 | This setting has no effect on Windows machines. 94 | 95 | The maximum path length for valid domain sockets is limited on many 96 | platforms. It is therefore recommended to use a short path as the domain 97 | socket directory. 98 | ::: 99 | 100 | #### use_tcp_port 101 | 102 | If this setting is set to a port number or the special `auto` value, 103 | Hyper will use the TCP protocol to communicate with clients. If `auto` 104 | is passed, Hyper will automatically pick an available port. Otherwise, 105 | the passed port number is used. If this setting is set to `off`, which 106 | is the default value, Hyper will use named pipes on Windows and domain 107 | sockets on Linux and macOS. 108 | 109 | If TCP communication is desired, it is recommended to use the automatic 110 | port detection by passing `auto` instead of an explicit port. 111 | 112 | Default value: `off` 113 | 114 | Accepted values: `auto`, `off` or a port number between 1 and 65535 115 | 116 | ### Logging Settings {#loggingsettings} 117 | 118 | These settings control how Hyper writes its activity logs. 119 | Note that these setting controls the activity log of Hyper and not a transactional 120 | write-ahead log. 121 | 122 | #### log_config 123 | 124 | Can be used to disable Hyper's logging by setting it to the empty 125 | string. By default, logging is enabled. 126 | 127 | #### log_dir 128 | 129 | Specifies the directory into which Hyper's log files will be written. 130 | 131 | #### log_file_max_count 132 | 133 | Specifies how many Hyper log files are kept until the oldest ones are 134 | deleted. This setting only has an effect if multiple log files will be 135 | created (see: [log_file_size_limit](#log_file_size_limit)). For example, 136 | if `log_file_max_count` is set to `2` and 137 | [log_file_size_limit](#log_file_size_limit) is set to `100M`, there will 138 | be at most two log files with a file size of up to 100 MB containing the 139 | most recent log information. 140 | 141 | It is not recommended to set the limit to `1`, since this can lead to 142 | situations in which very little log information is available. This is 143 | because the old log file will be deleted immediately when a new log file 144 | is started. 145 | 146 | When set to `0`, the number of log files is not limited. 147 | 148 | Default value: `0` 149 | 150 | #### log_file_size_limit 151 | 152 | Specifies how large a Hyper log file is allowed to grow before logging 153 | switches to a new log file. When this setting is set to a value greater 154 | than zero, the log files will be suffixed with a timestamp indicating 155 | the time at which the logging into this file started. The setting\'s 156 | value can be specified in `K`(KB), `M`(MB), `G`(GB) or `T`(TB) units. 157 | For example, you can specify `100M` to limit the file size of each log 158 | file to 100 MB. A limit on how many log files should be kept around can 159 | be specified with [log_file_max_count](#log_file_max_count). 160 | 161 | When set to `0`, the log file size is not limited and no timestamps are 162 | added to the log file name. 163 | 164 | Default value: `0` 165 | 166 | ### Database Settings {#databasesettings} 167 | 168 | These settings control Hyper's database files. 169 | 170 | #### default_database_version 171 | 172 | Specifies the default database file format version that will be used to 173 | create new database files. 174 | Every version builds on the improvements of the previous version(s) and 175 | adds some new functionality, like new data types. 176 | 177 | Default value: `2` 178 | 179 | Accepted values: `0`, `1` (writing this version is deprecated in favor 180 | of version 2 and will be removed in a future Hyper API release), `2`, `3`, and `4`. 181 | 182 | :::note 183 | Newer database file format versions than the initial version `0` are 184 | unsupported in older product versions. This means that you can use newer 185 | database versions with the latest Hyper API and newer product versions 186 | but you cannot open them in older product versions. For example, the 187 | database file format version `2` can be opened in Tableau Desktop 188 | 2020.4.15 but it cannot be opened in Tableau Desktop 2020.3. The 189 | complete compatibility matrix is documented in the version sections 190 | below. 191 | 192 | Opening a database file with an unsupported 193 | Tableau product version will produce an error message similar to: 194 | 195 | "There was an error during loading database '[...]/file.hyper': 196 | unsupported version 3 (max supported version: 2). To open this database, 197 | please update your product. (error code 0AS01)" 198 | ::: 199 | 200 | 201 | ##### version 0 202 | 203 | The default and initial database file format version is version `0`. It 204 | is supported by all product versions. To create a new Hyper database 205 | file with this version, set `default_database_version=0`. 206 | 207 | ##### version 1 (deprecated) 208 | 209 | Database file format version `1` improves Hyper's file format 210 | significantly. It contains a collection of improvements from the years 211 | since Hyper's initial release: 212 | 213 | - Hyper will compress database files more efficiently after rows have 214 | been deleted. The initial file format was not able to compress data 215 | blocks with deleted rows, so the file size increased significantly 216 | when rows were deleted. 217 | 218 | - Hyper will process queries on textual data with collations more 219 | efficiently. 220 | 221 | - Hyper will detect database files that have been corrupted externally 222 | more reliably. 223 | 224 | To create a new Hyper database file with this version, set 225 | `default_database_version=1`. Note: Writing file version 1 is deprecated 226 | and will be removed in a future Hyper API release. 227 | 228 | :::note 229 | The database file format version `1` is supported by Tableau 230 | Desktop/Server 2019.2.10, 2019.3.6, 2019.4.5, 2020.1.1 and newer product 231 | versions. It is supported by Tableau Prep 2020.2 and newer versions. 232 | ::: 233 | 234 | ##### version 2 235 | 236 | Database file format version `2` adds support for storing and querying 237 | textual data with arbitrary versions of the Unicode collation tables. 238 | 239 | To create a new Hyper database file with this version, set 240 | `default_database_version=2`. 241 | 242 | :::note 243 | The database file format version `2` is supported by Tableau 244 | Desktop/Server 2020.4.15, 2021.1.12, 2021.2.9, 2021.3.8, 2021.4.4, 245 | 2022.1.2 and newer product versions. It is supported by Tableau Prep 246 | 2022.3 and newer versions. 247 | ::: 248 | 249 | ##### version 3 250 | 251 | Database file format version `3` supports storing and querying 128-bit 252 | numerics. 128-bit numerics support a precision of up to 38 places. 253 | Additionally, file format `3` improves compression of all 128-bit data types. 254 | 255 | To create a new Hyper database file with this version, set 256 | `default_database_version=3`. 257 | 258 | :::note 259 | The database file format version `3` is supported by Tableau Desktop 260 | 2022.4.1 and Server 2023.1 and newer product versions. It is supported by 261 | Tableau Prep 2022.4.1 and newer versions. 262 | ::: 263 | 264 | #### version 4 265 | Database file format version `4` was introduced to support 266 | persisting and reading the new 32-bit floating point type. 267 | 268 | Starting with release 0.0.19484, Hyper uses 32-bit floats for 269 | the SQL types `real`, `float4`, and `float(p)` with `p <= 24`. 270 | The types `double precision`, `float`, `float8`, and `float(p)` with `p >= 25` 271 | still use 64-bit doubles. 272 | 273 | To create a new Hyper database file with this version, set 274 | `default_database_version=4`. 275 | 276 | :::note 277 | The database file format version `4` will be supported 278 | by Tableau Desktop/Server/Prep 2024.3 and newer product versions. 279 | ::: 280 | 281 | 299 | -------------------------------------------------------------------------------- /website/static/img/hyper-social-card.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 22 | 24 | 51 | 53 | 54 | 56 | image/svg+xml 57 | 59 | 60 | 61 | 62 | 67 | 74 | 78 | 82 | 88 | 92 | 97 | 102 | 107 | 112 | 113 | 114 | 115 | 127 | 139 | 143 | 147 | 151 | 155 | 159 | 163 | 164 | 168 | 172 | 176 | 180 | 184 | 188 | 192 | 196 | 200 | 204 | 208 | 212 | 216 | 220 | 221 | 222 | 223 | -------------------------------------------------------------------------------- /website/docs/releases.md: -------------------------------------------------------------------------------- 1 | # Releases 2 | 3 | Below you can find the latest downloads and the new functionalities and bug fixes which shipped with each version. 4 | 5 | ## Download {#download} 6 | 7 | ```mdx-code-block 8 | import {config} from '@site/src/config'; 9 | ``` 10 | 11 |

The latest available version is v{config.version_short}.

12 | 13 | ```mdx-code-block 14 | import {DownloadPicker} from '@site/src/components/DownloadPicker' 15 | 16 | 17 | ``` 18 | 19 | ## Release Notes 20 | 21 | :::note 22 | 23 | In case you are wondering why all our releases start with `0.0`, read [this FAQ entry](/docs/faq#why-does-hyperapi-only-have-00-versions). 24 | 25 | ::: 26 | 27 | ### 0.0.24081 [December 11 2025] 28 | * Added support for two new SQL functions: 29 | * `split` splits a string into an array based on a delimiter. If the delimiter is null, it returns null. If the delimiter is empty, it splits the string into individual characters. 30 | * `regexp_split_to_array` splits a string into an array by using a POSIX regular expression as the delimiter and returns the text from the end of the last match (or the beginning of the string) to the beginning of the match. When there are no more matches, it returns the text from the end of the last match to the end of the string. 31 | 32 | ### 0.0.23576 [October 16 2025] 33 | * The default_database_version is upgraded to version 2 34 | * This version better supports UPDATE and DELETE on extracts 35 | * All supported Tableau versions can read these files 36 | * Updated OpenSSL version from 3.4.1 to 3.4.3 37 | 38 | ### 0.0.23135 [August 28 2025] 39 | * This release adds three new regular expression functions: 40 | * `regexp_substr` to extract substrings 41 | * `regexp_count` to count matches 42 | * `regexp_instr` to find the position of a match 43 | 44 | ### 0.0.22502 [June 19 2025] 45 | * IANA released version 2025b of the Time Zone Database. Hyper’s time zone information is updated accordingly. Noteworthy changes: 46 | * New zone America/Coyhaique 47 | * Iran switched from +04 to +0330 on 1978-11-10 at 24:00, not at year end. 48 | * Support for the FILTER clause has been added to aggregation functions in Hyper SQL. 49 | * For example, you can now write queries such as `SELECT count(*), count(*) FILTER (WHERE color = 'blue') FROM phone_cases;`. 50 | * The FILTER clause is not supported for ordered-set aggregates. The FILTER clause is only supported for WINDOW functions, which are aggregates. 51 | 52 | ### 0.0.22106 [May 1 2025] 53 | 54 | * The following deprecated geospatial functions have been removed from Hyper SQL: 55 | * `geo_make_point` (superseded by `tableau.geo_make_point`) 56 | * `geo_make_line` (superseded by `tableau.geo_make_line`) 57 | * `geo_distance` (superseded by `tableau.geo_distance`) 58 | * `geo_buffer` (superseded by `tableau.geo_buffer`) 59 | * `geo_auto_vertex_order` (superseded by `tableau.geo_auto_vertex_order`) 60 | * `geo_invert_vertex_order` (superseded by `tableau.geo_invert_vertex_order`) 61 | 62 | ### 0.0.21408 [Feb 13 2025] 63 | 64 | * Hyper’s SQL for Hyper API is now documented as part of the [Salesforce Data Cloud SQL Reference](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/data-cloud-sql-context.html) 65 | * Please be aware that Data Cloud specific functions are not available in Hyper API. Those functions are marked accordingly in the “Applies To” headers of the individual documentation pages. 66 | * The `geography` type has been renamed to `tableau.tabgeography` and the geospatial functions have been moved to the `tableau` namespace. 67 | * Existing Hyper files will continue to work; however, SQL queries and HAPI programs will need to be adjusted. 68 | * For example, use `tableau.geo_make_point` in SQL queries instead of just `geo_make_point`. 69 | * Use `SqlType.tabgeography()` in Python and Java, and `SqlType::tabgeography()` in C++. 70 | * The plain `geography` type and all geospatial functions outside the `tableau` namespace are deprecated and will be removed in the near future. 71 | * IANA released version 2024a of the Time Zone Database. Hyper’s time zone information is updated accordingly. Noteworthy changes: 72 | * Paraguay adopts permanent -03 starting spring 2024. 73 | * Improve historical data for Mexico, Mongolia, Philippines, and Portugal. 74 | * Update syntax for [`ARRAY` literals](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/array.html) and fixed bugs with quoting and escaping of text arrays. 75 | 76 | ### 0.0.21200 [Jan 17 2025] 77 | 78 | * Support for Microsoft Azure Blob Storage using [`azure_location`](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/external-location.html#microsoft-azure-blob-storage) was added. 79 | * Documented `starts_with` and `ends_with`, as well as negative field positions for `split_part`. See [String Functions](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/string-func.html) for details 80 | * Fixed double free bug in Java Hyper API (reported in GitHub Issue [#133](https://github.com/tableau/hyper-db/issues/133)). 81 | * Improved performance of distinct aggregates (e.g., `SELECT COUNT(DISTINCT a) from t`). 82 | * Dropped Centos7/RHEL7 support (in line with Tableau Server 2025.1 dropping Centos7/RHEL7 support). Starting with v0.0.21200 Hyper API doesn't support Centos7/RHEL7 anymore. 83 | 84 | ### 0.0.20746 [Nov 7 2024] 85 | 86 | * Support for `array_contains`, `array_position` and `array_positions` was added 87 | * Support for the `unnest` set-returning function was added 88 | * Various performance and stability improvements 89 | 90 | ### 0.0.20027 [Aug 19 2024] 91 | 92 | * Introduced new [`array` SQL datatypes](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/array.html). 93 | * The options available for [EXPLAIN](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/explain.html) changed: 94 | * `EXPLAIN VERBOSE` was removed in favor of `EXPLAIN (FORMAT JSON)` 95 | * There are two new output formats `EXPLAIN (FORMAT TERSE_JSON)` and `EXPLAIN (FORMAT SCHEMA)` 96 | * Update Unicode support from Unicode 14.0 to 15.1 97 | * Update Hyper's collation tables from CLDR 71 to CLDR 74. Besides corrections this also adds new collations to Hyper. 98 | 99 | ### 0.0.19691 [July 4, 2024] 100 | 101 | * Hyper API is now available under the Apache 2.0 license. 102 | 103 | ### 0.0.19484 [June 6, 2024] 104 | 105 | * Introduced a new 32-bit floating point data type for `REAL`. 106 | * Previously, `REAL` was internally mapped to 64-bit `DOUBLE PRECISION`. 107 | * Introduced new [database file format version 4](hyper-api/hyper_process#version-4) to support reading and persisting the new 32-bit floats. 108 | * A `CAST(… AS double precision)` is needed to store such columns in older file formats. 109 | * Documented the new and improved [database file format version 3](hyper-api/hyper_process#version-3) that was introduced in version 0.0.16123. The new format supports 128-bit numerics. Refer to [Hyper Database Settings](/docs/hyper-api/hyper_process#default_database_version) for more information. 110 | * Documented the [regexp_replace](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/string-matching.html#regular-expression-functions) function which provides substitution of new text for substrings based on POSIX regular expressions. 111 | * Added native support for Apple Silicon processors (aarch64) for macOS 13.0 or newer 112 | 113 | :::warning 114 | Queries using `REAL`, `FLOAT4`, or `FLOAT(p)` with `p <= 24` are now treated as 32-bit floating points. 115 | This can lead to different results due to the reduced precision of 32-bit floating points. 116 | To preserve the old behavior, you need to use the types `DOUBLE PRECISION`, `FLOAT8`, or `FLOAT(p)` with `p >= 25`. These continue to be treated as 64-bit floating points. 117 | ::: 118 | 119 | 120 | 121 | ### 0.0.18825 [March 6, 2024] 122 | 123 | * Updated OpenSSL version from 3.1.4 to 3.2.1 124 | * IANA released version 2024a of the Time Zone Database. Hyper’s time zone information is updated accordingly. Noteworthy changes: 125 | * Kazakhstan unifies on UTC+5 beginning 2024-03-01. 126 | * Palestine springs forward a week later after Ramadan. 127 | 128 | 129 | ### 0.0.18618 [February 7, 2024] 130 | 131 | * Overhauled the [SQL type propagation rules for the `NUMERIC` data type](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/numeric.html). This can lead to differences in the precision and number of decimal digits of `NUMERIC` calculations in existing queries and their results. The changes are: 132 | * Multiplication and division of `NUMERIC` with integer and numeric types now follows different rules to determine the result's precision and scale. 133 | * The `AVG` aggregate function now adds 6 instead of 4 decimal digits. 134 | * The statistical aggregate functions (like `STDDEV_SAMP`) now always return `NUMERIC(38,6)` for integer and numeric inputs. If a scale of 6 is too low for your purposes consider casting the input to `double precision`. 135 | * Integer literals are now treated as if they were of type numeric with the minimal required precision when in arithmetic operations with `NUMERIC` values. 136 | * IANA released version 2023d of the Time Zone Database. Hyper’s time zone information is updated accordingly. Noteworthy changes: 137 | * Ittoqqortoormiit, Greenland changes time zones on 2024-03-31. 138 | * Vostok, Antarctica changed time zones on 2023-12-18. 139 | * Casey, Antarctica changed time zones five times since 2020. 140 | * Code and data fixes for Palestine timestamps starting in 2072. 141 | 142 | ### 0.0.18441 [January 10, 2024] 143 | 144 | * Various performance and stability improvements 145 | 146 | ### 0.0.18369 [December 8, 2023] 147 | 148 | * Added support for `localized_week` 149 | * The functions `EXTRACT`, `date_part` and `date_trunc` support a new field `localized_week`. 150 | * Localized week options for the new field are added. See [Localized Week Options](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/datetime-func.html#localized-week-options). 151 | 152 | ### 0.0.18161 [November 8, 2023] 153 | 154 | * Reduced memory consumption for `WITH RECURSIVE`: Up to 20,000x less memory usage in microbenchmarks. 155 | * Improved performance for `WITH RECURSIVE`: Roughly 2x improvement in microbenchmarks. 156 | * Improved heuristic rewrites for joins: 157 | * Remove semi/anti-joins with a constant FALSE conditions. 158 | * Apply the exact same "constant FALSE condition" rewrites also in cases where the join has an empty input on one of both sides. 159 | * Updated OpenSSL version from 1.1.1u to 3.0.10. 160 | 161 | ### 0.0.17971 [October 9, 2023] 162 | 163 | * .Net Deprecation: As announced last release, we are deprecating the .NET version of Hyper API. We decided to [open-source its source code](https://github.com/tableau/hyper-api-dotnet), so whoever might be interested in maintaining it can pick up where we left. 164 | * Some of the [Fiscal Calendar Options](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/datetime-func.html#fiscal-calendar-options) are renamed: 165 | * `start_month` is renamed to `fiscal_year_start_month`. 166 | * `first_day_of_week` is renamed to `first_day_of_fiscal_week`. 167 | * The [EXTRACT](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/datetime-func.html#extract) function was accepting (and ignoring) named arguments that were not required. Now it only accepts [Fiscal Calendar Options](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/datetime-func.html#fiscal-calendar-options) when fiscal function fields are used. A named argument that is not required by the `EXTRACT` function will be rejected. 168 | 169 | ### 0.0.17782 [September 6, 2023] 170 | 171 | * Sharing a Hyper process between multiple programs is now supported. This allows for better resource management and performance if multiple programs that interact with Hyper run at the same time. See [HyperProcess: Performance Best Practices](/docs/hyper-api/hyper_process#performance). 172 | * The HyperAPI Python zip package was removed from our download page. The package is available on the Python Package Index (PyPI) and we recommend installation through `pip install tableauhyperapi` instead. 173 | * .Net Deprecation: The .NET version of Hyper API is deprecated. This release will be the last update for .NET. In the future, .NET will not receive any new features or updates. The old packages will stay available on NuGET indefinitely, so you can keep using older versions in .NET. The other languages (Python, Java, C++) are not impacted by this in any way. In case this is causing issues for you, please reach out via [Slack](https://join.slack.com/t/tableau-datadev/shared_invite/zt-1q4rrimsh-lHHKzrhid1MR4aMOkrnAFQ). 174 | 175 | 176 | ### 0.0.17537 [August 2, 2023] 177 | 178 | * Support for fiscal calendar was added 179 | * The functions `EXTRACT`, `date_part` and `date_trunc` have the following fiscal calendar fields: `fiscal_week`, `fiscal_month`, `fiscal_quarter`, and `fiscal_year`. 180 | * Fiscal calendar options for the newly-added fields were added. See [Fiscal Calendar Options](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/datetime-func.html#fiscal-calendar-options). 181 | * Updated OpenSSL version from 1.1.1t to 1.1.1u. 182 | * Support for reading and writing [Arrow](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/external-formats.html#apache-arrow-format) is now stable and ready for use in production 183 | 184 | ### 0.0.17360 [July 5, 2023] 185 | 186 | * Support for `AT TIME ZONE` was added. See [documentation of Date/Time Functions](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/datetime-func.html) 187 | * Experimental support for reading and writing [Arrow](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/external-formats.html#apache-arrow-format) files (`.arrow`) and streams (`.arrows`) 188 | * Support for the `COPY TO` statement was added. See [documentation of COPY TO](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/copy-to.html) 189 | 190 | ### 0.0.17231 [June 7, 2023] 191 | 192 | * Support for multiple concurrent connections to the same Hyper file was added 193 | * Support for `FETCH [...] WITH TIES` was added. See [documentation of FETCH](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/select-limit-offset.html#fetch-and-offset) 194 | * Improved query planning for external formats (e.g., Parquet, CSV, ...) 195 | * Samples are used for selectivity estimation on external formats 196 | * Distinct counts, statistics, and samples are no longer computed eagerly when issuing a `CREATE TEMP EXTERNAL TABLE`. Instead, the first query that uses the external table updates them. 197 | 198 | ### 0.0.17002 [May 3, 2023] 199 | 200 | * Improved [documentation](https://tableau.github.io/hyper-db/docs/) 201 | * IANA released version 2023c of the Time Zone Database. Hyper’s time zone information is updated accordingly. Noteworthy changes: 202 | * Egypt now uses DST again, from April through October. 203 | * This year Morocco springs forward April 23, not April 30. 204 | * Palestine delays the start of DST this year. 205 | * Much of Greenland still uses DST from 2024 on. 206 | * America/Yellowknife now links to America/Edmonton. 207 | * When observing Moscow time, Europe/Kirov and Europe/Volgograd now 208 | use the abbreviations MSK/MSD instead of numeric abbreviations, 209 | for consistency with other timezones observing Moscow time. 210 | 211 | ### 0.0.16868 [April 5, 2023] 212 | 213 | * Introduced `approx_count_distinct` aggregate 214 | * It can be used to compute an approximation to exact count distinct with configurable relative error. 215 | * E.g., the query `select approx_count_distinct(x) from generate_series(1,pow(10,6)) s(x)` returns `960712`. 216 | * A relative error argument is supported as well, e.g., `select approx_count_distinct(x, 0.002) from generate_series(1,pow(10,6)) s(x)` returns `998192`, a much better estimate with relative error under `0.2%` (the default value if omitted is `2.3%` relative error accuracy). 217 | * In general, `approx_count_distinct(c, e)` uses less memory than `count(distinct c)` and is faster. This makes it a good option when exact distinct count is not required. 218 | 219 | ### 0.0.16638 [March 1, 2023] 220 | 221 | * Updated OpenSSL version from 1.1.1q to 1.1.1t. 222 | * IANA released version 2022g of the Time Zone Database. Hyper’s time zone information is updated accordingly. Noteworthy changes: 223 | * Jordan and Syria switched from +02/+03 with DST to year-round +03 224 | * Mexico no longer observes DST except near the US border 225 | * Chihuahua moved to year-round -06 on 2022-10-30 226 | * Fiji no longer observes DST 227 | * Simplified four Ontario zones 228 | * The northern edge of Chihuahua changed to US timekeeping 229 | * Much of Greenland stops changing clocks after March 2023 230 | * Fixed some pre-1996 timestamps in northern Canada 231 | 232 | ### 0.0.16491 [February 8, 2023] 233 | 234 | * Added support for the [`GROUPING SETS` SQL feature](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/select-group-by.html), including `ROLLUP` and `CUBE`. 235 | 236 | 237 | ### 0.0.16377 [January 18, 2023] 238 | 239 | * Minor improvements and bug fixes. 240 | 241 | ### 0.0.16123 [December 7, 2022] 242 | 243 | * Added support for 128-bit numerics. This allows a precision of up to 38 for the `NUMERIC` SQL type. 244 | * Added support to read 128-bit `DECIMAL` values from parquet files. 245 | * Overhauled the [SQL type propagation rules for the `NUMERIC` data type](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/numeric.html). 246 | * Improved partition pruning support when querying Apache Iceberg. This should speed up queries with 247 | equality predicates on Iceberg columns partitioned with bucket partitioning. 248 | * New `ANY_VALUE` aggregate function: The `ANY_VALUE` aggregate function returns an arbitrary, implementation-defined value from the set of input values within a group. 249 | 250 | ### 0.0.15888 [November 9, 2022] 251 | * IANA released version `2022d` of the Time Zone Database. Hyper's time zone information is updated accordingly. Noteworthy changes: 252 | * Palestine daylight savings time (DST) transitions are now Saturdays at 02:00 (24-hour clock). 253 | * Simplified three Ukrainian zones into one. 254 | 255 | ### 0.0.15735 [October 5, 2022] 256 | 257 | * New support for Apache Iceberg as an [external format](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/external-files.html). 258 | * Support for reading external files from S3 is now enabled by default. (The experimental_external_s3 setting has been removed. Specifying it now causes an unknown setting error.) 259 | * "New convenience functions to extract date and time units: 260 | * New [YEAR](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/datetime-func.html#functions) function: extracts the year of a timestamp or interval. 261 | * New [QUARTER](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/datetime-func.html#functions) function: extracts the quarter of a timestamp. 262 | * New [MONTH](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/datetime-func.html#functions) function: extracts the month of a timestamp or interval. 263 | * New [WEEK](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/datetime-func.html#functions) function: extracts the week of a timestamp. 264 | * New [DAY](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/datetime-func.html#functions) function: extracts the day of a timestamp or interval. 265 | * New [HOUR](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/datetime-func.html#functions) function: extracts the hour of a timestamp or interval. 266 | * New [MINUTE](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/datetime-func.html#functions) function: extracts the minute of a timestamp or interval. 267 | * New [SECOND](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/datetime-func.html#functions) function: extracts the second of a timestamp or interval. 268 | * IANA released version `2022c` of the Time Zone Database. Hyper's time zone information is updated accordingly. Noteworthy changes: 269 | * Chile's DST is delayed by a week in September 2022. 270 | * Iran no longer observes DST after 2022. 271 | * Renamed Europe/Kiev to Europe/Kyiv. 272 | * Fixes to support timestamps prior to 1970. 273 | 274 | ### 0.0.15530 [September 7, 2022] 275 | 276 | * To read external data from Amazon S3, you must now provide credentials when using the extended syntax `s3_location(...)`. Omitting credentials now causes an error. For anonymous access to S3 provide empty credentials (`""`) instead. 277 | 278 | ### 0.0.15305 [August 3, 2022] 279 | 280 | * Minor improvements and bug fixes. 281 | 282 | ### 0.0.15145 [July 13, 2022] 283 | 284 | * Added ZSTD and LZ4_RAW compression support for Parquet files. 285 | * Updated OpenSSL version from 1.1.1n to 1.1.1q. 286 | * IANA released version `2022a` of the Time Zone Database. Hyper's time zone information is updated accordingly. Noteworthy changes: 287 | * Palestine will spring forward on `2022-03-27`, not `2022-03-26`. 288 | * From 1992 through spring 1996, Ukraine's DST transitions were at `02:00 standard time`, not at `01:00 UTC`. 289 | * Chile's Santiago Mean Time and its LMT precursor have been adjusted eastward by 1 second to align with past and present law. 290 | * Updated Hyper's collation tables from CLDR 38.1 to CLDR 41. Besides corrections this also adds new collations to Hyper. 291 | * Updated Unicode support from Unicode 13.0.0 to 14.0.0. 292 | 293 | ### 0.0.14946 [June 1, 2022] 294 | 295 | * Updated OpenSSL version from 1.1.1l to 1.1.1n. 296 | * New [TRY_CAST](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/type-conversion.html) function: converts a value to a target type, returns NULL on failure. 297 | 298 | ### 0.0.14751 [May 4, 2022] 299 | 300 | * Restriction pushdown for Parquet files: Hyper now exploits min/max values in Parquet RowGroups to skip groups based on the predicates present in SQL queries. 301 | * Besides many corrections, this update also adds a significant number of new collations to Hyper. Previously, the collations Hyper used were based on CLDR 1.8 from March 2010. 302 | * C++: Fixed inserting and querying columns of type CHAR(1). 303 | * Fixed empty strings of type CHAR(1) being returned as a space instead of '\0'. 304 | * Fixed a defect which could lead to Hyper crashing when using outer joins with Parquet files. 305 | * Fixed a defect which prevented Hyper from opening external files from Amazon S3 if the S3 URL contained a whitespace character, e.g. "s3://bucket/filename with whitespace.csv" 306 | 307 | ### 0.0.14567 [March 23, 2022] 308 | 309 | * Result fetching of large results is up to 5x faster. 310 | * Fix a potential crash when reading multiple Parquet files with string columns. 311 | 312 | ### 0.0.14401 [March 2, 2022] 313 | 314 | * Introduced the new and improved database file format version 2 that can be used via [Hyper Process Settings](/docs/hyper-api/hyper_process). The new format stores data independent of collation versions. File format 1 is deprecated in favor of the new file format 2. Refer to [Hyper Database Settings](/docs/hyper-api/hyper_process#default_database_version) for more information. 315 | * Added support for S3 keys containing special characters (such as "=") 316 | * Implemented support for [external(source_location(...))](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/setreturning.html#external) syntax. 317 | 318 | ### 0.0.14265 [February 2, 2022] 319 | 320 | Improved external file format support (CSV & Apache Parquet): Now you can use Hyper as a SQL 321 | query engine directly on top of open formats and data lakes. 322 | * Hyper now has experimental support for reading external data directly from Amazon S3. ~~You need to enable the experimental_external_s3 323 | setting to use this feature and be aware that it can **change or be removed at any time without prior notice**.~~ The feature is considered stable as of 0.0.15735. The experimental flag is not necessary anymore. 324 | 325 | Hyper's S3 capabilities are highly optimized 326 | (using techniques such as concurrent requests, request hedging and prefetching). For maximum performance, 327 | ensure a high network bandwidth to Amazon S3, e.g., by running HyperAPI directly on an AWS EC2 instance. 328 | * Temporary external tables: The new 329 | [CREATE TEMPORARY EXTERNAL TABLE](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/create-external-table.html) 330 | command exposes external data to SQL as if it was a Hyper table, but the data is read directly 331 | from the external file whenever the external table is referenced in a query. 332 | * The new function [external](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/setreturning.html#external), 333 | enables reading external data directly in a SQL query without creating an external table. 334 | * Aligned the syntax of the [COPY](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/copy-from.html) 335 | statement with the syntax for external tables and the `external` function. 336 | The old syntax is still supported for PostgreSQL compatibility but its use is discouraged. 337 | * The new `ARRAY[...]` syntax enables reading from multiple files when using the `external` function, external tables, 338 | or the `COPY` command. 339 | * Graceful handling of invalid UTF-8 sequences: 340 | The new [SANITIZE](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/external-formats.html#format-options) 341 | option instructs Hyper to replace invalid UTF-8 sequence with the replacement character (�) instead of failing the query with an error. 342 | * Improved support for reading CSV files: 343 | * GZip-compressed CSV files: CSV files ending in `.gz` will automatically be assumed to be GZip-compressed. 344 | * UTF-16 encoded CSV files: UTF-16 reading can be enabled using the new 345 | [ENCODING](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/external-formats.html#format-options) 346 | option. 347 | * Graceful cast failure handling: When a value in the file cannot be cast 348 | to the target type, the new 349 | [ON_CAST_FAILURE](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/external-formats.html#format-options) 350 | option instructs Hyper to read the value as NULL instead of raising an error. 351 | 352 | ### 0.0.14109 [January 5, 2022] 353 | 354 | * Minor improvements and bug fixes. 355 | 356 | ### 0.0.13980 [December 8, 2021] 357 | * Smaller packages: Thanks to the removal of unused collation data from the Hyper binary, the package size was reduced. For example, the size of an unpacked Windows Python installation went from 157 MB to 145 MB. The download size of a packed Windows Python package was reduced from 47 MB to 42 MB. 358 | * Java: Improved the read performance of text and geography columns. 359 | * Fixed a defect that could lead to crashes when reading Parquet files with text columns that contain null values. 360 | * Fixed a defect that could lead to Hyper sorting and comparing text with the "ro" locale incorrectly. 361 | * IANA released new versions of the Time Zone Database. This commit updates Hyper's Time Zone Database to release 2021e. 362 | 363 | Noteworthy changes in the Time Zone Database: 364 | 365 | * Palestine will fall back 10-29 (not 10-30) at 01:00. 366 | * Fiji suspends DST for the 2021/2022 season. 367 | * Jordan now starts DST on February's last Thursday. 368 | * Samoa no longer observes DST. 369 | * Merge more location-based Zones whose timestamps agree since 1970. 370 | * Rename Pacific/Enderbury to Pacific/Kanton. 371 | 372 | ### 0.0.13821 [November 3, 2021] 373 | * Reading Apache Parquet files is now officially supported and no longer has to be enabled through the use of the process setting ‘experimental_external_format_parquet’. 374 | * Improved performance for window functions for large data sets on multi-core machines: 375 | * calls without `PARTITION BY` clause improved by 5% - 20% 376 | * calls without `PARTITION BY` and `ORDER BY` clauses, e.g. `ROW_NUMBER() OVER()`, by 10% - 25% 377 | * Fixed a defect that could cause crashes when you deleted a tuple multiple times, or could cause wrong query results after single deletion. 378 | * Updated OpenSSL version from 1.1.1k to 1.1.1l 379 | 380 | ### 0.0.13617 [October 6, 2021] 381 | 382 | * Fix a query compilation defect that led to reproducible crashes for a very small number of queries. 383 | * Upgraded Unicode support from Unicode 9.0.0 to 13.0.0 384 | 385 | ### 0.0.13394 [September 1, 2021] 386 | 387 | * Hyper API now runs on AWS lambda. 388 | * Parquet files with dots in their column names can now be read. 389 | * Column references can now be qualified with database alias (e.g., `SELECT db.schema.table.column ...`) 390 | * More actionable error messages for a wide range of invalid SQL queries. 391 | * Going forward, the Hyper API will only support the three most recent versions of Python. Currently, those versions are 3.7, 3.8, and 3.9. However, the Hyper API will continue to support Python 3.6 for a transition period of three months. 392 | 393 | ### 0.0.13287 [August 4, 2021] 394 | 395 | * Java: Dependencies are updated to newer versions. In particular, JNA was updated to 5.6.0. 396 | 397 | ### 0.0.13129 [July 7, 2021] 398 | 399 | * Minor improvements and bug fixes. 400 | 401 | ### 0.0.12982 [June 9, 2021] 402 | 403 | * Fixed a problem that could corrupt databases in rare cases when the [new file format](/docs/hyper-api/hyper_process#default_database_version) has explicitly been enabled by the user. This problem did not affect the default file format. 404 | * Updated OpenSSL dependency from 1.1.1g to 1.1.1k. 405 | 406 | ### 0.0.12805 [May 19, 2021] 407 | 408 | * Hyper now has experimental support for reading Apache Parquet files. See [Hyper API SQL documentation](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/copy-from.html#sql-copy-examples) for details. 409 | * Hyper now adjusts the resulting interval from a timestamp subtraction so that 24-hour time periods are represented as days. 410 | * Hyper now supports +/-13 and +/-14 as timezone offsets. 411 | * Python: The most commonly used Hyper API types now have `__repr__()` methods and will return a string representation of the object when printed, making interactive exploring of the Hyper API more fun. 412 | * Improved handling of geospatial types: 413 | * Parsing GEOGRAPHY values from Well-known text (WKT) format automatically adjusts the order of vertices in polygons. 414 | * During WKT parsing, additional vertices may be added to more closely resemble the original shape specified in the WKT. 415 | 416 | ### 0.0.12514 [April 7, 2021] 417 | 418 | * Fixed a rare defect where queries could return incorrect results after tuples at the end of a table were deleted. 419 | 420 | ### 0.0.12366 [March 10, 2021] 421 | 422 | * Improved performance for complex queries thanks to improved join ordering. 423 | * Fixed a defect where Hyper would use too much memory when executing string expressions in certain contexts. 424 | 425 | ### 0.0.12249 [February 17, 2021] 426 | 427 | * IANA released version `2021a` of the Time Zone Database. Hyper's time zone information is updated accordingly. Noteworthy changes: 428 | * Revised predictions for Morocco's changes starting in 2023. 429 | * Canada's Yukon changes to `-07` on `2020-11-01`, not `2020-03-08`. 430 | * Macquarie Island has stayed in sync with Tasmania since 2011. 431 | * Casey, Antarctica is at `+08` in winter and `+11` in summer. 432 | * Fiji starts DST later than usual, on `2020-12-20`. 433 | * Palestine ends DST earlier than predicted, on `2020-10-24`. 434 | * Volgograd switches to Moscow time on `2020-12-27` at `02:00`. 435 | * South Sudan changes from `+03` to `+02` on `2021-02-01` at `00:00`. 436 | * Added additional information to certain Hyper API exceptions that previously contained only their context id. 437 | 438 | ### 0.0.12005 [January 20, 2021] 439 | 440 | * Introduced a new and improved database file format that can be used via [Hyper Process Settings](/docs/hyper-api/hyper_process). Refer to [Hyper Database Settings](/docs/hyper-api/hyper_process#default_database_version) for more information. 441 | * Clarified the `Create hyper file from csv` example: We highlight the usage of the `HEADER` COPY option which ignores the first line in a csv file. 442 | * Java: Fixed the `getShort()` method to return a `short` instead of an `int`. 443 | 444 | ### 0.0.11952 [December 16, 2020] 445 | 446 | * When Hyper is running inside a container, such as Docker, Hyper now respects the memory limits that are set for the container. 447 | 448 | ### 0.0.11889 [December 2, 2020] 449 | 450 | * Fixed a parsing error that could lead to a failure to connect to a Hyper database. This error could occur with certain operating system configurations if you were using special UTF-8 characters as the database name. 451 | 452 | ### 0.0.11691 [November 9, 2020] 453 | 454 | * Faster initialization of `HyperProcess`: Starting Hyper is now 4x faster. For example, on our internal Linux computers, we measured 11 milliseconds startup time instead of previously 44 milliseconds. 455 | * Python: `TableDefinition.Column.collation` represents the default collation with `None` now. Previously, the results of `catalog.get_table_definition` used `''` for the default collation. This is a breaking change. 456 | 457 | ### 0.0.11556 [September 30, 2020] 458 | 459 | * C++: Fixed a bug that could have lead to wrong or missing results when multiple `ResultIterator` or `ChunkIterator` iterators are constructed over the same `hyperapi::Result` object. 460 | * C++: Interface fix: Removed an incorrect `noexcept` specification from the `ResultIterator()` and `ChunkedIterator()` constructors for begin iterators. These functions may fail by throwing `std::bad_alloc` or `hyperapi::HyperException`. Those were previously flagged as `noexcept` even though they could have thrown. 461 | * Removed support for the PostgreSQL legacy end-of-data marker `\.`. The marker could be used to mark the end of CSV and TEXT input. Hyper now solely relies on the end-of-file condition to determine the end. 462 | 463 | ### 0.0.11355 [August 26, 2020] 464 | 465 | * Removed the following settings for the `HyperProcess` class that were deprecated since version 0.0.10309: 466 | * `log-dir`: Use `log_dir` instead. 467 | * `:restrict_database_directory`: Not required since Hyper no longer creates database files in the working directory. 468 | * `:database_directory`: Not required since Hyper no longer creates database files in the working directory. 469 | * `:log_file_size_limit`: Use `log_file_size_limit` instead. 470 | * `:log_file_max_count`: Use `log_file_max_count` instead. 471 | 472 | ### 0.0.11249 [July 30, 2020] 473 | 474 | * Hyper now correctly checks for NOT NULL constraints when creating a table from a CSV file with the COPY statement. 475 | 476 | ### 0.0.11074 [June 24, 2020] 477 | 478 | * Adds several SQL functions for managing geospatial data: 479 | * For creating geography objects (`geo_make_point` and `geo_make_line`). 480 | * For performing calculations on geography objects (`geo_distance` and `geo_buffer`). 481 | * For manipulating the vertex order of polygons in geography objects (`geo_auto_vertex_order` and `geo_invert_vertex_order`). These functions can be used to address problems (for example, with geospatial joins or to automatically zoom) where data comes from a source that uses a different winding order for polygons than the one used by Tableau. In Tableau, the interior of the polygon is considered to be on the left of the path drawn by points of the polygon ring. 482 | * Prepared queries gained support for parallelized execution. See [PREPARE](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/prepare.html) and [EXECUTE](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/execute.html) for more information on prepared queries in Hyper. 483 | * Java: Fixed crashes that could occur when inserting more than 16 MB of data into a table. 484 | * Python: Fixed crashes of Python interpreter on shutdown by fixing reference counting. 485 | * .NET: Fixed broken Nuget packages. 486 | * Fixed a hanging query result fetch operation in the Hyper API when rows are consistently larger than 1 MB. 487 | * New Python sample file that shows how you can use the Hyper API to reduce the fragmentation of `.hyper` files. See [Optimize Hyper File Storage](/docs/guides/hyper_file/optimize) and the [defragment-data-of-existing-hyper-file](https://github.com/tableau/hyper-api-samples/tree/main/Community-Supported/defragment-hyper-file) sample on GitHub. 488 | 489 | ### 0.0.10899 [May 27, 2020] 490 | 491 | * Hyper now fully supports the options `FORCE_NULL` and `FORCE_NOT_NULL` for CSV parsing. By default, only unquoted values are compared to the null string to determine whether they represent a `NULL` value. `FORCE_NULL` toggles the same for quoted values. `FORCE_NOT_NULL` disables comparison of non-quoted values with the null string. See [COPY command](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/copy-from.html). 492 | 493 | * Updated the target framework of the Hyper API for .NET example from .NET Core 2.2 to .NET Core 3.1. .NET Core 2.2 has already reached its end of life at 2019-12-23 and increasingly surfaced stability problems. We continue to target the .NET Standard 2.0 in the Hyper API for .NET. 494 | 495 | * IANA released version `2020a` of the Time Zone Database. Hyper's time zone information is updated accordingly. Noteworthy changes: 496 | * Morocco springs forward on `2020-05-31`, not `2020-05-24`. 497 | * Canada's Yukon advanced to `-07` year-round on `2020-03-08`. 498 | * `America/Nuuk` was renamed from `America/Godthab`. 499 | 500 | ### 0.0.10622 [April 22, 2020] 501 | 502 | * If you use the Hyper API and accidentally open a file that is not a Hyper file, you now see a more informative error message. 503 | 504 | * C++: Fixed a memory leak in the constructor of `hyperapi::HyperProcess` when an invalid parameter was supplied. 505 | 506 | * The Python Hyper API now exposes a `__version__` attribute and thus supports PEP 396. 507 | 508 | ### 0.0.10309 [March 25, 2020] 509 | 510 | * The Hyper API `Inserter` class now allows SQL expressions to compute or transform data on the fly during insertion. 511 | 512 | * The Hyper API `Inserter` class now allows inserting Well-known text (WKT) into `Geography` columns. You can use the `CAST` expression to transform WKT data to the `Geography` type and provide WKT data as a string to the `Inserter` class. For more information, see [Add Geospatial Data to a Hyper File](/docs/guides/hyper_file/geodata). 513 | 514 | * Documented the available settings that can be passed to the `HyperProcess` and `Connection` constructors. See [Settings](/docs/hyper-api/hyper_process#passingprocesssettings). 515 | 516 | * Exposed settings for the `HyperProcess` class that give control over the way Hyper communicates with its clients. See [Connectivity Settings](/docs/hyper-api/hyper_process#connectivitysettings). 517 | 518 | * Exposed settings for the `HyperProcess` class that give control over its logging behavior. See [Logging Settings](/docs/hyper-api/hyper_process#loggingsettings). 519 | 520 | * Exposed settings for the `Connection` class that give control over date and time parsing. See [Date and Time Settings](/docs/hyper-api/connection#datetimesettings). 521 | 522 | * The Hyper API no longer creates database files in the working directory. Instead, they are placed in a temporary directory. This makes it easier to use the Hyper API in write-protected working directories. 523 | 524 | * Deprecated the following settings for the `HyperProcess` class: 525 | * `log-dir`: Now called `log_dir`. 526 | * `:restrict_database_directory`: Not required since Hyper no longer creates database files in the working directory. 527 | * `:database_directory`: Not required since Hyper no longer creates database files in the working directory. 528 | * `:log_file_size_limit`: Now called `log_file_size_limit`. 529 | * `:log_file_max_count`: Now called `log_file_max_count`. 530 | 531 | The deprecated settings will continue to work for at least three releases. Afterwards, the deprecated settings will be removed. 532 | 533 | * The C++ HAPI now expects all settings (i.e., the keys and values of the `parameters` map passed to the constructor of `HyperProcess` and `Connection`) to be passed in UTF-8 encoding. 534 | 535 | * Improved loading time for Python: `import tableauhyperapi` now takes 100 milliseconds instead of 250 milliseconds. 536 | 537 | * Added the `to_date` function. See [Data Type Formatting Functions](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/formatting.html). 538 | 539 | --- 540 | 541 | ### 0.0.10002 [February 26, 2020] 542 | 543 | * Reduced memory consumption for `INSERT`: When inserting a large number of tuples using INSERT, Hyper API now uses less RAM. This is particularly important when copying large tables using `INSERT INTO newtable SELECT * FROM oldtable`. 544 | 545 | * Simplified installation requirements on Windows: The Hyper API no longer requires that you install the Microsoft Visual C++ Runtime Library separately. 546 | 547 | * Smaller packages: Thanks to improvements to our build processes and packaging, the package size was reduced. For example, the size of an unpacked Python installation went from 186 MB to 174 MB. The download size of a packed Python package was reduced from 49 MB to 46 MB. 548 | 549 | * Bug fix for `VALUES` clauses: In rare cases, Hyper evaluated a join against a `LATERAL VALUES` clause incorrectly, leading to crashes or incorrect results. With this release, Hyper now evaluates such `VALUES` clauses correctly. 550 | 551 | * Deprecations around the `HyperExpection` class: The following changes were done to simplify the interface of the `HyperException` class across languages. All of the deprecated functions can be replaced by their newly introduced alternatives. In general, these changes should only impact power users. For most use cases, we recommend using `str()` (Python), `getMessage()` (Java), `ToString()` (C#) or `what()` (C++). 552 | * Python: `message` was deprecated in favor of `main_message` and `hint_message` was deprecated in favor of `hint`. Furthermore, `context_id` is now an instance of the `ContextId` class and no longer a plain integer. 553 | * Java: `getErrorMessage` was deprecated in favor of `getMainMessage`. 554 | * C#/.Net: The `PrimaryMessage` property was deprecated in favor of `MainMessage`. 555 | * C++: `getHintMessage` was deprecated in favor of `getHint`. `getMessage` was deprecated in favor of `getMainMessage`. Furthermore, `HyperException::getCause` now returns a `optional` instead of a `HyperException`. The method `hasCause` was deprecated. 556 | 557 | The old method names will stay unchanged and continue working for at least the next three releases of Hyper API. They will be removed at some point in future after that. 558 | 559 | --- 560 | 561 | ### 0.0.9746 [January 29, 2020] 562 | 563 | * Improved time zone support. In particular, the `TIMESTAMP WITH TIME ZONE` (or `TIMESTAMPTZ`) type is now properly supported. 564 | 565 | * This release includes documentation for several SQL features, including: 566 | * Manipulation and formatting of date/time values and intervals, also with full time zone support. See [Data Type Formatting Functions](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/formatting.html) and [Date/Time Functions and Operators](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/datetime-func.html). 567 | * Sub-query expressions (for example, `EXISTS`, `IN`, `ALL`). See [Subquery Expressions](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/subquery-comparison.html). 568 | * Window aggregate functions (for example, `RANK()`). See [Window Functions and Queries](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/window.html). 569 | * `generate_series` - See [Set Returning Functions](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/setreturning.html). 570 | * Data Types: boolean, binary, numeric types, character, date/time. See [Data Types](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/datatypes.html). 571 | 572 | * The Tableau Hyper API no longer requires write access in the working directory. 573 | 574 | * Improved error handling and messages. 575 | 576 | * The Tableau Hyper API is available on the Python Package Index (PyPI). You can now install the Tableau Hyper API using the package installer, `pip`. 577 | 578 | ``` 579 | pip install tableauhyperapi 580 | ``` 581 | 582 | Or, if you previously installed the package. 583 | 584 | ``` 585 | pip install --upgrade tableauhyperapi 586 | ``` 587 | 588 | Linux installations require `pip` version 19.3 or newer. Note that `pip` versions 20.0 and 20.1 are not working because of issues with `pip` and not the Tableau Hyper API package. 589 | 590 | * Support for macOS 10.15 (Catalina). You can now install the Hyper API on computers running macOS 10.13 and later. 591 | 592 | * The `HyperProcess` (hyperd.exe) on Windows no longer opens a terminal window (Issue 1039998). 593 | 594 | * Hyper is now reusing space freed by DELETE (Issue 1056751). In a rolling-window scenario (where old data is deleted in bulk before appending new data), previous versions of the Tableau Hyper API would not re-use the deleted space, causing the `.hyper` file to grow. This problem is fixed with this release. In addition to the simple rolling window scenario, the fix also applies to other bulk deletion patterns. 595 | 596 | * UPDATE now supports multi-column subqueries in SET clauses. See [UPDATE](https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/dc-sql-reference/update.html). 597 | 598 | * Standard-compliant natural join. 599 | 600 | --- 601 | 602 | ### 0.0.9273 [December 4, 2019] 603 | 604 | * NuGet package for the Tableau Hyper API for .NET. You can now reference the Tableau Hyper API library from your project file as you would for other NuGet packages. See [Install the Hyper API for .NET](http://localhost:3000/docs/installation?client-language=dotnet#instructions). 605 | 606 | * The Hyper API for Python now allows you to use `pathlib.Path` to specify the `hyper_path` when you start the HyperProcess. This is the path to the directory that contains the `hyperd` executable file. 607 | 608 | * Support added for the asterisk (`*`) in namespace-qualified column references. For example, you can select the columns from a table in a specified namespace using three-part names (`schema_name.table_name.*`). 609 | 610 | ```sql 611 | SELECT schema_name.table_name.*, schema_name2.table_name.* FROM schema_name.table_name, schema_name2.table_name ... 612 | ``` 613 | 614 | * Support for quoted strings in CSV headers. 615 | 616 | * When Hyper is launched inside a container (for example, Docker), Hyper now respects the memory limits that are set for the container. Previously, Hyper would assume that full system memory was available. 617 | 618 | * Updated requirements. The Hyper API requires Microsoft Visual C++ Runtime Library version 19.15.26726 or later. You can download the library from Microsoft: [Microsoft Visual C++ Redistributable for Visual Studio 2015, 2017 and 2019](https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads) 619 | 620 | 621 | 622 | --- 623 | 624 | ### 0.0.8953 [October 30, 2019] 625 | 626 | * Various bug fixes. See the **Resolved Issues** on the [Hyper API Product Release and Download](https://tableau.com/support/releases/hyper-api/latest) page. 627 | 628 | * Documentation updates to correct C++ installation instructions, platform support (macOS 10.15 not yet supported). 629 | 630 | **Changed in this release** 631 | 632 | * In the Hyper API (Python), the `name` parameter in the `TableDefinition` method changed to `table_name`. 633 | If you use keyword arguments to define tables in the previous release of the Hyper API, you need to modify your code. 634 | 635 | For example, if you were creating a table called `airports` in the `public` namespace (schema), you would need to make the following change. 636 | 637 | Change: 638 | 639 | ```python 640 | airports_table = TableDefinition(name=TableName( "public", "airports"), ...) 641 | ``` 642 | 643 | To the following: 644 | 645 | ```python 646 | airports_table = TableDefinition(table_name=TableName("public", "airports"), ... ) 647 | ``` 648 | 649 | Note, if you are using positional arguments, you can avoid this issue. 650 | 651 | ```python 652 | airports_table = TableDefinition(TableName("public", "airports"), ...) 653 | ``` 654 | 655 | --- 656 | 657 | ### 0.0.8707 [October 2019] 658 | 659 | The Hyper API replaces the Extract API 2.0 for building applications that create and update Tableau extract files (`.hyper`) for Tableau 10.5 and later. The Hyper API provides more capabilities and improved performance when compared to the previous API. 660 | 661 | * Use SQL statements to insert, read, update, and delete data in extract files 662 | 663 | * Copy data directly from CSV files 664 | 665 | * Create applications in Python, Java, C++, or .NET (C#) 666 | 667 | * Read data from `.hyper` files 668 | 669 | * Update data in existing `.hyper` files 670 | 671 | * Delete data from existing `.hyper` files 672 | 673 | * Drastic performance improvements for extract creation 674 | 675 | --- 676 | --------------------------------------------------------------------------------