├── .gitignore ├── Community-Supported ├── README.md ├── adjust-vertex-order │ ├── README.md │ ├── adjust_vertex_order.py │ ├── europe_interior_left.png │ ├── europe_interior_right.hyper │ └── europe_interior_right.png ├── clouddb-extractor │ ├── README.md │ ├── SAMPLE-config.yml │ ├── SAMPLE-delete_and_append_script.py │ ├── SAMPLE-upsert_script.py │ ├── azuresql_extractor.py │ ├── base_extractor.py │ ├── bigquery_extractor.py │ ├── delete_conditions_example.json │ ├── extractor_cli.py │ ├── mysql_extractor.py │ ├── postgres_extractor.py │ ├── redshift_extractor.py │ └── requirements.txt ├── convert-hyper-file │ ├── README.md │ └── convert_hyper_file.py ├── defragment-hyper-file │ ├── README.md │ └── defragment_data_of_existing_hyper_file.py ├── flights-data-incremental-refresh │ ├── README.md │ ├── flights-data-incremental-refresh.py │ └── requirements.txt ├── git-to-hyper │ ├── README.md │ ├── advanced-git-to-hyper.py │ ├── basic-git-to-hyper.py │ ├── ics.png │ ├── requirements.txt │ └── sloc.png ├── hyper-jupyter-kernel │ ├── README.md │ ├── hyper_kernel │ │ ├── __init__.py │ │ ├── __main__.py │ │ ├── data │ │ │ ├── __init__.py │ │ │ ├── logo-128x128.png │ │ │ ├── logo-32x32.png │ │ │ └── logo-64x64.png │ │ ├── install.py │ │ └── kernel.py │ ├── media │ │ ├── jupyter_console.png │ │ └── jupyterlab.png │ └── setup.py ├── hyper-to-csv │ ├── README.md │ ├── hyper-to-csv.py │ └── requirements.txt ├── list-hyper-contents │ ├── README.md │ └── list_hyper_contents.py ├── native-s3 │ ├── README.md │ ├── join-parquet-and-csv-on-s3.py │ ├── parquet-on-s3-to-hyper.py │ ├── query-csv-on-s3.py │ └── requirements.txt ├── pandas-to-hyper │ ├── README.md │ ├── create_hyper_from_pandas_dataframe.py │ └── requirements.txt ├── parquet-to-hyper │ ├── README.md │ ├── create_hyper_file_from_parquet.py │ └── orders_10rows.parquet ├── publish-hyper │ ├── README.md │ ├── publish-hyper-file.py │ └── requirements.txt ├── publish-multi-table-hyper-legacy │ ├── README.md │ ├── config.json │ ├── publish-multi-table-hyper.py │ └── requirements.txt ├── publish-multi-table-hyper │ ├── README.md │ ├── config.json │ ├── publish-multi-table-hyper.py │ └── requirements.txt ├── query-external-data │ ├── README.md │ ├── customers.csv │ ├── customers_2.csv │ ├── orders.parquet │ └── query_external_data.py ├── s3-compatible-services │ ├── README.md │ └── query-parquet-on-gs.py ├── s3-to-hyper │ ├── README.md │ ├── config.json │ ├── requirements.txt │ └── s3-hyper.py └── tde-to-hyper │ ├── README.md │ ├── requirements.txt │ └── tde_to_hyper.py ├── LICENSE ├── README.md └── Tableau-Supported ├── CPP ├── CMakeLists.txt ├── create_hyper_file_from_csv.cpp ├── data │ ├── Batters.csv │ ├── Staples.csv │ ├── Starbucks.csv │ ├── calcs.csv │ ├── sample_extracts │ │ ├── superstore_sample.hyper │ │ └── superstore_sample_denormalized.hyper │ ├── shunting_report.csv │ ├── superstore.csv │ └── superstore_normalized │ │ ├── addresses.csv │ │ ├── customers.csv │ │ ├── lineitems.csv │ │ ├── orders.csv │ │ ├── products.csv │ │ └── states.csv ├── delete_data_in_existing_hyper_file.cpp ├── insert_data_into_multiple_tables.cpp ├── insert_data_into_single_table.cpp ├── insert_data_with_expressions.cpp ├── insert_geospatial_data_to_a_hyper_file.cpp ├── read_and_print_data_from_existing_hyper_file.cpp └── update_data_in_existing_hyper_file.cpp ├── DotNet ├── CreateHyperFileFromCsv.cs ├── DeleteDataInExistingHyperFile.cs ├── Example.csproj ├── InsertDataIntoMultipleTables.cs ├── InsertDataIntoSingleTable.cs ├── InsertDataWithExpressions.cs ├── InsertSpatialDataToAHyperFile.cs ├── Program.cs ├── ReadDataFromExistingHyperFile.cs ├── UpdateDataInExistingHyperFile.cs ├── build.bat ├── build.sh └── data │ ├── customers.csv │ ├── superstore_sample.hyper │ └── superstore_sample_denormalized.hyper ├── Java ├── build.gradle ├── create-hyper-file-from-csv │ ├── build.gradle │ └── src │ │ └── main │ │ └── java │ │ └── examples │ │ └── CreateHyperFileFromCSV.java ├── data │ ├── addresses.csv │ ├── customers.csv │ ├── lineitems.csv │ ├── orders.csv │ ├── products.csv │ ├── states.csv │ ├── superstore_sample.hyper │ └── superstore_sample_denormalized.hyper ├── delete-data-in-existing-hyper-file │ ├── build.gradle │ └── src │ │ └── main │ │ └── java │ │ └── examples │ │ └── DeleteDataInExistingHyperFile.java ├── gradle │ └── wrapper │ │ ├── gradle-wrapper.jar │ │ └── gradle-wrapper.properties ├── gradlew ├── gradlew.bat ├── insert-data-into-multiple-tables │ ├── build.gradle │ └── src │ │ └── main │ │ └── java │ │ └── examples │ │ └── InsertDataIntoMultipleTables.java ├── insert-data-into-single-table │ ├── build.gradle │ └── src │ │ └── main │ │ └── java │ │ └── examples │ │ └── InsertDataIntoSingleTable.java ├── insert-data-with-expressions │ ├── build.gradle │ └── src │ │ └── main │ │ └── java │ │ └── examples │ │ └── InsertDataWithExpressions.java ├── insert-spatial-data-to-a-hyper-file │ ├── build.gradle │ └── src │ │ └── main │ │ └── java │ │ └── examples │ │ └── InsertSpatialDataToAHyperFile.java ├── read-and-print-data-from-existing-hyper-file │ ├── build.gradle │ └── src │ │ └── main │ │ └── java │ │ └── examples │ │ └── ReadAndPrintDataFromExistingHyperFile.java ├── settings.gradle └── update-data-in-existing-hyper-file │ ├── build.gradle │ └── src │ └── main │ └── java │ └── examples │ └── UpdateDataInExistingHyperFile.java ├── Python ├── create_hyper_file_from_csv.py ├── data │ ├── customers.csv │ ├── superstore_sample.hyper │ └── superstore_sample_denormalized.hyper ├── delete_data_in_existing_hyper_file.py ├── insert_data_into_multiple_tables.py ├── insert_data_into_single_table.py ├── insert_data_with_expressions.py ├── insert_spatial_data_to_a_hyper_file.py ├── read_and_print_data_from_existing_hyper_file.py └── update_data_in_existing_hyper_file.py └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | Community-Supported/clouddb-extractor/debug.log 3 | -------------------------------------------------------------------------------- /Community-Supported/adjust-vertex-order/README.md: -------------------------------------------------------------------------------- 1 | # adjust-vertex-order 2 | ## __adjust_vertex_order__ 3 | 4 | ![Community Supported](https://img.shields.io/badge/Support%20Level-Community%20Supported-53bd92.svg) 5 | 6 | __Current Version__: 1.0 7 | 8 | This sample demonstrates how you can adjust the vertex order of all polygons of an existing `.hyper` file. This is done by copying all of the tables and data into a new file while calling a SQL function (`geo_auto_vertex_order` or `geo_invert_vertex_order`) on all columns of type `GEOGRAPHY`. 9 | 10 | For more information on Hyper's SQL functions that operate on `GEOGRAPHY` values, see [Geographic Functions](https://tableau.github.io/hyper-db/docs/sql/scalar_func/geography). 11 | 12 | ## Background 13 | 14 | For the `GEOGRAPHY` type, in case of polygons, the order of vertices in the polygon's rings defines what is inside and what is outside the polygon. 15 | Tableau's convention is "interior-left", which means that as we progress through the vertices of the polygon in storage order, the interior of the polygon is to the left. 16 | In the "spheroidal" system, as used by Tableau, there is no such thing as "incorrect vertex ordering". Either ordering can be correct because in this system, reverse vertex ordering merely changes what space the polygon encloses. 17 | Hence, there is no way to automatically detect the intention of what space a polygon is intended to enclose. 18 | Depending on the data source used to import `GEOGRAPHY`, Tableau users may experience problems when their `.hyper` files contain `GEOGRAPHY` values with polygons whose vertices are in the reverse order of what is intended by the user. 19 | For example, depending on how the polygons are specified, Tableau may not automatically zoom the map to the data. Another example is with spatial joins which would also produce unexpected results. 20 | 21 | The image below depicts the problem with auto zoom for a collection of polygons, specified with an "interior-right" winding order, marking the boundaries of the European continent: 22 | 23 | ![Europe interior-right](europe_interior_right.png) 24 | 25 | The image below depicts the desired case where the polygons are specified with an "interior-left" winding order: 26 | 27 | ![Europe interior-left](europe_interior_left.png) 28 | 29 | ## Script semantics 30 | 31 | This sample script connects to two `.hyper` files, an input file and an output file, and copies all the schemas, tables and data from the input file into the (newly created) output file. During the copy, either `geo_auto_vertex_order` or `geo_invert_vertex_order` is called on all columns of type `GEOGRAPHY`. The copying of data from source to target tables is done entirely inside Hyper, i.e. no data (apart from table definitions) is exchanged between the Hyper and Python processes. 32 | The script also provides a command to list all tables inside a `.hyper` file and enumerate any columns of type `GEOGRAPHY`. 33 | 34 | ## Running the sample script 35 | This sample script accepts two commands `list` and `run`: 36 | ``` 37 | usage: adjust_vertex_order.py [-h] {list,run} ... 38 | 39 | commands: 40 | {list,run} Available commands 41 | list Lists all tables in a .hyper file and shows columns of type 42 | GEOGRAPHY 43 | run Copies tables from a .hyper file to a new file while adjusting 44 | vertex order of all polygons 45 | ``` 46 | 47 | The syntax for the `run` command is: 48 | ``` 49 | usage: adjust_vertex_order.py run [-h] -i -o -m 50 | {AdjustVertexOrderMode.AUTO,AdjustVertexOrderMode.INVERT} 51 | 52 | optional arguments: 53 | -h, --help show this help message and exit 54 | -i , --input_file 55 | Input .hyper file 56 | -o , --output_file 57 | Output .hyper file 58 | -m {AdjustVertexOrderMode.AUTO,AdjustVertexOrderMode.INVERT}, --mode {AdjustVertexOrderMode.AUTO,AdjustVertexOrderMode.INVERT} 59 | Vertex order adjustment mode: (auto | invert). Auto: 60 | assuming data comes from a source with a flat - earth 61 | topology, it automatically adjusts the vertex order 62 | according to the interior - left definition of 63 | polygons. Invert: inverts the vertex order for all 64 | polygons. 65 | ``` 66 | 67 | The syntax for the `list` command is: 68 | ``` 69 | usage: adjust_vertex_order.py list [-h] -i 70 | 71 | optional arguments: 72 | -h, --help show this help message and exit 73 | -i , --input_file 74 | Input .hyper file 75 | ``` 76 | 77 | Example: 78 | ``` 79 | $ python3 adjust_vertex_order.py run -i ./europe_interior_right.hyper -o ./europe_interior_left.hyper -m auto 80 | Adjusting vertex order of polygons assuming data source with flat - earth topology in spatial columns 81 | Copying table "input"."Extract"."Extract" with 1 spatial columns: [Name('Location')]... 82 | 1 rows copied 83 | ``` 84 | 85 | ## __Resources__ 86 | Check out these resources to learn more: 87 | 88 | - [Hyper API docs](https://tableau.github.io/hyper-db) 89 | 90 | - [Tableau Hyper API Reference (Python)](https://tableau.github.io/hyper-db/lang_docs/py/index.html) 91 | 92 | - [Hyper API SQL Reference](https://tableau.github.io/hyper-db/docs/sql/) 93 | 94 | - [Add Spatial Data to a Hyper File](https://tableau.github.io/hyper-db/docs/guides/hyper_file/geodata) 95 | 96 | - [Geographic Functions in Hyper's SQL API](https://tableau.github.io/hyper-db/docs/sql/scalar_func/geography) 97 | -------------------------------------------------------------------------------- /Community-Supported/adjust-vertex-order/europe_interior_left.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Community-Supported/adjust-vertex-order/europe_interior_left.png -------------------------------------------------------------------------------- /Community-Supported/adjust-vertex-order/europe_interior_right.hyper: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Community-Supported/adjust-vertex-order/europe_interior_right.hyper -------------------------------------------------------------------------------- /Community-Supported/adjust-vertex-order/europe_interior_right.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Community-Supported/adjust-vertex-order/europe_interior_right.png -------------------------------------------------------------------------------- /Community-Supported/clouddb-extractor/SAMPLE-config.yml: -------------------------------------------------------------------------------- 1 | #Configuration file for clouddb_extractor utilities 2 | default_extractor : "bigquery" 3 | sample_rows : 1000 # How many rows to extract when using load_sample to preview a table 4 | tableau_env: #Tableau environment configuration defaults 5 | server_address : "https://eu-west-1a.online.tableau.com/" # The address of target Tableau Server or Tableau Online 6 | site_id : "testsiteid" # This corresponds to the contentUrl attribute in the Tableau REST API. 7 | project : "hyperapitest" # The default Tableau project to store extract datasources 8 | token_name : hyperapitest # Personal access token name (Do not specify if using tableau_username and password) 9 | token_secretfile : hyperapitest.token # Personal access token name (Do not specify if using tableau_username and password) 10 | bigquery: #BigQueryExtractor configuration defaults 11 | staging_bucket : "HyperAPITestStaging" # Cloud blob storage bucket for extract staging if required 12 | postgres: #PostgreSQL configuration defaults 13 | connection: 14 | dbname : "dev" 15 | user : "test" 16 | password : "password" 17 | host : "postgres.test" 18 | port : 5432 19 | mysql: #Mysql configuration defaults 20 | connection: 21 | host : "mysql.test" 22 | database : "dev" 23 | port : 3306 24 | username : "test" 25 | password : "password" 26 | raise_on_warnings : True 27 | azuresql: #Azure SQL Database configuration defaults 28 | connection: 29 | host : "mydbserver.test" 30 | port : 1433 31 | database : "test" 32 | username : "test" 33 | #Recommended: use key vault instead of plain text password for production 34 | #key_vault_url : "https://.vault.azure.net" 35 | #secret_name : my-password-secret 36 | password : "password" 37 | connect_str_suffix : "Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30" 38 | raise_on_warnings : True 39 | # #Manually define target hyper column type 40 | # #Workaround if Precision,Scale not reported correctly for some SQL Server implementations 41 | # # For target types and args see tableauhyperapi.SqlType 42 | override_column_type: 43 | "example_decimal_col": 44 | sqltype: "numeric" 45 | precision: 18 46 | scale: 3 47 | "example_int_col" : 48 | sqltype: "int" 49 | redshift: #Redshift configuration defaults 50 | connection: 51 | host : 'redshift-cluster-1.XXX.eu-west-1.redshift.amazonaws.com' 52 | database : 'dev' 53 | user : 'test' 54 | password : 'password' -------------------------------------------------------------------------------- /Community-Supported/clouddb-extractor/SAMPLE-delete_and_append_script.py: -------------------------------------------------------------------------------- 1 | """ Sample script file shows how to execute multiple commands in a single batch operation 2 | 3 | Tableau Community supported Hyper API sample 4 | 5 | This script uses the database_extractor utilities to: 6 | - 1: Query records for the most recent UPDATE_WINDOW_DAYS days from source database and 7 | write to a changeset hyper file 8 | - 2: Delete the most recent N days from target datasource 9 | - 3: Append the changeset to target datasource 10 | 11 | Update "#Batch Configuration" and "#Query and action settings" below and 12 | the config.yml file with your site configuration before using. 13 | 14 | ----------------------------------------------------------------------------- 15 | 16 | This file is the copyrighted property of Tableau Software and is protected 17 | by registered patents and other applicable U.S. and international laws and 18 | regulations. 19 | 20 | You may adapt this file and modify it to fit into your context and use it 21 | as a template to start your own projects. 22 | 23 | ----------------------------------------------------------------------------- 24 | """ 25 | 26 | from azuresql_extractor import AzureSQLExtractor 27 | import os 28 | import yaml 29 | import logging 30 | from datetime import datetime, timedelta 31 | 32 | 33 | #Batch Configuration 34 | CONFIG_FILE = "config.yml" 35 | SOURCE_TABLE = "MY_SOURCE_TABLE" 36 | TARGET_DATASOURCE = "MY_TARGET_DATASOURCE" 37 | UPDATE_WINDOW_DAYS = 7 38 | TARGET_DATE_COLUMN = "LOAD_DATE" 39 | 40 | #Load Defaults from CONFIG_FILE 41 | config = yaml.safe_load(open(CONFIG_FILE)) 42 | tableau_env = config.get("tableau_env") 43 | db_env = config.get("azuresql") 44 | 45 | #Calculated datetime fields 46 | target_date_start = datetime.today() - timedelta(days=UPDATE_WINDOW_DAYS) 47 | 48 | #Query and Action Settings 49 | SOURCE_SQL = f"SELECT * from {SOURCE_TABLE} WHERE {TARGET_DATE_COLUMN} >= '{target_date_start.strftime('%Y-%m-%d')}'" 50 | ACTION_BATCH_JSON = [ 51 | { 52 | "action": "delete", 53 | "target-schema": "Extract", 54 | "target-table": "Extract", 55 | "condition": { 56 | "op": "gte", 57 | "target-col": TARGET_DATE_COLUMN, 58 | "const": {"type": "datetime", "v": target_date_start.astimezone().isoformat()} # datetime is in ISO 8601 format e.g. 2007-12-03T10:15:30Z (for UTC time) or 2007-12-03T10:15:30+0100 (for a timezone UTC+1:00) 59 | }, 60 | }, 61 | { 62 | "action": "insert", 63 | "source-schema": "Extract", 64 | "source-table": "Extract", 65 | "target-schema": "Extract", 66 | "target-table": "Extract", 67 | }, 68 | ] 69 | 70 | # Initialize logging 71 | consoleHandler = logging.StreamHandler() 72 | logging.basicConfig( 73 | level=logging.DEBUG, 74 | format="%(asctime)s - %(name)s %(funcName)s - %(levelname)s - %(message)s", 75 | handlers=[logging.FileHandler("debug.log"), consoleHandler], 76 | ) 77 | consoleHandler.setLevel(logging.INFO) 78 | 79 | logger = logging.getLogger("hyper_samples.batch_script") 80 | 81 | def main(): 82 | logger.info(f"Load private access token") 83 | with open(tableau_env.get("token_secretfile"), "r") as myfile: 84 | tableau_token_secret = myfile.read().strip() 85 | 86 | logger.info(f"Initialize extractor") 87 | extractor = AzureSQLExtractor( 88 | source_database_config=db_env, 89 | tableau_hostname=tableau_env.get("server_address"), 90 | tableau_project=tableau_env.get("project"), 91 | tableau_site_id=tableau_env.get("site_id"), 92 | tableau_token_name=tableau_env.get("token_name"), 93 | tableau_token_secret=tableau_token_secret, 94 | ) 95 | 96 | # Run Job 97 | logger.info(f"Connecting to source database") 98 | cursor = extractor.source_database_cursor() 99 | 100 | logger.info(f"Execute SQL:{SOURCE_SQL}") 101 | cursor.execute(SOURCE_SQL) 102 | 103 | logger.info(f"Extract to local changeset hyper file") 104 | path_to_database = extractor.query_result_to_hyper_file(cursor=cursor, hyper_table_name="Extract") 105 | 106 | logger.info(f"Transfer changeset hyper file {path_to_database} and execute batch {ACTION_BATCH_JSON}") 107 | extractor.patch_datasource( 108 | tab_ds_name=TARGET_DATASOURCE, 109 | actions_json=ACTION_BATCH_JSON, 110 | path_to_database=path_to_database, 111 | ) 112 | 113 | logger.info(f"Remove local changeset hyper file {path_to_database}") 114 | os.remove(path_to_database) 115 | 116 | if __name__ == "__main__": 117 | main() -------------------------------------------------------------------------------- /Community-Supported/clouddb-extractor/SAMPLE-upsert_script.py: -------------------------------------------------------------------------------- 1 | """ Sample script file showing how to call clouddb-extractor extractor class directly 2 | 3 | Tableau Community supported Hyper API sample 4 | 5 | This script uses the database_extractor utilities to: 6 | - Query records for the most recent UPDATE_WINDOW_DAYS days from source database and 7 | write to a changeset hyper file 8 | - Update TARGET_DATASOURCE where primary key columns identified by MATCHING_COLUMNS 9 | in changeset match corresponding columns in target datasource, else insert) 10 | 11 | Update "#Batch Configuration" and "#Query and action settings" below and 12 | the config.yml file with your site configuration before using. 13 | 14 | ----------------------------------------------------------------------------- 15 | 16 | This file is the copyrighted property of Tableau Software and is protected 17 | by registered patents and other applicable U.S. and international laws and 18 | regulations. 19 | 20 | You may adapt this file and modify it to fit into your context and use it 21 | as a template to start your own projects. 22 | 23 | ----------------------------------------------------------------------------- 24 | """ 25 | 26 | from azuresql_extractor import AzureSQLExtractor 27 | import os 28 | import yaml 29 | import logging 30 | from datetime import datetime, timedelta 31 | 32 | 33 | #Batch Configuration 34 | CONFIG_FILE = "config.yml" 35 | SOURCE_TABLE = "MY_SOURCE_TABLE" 36 | TARGET_DATASOURCE = "MY_TARGET_DATASOURCE" 37 | UPDATE_WINDOW_DAYS = 7 38 | TARGET_DATE_COLUMN = "LOAD_DATE" 39 | 40 | #Load Defaults from CONFIG_FILE 41 | config = yaml.safe_load(open(CONFIG_FILE)) 42 | tableau_env = config.get("tableau_env") 43 | db_env = config.get("azuresql") 44 | 45 | #Calculated datetime fields 46 | target_date_start = datetime.today() - timedelta(days=UPDATE_WINDOW_DAYS) 47 | 48 | #Query and Action Settings 49 | SOURCE_SQL = f"SELECT * from {SOURCE_TABLE} WHERE {TARGET_DATE_COLUMN} >= '{target_date_start.strftime('%Y-%m-%d')}'" 50 | MATCH_COLUMNS = [ 51 | ['PK_COLUMN_A','PK_COLUMN_A'], 52 | ['PK_COLUMN_B','PK_COLUMN_B'], 53 | ['PK_COLUMN_C','PK_COLUMN_C'], 54 | ] 55 | 56 | # Initialize logging 57 | consoleHandler = logging.StreamHandler() 58 | logging.basicConfig( 59 | level=logging.DEBUG, 60 | format="%(asctime)s - %(name)s %(funcName)s - %(levelname)s - %(message)s", 61 | handlers=[logging.FileHandler("debug.log"), consoleHandler], 62 | ) 63 | consoleHandler.setLevel(logging.INFO) 64 | 65 | logger = logging.getLogger("hyper_samples.batch_script") 66 | 67 | def main(): 68 | logger.info(f"Load private access token") 69 | with open(tableau_env.get("token_secretfile"), "r") as myfile: 70 | tableau_token_secret = myfile.read().strip() 71 | 72 | logger.info(f"Initialize extractor") 73 | extractor = AzureSQLExtractor( 74 | source_database_config=db_env, 75 | tableau_hostname=tableau_env.get("server_address"), 76 | tableau_project=tableau_env.get("project"), 77 | tableau_site_id=tableau_env.get("site_id"), 78 | tableau_token_name=tableau_env.get("token_name"), 79 | tableau_token_secret=tableau_token_secret, 80 | ) 81 | 82 | # Run Job 83 | logger.info(f"Run UPSERT action") 84 | extractor.upsert_to_datasource( 85 | tab_ds_name=TARGET_DATASOURCE, 86 | sql_query=SOURCE_SQL, 87 | match_columns=MATCH_COLUMNS, 88 | ) 89 | 90 | if __name__ == "__main__": 91 | main() -------------------------------------------------------------------------------- /Community-Supported/clouddb-extractor/delete_conditions_example.json: -------------------------------------------------------------------------------- 1 | { 2 | "op": "lt", 3 | "target-col": "ORDER_DATE", 4 | "const": {"type": "datetime", "v": "2018-02-01T00:00:00Z"} 5 | } 6 | -------------------------------------------------------------------------------- /Community-Supported/clouddb-extractor/requirements.txt: -------------------------------------------------------------------------------- 1 | filelock==3.0.12 2 | PyYAML==5.4.1 3 | toml==0.10.2 4 | types-filelock==0.1.3 5 | types-futures==0.1.3 6 | types-protobuf==0.1.11 7 | types-requests==0.1.9 8 | typing==3.7.4.3 9 | typing-extensions==3.10.0.0 10 | -------------------------------------------------------------------------------- /Community-Supported/convert-hyper-file/README.md: -------------------------------------------------------------------------------- 1 | # convert-hyper-file 2 | ## __convert_hyper_file__ 3 | 4 | 5 | 6 | ![Community Supported](https://img.shields.io/badge/Support%20Level-Community%20Supported-53bd92.svg) 7 | 8 | __Current Version__: 1.0 9 | 10 | This sample demonstrates how you can upgrade or downgrade an existing `.hyper` file to a newer or older Hyper file format by copying all of the tables and data into a new file. 11 | 12 | Fore more information on the Hyper file formats, see [Hyper Process Settings](https://tableau.github.io/hyper-db/docs/hyper-api/hyper_process#process-settings). 13 | 14 | # Get started 15 | 16 | ## __Prerequisites__ 17 | 18 | To run the script, you will need: 19 | 20 | - a computer running Windows, macOS, or Linux 21 | 22 | - Python 3.6 or 3.7 23 | 24 | ## Run the sample 25 | 26 | As a quick test of how the sample works, you don't need to make any changes to the `convert_hyper_file.py` file. You can simply run the sample on a `.hyper` file to see how it downgrades a Hyper file to the initial file format version 0. The Python sample reads an existing `.hyper` file and copies all the tables and data into a new file. 27 | 28 | Ensure that you have installed the requirements and then just run the sample Python file. 29 | The following instructions assume that you have set up a virtual environment for Python. For more information on creating virtual environments, see [venv - Creation of virtual environments](https://docs.python.org/3/library/venv.html) in the Python Standard Library. 30 | 31 | 1. Open a terminal and activate the Python virtual environment (`venv`). 32 | 33 | 1. Navigate to the folder where you installed the sample. 34 | 35 | 1. Run the Python script against an existing `.hyper` file. The syntax for running the script is: 36 | 37 | **python convert_hyper_file.py [-h] [--output-hyper-file-path OUTPUT_HYPER_FILE_PATH] [--output-hyper-file-version OUTPUT_HYPER_FILE_VERSION] input_hyper_file_path** 38 | 39 | If you do not specify an output file, a new file is created with the same name and with `.[new version].hyper` as the file name extension. The new version of the file contains all the data of the existing file with the specified version. 40 | 41 | Example: 42 | 43 | ```cli 44 | (venv)c:\mydir> python convert_hyper_file.py -o NewVersionFile.hyper -v 1 OldVersionFile.hyper 45 | Successfully converted table "input_database"."Extract"."Extract" 46 | Successfully converted OldVersionFile.hyper into NewVersionFile.hyper 47 | ``` 48 | 49 | ## __Resources__ 50 | Check out these resources to learn more: 51 | 52 | - [Hyper API docs](https://tableau.github.io/hyper-db) 53 | 54 | - [Tableau Hyper API Reference (Python)](https://tableau.github.io/hyper-db/lang_docs/py/index.html) 55 | 56 | - [Hyper API SQL Reference](https://tableau.github.io/hyper-db/docs/sql/) 57 | -------------------------------------------------------------------------------- /Community-Supported/convert-hyper-file/convert_hyper_file.py: -------------------------------------------------------------------------------- 1 | # ----------------------------------------------------------------------------- 2 | # 3 | # This file is the copyrighted property of Tableau Software and is protected 4 | # by registered patents and other applicable U.S. and international laws and 5 | # regulations. 6 | # 7 | # You may adapt this file and modify it to fit into your context and use it 8 | # as a template to start your own projects. 9 | # 10 | # ----------------------------------------------------------------------------- 11 | 12 | import argparse 13 | from pathlib import PurePath 14 | from tableauhyperapi import HyperProcess, Telemetry, Connection, TableDefinition, TableName 15 | 16 | if __name__ == "__main__": 17 | argparser = argparse.ArgumentParser(description="This tool converts the given Hyper file to the selected file format version. Per default, it will downgrade to the initial file format (version 0).") 18 | argparser.add_argument("input_hyper_file_path", type=PurePath, help="The input Hyper file path that will be converted to selected file format version") 19 | argparser.add_argument("--output-hyper-file-path", "-o", type=PurePath, help="The output Hyper file path for the converted output Hyper file") 20 | argparser.add_argument("--output-hyper-file-version", "-v", type=int, help="The output Hyper file version for the converted output Hyper file. Defaults to the initial file format (version 0)", default=0) 21 | args = argparser.parse_args() 22 | if not args.output_hyper_file_path: 23 | args.output_hyper_file_path = args.input_hyper_file_path.parent / (args.input_hyper_file_path.stem + f".version{args.output_hyper_file_version}.hyper") 24 | 25 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU, parameters={"default_database_version": f"{args.output_hyper_file_version}"}) as hyper: 26 | with Connection(endpoint=hyper.endpoint) as connection: 27 | # Create the output Hyper file or overwrite it 28 | catalog = connection.catalog 29 | catalog.drop_database_if_exists(args.output_hyper_file_path) 30 | catalog.create_database(args.output_hyper_file_path) 31 | catalog.attach_database(args.output_hyper_file_path, alias="output_database") 32 | catalog.attach_database(args.input_hyper_file_path, alias="input_database") 33 | 34 | # Process all tables of all schemas of the input Hyper file and copy them into the output Hyper file 35 | for input_schema_name in catalog.get_schema_names("input_database"): 36 | for input_table_name in catalog.get_table_names(input_schema_name): 37 | output_table_name = TableName("output_database", input_schema_name.name, input_table_name.name) 38 | output_table_definition = TableDefinition(output_table_name, catalog.get_table_definition(input_table_name).columns) 39 | catalog.create_schema_if_not_exists(output_table_name.schema_name) 40 | catalog.create_table(output_table_definition) 41 | connection.execute_command(f"INSERT INTO {output_table_name} (SELECT * FROM {input_table_name})") 42 | print(f"Successfully converted table {input_table_name}") 43 | print(f"Successfully converted {args.input_hyper_file_path} into {args.output_hyper_file_path}") 44 | -------------------------------------------------------------------------------- /Community-Supported/defragment-hyper-file/README.md: -------------------------------------------------------------------------------- 1 | # defragment-data-of-existing-hyper-file 2 | ## __defragment_data_of_existing_hyper_file__ 3 | 4 | 5 | 6 | ![Community Supported](https://img.shields.io/badge/Support%20Level-Community%20Supported-53bd92.svg) 7 | 8 | __Current Version__: 1.0 9 | 10 | This sample demonstrates how you can optimize the file storage of an existing `.hyper` file by copying all of the tables and data into a new file, in a continuous sequence, to eliminate any fragmentation that might have occurred. This method can improve the performance of your Hyper file when you have large amounts of data, as fragmentation can increase both file size and access times. 11 | 12 | This sample should serve as a starting point for anyone looking for a programmatic way to reduce the fragmentation of their `.hyper` file. 13 | 14 | For a description of how fragmentation can occur in the `.hyper` file and for ways of minimizing it's occurrence, see [Optimize Hyper File Storage](https://tableau.github.io/hyper-db/docs/guides/hyper_file/optimize). 15 | 16 | # Get started 17 | 18 | ## __Prerequisites__ 19 | 20 | To run the script, you will need: 21 | 22 | - a computer running Windows, macOS, or Linux 23 | 24 | - Tableau Desktop v10.5 or later 25 | 26 | - Python 3.6 or 3.7 27 | 28 | ## Run the sample 29 | 30 | As a quick test of how the sample works, you don't need to make any changes to the `defragment_data_of_existing_hyper_file.py` file. You can simply run the sample on a `.hyper` file to see how it can reduce fragmentation. The Python sample reads an existing `.hyper` file and copies all the tables and data into a new file. 31 | 32 | Ensure that you have installed the requirements and then just run the sample Python file. 33 | The following instructions assume that you have set up a virtual environment for Python. For more information on creating virtual environments, see [venv - Creation of virtual environments](https://docs.python.org/3/library/venv.html) in the Python Standard Library. 34 | 35 | 1. Open a terminal and activate the Python virtual environment (`venv`). 36 | 37 | 1. Navigate to the folder where you installed the sample. 38 | 39 | 1. Run the Python script against an existing `.hyper` file. The syntax for running the script is: 40 | 41 | **python defragment_data_of_existing_hyper_file.py [-h] [--output-hyper-file-path OUTPUT_HYPER_FILE_PATH] input_hyper_file_path** 42 | 43 | If you do not specify an output file, a new file is created with the same name and with `.new.hyper` as the file name extension. The new version of the file contains all the data of the existing file, without the fragmentation. 44 | 45 | Example: 46 | 47 | ```cli 48 | (venv)c:\mydir> python defragment_data_of_existing_hyper_file.py -o Newfile.hyper Oldfile.hyper 49 | Successfully converted table "input_database"."Extract"."Extract" 50 | Successfully converted Oldfile.hyper into Newfile.hyper 51 | ``` 52 | 53 | 54 | ## __Customization__ 55 | 56 | The script, `defragment_data_of_existing_hyper_file.py`, should work "as is" in most cases. If your Hyper file contains metadata that is not stored in the table definitions, for example, column descriptions or assumed constraints, the script will still work as expected and will create a new `.hyper` file with all the tables and data. However, the new file will not contain the metadata. In that case, you might need to modify the script to preserve that information. If you created the constraints and definitions yourself, using your own SQL statements, the Python script should give you a good starting point for adding your own code. 57 | 58 | If you use the script on an extract file created using Tableau or Prep, be aware that the metadata (if it exists) will not be copied to the new file. The script only copies the data contained in the table definitions. 59 | 60 | To learn more about what is possible with the Hyper API, see the [official Hyper API samples](https://github.com/tableau/hyper-api-samples/tree/master/Python). 61 | 62 | ## __Resources__ 63 | Check out these resources to learn more: 64 | 65 | - [Hyper API docs](https://tableau.github.io/hyper-db) 66 | 67 | - [Tableau Hyper API Reference (Python)](https://tableau.github.io/hyper-db/lang_docs/py/index.html) 68 | 69 | - [Hyper API SQL Reference](https://tableau.github.io/hyper-db/docs/sql/) 70 | -------------------------------------------------------------------------------- /Community-Supported/defragment-hyper-file/defragment_data_of_existing_hyper_file.py: -------------------------------------------------------------------------------- 1 | # ----------------------------------------------------------------------------- 2 | # 3 | # This file is the copyrighted property of Tableau Software and is protected 4 | # by registered patents and other applicable U.S. and international laws and 5 | # regulations. 6 | # 7 | # You may adapt this file and modify it to fit into your context and use it 8 | # as a template to start your own projects. 9 | # 10 | # ----------------------------------------------------------------------------- 11 | import argparse 12 | from pathlib import PurePath 13 | from tableauhyperapi import HyperProcess, Telemetry, Connection, TableDefinition, TableName 14 | 15 | # An example of how you can optimize the file storage of an existing `.hyper` file by copying 16 | # all of the tables and data into a new file. This reduces file fragmentation. 17 | 18 | if __name__ == "__main__": 19 | argparser = argparse.ArgumentParser(description="This tool rewrites a given Hyper file in an optimized, dense format.") 20 | argparser.add_argument("input_hyper_file_path", type=PurePath, help="The input Hyper file path that will be rewritten.") 21 | argparser.add_argument("--output-hyper-file-path", "-o", type=PurePath, help="The output Hyper file path for the rewritten output Hyper file.") 22 | args = argparser.parse_args() 23 | if not args.output_hyper_file_path: 24 | args.output_hyper_file_path = args.input_hyper_file_path.parent / (args.input_hyper_file_path.stem + f".new.hyper") 25 | 26 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 27 | with Connection(endpoint=hyper.endpoint) as connection: 28 | # Connect to the input and output databases 29 | # Create the output Hyper file or overwrite it 30 | catalog = connection.catalog 31 | catalog.drop_database_if_exists(args.output_hyper_file_path) 32 | catalog.create_database(args.output_hyper_file_path) 33 | catalog.attach_database(args.output_hyper_file_path, alias="output_database") 34 | catalog.attach_database(args.input_hyper_file_path, alias="input_database") 35 | 36 | # Process all tables of all schemas of the input Hyper file and copy them into the output Hyper file 37 | for input_schema_name in catalog.get_schema_names("input_database"): 38 | for input_table_name in catalog.get_table_names(input_schema_name): 39 | output_table_name = TableName("output_database", input_schema_name.name, input_table_name.name) 40 | output_table_definition = TableDefinition(output_table_name, catalog.get_table_definition(input_table_name).columns) 41 | catalog.create_schema_if_not_exists(output_table_name.schema_name) 42 | catalog.create_table(output_table_definition) 43 | connection.execute_command(f"INSERT INTO {output_table_name} (SELECT * FROM {input_table_name})") 44 | print(f"Successfully converted table {input_table_name}") 45 | print(f"Successfully converted {args.input_hyper_file_path} into {args.output_hyper_file_path}") -------------------------------------------------------------------------------- /Community-Supported/flights-data-incremental-refresh/README.md: -------------------------------------------------------------------------------- 1 | # flights-data-incremental-refresh 2 | ## __Incremental Refresh using the OpenSkyApi__ 3 | 4 | ![Community Supported](https://img.shields.io/badge/Support%20Level-Community%20Supported-53bd92.svg) 5 | 6 | This sample is based on the content the Hyper team presented in the Hands on Training session "Hands-on: Leverage the Hyper Update API and Hyper API to Keep Your Data Fresh on Tableau Server" at Tableau Conference 2022 ([slides available here](https://mkt.tableau.com/tc22/sessions/live/430-HOT-D1_Hands-onLeverageTheHyperUpdate.pdf)). 7 | 8 | This script pulls down flights data from the [OpenSkyAPI](https://github.com/openskynetwork/opensky-api), creates a hyper database with this data and uses the [Hyper Update API](https://help.tableau.com/current/api/rest_api/en-us/REST/rest_api_how_to_update_data_to_hyper.htm) to implement an incremental refresh on your Tableau Server/Cloud. The first time this script is executed, the database file is simply published. 9 | 10 | # Get started 11 | 12 | ## __Prerequisites__ 13 | To run the script, you will need: 14 | - Windows, Linux, or Mac 15 | - Python 3 16 | - Run `pip install -r requirements.txt` 17 | - Tableau Server Credentials, see below. 18 | 19 | ## Tableau Server Credentials 20 | To run this sample with your Tableau Server/Cloud, you first need to get the following information: 21 | - Tableau Server Url, e.g. 'https://us-west-2a.online.tableau.com' 22 | - Site name, e.g., use 'default' for your default site (note that you cannot use 'default' in Tableau Cloud but must use the site name) 23 | - Project name, e.g., use an empty string ('') for your default project 24 | - [Token Name and Token Value](https://help.tableau.com/current/server/en-us/security_personal_access_tokens.htm) 25 | 26 | Ensure that you have installed the requirements and then just run the sample Python file with the information from above. The syntax for running the script is: 27 | 28 | **python flights-data-incremental-refresh.py [-h] server_url site_name project_name token_name token_value** 29 | 30 | # Incremental Refresh using the OpenSkyApi 31 | The script consists of two parts: first it creates a Hyper database with flights data and then publishes the database to Tableau Server/Cloud. 32 | 33 | ## Create a database with flights data 34 | The `create_hyper_database_with_flights_data` method creates an instance of the `OpenSkyAPI` and then pulls down states within a specific bounding box. This example just uses a subset of the available data as we are using the free version of the OpenSkyApi. 35 | 36 | Then, a Hyper database is created with a table with name `TableName("public", "flights")`. Finally, an inserter is used to insert the flights data. 37 | 38 | ## Publish the hyper database to Tableau Server / Cloud 39 | The `publish_to_server` method first signs into Tableau Server / Cloud. Then, it finds the respective project to which the database should be published to. 40 | 41 | There are two cases for publishing the database to Server: 42 | - No datasource with name `datasource_name_on_server` exists on Tableau Server. In this case, the script simply creates the initial datasource on Tableau server. This datasource is needed for the subsequent incremental refreshes as the data will be added to this datasource. 43 | - The datasource with name `datasource_name_on_server` already exists on Tableau Server. In this case, the script uses the Hyper Update REST API to insert the data from the database into the respective table in the datasource on Tableau Server/Cloud. 44 | 45 | ## __Resources__ 46 | Check out these resources to learn more: 47 | - [Hyper API documentation](https://tableau.github.io/hyper-db) 48 | - [Hyper Update API documentation](https://help.tableau.com/current/api/rest_api/en-us/REST/rest_api_how_to_update_data_to_hyper.htm) 49 | - [Tableau Server Client Docs](https://tableau.github.io/server-client-python/docs/) 50 | - [REST API documentation](https://help.tableau.com/current/api/rest_api/en-us/REST/rest_api.htm) 51 | - [Tableau Tools](https://github.com/bryantbhowell/tableau_tools) 52 | -------------------------------------------------------------------------------- /Community-Supported/flights-data-incremental-refresh/requirements.txt: -------------------------------------------------------------------------------- 1 | tableauhyperapi>=0.0.14946 2 | tableauserverclient>=0.19.0 3 | https://github.com/openskynetwork/opensky-api/archive/master.zip#subdirectory=python -------------------------------------------------------------------------------- /Community-Supported/git-to-hyper/README.md: -------------------------------------------------------------------------------- 1 | # git-to-hyper 2 | ## __git_to_hyper__ 3 | 4 | ![Community Supported](https://img.shields.io/badge/Support%20Level-Community%20Supported-53bd92.svg) 5 | 6 | __Current Version__: 1.0 7 | 8 | This sample shows you how use Hyper and Tableau to gain insights into a software project using the git meta data (e.g. commit timestamps). The script extracts this data from a given git repository and stores is in a `git.hyper` file which can be used in Tableau. 9 | 10 | Here are some examples for the [bazel-remote cache](https://github.com/buchgr/bazel-remote) OSS project: 11 | 12 | ![Tableau plot of number of ICs](ics.png) 13 | 14 | ![Tableau plot of number of SLOC](sloc.png) 15 | 16 | 17 | ## Prerequisites 18 | 19 | To run the script, you will need: 20 | 21 | - a computer running Windows, macOS, or Linux 22 | - Python 3.9+ 23 | - install the dependencies from the `requirements.txt` file 24 | 25 | ## Run the basic sample 26 | 27 | ```bash 28 | $ git clone https://github.com/tableau/hyper-api-samples.git 29 | $ cd hyper-api-samples 30 | $ pip3 install -r Community-Supported/git-to-hyper/requirements.txt 31 | $ python Community-Supported/git-to-hyper/basic-git-to-hyper.py --path_to_repo ~/sample/repository 32 | ``` 33 | 34 | The basic sample is a very lightweight example of how to extract meta data from git and save it into a Hyper database. It extracts only very high-level information like timestamps of commits, their authors and metrics like number of added lines. 35 | 36 | The script offers the following options: 37 | 38 | ```bash 39 | $ python Community-Supported/git-to-hyper/basic-git-to-hyper.py --help 40 | usage: basic-git-to-hyper.py [-h] [--branch BRANCH] path_to_repo 41 | 42 | positional arguments: 43 | path_to_repo Path to the repository, e.g. ~/src/repo 44 | 45 | options: 46 | -h, --help show this help message and exit 47 | --branch BRANCH Branch to follow in the repository. Default: main 48 | ``` 49 | 50 | The script will generate a `git.hyper` file which can be used in Tableau for further analysis. 51 | 52 | ## Run the advanced sample 53 | 54 | ```bash 55 | $ git clone https://github.com/tableau/hyper-api-samples.git 56 | $ cd hyper-api-samples 57 | $ pip3 install -r Community-Supported/git-to-hyper/requirements.txt 58 | $ python Community-Supported/git-to-hyper/advanced-git-to-hyper.py --path_to_repo ~/sample/repository 59 | ``` 60 | 61 | The advanced sample gathers further data by e.g. running `git blame` for every changed file to count the source lines of code for every author. The results are stored in multiple tables inside the Hyper file and `FOREIGN_KEY` is used to link them together (see [here](https://github.com/tableau/hyper-api-samples/tree/main/Community-Supported/publish-multi-table-hyper) for a detailed explanation on how to use multiple tables in Hyper). 62 | 63 | In order to speed things up multiple processes are used for data extraction and a single injection process for the communication to the Hyper database. Even if you are not interested in the git meta data, this sample can still be of interest to you if you are looking for ways to speed up your data extraction by using multiple processes. 64 | 65 | The sample can optionally use a ram disk to speed up the extraction, as the duration of running `git blame` on every changed file is primarily limited by the I/O speed of your system. Moving the files into memory using a [ram disk](https://de.wikipedia.org/wiki/RAM-Disk) makes the execution time mostly independent from the I/O speed of your hard drive. Most Linux operating system provide a ram disk out-of-the box for you, mounted at `/dev/shm`. If the option `--ram_disk_dir` is not set no ram disk will used. 66 | 67 | The script offers the following options: 68 | 69 | ```bash 70 | $ python Community-Supported/git-to-hyper/advanced-git-to-hyper.py --help 71 | usage: advanced-git-to-hyper.py [-h] [--branch BRANCH] [--ram_disk_dir RAM_DISK_DIR] [--number_of_workers NUMBER_OF_WORKERS] 72 | [--file_size_limit FILE_SIZE_LIMIT] [--blame_only_for_head] [--verbose] 73 | path_to_repo 74 | 75 | positional arguments: 76 | path_to_repo Path to the repository, e.g. ~/src/repo 77 | 78 | options: 79 | -h, --help show this help message and exit 80 | --branch BRANCH Branch to follow in the repository. Default: main 81 | --ram_disk_dir RAM_DISK_DIR 82 | Path to ram disk on the host machine. Used to improve the execution time by speeding up I/O heavy git operations. using "/dev/shm" should work for most Linux OS, if you are using a different OS you might need to create the ram disk manually first. It needs to have at least the size of the repository. 83 | --number_of_workers NUMBER_OF_WORKERS 84 | How many parallel processes shall be used for the data extraction 85 | --file_size_limit FILE_SIZE_LIMIT 86 | Files bigger than this limit are not analyzed. The unit is byte. Can be turned off by setting it to None. Default: 10 MB 87 | --blame_only_for_head 88 | Run git blame only for the HEAD commit to speed up the data collection 89 | --verbose Increase verbosity, e.g. print filenames of git blame targets 90 | ``` 91 | 92 | The script will generate a `git.hyper` file which can be used in Tableau for further analysis. 93 | 94 | ## __Resources__ 95 | Check out these resources to learn more: 96 | 97 | - [Hyper API docs](https://tableau.github.io/hyper-db) 98 | 99 | - [Tableau Hyper API Reference (Python)](https://tableau.github.io/hyper-db/lang_docs/py/index.html) 100 | -------------------------------------------------------------------------------- /Community-Supported/git-to-hyper/basic-git-to-hyper.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | 3 | 4 | import tableauhyperapi as hapi 5 | 6 | 7 | from pathlib import Path 8 | from datetime import datetime 9 | from git import Repo 10 | from git.objects import Commit 11 | 12 | 13 | __database_file = Path("git.hyper") 14 | 15 | 16 | def create_table(con: hapi.Connection): 17 | columns = [ 18 | ('commit_sha', hapi.SqlType.text()), 19 | ('authored_date', hapi.SqlType.timestamp()), 20 | ('committed_date', hapi.SqlType.timestamp()), 21 | ('author_mail', hapi.SqlType.text()), 22 | ('committer_mail', hapi.SqlType.text()), 23 | ('insertions', hapi.SqlType.int()), 24 | ('deletions', hapi.SqlType.int()), 25 | ('number_of_changed_lines', hapi.SqlType.int()), 26 | ('number_of_changed_files', hapi.SqlType.int()), 27 | ('changed_files', hapi.SqlType.text()), 28 | ('message', hapi.SqlType.text()), 29 | ] 30 | hapi_columns = [hapi.TableDefinition.Column(name=col[0], type=col[1], nullability=hapi.NOT_NULLABLE) for col in columns] 31 | table = hapi.TableDefinition(table_name='commits', columns=hapi_columns) 32 | con.catalog.create_table(table_definition=table) 33 | return table 34 | 35 | 36 | def insert_commit_data(commit: Commit, table: hapi.TableDefinition, con: hapi.Connection): 37 | with hapi.Inserter(con, table) as inserter: 38 | inserter.add_row([commit.hexsha, 39 | datetime.utcfromtimestamp(commit.authored_date), 40 | datetime.utcfromtimestamp(commit.committed_date), 41 | commit.author.email, 42 | commit.committer.email, 43 | commit.stats.total['insertions'], 44 | commit.stats.total['deletions'], 45 | commit.stats.total['lines'], 46 | commit.stats.total['files'], 47 | ', '.join(list(commit.stats.files.keys())), 48 | commit.message]) 49 | inserter.execute() 50 | 51 | 52 | def main(path_to_repo: Path, branch: str): 53 | """ 54 | Extracting meta data of git repository into HYPER file. 55 | """ 56 | # set up Hyper Database 57 | with hapi.HyperProcess(telemetry=hapi.Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 58 | with hapi.Connection(endpoint=hyper.endpoint, database=__database_file, create_mode=hapi.CreateMode.CREATE_AND_REPLACE) as con: 59 | table = create_table(con) 60 | repo = Repo(path_to_repo) 61 | 62 | # Iterate over all commits 63 | for commit_idx, commit in enumerate(repo.iter_commits(branch)): 64 | print(f"Analyzing commit {commit_idx+1}/{len(list(repo.iter_commits(branch)))} ({commit.hexsha})") 65 | insert_commit_data(commit, table, con) 66 | 67 | # print statistics 68 | number_of_commits = con.execute_scalar_query(f"SELECT COUNT(*) FROM {table.table_name}") 69 | print(f"Analyzed {number_of_commits} commits") 70 | 71 | 72 | if __name__ == '__main__': 73 | parser = argparse.ArgumentParser() 74 | parser.add_argument("path_to_repo", help='Path to the repository, e.g. ~/src/repo') 75 | parser.add_argument("--branch", default='main', help='Branch to follow in the repository. Default: main') 76 | args = parser.parse_args() 77 | main(args.path_to_repo, args.branch) 78 | -------------------------------------------------------------------------------- /Community-Supported/git-to-hyper/ics.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Community-Supported/git-to-hyper/ics.png -------------------------------------------------------------------------------- /Community-Supported/git-to-hyper/requirements.txt: -------------------------------------------------------------------------------- 1 | GitPython 2 | tableauhyperapi -------------------------------------------------------------------------------- /Community-Supported/git-to-hyper/sloc.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Community-Supported/git-to-hyper/sloc.png -------------------------------------------------------------------------------- /Community-Supported/hyper-jupyter-kernel/hyper_kernel/__init__.py: -------------------------------------------------------------------------------- 1 | """A Hyper Jupyter kernel""" 2 | 3 | from .kernel import HyperKernel 4 | -------------------------------------------------------------------------------- /Community-Supported/hyper-jupyter-kernel/hyper_kernel/__main__.py: -------------------------------------------------------------------------------- 1 | from ipykernel.kernelapp import IPKernelApp 2 | from . import HyperKernel 3 | 4 | IPKernelApp.launch_instance(kernel_class=HyperKernel) 5 | -------------------------------------------------------------------------------- /Community-Supported/hyper-jupyter-kernel/hyper_kernel/data/__init__.py: -------------------------------------------------------------------------------- 1 | import os 2 | _data_root = os.path.abspath(os.path.dirname(__file__)) 3 | -------------------------------------------------------------------------------- /Community-Supported/hyper-jupyter-kernel/hyper_kernel/data/logo-128x128.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Community-Supported/hyper-jupyter-kernel/hyper_kernel/data/logo-128x128.png -------------------------------------------------------------------------------- /Community-Supported/hyper-jupyter-kernel/hyper_kernel/data/logo-32x32.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Community-Supported/hyper-jupyter-kernel/hyper_kernel/data/logo-32x32.png -------------------------------------------------------------------------------- /Community-Supported/hyper-jupyter-kernel/hyper_kernel/data/logo-64x64.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Community-Supported/hyper-jupyter-kernel/hyper_kernel/data/logo-64x64.png -------------------------------------------------------------------------------- /Community-Supported/hyper-jupyter-kernel/hyper_kernel/install.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | import json 3 | import os 4 | import sys 5 | from shutil import copyfile 6 | 7 | from jupyter_client.kernelspec import KernelSpecManager 8 | from IPython.utils.tempdir import TemporaryDirectory 9 | from hyper_kernel.data import _data_root 10 | 11 | 12 | kernel_json = { 13 | "argv": [sys.executable, "-m", "hyper_kernel", "-f", "{connection_file}"], 14 | "display_name": "Hyper SQL", 15 | "language": "sql", 16 | } 17 | 18 | 19 | def install_my_kernel_spec(user=True, prefix=None): 20 | with TemporaryDirectory() as td: 21 | os.chmod(td, 0o755) # Starts off as 700, not user readable 22 | with open(os.path.join(td, 'kernel.json'), 'w') as f: 23 | json.dump(kernel_json, f, sort_keys=True) 24 | copyfile(os.path.join(_data_root, 'logo-32x32.png'), os.path.join(td, 'logo-32x32.png')) 25 | copyfile(os.path.join(_data_root, 'logo-64x64.png'), os.path.join(td, 'logo-64x64.png')) 26 | 27 | print('Installing Jupyter kernel spec') 28 | KernelSpecManager().install_kernel_spec(td, 'hyper', user=user, prefix=prefix) 29 | 30 | 31 | def _is_root(): 32 | try: 33 | return os.geteuid() == 0 34 | except AttributeError: 35 | return False # assume not an admin on non-Unix platforms 36 | 37 | 38 | def main(argv=None): 39 | ap = argparse.ArgumentParser() 40 | ap.add_argument('--user', action='store_true', 41 | help="Install to the per-user kernels registry. Default if not root.") 42 | ap.add_argument('--sys-prefix', action='store_true', 43 | help="Install to sys.prefix (e.g. a virtualenv or conda env)") 44 | ap.add_argument('--prefix', 45 | help="Install to the given prefix. " 46 | "Kernelspec will be installed in {PREFIX}/share/jupyter/kernels/") 47 | args = ap.parse_args(argv) 48 | 49 | if args.sys_prefix: 50 | args.prefix = sys.prefix 51 | if not args.prefix and not _is_root(): 52 | args.user = True 53 | 54 | install_my_kernel_spec(user=args.user, prefix=args.prefix) 55 | 56 | 57 | if __name__ == '__main__': 58 | main() 59 | -------------------------------------------------------------------------------- /Community-Supported/hyper-jupyter-kernel/media/jupyter_console.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Community-Supported/hyper-jupyter-kernel/media/jupyter_console.png -------------------------------------------------------------------------------- /Community-Supported/hyper-jupyter-kernel/media/jupyterlab.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Community-Supported/hyper-jupyter-kernel/media/jupyterlab.png -------------------------------------------------------------------------------- /Community-Supported/hyper-jupyter-kernel/setup.py: -------------------------------------------------------------------------------- 1 | from distutils.core import setup 2 | from setuptools import find_packages 3 | 4 | 5 | with open('README.md') as f: 6 | readme = f.read() 7 | 8 | setup( 9 | name='hyper_kernel', 10 | version='0.0', 11 | packages=find_packages(), 12 | description='Hyper kernel for Jupyter', 13 | long_description=readme, 14 | author='Adrian Vogelsgesang', 15 | author_email='avogelsgesang@tableau.com', 16 | package_data={'hyper_kernel': ['data/*.png']}, 17 | install_requires=[ 18 | 'jupyter_client', 'IPython', 'ipykernel', 'tabulate' 19 | ], 20 | classifiers=[ 21 | 'Intended Audience :: Developers', 22 | 'Programming Language :: Python :: 3', 23 | ], 24 | ) 25 | -------------------------------------------------------------------------------- /Community-Supported/hyper-to-csv/README.md: -------------------------------------------------------------------------------- 1 | # hyper-to-csv 2 | ## Converts Hyper Databases to CSV 3 | 4 | ![Community Supported](https://img.shields.io/badge/Support%20Level-Community%20Supported-53bd92.svg) 5 | 6 | This simple Python script demonstrates how to transform a `.hyper` file into a CSV file. This example showcases how to fit .hyper files easily into ETL processes outside of the Tableau platform. 7 | 8 | 9 | # Get started 10 | 11 | ## __Prerequisites__ 12 | 13 | To run the script, you will need: 14 | 15 | - a computer running Windows, macOS, or Linux 16 | - Python >=3.6 17 | - `pip install requirements.txt` 18 | 19 | ## Run the sample 20 | 21 | Running hyper-to-csv.py will transform a given hyper file into a CSV. Simply change the values of `hyper_name`, `my_table`, and `output_name` to match your use case. Note that `my_table` is an object that can take both the table name, as well as the schema name. See more on this in the [TableName documentation](https://tableau.github.io/hyper-db/lang_docs/py/tableauhyperapi.html#tableauhyperapi.TableName). 22 | 23 | Instead of leveraging the `insert_data()` method, you will simply reference an existing `.hyper` file and use the `convert_to_csv()` method. The sample database created in the script is for demonstrative purposes only. 24 | 25 | ## __Notes__ 26 | 27 | This sample leverages the power of `pantab` to efficiently transform `.hyper` databases to dataframes. We recommend you check out the pantab docs [here](https://pantab.readthedocs.io/en/latest/index.html) to learn more about the wonderful library. 28 | 29 | As mentioned in pantab's [usage notes](https://pantab.readthedocs.io/en/latest/caveats.html), be sure to properly map the datatypes between Hyper's SQLTypes and pandas' dtypes. Importantly, the `NULLABILITY` noted in the doc must match in your `.hyper` file. 30 | 31 | 32 | ## __Resources__ 33 | Check out these resources to learn more: 34 | 35 | - [Hyper API docs](https://tableau.github.io/hyper-db) 36 | 37 | - [Tableau Hyper API Reference (Python)](https://tableau.github.io/hyper-db/lang_docs/py/index.html) 38 | 39 | - [Hyper API SQL Reference](https://tableau.github.io/hyper-db/docs/sql/) 40 | 41 | - [pantab](https://pantab.readthedocs.io/en/latest/index.html) 42 | 43 | - [pandas docs](https://pandas.pydata.org/docs/) 44 | -------------------------------------------------------------------------------- /Community-Supported/hyper-to-csv/hyper-to-csv.py: -------------------------------------------------------------------------------- 1 | # ----------------------------------------------------------------------------- 2 | # 3 | # This file is the copyrighted property of Tableau Software and is protected 4 | # by registered patents and other applicable U.S. and international laws and 5 | # regulations. 6 | # 7 | # You may adapt this file and modify it to fit into your context and use it 8 | # as a template to start your own projects. 9 | # 10 | # ----------------------------------------------------------------------------- 11 | from pathlib import Path 12 | import pandas as pd 13 | import pantab 14 | from tableauhyperapi import HyperProcess, Telemetry, \ 15 | Connection, CreateMode, \ 16 | NOT_NULLABLE, NULLABLE, SqlType, TableDefinition, \ 17 | Inserter, \ 18 | escape_name, escape_string_literal, \ 19 | TableName, \ 20 | HyperException 21 | 22 | # An example of how to turn a .hyper file into a csv to fit within potential ETL workflows. 23 | 24 | """ 25 | Note: you need to follow the pantab documentation to make sure columns line up with the 26 | appropriate datatypes. More here: 27 | https://pantab.readthedocs.io/en/latest/caveats.html 28 | 29 | """ 30 | 31 | # Change these to match your use case. 32 | hyper_file_path = "hyper_for_csv.hyper" 33 | table_name = TableName("Extract", "Extract") 34 | output_name = "output.csv" 35 | 36 | path_to_database = Path(hyper_file_path) 37 | 38 | 39 | # The table is called "Extract" and will be created in the "Extract" schema 40 | # and contains four columns. 41 | extract_table = TableDefinition( 42 | table_name=table_name, 43 | columns=[ 44 | TableDefinition.Column(name='Customer ID', type=SqlType.text(), nullability=NULLABLE), 45 | TableDefinition.Column(name='Customer Name', type=SqlType.text(), nullability=NULLABLE), 46 | TableDefinition.Column(name='Loyalty Reward Points', type=SqlType.big_int(), nullability=NOT_NULLABLE), 47 | TableDefinition.Column(name='Segment', type=SqlType.text(), nullability=NULLABLE) 48 | ] 49 | ) 50 | 51 | 52 | def insert_data(): 53 | """ 54 | Creates a simple .hyper file. For more on this, see the below example: 55 | https://github.com/tableau/hyper-api-samples/blob/main/Tableau-Supported/Python/insert_data_into_single_table.py 56 | """ 57 | print("Creating single table for conversion.") 58 | 59 | # Starts the Hyper Process with telemetry enabled to send data to Tableau. 60 | # To opt out, simply set telemetry=Telemetry.DO_NOT_SEND_USAGE_DATA_TO_TABLEAU. 61 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 62 | 63 | # Creates new Hyper file 64 | # Replaces file with CreateMode.CREATE_AND_REPLACE if it already exists. 65 | with Connection(endpoint=hyper.endpoint, 66 | database=path_to_database, 67 | create_mode=CreateMode.CREATE_AND_REPLACE) as connection: 68 | 69 | # Creates schema and table with table_definition. 70 | connection.catalog.create_schema(schema=extract_table.table_name.schema_name) 71 | connection.catalog.create_table(table_definition=extract_table) 72 | 73 | # The rows to insert into the "Extract"."Extract" table. 74 | data_to_insert = [ 75 | ["DK-13375", "Dennis Kane", 685, "Consumer"], 76 | ["EB-13705", "Ed Braxton", 815, "Corporate"] 77 | ] 78 | 79 | # Insert the data. 80 | with Inserter(connection, extract_table) as inserter: 81 | inserter.add_rows(rows=data_to_insert) 82 | inserter.execute() 83 | 84 | print("The connection to the Hyper file has been closed.") 85 | print("The Hyper process has been shut down.") 86 | 87 | 88 | def convert_to_csv(): 89 | """ 90 | Leverages pantab and pandas to convert a .hyper file to a df, and then convert 91 | the df to a csv file. 92 | """ 93 | 94 | # Uses pantab to convert the hyper file to a df. 95 | df = pantab.frame_from_hyper(hyper_file_path, table=table_name) 96 | print("Converting to CSV...") 97 | 98 | # Simple pandas->csv operation. 99 | df.to_csv(output_name) 100 | 101 | 102 | # Run it! 103 | if __name__ == '__main__': 104 | insert_data() 105 | convert_to_csv() 106 | print("All done!") 107 | -------------------------------------------------------------------------------- /Community-Supported/hyper-to-csv/requirements.txt: -------------------------------------------------------------------------------- 1 | tableauhyperapi==0.0.10899 2 | tableauserverclient==0.10 3 | pantab==1.1.0 4 | pandas>=0.25.1 -------------------------------------------------------------------------------- /Community-Supported/list-hyper-contents/README.md: -------------------------------------------------------------------------------- 1 | # list-hyper-contents 2 | 3 | ![Community Supported](https://img.shields.io/badge/Support%20Level-Community%20Supported-53bd92.svg) 4 | 5 | This little utility script lists the schemas, tables and columns inside a Hyper file, thereby providing you a first quick peak into a Hyper file before actually opening it, e.g., with Tableau. 6 | 7 | This sample can serve as a starting point for anyone who wants to, e.g., programmatically enumerate all data contained inside a `.hyper` file. 8 | 9 | But already on its own, it can be a pretty useful command line utility if you just need to know the table names/column names to write your own SQL queries. 10 | 11 | # Get started 12 | 13 | ## __Prerequisites__ 14 | 15 | To run the script, you will need: 16 | 17 | - Windows, Mac, or supported Linux distro 18 | 19 | - Python 3.6 or 3.7 20 | 21 | - Install tableauhyperapi: `pip install tableauhyperapi` 22 | 23 | ## How to use it 24 | 25 | Simply call it from the console, passing the file path of your Hyper file as the first (and only) parameter like this 26 | 27 | ``` 28 | python list_hyper_contents.py my_data.hyper 29 | ``` 30 | 31 | ## Exemplary usage 32 | 33 | Running 34 | 35 | ``` 36 | python list_hyper_contents.py "World Indicators.hyper" 37 | ``` 38 | 39 | where `World Indicators.hyper` is the Hyper file from the "World Indicators" demo workbook shipped with every Tableau Desktop version, we get 40 | 41 | ``` 42 | 2 schemas: 43 | * Schema "Extract": 1 tables 44 | -> Table "Extract": 26 columns 45 | -> "Birth Rate" DOUBLE 46 | -> "Business Tax Rate" DOUBLE 47 | -> "CO2 Emissions" BIG_INT 48 | -> "Country/Region" TEXT en_US 49 | -> "Days to Start Business" BIG_INT 50 | -> "Ease of Business" DOUBLE 51 | -> "Energy Usage" BIG_INT 52 | -> "GDP" DOUBLE 53 | -> "Health Exp % GDP" DOUBLE 54 | -> "Health Exp/Capita" BIG_INT 55 | -> "Hours to do Tax" DOUBLE 56 | -> "Infant Mortality Rate" DOUBLE 57 | -> "Internet Usage" DOUBLE 58 | -> "Lending Interest" DOUBLE 59 | -> "Life Expectancy Female" BIG_INT 60 | -> "Life Expectancy Male" BIG_INT 61 | -> "Mobile Phone Usage" DOUBLE 62 | -> "Population 0-14" DOUBLE 63 | -> "Population 15-64" DOUBLE 64 | -> "Population 65+" DOUBLE 65 | -> "Population Total" BIG_INT 66 | -> "Population Urban" DOUBLE 67 | -> "Region" TEXT en_US 68 | -> "Tourism Inbound" DOUBLE 69 | -> "Tourism Outbound" BIG_INT 70 | -> "Year" DATE 71 | * Schema "public": 0 tables 72 | ``` 73 | -------------------------------------------------------------------------------- /Community-Supported/list-hyper-contents/list_hyper_contents.py: -------------------------------------------------------------------------------- 1 | # ----------------------------------------------------------------------------- 2 | # 3 | # This file is the copyrighted property of Tableau Software and is protected 4 | # by registered patents and other applicable U.S. and international laws and 5 | # regulations. 6 | # 7 | # You may adapt this file and modify it to fit into your context and use it 8 | # as a template to start your own projects. 9 | # 10 | # ----------------------------------------------------------------------------- 11 | import argparse 12 | from pathlib import PurePath 13 | from tableauhyperapi import HyperProcess, Telemetry, Connection, CreateMode, Nullability 14 | 15 | # Lists the schemas, tables, and columns inside a Hyper file 16 | 17 | if __name__ == "__main__": 18 | argparser = argparse.ArgumentParser(description="List the schemas, tables, and columns inside a Hyper file") 19 | argparser.add_argument("file", type=PurePath, help="The input Hyper file") 20 | args = argparser.parse_args() 21 | 22 | # Start Hyper and connect to our Hyper file 23 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 24 | with Connection(hyper.endpoint, args.file, CreateMode.NONE) as connection: 25 | # The `connection.catalog` provides us with access to the meta-data we are interested in 26 | catalog = connection.catalog 27 | 28 | # Iterate over all schemas and print them 29 | schemas = catalog.get_schema_names() 30 | print(f"{len(schemas)} schemas:") 31 | for schema_name in schemas: 32 | # For each schema, iterate over all tables and print them 33 | tables = catalog.get_table_names(schema=schema_name) 34 | print(f" * Schema {schema_name}: {len(tables)} tables") 35 | for table in tables: 36 | # For each table, iterate over all columns and print them 37 | table_definition = catalog.get_table_definition(name=table) 38 | print(f" -> Table {table.name}: {len(table_definition.columns)} columns") 39 | for column in table_definition.columns: 40 | nullability = " NOT NULL" if column.nullability == Nullability.NOT_NULLABLE else "" 41 | collation = " " + column.collation if column.collation is not None else "" 42 | print(f" -> {column.name} {column.type}{nullability}{collation}") 43 | -------------------------------------------------------------------------------- /Community-Supported/native-s3/README.md: -------------------------------------------------------------------------------- 1 | # parquet-to-hyper 2 | ## __parquet_to_hyper__ 3 | 4 | ![Community Supported](https://img.shields.io/badge/Support%20Level-Community%20Supported-53bd92.svg) 5 | 6 | __Current Version__: 1.0 7 | 8 | These samples show you how Hyper can natively interact with Amazon S3, without the need to install any external dependencies like boto or aws-cli. 9 | They originate from the Tableau Conference 2022 Hands-on Training Use Hyper as your Cloud Lake Engine - you can [check out the slides here](https://mkt.tableau.com/tc22/sessions/live/428-HOT-D1_Hands-onUseTheHyperAPI.pdf). 10 | 11 | # Get started 12 | 13 | ## __Prerequisites__ 14 | 15 | To run the script, you will need: 16 | 17 | - a computer running Windows, macOS, or Linux 18 | 19 | - Python 3.9+ 20 | 21 | - install the dependencies from the `requirements.txt` file 22 | 23 | ## Run the samples 24 | 25 | The following instructions assume that you have set up a virtual environment for Python. For more information on 26 | creating virtual environments, see [venv - Creation of virtual environments](https://docs.python.org/3/library/venv.html) 27 | in the Python Standard Library. 28 | 29 | 1. Open a terminal and activate the Python virtual environment (`venv`). 30 | 31 | 1. Navigate to the folder where you installed the samples. 32 | 33 | 1. Then follow the steps to run one of the samples which are shown below. 34 | 35 | **Create a `.hyper` file from parquet file on S3** 36 | Run the Python script 37 | ```bash 38 | $ python parquet-on-s3-to-hyper.py 39 | ``` 40 | 41 | This script will read the parquet file from `s3://nyc-tlc/trip%20data/yellow_tripdata_2021-06.parquet`, visit [AWS OpenData](https://registry.opendata.aws/nyc-tlc-trip-records-pds/) for more details and license about the dataset and insert the records into a table named `taxi_rides` which is stored in a `.hyper` database file. 42 | 43 | This database file can then directly be opened with Tableau Desktop or Tableau Prep or it can be published to Tableau Online and Tableau Server as shown in [this example](https://github.com/tableau/hyper-api-samples/tree/main/Community-Supported/publish-hyper). 44 | 45 | **Live query against a `.csv` file which is stored on AWS S3** 46 | Run the Python script 47 | 48 | ```bash 49 | $ python query-csv-on-s3.py 50 | ``` 51 | 52 | This script will perform a live query on the CSV file which is stored in this public S3 bucket: `s3://hyper-dev-us-west-2-bucket/tc22-demo/orders_small.csv`. 53 | 54 | **Live query with multiple `.parquet` and `.csv` files which are stored on AWS S3** 55 | Run the Python script 56 | 57 | ```bash 58 | $ python https://tableau.github.io/hyper-db/lang_docs/pyhttps://tableau.github.io/hyper-db/lang_docs/py 59 | ``` 60 | 61 | This script will perform a live query on multiple `.parquet` files which are stored on AWS S3. It shows how to use the [`ARRAY` syntax](https://tableau.github.io/hyper-db/docs/sql/external/) to union multiple `.parquet` files and how `.parquet` files can be joined together with `.csv` files - as you would expect from normal database tables stored inside a `.hyper` file. 62 | 63 | ## __Resources__ 64 | Check out these resources to learn more: 65 | 66 | - [Hyper API docs](https://tableau.github.io/hyper-db) 67 | 68 | - [Tableau Hyper API Reference (Python)](https://tableau.github.io/hyper-db/lang_docs/py/index.html) 69 | 70 | - [The EXTERNAL function in the Hyper API SQL Reference](https://tableau.github.io/hyper-db/docs/sql/external/) 71 | 72 | - [AWS command line tools documentation](https://docs.aws.amazon.com/cli/latest/reference/s3/cp.html), e.g. if you want to download some of the sample files to your local machine and explore them 73 | -------------------------------------------------------------------------------- /Community-Supported/native-s3/join-parquet-and-csv-on-s3.py: -------------------------------------------------------------------------------- 1 | from tableauhyperapi import HyperProcess, Connection, Telemetry, CreateMode, SqlType, TableDefinition, TableName, Nullability, Inserter, escape_string_literal 2 | 3 | ORDERS_DATASET_2018 = escape_string_literal("s3://hyper-dev-us-west-2-bucket/tc22-demo/orders_2018.parquet") 4 | ORDERS_DATASET_2019 = escape_string_literal("s3://hyper-dev-us-west-2-bucket/tc22-demo/orders_2019.parquet") 5 | ORDERS_DATASET_2020 = escape_string_literal("s3://hyper-dev-us-west-2-bucket/tc22-demo/orders_2020.parquet") 6 | ORDERS_DATASET_2021 = escape_string_literal("s3://hyper-dev-us-west-2-bucket/tc22-demo/orders_2021.parquet") 7 | 8 | # CSV file which contains the orders that were returned by the customers 9 | RETURNS_DATASET = escape_string_literal("s3://hyper-dev-us-west-2-bucket/tc22-demo/returns.csv") 10 | 11 | EMPTY_S3_CREDENTIALS = "ACCESS_KEY_ID => '', SECRET_ACCESS_KEY => ''" 12 | 13 | # We need to manually enable S3 connectivity as this is still an experimental feature 14 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 15 | # Create a connection to the Hyper process - we do not connect to a database 16 | with Connection(endpoint=hyper.endpoint) as connection: 17 | 18 | # We use the `ARRAY` syntax in the CREATE TEMP EXTERNAL TABLE statement to specify multiple files to be unioned 19 | create_ext_orders_table = f""" 20 | CREATE TEMP EXTERNAL TABLE orders 21 | FOR ARRAY[ S3_LOCATION({ORDERS_DATASET_2018}, {EMPTY_S3_CREDENTIALS}, REGION => 'us-west-2'), 22 | S3_LOCATION({ORDERS_DATASET_2019}, {EMPTY_S3_CREDENTIALS}, REGION => 'us-west-2'), 23 | S3_LOCATION({ORDERS_DATASET_2020}, {EMPTY_S3_CREDENTIALS}, REGION => 'us-west-2'), 24 | S3_LOCATION({ORDERS_DATASET_2021}, {EMPTY_S3_CREDENTIALS}, REGION => 'us-west-2')] 25 | WITH (FORMAT => 'parquet') 26 | """ 27 | connection.execute_command(create_ext_orders_table) 28 | 29 | # Create the `returns` table also as EXTERNAL TABLE 30 | create_ext_returns_table = f""" 31 | CREATE TEMP EXTERNAL TABLE returns( 32 | returned TEXT, 33 | order_id TEXT 34 | ) 35 | FOR S3_LOCATION({RETURNS_DATASET}, {EMPTY_S3_CREDENTIALS}, REGION => 'us-west-2') 36 | WITH (FORMAT => 'csv', HEADER => 'true', DELIMITER => ';') 37 | """ 38 | connection.execute_command(create_ext_returns_table) 39 | 40 | # Select the total sales amount per category from the CSV file 41 | # and drill down by whether the orders were returned or not 42 | query = f"""SELECT category, 43 | (CASE WHEN returned IS NULL THEN 'Not Returned' ELSE 'Returned' END) AS return_info, 44 | SUM(sales) 45 | FROM orders 46 | LEFT OUTER JOIN returns on orders.order_id = returns.order_id 47 | GROUP BY 1, 2 48 | ORDER BY 1, 2""" 49 | 50 | # Execute the query with `execute_list_query` 51 | result = connection.execute_list_query(query) 52 | 53 | # Iterate over all rows in the result and print them 54 | print(f"{'Category':<20} {'Status':<20} Sales") 55 | print(f"{'--------':<20} {'------':<20} -----") 56 | for row in result: 57 | print(f"{row[0]:<20} {row[1]:<20} {row[2]:,.2f} USD") 58 | -------------------------------------------------------------------------------- /Community-Supported/native-s3/parquet-on-s3-to-hyper.py: -------------------------------------------------------------------------------- 1 | from tableauhyperapi import HyperProcess, Connection, Telemetry, CreateMode, SqlType, TableDefinition, TableName, Nullability, Inserter, escape_string_literal 2 | 3 | # Details and license of dataset: https://registry.opendata.aws/nyc-tlc-trip-records-pds/ 4 | # NOTE: This dataset is currently not accessible - see above website for more details and to check if it has become available again 5 | TAXI_DATASET = escape_string_literal("s3://nyc-tlc/trip%20data/yellow_tripdata_2021-06.parquet") # May release fixes a bug so that %20 doesn't need to be escaped manually 6 | TAXI_DATASET_TABLE_NAME = "taxi_rides" 7 | TAXI_DATASET_DBNAME = "taxi-rides-2021-06.hyper" 8 | TAXI_DATASET_REGION = "us-east-1" 9 | 10 | # Currently (last checked Aug 8, 2022) the NYC taxi dataset is not available on AWS OpenData, however access may get restored in the future 11 | # Therefore, we're providing an alternative using our own orders data set in parquet format 12 | ORDERS_DATASET = escape_string_literal("s3://hyper-dev-us-west-2-bucket/tc22-demo/orders_2018.parquet") 13 | ORDERS_DATASET_TABLE_NAME = "orders" 14 | ORDERS_DATASET_DBNAME = "orders-2018.hyper" 15 | ORDERS_DATASET_REGION = "us-west-2" 16 | 17 | # If AWS has restored access to the NYC taxi dataset, below config can be changed to reference the TAXI_DATASET when it becomes available again in the future 18 | CURRENT_DATASET = ORDERS_DATASET 19 | CURRENT_DATASET_TABLE_NAME = ORDERS_DATASET_TABLE_NAME 20 | CURRENT_DATASET_DBNAME = ORDERS_DATASET_DBNAME 21 | CURRENT_DATASET_REGION = ORDERS_DATASET_REGION 22 | 23 | # We need to manually enable S3 connectivity as this is still an experimental feature 24 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 25 | # Create a connection to the Hyper process and let it create a database file - if it exists, it's overwritten 26 | with Connection(endpoint=hyper.endpoint, database=CURRENT_DATASET_DBNAME, create_mode=CreateMode.CREATE_AND_REPLACE) as connection: 27 | 28 | # Use `TableName` so we do not have to worry about escaping in the SQL query we generate below 29 | # Note: This line does not create a table in Hyper, it just defines a name 30 | table_name = TableName("public", CURRENT_DATASET_TABLE_NAME) 31 | 32 | # Ingest the data from the parquet file into a Hyper Table 33 | # Since the schema is stored inside the parquet file, we don't need to specify it explicitly here 34 | cmd = f"CREATE TABLE {table_name}" \ 35 | f" AS ( SELECT * FROM EXTERNAL(S3_LOCATION({CURRENT_DATASET}, ACCESS_KEY_ID => '', SECRET_ACCESS_KEY => '', REGION => '{CURRENT_DATASET_REGION}')," \ 36 | f" FORMAT => 'parquet'))" 37 | 38 | # We use `execute_command` to send the CREATE TABLE statement to Hyper 39 | # This may take some time depending on your network connectivity so AWS S3 40 | connection.execute_command(cmd) 41 | 42 | # Let's check how many rows we loaded 43 | row_count = connection.execute_scalar_query(f"SELECT COUNT(*) FROM {table_name}") 44 | print (f"Loaded {row_count} rows") 45 | -------------------------------------------------------------------------------- /Community-Supported/native-s3/query-csv-on-s3.py: -------------------------------------------------------------------------------- 1 | from tableauhyperapi import HyperProcess, Connection, Telemetry, CreateMode, SqlType, TableDefinition, TableName, Nullability, Inserter, escape_string_literal 2 | 3 | ORDERS_DATASET_S3 = escape_string_literal("s3://hyper-dev-us-west-2-bucket/tc22-demo/orders_small.csv") 4 | 5 | # We need to manually enable S3 connectivity as this is still an experimental feature 6 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 7 | # Create a connection to the Hyper process - we do not connect to a database 8 | with Connection(endpoint=hyper.endpoint) as connection: 9 | 10 | # Use the CREATE TEMP EXTERNAL TABLE syntax - this allows us to use the CSV file like a normal table name in SQL queries 11 | # We specify empty credentials as the bucket is publicy accessible; this may be different when used with your own data 12 | create_external_table = f""" 13 | CREATE TEMP EXTERNAL TABLE orders( 14 | order_date DATE, 15 | product_id TEXT, 16 | category TEXT, 17 | sales DOUBLE PRECISION 18 | ) 19 | FOR S3_LOCATION({ORDERS_DATASET_S3}, 20 | ACCESS_KEY_ID => '', 21 | SECRET_ACCESS_KEY => '', 22 | REGION => 'us-west-2') 23 | WITH (FORMAT => 'csv', HEADER => true) 24 | """ 25 | # Create the external table using `execute_command` which sends an instruction to the database - we don't expect a result value 26 | connection.execute_command(create_external_table) 27 | 28 | # Select the total sales amount per category from the external table 29 | query = f"""SELECT category, SUM(sales) 30 | FROM orders 31 | GROUP BY category""" 32 | 33 | # Execute the query with `execute_list_query` as we expect multiple rows (one row per category) and two columns (category name and sum of sales) 34 | result = connection.execute_list_query(query) 35 | 36 | # Iterate over all rows in the result and print the category name and the sum of sales for that category 37 | for row in result: 38 | print(f"{row[0]}: {row[1]} USD") 39 | -------------------------------------------------------------------------------- /Community-Supported/native-s3/requirements.txt: -------------------------------------------------------------------------------- 1 | tableauhyperapi>=0.0.14946 -------------------------------------------------------------------------------- /Community-Supported/pandas-to-hyper/README.md: -------------------------------------------------------------------------------- 1 | 2 | # hyper-from-dataframe 3 | ## Create a Hyper File from a Pandas DataFrame 4 | 5 | ![Community Supported](https://img.shields.io/badge/Support%20Level-Community%20Supported-53bd92.svg) 6 | 7 | This Python script demonstrates how to create a `.hyper` file (Tableau's Hyper database format) from a pandas DataFrame. It uses Tableau's Hyper API to define a table structure, insert the data from the DataFrame, and save it as a `.hyper` file. 8 | 9 | This is example shows an alternative to [using pantab](https://tableau.github.io/hyper-db/docs/guides/pandas_integration#loading-data-through-pandas), in case pantab cannot be used. 10 | ## Get Started 11 | 12 | ### Prerequisites 13 | 14 | Before running the script, ensure you have the following installed: 15 | 16 | - Python >= 3.6 17 | - The required dependencies listed in `requirements.txt`. 18 | 19 | ### Install Dependencies 20 | 21 | To install the necessary dependencies, run the following command: 22 | 23 | ```bash 24 | pip install -r requirements.txt 25 | ``` 26 | 27 | ### Running the Script 28 | 29 | To run the script and generate the `.hyper` file, execute: 30 | 31 | ```bash 32 | python create_hyper_from_pandas_dataframe.py 33 | ``` 34 | 35 | ### What the Script Does 36 | 37 | 1. Creates a pandas DataFrame containing sample customer data. 38 | 2. Defines a table schema for the Hyper file, including columns like Customer ID, Customer Name, Loyalty Points, and Segment. 39 | 3. Inserts the DataFrame data into the Hyper file `customer.hyper`. 40 | 4. Verifies the number of rows inserted and prints a confirmation message. 41 | 42 | ### Modifying the Script 43 | 44 | You can easily modify the script to load your own data by: 45 | 46 | 1. Changing the data inside the `data` dictionary to match your own structure. 47 | 2. Adjusting the table schema in the `TableDefinition` object accordingly to reflect your columns. 48 | 49 | ### Example Output 50 | 51 | When you run the script, you should see output similar to this: 52 | 53 | ``` 54 | EXAMPLE - Load data from pandas DataFrame into table in new Hyper file 55 | The number of rows in table Customer is 3. 56 | The connection to the Hyper file has been closed. 57 | The Hyper process has been shut down. 58 | ``` 59 | 60 | ### Error Handling 61 | 62 | If any issues occur, such as problems connecting to the Hyper file or inserting data, the script will raise an exception and print an error message to the console. 63 | 64 | ## Notes 65 | 66 | This sample script demonstrates: 67 | 68 | - How to use Tableau's `HyperProcess` and `Connection` classes. 69 | - Defining table schemas using `TableDefinition`. 70 | - Inserting data into the Hyper table using the `Inserter` class. 71 | 72 | ### Resources 73 | 74 | - [Tableau Hyper API Documentation](https://tableau.github.io/hyper-db/lang_docs/py/index.html) 75 | - [Tableau Hyper API SQL Reference](https://tableau.github.io/hyper-db/docs/sql/) 76 | - [pandas Documentation](https://pandas.pydata.org/docs/) 77 | 78 | -------------------------------------------------------------------------------- /Community-Supported/pandas-to-hyper/create_hyper_from_pandas_dataframe.py: -------------------------------------------------------------------------------- 1 | # Import necessary standard libraries 2 | from pathlib import Path # For file path manipulations 3 | 4 | # Import pandas for DataFrame creation and manipulation 5 | import pandas as pd 6 | 7 | # Import necessary classes from the Tableau Hyper API 8 | from tableauhyperapi import ( 9 | HyperProcess, 10 | Telemetry, 11 | Connection, 12 | CreateMode, 13 | NOT_NULLABLE, 14 | NULLABLE, 15 | SqlType, 16 | TableDefinition, 17 | Inserter, 18 | HyperException, 19 | ) 20 | 21 | def run_create_hyper_file_from_dataframe(): 22 | """ 23 | An example demonstrating loading data from a pandas DataFrame into a new Hyper file. 24 | """ 25 | 26 | print("EXAMPLE - Load data from pandas DataFrame into table in new Hyper file") 27 | 28 | # Step 1: Create a sample pandas DataFrame. 29 | data = { 30 | "Customer ID": ["DK-13375", "EB-13705", "JH-13600"], 31 | "Customer Name": ["John Doe", "Jane Smith", "Alice Johnson"], 32 | "Loyalty Reward Points": [100, 200, 300], 33 | "Segment": ["Consumer", "Corporate", "Home Office"], 34 | } 35 | df = pd.DataFrame(data) 36 | 37 | # Step 2: Define the path where the Hyper file will be saved. 38 | path_to_database = Path("customer.hyper") 39 | 40 | # Step 3: Optional process parameters. 41 | # These settings limit the number of log files and their size. 42 | process_parameters = { 43 | "log_file_max_count": "2", # Limit the number of log files to 2 44 | "log_file_size_limit": "100M", # Limit the log file size to 100 megabytes 45 | } 46 | 47 | # Step 4: Start the Hyper Process. 48 | # Telemetry is set to send usage data to Tableau. 49 | with HyperProcess( 50 | telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU, parameters=process_parameters 51 | ) as hyper: 52 | 53 | # Step 5: Optional connection parameters. 54 | # This sets the locale for time formats to 'en_US'. 55 | connection_parameters = {"lc_time": "en_US"} 56 | 57 | # Step 6: Create a connection to the Hyper file. 58 | # If the file exists, it will be replaced. 59 | with Connection( 60 | endpoint=hyper.endpoint, 61 | database=path_to_database, 62 | create_mode=CreateMode.CREATE_AND_REPLACE, 63 | parameters=connection_parameters, 64 | ) as connection: 65 | 66 | # Step 7: Define the table schema. 67 | customer_table = TableDefinition( 68 | table_name="Customer", # Name of the table 69 | columns=[ 70 | TableDefinition.Column( 71 | "Customer ID", SqlType.text(), NOT_NULLABLE 72 | ), 73 | TableDefinition.Column( 74 | "Customer Name", SqlType.text(), NOT_NULLABLE 75 | ), 76 | TableDefinition.Column( 77 | "Loyalty Reward Points", SqlType.big_int(), NOT_NULLABLE 78 | ), 79 | TableDefinition.Column("Segment", SqlType.text(), NOT_NULLABLE), 80 | ], 81 | ) 82 | 83 | # Step 8: Create the table in the Hyper file. 84 | connection.catalog.create_table(table_definition=customer_table) 85 | 86 | # Step 9: Use the Inserter to insert data into the table. 87 | with Inserter(connection, customer_table) as inserter: 88 | # Iterate over the DataFrame rows as tuples. 89 | # 'itertuples' returns an iterator yielding named tuples. 90 | for row in df.itertuples(index=False, name=None): 91 | inserter.add_row(row) # Add each row to the inserter 92 | inserter.execute() # Execute the insertion into the Hyper file 93 | 94 | # Step 10: Verify the number of rows inserted. 95 | row_count = connection.execute_scalar_query( 96 | f"SELECT COUNT(*) FROM {customer_table.table_name}" 97 | ) 98 | print( 99 | f"The number of rows in table {customer_table.table_name} is {row_count}." 100 | ) 101 | print("Data has been successfully inserted into the Hyper file.") 102 | 103 | # The connection is automatically closed when exiting the 'with' block. 104 | print("The connection to the Hyper file has been closed.") 105 | 106 | # The Hyper process is automatically shut down when exiting the 'with' block. 107 | print("The Hyper process has been shut down.") 108 | 109 | 110 | if __name__ == "__main__": 111 | try: 112 | run_create_hyper_file_from_dataframe() 113 | except HyperException as ex: 114 | print(ex) 115 | exit(1) 116 | -------------------------------------------------------------------------------- /Community-Supported/pandas-to-hyper/requirements.txt: -------------------------------------------------------------------------------- 1 | tableauhyperapi 2 | pandas -------------------------------------------------------------------------------- /Community-Supported/parquet-to-hyper/README.md: -------------------------------------------------------------------------------- 1 | # parquet-to-hyper 2 | ## __parquet_to_hyper__ 3 | 4 | ![Community Supported](https://img.shields.io/badge/Support%20Level-Community%20Supported-53bd92.svg) 5 | 6 | __Current Version__: 1.0 7 | 8 | This sample demonstrates how you can create a `.hyper` file from an Apache Parquet file. 9 | 10 | # Get started 11 | 12 | ## __Prerequisites__ 13 | 14 | To run the script, you will need: 15 | 16 | - a computer running Windows, macOS, or Linux 17 | 18 | - Python 3.6 or 3.7 19 | 20 | ## Run the sample 21 | 22 | Ensure that you have installed the requirements and then just run the sample Python file. 23 | The following instructions assume that you have set up a virtual environment for Python. For more information on 24 | creating virtual environments, see [venv - Creation of virtual environments](https://docs.python.org/3/library/venv.html) 25 | in the Python Standard Library. 26 | 27 | 1. Open a terminal and activate the Python virtual environment (`venv`). 28 | 29 | 1. Navigate to the folder where you installed the sample. 30 | 31 | 1. Run the Python script: 32 | 33 | **python create_hyper_file_from_parquet.py** 34 | 35 | It will read the `orders_10rows.parquet` file from the working directory and create a new Hyper database 36 | named `orders.hyper` with a table named "orders", which will contain the 10 rows copied from the Parquet file. 37 | 38 | ## __Resources__ 39 | Check out these resources to learn more: 40 | 41 | - [Hyper API docs](https://tableau.github.io/hyper-db) 42 | 43 | - [Tableau Hyper API Reference (Python)](https://tableau.github.io/hyper-db/lang_docs/py/index.html) 44 | 45 | - [The COPY command in the Hyper API SQL Reference](https://tableau.github.io/hyper-db/docs/sql/command/copy_from) 46 | -------------------------------------------------------------------------------- /Community-Supported/parquet-to-hyper/create_hyper_file_from_parquet.py: -------------------------------------------------------------------------------- 1 | # ----------------------------------------------------------------------------- 2 | # 3 | # This file is the copyrighted property of Tableau Software and is protected 4 | # by registered patents and other applicable U.S. and international laws and 5 | # regulations. 6 | # 7 | # You may adapt this file and modify it to fit into your context and use it 8 | # as a template to start your own projects. 9 | # 10 | # ----------------------------------------------------------------------------- 11 | from datetime import date 12 | from pathlib import Path 13 | 14 | from tableauhyperapi import HyperProcess, Telemetry, \ 15 | Connection, CreateMode, \ 16 | NOT_NULLABLE, NULLABLE, SqlType, TableDefinition, \ 17 | Inserter, \ 18 | escape_name, escape_string_literal, \ 19 | HyperException 20 | 21 | 22 | def run_create_hyper_file_from_parquet( 23 | parquet_file_path: Path, 24 | table_definition: TableDefinition, 25 | hyper_database_path: Path): 26 | """ 27 | An example demonstrating how to load rows from an Apache Parquet file (`parquet_file_path`) 28 | into a new Hyper file (`hyper_database_path`) using the COPY command. Currently the 29 | table definition of the data to copy needs to be known and explicitly specified. 30 | 31 | Reading Parquet data is analogous to reading CSV data. For more details, see: 32 | https://tableau.github.io/hyper-db/docs/guides/hyper_file/insert_csv 33 | """ 34 | 35 | # Start the Hyper process. 36 | # 37 | # * Sending telemetry data to Tableau is encouraged when trying out an experimental feature. 38 | # To opt out, simply set `telemetry=Telemetry.DO_NOT_SEND_USAGE_DATA_TO_TABLEAU` below. 39 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 40 | 41 | # Open a connection to the Hyper process. This will also create the new Hyper file. 42 | # The `CREATE_AND_REPLACE` mode causes the file to be replaced if it 43 | # already exists. 44 | with Connection(endpoint=hyper.endpoint, 45 | database=hyper_database_path, 46 | create_mode=CreateMode.CREATE_AND_REPLACE) as connection: 47 | 48 | # Create the target table. 49 | connection.catalog.create_table(table_definition=table_definition) 50 | 51 | # Execute a COPY command to insert the data from the Parquet file. 52 | copy_command = f"COPY {table_definition.table_name} FROM {escape_string_literal(parquet_file_path)} WITH (FORMAT PARQUET)" 53 | print(copy_command) 54 | count_inserted = connection.execute_command(copy_command) 55 | print(f"-- {count_inserted} rows have been copied from '{parquet_file_path}' to the table {table_definition.table_name} in '{hyper_database_path}'.") 56 | 57 | 58 | if __name__ == '__main__': 59 | try: 60 | # The `orders` table to read from the Parquet file. 61 | table_definition = TableDefinition( 62 | table_name="orders", 63 | columns=[ 64 | TableDefinition.Column("o_orderkey", SqlType.int(), NOT_NULLABLE), 65 | TableDefinition.Column("o_custkey", SqlType.int(), NOT_NULLABLE), 66 | TableDefinition.Column("o_orderstatus", SqlType.text(), NOT_NULLABLE), 67 | TableDefinition.Column("o_totalprice", SqlType.numeric(8, 2), NOT_NULLABLE), 68 | TableDefinition.Column("o_orderdate", SqlType.date(), NOT_NULLABLE), 69 | TableDefinition.Column("o_orderpriority", SqlType.text(), NOT_NULLABLE), 70 | TableDefinition.Column("o_clerk", SqlType.text(), NOT_NULLABLE), 71 | TableDefinition.Column("o_shippriority", SqlType.int(), NOT_NULLABLE), 72 | TableDefinition.Column("o_comment", SqlType.text(), NOT_NULLABLE), 73 | ] 74 | ) 75 | 76 | run_create_hyper_file_from_parquet( 77 | "orders_10rows.parquet", 78 | table_definition, 79 | "orders.hyper") 80 | 81 | except HyperException as ex: 82 | print(ex) 83 | exit(1) 84 | -------------------------------------------------------------------------------- /Community-Supported/parquet-to-hyper/orders_10rows.parquet: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Community-Supported/parquet-to-hyper/orders_10rows.parquet -------------------------------------------------------------------------------- /Community-Supported/publish-hyper/README.md: -------------------------------------------------------------------------------- 1 | # publish-hyper 2 | ## __Publishing a Single-Table Hyper File Directly to Tableau Online/Server__ 3 | 4 | ![Community Supported](https://img.shields.io/badge/Support%20Level-Community%20Supported-53bd92.svg) 5 | 6 | This sample demonstrates how to leverage the Hyper API and Tableau Server Client Library to do the following: 7 | - Create a single-table `.hyper` file 8 | - Publish the file as a datasource directly to Tableau Server or Tableau Online, without a .tdsx file 9 | 10 | It should serve as a starting point for anyone looking to automate the publishing process of (single-table) hyper files to Tableau Server or Online. 11 | 12 | ## __Prerequisites__ 13 | To run the script, you will need: 14 | - Windows, Mac, or supported Linux distro 15 | - Python 3.6 - 3.7 16 | - Run `pip install -r requirements.txt` 17 | - Tableau Online/Server credentials or Personal Access Token 18 | 19 | ## __How to Use__ 20 | Edit the following: 21 | - Tableau Online or Server address, site, and project 22 | - Tableau Online or Server authentication credentials 23 | - Name of `.hyper` file 24 | - TableDefinition (columns and SQLTypes) 25 | 26 | Next, you'll need to determine how to insert the data into the `.hyper` file. This will vary depending on the shape of the data and how it is stored. Please see our other samples for more on best practices with the Hyper API. Make those changes in the `insert_data()` function. 27 | -------------------------------------------------------------------------------- /Community-Supported/publish-hyper/publish-hyper-file.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import tableauserverclient as TSC 3 | from tableauhyperapi import HyperProcess, Telemetry, \ 4 | Connection, CreateMode, \ 5 | NOT_NULLABLE, NULLABLE, SqlType, TableDefinition, \ 6 | Inserter, \ 7 | escape_name, escape_string_literal, \ 8 | TableName, \ 9 | HyperException 10 | 11 | 12 | # Configure for TSC to publish 13 | # Note: Do not store creds/tokens in plaintext, please use env vars :) 14 | hyper_name = 'customer.hyper' 15 | server_address = 'https://10ax.online.tableau.com/' 16 | site_name = 'mysitename' 17 | project_name = 'myproject' 18 | token_name = 'mytokenname' 19 | token_value = 'tokenherebutpleaseconsidersecuritypolicies' 20 | # For more on tokens, head here: 21 | # https://help.tableau.com/current/server/en-us/security_personal_access_tokens.htm 22 | 23 | path_to_database = Path(hyper_name) 24 | 25 | # The table is called "Extract" and will be created in the "Extract" schema 26 | # and contains four columns. 27 | extract_table = TableDefinition( 28 | table_name=TableName("Extract", "Extract"), 29 | columns=[ 30 | TableDefinition.Column(name='Customer ID', type=SqlType.text(), nullability=NOT_NULLABLE), 31 | TableDefinition.Column(name='Customer Name', type=SqlType.text(), nullability=NOT_NULLABLE), 32 | TableDefinition.Column(name='Loyalty Reward Points', type=SqlType.big_int(), nullability=NOT_NULLABLE), 33 | TableDefinition.Column(name='Segment', type=SqlType.text(), nullability=NOT_NULLABLE) 34 | ] 35 | ) 36 | 37 | 38 | def insert_data(): 39 | """ 40 | An example demonstrating a simple single-table Hyper file including table creation and data insertion with different types 41 | This code is lifted from the below example: 42 | https://github.com/tableau/hyper-api-samples/blob/main/Tableau-Supported/Python/insert_data_into_single_table.py 43 | """ 44 | print("Creating single table for publishing.") 45 | 46 | # Starts the Hyper Process with telemetry enabled to send data to Tableau. 47 | # To opt out, simply set telemetry=Telemetry.DO_NOT_SEND_USAGE_DATA_TO_TABLEAU. 48 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 49 | 50 | # Creates new Hyper file "customer.hyper". 51 | # Replaces file with CreateMode.CREATE_AND_REPLACE if it already exists. 52 | with Connection(endpoint=hyper.endpoint, 53 | database=path_to_database, 54 | create_mode=CreateMode.CREATE_AND_REPLACE) as connection: 55 | 56 | connection.catalog.create_schema(schema=extract_table.table_name.schema_name) 57 | connection.catalog.create_table(table_definition=extract_table) 58 | 59 | # The rows to insert into the "Extract"."Extract" table. 60 | data_to_insert = [ 61 | ["DK-13375", "Dennis Kane", 685, "Consumer"], 62 | ["EB-13705", "Ed Braxton", 815, "Corporate"] 63 | ] 64 | 65 | with Inserter(connection, extract_table) as inserter: 66 | inserter.add_rows(rows=data_to_insert) 67 | inserter.execute() 68 | 69 | # The table names in the "Extract" schema (the default schema). 70 | table_names = connection.catalog.get_table_names("Extract") 71 | print(f"Tables available in {path_to_database} are: {table_names}") 72 | 73 | # Number of rows in the "Extract"."Extract" table. 74 | # `execute_scalar_query` is for executing a query that returns exactly one row with one column. 75 | row_count = connection.execute_scalar_query(query=f"SELECT COUNT(*) FROM {extract_table.table_name}") 76 | print(f"The number of rows in table {extract_table.table_name} is {row_count}.") 77 | 78 | print("The connection to the Hyper file has been closed.") 79 | print("The Hyper process has been shut down.") 80 | 81 | 82 | def publish_hyper(): 83 | """ 84 | Shows how to leverage the Tableau Server Client (TSC) to sign in and publish an extract directly to Tableau Online/Server 85 | """ 86 | 87 | # Sign in to server 88 | tableau_auth = TSC.PersonalAccessTokenAuth(token_name=token_name, personal_access_token=token_value, site_id=site_name) 89 | server = TSC.Server(server_address, use_server_version=True) 90 | 91 | print(f"Signing into {site_name} at {server_address}") 92 | with server.auth.sign_in(tableau_auth): 93 | # Define publish mode - Overwrite, Append, or CreateNew 94 | publish_mode = TSC.Server.PublishMode.Overwrite 95 | 96 | # Get project_id from project_name 97 | all_projects, pagination_item = server.projects.get() 98 | for project in TSC.Pager(server.projects): 99 | if project.name == project_name: 100 | project_id = project.id 101 | 102 | # Create the datasource object with the project_id 103 | datasource = TSC.DatasourceItem(project_id) 104 | 105 | print(f"Publishing {hyper_name} to {project_name}...") 106 | # Publish datasource 107 | datasource = server.datasources.publish(datasource, path_to_database, publish_mode) 108 | print("Datasource published. Datasource ID: {0}".format(datasource.id)) 109 | 110 | 111 | if __name__ == '__main__': 112 | insert_data() 113 | publish_hyper() 114 | -------------------------------------------------------------------------------- /Community-Supported/publish-hyper/requirements.txt: -------------------------------------------------------------------------------- 1 | tableauhyperapi==0.0.10899 2 | tableauserverclient==0.10 -------------------------------------------------------------------------------- /Community-Supported/publish-multi-table-hyper-legacy/config.json: -------------------------------------------------------------------------------- 1 | { 2 | "hyper_name": "extract.hyper", 3 | "tdsx_name": "test.tdsx", 4 | "server_address": "my.tableau.server", 5 | "site_name": "my_site", 6 | "project_name": "my_project", 7 | "tableau_token_name": "token_name", 8 | "tableau_token": "token_goes_here" 9 | } -------------------------------------------------------------------------------- /Community-Supported/publish-multi-table-hyper-legacy/requirements.txt: -------------------------------------------------------------------------------- 1 | tableauhyperapi==0.0.10899 2 | tableauserverclient==0.10 3 | tableau_tools==5.1.3 -------------------------------------------------------------------------------- /Community-Supported/publish-multi-table-hyper/README.md: -------------------------------------------------------------------------------- 1 | # publish-multi-table-hyper 2 | ## __Publishing a Multi-Table Hyper File to Tableau Online/Server (for Tableau Server versions 2021.4+)__ 3 | 4 | ![Community Supported](https://img.shields.io/badge/Support%20Level-Community%20Supported-53bd92.svg) 5 | 6 | In contrast to single-table `.hyper` files, with multi-table `.hyper` files it is not obvious which data model you want to analyze in the data source on Tableau Server. Thus, when publishing to Tableau Server < 2021.4, you can only publish single-table `.hyper` files or need to wrap the `.hyper` file into a Packaged Data Source (.tdsx). Starting with version 2021.4, you can now publish multi-table `.hyper` files to Tableau Server and Tableau Online; the data model will be automatically inferred (as specified by assumed table constraints). 7 | 8 | This sample demonstrates how to create a multi-table `.hyper` file with constraints such that Tableau Server (2021.4+) can infer the data model. If you are looking for how to wrap a multi-table `.hyper` file into a Packaged Data Source (.tdsx), have a look at [this sample](https://github.com/tableau/hyper-api-samples/tree/main/Community-Supported/publish-multi-table-hyper-legacy). 9 | 10 | It should serve as a starting point for anyone looking to automate the publishing process of multi-table extracts and data sources to Tableau. 11 | 12 | # Get started 13 | 14 | ## __Prerequisites__ 15 | To run the script, you will need: 16 | - Windows, Linux, or Mac 17 | - Python 3 18 | - Run `pip install -r requirements.txt` 19 | - Tableau Online/Server credentials or Personal Access Token 20 | 21 | ## __Configuration File__ 22 | Modify `config.json` and add the following fields: 23 | - Name of the `.hyper` file 24 | - Server/Online url 25 | - Site name 26 | - Project name 27 | - Authentication information 28 | 29 | ## __Data and Table Definitions__ 30 | If you want to simply run the sample to test the publishing process, you do not need to make any changes to the python file. Ensure that you have installed the requirements, update the config file with authentication information and execute the python file. 31 | 32 | Once you are ready to use your own data, you will need to change the `create_hyper_file_and_insert_data()` function. This function could be a part of an existing ETL workflow, grab the data from an API request, or pull CSVs from cloud storage like AWS, Azure, or GCP. In any case, writing that code is up to you. You can [check out this doc](https://tableau.github.io/hyper-db/lang_docs/py/tableauhyperapi.html?tableauhyperapi.Inserter) for more information on how to pass data to Hyper's `inserter()` method and [this doc](https://tableau.github.io/hyper-db/lang_docs/py/tableauhyperapi.html?tableauhyperapi.SqlType) for more information on the the Hyper API's SqlType class. 33 | 34 | __Note:__ The current example features two tables, but in theory, this could support as many as you'd like. Just be sure to add the proper table definitions and make sure that the order in the list of table data and table definitions properly match. 35 | 36 | ## __How Does Tableau Server infer the data model for the data source?__ 37 | On Tableau Server, the data model for the data source is generated from the foreign keys in the `.hyper` file. In particular, a relationship between two tables is generated whenever they are connected with a foreign key (the resulting relationship may be a multi-expression if multiple columns are involved). No validation on foreign keys will be performed, for example, referential integrity is not enforced. Only simple relationship trees which span all tables in the database are supported; publishing will fail if this is violated, e.g., if there are multiple tables which do not have incoming foreign keys (multiple fact tables). 38 | 39 | The resulting data source will have a single connection to the Hyper file, with a set of objects which correspond to individual tables in the Hyper file. For compatibility and to avoid surprises this will be done only for multi-table Hyper files, i.e. no changes for publishing single-table Hyper files. 40 | ## __Additional Customization__ 41 | If you end up needing to change more about how the extract is built (e.g., inserting directly from a CSV file) then you will need to also change the `create_hyper_file_and_insert_data()` function, but most likely nothing else. 42 | 43 | Leverage the [official Hyper API samples](https://github.com/tableau/hyper-api-samples/tree/master/Python) to learn more about what's possible. 44 | 45 | 46 | ## __Resources__ 47 | Check out these resources to learn more: 48 | - [Hyper API docs](https://tableau.github.io/hyper-db) 49 | - [TSC Docs](https://tableau.github.io/server-client-python/docs/) 50 | - [REST API docs](https://help.tableau.com/current/api/rest_api/en-us/REST/rest_api.htm) 51 | - [Tableau Tools](https://github.com/bryantbhowell/tableau_tools) 52 | - [Another multi-table example](https://github.com/tableau/hyper-api-samples/tree/main/Community-Supported/git-to-hyper) 53 | -------------------------------------------------------------------------------- /Community-Supported/publish-multi-table-hyper/config.json: -------------------------------------------------------------------------------- 1 | { 2 | "hyper_name": "data.hyper", 3 | "server_address": "my.tableau.server", 4 | "site_name": "my_site", 5 | "project_name": "my_project", 6 | "tableau_token_name": "token_name", 7 | "tableau_token": "token_goes_here" 8 | } -------------------------------------------------------------------------------- /Community-Supported/publish-multi-table-hyper/requirements.txt: -------------------------------------------------------------------------------- 1 | tableauhyperapi 2 | tableauserverclient==0.10 -------------------------------------------------------------------------------- /Community-Supported/query-external-data/README.md: -------------------------------------------------------------------------------- 1 | # query-external-data 2 | 3 | ![Community Supported](https://img.shields.io/badge/Support%20Level-Community%20Supported-53bd92.svg) 4 | 5 | __Current Version__: 1.0 6 | 7 | This sample demonstrates how you can use Hyper to query external data like parquet or CSV files directly. This enables a variety of ETL capabilities like accessing multiple files at once, filtering the read data and creating additional calculated columns. 8 | 9 | # Get started 10 | 11 | ## __Prerequisites__ 12 | 13 | To run the script, you will need: 14 | 15 | - a computer running Windows, macOS, or Linux 16 | 17 | - Python 3.7 or newer 18 | 19 | ## Run the sample 20 | 21 | Ensure that you have installed the requirements and then just run the sample Python file. 22 | The following instructions assume that you have set up a virtual environment for Python. For more information on 23 | creating virtual environments, see [venv - Creation of virtual environments](https://docs.python.org/3/library/venv.html) 24 | in the Python Standard Library. 25 | 26 | 1. Open a terminal and activate the Python virtual environment (`venv`). 27 | 28 | 1. Navigate to the folder where you installed the sample. 29 | 30 | 1. Run the Python script: 31 | 32 | **python query_external_data.py** 33 | 34 | ## __Resources__ 35 | Check out these resources to learn more: 36 | 37 | - [Hyper API docs](https://tableau.github.io/hyper-db) 38 | 39 | - [Tableau Hyper API Reference (Python)](https://tableau.github.io/hyper-db/lang_docs/py/index.html) 40 | 41 | - [The Hyper API SQL Reference](https://tableau.github.io/hyper-db/docs/sql/) 42 | -------------------------------------------------------------------------------- /Community-Supported/query-external-data/customers.csv: -------------------------------------------------------------------------------- 1 | 2554,DE,Hansastrasse,15 2 | 3554,DE,Ganghoferstrasse,24 3 | 2654,US,180th Ave,174 4 | 2564,US,150th Ave,114 5 | 2114,US,80th Ave,74 6 | 9954,US,42th Ave,94 7 | 2444,EN,Oxford Rd,13 8 | 1004,EN,Dowells Cl,41 9 | 6454,DE,Radlkoferstrasse,75 -------------------------------------------------------------------------------- /Community-Supported/query-external-data/customers_2.csv: -------------------------------------------------------------------------------- 1 | 2954,DE,Hansastrasse,11 2 | 9664,DE,Ganghoferstrasse,14 3 | 8554,US,10th Ave,184 4 | -------------------------------------------------------------------------------- /Community-Supported/query-external-data/orders.parquet: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Community-Supported/query-external-data/orders.parquet -------------------------------------------------------------------------------- /Community-Supported/query-external-data/query_external_data.py: -------------------------------------------------------------------------------- 1 | # ----------------------------------------------------------------------------- 2 | # 3 | # This file is the copyrighted property of Tableau Software and is protected 4 | # by registered patents and other applicable U.S. and international laws and 5 | # regulations. 6 | # 7 | # You may adapt this file and modify it to fit into your context and use it 8 | # as a template to start your own projects. 9 | # 10 | # ----------------------------------------------------------------------------- 11 | from tableauhyperapi import HyperProcess, Telemetry, \ 12 | Connection, CreateMode, \ 13 | HyperException 14 | 15 | def print_list(l): 16 | for e in l: 17 | print(e) 18 | 19 | 20 | def run_hyper_query_external(): 21 | """ 22 | An example demonstrating how to use Hyper to read data directly from external sources. 23 | 24 | More information can be found here: 25 | https://tableau.github.io/hyper-db/docs/sql/external/ 26 | https://tableau.github.io/hyper-db/docs/sql/command/copy_from 27 | https://tableau.github.io/hyper-db/docs/sql/command/create_external_table 28 | https://tableau.github.io/hyper-db/docs/sql/external/ 29 | """ 30 | 31 | # Start the Hyper process. 32 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 33 | # Open a connection to the Hyper process. This will also create the new Hyper file. 34 | # The `CREATE_AND_REPLACE` mode causes the file to be replaced if it 35 | # already exists. 36 | with Connection(endpoint=hyper.endpoint, 37 | database="output_file.hyper", 38 | create_mode=CreateMode.CREATE_AND_REPLACE) as connection: 39 | 40 | print("Scenario 1: Create a table from filtered parquet data with a calculated extra column") 41 | # This SQL command queries a parquet file directly and creates the table 'low_prio_orders' in Hyper. 42 | # The created table contains the data that is returned from the 'SELECT' part of the query. I.e., only 43 | # a selection of columns, a new calculated column 'employee_nr' and only the rows with low order priority. 44 | command_1 = """CREATE TABLE low_prio_orders AS 45 | SELECT order_key, customer_key, price, CAST(SUBSTRING(employee from 0 for 6) AS int) as employee_nr 46 | FROM external('orders.parquet') 47 | WHERE priority = 'LOW'""" 48 | 49 | connection.execute_command(command_1) 50 | 51 | print("table content:") 52 | print_list(connection.execute_list_query("SELECT * FROM low_prio_orders")) 53 | print() 54 | 55 | print("\nScenario 2: Query multiple external data sources in one query.") 56 | # This query reads data from a parquet and a CSV file and joins it. Note that, for CSV files, the schema of the file 57 | # has to be provided and currently cannot be inferred form the file directly (see the `DESCRIPTOR` argument below). 58 | command_2 = """SELECT country, SUM(quantity * price) 59 | FROM external('orders.parquet') orders 60 | join external('customers.csv', 61 | COLUMNS => DESCRIPTOR(customer_key int, country text, street text, nr int), 62 | DELIMITER => ',', FORMAT => 'csv', HEADER => false) customers 63 | on orders.customer_key = customers.customer_key GROUP BY country 64 | ORDER BY country""" 65 | print("result:") 66 | print_list(connection.execute_list_query(command_2)) 67 | print() 68 | 69 | 70 | print("Scenario 3: Query multiple CSV files that have the same schema in one go.") 71 | # Note that, for CSV files, the schema of the file has to be provided and currently cannot be inferred form the file directly. 72 | # (see the `DESCRIPTOR` argument below). 73 | command_3 = """SELECT * 74 | FROM external(ARRAY['customers.csv','customers.csv'], 75 | COLUMNS => DESCRIPTOR(customer_key int, country text, street text, nr int), 76 | DELIMITER => ',', FORMAT => 'csv', HEADER => false) 77 | ORDER BY country""" 78 | 79 | print("result:") 80 | print_list(connection.execute_list_query(command_3)) 81 | 82 | 83 | 84 | if __name__ == '__main__': 85 | run_hyper_query_external() 86 | -------------------------------------------------------------------------------- /Community-Supported/s3-compatible-services/README.md: -------------------------------------------------------------------------------- 1 | # parquet-to-hyper 2 | 3 | ## __parquet_to_hyper__ 4 | 5 | ![Community Supported](https://img.shields.io/badge/Support%20Level-Community%20Supported-53bd92.svg) 6 | 7 | __Current Version__: 1.0 8 | 9 | These samples show you how Hyper can natively interact with S3 compatible services, such as Google Storage, without the need to install any external dependencies like `google-cloud-bigquery`. 10 | 11 | # Get started 12 | 13 | ## __Prerequisites__ 14 | 15 | To run the script, you will need: 16 | 17 | - a computer running Windows, macOS, or Linux 18 | 19 | - Python 3.9+ 20 | 21 | - install the dependencies from the `requirements.txt` file 22 | 23 | ## Run the samples 24 | 25 | The following instructions assume that you have set up a virtual environment for Python. For more information on 26 | creating virtual environments, see [venv - Creation of virtual environments](https://docs.python.org/3/library/venv.html) 27 | in the Python Standard Library. 28 | 29 | 1. Open a terminal and activate the Python virtual environment (`venv`). 30 | 31 | 1. Navigate to the folder where you installed the samples. 32 | 33 | 1. Then follow the steps to run one of the samples which are shown below. 34 | 35 | **Live query against a `.parquet` file which is stored on Google Storage** 36 | 37 | Run the Python script 38 | 39 | ```bash 40 | $ python query-parquet-on-gs.py 41 | ``` 42 | This script will perform a live query on the Parquet file which is stored in this public Google Storage bucket: `gs://cloud-samples-data/bigquery/us-states/us-states.parquet`. 43 | 44 | ## __Resources__ 45 | Check out these resources to learn more: 46 | 47 | - [Hyper API docs](https://tableau.github.io/hyper-db) 48 | 49 | - [Tableau Hyper API Reference (Python)](https://tableau.github.io/hyper-db/lang_docs/py/index.html) 50 | 51 | - [The EXTERNAL function in the Hyper API SQL Reference](https://tableau.github.io/hyper-db/docs/sql/external/) 52 | 53 | -------------------------------------------------------------------------------- /Community-Supported/s3-compatible-services/query-parquet-on-gs.py: -------------------------------------------------------------------------------- 1 | """Connect to Google Storage and query a parquet file located in a public bucket. 2 | 3 | Adapted from hyper-api-samples/Community-Supported/native-s3/query-csv-on-s3.py 4 | """ 5 | 6 | from tableauhyperapi import Connection, HyperProcess, Telemetry, escape_string_literal 7 | 8 | BUCKET_NAME = "cloud-samples-data" 9 | FILE_PATH = "bigquery/us-states/us-states.parquet" 10 | 11 | states_dataset_gs = escape_string_literal( 12 | f"s3://{BUCKET_NAME.strip('/')}/{FILE_PATH.strip('/')}" 13 | ) 14 | 15 | # Hyper Process parameters 16 | parameters = {} 17 | # endpoint URL 18 | parameters["external_s3_hostname"] = "storage.googleapis.com" 19 | # We do not need to specify credentials and bucket location as the GS bucket is 20 | # publicly accessible; this may be different when used with your own data 21 | 22 | with HyperProcess( 23 | telemetry=Telemetry.DO_NOT_SEND_USAGE_DATA_TO_TABLEAU, 24 | parameters=parameters, 25 | ) as hyper: 26 | # Create a connection to the Hyper process - we do not connect to a database 27 | with Connection( 28 | endpoint=hyper.endpoint, 29 | ) as connection: 30 | 31 | # Use the SELECT FROM EXTERNAL(S3_LOCATION()) syntax - this allows us to use 32 | # the parquet file like a normal table name in SQL queries 33 | sql_query = ( 34 | f"""SELECT COUNT(*) FROM EXTERNAL(S3_LOCATION({states_dataset_gs}))""" 35 | ) 36 | 37 | # Execute the query with `execute_scalar_query` as we expect a single number 38 | count = connection.execute_scalar_query(sql_query) 39 | print(f"number of rows : {count}") 40 | -------------------------------------------------------------------------------- /Community-Supported/s3-to-hyper/README.md: -------------------------------------------------------------------------------- 1 | # s3-to-hyper 2 | ## __Dynamically Creating and Publishing Hyper Files from S3__ 3 | 4 | ![Community Supported](https://img.shields.io/badge/Support%20Level-Community%20Supported-53bd92.svg) 5 | 6 | __Current Version__: 1.0 7 | 8 | This sample demonstrates how to, with little modification, leverage the Hyper API, Tableau Server Client Library, Tableau Tools, and Boto3 to do the following: 9 | - Download a header file and un-unioned CSVs to pass to hyper from a S3 bucket 10 | - Create an empty hyper file with columns defined in the header file 11 | - Copies the CSVs to the hyper file based on a wildcard pattern match 12 | - Swaps the newly created extract into a Packaged Data Source file (.tdsx) 13 | - Publishes the data source to a specified project on Tableau Online/Server 14 | 15 | It should serve as a starting point for anyone looking to automate the publishing process of datasources based on contents of S3 buckets. The advantage of leveraging this sample is that an end user should not need to open the Python script, instead simply edit the configuration file and the code handles the rest automatically. 16 | 17 | **Note:** As an alternative to using Boto3, you can also check out if [Hyper's Native S3 capabilities](https://github.com/tableau/hyper-api-samples/tree/main/Community-Supported/native-s3/README.md) are applicable to your use-case to ingest data from AWS S3 into Hyper. 18 | 19 | # Get started 20 | 21 | ## __Prerequisites__ 22 | To run the script, you will need: 23 | - Windows or Mac 24 | - Tableau Desktop v10.5 or higher 25 | - Python 3.6 - 3.7 26 | - Run `pip install -r requirements.txt` 27 | - Tableau Online/Server credentials or personal access token 28 | - AWS Credentials File ([more on this here](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html)) 29 | 30 | ## __Configuration File__ 31 | You will need to modify `config.json` and add fields that include: 32 | - `.hyper` file name, `.tdsx` file name 33 | - Tableau Online/Server address, site, project, and auth fields 34 | - Header file and wildcard name format 35 | - S3 bucket name and AWS credential profile name 36 | 37 | 38 | ## __Creating the .tdsx File__ 39 | As mentioned, one key step needed for the automatic publishing of multi-table hyper files is a Packaged Data Source, or .tdsx. As of now, this is a step that must be completed manually as a part of the setup process. _You will only need to do this once_. If a .tdsx is not present in the directory, the script will prompt you to create one. 40 | 41 | Packaged Data Sources contain important metadata needed for Tableau Desktop and Server/Online. This includes things like definted joins and join clauses, relationships, calculated fields, and more. 42 | 43 | To create the .tdsx, [follow these steps](https://help.tableau.com/current/pro/desktop/en-us/export_connection.htm): 44 | - Run the script without a data source present to create the initial hyper file 45 | - Double-click the hyper file to open it in Tableau Desktop 46 | - Click and drag the relevant tables and create the joins or relationships 47 | - Head to 'Sheet 1' 48 | - In the top-left corner, right-click on the data source and select 'Add to Saved Data Sources...' 49 | - Name the file to match the value in `config.json` 50 | - Select 'Tableau __Packaged__ Data Source (*.tdsx)' from the dropdown 51 | - Save it in the directory with the script and hyper file 52 | 53 | Now you are free to rerun the script and validate the swapping and publishing process. Unless you change how the hyper file is being created (schema, column names, joins, etc.), you will not need to remake the .tdsx again. 54 | 55 | 56 | Leverage the [official Hyper API samples](https://github.com/tableau/hyper-api-samples/tree/master/Python) to learn more about what's possible. 57 | 58 | 59 | ## __Resources__ 60 | Check out these resources to learn more: 61 | - [Hyper API docs](https://tableau.github.io/hyper-db) 62 | - [TSC docs](https://tableau.github.io/server-client-python/docs/) 63 | - [REST API docs](https://help.tableau.com/current/api/rest_api/en-us/REST/rest_api.htm) 64 | - [Tableau Tools](https://github.com/bryantbhowell/tableau_tools) 65 | - [Publishing Data Sources](https://help.tableau.com/current/pro/desktop/en-us/export_connection.htm) 66 | -------------------------------------------------------------------------------- /Community-Supported/s3-to-hyper/config.json: -------------------------------------------------------------------------------- 1 | { 2 | "hyper_name": "extract.hyper", 3 | "tdsx_name": "test.tdsx", 4 | "table_name": "s3_data", 5 | "server_address": "my.tableau.server", 6 | "site_name": "my_site", 7 | "project_name": "my_project", 8 | "tableau_token_name": "token_name", 9 | "tableau_token": "token_goes_here", 10 | "header_file": "header.csv", 11 | "name_format": "sales*.csv", 12 | "bucket_name": "bucket_name", 13 | "aws_cred_profile_name": "default", 14 | "contains_header": "False" 15 | } -------------------------------------------------------------------------------- /Community-Supported/s3-to-hyper/requirements.txt: -------------------------------------------------------------------------------- 1 | tableauhyperapi==0.0.10899 2 | tableauserverclient==0.10 3 | tableau_tools==5.1.3 4 | boto3==1.13.16 5 | botocore==1.16.16 -------------------------------------------------------------------------------- /Community-Supported/tde-to-hyper/README.md: -------------------------------------------------------------------------------- 1 | # tde-to-hyper 2 | 3 | ![Community Supported](https://img.shields.io/badge/Support%20Level-Community%20Supported-53bd92.svg) 4 | 5 | The `.tde` file format will be deprecated by end of 2023 (see [deprecation announcement](https://community.tableau.com/s/feed/0D54T00001BHiGwSAL)). 6 | This script upgrades `.tde` files to `.hyper` files. 7 | 8 | # Get started 9 | 10 | ## Prerequisites 11 | 12 | To run the script, you will need: 13 | 14 | - A computer running Windows, macOS, or Linux 15 | - Python (3.7 or newer) 16 | - Install the Hyper API version `0.0.17537` (`pip install -r requirements.txt`) 17 | 18 | ## Run the sample 19 | 20 | > **_NOTE:_** The following command lines are for Linux or macOS and need to be slightly adapted for Windows. 21 | 22 | The following instructions assume that you want to use a virtual environment for Python. For more information on 23 | creating virtual environments, see [venv - Creation of virtual environments](https://docs.python.org/3/library/venv.html). 24 | 25 | 1. Open a terminal and navigate to the folder of the `tde_to_hyper.py` file 26 | 1. Create a virtual environment and install Python Hyper API 27 | ``` 28 | $ python3 -m venv .venv/ 29 | $ .venv/bin/python3 -m pip install -r requirements.txt 30 | ``` 31 | 1. Run the Python script: 32 | The script requires a path to a tde file and will convert the tde to a `.hyper` file. The `.hyper` file will be created in the directory of the tde file. 33 | Example: 34 | ```cli 35 | $ .venv/bin/python3 tde_to_hyper.py extract.tde 36 | Successfully converted extract.tde to extract.hyper 37 | $ .venv/bin/python3 tde_to_hyper.py path/to/file/extract.tde 38 | Successfully converted path/to/file/extract.tde to path/to/file/extract.hyper 39 | ``` 40 | 41 | ## Advanced usage 42 | 43 | The script can also be invoked on a folder and will upgrade each `.tde` file within 44 | that folder. The upgraded `.hyper` files will be placed directly next to the `.tde` 45 | files. If the new `.hyper` files should be written to a differnt folder, the 46 | `--output` can be used to specify a location where the upgraded `.hyper` should be 47 | written. 48 | 49 | ## Resources 50 | 51 | - [Hyper API docs](https://tableau.github.io/hyper-db) 52 | - [Tableau Hyper API Reference (Python)](https://tableau.github.io/hyper-db/lang_docs/py/index.html) 53 | - [TDE deprecation announcement](https://community.tableau.com/s/feed/0D54T00001BHiGwSAL) 54 | - [August 2023 update on TDE deprecation](https://community.tableau.com/s/feed/0D58b0000BTEIShCQP) 55 | -------------------------------------------------------------------------------- /Community-Supported/tde-to-hyper/requirements.txt: -------------------------------------------------------------------------------- 1 | tableauhyperapi==0.0.17537 2 | -------------------------------------------------------------------------------- /Community-Supported/tde-to-hyper/tde_to_hyper.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | import os 3 | from pathlib import Path 4 | from tableauhyperapi import HyperProcess, Telemetry, Connection, CreateMode, escape_string_literal, TableName, HyperException 5 | 6 | def convert_tde_to_hyper(hyper_endpoint: str, tde_path: Path, hyper_path: Path): 7 | os.makedirs(hyper_path.parent.absolute(), exist_ok = True) 8 | with Connection(endpoint=hyper_endpoint, database=hyper_database, create_mode=CreateMode.CREATE_AND_REPLACE) as connection: 9 | try: 10 | existing_tables = connection.execute_list_query(f""" 11 | SELECT "SCHEMAS_NAME", "TABLES_NAME" 12 | FROM external({escape_string_literal(str(tde_path))}, format => 'tde', "table" => 'SYS.TABLES') 13 | JOIN external({escape_string_literal(str(tde_path))}, format => 'tde', "table" => 'SYS.SCHEMAS') 14 | ON "TABLES_PARENT"="SCHEMAS_ID" 15 | WHERE "SCHEMAS_NAME" <> 'SYS' AND "TABLES_NAME"<>'$TableauMetadata' AND "TABLES_ACTIVE" AND "SCHEMAS_ACTIVE" 16 | """) 17 | for schema, table in existing_tables: 18 | # Create the destination table in the Hyper database 19 | connection.catalog.create_schema_if_not_exists(schema) 20 | connection.execute_command(f""" 21 | CREATE TABLE {TableName(schema, table)} AS 22 | SELECT * FROM external({escape_string_literal(str(tde_path))}, format => 'tde', "table" => {escape_string_literal(f"{schema}.{table}")})""") 23 | except HyperException: 24 | os.unlink(hyper_database) 25 | print(f"FAILED conversion {tde_path} -> {hyper_database}") 26 | 27 | if __name__ == '__main__': 28 | argparser = argparse.ArgumentParser(description="Script to convert a TDE file to a Hyper file.") 29 | argparser.add_argument("input_tde_path", type=Path, help="The input TDE file path that will be converted to a Hyper file.") 30 | argparser.add_argument("--output", type=Path, help="The output path.", default=".") 31 | args = argparser.parse_args() 32 | 33 | input_tde_path = Path(args.input_tde_path) 34 | if not input_tde_path.exists(): 35 | raise Exception(f"{input_tde_path} not found") 36 | 37 | if input_tde_path.is_dir(): 38 | inputs = list(input_tde_path.glob("**/*.tde")) 39 | else: 40 | inputs = [input_tde_path] 41 | 42 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 43 | for tde_path in inputs: 44 | # Rename path with a .hyper extension in the directory of the tde file 45 | hyper_database = args.output / tde_path.with_name(tde_path.stem + '.hyper') 46 | convert_tde_to_hyper(hyper.endpoint, tde_path, hyper_database) 47 | 48 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Tableau 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | # Hyper API Samples 5 | [![Tableau Supported](https://img.shields.io/badge/Support%20Level-Tableau%20Supported-53bd92.svg)](https://www.tableau.com/support-levels-it-and-developer-tools) 6 | [![Community Supported](https://img.shields.io/badge/Support%20Level-Community%20Supported-457387.svg)](https://www.tableau.com/support-levels-it-and-developer-tools) 7 | 8 | This repo is the home of the Hyper API samples. It contains both __Tableau-Supported Samples__ and __Community-Supported Samples__: 9 | - The official samples are packaged and shipped within the Hyper API itself. All of the official samples are available for each language supported by the Hyper API: Python, Java, C++, and C#/.Net (.NET Standard 2.0) and are entirely supported and maintained by Tableau. 10 | - The community samples focus on individual use cases and are Python-only. They have been written by members of the Tableau development team but receive the level of 'Community Supported' as they may not be maintained in newer releases. Each of the samples has been manually tested and reviewed before publishing and will still be open to pull requests and issues. 11 | - See our [support page](https://www.tableau.com/support/itsupport) for more information. 12 | 13 |
14 | 15 | ### __If you are looking to learn more about the Hyper API, please check out the [official documentation](https://tableau.github.io/hyper-db).__ ### 16 | 17 |
18 | 19 | ## What is the Hyper API? 20 | For the unfamiliar, the Hyper API contains a set of functions you can use to automate your interactions with Tableau extract (.hyper) files. You can use the API to create new extract files, or to open existing files, and then insert, delete, update, or read data from those files. Using the Hyper API developers and administrators can: 21 | * Create extract files for data sources not currently supported by Tableau. 22 | * Automate custom extract, transform and load (ETL) processes (for example, implement rolling window updates or custom incremental updates). 23 | * Retrieve data from an extract file. 24 | 25 | 26 | 27 | ## How do I install the Hyper API? 28 | It is a prerequisite that to work with these code samples, the Hyper API is installed in your language of choice. Head to our [official Hyper API Documentation](https://tableau.github.io/hyper-db/docs) to get it up and running. 29 | 30 | 31 | 32 | ## How do I get help or give feedback? 33 | If you have questions, want to submit ideas, or give feedback on the code, please do so by submitting an issue on this project. 34 | 35 | 36 | 37 | ## Contributions 38 | Code contributions and improvements by the community are welcomed and accepted on a case-by-case basis. See the LICENSE file for current open-source licensing and use information. 39 | 40 | Before we can accept pull requests from contributors, we require a signed [Contributor License Agreement (CLA)](https://tableau.github.io/contributing.html). 41 | 42 | ## Hyper API License 43 | Note that the Hyper API comes with a proprietary license, see the `LICENSE` file in the Hyper API package. 44 | -------------------------------------------------------------------------------- /Tableau-Supported/CPP/create_hyper_file_from_csv.cpp: -------------------------------------------------------------------------------- 1 | /** 2 | * \example create_hyper_file_from_csv.cpp 3 | * 4 | * An example of how to load data from a CSV file into a new Hyper file. 5 | */ 6 | 7 | #include 8 | #include 9 | #include 10 | 11 | static const hyperapi::TableDefinition customerTable{ 12 | "Customer", // Since the table name is not prefixed with an explicit schema name, the table will reside in the default "public" namespace. 13 | {hyperapi::TableDefinition::Column{"Customer ID", hyperapi::SqlType::text(), hyperapi::Nullability::NotNullable}, 14 | hyperapi::TableDefinition::Column{"Customer Name", hyperapi::SqlType::text(), hyperapi::Nullability::NotNullable}, 15 | hyperapi::TableDefinition::Column{"Loyalty Reward Points", hyperapi::SqlType::bigInt(), hyperapi::Nullability::NotNullable}, 16 | hyperapi::TableDefinition::Column{"Segment", hyperapi::SqlType::text(), hyperapi::Nullability::NotNullable}}}; 17 | 18 | // An example demonstrating loading data from a csv into a new Hyper file 19 | // For more details, see https://tableau.github.io/hyper-db/docs/guides/hyper_file/insert_csv 20 | static void runCreateHyperFileFromCSV() { 21 | std::cout << "EXAMPLE - Load data from CSV into table in new Hyper file" << std::endl; 22 | const std::string pathToDatabase = "data/customer.hyper"; 23 | 24 | // Starts the Hyper Process with telemetry enabled to send data to Tableau. 25 | // To opt out, simply set telemetry=hyperapi::Telemetry::DoNotSendUsageDataToTableau. 26 | { 27 | // Optional process parameters. They are documented in the Tableau Hyper documentation, chapter "Process Settings" 28 | // (https://tableau.github.io/hyper-db/docs/hyper-api/hyper_process#process-settings). 29 | std::unordered_map processParameters = { 30 | // Limits the number of Hyper event log files to two. 31 | {"log_file_max_count", "2"}, 32 | // Limits the size of Hyper event log files to 100 megabytes. 33 | {"log_file_size_limit", "100M"}}; 34 | 35 | hyperapi::HyperProcess hyper(hyperapi::Telemetry::SendUsageDataToTableau, "example", processParameters); 36 | // Creates new Hyper file "customer.hyper". 37 | // Replaces existing file with hyperapi::CreateMode::CreateAndReplace if it already exists. 38 | { 39 | // Optional connection parameters. They are documented in the Tableau Hyper documentation, chapter "Connection Settings" 40 | // (https://tableau.github.io/hyper-db/docs/hyper-api/connection#connection-settings). 41 | std::unordered_map connectionParameters = {{"lc_time", "en_US"}}; 42 | 43 | hyperapi::Connection connection(hyper.getEndpoint(), pathToDatabase, hyperapi::CreateMode::CreateAndReplace, connectionParameters); 44 | const hyperapi::Catalog& catalog = connection.getCatalog(); 45 | 46 | catalog.createTable(customerTable); 47 | 48 | // Using path to current file, create a path that locates CSV file packaged with these examples. 49 | std::string pathToCSV = "data/customers.csv"; 50 | 51 | // Load all rows into "Customers" table from the CSV file. 52 | // `executeCommand` executes a SQL statement and returns the impacted row count. 53 | // 54 | // Note: 55 | // You might have to adjust the COPY parameters to the format of your specific csv file. 56 | // The example assumes that your columns are separated with the ',' character 57 | // and that NULL values are encoded via the string 'NULL'. 58 | // Also be aware that the `header` option is used in this example: 59 | // It treats the first line of the csv file as a header and does not import it. 60 | // 61 | // The parameters of the COPY command are documented in the Tableau Hyper SQL documentation 62 | // (https://tableau.github.io/hyper-db/docs/sql/command/copy_from). 63 | std::cout << "Issuing the SQL COPY command to load the csv file into the table. Since the first line" << std::endl; 64 | std::cout << "of our csv file contains the column names, we use the `header` option to skip it." << std::endl; 65 | int64_t rowCount = connection.executeCommand( 66 | "COPY " + customerTable.getTableName().toString() + " from " + hyperapi::escapeStringLiteral(pathToCSV) + 67 | " with (format csv, NULL 'NULL', delimiter ',', header)"); 68 | 69 | std::cout << "The number of rows in table " << customerTable.getTableName() << " is " << rowCount << "." << std::endl; 70 | } 71 | std::cout << "The connection to the Hyper file has been closed." << std::endl; 72 | } 73 | std::cout << "The Hyper Process has been shut down." << std::endl; 74 | } 75 | 76 | int main() { 77 | try { 78 | runCreateHyperFileFromCSV(); 79 | } catch (const hyperapi::HyperException& e) { 80 | std::cout << e.toString() << std::endl; 81 | return 1; 82 | } 83 | return 0; 84 | } 85 | -------------------------------------------------------------------------------- /Tableau-Supported/CPP/data/Staples.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Tableau-Supported/CPP/data/Staples.csv -------------------------------------------------------------------------------- /Tableau-Supported/CPP/data/calcs.csv: -------------------------------------------------------------------------------- 1 | key,num0,num1,num2,num3,num4,str0,str1,str2,str3,int0,int1,int2,int3,bool0_,bool1_,bool2_,bool3_,date0,date1,date2,date3,time0,time1,datetime0,datetime1,zzz 2 | "key00",12.300000000000001,8.4199999999999999,17.859999999999999,-11.52,,"FURNITURE","CLAMP ON LAMPS","one","e",1,-3,5,8,1,1,0,1,2004-04-15 00:00:00,2004-04-01 00:00:00,1977-04-20 00:00:00,1986-03-20 00:00:00,1899-12-30 21:07:32,1899-12-30 19:36:22,2004-07-09 10:17:35,"","a" 3 | "key01",-12.300000000000001,6.71,16.73,-9.3100000000000005,10.85,"FURNITURE","CLOCKS","two","e",,-6,-4,13,0,1,0,,1972-07-04 00:00:00,2004-04-02 00:00:00,1995-09-03 00:00:00,,1900-01-01 13:48:48,1899-12-30 02:05:25,2004-07-26 12:30:34,"","b" 4 | "key02",15.700000000000001,9.7799999999999994,,-12.17,-13.470000000000001,"OFFICE SUPPLIES","AIR PURIFIERS","three","e",,,5,2,,1,0,,1975-11-12 00:00:00,2004-04-03 00:00:00,1997-09-19 00:00:00,1997-02-02 00:00:00,1900-01-01 18:21:08,1899-12-30 09:33:31,2004-08-02 07:59:23,"","c" 5 | "key03",-15.700000000000001,7.4299999999999997,8.5099999999999998,-7.25,-6.0499999999999998,"OFFICE SUPPLIES","BINDER ACCESSORIES","","e",,-4,-5,5,1,0,0,,2004-06-04 00:00:00,2004-04-04 00:00:00,1980-07-26 00:00:00,,1900-01-01 18:51:48,1899-12-30 22:50:16,2004-07-05 13:14:20,"","d" 6 | "key04",3.5,9.0500000000000007,6.46,12.93,8.3200000000000003,"OFFICE SUPPLIES","BINDER CLIPS","five","",7,,3,9,0,0,1,1,2004-06-19 00:00:00,2004-04-05 00:00:00,1997-05-30 00:00:00,1996-03-07 00:00:00,1900-01-01 15:01:19,,2004-07-28 23:30:22,"","e" 7 | "key05",-3.5,9.3800000000000008,8.9800000000000004,-19.960000000000001,10.710000000000001,"OFFICE SUPPLIES","BINDING MACHINES","six","",3,,2,7,,0,1,0,,2004-04-06 00:00:00,1980-11-07 00:00:00,1979-04-01 00:00:00,1900-01-01 08:59:39,1899-12-30 19:57:33,2004-07-22 00:30:23,"","f" 8 | "key06",0,16.420000000000002,11.69,10.93,,"OFFICE SUPPLIES","BINDING SUPPLIES","","e",8,,9,18,1,,0,,,2004-04-07 00:00:00,1977-02-08 00:00:00,,1900-01-01 07:37:48,,2004-07-28 06:54:50,"","g" 9 | "key07",,11.380000000000001,17.25,3.6400000000000001,-10.24,"OFFICE SUPPLIES","BUSINESS ENVELOPES","eight","e",,2,0,3,0,,1,0,,2004-04-08 00:00:00,1974-05-03 00:00:00,,1900-01-01 19:45:54,1899-12-30 19:48:23,2004-07-12 17:30:16,"","h" 10 | "key08",10,9.4700000000000006,,-13.380000000000001,4.7700000000000005,"TECHNOLOGY","ANSWERING MACHINES","nine","",,3,-6,17,,,0,0,,2004-04-09 00:00:00,1976-09-09 00:00:00,1983-05-22 00:00:00,1900-01-01 09:00:59,1899-12-30 22:20:14,2004-07-04 22:49:28,"","i" 11 | "key09",,12.4,11.5,-10.56,,"TECHNOLOGY","BUSINESS COPIERS","ten","e",8,3,-9,2,,1,0,,,2004-04-10 00:00:00,1998-08-12 00:00:00,,1900-01-01 20:36:00,,2004-07-23 21:13:37,"","j" 12 | "key10",,10.32,6.7999999999999998,-4.79,19.390000000000001,"TECHNOLOGY","CD-R MEDIA","eleven","e",4,,-3,11,1,1,0,,,2004-04-11 00:00:00,1974-03-17 00:00:00,1999-08-20 00:00:00,1900-01-01 01:31:32,1899-12-30 00:05:57,2004-07-14 08:16:44,"","k" 13 | "key11",,2.4700000000000002,3.79,-10.81,3.8200000000000003,"TECHNOLOGY","CONFERENCE PHONES","twelve","",10,-8,-4,2,0,1,1,,,2004-04-12 00:00:00,1994-04-20 00:00:00,,1899-12-30 22:15:40,1899-12-30 04:40:49,2004-07-25 15:22:26,"","l" 14 | "key12",,12.050000000000001,,-6.6200000000000001,3.3799999999999999,"TECHNOLOGY","CORDED KEYBOARDS","","",,,0,11,,0,1,1,,2004-04-13 00:00:00,2001-02-04 00:00:00,,1900-01-01 13:53:46,1899-12-30 04:48:07,2004-07-17 14:01:56,"","m" 15 | "key13",,10.370000000000001,13.040000000000001,-18.43,,"TECHNOLOGY","CORDLESS KEYBOARDS","fourteen","",4,,4,18,,0,1,1,,2004-04-14 00:00:00,1988-01-05 00:00:00,1996-05-13 00:00:00,1900-01-01 04:57:51,,2004-07-19 22:21:31,"","n" 16 | "key14",,7.1000000000000005,,6.8399999999999999,-14.210000000000001,"TECHNOLOGY","DOT MATRIX PRINTERS","fifteen","e",11,,-8,18,1,0,1,,,2004-04-15 00:00:00,1972-07-12 00:00:00,1986-11-08 00:00:00,1899-12-30 22:42:43,1899-12-30 18:58:41,2004-07-31 11:57:52,"","o" 17 | "key15",,16.809999999999999,10.98,-10.98,6.75,"TECHNOLOGY","DVD","sixteen","e",4,,-9,11,0,,0,1,,2004-04-16 00:00:00,1995-06-04 00:00:00,,1899-12-30 22:24:08,,2004-07-14 07:43:00,"","p" 18 | "key16",,7.1200000000000001,7.8700000000000001,-2.6000000000000001,,"TECHNOLOGY","ERICSSON","","",8,-9,6,0,,,0,,,2004-04-17 00:00:00,2002-04-27 00:00:00,1992-01-18 00:00:00,1900-01-01 11:58:29,1899-12-30 12:33:57,2004-07-28 12:34:28,"","q" 19 | -------------------------------------------------------------------------------- /Tableau-Supported/CPP/data/sample_extracts/superstore_sample.hyper: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Tableau-Supported/CPP/data/sample_extracts/superstore_sample.hyper -------------------------------------------------------------------------------- /Tableau-Supported/CPP/data/sample_extracts/superstore_sample_denormalized.hyper: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Tableau-Supported/CPP/data/sample_extracts/superstore_sample_denormalized.hyper -------------------------------------------------------------------------------- /Tableau-Supported/CPP/data/superstore_normalized/states.csv: -------------------------------------------------------------------------------- 1 | Population,State 2 | 4833722,Alabama 3 | 735132,Alaska 4 | 6626624,Arizona 5 | 2959373,Arkansas 6 | 38332521,California 7 | 5268367,Colorado 8 | 3596080,Connecticut 9 | 925749,Delaware 10 | 646449,District of Columbia 11 | 19552860,Florida 12 | 9992167,Georgia 13 | 1404054,Hawaii 14 | 1612136,Idaho 15 | 12882135,Illinois 16 | 6570902,Indiana 17 | 3090416,Iowa 18 | 2893957,Kansas 19 | 4395295,Kentucky 20 | 4625470,Louisiana 21 | 1328302,Maine 22 | 5928814,Maryland 23 | 6692824,Massachusetts 24 | 9895622,Michigan 25 | 5420380,Minnesota 26 | 2991207,Mississippi 27 | 6044171,Missouri 28 | 1015165,Montana 29 | 1868516,Nebraska 30 | 2790136,Nevada 31 | 1323459,New Hampshire 32 | 8899339,New Jersey 33 | 2085287,New Mexico 34 | 19651127,New York 35 | 9848060,North Carolina 36 | 723393,North Dakota 37 | 11570808,Ohio 38 | 3850568,Oklahoma 39 | 3930065,Oregon 40 | 12773801,Pennsylvania 41 | 1051511,Rhode Island 42 | 4774839,South Carolina 43 | 844877,South Dakota 44 | 6495978,Tennessee 45 | 26448193,Texas 46 | 2900872,Utah 47 | 626630,Vermont 48 | 8260405,Virginia 49 | 6971406,Washington 50 | 1854304,West Virginia 51 | 5742713,Wisconsin 52 | 582658,Wyoming 53 | -------------------------------------------------------------------------------- /Tableau-Supported/CPP/delete_data_in_existing_hyper_file.cpp: -------------------------------------------------------------------------------- 1 | /** 2 | * \example delete_data_in_existing_hyper_file.cpp 3 | * 4 | * An example of how to delete data in an existing Hyper file. 5 | */ 6 | 7 | #include 8 | #include 9 | #include 10 | #include 11 | 12 | /** 13 | * Helper function to copy a file 14 | */ 15 | 16 | static void copy(const std::string& sourcePath, const std::string& destinationPath) { 17 | std::ifstream source(sourcePath, std::ios::binary); 18 | std::ofstream destination(destinationPath, std::ios::binary); 19 | destination << source.rdbuf(); 20 | source.close(); 21 | destination.close(); 22 | } 23 | 24 | static void runDeleteDataInExistingHyperFile() { 25 | std::cout << "EXAMPLE - Delete data from an existing Hyper file" << std::endl; 26 | 27 | // Path to a Hyper file containing all data inserted into Customer, Product, Orders and LineItems table 28 | // See "insert_data_into_multiple_tables.cpp" for an example that works with the complete schema. 29 | const std::string pathToSourceDatabase = "data/superstore_sample.hyper"; 30 | 31 | // Make a copy of the superstore example Hyper file. 32 | const std::string pathToDatabase = "data/superstore_sample_delete.hyper"; 33 | copy(pathToSourceDatabase, pathToDatabase); 34 | 35 | // Starts the Hyper Process with telemetry enabled to send data to Tableau. 36 | // To opt out, simply set telemetry=hyperapi::Telemetry::DoNotSendUsageDataToTableau. 37 | { 38 | hyperapi::HyperProcess hyper(hyperapi::Telemetry::SendUsageDataToTableau); 39 | 40 | // Connect to existing Hyper file "superstore_sample_delete.hyper". 41 | { 42 | hyperapi::Connection connection(hyper.getEndpoint(), pathToDatabase); 43 | 44 | std::cout << "Delete all rows from customer with the name 'Dennis Kane' from table " << hyperapi::escapeName("Orders") << "." << std::endl; 45 | // `executeCommand` executes a SQL statement and returns the impacted row count. 46 | int64_t rowCount = connection.executeCommand( 47 | "DELETE FROM " + hyperapi::escapeName("Orders") + " WHERE " + hyperapi::escapeName("Customer ID") + " = ANY(SELECT " + 48 | hyperapi::escapeName("Customer ID") + " FROM " + hyperapi::escapeName("Customer") + " WHERE " + hyperapi::escapeName("Customer Name") + " = " + 49 | hyperapi::escapeStringLiteral("Dennis Kane") + ")"); 50 | std::cout << "The number of deleted rows in table " << hyperapi::escapeName("Orders") << " is " << rowCount << "." << std::endl 51 | << std::endl; 52 | 53 | std::cout << "Delete all rows from customer with the name 'Dennis Kane' from table " << hyperapi::escapeName("Customer") << "." << std::endl; 54 | rowCount = connection.executeCommand( 55 | "DELETE FROM " + hyperapi::escapeName("Customer") + " WHERE " + hyperapi::escapeName("Customer Name") + " =" + 56 | hyperapi::escapeStringLiteral("Dennis Kane")); 57 | 58 | std::cout << "The number of deleted rows in table Customer is " << rowCount << "." << std::endl; 59 | } 60 | std::cout << "The connection to the Hyper file has been closed." << std::endl; 61 | } 62 | std::cout << "The Hyper Process has been shut down." << std::endl; 63 | } 64 | 65 | int main() { 66 | try { 67 | runDeleteDataInExistingHyperFile(); 68 | } catch (const hyperapi::HyperException& e) { 69 | std::cout << e.toString() << std::endl; 70 | return 1; 71 | } 72 | return 0; 73 | } 74 | -------------------------------------------------------------------------------- /Tableau-Supported/CPP/insert_data_into_single_table.cpp: -------------------------------------------------------------------------------- 1 | /** 2 | * \example insert_data_into_single_table.cpp 3 | * 4 | * An example of how to create and insert into a single-table Hyper file with different column types. 5 | */ 6 | 7 | #include 8 | #include 9 | #include 10 | #include 11 | 12 | // The table is called "Extract" and will be created in the "Extract" schema. 13 | // This has historically been the default table name and schema for extracts created by Tableau. 14 | static const hyperapi::TableDefinition extractTable{ 15 | {"Extract", "Extract"}, 16 | {hyperapi::TableDefinition::Column{"Customer ID", hyperapi::SqlType::text(), hyperapi::Nullability::NotNullable}, 17 | hyperapi::TableDefinition::Column{"Customer Name", hyperapi::SqlType::text(), hyperapi::Nullability::NotNullable}, 18 | hyperapi::TableDefinition::Column{"Loyalty Reward Points", hyperapi::SqlType::bigInt(), hyperapi::Nullability::NotNullable}, 19 | hyperapi::TableDefinition::Column{"Segment", hyperapi::SqlType::text(), hyperapi::Nullability::NotNullable}}}; 20 | 21 | static void runInsertDataIntoSingleTable() { 22 | std::cout << "EXAMPLE - Insert data into a single table within a new Hyper file" << std::endl; 23 | const std::string pathToDatabase = "data/customer.hyper"; 24 | 25 | // Starts the Hyper Process with telemetry enabled to send data to Tableau. 26 | // To opt out, simply set telemetry=hyperapi::Telemetry::DoNotSendUsageDataToTableau. 27 | { 28 | hyperapi::HyperProcess hyper(hyperapi::Telemetry::SendUsageDataToTableau); 29 | // Creates new Hyper file "customer.hyper". 30 | // Replaces existing file with hyperapi::CreateMode::CreateAndReplace if it already exists. 31 | { 32 | hyperapi::Connection connection(hyper.getEndpoint(), pathToDatabase, hyperapi::CreateMode::CreateAndReplace); 33 | const hyperapi::Catalog& catalog = connection.getCatalog(); 34 | 35 | // Create the schema and the table. 36 | catalog.createSchema("Extract"); 37 | catalog.createTable(extractTable); 38 | 39 | // Insert data into the "Extract"."Extract" table. 40 | { 41 | hyperapi::Inserter inserter(connection, extractTable); 42 | inserter.addRow("DK-13375", "Dennis Kane", 518, "Consumer"); 43 | inserter.addRow("EB-13705", "Ed Braxton", 815, "Corporate"); 44 | inserter.execute(); 45 | } 46 | 47 | // Print the table names in the "Extract" schema. 48 | std::unordered_set tableNames = catalog.getTableNames("Extract"); 49 | std::cout << "Tables available in " << pathToDatabase << " in the Extract schema are: "; 50 | for (auto& tableName : tableNames) 51 | std::cout << tableName.toString() << "\t"; 52 | std::cout << std::endl; 53 | 54 | // Number of rows in the "Extract"."Extract" table. 55 | // `executeScalarQuery` is for executing a query that returns exactly one row with one column. 56 | int64_t rowCount = connection.executeScalarQuery("SELECT COUNT(*) FROM " + extractTable.getTableName().toString()); 57 | std::cout << "The number of rows in table " << extractTable.getTableName() << " is " << rowCount << "." << std::endl; 58 | } 59 | std::cout << "The connection to the Hyper file has been closed." << std::endl; 60 | } 61 | std::cout << "The Hyper Process has been shut down." << std::endl; 62 | } 63 | 64 | int main() { 65 | try { 66 | runInsertDataIntoSingleTable(); 67 | } catch (const hyperapi::HyperException& e) { 68 | std::cout << e.toString() << std::endl; 69 | return 1; 70 | } 71 | return 0; 72 | } 73 | -------------------------------------------------------------------------------- /Tableau-Supported/CPP/read_and_print_data_from_existing_hyper_file.cpp: -------------------------------------------------------------------------------- 1 | /** 2 | * \example read_and_print_data_from_existing_hyper_file.cpp 3 | * 4 | * An example of how to read and print data from an existing Hyper file. 5 | */ 6 | 7 | #include 8 | #include 9 | #include 10 | #include 11 | #include 12 | 13 | /** 14 | * Helper function to copy a file 15 | */ 16 | static void copy(const std::string& sourcePath, const std::string& destinationPath) { 17 | std::ifstream source(sourcePath, std::ios::binary); 18 | std::ofstream destination(destinationPath, std::ios::binary); 19 | destination << source.rdbuf(); 20 | source.close(); 21 | destination.close(); 22 | } 23 | 24 | static void runReadAndPrintDataFromExistingHyperFile() { 25 | std::cout << "EXAMPLE - Read data from an existing Hyper file" << std::endl; 26 | 27 | // Path to a Hyper file containing all data inserted into "Extract"."Extract" table 28 | // See "insert_data_into_single_table.cpp" for an example that works with the complete schema. 29 | const std::string pathToSourceDatabase = "data/superstore_sample_denormalized.hyper"; 30 | 31 | // Make a copy of the superstore example Hyper file. 32 | const std::string pathToDatabase = "data/superstore_sample_denormalized_read.hyper"; 33 | copy(pathToSourceDatabase, pathToDatabase); 34 | 35 | // Starts the Hyper Process with telemetry enabled to send data to Tableau. 36 | // To opt out, simply set telemetry=hyperapi::Telemetry::DoNotSendUsageDataToTableau. 37 | { 38 | hyperapi::HyperProcess hyper(hyperapi::Telemetry::SendUsageDataToTableau); 39 | 40 | // Connect to existing Hyper file "superstore_sample_denormalized_read.hyper". 41 | { 42 | hyperapi::Connection connection(hyper.getEndpoint(), pathToDatabase); 43 | const hyperapi::Catalog& catalog = connection.getCatalog(); 44 | 45 | // The table names in the "Extract" schema. 46 | std::unordered_set tableNames = catalog.getTableNames("Extract"); 47 | for (auto& tableName : tableNames) { 48 | hyperapi::TableDefinition tableDefinition = catalog.getTableDefinition(tableName); 49 | std::cout << "Table " << tableName << " has qualified name: " << tableDefinition.getTableName() << std::endl; 50 | for (auto& column : tableDefinition.getColumns()) { 51 | std::cout << "\t Column " << column.getName() << " has type " << column.getType() << " and nullability " << column.getNullability() 52 | << std::endl; 53 | } 54 | std::cout << std::endl; 55 | } 56 | 57 | // Print all rows from the "Extract"."Extract" table. 58 | hyperapi::TableName extractTable("Extract", "Extract"); 59 | std::cout << "These are all rows in the table " << extractTable.toString() << ":" << std::endl; 60 | 61 | hyperapi::Result rowsInTable = connection.executeQuery("SELECT * FROM " + extractTable.toString()); 62 | for (const hyperapi::Row& row : rowsInTable) { 63 | for (const hyperapi::Value& value : row) { 64 | std::cout << value << '\t'; 65 | } 66 | std::cout << '\n'; 67 | } 68 | } 69 | std::cout << "The connection to the Hyper file has been closed." << std::endl; 70 | } 71 | std::cout << "The Hyper Process has been shut down." << std::endl; 72 | } 73 | 74 | int main() { 75 | try { 76 | runReadAndPrintDataFromExistingHyperFile(); 77 | } catch (const hyperapi::HyperException& e) { 78 | std::cout << e.toString() << std::endl; 79 | return 1; 80 | } 81 | return 0; 82 | } 83 | -------------------------------------------------------------------------------- /Tableau-Supported/CPP/update_data_in_existing_hyper_file.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | 6 | /** 7 | * \example update_data_in_existing_hyper_file.cpp 8 | * 9 | * An example of how to update data in an existing Hyper file. 10 | */ 11 | 12 | /** 13 | * Helper function to copy a file 14 | */ 15 | static void copy(const std::string& sourcePath, const std::string& destinationPath) { 16 | std::ifstream source(sourcePath, std::ios::binary); 17 | std::ofstream destination(destinationPath, std::ios::binary); 18 | destination << source.rdbuf(); 19 | source.close(); 20 | destination.close(); 21 | } 22 | 23 | static void runUpdateDataInExistingHyperFile() { 24 | std::cout << "EXAMPLE - Update existing data in a Hyper file" << std::endl; 25 | 26 | // Path to a Hyper file containing all data inserted into Customer, Product, Orders and LineItems table 27 | // See "insert_data_into_multiple_tables.cpp" for an example that works with the complete schema. 28 | const std::string pathToSourceDatabase = "data/superstore_sample.hyper"; 29 | 30 | // Make a copy of the superstore example Hyper file. 31 | const std::string pathToDatabase = "data/superstore_sample_update.hyper"; 32 | copy(pathToSourceDatabase, pathToDatabase); 33 | 34 | // Starts the Hyper Process with telemetry enabled to send data to Tableau. 35 | // To opt out, simply set telemetry=hyperapi::Telemetry::DoNotSendUsageDataToTableau. 36 | { 37 | hyperapi::HyperProcess hyper(hyperapi::Telemetry::SendUsageDataToTableau); 38 | 39 | // Connect to existing Hyper file "superstore_sample_update.hyper". 40 | { 41 | hyperapi::Connection connection(hyper.getEndpoint(), pathToDatabase); 42 | 43 | hyperapi::Result rowsPreUpdate = connection.executeQuery( 44 | "SELECT " + hyperapi::escapeName("Loyalty Reward Points") + ", " + hyperapi::escapeName("Segment") + " FROM " + hyperapi::escapeName("Customer")); 45 | std::cout << "Pre-Update: Individual rows showing 'Loyalty Reward Points' and 'Segment' columns: " << std::endl; 46 | for (const hyperapi::Row& row : rowsPreUpdate) { 47 | for (const hyperapi::Value& value : row) { 48 | std::cout << value << '\t'; 49 | } 50 | std::cout << '\n'; 51 | } 52 | std::cout << std::endl; 53 | 54 | std::cout << "Update 'Customers' table by adding 50 Loyalty Reward Points to all Corporate Customers." << std::endl; 55 | int64_t rowCount = connection.executeCommand( 56 | "UPDATE " + hyperapi::escapeName("Customer") + " SET " + hyperapi::escapeName("Loyalty Reward Points") + " = " + 57 | hyperapi::escapeName("Loyalty Reward Points") + " + 50 " + "WHERE " + hyperapi::escapeName("Segment") + " = " + 58 | hyperapi::escapeStringLiteral("Corporate")); 59 | 60 | std::cout << "The number of updated rows in table " << hyperapi::escapeName("Customer") << " is " << rowCount << "." << std::endl; 61 | 62 | hyperapi::Result rowsPostUpdate = connection.executeQuery( 63 | "SELECT " + hyperapi::escapeName("Loyalty Reward Points") + ", " + hyperapi::escapeName("Segment") + " FROM " + hyperapi::escapeName("Customer")); 64 | for (const hyperapi::Row& row : rowsPostUpdate) { 65 | for (const hyperapi::Value& value : row) { 66 | std::cout << value << '\t'; 67 | } 68 | std::cout << '\n'; 69 | } 70 | } 71 | std::cout << "The connection to the Hyper file has been closed." << std::endl; 72 | } 73 | std::cout << "The Hyper Process has been shut down." << std::endl; 74 | } 75 | 76 | int main() { 77 | try { 78 | runUpdateDataInExistingHyperFile(); 79 | } catch (const hyperapi::HyperException& e) { 80 | std::cout << e.toString() << std::endl; 81 | return 1; 82 | } 83 | return 0; 84 | } 85 | -------------------------------------------------------------------------------- /Tableau-Supported/DotNet/CreateHyperFileFromCsv.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using System.IO; 3 | using System.Collections.Generic; 4 | 5 | using Tableau.HyperAPI; 6 | 7 | namespace Example 8 | { 9 | internal class CreateHyperFileFromCsv : Example 10 | { 11 | /// 12 | /// Create a new Hyper file with a single table and load data from a CSV file into it. 13 | /// For more details, see https://tableau.github.io/hyper-db/docs/guides/hyper_file/insert_csv 14 | /// 15 | /// Path to the directory with example data. 16 | public override void Execute(string exampleDataDir) 17 | { 18 | Console.WriteLine("EXAMPLE - Load data from a CSV file into a table in a new Hyper file."); 19 | 20 | // Optional process parameters. They are documented in the Tableau Hyper documentation, chapter "Process Settings" 21 | // (https://tableau.github.io/hyper-db/docs/hyper-api/hyper_process#process-settings). 22 | var processParameters = new Dictionary 23 | { 24 | // Limits the number of Hyper event log files to two. 25 | { "log_file_max_count", "2" }, 26 | // Limits the size of Hyper event log files to 100 megabytes. 27 | { "log_file_size_limit", "100M" } 28 | }; 29 | 30 | // Start the Hyper process with telemetry enabled. 31 | using (HyperProcess hyper = new HyperProcess(Telemetry.SendUsageDataToTableau, "example", processParameters)) 32 | { 33 | // Optional connection parameters. They are documented in the Tableau Hyper documentation, chapter "Connection Settings" 34 | // (https://tableau.github.io/hyper-db/docs/hyper-api/connection#connection-settings). 35 | var connectionParameters = new Dictionary 36 | { 37 | { "lc_time", "en_US" } 38 | }; 39 | 40 | // Connect to Hyper and create new Hyper file "customer.hyper". 41 | // It replaces the file if it already exists when CreateMode.CreateAndReplace is set. 42 | using (Connection connection = new Connection(hyper.Endpoint, "customer.hyper", CreateMode.CreateAndReplace, connectionParameters)) 43 | { 44 | // Table definition - its name and the list of columns. 45 | // Since the table name is not prefixed with an explicit schema name, the table will reside in the default "public" namespace. 46 | TableDefinition customerTable = new TableDefinition("Customer") 47 | .AddColumn("Customer ID", SqlType.Text(), Nullability.NotNullable) 48 | .AddColumn("Customer Name", SqlType.Text(), Nullability.NotNullable) 49 | .AddColumn("Loyalty Reward Points", SqlType.BigInt(), Nullability.NotNullable) 50 | .AddColumn("Segment", SqlType.Text(), Nullability.NotNullable); 51 | 52 | // Create the table in the Hyper file. 53 | connection.Catalog.CreateTable(customerTable); 54 | 55 | string pathToCsv = Path.Join(exampleDataDir, "customers.csv"); 56 | 57 | // Load all rows into "Customers" table from the CSV file. 58 | // ExecuteCommand executes a SQL statement and returns the impacted row count. 59 | // TableDefinition.Name property is a QualifiedName object which is escaped properly when 60 | // converted to a string; but the path to the CSV file needs to be escaped. 61 | // 62 | // Note: 63 | // You might have to adjust the COPY parameters to the format of your specific csv file. 64 | // The example assumes that your columns are separated with the ',' character 65 | // and that NULL values are encoded via the string 'NULL'. 66 | // Also be aware that the `header` option is used in this example: 67 | // It treats the first line of the csv file as a header and does not import it. 68 | // 69 | // The parameters of the COPY command are documented in the Tableau Hyper SQL documentation 70 | // (https://tableau.github.io/hyper-db/docs/sql/command/copy_from). 71 | Console.WriteLine("Issuing the SQL COPY command to load the csv file into the table. Since the first line"); 72 | Console.WriteLine("of our csv file contains the column names, we use the `header` option to skip it."); 73 | int countInCustomerTable = connection.ExecuteCommand( 74 | $"COPY {customerTable.TableName} from {Sql.EscapeStringLiteral(pathToCsv)} with " + 75 | $"(format csv, NULL 'NULL', delimiter ',', header)"); 76 | 77 | Console.WriteLine($"The number of rows in table {customerTable.TableName} is {countInCustomerTable}"); 78 | } 79 | 80 | Console.WriteLine("The connection to the Hyper file has been closed."); 81 | } 82 | 83 | Console.WriteLine("The Hyper process has been shut down."); 84 | } 85 | } 86 | } 87 | -------------------------------------------------------------------------------- /Tableau-Supported/DotNet/DeleteDataInExistingHyperFile.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using System.Collections.Generic; 3 | using System.Text; 4 | using System.IO; 5 | 6 | using Tableau.HyperAPI; 7 | 8 | namespace Example 9 | { 10 | internal class DeleteDataInExistingHyperFile : Example 11 | { 12 | /// 13 | /// Open a Hyper file and delete some data from it. 14 | /// 15 | /// Path to the directory with example data. 16 | public override void Execute(string dataDir) 17 | { 18 | Console.WriteLine("EXAMPLE - Delete data from an existing Hyper file."); 19 | 20 | // Hyper file containing data in Customer, Product, Orders and LineItems tables. 21 | string sourceDatabase = Path.Join(dataDir, "superstore_sample.hyper"); 22 | 23 | // Make a copy of the Hyper file to modify. 24 | string database = "superstore_sample_delete.hyper"; 25 | File.Copy(sourceDatabase, database, true); 26 | 27 | // Start Hyper process with telemetry enabled. 28 | using (HyperProcess hyper = new HyperProcess(Telemetry.SendUsageDataToTableau)) 29 | { 30 | // Connect to the Hyper file. 31 | using (Connection connection = new Connection(hyper.Endpoint, database)) 32 | { 33 | Console.WriteLine("Delete all rows from customer with the name 'Dennis Kane' from table Orders"); 34 | // ExecuteCommand executes a SQL statement and returns the impacted row count 35 | int deletedRowCount = connection.ExecuteCommand( 36 | $"DELETE FROM {Sql.EscapeName("Orders")} " + 37 | $"WHERE {Sql.EscapeName("Customer ID")} = ANY(" + 38 | $"SELECT {Sql.EscapeName("Customer ID")} FROM {Sql.EscapeName("Customer")} " + 39 | $"WHERE {Sql.EscapeName("Customer Name")} = {Sql.EscapeStringLiteral("Dennis Kane")})"); 40 | Console.WriteLine($"The number of deleted rows in table Orders is {deletedRowCount}\n"); 41 | 42 | Console.WriteLine("Delete all rows from customer with the name 'Dennis Kane' from table Customer"); 43 | deletedRowCount = connection.ExecuteCommand( 44 | $"DELETE FROM {Sql.EscapeName("Customer")} " + 45 | $"WHERE {Sql.EscapeName("Customer Name")} = {Sql.EscapeStringLiteral("Dennis Kane")}"); 46 | Console.WriteLine($"The number of deleted rows in table Customer is {deletedRowCount}\n"); 47 | } 48 | 49 | Console.WriteLine("The connection to the Hyper file has been closed."); 50 | } 51 | 52 | Console.WriteLine("The Hyper process has been shut down."); 53 | } 54 | } 55 | } 56 | -------------------------------------------------------------------------------- /Tableau-Supported/DotNet/Example.csproj: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | Exe 5 | netcoreapp3.1 6 | true 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | ../lib/Tableau.HyperAPI.NET.dll 16 | 17 | 18 | 19 | 20 | -------------------------------------------------------------------------------- /Tableau-Supported/DotNet/InsertDataIntoSingleTable.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | 3 | using Tableau.HyperAPI; 4 | 5 | namespace Example 6 | { 7 | internal class InsertDataIntoSingleTable : Example 8 | { 9 | /// 10 | /// Create a new Hyper file with a single table and write some data into it. 11 | /// 12 | /// Path to the directory with example data. 13 | public override void Execute(string exampleDataDir) 14 | { 15 | Console.WriteLine("EXAMPLE - Insert data into a single tables within a new Hyper file."); 16 | 17 | // Start the Hyper process with telemetry enabled. 18 | using (HyperProcess hyper = new HyperProcess(Telemetry.SendUsageDataToTableau)) 19 | { 20 | // Connect to hyper and create new Hyper file "superstore.hyper". 21 | // Replaces file if it already exists when CreateMode.CreateAndReplace is set. 22 | using (Connection connection = new Connection(hyper.Endpoint, "superstore.hyper", CreateMode.CreateAndReplace)) 23 | { 24 | // The table is called "Extract" and will be created in the "Extract" schema. 25 | // This has historically been the default table name and schema for extracts created by Tableau. 26 | TableName extractTable = new TableName("Extract", "Extract"); 27 | TableDefinition extractTableDefinition = new TableDefinition(extractTable) 28 | .AddColumn("Customer ID", SqlType.Text(), Nullability.NotNullable) 29 | .AddColumn("Customer Name", SqlType.Text(), Nullability.NotNullable) 30 | .AddColumn("Loyalty Reward Points", SqlType.BigInt(), Nullability.NotNullable) 31 | .AddColumn("Segment", SqlType.Text(), Nullability.NotNullable); 32 | 33 | // Create the schema and the table 34 | connection.Catalog.CreateSchema("Extract"); 35 | connection.Catalog.CreateTable(extractTableDefinition); 36 | 37 | // Insert data into the "Extract"."Extract" table 38 | using (Inserter inserter = new Inserter(connection, extractTable)) 39 | { 40 | inserter.AddRow("DK-13375", "Dennis Kane", 518, "Consumer"); 41 | inserter.AddRow("EB-13705", "Ed Braxton", 815, "Corporate"); 42 | inserter.Execute(); 43 | } 44 | 45 | // ExecuteScalarQuery is for executing a query that returns exactly one row with one column 46 | long count = connection.ExecuteScalarQuery($"SELECT COUNT(*) FROM {extractTable}"); 47 | Console.WriteLine($"Table {extractTable} has a count of {count} rows"); 48 | } 49 | 50 | Console.WriteLine("The connection to the Hyper file has been closed."); 51 | } 52 | 53 | Console.WriteLine("The Hyper process has been shut down."); 54 | } 55 | } 56 | } 57 | -------------------------------------------------------------------------------- /Tableau-Supported/DotNet/Program.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using System.IO; 3 | 4 | using Microsoft.Extensions.CommandLineUtils; 5 | 6 | using Tableau.HyperAPI; 7 | 8 | namespace Example 9 | { 10 | public abstract class Example 11 | { 12 | public abstract void Execute(string dataDir); 13 | } 14 | 15 | public class Program 16 | { 17 | private static void FindDataDir(ref string dataDir) 18 | { 19 | if (string.IsNullOrEmpty(dataDir)) 20 | { 21 | dataDir = "data"; 22 | if (!Directory.Exists(dataDir)) 23 | { 24 | Console.WriteLine("could not find example data directory, use --data-dir to specify it"); 25 | Environment.Exit(1); 26 | } 27 | } 28 | } 29 | 30 | private static void AddExample(CommandLineApplication app, string name, string description, Example example, CommandOption dataDirOption) 31 | { 32 | app.Command(name, (cmd) => 33 | { 34 | cmd.Description = description; 35 | cmd.HelpOption("-?|-h|--help"); 36 | cmd.OnExecute(() => 37 | { 38 | string dataDir = dataDirOption.Value(); 39 | FindDataDir(ref dataDir); 40 | 41 | try 42 | { 43 | example.Execute(dataDir); 44 | } 45 | catch (Exception ex) 46 | { 47 | Console.WriteLine(ex); 48 | return 1; 49 | } 50 | 51 | return 0; 52 | }); 53 | }); 54 | } 55 | 56 | public static void Main(string[] args) 57 | { 58 | CommandLineApplication app = new CommandLineApplication(); 59 | app.Description = "Extract API Example Application"; 60 | app.HelpOption("-? | -h | --help"); 61 | 62 | CommandOption dataDir = app.Option("--data-dir", "Path to the directory with the example data", CommandOptionType.SingleValue); 63 | 64 | AddExample(app, "create-extract-from-csv", "Load data from a CSV into a new Hyper file.", new CreateHyperFileFromCsv(), dataDir); 65 | AddExample(app, "delete-data-in-extract", "Delete data in an existing Hyper file.", new DeleteDataInExistingHyperFile(), dataDir); 66 | AddExample(app, "insert-data-into-multiple-tables", "Insert data into multiple tables.", new InsertDataIntoMultipleTables(), dataDir); 67 | AddExample(app, "insert-data-into-single-table", "Insert data into a single tables.", new InsertDataIntoSingleTable(), dataDir); 68 | AddExample(app, "insert-data-with-expressions", "Push down computations to Hyper during Insertion using expressions.", new InsertDataWithExpressions(), dataDir); 69 | AddExample(app, "insert-spatial-data-to-a-hyper-file", "Insert spatial data into a hyper file.", new InsertSpatialDataToAHyperFile(), dataDir); 70 | AddExample(app, "read-data-from-existing-extract", "Read data from an existing Hyper file.", new ReadDataFromExistingHyperFile(), dataDir); 71 | AddExample(app, "update-data-in-existing-extract", "Update data in an existing Hyper file.", new UpdateDataInExistingHyperFile(), dataDir); 72 | 73 | Environment.Exit(app.Execute(args)); 74 | } 75 | } 76 | } 77 | -------------------------------------------------------------------------------- /Tableau-Supported/DotNet/ReadDataFromExistingHyperFile.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using System.IO; 3 | 4 | using Tableau.HyperAPI; 5 | 6 | namespace Example 7 | { 8 | internal class ReadDataFromExistingHyperFile : Example 9 | { 10 | /// 11 | /// Open a Hyper file and read data from it. 12 | /// 13 | /// Path to the directory with example data. 14 | public override void Execute(string exampleDataDir) 15 | { 16 | Console.WriteLine("EXAMPLE - Read data from an existing Hyper file."); 17 | 18 | // Start the Hyper process with telemetry enabled. 19 | using (HyperProcess hyper = new HyperProcess(Telemetry.SendUsageDataToTableau)) 20 | { 21 | // Connect to the Hyper file. 22 | using (Connection connection = new Connection(hyper.Endpoint, Path.Join(exampleDataDir, "superstore_sample_denormalized.hyper"))) 23 | { 24 | // Get all tables in the "Extract" schema of the Hyper file 25 | foreach (TableName table in connection.Catalog.GetTableNames("Extract")) 26 | { 27 | TableDefinition tableDef = connection.Catalog.GetTableDefinition(table); 28 | Console.WriteLine($"Table {table.Name} has qualified name: {tableDef.TableName}"); 29 | // Get all the columns in the table. 30 | foreach (TableDefinition.Column column in tableDef.Columns) 31 | { 32 | Console.WriteLine($"Column {column.Name} has type={column.Type} and nullabilty={column.Nullability}"); 33 | } 34 | } 35 | 36 | // Print all rows from the "Extract"."Extract" table. 37 | TableName tableName = new TableName("Extract", "Extract"); 38 | Console.WriteLine($"These are all rows in the table {tableName}"); 39 | using (Result result = connection.ExecuteQuery($"SELECT * FROM {tableName}")) 40 | { 41 | while (result.NextRow()) 42 | { 43 | Console.WriteLine($"[{string.Join(", ", result.GetValues())}]"); 44 | } 45 | } 46 | } 47 | 48 | Console.WriteLine("The connection to the Hyper file has been closed."); 49 | } 50 | 51 | Console.WriteLine("The Hyper process has been shut down."); 52 | } 53 | } 54 | } 55 | -------------------------------------------------------------------------------- /Tableau-Supported/DotNet/UpdateDataInExistingHyperFile.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using System.IO; 3 | 4 | using Tableau.HyperAPI; 5 | 6 | namespace Example 7 | { 8 | internal class UpdateDataInExistingHyperFile : Example 9 | { 10 | /// 11 | /// Open a Hyper file and modify some data in it. 12 | /// 13 | /// Path to the directory with example data. 14 | public override void Execute(string exampleDataDir) 15 | { 16 | Console.WriteLine("EXAMPLE - Update data in an existing Hyper file."); 17 | 18 | // Start the Hyper process with telemetry enabled. 19 | using (HyperProcess hyper = new HyperProcess(Telemetry.SendUsageDataToTableau)) 20 | { 21 | // Make a copy of the Hyper file to modify. 22 | string sourceDatabase = Path.Join(exampleDataDir, "superstore_sample.hyper"); 23 | string database = "superstore_sample_update.hyper"; 24 | File.Copy(sourceDatabase, database, true); 25 | 26 | // Connect to the Hyper file. 27 | using (Connection connection = new Connection(hyper.Endpoint, database)) 28 | { 29 | Console.WriteLine("Pre-Update: Individual rows showing 'Segment' and 'Loyalty Reward Points':"); 30 | using (Result result = connection.ExecuteQuery( 31 | $"SELECT {Sql.EscapeName("Loyalty Reward Points")}, {Sql.EscapeName("Segment")}" + 32 | $"FROM {Sql.EscapeName("Customer")}")) 33 | { 34 | while (result.NextRow()) 35 | { 36 | Console.WriteLine($"[{string.Join(", ", result.GetValues())}]"); 37 | } 38 | } 39 | 40 | Console.WriteLine("Update 'Customers' table by adding 50 Loyalty Reward Points to all Corporate Customers"); 41 | int updatedRowCount = connection.ExecuteCommand( 42 | $"UPDATE {Sql.EscapeName("Customer")} " + 43 | $"SET {Sql.EscapeName("Loyalty Reward Points")} = {Sql.EscapeName("Loyalty Reward Points")} + 50 " + 44 | $"WHERE {Sql.EscapeName("Segment")} = {Sql.EscapeStringLiteral("Corporate")}"); 45 | 46 | Console.WriteLine($"The number of updated rows in 'Customer' table is {updatedRowCount}"); 47 | 48 | Console.WriteLine("Post-Update: Individual rows showing 'Segment' and 'Loyalty Reward Points':"); 49 | using (Result result = connection.ExecuteQuery( 50 | $"SELECT {Sql.EscapeName("Loyalty Reward Points")}, {Sql.EscapeName("Segment")}" + 51 | $"FROM {Sql.EscapeName("Customer")}")) 52 | { 53 | while (result.NextRow()) 54 | { 55 | Console.WriteLine($"[{string.Join(", ", result.GetValues())}]"); 56 | } 57 | } 58 | } 59 | 60 | Console.WriteLine("The connection to the Hyper file has been closed."); 61 | } 62 | 63 | Console.WriteLine("The Hyper process has been shut down."); 64 | } 65 | } 66 | } 67 | -------------------------------------------------------------------------------- /Tableau-Supported/DotNet/build.bat: -------------------------------------------------------------------------------- 1 | REM This script uses a local copy of the Hyper API package to build the example project. Use "build-nuget.bat" to download the package from nuget instead. 2 | SETLOCAL EnableDelayedExpansion 3 | dotnet build Example.csproj || exit /b !ERRORLEVEL! 4 | xcopy /Y ..\lib\tableauhyperapi.dll bin\Debug\netcoreapp3.1\ || exit /b !ERRORLEVEL! 5 | xcopy /E /Y ..\lib\hyper bin\Debug\netcoreapp3.1\hyper\ || exit /b !ERRORLEVEL! 6 | -------------------------------------------------------------------------------- /Tableau-Supported/DotNet/build.sh: -------------------------------------------------------------------------------- 1 | # This script uses a local copy of the Hyper API package to build the example project. Use "build-nuget.sh" to download the package from nuget instead. 2 | set -e 3 | dotnet build Example.csproj 4 | cp -R ../lib/hyper bin/Debug/netcoreapp3.1 5 | cp -R ../lib/libtableauhyperapi.* bin/Debug/netcoreapp3.1/ 6 | -------------------------------------------------------------------------------- /Tableau-Supported/DotNet/data/superstore_sample.hyper: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Tableau-Supported/DotNet/data/superstore_sample.hyper -------------------------------------------------------------------------------- /Tableau-Supported/DotNet/data/superstore_sample_denormalized.hyper: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Tableau-Supported/DotNet/data/superstore_sample_denormalized.hyper -------------------------------------------------------------------------------- /Tableau-Supported/Java/build.gradle: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Tableau-Supported/Java/build.gradle -------------------------------------------------------------------------------- /Tableau-Supported/Java/create-hyper-file-from-csv/build.gradle: -------------------------------------------------------------------------------- 1 | plugins { 2 | id 'java' 3 | id 'application' 4 | } 5 | 6 | repositories { 7 | mavenCentral() 8 | } 9 | 10 | application { 11 | mainClassName = 'examples.CreateHyperFileFromCSV' 12 | } 13 | 14 | dependencies { 15 | implementation fileTree(dir: '../../lib', include: '*.jar') 16 | } -------------------------------------------------------------------------------- /Tableau-Supported/Java/data/states.csv: -------------------------------------------------------------------------------- 1 | Population,State 2 | 4833722,Alabama 3 | 735132,Alaska 4 | 6626624,Arizona 5 | 2959373,Arkansas 6 | 38332521,California 7 | 5268367,Colorado 8 | 3596080,Connecticut 9 | 925749,Delaware 10 | 646449,District of Columbia 11 | 19552860,Florida 12 | 9992167,Georgia 13 | 1404054,Hawaii 14 | 1612136,Idaho 15 | 12882135,Illinois 16 | 6570902,Indiana 17 | 3090416,Iowa 18 | 2893957,Kansas 19 | 4395295,Kentucky 20 | 4625470,Louisiana 21 | 1328302,Maine 22 | 5928814,Maryland 23 | 6692824,Massachusetts 24 | 9895622,Michigan 25 | 5420380,Minnesota 26 | 2991207,Mississippi 27 | 6044171,Missouri 28 | 1015165,Montana 29 | 1868516,Nebraska 30 | 2790136,Nevada 31 | 1323459,New Hampshire 32 | 8899339,New Jersey 33 | 2085287,New Mexico 34 | 19651127,New York 35 | 9848060,North Carolina 36 | 723393,North Dakota 37 | 11570808,Ohio 38 | 3850568,Oklahoma 39 | 3930065,Oregon 40 | 12773801,Pennsylvania 41 | 1051511,Rhode Island 42 | 4774839,South Carolina 43 | 844877,South Dakota 44 | 6495978,Tennessee 45 | 26448193,Texas 46 | 2900872,Utah 47 | 626630,Vermont 48 | 8260405,Virginia 49 | 6971406,Washington 50 | 1854304,West Virginia 51 | 5742713,Wisconsin 52 | 582658,Wyoming 53 | -------------------------------------------------------------------------------- /Tableau-Supported/Java/data/superstore_sample.hyper: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Tableau-Supported/Java/data/superstore_sample.hyper -------------------------------------------------------------------------------- /Tableau-Supported/Java/data/superstore_sample_denormalized.hyper: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Tableau-Supported/Java/data/superstore_sample_denormalized.hyper -------------------------------------------------------------------------------- /Tableau-Supported/Java/delete-data-in-existing-hyper-file/build.gradle: -------------------------------------------------------------------------------- 1 | plugins { 2 | id 'java' 3 | id 'application' 4 | } 5 | 6 | repositories { 7 | mavenCentral() 8 | } 9 | 10 | application { 11 | mainClassName = 'examples.DeleteDataInExistingHyperFile' 12 | } 13 | 14 | dependencies { 15 | implementation fileTree(dir: '../../lib', include: '*.jar') 16 | } -------------------------------------------------------------------------------- /Tableau-Supported/Java/delete-data-in-existing-hyper-file/src/main/java/examples/DeleteDataInExistingHyperFile.java: -------------------------------------------------------------------------------- 1 | package examples; 2 | 3 | import com.tableau.hyperapi.Connection; 4 | import com.tableau.hyperapi.HyperProcess; 5 | import com.tableau.hyperapi.Telemetry; 6 | 7 | import java.io.IOException; 8 | import java.io.UncheckedIOException; 9 | import java.nio.file.Files; 10 | import java.nio.file.Path; 11 | import java.nio.file.Paths; 12 | import java.nio.file.StandardCopyOption; 13 | 14 | import static com.tableau.hyperapi.Sql.escapeName; 15 | import static com.tableau.hyperapi.Sql.escapeStringLiteral; 16 | 17 | /** 18 | * An example demonstrating deleting data from an existing Hyper file 19 | */ 20 | public class DeleteDataInExistingHyperFile { 21 | /** 22 | * The main function 23 | * 24 | * @param args The args 25 | */ 26 | public static void main(String[] args) { 27 | System.out.println("EXAMPLE - Delete data from an existing Hyper file\n"); 28 | 29 | // Path to an Hyper file containing all data inserted into Customer, Product, Orders and LineItems table 30 | Path pathToOriginalDatabase = resolveExampleFile("superstore_sample.hyper"); 31 | 32 | // Make a copy of the superstore example Hyper file 33 | Path pathToCopiedDatabase = Paths.get(getWorkingDirectory(), "superstore_sample_delete.hyper").toAbsolutePath(); 34 | try { 35 | Files.copy(pathToOriginalDatabase, pathToCopiedDatabase, StandardCopyOption.REPLACE_EXISTING); 36 | } catch (IOException e) { 37 | throw new UncheckedIOException(e); 38 | } 39 | 40 | // Starts the Hyper Process with telemetry enabled to send data to Tableau. 41 | // To opt out, simply set telemetry=Telemetry.DO_NOT_SEND_USAGE_DATA_TO_TABLEAU. 42 | try (HyperProcess process = new HyperProcess(Telemetry.SEND_USAGE_DATA_TO_TABLEAU)) { 43 | 44 | // Connect to existing Hyper file "superstore_sample_delete.hyper" 45 | try (Connection connection = new Connection(process.getEndpoint(), 46 | pathToCopiedDatabase.toString())) { 47 | 48 | System.out.println("Delete all rows from customer with the name 'Dennis Kane' from 'Orders' table"); 49 | 50 | // executeCommand executes a SQL statement and returns the impacted row count 51 | long rowCountDeletedInOrdersTable = connection.executeCommand( 52 | "DELETE FROM " + escapeName("Orders") + " WHERE " + escapeName("Customer ID") + 53 | "= ANY(" + 54 | "SELECT " + escapeName("Customer ID") + " FROM " + escapeName("Customer") + 55 | " WHERE " + escapeName("Customer Name") + "=" + escapeStringLiteral("Dennis Kane") + 56 | ")" 57 | ).getAsLong(); 58 | 59 | System.out.println("The number of deleted rows in table 'Orders' is " + rowCountDeletedInOrdersTable + "\n"); 60 | 61 | System.out.println("Delete all rows from customer with the name 'Dennis Kane' from 'Customer' table"); 62 | long rowCountDeletedInCustomersTable = connection.executeCommand( 63 | "DELETE FROM " + escapeName("Customer") + 64 | "WHERE " + escapeName("Customer Name") + "=" + escapeStringLiteral("Dennis Kane") 65 | ).getAsLong(); 66 | 67 | System.out.println("The number of deleted rows in table 'Customer' is " + rowCountDeletedInCustomersTable + "\n"); 68 | } 69 | System.out.println("The connection to the Hyper file has been closed"); 70 | } 71 | System.out.println("The Hyper process has been shut down"); 72 | } 73 | 74 | /** 75 | * Resolve the example file 76 | * 77 | * @param filename The filename 78 | * @return A path to the resolved file 79 | */ 80 | private static Path resolveExampleFile(String filename) { 81 | for (Path path = Paths.get(getWorkingDirectory()).toAbsolutePath(); path != null; path = path.getParent()) { 82 | Path file = path.resolve("data/" + filename); 83 | if (Files.isRegularFile(file)) { 84 | return file; 85 | } 86 | } 87 | throw new IllegalAccessError("Could not find example file. Check the working directory."); 88 | } 89 | 90 | /** 91 | * Returns the current working directory 92 | * 93 | * @return The inferred working directory 94 | */ 95 | private static String getWorkingDirectory() { 96 | return System.getProperty("user.dir"); 97 | } 98 | } 99 | -------------------------------------------------------------------------------- /Tableau-Supported/Java/gradle/wrapper/gradle-wrapper.jar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Tableau-Supported/Java/gradle/wrapper/gradle-wrapper.jar -------------------------------------------------------------------------------- /Tableau-Supported/Java/gradle/wrapper/gradle-wrapper.properties: -------------------------------------------------------------------------------- 1 | distributionBase=GRADLE_USER_HOME 2 | distributionPath=wrapper/dists 3 | distributionUrl=https\://services.gradle.org/distributions/gradle-5.6.4-bin.zip 4 | zipStoreBase=GRADLE_USER_HOME 5 | zipStorePath=wrapper/dists 6 | -------------------------------------------------------------------------------- /Tableau-Supported/Java/gradlew.bat: -------------------------------------------------------------------------------- 1 | @rem 2 | @rem Copyright 2015 the original author or authors. 3 | @rem 4 | @rem Licensed under the Apache License, Version 2.0 (the "License"); 5 | @rem you may not use this file except in compliance with the License. 6 | @rem You may obtain a copy of the License at 7 | @rem 8 | @rem https://www.apache.org/licenses/LICENSE-2.0 9 | @rem 10 | @rem Unless required by applicable law or agreed to in writing, software 11 | @rem distributed under the License is distributed on an "AS IS" BASIS, 12 | @rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | @rem See the License for the specific language governing permissions and 14 | @rem limitations under the License. 15 | @rem 16 | 17 | @if "%DEBUG%" == "" @echo off 18 | @rem ########################################################################## 19 | @rem 20 | @rem Gradle startup script for Windows 21 | @rem 22 | @rem ########################################################################## 23 | 24 | @rem Set local scope for the variables with windows NT shell 25 | if "%OS%"=="Windows_NT" setlocal 26 | 27 | set DIRNAME=%~dp0 28 | if "%DIRNAME%" == "" set DIRNAME=. 29 | set APP_BASE_NAME=%~n0 30 | set APP_HOME=%DIRNAME% 31 | 32 | @rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. 33 | set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m" 34 | 35 | @rem Find java.exe 36 | if defined JAVA_HOME goto findJavaFromJavaHome 37 | 38 | set JAVA_EXE=java.exe 39 | %JAVA_EXE% -version >NUL 2>&1 40 | if "%ERRORLEVEL%" == "0" goto init 41 | 42 | echo. 43 | echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 44 | echo. 45 | echo Please set the JAVA_HOME variable in your environment to match the 46 | echo location of your Java installation. 47 | 48 | goto fail 49 | 50 | :findJavaFromJavaHome 51 | set JAVA_HOME=%JAVA_HOME:"=% 52 | set JAVA_EXE=%JAVA_HOME%/bin/java.exe 53 | 54 | if exist "%JAVA_EXE%" goto init 55 | 56 | echo. 57 | echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 58 | echo. 59 | echo Please set the JAVA_HOME variable in your environment to match the 60 | echo location of your Java installation. 61 | 62 | goto fail 63 | 64 | :init 65 | @rem Get command-line arguments, handling Windows variants 66 | 67 | if not "%OS%" == "Windows_NT" goto win9xME_args 68 | 69 | :win9xME_args 70 | @rem Slurp the command line arguments. 71 | set CMD_LINE_ARGS= 72 | set _SKIP=2 73 | 74 | :win9xME_args_slurp 75 | if "x%~1" == "x" goto execute 76 | 77 | set CMD_LINE_ARGS=%* 78 | 79 | :execute 80 | @rem Setup the command line 81 | 82 | set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar 83 | 84 | @rem Execute Gradle 85 | "%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %CMD_LINE_ARGS% 86 | 87 | :end 88 | @rem End local scope for the variables with windows NT shell 89 | if "%ERRORLEVEL%"=="0" goto mainEnd 90 | 91 | :fail 92 | rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of 93 | rem the _cmd.exe /c_ return code! 94 | if not "" == "%GRADLE_EXIT_CONSOLE%" exit 1 95 | exit /b 1 96 | 97 | :mainEnd 98 | if "%OS%"=="Windows_NT" endlocal 99 | 100 | :omega 101 | -------------------------------------------------------------------------------- /Tableau-Supported/Java/insert-data-into-multiple-tables/build.gradle: -------------------------------------------------------------------------------- 1 | plugins { 2 | id 'java' 3 | id 'application' 4 | } 5 | 6 | repositories { 7 | mavenCentral() 8 | } 9 | 10 | application { 11 | mainClassName = 'examples.InsertDataIntoMultipleTables' 12 | } 13 | 14 | dependencies { 15 | implementation fileTree(dir: '../../lib', include: '*.jar') 16 | } -------------------------------------------------------------------------------- /Tableau-Supported/Java/insert-data-into-single-table/build.gradle: -------------------------------------------------------------------------------- 1 | plugins { 2 | id 'java' 3 | id 'application' 4 | } 5 | 6 | repositories { 7 | mavenCentral() 8 | } 9 | 10 | application { 11 | mainClassName = 'examples.InsertDataIntoSingleTable' 12 | } 13 | 14 | dependencies { 15 | implementation fileTree(dir: '../../lib', include: '*.jar') 16 | } -------------------------------------------------------------------------------- /Tableau-Supported/Java/insert-data-into-single-table/src/main/java/examples/InsertDataIntoSingleTable.java: -------------------------------------------------------------------------------- 1 | package examples; 2 | 3 | import com.tableau.hyperapi.*; 4 | 5 | import javax.xml.validation.Schema; 6 | import java.nio.file.Path; 7 | import java.nio.file.Paths; 8 | import java.util.List; 9 | 10 | import static com.tableau.hyperapi.Nullability.NOT_NULLABLE; 11 | 12 | /** 13 | * An example demonstrating a simple single-table Hyper file including table creation and data insertion with different types 14 | */ 15 | public class InsertDataIntoSingleTable { 16 | /** 17 | * The table is called "Extract" and will be created in the "Extract" schema. 18 | * This has historically been the default table name and schema for extracts created by Tableau 19 | */ 20 | private static TableDefinition EXTRACT_TABLE = new TableDefinition( 21 | new TableName("Extract", "Extract")) 22 | .addColumn("Customer ID", SqlType.text(), NOT_NULLABLE) 23 | .addColumn("Customer Name", SqlType.text(), NOT_NULLABLE) 24 | .addColumn("Loyalty Reward Points", SqlType.bigInt(), NOT_NULLABLE) 25 | .addColumn("Segment", SqlType.text(), NOT_NULLABLE); 26 | 27 | /** 28 | * The main function 29 | * 30 | * @param args The args 31 | */ 32 | public static void main(String[] args) { 33 | System.out.println("EXAMPLE - Insert data into a single table into a new Hyper file"); 34 | 35 | Path customerDatabasePath = Paths.get(getWorkingDirectory(), "customers.hyper"); 36 | 37 | // Starts the Hyper Process with telemetry enabled to send data to Tableau. 38 | // To opt out, simply set telemetry=Telemetry.DO_NOT_SEND_USAGE_DATA_TO_TABLEAU. 39 | try (HyperProcess process = new HyperProcess(Telemetry.SEND_USAGE_DATA_TO_TABLEAU)) { 40 | // Creates new Hyper file "customer.hyper" 41 | // Replaces file with CreateMode.CREATE_AND_REPLACE if it already exists 42 | try (Connection connection = new Connection(process.getEndpoint(), 43 | customerDatabasePath.toString(), 44 | CreateMode.CREATE_AND_REPLACE)) { 45 | Catalog catalog = connection.getCatalog(); 46 | 47 | catalog.createSchema(new SchemaName("Extract")); 48 | catalog.createTable(EXTRACT_TABLE); 49 | 50 | // Insert data into "Extract"."Extract" table 51 | try (Inserter inserter = new Inserter(connection, EXTRACT_TABLE)) { 52 | inserter.add("DK-13375").add("Dennis Kane").add(518).add("Consumer").endRow(); 53 | inserter.add("EB-13705").add("Ed Braxton").add(815).add("Corporate").endRow(); 54 | inserter.execute(); 55 | } 56 | 57 | // The table names in the "Extract" schema 58 | List tablesInDatabase = catalog.getTableNames(new SchemaName("Extract")); 59 | System.out.println("Tables available in " + customerDatabasePath + " are: " + tablesInDatabase); 60 | 61 | // Number of rows in the "Extract"."Extract" table 62 | // executeScalarQuery is for executing a query that returns exactly one row with one column 63 | long rowCount = connection.executeScalarQuery( 64 | "SELECT COUNT(*) FROM " + EXTRACT_TABLE.getTableName() 65 | ).get(); 66 | System.out.println("The number of rows in table " + EXTRACT_TABLE.getTableName() + " is " + rowCount + "\n"); 67 | } 68 | System.out.println("The connection to the Hyper file has been closed"); 69 | } 70 | System.out.println("The Hyper process has been shut down"); 71 | } 72 | 73 | /** 74 | * Returns the current working directory 75 | * 76 | * @return The inferred working directory 77 | */ 78 | private static String getWorkingDirectory() { 79 | return System.getProperty("user.dir"); 80 | } 81 | } 82 | -------------------------------------------------------------------------------- /Tableau-Supported/Java/insert-data-with-expressions/build.gradle: -------------------------------------------------------------------------------- 1 | plugins { 2 | id 'java' 3 | id 'application' 4 | } 5 | 6 | repositories { 7 | mavenCentral() 8 | } 9 | 10 | application { 11 | mainClassName = 'examples.InsertDataWithExpressions' 12 | } 13 | 14 | dependencies { 15 | implementation fileTree(dir: '../../lib', include: '*.jar') 16 | } -------------------------------------------------------------------------------- /Tableau-Supported/Java/insert-spatial-data-to-a-hyper-file/build.gradle: -------------------------------------------------------------------------------- 1 | plugins { 2 | id 'java' 3 | id 'application' 4 | } 5 | 6 | repositories { 7 | mavenCentral() 8 | } 9 | 10 | application { 11 | mainClassName = 'examples.InsertSpatialDataToAHyperFile' 12 | } 13 | 14 | dependencies { 15 | implementation fileTree(dir: '../../lib', include: '*.jar') 16 | } -------------------------------------------------------------------------------- /Tableau-Supported/Java/read-and-print-data-from-existing-hyper-file/build.gradle: -------------------------------------------------------------------------------- 1 | plugins { 2 | id 'java' 3 | id 'application' 4 | } 5 | 6 | repositories { 7 | mavenCentral() 8 | } 9 | 10 | application { 11 | mainClassName = 'examples.ReadAndPrintDataFromExistingHyperFile' 12 | } 13 | 14 | dependencies { 15 | implementation fileTree(dir: '../../lib', include: '*.jar') 16 | } -------------------------------------------------------------------------------- /Tableau-Supported/Java/settings.gradle: -------------------------------------------------------------------------------- 1 | rootProject.name = 'Hyper API examples' 2 | 3 | include(':create-hyper-file-from-csv') 4 | include(':delete-data-in-existing-hyper-file') 5 | include(':insert-data-into-multiple-tables') 6 | include(':insert-data-into-single-table') 7 | include(':insert-data-with-expressions') 8 | include(':insert-spatial-data-to-a-hyper-file') 9 | include(':read-and-print-data-from-existing-hyper-file') 10 | include(':update-data-in-existing-hyper-file') 11 | -------------------------------------------------------------------------------- /Tableau-Supported/Java/update-data-in-existing-hyper-file/build.gradle: -------------------------------------------------------------------------------- 1 | plugins { 2 | id 'java' 3 | id 'application' 4 | } 5 | 6 | repositories { 7 | mavenCentral() 8 | } 9 | 10 | application { 11 | mainClassName = 'examples.UpdateDataInExistingHyperFile' 12 | } 13 | 14 | dependencies { 15 | implementation fileTree(dir: '../../lib', include: '*.jar') 16 | } -------------------------------------------------------------------------------- /Tableau-Supported/Python/create_hyper_file_from_csv.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | 3 | from tableauhyperapi import HyperProcess, Telemetry, \ 4 | Connection, CreateMode, \ 5 | NOT_NULLABLE, NULLABLE, SqlType, TableDefinition, \ 6 | Inserter, \ 7 | escape_name, escape_string_literal, \ 8 | HyperException 9 | 10 | 11 | customer_table = TableDefinition( 12 | # Since the table name is not prefixed with an explicit schema name, the table will reside in the default "public" namespace. 13 | table_name="Customer", 14 | columns=[ 15 | TableDefinition.Column("Customer ID", SqlType.text(), NOT_NULLABLE), 16 | TableDefinition.Column("Customer Name", SqlType.text(), NOT_NULLABLE), 17 | TableDefinition.Column("Loyalty Reward Points", SqlType.big_int(), NOT_NULLABLE), 18 | TableDefinition.Column("Segment", SqlType.text(), NOT_NULLABLE) 19 | ] 20 | ) 21 | 22 | 23 | def run_create_hyper_file_from_csv(): 24 | """ 25 | An example demonstrating loading data from a csv into a new Hyper file 26 | For more details, see https://tableau.github.io/hyper-db/docs/guides/hyper_file/insert_csv 27 | """ 28 | print("EXAMPLE - Load data from CSV into table in new Hyper file") 29 | 30 | path_to_database = Path("customer.hyper") 31 | 32 | # Optional process parameters. 33 | # They are documented in the Tableau Hyper documentation, chapter "Process Settings" 34 | # (https://tableau.github.io/hyper-db/docs/hyper-api/hyper_process#process-settings). 35 | process_parameters = { 36 | # Limits the number of Hyper event log files to two. 37 | "log_file_max_count": "2", 38 | # Limits the size of Hyper event log files to 100 megabytes. 39 | "log_file_size_limit": "100M" 40 | } 41 | 42 | # Starts the Hyper Process with telemetry enabled to send data to Tableau. 43 | # To opt out, simply set telemetry=Telemetry.DO_NOT_SEND_USAGE_DATA_TO_TABLEAU. 44 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU, parameters=process_parameters) as hyper: 45 | 46 | # Optional connection parameters. 47 | # They are documented in the Tableau Hyper documentation, chapter "Connection Settings" 48 | # (https://tableau.github.io/hyper-db/docs/hyper-api/connection#connection-settings). 49 | connection_parameters = {"lc_time": "en_US"} 50 | 51 | # Creates new Hyper file "customer.hyper". 52 | # Replaces file with CreateMode.CREATE_AND_REPLACE if it already exists. 53 | with Connection(endpoint=hyper.endpoint, 54 | database=path_to_database, 55 | create_mode=CreateMode.CREATE_AND_REPLACE, 56 | parameters=connection_parameters) as connection: 57 | 58 | connection.catalog.create_table(table_definition=customer_table) 59 | 60 | # Using path to current file, create a path that locates CSV file packaged with these examples. 61 | path_to_csv = str(Path(__file__).parent / "data" / "customers.csv") 62 | 63 | # Load all rows into "Customers" table from the CSV file. 64 | # `execute_command` executes a SQL statement and returns the impacted row count. 65 | # 66 | # Note: 67 | # You might have to adjust the COPY parameters to the format of your specific csv file. 68 | # The example assumes that your columns are separated with the ',' character 69 | # and that NULL values are encoded via the string 'NULL'. 70 | # Also be aware that the `header` option is used in this example: 71 | # It treats the first line of the csv file as a header and does not import it. 72 | # 73 | # The parameters of the COPY command are documented in the Tableau Hyper SQL documentation 74 | # (https://tableau.github.io/hyper-db/docs/sql/command/copy_from). 75 | print("Issuing the SQL COPY command to load the csv file into the table. Since the first line") 76 | print("of our csv file contains the column names, we use the `header` option to skip it.") 77 | count_in_customer_table = connection.execute_command( 78 | command=f"COPY {customer_table.table_name} from {escape_string_literal(path_to_csv)} with " 79 | f"(format csv, NULL 'NULL', delimiter ',', header)") 80 | 81 | print(f"The number of rows in table {customer_table.table_name} is {count_in_customer_table}.") 82 | 83 | print("The connection to the Hyper file has been closed.") 84 | print("The Hyper process has been shut down.") 85 | 86 | 87 | if __name__ == '__main__': 88 | try: 89 | run_create_hyper_file_from_csv() 90 | except HyperException as ex: 91 | print(ex) 92 | exit(1) 93 | -------------------------------------------------------------------------------- /Tableau-Supported/Python/data/superstore_sample.hyper: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Tableau-Supported/Python/data/superstore_sample.hyper -------------------------------------------------------------------------------- /Tableau-Supported/Python/data/superstore_sample_denormalized.hyper: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tableau/hyper-api-samples/61d5e3a778fa6f6e27d72b0317c6f209d170e7ea/Tableau-Supported/Python/data/superstore_sample_denormalized.hyper -------------------------------------------------------------------------------- /Tableau-Supported/Python/delete_data_in_existing_hyper_file.py: -------------------------------------------------------------------------------- 1 | import shutil 2 | 3 | from pathlib import Path 4 | 5 | from tableauhyperapi import HyperProcess, Telemetry, \ 6 | Connection, CreateMode, \ 7 | NOT_NULLABLE, NULLABLE, SqlType, TableDefinition, \ 8 | Inserter, \ 9 | escape_name, escape_string_literal, \ 10 | HyperException 11 | 12 | 13 | def run_delete_data_in_existing_hyper_file(): 14 | """ 15 | An example of how to delete data in an existing Hyper file. 16 | """ 17 | print("EXAMPLE - Delete data from an existing Hyper file") 18 | 19 | # Path to a Hyper file containing all data inserted into Customer, Product, Orders and LineItems table. 20 | # See "insert_data_into_multiple_tables.py" for an example that works with the complete schema. 21 | path_to_source_database = Path(__file__).parent / "data" / "superstore_sample.hyper" 22 | 23 | # Make a copy of the superstore example Hyper file. 24 | path_to_database = Path(shutil.copy(path_to_source_database, "superstore_sample_delete.hyper")).resolve() 25 | 26 | # Starts the Hyper Process with telemetry enabled to send data to Tableau. 27 | # To opt out, simply set telemetry=Telemetry.DO_NOT_SEND_USAGE_DATA_TO_TABLEAU. 28 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 29 | 30 | # Connect to existing Hyper file "superstore_sample_delete.hyper". 31 | with Connection(endpoint=hyper.endpoint, 32 | database=path_to_database) as connection: 33 | 34 | print(f"Delete all rows from customer with the name 'Dennis Kane' from table {escape_name('Orders')}.") 35 | # `execute_command` executes a SQL statement and returns the impacted row count. 36 | row_count = connection.execute_command( 37 | command=f"DELETE FROM {escape_name('Orders')} " 38 | f"WHERE {escape_name('Customer ID')} = ANY(" 39 | f"SELECT {escape_name('Customer ID')} FROM {escape_name('Customer')} " 40 | f"WHERE {escape_name('Customer Name')} = {escape_string_literal('Dennis Kane')})") 41 | 42 | print(f"The number of deleted rows in table {escape_name('Orders')} " 43 | f"is {row_count}.\n") 44 | 45 | print(f"Delete all rows from customer with the name 'Dennis Kane' from table {escape_name('Customer')}.") 46 | row_count = connection.execute_command( 47 | command=f"DELETE FROM {escape_name('Customer')} " 48 | f"WHERE {escape_name('Customer Name')} = {escape_string_literal('Dennis Kane')}") 49 | 50 | print(f"The number of deleted rows in table Customer is {row_count}.") 51 | 52 | print("The connection to the Hyper file has been closed.") 53 | print("The Hyper process has been shut down.") 54 | 55 | 56 | if __name__ == '__main__': 57 | try: 58 | run_delete_data_in_existing_hyper_file() 59 | except HyperException as ex: 60 | print(ex) 61 | exit(1) 62 | -------------------------------------------------------------------------------- /Tableau-Supported/Python/insert_data_into_single_table.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | 3 | from tableauhyperapi import HyperProcess, Telemetry, \ 4 | Connection, CreateMode, \ 5 | NOT_NULLABLE, NULLABLE, SqlType, TableDefinition, \ 6 | Inserter, \ 7 | escape_name, escape_string_literal, \ 8 | TableName, \ 9 | HyperException 10 | 11 | # The table is called "Extract" and will be created in the "Extract" schema. 12 | # This has historically been the default table name and schema for extracts created by Tableau 13 | extract_table = TableDefinition( 14 | table_name=TableName("Extract", "Extract"), 15 | columns=[ 16 | TableDefinition.Column(name='Customer ID', type=SqlType.text(), nullability=NOT_NULLABLE), 17 | TableDefinition.Column(name='Customer Name', type=SqlType.text(), nullability=NOT_NULLABLE), 18 | TableDefinition.Column(name='Loyalty Reward Points', type=SqlType.big_int(), nullability=NOT_NULLABLE), 19 | TableDefinition.Column(name='Segment', type=SqlType.text(), nullability=NOT_NULLABLE) 20 | ] 21 | ) 22 | 23 | 24 | def run_insert_data_into_single_table(): 25 | """ 26 | An example demonstrating a simple single-table Hyper file including table creation and data insertion with different types 27 | """ 28 | print("EXAMPLE - Insert data into a single table within a new Hyper file") 29 | path_to_database = Path("customer.hyper") 30 | 31 | # Starts the Hyper Process with telemetry enabled to send data to Tableau. 32 | # To opt out, simply set telemetry=Telemetry.DO_NOT_SEND_USAGE_DATA_TO_TABLEAU. 33 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 34 | 35 | # Creates new Hyper file "customer.hyper". 36 | # Replaces file with CreateMode.CREATE_AND_REPLACE if it already exists. 37 | with Connection(endpoint=hyper.endpoint, 38 | database=path_to_database, 39 | create_mode=CreateMode.CREATE_AND_REPLACE) as connection: 40 | 41 | connection.catalog.create_schema(schema=extract_table.table_name.schema_name) 42 | connection.catalog.create_table(table_definition=extract_table) 43 | 44 | # The rows to insert into the "Extract"."Extract" table. 45 | data_to_insert = [ 46 | ["DK-13375", "Dennis Kane", 518, "Consumer"], 47 | ["EB-13705", "Ed Braxton", 815, "Corporate"] 48 | ] 49 | 50 | with Inserter(connection, extract_table) as inserter: 51 | inserter.add_rows(rows=data_to_insert) 52 | inserter.execute() 53 | 54 | # The table names in the "Extract" schema (the default schema). 55 | table_names = connection.catalog.get_table_names("Extract") 56 | print(f"Tables available in {path_to_database} are: {table_names}") 57 | 58 | # Number of rows in the "Extract"."Extract" table. 59 | # `execute_scalar_query` is for executing a query that returns exactly one row with one column. 60 | row_count = connection.execute_scalar_query(query=f"SELECT COUNT(*) FROM {extract_table.table_name}") 61 | print(f"The number of rows in table {extract_table.table_name} is {row_count}.") 62 | 63 | print("The connection to the Hyper file has been closed.") 64 | print("The Hyper process has been shut down.") 65 | 66 | 67 | if __name__ == '__main__': 68 | try: 69 | run_insert_data_into_single_table() 70 | except HyperException as ex: 71 | print(ex) 72 | exit(1) 73 | -------------------------------------------------------------------------------- /Tableau-Supported/Python/insert_spatial_data_to_a_hyper_file.py: -------------------------------------------------------------------------------- 1 | import shutil 2 | 3 | from pathlib import Path 4 | 5 | from tableauhyperapi import HyperProcess, Telemetry, \ 6 | Connection, CreateMode, \ 7 | NOT_NULLABLE, NULLABLE, SqlType, TableDefinition, \ 8 | Inserter, \ 9 | escape_name, escape_string_literal, \ 10 | TableName, Name, \ 11 | HyperException 12 | 13 | # The table is called "Extract" and will be created in the "Extract" schema. 14 | # This has historically been the default table name and schema for extracts created by Tableau 15 | extract_table = TableDefinition( 16 | table_name=TableName("Extract", "Extract"), 17 | columns=[ 18 | TableDefinition.Column(name='Name', type=SqlType.text(), nullability=NOT_NULLABLE), 19 | TableDefinition.Column(name='Location', type=SqlType.geography(), nullability=NOT_NULLABLE) 20 | ] 21 | ) 22 | 23 | 24 | def run_insert_spatial_data_to_a_hyper_file(): 25 | """ 26 | An example of how to add spatial data to a Hyper file. 27 | """ 28 | print("EXAMPLE - Add spatial data to a Hyper file ") 29 | path_to_database = Path("spatial_data.hyper") 30 | 31 | # Starts the Hyper Process with telemetry enabled to send data to Tableau. 32 | # To opt out, simply set telemetry=Telemetry.DO_NOT_SEND_USAGE_DATA_TO_TABLEAU. 33 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 34 | 35 | # Creates new Hyper file "spatial_data.hyper". 36 | # Replaces file with CreateMode.CREATE_AND_REPLACE if it already exists. 37 | with Connection(endpoint=hyper.endpoint, 38 | database=path_to_database, 39 | create_mode=CreateMode.CREATE_AND_REPLACE) as connection: 40 | 41 | connection.catalog.create_schema(schema=extract_table.table_name.schema_name) 42 | connection.catalog.create_table(table_definition=extract_table) 43 | 44 | # Hyper API's Inserter allows users to transform data during insertion. 45 | # To make use of data transformation during insertion, the inserter requires the following inputs 46 | # 1. The connection to the Hyper instance containing the table. 47 | # 2. The table name or table defintion into which data is inserted. 48 | # 3. List of Inserter.ColumnMapping. 49 | # This list informs the inserter how each column in the target table is tranformed. 50 | # The list must contain all the columns into which data is inserted. 51 | # "Inserter.ColumnMapping" maps a valid SQL expression (if any) to a column in the target table. 52 | # For example Inserter.ColumnMapping('target_column_name', f'{escape_name("colA")}*{escape_name("colB")}') 53 | # The column "target_column" contains the product of "colA" and "colB" after successful insertion. 54 | # SQL expression string is optional in Inserter.ColumnMapping. 55 | # For a column without any transformation only the column name is required. 56 | # For example Inserter.ColumnMapping('no_data_transformation_column') 57 | # 4. The Column Definition of all input values provided to the Inserter 58 | 59 | # Inserter definition contains the column definition for the values that are inserted 60 | # The data input has two text values Name and Location_as_text 61 | inserter_definition = [ 62 | TableDefinition.Column(name='Name', type=SqlType.text(), nullability=NOT_NULLABLE), 63 | TableDefinition.Column(name='Location_as_text', type=SqlType.text(), nullability=NOT_NULLABLE)] 64 | 65 | # Column 'Name' is inserted into "Extract"."Extract" as-is. 66 | # Column 'Location' in "Extract"."Extract" of geography type is computed from Column 'Location_as_text' of text type 67 | # using the expression 'CAST("Location_as_text") AS GEOGRAPHY'. 68 | # Inserter.ColumnMapping is used for mapping the CAST expression to Column 'Location'. 69 | column_mappings = [ 70 | 'Name', 71 | Inserter.ColumnMapping('Location', f'CAST({escape_name("Location_as_text")} AS GEOGRAPHY)') 72 | ] 73 | 74 | # Data to be inserted. 75 | data_to_insert = [ 76 | ['Seattle', "point(-122.338083 47.647528)"], 77 | ['Munich', "point(11.584329 48.139257)"] 78 | ] 79 | 80 | # Insert data into "Extract"."Extract" table with CAST expression. 81 | with Inserter(connection, extract_table, column_mappings, inserter_definition=inserter_definition) as inserter: 82 | inserter.add_rows(rows=data_to_insert) 83 | inserter.execute() 84 | print("The data was added to the table.") 85 | 86 | print("The connection to the Hyper file has been closed.") 87 | print("The Hyper process has been shut down.") 88 | 89 | 90 | if __name__ == '__main__': 91 | try: 92 | run_insert_spatial_data_to_a_hyper_file() 93 | except HyperException as ex: 94 | print(ex) 95 | exit(1) 96 | -------------------------------------------------------------------------------- /Tableau-Supported/Python/read_and_print_data_from_existing_hyper_file.py: -------------------------------------------------------------------------------- 1 | import shutil 2 | 3 | from pathlib import Path 4 | 5 | from tableauhyperapi import HyperProcess, Telemetry, \ 6 | Connection, CreateMode, \ 7 | NOT_NULLABLE, NULLABLE, SqlType, TableDefinition, \ 8 | Inserter, \ 9 | escape_name, escape_string_literal, \ 10 | TableName, \ 11 | HyperException 12 | 13 | 14 | def run_read_data_from_existing_hyper_file(): 15 | """ 16 | An example of how to read and print data from an existing Hyper file. 17 | """ 18 | print("EXAMPLE - Read data from an existing Hyper file") 19 | 20 | # Path to a Hyper file containing all data inserted into Customer, Product, Orders and LineItems table. 21 | # See "insert_data_into_multiple_tables.py" for an example that works with the complete schema. 22 | path_to_source_database = Path(__file__).parent / "data" / "superstore_sample_denormalized.hyper" 23 | 24 | # Make a copy of the superstore denormalized sample Hyper file 25 | path_to_database = Path(shutil.copy(src=path_to_source_database, 26 | dst="superstore_sample_denormalized_read.hyper")).resolve() 27 | 28 | # Starts the Hyper Process with telemetry enabled to send data to Tableau. 29 | # To opt out, simply set telemetry=Telemetry.DO_NOT_SEND_USAGE_DATA_TO_TABLEAU. 30 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 31 | 32 | # Connect to existing Hyper file "superstore_sample_denormalized_read.hyper". 33 | with Connection(endpoint=hyper.endpoint, 34 | database=path_to_database) as connection: 35 | # The table names in the "Extract" schema (the default schema). 36 | table_names = connection.catalog.get_table_names(schema="Extract") 37 | 38 | for table in table_names: 39 | table_definition = connection.catalog.get_table_definition(name=table) 40 | print(f"Table {table.name} has qualified name: {table}") 41 | for column in table_definition.columns: 42 | print(f"Column {column.name} has type={column.type} and nullability={column.nullability}") 43 | print("") 44 | 45 | # Print all rows from the "Extract"."Extract" table. 46 | table_name = TableName("Extract", "Extract") 47 | print(f"These are all rows in the table {table_name}:") 48 | # `execute_list_query` executes a SQL query and returns the result as list of rows of data, 49 | # each represented by a list of objects. 50 | rows_in_table = connection.execute_list_query(query=f"SELECT * FROM {table_name}") 51 | print(rows_in_table) 52 | 53 | print("The connection to the Hyper file has been closed.") 54 | print("The Hyper process has been shut down.") 55 | 56 | 57 | if __name__ == '__main__': 58 | try: 59 | run_read_data_from_existing_hyper_file() 60 | except HyperException as ex: 61 | print(ex) 62 | exit(1) 63 | -------------------------------------------------------------------------------- /Tableau-Supported/Python/update_data_in_existing_hyper_file.py: -------------------------------------------------------------------------------- 1 | import shutil 2 | 3 | from pathlib import Path 4 | 5 | from tableauhyperapi import HyperProcess, Telemetry, \ 6 | Connection, CreateMode, \ 7 | NOT_NULLABLE, NULLABLE, SqlType, TableDefinition, \ 8 | Inserter, \ 9 | escape_name, escape_string_literal, \ 10 | TableName, \ 11 | HyperException 12 | 13 | 14 | def run_update_data_in_existing_hyper_file(): 15 | """ 16 | An example of how to update data in an existing Hyper file. 17 | """ 18 | print("EXAMPLE - Update existing data in an Hyper file") 19 | 20 | # Path to a Hyper file containing all data inserted into Customer, Product, Orders and LineItems table. 21 | # See "insert_data_into_multiple_tables.py" for an example that works with the complete schema. 22 | path_to_source_database = Path(__file__).parent / "data" / "superstore_sample.hyper" 23 | 24 | # Make a copy of the superstore sample Hyper file. 25 | path_to_database = Path(shutil.copy(path_to_source_database, "superstore_sample_update.hyper")).resolve() 26 | 27 | # Starts the Hyper Process with telemetry enabled to send data to Tableau. 28 | # To opt out, simply set telemetry=Telemetry.DO_NOT_SEND_USAGE_DATA_TO_TABLEAU. 29 | with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper: 30 | 31 | # Connect to existing Hyper file "superstore_sample_update.hyper". 32 | with Connection(endpoint=hyper.endpoint, 33 | database=path_to_database) as connection: 34 | 35 | rows_pre_update = connection.execute_list_query( 36 | query=f"SELECT {escape_name('Loyalty Reward Points')}, {escape_name('Segment')}" 37 | f"FROM {escape_name('Customer')}") 38 | print(f"Pre-Update: Individual rows showing 'Loyalty Reward Points' and 'Segment' " 39 | f"columns: {rows_pre_update}\n") 40 | 41 | print("Update 'Customers' table by adding 50 Loyalty Reward Points to all Corporate Customers.") 42 | row_count = connection.execute_command( 43 | command=f"UPDATE {escape_name('Customer')} " 44 | f"SET {escape_name('Loyalty Reward Points')} = {escape_name('Loyalty Reward Points')} + 50 " 45 | f"WHERE {escape_name('Segment')} = {escape_string_literal('Corporate')}") 46 | 47 | print(f"The number of updated rows in table {escape_name('Customer')} is {row_count}") 48 | 49 | rows_post_update = connection.execute_list_query( 50 | query=f"SELECT {escape_name('Loyalty Reward Points')}, {escape_name('Segment')} " 51 | f"FROM {escape_name('Customer')}") 52 | print(f"Post-Update: Individual rows showing 'Loyalty Reward Points' and 'Segment' " 53 | f"columns: {rows_post_update}") 54 | 55 | print("The connection to the Hyper file has been closed.") 56 | print("The Hyper process has been shut down.") 57 | 58 | 59 | if __name__ == '__main__': 60 | try: 61 | run_update_data_in_existing_hyper_file() 62 | except HyperException as ex: 63 | print(ex) 64 | exit(1) 65 | -------------------------------------------------------------------------------- /Tableau-Supported/README.md: -------------------------------------------------------------------------------- 1 | # Tableau-Supported Samples 2 | [![Tableau Supported](https://img.shields.io/badge/Support%20Level-Tableau%20Supported-53bd92.svg)](https://www.tableau.com/support-levels-it-and-developer-tools) 3 | 4 | Each of these samples are officially supported, maintained, and tested by our development team. They are available in each supported languages of the Hyper API and may be updated with every new release. 5 | 6 |
7 | 8 | ## What do these samples do? 9 | There are currently eight samples that are available in each language supported: 10 | 11 | * __create_hyper_file_from_csv__ 12 | * Demonstrates how you can use the Hyper SQL COPY command to quickly populate a table in the `.hyper` file from the contents of a comma-separated value (CSV) file. This technique is the fastest way to bring data into an extract, as Hyper is reading data directly from the CSV file. 13 | 14 | * __delete_data_from_existing_hyper_file__ 15 | * Demonstrates how you can use the Hyper SQL DELETE command to remove data from a table in the `.hyper` file. 16 | 17 | * __insert_data_into_multiple_tables__ 18 | * This example shows how you can create and insert data into a `.hyper` file that contains multiple tables. Uses the Inserter class to add data to the tables and a Hyper SQL query to report the number of rows in the tables. 19 | 20 | * __insert_data_into_single_table__ 21 | * Demonstrates how you to add data to a single table named Extract that uses the Extract schema. Uses the Inserter class to add data to the table and a Hyper SQL query to report the number of rows in the table. 22 | 23 | * __read_and_print_data_from_existing_hyper_file__ 24 | * Demonstrates how to read data from an existing `.hyper` file and print the output to the console. 25 | 26 | * __update_data_in_existing_hyper_file__ 27 | * This example demonstrates how you can use the Hyper SQL UPDATE command to change values in a `.hyper` file. 28 | 29 | * __insert_spatial_data_to_a_hyper_file__ 30 | * This example demonstrates how to insert spatial data (WKT) to a `.hyper` file. 31 | 32 | * __insert_data_using_expressions__ 33 | * This example shows how you can use SQL expressions in Hyper API Inserter to transform or compute data on the fly during data insertion. 34 | 35 |
36 |
37 | 38 | ## How do I get help or give feedback? 39 | If you have questions, want to submit ideas, or give feedback on the code, please do so by submitting an issue on this project. 40 | --------------------------------------------------------------------------------