├── .gitignore ├── LICENSE ├── Makefile ├── README.md ├── cass_connection.c ├── cassandra2_fdw--2.0.sql ├── cassandra2_fdw.c ├── cassandra2_fdw.control ├── cassandra2_fdw.h └── test ├── README.md ├── build.cfg ├── cassandra.sh ├── cleanup.sh ├── postgresql.sh ├── setup_img.sh └── test.sh /.gitignore: -------------------------------------------------------------------------------- 1 | # Ignore file 2 | cass_connection.bc 3 | cass_connection.o 4 | cassandra2_fdw.bc 5 | cassandra2_fdw.o 6 | cassandra2_fdw.so 7 | .vscode/ 8 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 Jaimin Pan (jaimin.pan@gmail.com) 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of 6 | this software and associated documentation files (the "Software"), to deal in 7 | the Software without restriction, including without limitation the rights to 8 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 9 | the Software, and to permit persons to whom the Software is furnished to do so, 10 | subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 17 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 18 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 19 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 20 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | # contrib/cassandra2_fdw/Makefile 2 | 3 | MODULE_big = cassandra2_fdw 4 | OBJS = cassandra2_fdw.o cass_connection.o 5 | 6 | SHLIB_LINK = -lcassandra 7 | 8 | EXTENSION = cassandra2_fdw 9 | DATA = cassandra2_fdw--2.0.sql 10 | 11 | REGRESS = cassandra2_fdw 12 | 13 | ifdef USE_PGXS 14 | PG_CONFIG = pg_config 15 | PGXS := $(shell $(PG_CONFIG) --pgxs) 16 | include $(PGXS) 17 | else 18 | subdir = contrib/cassandra2_fdw 19 | top_builddir = ../.. 20 | include $(top_builddir)/src/Makefile.global 21 | include $(top_srcdir)/contrib/contrib-global.mk 22 | endif 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | cassandra2_fdw 2 | ============== 3 | [![Lang](https://img.shields.io/badge/Language-C%2FC%2B%2B-green.svg)]() 4 | [![MIT](https://img.shields.io/badge/License-MIT-green.svg)]() 5 | [![Extension](https://img.shields.io/badge/Extension-PostgreSQL-green.svg)]() 6 | 7 | Foreign Data Wrapper (FDW) that facilitates access to Cassandra 2.x, 3.x, and 4.x from within PostgreSQL. 8 | 9 | Only functions for reading data originated in Cassandra have been implemented. Writing data back to Cassandra is not supported. 10 | 11 | Cassandra: http://cassandra.apache.org/ 12 | 13 | __PostgreSQL Support__: 14 | [![version](https://img.shields.io/badge/PostgreSQL-11-blue.svg)]() 15 | [![version](https://img.shields.io/badge/PostgreSQL-12-blue.svg)]() 16 | [![version](https://img.shields.io/badge/PostgreSQL-13-blue.svg)]() 17 | [![version](https://img.shields.io/badge/PostgreSQL-14-blue.svg)]() 18 | [![version](https://img.shields.io/badge/PostgreSQL-15-blue.svg)]() 19 | 20 | __Cassandra Support__: 21 | [![version](https://img.shields.io/badge/Cassandra-3.0-blue.svg)]() 22 | [![version](https://img.shields.io/badge/Cassandra-3.1-blue.svg)]() 23 | [![version](https://img.shields.io/badge/Cassandra-4.0-blue.svg)]() 24 | 25 | ## Prepare 26 | 27 | ### Cassandra CPP driver 28 | 29 | Firstly, the latest Cassandra2 cpp driver needs be installed from https://github.com/datastax/cpp-driver. 30 | 31 | ### PostgreSQL Header 32 | 33 | You'll need the following packages to be installed with either, apt, or rpm, depending on your Linux distribution 34 | 35 | - postgresql-server 36 | - postgresql-contrib 37 | - postgresql-server-dev-{version number} 38 | 39 | ## Build 40 | 41 | ```bash 42 | # Clone the repository from GitHub 43 | git clone https://github.com/jaiminpan/cassandra2_fdw 44 | 45 | cd cassandra2_fdw 46 | USE_PGXS=1 make 47 | sudo USE_PGXS=1 make install 48 | ``` 49 | 50 | ## Usage 51 | 52 | The following parameters can be set on a Cassandra foreign server object: 53 | 54 | * **`host`**: the address or host name of the Cassandra server, Examples: "127.0.0.1" "127.0.0.1,127.0.0.2", "server1.domain.com" 55 | * **`port`**: the port number of the Cassandra server. Defaults is 9042 56 | * **`protocol`**: the protocol connect to Cassandra server. Defaults depends on cassandra-cpp-driver. (Default is "2" of v2.1.0 and default is "4" of v2.2.2) 57 | 58 | The following parameters can be set on a Cassandra foreign table object: 59 | 60 | * **`schema_name`**: the name of the Cassandra keyspace to query. Defaults to public 61 | * **`table_name`**: the name of the Cassandra table to query. Defaults to the foreign table name used in the relevant CREATE command 62 | 63 | Here is an example 64 | ```sql 65 | -- Create the extension inside a database 66 | CREATE EXTENSION cassandra2_fdw; 67 | 68 | -- Create the server object 69 | CREATE SERVER cass_serv FOREIGN DATA WRAPPER cassandra2_fdw 70 | OPTIONS(host '127.0.0.1,127.0.0.2', port '9042', protocol '2'); 71 | 72 | -- Create a user mapping for the server 73 | CREATE USER MAPPING FOR public SERVER cass_serv OPTIONS(username 'test', password 'test'); 74 | 75 | 76 | -- Create a foreign table on the server 77 | CREATE FOREIGN TABLE test (id int) SERVER cass_serv OPTIONS (schema_name 'example', table_name 'order'); 78 | 79 | -- Query the foreign table 80 | SELECT * FROM test limit 5; 81 | ``` 82 | 83 | ## Supported Data Types 84 | 85 | | Cassandra | PostgreSQL | 86 | | --------- | :--------------------------------------------------- | 87 | | tynint | smallint | 88 | | smallint | smallint | 89 | | int | integer | 90 | | bigint | bigint | 91 | | boolean | boolean | 92 | | float | float | 93 | | double | double precision | 94 | | timstamp | bigint (documented int the cpp-driver documentation) | 95 | | text | text | 96 | | ascii | text | 97 | | varchar | text | 98 | | uuid | uuid | 99 | 100 | ## Unsupported Data Types 101 | 102 | Unsupported data types will return an info text, that the data type is not supported instead of data. 103 | 104 | - list: Not implemented in this FDW 105 | - map: Not implemented in this FDW 106 | - user defined types: Not implemented in the cpp-driver 107 | 108 | ## Test 109 | 110 | With cpp-driver 2.16.0 against mentioned PostgreSQL versions and against Cassandra 3.2. 111 | 112 | ## Authors 113 | 114 | - [Jaimin Pan](mailto:jaimin.pan@gmail.com) 115 | - [Stefanie Janine Stölting](mailto:stefanie@ProOpenSource.eu) 116 | -------------------------------------------------------------------------------- /cass_connection.c: -------------------------------------------------------------------------------- 1 | /*------------------------------------------------------------------------- 2 | * 3 | * cass_connection.c 4 | * 5 | * Created on: Aug 8, 2015 6 | * Author: jaimin 7 | * 8 | * IDENTIFICATION 9 | * contrib/cassandra2_fdw/cass_connection.c 10 | * 11 | *------------------------------------------------------------------------- 12 | */ 13 | #include "postgres.h" 14 | 15 | #include "cassandra2_fdw.h" 16 | 17 | #include "access/hash.h" 18 | #include "access/xact.h" 19 | #include "commands/defrem.h" 20 | #include "mb/pg_wchar.h" 21 | #include "miscadmin.h" 22 | #include "storage/ipc.h" 23 | #include "utils/hsearch.h" 24 | #include "utils/memutils.h" 25 | 26 | 27 | typedef struct ConnCacheKey 28 | { 29 | Oid serverid; /* OID of foreign server */ 30 | Oid userid; /* OID of local user whose mapping we use */ 31 | } ConnCacheKey; 32 | 33 | typedef struct ConnCacheEntry 34 | { 35 | ConnCacheKey key; /* hash key (must be first) */ 36 | CassSession *conn; /* connection to foreign server, or NULL */ 37 | int xact_depth; /* 0 = no xact open, 1 = main xact open, 2 = 38 | * one level of subxact open, etc */ 39 | bool have_prep_stmt; /* have we prepared any stmts in this xact? */ 40 | bool have_error; /* have any subxacts aborted in this xact? */ 41 | } ConnCacheEntry; 42 | 43 | /* 44 | * Connection cache (initialized on first use) 45 | */ 46 | static HTAB *ConnectionHash = NULL; 47 | 48 | /* tracks whether any work is needed in callback functions */ 49 | static bool xact_got_connection = false; 50 | 51 | /* prototypes of private functions */ 52 | static CassSession *connect_cass_server(ForeignServer *server, UserMapping *user); 53 | static int ExtractOptions(List *defelems, const char **keywords, 54 | const char **values); 55 | 56 | 57 | static CassCluster* cluster; 58 | 59 | static void pgcass_close(int code, Datum arg); 60 | 61 | 62 | CassSession * 63 | pgcass_GetConnection(ForeignServer *server, UserMapping *user, 64 | bool will_prep_stmt) 65 | { 66 | bool found; 67 | ConnCacheEntry *entry; 68 | ConnCacheKey key; 69 | 70 | /* First time through, initialize connection cache hashtable */ 71 | if (ConnectionHash == NULL) 72 | { 73 | HASHCTL ctl; 74 | 75 | MemSet(&ctl, 0, sizeof(ctl)); 76 | ctl.keysize = sizeof(ConnCacheKey); 77 | ctl.entrysize = sizeof(ConnCacheEntry); 78 | ctl.hash = tag_hash; 79 | /* allocate ConnectionHash in the cache context */ 80 | ctl.hcxt = CacheMemoryContext; 81 | ConnectionHash = hash_create("cassandra2_fdw connections", 8, 82 | &ctl, 83 | HASH_ELEM | HASH_FUNCTION | HASH_CONTEXT); 84 | 85 | /* 86 | * Register some callback functions that manage connection cleanup. 87 | * This should be done just once in each backend. 88 | */ 89 | // RegisterXactCallback(pgfdw_xact_callback, NULL); 90 | // RegisterSubXactCallback(pgfdw_subxact_callback, NULL); 91 | } 92 | 93 | /* Set flag that we did GetConnection during the current transaction */ 94 | xact_got_connection = true; 95 | 96 | /* Create hash key for the entry. Assume no pad bytes in key struct */ 97 | key.serverid = server->serverid; 98 | key.userid = user->userid; 99 | 100 | /* 101 | * Find or create cached entry for requested connection. 102 | */ 103 | entry = hash_search(ConnectionHash, &key, HASH_ENTER, &found); 104 | if (!found) 105 | { 106 | /* initialize new hashtable entry (key is already filled in) */ 107 | entry->conn = NULL; 108 | entry->xact_depth = 0; 109 | entry->have_prep_stmt = false; 110 | entry->have_error = false; 111 | } 112 | 113 | /* 114 | * We don't check the health of cached connection here, because it would 115 | * require some overhead. Broken connection will be detected when the 116 | * connection is actually used. 117 | */ 118 | 119 | /* 120 | * If cache entry doesn't have a connection, we have to establish a new 121 | * connection. (If connect_pg_server throws an error, the cache entry 122 | * will be left in a valid empty state.) 123 | */ 124 | if (entry->conn == NULL) 125 | { 126 | entry->xact_depth = 0; /* just to be sure */ 127 | entry->have_prep_stmt = false; 128 | entry->have_error = false; 129 | entry->conn = connect_cass_server(server, user); 130 | elog(DEBUG3, "new cassandra2_fdw connection %p for server \"%s\"", 131 | entry->conn, server->servername); 132 | } 133 | 134 | /* 135 | * do nothing for connection pre-cmd execute till now. 136 | */ 137 | 138 | /* Remember if caller will prepare statements */ 139 | entry->have_prep_stmt |= will_prep_stmt; 140 | 141 | return entry->conn; 142 | } 143 | 144 | /* 145 | * Release connection reference count created by calling GetConnection. 146 | */ 147 | void 148 | pgcass_ReleaseConnection(CassSession *session) 149 | { 150 | /* 151 | * Currently, we don't actually track connection references because all 152 | * cleanup is managed on a transaction or subtransaction basis instead. So 153 | * there's nothing to do here. 154 | */ 155 | CassFuture* close_future = NULL; 156 | 157 | return; 158 | 159 | /* Close the session */ 160 | close_future = cass_session_close(session); 161 | cass_future_wait(close_future); 162 | cass_future_free(close_future); 163 | } 164 | 165 | /* 166 | * Connect to remote server using specified server and user mapping properties. 167 | */ 168 | static CassSession * 169 | connect_cass_server(ForeignServer *server, UserMapping *user) 170 | { 171 | CassFuture* conn_future = NULL; 172 | CassSession* session = NULL; 173 | 174 | if (!cluster) 175 | { 176 | cluster = cass_cluster_new(); 177 | on_proc_exit(pgcass_close, 0); 178 | } 179 | 180 | /* 181 | * Use PG_TRY block to ensure closing connection on error. 182 | */ 183 | PG_TRY(); 184 | { 185 | const char **keywords; 186 | const char **values; 187 | int n; 188 | int i; 189 | const char *svr_host = NULL; 190 | int svr_port = 0; 191 | int svr_proto = 0; 192 | const char *svr_username = NULL; 193 | const char *svr_password = NULL; 194 | 195 | /* 196 | * Construct connection params from generic options of ForeignServer 197 | * and UserMapping. 198 | */ 199 | n = list_length(server->options) + list_length(user->options) + 1; 200 | keywords = (const char **) palloc(n * sizeof(char *)); 201 | values = (const char **) palloc(n * sizeof(char *)); 202 | 203 | n = 0; 204 | n += ExtractOptions(server->options, 205 | keywords + n, values + n); 206 | n += ExtractOptions(user->options, 207 | keywords + n, values + n); 208 | 209 | keywords[n] = values[n] = NULL; 210 | 211 | /* Add contact points */ 212 | for (i = 0; keywords[i] != NULL; i++) 213 | { 214 | if (strcmp(keywords[i], "host") == 0) 215 | { 216 | svr_host = values[i]; 217 | } 218 | else if (strcmp(keywords[i], "port") == 0) 219 | { 220 | svr_port = atoi(values[i]); 221 | } 222 | else if (strcmp(keywords[i], "protocol") == 0) 223 | { 224 | svr_proto = atoi(values[i]); 225 | } 226 | else if (strcmp(keywords[i], "username") == 0) 227 | { 228 | svr_username = values[i]; 229 | } 230 | else if (strcmp(keywords[i], "password") == 0) 231 | { 232 | svr_password = values[i]; 233 | } 234 | } 235 | 236 | if (svr_host) 237 | cass_cluster_set_contact_points(cluster, svr_host); 238 | if (svr_port) 239 | cass_cluster_set_port(cluster, svr_port); 240 | if (svr_proto) 241 | cass_cluster_set_protocol_version(cluster, svr_proto); 242 | if (svr_username && svr_password) 243 | cass_cluster_set_credentials(cluster, svr_username, svr_password); 244 | 245 | session = cass_session_new(); 246 | 247 | /* Provide the cluster object as configuration to connect the session */ 248 | conn_future = cass_session_connect(session, cluster); 249 | if (cass_future_error_code(conn_future) != CASS_OK) 250 | { 251 | /* Handle error */ 252 | const char* message; 253 | size_t message_length; 254 | 255 | cass_future_error_message(conn_future, &message, &message_length); 256 | 257 | ereport(ERROR, 258 | (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION), 259 | errmsg("could not connect to server \"%s\"", 260 | server->servername), 261 | errdetail_internal("%.*s", (int)message_length, message))); 262 | } 263 | 264 | cass_future_free(conn_future); 265 | 266 | pfree(keywords); 267 | pfree(values); 268 | } 269 | PG_CATCH(); 270 | { 271 | /* Release data structure if we managed to create one */ 272 | if (conn_future) 273 | cass_future_free(conn_future); 274 | 275 | if (session) 276 | cass_session_free(session); 277 | PG_RE_THROW(); 278 | } 279 | PG_END_TRY(); 280 | 281 | return session; 282 | } 283 | 284 | 285 | /* 286 | * pgcass_close 287 | * Shuts down the Connection. 288 | */ 289 | static void 290 | pgcass_close(int code, Datum arg) 291 | { 292 | cass_cluster_free(cluster); 293 | } 294 | 295 | 296 | /* 297 | * Report an error we got from the remote server. 298 | * 299 | * elevel: error level to use (typically ERROR, but might be less) 300 | * clear: if true, clear the result (otherwise caller will handle it) 301 | * sql: NULL, or text of remote command we tried to execute 302 | * 303 | * Note: callers that choose not to throw ERROR for a remote error are 304 | * responsible for making sure that the associated ConnCacheEntry gets 305 | * marked with have_error = true. 306 | */ 307 | void 308 | pgcass_report_error(int elevel, CassFuture* result_future, 309 | bool clear, const char *sql) 310 | { 311 | PG_TRY(); 312 | { 313 | const char *message_primary = NULL; 314 | const char *message_detail = NULL; 315 | const char *message_hint = NULL; 316 | const char *message_context = NULL; 317 | int sqlstate; 318 | size_t message_length; 319 | 320 | sqlstate = ERRCODE_CONNECTION_FAILURE; 321 | 322 | cass_future_error_message(result_future, &message_primary, &message_length); 323 | 324 | ereport(elevel, 325 | (errcode(sqlstate), 326 | message_primary ? errmsg_internal("%s", message_primary) : 327 | errmsg("unknown error"), 328 | message_detail ? errdetail_internal("%s", message_detail) : 0, 329 | message_hint ? errhint("%s", message_hint) : 0, 330 | message_context ? errcontext("%s", message_context) : 0, 331 | sql ? errcontext("Remote SQL command: %s", sql) : 0)); 332 | } 333 | PG_CATCH(); 334 | { 335 | if (clear) 336 | cass_future_free(result_future); 337 | PG_RE_THROW(); 338 | } 339 | PG_END_TRY(); 340 | if (clear) 341 | cass_future_free(result_future); 342 | } 343 | 344 | /* 345 | * Generate key-value arrays which include only libpq options from the 346 | * given list (which can contain any kind of options). Caller must have 347 | * allocated large-enough arrays. Returns number of options found. 348 | */ 349 | static int 350 | ExtractOptions(List *defelems, const char **keywords, 351 | const char **values) 352 | { 353 | ListCell *lc; 354 | int i; 355 | 356 | i = 0; 357 | foreach(lc, defelems) 358 | { 359 | DefElem *d = (DefElem *) lfirst(lc); 360 | 361 | keywords[i] = d->defname; 362 | values[i] = defGetString(d); 363 | i++; 364 | } 365 | return i; 366 | } 367 | -------------------------------------------------------------------------------- /cassandra2_fdw--2.0.sql: -------------------------------------------------------------------------------- 1 | /*------------------------------------------------------------------------- 2 | * 3 | * Copyright (c) 2014, Open Source Consulting Group 4 | * 5 | *------------------------------------------------------------------------- 6 | */ 7 | 8 | CREATE FUNCTION cassandra2_fdw_handler() 9 | RETURNS fdw_handler 10 | AS 'MODULE_PATHNAME' 11 | LANGUAGE C STRICT; 12 | 13 | CREATE FUNCTION cassandra2_fdw_validator(text[], oid) 14 | RETURNS void 15 | AS 'MODULE_PATHNAME' 16 | LANGUAGE C STRICT; 17 | 18 | CREATE FOREIGN DATA WRAPPER cassandra2_fdw 19 | HANDLER cassandra2_fdw_handler 20 | VALIDATOR cassandra2_fdw_validator; 21 | -------------------------------------------------------------------------------- /cassandra2_fdw.c: -------------------------------------------------------------------------------- 1 | /*------------------------------------------------------------------------- 2 | * 3 | * Cassandra2 Foreign Data Wrapper for PostgreSQL 4 | * 5 | * Copyright (c) 2015 Jaimin Pan 6 | * 7 | * This software is released under the PostgreSQL Licence 8 | * 9 | * Author: Jaimin Pan 10 | * 11 | * IDENTIFICATION 12 | * cassandra2_fdw/cassandra2_fdw.c 13 | * 14 | *------------------------------------------------------------------------- 15 | */ 16 | #include "postgres.h" 17 | 18 | #if (PG_VERSION_NUM < 100000) 19 | #error Oldest supported PostgreSQL version is 10 (100000) 20 | #endif /* PG_VERSION_NUM < 100000 */ 21 | 22 | #include 23 | 24 | #include "cassandra2_fdw.h" 25 | 26 | #include "access/htup_details.h" 27 | #include "access/reloptions.h" 28 | #include "access/sysattr.h" 29 | #include "executor/spi.h" 30 | #include "foreign/fdwapi.h" 31 | #include "common/md5.h" 32 | #include "funcapi.h" 33 | #include "miscadmin.h" 34 | #include "mb/pg_wchar.h" 35 | #include "optimizer/cost.h" 36 | #include "optimizer/restrictinfo.h" 37 | #include "catalog/pg_foreign_server.h" 38 | #include "catalog/pg_foreign_table.h" 39 | #include "catalog/pg_user_mapping.h" 40 | #include "catalog/pg_type.h" 41 | #include "commands/defrem.h" 42 | #include "commands/explain.h" 43 | #include "commands/vacuum.h" 44 | #include "nodes/makefuncs.h" 45 | #include "nodes/nodeFuncs.h" 46 | #include "optimizer/pathnode.h" 47 | #if PG_VERSION_NUM >= 130000 48 | #include "optimizer/paths.h" 49 | #endif /* PG_VERSION_NUM 130000 */ 50 | #include "optimizer/planmain.h" 51 | #include "optimizer/restrictinfo.h" 52 | #if PG_VERSION_NUM < 120000 53 | #include "nodes/relation.h" 54 | #include "optimizer/var.h" 55 | #include "utils/tqual.h" 56 | #else 57 | #include "nodes/pathnodes.h" 58 | #include "optimizer/optimizer.h" 59 | #include "access/heapam.h" 60 | #endif /* PG_VERSION_NUM 120000 */ 61 | #include "optimizer/paths.h" 62 | #include "parser/parsetree.h" 63 | #include "storage/fd.h" 64 | #include "storage/ipc.h" 65 | #include "utils/acl.h" 66 | #include "utils/array.h" 67 | #include "utils/builtins.h" 68 | #include "utils/guc.h" 69 | #include "utils/lsyscache.h" 70 | #include "utils/memutils.h" 71 | 72 | /* defined in backend/commands/analyze.c */ 73 | #ifndef WIDTH_THRESHOLD 74 | #define WIDTH_THRESHOLD 1024 75 | #endif /* WIDTH_THRESHOLD */ 76 | 77 | #if PG_VERSION_NUM >= 90500 78 | #define IMPORT_API 79 | 80 | /* array_create_iterator has a new signature from 9.5 on */ 81 | #define array_create_iterator(arr, slice_ndim) array_create_iterator(arr, slice_ndim, NULL) 82 | #else 83 | #undef IMPORT_API 84 | #endif /* PG_VERSION_NUM */ 85 | 86 | #if PG_VERSION_NUM >= 90600 87 | #define JOIN_API 88 | 89 | /* the useful macro IS_SIMPLE_REL is defined in v10, backport */ 90 | #ifndef IS_SIMPLE_REL 91 | #define IS_SIMPLE_REL(rel) \ 92 | ((rel)->reloptkind == RELOPT_BASEREL || \ 93 | (rel)->reloptkind == RELOPT_OTHER_MEMBER_REL) 94 | #endif 95 | 96 | #if (PG_VERSION_NUM < 110000) 97 | #define TupleDescAttr(tupdesc, i) ((tupdesc)->attrs[(i)]) 98 | #else 99 | #define get_relid_attribute_name(relid, varattno) get_attname((relid), (varattno), false) 100 | #endif 101 | 102 | /* GetConfigOptionByName has a new signature from 9.6 on */ 103 | #define GetConfigOptionByName(name, varname) GetConfigOptionByName(name, varname, false) 104 | #else 105 | #undef JOIN_API 106 | #endif /* PG_VERSION_NUM */ 107 | 108 | #if PG_VERSION_NUM < 110000 109 | /* backport macro from V11 */ 110 | #define TupleDescAttr(tupdesc, i) ((tupdesc)->attrs[(i)]) 111 | #endif /* PG_VERSION_NUM */ 112 | 113 | /* list API has changed in v13 */ 114 | #if PG_VERSION_NUM < 130000 115 | #define list_next(l, e) lnext((e)) 116 | #define do_each_cell(cell, list, element) for_each_cell(cell, (element)) 117 | #else 118 | #define list_next(l, e) lnext((l), (e)) 119 | #define do_each_cell(cell, list, element) for_each_cell(cell, (list), (element)) 120 | #endif /* PG_VERSION_NUM */ 121 | 122 | /* "table_open" was "heap_open" before v12 */ 123 | #if PG_VERSION_NUM < 120000 124 | #define table_open(x, y) heap_open(x, y) 125 | #define table_close(x, y) heap_close(x, y) 126 | #endif /* PG_VERSION_NUM */ 127 | 128 | PG_MODULE_MAGIC; 129 | 130 | /* Default CPU cost to start up a foreign query. */ 131 | #define DEFAULT_FDW_STARTUP_COST 100.0 132 | 133 | /* Default CPU cost to process 1 row (above and beyond cpu_tuple_cost). */ 134 | #define DEFAULT_FDW_TUPLE_COST 0.01 135 | 136 | 137 | /* 138 | * Describes the valid options for objects that use this wrapper. 139 | */ 140 | struct CassFdwOption 141 | { 142 | const char *optname; 143 | Oid optcontext; /* Oid of catalog in which option may appear */ 144 | }; 145 | 146 | /* 147 | * Valid options for cassandra2_fdw. 148 | */ 149 | static struct CassFdwOption valid_options[] = 150 | { 151 | /* Connection options */ 152 | { "host", ForeignServerRelationId }, 153 | { "port", ForeignServerRelationId }, 154 | { "protocol", ForeignServerRelationId }, 155 | { "username", UserMappingRelationId }, 156 | { "password", UserMappingRelationId }, 157 | { "query", ForeignTableRelationId }, 158 | { "schema_name", ForeignTableRelationId }, 159 | { "table_name", ForeignTableRelationId }, 160 | 161 | /* Sentinel */ 162 | { NULL, InvalidOid } 163 | }; 164 | 165 | /* 166 | * FDW-specific information for RelOptInfo.fdw_private. 167 | */ 168 | typedef struct CassFdwPlanState 169 | { 170 | /* baserestrictinfo clauses, broken down into safe and unsafe subsets. */ 171 | List *remote_conds; 172 | List *local_conds; 173 | 174 | /* Bitmap of attr numbers we need to fetch from the remote server. */ 175 | Bitmapset *attrs_used; 176 | 177 | /* Estimated size and cost for a scan with baserestrictinfo quals. */ 178 | double rows; 179 | int width; 180 | Cost startup_cost; 181 | Cost total_cost; 182 | } CassFdwPlanState; 183 | 184 | /* 185 | * FDW-specific information for ForeignScanState.fdw_state. 186 | */ 187 | typedef struct CassFdwScanState 188 | { 189 | Relation rel; /* relcache entry for the foreign table */ 190 | AttInMetadata *attinmeta; /* attribute datatype conversion metadata */ 191 | 192 | /* extracted fdw_private data */ 193 | char *query; /* text of SELECT command */ 194 | List *retrieved_attrs; /* list of retrieved attribute numbers */ 195 | 196 | int NumberOfColumns; 197 | 198 | /* for remote query execution */ 199 | CassSession *cass_conn; /* connection for the scan */ 200 | bool sql_sended; 201 | CassStatement *statement; 202 | 203 | /* for storing result tuples */ 204 | HeapTuple *tuples; /* array of currently-retrieved tuples */ 205 | int num_tuples; /* # of tuples in array */ 206 | int next_tuple; /* index of next one to return */ 207 | 208 | /* batch-level state, for optimizing rewinds and avoiding useless fetch */ 209 | int fetch_ct_2; /* Min(# of fetches done, 2) */ 210 | bool eof_reached; /* true if last fetch reached EOF */ 211 | 212 | /* working memory contexts */ 213 | MemoryContext batch_cxt; /* context holding current batch of tuples */ 214 | MemoryContext temp_cxt; /* context for per-tuple temporary data */ 215 | } CassFdwScanState; 216 | 217 | enum CassFdwScanPrivateIndex 218 | { 219 | /* SQL statement to execute remotely (as a String node) */ 220 | CassFdwScanPrivateSelectSql, 221 | /* Integer list of attribute numbers retrieved by the SELECT */ 222 | CassFdwScanPrivateRetrievedAttrs 223 | }; 224 | 225 | 226 | /* 227 | * SQL functions 228 | */ 229 | extern Datum cassandra2_fdw_handler(PG_FUNCTION_ARGS); 230 | extern Datum cassandra2_fdw_validator(PG_FUNCTION_ARGS); 231 | 232 | PG_FUNCTION_INFO_V1(cassandra2_fdw_handler); 233 | PG_FUNCTION_INFO_V1(cassandra2_fdw_validator); 234 | 235 | 236 | /* 237 | * FDW callback routines 238 | */ 239 | static void cassGetForeignRelSize(PlannerInfo *root, RelOptInfo *baserel, Oid foreigntableid); 240 | static void cassGetForeignPaths(PlannerInfo *root, RelOptInfo *baserel, Oid foreigntableid); 241 | static ForeignScan *cassGetForeignPlan( 242 | PlannerInfo *root, 243 | RelOptInfo *baserel, 244 | Oid foreigntableid, 245 | ForeignPath *best_path, 246 | List *tlist, 247 | List *scan_clauses 248 | #if (PG_VERSION_NUM >= 90500) 249 | , 250 | Plan *outer_plan 251 | #endif 252 | ); 253 | static void cassExplainForeignScan(ForeignScanState *node, ExplainState *es); 254 | static void cassBeginForeignScan(ForeignScanState *node, int eflags); 255 | static TupleTableSlot *cassIterateForeignScan(ForeignScanState *node); 256 | static void cassReScanForeignScan(ForeignScanState *node); 257 | static void cassEndForeignScan(ForeignScanState *node); 258 | 259 | /* 260 | * Helper functions 261 | */ 262 | static void estimate_path_cost_size(PlannerInfo *root, 263 | RelOptInfo *baserel, 264 | List *join_conds, 265 | double *p_rows, int *p_width, 266 | Cost *p_startup_cost, Cost *p_total_cost); 267 | static bool cassIsValidOption(const char *option, Oid context); 268 | static void cassGetOptions(Oid foreigntableid, 269 | char **host, int *port, 270 | char **username, char **password, 271 | char **query, char **tablename); 272 | 273 | static void create_cursor(ForeignScanState *node); 274 | static void close_cursor(CassFdwScanState *fsstate); 275 | static void fetch_more_data(ForeignScanState *node); 276 | static void pgcass_transferValue(StringInfo buf, const CassValue* value); 277 | static HeapTuple make_tuple_from_result_row(const CassRow* row, 278 | int ncolumn, 279 | Relation rel, 280 | AttInMetadata *attinmeta, 281 | List *retrieved_attrs, 282 | MemoryContext temp_context); 283 | 284 | static void classifyConditions(PlannerInfo *root, 285 | RelOptInfo *baserel, 286 | List *input_conds, 287 | List **remote_conds, 288 | List **local_conds); 289 | static void 290 | deparseSelectSql(StringInfo buf, 291 | PlannerInfo *root, 292 | RelOptInfo *baserel, 293 | Bitmapset *attrs_used, 294 | List **retrieved_attrs); 295 | 296 | /* 297 | * Foreign-data wrapper handler function: return a struct with pointers 298 | * to my callback routines. 299 | */ 300 | Datum 301 | cassandra2_fdw_handler(PG_FUNCTION_ARGS) 302 | { 303 | FdwRoutine *fdwroutine = makeNode(FdwRoutine); 304 | 305 | fdwroutine->GetForeignRelSize = cassGetForeignRelSize; 306 | fdwroutine->GetForeignPaths = cassGetForeignPaths; 307 | fdwroutine->GetForeignPlan = cassGetForeignPlan; 308 | 309 | fdwroutine->ExplainForeignScan = cassExplainForeignScan; 310 | fdwroutine->BeginForeignScan = cassBeginForeignScan; 311 | fdwroutine->IterateForeignScan = cassIterateForeignScan; 312 | fdwroutine->ReScanForeignScan = cassReScanForeignScan; 313 | fdwroutine->EndForeignScan = cassEndForeignScan; 314 | fdwroutine->AnalyzeForeignTable = NULL; 315 | 316 | PG_RETURN_POINTER(fdwroutine); 317 | } 318 | 319 | /* 320 | * Validate the generic options given to a FOREIGN DATA WRAPPER, SERVER, 321 | * USER MAPPING or FOREIGN TABLE that uses file_fdw. 322 | * 323 | * Raise an ERROR if the option or its value is considered invalid. 324 | */ 325 | Datum 326 | cassandra2_fdw_validator(PG_FUNCTION_ARGS) 327 | { 328 | List *options_list = untransformRelOptions(PG_GETARG_DATUM(0)); 329 | Oid catalog = PG_GETARG_OID(1); 330 | char *svr_host = NULL; 331 | int svr_port = 0; 332 | char *svr_username = NULL; 333 | char *svr_password = NULL; 334 | char *svr_query = NULL; 335 | char *svr_schema = NULL; 336 | char *svr_table = NULL; 337 | ListCell *cell; 338 | 339 | /* 340 | * Check that only options supported by cassandra2_fdw, 341 | * and allowed for the current object type, are given. 342 | */ 343 | foreach(cell, options_list) 344 | { 345 | DefElem *def = (DefElem *) lfirst(cell); 346 | 347 | if (!cassIsValidOption(def->defname, catalog)) 348 | { 349 | const struct CassFdwOption *opt; 350 | StringInfoData buf; 351 | 352 | /* 353 | * Unknown option specified, complain about it. Provide a hint 354 | * with list of valid options for the object. 355 | */ 356 | initStringInfo(&buf); 357 | for (opt = valid_options; opt->optname; opt++) 358 | { 359 | if (catalog == opt->optcontext) 360 | appendStringInfo(&buf, "%s%s", (buf.len > 0) ? ", " : "", 361 | opt->optname); 362 | } 363 | 364 | ereport(ERROR, 365 | (errcode(ERRCODE_FDW_INVALID_OPTION_NAME), 366 | errmsg("invalid option \"%s\"", def->defname), 367 | buf.len > 0 368 | ? errhint("Valid options in this context are: %s", 369 | buf.data) 370 | : errhint("There are no valid options in this context."))); 371 | } 372 | 373 | if (strcmp(def->defname, "host") == 0) 374 | { 375 | if (svr_host) 376 | ereport(ERROR, 377 | (errcode(ERRCODE_SYNTAX_ERROR), 378 | errmsg("conflicting or redundant options"))); 379 | svr_host = defGetString(def); 380 | } 381 | if (strcmp(def->defname, "port") == 0) 382 | { 383 | if (svr_port) 384 | ereport(ERROR, 385 | (errcode(ERRCODE_SYNTAX_ERROR), 386 | errmsg("conflicting or redundant options"))); 387 | svr_port = atoi(defGetString(def)); 388 | } 389 | else if (strcmp(def->defname, "username") == 0) 390 | { 391 | if (svr_username) 392 | ereport(ERROR, 393 | (errcode(ERRCODE_SYNTAX_ERROR), 394 | errmsg("conflicting or redundant options"))); 395 | svr_username = defGetString(def); 396 | } 397 | else if (strcmp(def->defname, "password") == 0) 398 | { 399 | if (svr_password) 400 | ereport(ERROR, 401 | (errcode(ERRCODE_SYNTAX_ERROR), 402 | errmsg("conflicting or redundant options"))); 403 | svr_password = defGetString(def); 404 | } 405 | else if (strcmp(def->defname, "query") == 0) 406 | { 407 | if (svr_table) 408 | ereport(ERROR, 409 | (errcode(ERRCODE_SYNTAX_ERROR), 410 | errmsg("conflicting or redundant options: query cannot be used with table"))); 411 | 412 | if (svr_query) 413 | ereport(ERROR, 414 | (errcode(ERRCODE_SYNTAX_ERROR), 415 | errmsg("conflicting or redundant options"))); 416 | 417 | svr_query = defGetString(def); 418 | } 419 | else if (strcmp(def->defname, "schema_name") == 0) 420 | { 421 | if (svr_schema) 422 | ereport(ERROR, 423 | (errcode(ERRCODE_SYNTAX_ERROR), 424 | errmsg("conflicting or redundant options"))); 425 | 426 | svr_schema = defGetString(def); 427 | } 428 | else if (strcmp(def->defname, "table_name") == 0) 429 | { 430 | if (svr_query) 431 | ereport(ERROR, 432 | (errcode(ERRCODE_SYNTAX_ERROR), 433 | errmsg("conflicting or redundant options: table_name cannot be used with query"))); 434 | 435 | if (svr_table) 436 | ereport(ERROR, 437 | (errcode(ERRCODE_SYNTAX_ERROR), 438 | errmsg("conflicting or redundant options"))); 439 | 440 | svr_table = defGetString(def); 441 | } 442 | } 443 | 444 | if (catalog == ForeignServerRelationId && svr_host == NULL) 445 | ereport(ERROR, 446 | (errcode(ERRCODE_SYNTAX_ERROR), 447 | errmsg("host must be specified"))); 448 | 449 | if (catalog == ForeignTableRelationId && 450 | svr_query == NULL && svr_table == NULL) 451 | ereport(ERROR, 452 | (errcode(ERRCODE_SYNTAX_ERROR), 453 | errmsg("either table_name or query must be specified"))); 454 | 455 | PG_RETURN_VOID(); 456 | } 457 | 458 | 459 | /* 460 | * Check if the provided option is one of the valid options. 461 | * context is the Oid of the catalog holding the object the option is for. 462 | */ 463 | static bool 464 | cassIsValidOption(const char *option, Oid context) 465 | { 466 | const struct CassFdwOption *opt; 467 | 468 | for (opt = valid_options; opt->optname; opt++) 469 | { 470 | if (context == opt->optcontext && strcmp(opt->optname, option) == 0) 471 | return true; 472 | } 473 | return false; 474 | } 475 | 476 | 477 | /* 478 | * Fetch the options for a fdw foreign table. 479 | */ 480 | static void 481 | cassGetOptions(Oid foreigntableid, char **host, int *port, 482 | char **username, char **password, char **query, char **tablename) 483 | { 484 | ForeignTable *table; 485 | ForeignServer *server; 486 | UserMapping *user; 487 | List *options; 488 | ListCell *lc; 489 | 490 | /* 491 | * Extract options from FDW objects. 492 | */ 493 | table = GetForeignTable(foreigntableid); 494 | server = GetForeignServer(table->serverid); 495 | user = GetUserMapping(GetUserId(), server->serverid); 496 | 497 | options = NIL; 498 | options = list_concat(options, table->options); 499 | options = list_concat(options, server->options); 500 | options = list_concat(options, user->options); 501 | 502 | /* Loop through the options, and get the server/port */ 503 | foreach(lc, options) 504 | { 505 | DefElem *def = (DefElem *) lfirst(lc); 506 | 507 | if (strcmp(def->defname, "username") == 0) 508 | { 509 | *username = defGetString(def); 510 | } 511 | else if (strcmp(def->defname, "password") == 0) 512 | { 513 | *password = defGetString(def); 514 | } 515 | else if (strcmp(def->defname, "query") == 0) 516 | { 517 | *query = defGetString(def); 518 | } 519 | else if (strcmp(def->defname, "table_name") == 0) 520 | { 521 | *tablename = defGetString(def); 522 | } 523 | else if (strcmp(def->defname, "host") == 0) 524 | { 525 | *host = defGetString(def); 526 | } 527 | else if (strcmp(def->defname, "port") == 0) 528 | { 529 | *port = atoi(defGetString(def)); 530 | } 531 | 532 | } 533 | } 534 | 535 | //#if (PG_VERSION_NUM >= 90200) 536 | 537 | /* 538 | * cassGetForeignRelSize 539 | * Obtain relation size estimates for a foreign table 540 | */ 541 | static void 542 | cassGetForeignRelSize(PlannerInfo *root, 543 | RelOptInfo *baserel, 544 | Oid foreigntableid) 545 | { 546 | CassFdwPlanState *fpinfo; 547 | ListCell *lc; 548 | 549 | fpinfo = (CassFdwPlanState *) palloc0(sizeof(CassFdwPlanState)); 550 | baserel->fdw_private = (void *) fpinfo; 551 | 552 | /* 553 | * Identify which baserestrictinfo clauses can be sent to the remote 554 | * server and which can't. 555 | */ 556 | classifyConditions(root, baserel, baserel->baserestrictinfo, 557 | &fpinfo->remote_conds, &fpinfo->local_conds); 558 | 559 | fpinfo->attrs_used = NULL; 560 | #if (PG_VERSION_NUM >= 90600) 561 | pull_varattnos((Node *) baserel->reltarget->exprs, baserel->relid, 562 | &fpinfo->attrs_used); 563 | #else 564 | pull_varattnos((Node *) baserel->reltargetlist, baserel->relid, 565 | &fpinfo->attrs_used); 566 | #endif 567 | foreach(lc, fpinfo->local_conds) 568 | { 569 | RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc); 570 | 571 | pull_varattnos((Node *) rinfo->clause, baserel->relid, 572 | &fpinfo->attrs_used); 573 | } 574 | 575 | //TODO 576 | /* Fetch options */ 577 | 578 | /* Estimate relation size */ 579 | { 580 | /* 581 | * If the foreign table has never been ANALYZEd, it will have relpages 582 | * and reltuples equal to zero, which most likely has nothing to do 583 | * with reality. We can't do a whole lot about that if we're not 584 | * allowed to consult the remote server, but we can use a hack similar 585 | * to plancat.c's treatment of empty relations: use a minimum size 586 | * estimate of 10 pages, and divide by the column-datatype-based width 587 | * estimate to get the corresponding number of tuples. 588 | */ 589 | if (baserel->pages == 0 && baserel->tuples == 0) 590 | { 591 | baserel->pages = 10; 592 | baserel->tuples = 593 | #if (PG_VERSION_NUM >= 90600) 594 | (10 * BLCKSZ) / (baserel->reltarget->width + sizeof(HeapTupleHeaderData)); 595 | #else 596 | (10 * BLCKSZ) / (baserel->width + sizeof(HeapTupleHeaderData)); 597 | #endif 598 | } 599 | 600 | /* Estimate baserel size as best we can with local statistics. */ 601 | set_baserel_size_estimates(root, baserel); 602 | 603 | /* Fill in basically-bogus cost estimates for use later. */ 604 | estimate_path_cost_size(root, baserel, NIL, 605 | &fpinfo->rows, &fpinfo->width, 606 | &fpinfo->startup_cost, &fpinfo->total_cost); 607 | } 608 | } 609 | 610 | static void 611 | estimate_path_cost_size(PlannerInfo *root, 612 | RelOptInfo *baserel, 613 | List *join_conds, 614 | double *p_rows, int *p_width, 615 | Cost *p_startup_cost, Cost *p_total_cost) 616 | { 617 | *p_rows = baserel->rows; 618 | #if (PG_VERSION_NUM >= 90600) 619 | *p_width = baserel->reltarget->width; 620 | #else 621 | *p_width = baserel->width; 622 | #endif 623 | 624 | *p_startup_cost = DEFAULT_FDW_STARTUP_COST; 625 | *p_total_cost = DEFAULT_FDW_TUPLE_COST * 100; 626 | } 627 | 628 | /* 629 | * cassGetForeignPaths 630 | * (9.2+) Get the foreign paths 631 | */ 632 | static void 633 | cassGetForeignPaths(PlannerInfo *root, 634 | RelOptInfo *baserel, 635 | Oid foreigntableid) 636 | { 637 | CassFdwPlanState *fpinfo = (CassFdwPlanState *) baserel->fdw_private; 638 | ForeignPath *path; 639 | 640 | /* 641 | * Create simplest ForeignScan path node and add it to baserel. This path 642 | * corresponds to SeqScan path of regular tables (though depending on what 643 | * baserestrict conditions we were able to send to remote, there might 644 | * actually be an indexscan happening there). We already did all the work 645 | * to estimate cost and size of this path. 646 | */ 647 | path = create_foreignscan_path(root, baserel, 648 | #if (PG_VERSION_NUM >= 10000) 649 | NULL, 650 | #endif 651 | fpinfo->rows + baserel->rows, 652 | fpinfo->startup_cost, 653 | fpinfo->total_cost, 654 | NIL, /* no pathkeys */ 655 | NULL, /* no outer rel either */ 656 | #if (PG_VERSION_NUM >= 90500) 657 | NULL, /* no fdw_outerpath */ 658 | #endif 659 | NIL); /* no fdw_private list */ 660 | 661 | add_path(baserel, (Path *) path); 662 | 663 | //TODO 664 | } 665 | 666 | /* 667 | * cassGetForeignPlan 668 | * Create ForeignScan plan node which implements selected best path 669 | */ 670 | static ForeignScan * 671 | cassGetForeignPlan(PlannerInfo *root, 672 | RelOptInfo *baserel, 673 | Oid foreigntableid, 674 | ForeignPath *best_path, 675 | List *tlist, 676 | List *scan_clauses 677 | #if (PG_VERSION_NUM >= 90500) 678 | , 679 | Plan *outer_plan 680 | #endif 681 | ) 682 | { 683 | CassFdwPlanState *fpinfo = (CassFdwPlanState *) baserel->fdw_private; 684 | Index scan_relid = baserel->relid; 685 | List *fdw_private; 686 | List *local_exprs = NIL; 687 | StringInfoData sql; 688 | List *retrieved_attrs; 689 | 690 | local_exprs = extract_actual_clauses(scan_clauses, false); 691 | 692 | /* 693 | * Build the query string to be sent for execution, and identify 694 | * expressions to be sent as parameters. 695 | */ 696 | initStringInfo(&sql); 697 | deparseSelectSql(&sql, root, baserel, fpinfo->attrs_used, 698 | &retrieved_attrs); 699 | 700 | /* 701 | * Build the fdw_private list that will be available to the executor. 702 | * Items in the list must match enum CassFdwScanPrivateIndex, above. 703 | */ 704 | fdw_private = list_make2(makeString(sql.data), 705 | retrieved_attrs); 706 | 707 | /* 708 | * Create the ForeignScan node from target list, local filtering 709 | * expressions, remote parameter expressions, and FDW private information. 710 | * 711 | * Note that the remote parameter expressions are stored in the fdw_exprs 712 | * field of the finished plan node; we can't keep them in private state 713 | * because then they wouldn't be subject to later planner processing. 714 | */ 715 | return make_foreignscan(tlist, 716 | local_exprs, 717 | scan_relid, 718 | NIL, 719 | fdw_private 720 | #if (PG_VERSION_NUM >= 90500) 721 | , 722 | NIL, /* no custom tlist */ 723 | NIL, /* no remote quals */ 724 | outer_plan 725 | #endif 726 | ); 727 | } 728 | 729 | //#endif /* #if (PG_VERSION_NUM >= 90200) */ 730 | 731 | /* 732 | * cassExplainForeignScan 733 | * Produce extra output for EXPLAIN 734 | */ 735 | static void 736 | cassExplainForeignScan(ForeignScanState *node, ExplainState *es) 737 | { 738 | List *fdw_private; 739 | char *sql; 740 | 741 | char *svr_host = NULL; 742 | int svr_port = 0; 743 | char *svr_username = NULL; 744 | char *svr_password = NULL; 745 | char *svr_query = NULL; 746 | char *svr_table = NULL; 747 | 748 | if (es->verbose) 749 | { 750 | /* Fetch options */ 751 | cassGetOptions(RelationGetRelid(node->ss.ss_currentRelation), 752 | &svr_host, &svr_port, 753 | &svr_username, &svr_password, 754 | &svr_query, &svr_table); 755 | 756 | fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private; 757 | sql = strVal(list_nth(fdw_private, CassFdwScanPrivateSelectSql)); 758 | ExplainPropertyText("Remote SQL", sql, es); 759 | } 760 | } 761 | 762 | 763 | /* 764 | * cassBeginForeignScan 765 | * Initiate access to the database 766 | */ 767 | static void 768 | cassBeginForeignScan(ForeignScanState *node, int eflags) 769 | { 770 | ForeignScan *fsplan = (ForeignScan *) node->ss.ps.plan; 771 | EState *estate = node->ss.ps.state; 772 | CassFdwScanState *fsstate; 773 | RangeTblEntry *rte; 774 | Oid userid; 775 | ForeignTable *table; 776 | ForeignServer *server; 777 | UserMapping *user; 778 | 779 | /* 780 | * Do nothing in EXPLAIN (no ANALYZE) case. node->fdw_state stays NULL. 781 | */ 782 | if (eflags & EXEC_FLAG_EXPLAIN_ONLY) 783 | return; 784 | 785 | /* 786 | * We'll save private state in node->fdw_state. 787 | */ 788 | fsstate = (CassFdwScanState *) palloc0(sizeof(CassFdwScanState)); 789 | node->fdw_state = (void *) fsstate; 790 | 791 | /* 792 | * Identify which user to do the remote access as. This should match what 793 | * ExecCheckRTEPerms() does. 794 | */ 795 | rte = rt_fetch(fsplan->scan.scanrelid, estate->es_range_table); 796 | userid = rte->checkAsUser ? rte->checkAsUser : GetUserId(); 797 | 798 | /* Get info about foreign table. */ 799 | fsstate->rel = node->ss.ss_currentRelation; 800 | table = GetForeignTable(RelationGetRelid(fsstate->rel)); 801 | server = GetForeignServer(table->serverid); 802 | user = GetUserMapping(userid, server->serverid); 803 | 804 | /* 805 | * Get connection to the foreign server. Connection manager will 806 | * establish new connection if necessary. 807 | */ 808 | fsstate->cass_conn = pgcass_GetConnection(server, user, false); 809 | fsstate->sql_sended = false; 810 | 811 | /* Get private info created by planner functions. */ 812 | fsstate->query = strVal(list_nth(fsplan->fdw_private, 813 | CassFdwScanPrivateSelectSql)); 814 | fsstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private, 815 | CassFdwScanPrivateRetrievedAttrs); 816 | 817 | /* Create contexts for batches of tuples and per-tuple temp workspace. */ 818 | fsstate->batch_cxt = AllocSetContextCreate(estate->es_query_cxt, 819 | "cassandra2_fdw tuple data", 820 | ALLOCSET_DEFAULT_MINSIZE, 821 | ALLOCSET_DEFAULT_INITSIZE, 822 | ALLOCSET_DEFAULT_MAXSIZE); 823 | 824 | fsstate->temp_cxt = AllocSetContextCreate(estate->es_query_cxt, 825 | "cassandra2_fdw temporary data", 826 | ALLOCSET_SMALL_MINSIZE, 827 | ALLOCSET_SMALL_INITSIZE, 828 | ALLOCSET_SMALL_MAXSIZE); 829 | 830 | /* Get info we'll need for input data conversion. */ 831 | fsstate->attinmeta = TupleDescGetAttInMetadata(RelationGetDescr(fsstate->rel)); 832 | } 833 | 834 | 835 | /* 836 | * cassIterateForeignScan 837 | * Read next record from the data file and store it into the 838 | * ScanTupleSlot as a virtual tuple 839 | */ 840 | static TupleTableSlot* 841 | cassIterateForeignScan(ForeignScanState *node) 842 | { 843 | CassFdwScanState *fsstate = (CassFdwScanState *) node->fdw_state; 844 | TupleTableSlot *slot = node->ss.ss_ScanTupleSlot; 845 | 846 | /* 847 | * If this is the first call after Begin or ReScan, we need to create the 848 | * cursor on the remote side. 849 | */ 850 | if (!fsstate->sql_sended) 851 | create_cursor(node); 852 | 853 | /* 854 | * Get some more tuples, if we've run out. 855 | */ 856 | if (fsstate->next_tuple >= fsstate->num_tuples) 857 | { 858 | /* No point in another fetch if we already detected EOF, though. */ 859 | if (!fsstate->eof_reached) 860 | fetch_more_data(node); 861 | /* If we didn't get any tuples, must be end of data. */ 862 | if (fsstate->next_tuple >= fsstate->num_tuples) 863 | return ExecClearTuple(slot); 864 | } 865 | 866 | /* 867 | * Return the next tuple. 868 | */ 869 | #if (PG_VERSION_NUM < 120000) 870 | ExecStoreTuple(fsstate->tuples[fsstate->next_tuple++], 871 | slot, 872 | InvalidBuffer, 873 | false); 874 | #else 875 | ExecStoreHeapTuple(fsstate->tuples[fsstate->next_tuple++], 876 | slot, 877 | false); 878 | #endif 879 | 880 | return slot; 881 | } 882 | 883 | /* 884 | * cassReScanForeignScan 885 | * Rescan table, possibly with new parameters 886 | */ 887 | static void 888 | cassReScanForeignScan(ForeignScanState *node) 889 | { 890 | CassFdwScanState *fsstate = (CassFdwScanState *) node->fdw_state; 891 | 892 | /* If we haven't created the cursor yet, nothing to do. */ 893 | if (!fsstate->sql_sended) 894 | return; 895 | 896 | { 897 | /* Easy: just rescan what we already have in memory, if anything */ 898 | fsstate->next_tuple = 0; 899 | return; 900 | } 901 | 902 | /* Now force a fresh FETCH. */ 903 | fsstate->tuples = NULL; 904 | fsstate->num_tuples = 0; 905 | fsstate->next_tuple = 0; 906 | fsstate->fetch_ct_2 = 0; 907 | fsstate->eof_reached = false; 908 | } 909 | 910 | /* 911 | * cassEndForeignScan 912 | * Finish scanning foreign table and dispose objects used for this scan 913 | */ 914 | static void 915 | cassEndForeignScan(ForeignScanState *node) 916 | { 917 | CassFdwScanState *fsstate = (CassFdwScanState *) node->fdw_state; 918 | 919 | /* if fsstate is NULL, we are in EXPLAIN; nothing to do */ 920 | if (fsstate == NULL) 921 | return; 922 | 923 | /* Close the cursor if open, to prevent accumulation of cursors */ 924 | if (fsstate->sql_sended) 925 | close_cursor(fsstate); 926 | 927 | /* Release remote connection */ 928 | pgcass_ReleaseConnection(fsstate->cass_conn); 929 | fsstate->cass_conn = NULL; 930 | 931 | /* MemoryContexts will be deleted automatically. */ 932 | } 933 | 934 | 935 | /* 936 | * Create cursor for node's query with current parameter values. 937 | */ 938 | static void 939 | create_cursor(ForeignScanState *node) 940 | { 941 | CassFdwScanState *fsstate = (CassFdwScanState *) node->fdw_state; 942 | 943 | /* Build statement and execute query */ 944 | fsstate->statement = cass_statement_new(fsstate->query, 0); 945 | 946 | /* Mark the cursor as created, and show no tuples have been retrieved */ 947 | fsstate->sql_sended = true; 948 | fsstate->tuples = NULL; 949 | fsstate->num_tuples = 0; 950 | fsstate->next_tuple = 0; 951 | fsstate->fetch_ct_2 = 0; 952 | fsstate->eof_reached = false; 953 | } 954 | 955 | /* 956 | * Utility routine to close a cursor. 957 | */ 958 | static void 959 | close_cursor(CassFdwScanState *fsstate) 960 | { 961 | if (fsstate->statement) 962 | cass_statement_free(fsstate->statement); 963 | } 964 | 965 | /* 966 | * Fetch some more rows from the node's cursor. 967 | */ 968 | static void 969 | fetch_more_data(ForeignScanState *node) 970 | { 971 | CassFdwScanState *fsstate = (CassFdwScanState *) node->fdw_state; 972 | 973 | /* 974 | * We'll store the tuples in the batch_cxt. First, flush the previous 975 | * batch. 976 | */ 977 | fsstate->tuples = NULL; 978 | MemoryContextReset(fsstate->batch_cxt); 979 | MemoryContextSwitchTo(fsstate->batch_cxt); 980 | 981 | { 982 | CassFuture* result_future = cass_session_execute(fsstate->cass_conn, fsstate->statement); 983 | if (cass_future_error_code(result_future) == CASS_OK) 984 | { 985 | const CassResult* res; 986 | int numrows; 987 | CassIterator* rows; 988 | int k; 989 | 990 | /* Retrieve result set and iterate over the rows */ 991 | res = cass_future_get_result(result_future); 992 | 993 | /* Stash away the state info we have already */ 994 | fsstate->NumberOfColumns = cass_result_column_count(res); 995 | 996 | /* Convert the data into HeapTuples */ 997 | numrows = cass_result_row_count(res); 998 | fsstate->tuples = (HeapTuple *) palloc0(numrows * sizeof(HeapTuple)); 999 | fsstate->num_tuples = numrows; 1000 | fsstate->next_tuple = 0; 1001 | 1002 | rows = cass_iterator_from_result(res); 1003 | k = 0; 1004 | while (cass_iterator_next(rows)) 1005 | { 1006 | const CassRow* row = cass_iterator_get_row(rows); 1007 | 1008 | fsstate->tuples[k] = make_tuple_from_result_row(row, 1009 | fsstate->NumberOfColumns, 1010 | fsstate->rel, 1011 | fsstate->attinmeta, 1012 | fsstate->retrieved_attrs, 1013 | fsstate->temp_cxt); 1014 | 1015 | Assert(k < numrows); 1016 | k++; 1017 | } 1018 | 1019 | fsstate->eof_reached = true; 1020 | 1021 | cass_result_free(res); 1022 | cass_iterator_free(rows); 1023 | } 1024 | else 1025 | { 1026 | /* On error, report the original query. */ 1027 | pgcass_report_error(ERROR, result_future, true, fsstate->query); 1028 | 1029 | fsstate->eof_reached = true; 1030 | } 1031 | 1032 | cass_future_free(result_future); 1033 | } 1034 | } 1035 | 1036 | static HeapTuple 1037 | make_tuple_from_result_row(const CassRow* row, 1038 | int ncolumn, 1039 | Relation rel, 1040 | AttInMetadata *attinmeta, 1041 | List *retrieved_attrs, 1042 | MemoryContext temp_context) 1043 | { 1044 | HeapTuple tuple; 1045 | TupleDesc tupdesc = RelationGetDescr(rel); 1046 | Datum *values; 1047 | bool *nulls; 1048 | MemoryContext oldcontext; 1049 | ListCell *lc; 1050 | int j; 1051 | StringInfoData buf; 1052 | 1053 | /* 1054 | * Do the following work in a temp context that we reset after each tuple. 1055 | * This cleans up not only the data we have direct access to, but any 1056 | * cruft the I/O functions might leak. 1057 | */ 1058 | oldcontext = MemoryContextSwitchTo(temp_context); 1059 | 1060 | values = (Datum *) palloc0(tupdesc->natts * sizeof(Datum)); 1061 | nulls = (bool *) palloc(tupdesc->natts * sizeof(bool)); 1062 | /* Initialize to nulls for any columns not present in result */ 1063 | memset(nulls, true, tupdesc->natts * sizeof(bool)); 1064 | 1065 | initStringInfo(&buf); 1066 | 1067 | /* 1068 | * i indexes columns in the relation, j indexes columns in the PGresult. 1069 | */ 1070 | j = 0; 1071 | foreach(lc, retrieved_attrs) 1072 | { 1073 | int i = lfirst_int(lc); 1074 | const char *valstr; 1075 | 1076 | const CassValue* cassVal = cass_row_get_column(row, j); 1077 | if (cass_true == cass_value_is_null(cassVal)) 1078 | valstr = NULL; 1079 | else 1080 | { 1081 | pgcass_transferValue(&buf, cassVal); 1082 | valstr = buf.data; 1083 | } 1084 | 1085 | if (i > 0) 1086 | { 1087 | /* ordinary column */ 1088 | Assert(i <= tupdesc->natts); 1089 | nulls[i - 1] = (valstr == NULL); 1090 | /* Apply the input function even to nulls, to support domains */ 1091 | values[i - 1] = InputFunctionCall(&attinmeta->attinfuncs[i - 1], 1092 | (char *) valstr, 1093 | attinmeta->attioparams[i - 1], 1094 | attinmeta->atttypmods[i - 1]); 1095 | } 1096 | 1097 | resetStringInfo(&buf); 1098 | 1099 | j++; 1100 | } 1101 | 1102 | /* 1103 | * Check we got the expected number of columns. Note: j == 0 and 1104 | * PQnfields == 1 is expected, since deparse emits a NULL if no columns. 1105 | */ 1106 | if (j > 0 && j != ncolumn) 1107 | elog(ERROR, "remote query result does not match the foreign table"); 1108 | 1109 | /* 1110 | * Build the result tuple in caller's memory context. 1111 | */ 1112 | MemoryContextSwitchTo(oldcontext); 1113 | 1114 | tuple = heap_form_tuple(tupdesc, values, nulls); 1115 | 1116 | /* Clean up */ 1117 | MemoryContextReset(temp_context); 1118 | 1119 | return tuple; 1120 | } 1121 | 1122 | static void 1123 | pgcass_transferValue(StringInfo buf, const CassValue* value) 1124 | { 1125 | CassValueType type = cass_value_type(value); 1126 | switch (type) 1127 | { 1128 | case CASS_VALUE_TYPE_SMALL_INT: 1129 | case CASS_VALUE_TYPE_TINY_INT: 1130 | { 1131 | cass_int16_t i; 1132 | cass_value_get_int16(value, &i); 1133 | appendStringInfo(buf, "%d", i); 1134 | break; 1135 | } 1136 | case CASS_VALUE_TYPE_INT: 1137 | { 1138 | cass_int32_t i; 1139 | cass_value_get_int32(value, &i); 1140 | appendStringInfo(buf, "%d", i); 1141 | break; 1142 | } 1143 | case CASS_VALUE_TYPE_BIGINT: 1144 | case CASS_VALUE_TYPE_TIMESTAMP: 1145 | { 1146 | cass_int64_t i; 1147 | cass_value_get_int64(value, &i); 1148 | appendStringInfo(buf, "%ld ", i); 1149 | break; 1150 | } 1151 | case CASS_VALUE_TYPE_BOOLEAN: 1152 | { 1153 | cass_bool_t b; 1154 | cass_value_get_bool(value, &b); 1155 | appendStringInfoString(buf, b ? "true" : "false"); 1156 | break; 1157 | } 1158 | case CASS_VALUE_TYPE_FLOAT: 1159 | { 1160 | cass_float_t f; 1161 | cass_value_get_float(value, &f); 1162 | appendStringInfo(buf, "%f", f); 1163 | break; 1164 | } 1165 | case CASS_VALUE_TYPE_DOUBLE: 1166 | { 1167 | cass_double_t d; 1168 | cass_value_get_double(value, &d); 1169 | appendStringInfo(buf, "%f", d); 1170 | break; 1171 | } 1172 | 1173 | case CASS_VALUE_TYPE_TEXT: 1174 | case CASS_VALUE_TYPE_ASCII: 1175 | case CASS_VALUE_TYPE_VARCHAR: 1176 | { 1177 | const char* s; 1178 | size_t s_length; 1179 | cass_value_get_string(value, &s, &s_length); 1180 | appendStringInfo(buf, "%.*s", (int)s_length, s); 1181 | break; 1182 | } 1183 | case CASS_VALUE_TYPE_UUID: 1184 | { 1185 | CassUuid u; 1186 | 1187 | cass_value_get_uuid(value, &u); 1188 | cass_uuid_string(u, buf->data + buf->len); 1189 | buf->len += CASS_UUID_STRING_LENGTH; 1190 | buf->data[buf->len] = '\0'; 1191 | break; 1192 | } 1193 | case CASS_VALUE_TYPE_SET: 1194 | case CASS_VALUE_TYPE_LIST: 1195 | case CASS_VALUE_TYPE_MAP: 1196 | default: 1197 | appendStringInfoString(buf, ""); 1198 | break; 1199 | } 1200 | } 1201 | 1202 | 1203 | static void deparseTargetList(StringInfo buf, 1204 | PlannerInfo *root, 1205 | Index rtindex, 1206 | Relation rel, 1207 | Bitmapset *attrs_used, 1208 | List **retrieved_attrs); 1209 | static void deparseColumnRef(StringInfo buf, int varno, int varattno, 1210 | PlannerInfo *root); 1211 | static void deparseRelation(StringInfo buf, Relation rel); 1212 | 1213 | /* 1214 | * Examine each qual clause in input_conds, and classify them into two groups, 1215 | * which are returned as two lists: 1216 | * - remote_conds contains expressions that can be evaluated remotely 1217 | * - local_conds contains expressions that can't be evaluated remotely 1218 | */ 1219 | static void 1220 | classifyConditions(PlannerInfo *root, 1221 | RelOptInfo *baserel, 1222 | List *input_conds, 1223 | List **remote_conds, 1224 | List **local_conds) 1225 | { 1226 | ListCell *lc; 1227 | 1228 | *remote_conds = NIL; 1229 | *local_conds = NIL; 1230 | 1231 | foreach(lc, input_conds) 1232 | { 1233 | RestrictInfo *ri = (RestrictInfo *) lfirst(lc); 1234 | 1235 | *local_conds = lappend(*local_conds, ri); 1236 | } 1237 | } 1238 | 1239 | /* 1240 | * Construct a simple SELECT statement that retrieves desired columns 1241 | * of the specified foreign table, and append it to "buf". The output 1242 | * contains just "SELECT ... FROM tablename". 1243 | * 1244 | * We also create an integer List of the columns being retrieved, which is 1245 | * returned to *retrieved_attrs. 1246 | */ 1247 | static void 1248 | deparseSelectSql(StringInfo buf, 1249 | PlannerInfo *root, 1250 | RelOptInfo *baserel, 1251 | Bitmapset *attrs_used, 1252 | List **retrieved_attrs) 1253 | { 1254 | RangeTblEntry *rte = planner_rt_fetch(baserel->relid, root); 1255 | Relation rel; 1256 | 1257 | /* 1258 | * Core code already has some lock on each rel being planned, so we can 1259 | * use NoLock here. 1260 | */ 1261 | rel = table_open(rte->relid, NoLock); 1262 | 1263 | /* 1264 | * Construct SELECT list 1265 | */ 1266 | appendStringInfoString(buf, "SELECT "); 1267 | deparseTargetList(buf, root, baserel->relid, rel, attrs_used, 1268 | retrieved_attrs); 1269 | 1270 | /* 1271 | * Construct FROM clause 1272 | */ 1273 | appendStringInfoString(buf, " FROM "); 1274 | deparseRelation(buf, rel); 1275 | 1276 | #if PG_VERSION_NUM < 120000 1277 | heap_close(rel, NoLock); 1278 | #else 1279 | table_close(rel, NoLock); 1280 | #endif 1281 | } 1282 | 1283 | 1284 | /* 1285 | * Emit a target list that retrieves the columns specified in attrs_used. 1286 | * This is used for both SELECT and RETURNING targetlists. 1287 | * 1288 | * The tlist text is appended to buf, and we also create an integer List 1289 | * of the columns being retrieved, which is returned to *retrieved_attrs. 1290 | */ 1291 | static void 1292 | deparseTargetList(StringInfo buf, 1293 | PlannerInfo *root, 1294 | Index rtindex, 1295 | Relation rel, 1296 | Bitmapset *attrs_used, 1297 | List **retrieved_attrs) 1298 | { 1299 | TupleDesc tupdesc = RelationGetDescr(rel); 1300 | bool have_wholerow; 1301 | bool first; 1302 | int i; 1303 | 1304 | *retrieved_attrs = NIL; 1305 | 1306 | /* If there's a whole-row reference, we'll need all the columns. */ 1307 | have_wholerow = bms_is_member(0 - FirstLowInvalidHeapAttributeNumber, 1308 | attrs_used); 1309 | 1310 | first = true; 1311 | for (i = 1; i <= tupdesc->natts; i++) 1312 | { 1313 | Form_pg_attribute attr = TupleDescAttr(tupdesc, i - 1); 1314 | 1315 | /* Ignore dropped attributes. */ 1316 | if (attr->attisdropped) 1317 | continue; 1318 | 1319 | if (have_wholerow || 1320 | bms_is_member(i - FirstLowInvalidHeapAttributeNumber, 1321 | attrs_used)) 1322 | { 1323 | if (!first) 1324 | appendStringInfoString(buf, ", "); 1325 | first = false; 1326 | 1327 | deparseColumnRef(buf, rtindex, i, root); 1328 | 1329 | *retrieved_attrs = lappend_int(*retrieved_attrs, i); 1330 | } 1331 | } 1332 | 1333 | /* 1334 | * Add ctid if needed. We currently don't support retrieving any other 1335 | * system columns. 1336 | */ 1337 | if (bms_is_member(SelfItemPointerAttributeNumber - FirstLowInvalidHeapAttributeNumber, 1338 | attrs_used)) 1339 | { 1340 | if (!first) 1341 | appendStringInfoString(buf, ", "); 1342 | first = false; 1343 | 1344 | appendStringInfoString(buf, "ctid"); 1345 | 1346 | *retrieved_attrs = lappend_int(*retrieved_attrs, 1347 | SelfItemPointerAttributeNumber); 1348 | } 1349 | 1350 | /* Don't generate bad syntax if no undropped columns */ 1351 | if (first) 1352 | appendStringInfoString(buf, "NULL"); 1353 | } 1354 | 1355 | /* 1356 | * Construct name to use for given column, and emit it into buf. 1357 | * If it has a column_name FDW option, use that instead of attribute name. 1358 | */ 1359 | static void 1360 | deparseColumnRef(StringInfo buf, int varno, int varattno, PlannerInfo *root) 1361 | { 1362 | RangeTblEntry *rte; 1363 | char *colname = NULL; 1364 | List *options; 1365 | ListCell *lc; 1366 | 1367 | /* varno must not be any of OUTER_VAR, INNER_VAR and INDEX_VAR. */ 1368 | Assert(!IS_SPECIAL_VARNO(varno)); 1369 | 1370 | /* Get RangeTblEntry from array in PlannerInfo. */ 1371 | rte = planner_rt_fetch(varno, root); 1372 | 1373 | /* 1374 | * If it's a column of a foreign table, and it has the column_name FDW 1375 | * option, use that value. 1376 | */ 1377 | options = GetForeignColumnOptions(rte->relid, varattno); 1378 | foreach(lc, options) 1379 | { 1380 | DefElem *def = (DefElem *) lfirst(lc); 1381 | 1382 | if (strcmp(def->defname, "column_name") == 0) 1383 | { 1384 | colname = defGetString(def); 1385 | break; 1386 | } 1387 | } 1388 | 1389 | /* 1390 | * If it's a column of a regular table or it doesn't have column_name FDW 1391 | * option, use attribute name. 1392 | */ 1393 | if (colname == NULL) 1394 | colname = get_relid_attribute_name(rte->relid, varattno); 1395 | 1396 | appendStringInfoString(buf, quote_identifier(colname)); 1397 | } 1398 | 1399 | 1400 | /* 1401 | * Append remote name of specified foreign table to buf. 1402 | * Use value of table_name FDW option (if any) instead of relation's name. 1403 | * Similarly, schema_name FDW option overrides schema name. 1404 | */ 1405 | static void 1406 | deparseRelation(StringInfo buf, Relation rel) 1407 | { 1408 | ForeignTable *table; 1409 | const char *nspname = NULL; 1410 | const char *relname = NULL; 1411 | ListCell *lc; 1412 | 1413 | /* obtain additional catalog information. */ 1414 | table = GetForeignTable(RelationGetRelid(rel)); 1415 | 1416 | /* 1417 | * Use value of FDW options if any, instead of the name of object itself. 1418 | */ 1419 | foreach(lc, table->options) 1420 | { 1421 | DefElem *def = (DefElem *) lfirst(lc); 1422 | 1423 | if (strcmp(def->defname, "schema_name") == 0) 1424 | nspname = defGetString(def); 1425 | else if (strcmp(def->defname, "table_name") == 0) 1426 | relname = defGetString(def); 1427 | } 1428 | 1429 | /* 1430 | * Note: we could skip printing the schema name if it's pg_catalog, but 1431 | * that doesn't seem worth the trouble. 1432 | */ 1433 | if (nspname == NULL) 1434 | nspname = get_namespace_name(RelationGetNamespace(rel)); 1435 | if (relname == NULL) 1436 | relname = RelationGetRelationName(rel); 1437 | 1438 | appendStringInfo(buf, "%s.%s", 1439 | quote_identifier(nspname), quote_identifier(relname)); 1440 | } 1441 | -------------------------------------------------------------------------------- /cassandra2_fdw.control: -------------------------------------------------------------------------------- 1 | # cassandra2_fdw extension 2 | comment = 'foreign-data wrapper for querying Cassandra 2+' 3 | default_version = '2.0' 4 | module_pathname = '$libdir/cassandra2_fdw' 5 | relocatable = true 6 | -------------------------------------------------------------------------------- /cassandra2_fdw.h: -------------------------------------------------------------------------------- 1 | /*------------------------------------------------------------------------- 2 | * 3 | * cassandra2_fdw.h 4 | * 5 | * Copyright (c) 2015 Jaimin Pan 6 | * 7 | * This software is released under the PostgreSQL Licence 8 | * 9 | * Author: Jaimin Pan 10 | * 11 | * IDENTIFICATION 12 | * cassandra2_fdw/cassandra2_fdw.h 13 | * 14 | *------------------------------------------------------------------------- 15 | */ 16 | 17 | #ifndef CASSANDRA2_FDW_H_ 18 | #define CASSANDRA2_FDW_H_ 19 | 20 | #include 21 | 22 | #include "foreign/foreign.h" 23 | #include "lib/stringinfo.h" 24 | #include "nodes/pathnodes.h" 25 | #include "utils/rel.h" 26 | 27 | /* in cass_connection.c */ 28 | extern CassSession *pgcass_GetConnection(ForeignServer *server, UserMapping *user, 29 | bool will_prep_stmt); 30 | extern void pgcass_ReleaseConnection(CassSession *session); 31 | 32 | extern void pgcass_report_error(int elevel, CassFuture* result_future, 33 | bool clear, const char *sql); 34 | 35 | #endif /* CASSANDRA2_FDW_H_ */ 36 | -------------------------------------------------------------------------------- /test/README.md: -------------------------------------------------------------------------------- 1 | # Testing the Foreign Data Wraper 2 | 3 | ## Source Of The Cassandra Docker Images 4 | 5 | The Cassandra Docker images are available at [cassandra|Docker Hub](https://hub.docker.com/_/cassandra/). 6 | 7 | ## Source Of The PostgreSQL Docker Images 8 | 9 | The Cassandra Docker images are available at [PostgreSQL|Docker Hub](https://hub.docker.com/_/postgres/). 10 | 11 | ## Test Scipts 12 | ### build.cfg 13 | 14 | Contains the configuration of all parameters used in the tests. 15 | 16 | #### Parameters 17 | 18 | - POSTGRESQL_VERSIONS: An array with all PostgreSQL version numbers, that should be tested against 19 | - POSTGRESQL_HOST: The host where PostgreSQL connections are opened: _localhost_ 20 | - POSTGRESQL_PORT: The port where PostgreSQL is listening: _5432_ 21 | - POSTGRESQL_USER: The user, that is used to connect to PostgreSQL: fdw_tester 22 | - POSTGRESQL_PWD: The password, which is used to create the user in **POSTGRESQL_USER** 23 | - POSTGRESQL_DB1: Default database: _postgres_ 24 | - POSTGRESQL_DB2: Test database 25 | - PostgreSQL_RESULT_VERSION: The PostgreSQL version of the database, where the query results will be stored: _15_ 26 | - POSTGRESQL_RESULT_DB: The PostgreSQL database, where the query results will be stored: _results_ 27 | - POSTGRESQL_RESULT_SCHEMA: The schema inside the PostgreSQL database, where the query results will be stored: _result_data_ 28 | - CASSANDRA_PORT: The port where Cassandra is listening: _9042_ 29 | - CASSANDRA_VERSIONS: An array with all Cassandra version numbers, that should be tested against 30 | - DOCKER_NETWORK: The network, that is used by all test containers 31 | 32 | ### test.sh 33 | 34 | This script executes all tests. 35 | 36 | ```bash 37 | # Execute the tests 38 | ./test.sh 39 | ``` 40 | 41 | ### setup_img.sh 42 | 43 | The script creates all Docker images as configured in [build.cfg](#buildcfg). Within the PostgreSQL container the packages are updated and all needed packages will be installed, that are needed to compile the foreign data wrapper. It will compile the datastax Cassandra lib, that is needed to compile the extension.
44 | The installation runs in all PostgreSQL containers. 45 | 46 | ### cassandra.sh 47 | 48 | The scripts to initialize the Cassandra data for every Container created in setup_img.sh and configured in [build.cfg](#buildcfg). 49 | 50 | Within a loop for every Cassandra container configured, it will call the [postgresql.sh](#postgresqlsh). 51 | 52 | ### postgresql.sh 53 | 54 | The script creates a database in a PostgreSQL container and adds all objects to access the Cassandra container given as IP address as parameter within [cassandra.sh](#cassandrash). 55 | 56 | ### cleanup.sh 57 | 58 | The script stops and deletes all containers configured in [build.cfg](#buildcfg). It can also delete the Docker network, where all containers are running with. 59 | 60 | By default the result container is not removed. The shell script interprets **1** given as parameter as to remove the container with the result database, too.
61 | As there are no contianers left when started with **1**, the docker network will be removed, too. 62 | 63 | This can be achieved by calling this script manually: 64 | 65 | ```bash 66 | # Remove all containers used for testing and the network 67 | ./cleanup.sh 1 68 | ``` 69 | 70 | ## Checking Test Results 71 | 72 | The test results are stored in a container with PostgreSQL as database. 73 | 74 | For every test the following data is stored: 75 | 76 | - created: TIMESTAMP WITH TIME ZONE: Execution timestamp 77 | - postgresql_version: TEXT: Name of the PostgreSQL container which does by default contain the version number 78 | - cassandra_version: TEXT: Name of the Cassandra container which does by default contain the version number 79 | - test_description: TEXT: Name/description of the test 80 | - test_result: TEXT: Data read from the Cassandra database, fields are concatinated, separator is pipe 81 | 82 | ```sql 83 | -- Count of tests for each version 84 | SELECT postgresql_version 85 | , cassandra_version 86 | , count(*) AS count_of_tests 87 | FROM result_data.results 88 | GROUP BY postgresql_version 89 | , cassandra_version 90 | ORDER BY postgresql_version 91 | , cassandra_version 92 | ; 93 | ``` 94 | -------------------------------------------------------------------------------- /test/build.cfg: -------------------------------------------------------------------------------- 1 | # Configuration for running tests of the cassandra2_fdw, details are in the README 2 | ## PostgreSQL 3 | POSTGRESQL_VERSIONS=("12" "13" "14" "15") 4 | POSTGRESQL_HOST="localhost" 5 | POSTGRESQL_PORT="5432" 6 | POSTGRESQL_USER="fdw_tester" 7 | POSTGRESQL_PWD="KliewuioPkk83j" 8 | POSTGRESQL_DB1="postgres" 9 | POSTGRESQL_DB2="cassandra_fdw" 10 | PostgreSQL_RESULT_VERSION="15" 11 | POSTGRESQL_RESULT_DB="results" 12 | POSTGRESQL_RESULT_SCHEMA="result_data" 13 | ## Cassandra 14 | CASSANDRA_PORT="9042" 15 | CASSANDRA_VERSIONS=("4.0" "3.11" "3.0") 16 | ## Docker 17 | DOCKER_NETWORK="fdw_test_network" 18 | -------------------------------------------------------------------------------- /test/cassandra.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Set variables 4 | source ./build.cfg 5 | 6 | for CASSANDRA_VERSION in "${CASSANDRA_VERSIONS[@]}" 7 | do 8 | # Set container name and image 9 | CASSANDRA_CONTAINER="Cassandra_${CASSANDRA_VERSION}_N" 10 | CASSANDRA_CONTAINER=${CASSANDRA_CONTAINER/./_} 11 | 12 | echo "Creating keyspace and tables in $CASSANDRA_CONTAINER" 13 | 14 | # Drop keyspace if exists to start from scratch 15 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --execute="drop keyspace if exists cassandra_fdw;" 16 | 17 | # Create the cassandra_fdw keyspace 18 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --execute="CREATE KEYSPACE cassandra_fdw WITH replication = {'class':'SimpleStrategy', 'replication_factor': '1'} AND durable_writes = true;" 19 | 20 | # Create a tables 21 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="CREATE TABLE fdw_test_int(id int, some_text text, PRIMARY KEY (id));" 22 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="CREATE TABLE fdw_test_bigint(id bigint, some_text text, PRIMARY KEY (id));" 23 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="CREATE TABLE fdw_test_smallint(id smallint, some_text text, PRIMARY KEY (id));" 24 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="CREATE TABLE fdw_test_tinyint(id tinyint, some_text text, PRIMARY KEY (id));" 25 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="CREATE TABLE fdw_test_boolean(id boolean, some_text text, PRIMARY KEY (id));" 26 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="CREATE TABLE fdw_test_float(id float, some_text text, PRIMARY KEY (id));" 27 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="CREATE TABLE fdw_test_double(id double, some_text text, PRIMARY KEY (id));" 28 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="CREATE TABLE fdw_test_uuid(id uuid, some_text text, PRIMARY KEY (id));" 29 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="CREATE TABLE fdw_test_ascii(id uuid, some_text ascii, PRIMARY KEY (id));" 30 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="CREATE TABLE fdw_test_varchar(id uuid, some_text varchar, PRIMARY KEY (id));" 31 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="CREATE TABLE fdw_test_set(id uuid, some_text set , PRIMARY KEY (id));" 32 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="CREATE TABLE fdw_test_timestamp(id uuid, some_timestamp timestamp, PRIMARY KEY (id));" 33 | 34 | echo "Inserting data into tables in $CASSANDRA_CONTAINER" 35 | 36 | # Insert some records 37 | # bigint 38 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_bigint(id, some_text) VALUES (1, 'test 1');" 39 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_bigint(id, some_text) VALUES (2, 'test 2');" 40 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_bigint(id, some_text) VALUES (3, 'test 3');" 41 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_bigint(id, some_text) VALUES (4, 'test 4');" 42 | 43 | # integer 44 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_int(id, some_text) VALUES (1, 'test 1');" 45 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_int(id, some_text) VALUES (2, 'test 2');" 46 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_int(id, some_text) VALUES (3, 'test 3');" 47 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_int(id, some_text) VALUES (4, 'test 4');" 48 | 49 | # smallint 50 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_smallint(id, some_text) VALUES (1, 'test 1');" 51 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_smallint(id, some_text) VALUES (2, 'test 2');" 52 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_smallint(id, some_text) VALUES (3, 'test 3');" 53 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_smallint(id, some_text) VALUES (4, 'test 4');" 54 | 55 | # tinyint 56 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_tinyint(id, some_text) VALUES (1, 'test 1');" 57 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_tinyint(id, some_text) VALUES (2, 'test 2');" 58 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_tinyint(id, some_text) VALUES (3, 'test 3');" 59 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_tinyint(id, some_text) VALUES (4, 'test 4');" 60 | 61 | # boolean 62 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_boolean(id, some_text) VALUES (true, 'test 1');" 63 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_boolean(id, some_text) VALUES (false, 'test 2');" 64 | 65 | # float 66 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_float(id, some_text) VALUES (1.1, 'test 1');" 67 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_float(id, some_text) VALUES (2e10, 'test 2');" 68 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_float(id, some_text) VALUES (3, 'test 3');" 69 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_float(id, some_text) VALUES (4, 'test 4');" 70 | 71 | # double 72 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_double(id, some_text) VALUES (1.1, 'test 1');" 73 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_double(id, some_text) VALUES (2e10, 'test 2');" 74 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_double(id, some_text) VALUES (3, 'test 3');" 75 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_double(id, some_text) VALUES (4, 'test 4');" 76 | 77 | # uuid 78 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_uuid(id, some_text) VALUES (uuid(), 'test 1');" 79 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_uuid(id, some_text) VALUES (uuid(), 'test 2');" 80 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_uuid(id, some_text) VALUES (uuid(), 'test 3');" 81 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_uuid(id, some_text) VALUES (uuid(), 'test 4');" 82 | 83 | # ascii 84 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_ascii(id, some_text) VALUES (uuid(), 'test 1');" 85 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_ascii(id, some_text) VALUES (uuid(), 'test 2');" 86 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_ascii(id, some_text) VALUES (uuid(), 'test 3');" 87 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_ascii(id, some_text) VALUES (uuid(), 'test 4');" 88 | 89 | # varchar 90 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_varchar(id, some_text) VALUES (uuid(), 'test 1');" 91 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_varchar(id, some_text) VALUES (uuid(), 'test 2');" 92 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_varchar(id, some_text) VALUES (uuid(), 'test 3');" 93 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_varchar(id, some_text) VALUES (uuid(), 'test 4');" 94 | 95 | # timestamp 96 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_timestamp(id, some_timestamp) VALUES (uuid(), toTimestamp(now()));" 97 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_timestamp(id, some_timestamp) VALUES (uuid(), toTimestamp(now()));" 98 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_timestamp(id, some_timestamp) VALUES (uuid(), toTimestamp(now()));" 99 | sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --keyspace=cassandra_fdw --execute="INSERT INTO fdw_test_timestamp(id, some_timestamp) VALUES (uuid(), toTimestamp(now()));" 100 | 101 | # Getting IP address of the Cassandra server 102 | CASSANDRA_HOST=$(sudo docker exec $CASSANDRA_CONTAINER hostname -i) 103 | 104 | # Starting tests in PostgreSQL 105 | echo "Starting tests in PostgreSQL container with Cassandra using $CASSANDRA_HOST" 106 | ./postgresql.sh $CASSANDRA_HOST $CASSANDRA_CONTAINER 107 | 108 | # Drop keyspace 109 | #sudo docker exec -it $CASSANDRA_CONTAINER cqlsh --execute="drop keyspace if exists cassandra_fdw;" 110 | 111 | done 112 | -------------------------------------------------------------------------------- /test/cleanup.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Set variables 4 | source ./build.cfg 5 | 6 | # Get the IP address of the Cassandra server 7 | if [ -z "$1" ]; then 8 | REMOVE_RESULT_SERVER=0 9 | else 10 | REMOVE_RESULT_SERVER=1 11 | fi 12 | 13 | # Remove containers 14 | ## Cassandra 15 | for CASSANDRA_VERSION in "${CASSANDRA_VERSIONS[@]}" 16 | do 17 | # Set container name and image 18 | CASSANDRA_CONTAINER="Cassandra_${CASSANDRA_VERSION}_N" 19 | CASSANDRA_CONTAINER=${CASSANDRA_CONTAINER/./_} 20 | 21 | # Deleting the cassandra container 22 | echo "Deleting container $CASSANDRA_CONTAINER ..." 23 | sudo docker stop $CASSANDRA_CONTAINER 24 | sudo docker rm $CASSANDRA_CONTAINER 25 | done 26 | 27 | ## PostgreSQL 28 | for PG_VERSION in "${POSTGRESQL_VERSIONS[@]}" 29 | do 30 | # Set container names and PostgreSQL images 31 | POSTGRESQL_CONTAINER="PostgreSQL_${PG_VERSION}_N" 32 | 33 | echo "Deleting container $POSTGRESQL_CONTAINER ..." 34 | sudo docker stop $POSTGRESQL_CONTAINER 35 | sudo docker rm $POSTGRESQL_CONTAINER 36 | done 37 | 38 | # PostgreSQL result server 39 | if [ $REMOVE_RESULT_SERVER -eq 1 ]; then 40 | POSTGRESQL_RESULT_CONTAINER="PostgreSQL_RESULT_${PostgreSQL_RESULT_VERSION}_N" 41 | sudo docker stop $POSTGRESQL_RESULT_CONTAINER 42 | sudo docker rm $POSTGRESQL_RESULT_CONTAINER 43 | 44 | # Remove the network 45 | echo "Removing network $DOCKER_NETWORK ..." 46 | sudo docker network rm $DOCKER_NETWORK 47 | else 48 | echo "Result container $POSTGRESQL_RESULT_CONTAINER and docker network $DOCKER_NETWORK are still up!" 49 | fi 50 | -------------------------------------------------------------------------------- /test/postgresql.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Set variables 4 | source ./build.cfg 5 | 6 | # Get the IP address of the Cassandra server 7 | if [ -z "$1" ]; then 8 | echo "No Cassandra host IP address passed, will exit now" 9 | exit 1 10 | else 11 | CASSANDRA_HOST=$1 12 | fi 13 | 14 | # Get the container name of the Cassandra server 15 | if [ -z "$2" ]; then 16 | echo "No Cassandra container name passed, will exit now" 17 | exit 1 18 | else 19 | CASSANDRA_CONTAINER=$2 20 | fi 21 | 22 | 23 | for PG_VERSION in "${POSTGRESQL_VERSIONS[@]}" 24 | do 25 | # Set container names and PostgreSQL imagest 26 | POSTGRESQL_CONTAINER="PostgreSQL_${PG_VERSION}_N" 27 | POSTGRESQL_RESULT_CONTAINER="PostgreSQL_RESULT_${PostgreSQL_RESULT_VERSION}_N" 28 | POSTGRESQL_RESULT_HOST=$(sudo docker exec $POSTGRESQL_RESULT_CONTAINER hostname -i) 29 | 30 | 31 | echo "Starting PostgreSQL tests with container $POSTGRESQL_CONTAINER against Cassandra container with IP address $CASSANDRA_HOST" 32 | 33 | # Show PostgreSQL version 34 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB1 --command "SELECT version();" 35 | 36 | # Database creation 37 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB1 --command "DROP DATABASE IF EXISTS $POSTGRESQL_DB2;" 38 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB1 --command "CREATE DATABASE $POSTGRESQL_DB2;" 39 | 40 | # Foreign data wrapper 41 | ## PostgreSQL result server 42 | echo "Creating foreign connection to the PostgreSQL result container $POSTGRESQL_RESULT_CONTAINER with IP address $POSTGRESQL_RESULT_HOST" 43 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "CREATE EXTENSION postgres_fdw;" 44 | 45 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "CREATE SERVER result_serv FOREIGN DATA WRAPPER postgres_fdw OPTIONS (host '$POSTGRESQL_RESULT_HOST', port '$POSTGRESQL_PORT', dbname '$POSTGRESQL_RESULT_DB');" 46 | 47 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "CREATE USER MAPPING FOR $POSTGRESQL_USER server result_serv options (user '$POSTGRESQL_USER', password '$POSTGRESQL_PWD');" 48 | 49 | ## Create a schema for the foreign tables 50 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "CREATE SCHEMA $POSTGRESQL_RESULT_SCHEMA;" 51 | 52 | # Create foreign table for results 53 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "CREATE FOREIGN TABLE $POSTGRESQL_RESULT_SCHEMA.fdw_test_results(created TIMESTAMP WITH TIME ZONE, postgresql_version TEXT, cassandra_version TEXT, test_description TEXT, test_result TEXT) server result_serv options (schema_name '$POSTGRESQL_RESULT_SCHEMA', table_name 'results');" 54 | 55 | ## Current Cassandra server 56 | 57 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "CREATE EXTENSION cassandra2_fdw;" 58 | 59 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "CREATE SERVER cass_serv FOREIGN DATA WRAPPER cassandra2_fdw OPTIONS (host '$CASSANDRA_HOST', port '$CASSANDRA_PORT');" 60 | 61 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "CREATE USER MAPPING FOR $POSTGRESQL_USER server cass_serv options (username '', password '');" 62 | 63 | # Create a schema for the foreign tables 64 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "CREATE SCHEMA cassandra;" 65 | 66 | # Create foreign tables 67 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "CREATE FOREIGN TABLE cassandra.fdw_test_bigint(id bigint, some_text text) server cass_serv options (schema_name 'cassandra_fdw', table_name 'fdw_test_bigint');" 68 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "CREATE FOREIGN TABLE cassandra.fdw_test_int(id int, some_text text) server cass_serv options (schema_name 'cassandra_fdw', table_name 'fdw_test_int');" 69 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "CREATE FOREIGN TABLE cassandra.fdw_test_smallint(id smallint, some_text text) server cass_serv options (schema_name 'cassandra_fdw', table_name 'fdw_test_smallint');" 70 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "CREATE FOREIGN TABLE cassandra.fdw_test_tinyint(id smallint, some_text text) server cass_serv options (schema_name 'cassandra_fdw', table_name 'fdw_test_smallint');" 71 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "CREATE FOREIGN TABLE cassandra.fdw_test_boolean(id boolean, some_text text) server cass_serv options (schema_name 'cassandra_fdw', table_name 'fdw_test_boolean');" 72 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "CREATE FOREIGN TABLE cassandra.fdw_test_float(id float, some_text text) server cass_serv options (schema_name 'cassandra_fdw', table_name 'fdw_test_float');" 73 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "CREATE FOREIGN TABLE cassandra.fdw_test_double(id double precision, some_text text) server cass_serv options (schema_name 'cassandra_fdw', table_name 'fdw_test_double');" 74 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "CREATE FOREIGN TABLE cassandra.fdw_test_uuid(id uuid, some_text text) server cass_serv options (schema_name 'cassandra_fdw', table_name 'fdw_test_uuid');" 75 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "CREATE FOREIGN TABLE cassandra.fdw_test_ascii(id uuid, some_text text) server cass_serv options (schema_name 'cassandra_fdw', table_name 'fdw_test_ascii');" 76 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "CREATE FOREIGN TABLE cassandra.fdw_test_varchar(id uuid, some_text text) server cass_serv options (schema_name 'cassandra_fdw', table_name 'fdw_test_varchar');" 77 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "CREATE FOREIGN TABLE cassandra.fdw_test_timestamp(id uuid, some_timestamp bigint) server cass_serv options (schema_name 'cassandra_fdw', table_name 'fdw_test_timestamp');" 78 | 79 | # Test by running a queries 80 | echo "Testing bigint" 81 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "INSERT INTO $POSTGRESQL_RESULT_SCHEMA.fdw_test_results(created, postgresql_version, cassandra_version, test_description, test_result) SELECT current_timestamp, '$POSTGRESQL_CONTAINER', '$CASSANDRA_CONTAINER', 'Testing bigint', id::text || '|' || some_text from cassandra.fdw_test_bigint;" 82 | 83 | echo "Testing integer" 84 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "INSERT INTO $POSTGRESQL_RESULT_SCHEMA.fdw_test_results(created, postgresql_version, cassandra_version, test_description, test_result) SELECT current_timestamp, '$POSTGRESQL_CONTAINER', '$CASSANDRA_CONTAINER', 'Testing integer', id::text || '|' || some_text from cassandra.fdw_test_int;" 85 | 86 | echo "Testing smallint" 87 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "INSERT INTO $POSTGRESQL_RESULT_SCHEMA.fdw_test_results(created, postgresql_version, cassandra_version, test_description, test_result) SELECT current_timestamp, '$POSTGRESQL_CONTAINER', '$CASSANDRA_CONTAINER', 'Testing smallint', id::text || '|' || some_text from cassandra.fdw_test_smallint;" 88 | 89 | echo "Testing tinyint" 90 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "INSERT INTO $POSTGRESQL_RESULT_SCHEMA.fdw_test_results(created, postgresql_version, cassandra_version, test_description, test_result) SELECT current_timestamp, '$POSTGRESQL_CONTAINER', '$CASSANDRA_CONTAINER', 'Testing tinyint', id::text || '|' || some_text from cassandra.fdw_test_tinyint;" 91 | 92 | echo "Testing boolean" 93 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "INSERT INTO $POSTGRESQL_RESULT_SCHEMA.fdw_test_results(created, postgresql_version, cassandra_version, test_description, test_result) SELECT current_timestamp, '$POSTGRESQL_CONTAINER', '$CASSANDRA_CONTAINER', 'Testing boolean', id::text || '|' || some_text from cassandra.fdw_test_boolean;" 94 | 95 | echo "Testing float" 96 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "INSERT INTO $POSTGRESQL_RESULT_SCHEMA.fdw_test_results(created, postgresql_version, cassandra_version, test_description, test_result) SELECT current_timestamp, '$POSTGRESQL_CONTAINER', '$CASSANDRA_CONTAINER', 'Testing float', id::text || '|' || some_text from cassandra.fdw_test_float;" 97 | 98 | echo "Testing double precision" 99 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "INSERT INTO $POSTGRESQL_RESULT_SCHEMA.fdw_test_results(created, postgresql_version, cassandra_version, test_description, test_result) SELECT current_timestamp, '$POSTGRESQL_CONTAINER', '$CASSANDRA_CONTAINER', 'Testing double precision', id::text || '|' || some_text from cassandra.fdw_test_double;" 100 | 101 | echo "Testing uuid" 102 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "INSERT INTO $POSTGRESQL_RESULT_SCHEMA.fdw_test_results(created, postgresql_version, cassandra_version, test_description, test_result) SELECT current_timestamp, '$POSTGRESQL_CONTAINER', '$CASSANDRA_CONTAINER', 'Testing uuid', id::text || '|' || some_text from cassandra.fdw_test_uuid;" 103 | 104 | echo "Testing ascii" 105 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "INSERT INTO $POSTGRESQL_RESULT_SCHEMA.fdw_test_results(created, postgresql_version, cassandra_version, test_description, test_result) SELECT current_timestamp, '$POSTGRESQL_CONTAINER', '$CASSANDRA_CONTAINER', 'Testing ascii', id::text || '|' || some_text from cassandra.fdw_test_ascii;" 106 | 107 | echo "Testing varchar" 108 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "INSERT INTO $POSTGRESQL_RESULT_SCHEMA.fdw_test_results(created, postgresql_version, cassandra_version, test_description, test_result) SELECT current_timestamp, '$POSTGRESQL_CONTAINER', '$CASSANDRA_CONTAINER', 'Testing varchar', id::text || '|' || some_text from cassandra.fdw_test_varchar;" 109 | 110 | echo "Testing timestamp" 111 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB2 --command "INSERT INTO $POSTGRESQL_RESULT_SCHEMA.fdw_test_results(created, postgresql_version, cassandra_version, test_description, test_result) SELECT current_timestamp, '$POSTGRESQL_CONTAINER', '$CASSANDRA_CONTAINER', 'Testing timestamp', id::text || '|' || some_timestamp from cassandra.fdw_test_timestamp;" 112 | 113 | # Clear data and objects from testing 114 | sudo docker exec -it $POSTGRESQL_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB1 --command "DROP DATABASE IF EXISTS $POSTGRESQL_DB2;" 115 | 116 | done 117 | -------------------------------------------------------------------------------- /test/setup_img.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Set variables 4 | source ./build.cfg 5 | 6 | # Create the network 7 | sudo docker network create $DOCKER_NETWORK 8 | 9 | # PostgreSQL 10 | ## Create the server, where the results are stored 11 | POSTGRESQL_RESULT_CONTAINER="PostgreSQL_RESULT_${PostgreSQL_RESULT_VERSION}_N" 12 | POSTGRESQL_RESULT_IMAGE="postgres:${PostgreSQL_RESULT_VERSION}" 13 | 14 | ## Create the result container 15 | echo "Creating container $POSTGRESQL_RESULT_CONTAINER ..." 16 | sudo docker run -d --name $POSTGRESQL_RESULT_CONTAINER --network $DOCKER_NETWORK -e POSTGRES_USER=$POSTGRESQL_USER -e POSTGRES_PASSWORD=$POSTGRESQL_PWD -e POSTGRES_DB=$POSTGRESQL_USER -d ${POSTGRESQL_RESULT_IMAGE} 17 | sudo docker start $POSTGRESQL_RESULT_CONTAINER 18 | 19 | ## Wait for five seconds to be sure, that PostgreSQL is up and running 20 | echo "Waiting for five seconds until PostgreSQL is up in $POSTGRESQL_RESULT_CONTAINER ..." 21 | sleep 5 22 | 23 | sudo docker exec -it $POSTGRESQL_RESULT_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB1 --command "DROP DATABASE IF EXISTS $POSTGRESQL_RESULT_DB;" 24 | sudo docker exec -it $POSTGRESQL_RESULT_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_DB1 --command "CREATE DATABASE $POSTGRESQL_RESULT_DB;" 25 | sudo docker exec -it $POSTGRESQL_RESULT_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_RESULT_DB --command "CREATE SCHEMA $POSTGRESQL_RESULT_SCHEMA;" 26 | 27 | sudo docker exec -it $POSTGRESQL_RESULT_CONTAINER psql --host=$POSTGRESQL_HOST --port=$POSTGRESQL_PORT --username=$POSTGRESQL_USER $POSTGRESQL_RESULT_DB --command "CREATE TABLE $POSTGRESQL_RESULT_SCHEMA.results (created TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT current_timestamp, postgresql_version TEXT NOT NULL, cassandra_version TEXT NOT NULL, test_description TEXT NOT NULL, test_result TEXT NOT NULL);" 28 | 29 | 30 | for PG_VERSION in "${POSTGRESQL_VERSIONS[@]}" 31 | do 32 | # Set container names and PostgreSQL images 33 | POSTGRESQL_CONTAINER="PostgreSQL_${PG_VERSION}_N" 34 | POSTGRESQL_IMAGE="postgres:${PG_VERSION}" 35 | 36 | ## Create the container 37 | echo "Creating container $POSTGRESQL_CONTAINER ..." 38 | sudo docker run -d --name $POSTGRESQL_CONTAINER --network $DOCKER_NETWORK -e POSTGRES_USER=$POSTGRESQL_USER -e POSTGRES_PASSWORD=$POSTGRESQL_PWD -e POSTGRES_DB=$POSTGRESQL_USER -d ${POSTGRESQL_IMAGE} 39 | 40 | sudo docker start $POSTGRESQL_CONTAINER 41 | 42 | ## Update packages 43 | sudo docker exec -it $POSTGRESQL_CONTAINER bash -c "apt-get update > /dev/null; apt-get upgrade -y > /dev/null" 44 | 45 | ## Install additional packages for the cpp-driver and PostgreSQL 46 | sudo docker exec -it $POSTGRESQL_CONTAINER bash -c "apt-get install locales-all locales -y > /dev/null" 47 | sudo docker exec -it $POSTGRESQL_CONTAINER bash -c "apt-get install git make cmake build-essential lsb-release wget libuv1 libuv1-dev openssl libssl-dev zlib1g-dev zlib1g libkrb5-dev wget pgxnclient postgresql-contrib postgresql-server-dev-${PG_VERSION} -y > /dev/null" 48 | 49 | ## Get the cpp-driver sources 50 | sudo docker exec -it $POSTGRESQL_CONTAINER bash -c "mkdir /tmp ; cd /tmp ; git clone https://github.com/datastax/cpp-driver.git" 51 | 52 | ## Compile and install the ccp-driver 53 | sudo docker exec -it $POSTGRESQL_CONTAINER bash -c "cd /tmp/cpp-driver ; mkdir build ; pushd build ; cmake .. ; make ; make install ; popd" 54 | 55 | ## Get the cassandra2_fdw sources 56 | sudo docker exec -it $POSTGRESQL_CONTAINER bash -c "cd /tmp ; git clone https://github.com/sjstoelting/cassandra2_fdw.git" 57 | 58 | ## Change pg_config to use the right version in the Makefile 59 | CONFIGFILE="\"\/usr\/lib\/postgresql\/${PG_VERSION}\/bin\/pg_config\"" 60 | 61 | sudo docker exec -it $POSTGRESQL_CONTAINER bash -c "cd /tmp/cassandra2_fdw ; sed -i 's/pg_config/${CONFIGFILE}/g' Makefile" 62 | 63 | ## Compile the cassandra2_fdw 64 | sudo docker exec -it $POSTGRESQL_CONTAINER bash -c "cd /tmp/cassandra2_fdw ; USE_PGXS=1 make ; USE_PGXS=1 make install" 65 | 66 | ## Update libraries 67 | sudo docker exec -it $POSTGRESQL_CONTAINER bash -c "ldconfig" 68 | 69 | done 70 | 71 | 72 | # Cassandra 73 | for CASSANDRA_VERSION in "${CASSANDRA_VERSIONS[@]}" 74 | do 75 | # Set container name and image 76 | CASSANDRA_CONTAINER="Cassandra_${CASSANDRA_VERSION}_N" 77 | CASSANDRA_CONTAINER=${CASSANDRA_CONTAINER/./_} 78 | 79 | CASSNADRA_IMAGE="cassandra:$CASSANDRA_VERSION" 80 | 81 | # Creating and starting the cassandra container 82 | echo "Creating container $CASSANDRA_CONTAINER from $CASSNADRA_IMAGE ..." 83 | sudo docker run --name $CASSANDRA_CONTAINER --network $DOCKER_NETWORK -d "cassandra:$CASSANDRA_VERSION" 84 | 85 | done 86 | 87 | # Cassandra needs some time to be started, therefore the script waits for 30 seconds 88 | echo "Waiting for 60 seconds until Cassandra servers have started and are up..." 89 | sleep 60 90 | -------------------------------------------------------------------------------- /test/test.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Initialize all Docker images 4 | ./setup_img.sh 5 | 6 | # Run the Cassandra part, will also run the PostgreSQL parts for each Cassandra container 7 | ./cassandra.sh 8 | 9 | # Clean up everything 10 | ./cleanup.sh 11 | --------------------------------------------------------------------------------