├── LICENSE ├── README.md ├── RegistExt.cpp ├── RegistExt.h ├── SQLiteExt.cpp ├── helpers.cpp ├── helpers.h ├── histo.jpg ├── histogram.cpp ├── mean.jpg ├── meanhistogram.cpp ├── ratio1.jpg ├── ratio2.jpg ├── ratiohistogram.cpp ├── scatter.jpg ├── sqlite3.h └── sqlite3ext.h /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Robert Oeffner 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | # SQLiteHistograms 3 | 4 | This SQLite extension library is based on the 'ext/misc/series.c example from the SQLite source files. 5 | It uses the virtual table feature of SQLite to generate tables on the fly. 6 | 7 | The library contains more SQLite extensions: HISTO for calculating histograms of data, RATIOHISTO 8 | for calculating 9 | ratios of two histograms and MEANHISTO for calculating interpolated values of 2D scatter data. 10 | SQRT, LOG, EXP and POW are provided for calculating squareroot, logarithm, exponential and raising 11 | column values to a power, respectively. The Pearson correlation value and the Spearman rank 12 | correlation can be calculated between two columns with the CORREL and the SPEARMANCORREL functions. 13 | 14 | ## HISTO function: 15 | 16 | The signature of the HISTO function is as follows: 17 | `HISTO("tablename", "columnname", nbins, minbin, maxbin)` 18 | The tablename and the columnname must be entered with quotes or double quotes. The arguments, nbins, minbin and maxbin are the 19 | number of histogram bins, the minimum bin value and the maximum bin value respectively. 20 | An example is an SQLite table, "AllProteins", that contains a column of integers labelled "NumberofResiduesInModel". Rows in the table are labelled with this 21 | number which goes up to about 1500. A histogram can be computed using the HISTO function like this: 22 | `SELECT * FROM HISTO("AllProteins", "NumberofResiduesInModel", 15, 0, 1500);` 23 | This produces a histogram with 15 bins between 0 and 1500. In the SQLitebrowser the result can be visualised graphically as below: 24 | ![alt text](histo.jpg) 25 | 26 | ## RATIOHISTO function: 27 | 28 | The signature of the RATIOHISTO function is as follows: 29 | `RATIOHISTO("tablename", "columnname", nbins, minbin, maxbin, "discrcolid", discrval);` 30 | The tablename and the columnname must be entered with quotes or double quotes. The arguments, nbins, minbin and maxbin are the 31 | number of histogram bins, the minimum bin value and the maximum bin value respectively. The "discrcolid" is the name of a 32 | column with values that may be a function of the columnname values. 33 | Given two columns where the values in one column is a function of the values in the other RATIOHISTO creates two 34 | histograms, count1 and count2. The bin count in count1 are those "columnname" values where the corresponding "discrcolid" 35 | is smaller than discrval. The bin count in count2 are those "columnname" values where the corresponding "discrcolid" is 36 | greater than discrval. RATIOHISTO then computes a column named ratio given as count2/(count1+count2) for each bin. 37 | 38 | An example is an SQLite table, "AllProteins", containing a column labelled "LLGvrms". It also has a column, "CCglobal", 39 | which is correlated to the values in "LLGvrms" in the sense that whenever values of CCglobal are below 0.2 the 40 | corresponding LLGvrms value is 41 | below 50. Two different histograms, "count1" and "count2", can be produced when issuing the statement: 42 | `SELECT * FROM RATIOHISTO("AllProteins", "LLGvrms", 20, 0, 90, "CCglobal", 0.2);` 43 | as illustrated below: 44 | ![alt text](ratio1.jpg) 45 | 46 | Although the two histograms peak at different bin values this can be better illustrated by computing the ratio of the "count2" 47 | histogram divided by the total count. These values are stored in the "ratio" column which for this particular table 48 | happens to be a sigmoidal curve as illustrated below: 49 | ![alt text](ratio2.jpg) 50 | 51 | ## MEANHISTO function 52 | 53 | The signature for the MEANHISTO function is as follows: 54 | `MEANHISTO('tablename', 'xcolumnname', 'ycolumnname', nbins, minbin, maxbin);` 55 | The "tablename", the "xcolumnname" and the "ycolumnname" must be entered in quotes. The arguments, nbins, minbin and maxbin are the 56 | number of histogram bins, the minimum bin value and the maximum bin value respectively. This function takes a 2 dimensional table 57 | and sorts the data into nbins number of bins with respect to the xcolumnname values. For each bin it then computes the average of 58 | the corresponding ycolumnname values. Thus creating an interpolated curve through the 2 dimensional scatter data. 59 | In addition it also computes the standard deviation, sigma, and the standard error margin, sem, and the total count of each bin. 60 | 61 | To demonstrate the use of MEANHISTO consider 62 | an SQLite table, "AllProteins", containing two columns labelled "FracvarVRMS1" and "LLGrefl_vrms" 63 | respectively. The LLGrefl_vrms values are a function of the FracvarVRMS1 values albeit not an exact function. 64 | So the following plain SQLite statement: 65 | `SELECT FracvarVRMS1, LLGrefl_vrms FROM AllProteins WHERE FracvarVRMS1 <= 0.6;` 66 | produces the table and the scatter plot below 67 | ![alt text](scatter.jpg) 68 | 69 | An interpolated curve through this scatter plot can be created with the following statement: 70 | `SELECT * FROM MEANHISTO("AllProteins", "FracvarVRMS1", "LLGrefl_vrms", 30, 0, 0.6);` 71 | which produces the table of average bin values below: 72 | ![alt text](mean.jpg) 73 | 74 | 75 | ## SQRT, LOG, EXP and POW functions 76 | 77 | The squareroot, logarithm, exponential and the power function act on column values and are 78 | invoked as in 79 | `SELECT SQRT(LLGvrms), LOG(LLGvrms), EXP(CCglobal), POW(TFZequiv, 2.0) FROM AllProteins;` 80 | 81 | ## Correlation values 82 | 83 | The Pearson correlation value and the Spearman rank correlation value can be calculated as in: 84 | `SELECT CORREL(LLGvrms, CCglobal) FROM AllProteins;` 85 | or 86 | `SELECT SPEARMANCORREL(LLGvrms, CCglobal) FROM AllProteins;` 87 | 88 | ## Compile on Windows with Visual Studio 2015 89 | 90 | cl /Ox /EHsc /GL /Fohelpers.obj /c helpers.cpp ^ 91 | && cl /Ox /EHsc /GL /FoSQLiteExt.obj /c SQLiteExt.cpp ^ 92 | && cl /Ox /EHsc /GL /Foratiohistogram.obj /c ratiohistogram.cpp ^ 93 | && cl /Ox /EHsc /GL /Fohistogram.obj /c histogram.cpp ^ 94 | && cl /Ox /EHsc /GL /Fomeanhistogram.obj /c meanhistogram.cpp ^ 95 | && cl /Ox /EHsc /GL /FoRegistExt.obj /c RegistExt.cpp ^ 96 | && link /DLL /LTCG /OUT:histograms.dll helpers.obj SQLiteExt.obj RegistExt.obj meanhistogram.obj histogram.obj ratiohistogram.obj 97 | 98 | 99 | ## Compile on Linux with g++ 100 | 101 | g++ -O3 -fPIC -lm -shared histogram.cpp helpers.cpp meanhistogram.cpp ratiohistogram.cpp SQLiteExt.cpp RegistExt.cpp -o histograms.so 102 | 103 | ## Compile on macOS with clang++ 104 | 105 | clang++ -O3 -fPIC -lm -shared histogram.cpp helpers.cpp meanhistogram.cpp ratiohistogram.cpp SQLiteExt.cpp RegistExt.cpp -o histograms.dylib 106 | 107 | ## Loading the extension from the sqlite3 commandline 108 | 109 | ### on Windows: 110 | 111 | sqlite> .load histograms.dll 112 | 113 | ### on Linux: 114 | 115 | sqlite> .load ./histograms.so 116 | 117 | ### on macOS 118 | 119 | sqlite> .load ./histograms.dylib 120 | 121 | ## Bugs and feedback 122 | Feel free to get in touch with me. 123 | 124 | ## Acknowledgements 125 | This work would not have been possible without funding by the US National Institutes of Health (NIH). 126 | Thanks go to all contributors and users reporting bugs or fixes. 127 | 128 | 129 | -------------------------------------------------------------------------------- /RegistExt.cpp: -------------------------------------------------------------------------------- 1 | /* 2 | RegistExt.cpp, Robert Oeffner 2018 3 | 4 | The MIT License (MIT) 5 | 6 | Copyright (c) 2017 Robert Oeffner 7 | 8 | Permission is hereby granted, free of charge, to any person obtaining a copy 9 | of this software and associated documentation files (the "Software"), to deal 10 | in the Software without restriction, including without limitation the rights 11 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 12 | copies of the Software, and to permit persons to whom the Software is 13 | furnished to do so, subject to the following conditions: 14 | 15 | The above copyright notice and this permission notice shall be included in all 16 | copies or substantial portions of the Software. 17 | 18 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 19 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 20 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 21 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 22 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 23 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 24 | SOFTWARE. 25 | 26 | 27 | 28 | 29 | Compile on Windows linking with whole program optimisation: 30 | 31 | cl /Ox /EHsc /GL /Fohelpers.obj /c helpers.cpp ^ 32 | && cl /Ox /EHsc /GL /FoSQLiteExt.obj /c SQLiteExt.cpp ^ 33 | && cl /Ox /EHsc /GL /Foratiohistogram.obj /c ratiohistogram.cpp ^ 34 | && cl /Ox /EHsc /GL /Fohistogram.obj /c histogram.cpp ^ 35 | && cl /Ox /EHsc /GL /Fomeanhistogram.obj /c meanhistogram.cpp ^ 36 | && cl /Ox /EHsc /GL /FoRegistExt.obj /c RegistExt.cpp ^ 37 | && link /DLL /LTCG /OUT:histograms.dll helpers.obj SQLiteExt.obj RegistExt.obj meanhistogram.obj histogram.obj ratiohistogram.obj 38 | 39 | With debug info: 40 | 41 | cl /Fohelpers.obj /c helpers.cpp /DDEBUG /ZI /EHsc ^ 42 | && cl /FoSQLiteExt.obj /c SQLiteExt.cpp /DDEBUG /ZI /EHsc ^ 43 | && cl /Foratiohistogram.obj /c ratiohistogram.cpp /DDEBUG /ZI /EHsc ^ 44 | && cl /Fomeanhistogram.obj /c meanhistogram.cpp /DDEBUG /ZI /EHsc ^ 45 | && cl /Fohistogram.obj /c histogram.cpp /DDEBUG /ZI /EHsc ^ 46 | && cl /FoRegistExt.obj /c RegistExt.cpp /DDEBUG /ZI /EHsc ^ 47 | && link /DLL /DEBUG /debugtype:cv /OUT:histograms.dll helpers.obj SQLiteExt.obj meanhistogram.obj RegistExt.obj histogram.obj ratiohistogram.obj 48 | 49 | 50 | Compile on Linux: 51 | 52 | g++ -fPIC -lm -shared histogram.cpp helpers.cpp meanhistogram.cpp ratiohistogram.cpp RegistExt.cpp -o libhistograms.so 53 | 54 | From the sqlite commandline load the extension 55 | 56 | on Windows 57 | sqlite> .load histograms.dll 58 | sqlite> 59 | 60 | on Linux 61 | sqlite> .load ./histograms.so 62 | sqlite> 63 | 64 | */ 65 | 66 | 67 | #include "RegistExt.h" 68 | 69 | 70 | #ifdef __cplusplus 71 | extern "C" { 72 | #endif 73 | 74 | 75 | SQLITE_EXTENSION_INIT1 76 | 77 | 78 | 79 | sqlite3 *thisdb = NULL; 80 | 81 | #ifdef _WIN32 82 | __declspec(dllexport) 83 | #endif 84 | /* The built library file name excluding its file extension must be part of the 85 | function name below as documented on http://www.sqlite.org/loadext.html 86 | */ 87 | int sqlite3_histograms_init( // always use lower case 88 | sqlite3 *db, 89 | char **pzErrMsg, 90 | const sqlite3_api_routines *pApi 91 | ) 92 | { 93 | int rc = SQLITE_OK; 94 | SQLITE_EXTENSION_INIT2(pApi); 95 | #ifndef SQLITE_OMIT_VIRTUALTABLE 96 | if (sqlite3_libversion_number()<3008012) 97 | { 98 | *pzErrMsg = sqlite3_mprintf("Histogram extension requires SQLite 3.8.12 or later"); 99 | return SQLITE_ERROR; 100 | } 101 | rc = sqlite3_create_module(db, "HISTO", &histoModule, 0); 102 | rc = sqlite3_create_module(db, "RATIOHISTO", &ratiohistoModule, 0); 103 | rc = sqlite3_create_module(db, "MEANHISTO", &meanhistoModule, 0); 104 | 105 | // 3. parameter is the number of arguments the functions take 106 | sqlite3_create_function(db, "SQRT", 1, SQLITE_ANY, 0, sqrtFunc, 0, 0); 107 | sqlite3_create_function(db, "LOG", 1, SQLITE_ANY, 0, logFunc, 0, 0); 108 | sqlite3_create_function(db, "EXP", 1, SQLITE_ANY, 0, expFunc, 0, 0); 109 | sqlite3_create_function(db, "POW", 2, SQLITE_ANY, 0, powFunc, 0, 0); 110 | 111 | sqlite3_create_function(db, "CORREL", 2, SQLITE_ANY, db, NULL, CorrelStep, CorrelFinal); 112 | sqlite3_create_function(db, "SPEARMANCORREL", 2, SQLITE_ANY, db, NULL, SpCorrelStep, SpCorrelFinal); 113 | 114 | #endif 115 | return rc; 116 | } 117 | 118 | 119 | 120 | #ifdef __cplusplus 121 | } 122 | #endif 123 | -------------------------------------------------------------------------------- /RegistExt.h: -------------------------------------------------------------------------------- 1 | /* 2 | RegistExt.h, Robert Oeffner 2018 3 | 4 | The MIT License (MIT) 5 | 6 | Copyright (c) 2017 Robert Oeffner 7 | 8 | Permission is hereby granted, free of charge, to any person obtaining a copy 9 | of this software and associated documentation files (the "Software"), to deal 10 | in the Software without restriction, including without limitation the rights 11 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 12 | copies of the Software, and to permit persons to whom the Software is 13 | furnished to do so, subject to the following conditions: 14 | 15 | The above copyright notice and this permission notice shall be included in all 16 | copies or substantial portions of the Software. 17 | 18 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 19 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 20 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 21 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 22 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 23 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 24 | SOFTWARE. 25 | 26 | */ 27 | 28 | #include "sqlite3ext.h" 29 | #include 30 | 31 | 32 | #ifdef __cplusplus 33 | extern "C" { 34 | #endif 35 | 36 | 37 | // scalar functions 38 | 39 | void sqrtFunc(sqlite3_context *context, int argc, sqlite3_value **argv); 40 | void logFunc(sqlite3_context *context, int argc, sqlite3_value **argv); 41 | void expFunc(sqlite3_context *context, int argc, sqlite3_value **argv); 42 | void powFunc(sqlite3_context *context, int argc, sqlite3_value **argv); 43 | 44 | // aggregate functions 45 | 46 | void CorrelStep(sqlite3_context *context, int argc, sqlite3_value **argv); 47 | void CorrelFinal(sqlite3_context *context); 48 | void SpCorrelStep(sqlite3_context *context, int argc, sqlite3_value **argv); 49 | void SpCorrelFinal(sqlite3_context *context); 50 | 51 | 52 | 53 | extern const sqlite3_api_routines *sqlite3_api; 54 | extern sqlite3 *thisdb; 55 | 56 | 57 | // module functions for some virtual tables 58 | 59 | extern sqlite3_module histoModule; 60 | 61 | int histoConnect( 62 | sqlite3 *db, 63 | void *pAux, 64 | int argc, const char *const*argv, 65 | sqlite3_vtab **ppVtab, 66 | char **pzErr 67 | ); 68 | int histoDisconnect(sqlite3_vtab *pVtab); 69 | int histoOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor); 70 | int histoClose(sqlite3_vtab_cursor *cur); 71 | int histoNext(sqlite3_vtab_cursor *cur); 72 | int histoColumn(sqlite3_vtab_cursor *cur, sqlite3_context *ctx, int i); 73 | int histoRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid); 74 | int histoEof(sqlite3_vtab_cursor *cur); 75 | int histoFilter( 76 | sqlite3_vtab_cursor *pVtabCursor, 77 | int idxNum, const char *idxStr, 78 | int argc, sqlite3_value **argv 79 | ); 80 | int histoBestIndex(sqlite3_vtab *tab, sqlite3_index_info *pIdxInfo); 81 | 82 | 83 | 84 | 85 | extern sqlite3_module ratiohistoModule; 86 | 87 | int ratiohistoConnect( 88 | sqlite3 *db, 89 | void *pAux, 90 | int argc, const char *const*argv, 91 | sqlite3_vtab **ppVtab, 92 | char **pzErr 93 | ); 94 | int ratiohistoDisconnect(sqlite3_vtab *pVtab); 95 | int ratiohistoOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor); 96 | int ratiohistoClose(sqlite3_vtab_cursor *cur); 97 | int ratiohistoNext(sqlite3_vtab_cursor *cur); 98 | int ratiohistoColumn(sqlite3_vtab_cursor *cur, sqlite3_context *ctx, int i); 99 | int ratiohistoRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid); 100 | int ratiohistoEof(sqlite3_vtab_cursor *cur); 101 | int ratiohistoFilter( 102 | sqlite3_vtab_cursor *pVtabCursor, 103 | int idxNum, const char *idxStr, 104 | int argc, sqlite3_value **argv 105 | ); 106 | int ratiohistoBestIndex(sqlite3_vtab *tab, sqlite3_index_info *pIdxInfo); 107 | 108 | 109 | 110 | extern sqlite3_module meanhistoModule; 111 | 112 | int meanhistoConnect( 113 | sqlite3 *db, 114 | void *pAux, 115 | int argc, const char *const*argv, 116 | sqlite3_vtab **ppVtab, 117 | char **pzErr 118 | ); 119 | int meanhistoDisconnect(sqlite3_vtab *pVtab); 120 | int meanhistoOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor); 121 | int meanhistoClose(sqlite3_vtab_cursor *cur); 122 | int meanhistoNext(sqlite3_vtab_cursor *cur); 123 | int meanhistoColumn(sqlite3_vtab_cursor *cur, sqlite3_context *ctx, int i); 124 | int meanhistoRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid); 125 | int meanhistoEof(sqlite3_vtab_cursor *cur); 126 | int meanhistoFilter( 127 | sqlite3_vtab_cursor *pVtabCursor, 128 | int idxNum, const char *idxStr, 129 | int argc, sqlite3_value **argv 130 | ); 131 | int meanhistoBestIndex(sqlite3_vtab *tab, sqlite3_index_info *pIdxInfo); 132 | 133 | 134 | 135 | 136 | #ifdef __cplusplus 137 | } 138 | #endif 139 | 140 | 141 | #pragma once 142 | 143 | -------------------------------------------------------------------------------- /SQLiteExt.cpp: -------------------------------------------------------------------------------- 1 | /* 2 | SQLiteExt.cpp, Robert Oeffner 2018 3 | 4 | SQLite extension for providing useful math functions not present in vanilla SQLite. 5 | 6 | The MIT License (MIT) 7 | 8 | Copyright (c) 2017 Robert Oeffner 9 | 10 | Permission is hereby granted, free of charge, to any person obtaining a copy 11 | of this software and associated documentation files (the "Software"), to deal 12 | in the Software without restriction, including without limitation the rights 13 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 14 | copies of the Software, and to permit persons to whom the Software is 15 | furnished to do so, subject to the following conditions: 16 | 17 | The above copyright notice and this permission notice shall be included in all 18 | copies or substantial portions of the Software. 19 | 20 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 21 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 22 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 23 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 24 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 25 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 26 | SOFTWARE. 27 | 28 | */ 29 | 30 | 31 | #include "RegistExt.h" 32 | #include "sqlite3ext.h" 33 | #include 34 | #include 35 | 36 | 37 | 38 | void sqrtFunc(sqlite3_context *context, int argc, sqlite3_value **argv) 39 | { 40 | sqlite3_result_double(context, sqrt(sqlite3_value_double(argv[0]))); 41 | } 42 | 43 | void logFunc( sqlite3_context *context, int argc, sqlite3_value **argv) 44 | { 45 | sqlite3_result_double(context, log(sqlite3_value_double(argv[0]))); 46 | } 47 | 48 | void expFunc(sqlite3_context *context, int argc, sqlite3_value **argv) 49 | { 50 | sqlite3_result_double(context, exp(sqlite3_value_double(argv[0]))); 51 | } 52 | 53 | void powFunc(sqlite3_context *context, int argc, sqlite3_value **argv) 54 | { 55 | sqlite3_result_double(context, pow(sqlite3_value_double(argv[0]), sqlite3_value_double(argv[1]))); 56 | } 57 | 58 | 59 | struct StdevCtx 60 | { 61 | std::vector X; 62 | std::vector Y; 63 | }; 64 | 65 | 66 | void CorrelStep(sqlite3_context *context, int argc, sqlite3_value **argv) 67 | { 68 | StdevCtx *p = (StdevCtx*)sqlite3_aggregate_context(context, sizeof(*p)); 69 | // only consider non-null values 70 | if (SQLITE_NULL != sqlite3_value_numeric_type(argv[0]) 71 | && SQLITE_NULL != sqlite3_value_numeric_type(argv[1])) 72 | { 73 | p->X.push_back(sqlite3_value_double(argv[0])); 74 | p->Y.push_back(sqlite3_value_double(argv[1])); 75 | } 76 | } 77 | 78 | 79 | void CorrelFinal(sqlite3_context *context) 80 | { 81 | StdevCtx *p = (StdevCtx*)sqlite3_aggregate_context(context, 0); 82 | if (p && p->X.size() > 0) 83 | { 84 | double Xsum = 0.0, Ysum = 0.0; 85 | for (unsigned j = 0; j < p->X.size(); j++) 86 | { 87 | Xsum += p->X[j]; 88 | Ysum += p->Y[j]; 89 | } 90 | double Xavg = Xsum / p->X.size(); 91 | double Yavg = Ysum / p->Y.size(); 92 | 93 | double covar = 0.0, Xvar = 0.0, Yvar = 0.0; 94 | for (unsigned j = 0; j < p->X.size(); j++) 95 | { 96 | double xd = (p->X[j] - Xavg); 97 | double yd = (p->Y[j] - Yavg); 98 | Xvar += xd * xd; 99 | Yvar += yd * yd; 100 | covar += xd * yd; 101 | } 102 | covar /= p->X.size(); 103 | Xvar /= p->X.size(); 104 | Yvar /= p->X.size(); 105 | double Xsig = sqrt(Xvar); 106 | double Ysig = sqrt(Yvar); 107 | double correlation = covar / (Xsig*Ysig); 108 | sqlite3_result_double(context, correlation); 109 | } 110 | else 111 | { 112 | sqlite3_result_double(context, 0.0); 113 | } 114 | p->X.clear(); 115 | p->Y.clear(); 116 | } 117 | 118 | 119 | 120 | struct spcorval 121 | { 122 | double val, rank; 123 | }; 124 | 125 | 126 | struct v2Ctx 127 | { 128 | std::vector X; 129 | std::vector Y; 130 | }; 131 | 132 | 133 | 134 | // Function to find ranks in array of spcorval elements 135 | void Rankify(std::vector &A) 136 | { 137 | // Sweep through all elements in A for each 138 | // element count the number of less than and 139 | // equal elements separately in r and s. 140 | for (int i = 0; i < A.size(); i++) 141 | { 142 | int r = 1, s = 1; 143 | for (int j = 0; j < A.size(); j++) 144 | { 145 | if (j != i && A[j].val < A[i].val) 146 | r += 1; 147 | if (j != i && A[j].val == A[i].val) 148 | s += 1; 149 | } 150 | // Use formula to obtain rank 151 | A[i].rank = r + (float)(s - 1)/2.0; 152 | } 153 | } 154 | 155 | 156 | void SpCorrelStep(sqlite3_context *context, int argc, sqlite3_value **argv) 157 | { 158 | v2Ctx *p = (v2Ctx*)sqlite3_aggregate_context(context, sizeof(*p)); 159 | // only consider non-null values 160 | if (SQLITE_NULL != sqlite3_value_numeric_type(argv[0]) 161 | && SQLITE_NULL != sqlite3_value_numeric_type(argv[1])) 162 | { 163 | spcorval x, y; 164 | x.val = sqlite3_value_double(argv[0]); 165 | y.val = sqlite3_value_double(argv[1]); 166 | p->X.push_back(x); 167 | p->Y.push_back(y); 168 | } 169 | } 170 | 171 | 172 | void SpCorrelFinal(sqlite3_context *context) { 173 | v2Ctx *p = (v2Ctx*)sqlite3_aggregate_context(context, 0); 174 | if (p && p->X.size() > 0) 175 | { 176 | // now rank the arrays first 177 | Rankify(p->X); 178 | Rankify(p->Y); 179 | 180 | double Xsum = 0.0, Ysum = 0.0; 181 | for (unsigned j = 0; j < p->X.size(); j++) 182 | { 183 | Xsum += p->X[j].rank; 184 | Ysum += p->Y[j].rank; 185 | } 186 | double Xavg = Xsum / p->X.size(); 187 | double Yavg = Ysum / p->Y.size(); 188 | double covar = 0.0, Xvar = 0.0, Yvar = 0.0; 189 | for (unsigned j = 0; j < p->X.size(); j++) 190 | { 191 | double xd = (p->X[j].rank - Xavg); 192 | double yd = (p->Y[j].rank - Yavg); 193 | Xvar += xd * xd; 194 | Yvar += yd * yd; 195 | covar += xd * yd; 196 | } 197 | covar /= p->X.size(); 198 | Xvar /= p->X.size(); 199 | Yvar /= p->X.size(); 200 | double Xsig = sqrt(Xvar); 201 | double Ysig = sqrt(Yvar); 202 | double correlation = covar/(Xsig*Ysig); 203 | sqlite3_result_double(context, correlation); 204 | } 205 | else 206 | { 207 | sqlite3_result_double(context, 0.0); 208 | } 209 | p->X.clear(); 210 | p->Y.clear(); 211 | } 212 | 213 | 214 | 215 | 216 | -------------------------------------------------------------------------------- /helpers.cpp: -------------------------------------------------------------------------------- 1 | /* 2 | helpers.cpp, Robert Oeffner 2018 3 | 4 | The MIT License (MIT) 5 | 6 | Copyright (c) 2017 Robert Oeffner 7 | 8 | Permission is hereby granted, free of charge, to any person obtaining a copy 9 | of this software and associated documentation files (the "Software"), to deal 10 | in the Software without restriction, including without limitation the rights 11 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 12 | copies of the Software, and to permit persons to whom the Software is 13 | furnished to do so, subject to the following conditions: 14 | 15 | The above copyright notice and this permission notice shall be included in all 16 | copies or substantial portions of the Software. 17 | 18 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 19 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 20 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 21 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 22 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 23 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 24 | SOFTWARE. 25 | 26 | */ 27 | 28 | 29 | #include "helpers.h" 30 | 31 | 32 | 33 | std::vector< std::vector > GetColumns(sqlite3* db, std::string sqlxprs, int *rc) 34 | { 35 | // get column from sql expression and return it 36 | char *zErrMsg; 37 | char **result; 38 | int nrow, ncol; 39 | int db_open; 40 | *rc = sqlite3_get_table( 41 | db, /* An open database */ 42 | sqlxprs.c_str(), /* SQL to be executed */ 43 | &result, /* Result written to a char *[] that this points to */ 44 | &nrow, /* Number of result rows written here */ 45 | &ncol, /* Number of result columns written here */ 46 | &zErrMsg /* Error msg written here */ 47 | ); 48 | /* If SQL table is structured as: 49 | Name | Age 50 | ----------------------- 51 | Alice | 43 52 | Bob | 28 53 | Cindy | 21 54 | 55 | then &result will look like: 56 | 57 | result[0] = "Name"; 58 | result[1] = "Age"; 59 | result[2] = "Alice"; 60 | result[3] = "43"; 61 | result[4] = "Bob"; 62 | result[5] = "28"; 63 | result[6] = "Cindy"; 64 | result[7] = "21"; 65 | */ 66 | std::vector< std::vector > columns; 67 | columns.clear(); 68 | columns.resize(ncol); 69 | 70 | for (int n = 0; n < ncol; n++) 71 | { 72 | columns[n].resize(nrow); 73 | } 74 | if (*rc == SQLITE_OK) { 75 | for (int i = 0; i < nrow*ncol; i++) 76 | { 77 | int col = i % ncol; 78 | int row = floor(i / ncol); 79 | if (result[ncol + i]) // skip column names 80 | { 81 | std::string val(result[ncol + i]); 82 | columns[col][row] = atof(val.c_str()); 83 | } 84 | } 85 | } 86 | sqlite3_free_table(result); 87 | 88 | return columns; 89 | } 90 | 91 | 92 | 93 | /* Caclulate a histogram from the col array with bins number of bins and values between 94 | minbin and maxbin 95 | */ 96 | std::vector CalcHistogram(std::vector< std::vector > Yvals, 97 | int bins, double minbin, double maxbin, int *rc) 98 | { 99 | std::vector histo; 100 | 101 | if (bins < 1 || minbin >= maxbin) 102 | { 103 | std::cerr << "Nonsensical value for either bins, minbin or maxbin" << std::endl; 104 | *rc = SQLITE_ERROR; 105 | return histo; 106 | } 107 | 108 | histo.resize(bins); 109 | double bindomain = maxbin - minbin; 110 | double binwidth = bindomain / bins; 111 | int accumcount = 0; 112 | for (unsigned i = 0; i < histo.size(); i++) 113 | { 114 | double upper = binwidth * (i + 1) + minbin; 115 | double middle = binwidth * (i + 0.5) + minbin; 116 | double lower = binwidth * i + minbin; 117 | histo[i].binval = middle; 118 | histo[i].count = 0; 119 | histo[i].accumcount = 0; 120 | histo[i].accumcount = accumcount; 121 | } 122 | 123 | if (Yvals.size() > 0) 124 | { 125 | for (unsigned j = 0; j < Yvals[0].size(); j++) 126 | { 127 | int ibin = (Yvals[0][j] - minbin) / binwidth; 128 | if (ibin < 0 || ibin >(bins - 1)) 129 | continue; 130 | histo[ibin].count++; 131 | accumcount++; 132 | histo[ibin].accumcount = accumcount; 133 | } 134 | } 135 | 136 | return histo; 137 | }; 138 | 139 | 140 | /* Calculate interpolation values for scatter data within bin values between 141 | minbin and maxbin 142 | */ 143 | std::vector CalcInterpolations(std::vector< std::vector > XYvals, 144 | int bins, double minbin, double maxbin, int *rc) 145 | { 146 | std::vector interpol; 147 | std::vector< std::vector > shiftval; 148 | 149 | if (bins < 1 || minbin >= maxbin) 150 | { 151 | std::cerr << "Nonsensical value for either bins, minbin or maxbin" << std::endl; 152 | *rc = SQLITE_ERROR; 153 | return interpol; 154 | } 155 | 156 | shiftval.resize(2); 157 | shiftval[0].resize(bins); 158 | shiftval[1].resize(bins); 159 | 160 | interpol.resize(bins); 161 | double bindomain = maxbin - minbin; 162 | double binwidth = bindomain / bins; 163 | if (XYvals.size() < 1) 164 | return interpol; 165 | 166 | for (unsigned i = 0; i < interpol.size(); i++) 167 | { 168 | double upper = binwidth * (i + 1) + minbin; 169 | double middle = binwidth * (i + 0.5) + minbin; 170 | double lower = binwidth * i + minbin; 171 | interpol[i].xval = middle; 172 | interpol[i].count = 0; 173 | } 174 | 175 | for (unsigned j = 0; j < XYvals[0].size(); j++) 176 | { 177 | int ibin = (XYvals[0][j] - minbin) / binwidth; 178 | if (ibin < 0 || ibin >(bins - 1)) 179 | continue; 180 | interpol[ibin].count++; 181 | shiftval[0][ibin] = shiftval[1][ibin] = 0.0; 182 | } 183 | 184 | for (unsigned j = 0; j < XYvals[0].size(); j++) 185 | { 186 | int ibin = (XYvals[0][j] - minbin) / binwidth; 187 | if (ibin < 0 || ibin >(bins - 1)) 188 | continue; 189 | // First calculate shiftconst used for avoiding numerical instability of sigma 190 | // by assigning it to the mean value of values in the bin. 191 | // See https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Algorithm_II 192 | double shiftconst = 0.0; 193 | if (interpol[ibin].count) 194 | shiftconst = XYvals[1][j]/interpol[ibin].count; 195 | 196 | double shifted = XYvals[1][j] - shiftconst; 197 | shiftval[0][ibin] += shifted; // avoid numerical instability close to zero 198 | shiftval[1][ibin] += shifted * shifted; 199 | interpol[ibin].yval += XYvals[1][j]; 200 | } 201 | 202 | for (unsigned i = 0; i < interpol.size(); i++) 203 | { 204 | interpol[i].yval /= interpol[i].count; 205 | interpol[i].sigma = sqrt((shiftval[1][i] 206 | - (shiftval[0][i] * shiftval[0][i]) / interpol[i].count) / interpol[i].count); 207 | /* 208 | Margin of Error (MOE) of a mean value is based on Z*sigma/sqrt(N) Z=1.96 corresponds to 95% confidence 209 | Z-Score Confidence Limit (%) 210 | 3 99.73 211 | 2.58 99 212 | 2.33 98 213 | 2.17 97 214 | 2.05 96 215 | 2.0 95.45 216 | 1.96 95 217 | 1.64 90 218 | */ 219 | interpol[i].sem = interpol[i].sigma / sqrt(interpol[i].count); 220 | } 221 | 222 | return interpol; 223 | } 224 | 225 | -------------------------------------------------------------------------------- /helpers.h: -------------------------------------------------------------------------------- 1 | /* 2 | helpers.h, Robert Oeffner 2018 3 | 4 | The MIT License (MIT) 5 | 6 | Copyright (c) 2017 Robert Oeffner 7 | 8 | Permission is hereby granted, free of charge, to any person obtaining a copy 9 | of this software and associated documentation files (the "Software"), to deal 10 | in the Software without restriction, including without limitation the rights 11 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 12 | copies of the Software, and to permit persons to whom the Software is 13 | furnished to do so, subject to the following conditions: 14 | 15 | The above copyright notice and this permission notice shall be included in all 16 | copies or substantial portions of the Software. 17 | 18 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 19 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 20 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 21 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 22 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 23 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 24 | SOFTWARE. 25 | 26 | */ 27 | 28 | #include 29 | #include 30 | #include 31 | #include 32 | 33 | #include "RegistExt.h" 34 | #include 35 | #include 36 | 37 | 38 | struct histobin 39 | { 40 | double binval; 41 | int count, accumcount; 42 | histobin(double b = 0, int c = 0, int ac = 0) 43 | { 44 | binval = b; 45 | count = c; 46 | accumcount = ac; 47 | } 48 | }; 49 | 50 | 51 | struct interpolatebin 52 | { 53 | double xval, yval, sigma, sem; 54 | int count; 55 | interpolatebin(double x = 0, double y = 0, double s = 0, int c = 0) 56 | { 57 | xval = x; 58 | yval = y; 59 | count = c; 60 | sem = sigma = 0.0; // Standard Error of Mean (SEM) is sigma/sqrt(N) 61 | // Margin of error is then Z*sigma/sqrt(N) for a given Z score 62 | } 63 | }; 64 | 65 | 66 | std::vector< std::vector > GetColumns(sqlite3* db, std::string sqlxprs, int *rc); 67 | 68 | std::vector CalcHistogram(std::vector< std::vector > Yvals, 69 | int bins, double minbin, double maxbin, int *rc); 70 | 71 | std::vector CalcInterpolations(std::vector< std::vector > XYvals, 72 | int bins, double minbin, double maxbin, int *rc); 73 | 74 | 75 | 76 | #pragma once 77 | -------------------------------------------------------------------------------- /histo.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Oeffner/SQLiteHistograms/374b0645e7ac6efdf1e5f66d83102cc57052a037/histo.jpg -------------------------------------------------------------------------------- /histogram.cpp: -------------------------------------------------------------------------------- 1 | /* 2 | histogram.cpp, Robert Oeffner 2018 3 | 4 | The MIT License (MIT) 5 | 6 | Copyright (c) 2017 Robert Oeffner 7 | 8 | Permission is hereby granted, free of charge, to any person obtaining a copy 9 | of this software and associated documentation files (the "Software"), to deal 10 | in the Software without restriction, including without limitation the rights 11 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 12 | copies of the Software, and to permit persons to whom the Software is 13 | furnished to do so, subject to the following conditions: 14 | 15 | The above copyright notice and this permission notice shall be included in all 16 | copies or substantial portions of the Software. 17 | 18 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 19 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 20 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 21 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 22 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 23 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 24 | SOFTWARE. 25 | 26 | */ 27 | 28 | #include 29 | #include 30 | #include 31 | 32 | #include "RegistExt.h" 33 | #include "helpers.h" 34 | #include 35 | #include 36 | 37 | 38 | #ifndef SQLITE_OMIT_VIRTUALTABLE 39 | 40 | 41 | 42 | #ifdef __cplusplus 43 | extern "C" { 44 | #endif 45 | 46 | 47 | 48 | /* histo_cursor is a subclass of sqlite3_vtab_cursor which will 49 | ** serve as the underlying representation of a cursor that scans 50 | ** over rows of the result 51 | */ 52 | typedef struct histo_cursor histo_cursor; 53 | struct histo_cursor { 54 | sqlite3_vtab_cursor base; /* Base class - must be first */ 55 | int isDesc; /* True to count down rather than up */ 56 | sqlite3_int64 iRowid; /* The rowid */ 57 | double bin; 58 | sqlite3_int64 count1; 59 | sqlite3_int64 count2; 60 | double ratio; 61 | sqlite3_int64 totalcount; 62 | std::string tblname; 63 | std::string colid; 64 | int nbins; 65 | double minbin; 66 | double maxbin; 67 | std::vector histogram; 68 | }; 69 | 70 | 71 | 72 | enum ColNum 73 | { /* Column numbers. The order determines the order of columns in the table output 74 | and must match the order of columns in the CREATE TABLE statement below 75 | */ 76 | HISTO_BIN = 0, 77 | HISTO_COUNT1, 78 | HISTO_COUNT2, 79 | HISTO_TBLNAME, 80 | HISTO_COLID, 81 | HISTO_NBINS, 82 | HISTO_MINBIN, 83 | HISTO_MAXBIN 84 | }; 85 | 86 | 87 | /* 88 | ** The histoConnect() method is invoked to create a new 89 | ** histo_vtab that describes the generate_histo virtual table. 90 | ** As the histoCreate method is set to NULL this virtual table is 91 | ** an Eponymous-only virtual table, i.e. useful as a table-valued function. 92 | ** The hidden columns are the arguments to the function and won't show up 93 | ** in the SQL tables. 94 | ** Think of this routine as the constructor for histo_vtab objects. 95 | ** 96 | ** All this routine needs to do is: 97 | ** 98 | ** (1) Allocate the histo_vtab object and initialize all fields. 99 | ** 100 | ** (2) Tell SQLite (via the sqlite3_declare_vtab() interface) what the 101 | ** result set of queries against generate_histo will look like. 102 | */ 103 | int histoConnect( 104 | sqlite3 *db, 105 | void *pAux, 106 | int argc, const char *const*argv, 107 | sqlite3_vtab **ppVtab, 108 | char **pzErr 109 | ) 110 | { 111 | sqlite3_vtab *pNew; 112 | int rc; 113 | // The hidden columns serves as arguments to the HISTO function as in: 114 | // SELECT * FROM HISTO('tblname', 'colid', nbins, minbin, maxbin, 'discrcolid', discrval); 115 | // They won't show up in the SQL tables. 116 | rc = sqlite3_declare_vtab(db, 117 | // Order of columns MUST match the order of the above enum ColNum 118 | "CREATE TABLE x(bin REAL, bincount INTEGER, accumcount INTEGER, " \ 119 | "tblname hidden, colid hidden, nbins hidden, minbin hidden, maxbin hidden)"); 120 | if( rc==SQLITE_OK ) 121 | { 122 | pNew = *ppVtab = (sqlite3_vtab *)sqlite3_malloc( sizeof(*pNew) ); 123 | if( pNew==0 ) return SQLITE_NOMEM; 124 | memset(pNew, 0, sizeof(*pNew)); 125 | } 126 | thisdb = db; 127 | return rc; 128 | } 129 | 130 | /* 131 | ** This method is the destructor for histo_cursor objects. 132 | */ 133 | int histoDisconnect(sqlite3_vtab *pVtab){ 134 | sqlite3_free(pVtab); 135 | return SQLITE_OK; 136 | } 137 | 138 | /* 139 | ** Constructor for a new histo_cursor object. 140 | */ 141 | int histoOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){ 142 | histo_cursor *pCur; 143 | // allocate c++ object with new rather than sqlite3_malloc which doesn't call constructors 144 | pCur = new histo_cursor; 145 | if( pCur==NULL ) return SQLITE_NOMEM; 146 | *ppCursor = &pCur->base; 147 | return SQLITE_OK; 148 | } 149 | 150 | /* 151 | ** Destructor for a histo_cursor. 152 | */ 153 | int histoClose(sqlite3_vtab_cursor *cur){ 154 | delete cur; 155 | return SQLITE_OK; 156 | } 157 | 158 | 159 | /* 160 | ** Advance a histo_cursor to its next row of output. 161 | */ 162 | int histoNext(sqlite3_vtab_cursor *cur){ 163 | histo_cursor *pCur = (histo_cursor*)cur; 164 | pCur->iRowid++; 165 | int i = pCur->iRowid - 1; 166 | pCur->bin = pCur->histogram[i].binval; 167 | pCur->count1 = pCur->histogram[i].count; 168 | pCur->count2 = pCur->histogram[i].accumcount; 169 | return SQLITE_OK; 170 | } 171 | 172 | /* 173 | ** Return values of columns for the row at which the histo_cursor 174 | ** is currently pointing. 175 | */ 176 | int histoColumn( 177 | sqlite3_vtab_cursor *cur, /* The cursor */ 178 | sqlite3_context *ctx, /* First argument to sqlite3_result_...() */ 179 | int i /* Which column to return */ 180 | ){ 181 | histo_cursor *pCur = (histo_cursor*)cur; 182 | sqlite3_int64 x = 123456; 183 | std::string c = "waffle"; 184 | double d = -42.24; 185 | switch( i ){ 186 | case HISTO_BIN: d = pCur->bin; sqlite3_result_double(ctx, d); break; 187 | case HISTO_COUNT1: x = pCur->count1; sqlite3_result_int64(ctx, x); break; 188 | case HISTO_COUNT2: x = pCur->count2; sqlite3_result_int64(ctx, x); break; 189 | case HISTO_TBLNAME: c = pCur->tblname; sqlite3_result_text(ctx, c.c_str(), -1, NULL); break; 190 | case HISTO_COLID: c = pCur->colid; sqlite3_result_text(ctx, c.c_str(), -1, NULL); break; 191 | case HISTO_NBINS: x = pCur->nbins; sqlite3_result_double(ctx, x); break; 192 | case HISTO_MINBIN: d = pCur->minbin; sqlite3_result_double(ctx, d); break; 193 | case HISTO_MAXBIN: d = pCur->maxbin; sqlite3_result_double(ctx, d); break; 194 | default: x = pCur->count1; sqlite3_result_int64(ctx, x); break; 195 | } 196 | return SQLITE_OK; 197 | } 198 | 199 | /* 200 | ** Return the rowid for the current row. In this implementation, the 201 | ** rowid is the same as the output value. 202 | */ 203 | int histoRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){ 204 | histo_cursor *pCur = (histo_cursor*)cur; 205 | *pRowid = pCur->iRowid; 206 | return SQLITE_OK; 207 | } 208 | 209 | /* 210 | ** Return TRUE if the cursor has been moved off of the last 211 | ** row of output. 212 | */ 213 | 214 | int histoEof(sqlite3_vtab_cursor *cur) { 215 | histo_cursor *pCur = (histo_cursor*)cur; 216 | if (pCur->isDesc) { 217 | return pCur->iRowid < 1; 218 | } 219 | else { 220 | return pCur->iRowid > pCur->histogram.size(); 221 | } 222 | } 223 | 224 | 225 | /* 226 | ** This method is called to "rewind" the histo_cursor object back 227 | ** to the first row of output. This method is always called at least 228 | ** once prior to any call to histoColumn() or histoRowid() or 229 | ** histoEof(). 230 | ** 231 | ** The query plan selected by histoBestIndex is passed in the idxNum 232 | ** parameter. (idxStr is not used in this implementation.) idxNum 233 | ** is one of the enum ColNum values above. 234 | ** This routine should initialize the cursor and position it so that it 235 | ** is pointing at the first row, or pointing off the end of the table 236 | ** (so that histoEof() will return true) if the table is empty. 237 | */ 238 | int histoFilter( 239 | sqlite3_vtab_cursor *pVtabCursor, 240 | int idxNum, const char *idxStr, 241 | int argc, sqlite3_value **argv 242 | ){ 243 | histo_cursor *pCur = (histo_cursor *)pVtabCursor; 244 | int i = 0, rc = SQLITE_OK; 245 | pCur->tblname = ""; 246 | pCur->colid = ""; 247 | pCur->nbins = 1.0; 248 | pCur->minbin = 1.0; 249 | pCur->maxbin = 1.0; 250 | 251 | if( idxNum >= HISTO_MAXBIN) 252 | { 253 | pCur->tblname = (const char*)sqlite3_value_text(argv[i++]); 254 | pCur->colid = (const char*)sqlite3_value_text(argv[i++]); 255 | pCur->nbins = sqlite3_value_double(argv[i++]); 256 | pCur->minbin = sqlite3_value_double(argv[i++]); 257 | pCur->maxbin = sqlite3_value_double(argv[i++]); 258 | } 259 | else 260 | { 261 | const char *zText = "Incorrect arguments for function HISTO which must be called as:\n" \ 262 | " HISTO('tablename', 'columnname', nbins, minbin, maxbin)\n"; 263 | pCur->base.pVtab->zErrMsg = sqlite3_mprintf( zText); 264 | 265 | return SQLITE_ERROR; 266 | } 267 | 268 | std::vector< std::vector > mybins; 269 | std::string s_exe("SELECT "); 270 | s_exe += pCur->colid + " FROM " + pCur->tblname; 271 | mybins.clear(); 272 | mybins = GetColumns(thisdb, s_exe, &rc); 273 | if (rc != SQLITE_OK) 274 | { 275 | pCur->base.pVtab->zErrMsg = sqlite3_mprintf(sqlite3_errmsg(thisdb)); 276 | return rc; 277 | } 278 | pCur->histogram = CalcHistogram(mybins, pCur->nbins, pCur->minbin, pCur->maxbin, &rc); 279 | if (rc != SQLITE_OK) 280 | return rc; 281 | 282 | pCur->bin = pCur->histogram[0].binval; 283 | pCur->count1 = pCur->histogram[0].count; 284 | pCur->count2 = pCur->histogram[0].accumcount; 285 | pCur->isDesc = 0; 286 | pCur->iRowid = 1; 287 | 288 | return rc; 289 | } 290 | 291 | 292 | /* 293 | ** SQLite will invoke this method one or more times while planning a query 294 | ** that uses the generate_histo virtual table. This routine needs to create 295 | ** a query plan for each invocation and compute an estimated cost for that 296 | ** plan. 297 | ** 298 | ** In this implementation idxNum is used to represent the 299 | ** query plan. idxStr is unused. 300 | ** 301 | ** The query plan is represented by bits in idxNum: 302 | ** 303 | ** (1) start = $value -- constraint exists 304 | ** (2) stop = $value -- constraint exists 305 | ** (4) step = $value -- constraint exists 306 | ** (8) output in descending order 307 | */ 308 | int histoBestIndex( 309 | sqlite3_vtab *tab, 310 | sqlite3_index_info *pIdxInfo 311 | ){ 312 | int i; /* Loop over constraints */ 313 | int idxNum = 0; /* The query plan bitmask */ 314 | int tblnameidx = -1; /* Index of the start= constraint, or -1 if none */ 315 | int colididx = -1; /* Index of the stop= constraint, or -1 if none */ 316 | int binsidx = -1; /* Index of the step= constraint, or -1 if none */ 317 | int minbinidx = -1; 318 | int maxbinidx = -1; 319 | int nArg = 0; /* Number of arguments that histoFilter() expects */ 320 | 321 | sqlite3_index_info::sqlite3_index_constraint *pConstraint; 322 | pConstraint = pIdxInfo->aConstraint; 323 | for(i=0; inConstraint; i++, pConstraint++){ 324 | if( pConstraint->usable==0 ) continue; 325 | if( pConstraint->op!=SQLITE_INDEX_CONSTRAINT_EQ ) continue; 326 | switch( pConstraint->iColumn ){ 327 | case HISTO_TBLNAME: 328 | tblnameidx = i; 329 | idxNum = HISTO_TBLNAME; 330 | break; 331 | case HISTO_COLID: 332 | colididx = i; 333 | idxNum = HISTO_COLID; 334 | break; 335 | case HISTO_NBINS: 336 | binsidx = i; 337 | idxNum = HISTO_NBINS; 338 | break; 339 | case HISTO_MINBIN: 340 | minbinidx = i; 341 | idxNum = HISTO_MINBIN; 342 | break; 343 | case HISTO_MAXBIN: 344 | maxbinidx = i; 345 | idxNum = HISTO_MAXBIN; 346 | break; 347 | } 348 | } 349 | if(tblnameidx >=0 ){ 350 | pIdxInfo->aConstraintUsage[tblnameidx].argvIndex = ++nArg; 351 | pIdxInfo->aConstraintUsage[tblnameidx].omit= 1; 352 | } 353 | if(colididx >=0 ){ 354 | pIdxInfo->aConstraintUsage[colididx].argvIndex = ++nArg; 355 | pIdxInfo->aConstraintUsage[colididx].omit = 1; 356 | } 357 | if (binsidx >= 0) { 358 | pIdxInfo->aConstraintUsage[binsidx].argvIndex = ++nArg; 359 | pIdxInfo->aConstraintUsage[binsidx].omit = 1; 360 | } 361 | if (minbinidx >= 0) { 362 | pIdxInfo->aConstraintUsage[minbinidx].argvIndex = ++nArg; 363 | pIdxInfo->aConstraintUsage[minbinidx].omit = 1; 364 | } 365 | if (maxbinidx >= 0) { 366 | pIdxInfo->aConstraintUsage[maxbinidx].argvIndex = ++nArg; 367 | pIdxInfo->aConstraintUsage[maxbinidx].omit = 1; 368 | } 369 | pIdxInfo->estimatedCost = 2.0; 370 | pIdxInfo->estimatedRows = 500; 371 | if( pIdxInfo->nOrderBy==1 ) 372 | { 373 | pIdxInfo->orderByConsumed = 1; 374 | } 375 | pIdxInfo->idxNum = idxNum; 376 | return SQLITE_OK; 377 | } 378 | 379 | 380 | 381 | /* 382 | ** This following structure defines all the methods for the 383 | ** generate_histo virtual table. 384 | */ 385 | sqlite3_module histoModule = { 386 | 0, /* iVersion */ 387 | 0, /* xCreate */ 388 | histoConnect, /* xConnect */ 389 | histoBestIndex, /* xBestIndex */ 390 | histoDisconnect, /* xDisconnect */ 391 | 0, /* xDestroy */ 392 | histoOpen, /* xOpen - open a cursor */ 393 | histoClose, /* xClose - close a cursor */ 394 | histoFilter, /* xFilter - configure scan constraints */ 395 | histoNext, /* xNext - advance a cursor */ 396 | histoEof, /* xEof - check for end of scan */ 397 | histoColumn, /* xColumn - read data */ 398 | histoRowid, /* xRowid - read data */ 399 | 0, /* xUpdate */ 400 | 0, /* xBegin */ 401 | 0, /* xSync */ 402 | 0, /* xCommit */ 403 | 0, /* xRollback */ 404 | 0, /* xFindMethod */ 405 | 0, /* xRename */ 406 | }; 407 | 408 | 409 | 410 | #endif /* SQLITE_OMIT_VIRTUALTABLE */ 411 | 412 | 413 | 414 | 415 | #ifdef __cplusplus 416 | } 417 | #endif 418 | -------------------------------------------------------------------------------- /mean.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Oeffner/SQLiteHistograms/374b0645e7ac6efdf1e5f66d83102cc57052a037/mean.jpg -------------------------------------------------------------------------------- /meanhistogram.cpp: -------------------------------------------------------------------------------- 1 | /* 2 | meanhistogram.cpp, Robert Oeffner 2018 3 | 4 | SQLite extension for calculating interpolated curve of one dimensional scatterplot 5 | of column values as well as corresponding standard deviations. 6 | 7 | The MIT License (MIT) 8 | 9 | Copyright (c) 2017 Robert Oeffner 10 | 11 | Permission is hereby granted, free of charge, to any person obtaining a copy 12 | of this software and associated documentation files (the "Software"), to deal 13 | in the Software without restriction, including without limitation the rights 14 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 15 | copies of the Software, and to permit persons to whom the Software is 16 | furnished to do so, subject to the following conditions: 17 | 18 | The above copyright notice and this permission notice shall be included in all 19 | copies or substantial portions of the Software. 20 | 21 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 22 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 23 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 24 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 25 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 26 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 27 | SOFTWARE. 28 | 29 | */ 30 | 31 | #include 32 | #include 33 | #include 34 | 35 | #include "RegistExt.h" 36 | #include "helpers.h" 37 | #include 38 | #include 39 | 40 | 41 | #ifndef SQLITE_OMIT_VIRTUALTABLE 42 | 43 | 44 | 45 | 46 | #ifdef __cplusplus 47 | extern "C" { 48 | #endif 49 | 50 | 51 | 52 | /* meanhisto_cursor is a subclass of sqlite3_vtab_cursor which will 53 | ** serve as the underlying representation of a cursor that scans 54 | ** over rows of the result 55 | */ 56 | typedef struct meanhisto_cursor meanhisto_cursor; 57 | struct meanhisto_cursor { 58 | sqlite3_vtab_cursor base; /* Base class - must be first */ 59 | int isDesc; /* True to count down rather than up */ 60 | sqlite3_int64 iRowid; /* The rowid */ 61 | double x; 62 | double y; 63 | double sigma; 64 | double sem; 65 | sqlite3_int64 count; 66 | std::string tblname; 67 | std::string xcolid; 68 | std::string ycolid; 69 | int nbins; 70 | double minbin; 71 | double maxbin; 72 | std::vector meanhistobins; 73 | }; 74 | 75 | 76 | 77 | enum ColNum 78 | { /* Column numbers. The order determines the order of columns in the table output 79 | and must match the order of columns in the CREATE TABLE statement below 80 | */ 81 | MEANHISTO_X = 0, 82 | MEANHISTO_Y, 83 | MEANHISTO_SIGMA, 84 | MEANHISTO_SEM, 85 | MEANHISTO_COUNT, 86 | MEANHISTO_TBLNAME, 87 | MEANHISTO_XCOLID, 88 | MEANHISTO_YCOLID, 89 | MEANHISTO_NBINS, 90 | MEANHISTO_MINBIN, 91 | MEANHISTO_MAXBIN 92 | }; 93 | 94 | 95 | int meanhistoConnect( 96 | sqlite3 *db, 97 | void *pAux, 98 | int argc, const char *const*argv, 99 | sqlite3_vtab **ppVtab, 100 | char **pzErr 101 | ) 102 | { 103 | sqlite3_vtab *pNew; 104 | int rc; 105 | /* The hidden columns serves as arguments to the MEANHISTO function as in: 106 | SELECT * FROM MEANHISTO('tblname', 'xcolid', 'ycolid', nbins, minbin, maxbin); 107 | They won't show up in the SQL tables. 108 | */ 109 | rc = sqlite3_declare_vtab(db, 110 | // Order of columns MUST match the order of the above enum ColNum 111 | "CREATE TABLE x(xbin REAL, yval REAL, sigma REAL, sem REAL, bincount INTEGER, " \ 112 | "tblname hidden, xcolid hidden, ycolid hidden, nbins hidden, minbin hidden, maxbin hidden)"); 113 | if( rc==SQLITE_OK ) 114 | { 115 | pNew = *ppVtab = (sqlite3_vtab *)sqlite3_malloc( sizeof(*pNew) ); 116 | if( pNew==0 ) return SQLITE_NOMEM; 117 | memset(pNew, 0, sizeof(*pNew)); 118 | } 119 | thisdb = db; 120 | return rc; 121 | } 122 | 123 | /* 124 | ** This method is the destructor for meanhisto_cursor objects. 125 | */ 126 | int meanhistoDisconnect(sqlite3_vtab *pVtab){ 127 | sqlite3_free(pVtab); 128 | return SQLITE_OK; 129 | } 130 | 131 | /* 132 | ** Constructor for a new meanhisto_cursor object. 133 | */ 134 | int meanhistoOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){ 135 | meanhisto_cursor *pCur; 136 | // allocate c++ object with new rather than sqlite3_malloc which doesn't call constructors 137 | pCur = new meanhisto_cursor; 138 | if (pCur == NULL) return SQLITE_NOMEM; 139 | *ppCursor = &pCur->base; 140 | return SQLITE_OK; 141 | } 142 | 143 | /* 144 | ** Destructor for a meanhisto_cursor. 145 | */ 146 | int meanhistoClose(sqlite3_vtab_cursor *cur){ 147 | delete cur; 148 | return SQLITE_OK; 149 | } 150 | 151 | 152 | /* 153 | ** Advance a meanhisto_cursor to its next row of output. 154 | */ 155 | int meanhistoNext(sqlite3_vtab_cursor *cur){ 156 | meanhisto_cursor *pCur = (meanhisto_cursor*)cur; 157 | pCur->iRowid++; 158 | int i = pCur->iRowid - 1; 159 | pCur->x = pCur->meanhistobins[i].xval; 160 | pCur->y = pCur->meanhistobins[i].yval; 161 | pCur->sigma = pCur->meanhistobins[i].sigma; 162 | pCur->sem = pCur->meanhistobins[i].sem; 163 | pCur->count = pCur->meanhistobins[i].count; 164 | return SQLITE_OK; 165 | } 166 | 167 | /* 168 | ** Return values of columns for the row at which the meanhisto_cursor 169 | ** is currently pointing. 170 | */ 171 | int meanhistoColumn( 172 | sqlite3_vtab_cursor *cur, /* The cursor */ 173 | sqlite3_context *ctx, /* First argument to sqlite3_result_...() */ 174 | int i /* Which column to return */ 175 | ){ 176 | meanhisto_cursor *pCur = (meanhisto_cursor*)cur; 177 | sqlite3_int64 x = 123456; 178 | std::string c = "waffle"; 179 | double d = -42.24; 180 | switch( i ){ 181 | case MEANHISTO_X: d = pCur->x; sqlite3_result_double(ctx, d); break; 182 | case MEANHISTO_Y: d = pCur->y; sqlite3_result_double(ctx, d); break; 183 | case MEANHISTO_SIGMA: d = pCur->sigma; sqlite3_result_double(ctx, d); break; 184 | case MEANHISTO_SEM: d = pCur->sem; sqlite3_result_double(ctx, d); break; 185 | case MEANHISTO_COUNT: x = pCur->count; sqlite3_result_int64(ctx, x); break; 186 | case MEANHISTO_TBLNAME: c = pCur->tblname; sqlite3_result_text(ctx, c.c_str(), -1, NULL); break; 187 | case MEANHISTO_XCOLID: c = pCur->xcolid; sqlite3_result_text(ctx, c.c_str(), -1, NULL); break; 188 | case MEANHISTO_YCOLID: c = pCur->ycolid; sqlite3_result_text(ctx, c.c_str(), -1, NULL); break; 189 | case MEANHISTO_NBINS: x = pCur->nbins; sqlite3_result_double(ctx, x); break; 190 | case MEANHISTO_MINBIN: d = pCur->minbin; sqlite3_result_double(ctx, d); break; 191 | case MEANHISTO_MAXBIN: d = pCur->maxbin; sqlite3_result_double(ctx, d); break; 192 | default: sqlite3_result_double(ctx, 0); break; 193 | } 194 | return SQLITE_OK; 195 | } 196 | 197 | /* 198 | ** Return the rowid for the current row. In this implementation, the 199 | ** rowid is the same as the output value. 200 | */ 201 | int meanhistoRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){ 202 | meanhisto_cursor *pCur = (meanhisto_cursor*)cur; 203 | *pRowid = pCur->iRowid; 204 | return SQLITE_OK; 205 | } 206 | 207 | /* 208 | ** Return TRUE if the cursor has been moved off of the last 209 | ** row of output. 210 | */ 211 | 212 | int meanhistoEof(sqlite3_vtab_cursor *cur) { 213 | meanhisto_cursor *pCur = (meanhisto_cursor*)cur; 214 | if (pCur->isDesc) { 215 | return pCur->iRowid < 1; 216 | } 217 | else { 218 | return pCur->iRowid > pCur->meanhistobins.size(); 219 | } 220 | } 221 | 222 | 223 | 224 | int meanhistoFilter( 225 | sqlite3_vtab_cursor *pVtabCursor, 226 | int idxNum, const char *idxStr, 227 | int argc, sqlite3_value **argv 228 | ){ 229 | meanhisto_cursor *pCur = (meanhisto_cursor *)pVtabCursor; 230 | int i = 0, rc = SQLITE_OK; 231 | pCur->tblname = ""; 232 | pCur->xcolid = ""; 233 | pCur->ycolid = ""; 234 | pCur->nbins = 1.0; 235 | pCur->minbin = 1.0; 236 | pCur->maxbin = 1.0; 237 | 238 | if( idxNum >= MEANHISTO_MAXBIN) 239 | { 240 | pCur->tblname = (const char*)sqlite3_value_text(argv[i++]); 241 | pCur->xcolid = (const char*)sqlite3_value_text(argv[i++]); 242 | pCur->ycolid = (const char*)sqlite3_value_text(argv[i++]); 243 | pCur->nbins = sqlite3_value_double(argv[i++]); 244 | pCur->minbin = sqlite3_value_double(argv[i++]); 245 | pCur->maxbin = sqlite3_value_double(argv[i++]); 246 | } 247 | else 248 | { 249 | const char *zText = "Incorrect arguments for function MEANHISTO which must be called as:\n" \ 250 | " MEANHISTO('tablename', 'xcolumnname', 'ycolumnname', nbins, minbin, maxbin)\n"; 251 | pCur->base.pVtab->zErrMsg = sqlite3_mprintf(zText); 252 | return SQLITE_ERROR; 253 | } 254 | 255 | std::string s_exe("SELECT "); 256 | s_exe += pCur->xcolid + ", " + pCur->ycolid + " FROM " + pCur->tblname; 257 | std::vector< std::vector > myXYs = GetColumns(thisdb, s_exe, &rc); 258 | if (rc != SQLITE_OK) 259 | { 260 | pCur->base.pVtab->zErrMsg = sqlite3_mprintf(sqlite3_errmsg(thisdb)); 261 | return rc; 262 | } 263 | pCur->meanhistobins = CalcInterpolations(myXYs, pCur->nbins, pCur->minbin, pCur->maxbin, &rc); 264 | if (rc != SQLITE_OK) 265 | return rc; 266 | 267 | pCur->x = pCur->meanhistobins[0].xval; 268 | pCur->y = pCur->meanhistobins[0].yval; 269 | pCur->sigma = pCur->meanhistobins[0].sigma; 270 | pCur->sem = pCur->meanhistobins[0].sem; 271 | pCur->count = pCur->meanhistobins[0].count; 272 | pCur->isDesc = 0; 273 | pCur->iRowid = 1; 274 | 275 | return SQLITE_OK; 276 | } 277 | 278 | 279 | int meanhistoBestIndex( 280 | sqlite3_vtab *tab, 281 | sqlite3_index_info *pIdxInfo 282 | ){ 283 | int i; /* Loop over constraints */ 284 | int idxNum = 0; /* The query plan bitmask */ 285 | int tblnameidx = -1; /* Index of the start= constraint, or -1 if none */ 286 | int xcolididx = -1; /* Index of the stop= constraint, or -1 if none */ 287 | int ycolididx = -1; /* Index of the stop= constraint, or -1 if none */ 288 | int binsidx = -1; /* Index of the step= constraint, or -1 if none */ 289 | int minbinidx = -1; 290 | int maxbinidx = -1; 291 | int nArg = 0; /* Number of arguments that meanhistoFilter() expects */ 292 | 293 | sqlite3_index_info::sqlite3_index_constraint *pConstraint; 294 | pConstraint = pIdxInfo->aConstraint; 295 | for(i=0; inConstraint; i++, pConstraint++){ 296 | if( pConstraint->usable==0 ) continue; 297 | if( pConstraint->op!=SQLITE_INDEX_CONSTRAINT_EQ ) continue; 298 | switch( pConstraint->iColumn ){ 299 | case MEANHISTO_TBLNAME: 300 | tblnameidx = i; 301 | idxNum = MEANHISTO_TBLNAME; 302 | break; 303 | case MEANHISTO_XCOLID: 304 | xcolididx = i; 305 | idxNum = MEANHISTO_XCOLID; 306 | break; 307 | case MEANHISTO_YCOLID: 308 | ycolididx = i; 309 | idxNum = MEANHISTO_YCOLID; 310 | break; 311 | case MEANHISTO_NBINS: 312 | binsidx = i; 313 | idxNum = MEANHISTO_NBINS; 314 | break; 315 | case MEANHISTO_MINBIN: 316 | minbinidx = i; 317 | idxNum = MEANHISTO_MINBIN; 318 | break; 319 | case MEANHISTO_MAXBIN: 320 | maxbinidx = i; 321 | idxNum = MEANHISTO_MAXBIN; 322 | break; 323 | } 324 | } 325 | if(tblnameidx >=0 ){ 326 | pIdxInfo->aConstraintUsage[tblnameidx].argvIndex = ++nArg; 327 | pIdxInfo->aConstraintUsage[tblnameidx].omit= 1; 328 | } 329 | if (xcolididx >= 0) { 330 | pIdxInfo->aConstraintUsage[xcolididx].argvIndex = ++nArg; 331 | pIdxInfo->aConstraintUsage[xcolididx].omit = 1; 332 | } 333 | if (ycolididx >= 0) { 334 | pIdxInfo->aConstraintUsage[ycolididx].argvIndex = ++nArg; 335 | pIdxInfo->aConstraintUsage[ycolididx].omit = 1; 336 | } 337 | if (binsidx >= 0) { 338 | pIdxInfo->aConstraintUsage[binsidx].argvIndex = ++nArg; 339 | pIdxInfo->aConstraintUsage[binsidx].omit = 1; 340 | } 341 | if (minbinidx >= 0) { 342 | pIdxInfo->aConstraintUsage[minbinidx].argvIndex = ++nArg; 343 | pIdxInfo->aConstraintUsage[minbinidx].omit = 1; 344 | } 345 | if (maxbinidx >= 0) { 346 | pIdxInfo->aConstraintUsage[maxbinidx].argvIndex = ++nArg; 347 | pIdxInfo->aConstraintUsage[maxbinidx].omit = 1; 348 | } 349 | pIdxInfo->estimatedCost = 2.0; 350 | pIdxInfo->estimatedRows = 500; 351 | if( pIdxInfo->nOrderBy==1 ) 352 | { 353 | pIdxInfo->orderByConsumed = 1; 354 | } 355 | pIdxInfo->idxNum = idxNum; 356 | return SQLITE_OK; 357 | } 358 | 359 | 360 | 361 | 362 | /* 363 | ** This following structure defines all the methods for the 364 | ** generate_meanhisto virtual table. 365 | */ 366 | sqlite3_module meanhistoModule = { 367 | 0, /* iVersion */ 368 | 0, /* xCreate */ 369 | meanhistoConnect, /* xConnect */ 370 | meanhistoBestIndex, /* xBestIndex */ 371 | meanhistoDisconnect, /* xDisconnect */ 372 | 0, /* xDestroy */ 373 | meanhistoOpen, /* xOpen - open a cursor */ 374 | meanhistoClose, /* xClose - close a cursor */ 375 | meanhistoFilter, /* xFilter - configure scan constraints */ 376 | meanhistoNext, /* xNext - advance a cursor */ 377 | meanhistoEof, /* xEof - check for end of scan */ 378 | meanhistoColumn, /* xColumn - read data */ 379 | meanhistoRowid, /* xRowid - read data */ 380 | 0, /* xUpdate */ 381 | 0, /* xBegin */ 382 | 0, /* xSync */ 383 | 0, /* xCommit */ 384 | 0, /* xRollback */ 385 | 0, /* xFindMethod */ 386 | 0, /* xRename */ 387 | }; 388 | 389 | 390 | 391 | #endif /* SQLITE_OMIT_VIRTUALTABLE */ 392 | 393 | 394 | 395 | 396 | #ifdef __cplusplus 397 | } 398 | #endif 399 | -------------------------------------------------------------------------------- /ratio1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Oeffner/SQLiteHistograms/374b0645e7ac6efdf1e5f66d83102cc57052a037/ratio1.jpg -------------------------------------------------------------------------------- /ratio2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Oeffner/SQLiteHistograms/374b0645e7ac6efdf1e5f66d83102cc57052a037/ratio2.jpg -------------------------------------------------------------------------------- /ratiohistogram.cpp: -------------------------------------------------------------------------------- 1 | /* 2 | ratiohistogram.cpp, Robert Oeffner 2018 3 | 4 | SQLite extension for calculating ratiohistogram of column values as well as calculating ratios of 5 | two histograms deduced from the same table. 6 | 7 | The MIT License (MIT) 8 | 9 | Copyright (c) 2017 Robert Oeffner 10 | 11 | Permission is hereby granted, free of charge, to any person obtaining a copy 12 | of this software and associated documentation files (the "Software"), to deal 13 | in the Software without restriction, including without limitation the rights 14 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 15 | copies of the Software, and to permit persons to whom the Software is 16 | furnished to do so, subject to the following conditions: 17 | 18 | The above copyright notice and this permission notice shall be included in all 19 | copies or substantial portions of the Software. 20 | 21 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 22 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 23 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 24 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 25 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 26 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 27 | SOFTWARE. 28 | 29 | */ 30 | 31 | #include 32 | #include 33 | #include 34 | 35 | #include "RegistExt.h" 36 | #include "helpers.h" 37 | #include 38 | #include 39 | 40 | 41 | #ifndef SQLITE_OMIT_VIRTUALTABLE 42 | 43 | 44 | #ifdef __cplusplus 45 | extern "C" { 46 | #endif 47 | 48 | /* histo_cursor is a subclass of sqlite3_vtab_cursor which will 49 | ** serve as the underlying representation of a cursor that scans 50 | ** over rows of the result 51 | */ 52 | typedef struct ratiohisto_cursor ratiohisto_cursor; 53 | struct ratiohisto_cursor { 54 | sqlite3_vtab_cursor base; /* Base class - must be first */ 55 | int isDesc; /* True to count down rather than up */ 56 | sqlite3_int64 iRowid; /* The rowid */ 57 | double bin; 58 | sqlite3_int64 count1; 59 | sqlite3_int64 count2; 60 | double ratio; 61 | sqlite3_int64 totalcount; 62 | std::string tblname; 63 | std::string colid; 64 | int nbins; 65 | double minbin; 66 | double maxbin; 67 | std::string discrcolid; 68 | std::string discrval; 69 | std::vector ratiohistogram1; 70 | std::vector ratiohistogram2; 71 | }; 72 | 73 | 74 | 75 | enum ColNum 76 | { /* Column numbers. The order determines the order of columns in the table output 77 | and must match the order of columns in the CREATE TABLE statement below 78 | */ 79 | RATIOHISTO_BIN = 0, 80 | RATIOHISTO_COUNT1, 81 | RATIOHISTO_COUNT2, 82 | RATIOHISTO_RATIO, 83 | RATIOHISTO_TOTALCOUNT, 84 | RATIOHISTO_TBLNAME, 85 | RATIOHISTO_COLID, 86 | RATIOHISTO_NBINS, 87 | RATIOHISTO_MINBIN, 88 | RATIOHISTO_MAXBIN, 89 | RATIOHISTO_DISCRCOLID, 90 | RATIOHISTO_DISCRVAL 91 | }; 92 | 93 | 94 | /* 95 | ** The ratiohistoConnect() method is invoked to create a new 96 | ** ratiohisto_vtab that describes the generate_ratiohisto virtual table. 97 | ** As the ratiohistoCreate method is set to NULL this virtual table is 98 | ** an Eponymous-only virtual table, i.e. useful as a table-valued function. 99 | ** The hidden columns are the arguments to the function and won't show up 100 | ** in the SQL tables. 101 | ** Think of this routine as the constructor for ratiohisto_vtab objects. 102 | ** 103 | ** All this routine needs to do is: 104 | ** 105 | ** (1) Allocate the ratiohisto_vtab object and initialize all fields. 106 | ** 107 | ** (2) Tell SQLite (via the sqlite3_declare_vtab() interface) what the 108 | ** result set of queries against generate_ratiohisto will look like. 109 | */ 110 | int ratiohistoConnect( 111 | sqlite3 *db, 112 | void *pAux, 113 | int argc, const char *const*argv, 114 | sqlite3_vtab **ppVtab, 115 | char **pzErr 116 | ) 117 | { 118 | sqlite3_vtab *pNew; 119 | int rc; 120 | // The hidden columns serves as arguments to the RATIOHISTO function as in: 121 | // SELECT * FROM RATIOHISTO('tblname', 'colid', nbins, minbin, maxbin, 'discrcolid', discrval); 122 | // They won't show up in the SQL tables. 123 | rc = sqlite3_declare_vtab(db, 124 | // Order of columns MUST match the order of the above enum ColNum 125 | "CREATE TABLE x(bin REAL, count1 INTEGER, count2 INTEGER, ratio REAL, totalcount INTEGER, " \ 126 | "tblname hidden, colid hidden, nbins hidden, minbin hidden, maxbin hidden, discrcolid hidden, discrval hidden)"); 127 | if( rc==SQLITE_OK ) 128 | { 129 | pNew = *ppVtab = (sqlite3_vtab *)sqlite3_malloc( sizeof(*pNew) ); 130 | if( pNew==0 ) return SQLITE_NOMEM; 131 | memset(pNew, 0, sizeof(*pNew)); 132 | } 133 | thisdb = db; 134 | return rc; 135 | } 136 | 137 | /* 138 | ** This method is the destructor for ratiohisto_cursor objects. 139 | */ 140 | int ratiohistoDisconnect(sqlite3_vtab *pVtab){ 141 | sqlite3_free(pVtab); 142 | return SQLITE_OK; 143 | } 144 | 145 | /* 146 | ** Constructor for a new ratiohisto_cursor object. 147 | */ 148 | int ratiohistoOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){ 149 | ratiohisto_cursor *pCur; 150 | // allocate c++ object with new rather than sqlite3_malloc which doesn't call constructors 151 | pCur = new ratiohisto_cursor; 152 | if (pCur == NULL) return SQLITE_NOMEM; 153 | *ppCursor = &pCur->base; 154 | return SQLITE_OK; 155 | } 156 | 157 | /* 158 | ** Destructor for a ratiohisto_cursor. 159 | */ 160 | int ratiohistoClose(sqlite3_vtab_cursor *cur){ 161 | delete cur; 162 | return SQLITE_OK; 163 | } 164 | 165 | 166 | /* 167 | ** Advance a ratiohisto_cursor to its next row of output. 168 | */ 169 | int ratiohistoNext(sqlite3_vtab_cursor *cur){ 170 | ratiohisto_cursor *pCur = (ratiohisto_cursor*)cur; 171 | pCur->iRowid++; 172 | int i = pCur->iRowid - 1; 173 | pCur->bin = pCur->ratiohistogram1[i].binval; 174 | pCur->count1 = pCur->ratiohistogram1[i].count; 175 | pCur->count2 = pCur->ratiohistogram2[i].count; 176 | pCur->totalcount = pCur->ratiohistogram1[i].count + pCur->ratiohistogram2[i].count; 177 | if (pCur->totalcount > 0) 178 | pCur->ratio = ((double) pCur->count1 ) / pCur->totalcount; 179 | else 180 | pCur->ratio = 0.0; 181 | return SQLITE_OK; 182 | } 183 | 184 | /* 185 | ** Return values of columns for the row at which the ratiohisto_cursor 186 | ** is currently pointing. 187 | */ 188 | int ratiohistoColumn( 189 | sqlite3_vtab_cursor *cur, /* The cursor */ 190 | sqlite3_context *ctx, /* First argument to sqlite3_result_...() */ 191 | int i /* Which column to return */ 192 | ){ 193 | ratiohisto_cursor *pCur = (ratiohisto_cursor*)cur; 194 | sqlite3_int64 x = 123456; 195 | std::string c = "waffle"; 196 | double d = -42.24; 197 | switch( i ){ 198 | case RATIOHISTO_BIN: d = pCur->bin; sqlite3_result_double(ctx, d); break; 199 | case RATIOHISTO_COUNT1: x = pCur->count1; sqlite3_result_int64(ctx, x); break; 200 | case RATIOHISTO_COUNT2: { 201 | if (pCur->discrcolid == "") 202 | { // if not computing ratio ratiohistogram then return empty column values 203 | sqlite3_result_null(ctx); 204 | } 205 | else 206 | { 207 | x = pCur->count2; sqlite3_result_int64(ctx, x); 208 | } 209 | } break; 210 | 211 | case RATIOHISTO_RATIO: { 212 | if (pCur->discrcolid == "") 213 | { // if not computing ratio ratiohistogram then return empty column values 214 | sqlite3_result_null(ctx); 215 | } 216 | else 217 | { 218 | d = pCur->ratio; sqlite3_result_double(ctx, d); 219 | } 220 | } break; 221 | 222 | case RATIOHISTO_TOTALCOUNT: { 223 | if (pCur->discrcolid == "") 224 | { // if not computing ratio ratiohistogram then return empty column values 225 | sqlite3_result_null(ctx); 226 | } 227 | else 228 | { 229 | x = pCur->totalcount; sqlite3_result_int64(ctx, x); 230 | } 231 | } break; 232 | 233 | case RATIOHISTO_TBLNAME: c = pCur->tblname; sqlite3_result_text(ctx, c.c_str(), -1, NULL); break; 234 | case RATIOHISTO_COLID: c = pCur->colid; sqlite3_result_text(ctx, c.c_str(), -1, NULL); break; 235 | case RATIOHISTO_NBINS: x = pCur->nbins; sqlite3_result_double(ctx, x); break; 236 | case RATIOHISTO_MINBIN: d = pCur->minbin; sqlite3_result_double(ctx, d); break; 237 | case RATIOHISTO_MAXBIN: d = pCur->maxbin; sqlite3_result_double(ctx, d); break; 238 | case RATIOHISTO_DISCRCOLID: c = pCur->discrcolid; sqlite3_result_text(ctx, c.c_str(), -1, NULL); break; 239 | case RATIOHISTO_DISCRVAL: c = pCur->discrval; sqlite3_result_text(ctx, c.c_str(), -1, NULL); break; 240 | default: x = pCur->count1; sqlite3_result_int64(ctx, x); break; 241 | } 242 | return SQLITE_OK; 243 | } 244 | 245 | /* 246 | ** Return the rowid for the current row. In this implementation, the 247 | ** rowid is the same as the output value. 248 | */ 249 | int ratiohistoRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){ 250 | ratiohisto_cursor *pCur = (ratiohisto_cursor*)cur; 251 | *pRowid = pCur->iRowid; 252 | return SQLITE_OK; 253 | } 254 | 255 | /* 256 | ** Return TRUE if the cursor has been moved off of the last 257 | ** row of output. 258 | */ 259 | 260 | int ratiohistoEof(sqlite3_vtab_cursor *cur) { 261 | ratiohisto_cursor *pCur = (ratiohisto_cursor*)cur; 262 | if (pCur->isDesc) { 263 | return pCur->iRowid < 1; 264 | } 265 | else { 266 | return pCur->iRowid > pCur->ratiohistogram1.size(); 267 | } 268 | } 269 | 270 | 271 | /* 272 | ** This method is called to "rewind" the ratiohisto_cursor object back 273 | ** to the first row of output. This method is always called at least 274 | ** once prior to any call to ratiohistoColumn() or ratiohistoRowid() or 275 | ** ratiohistoEof(). 276 | ** 277 | ** The query plan selected by ratiohistoBestIndex is passed in the idxNum 278 | ** parameter. (idxStr is not used in this implementation.) idxNum 279 | ** is one of the enum ColNum values above. 280 | ** This routine should initialize the cursor and position it so that it 281 | ** is pointing at the first row, or pointing off the end of the table 282 | ** (so that ratiohistoEof() will return true) if the table is empty. 283 | */ 284 | int ratiohistoFilter( 285 | sqlite3_vtab_cursor *pVtabCursor, 286 | int idxNum, const char *idxStr, 287 | int argc, sqlite3_value **argv 288 | ){ 289 | ratiohisto_cursor *pCur = (ratiohisto_cursor *)pVtabCursor; 290 | int i = 0, rc = SQLITE_OK; 291 | pCur->tblname = ""; 292 | pCur->colid = ""; 293 | pCur->nbins = 1.0; 294 | pCur->minbin = 1.0; 295 | pCur->maxbin = 1.0; 296 | pCur->discrcolid = ""; 297 | pCur->discrval = "0.0"; 298 | 299 | if (idxNum >= RATIOHISTO_DISCRVAL) 300 | { 301 | pCur->tblname = (const char*)sqlite3_value_text(argv[i++]); 302 | pCur->colid = (const char*)sqlite3_value_text(argv[i++]); 303 | pCur->nbins = sqlite3_value_double(argv[i++]); 304 | pCur->minbin = sqlite3_value_double(argv[i++]); 305 | pCur->maxbin = sqlite3_value_double(argv[i++]); 306 | pCur->discrcolid = (const char*)sqlite3_value_text(argv[i++]); 307 | pCur->discrval = (const char*)sqlite3_value_text(argv[i++]); 308 | } 309 | else 310 | { 311 | const char *zText = "Incorrect arguments for function RATIOHISTO which must be called as:\n" \ 312 | "RATIOHISTO('tablename', 'columnname', nbins, minbin, maxbin, 'discrcolid', discrval)\n"; 313 | pCur->base.pVtab->zErrMsg = sqlite3_mprintf(zText); 314 | return SQLITE_ERROR; 315 | } 316 | 317 | std::vector< std::vector > mybins; 318 | std::string s_exe("SELECT "); 319 | s_exe += pCur->colid + " FROM " + pCur->tblname; 320 | mybins.clear(); 321 | mybins = GetColumns(thisdb, s_exe, &rc); 322 | if (rc != SQLITE_OK) 323 | { 324 | pCur->base.pVtab->zErrMsg = sqlite3_mprintf(sqlite3_errmsg(thisdb)); 325 | return rc; 326 | } 327 | pCur->ratiohistogram1 = CalcHistogram(mybins, pCur->nbins, pCur->minbin, pCur->maxbin, &rc); 328 | if (rc != SQLITE_OK) 329 | return rc; 330 | pCur->ratiohistogram2.resize(pCur->nbins); 331 | pCur->totalcount = pCur->ratiohistogram1[0].count; 332 | pCur->ratio = 0.0; 333 | 334 | if (pCur->discrcolid != "") // make two ratiohistograms for values above and below discrval 335 | { 336 | std::string s_exe("SELECT "); // get first ratiohistogram where values are above discrval 337 | s_exe += pCur->colid + " FROM " + pCur->tblname 338 | + " WHERE " + pCur->discrcolid + " >= " + pCur->discrval; 339 | mybins.clear(); 340 | mybins = GetColumns(thisdb, s_exe, &rc); 341 | if (rc != SQLITE_OK) 342 | { 343 | pCur->base.pVtab->zErrMsg = sqlite3_mprintf(sqlite3_errmsg(thisdb)); 344 | return rc; 345 | } 346 | pCur->ratiohistogram1 = CalcHistogram(mybins, pCur->nbins, pCur->minbin, pCur->maxbin, &rc); 347 | if (rc != SQLITE_OK) 348 | return rc; 349 | 350 | // get second ratiohistogram where values are below discrval 351 | s_exe = "SELECT " + pCur->colid + " FROM " + pCur->tblname 352 | + " WHERE " + pCur->discrcolid + " < " + pCur->discrval; 353 | mybins.clear(); 354 | mybins = GetColumns(thisdb, s_exe, &rc); 355 | if (rc != SQLITE_OK) 356 | { 357 | pCur->base.pVtab->zErrMsg = sqlite3_mprintf(sqlite3_errmsg(thisdb)); 358 | return rc; 359 | } 360 | pCur->ratiohistogram2 = CalcHistogram(mybins, pCur->nbins, pCur->minbin, pCur->maxbin, &rc); 361 | if (rc != SQLITE_OK) 362 | return rc; 363 | } 364 | pCur->bin = pCur->ratiohistogram1[0].binval; 365 | pCur->count1 = pCur->ratiohistogram1[0].count; 366 | pCur->count2 = pCur->ratiohistogram2[0].count; 367 | pCur->totalcount = pCur->ratiohistogram1[0].count + pCur->ratiohistogram2[0].count; 368 | if (pCur->totalcount > 0) 369 | pCur->ratio = ((double)pCur->count1) / pCur->totalcount; 370 | 371 | pCur->isDesc = 0; 372 | pCur->iRowid = 1; 373 | 374 | return rc; 375 | } 376 | 377 | 378 | /* 379 | ** SQLite will invoke this method one or more times while planning a query 380 | ** that uses the generate_ratiohisto virtual table. This routine needs to create 381 | ** a query plan for each invocation and compute an estimated cost for that 382 | ** plan. 383 | ** 384 | ** In this implementation idxNum is used to represent the 385 | ** query plan. idxStr is unused. 386 | ** 387 | ** The query plan is represented by bits in idxNum: 388 | ** 389 | ** (1) start = $value -- constraint exists 390 | ** (2) stop = $value -- constraint exists 391 | ** (4) step = $value -- constraint exists 392 | ** (8) output in descending order 393 | */ 394 | int ratiohistoBestIndex( 395 | sqlite3_vtab *tab, 396 | sqlite3_index_info *pIdxInfo 397 | ){ 398 | int i; /* Loop over constraints */ 399 | int idxNum = 0; /* The query plan bitmask */ 400 | int tblnameidx = -1; /* Index of the start= constraint, or -1 if none */ 401 | int colididx = -1; /* Index of the stop= constraint, or -1 if none */ 402 | int binsidx = -1; /* Index of the step= constraint, or -1 if none */ 403 | int minbinidx = -1; 404 | int maxbinidx = -1; 405 | int discrcolididx = -1; 406 | int discrvalidx = -1; 407 | int nArg = 0; /* Number of arguments that ratiohistoFilter() expects */ 408 | 409 | sqlite3_index_info::sqlite3_index_constraint *pConstraint; 410 | pConstraint = pIdxInfo->aConstraint; 411 | for(i=0; inConstraint; i++, pConstraint++){ 412 | if( pConstraint->usable==0 ) continue; 413 | if( pConstraint->op!=SQLITE_INDEX_CONSTRAINT_EQ ) continue; 414 | switch( pConstraint->iColumn ){ 415 | case RATIOHISTO_TBLNAME: 416 | tblnameidx = i; 417 | idxNum = RATIOHISTO_TBLNAME; 418 | break; 419 | case RATIOHISTO_COLID: 420 | colididx = i; 421 | idxNum = RATIOHISTO_COLID; 422 | break; 423 | case RATIOHISTO_NBINS: 424 | binsidx = i; 425 | idxNum = RATIOHISTO_NBINS; 426 | break; 427 | case RATIOHISTO_MINBIN: 428 | minbinidx = i; 429 | idxNum = RATIOHISTO_MINBIN; 430 | break; 431 | case RATIOHISTO_MAXBIN: 432 | maxbinidx = i; 433 | idxNum = RATIOHISTO_MAXBIN; 434 | break; 435 | case RATIOHISTO_DISCRCOLID: 436 | discrcolididx = i; 437 | idxNum = RATIOHISTO_DISCRCOLID; 438 | break; 439 | case RATIOHISTO_DISCRVAL: 440 | discrvalidx = i; 441 | idxNum = RATIOHISTO_DISCRVAL; 442 | break; 443 | } 444 | } 445 | if(tblnameidx >=0 ){ 446 | pIdxInfo->aConstraintUsage[tblnameidx].argvIndex = ++nArg; 447 | pIdxInfo->aConstraintUsage[tblnameidx].omit= 1; 448 | } 449 | if(colididx >=0 ){ 450 | pIdxInfo->aConstraintUsage[colididx].argvIndex = ++nArg; 451 | pIdxInfo->aConstraintUsage[colididx].omit = 1; 452 | } 453 | if (binsidx >= 0) { 454 | pIdxInfo->aConstraintUsage[binsidx].argvIndex = ++nArg; 455 | pIdxInfo->aConstraintUsage[binsidx].omit = 1; 456 | } 457 | if (minbinidx >= 0) { 458 | pIdxInfo->aConstraintUsage[minbinidx].argvIndex = ++nArg; 459 | pIdxInfo->aConstraintUsage[minbinidx].omit = 1; 460 | } 461 | if (maxbinidx >= 0) { 462 | pIdxInfo->aConstraintUsage[maxbinidx].argvIndex = ++nArg; 463 | pIdxInfo->aConstraintUsage[maxbinidx].omit = 1; 464 | } 465 | if (discrcolididx >= 0) { 466 | pIdxInfo->aConstraintUsage[discrcolididx].argvIndex = ++nArg; 467 | pIdxInfo->aConstraintUsage[discrcolididx].omit = 1; 468 | } 469 | if (discrvalidx >= 0) { 470 | pIdxInfo->aConstraintUsage[discrvalidx].argvIndex = ++nArg; 471 | pIdxInfo->aConstraintUsage[discrvalidx].omit = 1; 472 | } 473 | pIdxInfo->estimatedCost = 2.0; 474 | pIdxInfo->estimatedRows = 500; 475 | if( pIdxInfo->nOrderBy==1 ) 476 | { 477 | pIdxInfo->orderByConsumed = 1; 478 | } 479 | pIdxInfo->idxNum = idxNum; 480 | return SQLITE_OK; 481 | } 482 | 483 | 484 | 485 | /* 486 | ** This following structure defines all the methods for the 487 | ** generate_ratiohisto virtual table. 488 | */ 489 | sqlite3_module ratiohistoModule = { 490 | 0, /* iVersion */ 491 | 0, /* xCreate */ 492 | ratiohistoConnect, /* xConnect */ 493 | ratiohistoBestIndex, /* xBestIndex */ 494 | ratiohistoDisconnect, /* xDisconnect */ 495 | 0, /* xDestroy */ 496 | ratiohistoOpen, /* xOpen - open a cursor */ 497 | ratiohistoClose, /* xClose - close a cursor */ 498 | ratiohistoFilter, /* xFilter - configure scan constraints */ 499 | ratiohistoNext, /* xNext - advance a cursor */ 500 | ratiohistoEof, /* xEof - check for end of scan */ 501 | ratiohistoColumn, /* xColumn - read data */ 502 | ratiohistoRowid, /* xRowid - read data */ 503 | 0, /* xUpdate */ 504 | 0, /* xBegin */ 505 | 0, /* xSync */ 506 | 0, /* xCommit */ 507 | 0, /* xRollback */ 508 | 0, /* xFindMethod */ 509 | 0, /* xRename */ 510 | }; 511 | 512 | 513 | 514 | #endif /* SQLITE_OMIT_VIRTUALTABLE */ 515 | 516 | 517 | 518 | 519 | #ifdef __cplusplus 520 | } 521 | #endif 522 | -------------------------------------------------------------------------------- /scatter.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Oeffner/SQLiteHistograms/374b0645e7ac6efdf1e5f66d83102cc57052a037/scatter.jpg -------------------------------------------------------------------------------- /sqlite3ext.h: -------------------------------------------------------------------------------- 1 | /* 2 | ** 2006 June 7 3 | ** 4 | ** The author disclaims copyright to this source code. In place of 5 | ** a legal notice, here is a blessing: 6 | ** 7 | ** May you do good and not evil. 8 | ** May you find forgiveness for yourself and forgive others. 9 | ** May you share freely, never taking more than you give. 10 | ** 11 | ************************************************************************* 12 | ** This header file defines the SQLite interface for use by 13 | ** shared libraries that want to be imported as extensions into 14 | ** an SQLite instance. Shared libraries that intend to be loaded 15 | ** as extensions by SQLite should #include this file instead of 16 | ** sqlite3.h. 17 | */ 18 | #ifndef SQLITE3EXT_H 19 | #define SQLITE3EXT_H 20 | #include "sqlite3.h" 21 | 22 | /* 23 | ** The following structure holds pointers to all of the SQLite API 24 | ** routines. 25 | ** 26 | ** WARNING: In order to maintain backwards compatibility, add new 27 | ** interfaces to the end of this structure only. If you insert new 28 | ** interfaces in the middle of this structure, then older different 29 | ** versions of SQLite will not be able to load each other's shared 30 | ** libraries! 31 | */ 32 | struct sqlite3_api_routines { 33 | void * (*aggregate_context)(sqlite3_context*,int nBytes); 34 | int (*aggregate_count)(sqlite3_context*); 35 | int (*bind_blob)(sqlite3_stmt*,int,const void*,int n,void(*)(void*)); 36 | int (*bind_double)(sqlite3_stmt*,int,double); 37 | int (*bind_int)(sqlite3_stmt*,int,int); 38 | int (*bind_int64)(sqlite3_stmt*,int,sqlite_int64); 39 | int (*bind_null)(sqlite3_stmt*,int); 40 | int (*bind_parameter_count)(sqlite3_stmt*); 41 | int (*bind_parameter_index)(sqlite3_stmt*,const char*zName); 42 | const char * (*bind_parameter_name)(sqlite3_stmt*,int); 43 | int (*bind_text)(sqlite3_stmt*,int,const char*,int n,void(*)(void*)); 44 | int (*bind_text16)(sqlite3_stmt*,int,const void*,int,void(*)(void*)); 45 | int (*bind_value)(sqlite3_stmt*,int,const sqlite3_value*); 46 | int (*busy_handler)(sqlite3*,int(*)(void*,int),void*); 47 | int (*busy_timeout)(sqlite3*,int ms); 48 | int (*changes)(sqlite3*); 49 | int (*close)(sqlite3*); 50 | int (*collation_needed)(sqlite3*,void*,void(*)(void*,sqlite3*, 51 | int eTextRep,const char*)); 52 | int (*collation_needed16)(sqlite3*,void*,void(*)(void*,sqlite3*, 53 | int eTextRep,const void*)); 54 | const void * (*column_blob)(sqlite3_stmt*,int iCol); 55 | int (*column_bytes)(sqlite3_stmt*,int iCol); 56 | int (*column_bytes16)(sqlite3_stmt*,int iCol); 57 | int (*column_count)(sqlite3_stmt*pStmt); 58 | const char * (*column_database_name)(sqlite3_stmt*,int); 59 | const void * (*column_database_name16)(sqlite3_stmt*,int); 60 | const char * (*column_decltype)(sqlite3_stmt*,int i); 61 | const void * (*column_decltype16)(sqlite3_stmt*,int); 62 | double (*column_double)(sqlite3_stmt*,int iCol); 63 | int (*column_int)(sqlite3_stmt*,int iCol); 64 | sqlite_int64 (*column_int64)(sqlite3_stmt*,int iCol); 65 | const char * (*column_name)(sqlite3_stmt*,int); 66 | const void * (*column_name16)(sqlite3_stmt*,int); 67 | const char * (*column_origin_name)(sqlite3_stmt*,int); 68 | const void * (*column_origin_name16)(sqlite3_stmt*,int); 69 | const char * (*column_table_name)(sqlite3_stmt*,int); 70 | const void * (*column_table_name16)(sqlite3_stmt*,int); 71 | const unsigned char * (*column_text)(sqlite3_stmt*,int iCol); 72 | const void * (*column_text16)(sqlite3_stmt*,int iCol); 73 | int (*column_type)(sqlite3_stmt*,int iCol); 74 | sqlite3_value* (*column_value)(sqlite3_stmt*,int iCol); 75 | void * (*commit_hook)(sqlite3*,int(*)(void*),void*); 76 | int (*complete)(const char*sql); 77 | int (*complete16)(const void*sql); 78 | int (*create_collation)(sqlite3*,const char*,int,void*, 79 | int(*)(void*,int,const void*,int,const void*)); 80 | int (*create_collation16)(sqlite3*,const void*,int,void*, 81 | int(*)(void*,int,const void*,int,const void*)); 82 | int (*create_function)(sqlite3*,const char*,int,int,void*, 83 | void (*xFunc)(sqlite3_context*,int,sqlite3_value**), 84 | void (*xStep)(sqlite3_context*,int,sqlite3_value**), 85 | void (*xFinal)(sqlite3_context*)); 86 | int (*create_function16)(sqlite3*,const void*,int,int,void*, 87 | void (*xFunc)(sqlite3_context*,int,sqlite3_value**), 88 | void (*xStep)(sqlite3_context*,int,sqlite3_value**), 89 | void (*xFinal)(sqlite3_context*)); 90 | int (*create_module)(sqlite3*,const char*,const sqlite3_module*,void*); 91 | int (*data_count)(sqlite3_stmt*pStmt); 92 | sqlite3 * (*db_handle)(sqlite3_stmt*); 93 | int (*declare_vtab)(sqlite3*,const char*); 94 | int (*enable_shared_cache)(int); 95 | int (*errcode)(sqlite3*db); 96 | const char * (*errmsg)(sqlite3*); 97 | const void * (*errmsg16)(sqlite3*); 98 | int (*exec)(sqlite3*,const char*,sqlite3_callback,void*,char**); 99 | int (*expired)(sqlite3_stmt*); 100 | int (*finalize)(sqlite3_stmt*pStmt); 101 | void (*free)(void*); 102 | void (*free_table)(char**result); 103 | int (*get_autocommit)(sqlite3*); 104 | void * (*get_auxdata)(sqlite3_context*,int); 105 | int (*get_table)(sqlite3*,const char*,char***,int*,int*,char**); 106 | int (*global_recover)(void); 107 | void (*interruptx)(sqlite3*); 108 | sqlite_int64 (*last_insert_rowid)(sqlite3*); 109 | const char * (*libversion)(void); 110 | int (*libversion_number)(void); 111 | void *(*malloc)(int); 112 | char * (*mprintf)(const char*,...); 113 | int (*open)(const char*,sqlite3**); 114 | int (*open16)(const void*,sqlite3**); 115 | int (*prepare)(sqlite3*,const char*,int,sqlite3_stmt**,const char**); 116 | int (*prepare16)(sqlite3*,const void*,int,sqlite3_stmt**,const void**); 117 | void * (*profile)(sqlite3*,void(*)(void*,const char*,sqlite_uint64),void*); 118 | void (*progress_handler)(sqlite3*,int,int(*)(void*),void*); 119 | void *(*realloc)(void*,int); 120 | int (*reset)(sqlite3_stmt*pStmt); 121 | void (*result_blob)(sqlite3_context*,const void*,int,void(*)(void*)); 122 | void (*result_double)(sqlite3_context*,double); 123 | void (*result_error)(sqlite3_context*,const char*,int); 124 | void (*result_error16)(sqlite3_context*,const void*,int); 125 | void (*result_int)(sqlite3_context*,int); 126 | void (*result_int64)(sqlite3_context*,sqlite_int64); 127 | void (*result_null)(sqlite3_context*); 128 | void (*result_text)(sqlite3_context*,const char*,int,void(*)(void*)); 129 | void (*result_text16)(sqlite3_context*,const void*,int,void(*)(void*)); 130 | void (*result_text16be)(sqlite3_context*,const void*,int,void(*)(void*)); 131 | void (*result_text16le)(sqlite3_context*,const void*,int,void(*)(void*)); 132 | void (*result_value)(sqlite3_context*,sqlite3_value*); 133 | void * (*rollback_hook)(sqlite3*,void(*)(void*),void*); 134 | int (*set_authorizer)(sqlite3*,int(*)(void*,int,const char*,const char*, 135 | const char*,const char*),void*); 136 | void (*set_auxdata)(sqlite3_context*,int,void*,void (*)(void*)); 137 | char * (*snprintf)(int,char*,const char*,...); 138 | int (*step)(sqlite3_stmt*); 139 | int (*table_column_metadata)(sqlite3*,const char*,const char*,const char*, 140 | char const**,char const**,int*,int*,int*); 141 | void (*thread_cleanup)(void); 142 | int (*total_changes)(sqlite3*); 143 | void * (*trace)(sqlite3*,void(*xTrace)(void*,const char*),void*); 144 | int (*transfer_bindings)(sqlite3_stmt*,sqlite3_stmt*); 145 | void * (*update_hook)(sqlite3*,void(*)(void*,int ,char const*,char const*, 146 | sqlite_int64),void*); 147 | void * (*user_data)(sqlite3_context*); 148 | const void * (*value_blob)(sqlite3_value*); 149 | int (*value_bytes)(sqlite3_value*); 150 | int (*value_bytes16)(sqlite3_value*); 151 | double (*value_double)(sqlite3_value*); 152 | int (*value_int)(sqlite3_value*); 153 | sqlite_int64 (*value_int64)(sqlite3_value*); 154 | int (*value_numeric_type)(sqlite3_value*); 155 | const unsigned char * (*value_text)(sqlite3_value*); 156 | const void * (*value_text16)(sqlite3_value*); 157 | const void * (*value_text16be)(sqlite3_value*); 158 | const void * (*value_text16le)(sqlite3_value*); 159 | int (*value_type)(sqlite3_value*); 160 | char *(*vmprintf)(const char*,va_list); 161 | /* Added ??? */ 162 | int (*overload_function)(sqlite3*, const char *zFuncName, int nArg); 163 | /* Added by 3.3.13 */ 164 | int (*prepare_v2)(sqlite3*,const char*,int,sqlite3_stmt**,const char**); 165 | int (*prepare16_v2)(sqlite3*,const void*,int,sqlite3_stmt**,const void**); 166 | int (*clear_bindings)(sqlite3_stmt*); 167 | /* Added by 3.4.1 */ 168 | int (*create_module_v2)(sqlite3*,const char*,const sqlite3_module*,void*, 169 | void (*xDestroy)(void *)); 170 | /* Added by 3.5.0 */ 171 | int (*bind_zeroblob)(sqlite3_stmt*,int,int); 172 | int (*blob_bytes)(sqlite3_blob*); 173 | int (*blob_close)(sqlite3_blob*); 174 | int (*blob_open)(sqlite3*,const char*,const char*,const char*,sqlite3_int64, 175 | int,sqlite3_blob**); 176 | int (*blob_read)(sqlite3_blob*,void*,int,int); 177 | int (*blob_write)(sqlite3_blob*,const void*,int,int); 178 | int (*create_collation_v2)(sqlite3*,const char*,int,void*, 179 | int(*)(void*,int,const void*,int,const void*), 180 | void(*)(void*)); 181 | int (*file_control)(sqlite3*,const char*,int,void*); 182 | sqlite3_int64 (*memory_highwater)(int); 183 | sqlite3_int64 (*memory_used)(void); 184 | sqlite3_mutex *(*mutex_alloc)(int); 185 | void (*mutex_enter)(sqlite3_mutex*); 186 | void (*mutex_free)(sqlite3_mutex*); 187 | void (*mutex_leave)(sqlite3_mutex*); 188 | int (*mutex_try)(sqlite3_mutex*); 189 | int (*open_v2)(const char*,sqlite3**,int,const char*); 190 | int (*release_memory)(int); 191 | void (*result_error_nomem)(sqlite3_context*); 192 | void (*result_error_toobig)(sqlite3_context*); 193 | int (*sleep)(int); 194 | void (*soft_heap_limit)(int); 195 | sqlite3_vfs *(*vfs_find)(const char*); 196 | int (*vfs_register)(sqlite3_vfs*,int); 197 | int (*vfs_unregister)(sqlite3_vfs*); 198 | int (*xthreadsafe)(void); 199 | void (*result_zeroblob)(sqlite3_context*,int); 200 | void (*result_error_code)(sqlite3_context*,int); 201 | int (*test_control)(int, ...); 202 | void (*randomness)(int,void*); 203 | sqlite3 *(*context_db_handle)(sqlite3_context*); 204 | int (*extended_result_codes)(sqlite3*,int); 205 | int (*limit)(sqlite3*,int,int); 206 | sqlite3_stmt *(*next_stmt)(sqlite3*,sqlite3_stmt*); 207 | const char *(*sql)(sqlite3_stmt*); 208 | int (*status)(int,int*,int*,int); 209 | int (*backup_finish)(sqlite3_backup*); 210 | sqlite3_backup *(*backup_init)(sqlite3*,const char*,sqlite3*,const char*); 211 | int (*backup_pagecount)(sqlite3_backup*); 212 | int (*backup_remaining)(sqlite3_backup*); 213 | int (*backup_step)(sqlite3_backup*,int); 214 | const char *(*compileoption_get)(int); 215 | int (*compileoption_used)(const char*); 216 | int (*create_function_v2)(sqlite3*,const char*,int,int,void*, 217 | void (*xFunc)(sqlite3_context*,int,sqlite3_value**), 218 | void (*xStep)(sqlite3_context*,int,sqlite3_value**), 219 | void (*xFinal)(sqlite3_context*), 220 | void(*xDestroy)(void*)); 221 | int (*db_config)(sqlite3*,int,...); 222 | sqlite3_mutex *(*db_mutex)(sqlite3*); 223 | int (*db_status)(sqlite3*,int,int*,int*,int); 224 | int (*extended_errcode)(sqlite3*); 225 | void (*log)(int,const char*,...); 226 | sqlite3_int64 (*soft_heap_limit64)(sqlite3_int64); 227 | const char *(*sourceid)(void); 228 | int (*stmt_status)(sqlite3_stmt*,int,int); 229 | int (*strnicmp)(const char*,const char*,int); 230 | int (*unlock_notify)(sqlite3*,void(*)(void**,int),void*); 231 | int (*wal_autocheckpoint)(sqlite3*,int); 232 | int (*wal_checkpoint)(sqlite3*,const char*); 233 | void *(*wal_hook)(sqlite3*,int(*)(void*,sqlite3*,const char*,int),void*); 234 | int (*blob_reopen)(sqlite3_blob*,sqlite3_int64); 235 | int (*vtab_config)(sqlite3*,int op,...); 236 | int (*vtab_on_conflict)(sqlite3*); 237 | /* Version 3.7.16 and later */ 238 | int (*close_v2)(sqlite3*); 239 | const char *(*db_filename)(sqlite3*,const char*); 240 | int (*db_readonly)(sqlite3*,const char*); 241 | int (*db_release_memory)(sqlite3*); 242 | const char *(*errstr)(int); 243 | int (*stmt_busy)(sqlite3_stmt*); 244 | int (*stmt_readonly)(sqlite3_stmt*); 245 | int (*stricmp)(const char*,const char*); 246 | int (*uri_boolean)(const char*,const char*,int); 247 | sqlite3_int64 (*uri_int64)(const char*,const char*,sqlite3_int64); 248 | const char *(*uri_parameter)(const char*,const char*); 249 | char *(*vsnprintf)(int,char*,const char*,va_list); 250 | int (*wal_checkpoint_v2)(sqlite3*,const char*,int,int*,int*); 251 | /* Version 3.8.7 and later */ 252 | int (*auto_extension)(void(*)(void)); 253 | int (*bind_blob64)(sqlite3_stmt*,int,const void*,sqlite3_uint64, 254 | void(*)(void*)); 255 | int (*bind_text64)(sqlite3_stmt*,int,const char*,sqlite3_uint64, 256 | void(*)(void*),unsigned char); 257 | int (*cancel_auto_extension)(void(*)(void)); 258 | int (*load_extension)(sqlite3*,const char*,const char*,char**); 259 | void *(*malloc64)(sqlite3_uint64); 260 | sqlite3_uint64 (*msize)(void*); 261 | void *(*realloc64)(void*,sqlite3_uint64); 262 | void (*reset_auto_extension)(void); 263 | void (*result_blob64)(sqlite3_context*,const void*,sqlite3_uint64, 264 | void(*)(void*)); 265 | void (*result_text64)(sqlite3_context*,const char*,sqlite3_uint64, 266 | void(*)(void*), unsigned char); 267 | int (*strglob)(const char*,const char*); 268 | /* Version 3.8.11 and later */ 269 | sqlite3_value *(*value_dup)(const sqlite3_value*); 270 | void (*value_free)(sqlite3_value*); 271 | int (*result_zeroblob64)(sqlite3_context*,sqlite3_uint64); 272 | int (*bind_zeroblob64)(sqlite3_stmt*, int, sqlite3_uint64); 273 | /* Version 3.9.0 and later */ 274 | unsigned int (*value_subtype)(sqlite3_value*); 275 | void (*result_subtype)(sqlite3_context*,unsigned int); 276 | /* Version 3.10.0 and later */ 277 | int (*status64)(int,sqlite3_int64*,sqlite3_int64*,int); 278 | int (*strlike)(const char*,const char*,unsigned int); 279 | int (*db_cacheflush)(sqlite3*); 280 | /* Version 3.12.0 and later */ 281 | int (*system_errno)(sqlite3*); 282 | /* Version 3.14.0 and later */ 283 | int (*trace_v2)(sqlite3*,unsigned,int(*)(unsigned,void*,void*,void*),void*); 284 | char *(*expanded_sql)(sqlite3_stmt*); 285 | /* Version 3.18.0 and later */ 286 | void (*set_last_insert_rowid)(sqlite3*,sqlite3_int64); 287 | /* Version 3.20.0 and later */ 288 | int (*prepare_v3)(sqlite3*,const char*,int,unsigned int, 289 | sqlite3_stmt**,const char**); 290 | int (*prepare16_v3)(sqlite3*,const void*,int,unsigned int, 291 | sqlite3_stmt**,const void**); 292 | int (*bind_pointer)(sqlite3_stmt*,int,void*,const char*,void(*)(void*)); 293 | void (*result_pointer)(sqlite3_context*,void*,const char*,void(*)(void*)); 294 | void *(*value_pointer)(sqlite3_value*,const char*); 295 | }; 296 | 297 | /* 298 | ** This is the function signature used for all extension entry points. It 299 | ** is also defined in the file "loadext.c". 300 | */ 301 | typedef int (*sqlite3_loadext_entry)( 302 | sqlite3 *db, /* Handle to the database. */ 303 | char **pzErrMsg, /* Used to set error string on failure. */ 304 | const sqlite3_api_routines *pThunk /* Extension API function pointers. */ 305 | ); 306 | 307 | /* 308 | ** The following macros redefine the API routines so that they are 309 | ** redirected through the global sqlite3_api structure. 310 | ** 311 | ** This header file is also used by the loadext.c source file 312 | ** (part of the main SQLite library - not an extension) so that 313 | ** it can get access to the sqlite3_api_routines structure 314 | ** definition. But the main library does not want to redefine 315 | ** the API. So the redefinition macros are only valid if the 316 | ** SQLITE_CORE macros is undefined. 317 | */ 318 | #if !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION) 319 | #define sqlite3_aggregate_context sqlite3_api->aggregate_context 320 | #ifndef SQLITE_OMIT_DEPRECATED 321 | #define sqlite3_aggregate_count sqlite3_api->aggregate_count 322 | #endif 323 | #define sqlite3_bind_blob sqlite3_api->bind_blob 324 | #define sqlite3_bind_double sqlite3_api->bind_double 325 | #define sqlite3_bind_int sqlite3_api->bind_int 326 | #define sqlite3_bind_int64 sqlite3_api->bind_int64 327 | #define sqlite3_bind_null sqlite3_api->bind_null 328 | #define sqlite3_bind_parameter_count sqlite3_api->bind_parameter_count 329 | #define sqlite3_bind_parameter_index sqlite3_api->bind_parameter_index 330 | #define sqlite3_bind_parameter_name sqlite3_api->bind_parameter_name 331 | #define sqlite3_bind_text sqlite3_api->bind_text 332 | #define sqlite3_bind_text16 sqlite3_api->bind_text16 333 | #define sqlite3_bind_value sqlite3_api->bind_value 334 | #define sqlite3_busy_handler sqlite3_api->busy_handler 335 | #define sqlite3_busy_timeout sqlite3_api->busy_timeout 336 | #define sqlite3_changes sqlite3_api->changes 337 | #define sqlite3_close sqlite3_api->close 338 | #define sqlite3_collation_needed sqlite3_api->collation_needed 339 | #define sqlite3_collation_needed16 sqlite3_api->collation_needed16 340 | #define sqlite3_column_blob sqlite3_api->column_blob 341 | #define sqlite3_column_bytes sqlite3_api->column_bytes 342 | #define sqlite3_column_bytes16 sqlite3_api->column_bytes16 343 | #define sqlite3_column_count sqlite3_api->column_count 344 | #define sqlite3_column_database_name sqlite3_api->column_database_name 345 | #define sqlite3_column_database_name16 sqlite3_api->column_database_name16 346 | #define sqlite3_column_decltype sqlite3_api->column_decltype 347 | #define sqlite3_column_decltype16 sqlite3_api->column_decltype16 348 | #define sqlite3_column_double sqlite3_api->column_double 349 | #define sqlite3_column_int sqlite3_api->column_int 350 | #define sqlite3_column_int64 sqlite3_api->column_int64 351 | #define sqlite3_column_name sqlite3_api->column_name 352 | #define sqlite3_column_name16 sqlite3_api->column_name16 353 | #define sqlite3_column_origin_name sqlite3_api->column_origin_name 354 | #define sqlite3_column_origin_name16 sqlite3_api->column_origin_name16 355 | #define sqlite3_column_table_name sqlite3_api->column_table_name 356 | #define sqlite3_column_table_name16 sqlite3_api->column_table_name16 357 | #define sqlite3_column_text sqlite3_api->column_text 358 | #define sqlite3_column_text16 sqlite3_api->column_text16 359 | #define sqlite3_column_type sqlite3_api->column_type 360 | #define sqlite3_column_value sqlite3_api->column_value 361 | #define sqlite3_commit_hook sqlite3_api->commit_hook 362 | #define sqlite3_complete sqlite3_api->complete 363 | #define sqlite3_complete16 sqlite3_api->complete16 364 | #define sqlite3_create_collation sqlite3_api->create_collation 365 | #define sqlite3_create_collation16 sqlite3_api->create_collation16 366 | #define sqlite3_create_function sqlite3_api->create_function 367 | #define sqlite3_create_function16 sqlite3_api->create_function16 368 | #define sqlite3_create_module sqlite3_api->create_module 369 | #define sqlite3_create_module_v2 sqlite3_api->create_module_v2 370 | #define sqlite3_data_count sqlite3_api->data_count 371 | #define sqlite3_db_handle sqlite3_api->db_handle 372 | #define sqlite3_declare_vtab sqlite3_api->declare_vtab 373 | #define sqlite3_enable_shared_cache sqlite3_api->enable_shared_cache 374 | #define sqlite3_errcode sqlite3_api->errcode 375 | #define sqlite3_errmsg sqlite3_api->errmsg 376 | #define sqlite3_errmsg16 sqlite3_api->errmsg16 377 | #define sqlite3_exec sqlite3_api->exec 378 | #ifndef SQLITE_OMIT_DEPRECATED 379 | #define sqlite3_expired sqlite3_api->expired 380 | #endif 381 | #define sqlite3_finalize sqlite3_api->finalize 382 | #define sqlite3_free sqlite3_api->free 383 | #define sqlite3_free_table sqlite3_api->free_table 384 | #define sqlite3_get_autocommit sqlite3_api->get_autocommit 385 | #define sqlite3_get_auxdata sqlite3_api->get_auxdata 386 | #define sqlite3_get_table sqlite3_api->get_table 387 | #ifndef SQLITE_OMIT_DEPRECATED 388 | #define sqlite3_global_recover sqlite3_api->global_recover 389 | #endif 390 | #define sqlite3_interrupt sqlite3_api->interruptx 391 | #define sqlite3_last_insert_rowid sqlite3_api->last_insert_rowid 392 | #define sqlite3_libversion sqlite3_api->libversion 393 | #define sqlite3_libversion_number sqlite3_api->libversion_number 394 | #define sqlite3_malloc sqlite3_api->malloc 395 | #define sqlite3_mprintf sqlite3_api->mprintf 396 | #define sqlite3_open sqlite3_api->open 397 | #define sqlite3_open16 sqlite3_api->open16 398 | #define sqlite3_prepare sqlite3_api->prepare 399 | #define sqlite3_prepare16 sqlite3_api->prepare16 400 | #define sqlite3_prepare_v2 sqlite3_api->prepare_v2 401 | #define sqlite3_prepare16_v2 sqlite3_api->prepare16_v2 402 | #define sqlite3_profile sqlite3_api->profile 403 | #define sqlite3_progress_handler sqlite3_api->progress_handler 404 | #define sqlite3_realloc sqlite3_api->realloc 405 | #define sqlite3_reset sqlite3_api->reset 406 | #define sqlite3_result_blob sqlite3_api->result_blob 407 | #define sqlite3_result_double sqlite3_api->result_double 408 | #define sqlite3_result_error sqlite3_api->result_error 409 | #define sqlite3_result_error16 sqlite3_api->result_error16 410 | #define sqlite3_result_int sqlite3_api->result_int 411 | #define sqlite3_result_int64 sqlite3_api->result_int64 412 | #define sqlite3_result_null sqlite3_api->result_null 413 | #define sqlite3_result_text sqlite3_api->result_text 414 | #define sqlite3_result_text16 sqlite3_api->result_text16 415 | #define sqlite3_result_text16be sqlite3_api->result_text16be 416 | #define sqlite3_result_text16le sqlite3_api->result_text16le 417 | #define sqlite3_result_value sqlite3_api->result_value 418 | #define sqlite3_rollback_hook sqlite3_api->rollback_hook 419 | #define sqlite3_set_authorizer sqlite3_api->set_authorizer 420 | #define sqlite3_set_auxdata sqlite3_api->set_auxdata 421 | #define sqlite3_snprintf sqlite3_api->snprintf 422 | #define sqlite3_step sqlite3_api->step 423 | #define sqlite3_table_column_metadata sqlite3_api->table_column_metadata 424 | #define sqlite3_thread_cleanup sqlite3_api->thread_cleanup 425 | #define sqlite3_total_changes sqlite3_api->total_changes 426 | #define sqlite3_trace sqlite3_api->trace 427 | #ifndef SQLITE_OMIT_DEPRECATED 428 | #define sqlite3_transfer_bindings sqlite3_api->transfer_bindings 429 | #endif 430 | #define sqlite3_update_hook sqlite3_api->update_hook 431 | #define sqlite3_user_data sqlite3_api->user_data 432 | #define sqlite3_value_blob sqlite3_api->value_blob 433 | #define sqlite3_value_bytes sqlite3_api->value_bytes 434 | #define sqlite3_value_bytes16 sqlite3_api->value_bytes16 435 | #define sqlite3_value_double sqlite3_api->value_double 436 | #define sqlite3_value_int sqlite3_api->value_int 437 | #define sqlite3_value_int64 sqlite3_api->value_int64 438 | #define sqlite3_value_numeric_type sqlite3_api->value_numeric_type 439 | #define sqlite3_value_text sqlite3_api->value_text 440 | #define sqlite3_value_text16 sqlite3_api->value_text16 441 | #define sqlite3_value_text16be sqlite3_api->value_text16be 442 | #define sqlite3_value_text16le sqlite3_api->value_text16le 443 | #define sqlite3_value_type sqlite3_api->value_type 444 | #define sqlite3_vmprintf sqlite3_api->vmprintf 445 | #define sqlite3_vsnprintf sqlite3_api->vsnprintf 446 | #define sqlite3_overload_function sqlite3_api->overload_function 447 | #define sqlite3_prepare_v2 sqlite3_api->prepare_v2 448 | #define sqlite3_prepare16_v2 sqlite3_api->prepare16_v2 449 | #define sqlite3_clear_bindings sqlite3_api->clear_bindings 450 | #define sqlite3_bind_zeroblob sqlite3_api->bind_zeroblob 451 | #define sqlite3_blob_bytes sqlite3_api->blob_bytes 452 | #define sqlite3_blob_close sqlite3_api->blob_close 453 | #define sqlite3_blob_open sqlite3_api->blob_open 454 | #define sqlite3_blob_read sqlite3_api->blob_read 455 | #define sqlite3_blob_write sqlite3_api->blob_write 456 | #define sqlite3_create_collation_v2 sqlite3_api->create_collation_v2 457 | #define sqlite3_file_control sqlite3_api->file_control 458 | #define sqlite3_memory_highwater sqlite3_api->memory_highwater 459 | #define sqlite3_memory_used sqlite3_api->memory_used 460 | #define sqlite3_mutex_alloc sqlite3_api->mutex_alloc 461 | #define sqlite3_mutex_enter sqlite3_api->mutex_enter 462 | #define sqlite3_mutex_free sqlite3_api->mutex_free 463 | #define sqlite3_mutex_leave sqlite3_api->mutex_leave 464 | #define sqlite3_mutex_try sqlite3_api->mutex_try 465 | #define sqlite3_open_v2 sqlite3_api->open_v2 466 | #define sqlite3_release_memory sqlite3_api->release_memory 467 | #define sqlite3_result_error_nomem sqlite3_api->result_error_nomem 468 | #define sqlite3_result_error_toobig sqlite3_api->result_error_toobig 469 | #define sqlite3_sleep sqlite3_api->sleep 470 | #define sqlite3_soft_heap_limit sqlite3_api->soft_heap_limit 471 | #define sqlite3_vfs_find sqlite3_api->vfs_find 472 | #define sqlite3_vfs_register sqlite3_api->vfs_register 473 | #define sqlite3_vfs_unregister sqlite3_api->vfs_unregister 474 | #define sqlite3_threadsafe sqlite3_api->xthreadsafe 475 | #define sqlite3_result_zeroblob sqlite3_api->result_zeroblob 476 | #define sqlite3_result_error_code sqlite3_api->result_error_code 477 | #define sqlite3_test_control sqlite3_api->test_control 478 | #define sqlite3_randomness sqlite3_api->randomness 479 | #define sqlite3_context_db_handle sqlite3_api->context_db_handle 480 | #define sqlite3_extended_result_codes sqlite3_api->extended_result_codes 481 | #define sqlite3_limit sqlite3_api->limit 482 | #define sqlite3_next_stmt sqlite3_api->next_stmt 483 | #define sqlite3_sql sqlite3_api->sql 484 | #define sqlite3_status sqlite3_api->status 485 | #define sqlite3_backup_finish sqlite3_api->backup_finish 486 | #define sqlite3_backup_init sqlite3_api->backup_init 487 | #define sqlite3_backup_pagecount sqlite3_api->backup_pagecount 488 | #define sqlite3_backup_remaining sqlite3_api->backup_remaining 489 | #define sqlite3_backup_step sqlite3_api->backup_step 490 | #define sqlite3_compileoption_get sqlite3_api->compileoption_get 491 | #define sqlite3_compileoption_used sqlite3_api->compileoption_used 492 | #define sqlite3_create_function_v2 sqlite3_api->create_function_v2 493 | #define sqlite3_db_config sqlite3_api->db_config 494 | #define sqlite3_db_mutex sqlite3_api->db_mutex 495 | #define sqlite3_db_status sqlite3_api->db_status 496 | #define sqlite3_extended_errcode sqlite3_api->extended_errcode 497 | #define sqlite3_log sqlite3_api->log 498 | #define sqlite3_soft_heap_limit64 sqlite3_api->soft_heap_limit64 499 | #define sqlite3_sourceid sqlite3_api->sourceid 500 | #define sqlite3_stmt_status sqlite3_api->stmt_status 501 | #define sqlite3_strnicmp sqlite3_api->strnicmp 502 | #define sqlite3_unlock_notify sqlite3_api->unlock_notify 503 | #define sqlite3_wal_autocheckpoint sqlite3_api->wal_autocheckpoint 504 | #define sqlite3_wal_checkpoint sqlite3_api->wal_checkpoint 505 | #define sqlite3_wal_hook sqlite3_api->wal_hook 506 | #define sqlite3_blob_reopen sqlite3_api->blob_reopen 507 | #define sqlite3_vtab_config sqlite3_api->vtab_config 508 | #define sqlite3_vtab_on_conflict sqlite3_api->vtab_on_conflict 509 | /* Version 3.7.16 and later */ 510 | #define sqlite3_close_v2 sqlite3_api->close_v2 511 | #define sqlite3_db_filename sqlite3_api->db_filename 512 | #define sqlite3_db_readonly sqlite3_api->db_readonly 513 | #define sqlite3_db_release_memory sqlite3_api->db_release_memory 514 | #define sqlite3_errstr sqlite3_api->errstr 515 | #define sqlite3_stmt_busy sqlite3_api->stmt_busy 516 | #define sqlite3_stmt_readonly sqlite3_api->stmt_readonly 517 | #define sqlite3_stricmp sqlite3_api->stricmp 518 | #define sqlite3_uri_boolean sqlite3_api->uri_boolean 519 | #define sqlite3_uri_int64 sqlite3_api->uri_int64 520 | #define sqlite3_uri_parameter sqlite3_api->uri_parameter 521 | #define sqlite3_uri_vsnprintf sqlite3_api->vsnprintf 522 | #define sqlite3_wal_checkpoint_v2 sqlite3_api->wal_checkpoint_v2 523 | /* Version 3.8.7 and later */ 524 | #define sqlite3_auto_extension sqlite3_api->auto_extension 525 | #define sqlite3_bind_blob64 sqlite3_api->bind_blob64 526 | #define sqlite3_bind_text64 sqlite3_api->bind_text64 527 | #define sqlite3_cancel_auto_extension sqlite3_api->cancel_auto_extension 528 | #define sqlite3_load_extension sqlite3_api->load_extension 529 | #define sqlite3_malloc64 sqlite3_api->malloc64 530 | #define sqlite3_msize sqlite3_api->msize 531 | #define sqlite3_realloc64 sqlite3_api->realloc64 532 | #define sqlite3_reset_auto_extension sqlite3_api->reset_auto_extension 533 | #define sqlite3_result_blob64 sqlite3_api->result_blob64 534 | #define sqlite3_result_text64 sqlite3_api->result_text64 535 | #define sqlite3_strglob sqlite3_api->strglob 536 | /* Version 3.8.11 and later */ 537 | #define sqlite3_value_dup sqlite3_api->value_dup 538 | #define sqlite3_value_free sqlite3_api->value_free 539 | #define sqlite3_result_zeroblob64 sqlite3_api->result_zeroblob64 540 | #define sqlite3_bind_zeroblob64 sqlite3_api->bind_zeroblob64 541 | /* Version 3.9.0 and later */ 542 | #define sqlite3_value_subtype sqlite3_api->value_subtype 543 | #define sqlite3_result_subtype sqlite3_api->result_subtype 544 | /* Version 3.10.0 and later */ 545 | #define sqlite3_status64 sqlite3_api->status64 546 | #define sqlite3_strlike sqlite3_api->strlike 547 | #define sqlite3_db_cacheflush sqlite3_api->db_cacheflush 548 | /* Version 3.12.0 and later */ 549 | #define sqlite3_system_errno sqlite3_api->system_errno 550 | /* Version 3.14.0 and later */ 551 | #define sqlite3_trace_v2 sqlite3_api->trace_v2 552 | #define sqlite3_expanded_sql sqlite3_api->expanded_sql 553 | /* Version 3.18.0 and later */ 554 | #define sqlite3_set_last_insert_rowid sqlite3_api->set_last_insert_rowid 555 | /* Version 3.20.0 and later */ 556 | #define sqlite3_prepare_v3 sqlite3_api->prepare_v3 557 | #define sqlite3_prepare16_v3 sqlite3_api->prepare16_v3 558 | #define sqlite3_bind_pointer sqlite3_api->bind_pointer 559 | #define sqlite3_result_pointer sqlite3_api->result_pointer 560 | #define sqlite3_value_pointer sqlite3_api->value_pointer 561 | #endif /* !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION) */ 562 | 563 | #if !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION) 564 | /* This case when the file really is being compiled as a loadable 565 | ** extension */ 566 | # define SQLITE_EXTENSION_INIT1 const sqlite3_api_routines *sqlite3_api=0; 567 | # define SQLITE_EXTENSION_INIT2(v) sqlite3_api=v; 568 | # define SQLITE_EXTENSION_INIT3 \ 569 | extern const sqlite3_api_routines *sqlite3_api; 570 | #else 571 | /* This case when the file is being statically linked into the 572 | ** application */ 573 | # define SQLITE_EXTENSION_INIT1 /*no-op*/ 574 | # define SQLITE_EXTENSION_INIT2(v) (void)v; /* unused parameter */ 575 | # define SQLITE_EXTENSION_INIT3 /*no-op*/ 576 | #endif 577 | 578 | #endif /* SQLITE3EXT_H */ 579 | --------------------------------------------------------------------------------