├── .travis.yml ├── CHANGES ├── HowToUseLittletable.txt ├── LICENSE ├── MANIFEST.in ├── README.md ├── _config.yml ├── docs └── accessor_pattern_diagrams.md ├── examples ├── Fixed Width data demo.ipynb ├── Littletable PPL demo.ipynb ├── PEPs data demo.ipynb ├── area_codes.py ├── csv_import_examples.py ├── dune_casts.py ├── excel_data_types.py ├── explore_unicode.py ├── fixed_width_demo.py ├── food_data.py ├── food_data_set │ ├── FOOD-DATA-GROUP1.csv │ ├── FOOD-DATA-GROUP2.csv │ ├── FOOD-DATA-GROUP3.csv │ ├── FOOD-DATA-GROUP4.csv │ ├── FOOD-DATA-GROUP5.csv │ ├── README.md │ ├── combined_food_data.csv │ └── food_data_merge.py ├── full_text_search.py ├── littletable_demo.py ├── nfkc_normalization.py ├── peps.json.zip ├── peps.py ├── pivot_demo.py ├── present_groupby_example.py ├── pyscript │ └── matplotlib.html ├── sliced_indexing.py ├── star_trek_tos.py ├── star_trek_tos_eps.csv ├── table_to_dataframe.py ├── time_conversions.py ├── time_zone_db.py ├── unicode_15.1.0.txt.zip ├── unicode_320.txt.zip ├── us_ppl.zip └── zen_of_python_pep20.py ├── how_to_use_littletable.md ├── littletable.py ├── pyproject.toml ├── requirements.dev ├── test ├── abc.csv ├── abc.csv.gz ├── abc.csv.tar.gz ├── abc.csv.xz ├── abc.csv.zip ├── abc.json.gz ├── abc.tar.gz ├── abc.tsv ├── abc.xlsx ├── abc.zip ├── csv_import_http_server.py ├── data_types.xlsx └── star_trek_tos_eps.jsonl ├── tox.ini ├── unit_tests.py └── update_littletable_timestamp.py /.travis.yml: -------------------------------------------------------------------------------- 1 | sudo: false 2 | 3 | language: python 4 | 5 | matrix: 6 | include: 7 | - python: 3.7 8 | - python: 3.8 9 | - python: 3.9 10 | - python: 3.10-dev 11 | dist: xenial 12 | sudo: true 13 | fast_finish: true 14 | 15 | before_install: 16 | - export verint="`python -c "import platform; a,b,c=map(int,platform.python_version().split('.')); print(((a*100+b)*100)+c)"`" 17 | - if [ $verint -ge 30600 ]; then pip install rich; fi 18 | 19 | script: 20 | - python littletable.py 21 | - python unit_tests.py 22 | - PYTHONPATH=. python examples/pivot_demo.py 23 | - PYTHONPATH=. python examples/fixed_width_demo.py 24 | -------------------------------------------------------------------------------- /HowToUseLittletable.txt: -------------------------------------------------------------------------------- 1 | littletable - Usage Notes 2 | ========================= 3 | 4 | (This file is superseded by the Markdown version, https://github.com/ptmcg/littletable/blob/master/how_to_use_littletable.md - refer to that file for more current 5 | information on `littletable`.) 6 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2010 Paul T. McGuire 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining 4 | a copy of this software and associated documentation files (the 5 | "Software"), to deal in the Software without restriction, including 6 | without limitation the rights to use, copy, modify, merge, publish, 7 | distribute, sublicense, and/or sell copies of the Software, and to 8 | permit persons to whom the Software is furnished to do so, subject to 9 | the following conditions: 10 | 11 | The above copyright notice and this permission notice shall be 12 | included in all copies or substantial portions of the Software. 13 | 14 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 15 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 16 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 17 | IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY 18 | CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, 19 | TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 20 | SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 21 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include littletable.py 2 | include littletable_demo.py 3 | include pivot_demo.py 4 | include setup.py 5 | include unit_tests.py 6 | include CHANGES 7 | include LICENSE 8 | include README.md 9 | include htmldoc\*.* 10 | include HowToUseLittletable.txt -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # littletable - a Python module to give ORM-like access to a collection of objects 2 | [![Build Status](https://travis-ci.org/ptmcg/littletable.svg?branch=master)](https://travis-ci.org/ptmcg/littletable) [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/ptmcg/littletable/master) 3 | 4 | - [Introduction](#introduction) 5 | - [Optional dependencies](#optional-dependencies) 6 | - [Importing data from CSV files](#importing-data-from-csv-files) 7 | - [Tabular output](#tabular-output) 8 | - [For More Info](#for-more-info) 9 | - [Sample Demo](#sample-demo) 10 | 11 | Introduction 12 | ------------ 13 | The `littletable` module provides a low-overhead, schema-less, in-memory database access to a collection 14 | of user objects. `littletable` Tables will accept Python `dict`s or any user-defined object type, including: 15 | 16 | - `namedtuples` and `typing.NamedTuples` 17 | - `dataclasses` 18 | - `types.SimpleNamespaces` 19 | - `attrs` classes 20 | - `PyDantic` data models 21 | - `traitlets` 22 | 23 | `littletable` infers the Table's "columns" from those objects' `__dict__`, `__slots__`, or `_fields` mappings to access 24 | object attributes. 25 | 26 | If populated with Python `dict`s, they get stored as `SimpleNamespace`s. 27 | 28 | In addition to basic ORM-style insert/remove/query/delete access to the contents of a `Table`, `littletable` offers: 29 | * simple indexing for improved retrieval performance, and optional enforcing key uniqueness 30 | * access to objects using indexed attributes 31 | * direct import/export to CSV, TSV, JSON, and Excel .xlsx files 32 | * clean tabular output for data presentation 33 | * simplified joins using `"+"` operator syntax between annotated `Table`s 34 | * the result of any query or join is a new first-class `littletable` `Table` 35 | * simple full-text search against multi-word text attributes 36 | * access like a standard Python list to the records in a `Table`, including indexing/slicing, `iter`, `zip`, `len`, `groupby`, etc. 37 | * access like a standard Python `dict` to attributes with a unique index, or like a standard Python `defaultdict(list)` to attributes with a non-unique index 38 | 39 | `littletable` `Table`s do not require an upfront schema definition, but simply work off of the attributes in 40 | the stored values, and those referenced in any query parameters. 41 | 42 | 43 | Optional dependencies 44 | --------------------- 45 | The base `littletable` code has no dependencies outside of the Python stdlib. However, some operations 46 | require additional package installs: 47 | 48 | | operation | additional install required | 49 | |-----------------------------|--------------------------------------------------------------------| 50 | | `Table.present` | `rich` | 51 | | `Table.excel_import/export` | `openpyxl` (plus `defusedxml` or `lxml`, `defusedxml` recommended) | 52 | | `Table.as_dataframe` | `pandas` | 53 | 54 | 55 | Importing data from CSV files 56 | ----------------------------- 57 | You can easily import a CSV file into a `Table` using `Table.csv_import()`: 58 | 59 | ```python 60 | import littletable as lt 61 | t = lt.Table().csv_import("my_data.csv") 62 | # or 63 | t = lt.csv_import("my_data.csv") 64 | ``` 65 | 66 | In place of a local file name, you can also specify an HTTP url: 67 | 68 | ```python 69 | url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/iris.csv" 70 | names = ["sepal-length", "sepal-width", "petal-length", "petal-width", "class"] 71 | iris_table = Table('iris').csv_import(url, fieldnames=names) 72 | ``` 73 | 74 | You can also directly import CSV data as a string: 75 | 76 | ```python 77 | catalog = Table("catalog") 78 | 79 | catalog_data = """\ 80 | sku,description,unitofmeas,unitprice 81 | BRDSD-001,Bird seed,LB,3 82 | BBS-001,Steel BB's,LB,5 83 | MGNT-001,Magnet,EA,8""" 84 | 85 | catalog.csv_import(catalog_data, transforms={'unitprice': int}) 86 | ``` 87 | 88 | Data can also be directly imported from compressed .zip, .gz, and .xz files. 89 | 90 | Files containing JSON-formatted records can be similarly imported using `json_import()`. 91 | 92 | 93 | Tabular output 94 | -------------- 95 | To produce a nice tabular output for a table, you can use the embedded support for 96 | the [rich](https://github.com/willmcgugan/rich) module, `as_html()` in [Jupyter Notebook](https://jupyter.org/), 97 | or the [tabulate](https://github.com/astanin/python-tabulate) module: 98 | 99 | Using `table.present()` (implemented using `rich`; `present()` accepts `rich` `Table` keyword args): 100 | 101 | ```python 102 | table(title_str).present(fields=["col1", "col2", "col3"]) 103 | or 104 | table.select("col1 col2 col3")(title_str).present(caption="caption text", 105 | caption_justify="right") 106 | ``` 107 | 108 | Using `Jupyter Notebook`: 109 | 110 | ```python 111 | from IPython.display import HTML, display 112 | display(HTML(table.as_html())) 113 | ``` 114 | 115 | Using `tabulate`: 116 | 117 | ```python 118 | from tabulate import tabulate 119 | print(tabulate((vars(rec) for rec in table), headers="keys")) 120 | ``` 121 | 122 | For More Info 123 | ------------- 124 | Extended "getting started" notes at [how_to_use_littletable.md](https://github.com/ptmcg/littletable/blob/master/how_to_use_littletable.md). 125 | 126 | Sample Demo 127 | ----------- 128 | Here is a simple `littletable` data storage/retrieval example: 129 | 130 | ```python 131 | from littletable import Table 132 | 133 | customers = Table('customers') 134 | customers.create_index("id", unique=True) 135 | customers.csv_import("""\ 136 | id,name 137 | 0010,George Jetson 138 | 0020,Wile E. Coyote 139 | 0030,Jonny Quest 140 | """) 141 | 142 | catalog = Table('catalog') 143 | catalog.create_index("sku", unique=True) 144 | catalog.insert({"sku": "ANVIL-001", "descr": "1000lb anvil", "unitofmeas": "EA","unitprice": 100}) 145 | catalog.insert({"sku": "BRDSD-001", "descr": "Bird seed", "unitofmeas": "LB","unitprice": 3}) 146 | catalog.insert({"sku": "MAGNT-001", "descr": "Magnet", "unitofmeas": "EA","unitprice": 8}) 147 | catalog.insert({"sku": "MAGLS-001", "descr": "Magnifying glass", "unitofmeas": "EA","unitprice": 12}) 148 | 149 | wishitems = Table('wishitems') 150 | wishitems.create_index("custid") 151 | wishitems.create_index("sku") 152 | 153 | # easy to import CSV data from a string or file 154 | wishitems.csv_import("""\ 155 | custid,sku 156 | 0020,ANVIL-001 157 | 0020,BRDSD-001 158 | 0020,MAGNT-001 159 | 0030,MAGNT-001 160 | 0030,MAGLS-001 161 | """) 162 | 163 | # print a particular customer name 164 | # (unique indexes will return a single item; non-unique 165 | # indexes will return a new Table of all matching items) 166 | print(customers.by.id["0030"].name) 167 | 168 | # see all customer names 169 | for name in customers.all.name: 170 | print(name) 171 | 172 | # print all items sold by the pound 173 | for item in catalog.where(unitofmeas="LB"): 174 | print(item.sku, item.descr) 175 | 176 | # print all items that cost more than 10 177 | for item in catalog.where(lambda o: o.unitprice > 10): 178 | print(item.sku, item.descr, item.unitprice) 179 | 180 | # join tables to create queryable wishlists collection 181 | wishlists = customers.join_on("id") + wishitems.join_on("custid") + catalog.join_on("sku") 182 | 183 | # print all wishlist items with price > 10 (can use Table.gt comparator instead of lambda) 184 | bigticketitems = wishlists().where(unitprice=Table.gt(10)) 185 | for item in bigticketitems: 186 | print(item) 187 | 188 | # list all wishlist items in descending order by price 189 | for item in wishlists().sort("unitprice desc"): 190 | print(item) 191 | 192 | # print output as a nicely-formatted table 193 | wishlists().sort("unitprice desc")("Wishlists").present() 194 | 195 | # print output as an HTML table 196 | print(wishlists().sort("unitprice desc")("Wishlists").as_html()) 197 | 198 | # print output as a Markdown table 199 | print(wishlists().sort("unitprice desc")("Wishlists").as_markdown()) 200 | 201 | ``` 202 | -------------------------------------------------------------------------------- /_config.yml: -------------------------------------------------------------------------------- 1 | theme: jekyll-theme-slate -------------------------------------------------------------------------------- /docs/accessor_pattern_diagrams.md: -------------------------------------------------------------------------------- 1 | Accessor classes 2 | 3 | ```mermaid 4 | classDiagram 5 | 6 | class Object { 7 | attribute 8 | access()* 9 | } 10 | 11 | class Accessor { 12 | object 13 | __getattr__(attrname) 14 | } 15 | 16 | class ObjectAttributeWrapper { 17 | object 18 | object_attribute 19 | action()* 20 | } 21 | 22 | Accessor --> Object 23 | Object ..> Accessor: access (creates) 24 | Accessor ..> ObjectAttributeWrapper: __getattr__ (creates) 25 | ObjectAttributeWrapper --> Object 26 | 27 | 28 | ``` 29 | 30 | ```mermaid 31 | sequenceDiagram 32 | actor client as client
33 | participant tbl as catalog:
Table 34 | participant acc as accessor:
IndexAccessor 35 | participant w as wrapper:
IndexWrapper 36 | 37 | note over client,w: create index on "sku" 38 | 39 | client ->> tbl: create_index("sku", unique=True) 40 | note over tbl: build "sku" index dict
mapping obj.sku -> obj
for all obj in tbl 41 | tbl ->> tbl: index =
{getattr(obj, "sku"): obj
for obj in self.obj_list} 42 | tbl ->> tbl: self.indexes["sku"] = index 43 | 44 | 45 | note over client,w: catalog.by.sku["001"] 46 | 47 | note right of client: get "by" accessor 48 | client ->> tbl: by 49 | tbl ->> acc: create(catalog) 50 | tbl -->> client: accessor 51 | 52 | note right of client: get "sku" index wrapper 53 | client ->> acc: __getattr__("sku") 54 | acc ->> w: create(idx=catalog.indexes["sku"]) 55 | w ->> w: self.index = idx 56 | acc -->> client: wrapper 57 | 58 | note right of client: get "001" element in index 59 | client ->> w: __getitem__("001") 60 | w ->> w: self.index["001"] 61 | w -->> client: ["001"] object or KeyError 62 | 63 | ``` -------------------------------------------------------------------------------- /examples/Fixed Width data demo.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": { 7 | "collapsed": true 8 | }, 9 | "outputs": [], 10 | "source": [ 11 | "import littletable as lt\n", 12 | "from IPython.display import display, HTML\n", 13 | "_HTML = lambda tbl, fields='*': display(HTML(tbl.as_html(fields)))" 14 | ] 15 | }, 16 | { 17 | "cell_type": "code", 18 | "execution_count": 2, 19 | "metadata": { 20 | "collapsed": true 21 | }, 22 | "outputs": [], 23 | "source": [ 24 | "data = \"\"\"\\\n", 25 | "0010GEORGE JETSON 12345 SPACESHIP ST HOUSTON TX 4.9\n", 26 | "0020WILE E COYOTE 312 ACME BLVD TUCSON AZ 7.3\n", 27 | "0030FRED FLINTSTONE 246 GRANITE LANE BEDROCK CA 2.6\n", 28 | "0040JONNY QUEST 31416 SCIENCE AVE PALO ALTO CA 8.1\n", 29 | "\"\"\"\n", 30 | "\n", 31 | "columns = [\n", 32 | " (\"id_no\", 0, ),\n", 33 | " (\"name\", 4, ),\n", 34 | " (\"address\", 21, ),\n", 35 | " (\"city\", 42, ),\n", 36 | " (\"state\", 56, 58, ),\n", 37 | " (\"tech_skill_score\", 59, None, float),\n", 38 | " ]\n", 39 | "\n", 40 | "reader = lt.FixedWidthReader(columns, data)" 41 | ] 42 | }, 43 | { 44 | "cell_type": "code", 45 | "execution_count": 3, 46 | "metadata": { 47 | "collapsed": false 48 | }, 49 | "outputs": [], 50 | "source": [ 51 | "characters_table = lt.Table().insert_many(reader)" 52 | ] 53 | }, 54 | { 55 | "cell_type": "code", 56 | "execution_count": 4, 57 | "metadata": { 58 | "collapsed": false 59 | }, 60 | "outputs": [ 61 | { 62 | "data": { 63 | "text/html": [ 64 | "\n", 65 | "\n", 66 | "\n", 67 | "\n", 68 | "\n", 69 | "\n", 70 | "
id_no
name
address
city
state
tech_skill_score
0010
GEORGE JETSON
12345 SPACESHIP ST
HOUSTON
TX
4.9
0020
WILE E COYOTE
312 ACME BLVD
TUCSON
AZ
7.3
0030
FRED FLINTSTONE
246 GRANITE LANE
BEDROCK
CA
2.6
0040
JONNY QUEST
31416 SCIENCE AVE
PALO ALTO
CA
8.1
" 71 | ], 72 | "text/plain": [ 73 | "" 74 | ] 75 | }, 76 | "metadata": {}, 77 | "output_type": "display_data" 78 | } 79 | ], 80 | "source": [ 81 | "_HTML(characters_table, \"id_no name address city state *\")" 82 | ] 83 | }, 84 | { 85 | "cell_type": "code", 86 | "execution_count": null, 87 | "metadata": { 88 | "collapsed": true 89 | }, 90 | "outputs": [], 91 | "source": [] 92 | } 93 | ], 94 | "metadata": { 95 | "kernelspec": { 96 | "display_name": "Python 3", 97 | "language": "python", 98 | "name": "python3" 99 | }, 100 | "language_info": { 101 | "codemirror_mode": { 102 | "name": "ipython", 103 | "version": 3 104 | }, 105 | "file_extension": ".py", 106 | "mimetype": "text/x-python", 107 | "name": "python", 108 | "nbconvert_exporter": "python", 109 | "pygments_lexer": "ipython3", 110 | "version": "3.5.2" 111 | } 112 | }, 113 | "nbformat": 4, 114 | "nbformat_minor": 2 115 | } 116 | -------------------------------------------------------------------------------- /examples/area_codes.py: -------------------------------------------------------------------------------- 1 | """ 2 | What do telephone cords have to do with area codes? 3 | 4 | Telephone cords always had 4 wires: red, green, black, and white. But for 5 | most house and office phones back in the day, all you needed to connect were 6 | the red and green ones. 7 | 8 | In analog times, "dialing" each number involved opening and closing the circuit 9 | N number of times (to dial a "0" required 10 circuit open/closes). Rotary phones 10 | did this for you, but you could do it yourself by tapping the wire on the 11 | mounting screw (I think this was actually used as an escape in a 70's era spy 12 | movie or TV show). 13 | 14 | Phone exchanges did the same thing with physical switches to open and close 15 | circuits. When area codes were added, their 3-digit pattern had to be detectable 16 | as different from regular phone numbers. The middle number had to be a 0 or 1, 17 | and the first and third numbers had to be digits 2-9. Since the switches were 18 | physical, area codes were assigned by number and activity of phones, so that the 19 | most phone-active areas used the lowest possible numbers for the fewest number 20 | of circuit open/close sequences. So who got 212? New York City (Manhattan). 21 | 213? Los Angeles. 312? Chicago. 214? Dallas. 412? Pittsburgh. 22 | 23 | 1947 map of initial area codes: 24 | https://upload.wikimedia.org/wikipedia/en/thumb/7/72/North_American_Numbering_Plan_NPA_map_BTM_1947.png/1400px-North_American_Numbering_Plan_NPA_map_BTM_1947.png 25 | 26 | Now that the phone system is completely digital, these area code constraints 27 | have largely been lifted (still no area codes start with 0 or 1, and the x00 and 28 | x11 codes are reserved for special uses). 29 | """ 30 | import littletable as lt 31 | from typing import Union 32 | 33 | try: 34 | area_codes = lt.csv_import( 35 | "area_codes.csv", 36 | transforms = {"latitude": float, "longitude": float}, 37 | ) 38 | except FileNotFoundError: 39 | area_codes = lt.csv_import( 40 | "https://raw.githubusercontent.com/ravisorg/Area-Code-Geolocation-Database/refs/heads/master/us-area-code-cities.csv", 41 | fieldnames = "area_code,city,state,country,latitude,longitude".split(","), 42 | transforms = {"latitude": float, "longitude": float}, 43 | ) 44 | area_codes.csv_export("area_codes.csv") 45 | 46 | # collapse duplicate entries for the same area code 47 | area_codes = area_codes.unique("area_code") 48 | print(len(area_codes)) 49 | 50 | 51 | class AreaCode: 52 | def __init__(self, digits: Union[tuple[int, int, int], str]): 53 | if isinstance(digits, str): 54 | digits = tuple(int(d) for d in digits) 55 | self.digits: tuple[int, int, int] = digits 56 | 57 | @property 58 | def is_analog(self): 59 | return self.digits[1] in (0, 1) and not set(self.digits[::2]) & {0, 1} 60 | 61 | def __len__(self): 62 | return sum(self.digits) + (10 * (self.digits.count(0))) 63 | 64 | def __str__(self): 65 | return "".join(str(d) for d in self.digits) 66 | 67 | 68 | area_codes.compute_field("area_code", lambda rec: AreaCode(rec.area_code)) 69 | 70 | # analog area codes must start and end with digit 2-9, and have middle digit of 0 or 1 71 | analog_codes: lt.Table = area_codes.where(lambda rec: rec.area_code.is_analog) 72 | print(len(analog_codes)) 73 | 74 | # find those area codes that require the fewest clicks 75 | analog_codes.sort(key=lambda rec: len(rec.area_code)) 76 | 77 | analog_codes.compute_field("clicks", lambda rec: len(rec.area_code)) 78 | analog_codes.where(clicks=lt.Table.le(14)).present(width=120) 79 | -------------------------------------------------------------------------------- /examples/csv_import_examples.py: -------------------------------------------------------------------------------- 1 | # 2 | # csv_import_examples.py 3 | # 4 | # Some examples of importing data from CSV sources. 5 | # 6 | # CSV data may be imported from: 7 | # - a Python string 8 | # - a local .csv file 9 | # - a .zip file containing a .csv file 10 | # - a .gz file containing a .csv file 11 | # - a .xz or .lzma file containing a .csv file 12 | # - an http or https URL to a page containing raw CSV data 13 | # 14 | # littletable tables can also import data from: 15 | # - TSV files (tab-separated values) 16 | # - JSON files 17 | # . a single JSON list of objects, or 18 | # . multiple JSON objects 19 | # - Excel spreadsheets 20 | # 21 | 22 | import littletable as lt 23 | 24 | # read from CSV data in a string 25 | catalog_data = """\ 26 | sku,descr,unitofmeas,unitprice 27 | BRDSD-001,Bird seed,LB,3 28 | BBS-001,Steel BB's,LB,5 29 | MGNT-001,Magnet,EA,8 30 | MAGLS-001,Magnifying glass,EA,12 31 | ANVIL-001,1000lb anvil,EA,100 32 | ROPE-001,1 in. heavy rope,100FT,10 33 | ROBOT-001,Domestic robot,EA,5000""" 34 | catalog = lt.csv_import(catalog_data, transforms={"unitprice": int}) 35 | 36 | # read from a CSV file 37 | data = lt.csv_import("my_data.csv") 38 | 39 | # read from a ZIP file containing a single CSV file 40 | data = lt.csv_import("my_data.csv.zip") 41 | 42 | # read from a GZIP file containing a single CSV file 43 | data = lt.csv_import("my_data.csv.gz") 44 | 45 | # read from CSV data in a remote URL 46 | url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/iris.csv" 47 | names = ["sepal-length", "sepal-width", "petal-length", "petal-width", "class"] 48 | iris_transforms = dict.fromkeys( 49 | ["petal-length", "petal-width", "sepal-length", "sepal-width"], float 50 | ) 51 | iris_table = lt.csv_import( 52 | url, fieldnames=names, transforms=iris_transforms 53 | )("iris") 54 | print(iris_table.info()) 55 | 56 | # accumulate data from multiple CSVs into a single table, and then resave as a single CSV 57 | from pathlib import Path 58 | csv_files = Path(".").glob("*.csv") 59 | 60 | all_data = lt.Table() 61 | for csv in csv_files: 62 | # repeated calls to csv_import appends data to an existing table 63 | all_data.csv_import(csv) 64 | 65 | all_data.csv_export("accumulated_data.csv") 66 | -------------------------------------------------------------------------------- /examples/dune_casts.py: -------------------------------------------------------------------------------- 1 | # 2 | # dune_casts.py 3 | # 4 | # Use join() to create a consolidated table of the casts of Dune, 5 | # from the 1984 movie, the 2000 mini-series, and the 2021 movie. 6 | # 7 | import littletable as lt 8 | from operator import attrgetter 9 | 10 | dune_casts_csv = """\ 11 | character,actor,year 12 | Lady Jessica Atreides,Francesca Annis,1984 13 | Piter De Vries,Brad Dourif,1984 14 | Padishah Emperor Shaddam IV,José Ferrer,1984 15 | Shadout Mapes,Linda Hunt,1984 16 | Thufir Hawat,Freddie Jones,1984 17 | Duncan Idaho,Richard Jordan,1984 18 | Paul Atreides,Kyle MacLachlan,1984 19 | Princess Irulan,Virginia Madsen,1984 20 | Reverend Mother Ramallo,Silvana Mangano,1984 21 | Stilgar,Everett McGill,1984 22 | Baron Vladimir Harkonnen,Kenneth McMillan,1984 23 | Reverend Mother Gaius Helen Mohiam,Siân Phillips,1984 24 | Duke Leto Atreides,Jürgen Prochnow,1984 25 | Glossu Beast Rabban,Paul L. Smith,1984 26 | Gurney Halleck,Patrick Stewart,1984 27 | Feyd-Rautha Harkonnen,Sting,1984 28 | Dr. Yueh,Dean Stockwell,1984 29 | Dr. Liet-Kynes,Max von Sydow,1984 30 | Alia Atreides,Alicia Witt,1984 31 | Chani,Sean Young,1984 32 | Otheym,Honorato Magaloni,1984 33 | Jamis,Judd Omen,1984 34 | Harah,Molly Wryn,1984 35 | character,actor 36 | Duke Leto Atreides,William Hurt,2000 37 | Paul Atreides,Alec Newman,2000 38 | Lady Jessica Atreides,Saskia Reeves,2000 39 | Gurney Halleck,P.H. Moriarty,2000 40 | Baron Vladimir Harkonnen,Ian McNeice,2000 41 | Feyd-Rautha Harkonnen,Matt Keeslar,2000 42 | Glossu Beast Rabban,László I. Kish,2000 43 | Padishah Emperor Shaddam IV,Giancarlo Giannini,2000 44 | Princess Irulan,Julie Cox,2000 45 | Stilgar,Uwe Ochsenknecht,2000 46 | Reverend Mother Gaius Helen Mohiam,Zuzana Geislerová,2000 47 | Alia Atreides,Laura Burton,2000 48 | Duncan Idaho,James Watson,2000 49 | Chani,Barbora Kodetová,2000 50 | Otheym,Jakob Schwarz,2000 51 | Dr. Liet-Kynes,Karel Dobrý,2000 52 | Thufir Hawat,Jan Vlasák,2000 53 | Dr. Yueh,Robert Russell,2000 54 | Piter De Vries,Jan Unger,2000 55 | Jamis,Christopher Lee Brown,2000 56 | Shadout Mapes,Jaroslava Siktancova,2000 57 | Reverend Mother Ramallo,Drahomíra Fialková,2000 58 | Young Mother Ramallo,Petra Spindlerová,2000 59 | Harah,...,2000 60 | Duncan Idaho,Jason Momoa,2021 61 | Paul Atreides,Timothée Chalamet,2021 62 | Chani,Zendaya,2021 63 | Lady Jessica Atreides,Rebecca Ferguson,2021 64 | Gurney Halleck,Josh Brolin,2021 65 | Glossu Beast Rabban,Dave Bautista,2021 66 | Duke Leto Atreides,Oscar Isaac,2021 67 | Stilgar,Javier Bardem,2021 68 | Piter De Vries,David Dastmalchian,2021 69 | Baron Vladimir Harkonnen,Stellan Skarsgård,2021 70 | Reverend Mother Gaius Helen Mohiam,Charlotte Rampling,2021 71 | Thufir Hawat,Stephen McKinley Henderson,2021 72 | Dr. Yueh,Chen Chang,2021 73 | Dr. Liet-Kynes,Sharon Duncan-Brewster,2021 74 | Jamis,Babs Olusanmokun,2021 75 | Harah,Gloria Obianyo,2021 76 | Padishah Emperor Shaddam IV,Christopher Walken,2021 77 | Shadout Mapes,Golda Rosheuvel,2021 78 | Princess Irulan,Florence Pugh,2021 79 | Reverend Mother Ramallo,Giusi Merli,2021 80 | Feyd-Rautha Harkonnen,Austin Butler,2021 81 | Alia Atreides,Anya Taylor-Joy,2021 82 | Otheym,...,2021 83 | """ 84 | 85 | dune_casts = lt.Table("dune_1984").csv_import(dune_casts_csv).create_index("character") 86 | dune_1984 = dune_casts.where(year="1984").compute_field("actor_1984", attrgetter("actor")) 87 | dune_2000 = dune_casts.where(year="2000").compute_field("actor_2000", attrgetter("actor")) 88 | dune_2021 = dune_casts.where(year="2021").compute_field("actor_2021", attrgetter("actor")) 89 | 90 | join = dune_1984.join_on("character") + dune_2000 + dune_2021 91 | dune_combined = join()("Dune Casts (combined)") 92 | dune_combined.present( 93 | fields=[ 94 | "character", 95 | ("actor_1984", {"header": "Actor (1984)"}), 96 | ("actor_2000", {"header": "Actor (2000)"}), 97 | ("actor_2021", {"header": "Actor (2021)"}), 98 | ] 99 | ) 100 | 101 | print() 102 | print(dune_combined.select("character actor_1984 actor_2000 actor_2021").json_export()) 103 | print(len(dune_combined)) -------------------------------------------------------------------------------- /examples/excel_data_types.py: -------------------------------------------------------------------------------- 1 | # 2 | # excel_data_types.py 3 | # 4 | # Demonstrate data type conversions done automatically when importing from Excel 5 | # 6 | import littletable as lt 7 | 8 | xl = lt.Table().excel_import("../test/data_types.xlsx") 9 | 10 | xl.present() 11 | for row in xl: 12 | print(row.name, repr(row.value), type(row.value), row.type) 13 | -------------------------------------------------------------------------------- /examples/explore_unicode.py: -------------------------------------------------------------------------------- 1 | # explore_unicode.py 2 | # 3 | # Read and process the standard Unicode table (from http://www.unicode.org/L2/L1999/UnicodeData.html) 4 | # 5 | # Field Title Normative/Informative Desc 6 | # ────────────────────────────────────────────────────────────────────────────────────────────────────── 7 | # 0 Code value normative Code value in 4-digit 8 | # hexadecimal format. 9 | # 1 Character name normative These names match exactly the 10 | # names published in Chapter 7 of 11 | # the Unicode Standard, Version 12 | # 2.0, except for the two 13 | # additional characters. 14 | # 2 General category normative / informative This is a useful breakdown into 15 | # various "character types" which 16 | # can be used as a default 17 | # categorization in 18 | # implementations. See below for 19 | # a brief explanation. 20 | # 3 Canonical combining classes normative The classes used for the 21 | # Canonical Ordering Algorithm in 22 | # the Unicode Standard. These 23 | # classes are also printed in 24 | # Chapter 4 of the Unicode 25 | # Standard. 26 | # 4 Bidirectional category normative See the list below for an 27 | # explanation of the 28 | # abbreviations used in this 29 | # field. These are the categories 30 | # required by the Bidirectional 31 | # Behavior Algorithm in the 32 | # Unicode Standard. These 33 | # categories are summarized in 34 | # Chapter 3 of the Unicode 35 | # Standard. 36 | # 5 Character decomposition mapping normative In the Unicode Standard, not 37 | # all of the mappings are full 38 | # (maximal) decompositions. 39 | # Recursive application of 40 | # look-up for decompositions 41 | # will, in all cases, lead to a 42 | # maximal decomposition. The 43 | # decomposition mappings match 44 | # exactly the decomposition 45 | # mappings published with the 46 | # character names in the Unicode 47 | # Standard. 48 | # 6 Decimal digit value normative This is a numeric field. If the 49 | # character has the decimal digit 50 | # property, as specified in 51 | # Chapter 4 of the Unicode 52 | # Standard, the value of that 53 | # digit is represented with an 54 | # integer value in this field 55 | # 7 Digit value normative This is a numeric field. If the 56 | # character represents a digit, 57 | # not necessarily a decimal 58 | # digit, the value is here. This 59 | # covers digits which do not form 60 | # decimal radix forms, such as 61 | # the compatibility superscript 62 | # digits 63 | # 8 Numeric value normative This is a numeric field. If the 64 | # character has the numeric 65 | # property, as specified in 66 | # Chapter 4 of the Unicode 67 | # Standard, the value of that 68 | # character is represented with 69 | # an integer or rational number 70 | # in this field. This includes 71 | # fractions as, e.g., "1/5" for 72 | # U+2155 VULGAR FRACTION ONE 73 | # FIFTH Also included are 74 | # numerical values for 75 | # compatibility characters such 76 | # as circled numbers. 77 | # 9 Mirrored normative If the character has been 78 | # identified as a "mirrored" 79 | # character in bidirectional 80 | # text, this field has the value 81 | # "Y"; otherwise "N". The list of 82 | # mirrored characters is also 83 | # printed in Chapter 4 of the 84 | # Unicode Standard. 85 | # 10 Unicode 1.0 Name informative This is the old name as 86 | # published in Unicode 1.0. This 87 | # name is only provided when it 88 | # is significantly different from 89 | # the Unicode 3.0 name for the 90 | # character. 91 | # 11 10646 comment field informative This is the ISO 10646 comment 92 | # field. It is in parantheses in 93 | # the 10646 names list. 94 | # 12 Uppercase mapping informative Upper case equivalent mapping. 95 | # If a character is part of an 96 | # alphabet with case 97 | # distinctions, and has an upper 98 | # case equivalent, then the upper 99 | # case equivalent is in this 100 | # field. See the explanation 101 | # below on case distinctions. 102 | # These mappings are always 103 | # one-to-one, not one-to-many or 104 | # many-to-one. This field is 105 | # informative. 106 | # 13 Lowercase mapping informative Similar to Uppercase mapping 107 | # 14 Titlecase mapping informative Similar to Uppercase mapping 108 | # """ 109 | # 110 | import re 111 | import littletable as lt 112 | 113 | 114 | fieldnames = [ 115 | "code_value_hex", 116 | "name", 117 | "category", 118 | "combining_classes", 119 | "bidirectional_category", 120 | "char_decomposition_mapping", 121 | "decimal_digit_value", 122 | "digit_value", 123 | "numeric_value", 124 | "mirrored", 125 | "unicode_1_name", 126 | "iso10646_comment", 127 | "uppercase_hex", 128 | "lowercase_hex", 129 | "titlecase_hex", 130 | ] 131 | unicode_url = "https://www.unicode.org/Public/15.1.0/ucd/UnicodeData.txt" 132 | unicode_file = "unicode_15.1.0.txt.zip" 133 | unicode = lt.Table().csv_import( 134 | # unicode_url, 135 | unicode_file, 136 | delimiter=";", 137 | transforms={ 138 | "decimal_digit_value": int, 139 | "digit_value": int, 140 | "numeric_value": int, 141 | }, 142 | fieldnames=fieldnames, 143 | ) 144 | 145 | # remove surrogates 146 | unicode.delete(category="Cs") 147 | 148 | unicode.compute_field("code_value", lambda r: int(r.code_value_hex, 16)) 149 | unicode.compute_field("uppercase_int", lambda r: int(r.uppercase_hex, 16)) 150 | unicode.compute_field("lowercase_int", lambda r: int(r.lowercase_hex, 16)) 151 | unicode.compute_field("titlecase_int", lambda r: int(r.titlecase_hex, 16)) 152 | unicode.compute_field("character", lambda r: chr(r.code_value)) 153 | unicode.compute_field("uppercase", lambda r: chr(r.uppercase_int)) 154 | unicode.compute_field("lowercase", lambda r: chr(r.lowercase_int)) 155 | unicode.compute_field("titlecase", lambda r: chr(r.titlecase_int)) 156 | unicode.compute_field("is_identifier", lambda r: r.character.isidentifier()) 157 | 158 | unicode.create_index("code_value_hex", unique=True) 159 | unicode.create_index("code_value", unique=True) 160 | 161 | # 162 | # Explore some interesting groups of symbols in the Unicode set 163 | # 164 | def present_table( 165 | title: str, source_table: lt.Table = unicode 166 | ) -> lt.Table: 167 | """ 168 | Function to search for Unicode characters that match a starting string, and 169 | presents a table showing name, character, and decimal code value 170 | """ 171 | tbl = source_table.select("name character code_value code_value_hex") 172 | tbl.present( 173 | caption="Total {} symbols".format(len(tbl)), 174 | caption_justify="left", 175 | ) 176 | return tbl 177 | 178 | def present_symbol_group( 179 | start_str: str, title: str, source_table: lt.Table = unicode 180 | ) -> lt.Table: 181 | """ 182 | Function to search for Unicode characters that match a starting string, and 183 | presents a table showing name, character, and decimal code value 184 | """ 185 | tbl = source_table.where(name=lt.Table.startswith(start_str))(title) 186 | return present_table(title, tbl) 187 | 188 | def present_symbol_search_result( 189 | query: str, title: str, source_table: lt.Table = unicode 190 | ) -> lt.Table: 191 | """ 192 | Function to search for Unicode characters that match a search query, and 193 | presents a table showing name, character, and decimal code value 194 | """ 195 | search_result = source_table.search.name_words(query,as_table=True)(title) 196 | return present_table(title, search_result) 197 | 198 | 199 | # display the characters of the I Ching 200 | i_ching = present_symbol_group("HEXAGRAM FOR", "I Ching") 201 | 202 | # display the characters of the Tai Xuan Jing 203 | tai_xuan_jing = present_symbol_group("TETRAGRAM FOR", "Tai Xuan Jing") 204 | 205 | # display all the Roman numerals 206 | numerics = unicode.where(numeric_value=lt.Table.ne(None)).orderby("numeric_value") 207 | roman_numerals = present_symbol_group("ROMAN NUMERAL", "Roman Numerals", numerics) 208 | 209 | # display all Braille characters 210 | braille = present_symbol_group("BRAILLE PATTERN", "Braille") 211 | 212 | # display all Box Drawing characters 213 | box_drawing = present_symbol_group( 214 | "BOX DRAWINGS", "Box Drawing", unicode.where(code_value=lt.Table.lt(10000)) 215 | ) 216 | 217 | # clock faces 218 | clock_faces = present_symbol_group("CLOCK FACE", "Clock Faces") 219 | 220 | # die faces 221 | die_faces = present_symbol_group("DIE FACE", "Die Faces") 222 | 223 | # create search index for random search for words in code point names or comments 224 | unicode.create_search_index("name_words", using="name unicode_1_name iso10646_comment") 225 | 226 | dots = present_symbol_search_result("DOT", "Dots") 227 | 228 | arrows = present_symbol_search_result("ARROW", "Arrows") 229 | 230 | faces = present_symbol_search_result("FACE", "Faces") 231 | 232 | # chess pieces 233 | chess_pieces = present_symbol_search_result( 234 | r"white black ++chess --rotated --neutral --turned", 235 | "Chess Pieces", 236 | ) 237 | -------------------------------------------------------------------------------- /examples/fixed_width_demo.py: -------------------------------------------------------------------------------- 1 | import littletable as lt 2 | 3 | data = """\ 4 | 0010GEORGE JETSON 12345 SPACESHIP ST HOUSTON TX 4.9 5 | 0020WILE E COYOTE 312 ACME BLVD TUCSON AZ 7.3 6 | 0030FRED FLINTSTONE 246 GRANITE LANE BEDROCK CA 2.6 7 | 0040JONNY QUEST 31416 SCIENCE AVE PALO ALTO CA 8.1 8 | """ 9 | 10 | columns = [ 11 | ("id_no", 0, ), 12 | ("name", 4, ), 13 | ("address", 21, ), 14 | ("city", 42, ), 15 | ("state", 56, 58, ), 16 | ("tech_skill_score", 59, None, float), 17 | ] 18 | 19 | characters_table = lt.Table().insert_many( 20 | lt.FixedWidthReader(columns, data) 21 | ) 22 | 23 | print(len(characters_table)) 24 | print(characters_table[0]) 25 | print(max(characters_table, key=lambda rec: rec.tech_skill_score)) -------------------------------------------------------------------------------- /examples/food_data.py: -------------------------------------------------------------------------------- 1 | # 2 | # food_data.py 3 | # 4 | # Work with food data to do some Table functions. 5 | # (Using present() requires installation of the rich module.) 6 | # 7 | from pathlib import Path 8 | 9 | import littletable as lt 10 | 11 | data_dir = Path("food_data_set") 12 | food_data: lt.Table = lt.csv_import( 13 | data_dir / "combined_food_data.csv", 14 | transforms={"*": lt.Table.convert_numeric} 15 | )("Food Data") 16 | 17 | # create some useful indexes 18 | food_data.create_index("id", unique=True) 19 | food_data.create_search_index("food") 20 | 21 | # find foods that have "cheese" in the name, but not "pizza" or "bread" 22 | search_query = "cheese --pizza --bread" 23 | food_data.search.food(search_query).orderby("id").select("id food caloric_value").present() 24 | 25 | # 10 foods with the highest caloric value 26 | food_data.orderby("caloric_value desc")[:10].select("id food caloric_value").present() 27 | -------------------------------------------------------------------------------- /examples/food_data_set/FOOD-DATA-GROUP4.csv: -------------------------------------------------------------------------------- 1 | ,Unnamed: 0,food,Caloric Value,Fat,Saturated Fats,Monounsaturated Fats,Polyunsaturated Fats,Carbohydrates,Sugars,Protein,Dietary Fiber,Cholesterol,Sodium,Water,Vitamin A,Vitamin B1,Vitamin B11,Vitamin B12,Vitamin B2,Vitamin B3,Vitamin B5,Vitamin B6,Vitamin C,Vitamin D,Vitamin E,Vitamin K,Calcium,Copper,Iron,Magnesium,Manganese,Phosphorus,Potassium,Selenium,Zinc,Nutrition Density 2 | 0,0,chocolate pudding fat free,105,0.3,0,0,0,23.6,17.8,2.2,0.3,1.1,0.1,86.1,0,0.021,0.060,0.069,0.046,0.022,0.2,0.084,0,0,0,0.098,44.1,0.035,1.9,17,0.040,61,235,0.052,0.3,72.4 3 | 1,1,tapioca pudding,143,4.3,1.1,2.8,0.088,23.9,16.4,2.1,0,1.1,0.2,79,0,0.025,0.003,0.021,0.1,0.031,0.3,0.052,0.3,0,0.2,0.010,78.1,0.026,0.1,6.6,0.096,66,101.2,0,0.2,108.8 4 | 2,2,tapioca pudding fat free,105,0.4,0.1,0.080,0.067,23.9,15.9,1.6,0,1.1,0.2,85.4,0,0.044,0.096,0.012,0.043,0.1,0.056,0.039,0.3,0,0.072,0,58.2,0.004,0.1,5.6,0.023,73.9,78.4,0,0.2,84.5 5 | 3,3,rice pudding,122,2.4,1.4,0.6,0.1,20.8,13.1,3.6,0.3,13.6,0.1,84.9,0.043,0.002,0.098,0.089,0.098,0.2,0.2,0.4,0.074,1.5,0,0.011,0.063,107.4,0.014,0.1,9,0.1,92.7,141.3,0.083,27.329 6 | 4,4,corn pudding,328,12.6,6.3,3.9,1.4,42.4,16.5,11.1,3,180,0.7,180.8,0.2,0.1,0.2,0.074,0.026,0.4,2.6,1.3,0.3,9.3,0.048,0.7,0.066,97.5,0.1,1.3,37.5,0.2,225,440,0.069,69.795 7 | 5,5,chocolate pudding,213,6.9,1.9,4.1,0.2,34.5,25.8,3.1,0,1.5,0.2,104.2,0.045,0.003,0.061,0.044,0.1,0.2,0.4,0.062,0.5,0,0.5,0.066,76.5,0.1,1.9,27,0.2,84,276,0,0.5,123.424 8 | 6,6,lemon pudding,173,4.4,2.6,1.3,0.2,30.2,0,4.1,0,16.5,0.4,109.8,0.026,0.092,0.071,0.028,0.2,0.1,0.4,0.083,1.2,0,0,0,148.5,0.070,0.058,16.5,0.007,307.5,190.5,0.041,0.5,188.516 9 | 7,7,vanilla pudding,195,5.7,1.5,3.4,0.1,33.9,25.5,2.2,0,1.5,0.3,107.5,0.018,0.012,0.084,0.092,0.1,0.047,0.2,0.048,0.3,0,0.5,0.034,73.5,0.080,0.1,6,0.031,61.5,97.5,0,0.2,115.796 10 | 8,8,profeel proteiinirahka valio,140,0.4,0.2,16.8,16.8,19.3,0.2,175,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,194.7 11 | 9,9,cottage cheese creamed,20,0.9,0.3,0.2,0.084,0.7,0.5,2.2,0,3.4,0.095,16,0.022,0.089,0.039,0.060,0.060,0.059,0.1,0.036,0,0.085,0.082,0,16.6,0.019,0.071,1.6,0.009,31.8,20.8,0.045,0.018,20.576 12 | 10,10,zaziki milfina,51,4.4,1.6,1.2,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,4.4 13 | 11,11,cottage cheese low fat,163,2.3,1.5,0.7,0.041,6.1,6.1,28,0,9,0.9,186.4,0.021,0.081,0.005,0.072,0.4,0.3,0.5,0.2,0,0,0.035,0.024,137.9,0.005,0.3,11.3,0.041,302.8,194.4,0.055,0.9,174.683 14 | 12,12,cottage cheese nonfat,104,0.4,0.2,0.1,0.066,9.7,2.7,15,0,10.2,0.5,117.5,0.093,0.063,0.013,0.006,0.3,0.2,0.6,0.039,0,0,0.096,0,124.7,0.071,0.2,16,0.063,275.5,198.7,0.063,0.7,150.072 15 | 13,13,vanilla yogurt low fat,208,3.1,2,0.8,0.036,33.8,33.8,12.1,0,12.3,0.2,193.6,0.051,0.1,0.054,0.010,0.5,0.3,1.4,0.1,2,0,0.029,0.024,419,0.012,0.2,39.2,0.023,330.8,536.6,0.049,2,470.265 16 | 14,14,classique yogurt plain 2 liberte,30,1.2,0.7,0.7,0,1.7,1.7,3.3,0,0.017,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,6.2 17 | 15,15,yoplait light strawberry yogurt yoplait,79,0,14.1,8.8,4.4,0.074,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.061 18 | 16,16,vanilla yogurt,145,2.1,1.3,0.5,0.069,23.5,23.5,8.4,0,8.5,0.1,134.3,0.005,0.091,0.068,0.026,0.3,0.2,0.9,0.094,1.4,0,0.003,0.037,290.7,0.089,0.1,27.2,0.087,229.5,372.3,0.080,1.4,326.243 19 | 17,17,chocolate yogurt,168,0,0,0,0,35.3,22.5,5.3,1.8,1.5,0.2,107.4,0,0.083,0.019,0.042,0.3,0.3,0,0.050,0,0,0,0,132,0.3,0.6,60,0,249,508.5,0.069,1.7,175.0 20 | 18,18,yogurt parfait,125,1.5,0.8,0.5,0.2,23.6,17.4,5,1.6,4.5,0.048,117.7,0.073,0.1,0.018,0.100,0.3,1.1,0,0.3,21.3,0.042,0.4,0.085,156.5,0.025,0.7,25.3,0,138.6,281.6,0.088,1.3,210.272 21 | 19,19,yogurt,92,4.9,3.1,1.3,0.1,7,7,5.2,0,19.5,0.005,131.9,0.026,0.017,0.060,0.081,0.2,0.1,0.6,0.026,0.8,0.076,0.035,0.020,181.5,0.049,0.001,18,0.084,142.5,232.5,0.075,0.9,199.51 22 | 20,20,chocolate frozen yogurt,221,6.3,4,1.7,0.2,37.6,33.5,5.2,4,22.6,0.1,123.9,0.036,0.080,0.008,0.088,0.3,0.2,0,0.021,0,0.057,0.2,0.071,174,0.028,0.8,43.5,0,154.9,407.2,0.059,0.5,227.929 23 | 21,21,frozen yogurt,221,6.3,4,1.7,0.2,37.6,34.7,5.2,0,22.6,0.1,123.9,0.013,0.062,0.095,0.050,0.3,0.1,0,0.072,1.2,0.010,0.2,0.054,174,0.074,0.8,17.4,0,154.9,271.4,0.005,0.5,225.163 24 | 22,22,yogurt low fat,154,3.8,2.5,1,0.1,17.2,17.2,12.9,0,14.7,0.2,208.4,0.021,0.1,0.045,0.034,0.5,0.3,1.4,0.1,2,0,0.007,0.007,448.4,0.005,0.2,41.7,0.039,352.8,573.3,0.071,2.2,484.517 25 | 23,23,vanilla yoghurt oikos,120,1.8,24,4.9,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,1.8 26 | 24,24,fruit yogurt,149,1.7,1.1,0.5,0.024,28,28,6,0,7.5,0.013,113,0.006,0.081,0.032,0.087,0.2,0.1,0.7,0.099,0.9,0,0.079,0.031,207,0.1,0.066,19.5,0.029,163.5,265.5,0.026,1,243.671 27 | 25,25,danone low fat alsafi,88,1.7,1,11.1,7.1,0.089,255,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,1.743 28 | 26,26,fruit n yogurt parfait mcdonalds,156,1.9,0.044,0.2,0.1,30.9,19.1,4.1,1.5,7.5,0.065,111.2,0,0.004,0.029,0.099,0.2,0.4,0.6,0,20.7,0,0,0,128.1,0.092,0.7,20.9,0.3,125.2,248.8,0,0.5,187.9 29 | 27,27,triple zero greek yogurt dannon oikos,120,0,0,15,7,22.5,6,0.038,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,22.598 30 | 28,28,greek yogurt,100,0.7,0.2,0.018,0.010,6.1,5.5,17.3,0,8.5,0.081,144.7,0.035,0.009,0.045,0.036,0.5,0.4,0.6,0.1,0,0,0.089,0.010,187,0.021,0.1,18.7,0.004,229.5,239.7,0.032,0.9,211.251 31 | 29,29,chocolate rasberry protein mini proteinfx,102,2.9,1.8,9.1,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,2.9 32 | 30,30,gushers fruit gushers,360,6,80,40,0,0.2,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,6.2 33 | 31,31,egg fried,90,6.8,2,2.8,1.5,0.4,0.2,6.3,0,184.5,0.043,32,0.016,0.049,0.035,0.029,0.2,0.093,0.8,0.093,0,0.081,0.6,0.079,28.5,0.065,0.9,6,0.046,98.9,69.9,0.078,0.6,42.904 34 | 32,32,glucerna abbott,1010,35.6,2.8,0,0,120.9,49.8,0,12.8,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,169.3 35 | 33,33,sports drink,26,0,0,0,0,7.2,0,0,0,0,0.073,232.3,0,0,0,0,0,0,0,0,15.1,0,0,0,0,0.098,0.1,2.4,0,21.6,24,0.088,0.056,22.4 36 | 34,34,peanut butter cookie protein syntha 6,200,6,15,2,22,5,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,11.0 37 | 35,35,hanfprotein pulver rohkostqualitat veganz,107,3.1,0.3,2.3,1.6,14.2,0.057,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,17.3 38 | 36,36,smoothie high protein creamy chocolate slimfast,110,3.5,0.5,0,0,2.1,1,12,0,5,0.2,15,45,25,100,25,35,25,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,62.6 39 | 37,37,egg large,138,9.3,0.5,11.8,0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,9.3 40 | 38,38,boysenberries frozen,66,0.3,0.049,0.059,0.2,16.1,9.1,1.5,7,0,0.064,113.4,0,0.063,0.070,0,0.045,1,0.3,0.054,4.1,0,1.1,0.002,35.6,0.1,1.1,21.1,0.7,35.6,183.5,0.058,0.3,65.7 41 | 39,39,malt beverage,88,0.3,0.005,0.032,0.1,19.1,19.1,0.5,0,0,0.020,216,0,0.022,0.021,0,0.1,2.6,0.020,0.044,1.2,0,0,0,16.6,0.048,0.1,16.6,0.046,37.9,19,0.049,0.025,37.8 42 | 40,40,chocolate powder for milk,88,0.5,0.5,0,0,19.9,18,1,1,0,0.012,0.2,0,0.041,0.079,0,0.090,0.064,0,0.2,6,0,0.038,0.041,100.1,0.2,0,11.7,0.2,18.9,61.6,0.039,1.5,128.5 43 | 41,41,vegetable fruit juice,9,0.017,0,0,0,2.2,0.6,0.071,0,0,0.043,27.2,0,0.047,0,0,0.075,0.023,0.025,0.050,9.6,0,0.5,0.036,0.9,0.031,0.013,0.3,0.032,0.6,5.6,0.027,0.069,12.912 44 | 42,42,cocoa powder unsweetened,12,0.7,0.4,0.2,0.048,3.1,0.087,1.1,1.8,0,0.062,0.2,0,0.049,0.078,0,0.058,0.1,0.054,0.028,0,0,0.085,0.056,6.9,0.2,0.7,26.9,0.2,39.6,82.3,0.035,0.4,14.3 45 | 43,43,coconut milk,35,3.6,3.2,0.2,0.010,0.8,0.5,0.3,0.3,0,0.001,10.1,0,0.058,0.094,0,0,0.1,0.045,0.069,0.4,0,0.063,0.014,2.4,0.023,0.2,5.6,0.1,15,39.5,0.096,0.1,8.0 46 | 44,44,cocoa powder,28,0.3,0.2,0.061,0.055,5.9,4.6,0.5,0.3,0,0.034,0.1,0.066,0.045,0.096,0.051,0.033,0.048,0.066,0.085,0.016,0,0.094,0.001,9.3,0.049,0.075,5.8,0.010,22.1,49.8,0.069,0.1,16.393 47 | 45,45,rice drink,113,2.3,0,1.5,0.8,22,12.7,0.7,0.7,0,0.058,214.3,0,0.2,0.010,0.000,0.008,0.3,0.9,0.4,0.059,0,0.042,1.1,0.078,283.2,0.013,0.5,26.4,0.7,134.4,64.8,0.009,25.883 48 | 46,46,coconut water,46,0.5,0.4,0.091,0.030,8.9,6.3,1.7,2.6,0,0.3,228,0,0.002,0.050,0,0.1,0.2,0.1,0.082,5.8,0,0,0,57.6,0.029,0.7,60,0.3,48,600,0.045,0.2,77.8 49 | 47,47,caramel med iced coffee dunkin donuts,268,9,41,37,5,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,9.0 50 | 48,48,instant coffee with water decaffeinated,4,0,0.024,0,0.068,0.8,0,0.2,0,0,0.093,177.1,0,0,0,0,0.058,0.5,0.092,0,0,0,0,0,7.2,0.046,0.042,7.2,0.098,5.4,64.4,0.020,0.053,8.247 51 | 49,49,instant coffee powder,353,0.5,0.2,0.090,0.2,75.4,0,12.2,0,0,0.051,3.1,0,0.090,0,0,0.031,28.2,0.019,0.073,0,0,0,0.077,141,0.1,4.4,327,1.7,303,3535.00,0.024,0.4,233.5 52 | 50,50,instant coffee powder decaffeinated,6,0.069,0.015,0.031,0.075,1.4,0,0.2,0,0,0.090,0.051,0,0,0,0,0.072,0.5,0.077,0,0,0,0,0.015,2.5,0.031,0.056,5.6,0.084,5.1,63,0.015,0.004,4.121 53 | 51,51,instant cappuccino powder,53,0.7,0.4,0.2,0.098,11.2,9,0.4,0.2,0.3,0.074,0.2,0,0.073,0.067,0.037,0.087,0.3,0.037,0.011,0.100,0,0.080,0.062,17.3,0.078,0.036,5.6,0.014,23.8,108,0.064,0.058,29.908 54 | 52,52,coffee decaffeinated,0,0,0.087,0,0.099,0,0,0.2,0,0,0.058,235.3,0,0,0,0,0,0.5,0.091,0,0,0,0,0.023,4.7,0.028,0.1,11.9,0.083,2.4,128,0.042,0.084,5.0 55 | 53,53,espresso decaffeinated,3,0.011,0.037,0,0.057,0.5,0,0.065,0,0,0.011,29.3,0,0.087,0.054,0,0.003,1.6,0.086,0.035,0.026,0,0,0.047,0.6,0.085,0.087,24,0.093,2.1,34.5,0,0.036,1.391 56 | 54,54,instant coffee with water,4,0,0.051,0,0.025,0.6,0,0.2,0,0,0.015,177.4,0,0,0,0,0.073,0.4,0.089,0,0,0,0,0,7.2,0.026,0.015,7.2,0.001,5.4,53.7,0.054,0.013,8.027 57 | 55,55,coffee,2,0.069,0.011,0.016,0.011,0,0,0.3,0,0,0.075,235.6,0,0.034,0.076,0,0.2,0.5,0.6,0.022,0,0,0.068,0.025,4.7,0.045,0.051,7.1,0.076,7.1,116.1,0,0.039,5.09 58 | 56,56,espresso,3,0.086,0.068,0,0.054,0.5,0,0.021,0,0,0.083,29.3,0,0.088,0.073,0,0.080,1.6,0.075,0.099,0.099,0,0.005,0.027,0.6,0.022,0.049,24,0.023,2.1,34.5,0,0.020,1.263 59 | 57,57,whey protein powder vanilla eas,108,1.7,1.4,1.3,20.2,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,1.7 60 | 58,58,energy drink sugar free red bull,13,0.2,0,0,0,1.8,0,0.6,0,0,0.008,245.9,0,0.031,0,0.090,1.4,21.3,4.8,5,0,0,0,0,32.5,0.048,0.023,7.5,0.016,0,7.5,0.067,0,35.19 61 | 59,59,powerade coca cola,9,0.009,0.076,0.078,0.054,2.3,1.8,0,0,0,0.005,27.1,0,0.021,0,0.082,0,0.5,0,0.013,0,0,0,0,0.3,0,0.096,0,0,0.3,5.3,0,0.058,2.671 62 | 60,60,amp energy pepsi,110,0.2,0,0,0,29,29,0.6,0,0,0.093,209.5,0,0.078,0,0.083,0.8,4.8,2.4,0.5,0,0,0,0,31.2,0.044,0.053,7.2,0,40.8,7.2,0.075,0,61.067 63 | 61,61,gatorade g2 pepsi,2,0.052,0,0,0,0.6,0.4,0.069,0,0,0.011,28.9,0,0,0,0.091,0,0,0,0,0,0,0,0.038,0,0,0,0,0,2.1,3.8,0,0,0.69 64 | 62,62,amp energy sugar free pepsi,5,0,0,0,0,2.5,0,0,0,0,0.093,236,0,0,0,0.084,0.3,2,1,0.2,0,0,0,0,0,0.095,0,7.2,0,40.8,4.8,0.038,0,2.5 65 | 63,63,energy drink red bull,111,0,0,0,0,26.4,26.4,1.2,0,0,0.1,230.2,0,0,0.1,0,0.068,0.3,25.4,4.9,5.6,0,0,0,0,15.5,0.049,0.2,49,0.086,0,7.7,0.072,33.224 66 | 64,64,powerade zero ion4 coca cola,0,0,0,0,0,0,0,0,0,0,0.030,29.5,0,0.098,0,0.028,0,0.2,0,0.064,0.7,0,0.2,0.057,0.6,0.051,0.072,0.3,0,0,3,0,0.026,1.393 67 | 65,65,full throttle coca cola,110,0.2,0,0,0,29,29,0.6,0,0,0.071,209.5,0,0.037,0,0.031,0,4,0,0.4,0,0,0,0,31.2,0.054,0.037,7.2,0,0,7.2,0.083,0,61.049 68 | 66,66,passion fruit juice purple,148,0.4,0.064,0.037,0.3,35.7,35.2,1.7,0.5,0,0.084,208,0,0,0.039,0,0.2,5.5,0,0.1,45,0,0.090,0.011,9.9,0.1,0.9,42,0,61.8,686.7,0.001,0.1,94.1 69 | 67,67,pomegranate juice,134,0.7,0.2,0.1,0.1,32.7,31.5,0.4,0.2,0,0.089,214,0,0.037,0.014,0,0.006,0.6,0.7,0.019,0.2,0,0.9,0.081,27.4,0.060,0.2,17.4,0.2,27.4,532.9,0.065,0.2,61.8 70 | 68,68,cranberry apricot juice,157,0,0,0,0,39.7,0,0.5,0.2,0,0.007,204.6,0,0.089,0.071,0,0.010,0.3,0.071,0.082,0,0,0,0,22.1,0.051,0.4,7.4,0.3,12.3,149.5,0,0.050,62.9 71 | 69,69,pineapple juice,133,0.3,0.071,0.027,0.1,32.2,25,0.9,0.5,0,0.075,215.9,0,0,0.1,0.004,0,0.064,0.5,0.1,0.3,109.5,0,0.048,0.062,32.5,0.2,0.8,30,1.3,20,325,0.099,34.477 72 | 70,70,passion fruit juice yellow,126,0.1,0.008,0.020,0.094,33.6,33.1,1,0.5,0,0.003,211.5,0,0,0.100,0,0.3,3.6,0,0.1,73.6,0,0.036,0.090,9.9,0.1,0.6,42,0,32.1,686.7,0.098,0.1,119.3 73 | 71,71,orange apricot juice,128,0.3,0.001,0.078,0.063,31.8,30.2,0.8,0.3,0,0.096,216.8,0,0.022,0.094,0,0.021,0.5,0.2,0.080,50,0,0.3,0.054,12.5,0.024,0.3,12.5,0.091,20,200,0.086,0.1,96.0 74 | 72,72,tangerine juice,106,0.5,0.082,0.015,0.054,24.9,24.5,1.2,0.5,0,0.022,219.6,0,0,0.1,0.086,0,0.094,0.2,0.3,0.1,76.6,0,0.3,0,44.5,0.060,0.5,19.8,0.031,34.6,439.7,0.016,27.29 75 | 73,73,orange juice,90,0.4,0.079,0.046,0.037,20.8,16.8,1.4,0.4,0,0.067,176.6,0,0,0.2,0.002,0,0.010,0.8,0.4,0.052,100,0,0.013,0.053,22,0.060,0.4,22,0.056,34,400,0.067,23.177 76 | 74,74,grapefruit juice,96,0.2,0.011,0.090,0.008,22.7,0,1.2,0,0,0.061,222.3,0,0,0.060,0.038,0,0.010,0.5,0.5,0.1,93.9,0,0,0,22.2,0.004,0.5,29.6,0.037,37.1,400.1,0,24.254 77 | 75,75,pineapple grapefruit juice,118,0.3,0.077,0.000,0.022,29,28.8,0.5,0.3,0,0.054,219.8,0,0.042,0.047,0,0.070,0.7,0.1,0.1,115,0,0.001,0.046,17.5,0.1,0.8,15,1,15,152.5,0.025,0.2,163.4 78 | 76,76,tamarind nectar,143,0.3,0,0,0,37,31.9,0.2,1.3,0,0.046,213.3,0,0.077,0.055,0,0.024,0.2,0.025,0.048,17.8,0,0.3,0.069,25.1,0.054,1.9,10,0.079,5,67.8,0,0.089,83.6 79 | 77,77,apple juice,114,0.3,0.064,0.078,0.090,28,23.9,0.2,0.5,0,0.038,218.8,0,0,0.083,0,0,0.019,0.2,0.1,0.025,2.2,0,0.057,0,19.8,0.065,0.3,12.4,0.2,17.4,250.5,0.039,29.087 80 | 78,78,limeade,128,0,0,0,0,34.1,32.8,0,0,0,0.071,212.6,0,0.047,0.058,0,0.034,0.050,0.006,0.051,7.7,0,0,0,4.9,0.058,0,4.9,0.068,2.5,24.7,0.081,0.024,46.7 81 | 79,79,guava nectar,158,0.2,0.050,0.071,0.054,40.8,32.5,0.2,2.5,0,0.063,209.6,0,0,0.060,0.075,0,0.073,0.4,0.2,0.057,53,0,0.1,0.087,20.1,0.055,1,5,0.1,5,97.9,0,43.83 82 | 80,80,orange grapefruit juice,106,0.2,0.009,0.001,0.098,25.4,25.1,1.5,0.2,0,0.082,218.9,0,0.1,0.005,0,0.040,0.8,0.3,0.079,71.9,0,0.3,0.066,19.8,0.2,1.1,24.7,0.067,34.6,390.3,0.005,0.2,120.1 83 | 81,81,orange juice sainsburys,14,0.088,2.5,0.1,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.049 84 | 82,82,cranberry apple juice,154,0.3,0.066,0.014,0.1,38.8,35.5,0,0,0,0.061,205.3,0,0,0,0,0,0,0,0,96.8,0,0.3,0.075,7.4,0.070,0.2,2.5,0.100,4.9,41.7,0,0.015,143.5 85 | 83,83,pineapple orange juice,125,0,0,0,0,29.5,29,3.3,0.3,0,0.069,217.3,0,0.035,0.046,0,0.020,0.5,0.1,0.1,56.3,0,0.079,0.085,12.5,0.1,0.7,15,0.9,10,115,0.071,0.2,102.6 86 | 84,84,pear nectar,150,0.062,0.032,0.086,0.038,39.4,0,0.3,1.5,0,0.056,210,0,0,0.032,0.049,0,0.033,0.3,0.087,0.017,67.5,0,0,0,12.5,0.2,0.7,7.5,0.041,7.5,32.5,0.065,41.466 87 | 85,85,blackberry juice,95,1.5,0.095,0.1,0.9,19.5,19.3,0.8,0.3,0,0.072,227.3,0,0.076,0.091,0,0.053,1.1,0,0.075,28.3,0,2.3,0.088,30,0.3,1.2,52.5,0,30,337.5,0.093,1,81.6 88 | 86,86,lemon juice,54,0.6,0.060,0.073,0.038,16.6,6.1,0.9,0.7,0,0.046,225.2,0,0,0.015,0.093,0,0.084,0.2,0.2,0.1,94.4,0,0.4,0,14.6,0.073,0.2,14.6,0.091,19.5,251.3,0.090,18.9 89 | 87,87,cranberry grape juice,137,0.2,0.053,0.092,0.080,34.3,0,0.5,0.2,0,0.079,209.7,0,0.094,0.090,0,0.056,0.3,0.1,0.027,78.4,0,0,0,19.6,0.051,0.040,7.4,0.4,9.8,58.8,0,0.055,133.2 90 | 88,88,citrus fruit juice,114,0.057,0.088,0.048,0.085,28.3,20.2,0.8,0.2,0,0.051,217.8,0,0.043,0.021,0,0.096,0.2,0.2,0.054,67.2,0,0.1,0.042,22.3,0.065,2.8,14.9,0.2,24.8,277.8,0.036,0.011,121.7 91 | 89,89,prune juice,182,0.006,0.033,0.082,0.011,44.7,42.1,1.6,2.6,0,0.100,208,0,0.038,0,0,0.2,2,0.3,0.6,10.5,0,0.3,0.007,30.7,0.2,3,35.8,0.4,64,706.6,0.083,0.5,93.105 92 | 90,90,guanabana nectar,148,0.4,0.096,0.1,0.006,37.5,32.8,0.3,0.3,0,0.061,212.6,0,0.086,0.050,0,0.027,0.3,0.2,0.050,27.9,0,0.036,0,17.6,0.067,0.9,7.5,0.056,5,62.8,0.067,0.011,84.9 93 | 91,91,fruit juice,108,0.1,0.059,0.048,0.036,26.3,24.2,0.3,0.2,0,0.1,172.7,0,0.071,0.2,0,0,0.096,0.028,0,0.001,50.8,0,0.1,0,6,0.039,0.5,10,0,14,244,0.089,26.937 94 | 92,92,horchata,15,0.2,0,0,0,3.3,2.6,0.1,0,0,0.013,24.6,0,0,0,0,0,0.016,0,0.029,0,0,0.022,0,5.1,0.028,0.057,0.8,0.045,4.2,9.6,0.005,0.058,8.717 95 | 93,93,fruit punch,98,0.4,0.048,0.011,0.039,24.4,0,0.2,0,0,0.080,208.6,0,0.092,0.061,0,0.1,0.1,0.035,0.047,11.2,0,0,0,16.4,0.097,0.5,7,0.1,7,154.4,0,0.4,53.1 96 | 94,94,apricot nectar,139,0.2,0.070,0.085,0.039,35.7,34.2,0.9,1.5,0,0.025,210.5,0,0.044,0.059,0,0.014,0.6,0.2,0.036,1.5,0,0.8,0.008,17.4,0.2,0.9,12.4,0.008,22.3,282.7,0,0.2,58.1 97 | 95,95,orangensaft ja,86,0,0,17.8,17.8,1.4,0.024,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,1.4 98 | 96,96,fruchtcocktail granini,102,0.4,0.2,23,23,1,0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,1.4 99 | 97,97,peach nectar,125,1.3,0.6,0.2,0.5,29.7,27.8,0.5,0.2,0,0.077,217.3,0,0.038,0.1,0.005,0,0.049,0.5,0.2,0.091,75.2,0,1.5,0,14.9,0.2,0.2,12.5,0.058,12.5,124.5,0.071,31.949 100 | 98,98,orange pineapple juice,15,0.080,0.056,0.026,0.024,3.6,3.1,0.1,0.032,0,0.044,25.7,0,0.044,0.010,0,0.069,0.054,0,0.053,12,0,0.036,0.016,2.4,0.090,0.086,3,0.1,3.2,44.3,0.076,0.072,18.261 101 | 99,99,cranberry juice,92,0.3,0.061,0.098,0.1,24.4,24.2,0.8,0.2,0,0.093,174.3,0,0,0.069,0.011,0,0.016,0.2,0,0.1,18.6,0,2.4,0.099,16,0.1,0.5,12,0,26,154,0.036,25.975 102 | 100,100,white grapefruit juice,96,0.2,0.083,0.091,0.007,22.7,22.5,1.2,0.2,0,0.018,222.3,0,0.015,0.029,0,0.005,0.5,0.5,0.1,93.9,0,0.5,0,22.2,0.035,0.5,29.6,0.032,37.1,400.1,0.019,0.1,140.9 103 | 101,101,mango nectar,128,0.2,0.040,0.092,0.066,32.9,31.2,0.3,0.8,0,0.061,217.4,0,0,0.039,0.055,0,0.060,0.2,0.2,0.038,38.2,0,0.5,0.047,42.7,0.087,0.9,7.5,0.007,5,60.2,0.025,34.447 104 | 102,102,acerola cherry juice,56,0.7,0.2,0.2,0.2,11.6,10.9,1,0.7,0,0.082,228.2,0,0.067,0.084,0,0.1,1,0.5,0.051,3872.00,0,0.4,0.057,24.2,0.2,1.2,29,0,21.8,234.7,0.061,0.2,3911.4 105 | 103,103,apple juice concentrate,350,0.8,0.1,0.053,0.2,86.5,81.9,1.1,0.8,0,0.049,120.3,0,0.029,0.005,0,0.1,0.3,0.5,0.2,4.4,0,0.022,0,42.2,0.1,1.9,35.9,0.5,52.8,945.3,0.067,0.3,137.7 106 | 104,104,papaya nectar,143,0.4,0.1,0.1,0.070,36.3,34.8,0.4,1.5,0,0.082,212.6,0,0,0.053,0.039,0,0.077,0.4,0.1,0.017,7.5,0,0.6,0.076,25,0.029,0.9,7.5,0.012,0,77.5,0.072,38.657 107 | 105,105,apple juice martinellis,122,0,0,29.1,0.7,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 108 | 106,106,lime juice,61,0.2,0.052,0.087,0.071,20.4,4.1,1,1,0,0.014,219.7,0,0.015,0.010,0,0.078,0.3,0.3,0.040,72.6,0,0.5,0.074,33.9,0.027,0.2,19.4,0.011,33.9,283.1,0.052,0.2,129.3 109 | 107,107,grape juice,152,0.3,0.060,0.030,0.059,37.4,35.9,0.9,0.5,0,0.003,213.8,0,0.050,0,0,0.025,0.3,0.1,0.086,0.3,0,0,0.025,27.8,0.025,0.6,25.3,0.6,35.4,263.1,0,0.2,67.8 110 | 108,108,water dannon,0,0,0,0,0,0,0,0,0,0,0,330.9,0,0,0,0,0,0,0,0,0,0,0,0,9.9,0,0,3.3,0,0,0,0,0,9.9 111 | 109,109,tap water,0,0,0,0,0,0,0,0,0,0,0.018,236.8,0,0,0,0,0,0,0,0,0,0,0,0,7.1,0.086,0,2.4,0,0,0,0,0.042,7.1 112 | 110,110,table water,0,0,0,0,0,0,0,0,0,0,0.044,499.5,0,0,0,0,0,0,0,0,0,0,0,0,70,0,0,0,0,0,0,0,0,70.0 113 | 111,111,fruit flavored water,52,0,0,0,0,13,13,0,0,0,0,223.8,0.050,0,0.016,0.049,0,2,1,0.2,30.1,0,4.5,0,40.3,0.061,0,7.1,0,0,0,0,0.8,83.471 114 | 112,112,water,0,0,0,0,0,0,0,0,0,0,0.024,237,0,0,0,0,0,0,0,0,0,0,0,0,23.7,0.098,0,4.7,0,0,0,0,0,23.7 115 | 113,113,cola light coca cola,0,0,0,0,0,0,0,0,0,0,0.087,336.5,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 116 | 114,114,lemon lime soda,151,0,0,0,0,38.4,37.6,0.3,0,0,0.024,330.2,0,0,0,0,0,0.055,0,0,0,0,0,0,7.4,0,0.061,3.7,0,0,3.7,0,0.065,46.113 117 | 115,115,tonic water,114,0,0,0,0,29.6,29.6,0,0,0,0.049,306.1,0,0,0,0,0,0,0,0,0,0,0,0,3.4,0.031,0.090,0,0.050,0,0,0,0.3,33.038 118 | 116,116,lemonade low calorie,7,0,0,0,0.000,1.6,0,0.092,0,0,0.013,236.8,0,0,0,0,0,0,0,0,7.6,0,0,0,66.9,0.040,0.1,2.4,0.057,31.1,2.4,0,0.011,76.216 119 | 117,117,ginger ale,68,0,0,0,0,17.5,17.4,0,0,0,0.043,182.5,0,0,0,0,0,0,0,0,0,0,0,0,0,6,0.024,0.4,2,0.087,0,2,0.078,17.528 120 | 118,118,ice tea lemon flavor nestle,88,0,0,0,0,22.3,22.3,0,0,0,0.063,223.4,0,0,0,0,0,0,0,0,0,0,0,0,7.4,0.068,0,2.5,0.3,88.2,46.6,0,0.1,29.7 121 | 119,119,sprite coca cola,80,0.063,0,0,0,20.3,18,0.1,0,0,0.021,179.6,0,0,0,0,0,0,0.032,0,0,0,0,0,0,4,0.042,0.2,2,0.017,0,2,0,20.538 122 | 120,120,cola coca cola,151,0,0,0,0,38.9,38.9,0,0,0,0.045,329.8,0,0,0,0,0,0,0,0,0,0,0,0,7.4,0,0.048,0,0,40.5,11,0.045,0.052,46.4 123 | 121,121,cola,74,0.070,0,0,0,19.1,17.9,0.1,0,0,0.072,180.6,0,0,0,0,0,0,0,0,0,0,0,0,0,4,0.012,0.2,0,0.081,20,4,0.038,19.336 124 | 122,122,cream soda,252,0,0,0,0,65.7,65.7,0,0,0,0.098,428.3,0,0,0,0,0,0,0,0,0,0,0,0,24.7,0.036,0.2,4.9,0.080,0,4.9,0,0.3,90.6 125 | 123,123,club soda,0,0,0,0,0,0,0,0,0,0,0.046,199.8,0,0,0,0,0,0,0,0,0,0,0,0,0,10,0.015,0.078,2,0.052,0,4,0,0.025 126 | 124,124,pink lemonade,86,0.3,0.079,0.030,0.034,21.6,20.6,0.1,0.2,0,0.010,177.8,0,0.003,0.078,0,0,0.047,0.035,0.061,6.2,0,0.062,0,8,0.011,0.092,4,0.094,4,34,0,0.093,36.488 127 | 125,125,grape soda,160,0,0,0,0,41.7,0,0,0,0,0.042,330.3,0,0,0,0,0,0,0,0,0,0,0,0,11.2,0.038,0.3,3.7,0.054,0,3.7,0,0.3,53.2 128 | 126,126,cola without caffeine coca cola,151,0,0,0,0,38.9,38.9,0,0,0,0.040,329.8,0,0,0,0,0,0,0,0,0,0,0,0,7.4,0,0.026,0,0,40.5,11,0.063,0.032,46.381 129 | 127,127,white lemonade,80,0.003,0.093,0.024,0.076,20.8,20,0.1,0,0,0.020,178.7,0,0,0.004,0.043,0,0.020,0.094,0.094,0.007,7.8,0,0.040,0,8,0.013,0.3,4,0.083,4,30,0.066,21.02 130 | 128,128,chocolate soda,207,0,0,0,0,52.6,52.6,0,0,0,0.031,438.9,0,0,0,0,0,0,0,0,0,0,0,0,19.7,0.022,0.5,4.9,0.2,4.9,246,0.015,0.8,72.8 131 | 129,129,sweet tea lipton,130,0,0,0,0,35,35,0,0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,35.0 132 | 130,130,hibiscus tea,74,1.3,0.5,0.2,0.040,14.8,12,0.9,0.6,0,0.091,179.3,0,2.6,0.030,0,0.2,0,0,0,36.8,0,0,0,2,0.1,17.3,2,0,6,18,0,0.2,73.7 133 | 131,131,instant lemon tea,2,0,0,0,0,0.3,0.047,0.1,0,0,0.046,199.2,0,0,0,0,0,0.003,0.023,0.060,0.009,0,0,0,0,6,0.026,0.020,4,0.8,2,36,0,0.494 134 | 132,132,pesca tea santal,116,0,28.1,0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 135 | 133,133,instant lemon tea sweetened,91,0.2,0.018,0.046,0.093,22.3,21.6,0.071,0.3,0,0.089,236.2,0,0.083,0,0,0,0.006,0,0.044,0,0,0,0,5.2,0.076,0.000,2.6,0.3,0,38.9,0.017,0.048,28.147 136 | 134,134,tea with milk and sugar yorkshire,15,0.5,0.3,0.7,0.7,0.4,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.9 137 | 135,135,tea blueberry muffin tekanne,4,0.2,0.2,0.6,0.4,0.4,0.007,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.6 138 | 136,136,chamomile tea,2,0,0.028,0.032,0.073,0.4,0,0,0,0,0.003,199.4,0,0.060,0.050,0,0.022,0,0.027,0,0,0,0,0,4,0.097,0.2,2,0.078,0,18,0,0.006,4.6 139 | 137,137,black tea,2,0,0.072,0.032,0.029,0.7,0,0,0,0,0.091,236.3,0,0,0.041,0,0.018,0,0.040,0,0,0,0,0,0,0.088,0.088,7.1,0.5,2.4,87.7,0,0.081,0.72 140 | 138,138,herb tea,2,0,0.014,0.043,0.071,0.4,0,0,0,0,0.064,199.4,0,0.007,0.093,0,0.051,0,0.096,0,0,0,0,0,4,0.059,0.2,2,0.019,0,18,0,0.006,4.6 141 | 139,139,clam tomato juice,80,0.3,0,0,0,18.2,5.5,1,0.7,0,0.6,145.1,0,0.015,0.079,0,0.064,0.4,0.1,0.1,8.3,0,0.2,0,13.3,0.011,0.2,8.3,0.073,18.3,147.7,0.097,0.1,42.0 142 | 140,140,tomato juice,41,0.7,0.043,0.062,0.043,8.6,6.3,2.1,1,0,0.063,229,0,0.069,0.2,0.003,0,0.2,1.6,0.6,0.2,170.3,0,0.8,0.056,24.3,0.1,0.9,26.7,0.2,46.2,527.3,0.067,12.742 143 | 141,141,vegetable juice,53,0.2,0.071,0.016,0.1,11.1,7.9,1.5,1.9,0,0.1,226.3,0,0,0.095,0.039,0,0.091,1.8,0.6,0.3,67,0,0.8,0.007,26.6,0.5,1,26.6,0.2,41.1,467.1,0.026,15.538 144 | 142,142,tomato vegetable juice,53,0.2,0.014,0.014,0.1,11.1,9.2,1.5,1.9,0,0.2,226.3,0,0.034,0.047,0,0.097,1.8,0,0.3,67,0,0.8,0.085,26.6,0.5,1,26.6,0,41.1,467.1,0.046,0.5,109.3 145 | 143,143,carrot juice,80,0.3,0.055,0.027,0.1,18.6,7.8,1.9,1.6,0,0.1,177.7,0,0.2,0.077,0,0.1,0.8,0.5,0.4,17,0,2.3,0.047,48,0.087,0.9,28,0.3,84,584,0.046,0.4,88.3 146 | 144,144,acorn dried,144,8.9,1.2,5.6,1.7,15.2,0,2.3,0,0,0,1.4,0,0.005,0.085,0,0.021,0.7,0.3,0.2,0,0,0,0,15.3,0.2,0.3,23.2,0.4,29.1,200.6,0,0.2,42.0 147 | 145,145,fenugreek seeds,36,0.7,0.2,0,0,6.5,0,2.6,2.7,0,0.090,1,0,0.036,0.046,0,0.019,0.2,0,0.064,0.3,0,0,0,19.5,0.1,3.7,21.2,0.1,32.9,85.5,0.025,0.3,36.0 148 | 146,146,peanut topping mcdonalds,181,15,2.4,7.3,4,4.6,1.2,7.9,0,0,0,0.2,0,0.070,0.015,0,0.099,4.7,0.3,0.2,0,0,2.3,0,13,0.1,0.4,54.6,0.6,103,179.4,0,0.8,40.9 149 | 147,147,beechnuts dried,173,15,1.7,6.6,6,10.1,0,1.9,0,0,0.068,2,0,0.053,0.087,0,0.1,0.3,0.3,0.2,4.7,0,0,0,0.3,0.2,0.7,0,0.4,0,305.1,0,0.1,32.7 150 | 148,148,breadnut tree seeds raw,61,0.3,0.089,0.061,0.1,13.1,0,1.7,0,0,0.084,12.7,0,0.040,0.077,0,0.053,0.2,0.3,0.1,7.8,0,0,0,27.7,0.4,0.6,19.2,0.015,19,334.8,0,0.3,51.2 151 | 149,149,cumin seeds,8,0.5,0.096,0.3,0.013,0.9,0.063,0.4,0.2,0,0.073,0.2,0,0.079,0.015,0,0.093,0.064,0,0.021,0.2,0,0.096,0.001,19.6,0.058,1.4,7.7,0.087,10.5,37.5,0.072,0.1,23.2 152 | 150,150,pili nuts dried,863,95.5,37.4,44.7,9.1,4.8,0,13,0,0,0.014,3.3,0,1.1,0.066,0,0.1,0.6,0.6,0.1,0.7,0,0,0,174,1.1,4.2,362.4,2.8,690,608.4,0,3.6,292.2 153 | 151,151,hazelnuts roasted,194,18.7,1.4,14,2.5,5.3,1.5,4.5,2.8,0,0,0.8,0,0.1,0.031,0,0.074,0.6,0.3,0.2,1.1,0,4.6,0,36.9,0.5,1.3,51.9,1.7,93,226.5,0.061,0.8,70.6 154 | 152,152,lupins cooked,193,4.8,0.6,2,1.2,15.4,0,25.8,4.6,0,0.4,118,0,0.2,0.083,0,0.071,0.8,0.3,0.016,1.8,0,0,0,84.7,0.4,2,89.6,1.1,212.5,406.7,0.084,2.3,139.1 155 | 153,153,hazelnuts blanched,178,17.3,1.3,13.6,1.6,4.8,1,3.9,3.1,0,0,1.6,0,0.1,0.021,0,0.049,0.4,0.2,0.2,0.6,0,5,0,42.2,0.5,0.9,45.3,3.6,87.7,186.2,0.015,0.6,72.8 156 | 154,154,brazilnuts dried,33,3.3,0.8,1.2,1,0.6,0.1,0.7,0.4,0,0.073,0.2,0,0.079,0.035,0,0.013,0.083,0.051,0.041,0.024,0,0.3,0,8,0.074,0.1,18.8,0.084,36.3,33,0.035,0.2,13.139 157 | 155,155,sunflower seeds dry roasted,47,4,0.4,0.8,2.6,1.9,0.2,1.5,0.9,0,0.067,0.076,0,0.087,0.072,0,0.086,0.6,0.6,0.010,0.1,0,2.1,0.037,5.6,0.1,0.3,10.3,0.2,92.4,68,0.021,0.4,14.3 158 | 156,156,ginkgo nuts canned,172,2.5,0.5,0.9,0.9,34.3,0,3.5,14.4,0,0.5,113.1,0,0.2,0.060,0,0.052,5.6,0.2,0.3,14.1,0,0,0,6.2,0.3,0.4,24.8,0.1,83.7,279,0.067,0.3,75.4 159 | 157,157,peanuts cooked,572,39.6,5.5,19.7,12.5,38.3,4.4,24.3,15.8,0,1.4,75.2,0,0.5,0.1,0,0.1,9.5,1.5,0.3,0,0,7.4,0,99,0.9,1.8,183.6,1.8,356.4,324,0.027,3.3,218.8 160 | 158,158,pumpkin squash seeds dried,721,63.3,11.2,21,27.1,13.8,1.8,39,7.7,0,0.013,6.7,0,0.4,0.024,0,0.2,6.4,1,0.2,2.5,0,2.8,0.030,59.3,1.7,11.4,763.7,5.9,1590.60,1043.60,0.078,10.1,197.0 161 | 159,159,cashew nuts raw,156,12.4,2.2,6.7,2.2,8.5,1.7,5.2,0.9,0,0.020,1.5,0,0.1,0.076,0,0.071,0.3,0.2,0.1,0.1,0,0.3,0.040,10.5,0.6,1.9,82.6,0.5,167.8,186.8,0.039,1.6,39.5 162 | 160,160,sunflower seeds dried,269,23.7,2,8.5,10.6,9.2,1.2,9.6,4,0,0.014,2.2,0,0.7,0.1,0,0.2,3.8,0.5,0.6,0.6,0,16.2,0,35.9,0.8,2.4,149.5,0.9,303.6,296.7,0.077,2.3,85.4 163 | 161,161,lotus seeds dried,106,0.6,0.1,0.1,0.4,20.6,0,4.9,0,0,0.024,4.5,0,0.2,0.019,0,0.059,0.5,0.3,0.2,0,0,0,0,52.2,0.1,1.1,67.2,0.7,200.3,437.8,0,0.3,79.4 164 | 162,162,butternuts dried,734,68.4,1.6,12.5,51.3,14.5,0,29.9,5.6,0,0.062,4,0,0.5,0.088,0,0.2,1.3,0.8,0.7,3.8,0,0,0,63.6,0.5,4.8,284.4,7.9,535.2,505.2,0.028,3.8,190.6 165 | 163,163,sesame seeds dried,52,4.5,0.6,1.7,2,2.1,0.031,1.6,1.1,0,0.049,0.4,0,0.037,0.050,0,0.057,0.4,0.082,0.096,0,0,0.099,0,87.8,0.4,1.3,31.6,0.2,56.6,42.1,0.047,0.7,98.4 166 | 164,164,breadfruit seeds,57,1.7,0.5,0.2,0.9,8.8,0,2.2,1.6,0,0.040,16.9,0,0.1,0.095,0,0.066,0.1,0.3,0.024,2,0,0,0,10.8,0.3,1.1,16.2,0.049,52.5,282.3,0,0.3,28.2 167 | 165,165,pinyon pine nuts dried,1,0.073,0.022,0.044,0.074,0.010,0,0.083,0.005,0,0.070,0.045,0,0.020,0.066,0,0.040,0.027,0.000,0.048,0.004,0,0,0,0.022,0.078,0.007,0.2,0.012,0.098,0.6,0,0.049,0.359 168 | 166,166,pine nuts dried,909,92.3,6.6,25.3,46,17.7,4.8,18.5,5,0,0.086,3.1,0,0.5,0.018,0,0.3,5.9,0.4,0.1,1.1,0,12.6,0.002,21.6,1.8,7.5,338.9,11.9,776.3,806,0.061,8.7,163.7 169 | 167,167,flaxseeds,55,4.3,0.4,0.8,3,3,0.2,1.9,2.8,0,0.035,0.7,0,0.2,0.015,0,0.095,0.3,0.1,0.056,0.023,0,0.042,0.053,26.3,0.1,0.6,40.4,0.3,66.1,83.7,0.050,0.4,38.994 170 | 168,168,safflower seed kernels dried,146,10.9,1,1.4,8,9.7,0,4.6,0,0,0.042,1.6,0,0.3,0.055,0,0.1,0.6,1.1,0.3,0,0,0,0,22.1,0.5,1.4,99.9,0.6,182.3,194.4,0,1.4,48.7 171 | 169,169,pistachio nuts raw,691,55.8,6.8,29.3,16.9,33.8,9.4,24.9,12.7,0,0.037,4.8,0,1.1,0.087,0,0.2,1.6,0.6,2.1,6.9,0,2.8,0,129.2,1.6,4.8,148.8,1.5,602.7,1260.80,0.078,2.7,268.1 172 | 170,170,peanuts raw,828,71.9,10,35.7,22.7,23.5,5.8,37.7,12.4,0,0.038,9.5,0,0.9,0.4,0,0.2,17.6,2.6,0.5,0,0,12.2,0,134.3,1.7,6.7,245.3,2.8,549,1029.30,0.032,4.8,286.5 173 | 171,171,poppy seeds,46,3.7,0.4,0.5,2.5,2.5,0.3,1.6,1.7,0,0.079,0.5,0,0.034,0.044,0,0.040,0.069,0.071,0.090,0.061,0,0.2,0,126.5,0.1,0.9,30.5,0.6,76.6,63.3,0.059,0.7,136.934 174 | 172,172,valencia peanuts raw,832,69.5,10.7,31.3,24.1,30.5,0,36.6,12.7,0,0.094,6.2,0,0.9,0.4,0,0.4,18.8,2.6,0.5,0,0,0,0,90.5,1.7,3.1,268.6,2.9,490.6,484.7,0.056,4.9,242.9 175 | 173,173,spanish peanuts,832,72.4,11.2,32.6,25.1,23.1,0,38.2,13.9,0,0.036,9.3,0,1,0.4,0,0.2,23.3,2.6,0.5,0,0,0,0,154.8,1.3,5.7,274.5,3.9,566.5,1086.20,0.077,3.1,308.1 176 | 174,174,dill seeds,20,1,0.018,0.6,0.041,3.6,0,1.1,1.4,0,0.037,0.5,0,0.011,0.085,0,0.059,0.2,0,0.001,1.4,0,0,0,100.1,0.009,1.1,16.9,0.1,18.3,78.3,0.075,0.3,109.7 177 | 175,175,caraway seeds,22,1,0.030,0.5,0.2,3.3,0.038,1.3,2.5,0,0.014,0.7,0,0.096,0.052,0,0.075,0.2,0,0.099,1.4,0,0.2,0,46.2,0.077,1.1,17.3,0.034,38.1,90.5,0.062,0.4,56.8 178 | 176,176,acorn raw,110,6.8,0.9,4.3,1.3,11.5,0,1.7,0,0,0,7.9,0,0.016,0.060,0,0.045,0.5,0.2,0.1,0,0,0,0,11.6,0.2,0.2,17.5,0.4,22.4,152.5,0,0.1,31.8 179 | 177,177,almonds roasted,182,16.6,1.3,10.4,4.1,5.3,1.4,6.4,3.2,0,0.029,0.8,0,0.055,0.032,0,0.2,1.1,0.049,0.065,0,0,7.8,0,87.3,0.3,1.1,82.2,0.7,139.8,209.7,0.017,0.9,119.9 180 | 178,178,lotus seeds raw,25,0.1,0.097,0.011,0.002,4.9,0,1.2,0,0,0.020,21.8,0,0.073,0.052,0,0.070,0.1,0.088,0.074,0,0,0,0,12.5,0.079,0.3,15.8,0.2,47.5,103.9,0,0.043,19.0 181 | 179,179,chestnuts cooked,37,0.4,0.016,0.1,0.2,7.9,0,0.6,0,0,0.046,19.3,0,0.095,0.099,0,0.053,0.2,0.046,0.021,7.6,0,0,0,13,0.1,0.5,15.3,0.2,28,202.3,0,0.020,30.0 182 | 180,180,alfalfa seeds,1,0.075,0.029,0.067,0.057,0.002,0.058,0.1,0.015,0,0.048,2.8,0,0.065,0.008,0,0.078,0.020,0.051,0.021,0.2,0,0.036,0.042,1,0.048,0.034,0.8,0.024,2.1,2.4,0.076,0.071,1.435 183 | 181,181,virginia peanuts roasted,173,14.6,1.9,7.6,4.4,6,0,7.8,2.7,0,0.018,0.7,0,0.066,0.061,0,0.022,4.4,0.4,0.099,0,0,0,0,25.8,0.4,0.5,56.4,0.6,151.8,195.6,0.006,2,57.4 184 | 182,182,hazelnuts raw,471,45.6,3.3,34.2,5.9,12.5,3.3,11.2,7.3,0,0,4,0,0.5,0.034,0,0.022,1.4,0.7,0.4,4.7,0,11.3,0.097,85.5,1.3,3.5,122.3,4.6,217.5,510,0.012,1.8,170.3 185 | 183,183,anise seeds,7,0.3,0.060,0.2,0.066,1.1,0,0.4,0.3,0,0.100,0.2,0,0.058,0.002,0,0.091,0.090,0.058,0.010,0.4,0,0,0,13.6,0.024,0.8,3.6,0.031,9.2,30.3,0.050,0.1,16.9 186 | 184,184,macadamia nuts roasted,215,22.8,3.6,17.8,0.4,4,1.2,2.3,2.4,0,0.064,0.5,0,0.2,0.002,0,0.086,0.7,0.2,0.1,0.2,0,0.2,0,21,0.2,0.8,35.4,0.9,59.4,108.9,0.085,0.4,53.5 187 | 185,185,studentenfutter alnatura,139,7.8,0.9,4.8,2,13.2,11.7,3.1,0.031,1.7,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,24.152 188 | 186,186,pumpkin squash seeds roasted,46,3.9,0.7,1.3,1.6,1.2,0.1,2.4,0.5,0,0.045,0.2,0,0.046,0.041,0,0.046,0.4,0.020,0.012,0.1,0,0.049,0.002,4.2,0.1,0.6,44,0.4,93.9,63,0.050,0.6,12.9 189 | 187,187,water chestnut raw,120,0.1,0.039,0.055,0.089,29.7,6,1.7,3.7,0,0.097,91.1,0,0.2,0.011,0,0.2,1.2,0.6,0.4,5,0,1.5,0.058,13.6,0.4,0.034,27.3,0.4,78.1,724.2,0.004,0.6,53.84 190 | 188,188,chestnuts roasted,37,0.3,0.009,0.1,0.1,7.9,1.6,0.5,0.8,0,0.043,6.1,0,0.080,0.096,0,0.011,0.2,0.046,0.075,3.9,0,0.080,0.008,4.4,0.075,0.1,5,0.2,16.1,88.8,0.037,0.086,17.9 191 | 189,189,ginkgo nuts raw,52,0.5,0.029,0.2,0.2,10.6,0,1.2,0,0,0.067,15.6,0,0.013,0.085,0,0.040,1.7,0.078,0.098,4.2,0,0,0,0.6,0.029,0.3,7.6,0.027,35.1,144.3,0,0.079,17.4 192 | 190,190,almonds raw,9,0.7,0.040,0.5,0.2,0.3,0.026,0.3,0.2,0,0.043,0.026,0,0.079,0.073,0,0.066,0.052,0.068,0.029,0,0,0.4,0,4,0.007,0.082,4,0.007,7.3,10.6,0.013,0.061,5.554 193 | 191,191,peanuts roasted,180,15.8,2.6,7.8,4.6,4.6,1.3,8.4,2.8,0,0.069,0.4,0,0.057,0.051,0,0.020,4.1,0.4,0.1,0.2,0,2.1,0,18.3,0.2,0.5,52.8,0.6,119.1,217.8,0.066,1,50.6 194 | 192,192,sunflower seeds toasted,829,76.1,8,14.5,50.3,27.6,0,23.1,15.4,0,0.8,1.3,0,0.4,0.3,0,0.4,5.6,9.5,1.1,1.9,0,0,0,76.4,2.5,9.1,172.9,2.8,1551.70,657.9,0.066,7.1,229.6 195 | 193,193,sisymbrium seeds,235,3.4,0.7,1.1,1.5,43.1,0,9,0,0,0.047,4.5,0,0.1,0.072,0,0.3,12.5,0.7,0.6,22.7,0,0,0,1208.40,0.052,0.098,232.4,1.1,4.4,1576.20,0,0.2,1286.643 196 | 194,194,breadnut tree seeds dried,587,2.7,0.7,0.3,1.4,127,0,13.8,23.8,0,0.091,10.4,0,0.040,0.2,0,0.2,3.4,3,1.1,74.6,0,0,0,150.4,3.9,7.4,184,0.5,284.8,3217.60,0.011,3.1,399.7 197 | 195,195,pecans raw,753,78.4,6.7,44.5,23.6,15.1,4.3,10,10.5,0,0,3.8,0,0.7,0.019,0,0.1,1.3,0.9,0.2,1.2,0,1.5,0.033,76.3,1.3,2.8,131.9,4.9,301.9,446.9,0.039,4.9,194.3 198 | 196,196,chia seeds dried,138,8.7,0.9,0.7,6.7,11.9,0,4.7,9.7,0,0.047,1.6,0,0.2,0.098,0,0.080,2.5,0,0,0.5,0,0.1,0,178.6,0.3,2.2,94.8,0.8,243.4,115.2,0.035,1.3,216.3 199 | 197,197,japanese chestnuts roasted,57,0.2,0.080,0.1,0.094,12.8,0,0.8,0,0,0.079,14.1,0,0.1,0.022,0,0,0.2,0.1,0.1,7.9,0,0,0,9.9,0.2,0.6,18.1,0.6,26.3,120.8,0,0.4,32.2 200 | 198,198,sesame seeds toasted,45,3.8,0.5,1.5,1.7,2.1,0,1.4,1.1,0,0.070,0.3,0,0.084,0.075,0,0.022,0.4,0.074,0.002,0,0,0,0,79.1,0.2,1.2,28.5,0.2,51,38,0.030,0.6,88.7 201 | 199,199,cashew nuts roasted,174,14.3,2.5,7.8,2.6,9,1.5,5.1,1,0,0.091,0.7,0,0.1,0.080,0,0.025,0.5,0.3,0.056,0.071,0,0.3,0.041,12.9,0.6,1.8,81.9,0.5,159.3,189.6,0.020,1.6,44.12 202 | 200,200,lupins raw,668,17.5,2.1,7.1,4.4,72.7,0,65.1,34,0,0.009,18.8,0,1.2,0.6,0,0.4,3.9,1.4,0.6,8.6,0,0,0,316.8,1.8,7.8,356.4,4.3,792,1823.40,0.090,8.6,522.5 203 | 201,201,mustard seeds ground,10,0.7,0.012,0.5,0.2,0.6,0.1,0.5,0.2,0,0.017,0.1,0,0.072,0.093,0,0.020,0.093,0.018,0.000,0.1,0,0.1,0.086,5.3,0.055,0.2,7.4,0.078,16.6,14.8,0.003,0.1,7.6 204 | 202,202,mixed nuts roasted,178,15.4,2,9.4,3.2,7.6,1.4,5.2,2.7,0,0.1,0.5,0,0.063,0.008,0,0.074,1.4,0.4,0.073,0.1,0,3.3,0.017,21,0.4,1.1,67.5,0.6,130.5,207.9,0.053,1.1,53.1 205 | 203,203,salat mix alestro,61,5.2,0.9,0.1,0.3,2.4,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,7.6 206 | 204,204,chestnuts raw,309,3.3,0.6,1.1,1.3,66,0,3.5,11.7,0,0.020,70.5,0,0.3,0.030,0,0.2,1.7,0.7,0.5,62.4,0,0,0,39.2,0.6,1.5,46.4,1.4,134.9,751.1,0,0.8,187.6 207 | 205,205,pistachio nuts roasted,172,13.7,1.7,7.4,4,8.5,2.3,6.3,3.1,0,0.008,0.6,0,0,0.2,0.009,0,0.072,0.4,0.2,0.3,0.9,0,0.7,0.054,32.1,0.4,1.2,32.7,0.4,140.7,302.1,0.021,32.348 208 | 206,206,walnut,33,3.3,0.3,0.4,2.4,0.7,0.1,0.8,0.3,0,0.041,0.2,0,0.064,0.004,0,0.037,0.075,0.029,0.018,0.037,0,0.060,0.029,4.9,0.058,0.1,7.9,0.2,17.3,22.1,0.060,0.2,10.109 209 | 207,207,celery seeds,25,1.6,0.1,1,0.2,2.7,0.078,1.2,0.8,0,0.068,0.4,0,0.098,0.050,0,0.022,0.2,0,0.039,1.1,0,0.060,0,114.9,0.029,2.9,28.6,0.5,35.6,91,0.057,0.5,125.2 210 | 208,208,hickorynuts dried,788,77.2,8.4,39.1,26.3,21.9,0,15.3,7.7,0,0.053,3.2,0,1,0.042,0,0.2,1.1,2.1,0.2,2.4,0,0,0,73.2,0.9,2.5,207.6,5.5,403.2,523.2,0.038,5.2,200.2 211 | 209,209,spanish peanuts roasted,851,72.1,11.1,32.4,25,25.7,0,41.2,13.1,0,0.6,2.6,0,0.5,0.2,0,0.1,22,2,0.4,0,0,0,0,147,1,3.4,247,3.5,568.9,1140.70,0.083,2.9,302.5 212 | 210,210,woca seeds dried,102,0.3,0.025,0.044,0.059,22.6,0.1,2.2,5.4,0,0.048,2.6,0,0.029,0.092,0,0.042,1.2,0.4,0.3,0,0,10.1,0,7.6,0.041,0.4,24.3,0.3,64.8,156.5,0.021,1.8,38.5 213 | 211,211,pecans roasted,215,22.6,2.2,12.3,7.1,3.9,1.2,2.8,2.9,0,0.062,0.3,0,0.1,0.053,0,0.020,0.4,0.2,0.071,0.2,0,0.8,0,20.1,0.4,0.7,36.3,1.1,78.9,117.6,0.004,1.3,53.2 214 | 212,212,fennel seeds,7,0.3,0.008,0.2,0.065,1,0,0.3,0.8,0,0.097,0.2,0,0.021,0,0,0.092,0.1,0,0.045,0.4,0,0,0,23.9,0.051,0.4,7.7,0.1,9.7,33.9,0,0.042,27.1 215 | 213,213,chinese chestnuts roasted,68,0.3,0.027,0.2,0.073,14.8,0,1.3,0,0,0.052,11.4,0,0.036,0.100,0,0.037,0.4,0.2,0.1,10.9,0,0,0,5.4,0.1,0.4,25.5,0.5,28.9,135,0,0.3,33.1 216 | 214,214,watermelon seed kernels dried,602,51.2,10.6,8,30.3,16.5,0,30.6,0,0,0.1,5.5,0,0.2,0.038,0,0.2,3.8,0.4,0.032,0,0,0,0,58.3,0.7,7.9,556.2,1.7,815.4,699.8,0,11.1,164.5 217 | 215,215,macadamia nuts raw,38,4,0.6,3.1,0.059,0.7,0.2,0.4,0.5,0,0.011,0.011,0,0.032,0.061,0,0.042,0.1,0.066,0.035,0.062,0,0.058,0,4.5,0.009,0.2,6.8,0.2,9.9,19.3,0.064,0.034,10.387 218 | 216,216,pectin,163,0.2,0.052,0.093,0.076,45.2,0,0.2,4.3,0,0.1,4.4,0,0.042,0.025,0,0.021,0.017,0.023,0.063,0,0,0,0,3.5,0.2,1.4,0.5,0.094,1,3.5,0,0.2,54.8 219 | 217,217,black walnut dried,31,3,0.2,0.8,1.8,0.5,0.018,1.2,0.3,0,0.028,0.2,0,0.014,0.083,0,0.097,0.036,0.056,0.067,0.032,0,0.091,0.029,3.1,0.071,0.2,10.1,0.2,25.7,26.2,0.046,0.2,8.347 220 | 218,218,water chestnut canned,70,0.015,0.050,0.068,0.053,17.2,3.4,1.2,3.5,0,0.011,121,0,0.035,0.016,0,0.030,0.5,0.3,0.2,1.8,0,0.7,0.043,5.6,0.1,1.2,7,0.2,26.6,165.2,0.099,0.5,30.594 221 | 219,219,ginkgo nuts dried,98,0.6,0.1,0.2,0.2,20.5,0,2.9,0,0,0.045,3.5,0,0.1,0.054,0,0.010,3.3,0.4,0.2,8.3,0,0,0,5.7,0.2,0.5,15,0.059,76.1,282.4,0,0.2,38.5 222 | 220,220,virginia peanuts raw,822,71.2,9.3,36.9,21.5,24.1,5.8,36.8,12.4,0,0.008,10.1,0,1,0.3,0,0.2,18.1,2.6,0.5,0,0,9.6,0,129.9,1.6,3.7,249.7,2.5,554.8,1007.40,0.042,6.5,278.1 223 | 221,221,valencia peanuts roasted,177,15.4,2.4,6.9,5.3,4.9,0,8.1,2.7,0,0.004,0.6,0,0.052,0.003,0,0.068,4.3,0.4,0.072,0,0,0,0,16.2,0.3,0.5,48,0.5,95.7,183.6,0.046,0.9,47.8 224 | 222,222,nesfit diet cereal nestle,75,0,14.5,2.8,0,0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 225 | 223,223,corn grits cooked,182,1.2,0.2,0.2,0.4,37.9,0.3,4.4,2.1,0,0.6,213.1,0,0.2,0.002,0,0.1,2.1,0.1,0.1,0,0,0.030,0,2.6,0,1.5,18,0.008,51.4,69.4,0.052,0.5,49.7 226 | 224,224,amaranth flakes,134,2.7,0.5,0.8,1,27,0.2,5.9,3.6,0,0.067,1.1,0,0.033,0.025,0,0.000,1,0,0.006,1,0,0.5,0.007,6.5,0.011,0.7,9.5,0,125.8,134.1,0.009,0.011,47.4 227 | 225,225,granola,293,14.6,2.4,6.6,4.7,32.3,11.9,8.2,5.3,0,0.010,3.5,0.008,0.3,0.017,0,0.2,1.6,0.9,0.2,0.7,0,6.7,0.035,45.6,0.4,2.4,100.8,2.4,258.6,323.4,0.087,2.5,109.142 228 | 226,226,muesli with fruit nuts,231,3.4,0.5,1.2,1.4,45.7,14.2,4.3,5.8,0,0.1,5.1,0.2,0.4,177.6,0.047,0.5,5.5,2.8,0.5,0,0.028,1.4,0.038,25.8,0.2,5.9,72,1.3,176.4,229.2,0.083,1.6,91.1 229 | 227,227,muesli master crumble,124,2.4,1,19.8,8.5,4.5,0.042,2.4,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,9.3 230 | 228,228,bran flakes,131,0.8,0.2,0.1,0.5,32.2,7.4,4,7.3,0,0.2,1.4,0,0.3,0.5,0.4,0.011,0.6,6.7,0.3,0.7,0,0.090,0.3,0.025,17.6,0.3,11.2,91.6,1.4,179.6,213.2,0.083,45.358 231 | 229,229,nut cereal,245,5.9,1,3.9,0.5,46.2,17.2,4,3.6,0,0.3,2.6,0.2,0.5,0.3,0,0.6,6.5,0.3,0.7,10.7,0.002,0,0,75.6,0.2,10.3,39,1,106.8,134.4,0.066,3,156.5 232 | 230,230,corn flakes,108,0.3,0.098,0.071,0.018,24.6,2.2,1.7,0.8,0,0.2,0.9,0,0.3,1.3,0.026,0.056,0.5,5.9,0.022,0.5,18.2,0.011,0.011,0,0.6,0.083,5.4,2,0.060,9.2,30,0.077,27.947 233 | 231,231,corn grits dry,36,0.2,0.022,0.018,0.032,7.7,0.008,0.7,0.4,0,0.054,1.1,0,0.1,0.080,0,0.032,0.4,0.072,0.097,0,0,0.070,0,0.4,0.086,0.3,3.5,0.029,10.8,13.7,0.024,0.046,9.7 234 | -------------------------------------------------------------------------------- /examples/food_data_set/README.md: -------------------------------------------------------------------------------- 1 | # Food Data Set 2 | 3 | Data files downloaded from https://www.kaggle.com/datasets/utsavdey1410/food-nutrition-dataset, published 4 | under Creative Commons license 0: Public Domain. 5 | -------------------------------------------------------------------------------- /examples/food_data_set/food_data_merge.py: -------------------------------------------------------------------------------- 1 | # 2 | # food_data_merge.py 3 | # 4 | # Work with data files describing nutritional content of common foods, 5 | # downloaded from https://www.kaggle.com/datasets/utsavdey1410/food-nutrition-dataset. 6 | # 7 | # Rename columns to be valid Python identifiers, and re-rank so that 8 | # id's are unique. 9 | # 10 | 11 | from pathlib import Path 12 | 13 | import littletable as lt 14 | 15 | data_dir = Path(__file__).parent 16 | 17 | # read all CSVs into a single table - multiple calls to csv_import 18 | # on the same Table will concatenate all into a single group 19 | foods = lt.Table() 20 | for data_file in data_dir.glob("FOOD-DATA-GROUP*.csv"): 21 | foods.csv_import(data_file, transforms={"*": lt.Table.convert_numeric}) 22 | 23 | # re-number food items into an "id" field 24 | foods.rank("id") 25 | 26 | # convert field names with capital letters and spaces to snake case, 27 | # to make them easier to work with as valid Python identifiers 28 | field_names = foods.info()["fields"] 29 | py_names = [nm.lower().replace(" ", "_") for nm in field_names] 30 | for py_name, field_name in zip(py_names, field_names): 31 | if not py_name: 32 | continue 33 | foods.compute_field(py_name, field_name) 34 | 35 | # change the initial field name from "" to "id", and remove "unnamed" field 36 | py_names[0] = "id" 37 | py_names.remove("unnamed:_0") 38 | 39 | # build single CSV file from all data 40 | foods.select(py_names).csv_export(data_dir / "combined_food_data.csv") 41 | -------------------------------------------------------------------------------- /examples/full_text_search.py: -------------------------------------------------------------------------------- 1 | # 2 | # full_text_search.py 3 | # 4 | # Example code using littletable with a search index to do full text searching 5 | # of a table attribute. 6 | # 7 | # Copyright (c) 2020 Paul T. McGuire 8 | # 9 | 10 | import littletable as lt 11 | import textwrap 12 | 13 | # create table of recipes, by title and ingredients 14 | recipe_data = textwrap.dedent( 15 | """\ 16 | title,ingredients 17 | Tuna casserole,tuna noodles cream of mushroom soup 18 | Hawaiian pizza,pizza dough pineapple ham tomato sauce 19 | Margherita pizza,pizza dough cheese pesto artichoke hearts 20 | Pepperoni pizza,pizza dough cheese tomato sauce pepperoni 21 | Grilled cheese sandwich,bread cheese butter 22 | Tuna melt,tuna mayonnaise tomato bread cheese 23 | Chili dog,hot dog chili onion bun 24 | French toast,egg milk vanilla bread maple syrup 25 | BLT,bread bacon lettuce tomato mayonnaise 26 | Reuben sandwich,rye bread sauerkraut corned beef swiss cheese russian dressing thousand island 27 | Hamburger,ground beef bun lettuce ketchup mustard pickle 28 | Cheeseburger,ground beef bun lettuce ketchup mustard pickle cheese 29 | Bacon cheeseburger,ground beef bun lettuce ketchup mustard pickle cheese bacon 30 | """ 31 | ) 32 | recipes = lt.Table().csv_import(recipe_data) 33 | 34 | # define search index on "ingredients" attribute 35 | search_attr = "ingredients" 36 | recipes.create_search_index(search_attr) 37 | 38 | # run sample queries 39 | queries = """\ 40 | tuna 41 | tuna +cheese 42 | pineapple +bacon lettuce beef -sauerkraut tomato 43 | pizza dough -pineapple 44 | pizza dough --pineapple 45 | bread bacon 46 | bread ++bacon""".splitlines() 47 | 48 | # run each query, listing top 5 matches 49 | for query in queries: 50 | query = query.strip() 51 | if not query: 52 | continue 53 | 54 | matches = recipes.search.ingredients(query, limit=5, min_score=-100000) 55 | 56 | print(query) 57 | if matches: 58 | for rec in matches: 59 | print(" -", rec.title, rec.ingredients_search_score) 60 | else: 61 | print(" ") 62 | print() 63 | 64 | # redo last match, getting the words for each match 65 | print("repeat last match, including matched words") 66 | matches = recipes.search.ingredients( 67 | query, limit=5, min_score=-100000, include_words=True 68 | ) 69 | print(query) 70 | for rec in matches: 71 | print(" -", rec.title,rec.ingredients_search_score, rec.ingredients_search_words) 72 | 73 | # build a more complex search index, using words from multiple fields 74 | recipes.create_search_index("recipe_terms", using="ingredients title") 75 | matches = recipes.search.recipe_terms("casserole Hawaiian", as_table=True) 76 | matches.present(fields="title ingredients".split()) 77 | 78 | # exception gets raised if search() is called after the table has been modified without 79 | # rebuilding the search index 80 | print("\nmodify source table to show exception raised when search index is no longer valid") 81 | recipes.pop(0) 82 | matches = recipes.search.ingredients(query) 83 | -------------------------------------------------------------------------------- /examples/littletable_demo.py: -------------------------------------------------------------------------------- 1 | # 2 | # littletable_demo.py 3 | # 4 | # Copyright 2010, Paul T. McGuire 5 | # 6 | from collections import namedtuple 7 | import littletable as lt 8 | 9 | # load catalog of products 10 | catalog = lt.Table("ACME catalog") 11 | catalog.create_index("sku", unique=True) 12 | catalog_data = """\ 13 | sku,descr,unitofmeas,unitprice 14 | BRDSD-001,Bird seed,LB,3 15 | BBS-001,Steel BB's,LB,5 16 | MGNT-001,Magnet,EA,8 17 | MAGLS-001,Magnifying glass,EA,12 18 | ANVIL-001,1000lb anvil,EA,100 19 | ROPE-001,1 in. heavy rope,100FT,10 20 | ROBOT-001,Domestic robot,EA,5000""" 21 | catalog.csv_import(catalog_data, transforms={"unitprice": int}) 22 | catalog.present() 23 | 24 | print(catalog.by.sku["ANVIL-001"].descr) 25 | catalog.create_index("unitofmeas") 26 | catalog.by.unitofmeas["EA"].present() 27 | 28 | # load table of customers 29 | Customer = namedtuple("Customer", "id name") 30 | customers = lt.Table("customers") 31 | customers.create_index("id", unique=True) 32 | customer_data = """\ 33 | id,name 34 | 0010,George Jetson 35 | 0020,Wile E. Coyote 36 | 0030,Jonny Quest""" 37 | customers.csv_import(customer_data, row_class=Customer) 38 | customers.present() 39 | 40 | # load wishlist items for each customer 41 | wishitems = lt.Table("wishitems") 42 | wishitems.create_index("custid") 43 | wishitems.create_index("sku") 44 | 45 | # there is no user-defined type for these items, the default is SimpleNamespace 46 | wishlist_data = """\ 47 | custid,sku 48 | 0030,MAGLS-001 49 | 0020,MAGLS-001 50 | 0020,ANVIL-001 51 | 0020,ROPE-001 52 | 0020,BRDSD-001 53 | 0020,BBS-001 54 | 0020,MAGNT-001 55 | 0030,MAGNT-001 56 | 0030,ROBOT-001 57 | 0010,ROBOT-001""" 58 | wishitems.csv_import(wishlist_data) 59 | 60 | # print a particular customer name 61 | print(customers.by.id["0030"].name) 62 | print() 63 | 64 | # print all items sold by the pound 65 | for item in catalog.where(unitofmeas="LB"): 66 | print(item.sku, item.descr) 67 | print() 68 | 69 | # if querying on an indexed item, use ".by.attribute-name[key]" 70 | # catalog.create_index("unitofmeas") 71 | for item in catalog.by.unitofmeas["LB"]: 72 | print(item.sku, item.descr) 73 | print() 74 | 75 | # print all items that cost more than 10 76 | for item in catalog.where(lambda ob: ob.unitprice > 10): 77 | print(item.sku, item.descr, item.unitprice) 78 | print() 79 | 80 | # join tables to create queryable wishlists collection - the following are all equivalent 81 | wishlists = (customers.join_on("id") + wishitems.join_on("custid")).join_on("sku") + catalog.join_on("sku") 82 | wishlists = (customers.join_on("id") + wishitems.join_on("custid")).join_on("sku") + catalog 83 | wishlists = catalog + (customers.join_on("id") + wishitems.join_on("custid")).join_on("sku") 84 | wishlists = catalog.join_on("sku") + (customers.join_on("id") + wishitems.join_on("custid")) 85 | wishlists = customers.join_on("id") + wishitems.join_on("custid") + catalog.join_on("sku") 86 | print(wishlists().table_name) 87 | print(wishlists()("wishlists").table_name) 88 | 89 | # print all wishlist items with price > 10 (use Table.gt instead of lambda) 90 | # bigticketitems = wishlists().where(lambda ob : ob.unitprice > 10) 91 | bigticketitems = wishlists().where(unitprice=lt.Table.gt(10)) 92 | for bti in bigticketitems: 93 | print(bti) 94 | print() 95 | 96 | # list all wishlist items by customer, then in descending order by unit price 97 | for item in wishlists().orderby("custid, unitprice desc"): 98 | print(item) 99 | print() 100 | 101 | # display formatted tabular output 102 | wishlists().orderby("custid, unitprice desc")("Wishlists").select( 103 | "custid name sku descr" 104 | ).present(groupby="custid name") 105 | 106 | # create simple pivot table, grouping wishlist data by customer name 107 | wishlistsdata = wishlists() 108 | wishlistsdata.create_index("name") 109 | pivot = wishlistsdata.pivot("name") 110 | pivot.dump(row_fn=lambda o: "%s %s" % (o.sku, o.descr)) 111 | print() 112 | 113 | # pivot on both sku number and customer name, giving tabular output 114 | piv2 = wishlistsdata.pivot("sku name") 115 | piv2.dump_counts() 116 | print() 117 | 118 | # pivot on both sku number and customer name, giving tabular output 119 | # tabulate by sum(unitprice) for all items in each pivot table cell 120 | piv2.dump_counts(count_fn=lambda recs: sum(r.unitprice for r in recs)) 121 | print() 122 | -------------------------------------------------------------------------------- /examples/nfkc_normalization.py: -------------------------------------------------------------------------------- 1 | # NFKC_normalization.py 2 | # 3 | # List out Unicode characters that normalize to the ASCII character set. 4 | # 5 | # Copyright Paul McGuire, 2022 6 | # 7 | import sys 8 | import unicodedata 9 | 10 | import littletable as lt 11 | 12 | ALL_UNICODE_CHARS = False 13 | 14 | # initialize accumulator for ASCII character to collect Unicode characters 15 | # that normalize back to ASCII characters that can be used in Python 16 | # identifiers (A-Z, a-z, 0-9, _, and ·) 17 | _· = "_·" 18 | ident_chars = ("ABCDEFGHIJKLMNOPQRSTUVWXYZ" 19 | "abcdefghijklmnopqrstuvwxyz" 20 | "0123456789" + _·) 21 | accum = {ch: [] for ch in ident_chars} 22 | 23 | # build up accumulator by walking Unicode range 24 | unicode_upper_limit = sys.maxunicode+1 if ALL_UNICODE_CHARS else 65536 25 | for i in range(32, unicode_upper_limit): 26 | ch = chr(i) 27 | norm = unicodedata.normalize("NFKC", ch) 28 | if norm in accum: 29 | accum[norm].append(ch) 30 | 31 | # convert accumulator to a littletable Table for presentation 32 | normalizations = lt.Table() 33 | for asc_char, normalizing_chars in accum.items(): 34 | normalizations.insert_many( 35 | {"ASCII": asc_char, 36 | "ord": ord(asc_char), 37 | "Unicode": norm_char, 38 | "code_point": ord(norm_char), 39 | "name": unicodedata.name(norm_char), 40 | } 41 | for norm_char in normalizing_chars 42 | ) 43 | normalizations.orderby("ASCII") 44 | normalizations.present(groupby="ASCII ord") 45 | -------------------------------------------------------------------------------- /examples/peps.json.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ptmcg/littletable/77c64149e778923ce1feea141b279710243393a5/examples/peps.json.zip -------------------------------------------------------------------------------- /examples/peps.py: -------------------------------------------------------------------------------- 1 | # 2 | # peps.py 3 | # 4 | # Using PEP data extracted from Python.org, this program demonstrates 5 | # pulling data from a compressed JSON file, and accessing PEP entries 6 | # by id, year created, and text search of the PEP abstracts. 7 | # 8 | # Copyright Paul McGuire, 2022 9 | # 10 | import json 11 | from operator import attrgetter 12 | from pathlib import Path 13 | import littletable as lt 14 | 15 | 16 | # import PEP data from JSON, converting id's to ints and created 17 | # date stings to Python datetimes 18 | peps = lt.Table().json_import( 19 | Path(__file__).parent / "peps.json.zip", 20 | transforms={ 21 | "id": int, 22 | "created": lt.Table.parse_date("%d-%b-%Y"), 23 | } 24 | ) 25 | 26 | # print metadata of imported records 27 | print(peps.info()) 28 | 29 | # access records by unique PEP id 30 | peps.create_index("id", unique=True) 31 | print("PEP20:", peps.by.id[20].title) 32 | 33 | # add a numeric "year" field, and index it 34 | get_year = attrgetter("created.year") 35 | peps.compute_field("year", get_year) 36 | peps.create_index("year") 37 | 38 | # present PEPs created in 2016 39 | peps.by.year[2016]("PEPs Created in 2016").select("id python_version title status url").present() 40 | 41 | # how many PEPs since 2020? 42 | print("Number of PEPs since 2020", len(peps.by.year[2020:])) 43 | print() 44 | 45 | # pivot by year and dump counts, or present as nice table 46 | peps.pivot("year").dump_counts() 47 | peps.pivot("year").as_table().present() 48 | 49 | # create full text search on PEP abstracts 50 | peps.create_search_index("abstract_terms", using="abstract title authors status year") 51 | 52 | # search for PEPs referring to the walrus operator 53 | walrus_pep = peps.search.abstract_terms("walrus", as_table=True)("'walrus' Search Results") 54 | walrus_pep.select("id title year authors").present() 55 | print(walrus_pep.select("id title year authors").json_export()) 56 | 57 | # search for PEPs referring to GvR or Guido or BDFL 58 | bdfl_peps = peps.search.abstract_terms("gvr guido bdfl", as_table=True)("GvR PEPs") 59 | bdfl_peps.orderby("id") 60 | bdfl_peps.select("id title year url authors").present() 61 | 62 | 63 | # define a custom JSON encoder for datetime.date field 64 | class JsonDateEncoder(json.JSONEncoder): 65 | def default(self, o): 66 | import datetime 67 | if isinstance(o, datetime.date): 68 | return str(o) 69 | return super().default(o) 70 | 71 | 72 | print(bdfl_peps.select("id title created").json_export(json_encoder=(JsonDateEncoder,))) 73 | -------------------------------------------------------------------------------- /examples/pivot_demo.py: -------------------------------------------------------------------------------- 1 | # 2 | # pivot_demo.py 3 | # 4 | # Using littletable, performs summaries on one and two attribute pivots 5 | # of a table of US populated place names. 6 | # 7 | # Uses data excerpted from the US.TXT geographic names data file, provided 8 | # under Creative Commons Attribution 3.0 License by GeoNames (www.geonames.org) 9 | # 10 | # Copyright (c) 2011, 2024 Paul T. McGuire 11 | # 12 | import littletable as lt 13 | from pathlib import Path 14 | 15 | 16 | this_dir = Path(__file__).parent 17 | 18 | # import from csv, convert elevation from meters to feet 19 | places: lt.Table = lt.csv_import( 20 | this_dir / "us_ppl.zip", 21 | transforms={ 22 | "elev": lambda s: int(s) * 33 / 10, 23 | "pop": int 24 | }, 25 | ) 26 | print(places.info()) 27 | 28 | # add computed field, elevation rounded down by 1000's 29 | places.compute_field("elev_000", lambda x: int(x.elev / 1000) * 1000, 0) 30 | 31 | # create indexes for use by pivots 32 | places.create_index("state") 33 | places.create_index("elev_000") 34 | 35 | print("summarize population by state") 36 | piv = places.pivot("state") 37 | ppl_by_state = piv.as_table(sum, "pop").orderby("pop desc") 38 | for rec in ppl_by_state: 39 | print(rec.state, rec.pop) 40 | 41 | print() 42 | piv.dump_counts(count_fn=lambda recs: sum(r.pop for r in recs)) 43 | 44 | print() 45 | print("summarize population by elevation") 46 | piv = places.pivot("elev_000") 47 | ppl_by_elev = piv.as_table(sum, "pop") 48 | for rec in ppl_by_elev: 49 | print(rec.elev_000, rec.pop) 50 | 51 | print() 52 | print("summarize population by state and elevation") 53 | piv = places.pivot("state elev_000") 54 | 55 | # dump all the sum of all population attributes for each row in each subtable 56 | piv.dump_counts(count_fn=lambda recs: sum(r.pop for r in recs)) 57 | print() 58 | 59 | # select a subtable from the pivot table 60 | sd0 = piv["SD"][0]("SD places below 1000 ft elev") 61 | sd0.select("name pop elev elev_000").present() 62 | print() 63 | 64 | # pplByElev = piv.as_table(sum, "pop") 65 | # for rec in pplByElev[:100]: 66 | # print(rec.state, rec.elev_000, rec.pop) 67 | 68 | # find average elevation of person by state 69 | print("Average elevation of each person by state") 70 | piv = places.pivot("state") 71 | piv.dump_counts( 72 | count_fn=lambda recs: int( 73 | sum(r.pop * r.elev for r in recs) / sum(r.pop for r in recs) 74 | ) 75 | ) 76 | 77 | low_liers = places.where(elev=lt.Table.le(20)) 78 | print(f"\nPopulation at or below 20 feet sea level: {sum(low_liers.all.pop):,}") -------------------------------------------------------------------------------- /examples/present_groupby_example.py: -------------------------------------------------------------------------------- 1 | import littletable as lt 2 | 3 | tbl = lt.Table("Academy Awards 1960-1969").csv_import("""\ 4 | year,award,movie,recipient 5 | 1960,Best Picture,Ben-Hur, 6 | 1960,Best Actor,Ben-Hur,Charlton Heston 7 | 1960,Best Actress,The Heiress,Simone Signoret 8 | 1960,Best Director,Ben-Hur,William Wyler 9 | 1961,Best Picture,The Apartment, 10 | 1961,Best Actor,Elmer Gantry,Burt Lancaster 11 | 1961,Best Actress,Butterfield 8,Elizabeth Taylor 12 | 1961,Best Director,The Apartment,Billy Wilder 13 | 1962,Best Picture,West Side Story, 14 | 1962,Best Actor,Judgment at Nuremberg,Maximilian Schell 15 | 1962,Best Actress,Two Women,Sophia Loren 16 | 1962,Best Director,West Side Story,Willian Wise/Jerome Robbins 17 | 1963,Best Picture,Lawrence of Arabia, 18 | 1963,Best Actor,To Kill A Mockingbird,Gregory Peck 19 | 1963,Best Actress,The Miracle Worker,Anne Bancroft 20 | 1963,Best Director,Lawrence of Arabia,David Lean 21 | 1964,Best Picture,Tom Jones, 22 | 1964,Best Actor,Lilies of the Field,Sidney Poitier 23 | 1964,Best Actress,Hud,Patricia Neal 24 | 1964,Best Director,Tom Jones,Tony Richardson 25 | 1965,Best Picture,My Fair Lady, 26 | 1965,Best Actor,My Fair Lady,Rex Harrison 27 | 1965,Best Actress,Mary Poppins,Julie Andrews 28 | 1965,Best Director,My Fair Lady,George Kukor 29 | 1966,Best Picture,The Sound of Music, 30 | 1966,Best Actor,Cat Ballou,Lee Marvin 31 | 1966,Best Actress,Darling,Julie Christie 32 | 1966,Best Director,The Sound of Music,Robert Wise 33 | 1967,Best Picture,A Man for All Season, 34 | 1967,Best Actor,A Man for All Seasons,Paul Scofield 35 | 1967,Best Actress,Who's Afraid of Virginia Woolf,Elizabeth Taylor 36 | 1967,Best Director,A Man for All Seasons,Fred Zinnemann 37 | 1968,Best Picture,In The Heat of The Night, 38 | 1968,Best Actor,In The Heat of The Night,Rod Steiger 39 | 1968,Best Actress,Guess Who's Coming to Dinner,Katherine Hepburn 40 | 1968,Best Director,The Graduate,Mike Nichols 41 | 1969,Best Picture,Oliver!, 42 | 1969,Best Actor,Charly,Cliff Robertson 43 | 1969,Best Actress,Funny Girl,Barbra Streisand 44 | 1969,Best Director,Oliver!,Carol Reed 45 | """) 46 | 47 | tbl.present(groupby="year") 48 | -------------------------------------------------------------------------------- /examples/pyscript/matplotlib.html: -------------------------------------------------------------------------------- 1 | 9 | 10 | 11 | 12 | 13 | 14 | Pyscript + littletable Demo 15 | 16 | 17 | 18 | 19 | packages = [ 20 | "matplotlib", 21 | "littletable", 22 | ] 23 | 24 | 25 | 26 | 27 | 34 | 114 | 115 | 
116 |     


117 |
118 |
119 | 120 | 121 | -------------------------------------------------------------------------------- /examples/sliced_indexing.py: -------------------------------------------------------------------------------- 1 | # sliced_indexing.py 2 | # 3 | # Demonstration of using slices on indexed attributes to do range filtering. 4 | # Even non-integer types are supported (as long as they support >= and < 5 | # operations). 6 | # 7 | import littletable as lt 8 | import textwrap 9 | import datetime 10 | 11 | 12 | def main(): 13 | 14 | # read data from US place names CSV file 15 | transforms = { 16 | 'pop': int, 17 | # convert from meters to feet 18 | 'elev': lambda s: int(float(s) * 3.28084), 19 | 'lat': float, 20 | 'long': float, 21 | } 22 | us_ppl = lt.Table().csv_import( 23 | "./us_ppl.zip", 24 | transforms=transforms 25 | ).select("id name state elev lat long pop") 26 | 27 | print(us_ppl.info()) 28 | us_ppl.create_index("elev") 29 | 30 | test = "elev < 0" 31 | us_ppl.by.elev[:0](f"{test} (sliced)")[:100].present() 32 | 33 | test = "elev >= 1000" 34 | us_ppl.by.elev[1000:](f"{test} (sliced)").orderby("elev desc")[:100].present() 35 | 36 | test = "0 <= elev < 100" 37 | us_ppl.by.elev[0:100](f"{test} (sliced)")[:100].present() 38 | 39 | # slice using non-integer types 40 | 41 | us_ppl.create_index("name") 42 | a_ppl_slice = us_ppl.by.name["A":"C"] 43 | a_ppl_slice[:100].present() 44 | 45 | a_ppl_slice = us_ppl.by.name["Z":] 46 | a_ppl_slice[:100].present() 47 | 48 | # load some sales data from Acme, Inc. 49 | sales_data = textwrap.dedent("""\ 50 | date,customer,sku,qty 51 | 2000/01/01,0020,ANVIL-001,1 52 | 2000/01/01,0020,BRDSD-001,5 53 | 2000/02/15,0020,BRDSD-001,5 54 | 2000/03/31,0020,BRDSD-001,5 55 | 2000/03/31,0020,MAGNT-001,1 56 | 2000/04/01,0020,ROBOT-001,1 57 | 2000/04/15,0020,BRDSD-001,5 58 | 1900/02/29,0020,BRDSD-001,5 59 | """) 60 | 61 | # load data from CSV, converting dates to datetime.date 62 | transforms = {'date': lt.Table.parse_date("%Y/%m/%d"), 63 | 'qty': int} 64 | sales = lt.Table("All 2000 Sales").csv_import( 65 | sales_data, 66 | transforms=transforms, 67 | ) 68 | sales.present() 69 | 70 | # get sales from the first quarter only 71 | sales.create_index("date") 72 | jan_01 = datetime.date(2000, 1, 1) 73 | apr_01 = datetime.date(2000, 4, 1) 74 | first_qtr_sales = sales.by.date[jan_01: apr_01]("2000 Q1 Sales") 75 | first_qtr_sales.present() 76 | 77 | # load data from CSV, leave dates as strings (still sortable when in YYYY/MM/DD form) 78 | transforms = {'qty': int} 79 | sales = lt.Table("All 2000 Sales").csv_import( 80 | sales_data, 81 | transforms=transforms, 82 | ) 83 | 84 | # get sales from the first quarter only 85 | sales.create_index("date") 86 | first_qtr_sales = sales.by.date["2000/01/01": "2000/04/01"]("2000 Q1 Sales") 87 | first_qtr_sales.present() 88 | 89 | 90 | if __name__ == '__main__': 91 | main() 92 | -------------------------------------------------------------------------------- /examples/star_trek_tos.py: -------------------------------------------------------------------------------- 1 | import littletable as lt 2 | 3 | st = lt.Table().csv_import( 4 | "star_trek_tos_eps.csv", 5 | transforms={"rating": float, "votes": int} 6 | ) 7 | 8 | # sort by rating, and add "rank" field 9 | st.orderby(["rating desc", "votes desc"]) 10 | st.rank("rank") 11 | 12 | # display 10 best and worst episodes 13 | fields = "rank date title rating votes".split() 14 | count = 10 15 | best_and_worst = (st[:count] + st[-count:])(f"{count} Best and Worst Star Trek Episodes") 16 | best_and_worst.present( 17 | fields, 18 | caption="data from IMDB.com", 19 | caption_justify="right" 20 | ) 21 | 22 | # full-text search of descriptions 23 | st.create_search_index("description") 24 | for query in [ 25 | "--kirk ++spock", 26 | "--kirk ++mccoy", 27 | ]: 28 | st.search.description(query, as_table=True).select("date title description").present() 29 | -------------------------------------------------------------------------------- /examples/star_trek_tos_eps.csv: -------------------------------------------------------------------------------- 1 | date,description,href,rating,season,title,votes 2 | 1988-11-27,Capt. Pike is held prisoner and tested by aliens who have the power to project incredibly lifelike illusions.,/title/tt0059753/?ref_=ttep_ep1,7.7,1,The Cage,5100 3 | 1966-09-08,Dr. McCoy discovers his old flame is not what she seems after crew members begin dying from a sudden lack of salt in their bodies.,/title/tt0708469/?ref_=ttep_ep2,7.3,1,The Man Trap,3892 4 | 1966-09-15,Captain Kirk must learn the limits to the power of a 17-year-old boy with the psychic ability to create anything and destroy anyone.,/title/tt0708424/?ref_=ttep_ep3,7.1,1,Charlie X,3670 5 | 1966-09-22,The flight recorder of the 200-year-old U.S.S. Valiant relays a tale of terror--a magnetic storm at the edge of the galaxy!,/title/tt0061027/?ref_=ttep_ep4,7.8,1,Where No Man Has Gone Before,4570 6 | 1966-09-29,The crew is infected with a mysterious disease that removes people's emotional inhibitions to a dangerous degree.,/title/tt0708473/?ref_=ttep_ep5,7.9,1,The Naked Time,3445 7 | 1966-10-06,"A transporter malfunction splits Captain Kirk into two halves: one meek and indecisive, the other violent and ill tempered. The remaining crew members stranded on the planet cannot be beamed up to the ship until a problem is fixed.",/title/tt0708463/?ref_=ttep_ep6,7.7,1,The Enemy Within,3282 8 | 1966-10-13,The Enterprise picks up untrustworthy entrepreneur Harry Mudd accompanied by three beautiful women who immediately put a spell on all the male crew members.,/title/tt0708439/?ref_=ttep_ep7,6.8,1,Mudd's Women,3210 9 | 1966-10-20,Nurse Chapel is reunited with her fianc�; but his new obsession leads him to make an android duplicate of Captain Kirk.,/title/tt0708486/?ref_=ttep_ep8,7.5,1,What Are Little Girls Made Of?,3056 10 | 1966-10-27,"The Enterprise discovers a planet exactly like Earth, but the only inhabitants are children who contract a fatal disease upon entering puberty.",/title/tt0394905/?ref_=ttep_ep9,7.1,1,Miri,3038 11 | 1966-11-03,Kirk and psychiatrist Helen Noel are trapped on a maximum security penal colony that experiments with mind control and Spock must use the Vulcan mind-meld to find a way to save them.,/title/tt0708426/?ref_=ttep_ep10,7.5,1,Dagger of the Mind,2870 12 | 1966-11-10,"After the Enterprise is forced to destroy a dangerous marker buoy, a gigantic alien ship arrives to capture and condemn the crew as trespassers.",/title/tt0708458/?ref_=ttep_ep11,8.1,1,The Corbomite Maneuver,3219 13 | 1966-11-17,"Spock kidnaps the crippled Capt. Pike, hijacks the Enterprise and then surrenders for court martial.",/title/tt0394904/?ref_=ttep_ep12,8.4,1,The Menagerie: Part I,3171 14 | 1966-11-24,"At Spock's court martial, he explains himself with mysterious footage about when Capt. Pike was kidnapped by powerful illusion casting aliens.",/title/tt0708472/?ref_=ttep_ep13,8.3,1,The Menagerie: Part II,3073 15 | 1966-12-08,"While Captain Kirk investigates whether an actor is actually a presumed dead mass murderer, a mysterious assailant is killing the people who could identify the fugitive.",/title/tt0394903/?ref_=ttep_ep14,7.2,1,The Conscience of the King,2789 16 | 1966-12-15,The Enterprise must decide on its response when a Romulan ship makes a destructively hostile armed probe of Federation territory.,/title/tt0708420/?ref_=ttep_ep15,8.9,1,Balance of Terror,3710 17 | 1966-12-29,"The past three months has left the crew of the Enterprise exhausted and in desperate need of a break, but does this explain McCoy's encounter with a human-sized white rabbit or Kirk crossing paths with the prankster who plagued his days at Starfleet Academy?",/title/tt0708446/?ref_=ttep_ep16,7.6,1,Shore Leave,2957 18 | 1967-01-05,"The Galileo, under Spock's command, crash-lands on a hostile planet. As the Enterprise races against time to find the shuttlecraft, Spock's strictly logical leadership clashes with the fear and resentment of his crew.",/title/tt0708465/?ref_=ttep_ep17,7.7,1,The Galileo Seven,2834 19 | 1967-01-12,A being that controls matter and creates planets wants to play with the Enterprise crew.,/title/tt0708478/?ref_=ttep_ep18,7.4,1,The Squire of Gothos,2770 20 | 1967-01-19,"For bringing hostility into their solar system, a superior alien race brings Captain Kirk in mortal combat against the reptilian captain of an alien ship he was pursuing.",/title/tt0708418/?ref_=ttep_ep19,8.0,1,Arena,3005 21 | 1967-01-26,The Enterprise is thrown back in time to 1960s Earth.,/title/tt0708484/?ref_=ttep_ep20,8.0,1,Tomorrow Is Yesterday,2854 22 | 1967-02-02,Kirk draws a court martial in the negligent death of a crewman.,/title/tt0708425/?ref_=ttep_ep21,7.5,1,Court Martial,2650 23 | 1967-02-09,"Seeking the answer to a century-old mystery, Kirk and crew encounter a vacantly peaceful society under a 6000-year autocratic rule that kills all those it can't absorb.",/title/tt0708476/?ref_=ttep_ep22,7.2,1,The Return of the Archons,2573 24 | 1967-02-16,"While on patrol in deep space, Captain Kirk and his crew find and revive a genetically engineered world conqueror and his compatriots from Earth's 20th century.",/title/tt0708447/?ref_=ttep_ep23,8.8,1,Space Seed,3647 25 | 1967-02-23,Kirk and Spock must save their ship's crew when they are declared all killed in action in a bizarre computer simulated war where the actual deaths must occur to continue.,/title/tt0708414/?ref_=ttep_ep24,8.1,1,A Taste of Armageddon,2773 26 | 1967-03-02,"The Enterprise investigates a planet whose colonists should be dead, but are not.",/title/tt0708483/?ref_=ttep_ep25,8.0,1,This Side of Paradise,2712 27 | 1967-03-09,"The Enterprise is sent to a mining colony that is being terrorized by a mysterious monster, only to find that the situation is not that simple.",/title/tt0708460/?ref_=ttep_ep26,8.4,1,The Devil in the Dark,2925 28 | 1967-03-23,"With a war with Klingons raging, Kirk and Spock attempt to resist an occupation of a planet with incomprehensibly placid natives.",/title/tt0708429/?ref_=ttep_ep27,8.2,1,Errand of Mercy,2668 29 | 1967-03-30,"Existence itself comes under threat from a man's power-struggle with his alternate self, with the Enterprise's strained dilithium crystals presenting his key to a final solution.",/title/tt0708451/?ref_=ttep_ep28,5.8,1,The Alternative Factor,2601 30 | 1967-04-06,"When a temporarily insane Dr. McCoy accidentally changes history and destroys his time, Kirk and Spock follow him to prevent the disaster, but the price to do so is high.",/title/tt0708455/?ref_=ttep_ep29,9.3,1,The City on the Edge of Forever,4393 31 | 1967-04-13,The Enterprise crew attempts to stop a plague of amoeba-like creatures from possessing human hosts and spreading throughout the galaxy.,/title/tt0708441/?ref_=ttep_ep30,7.6,1,Operation - Annihilate!,2430 32 | 1967-09-15,"In the throes of his Pon Farr mating period, Spock must return to Vulcan to meet his intended future wife, betrothed from childhood.",/title/tt0708416/?ref_=ttep_ep1,8.7,2,Amok Time,3175 33 | 1967-09-22,A powerful being claiming to be the Greek god Apollo appears and demands that the crew of the Enterprise disembark onto his planet to worship him.,/title/tt0708488/?ref_=ttep_ep2,7.2,2,Who Mourns for Adonais?,2674 34 | 1967-09-29,"A powerful artificially intelligent Earth probe, with a murderously twisted imperative, comes aboard the Enterprise and mistakes Capt. Kirk for its creator.",/title/tt0708454/?ref_=ttep_ep3,7.7,2,The Changeling,2422 35 | 1967-10-06,"A transporter accident places Captain Kirk's landing party in an alternate universe, where the Enterprise is in the service of a barbarically brutal empire.",/title/tt0708438/?ref_=ttep_ep4,9.1,2,"Mirror, Mirror",3434 36 | 1967-10-13,"Primitive inhabitants of Gamma Trianguli VI worship a God who orders them to kill visitors, from the Enterprise.",/title/tt0708452/?ref_=ttep_ep5,6.4,2,The Apple,2342 37 | 1967-10-20,The USS Enterprise encounters the wrecked USS Constellation and its distraught commodore who's determined to stop the giant planet-destroying robot ship that killed his crew.,/title/tt0708461/?ref_=ttep_ep6,8.8,2,The Doomsday Machine,3039 38 | 1967-10-27,"Very alien visitors to our galaxy attempt to connect with human consciousness but miss, winding up tapping into the regions of human nightmares instead.",/title/tt0708423/?ref_=ttep_ep7,6.3,2,Catspaw,2379 39 | 1967-11-03,Harry Mudd returns with a plot to take over the Enterprise by stranding the crew on a planet populated by androids under his command.,/title/tt0708432/?ref_=ttep_ep8,7.5,2,"I, Mudd",2480 40 | 1967-11-10,"While returning to the Enterprise aboard the shuttlecraft, Kirk, Spock, McCoy and a seriously ill Federation diplomat find themselves kidnapped by an energized cloud.",/title/tt0708436/?ref_=ttep_ep9,7.3,2,Metamorphosis,2399 41 | 1967-11-17,"The Enterprise hosts a number of quarrelling diplomats, including Spock's father, but someone on board has murder in mind.",/title/tt0708434/?ref_=ttep_ep10,8.6,2,Journey to Babel,2663 42 | 1967-12-01,The Federation clashes with the Klingon Empire over mining rights to Capella IV. A sudden coup between its warrior-minded inhabitants forces Kirk's party to flee with the now dead leader's pregnant wife.,/title/tt0708431/?ref_=ttep_ep11,6.9,2,Friday's Child,2249 43 | 1967-12-08,A landing party from the Enterprise is exposed to strange form of radiation which rapidly ages them.,/title/tt0708459/?ref_=ttep_ep12,7.4,2,The Deadly Years,2256 44 | 1967-12-15,Capt. Kirk obsessively hunts for a mysterious cloud creature he encountered in his youth.,/title/tt0708440/?ref_=ttep_ep13,7.3,2,Obsession,2173 45 | 1967-12-22,Kirk and the Enterprise computer become detectives after Scotty is accused of murdering women on a pleasure planet.,/title/tt0708491/?ref_=ttep_ep14,7.4,2,Wolf in the Fold,2276 46 | 1967-12-29,"To protect a space station with a vital grain shipment, Kirk must deal with Federation bureaucrats, a Klingon battle cruiser and a peddler who sells furry, purring, hungry little creatures as pets.",/title/tt0708480/?ref_=ttep_ep15,8.9,2,The Trouble with Tribbles,3337 47 | 1968-01-05,"Kirk, Uhura and Chekov are trapped on a planet where abducted aliens are enslaved and trained to perform as gladiators for the amusement of bored, faceless aliens.",/title/tt0708466/?ref_=ttep_ep16,7.1,2,The Gamesters of Triskelion,2316 48 | 1968-01-12,The crew of the Enterprise struggles to cope with a planet of imitative people who have modeled their society on 1920s gangsters.,/title/tt0708412/?ref_=ttep_ep17,7.8,2,A Piece of the Action,2534 49 | 1968-01-19,The Enterprise encounters a gigantic energy draining space organism that threatens the galaxy.,/title/tt0708467/?ref_=ttep_ep18,7.6,2,The Immunity Syndrome,2171 50 | 1968-02-02,"Peaceful, primitive peoples get caught up in the struggle between superpowers, with Kirk unhappily trying to restore the balance of power disrupted by the Klingons.",/title/tt0708413/?ref_=ttep_ep19,6.9,2,A Private Little War,2155 51 | 1968-02-09,"The Enterprise is guided to a distant, long-dead world where survivors of an extremely ancient race - existing only as disembodied energy - desiring the bodies of Kirk, Spock and astro-biologist Ann Mulhall so that they may live again.",/title/tt0708445/?ref_=ttep_ep20,7.6,2,Return to Tomorrow,2173 52 | 1968-02-16,"Looking for a missing Federation cultural observer, Kirk and Spock find themselves on a planet whose culture has been completely patterned after Nazi Germany.",/title/tt0708442/?ref_=ttep_ep21,7.6,2,Patterns of Force,2285 53 | 1968-02-23,Galactic alien scouts capture the Enterprise for a return voyage and a prelude to invasion. Kirk's one advantage - they're not used to their adopted human form.,/title/tt0708422/?ref_=ttep_ep22,7.7,2,By Any Other Name,2228 54 | 1968-03-01,"Responding to a distress signal, Kirk finds Captain Tracey of the U.S.S. Exeter violating the prime directive and interfering with a war between the Yangs and the Kohms to find the secret of their longevity.",/title/tt0708474/?ref_=ttep_ep23,6.2,2,The Omega Glory,2251 55 | 1968-03-08,"Kirk and a sub-skeleton crew are ordered to test out an advanced artificially intelligent control system - the M-5 Multitronic system, which could potentially render them all redundant.",/title/tt0708481/?ref_=ttep_ep24,8.1,2,The Ultimate Computer,2280 56 | 1968-03-15,The Enterprise crew investigates the disappearance of a ship's crew on a planet that is a modern version of the Roman Empire.,/title/tt0708421/?ref_=ttep_ep25,7.2,2,Bread and Circuses,2185 57 | 1968-03-29,"While back in time observing Earth in 1968, the Enterprise crew encounters the mysterious Gary Seven who has his own agenda on the planet.",/title/tt0708419/?ref_=ttep_ep26,7.6,2,Assignment: Earth,2428 58 | 1968-09-20,The crew of the Enterprise pursues a mysterious woman who has abducted Spock's brain.,/title/tt0708449/?ref_=ttep_ep1,5.7,3,Spock's Brain,2391 59 | 1968-09-27,An apparently insane Capt. Kirk has the Enterprise deliberately enter the Romulan Neutral Zone where the ship is immediately captured by the enemy.,/title/tt0708464/?ref_=ttep_ep2,8.5,3,The Enterprise Incident,2549 60 | 1968-10-04,"Trapped on a planet whose inhabitants are descended from Northwestern Native Americans, Kirk loses his memory and is proclaimed a God while the crippled Enterprise races back to the planet before it is destroyed by an asteroid.",/title/tt0708475/?ref_=ttep_ep3,6.8,3,The Paradise Syndrome,2209 61 | 1968-10-11,The Enterprise reaches a Federation colony where the adults have all killed themselves but the children play without care.,/title/tt0708417/?ref_=ttep_ep4,5.2,3,And the Children Shall Lead,2250 62 | 1968-10-18,"Lovely telepath Miranda is aide to Ambassador Kollos, in a box to stop insanity when humans see Medusans. She rejects Larry, a designer of Enterprise, and senses murderous intent nearby.",/title/tt0708433/?ref_=ttep_ep5,7.0,3,Is There in Truth No Beauty?,2093 63 | 1968-10-25,"As punishment for ignoring their warning and trespassing on their planet, the Melkot condemn Capt. Kirk and his landing party to the losing side of a surreal recreation of the 1881 historic gunfight at the OK Corral.",/title/tt0708448/?ref_=ttep_ep6,7.3,3,Spectre of the Gun,2216 64 | 1968-11-01,Both humans and Klingons have been lured to a planet by a formless entity that feeds on hatred and has set about to fashion them into a permanent food supply for itself.,/title/tt0708427/?ref_=ttep_ep7,7.9,3,Day of the Dove,2194 65 | 1968-11-08,The Enterprise discovers an apparent asteroid that is on a collision course with a planet is actually an ancient populated generation ship.,/title/tt0708430/?ref_=ttep_ep8,7.2,3,For the World Is Hollow and I Have Touched the Sky,2038 66 | 1968-11-15,"With Capt. Kirk and the derelict USS Defiant apparently lost, the Enterprise grapples with an insanity causing plague and an attack by the Tholians.",/title/tt0708479/?ref_=ttep_ep9,8.1,3,The Tholian Web,2227 67 | 1968-11-22,"After Dr. McCoy helps the leader of a planet populated by people with powerful psionic abilities, they decide to force him to stay by torturing his comrades until he submits.",/title/tt0708443/?ref_=ttep_ep10,6.6,3,Plato's Stepchildren,2116 68 | 1968-11-29,A group of aliens who exist in a state of incredible acceleration invade the Enterprise and abduct Capt. Kirk.,/title/tt0708490/?ref_=ttep_ep11,7.4,3,Wink of an Eye,2034 69 | 1968-12-06,"Trapped in an alien laboratory, Kirk, Spock and McCoy meet an empath and are involved in a series of experiments.",/title/tt0708462/?ref_=ttep_ep12,6.6,3,The Empath,2134 70 | 1968-12-20,"While transporting an arrogant, demanding princess for a political marriage, Captain Kirk must cope both with her biochemical ability to force him to love her and sabotage on his ship.",/title/tt0708428/?ref_=ttep_ep13,7.1,3,Elaan of Troyius,2054 71 | 1969-01-03,"Kirk and Spock are taken prisoners by a former starship captain named Garth, who now resides at, and has taken over, a high security asylum for the criminally insane.",/title/tt0708489/?ref_=ttep_ep14,7.0,3,Whom Gods Destroy,1986 72 | 1969-01-10,The Enterprise encounters two duo-chromatic and mutually belligerent aliens who put the ship in the middle of their old conflict.,/title/tt0708435/?ref_=ttep_ep15,7.3,3,Let That Be Your Last Battlefield,2132 73 | 1969-01-17,Kirk beams down to the planet Gideon and appears to find himself trapped on a deserted Enterprise. Spock on the real Enterprise must use his diplomatic skills to deal with the uncooperative inhabitants of Gideon and find the Captain.,/title/tt0708470/?ref_=ttep_ep16,6.6,3,The Mark of Gideon,1939 74 | 1969-01-24,"After the Enterprise landing party beams down to investigate a geologically interesting planet, their ship is hurled across the galaxy. Kirk and company find a deserted outpost guarded by the deadly image of a beautiful woman.",/title/tt0708450/?ref_=ttep_ep17,6.6,3,That Which Survives,1908 75 | 1969-01-31,"A mysterious, twinkling mass of sapient energy ravages an important archive and Scotty's new girlfriend may be linked to it.",/title/tt0708468/?ref_=ttep_ep18,6.2,3,The Lights of Zetar,1869 76 | 1969-02-14,"On a planet, looking for an urgent medicinal cure, Kirk, Spock and McCoy come across a dignified recluse living privately but in splendor with his sheltered ward and a very protective robot servant.",/title/tt0708444/?ref_=ttep_ep19,7.5,3,Requiem for Methuselah,2105 77 | 1969-02-21,"A group of idealistic hippies, led by an irrational leader, come aboard the U.S.S. Enterprise.",/title/tt0708482/?ref_=ttep_ep20,5.5,3,The Way to Eden,2240 78 | 1969-02-28,Kirk and Spock are caught up in a revolution on a planet where intellectuals and artists live on a utopian city in the sky while the rest of the population toils in mines on the barren surface below.,/title/tt0708456/?ref_=ttep_ep21,7.1,3,The Cloud Minders,1949 79 | 1969-03-07,"Kirk, Spock, Abraham Lincoln and Vulcan legend Surak are pitted in battle against notorious villains from history for the purpose of helping a conscious rock creature's understanding of a concept he does not understand, ""good vs. evil"".",/title/tt0708477/?ref_=ttep_ep22,6.8,3,The Savage Curtain,1977 80 | 1969-03-14,"When Kirk, Spock and McCoy investigate the disappearance of a doomed planet's population, they find themselves trapped in different periods of that world's past.",/title/tt0708415/?ref_=ttep_ep23,8.3,3,All Our Yesterdays,2283 81 | 1969-06-03,Captain Kirk's insane ex-lover Dr. Janice Lester forcibly switches bodies with him in order to take command of the Enterprise.,/title/tt0708485/?ref_=ttep_ep24,6.9,3,Turnabout Intruder,1994 82 | -------------------------------------------------------------------------------- /examples/table_to_dataframe.py: -------------------------------------------------------------------------------- 1 | # 2 | # table_to_dataframe.py 3 | # 4 | # Short example showing how to create a pandas.DataFrame from 5 | # a littletable.Table. 6 | # 7 | import itertools 8 | 9 | import littletable as lt 10 | 11 | 12 | # make a Table 13 | counter = itertools.count() 14 | labels = "ABCDEFGHIJKLMNOPQRSTUVWXYZ" 15 | tbl = lt.Table().insert_many( 16 | (dict(zip("abc", (next(counter), next(counter), next(counter),))) 17 | | {"label": label}) for label in labels 18 | ) 19 | 20 | # pretty output of tbl.info() 21 | lt.Table().insert_many( 22 | {'property': k, 'value': v} for k, v in tbl.info().items() 23 | )("tbl.info()").present() 24 | 25 | # print first 5 rows starting table 26 | tbl[:5].present() 27 | 28 | # create DataFrame using extracted values with fieldnames as columns 29 | df = tbl.as_dataframe() 30 | 31 | # print first 5 rows of df 32 | print(df[:5]) 33 | -------------------------------------------------------------------------------- /examples/time_conversions.py: -------------------------------------------------------------------------------- 1 | # time_conversions.py 2 | import datetime 3 | import textwrap 4 | 5 | import littletable as lt 6 | 7 | process_data = textwrap.dedent("""\ 8 | timestamp,eqpt,event,lot 9 | 2020/01/01 04:15:22,DRILL01,LotStart,PCB146 10 | 2020/01/01 04:16:02,DRILL01,Tool1,PCB146 11 | 2020/01/01 04:19:47,DRILL01,Tool2,PCB146 12 | 2020/01/01 04:26:03,DRILL01,LotEnd,PCB146 13 | """) 14 | 15 | transforms = {'timestamp': lt.Table.parse_datetime("%Y/%m/%d %H:%M:%S")} 16 | 17 | # get timestamp of first event 18 | data = lt.Table().csv_import(process_data, transforms=transforms, limit=1) 19 | start_time = data[0].timestamp 20 | 21 | # read in values, converting timestamps to offset from start time 22 | transforms = {'timestamp': lt.Table.parse_timedelta("%Y/%m/%d %H:%M:%S", reference_time=start_time)} 23 | data = lt.Table(f"Events relative to {start_time}").csv_import(process_data, transforms=transforms) 24 | 25 | data.present() 26 | 27 | 28 | process_data = textwrap.dedent("""\ 29 | elapsed_time,eqpt,event,lot 30 | 0:00:00,DRILL01,LotStart,PCB146 31 | 0:00:40,DRILL01,Tool1,PCB146 32 | 0:03:45,DRILL01,Tool2,PCB146 33 | 0:06:16,DRILL01,LotEnd,PCB146 34 | """) 35 | 36 | transforms = {'elapsed_time': lt.Table.parse_timedelta("%H:%M:%S")} 37 | data = lt.Table(f"Process step elapsed times").csv_import(process_data, transforms=transforms) 38 | 39 | data.present() 40 | -------------------------------------------------------------------------------- /examples/time_zone_db.py: -------------------------------------------------------------------------------- 1 | # 2 | # time_zone_db.py 3 | # 4 | # Read data from a .zip archive containing 2 CSV files, describing 5 | # time zone definitions and country code definitions. 6 | # 7 | # Uses data fetched from https://timezonedb.com/files/TimeZoneDB.csv.zip, 8 | # The database is licensed under Creative Commons Attribution 3.0 License. 9 | # More info at https://timezonedb.com/download 10 | # 11 | from datetime import datetime, timedelta 12 | from pathlib import Path 13 | import sys 14 | 15 | import littletable as lt 16 | 17 | 18 | tzdb_zip_file = Path(__file__).parent / "TimeZoneDB.csv.zip" 19 | 20 | if not tzdb_zip_file.exists(): 21 | print("File not found: Download TimeZoneDB.csv.zip from https://timezonedb.com/files/TimeZoneDB.csv.zip") 22 | sys.exit() 23 | 24 | 25 | # read in all country codes and display the first 20 26 | country_codes = lt.csv_import( 27 | tzdb_zip_file, 28 | zippath="country.csv", 29 | fieldnames="country_code,country_name".split(","), 30 | ) 31 | country_codes[:20].present() 32 | 33 | 34 | def str_to_datetime(s: str) -> datetime: 35 | """ 36 | Function to transform timestamp seconds into a datetime object. 37 | Need special handling since times before Jan 1, 1970 are recorded as 38 | negative values, which datetime.fromtimestamp cannot handle directly. 39 | """ 40 | timestamp = int(s) 41 | if timestamp < 0: 42 | return datetime.fromtimestamp(0) + timedelta(seconds=timestamp) 43 | else: 44 | return datetime.fromtimestamp(timestamp) 45 | 46 | 47 | # read in all timezone definitions and present the first 20 48 | time_zones = lt.csv_import( 49 | tzdb_zip_file, 50 | zippath="time_zone.csv", 51 | fieldnames="zone_name,country_code,abbreviation,time_start,gmt_offset,dst".split(","), 52 | transforms={ 53 | "dst": lambda x: x == "1", 54 | "gmt_offset": int, 55 | "time_start": str_to_datetime 56 | }, 57 | ) 58 | time_zones[:20].present( 59 | width=120, 60 | caption=f"Total {len(time_zones):,} time zone records", 61 | caption_justify="left" 62 | ) 63 | 64 | # query for time zone records for America/Los_Angeles in 2025 65 | start_time = datetime(2025, 1, 1) 66 | end_time = datetime(2025, 12, 31) 67 | time_zones.where( 68 | zone_name="America/Los_Angeles", 69 | time_start=lt.Table.in_range(start_time, end_time) 70 | )("Time zone records for America/Los_Angeles in 2025").present(width=120) 71 | 72 | # how many different time zone names are there? 73 | time_zone_names = list(time_zones.all.zone_name.unique) 74 | print("Total time zone names:", len(time_zone_names)) 75 | -------------------------------------------------------------------------------- /examples/unicode_15.1.0.txt.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ptmcg/littletable/77c64149e778923ce1feea141b279710243393a5/examples/unicode_15.1.0.txt.zip -------------------------------------------------------------------------------- /examples/unicode_320.txt.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ptmcg/littletable/77c64149e778923ce1feea141b279710243393a5/examples/unicode_320.txt.zip -------------------------------------------------------------------------------- /examples/us_ppl.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ptmcg/littletable/77c64149e778923ce1feea141b279710243393a5/examples/us_ppl.zip -------------------------------------------------------------------------------- /examples/zen_of_python_pep20.py: -------------------------------------------------------------------------------- 1 | # 2 | # zen_of_python_pep20.py 3 | # 4 | # read the Zen of Python into a littletable.Table 5 | # 6 | from contextlib import redirect_stdout 7 | import io 8 | import littletable as lt 9 | 10 | # get the Zen of Python content 11 | import_this_stdout = io.StringIO() 12 | with redirect_stdout(import_this_stdout): 13 | import this 14 | import_this_lines = import_this_stdout.getvalue().splitlines()[2:] 15 | 16 | # load into a Table and present 17 | pep20 = lt.Table().insert_many( 18 | {'id': num, 'zen': zen} 19 | for num, zen in enumerate(import_this_lines, start=1) 20 | )("The Zen of Python") 21 | pep20.present() 22 | 23 | # use text search for "better" 24 | pep20.create_search_index("zen") 25 | for entry in pep20.search.zen("better"): 26 | print(entry.zen) 27 | -------------------------------------------------------------------------------- /how_to_use_littletable.md: -------------------------------------------------------------------------------- 1 | How to Use littletable 2 | ====================== 3 | 4 | * [Introduction](#introduction) 5 | * [Creating a table](#creating-a-table) 6 | * [Inserting objects](#inserting-objects) 7 | * [Importing data from CSV files](#importing-data-from-csv-files) 8 | * [Transforming data while importing](#transforming-data-while-importing) 9 | * [Import/export data to Excel files](#importexport-to-excel-files-xlsx) 10 | * [Importing from remote sources using HTTP](#importing-from-remote-sources-using-http) 11 | * [Tabular output](#tabular-output) 12 | * [types.SimpleNamespace](#typessimplenamespace) 13 | * [Removing objects](#removing-objects) 14 | * [Indexing attributes](#indexing-attributes) 15 | * [Querying with indexed attributes](#querying-with-indexed-attributes) 16 | * [Querying for exact matching attribute values](#querying-for-exact-matching-attribute-values) 17 | * [Querying for attribute value ranges](#querying-for-attribute-value-ranges) 18 | * [Range querying on index attributes using slice notation](#range-querying-on-indexed-attributes-using-slice-notation) 19 | * [Grouping a table](#returning-grouped-tables-from-a-table) 20 | * [Splitting a table using a criteria function](#splitting-a-table-using-a-criteria-function) 21 | * [Full-text search on text attributes](#full-text-search-on-text-attributes) 22 | * [Basic statistics on Tables of numeric values](#basic-statistics-on-tables-of-numeric-values) 23 | * [Importing data from fixed-width text files](#importing-data-from-fixed-width-text-files) 24 | * [Joining tables](#joining-tables) 25 | * [Pivoting a table](#pivoting-a-table) 26 | * [Optional dependencies](#optional-dependencies) 27 | * [littletable and pandas](#littletable-and-pandas) 28 | * [littletable and SQLite](#littletable-and-sqlite) 29 | * [Some simple littletable recipes](#some-simple-littletable-recipes) 30 | 31 | 32 | Introduction 33 | ------------ 34 | 35 | `littletable` is a simple Python module to make it easy to work with collections 36 | of objects as if they were records in a database table. `littletable` augments 37 | normal Python list access with: 38 | - indexing by key attributes 39 | - joining multiple tables by common attribute values 40 | - querying for matching objects by one or more attributes 41 | - data pivoting on 1 or more attributes 42 | - full-text search on indexed attributes 43 | - easy CSV import/export (and TSV, JSON, and Excel) 44 | 45 | `littletable` supports storage of: 46 | - user-defined objects 47 | - user-defined objects using `__slots__` 48 | - `dataclasses` 49 | - `collections.namedtuples` and `typing.NamedTuples` 50 | - `types.SimpleNamespaces` 51 | - `dicts` and `typing.TypedDicts` (converted internally to `SimpleNamespaces` for attribute-style access) 52 | - `pydantic` models (including `Model`, `ImmutableModel`, and `ORMModel`) 53 | - `traits` / `traitlets` classes 54 | - `attrs` classes 55 | 56 | It is not necessary to define a table schema for tables in `littletable`; the 57 | schema of the data emerges from the attributes of the stored objects, and those 58 | used to define indexes and queries. 59 | 60 | Indexes can be created and dropped at any time. An index can be defined to have 61 | unique or non-unique key values, and whether or not to allow null values. 62 | 63 | Instead of returning DataSets or rows of structured values, `littletable` queries 64 | return new Tables. This makes it easy to arrive at a complex query by a sequence 65 | of smaller steps. The resulting values are also easily saved to a CSV file, 66 | like any other `littletable` table. 67 | 68 | 69 | Creating a table 70 | ---------------- 71 | Creating a table is simple, just create an instance of `Table`: 72 | ```python 73 | t = Table() 74 | ``` 75 | 76 | If you want, you can name the table at creation time, or any time later. 77 | ```python 78 | t = Table("customers") 79 | ``` 80 | 81 | or 82 | 83 | ```python 84 | t = Table() 85 | t("customers") 86 | ``` 87 | 88 | Table names are not necessary for queries or updates, as they would be in SQL. 89 | Table names can be useful in diagnosing problems, as they will be included in 90 | exception messages. Table joins also use the names of the source tables to 91 | create a helpful name for the resulting data table. 92 | 93 | Once you have created the Table, you can then use `insert` or `insert_many` to 94 | populate the table, or use one of the `import` methods (such as `csv_import`, 95 | `tsv_import`, or `json_import`) to load the table from a data string, external file, 96 | compressed file, Excel spreadsheet, or Internet URL. 97 | 98 | 99 | Inserting objects 100 | ----------------- 101 | From within your Python code, you can create objects and add them to the table using 102 | `insert()` and `insert_many()`. Any object can be inserted into a table, using: 103 | 104 | ```python 105 | t.insert(obj) 106 | t.insert_many(objlist) 107 | ``` 108 | 109 | Performance tip: Calling `insert_many()` with a list of objects will perform better than calling 110 | `insert()` in a loop. 111 | 112 | Importing data from CSV files 113 | ----------------------------- 114 | You can easily import a CSV file into a `Table` using `Table.csv_import()`: 115 | 116 | ```python 117 | t = Table().csv_import("my_data.csv") 118 | ``` 119 | 120 | or: 121 | 122 | ```python 123 | import littletable as lt 124 | t = lt.csv_import("my_data.csv") 125 | ``` 126 | 127 | In place of a local file name, you can specify an HTTP url (see 128 | [Importing from remote sources using HTTP](#importing-from-remote-sources-using-http) for more 129 | details): 130 | 131 | ```python 132 | url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/iris.csv" 133 | names = ["sepal-length", "sepal-width", "petal-length", "petal-width", "class"] 134 | iris_table = Table('iris').csv_import(url, fieldnames=names) 135 | ``` 136 | 137 | You can directly import CSV data as a string: 138 | ```python 139 | catalog_data = """\ 140 | sku,description,unitofmeas,unitprice 141 | BRDSD-001,Bird seed,LB,3 142 | BBS-001,Steel BB's,LB,5 143 | MGNT-001,Magnet,EA,8""" 144 | 145 | catalog = Table("catalog") 146 | catalog.create_index("sku", unique=True) 147 | catalog.csv_import(catalog_data, transforms={'unitprice': int}) 148 | ``` 149 | 150 | If you are working with a very large CSV file and just trying to see 151 | what the structure is, add `limit=100` to only read the first 100 rows. 152 | 153 | You can pre-screen data as it is read from the input file by passing 154 | a `filters={attr: filter_fn, ...}` argument. Each filter function is called 155 | on the newly-read object _before_ it is added to the table. `filter_fn` 156 | can be any function that takes a single argument of the type of the given 157 | attribute, and returns `True` or `False`. If `False`, the record does not get 158 | added to the table. 159 | 160 | ```python 161 | # import only the first 100 items that match the filter 162 | # (product_category == "Home and Garden") 163 | catalog.csv_import(catalog_data, 164 | filters={"product_category": Table.eq("Home and Garden")}, 165 | limit=100) 166 | ``` 167 | 168 | Since CSV files do not keep any type information, `littletable` will use the 169 | `SimpleNamespace` type for imported records. You can specify your own type by 170 | passing `row_class=MyType` to `csv_import`. The type must be initializable 171 | using the form `MyType(**attributes_dict)`. `namedtuples` and `SimpleNamespace` 172 | both support this form. 173 | 174 | Performance tip: For very large files, it is faster to load data using 175 | a `dataclass` or `namedtuple` than to use the default `SimpleNamespace` class. 176 | Get the fields using 177 | a 10-item import using `limit=10`, and then define the `namedtuple` using the 178 | fields from `table.info()["fields"]`. 179 | 180 | Files containing JSON-formatted records can be similarly imported using 181 | `Table.json_import()`, and tab-separated files can be imported using 182 | `Table.tsv_import()`. 183 | 184 | `littletable` can also read CSV, TSV, etc. content directly from a simple `.zip`, 185 | `.gz`, `.tar.gz`, or `.xz` archive. If the archive contains multiple files, it 186 | will try to read the contained file that matches the name of the archive, after 187 | dropping the compression extension string - otherwise it will raise a `ValueError`. 188 | (For `.zip` files, an optional `zippath` argument can be used to select a specific 189 | CSV, TSV, or JSON file by its path into the zip archive.) 190 | 191 | Calling `csv_import` on the same `Table` multiple times with different CSV files 192 | accumulates all the data into a single table. 193 | 194 | Note: if you find you cannot import `.xz` or `.lzma` files, getting the Python error 195 | `ModuleNotFoundError :_lzma`, you can remedy this by rebuilding Python after 196 | installing the `lzma-dev` library. On Ubuntu for example, this is done using: 197 | 198 | $ sudo apt-get install liblzma-dev 199 | 200 | Then rebuild and reinstall Python. 201 | 202 | 203 | Transforming data while importing 204 | --------------------------------- 205 | By default, all data imported from CSV (and related untyped data formats) is loaded 206 | as Python strings (`str` type). To convert data of other types, add the `transforms` argument, 207 | defining conversion functions to use on specific fields of the imported data. Conversion 208 | functions must take a single `str` argument, and return the converted value or raise an 209 | exception. 210 | 211 | Given input data of this form: 212 | 213 | ``` 214 | label,a,b,c,date_created 215 | A,1,2,3,2020/01/01 216 | B,4,5,6,2020/01/02 217 | C,7,8,9,2020/01/03 218 | ``` 219 | 220 | You can convert the numeric and date fields during the import using: 221 | 222 | ```python 223 | table = lt.csv_import( 224 | data, 225 | transforms={ 226 | "a": int, 227 | "b": int, 228 | "c": int, 229 | "date_created": lambda s: datetime.datetime.strptime(s, "%Y/%m/%d"), 230 | } 231 | ) 232 | ``` 233 | 234 | This can be a lot of work when an input file contains many numeric fields to be converted. 235 | To simplify this case, you can use a "*" wildcard value to define a conversion function 236 | for all fields that do not have other defined transforms. For the above case, you can write: 237 | 238 | ```python 239 | table = lt.csv_import( 240 | data, 241 | transforms={ 242 | "*": int, 243 | "date_created": lambda s: datetime.datetime.strptime(s, "%Y/%m/%d"), 244 | } 245 | ) 246 | ``` 247 | 248 | By default, fields whose values raise an exception during version remain unchanged (such as 249 | all the "label" values in the above example). You can specify an error default value to be 250 | used instead by passing a `(function, default_value)` tuple as the transform value. 251 | 252 | Aside from the Python builtin functions `int` and `float`, the `littletable.Table` class 253 | includes some useful transform staticmethods: 254 | - `convert_numeric` 255 | - `parse_date` 256 | - `parse_datetime` 257 | - `parse_timedelta` 258 | 259 | See the help text for these methods for more details. 260 | 261 | 262 | Import/export to Excel files (.xlsx) 263 | ------------------------------------ 264 | `littletable` can read and write local Excel spreadsheet files: 265 | 266 | ```python 267 | tbl = lt.Table().excel_import("data_table.xlsx") 268 | tbl.excel_export("new_table.xlsx") 269 | ``` 270 | 271 | Data values from Excel get converted to standard Python types where possible. 272 | A spreadsheet containing the following data: 273 | 274 | | name | value | type | 275 | |------|---------------------|---------| 276 | | a | 100 | int | 277 | | b | 3.14159 | float | 278 | | c | None | null | 279 | | d | 2021-12-25 00:00:00 | date | 280 | | e | Floyd | str | 281 | | f | | space | 282 | | g | 𝚃𝖞𝐩𝓮𝖤𝔯𝘳º𝗿 | str | 283 | | h | True | bool | 284 | | i | =TODAY() | formula | 285 | | j | 0 | None | 286 | | k | None | None | 287 | 288 | Can be imported and the data values will be automatically 289 | converted as shown below: 290 | 291 | ```python 292 | xl = lt.Table().excel_import("../test/data_types.xlsx") 293 | 294 | for row in xl: 295 | print(row.name, repr(row.value), type(row.value), row.type) 296 | 297 | 298 | a 100 int 299 | b 3.14159 float 300 | c None null 301 | d datetime.datetime(2021, 12, 25, 0, 0) date 302 | e 'Floyd' str 303 | f ' ' space 304 | g '𝚃𝖞𝐩𝓮𝖤𝔯𝘳º𝗿' str 305 | h True bool 306 | i '=TODAY()' formula 307 | j 0 None 308 | k None None 309 | ``` 310 | 311 | `littletable` uses `openpyxl` package to read and write Excel files. When writing 312 | using `littletable.Table.excel_export`, `openpyxl` is optimized when an XML 313 | serialization package such as `defusedxml` or `lxml` is installed. `defusedxml` is 314 | preferred, as it guards against some known XML data attacks. 315 | 316 | 317 | Importing from remote sources using HTTP 318 | ---------------------------------------- 319 | When importing from an HTTP or HTTPS source, you can optionally specify the following 320 | named arguments: 321 | 322 | | Argument | Type | Description | 323 | |----------|----------------|-------------------------------------------------------------------------------------------------------------------| 324 | | headers | dict | Headers to be passed as part of the HTTP request | 325 | | data | bytes | Data to pass as the body of the request (will use the HTTP POST method in place of the default GET) | 326 | | username | str | Username to pass using Basic Authentication; will generate `Authorization` header | 327 | | password | str | Password for Basic Authentication, default = "" | 328 | | cafile | str | SSL certificate file, default = None | 329 | | capath | str | Directory of SSL certificate files, default = None | 330 | | cadata | str or bytes | Either an ASCII string of one or more PEM-encoded certificates or a bytes-like object of DER-encoded certificates | 331 | | context | ssl.SSLContext | SSLContext instance, default = None | 332 | | timeout | int | Timeout in seconds, default = littletable.DEFAULT_HTTP_TIMEOUT (initialized to 60) | 333 | 334 | _(Note: when passing username and password credentials, HTTPS URLs are strongly encouraged, to prevent 335 | exposure of credentials in unencrypted requests. `littletable` will emit a warning when importing with authentication 336 | using an HTTP URL.)_ 337 | 338 | Some sites will use the "User-Agent" HTTP header to filter out unrestricted automated access to their site. You can add this header to your 339 | import statement using code as in the following: 340 | 341 | user_agent_string = ( 342 | "Mozilla/5.0" 343 | " (Windows NT 10.0; Win64; x64)" 344 | " AppleWebKit/537.36" 345 | " (KHTML, like Gecko)" 346 | " Chrome/128.0.0.0" 347 | " Safari/537.36" 348 | " Edg/128.0.0.0" 349 | ) 350 | table = littletable.csv_import(url, headers={"User-Agent": user_agent_string}) 351 | 352 | Tabular output 353 | -------------- 354 | To produce a nice tabular output for a table, you can use the embedded support for 355 | the `rich` module, `as_html()` in Jupyter Notebook, or the `tabulate` module: 356 | 357 | - Using `table.present()` (implemented using `rich`; `present()` accepts `rich` Table 358 | keyword args): 359 | 360 | ```python 361 | table(title_str).present(fields=["col1", "col2", "col3"]) 362 | ``` 363 | 364 | or 365 | 366 | ```python 367 | # use select() to limit the columns to be shown 368 | table.select("col1 col2 col3")(title_str).present(caption="caption text") 369 | ``` 370 | 371 | `table.present()` also accepts a `width` argument, to override any terminal default width setting: 372 | 373 | ```python 374 | table(title_str).present(width=400) 375 | ``` 376 | 377 | - Using `Jupyter Notebook`: 378 | 379 | ```python 380 | from IPython.display import HTML, display 381 | display(HTML(table.as_html(table_properties={"border": 1, "cellpadding": 5}))) 382 | ``` 383 | 384 | - Using `tabulate`: 385 | ```python 386 | # use map(vars, table) to get each table record as a dict, then pass to tabulate with 387 | # headers="keys" to auto-define headers 388 | print(tabulate(map(vars, table), headers="keys")) 389 | ``` 390 | 391 | - Output as Markdown 392 | 393 | ```python 394 | print(table.as_markdown()) 395 | ``` 396 | 397 | You can display groups in your tables by specifying a particular field on which to group. 398 | Pass the `groupby` argument to `present()`, `as_html()` or `as_markdown()` with the name of the 399 | field or fields, and consecutive duplicate values for that field will be suppressed. 400 | 401 | 402 | types.SimpleNamespace 403 | --------------------- 404 | If your program does not have a type for inserting into your table, you can use 405 | `types.SimpleNamespace`: 406 | 407 | ```python 408 | from types import SimpleNamespace 409 | bob = {"name": "Bob", "age": 19} 410 | t.insert(SimpleNamespace(**bob)) 411 | ``` 412 | 413 | Or just use `dict`s directly (which `littletable` will convert to `SimpleNamespace`s) 414 | 415 | ```python 416 | bob = {"name": "Bob", "age": 19} 417 | t.insert(bob) 418 | ``` 419 | 420 | Removing objects 421 | ---------------- 422 | Objects can be removed individually or by passing a list (or `Table`) of 423 | objects: 424 | 425 | ```python 426 | t.remove(obj) 427 | t.remove_many(objlist) 428 | t.remove_many(t.where(a=100)) 429 | ``` 430 | 431 | They can also be removed by numeric index or slice using the Python `del` statement, just 432 | like removing from a list: 433 | 434 | ```python 435 | del t[0] 436 | del t[-1] 437 | del t[3:10] 438 | ``` 439 | 440 | Finally, items can be removed using `Table.pop()`, which like `list.pop` defaults to removing 441 | the last item in the table, and returns the removed item: 442 | 443 | ```python 444 | obj = t.pop(12) 445 | obj = t.pop() 446 | ``` 447 | 448 | Adding new fields to existing rows 449 | ---------------------------------- 450 | If the underlying storage types for the rows in the `Table` permit the addition of 451 | new attributes and/or modification of existing attributes, you can do so using 452 | `Table.compute_field`, passing in a callable that `compute_field` will use to 453 | compute the value for this field for each record. 454 | 455 | You can see an example of this in `examples/explore_unicode.py`, which builds a table 456 | from the contents of the official Unicode CSV file: 457 | 458 | ```python 459 | # import the Unicode definitions, including the name and hex character value for each code point 460 | unicode = lt.csv_import(unicode_csv_file, delimiter=';') 461 | 462 | # add a field containing the integer code point value, computed from the given code_value_hex field 463 | unicode.compute_field("code_value", lambda r: int(r.code_value_hex, 16)) 464 | 465 | # add a field containing the actual Unicode character for each row 466 | unicode.compute_field("character", lambda r: chr(r.code_value)) 467 | 468 | # add a boolean field indicating whether the character can be used as an identifier character 469 | unicode.compute_field("is_identifier", lambda r: r.character.isidentifier()) 470 | ``` 471 | 472 | You can also overwrite existing fields using `compute_field`. 473 | 474 | `compute_field` also accepts a `default` argument, to be used if the given callable raises 475 | an exception for a particular row; if no default is specified, `None` is used. 476 | 477 | (`compute_field` was formerly named `add_field`, which is still maintained for compatibility.) 478 | 479 | 480 | Indexing attributes 481 | ------------------- 482 | Use `create_index` to add an index to a `Table`. Indexes can be unique or 483 | non-unique. If the table is not empty and the index to be created is 484 | `unique=True`, the uniqueness of the index attribute across the existing 485 | records is verified before creating the index, raising `KeyError` and 486 | listing the duplicated value. 487 | 488 | If a unique index is created, then retrieving using that index will 489 | return the single matching object, or raise `KeyError`. 490 | 491 | If a non-unique index is created, a `Table` is returned of all the matching 492 | objects. If no objects match, an empty `Table` is returned. 493 | 494 | ```python 495 | employees.create_index('employee_id', unique=True) 496 | employees.create_index('zipcode') 497 | 498 | # unique indexes return a single object 499 | print(employees.by.employee_id["D1729"].name) 500 | 501 | # non unique indexes return a new Table 502 | for emp in employees.by.zipcode["12345"]: 503 | print(e.name) 504 | ``` 505 | 506 | 507 | Querying with indexed attributes 508 | -------------------------------- 509 | 510 | If accessing a table using a unique index, giving a key value will 511 | return the single matching record, or raise `KeyError`. 512 | 513 | ```python 514 | employees.by.employee_id['00086'] 515 | employees.by.employee_id['invalid_id'] 516 | # raises KeyError: "no such value 'invalid_id' in index 'employee_id'" 517 | ``` 518 | 519 | If accessing a table using a non-unique index, will return a new `Table` 520 | containing all matching records. If there are no matching records, the 521 | returned table will be empty. 522 | 523 | ```python 524 | employees.by.state['CA'] 525 | employees.by.dept['Sales'] 526 | employees.by.dept['Salex'] # no such department 527 | # returns empty table 528 | ``` 529 | 530 | You can also use the get() method, similar to Python's `dict.get()`: 531 | ```python 532 | # using the unique index on "employee_id" 533 | employees.by.employee_id.get('00086') # returns the matching record 534 | employees.by.employee_id.get('invalid_id') # returns None 535 | 536 | # using the non-unique index on "dept" 537 | employees.by.dept['Sales'] # returns a Table of matching records 538 | employees.by.dept['Salex'] # returns None 539 | ``` 540 | 541 | Querying for exact matching attribute values 542 | -------------------------------------------- 543 | Calling `Table.where()` with named attributes will return a Table of 544 | all records matching all the arguments: 545 | 546 | ```python 547 | employees.where(zipcode="12345", title="Manager") 548 | 549 | # Python keywords have to be passed in an unpacked dict. 550 | student.where(**{"class": "Algebra"}) 551 | ``` 552 | 553 | It is not necessary for the attributes to be indexed to use `Table.where()`. 554 | 555 | 556 | Querying for attribute value ranges 557 | ----------------------------------- 558 | `Table.where()` supports performing queries on one or more exact 559 | matches against entries in the table: 560 | 561 | ```python 562 | employees.where(dept="Engineering") 563 | ``` 564 | 565 | `Table.where()` will also accept a callable that takes a record and 566 | returns a bool to indicate if the record is a match: 567 | 568 | ```python 569 | employees.where(lambda emp: emp.salary > 50000) 570 | ``` 571 | 572 | `littletable` includes _comparators_ to make range-checking easier to 573 | write. The following table lists the comparators, plus examples of their 574 | usage: 575 | 576 | | Comparator | Example | Comparison performed | 577 | |---------------|:--------------------------------|:-------------------------------| 578 | | `lt` | `attr=Table.lt(100)` | `attr < 100` | 579 | | `le` | `attr=Table.le(100)` | `attr <= 100` | 580 | | `gt` | `attr=Table.gt(100)` | `attr > 100` | 581 | | `ge` | `attr=Table.ge(100)` | `attr >= 100` | 582 | | `eq` | `attr=Table.eq(100)` | `attr == 100` | 583 | | `ne` | `attr=Table.ne(100)` | `attr != 100` | 584 | | `is_none` | `attr=Table.is_none())` | `attr is None` | 585 | | `is_not_none` | `attr=Table.is_not_none())` | `attr is not None` | 586 | | `is_null` | `attr=Table.is_null())` | `attr is None, "", or omitted` | 587 | | `is_not_null` | `attr=Table.is_not_null())` | `attr is not None or ""` | 588 | | `startswith` | `attr=Table.startswith("ABC")` | `attr.startswith("ABC")` | 589 | | `endswith` | `attr=Table.endswith("XYZ")` | `attr.endswith("XYZ")` | 590 | | `between` | `attr=Table.between(100, 200)` | `100 < attr < 200` | 591 | | `within` | `attr=Table.within(100, 200)` | `100 <= attr <= 200` | 592 | | `in_range` | `attr=Table.in_range(100, 200)` | `100 <= attr < 200` | 593 | | `is_in` | `attr=Table.is_in((1, 2, 3))` | `attr in (1,2,3)` | 594 | | `not_in` | `attr=Table.not_in((1, 2, 3))` | `attr not in (1,2,3)` | 595 | 596 | More examples of comparators in actual Python code: 597 | 598 | ```python 599 | employees.where(salary=Table.gt(50000)) 600 | employees.where(dept=Table.is_in(["Sales", "Marketing"])) 601 | 602 | jan_01 = date(2000, 1, 1) 603 | mar_31 = date(2000, 3, 31) 604 | apr_01 = date(2000, 4, 1) 605 | 606 | first_qtr_sales = sales.where(date=Table.within(jan_01, mar_31)) 607 | first_qtr_sales = sales.where(date=Table.in_range(jan_01, apr_01)) 608 | 609 | # get customers whose address includes an apartment number 610 | has_apt = customers.where(address_apt_no=Table.is_not_null()) 611 | 612 | # get employees whose first name starts with "X" 613 | x_names = employees.where(name=Table.startswith("X")) 614 | 615 | # get log records that match a regex (any word starts with 616 | # "warn" in the log description) 617 | import re 618 | warnings = log.where(description = re.compile(r"\bwarn", flags=re.I).search) 619 | ``` 620 | 621 | Comparators can also be used as filter functions for import methods. 622 | 623 | [Added in version 2.0.6] 624 | 625 | You can write your own comparator functions also. Define a function that 626 | takes a single value as would be stored in an attribute, and call 627 | `Table.where()` using `attribute=function`: 628 | 629 | ```python 630 | # For a table with an int field 'a', select all the records where 631 | # 'a' is odd. 632 | 633 | def is_odd(x): 634 | return bool(x % 2) 635 | 636 | # formerly 637 | tbl.where(lambda rec: is_odd(rec.a)) 638 | 639 | # new simplified form 640 | tbl.where(a=is_odd) 641 | 642 | # a comparator using regex 643 | import re 644 | # products whose name starts with "cotton" or "linen" 645 | tbl.where(product_name=re.compile(r"(cotton|linen)\b", re.I).match) 646 | # products whose name contains the word "cotton" or "linen" 647 | tbl.where(product_name=re.compile(r"\b(cotton|linen)\b", re.I).search) 648 | ``` 649 | 650 | Range querying on indexed attributes using slice notation 651 | --------------------------------------------------------- 652 | 653 | [Added in version 2.0.7] 654 | 655 | For indexed fields, range queries can also be done using slice notation. 656 | Compare these examples with operations shown [above](#querying-for-attribute-value-ranges) 657 | (given in comments): 658 | 659 | ```python 660 | # employees.where(salary=Table.ge(50000)) 661 | employees.create_index("salary") 662 | employees.by.salary[50000:] 663 | ``` 664 | 665 | Unlike Python list slices, `Table` index slices can use non-integer data 666 | types (as long as they support `>=` and `<` comparison operations): 667 | 668 | ```python 669 | jan_01 = datetime.date(2000, 1, 1) 670 | apr_01 = datetime.date(2000, 4, 1) 671 | 672 | # first_qtr_sales = sales.where(date=Table.in_range(jan_01, apr_01)) 673 | sales.create_index("date") 674 | first_qtr_sales = sales.by.date[jan_01: apr_01] 675 | ``` 676 | 677 | Note that slices with a step field (as in `[start : stop : step]`) are not supported. 678 | 679 | 680 | Splitting a table using a criteria function 681 | ------------------------------------------- 682 | You can divide a `littletable.Table` into 2 new tables using 683 | `Table.splitby`. `Table.splitby` takes a predicate function that takes 684 | a table record and returns True or False, and returns two tables: 685 | a table with all the rows that returned False and a table with all the 686 | rows that returned True. Will also accept a string indicating a particular 687 | field name, and uses `bool(getattr(rec, field_name))` for the predicate 688 | function. 689 | 690 | ```python 691 | # split on records based on even/odd of a value attribute 692 | is_odd = lambda x: bool(x % 2) 693 | evens, odds = tbl.splitby(lambda rec: is_odd(rec.value)) 694 | 695 | # split on an integer field: 0 will be treated as False, non-zero as True 696 | has_no_cars, has_cars = tbl.splitby("number_of_cars_owned") 697 | 698 | # split on a field that may be None or "" 699 | nulls, not_nulls = tbl.splitby("optional_data_field") 700 | ``` 701 | 702 | A shorthand for specifying a predicate function can be used if the predicate 703 | is solely a test for a specific value for one or more attributes: 704 | 705 | ```python 706 | qa_data, production_assembly_data = tbl.splitby( 707 | lambda rec: rec.env == "prod" and rec.dept == "assembly" 708 | ) 709 | # can be written as 710 | qa_data, production_data = tbl.splitby(env="prod", dept="assembly") 711 | ``` 712 | 713 | An optional `errors` argument allows you to define what action 714 | to take if an exception occurs while evaluating the predicate function. 715 | Valid values for `errors` are: 716 | 717 | - True : return exceptions as True 718 | - False: return exceptions as False 719 | - 'discard': do not return table rows that raise exceptions 720 | - 'return': return a third table containing rows that raise exceptions 721 | - 'raise': raise the exception 722 | 723 | `errors` can also be given as a dict mapping Exception types to one of 724 | these 4 values. 725 | 726 | The default value for `errors` (if omitted, or if an exception is raised 727 | that is not listed in an `errors` dict), is to discard the row. 728 | 729 | 730 | Returning grouped tables from a table 731 | ------------------------------------- 732 | `Table.groupby` is very similar to `itertools.groupby`, yielding a `(key, Table)` tuple for 733 | each grouped list of rows. Includes an optional `sort` argument to sort the table before grouping, 734 | using the same key attributes or function used for grouping. 735 | 736 | 737 | Splitting a table into smaller batches 738 | -------------------------------------- 739 | You can break a `Table` into smaller-sized tables (such as might be 740 | needed for paginated reporting), using `Table.batched`, similar to 741 | `itertools.batched` added in Python 3.13. Returns a generator that 742 | yields tables sliced into n-sized batches: 743 | 744 | for mini_table in tbl.batched(10): 745 | ... work with table containing only 10 entries ... 746 | 747 | 748 | Full-text search on text attributes 749 | ----------------------------------- 750 | `littletable` can perform a rudimentary version of full-text search against 751 | attributes composed of multiple-word contents (such as item descriptions, 752 | comments, etc.). To perform a full-text search, a search index must first be 753 | created, using `Table.create_search_index()`, naming the attribute to be 754 | indexed, and optionally any stop words that should be ignored. 755 | 756 | Afterward, queries can be run using `table.search.(query)`, 757 | where `attribute` is the attribute that was indexed, and `query` is 758 | a list or space-delimited string of search terms. Search terms may be 759 | prefixed by '++' or '--' to indicate required or prohibited terms, or 760 | '+' or '-' for preferred or non-preferred terms. The search function 761 | uses these prefixes to compute a matching score, and the matching records 762 | are returned in descending score order, along with their scores, and optionally 763 | each record's parsed keywords. 764 | 765 | In addition to the query, you may also specify a limit; whether to include 766 | each entry's indexed search words; and whether to return the results as a new 767 | table or as a list of (record, search_score) tuples. 768 | 769 | Example: 770 | 771 | ```python 772 | recipe_data = textwrap.dedent("""\ 773 | title,ingredients 774 | Tuna casserole,tuna noodles cream of mushroom soup 775 | Hawaiian pizza,pizza dough pineapple ham tomato sauce 776 | BLT,bread bacon lettuce tomato mayonnaise 777 | Bacon cheeseburger,ground beef bun lettuce ketchup mustard pickle cheese bacon 778 | """) 779 | recipes = lt.Table().csv_import(recipe_data) 780 | 781 | recipes.create_search_index("ingredients") 782 | matches = recipes.search.ingredients("+bacon tomato --pineapple", as_table=True) 783 | matches.present() 784 | ``` 785 | 786 | Will display: 787 | 788 | ``` 789 | +bacon tomato --pineapple 790 | 791 | Title Ingredients Ingredients Search Score 792 | ─────────────────────────────────────────────────────────────────────────── 793 | BLT bread bacon lettuce 1100 794 | tomato mayonnaise 795 | Bacon cheeseburger ground beef bun lettuce 1000 796 | ketchup mustard pickle 797 | cheese bacon 798 | 799 | ``` 800 | 801 | In some cases, the terms to search for are spread across multiple attributes. 802 | For these cases, create_search_index accepts an optional `using` argument, 803 | containing a list of string attributes to concatenate into a single searchable 804 | attribute: 805 | 806 | ```python 807 | recipes.create_search_index("recipe_terms", using="ingredients title") 808 | matches = recipes.search.recipe_terms("casserole Hawaiian", as_table=True) 809 | matches.present(fields="title ingredients".split()) 810 | ``` 811 | 812 | Will display: 813 | ``` 814 | Title Ingredients 815 | ───────────────────────────────────────────────────────── 816 | Tuna casserole tuna noodles cream of mushroom soup 817 | Hawaiian pizza pizza dough pineapple ham tomato sauce 818 | ``` 819 | 820 | Search indexes will become invalid if records are added or removed from the table 821 | after the index has been created. If they are not rebuilt, subsequent searches 822 | will raise the `SearchIndexInconsistentError` exception. 823 | 824 | 825 | Basic statistics on Tables of numeric values 826 | --------------------------------------------- 827 | `Table.stats()` will perform basic mean, variance, and standard deviation 828 | calculations by attribute on records in a table. The results are returned 829 | in a new `Table` that can be keyed by attribute (with "mean", "variance", etc. 830 | attributes), or by statistic (keyed by "mean", etc., with attributes matching 831 | those in the source `Table`). Non-numeric values are implicitly omitted from 832 | the statistics calculations. 833 | 834 | ```python 835 | import littletable as lt 836 | 837 | t1 = lt.Table() 838 | t1.csv_import("""\ 839 | a,b,c 840 | 100,101,102 841 | 110,220,99 842 | 108,130,109""", transforms=dict(a=int, b=int, c=int)) 843 | 844 | t1_stats = t1.stats() 845 | t1_stats.present() 846 | print(t1_stats.by.name["a"].mean) 847 | ``` 848 | 849 | Prints: 850 | 851 | ``` 852 | Name Mean Median Min Max Variance Std Dev Count Missing 853 | ───────────────────────────────────────────────────────────────────────── 854 | a 106.0 108 100 110 28 5.292 3 0 855 | b 150.3 130 101 220 3850.0 62.05 3 0 856 | c 103.3 102 99 109 26.33 5.132 3 0 857 | 858 | 106.0 859 | ``` 860 | 861 | This same data computed by stat instead of by field name. 862 | 863 | ```python 864 | t1_stats = t1.stats(by_field=False) 865 | t1_stats.present(box=lt.box.ASCII) # uses ASCII table borders 866 | print(t1_stats.by.stat["mean"].a) 867 | ``` 868 | 869 | Prints: 870 | 871 | ``` 872 | +-----------------------------------+ 873 | | Stat | A | B | C | 874 | |----------+-------+--------+-------| 875 | | mean | 106.0 | 150.3 | 103.3 | 876 | | median | 108 | 130 | 102 | 877 | | min | 100 | 101 | 99 | 878 | | max | 110 | 220 | 109 | 879 | | variance | 28 | 3850.0 | 26.33 | 880 | | std_dev | 5.292 | 62.05 | 5.132 | 881 | | count | 3 | 3 | 3 | 882 | | missing | 0 | 0 | 0 | 883 | +-----------------------------------+ 884 | 106.0 885 | ``` 886 | 887 | 888 | Importing data from fixed-width text files 889 | ---------------------------------------------- 890 | Some files contain fixed-width columns, you can use the `FixedWidthReader` 891 | class to import these into `littletable` `Table`s. 892 | 893 | For data in this data file (not including the leading rows showing 894 | column numbers): 895 | 896 | 1 2 3 4 5 6 897 | 0123456789012345678901234567890123456789012345678901234567890123456789 898 | --- 899 | 0010GEORGE JETSON 12345 SPACESHIP ST HOUSTON TX 4.9 900 | 0020WILE E COYOTE 312 ACME BLVD TUCSON AZ 7.3 901 | 0030FRED FLINTSTONE 246 GRANITE LANE BEDROCK CA 2.6 902 | 0040JONNY QUEST 31416 SCIENCE AVE PALO ALTO CA 8.1 903 | 904 | Define the columns to import as: 905 | 906 | ```python 907 | columns = [ 908 | ("id_no", 0, ), 909 | ("name", 4, ), 910 | ("address", 21, ), 911 | ("city", 42, ), 912 | ("state", 56, 58, ), 913 | ("tech_skill_score", 59, None, float), 914 | ] 915 | ``` 916 | 917 | And use a `FixedWidthReader` to read the file and pass a list of 918 | dicts to a `Table.insert_many`: 919 | 920 | ```python 921 | characters = lt.Table() 922 | reader = lt.FixedWidthReader(columns, "cartoon_characters.txt") 923 | characters.insert_many(reader) 924 | ``` 925 | 926 | For each column, define: 927 | - the attribute name 928 | - the starting column 929 | - (optional) the ending column (the start of the next column is 930 | the default) 931 | - (optional) a function to transform the input string (in this 932 | example, 'tech_skill_score' gets converted to a float); if no 933 | function is specified, str.strip() is used 934 | 935 | 936 | Joining tables 937 | -------------- 938 | Joining tables is one of the basic functions of relational databases. 939 | To join two tables, you must specify: 940 | - the left source table and join attribute 941 | - the right source table and join attribute 942 | - whether the join should be performed as an inner join, left outer 943 | join, right outer join, or full outer join (default is an inner join) 944 | - optionally, a list of the attributes to include in the resulting join 945 | data (returned in `littletable` as a new `Table`) 946 | 947 | `littletable` provides two different coding styles for joining tables. 948 | The first uses conventional object notation, with the `table.join()` 949 | method: 950 | 951 | ```python 952 | customers.join(orders, custid="custid") 953 | ``` 954 | 955 | creates an inner join between the table of customers and the table of 956 | their respective orders, joining on both tables' `custid` attributes. 957 | 958 | More than 2 tables can be joined in succession, since the result of a 959 | join is itself a `Table`: 960 | 961 | ```python 962 | customers.join(orders, custid="custid").join(orderitems, orderid="orderid") 963 | ``` 964 | 965 | In this case a third table has been added, to include the actual items 966 | that comprise each customer's order. The `orderitems` are associated with 967 | each order by `orderid`, and so the additional join uses that field to 968 | associate the joined customer-orders table with the `orderitems` table. 969 | 970 | The second coding style takes advantage of Python's support for customizing 971 | the behavior of arithmetic operators. A natural operator for joining two 972 | tables would be the '+' operator. To complete the join specification, we 973 | need not only the tables to be joined (the left and right terms of the '+' 974 | operation), but also the attributes to use to know which objects of each 975 | table to join together. To support this, tables have the join_on() method, 976 | which return a `JoinTerm` object: 977 | 978 | ```python 979 | customers.join_on("custid") + orders.join_on("custid") 980 | ``` 981 | 982 | This returns a join expression, which when called, performs the join and 983 | returns the data as a new `Table`: 984 | 985 | ```python 986 | customerorders = (customers.join_on("custid") + orders.join_on("custid"))() 987 | ``` 988 | 989 | JoinTerms can be added to tables directly when the join table and the added 990 | table are to join using the same attribute name. The 3-table join above 991 | can be written as: 992 | 993 | ```python 994 | customerorderitems = ((customers.join_on("custid") 995 | + orders 996 | + orderitems.join_on("orderid"))()) 997 | ``` 998 | 999 | A classic example of performing an outer join is, given a table of students 1000 | and a table of student->course registrations, find the students who 1001 | are not registered for any courses. The solution is to perform an outer 1002 | join, and select those students where their course registration is NULL. 1003 | 1004 | Here is how that looks with littletable: 1005 | 1006 | ```python 1007 | # define student and registration data 1008 | students = lt.Table().csv_import("""\ 1009 | student_id,name 1010 | 0001,Alice 1011 | 0002,Bob 1012 | 0003,Charlie 1013 | 0004,Dave 1014 | 0005,Enid 1015 | """) 1016 | 1017 | registrations = lt.Table().csv_import("""\ 1018 | student_id,course 1019 | 0001,PSYCH101 1020 | 0001,CALC1 1021 | 0003,BIO200 1022 | 0005,CHEM101 1023 | """) 1024 | 1025 | # perform outer join and show results: 1026 | non_reg = students.outer_join(lt.Table.RIGHT_OUTER_JOIN, 1027 | registrations, 1028 | student_id="student_id").where(course=None) 1029 | non_reg.select("student_id name").present() 1030 | print(list(non_reg.all.name)) 1031 | ``` 1032 | 1033 | Displays: 1034 | 1035 | ``` 1036 | Student Id Name 1037 | ─────────────────── 1038 | 0002 Bob 1039 | 0004 Dave 1040 | 1041 | ['Bob', 'Dave'] 1042 | ``` 1043 | 1044 | 1045 | Pivoting a table 1046 | ---------------- 1047 | Pivoting is a useful function for extracting overall distributions of values 1048 | within a table. Tables can be pivoted on 1, 2, or 3 attributes. The pivot 1049 | tallies up the number of objects in a given table with each of the different 1050 | key values. A single attribute pivot gives the same results as a histogram - 1051 | each key for the attribute is given, along with the count of objects having 1052 | that key value. (Of course, pivots are most interesting for attributes 1053 | with non-unique indexes.) 1054 | 1055 | Pivoting on 2 attributes extends the concept, getting the range of key values 1056 | for each attribute, and then tallying the number of objects containing each 1057 | possible pair of key values. The results can be reported as a two-dimensional 1058 | table, with the primary attribute keys down the leftmost column, and the 1059 | secondary attribute keys as headers across the columns. Subtotals will 1060 | be reported at the far right column and at the bottom of each column. 1061 | 1062 | 1063 | Optional dependencies 1064 | --------------------- 1065 | The base `littletable` code has no dependencies outside of the Python stdlib. However, some operations 1066 | require additional package installs: 1067 | 1068 | | operation | additional install required | 1069 | |-----------------------------|--------------------------------------------------------------------| 1070 | | `Table.present` | `rich` | 1071 | | `Table.excel_import/export` | `openpyxl` (plus `defusedxml` or `lxml`, `defusedxml` recommended) | 1072 | | `Table.as_dataframe` | `pandas` | 1073 | 1074 | 1075 | littletable and pandas 1076 | ---------------------- 1077 | The `pandas` package is a mature and powerful data analysis module for Python, 1078 | with extensive analytical, statistical, and query features. It supports a 1079 | vectorized approach to many common data operations, so that operations on an 1080 | entire column of values can be done very quickly. 1081 | 1082 | However, the `pandas` package is rather heavyweight, and for simple table 1083 | operations (such as reading and accessing values in a CSV file), it may be 1084 | overpowered or overcomplicated to install and learn. 1085 | 1086 | `littletable` is lightweight in comparison, and, as a pure Python module, is 1087 | not optimized for vector operations on its "columns". As a single module 1088 | Python file, its installation and distribution can be very easy, so 1089 | well suited to small data projects. Its design philosophy is to make simple 1090 | tasks (such as CSV import and tabular display) simple, make some difficult 1091 | tasks (such as pivot, join, and full text search) possible, but leave the 1092 | really difficult tasks to other packages, such as `pandas`. 1093 | 1094 | `littletable` is useful even with tables of up to 1 or 2 million rows (especially 1095 | with defined indexes); `pandas` can handle much larger datasets, and so is 1096 | more suited to large Data Science projects. 1097 | 1098 | Use `pandas` if: 1099 | - your data set is large 1100 | - your problem involves extensive vector operations on entire columns 1101 | of data in your table 1102 | - your problem is simple (CSV import), but you already have installed 1103 | `pandas` and are familiar with its usage 1104 | 1105 | Consider `littletable` if: 1106 | - you do not already have `pandas` installed, or are unfamiliar with using 1107 | DataFrames 1108 | - your dataset is no more than 1-2 million rows 1109 | - your data needs are simple CSV import/export, filtering, sorting, 1110 | join/pivot, text search, and presentation 1111 | 1112 | 1113 | littletable and SQLite 1114 | ---------------------- 1115 | `littletable` and `SQLite` have many similarities in intent; lightweight, 1116 | easy to deploy and distribute. But `SQLite` requires some of the same 1117 | schema design steps and SQL language learning of any relational database. 1118 | Mapping from Python program objects to database tables often requires use 1119 | of an ORM like `SQLAlchemy`. `littletable` allows you to store your Python 1120 | objects directly in a Table, and query against it using similar ORM-style 1121 | methods. 1122 | 1123 | `littletable` might even be useful as a prototyping tool, to work through 1124 | some data modeling and schema design concepts before making a commitment 1125 | to a specific schema of tables and indexes. 1126 | 1127 | 1128 | Some simple littletable recipes 1129 | ------------------------------- 1130 | 1131 | - Find objects with NULL attribute values (an object's attribute is considered 1132 | NULL if the object does not have that attribute, or if its value is None or ""): 1133 | 1134 | ```python 1135 | table.where(keyattr=Table.is_null()) 1136 | ``` 1137 | 1138 | 1139 | - Histogram of values of a particular attribute: 1140 | 1141 | ```python 1142 | # returns a table 1143 | table.pivot(attribute).summary_counts() 1144 | ``` 1145 | or 1146 | ```python 1147 | # prints the values to stdout in tabular form 1148 | table.pivot(attribute).dump_counts() 1149 | ``` 1150 | 1151 | 1152 | - Get a list of all key values for an indexed attribute: 1153 | 1154 | ```python 1155 | customers.by.zipcode.keys() 1156 | ``` 1157 | 1158 | 1159 | - Get a list of all values for any attribute: 1160 | 1161 | ```python 1162 | list(customers.all.first_name) 1163 | 1164 | # or get just the unique values 1165 | list(customers.all.first_name.unique) 1166 | 1167 | # get a tally of all the first names 1168 | from collections import Counter 1169 | tally = Counter(customers.all.first_name) 1170 | ``` 1171 | 1172 | 1173 | - Get a count of entries for each key value: 1174 | 1175 | ```python 1176 | customers.pivot("zipcode").dump_counts() 1177 | ``` 1178 | 1179 | - Sort table by attribute x 1180 | 1181 | ```python 1182 | employees.orderby("salary") 1183 | 1184 | # sort in descending order 1185 | employees.orderby("salary desc") 1186 | ``` 1187 | 1188 | - Sorted table by primary attribute x, secondary attribute y 1189 | 1190 | ```python 1191 | sales_employees = employees.where(dept="Sales").orderby("salary,commission") 1192 | ``` 1193 | 1194 | or 1195 | 1196 | ```python 1197 | employees.create_index("dept") 1198 | sales_employees = employees.by.dept["Sales"].orderby("salary,commission") 1199 | ``` 1200 | 1201 | - Get top 5 objects in table by value of attribute x, and add ranking field 1202 | ```python 1203 | # top 5 sales employees 1204 | employees.where(dept="Sales").orderby("sales desc")[:5].rank() 1205 | ``` 1206 | 1207 | - Find all employees whose first name starts with "X" 1208 | 1209 | ```python 1210 | employees.where(first_name=Table.startswith("X")) 1211 | ``` 1212 | 1213 | (or using sliced indexing): 1214 | 1215 | ```python 1216 | employees.create_index("first_name") 1217 | employees.by.first_name["X": "Y"] # same as `"X" <= first_name < "Y"` 1218 | ``` 1219 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | [build-system] 2 | requires = ["setuptools>=61"] 3 | build-backend = "setuptools.build_meta" 4 | 5 | [project] 6 | name = "littletable" 7 | version = "3.0.3" 8 | description = "Python in-memory ORM database" 9 | readme = { file = "README.md", content-type = "text/markdown" } 10 | requires-python = ">=3.9" 11 | license = "MIT" 12 | authors = [ 13 | { name = "Paul McGuire", email = "ptmcg@austin.rr.com" } 14 | ] 15 | maintainers = [ 16 | { name = "Paul McGuire", email = "ptmcg@austin.rr.com" } 17 | ] 18 | urls = {Homepage = "https://github.com/ptmcg/littletable/", Download = "https://pypi.org/project/littletable/"} 19 | classifiers = [ 20 | "Development Status :: 5 - Production/Stable", 21 | "Intended Audience :: Developers", 22 | "Intended Audience :: Information Technology", 23 | "Intended Audience :: Science/Research", 24 | "Operating System :: OS Independent", 25 | "Programming Language :: Python", 26 | "Programming Language :: Python :: 3", 27 | "Programming Language :: Python :: 3 :: Only", 28 | "Programming Language :: Python :: 3.9", 29 | "Programming Language :: Python :: 3.10", 30 | "Programming Language :: Python :: 3.11", 31 | "Programming Language :: Python :: 3.12", 32 | "Programming Language :: Python :: 3.13", 33 | "Programming Language :: Python :: 3.14", 34 | "Topic :: Database" 35 | ] 36 | 37 | [tool.setuptools] 38 | py-modules = ["littletable"] 39 | -------------------------------------------------------------------------------- /requirements.dev: -------------------------------------------------------------------------------- 1 | attrs 2 | rich 3 | openpyxl 4 | traits 5 | traits-stubs 6 | traitlets 7 | pydantic 8 | pytest 9 | pyparsing 10 | tox 11 | bandit 12 | defusedxml 13 | -------------------------------------------------------------------------------- /test/abc.csv: -------------------------------------------------------------------------------- 1 | a,b,c 2 | 0,0,0 3 | 0,0,1 4 | 0,0,2 5 | 0,1,0 6 | 0,1,1 7 | 0,1,2 8 | 0,2,0 9 | 0,2,1 10 | 0,2,2 11 | 1,0,0 12 | 1,0,1 13 | 1,0,2 14 | 1,1,0 15 | 1,1,1 16 | 1,1,2 17 | 1,2,0 18 | 1,2,1 19 | 1,2,2 20 | 2,0,0 21 | 2,0,1 22 | 2,0,2 23 | 2,1,0 24 | 2,1,1 25 | 2,1,2 26 | 2,2,0 27 | 2,2,1 28 | 2,2,2 29 | -------------------------------------------------------------------------------- /test/abc.csv.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ptmcg/littletable/77c64149e778923ce1feea141b279710243393a5/test/abc.csv.gz -------------------------------------------------------------------------------- /test/abc.csv.tar.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ptmcg/littletable/77c64149e778923ce1feea141b279710243393a5/test/abc.csv.tar.gz -------------------------------------------------------------------------------- /test/abc.csv.xz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ptmcg/littletable/77c64149e778923ce1feea141b279710243393a5/test/abc.csv.xz -------------------------------------------------------------------------------- /test/abc.csv.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ptmcg/littletable/77c64149e778923ce1feea141b279710243393a5/test/abc.csv.zip -------------------------------------------------------------------------------- /test/abc.json.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ptmcg/littletable/77c64149e778923ce1feea141b279710243393a5/test/abc.json.gz -------------------------------------------------------------------------------- /test/abc.tar.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ptmcg/littletable/77c64149e778923ce1feea141b279710243393a5/test/abc.tar.gz -------------------------------------------------------------------------------- /test/abc.tsv: -------------------------------------------------------------------------------- 1 | a b c 2 | 0 0 0 3 | 0 0 1 4 | 0 0 2 5 | 0 1 0 6 | 0 1 1 7 | 0 1 2 8 | 0 2 0 9 | 0 2 1 10 | 0 2 2 11 | 1 0 0 12 | 1 0 1 13 | 1 0 2 14 | 1 1 0 15 | 1 1 1 16 | 1 1 2 17 | 1 2 0 18 | 1 2 1 19 | 1 2 2 20 | 2 0 0 21 | 2 0 1 22 | 2 0 2 23 | 2 1 0 24 | 2 1 1 25 | 2 1 2 26 | 2 2 0 27 | 2 2 1 28 | 2 2 2 29 | -------------------------------------------------------------------------------- /test/abc.xlsx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ptmcg/littletable/77c64149e778923ce1feea141b279710243393a5/test/abc.xlsx -------------------------------------------------------------------------------- /test/abc.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ptmcg/littletable/77c64149e778923ce1feea141b279710243393a5/test/abc.zip -------------------------------------------------------------------------------- /test/csv_import_http_server.py: -------------------------------------------------------------------------------- 1 | from http.server import HTTPServer, BaseHTTPRequestHandler 2 | from http import HTTPStatus 3 | import threading 4 | import time 5 | import urllib.error 6 | import urllib.request 7 | 8 | 9 | # noinspection PyPep8Naming 10 | class CSVTestRequestHandler(BaseHTTPRequestHandler): 11 | def do_GET(self): 12 | self.log_message("received %s %s", self.command, self.path) 13 | path = self.path 14 | if path == "/EXIT": 15 | self.send_response(HTTPStatus.OK) 16 | self.end_headers() 17 | threading.Thread(target=lambda: self.server.shutdown()).start() 18 | 19 | elif path == "/": 20 | self.send_response(HTTPStatus.OK) 21 | self.end_headers() 22 | 23 | elif path.startswith("/abc.csv"): 24 | send_bytes = b"a,b,c\n1,2,3\n" 25 | self.send_response(HTTPStatus.OK) 26 | self.end_headers() 27 | self.wfile.write(send_bytes) 28 | 29 | 30 | def run(server_class=HTTPServer, handler_class=CSVTestRequestHandler): 31 | server_address = ('', 8888) 32 | httpd = server_class(server_address, handler_class) 33 | httpd.serve_forever() 34 | 35 | 36 | def run_background_test_server(): 37 | p = threading.Thread(target=run) 38 | p.start() 39 | 40 | for tries_remaining in reversed(range(20)): 41 | try: 42 | with urllib.request.urlopen("http://localhost:8888/"): 43 | break 44 | except urllib.error.URLError: 45 | time.sleep(0.5) 46 | 47 | return p 48 | 49 | 50 | if __name__ == '__main__': 51 | run() 52 | -------------------------------------------------------------------------------- /test/data_types.xlsx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ptmcg/littletable/77c64149e778923ce1feea141b279710243393a5/test/data_types.xlsx -------------------------------------------------------------------------------- /test/star_trek_tos_eps.jsonl: -------------------------------------------------------------------------------- 1 | {"date":"8 Sep 1966","title":"The Man Trap","description":"The crew encounters a deadly shapeshifter.","cast":{"Kirk":"William Shatner","Spock":"Leonard Nimoy"}} 2 | {"date":"15 Sep 1966","title":"Charlie X","description":"A teenager with dangerous powers boards the Enterprise.","cast":{"Kirk":"William Shatner","Spock":"Leonard Nimoy"}} 3 | {"date":"22 Sep 1966","title":"Where No Man Has Gone Before","description":"The Enterprise crew faces godlike powers in deep space.","cast":{"Kirk":"William Shatner","Spock":"Leonard Nimoy"}} 4 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [tox] 2 | env_list = 3 | py3{9,10,11,12,13} 4 | minversion = 4.15.1 5 | skip_missing_interpreters = True 6 | 7 | [testenv] 8 | description = run the tests with pytest 9 | package = wheel 10 | wheel_build_env = .pkg 11 | deps = 12 | pytest>=6 13 | -r requirements.dev 14 | commands = 15 | pytest unit_tests.py {tty:--color=yes} {posargs} 16 | -------------------------------------------------------------------------------- /update_littletable_timestamp.py: -------------------------------------------------------------------------------- 1 | from datetime import datetime, UTC 2 | from pathlib import Path 3 | from pyparsing import quoted_string 4 | 5 | nw = datetime.now(tz=UTC) 6 | now_string = '"%s"' % (nw.strftime("%d %b %Y %X")[:-3] + " UTC") 7 | print(now_string) 8 | 9 | quoted_time = quoted_string() 10 | quoted_time.set_parse_action(lambda: now_string) 11 | 12 | version_time = "__version_time__ = " + quoted_time 13 | 14 | pp_init = Path("littletable.py") 15 | orig_code = pp_init.read_text() 16 | new_code = version_time.transform_string(orig_code) 17 | pp_init.write_text(new_code) 18 | --------------------------------------------------------------------------------