├── .gitignore ├── MANIFEST.in ├── README.md ├── ReleaseNotes.md ├── __init__.py ├── setup.py └── ukbb_parser ├── __init__.py ├── scripts ├── __init__.py ├── category_tree.py ├── cli.py ├── condition_filtering.py ├── config.py ├── create_header_key.py ├── data │ ├── careers_level_map.csv │ ├── field.txt │ ├── icd10_level_map.csv │ ├── mapped_category_tree.json │ └── self_report_level_map.csv ├── demo_conversion.py ├── level_processing.py └── utils.py └── tests ├── __init__.py └── test_installation.py /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | .Rhistory 3 | scripts/config.py 4 | scripts/category_tree.py 5 | *pyc 6 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include README.md 2 | include ukbb_parser/scripts/data/field.txt 3 | include ukbb_parser/scripts/data/self_report_level_map.csv 4 | include ukbb_parser/scripts/data/careers_level_map.csv 5 | include ukbb_parser/scripts/data/icd10_level_map.csv 6 | include ukbb_parser/scripts/data/mapped_category_tree.json 7 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # UK Biobank Data Parser 2 | #### Contact: Alyssa Zhu 3 | 4 | The UK Biobank Data Parser (**ukbb_parser**) is a python-based package that allows easy interfacing with the large UK Biobank dataset (Zhu et al., OHBM 2019). 5 | 6 | To avoid repetitive calls for standard demographic variables, we created default output columns for sex (self-reported and genetic), age at the time of the neuroimaging scan, race/ethnicity, and education. Self-reported race (Data-Field 21000) is coded first by the larger race group due to inconsistent reporting of ethnicity. Genetic ethnic grouping (Data-Field 22006) and the first 10 genetic principal components (Data-Field 22009) are also provided. Education (Data-Field 6138) is coded according to educational levels in the United Kingdom. These levels are converted to standardized education levels based on the International Standard Classification of Education framework, and estimated years of education is derived using the conversion strategy published by Okbay et al. (2016). 7 | 8 | See **ReleaseNotes.md** for version updates. 9 | 10 | ## How to cite 11 | 12 | Zhu, A.H., Salminen, L.E., Thompson, P.M., Jahanshad, N. "The UK Biobank Data Parser: a tool with built in and customizable filters for brain studies." (2019) Organization for Human Brain Mapping. Rome, Italy, June 9-13, 2019. 13 | 14 | ## Requirements 15 | 16 | Please note that this parser uses the Click package, which sometimes has problems installing with Python 3.7. Any Python version <=3.6 will be compatible. 17 | 18 | ## Installation 19 | 20 | As this package has not been added to PyPI, please follow the instructions below carefully. 21 | 22 | Two-step process: 23 | 1. To clone the repository: 24 | `git clone https://github.com/USC-IGC/ukbb_parser.git` 25 | 26 | 2. After downloading or cloning (in the same directory that `git clone` was run in): 27 | `pip install ./ukbb_parser` 28 | 29 | One-step process: 30 | `pip install git+https://github.com/USC-IGC/ukbb_parser.git` 31 | 32 | N.B. The above installation commands will require administrator/root privileges of the system. If you don't have administrator/root privileges, please try adding the `--user` flag. 33 | 34 | Example: 35 | `pip install --user ./ukbb_parser` 36 | 37 | ## Usages 38 | 39 | ### Display available subcommands 40 | `ukbb_parser --help` 41 | 42 | ### Display flags/options for a specific subcommand 43 | `ukbb_parser [subcommand] --help` 44 | 45 | ### Use subcommand 46 | `ukbb_parser [subcommand] [--flags inputs]` 47 | 48 | ## Subcommands 49 | 50 | ### parse 51 | 52 | The **parse** subcommand will filter the input CSV file with the given input parameters. 53 | 54 | The output csv will contain a set of default columns: eid, age at visits, race, education levels, healthy control cohorts (see output HTML for more details), and sex. 55 | 56 | Required Inputs: 57 | * `-i, --incsv CSV` File path of the downloaded UK Biobank data csv 58 | * `-o, --out TEXT` Output prefix 59 | 60 | Optional Inputs: 61 | * `--incon ICD10Code` ICD10 Diagnosis codes you wish to include. Least 62 | common denominators are acceptable, e.g. --incon F. 63 | Ranges are also acceptable inputs, e.g., F10-F20, 64 | and can span across letters 65 | * `--excon ICD10Code` ICD10 Diagnosis codes you wish to exclude. Least 66 | common denominators are acceptable, e.g. --incon F. 67 | Ranges are also acceptable inputs, e.g., F10-F20, 68 | and can span across letters 69 | * `--insr SRCode` Self-Report codes you wish to include. Ranges are 70 | acceptable inputs. 71 | * `--exsr SRCode` Self-Report codes you wish to exclude. Ranges are 72 | acceptable inputs. 73 | * `--incat Category` Categories you wish to include. Ranges are 74 | acceptable inputs, e.g. 100-200 75 | * `--excat Category` Categories you wish to exclude. Ranges are 76 | acceptable inputs, e.g. 100-200 77 | * `--inhdr HeaderName` Columns names you wish to include. Ranges are 78 | acceptable inputs, e.g. 100-200 79 | If you wish to have 80 | all the available data, please use --inhdr all 81 | * `--exhdr HeaderName` Columns names you wish to exclude. Ranges are 82 | acceptable inputs, e.g. 100-200 83 | * `--subjects SubID` A list of participant IDs to include 84 | * `--dropouts dropouts` CSV(s) containing eids of participants who have 85 | dropped out of the study 86 | * `--img_subs_only` Use this flag to only keep data of participants 87 | with an imaging visit. 88 | * `--img_visit_only` Use this flag to only keep data acquired during the 89 | imaging visit. 90 | * `--no_convert` Use this flag if you don't want the demographic conversions run 91 | * `--long_names` Use this flag to replace datafield numbers in column names with datafield titles 92 | * `--rcols` Use this flag if spreadsheet has columns names under R convention (e.g., "X31.0.0") 93 | * `--fillna TEXT` Use this flag to fill blank cells with the flag 94 | input, e.g., NA 95 | * `--chunksize INTEGER` Use this flag to set the processing chunk size, 96 | Default:1000 97 | 98 | 99 | The incon/excon/insr/exsr/incat/excat/inhdr/exhdr/combine flags can all be used multiple times, e.g., `--incat 110 --incat 134`. 100 | 101 | Example usage: 102 | 103 | `ukbb_parser parse --incsv ukbb_spreadsheet.csv -o ukbb_subset --incat 100 --excat 135 --incon G35` 104 | 105 | ### inventory 106 | 107 | The **inventory** subcommand will create binary columns indicating the presence of specified data. The data that can be inventoried are _ICD-10 conditions, self-report (non-cancer) conditions, and careers_. 108 | 109 | As can be viewed on [the UK Biobank data showcase](http://biobank.ctsu.ox.ac.uk/crystal/index.cgi), these data are organized in a hierarchical tree structure with five levels. At the end of each of the branches are selectable codes, i.e., codes that are used in and can be queried for in the UK Biobank data spreadsheet. By default, the **inventory** subcommand will aggregate all selectable codes within the requested level/code input into a single binary column. To obtain a column for the aggregate column _and_ an additional column for each of the selectable codes within the requested level/code combination, please use `--all_codes`. All selectable codes regardless of level can also be obtained using `--level S --code all`. 110 | 111 | Code inputs should be derived from the datafield-specific coding pages available on the UK Biobank showcase. If there are spaces in the codes, please use quotes around the entry (see example below). Multiple levels and codes can be used at once, but please ensure that levels and codes are provided in corresponding pairs. 112 | 113 | N.B. If `--long_names` has been used in the creation of an input spreadsheet for **inventory**, the code will not work. 114 | 115 | #### ICD-10 116 | 117 | The coding tree for the ICD-10 conditions can be viewed in part on the pages of either of the datafields ([41202](http://biobank.ctsu.ox.ac.uk/crystal/field.cgi?id=41202) and [41204](http://biobank.ctsu.ox.ac.uk/crystal/field.cgi?id=41204)) or on the [coding page](http://biobank.ctsu.ox.ac.uk/crystal/coding.cgi?id=19). 118 | 119 | _Notes_ 120 | * For top level codes (`--level 0`), use the chapter number as the `--code` input, e.g., "Chapter II" to indicate "Neoplasms". 121 | * For Level 1 (`--level 1`), add "Block" to the beginning of the `--code` input, e.g., "Block A00-09" to indicate "Intestinal infectious diseases" 122 | 123 | #### Self-Report (non-cancer) 124 | 125 | The coding tree for the self-report (non-cancer) conditions can be viewed on [the datafield page](http://biobank.ctsu.ox.ac.uk/crystal/field.cgi?id=20002) or on the [coding page](http://biobank.ctsu.ox.ac.uk/crystal/coding.cgi?id=6). 126 | 127 | _Notes_ 128 | * As can be seen on the coding page, the upper levels of the self-report conditions have non-unique codes (-1). Please use the "Meaning" as the `--code` input instead. 129 | 130 | #### Careers 131 | 132 | The coding tree for careers can be viewed on the datafield page [22617](http://biobank.ctsu.ox.ac.uk/crystal/field.cgi?id=22617)) or on the [coding page](http://biobank.ctsu.ox.ac.uk/crystal/coding.cgi?id=2). Please note that the coding on the coding page is only available to logged-in users. 133 | 134 | Also note that this inventory binarization only produces one column per job code, i.e., a job code will be noted as present so long as it has been indicated in any of the visits/instances. 135 | 136 | ##### Flags and Usage 137 | 138 | Inputs: 139 | * `--incsv CSV` File path of downloaded UK Biobank CSV 140 | * `--outcsv CSV` File path to write out to 141 | * `--rcols` Use this flag if spreadsheet has columns names under R convention (e.g., "X31.0.0") 142 | * `--datatype type` Data to inventory; Valid choices include: icd10, 143 | self_report, careers 144 | * `--level level` Level to inventory by; N.B. Please input 0 for the Top 145 | level or S for selectable codes 146 | * `--code code` Codes to inventory; Use the option 'all' to inventory all 147 | categories in the given level; Please use level-appropriate 148 | codes; Ranges are allowed for selectable codes 149 | * `--all_codes` (optional) Use this flag if you'd like to additionally obtain 150 | individual inventories of all codes 151 | * `--chunksize INTEGER` Use this flag to set the processing chunk size, 152 | Default:1000 153 | 154 | Example usage: 155 | 156 | `ukbb_parser inventory --incsv ukbb_spreadsheet.csv --outcsv jobs_inventory.csv --datatype careers --level 3 --code 6113` 157 | 158 | `ukbb_parser inventory --incsv ukbb_spreadsheet.csv --outcsv jobs_inventory.csv --datatype careers --level 3 --code 6113 --level 4 --code 1151010 --all_codes` 159 | 160 | `ukbb_parser inventory --incsv ukbb_spreadsheet.csv --outcsv icd10_inventory.csv --datatype icd10 --level 0 --code "Chapter V" --level 1 --code "Block G00-G09"` 161 | 162 | `ukbb_parser inventory --incsv ukbb_spreadsheet.csv --outcsv jobs_inventory.csv --datatype self_report --level S --code all` 163 | 164 | ### update 165 | 166 | The **update** subcommand will combine and/or update CSV files that were downloaded at different times (e.g., after a refresh). 167 | 168 | Inputs: 169 | * `--previous CSV` File path of previous downloaded UK Biobank CSV 170 | * `--new CSV` File path of new downloaded UK Biobank CSV or a CSV of processed results 171 | * `--outcsv CSV` File path to write newly updated CSV to 172 | 173 | Example usage: 174 | 175 | `ukbb_parser update --previous ukbb_2018_spreadsheet.csv --new ukbb_2019_spreadsheet.csv --outcsv combined_ukbb_2018_2019_spreadsheet.csv` 176 | 177 | ### check 178 | 179 | The **check** subcommand will determine if the queried Datafield or Category is included in the downloaded CSV file. 180 | 181 | Inputs: 182 | * `--incsv CSV` File path of downloaded UK Biobank CSV 183 | * `--datafield df` Datafield to check for 184 | * `--category cat` Category to check for 185 | 186 | Example usage: 187 | 188 | `ukbb_parser check --incsv ukbb_spreadsheet.csv --datafield 31` 189 | `ukbb_parser check --incsv ukbb_spreadsheet.csv --category 100` 190 | 191 | ### create_cohorts 192 | 193 | In Development 194 | 195 | ## References 196 | 197 | Zhu, A.H., Salminen, L.E., Thompson P.M., Jahanshad N. "The UK Biobank Data Parser: a tool with built in and customizable filters for brain studies." (2019) Organization for Human Brain Mapping. Rome, Italy, June 9-13, 2019. 198 | 199 | Okbay, A., Beauchamp, J.P., Fontana, M.A., Lee, J.J., Pers, T.H., Rietveld, C.A., ... & Oskarsson, S. (2016). Genome-wide association study identifies 74 loci associated with educational attainment. Nature, 533(7604), 539. 200 | 201 | ## Acknowledgments 202 | 203 | This package is developed using the UK Biobank Resource under Application Number 11559. 204 | 205 | Funding was provided in part by NIH R01-AG059874 (Jahanshad). 206 | -------------------------------------------------------------------------------- /ReleaseNotes.md: -------------------------------------------------------------------------------- 1 | ### Release 0.8.0 2 | 3 | Updates: 4 | * [January 3, 2024] Updated the category tree to account for newly available data fields and categories 5 | * [January 3, 2024] Removed datafield 132 as a necessary datafield from the job inventory script 6 | 7 | ### Release 0.7.0 8 | 9 | New Features: 10 | * [Oct 15, 2020] Changed `ukbb_parser check` to accept multiples and combinations of `--datafield` and `--category` arguments 11 | 12 | Bug Fixes: 13 | 14 | * [Oct 15, 2020] Fixed time difference calculation for imaging and online cognitive test dates 15 | 16 | ### Release 0.6.0 17 | 18 | New Features: 19 | 20 | * [Sep 23, 2020] Added `--subjects` flag to `ukbb_parser inventory` 21 | * [Sep 23, 2020] Added `--chunksize` flag to `ukbb_parser inventory` and `ukbb_parser parse` 22 | * [Sep 23, 2020] Added `--long_names` flag to `ukbb_parser parse` 23 | 24 | Removed Features: 25 | 26 | * [Sep 23, 2020] The `--combine` flag has been removed from the parse command 27 | 28 | ### Release 0.5.1 29 | 30 | * [Feb 26, 2020] Added `--rcols` to `ukbb_parser inventory` 31 | * [Feb 26, 2020] Added funcionality for fourth visit (i.e., second imaging visit) 32 | * [Feb 26, 2020] Edited `ukbb_parser parse` control assignments to account for missing datafields 33 | 34 | ### Release 0.5.0 35 | 36 | * [Jan 15, 2020] Updated datafield reference for `ukbb_parser parse` output HTML 37 | * [Jan 16, 2020] Updated category map json 38 | * Introduced dynamic setting of chunk size reading for input CSVs (better for lower memory environments) 39 | -------------------------------------------------------------------------------- /__init__.py: -------------------------------------------------------------------------------- 1 | __author__ = 'azhu' 2 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | try: 2 | from setuptools import setup, find_packages 3 | except (ImportError, ModuleNotFoundError) as e: 4 | print("Please install setuptools and try again.\npip install setuptools") 5 | import sys 6 | sys.exit(1) 7 | else: 8 | setup( 9 | name='ukbb_parser', 10 | author="Alyssa H. Zhu", 11 | description="ukbb_parser is a python-based parser for UK Biobank data", 12 | version='0.8.0', 13 | packages=find_packages(), 14 | include_package_data=True, 15 | install_requires=[ 16 | 'click', 17 | 'html5lib', 18 | 'numpy>=1.13.3', 19 | 'pandas>=0.21.0' 20 | ], 21 | entry_points={ 22 | 'console_scripts': [ 23 | 'ukbb_parser = ukbb_parser.scripts.cli:ukbb_parser', 24 | 'ukbb_parser_test = ukbb_parser.tests.test_installation:main' 25 | ] 26 | } 27 | ) 28 | -------------------------------------------------------------------------------- /ukbb_parser/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/USC-IGC/ukbb_parser/0aafa84bd427253c202ce8a03d9a3d3abd898501/ukbb_parser/__init__.py -------------------------------------------------------------------------------- /ukbb_parser/scripts/__init__.py: -------------------------------------------------------------------------------- 1 | __author__ = 'azhu' 2 | -------------------------------------------------------------------------------- /ukbb_parser/scripts/category_tree.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import pandas as pd 3 | 4 | map_key_error_msg = """ 5 | 6 | 7 | Hmm... We couldn't find %s in the pre-parsed tree. 8 | We're searching through the website now. 9 | This might take some extra time. 10 | 11 | Please note that the website has a few infinite loops, so if the parsing takes more than an hour, please kill the command and email Alyssa Zhu the requested category numbers. Thanks! 12 | 13 | 14 | """ 15 | 16 | def get_category_tree(category, mapped_json=cattree): 17 | fields = [] 18 | 19 | if "categories" in mapped_json[category].keys(): 20 | subcategories = mapped_json[category]["categories"] 21 | while len(subcategories) != 0: 22 | subcategories1 = [] 23 | for s in subcategories: 24 | if "categories" in mapped_json[s].keys(): 25 | subcategories1 += mapped_json[s]["categories"] 26 | if "fields" in mapped_json[s].keys(): 27 | fields += mapped_json[s]["fields"] 28 | subcategories = subcategories1 29 | if "fields" in mapped_json[category].keys(): 30 | fields += mapped_json[category]["fields"] 31 | 32 | return fields 33 | 34 | 35 | def get_category_tree_web(category): 36 | 37 | fields = [] 38 | 39 | df = pd.read_html("http://biobank.ctsu.ox.ac.uk/crystal/label.cgi?id="+category) 40 | if df[0].ix[0][0] == "Field ID": 41 | fields += list(df[0][0].values[1:]) 42 | else: 43 | subcategories = df[0][0].values[1:] 44 | while len(subcategories) != 0: 45 | subcategories1 = [] 46 | for s in subcategories: 47 | df_sub = pd.read_html("http://biobank.ctsu.ox.ac.uk/crystal/label.cgi?id="+s) 48 | if df_sub[0].ix[0][0] == "Field ID": 49 | fields += list(df_sub[0][0].values[1:]) 50 | else: 51 | subcategories1 += list(df_sub[0][0].values[1:]) 52 | subcategories = subcategories1 53 | 54 | return fields 55 | -------------------------------------------------------------------------------- /ukbb_parser/scripts/cli.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import ukbb_parser.scripts.demo_conversion as dc 3 | import ukbb_parser.scripts.condition_filtering as cf 4 | import ukbb_parser.scripts.create_header_key as chk 5 | import ukbb_parser.scripts.level_processing as lp 6 | from ukbb_parser.scripts.utils import read_spreadsheet, find_icd10_ix_range, find_icd10_letter_ixs, parse_cat_tree, create_long_bois 7 | import pkg_resources 8 | import pandas as pd 9 | import numpy as np 10 | import datetime 11 | import click 12 | import json 13 | import sys 14 | import os 15 | # import category_tree as ct 16 | # import config 17 | 18 | ### These are to get rid of warning messages that might worry people. Comment them out when developing and debugging 19 | import warnings 20 | warnings.filterwarnings("ignore") 21 | 22 | @click.group() 23 | def ukbb_parser(): 24 | pass 25 | 26 | # ### Create Custom Cohorts 27 | # 28 | # @ukbb_parser.command() 29 | # @click.option("--cohort", metavar="path", help="""Name of text file to write out""") 30 | # @click.option("--incsv", metavar="csv", help="""CSV(s) from which the cohort subject IDs are to be derived""") 31 | # def create_cohort(cohort, incsv): 32 | # subjids = [] 33 | # for csv in incsv: 34 | # subjids += csv.eid.values.tolist() 35 | # 36 | # subjids = set(subjids) # Remove Duplicates 37 | # subjids = sorted(list(subjids)) # Sort them in increasing order 38 | # subjids = [str(subjid)+"\n" for subjid in subjids] # Convert IDs to strings and add prepare for writing to text file 39 | # 40 | # with open(cohort, "w") as f: 41 | # f.writelines(subjids) 42 | 43 | @ukbb_parser.command() 44 | @click.option("--incsv", metavar="CSV", help="""File path of downloaded UK Biobank CSV""") 45 | @click.option("--datafield", multiple=True, metavar="df", help="""Datafield to check for""") 46 | @click.option("--category", multiple=True, metavar="cat", help="""Category to check for""") 47 | def check(incsv, datafield, category): 48 | ''' 49 | Determine if the queried Datafield or Category is included in the downloaded CSV file. 50 | 51 | Please see https://github.com/USC-IGC/ukbb_parser for additional documentation. 52 | ''' 53 | with open(incsv, 'r') as f: 54 | first_line = f.readline() 55 | columns = first_line.strip().split(",") 56 | datafields = set([col.split("-")[0] for col in columns]) 57 | datafields = list(datafields) 58 | if datafields[0].startswith('"'): 59 | datafields = [df[1:] for df in datafields] 60 | if len(datafield) > 0: 61 | for dfield in datafield: 62 | if str(dfield) in datafields: 63 | click.echo("\nFound {} in CSV\n".format(dfield)) 64 | else: 65 | click.echo("\nDid not find {} in CSV\n".format(dfield)) 66 | if len(category) > 0: 67 | with open(pkg_resources.resource_filename(__name__, 'data/mapped_category_tree.json'), 'r') as f: 68 | entire_cat_tree = json.load(f) 69 | cat_tree = entire_cat_tree["tree"] 70 | for cat in category: 71 | cat_dfs = parse_cat_tree(cat, cat_tree) 72 | if len(cat_dfs) == 0: 73 | click.echo("\n{} does not appear to be a valid or mapped category\n".format(cat)) 74 | sys.exit(1) 75 | else: 76 | y = 0 77 | missing = [] 78 | for dfield in cat_dfs: 79 | if str(dfield) in datafields: 80 | y += 1 81 | else: 82 | missing.append(dfield) 83 | click.echo("\nFound {} data-fields.\nMissing {} data-fields\n".format(y, len(missing))) 84 | if len(missing) > 0: 85 | click.echo(", ".join(missing)+'\n') 86 | 87 | 88 | @ukbb_parser.command() 89 | @click.option("--previous", metavar="CSV", help="""File path of previous downloaded UK Biobank CSV""") 90 | @click.option("--new", metavar="CSV", help="""File path of new downloaded UK Biobank CSV or CSV of processed results""") 91 | @click.option("--outcsv", metavar="CSV", help="""File path to write newly updated CSV to""") 92 | def update(previous, new, outcsv): 93 | ''' 94 | Combine multiple spreadsheet files. 95 | 96 | Please see https://github.com/USC-IGC/ukbb_parser for additional documentation. 97 | ''' 98 | click.echo("Loading "+previous) 99 | pc = read_spreadsheet(previous, 'csv') 100 | click.echo("Loading "+new) 101 | nc = read_spreadsheet(new, 'csv') 102 | 103 | keep = ['eid'] 104 | for col in pc.columns: 105 | if col not in nc.columns: 106 | keep.append(col) 107 | old_columns = " ".join(set([dfn.split("-")[0] for dfn in keep])) 108 | click.echo("\nKeeping old columns: " + old_columns) 109 | 110 | new_cols = [] 111 | for col in nc.columns: 112 | if col not in pc.columns: 113 | new_cols.append(col) 114 | new_columns = " ".join(set([dfn.split("-")[0] for dfn in new_cols])) 115 | click.echo("\nNew columns: " + new_columns) 116 | 117 | outdf = pd.merge(pc[keep], nc, how="outer", on="eid") 118 | except_eid = list(outdf.columns) 119 | str_columns = [] 120 | for c in outdf.columns.tolist(): 121 | try: 122 | int(c.split("-")[0]) 123 | except ValueError: 124 | except_eid.remove(c) 125 | str_columns.append(c) 126 | outdf.dropna(axis=1, how="all", inplace=True) 127 | outdf[str_columns+sorted(except_eid, key=lambda x: int(x.split("-")[0]))] 128 | click.echo("\nWriting "+outcsv) 129 | outdf.to_csv(outcsv, chunksize=15000, index=False) 130 | 131 | parser_desc = """ 132 | """ 133 | 134 | @ukbb_parser.command() 135 | @click.option("-i", "--incsv", metavar="CSV", help="""File path of the downloaded UK Biobank data csv""") 136 | @click.option("-o", "--out", help="""Output prefix""") 137 | @click.option("--incon", multiple=True, metavar="ICD10Code", 138 | help="ICD10 Diagnosis codes you wish to include. Least common denominators are acceptable, e.g. --incon F. Ranges are also acceptable inputs, e.g. F10-F20, and can span across letters") 139 | @click.option("--excon", multiple=True, metavar="ICD10Code", 140 | help="ICD10 Diagnosis codes you wish to exclude. Least common denominators are acceptable, e.g. --excon F. Ranges are also acceptable inputs, e.g. F10-F20, and can span across letters") 141 | @click.option("--insr", multiple=True, metavar="SRCode", 142 | help="Self-Report codes you wish to include. Ranges are acceptable inputs.") 143 | @click.option("--exsr", multiple=True, metavar="SRCode", 144 | help="Self-Report codes you wish to exclude. Ranges are acceptable inputs.") 145 | # @click.option("--filter", metavar="column=value", multiple=True, 146 | # help="Filters to only give rows where the column value is as specified") 147 | @click.option("--incat", multiple=True, metavar="Category", 148 | help="Categories you wish to include. Ranges are acceptable inputs, e.g. 100-200") 149 | @click.option("--excat", multiple=True, metavar="Category", 150 | help="Categories you wish to exclude. Ranges are acceptable inputs, e.g. 100-200") 151 | @click.option("--inhdr", multiple=True, metavar="HeaderName", 152 | help="Columns names you wish to include. Ranges are acceptable inputs, e.g. 100-200\nIf you wish to have all the available data, please use --inhdr all") 153 | @click.option("--exhdr", multiple=True, metavar="HeaderName", 154 | help="Columns names you wish to exclude. Ranges are acceptable inputs, e.g. 100-200") 155 | # @click.option("--cohort", metavar="cohort", multiple=True, 156 | # help="""Output csv will be limited to the data of the cohort(s)""") 157 | @click.option("--subjects", multiple=True, metavar="SubID", help="""A list of participant IDs to include""") 158 | @click.option("--dropouts", multiple=True, metavar="dropouts", help="""CSV(s) containing eids of participants who have dropped out of the study""") 159 | @click.option("--img_subs_only", is_flag=True, help="Use this flag to only keep data of participants with an imaging visit.") 160 | @click.option("--img_visit_only", is_flag=True, help="Use this flag to only keep data acquired during the imaging visit.") 161 | @click.option("--no_convert", is_flag=True, help="Use this flag if you don't want the demographic conversions run") 162 | @click.option("--long_names", is_flag=True, help='Use this flag to replace datafield numbers in column names with datafield titles') 163 | @click.option("--rcols", is_flag=True, help='Use this flag if spreadsheet has columns names under R convention (e.g., "X31.0.0")') 164 | @click.option("--fillna", help="Use this flag to fill blank cells with the flag input, e.g., NA") 165 | @click.option("--chunksize", help="Use this flag to set the processing chunk size, Default:1000", default=1000) 166 | # @click.option("--combine", metavar="Spreadsheet", multiple=True, help="""Spreadsheets to combine to output; Please make sure all spreadsheets have an identifier column 'eid'; These can be in csv, xls(x) or table formats""") 167 | def parse(incsv, out, incon, excon, insr, exsr, incat, excat, inhdr, exhdr, subjects, dropouts, img_subs_only, img_visit_only, no_convert, long_names, rcols, fillna, chunksize): 168 | ''' 169 | Filter the input CSV file with the given input parameters. 170 | 171 | Please see https://github.com/USC-IGC/ukbb_parser for additional documentation. 172 | ''' 173 | 174 | ################## 175 | ### Setting Up ### 176 | ################## 177 | 178 | if not os.path.exists(incsv): 179 | click.echo("{} does not exist. Please double check the provided file path".format(incsv)) 180 | sys.exit(1) 181 | 182 | if out is None: 183 | click.echo("An out prefix is required. Please provide one and run again.") 184 | sys.exit(1) 185 | 186 | if os.path.exists(out+'.csv'): 187 | click.echo("The output CSV already exists.\nIf %s is the intended name, please delete it and run again." %(out+'.csv')) 188 | sys.exit(1) 189 | 190 | arglist = ' '.join(sys.argv) 191 | pd.set_option("display.max_colwidth", 500) 192 | 193 | ### Functions... We like functions 194 | 195 | time_between_online_cognitive_test_and_imaging = { 196 | '20134-0.0': 'time_between_pairs_matching_imaging', 197 | '20135-0.0': 'time_between_FI_imaging', 198 | '20136-0.0': 'time_between_Trails_imaging', 199 | '20137-0.0': 'time_between_SDMT_imaging', 200 | '20138-0.0': 'time_between_digit_span_imaging' 201 | } 202 | 203 | def delta_t_days(datafield, dataframe): 204 | imaging_date = pd.Series([datetime.datetime.strptime(v, '%Y-%m-%d') if isinstance(v, str) else np.nan for v in dataframe['53-2.0']]) 205 | online_test_date = pd.Series([datetime.datetime.strptime(v.split('T')[0], '%Y-%m-%d') if isinstance(v, str) else np.nan for v in dataframe[datafield]]) 206 | try: 207 | time_between = online_test_date - imaging_date 208 | return np.abs(time_between.dt.days) 209 | except TypeError: 210 | return np.array([np.nan]*len(imaging_date)) 211 | 212 | #################################### 213 | ### Filter data columns, Part I ### 214 | #################################### 215 | 216 | all_columns = pd.read_csv(incsv, encoding='ISO-8859-1', nrows=2) 217 | all_columns = list(all_columns.columns) 218 | defcols = ["eid"] 219 | covariate_columns = ['eid'] 220 | 221 | # R cols 222 | 223 | if rcols: 224 | revert_names = {} 225 | for c in all_columns: 226 | if (len(c.split(".")) == 3) and c.startswith("X"): 227 | dfr = c.split(".")[0][1:] 228 | instr = c.split(".")[1] 229 | entryr = c.split(".")[2] 230 | revert_names[c] = "{}-{}.{}".format(dfr, instr, entryr) 231 | all_columns = ['eid'] + list(revert_names.values()) 232 | 233 | ### Loading in Mapped Category and Datafields Tree 234 | 235 | with open(pkg_resources.resource_filename(__name__, 'data/mapped_category_tree.json'), 'r') as f: 236 | entire_cat_tree = json.load(f) 237 | cat_tree = entire_cat_tree["tree"] 238 | 239 | ### Creating Include and Exclude Datafield Lists 240 | 241 | click.echo("Determining Output Datafields") 242 | 243 | if len(inhdr) > 0: 244 | if inhdr[0] == "all": 245 | to_include = [oc.split("-")[0] for oc in all_columns] 246 | else: 247 | to_include = [] 248 | for ih in inhdr: 249 | if "-" in ih: 250 | inhdr_range = list(range(int(ih.split("-")[0]), int(ih.split('-')[1])+1)) 251 | to_include += [str(hdr) for hdr in inhdr_range] 252 | else: 253 | to_include.append(ih) 254 | else: 255 | to_include = [] 256 | 257 | to_exclude = [] 258 | if len(exhdr) > 0: 259 | for eh in exhdr: 260 | if "-" in eh: 261 | exhdr_range = list(range(int(eh.split("-")[0]), int(eh.split('-')[1])+1)) 262 | to_exclude += [str(hdr) for hdr in exhdr_range] 263 | else: 264 | to_exclude.append(eh) 265 | 266 | if len(incat) > 0: 267 | for ic in incat: 268 | if "-" in ic: 269 | for n in range(int(ic.split("-")[0]), int(ic.split("-")[1])+1): 270 | to_include += parse_cat_tree(str(n), cat_tree) 271 | else: 272 | to_include += parse_cat_tree(str(ic), cat_tree) 273 | 274 | if len(excat) > 0: 275 | for ec in excat: 276 | if "-" in ec: 277 | for n in range(int(ec.split("-")[0]), int(ec.split("-")[1])+1): 278 | to_exclude += parse_cat_tree(str(n), cat_tree) 279 | else: 280 | to_exclude += parse_cat_tree(str(ec), cat_tree) 281 | 282 | to_include = list(set(sorted(to_include))) 283 | to_include = [hdr for hdr in to_include if hdr not in to_exclude] 284 | 285 | ### Check if Imaging Time Point Data Only is Requested 286 | 287 | if img_visit_only or long_names: 288 | field_txt = pd.read_csv(pkg_resources.resource_filename(__name__, 'data/field.txt'), index_col='field_id', sep='\t') 289 | 290 | instance2s = [] 291 | if img_visit_only: 292 | click.echo("Selecting data acquired at Imaging time point for those datafields using instance 2 codes") 293 | instance2s = field_txt.loc[field_txt.instance_id == 2].index.tolist() 294 | instance2s = [str(in2) for in2 in instance2s] 295 | 296 | ### Default Covariates 297 | 298 | # Sex 299 | 300 | if "31-0.0" in all_columns: 301 | covariate_columns.append("31-0.0") 302 | defcols.append("31-0.0") 303 | if "22001-0.0" in all_columns: 304 | defcols.append("22001-0.0") 305 | covariate_columns.append("22001-0.0") 306 | 307 | # Genetic Race 308 | 309 | if "22006-0.0" in all_columns: 310 | defcols.append("22006-0.0") 311 | covariate_columns.append("22006-0.0") 312 | 313 | # And More 314 | 315 | mds_components = [] 316 | icd_columns = [] 317 | sr_columns = [] 318 | 319 | for c in all_columns: 320 | 321 | # Other Demographics 322 | if c.split("-")[0] in ["6138", "21000", "21003", "34", "52", "53"]: 323 | defcols.append(c) 324 | continue 325 | 326 | # Genetics 327 | 328 | if c.startswith("22009-") and (len(mds_components) < 10): 329 | defcols.append(c) 330 | mds_components.append(c) 331 | continue 332 | 333 | # ICD10 Conditions 334 | 335 | if c.startswith("41202-") or c.startswith("41204-") or c.startswith("41270-"): 336 | icd_columns.append(c) 337 | defcols.append(c) 338 | continue 339 | 340 | # Self-Report Conditions 341 | 342 | if c.startswith("20002-"): 343 | sr_columns.append(c) 344 | defcols.append(c) 345 | continue 346 | 347 | # DataField Filter 348 | 349 | datafield = c.split("-")[0] 350 | if datafield in to_include: 351 | # Time Point Filter 352 | if img_visit_only and (datafield in instance2s): 353 | if c.split("-")[1].split(".")[0] in ["2", "3"]: 354 | defcols.append(c) 355 | else: 356 | defcols.append(c) 357 | 358 | ##################### 359 | ### Preprocessing ### 360 | ##################### 361 | 362 | ### ICD10 363 | 364 | if (len(icd_columns) == 0) and ((len(incon) > 0) or (len(excon) > 0)): 365 | click.echo("ICD 10 columns are not available for diagnosis condition filtering. Please check input spreadsheet before trying again.") 366 | sys.exit(1) 367 | 368 | coding19 = pd.read_csv(pkg_resources.resource_filename(__name__, 'data/icd10_level_map.csv'), index_col="Coding") 369 | selectable_icd10 = coding19.loc[coding19.Selectable == "Y"].index.tolist() 370 | 371 | include_icd = [] 372 | 373 | if len(incon) > 0: 374 | for icd in incon: 375 | if icd in selectable_icd10: 376 | include_icd.append(icd) 377 | elif "-" in icd: 378 | start_loc, end_loc = find_icd10_ix_range(coding19, icd.split("-")[0], icd.split("-")[1]) 379 | include_icd += coding19.loc[start_loc : end_loc].index.tolist() 380 | else: 381 | include_icd += find_icd10_letter_ixs(coding19, icd) 382 | 383 | exclude_icd = [] 384 | 385 | if len(excon) > 0: 386 | 387 | for icd in excon: 388 | if icd in selectable_icd10: 389 | exclude_icd.append(icd) 390 | elif "-" in icd: 391 | start_loc, end_loc = find_icd10_ix_range(coding19, icd.split("-")[0], icd.split("-")[1]) 392 | exclude_icd += coding19.loc[start_loc : end_loc].index.tolist() 393 | else: 394 | exclude_icd += find_icd10_letter_ixs(coding19, icd) 395 | 396 | # N.B. Actual filtering happens below in Self-Report section 397 | 398 | ### Filter subjects by Self-Report conditions 399 | 400 | if (len(sr_columns) == 0) and ((len(insr) > 0) or (len(exsr) > 0)): 401 | click.echo("Self-report columns are not available for filtering. Please check input spreadsheet before trying again.") 402 | sys.exit(1) 403 | 404 | if (len(insr) > 0) or (len(exsr) > 0): 405 | click.echo("Filtering by Self-Report conditions") 406 | 407 | include_srs = [] 408 | 409 | if len(insr) > 0: 410 | for sr in insr: 411 | if "-" in sr: 412 | include_srs += list(range(int(sr.split("-")[0]), int(sr.split("-")[1]) + 1)) 413 | else: 414 | include_srs.append(int(sr)) 415 | 416 | exclude_srs = [] 417 | 418 | if len(exsr) > 0: 419 | for sr in exsr: 420 | if "-" in sr: 421 | exclude_srs += list(range(int(sr.split("-")[0]), int(sr.split("-")[1]) + 1)) 422 | else: 423 | exclude_srs.append(int(sr)) 424 | 425 | ### Filter subjects by IDs 426 | 427 | sublist = [] 428 | if len(subjects) > 0: 429 | click.echo("Obtaining subset of provided subject IDs") 430 | for sub in subjects: 431 | try: 432 | with open(sub, 'r') as f: 433 | lines = [l.strip() for l in f.readlines()] 434 | sublist += [int(l) for l in lines] 435 | del lines 436 | except (FileNotFoundError, UnicodeDecodeError) as e: 437 | sublist.append(int(sub)) 438 | except ValueError: 439 | print("Encountered invalid --subjects entry. Please check before trying again.") 440 | sys.exit(1) 441 | 442 | sublist = [int(subjid) for subjid in sublist] 443 | 444 | ### Remove study dropouts 445 | 446 | dropids = [] 447 | 448 | if len(dropouts) > 0: 449 | for drop in dropouts: 450 | try: 451 | with open(drop, 'r') as f: 452 | lines = [l.strip() for l in f.readlines()] 453 | dropids += [int(l) for l in lines] 454 | del lines 455 | except (FileNotFoundError, UnicodeDecodeError) as e: 456 | dropids.append(int(drop)) 457 | except ValueError: 458 | print("Encountered invalid --dropouts entry. Please check before trying again.") 459 | sys.exit(1) 460 | 461 | ############### 462 | ### COHORTS ### 463 | ############### 464 | 465 | ### Cohort Flag 466 | 467 | # if len(cohort) > 0: 468 | # for coh in cohort: 469 | # click.echo("Obtaining data of individuals listed in " + coh) 470 | # with open(coh, "r") as f: 471 | # sublist += [l.strip() for l in f.readlines()] 472 | 473 | cohorts = {"NP_controls_1": {"icd10": ['A8', 'B20', 'B21', 'B22', 'B23', 'B24', 'B65-B83', 474 | 'C', 'F','G','I6','Q0','S04','S06','S07','S08','S09', 475 | 'T36-T46', 'T48-T50']}, 476 | "NP_controls_2": {"icd10": ['A8', 'B20', 'B21', 'B22', 'B23', 'B24', 'B65-B83', 477 | 'C', 'F','G','I6','Q0','S04','S06','S07','S08','S09', 478 | 'T36-T46', 'T48-T50'], 479 | "sr": [1075, 1081, 1086, 1220, 1234, 1243, 1246, 1247, 1250, 1256, 1258, 1260, 480 | 1261, 1262, 1264, 1266, 1267, 1286, 1287, 1288, 1291, 1371, 1408, 481 | 1434, 1437, 1439]}, 482 | "CNS_controls_1": {"icd10": ['C70', 'C71', 'C72', 'G', 'I6', 'Q0', 'R90', 'R940', 483 | 'S01', 'S02', 'S03', 'S04', 'S05', 'S06', 'S07', 'S08', 'S09'], 484 | "sr": [1081, 1086, 1266, 1267, 1397, 1434, 1437, 1491, 1626]\ 485 | + list(range(1244,1252)) + list(range(1258, 1265))}, 486 | "CNS_controls_2": {"icd10": ['C70', 'C71', 'C72', 'F2', 'F31', 'F7', 'G', 'I6', 'Q0', 'R90', 'R940', 487 | 'S01', 'S02', 'S03', 'S04', 'S05', 'S06', 'S07', 'S08', 'S09'], 488 | "sr": [1081, 1086, 1243, 1266, 1267, 1289, 1291, 489 | 1371, 1397, 1434, 1437, 1491, 1626]\ 490 | + list(range(1244,1252)) + list(range(1258, 1265))} 491 | } 492 | 493 | ### TODO: add Healthy cohort 494 | 495 | 496 | ####################### 497 | ### Processing Time ### 498 | ####################### 499 | 500 | ### Reading in the Data Source 501 | 502 | click.echo("Loading "+incsv) 503 | # df = read_spreadsheet(incsv, 'csv') 504 | 505 | ### Delete empty columns 506 | 507 | # df.dropna(axis=1, how="all", inplace=True) 508 | 509 | for i, df in enumerate(pd.read_csv(incsv, encoding='ISO-8859-1', chunksize=chunksize, usecols=defcols)): 510 | 511 | if rcols: 512 | df.rename(columns=revert_names, inplace=True) 513 | 514 | click.echo("\nProcessing Chunk %i\n" %i) 515 | 516 | ### Filter subjects by IDs 517 | if len(sublist) > 0: 518 | df = df[df.eid.isin(sublist)] 519 | if df.empty: 520 | continue 521 | 522 | if img_subs_only: 523 | if "53-2.0" in df.columns: 524 | df = df[df["53-2.0"].notnull()] 525 | else: 526 | click.echo("Cannot select imaged subset of participants") 527 | sys.exit(1) 528 | if df.empty: 529 | continue 530 | 531 | ### Remove study dropouts 532 | 533 | if len(dropids) > 0: 534 | click.echo("Removing data of individuals listed in {} who have withdrawn from the study".format(drop)) 535 | df = df[~df.eid.isin(dropids)] 536 | if df.empty: 537 | continue 538 | 539 | ### Filter subjects by ICD10 conditions 540 | 541 | if (len(incon) > 0) or (len(excon) > 0): 542 | click.echo("Filtering by ICD10 conditions") 543 | 544 | 545 | if (len(insr) > 0) or (len(incon) > 0): 546 | found_icd_includes = df[icd_columns].isin(include_icd).any(axis=1) 547 | found_sr_includes = df[sr_columns].isin(include_srs).any(axis=1) 548 | found_includes = np.logical_or(found_icd_includes, found_sr_includes) 549 | df = df.loc[found_includes] 550 | 551 | found_icd_excludes = df[icd_columns].isin(exclude_icd).any(axis=1) 552 | found_sr_excludes = df[sr_columns].isin(exclude_srs).any(axis=1) 553 | found_excludes = np.logical_or(found_icd_excludes, found_sr_excludes) 554 | df = df.loc[~found_excludes] 555 | if df.empty: 556 | continue 557 | 558 | ### Control Time 559 | 560 | if (len(icd_columns) > 0) and (len(sr_columns) > 0): 561 | click.echo("Indicating Control Cohorts") 562 | for k, v in cohorts.items(): 563 | icd10_excludes = [] 564 | for icd in v['icd10']: 565 | if icd in selectable_icd10: 566 | icd10_excludes.append(icd) 567 | elif "-" in icd: 568 | excon_ix_0 = icd.split("-")[0] 569 | excon_ix_1 = icd.split("-")[1] 570 | start_loc, end_loc = find_icd10_ix_range(coding19, excon_ix_0, excon_ix_1) 571 | icd10_excludes += coding19.loc[start_loc : end_loc].index.tolist() 572 | else: 573 | icd10_excludes += find_icd10_letter_ixs(coding19, icd) 574 | 575 | icd10_series = df[icd_columns].isin(icd10_excludes).any(axis=1) 576 | sr_series = np.zeros_like(icd10_series) 577 | if 'sr' in v.keys(): 578 | sr_series = df[sr_columns].isin(v['sr']).any(axis=1) 579 | df.loc[~np.logical_or(icd10_series, sr_series), k] = 1 580 | else: 581 | click.echo("Warning: Cannot indicate control subjects. ICD-10 or self-report data fields are missing") 582 | 583 | ### Default Covariates 584 | 585 | if no_convert: 586 | for c in df.columns: 587 | if c.startswith("6138-") or c.startswith("21000-") or c.startswith("21003-"): 588 | covariate_columns.append(c) 589 | else: 590 | click.echo("Adding Converted Demographic Information") 591 | 592 | ### Calculate Float Ages 593 | 594 | click.echo(" * float ages") 595 | df, convert_status = dc.calculate_float_ages(df) 596 | if convert_status is True: 597 | age_cols = ["Age1stVisit", "AgeRepVisit", "AgeAtScan", "AgeAt2ndScan"] 598 | else: 599 | age_cols = ["21003-0.0", "21003-1.0", "21003-2.0", "21003-3.0"] 600 | for c in age_cols: 601 | if c in df.columns: 602 | covariate_columns.append(c) 603 | 604 | ### Create Race Column 605 | ### TODO: create separate columns for race and ethnicity? 606 | 607 | click.echo(" * ethnicity") 608 | df, convert_status = dc.add_ethnicity_columns(df) 609 | if convert_status is True: 610 | covariate_columns.append("Race") 611 | 612 | ### Create Eductation Columns 613 | 614 | click.echo(" * education") 615 | df, convert_status = dc.add_education_columns(df) 616 | if convert_status is True: 617 | covariate_columns += ["ISCED", "YearsOfEducation"] 618 | 619 | #################################### 620 | ### Filter data columns, Part II ### 621 | #################################### 622 | 623 | click.echo("Filtering Data Columns") 624 | 625 | includes = [c for c in defcols if c.split("-")[0] in to_include] 626 | for datafield in ["21003-0.0", "21003-1.0", "21003-2.0", "21003-3.0", "31-0.0", "22001-0.0", "22006-0.0"] + mds_components: 627 | if (datafield in includes) and (datafield in covariate_columns): 628 | includes.remove(datafield) 629 | 630 | covariate_columns += mds_components 631 | covariate_columns += sorted(list(cohorts.keys())) 632 | includes = [c for c in df.columns if c in covariate_columns + includes] 633 | df = df[includes] 634 | # df.dropna(axis=1, how="all", inplace=True) 635 | 636 | if "53-2.0" in df.columns: 637 | for k in time_between_online_cognitive_test_and_imaging.keys(): 638 | if k in includes: 639 | df[time_between_online_cognitive_test_and_imaging[k]] = np.array(delta_t_days(k, df)) 640 | 641 | ######################## 642 | ### Finishing Up Now ### 643 | ######################## 644 | 645 | ### Long Bois Time 646 | 647 | if long_names: 648 | name_key = create_long_bois(df.columns, field_txt) 649 | df.rename(columns=name_key, inplace=True) 650 | 651 | if fillna is not None: 652 | df.fillna(fillna, inplace=True) 653 | 654 | if os.path.exists(out+".csv"): 655 | df.to_csv(out+".csv", mode="a", header=False, index=False) 656 | else: 657 | df.to_csv(out+".csv", index=False) 658 | 659 | ### Create Header Key 660 | 661 | click.echo("Creating HTML header key") 662 | chk.create_html_key(df, arglist, out) 663 | 664 | ### Add Additional Spreadsheets 665 | 666 | # if len(combine) > 0: 667 | # for com in combine: 668 | # click.echo('Adding {} data to output spreadsheet'.format(com)) 669 | # add_df = read_spreadsheet(com, 'unknown') 670 | # if "eid" not in add_df.columns: 671 | # click.echo("eid was not found in {}. Skipping for now.".format(com)) 672 | # continue 673 | # else: 674 | # if not pd.api.types.is_numeric_dtype(add_df.eid): 675 | # valid_eids = [eid for eid in add_df.eid if eid.isdigit()] 676 | # add_df = add_df[add_df.eid.isin(valid_eids)] 677 | # add_df.eid = add_df.eid.astype(int) 678 | # df = df.merge(add_df, on='eid', how='left', suffixes=("", "_"+com[:5])) 679 | 680 | click.echo("Done!") 681 | 682 | @ukbb_parser.command() 683 | @click.option("--incsv", metavar="CSV", help="""File path of downloaded UK Biobank CSV""") 684 | @click.option("--outcsv", metavar="CSV", help="""File path to write out to""") 685 | @click.option("--subjects", multiple=True, metavar="SubID", help="""A list of participant IDs to include""") 686 | @click.option("--rcols", is_flag=True, help='Use this flag if spreadsheet has columns names under R convention (e.g., "X31.0.0")') 687 | @click.option("--datatype", metavar="type", 688 | type=click.Choice(['icd10', 'self_report', 'careers']), 689 | help="""Data to inventory; Valid choices include: icd10, self_report, careers""") 690 | @click.option("--level", multiple=True, metavar="level", help="""Level to inventory by; N.B. Please input 0 for the Top level or S for selectable codes""") 691 | @click.option("--code", multiple=True, metavar="code", help="""Codes to inventory; Use the option 'all' to inventory all categories in the given level; Please use level-appropriate codes; Ranges are allowed""") 692 | @click.option("--all_codes", is_flag=True, help="""(optional) Use this flag if you'd like to obtain additionally obtain individual inventories of all codes""") 693 | @click.option("--chunksize", help="Use this flag to set the processing chunk size, Default:1000", default=1000) 694 | def inventory(incsv, outcsv, subjects, rcols, datatype, code, level, all_codes, chunksize): 695 | ''' 696 | Create binary columns indicating the presence of specified data. 697 | 698 | Please see https://github.com/USC-IGC/ukbb_parser for additional documentation. 699 | ''' 700 | 701 | # Check Inputs First 702 | 703 | if not os.path.exists(incsv): 704 | click.echo("{} does not exist. Please double check the provided file path".format(incsv)) 705 | sys.exit(1) 706 | 707 | if os.path.exists(outcsv): 708 | click.echo("The output CSV already exists.\nIf %s is the intended name, please delete it and run again." %outcsv) 709 | sys.exit(1) 710 | 711 | if len(level) != len(code): 712 | click.echo("Number of --level and --code flags used should match. Please double check your inputs before trying again.") 713 | sys.exit(1) 714 | 715 | # Load Datafields from Column Headers 716 | all_columns = pd.read_csv(incsv, encoding='ISO-8859-1', nrows=2) 717 | all_columns = list(all_columns.columns) 718 | 719 | # R columns 720 | if rcols: 721 | revert_names = {} 722 | for c in all_columns: 723 | if (len(c.split(".")) == 3) and c.startswith("X"): 724 | dfr = c.split(".")[0][1:] 725 | instr = c.split(".")[1] 726 | entryr = c.split(".")[2] 727 | revert_names[c] = "{}-{}.{}".format(dfr, instr, entryr) 728 | all_columns = ['eid']+list(revert_names.values()) 729 | 730 | # Isolate the Necessary DataFields 731 | 732 | necessary = { 733 | 'icd10': ['41202', '41204', '41270'], 734 | 'self_report': ['20002'], 735 | 'careers': ['22617'] # 132 now restricted 736 | } 737 | 738 | # The following shouldn't be necessary but just in case 739 | if datatype not in list(necessary.keys()): 740 | click.echo("--datatype was not specified correctly. Please check inputs and try again.") 741 | sys.exit(1) 742 | 743 | # Check to make sure necessary datafields are present 744 | click.echo("Currently checking for the required datafields") 745 | missing_df = 0 746 | defcols=['eid'] 747 | datafields = [] 748 | for c in all_columns: 749 | datafield = c.split("-")[0] 750 | if datafield in necessary[datatype]: 751 | if datafield not in datafields: 752 | datafields.append(datafield) 753 | defcols.append(c) 754 | 755 | if len(datafields) == 0: 756 | click.echo("The relevant datafield(s) (%s) have not been found. Please double check your input spreadsheet for the datafields before trying again." % ", ".join(necessary[datatype])) 757 | sys.exit(1) 758 | elif len(datafields) < len(necessary[datatype]): 759 | click.echo("Caution: Some datafields (%s) are not included in the input spreadsheet. Proceeding with the other(s)" % ", ".join([f for f in necessary[datatype] if f not in datafields])) 760 | 761 | # Load in relevant level map 762 | level_map = pkg_resources.resource_filename(__name__, 'data/{}_level_map.csv'.format(datatype)) 763 | 764 | ### Filter subjects by IDs 765 | 766 | sublist = [] 767 | if len(subjects) > 0: 768 | click.echo("Obtaining subset of provided subject IDs") 769 | for sub in subjects: 770 | try: 771 | with open(sub, 'r') as f: 772 | lines = [l.strip() for l in f.readlines()] 773 | sublist += [int(l) for l in lines] 774 | del lines 775 | except (FileNotFoundError, UnicodeDecodeError) as e: 776 | sublist.append(int(sub)) 777 | except ValueError: 778 | print("Encountered invalid --subjects entry. Please double check before trying again.") 779 | sys.exit(1) 780 | 781 | # Processing 782 | 783 | # df = read_spreadsheet(incsv) 784 | reldfs = list(defcols) 785 | reldfs.remove('eid') 786 | for i, df in enumerate(pd.read_csv(incsv, encoding='ISO-8859-1', chunksize=chunksize, usecols=defcols)): 787 | 788 | # Filter Subjects 789 | if len(sublist) > 0: 790 | df = df[df.eid.isin(sublist)] 791 | 792 | if rcols: 793 | df.rename(columns=revert_names, inplace=True) 794 | 795 | for i, l in enumerate(level): 796 | click.echo("Currently conducting an entry inventory of level {} / code {}".format(l, code[i])) 797 | df = lp.level_processing(df, datatype, reldfs, level_map, code[i], l, all_codes) 798 | new_columns = [col for col in df.columns if col not in reldfs] 799 | df = df[new_columns] 800 | 801 | if os.path.exists(outcsv): 802 | df.to_csv(outcsv, mode="a", header=False, index=False) 803 | else: 804 | df.to_csv(outcsv, index=False) 805 | 806 | click.echo("Done!") 807 | 808 | if __name__ == "__main__": 809 | ukbb_parser() 810 | -------------------------------------------------------------------------------- /ukbb_parser/scripts/condition_filtering.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import pandas as pd 3 | 4 | def cohort_filter(df, exc=None, inc=None, sr=None, count=None): 5 | icd_cols = [] 6 | 7 | for c in df.columns: 8 | if c.startswith('41202') or c.startswith('41204'): 9 | icd_cols.append(c) 10 | if exc['sr'] or inc['sr'] or sr: 11 | if c.startswith('20002'): 12 | icd_cols.append(c) 13 | 14 | decision = {} 15 | to_keep = [] 16 | to_discard = [] 17 | 18 | df2 = df.copy() 19 | icd10_conditions = [] 20 | sr_conditions = [] 21 | 22 | print("\n\nIt's time to filter by conditions now. This will usually take anywhere between 2-20 minutes... Doo De Doo...\n\n") 23 | print("Although please note if you haven't created a conditions database and asked for --icd10_count all this can take about an hour.\n\n") 24 | 25 | for i, row in df.iterrows(): 26 | exclude_subject = 0 27 | for c in icd_cols: 28 | if exclude_subject == 1: 29 | break 30 | try: 31 | if np.isnan(row[c]): 32 | continue 33 | except TypeError: 34 | pass 35 | if c.startswith('41202') or c.startswith('41204'): 36 | if (count == 'all') or (isinstance(count, list) and row[c] in count): 37 | df2 = df2.set_value(i, row[c], 1) 38 | if row[c] not in icd10_conditions: 39 | icd10_conditions.append(row[c]) 40 | for e in exc['icd']: 41 | if row[c].startswith(e): 42 | decision[row['eid']] = "exclude" 43 | exclude_subject = 1 44 | break 45 | for n in inc['icd']: 46 | if row[c].startswith(n): 47 | decision[row['eid']] = "include" 48 | break 49 | if c.startswith('20002'): 50 | if (sr == 'all') or (isinstance(sr, list) and row[c] in sr): 51 | df2 = df2.set_value(i, int(row[c]), 1) 52 | if row[c] not in sr_conditions: 53 | sr_conditions.append(int(row[c])) 54 | if row[c] in exc['sr']: 55 | decision[row['eid']] = "exclude" 56 | exclude_subject = 1 57 | elif row[c] in inc['sr']: 58 | decision[row['eid']] = "include" 59 | 60 | for (k,i) in decision.items(): 61 | if i == "exclude": 62 | to_discard.append(k) 63 | elif i == "include": 64 | to_keep.append(k) 65 | 66 | df2 = df2[~df2['eid'].isin(to_discard)] 67 | if (len(inc['icd']) != 0) or (len(inc['sr']) != 0): 68 | df2 = df2[df2['eid'].isin(to_keep)] 69 | 70 | return df2, icd10_conditions, sr_conditions 71 | 72 | -------------------------------------------------------------------------------- /ukbb_parser/scripts/config.py: -------------------------------------------------------------------------------- 1 | __author__ = 'azhu' 2 | 3 | REFERENCE_FILES = { 4 | 'ukbb_html': "/ifs/loni/faculty/thompson/four_d/azhu/ukbb/updated_header_keys.html", 5 | # ^^^ I need to account for this (ours is preprocessed) 6 | 'incsv': '/ifs/loni/faculty/thompson/four_d/azhu/ukbb/updated_oct2017_imaging_subset.csv', 7 | } 8 | 9 | ADDITIONAL_SPREADSHEETS = { 10 | 'GWAS': { 11 | 'spreadsheet_file': '/ifs/loni/faculty/thompson/four_d/azhu/ukbb/ukb1155_GWASed_HM3_b37mds.mds', 12 | 'delimiter': 'whitespace' 13 | } 14 | 'freesurfer': { 15 | 'spreadsheet_file': "/ifs/loni/faculty/thompson/four_d/azhu/ukbb/CorticalMeasuresENIGMA_SurfAvg-ThickAvg_ALL_v7_ROI_fails_as_NAs_NEDA_RL_combined.xlsx" 16 | #'join_on': '', 17 | } 18 | } 19 | 20 | self_report_excludes = [1075, 1081, 1086, 1220, 1234, 1243, 1246, 1247, 1250, 1256, 1258, 1260, 21 | 1261, 1262, 1264, 1266, 1267, 1286, 1287, 1288, 1291, 1371, 1408, 22 | 1434, 1437, 1439] 23 | 24 | # Maybe take this out since we'll have the other function 25 | COHORT= { 26 | "CNSFree_Broad": { 27 | 'cohort_file': '' 28 | }, 29 | "CNSFree_Strict": { 30 | 'cohort_file': '' 31 | }, 32 | "NeuroFree_Broad": { 33 | 'cohort_file': '' 34 | }, 35 | "NeuroFree_Strict": { 36 | 'cohort_file': '' 37 | }, 38 | "Healthy": { 39 | 'cohort_file': '' 40 | }, 41 | "SuperHealthy": { 42 | 'cohort_file': '' 43 | } 44 | } 45 | 46 | defcols = ["eid", "31-0.0", "Age1stVisit", "AgeRepVisit", "AgeAtScan", "Race", "ISCED", "YearsOfEducation"] 47 | + list(COHORT.keys()) 48 | 49 | if len(ADDITIONAL_SPREADSHEETS) > 0: 50 | defcols += list(ADDITIONAL_SPREADSHEETS.keys()) # This doesn't actually make sense 51 | -------------------------------------------------------------------------------- /ukbb_parser/scripts/create_header_key.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import pkg_resources 3 | import pandas as pd 4 | import datetime 5 | 6 | def create_html_key(df, arglist, outcsv): 7 | info_df = pd.read_csv(pkg_resources.resource_filename(__name__, 'data/field.txt'), sep='\t') 8 | info_df = info_df[['field_id', 'title', 'units', 'encoding_id', 'instance_id', 'notes']] 9 | info_df['ix'] = info_df.field_id 10 | info_df.set_index('ix', inplace=True) 11 | 12 | columns = [] 13 | cumu_columns = [columns.append(c.split("-")[0]) for c in df.columns if c.split("-")[0] not in columns] 14 | info_df = info_df[info_df.field_id.isin([int(c) for c in columns if c.isdigit()])] 15 | 16 | reverse_tbocti = {'time_between_pairs_matching_imaging': '20134-0.0', 17 | 'time_between_FI_imaging': '20135-0.0', 18 | 'time_between_Trails_imaging': '20136-0.0', 19 | 'time_between_SDMT_imaging': '20137-0.0', 20 | 'time_between_digit_span_imaging': '20138-0.0'} 21 | 22 | out_dict = {} 23 | 24 | for i, col in enumerate(columns): 25 | setup = {"field_id": col} 26 | if col == "eid": 27 | setup["title"] = 'Participant ID' 28 | setup["notes"] = "Encoded anonymised participant ID" 29 | out_dict[i] = setup 30 | elif col == "Age1stVisit": 31 | setup["title"] = "Age during Initial Visit to Assessment Center" 32 | setup["notes"] = 'Approximated float version of 21003-0.0"' 33 | out_dict[i] = setup 34 | elif col == "AgeRepVisit": 35 | setup["title"] = "Age during Repeat Visit to Assessment Center" 36 | setup["notes"] = 'Approximated float version of 21003-1.0"' 37 | out_dict[i] = setup 38 | elif col == "AgeAtScan": 39 | setup["title"] = "Age during Scan Visit to Assessment Center" 40 | setup["notes"] = 'Approximated float version of 21003-2.0"' 41 | out_dict[i] = setup 42 | elif col == "AgeAt2ndScan": 43 | setup["title"] = "Age during Repeat/Second Scan Visit to Assessment Center" 44 | setup["notes"] = 'Approximated float version of 21003-2.0"' 45 | out_dict[i] = setup 46 | elif col == "Race": 47 | setup["title"] = "Participant Race" 48 | setup["notes"] = 'Converted from and uses same data encoding as 21000-*.*"' 49 | out_dict[i] = setup 50 | elif col == "ISCED": 51 | setup["title"] = "Encoded anonymised participant ID" 52 | setup["notes"] = 'Converted from 6138-*.*"' 53 | out_dict[i] = setup 54 | elif col == "YearsOfEducation": 55 | setup["title"] = "Estimated Years of Education" 56 | setup["notes"] = 'Converted from 6138-*.*"' 57 | out_dict[i] = setup 58 | elif col == 'NP_controls_1': 59 | setup["title"] = "NeuroPsych Controls 1" 60 | setup["notes"] = "Exclusion ICD-10 Conditions: A8, B20-B24, B65-B83, C, F, G, I6, Q0, S04, S06, S07, S08, S09, T36-T46, T48-50; Exclusion SR Conditions: None" 61 | out_dict[i] = setup 62 | elif col == 'NP_controls_2': 63 | setup["title"] = "NeuroPsych Controls 2" 64 | setup["notes"] = "Exclusion ICD-10 Conditions: Same as NP_controls_1; Exclusion SR Conditions: 1075, 1081, 1086, 1220, 1234, 1243, 1246, 1247, 1250, 1256, 1258, 1260, 1261, 1262, 1264, 1266, 1267, 1286, 1287, 1288, 1291, 1371, 1408, 1434, 1437, 1439" 65 | out_dict[i] = setup 66 | elif col == 'CNS_controls_1': 67 | setup["title"] = "NeuroImaging Controls 1" 68 | setup["notes"] = "Exclusion ICD-10 Conditions: C70-C72, G, I6, Q0, R90, R940, S01-S09; Exclusion SR Conditions: 1081, 1086, 1266, 1267, 1397, 1434, 1437, 1491, 1626" 69 | out_dict[i] = setup 70 | elif col == 'CNS_controls_2': 71 | setup["title"] = "NeuroImaging Controls 2" 72 | setup["notes"] = "Exclusion ICD-10 Conditions: ICD-10 conditions from CNS_controls_1 as well as F20-F29, F31, F70-F79; Exclusion SR Conditions: Self-report conditions from CNS_controls_1 as well as 1243, 1289, 1291, 1371" 73 | out_dict[i] = setup 74 | elif col in reverse_tbocti.keys(): 75 | label = reverse_tbocti[col].split("-")[0] 76 | setup["title"] = reverse_tbocti[col] 77 | setup["units"] = "days" 78 | setup["notes"] = 'Number of days between online cognitive test ({label}-*.*) and imaging visit'.format(label=label) 79 | out_dict[i] = setup 80 | else: 81 | try: 82 | if int(col) in info_df.index.tolist(): 83 | info_df.loc[int(col), "field_id"] = '{fi}-*.*'.format(fi=info_df.loc[int(col), "field_id"]) 84 | info_df.loc[int(col), "encoding_id"] = '{fi}'.format(fi=info_df.loc[int(col), "encoding_id"]) 85 | info_df.loc[int(col), "instance_id"] = '{fi}'.format(fi=info_df.loc[int(col), "instance_id"]) 86 | if not isinstance(info_df.loc[int(col), "notes"], float): 87 | info_df.loc[int(col), "notes"] = info_df.loc[int(col), "notes"].replace("<", "\<").replace(">", "\>") 88 | else: 89 | info_df.loc[int(col), "field_id"] = '{fi}-*.*'.format(fi=col) 90 | except ValueError: 91 | continue 92 | 93 | out_df = pd.DataFrame.from_dict(out_dict, orient="index") 94 | info_df.sort_index(inplace=True) 95 | try: 96 | out_df = pd.concat([out_df, info_df], ignore_index=True, sort=True) 97 | except TypeError: 98 | out_df = pd.concat([out_df, info_df], ignore_index=True) 99 | out_df = out_df[['field_id', 'title', 'units', 'encoding_id', 'instance_id', 'notes']] 100 | out_df.fillna("", inplace=True) 101 | html = outcsv.split(".csv")[0] + "_header_key.html" 102 | out_df.to_html(html, escape=False, index=False) 103 | 104 | with open(html, 'r') as f: 105 | ll = f.read() 106 | with open(html, 'w') as f: 107 | todays_date = datetime.datetime.today() 108 | f.write('\n{arglist}
was run on {year}/{month}/{day} at {hour}:{minute}:{second}\n


'.format(arglist=arglist, 109 | year=todays_date.year, month=todays_date.month, day=todays_date.day, hour=todays_date.hour, minute=todays_date.minute, second=todays_date.second)) 110 | f.write(ll) 111 | return 112 | -------------------------------------------------------------------------------- /ukbb_parser/scripts/data/mapped_category_tree.json: -------------------------------------------------------------------------------- 1 | {"description": "This category tree was created using schema 1 and 13 as downloaded from http://biobank.ctsu.ox.ac.uk/crystal/schema.cgi", "updated": "20231221", "tree": {"1": {"categories": ["100094", "2"]}, "42": {"categories": ["45", "46", "47", "48", "49", "44", "50", "43"]}, "100": {"categories": ["108", "110", "112", "119", "106", "107", "111", "109", "200"], "datafields": ["12139", "12187", "12188", "12652", "12663", "12704", "24419", "25780"]}, "102": {"categories": ["128", "133", "157", "162"], "datafields": ["12624", "20207", "20208", "20209", "20210", "20211", "20212", "20213", "20214"]}, "103": {"categories": ["125", "124"], "datafields": ["12141", "12253", "12254", "20158", "21139"]}, "105": {"categories": ["156", "126", "131", "149", "158"], "datafields": ["12140", "12223", "12224", "20201"]}, "107": {"categories": ["134", "135", "204"], "datafields": ["20218", "20250", "24450", "24451", "24452", "24453", "24454", "24455", "24456", "25737", "25746", "25921", "25922", "25928"]}, "110": {"categories": ["1101", "1102", "190", "195", "197", "196", "194", "193", "192", "191"], "datafields": ["20216", "20252", "20263", "25000", "25001", "25002", "25003", "25004", "25005", "25006", "25007", "25008", "25009", "25010", "25025", "25731", "25732", "25733", "25734", "25735", "25756", "25757", "25758", "25759", "25925", "26500"]}, "111": {"categories": ["203"], "datafields": ["20225", "20227", "24420", "24421", "24422", "24423", "24424", "24425", "24426", "24427", "24428", "24429", "24430", "24431", "24432", "24433", "24434", "24435", "24436", "24437", "24438", "24439", "24440", "24441", "24442", "24443", "24457", "24458", "24459", "24460", "24461", "24462", "24463", "24464", "24465", "24466", "25739", "25741", "25743", "25744", "25750", "25751", "25752", "25753", "25754", "25755", "25923", "25929"]}, "113": {"categories": ["150", "114", "115", "151", "100115", "603"], "datafields": ["24025"]}, "116": {"categories": ["164", "155", "118", "121", "122", "117", "120", "161"], "datafields": ["22017"]}, "123": {"categories": ["130", "132"], "datafields": ["22016", "22499", "22500", "22501"]}, "136": {"categories": ["137", "138", "139", "140", "141", "142", "143", "144", "145", "146", "147"], "datafields": ["20400"]}, "156": {"categories": ["159"], "datafields": ["20243", "20264", "20265", "20267"]}, "170": {"categories": ["171", "172"], "datafields": ["23141", "23142", "23143", "23144", "23157", "23158", "23159", "23170", "23171", "23172", "23173", "23174", "32050"]}, "180": {"categories": ["185", "270", "184", "187", "186"], "datafields": ["24302", "24303"]}, "183": {"categories": ["181", "182"]}, "186": {"categories": ["183", "271", "272", "273", "274"], "datafields": ["23375"]}, "200": {"categories": ["201", "202"], "datafields": ["31000"]}, "220": {"categories": ["221", "222"], "datafields": ["20280", "20281", "23400", "23401", "23402", "23403", "23404", "23405", "23406", "23407", "23408", "23409", "23410", "23411", "23412", "23413", "23414", "23415", "23416", "23417", "23418", "23419", "23420", "23421", "23422", "23423", "23424", "23425", "23426", "23427", "23428", "23429", "23430", "23431", "23432", "23433", "23434", "23435", "23436", "23437", "23438", "23439", "23440", "23441", "23442", "23443", "23444", "23445", "23446", "23447", "23448", "23449", "23450", "23451", "23452", "23453", "23454", "23455", "23456", "23457", "23458", "23459", "23460", "23461", "23462", "23463", "23464", "23465", "23466", "23467", "23468", "23469", "23470", "23471", "23472", "23473", "23474", "23475", "23476", "23477", "23478", "23479", "23480", "23481", "23482", "23483", "23484", "23485", "23486", "23487", "23488", "23489", "23490", "23491", "23492", "23493", "23494", "23495", "23496", "23497", "23498", "23499", "23500", "23501", "23502", "23503", "23504", "23505", "23506", "23507", "23508", "23509", "23510", "23511", "23512", "23513", "23514", "23515", "23516", "23517", "23518", "23519", "23520", "23521", "23522", "23523", "23524", "23525", "23526", "23527", "23528", "23529", "23530", "23531", "23532", "23533", "23534", "23535", "23536", "23537", "23538", "23539", "23540", "23541", "23542", "23543", "23544", "23545", "23546", "23547", "23548", "23549", "23550", "23551", "23552", "23553", "23554", "23555", "23556", "23557", "23558", "23559", "23560", "23561", "23562", "23563", "23564", "23565", "23566", "23567", "23568", "23569", "23570", "23571", "23572", "23573", "23574", "23575", "23576", "23577", "23578", "23579", "23580", "23581", "23582", "23583", "23584", "23585", "23586", "23587", "23588", "23589", "23590", "23591", "23592", "23593", "23594", "23595", "23596", "23597", "23598", "23599", "23600", "23601", "23602", "23603", "23604", "23605", "23606", "23607", "23608", "23609", "23610", "23611", "23612", "23613", "23614", "23615", "23616", "23617", "23618", "23619", "23620", "23621", "23622", "23623", "23624", "23625", "23626", "23627", "23628", "23629", "23630", "23631", "23632", "23633", "23634", "23635", "23636", "23637", "23638", "23639", "23640", "23641", "23642", "23643", "23644", "23645", "23646", "23647", "23648"]}, "263": {"categories": ["100315", "100035", "100313", "199001"]}, "300": {"categories": ["301", "302"], "datafields": ["26200", "26201"]}, "347": {"categories": ["348", "349"]}, "994": {"categories": ["989", "990"], "datafields": ["28175", "28176", "28177", "28178"]}, "995": {"categories": ["993", "992", "991"], "datafields": ["28170", "28171", "28172", "28173", "28174"]}, "999": {"categories": ["997", "996", "998", "995", "994"], "datafields": ["32040", "32041", "40100", "41000", "41001"]}, "1008": {"categories": ["1020", "1009", "1010", "1011", "1012", "1013"], "datafields": ["90001", "90004", "110005", "110006"]}, "1306": {"categories": ["1419", "100014"], "datafields": ["5268"]}, "1500": {"categories": ["1501", "1502", "1503", "1504", "1505", "1506", "1507", "1508", "1509", "1510", "1511", "1512", "1513"], "datafields": ["29183"]}, "1712": {"categories": ["2401", "2403", "2404", "2405", "2406", "2407", "2408", "2409", "2410", "2411", "2412", "2413", "2414", "2415", "2416", "2417"]}, "1838": {"categories": ["1839"]}, "2000": {"categories": ["2006", "2002", "2005", "2003", "2004", "2001"], "datafields": ["41083", "41084", "41085", "41086", "41087", "41088", "41089", "41090", "41091", "41092", "41093", "41094", "41095", "41096", "41097", "41098", "41099", "41100", "41101", "41102", "41103", "41109", "41110", "41111", "41112", "41132", "41236", "41237", "41238", "41239", "41240", "41241", "41242", "41243", "41252", "41254"]}, "3000": {"categories": ["3001"]}, "17518": {"categories": ["18518"], "datafields": ["30600", "30610", "30620", "30630", "30640", "30650", "30660", "30670", "30680", "30690", "30700", "30710", "30720", "30730", "30740", "30750", "30760", "30770", "30780", "30790", "30800", "30810", "30820", "30830", "30840", "30850", "30860", "30870", "30880", "30890"]}, "51428": {"categories": ["1307"], "datafields": ["23050", "23051", "23052", "23053", "23054", "23055", "23056", "23057", "23058", "23059", "23060", "23061", "23062", "23063", "23064", "23065", "23066", "23067", "23068", "23069", "23070", "23071", "23073", "23074", "23075"]}, "100000": {"categories": ["100021", "100025", "100026", "100071", "100006", "100013", "100003", "100001", "100004"]}, "100001": {"categories": ["100002", "100095", "100096"]}, "100003": {"categories": ["105", "100", "102", "103"]}, "100004": {"categories": ["129", "152", "127"], "datafields": ["62", "111", "12148"]}, "100006": {"categories": ["100011", "101", "100007", "100049", "100019", "100008", "100018", "100020", "104", "100012"]}, "100008": {"categories": ["100010", "100009"]}, "100013": {"categories": ["1306", "100099", "100016", "100017", "100015"], "datafields": ["5182", "5183"]}, "100016": {"categories": ["100079", "100116"], "datafields": ["5270", "6070", "6071", "6072", "6073", "20061", "20062", "21011", "21012", "21013", "21014", "21015", "21016", "21017", "21018"]}, "100021": {"categories": ["100024", "100023", "100022"], "datafields": ["110003", "110008"]}, "100025": {"categories": ["100062", "100050", "100033", "100034", "100059", "100036", "100068"]}, "100026": {"categories": ["100032", "100029", "100027", "505", "501", "502", "1358", "504", "503", "506", "100031", "100030", "100028", "100077"]}, "100036": {"categories": ["100041", "100046", "100042", "100037", "100038", "100048", "100039", "100040", "100047", "100044", "100045", "100043"]}, "100050": {"categories": ["100054", "100053", "100057", "100058", "100052", "100051", "100055", "100056"]}, "100054": {"categories": ["54"], "datafields": ["864", "874", "884", "894", "904", "914", "924", "943", "971", "981", "991", "1001", "1011", "1021", "1070", "1080", "1090", "1100", "2624", "2634", "3637", "3647", "6162", "6164", "10953", "10962", "10971"]}, "100059": {"categories": ["100061", "100060"]}, "100062": {"categories": ["100066", "100064", "100063", "100065", "100067"]}, "100068": {"categories": ["100070", "100069"]}, "100071": {"categories": ["100072", "100073", "100074", "100075", "100076"]}, "100078": {"categories": ["100080", "100082", "100084", "100083"]}, "100080": {"categories": ["100081", "17518", "51428", "220", "1838", "163"]}, "100081": {"categories": ["9081"], "datafields": ["30000", "30010", "30020", "30030", "30040", "30050", "30060", "30070", "30080", "30090", "30100", "30110", "30120", "30130", "30140", "30150", "30160", "30170", "30180", "30190", "30200", "30210", "30220", "30230", "30240", "30250", "30260", "30270", "30280", "30290", "30300"]}, "100083": {"categories": ["148"], "datafields": ["30500", "30505", "30510", "30515", "30520", "30525", "30530", "30535"]}, "100084": {"categories": ["100085", "100086", "100087"]}, "100088": {"categories": ["113", "1008", "347"]}, "100089": {"categories": ["160", "1500", "116", "100090", "153", "154", "1039", "136", "123"]}, "100090": {"categories": ["100117", "100118", "100097", "100104", "100100", "100102", "100107", "100101", "100109", "100106", "100111", "100110", "100103", "100105", "100112", "100108", "100114", "100098"], "datafields": ["100026", "110001", "110002"]}, "100091": {"categories": ["999", "3000", "2000", "100093", "100092", "42", "1712"], "datafields": ["20142"]}, "100094": {"categories": ["76"], "datafields": ["31", "33", "34", "52", "189", "21022", "22189"]}, "100314": {"categories": ["300", "264", "263", "100319", "170", "180", "265"]}, "199001": {"categories": ["100316", "100317"], "datafields": ["22010", "22011", "22012", "22013", "22018", "22050", "22051", "22052"]}, "152": {"datafields": ["3", "4", "5", "6", "630", "5198", "6022", "10241", "21611", "21621", "21622", "21623", "21625", "21631", "21632", "21633", "21634", "21636", "21638", "21641", "21642", "21651", "21661", "21662", "21663", "21664", "21665", "21666", "21671"]}, "100018": {"datafields": ["19", "45", "77", "78", "3081", "3082", "3083", "3084", "3085", "3086", "3143", "3144", "3146", "3147", "3148", "4092", "4093", "4095", "4096", "4100", "4101", "4103", "4104", "4105", "4106", "4119", "4120", "4122", "4123", "4124", "4125", "4138", "4139", "4140", "4141", "4142", "4143", "4144", "4145", "4146", "4147"]}, "100010": {"datafields": ["21", "39", "40", "41", "44", "48", "49", "50", "51", "3077", "3160", "12143", "12144", "20015", "20041", "20045", "20046", "20047", "20048", "21001", "21002"]}, "100020": {"datafields": ["23", "42", "3059", "3060", "3061", "3062", "3063", "3064", "3065", "3066", "3088", "3089", "3090", "3132", "3137", "3159", "10691", "10693", "10694", "10695", "10696", "10697", "10711", "10714", "10717", "20031", "20032", "20042", "20150", "20151", "20152", "20153", "20154", "20255", "20256", "20257", "20258"]}, "100002": {"datafields": ["35", "68", "74", "3166", "20049", "20050"]}, "100011": {"datafields": ["36", "37", "93", "94", "95", "96", "102", "4079", "4080", "4081"]}, "100019": {"datafields": ["38", "46", "47", "20043", "20044"]}, "100009": {"datafields": ["43", "6218", "6219", "6220", "6221", "6222", "23098", "23099", "23100", "23101", "23102", "23104", "23105", "23106", "23107", "23108", "23109", "23110", "23111", "23112", "23113", "23114", "23115", "23116", "23117", "23118", "23119", "23120", "23121", "23122", "23123", "23124", "23125", "23126", "23127", "23128", "23129", "23130"]}, "100024": {"datafields": ["53", "54", "55", "20033", "20034", "20074", "20075", "20118", "20269", "20270", "20271", "20272", "20273", "20274", "20275", "20276", "21003", "22686", "22687", "22688", "22689"]}, "100074": {"datafields": ["84", "87", "134", "135", "3140", "20001", "20002", "20006", "20007", "20008", "20009", "20012", "20013"]}, "100076": {"datafields": ["92", "136", "3079", "20004", "20010", "20011", "20014"]}, "100072": {"datafields": ["115", "117", "118", "120", "129", "130", "146", "20022", "20115"]}, "100073": {"datafields": ["132", "20024", "20121", "20277"]}, "100075": {"datafields": ["137", "6671", "20003", "20199"]}, "2": {"datafields": ["190", "191", "20005", "20143", "20144", "20145", "110007"]}, "100023": {"datafields": ["200", "393"]}, "100030": {"datafields": ["396", "397", "398", "399", "400", "6334", "10133", "10134", "10136", "10137", "10138"]}, "100032": {"datafields": ["401", "402", "403", "404", "10139", "10140", "10141", "10147", "20023"]}, "100066": {"datafields": ["670", "680", "699", "709", "728", "738", "6139", "6140", "6141", "10860", "10877"]}, "100064": {"datafields": ["757", "767", "777", "796", "806", "816", "826", "3426", "6142", "6143", "20119"]}, "100063": {"datafields": ["845", "6138", "10722"]}, "100061": {"datafields": ["1031", "2110", "6160", "10740"]}, "100055": {"datafields": ["1050", "1060", "1717", "1727", "1737", "1747", "1757", "2267", "2277"]}, "100053": {"datafields": ["1110", "1120", "1130", "1140", "1150", "2237", "10016", "10105", "10114", "10749", "10886"]}, "100057": {"datafields": ["1160", "1170", "1180", "1190", "1200", "1210", "1220"]}, "100058": {"datafields": ["1239", "1249", "1259", "1269", "1279", "2644", "2867", "2877", "2887", "2897", "2907", "2926", "2936", "3436", "3446", "3456", "3466", "3476", "3486", "3496", "3506", "5959", "6157", "6158", "6183", "6194", "10115", "10827", "10895", "20116", "20160", "20161", "20162"]}, "100052": {"datafields": ["1289", "1299", "1309", "1319", "1329", "1339", "1349", "1359", "1369", "1379", "1389", "1408", "1418", "1428", "1438", "1448", "1458", "1468", "1478", "1488", "1498", "1508", "1518", "1528", "1538", "1548", "2654", "3680", "6144", "10767", "10776", "10855", "10912"]}, "100051": {"datafields": ["1558", "1568", "1578", "1588", "1598", "1608", "1618", "1628", "2664", "3731", "3859", "4407", "4418", "4429", "4440", "4451", "4462", "5364", "10818", "10853", "20117"]}, "100033": {"datafields": ["1647", "1677", "1687", "1697", "1707", "1767", "1777", "1787"]}, "100034": {"datafields": ["1797", "1807", "1835", "1845", "1873", "1883", "2946", "3526", "3912", "3942", "3972", "3982", "4501", "5057", "20107", "20110", "20111", "20112", "20113", "20114"]}, "100060": {"datafields": ["1920", "1930", "1940", "1950", "1960", "1970", "1980", "1990", "2000", "2010", "2020", "2030", "2040", "2050", "2060", "2070", "2080", "2090", "2100", "4526", "4537", "4548", "4559", "4570", "4581", "4598", "4609", "4620", "4631", "4642", "4653", "5375", "5386", "5663", "5674", "6145", "6156", "10721", "20122", "20123", "20124", "20125", "20126", "20127"]}, "100056": {"datafields": ["2129", "2139", "2149", "2159", "3669"]}, "100042": {"datafields": ["2178", "2188", "2296", "2306"]}, "100041": {"datafields": ["2207", "2217", "2227", "4689", "4700", "5408", "5419", "5430", "5441", "5610", "5832", "5843", "5855", "5877", "5890", "5901", "5912", "5923", "5934", "5945", "6119", "6147", "6148", "6205"]}, "100043": {"datafields": ["2247", "2257", "3393", "4792", "4803", "4814", "4825", "4836", "10793"]}, "100037": {"datafields": ["2316", "4717"]}, "100039": {"datafields": ["2335", "3606", "3616", "3751"]}, "100040": {"datafields": ["2345", "2355", "2365", "3809"]}, "100070": {"datafields": ["2375", "2385", "2395", "2405"]}, "100047": {"datafields": ["2415", "2844"]}, "100044": {"datafields": ["2443", "2453", "2463", "2473", "2966", "2976", "2986", "3005", "3627", "3761", "3786", "3894", "3992", "4012", "4022", "4041", "4056", "6150", "6151", "6152", "10844"]}, "100045": {"datafields": ["2492", "6153", "6154", "6155", "6177", "6179", "10004", "10005", "10007", "10723", "10854"]}, "100069": {"datafields": ["2674", "2684", "2694", "2704", "2714", "2724", "2734", "2744", "2754", "2764", "2774", "2784", "2794", "2804", "2814", "2824", "2834", "3536", "3546", "3581", "3591", "3700", "3710", "3720", "3829", "3839", "3849", "3872", "3882", "10132"]}, "100048": {"datafields": ["2956", "3404", "3414", "3571", "3741", "3773", "3799", "4067", "6159"]}, "100065": {"datafields": ["3659", "21000"]}, "100007": {"datafields": ["4136", "4186", "4194", "4195", "4196", "4198", "4199", "4200", "4204", "4205", "4206", "4207", "20051", "21021"]}, "100049": {"datafields": ["4229", "4230", "4232", "4233", "4234", "4235", "4236", "4237", "4238", "4239", "4240", "4241", "4242", "4243", "4244", "4245", "4246", "4247", "4248", "4249", "4268", "4269", "4270", "4272", "4275", "4276", "4277", "4279", "4849", "20019", "20021"]}, "100029": {"datafields": ["4250", "4251", "4252", "4253", "4254", "4255", "4256", "4257", "4258", "4259", "4260", "4281", "4282", "4283", "4285"]}, "100031": {"datafields": ["4286", "4287", "4288", "4289", "4290", "4291", "4292", "4293", "4294", "4295", "20018"]}, "100067": {"datafields": ["4674", "6146"]}, "100038": {"datafields": ["4728", "5452", "5463", "5474", "5485", "5496", "5507", "5518", "5529", "5540"]}, "100027": {"datafields": ["4924", "4935", "4946", "4957", "4968", "4979", "4990", "5001", "5012", "5556", "5699", "5779", "5790", "5866", "20016", "20128"]}, "100017": {"datafields": ["5074", "5075", "5076", "5077", "5078", "5079", "5080", "5081", "5082", "5083", "5185", "5186", "5187", "5188", "5199", "5200", "5201", "5202", "5204", "5205", "5206", "5207", "5208", "5209", "5211", "5212", "6074", "6075", "20056", "20057", "20261", "20262"]}, "100014": {"datafields": ["5084", "5085", "5086", "5087", "5088", "5089", "5090", "5091", "5096", "5097", "5098", "5099", "5100", "5101", "5102", "5103", "5104", "5105", "5106", "5107", "5108", "5109", "5110", "5111", "5112", "5113", "5114", "5115", "5116", "5117", "5118", "5119", "5132", "5133", "5134", "5135", "5136", "5138", "5139", "5140", "5141", "5142", "5143", "5144", "5145", "5146", "5147", "5148", "5149", "5152", "5155", "5156", "5157", "5158", "5159", "5160", "5161", "5162", "5163", "5164", "5189", "5190", "5191", "5193", "5214", "5215", "5221", "5237", "5251", "5273", "5274", "5276", "5292", "5306", "20052", "20055"]}, "100099": {"datafields": ["5181", "5324", "5325", "5326", "5327", "5328"]}, "100015": {"datafields": ["5194", "5196", "5253", "5254", "5255", "5256", "5257", "5258", "5259", "5261", "5262", "5263", "5264", "5265", "5266", "5267", "20053", "20054"]}, "100012": {"datafields": ["5983", "5984", "5985", "5986", "5987", "5988", "5990", "5991", "5992", "5993", "6014", "6015", "6016", "6017", "6019", "6020", "6023", "6024", "6025", "6032", "6033", "6034", "6038", "6039", "20058", "20059", "20060"]}, "100046": {"datafields": ["6149", "10006"]}, "503": {"datafields": ["6312", "6313", "6383", "21004"]}, "504": {"datafields": ["6314", "6315", "6317", "6364", "6365", "26302", "26306"]}, "502": {"datafields": ["6325", "6362", "23321", "23322", "23323", "23324"]}, "501": {"datafields": ["6332", "6333", "6373", "6374"]}, "505": {"datafields": ["6348", "6349", "6350", "6351", "6770", "6771", "6772", "6773"]}, "506": {"datafields": ["6448", "6459", "6470", "6481", "6492", "6503", "6514", "6525", "6536", "6547", "20197"]}, "1419": {"datafields": ["7381", "7382", "7383", "7384", "7385", "7386", "7387", "7388", "7389", "7390", "7391", "7392", "7393", "7394", "7395", "7396", "7397", "7398", "7399", "7400", "7401", "7402", "7403", "7404", "7405", "7406", "7407", "7408", "7409", "7410", "7476", "7486", "7494", "7495", "7496", "7497", "7498", "7499", "7500", "7501", "7502", "7503", "7504", "7505", "7507", "7508", "7509", "7510", "7511", "7512", "7513", "7514", "7515", "7516", "7517", "7518", "7519", "7520", "7521", "7522", "7523", "7524", "7525", "7526", "7527", "7529", "7530", "7531", "7532", "7533", "7534", "7535", "7536", "7537", "7538", "7539", "7541", "7542", "7544"]}, "1358": {"datafields": ["7669", "7670", "7671", "7672", "7673", "7674", "7676", "7677", "7678", "20139", "20141"]}, "100028": {"datafields": ["10142", "10143", "10144", "10145", "10146"]}, "100077": {"datafields": ["10609", "10610", "10612"]}, "101": {"datafields": ["12291", "12292", "20222", "20223", "20226", "20241", "22670", "22671", "22672", "22673", "22674", "22675", "22676", "22677", "22678", "22679", "22680", "22681", "22682", "22683", "22684", "22685"]}, "104": {"datafields": ["12323", "12336", "12338", "12340", "12653", "12654", "12657", "12658", "20205", "22330", "22331", "22332", "22333", "22334", "22335", "22336", "22337", "22338"]}, "106": {"datafields": ["12651", "12706", "20217", "20249", "24444", "24445", "24446", "24447", "24448", "24449", "25040", "25042", "25044", "25046", "25048", "25050", "25052", "25054", "25740", "25742", "25745", "25747", "25748", "25749", "25761", "25762", "25763", "25764", "25765", "25766", "25767", "25768", "25924", "25930"]}, "128": {"datafields": ["12671", "12673", "12674", "12675", "12676", "12677", "12678", "12679", "12680", "12681", "12682", "12683", "12684", "12685", "12686", "12687", "12688", "12695", "12697", "12698", "12699", "12700", "12702"]}, "100096": {"datafields": ["20025", "20071"]}, "100095": {"datafields": ["20035", "20072"]}, "100114": {"datafields": ["20077", "20078", "20079", "20080", "20081", "20082", "20083", "105010", "105030"]}, "100112": {"datafields": ["20084", "104670"]}, "100097": {"datafields": ["20085", "20086", "100010", "100020"]}, "100101": {"datafields": ["20087", "20091", "20092", "20093", "20094", "20098", "20099", "20100", "20101", "20102", "20103", "20104", "100940", "100950", "101020", "101090", "101160", "101230", "101240", "101250", "101260", "101270", "101300", "101310", "101350", "101390", "101430", "101470", "101510", "101550", "102700", "102710", "102720", "102730", "102740", "102750", "102760", "102770", "102780"]}, "100110": {"datafields": ["20088", "20090", "103310", "103980", "104660"]}, "100105": {"datafields": ["20089"]}, "100100": {"datafields": ["20095", "20096", "20097", "100580", "100590", "100630", "100670", "100710", "100720", "100730", "100740"]}, "100102": {"datafields": ["20105", "100760", "100770", "100800", "100810", "100820", "100830", "100840", "100850", "100860", "100880", "100890", "100900", "100910"]}, "100109": {"datafields": ["20106", "20108", "20109", "101970", "101980", "101990", "102000", "102010", "102020", "102030", "102040", "102050", "102060", "102070", "102080", "102090", "102120", "102130", "102140", "102150", "102170", "102180", "102190", "102200", "102210", "102220", "102230", "102250", "102260", "102270", "102280", "102290", "102300", "102310", "102320", "102330", "102340", "102350", "102360", "102370", "102380", "102400", "102410", "102420", "102430", "102440", "102450", "102460", "102470", "102480", "102490", "102500", "102520", "102530", "102540", "102620"]}, "117": {"datafields": ["20129", "20130", "20131", "20132", "20133", "20134", "20244"]}, "118": {"datafields": ["20135", "20165", "20166", "20167", "20168", "20169", "20170", "20171", "20172", "20173", "20174", "20175", "20176", "20177", "20178", "20179", "20180", "20181", "20182", "20183", "20184", "20185", "20186", "20187", "20188", "20189", "20190", "20191", "20192", "20193", "20194", "20242"]}, "121": {"datafields": ["20136", "20147", "20148", "20149", "20155", "20156", "20157", "20246", "20247", "20248"]}, "122": {"datafields": ["20137", "20159", "20195", "20196", "20198", "20200", "20229", "20230", "20245"]}, "120": {"datafields": ["20138", "20240"]}, "164": {"datafields": ["20140", "23077", "23078", "29250"]}, "131": {"datafields": ["20202", "20206", "20259", "20260"]}, "126": {"datafields": ["20203", "20204", "20254", "22400", "22401", "22402", "22417", "40060", "40061", "40062", "40063"]}, "108": {"datafields": ["20215", "20224"]}, "109": {"datafields": ["20219", "20251", "24418", "24467", "24468", "24469", "24470", "24471", "24472", "24473", "24474", "24475", "24476", "24477", "24478", "24479", "24480", "24481", "24482", "24483", "24484", "25026", "25027", "25028", "25029", "25030", "25031", "25032", "25033", "25034", "25035", "25036", "25037", "25038", "25039", "25738", "25927", "26301"]}, "112": {"datafields": ["20220", "20221", "20253", "24485", "24486", "25736", "25781", "25926"]}, "119": {"datafields": ["20266", "24360", "24361", "24362", "24363", "24364", "24365", "24366", "24367", "24368", "24369", "24370", "24371", "24372", "24373", "24374", "24375", "24376", "24377", "24378", "24379", "24380", "24381", "24382", "24383", "24384", "24385", "24386", "24387", "24388", "24389", "24390", "24391", "24392", "24393", "24394", "24395", "24396", "24397", "24398", "24399", "24400", "24401", "24402", "24403", "24404", "24405", "24406", "24407", "24408", "24409", "26300"]}, "271": {"datafields": ["20278", "20279", "23191", "23193", "23197", "23198", "23199", "23319", "23345", "23346", "23347", "23348", "23349", "23350", "23351", "23352", "23353", "23365", "24304", "24305", "24306"]}, "222": {"datafields": ["20282", "20283", "23649", "23650", "23651", "23652", "23653", "23654", "23655", "23658", "23659", "23660"]}, "125": {"datafields": ["20300", "20301", "20302", "20303", "20304", "20305", "20306", "20307", "20308", "20309", "20310", "20311", "20312", "20313", "20314", "20315", "20316", "20317", "20318", "20319", "20320", "20321", "21005", "21136", "21137", "21138", "23200", "23201", "23202", "23203", "23204", "23205", "23206", "23207", "23208", "23209", "23210", "23211", "23212", "23213", "23214", "23215", "23216", "23217", "23218", "23219", "23220", "23221", "23222", "23223", "23224", "23225", "23226", "23227", "23228", "23229", "23230", "23231", "23232", "23233", "23234", "23235", "23236", "23237", "23238", "23239", "23240", "23241", "23242", "23243", "23290", "23291", "23292", "23293", "23294", "23295", "23296", "23297", "23298", "23299", "23300", "23301", "23302", "23303", "23304", "23305", "23306", "23307", "23308", "23309", "23310", "23311", "23312", "23313", "23314", "23315", "23316", "23317", "23318", "23320", "23325", "23326", "23327", "23328", "23329", "23330", "23331", "23332", "23333", "23334", "23335", "23336", "23337", "23338", "23339", "23340", "23341", "23342", "23343", "23344"]}, "141": {"datafields": ["20401", "20404", "20406", "20415", "20431", "20432", "20456", "20457", "20503", "20504", "20551", "20552"]}, "142": {"datafields": ["20403", "20405", "20407", "20408", "20409", "20410", "20411", "20412", "20413", "20414", "20416"]}, "140": {"datafields": ["20417", "20418", "20419", "20420", "20421", "20422", "20423", "20425", "20426", "20427", "20428", "20429", "20505", "20506", "20509", "20512", "20515", "20516", "20520", "20537", "20538", "20539", "20540", "20541", "20542", "20543", "20549", "20550"]}, "138": {"datafields": ["20433", "20434", "20435", "20436", "20437", "20438", "20439", "20440", "20441", "20442", "20445", "20446", "20447", "20448", "20449", "20450", "20507", "20508", "20510", "20511", "20513", "20514", "20517", "20518", "20519", "20532", "20533", "20534", "20535", "20536", "20546", "20547"]}, "143": {"datafields": ["20453", "20454", "20455"]}, "147": {"datafields": ["20458", "20459", "20460"]}, "144": {"datafields": ["20461", "20462", "20463", "20465", "20466", "20467", "20468", "20470", "20471", "20473", "20474", "20476", "20477"]}, "146": {"datafields": ["20479", "20480", "20481", "20482", "20483", "20484", "20485", "20486", "20553", "20554"]}, "145": {"datafields": ["20487", "20488", "20489", "20490", "20491", "20494", "20495", "20496", "20497", "20498", "20521", "20522", "20523", "20524", "20525", "20526", "20527", "20528", "20529", "20530", "20531"]}, "139": {"datafields": ["20492", "20493", "20501", "20502", "20548"]}, "137": {"datafields": ["20499", "20500", "20544"]}, "1039": {"datafields": ["20599", "20600", "20601", "20602", "20603", "20604", "20605", "20606", "20607", "20608", "20609", "20610", "20611", "20612", "20613", "20614", "20615", "20616", "20617", "20618", "20619", "20620", "20621", "20622", "20623", "20624", "20625", "20626", "20627", "20628", "20629", "20630", "20631", "20632", "20633", "20634", "20635", "20636", "20637", "20638", "20639", "20640", "20641", "20642", "20643", "20644", "20645", "20646", "20647", "20648", "20649", "20650", "20651", "20652", "20653", "20654", "20655", "20656", "20657", "20658", "20659", "20660", "20661", "20662", "20663", "20664", "20665", "20666", "20667", "20668", "20669", "20670", "20671", "20672", "20673", "20674", "20675", "20676", "20677", "20678", "20679", "20680", "20681", "20682", "20683", "20684", "20685", "20686", "20687", "20688", "20689", "20690", "20691", "20692", "20693", "20694", "20695", "20696", "20697", "20698", "20699", "20700", "20701", "20702", "20703", "20704", "20705", "20706", "20707", "20708", "20709", "20710", "20711", "20712", "20713", "20714", "20715", "20716", "20717", "20718", "20719", "20720", "20721", "20722", "20723", "20724", "20725", "20726", "20727", "20728", "20729", "20730", "20731", "20732", "20733", "20734", "20735", "20736", "20737", "20738", "20739", "20740", "20741", "20742", "20743", "20744", "20745", "20746", "20747", "20748", "20749", "20750", "20751"]}, "161": {"datafields": ["20760", "20761", "20762", "20763", "20764", "20765"]}, "100319": {"datafields": ["21007", "21008", "22438", "22800", "22801", "22802", "22803", "22804", "22805", "22806", "22807", "22808", "22809", "22810", "22811", "22812", "22813", "22814", "22815", "22816", "22817", "22818", "22819", "22820", "22821", "22822", "22823", "22824", "22825", "22828"]}, "153": {"datafields": ["21023", "21024", "21025", "21026", "21027", "21028", "21029", "21030", "21031", "21032", "21033", "21034", "21035", "21036", "21037", "21038", "21039", "21040", "21041", "21042", "21043", "21044", "21045", "21046", "21047", "21048", "21049", "21050", "21051", "21052", "21053", "21054", "21055", "21056", "21057", "21058", "21059", "21060", "21061", "21062", "21063", "21064", "21065", "21066", "21067", "21068", "21069", "21070", "21071", "21072", "21073", "21074", "21075", "21076"]}, "158": {"datafields": ["21080", "21081", "21082", "21083", "21084", "21085", "21086", "21087", "21088", "21089", "21090", "21091", "21092", "21093", "21094", "21170", "21171", "21173", "21174"]}, "603": {"datafields": ["21100", "21101", "21102", "21103", "21104", "21105"]}, "124": {"datafields": ["21110", "21111", "21112", "21113", "21114", "21115", "21116", "21117", "21118", "21119", "21120", "21121", "21122", "21123", "21124", "21125", "21126", "21127", "21128", "21129", "21130", "21131", "21132", "21133", "21134", "21135", "23244", "23245", "23246", "23247", "23248", "23249", "23250", "23251", "23252", "23253", "23254", "23255", "23256", "23257", "23258", "23259", "23260", "23261", "23262", "23263", "23264", "23265", "23266", "23267", "23268", "23269", "23270", "23271", "23272", "23273", "23274", "23275", "23276", "23277", "23278", "23279", "23280", "23281", "23282", "23283", "23284", "23285", "23286", "23287", "23288", "23289"]}, "301": {"datafields": ["21150", "21151", "21152", "26202", "26204", "26206", "26210", "26212", "26214", "26216", "26218", "26220", "26223", "26225", "26227", "26229", "26232", "26234", "26238", "26240", "26242", "26244", "26246", "26248", "26250", "26252", "26254", "26258", "26260", "26265", "26267", "26269", "26273", "26275", "26278", "26283", "26285", "26287", "26289"]}, "159": {"datafields": ["21160", "21161", "21162", "21163", "21164"]}, "127": {"datafields": ["21711", "21721", "21722", "21723", "21725", "21731", "21732", "21733", "21734", "21736", "21738", "21741", "21742", "21751", "21761", "21762", "21763", "21764", "21765", "21766", "21771"]}, "129": {"datafields": ["21811", "21821", "21822", "21823", "21825", "21831", "21832", "21833", "21834", "21836", "21838", "21841", "21842", "21851", "21861", "21862", "21863", "21864", "21865", "21866", "21871"]}, "100313": {"datafields": ["22000", "22001", "22002", "22003", "22004", "22005", "22006", "22007", "22008", "22009", "22019", "22020", "22021", "22022", "22023", "22024", "22025", "22026", "22027", "22028", "22029", "22030"]}, "100317": {"datafields": ["22014", "22015", "22301", "22302", "22303", "22304", "22305", "22306", "22307", "22308", "22309", "22310", "22311", "22312", "22313", "22314", "22315", "22316", "22317", "22318", "22319", "22320", "22321", "22322", "22323", "22324", "22325"]}, "54": {"datafields": ["22032", "22033", "22034", "22035", "22036", "22037", "22038", "22039", "22040"]}, "100315": {"datafields": ["22100", "22101", "22102", "22103", "22104", "22105", "22106", "22107", "22108", "22109", "22110", "22111", "22112", "22113", "22114", "22115", "22116", "22117", "22118", "22119", "22120", "22121", "22122", "22123", "22124", "22125", "22418", "22419", "22430", "22431", "22437"]}, "132": {"datafields": ["22126", "22127", "22128", "22129", "22130", "22131", "22132", "22133", "22134", "22135", "22136", "22137", "22138", "22139", "22140", "22141", "22146", "22147", "22148", "22149", "22150", "22151", "22152", "22153", "22154", "22155", "22156", "22157", "22158", "22159", "22160", "22161", "22166", "22167", "22168", "22169", "22170", "22171", "22172", "22173", "22174", "22175", "22176", "22177", "22178", "22179", "22180", "22181", "22502", "22503", "22504", "22505", "22506", "22507", "22508"]}, "100035": {"datafields": ["22182"]}, "265": {"datafields": ["22190", "22191", "22192", "22193", "22194"]}, "130": {"datafields": ["22200", "22599", "22600", "22601", "22602", "22603", "22604", "22605", "22606", "22607", "22608", "22609", "22610", "22611", "22612", "22613", "22614", "22615", "22616", "22617", "22618", "22619", "22620", "22630", "22631", "22640", "22641", "22642", "22643", "22644", "22645", "22650", "22651", "22652", "22653", "22654", "22655", "22660", "22661", "22662", "22663", "22664"]}, "100316": {"datafields": ["22201", "22202", "22203", "22204", "22205", "22206", "22207", "22208", "22209", "22210", "22211", "22212", "22213", "22214", "22215", "22216", "22217", "22218", "22219", "22220", "22221", "22222", "22223", "22224", "22225"]}, "348": {"datafields": ["22297", "22298", "22299", "22300", "24700", "24701", "24702", "24703", "24704", "24705", "24706", "24707", "24708", "24709", "24710", "24711", "24712", "24713", "24714", "24715", "24716", "24717", "24718", "24719", "24720", "24721", "24722", "24723", "24724", "24725", "24726", "24727", "24728", "24729", "24730", "24731", "24732", "24733", "24734", "24735", "24736", "24737", "24738", "24739", "24740", "24741", "24742", "24743"]}, "149": {"datafields": ["22403", "22404", "22405", "22406", "22407", "22408", "22409", "22410", "22411", "22412", "22413", "22414", "22415", "22416", "22432", "22433", "22434", "22435", "22436", "23355", "23356", "23357", "23358", "23359", "23360", "23361", "23362", "23363", "23364", "24352", "24353", "24354"]}, "133": {"datafields": ["22420", "22421", "22422", "22423", "22424", "22425", "22426", "22427"]}, "150": {"datafields": ["22700", "22701", "22702", "22703", "22704"]}, "1307": {"datafields": ["23000", "23001", "23002", "23003", "23004", "23005", "23006", "23007", "23008", "23009", "23010", "23011", "23012", "23013", "23014", "23015", "23016", "23017", "23018", "23019", "23020", "23021", "23022", "23023", "23024", "23025", "23026", "23027", "23028", "23029", "23030", "23031", "23032", "23033", "23034", "23035", "23036", "23037", "23038", "23039", "23040", "23041", "23042", "23043", "23044", "23048", "23049"]}, "155": {"datafields": ["23045", "23046", "23047", "23072", "23076", "23079"]}, "272": {"datafields": ["23080", "23081", "23082", "23083", "23084", "23085", "23086", "23087", "23088", "23089", "23090", "23091", "23092", "23093", "23094"]}, "171": {"datafields": ["23145", "23146", "23147", "23148", "23149", "23150", "23151", "23152", "23153", "23154", "23155", "23156", "23160", "23161", "23162", "23163", "23164", "23176", "23177", "23178", "23179"]}, "264": {"datafields": ["23165"]}, "181": {"datafields": ["23181", "23182"]}, "182": {"datafields": ["23183", "23184"]}, "274": {"datafields": ["23195", "23196"]}, "270": {"datafields": ["23354", "23370", "23372", "23374", "23376", "23377", "23378", "23379", "23380", "23381", "23382", "23383", "23384", "32069"]}, "221": {"datafields": ["23700", "23701", "23702", "23703", "23704", "23705", "23706", "23707", "23708", "23709", "23710", "23711", "23712", "23713", "23714", "23715", "23716", "23717", "23718", "23719", "23720", "23721", "23722", "23723", "23724", "23725", "23726", "23727", "23728", "23729", "23730", "23731", "23732", "23733", "23734", "23735", "23736", "23737", "23738", "23739", "23740", "23741", "23742", "23743", "23744", "23745", "23746", "23747", "23748", "23749", "23750", "23751", "23752", "23753", "23754", "23755", "23756", "23757", "23758", "23759", "23760", "23761", "23762", "23763", "23764", "23765", "23766", "23767", "23768", "23769", "23770", "23771", "23772", "23773", "23774", "23775", "23776", "23777", "23778", "23779", "23780", "23781", "23782", "23783", "23784", "23785", "23786", "23787", "23788", "23789", "23790", "23791", "23792", "23793", "23794", "23795", "23796", "23797", "23798", "23799", "23800", "23801", "23802", "23803", "23804", "23805", "23806", "23807", "23808", "23809", "23810", "23811", "23812", "23813", "23814", "23815", "23816", "23817", "23818", "23819", "23820", "23821", "23822", "23823", "23824", "23825", "23826", "23827", "23828", "23829", "23830", "23831", "23832", "23833", "23834", "23835", "23836", "23837", "23838", "23839", "23840", "23841", "23842", "23843", "23844", "23845", "23846", "23847", "23848", "23849", "23850", "23851", "23852", "23853", "23854", "23855", "23856", "23857", "23858", "23859", "23860", "23861", "23862", "23863", "23864", "23865", "23866", "23867", "23868", "23869", "23870", "23871", "23872", "23873", "23874", "23875", "23876", "23877", "23878", "23879", "23880", "23881", "23882", "23883", "23884", "23885", "23886", "23887", "23888", "23889", "23890", "23891", "23892", "23893", "23894", "23895", "23896", "23897", "23898", "23899", "23900", "23901", "23902", "23903", "23904", "23905", "23906", "23907", "23908", "23909", "23910", "23911", "23912", "23913", "23914", "23915", "23916", "23917", "23918", "23919", "23920", "23921", "23922", "23923", "23924", "23925", "23926", "23927", "23928", "23929", "23930", "23931", "23932", "23933", "23934", "23935", "23936", "23937", "23938", "23939", "23940", "23941", "23942", "23943", "23944", "23945", "23946", "23947", "23948"]}, "114": {"datafields": ["24003", "24004", "24005", "24006", "24007", "24008", "24009", "24010", "24011", "24012", "24013", "24014", "24015", "24016", "24017", "24018", "24019"]}, "115": {"datafields": ["24020", "24021", "24022", "24023", "24024"]}, "273": {"datafields": ["24030", "24032", "24033", "24035", "24037", "24038", "24040", "24041", "24043", "24044", "24046", "24047"]}, "185": {"datafields": ["24048", "24050", "24051", "24053", "24055", "24056", "24058", "24059", "24061", "24062", "24064", "24065", "24308", "24309", "24310"]}, "172": {"datafields": ["24066", "24067", "24068", "24069"]}, "157": {"datafields": ["24100", "24101", "24102", "24103", "24104", "24105", "24106", "24107", "24108", "24109", "24110", "24111", "24112", "24113", "24114", "24115", "24116", "24117", "24118", "24119", "24120", "24121", "24122", "24123", "24124", "24125", "24126", "24127", "24128", "24129", "24130", "24131", "24132", "24133", "24134", "24135", "24136", "24137", "24138", "24139", "24140", "24141", "24142", "24143", "24144", "24145", "24146", "24147", "24148", "24149", "24150", "24151", "24152", "24153", "24154", "24155", "24156", "24157", "24158", "24159", "24160", "24161", "24162", "24163", "24164", "24165", "24166", "24167", "24168", "24169", "24170", "24171", "24172", "24173", "24174", "24175", "24176", "24177", "24178", "24179", "24180", "24181", "25931"]}, "151": {"datafields": ["24500", "24501", "24502", "24503", "24504", "24505", "24506", "24507", "24508"]}, "349": {"datafields": ["24600", "24601", "24602", "24603", "24604", "24605", "24606", "24607", "24608", "24609", "24610", "24611", "24612", "24613", "24614", "24615", "24616", "24617", "24618", "24619", "24620", "24621", "24622", "24623", "24624", "24625", "24626", "24627", "24628", "24629", "24630", "24631", "24632", "24633", "24634", "24635", "24636", "24637", "24638", "24639", "24640", "24641", "24642", "24643", "24644", "24645", "24646", "24647", "24648", "24649", "24650", "24651", "24652", "24653", "24654", "24655", "24656", "24657", "24658", "24659", "24660", "24661", "24662", "24663", "24664", "24665"]}, "1102": {"datafields": ["25011", "25012", "25013", "25014", "25015", "25016", "25017", "25018", "25019", "25020", "25021", "25022", "25023", "25024"]}, "134": {"datafields": ["25056", "25057", "25058", "25059", "25060", "25061", "25062", "25063", "25064", "25065", "25066", "25067", "25068", "25069", "25070", "25071", "25072", "25073", "25074", "25075", "25076", "25077", "25078", "25079", "25080", "25081", "25082", "25083", "25084", "25085", "25086", "25087", "25088", "25089", "25090", "25091", "25092", "25093", "25094", "25095", "25096", "25097", "25098", "25099", "25100", "25101", "25102", "25103", "25104", "25105", "25106", "25107", "25108", "25109", "25110", "25111", "25112", "25113", "25114", "25115", "25116", "25117", "25118", "25119", "25120", "25121", "25122", "25123", "25124", "25125", "25126", "25127", "25128", "25129", "25130", "25131", "25132", "25133", "25134", "25135", "25136", "25137", "25138", "25139", "25140", "25141", "25142", "25143", "25144", "25145", "25146", "25147", "25148", "25149", "25150", "25151", "25152", "25153", "25154", "25155", "25156", "25157", "25158", "25159", "25160", "25161", "25162", "25163", "25164", "25165", "25166", "25167", "25168", "25169", "25170", "25171", "25172", "25173", "25174", "25175", "25176", "25177", "25178", "25179", "25180", "25181", "25182", "25183", "25184", "25185", "25186", "25187", "25188", "25189", "25190", "25191", "25192", "25193", "25194", "25195", "25196", "25197", "25198", "25199", "25200", "25201", "25202", "25203", "25204", "25205", "25206", "25207", "25208", "25209", "25210", "25211", "25212", "25213", "25214", "25215", "25216", "25217", "25218", "25219", "25220", "25221", "25222", "25223", "25224", "25225", "25226", "25227", "25228", "25229", "25230", "25231", "25232", "25233", "25234", "25235", "25236", "25237", "25238", "25239", "25240", "25241", "25242", "25243", "25244", "25245", "25246", "25247", "25248", "25249", "25250", "25251", "25252", "25253", "25254", "25255", "25256", "25257", "25258", "25259", "25260", "25261", "25262", "25263", "25264", "25265", "25266", "25267", "25268", "25269", "25270", "25271", "25272", "25273", "25274", "25275", "25276", "25277", "25278", "25279", "25280", "25281", "25282", "25283", "25284", "25285", "25286", "25287", "25288", "25289", "25290", "25291", "25292", "25293", "25294", "25295", "25296", "25297", "25298", "25299", "25300", "25301", "25302", "25303", "25304", "25305", "25306", "25307", "25308", "25309", "25310", "25311", "25312", "25313", "25314", "25315", "25316", "25317", "25318", "25319", "25320", "25321", "25322", "25323", "25324", "25325", "25326", "25327", "25328", "25329", "25330", "25331", "25332", "25333", "25334", "25335", "25336", "25337", "25338", "25339", "25340", "25341", "25342", "25343", "25344", "25345", "25346", "25347", "25348", "25349", "25350", "25351", "25352", "25353", "25354", "25355", "25356", "25357", "25358", "25359", "25360", "25361", "25362", "25363", "25364", "25365", "25366", "25367", "25368", "25369", "25370", "25371", "25372", "25373", "25374", "25375", "25376", "25377", "25378", "25379", "25380", "25381", "25382", "25383", "25384", "25385", "25386", "25387", "25388", "25389", "25390", "25391", "25392", "25393", "25394", "25395", "25396", "25397", "25398", "25399", "25400", "25401", "25402", "25403", "25404", "25405", "25406", "25407", "25408", "25409", "25410", "25411", "25412", "25413", "25414", "25415", "25416", "25417", "25418", "25419", "25420", "25421", "25422", "25423", "25424", "25425", "25426", "25427", "25428", "25429", "25430", "25431", "25432", "25433", "25434", "25435", "25436", "25437", "25438", "25439", "25440", "25441", "25442", "25443", "25444", "25445", "25446", "25447", "25448", "25449", "25450", "25451", "25452", "25453", "25454", "25455", "25456", "25457", "25458", "25459", "25460", "25461", "25462", "25463", "25464", "25465", "25466", "25467", "25468", "25469", "25470", "25471", "25472", "25473", "25474", "25475", "25476", "25477", "25478", "25479", "25480", "25481", "25482", "25483", "25484", "25485", "25486", "25487"]}, "135": {"datafields": ["25488", "25489", "25490", "25491", "25492", "25493", "25494", "25495", "25496", "25497", "25498", "25499", "25500", "25501", "25502", "25503", "25504", "25505", "25506", "25507", "25508", "25509", "25510", "25511", "25512", "25513", "25514", "25515", "25516", "25517", "25518", "25519", "25520", "25521", "25522", "25523", "25524", "25525", "25526", "25527", "25528", "25529", "25530", "25531", "25532", "25533", "25534", "25535", "25536", "25537", "25538", "25539", "25540", "25541", "25542", "25543", "25544", "25545", "25546", "25547", "25548", "25549", "25550", "25551", "25552", "25553", "25554", "25555", "25556", "25557", "25558", "25559", "25560", "25561", "25562", "25563", "25564", "25565", "25566", "25567", "25568", "25569", "25570", "25571", "25572", "25573", "25574", "25575", "25576", "25577", "25578", "25579", "25580", "25581", "25582", "25583", "25584", "25585", "25586", "25587", "25588", "25589", "25590", "25591", "25592", "25593", "25594", "25595", "25596", "25597", "25598", "25599", "25600", "25601", "25602", "25603", "25604", "25605", "25606", "25607", "25608", "25609", "25610", "25611", "25612", "25613", "25614", "25615", "25616", "25617", "25618", "25619", "25620", "25621", "25622", "25623", "25624", "25625", "25626", "25627", "25628", "25629", "25630", "25631", "25632", "25633", "25634", "25635", "25636", "25637", "25638", "25639", "25640", "25641", "25642", "25643", "25644", "25645", "25646", "25647", "25648", "25649", "25650", "25651", "25652", "25653", "25654", "25655", "25656", "25657", "25658", "25659", "25660", "25661", "25662", "25663", "25664", "25665", "25666", "25667", "25668", "25669", "25670", "25671", "25672", "25673", "25674", "25675", "25676", "25677", "25678", "25679", "25680", "25681", "25682", "25683", "25684", "25685", "25686", "25687", "25688", "25689", "25690", "25691", "25692", "25693", "25694", "25695", "25696", "25697", "25698", "25699", "25700", "25701", "25702", "25703", "25704", "25705", "25706", "25707", "25708", "25709", "25710", "25711", "25712", "25713", "25714", "25715", "25716", "25717", "25718", "25719", "25720", "25721", "25722", "25723", "25724", "25725", "25726", "25727", "25728", "25729", "25730"]}, "1101": {"datafields": ["25782", "25783", "25784", "25785", "25786", "25787", "25788", "25789", "25790", "25791", "25792", "25793", "25794", "25795", "25796", "25797", "25798", "25799", "25800", "25801", "25802", "25803", "25804", "25805", "25806", "25807", "25808", "25809", "25810", "25811", "25812", "25813", "25814", "25815", "25816", "25817", "25818", "25819", "25820", "25821", "25822", "25823", "25824", "25825", "25826", "25827", "25828", "25829", "25830", "25831", "25832", "25833", "25834", "25835", "25836", "25837", "25838", "25839", "25840", "25841", "25842", "25843", "25844", "25845", "25846", "25847", "25848", "25849", "25850", "25851", "25852", "25853", "25854", "25855", "25856", "25857", "25858", "25859", "25860", "25861", "25862", "25863", "25864", "25865", "25866", "25867", "25868", "25869", "25870", "25871", "25872", "25873", "25874", "25875", "25876", "25877", "25878", "25879", "25880", "25881", "25882", "25883", "25884", "25885", "25886", "25887", "25888", "25889", "25890", "25891", "25892", "25893", "25894", "25895", "25896", "25897", "25898", "25899", "25900", "25901", "25902", "25903", "25904", "25905", "25906", "25907", "25908", "25909", "25910", "25911", "25912", "25913", "25914", "25915", "25916", "25917", "25918", "25919", "25920"]}, "100117": {"datafields": ["26000", "26001", "26002", "26003", "26004", "26005", "26006", "26007", "26008", "26009", "26010", "26011", "26012", "26013", "26014", "26015", "26016", "26017", "26018", "26019", "26020", "26021", "26022", "26023", "26024", "26025", "26026", "26027", "26028", "26029", "26030", "26031", "26032", "26033", "26034", "26035", "26036", "26037", "26038", "26039", "26040", "26041", "26042", "26043", "26044", "26045", "26046", "26047", "26048", "26049", "26050", "26051", "26052", "26053", "26054", "26055", "26056", "26057", "26058", "26059", "26060", "26061", "26155"]}, "100118": {"datafields": ["26062", "26063", "26064", "26065", "26066", "26067", "26068", "26069", "26070", "26071", "26072", "26073", "26074", "26075", "26076", "26077", "26078", "26079", "26080", "26081", "26082", "26083", "26084", "26085", "26086", "26087", "26088", "26089", "26090", "26091", "26092", "26093", "26094", "26095", "26096", "26097", "26098", "26099", "26100", "26101", "26102", "26103", "26104", "26105", "26106", "26107", "26108", "26109", "26110", "26111", "26112", "26113", "26114", "26115", "26116", "26117", "26118", "26119", "26120", "26121", "26122", "26123", "26124", "26125", "26126", "26127", "26128", "26129", "26130", "26131", "26132", "26133", "26134", "26135", "26136", "26137", "26138", "26139", "26140", "26141", "26142", "26143", "26144", "26145", "26146", "26147", "26148", "26149", "26150", "26151", "26152", "26153", "26154"]}, "302": {"datafields": ["26203", "26205", "26207", "26208", "26209", "26211", "26213", "26215", "26217", "26219", "26221", "26222", "26224", "26226", "26228", "26230", "26231", "26233", "26235", "26236", "26237", "26239", "26241", "26243", "26245", "26247", "26249", "26251", "26253", "26255", "26256", "26257", "26259", "26261", "26262", "26263", "26264", "26266", "26268", "26270", "26271", "26272", "26274", "26276", "26277", "26279", "26280", "26281", "26282", "26284", "26286", "26288", "26290"]}, "76": {"datafields": ["26410", "26411", "26412", "26413", "26414", "26415", "26416", "26417", "26418", "26419", "26420", "26421", "26422", "26423", "26424", "26425", "26426", "26427", "26428", "26429", "26430", "26431", "26432", "26433", "26434"]}, "190": {"datafields": ["26501", "26502", "26503", "26504", "26505", "26506", "26507", "26508", "26509", "26510", "26511", "26512", "26513", "26514", "26515", "26516", "26517", "26518", "26519", "26520", "26521", "26522", "26523", "26524", "26525", "26526", "26527", "26528", "26529", "26530", "26531", "26532", "26533", "26534", "26535", "26536", "26537", "26538", "26539", "26540", "26541", "26542", "26543", "26544", "26545", "26546", "26547", "26548", "26549", "26550", "26551", "26552", "26553", "26554", "26555", "26556", "26557", "26558", "26559", "26560", "26561", "26562", "26563", "26564", "26565", "26566", "26567", "26568", "26569", "26570", "26571", "26572", "26573", "26574", "26575", "26576", "26577", "26578", "26579", "26580", "26581", "26582", "26583", "26584", "26585", "26586", "26587", "26588", "26589", "26590", "26591", "26592", "26593", "26594", "26595", "26596", "26597", "26598", "26599"]}, "191": {"datafields": ["26600", "26601", "26602", "26603", "26604", "26605", "26606", "26607", "26608", "26609", "26610", "26611", "26612", "26613", "26614", "26615", "26616", "26617", "26618", "26619", "26620", "26621", "26622", "26623", "26624", "26625", "26626", "26627", "26628", "26629", "26630", "26631", "26632", "26633", "26634", "26635", "26636", "26637", "26638", "26639", "26640", "26641", "26642", "26643", "26644", "26645", "26646", "26647", "26648", "26649", "26650", "26651", "26652", "26653", "26654", "26655", "26656", "26657", "26658", "26659", "26660", "26661", "26662", "26663", "26664", "26665", "26666", "26667", "26668", "26669", "26670", "26671", "26672", "26673", "26674", "26675", "26676", "26677", "26678", "26679", "26680", "26681", "26682", "26683", "26684", "26685", "26686", "26687", "26688", "26689", "26690", "26691", "26692", "26693", "26694", "26695", "26696", "26697", "26698", "26699", "26700", "26701", "26702", "26703", "26704", "26705", "26706", "26707", "26708", "26709", "26710", "26711", "26712", "26713", "26714", "26715", "26716", "26717", "26718", "26719", "26720"]}, "192": {"datafields": ["26721", "26722", "26723", "26724", "26725", "26726", "26727", "26728", "26729", "26730", "26731", "26732", "26733", "26734", "26735", "26736", "26737", "26738", "26739", "26740", "26741", "26742", "26743", "26744", "26745", "26746", "26747", "26748", "26749", "26750", "26751", "26752", "26753", "26754", "26755", "26756", "26757", "26758", "26759", "26760", "26761", "26762", "26763", "26764", "26765", "26766", "26767", "26768", "26769", "26770", "26771", "26772", "26773", "26774", "26775", "26776", "26777", "26778", "26779", "26780", "26781", "26782", "26783", "26784", "26785", "26786", "26787", "26788", "26789", "26790", "26791", "26792", "26793", "26794", "26795", "26796", "26797", "26798", "26799", "26800", "26801", "26802", "26803", "26804", "26805", "26806", "26807", "26808", "26809", "26810", "26811", "26812", "26813", "26814", "26815", "26816", "26817", "26818", "26819", "26820", "26821", "26822", "26823", "26824", "26825", "26826", "26827", "26828", "26829", "26830", "26831", "26832", "26833", "26834", "26835", "26836", "26837", "26838", "26839", "26840", "26841", "26842", "26843", "26844", "26845", "26846", "26847", "26848", "26849", "26850", "26851", "26852", "26853", "26854", "26855", "26856", "26857", "26858", "26859", "26860", "26861", "26862", "26863", "26864", "26865", "26866", "26867", "26868", "26869", "26870", "26871", "26872", "26873", "26874", "26875", "26876", "26877", "26878", "26879", "26880", "26881", "26882", "26883", "26884", "26885", "26886", "26887", "26888", "26889", "26890", "26891", "26892", "26893", "26894", "26895", "26896", "26897", "26898", "26899", "26900", "26901", "26902", "26903", "26904", "26905", "26906", "26907", "26908", "26909", "26910", "26911", "26912", "26913", "26914", "26915", "26916", "26917", "26918", "26919", "26920", "26921", "26922"]}, "193": {"datafields": ["26923", "26924", "26925", "26926", "26927", "26928", "26929", "26930", "26931", "26932", "26933", "26934", "26935", "26936", "26937", "26938", "26939", "26940", "26941", "26942", "26943", "26944", "26945", "26946", "26947", "26948", "26949", "26950", "26951", "26952", "26953", "26954", "26955", "26956", "26957", "26958", "26959", "26960", "26961", "26962", "26963", "26964", "26965", "26966", "26967", "26968", "26969", "26970", "26971", "26972", "26973", "26974", "26975", "26976", "26977", "26978", "26979", "26980", "26981", "26982", "26983", "26984", "26985", "26986", "26987", "26988"]}, "194": {"datafields": ["26989", "26990", "26991", "26992", "26993", "26994", "26995", "26996", "26997", "26998", "26999", "27000", "27001", "27002", "27003", "27004", "27005", "27006", "27007", "27008", "27009", "27010", "27011", "27012", "27013", "27014", "27015", "27016", "27017", "27018", "27019", "27020", "27021", "27022", "27023", "27024", "27025", "27026", "27027", "27028", "27029", "27030", "27031", "27032", "27033", "27034", "27035", "27036", "27037", "27038", "27039", "27040", "27041", "27042", "27043", "27044", "27045", "27046", "27047", "27048", "27049", "27050", "27051", "27052", "27053", "27054", "27055", "27056", "27057", "27058"]}, "195": {"datafields": ["27059", "27060", "27061", "27062", "27063", "27064", "27065", "27066", "27067", "27068", "27069", "27070", "27071", "27072", "27073", "27074", "27075", "27076", "27077", "27078", "27079", "27080", "27081", "27082", "27083", "27084", "27085", "27086", "27087", "27088", "27089", "27090", "27091", "27092", "27093", "27094", "27095", "27096", "27097", "27098", "27099", "27100", "27101", "27102", "27103", "27104", "27105", "27106", "27107", "27108", "27109", "27110", "27111", "27112", "27113", "27114", "27115", "27116", "27117", "27118", "27119", "27120", "27121", "27122", "27123", "27124", "27125", "27126", "27127", "27128", "27129", "27130", "27131", "27132", "27133", "27134", "27135", "27136", "27137", "27138", "27139", "27140", "27141", "27142"]}, "196": {"datafields": ["27143", "27144", "27145", "27146", "27147", "27148", "27149", "27150", "27151", "27152", "27153", "27154", "27155", "27156", "27157", "27158", "27159", "27160", "27161", "27162", "27163", "27164", "27165", "27166", "27167", "27168", "27169", "27170", "27171", "27172", "27173", "27174", "27175", "27176", "27177", "27178", "27179", "27180", "27181", "27182", "27183", "27184", "27185", "27186", "27187", "27188", "27189", "27190", "27191", "27192", "27193", "27194", "27195", "27196", "27197", "27198", "27199", "27200", "27201", "27202", "27203", "27204", "27205", "27206", "27207", "27208", "27209", "27210", "27211", "27212", "27213", "27214", "27215", "27216", "27217", "27218", "27219", "27220", "27221", "27222", "27223", "27224", "27225", "27226", "27227", "27228", "27229", "27230", "27231", "27232", "27233", "27234", "27235", "27236", "27237", "27238", "27239", "27240", "27241", "27242", "27243", "27244", "27245", "27246", "27247", "27248", "27249", "27250", "27251", "27252", "27253", "27254", "27255", "27256", "27257", "27258", "27259", "27260", "27261", "27262", "27263", "27264", "27265", "27266", "27267", "27268", "27269", "27270", "27271", "27272", "27273", "27274", "27275", "27276", "27277", "27278", "27279", "27280", "27281", "27282", "27283", "27284", "27285", "27286", "27287", "27288", "27289", "27290", "27291", "27292", "27293", "27294", "27295", "27296", "27297", "27298", "27299", "27300", "27301", "27302", "27303", "27304", "27305", "27306", "27307", "27308", "27309", "27310", "27311", "27312", "27313", "27314", "27315", "27316", "27317", "27318", "27319", "27320", "27321", "27322", "27323", "27324", "27325", "27326", "27327", "27328"]}, "197": {"datafields": ["27329", "27330", "27331", "27332", "27333", "27334", "27335", "27336", "27337", "27338", "27339", "27340", "27341", "27342", "27343", "27344", "27345", "27346", "27347", "27348", "27349", "27350", "27351", "27352", "27353", "27354", "27355", "27356", "27357", "27358", "27359", "27360", "27361", "27362", "27363", "27364", "27365", "27366", "27367", "27368", "27369", "27370", "27371", "27372", "27373", "27374", "27375", "27376", "27377", "27378", "27379", "27380", "27381", "27382", "27383", "27384", "27385", "27386", "27387", "27388", "27389", "27390", "27391", "27392", "27393", "27394", "27395", "27396", "27397", "27398", "27399", "27400", "27401", "27402", "27403", "27404", "27405", "27406", "27407", "27408", "27409", "27410", "27411", "27412", "27413", "27414", "27415", "27416", "27417", "27418", "27419", "27420", "27421", "27422", "27423", "27424", "27425", "27426", "27427", "27428", "27429", "27430", "27431", "27432", "27433", "27434", "27435", "27436", "27437", "27438", "27439", "27440", "27441", "27442", "27443", "27444", "27445", "27446", "27447", "27448", "27449", "27450", "27451", "27452", "27453", "27454", "27455", "27456", "27457", "27458", "27459", "27460", "27461", "27462", "27463", "27464", "27465", "27466", "27467", "27468", "27469", "27470", "27471", "27472", "27473", "27474", "27475", "27476", "27477", "27478", "27479", "27480", "27481", "27482", "27483", "27484", "27485", "27486", "27487", "27488", "27489", "27490", "27491", "27492", "27493", "27494", "27495", "27496", "27497", "27498", "27499", "27500", "27501", "27502", "27503", "27504", "27505", "27506", "27507", "27508", "27509", "27510", "27511", "27512", "27513", "27514", "27515", "27516", "27517", "27518", "27519", "27520", "27521", "27522", "27523", "27524", "27525", "27526", "27527", "27528", "27529", "27530", "27531", "27532", "27533", "27534", "27535", "27536", "27537", "27538", "27539", "27540", "27541", "27542", "27543", "27544", "27545", "27546", "27547", "27548", "27549", "27550", "27551", "27552", "27553", "27554", "27555", "27556", "27557", "27558", "27559", "27560", "27561", "27562", "27563", "27564", "27565", "27566", "27567", "27568", "27569", "27570", "27571", "27572", "27573", "27574", "27575", "27576", "27577", "27578", "27579", "27580", "27581", "27582", "27583", "27584", "27585", "27586", "27587", "27588", "27589", "27590", "27591", "27592", "27593", "27594", "27595", "27596", "27597", "27598", "27599", "27600", "27601", "27602", "27603", "27604", "27605", "27606", "27607", "27608", "27609", "27610", "27611", "27612", "27613", "27614", "27615", "27616", "27617", "27618", "27619", "27620", "27621", "27622", "27623", "27624", "27625", "27626", "27627", "27628", "27629", "27630", "27631", "27632", "27633", "27634", "27635", "27636", "27637", "27638", "27639", "27640", "27641", "27642", "27643", "27644", "27645", "27646", "27647", "27648", "27649", "27650", "27651", "27652", "27653", "27654", "27655", "27656", "27657", "27658", "27659", "27660", "27661", "27662", "27663", "27664", "27665", "27666", "27667", "27668", "27669", "27670", "27671", "27672", "27673", "27674", "27675", "27676", "27677", "27678", "27679", "27680", "27681", "27682", "27683", "27684", "27685", "27686", "27687", "27688", "27689", "27690", "27691", "27692", "27693", "27694", "27695", "27696", "27697", "27698", "27699", "27700", "27701", "27702", "27703", "27704", "27705", "27706", "27707", "27708", "27709", "27710", "27711", "27712", "27713", "27714", "27715", "27716", "27717", "27718", "27719", "27720", "27721", "27722", "27723", "27724", "27725", "27726", "27727", "27728", "27729", "27730", "27731", "27732", "27733", "27734", "27735", "27736", "27737", "27738", "27739", "27740", "27741", "27742", "27743", "27744", "27745", "27746", "27747", "27748", "27749", "27750", "27751", "27752", "27753", "27754", "27755", "27756", "27757", "27758", "27759", "27760", "27761", "27762", "27763", "27764", "27765", "27766", "27767", "27768", "27769", "27770", "27771", "27772"]}, "100079": {"datafields": ["27800", "27801", "27802", "27803", "27804", "27805", "27806", "27807", "27808", "27809", "27810", "27811", "27812", "27813", "27814", "27815", "27816", "27817", "27818", "27819", "27820", "27821", "27822", "27823", "27824", "27825", "27826", "27827", "27828", "27829", "27830", "27831", "27832", "27833", "27834", "27835", "27836", "27837", "27838", "27839", "27840", "27841", "27851", "27852", "27853", "27854", "27855", "27856", "27857", "27858", "28500", "28501", "28502", "28503", "28504", "28505", "28506", "28507", "28508", "28509", "28510", "28511", "28512", "28513", "28514", "28515", "28516", "28517", "28518", "28519", "28520", "28521", "28522", "28523", "28524", "28525", "28526", "28527", "28528", "28529", "28530", "28531", "28532", "28533", "28534", "28535", "28536", "28537"]}, "998": {"datafields": ["27980", "27981", "27982", "27983", "27984", "27985", "27986", "27989"]}, "997": {"datafields": ["27990", "27991", "27992", "27993"]}, "993": {"datafields": ["28000", "28001", "28003", "28004", "28005", "28006", "28007", "28008", "28009"]}, "992": {"datafields": ["28010", "28011", "28012", "28013", "28014", "28015", "28016", "28017", "28018", "28019", "28020", "28021", "28022", "28023", "28024", "28025", "28026", "28027", "28028", "28029", "28030", "28031", "28032", "28033"]}, "991": {"datafields": ["28034", "28035", "28036", "28037", "28038", "28039", "28040", "28041", "28042", "28043", "28044", "28045", "28046", "28047", "28048", "28049", "28050", "28051", "28052", "28053", "28054", "28055", "28056", "28057", "28058", "28059", "28060", "28061", "28062", "28063", "28064", "28065", "28066", "28067", "28068", "28069", "28070", "28071", "28072", "28073", "28074", "28075", "28076", "28077", "28078", "28079", "28080", "28081", "28082", "28083", "28084", "28085", "28086", "28087", "28088", "28089", "28090", "28091", "28092", "28093", "28094", "28095", "28096", "28097", "28098", "28099", "28100", "28101", "28102", "28103", "28104", "28105", "28106", "28107", "28108", "28109", "28110", "28111", "28112", "28113", "28114", "28115", "28116", "28117", "28118", "28119", "28120", "28121", "28122", "28123", "28124", "28125", "28126", "28127"]}, "990": {"datafields": ["28140", "28141", "28142", "28143"]}, "989": {"datafields": ["28146", "28147", "28148", "28149", "28150", "28151", "28152", "28153", "28154", "28155", "28156", "28157", "28158", "28159", "28160", "28161", "28162", "28163", "28164", "28165", "28166", "28167"]}, "100116": {"datafields": ["28538", "28539", "28540", "28541", "28542", "28543", "28544", "28545", "28546", "28547", "28548", "28549", "28550", "28551", "28552", "28553"]}, "160": {"datafields": ["28600", "28601", "28602", "28603", "28604", "28605", "28606", "28607", "28608", "28609", "28610", "28611", "28612", "28613", "28614", "28615", "28616", "28617", "28618", "28619", "28620", "28621", "28622", "28623", "28624", "28625", "28626", "28627", "28628", "28629", "28630", "28631", "28632", "28633", "28634", "28635", "28636", "28637", "28638", "28639", "28640", "28641", "28642", "28643", "28644", "28645", "28646", "28647", "28648", "28649", "28650", "28651", "28652", "28653", "28654", "28655", "28656", "28657", "28658", "28659", "28660", "28661", "28662", "28663", "28664", "28665", "28666", "28667", "28668", "28669", "28670", "28671", "28672", "28673", "28674", "28675", "28676", "28677", "28678", "28679", "28680", "28681", "28682", "28683", "28684", "28685", "28686", "28687", "28688", "28689", "28690", "28691", "28692", "28693", "28694", "28695", "28696", "28697", "28698", "28699", "28700", "28701", "28702", "28703", "28704", "28705", "28706", "28707", "28708", "28709", "28710", "28711", "28712", "28713", "28714", "28715", "28716", "28717", "28718", "28719", "28720", "28721", "28722", "28723", "28724", "28725", "28726", "28727", "28728", "28729", "28730", "28731", "28732", "28733", "28734", "28735", "28736", "28737", "28738", "28739", "28740", "28741", "28742", "28743", "28744", "28745", "28746", "28747", "28748", "28749", "28750", "28751", "28752", "28753", "28754", "28755", "28756"]}, "1501": {"datafields": ["29000", "29001", "29184", "29197"]}, "1502": {"datafields": ["29002", "29003", "29004", "29005", "29006", "29007", "29008", "29009", "29010", "29011", "29012", "29013", "29014", "29015", "29016", "29017", "29018", "29019", "29020", "29021", "29022", "29023", "29024", "29025", "29026", "29027", "29028", "29029", "29030", "29031", "29032", "29033", "29034", "29035", "29036", "29037", "29038", "29039", "29040", "29041", "29042", "29043", "29044", "29045", "29046", "29047", "29048", "29185", "29198"]}, "1503": {"datafields": ["29049", "29050", "29051", "29052", "29053", "29054", "29055", "29056", "29057", "29186", "29199"]}, "1504": {"datafields": ["29058", "29059", "29060", "29061", "29062", "29063", "29064", "29065", "29066", "29067", "29068", "29069", "29070", "29071", "29072", "29073", "29074", "29075", "29187", "29200"]}, "1505": {"datafields": ["29076", "29077", "29078", "29079", "29080", "29081", "29082", "29083", "29084", "29085", "29086", "29087", "29088", "29089", "29090", "29188", "29201"]}, "1506": {"datafields": ["29091", "29092", "29093", "29094", "29095", "29096", "29097", "29098", "29099", "29100", "29101", "29102", "29103", "29189", "29202"]}, "1507": {"datafields": ["29104", "29105", "29106", "29107", "29190", "29203"]}, "1508": {"datafields": ["29108", "29109", "29110", "29111", "29112", "29113", "29114", "29115", "29116", "29117", "29118", "29119", "29191", "29204"]}, "1509": {"datafields": ["29120", "29121", "29122", "29123", "29124", "29125", "29126", "29127", "29128", "29129", "29130", "29131", "29132", "29133", "29134", "29135", "29136", "29137", "29138", "29139", "29140", "29141", "29142", "29143", "29144", "29145", "29146", "29147", "29148", "29149", "29192", "29205"]}, "1510": {"datafields": ["29150", "29151", "29152", "29153", "29154", "29155", "29193", "29206"]}, "1511": {"datafields": ["29156", "29157", "29158", "29159", "29160", "29161", "29194", "29207"]}, "1512": {"datafields": ["29162", "29163", "29164", "29165", "29166", "29167", "29168", "29169", "29170", "29171", "29172", "29173", "29174", "29175", "29176", "29177", "29178", "29179", "29180", "29195", "29208"]}, "1513": {"datafields": ["29181", "29182", "29196", "29209"]}, "9081": {"datafields": ["30001", "30002", "30003", "30004", "30011", "30012", "30013", "30014", "30021", "30022", "30023", "30024", "30031", "30032", "30033", "30034", "30041", "30042", "30043", "30044", "30051", "30052", "30053", "30054", "30061", "30062", "30063", "30064", "30071", "30072", "30073", "30074", "30081", "30082", "30083", "30084", "30091", "30092", "30093", "30094", "30101", "30102", "30103", "30104", "30111", "30112", "30113", "30114", "30121", "30122", "30123", "30124", "30131", "30132", "30133", "30134", "30141", "30142", "30143", "30144", "30151", "30152", "30153", "30154", "30161", "30162", "30163", "30164", "30171", "30172", "30173", "30174", "30181", "30182", "30183", "30184", "30191", "30192", "30193", "30194", "30201", "30202", "30203", "30204", "30211", "30212", "30213", "30214", "30221", "30222", "30223", "30224", "30231", "30232", "30233", "30234", "30241", "30242", "30243", "30244", "30251", "30252", "30253", "30254", "30261", "30262", "30263", "30264", "30271", "30272", "30273", "30274", "30281", "30282", "30283", "30284", "30291", "30292", "30293", "30294", "30301", "30302", "30303", "30304"]}, "100085": {"datafields": ["30314", "30324", "30334", "30344", "30354", "30364", "30374", "30384", "30404", "30414", "40425"]}, "100087": {"datafields": ["30394"]}, "100086": {"datafields": ["30424"]}, "148": {"datafields": ["30502", "30503", "30512", "30513", "30522", "30523", "30532", "30533"]}, "18518": {"datafields": ["30601", "30602", "30603", "30604", "30605", "30606", "30611", "30612", "30613", "30614", "30615", "30616", "30621", "30622", "30623", "30624", "30625", "30626", "30631", "30632", "30633", "30634", "30635", "30636", "30641", "30642", "30643", "30644", "30645", "30646", "30651", "30652", "30653", "30654", "30655", "30656", "30661", "30662", "30663", "30664", "30665", "30666", "30671", "30672", "30673", "30674", "30675", "30676", "30681", "30682", "30683", "30684", "30685", "30686", "30691", "30692", "30693", "30694", "30695", "30696", "30701", "30702", "30703", "30704", "30705", "30706", "30711", "30712", "30713", "30714", "30715", "30716", "30721", "30722", "30723", "30724", "30725", "30726", "30731", "30732", "30733", "30734", "30735", "30736", "30741", "30742", "30743", "30744", "30745", "30746", "30751", "30753", "30754", "30755", "30756", "30761", "30762", "30763", "30764", "30765", "30766", "30771", "30772", "30773", "30774", "30775", "30776", "30781", "30782", "30783", "30784", "30785", "30786", "30791", "30792", "30793", "30794", "30795", "30796", "30801", "30802", "30803", "30804", "30805", "30806", "30811", "30812", "30813", "30814", "30815", "30816", "30821", "30822", "30823", "30824", "30825", "30826", "30831", "30832", "30833", "30834", "30835", "30836", "30841", "30842", "30843", "30844", "30845", "30846", "30851", "30852", "30853", "30854", "30855", "30856", "30861", "30862", "30863", "30864", "30865", "30866", "30871", "30872", "30873", "30874", "30875", "30876", "30881", "30882", "30883", "30884", "30885", "30886", "30891", "30892", "30893", "30894", "30895", "30896", "30897"]}, "1839": {"datafields": ["30900", "30901", "30902", "30903", "30904"]}, "201": {"datafields": ["31001", "31002", "31003", "31004", "31005", "31006", "31007", "31008"]}, "202": {"datafields": ["31009", "31010", "31011", "31012", "31013"]}, "203": {"datafields": ["31014", "31015", "31016", "31017", "31018", "31019"]}, "204": {"datafields": ["31020", "31021", "31022", "31023", "31024", "31025", "31026", "31027", "31028"]}, "163": {"datafields": ["31040", "31041", "31042", "31043", "31044", "31045", "31046", "31047", "31048", "31049"]}, "162": {"datafields": ["31060", "31061", "31062", "31063", "31064", "31065", "31066", "31067", "31068", "31069", "31070", "31071", "31072", "31073", "31074", "31075", "31076", "31077", "31078", "31079", "31080", "31081", "31082", "31083", "31084", "31085", "31086", "31087", "31090", "31091", "31092", "31093", "31094", "31095", "31096", "31097", "31098", "31099", "31100", "31101", "31102", "31103", "31104", "31105", "31106", "31107", "31108", "31109", "31110", "31111", "31112", "31113", "31114", "31115", "31116", "31117", "31118", "31119", "31120", "31121", "31122", "31123", "31124", "31125", "31126", "31127", "31128", "31129", "31130", "31131", "31132", "31133"]}, "187": {"datafields": ["32051", "32052", "32053", "32054", "32055", "32056", "32057", "32058", "32059", "32060", "32061", "32062", "32063", "32064", "32065"]}, "100093": {"datafields": ["40000", "40001", "40002", "40007", "40010", "40018", "40020", "40023"]}, "100092": {"datafields": ["40005", "40006", "40008", "40009", "40011", "40012", "40013", "40019", "40021"]}, "2001": {"datafields": ["40022", "41206", "41207", "41208", "41209", "41211", "41212", "41213", "41229", "41230", "41231", "41232", "41233", "41235", "41244", "41245", "41246", "41247", "41248", "41249", "41250", "41251", "41253"]}, "1020": {"datafields": ["40030", "40031", "40032", "40033", "40034", "40035", "40036", "40037", "40038", "40039", "40040", "40041", "40042", "40043", "40044", "40045", "40046", "40047", "40048", "40049"]}, "996": {"datafields": ["40101", "40102", "40103", "40104"]}, "2003": {"datafields": ["41066", "41067", "41068", "41069", "41070", "41071", "41072", "41073", "41074", "41075", "41113", "41114", "41115", "41116", "41117", "41118", "41119", "41120", "41121", "41122", "41123", "41124", "41125", "41219", "41220", "41221", "41222", "41223", "41224", "41225", "41226", "41227", "41228", "41266", "41267", "41268", "41269", "41275", "41276", "41277", "41278", "41284", "41285", "41286", "41288"]}, "2002": {"datafields": ["41076", "41077", "41078", "41079", "41104", "41105", "41142", "41143", "41144", "41145", "41201", "41202", "41203", "41204", "41205", "41262", "41263", "41270", "41271", "41280", "41281"]}, "2005": {"datafields": ["41080", "41081", "41082", "41106", "41107", "41108", "41146", "41147", "41148", "41200", "41210", "41255", "41256", "41257", "41258", "41260", "41272", "41273", "41274", "41282", "41283"]}, "2004": {"datafields": ["41126", "41127", "41128", "41129", "41130", "41131", "41150", "41151", "41214", "41215", "41216", "41217", "41218", "41279"]}, "2006": {"datafields": ["41149", "41234", "41259", "41261", "41264", "41265", "41289", "41290"]}, "44": {"datafields": ["42000", "42001", "42002", "42003", "42004", "42005"]}, "43": {"datafields": ["42006", "42007", "42008", "42009", "42010", "42011", "42012", "42013"]}, "45": {"datafields": ["42014", "42015"]}, "46": {"datafields": ["42016", "42017"]}, "47": {"datafields": ["42018", "42019", "42020", "42021", "42022", "42023", "42024", "42025"]}, "48": {"datafields": ["42026", "42027"]}, "49": {"datafields": ["42028", "42029"]}, "50": {"datafields": ["42030", "42031", "42032", "42033", "42034", "42035", "42036", "42037"]}, "3001": {"datafields": ["42038", "42039", "42040"]}, "1013": {"datafields": ["90002", "90179", "90180", "90181", "90182", "90183", "90184", "90185", "90186", "90187", "90188", "90189", "90190", "90191", "90192", "90193", "90194", "90195"]}, "1011": {"datafields": ["90003", "90010", "90011", "90015", "90018", "90026", "90051", "90052", "90053", "90054", "90055", "90056", "90057", "90058", "90059", "90060", "90061", "90062", "90063", "90064", "90065", "90066", "90067", "90068", "90069", "90070", "90071", "90072", "90073", "90074", "90075", "90076", "90077", "90078", "90079", "90080", "90081", "90082", "90083", "90084", "90085", "90086"]}, "1009": {"datafields": ["90012", "90013", "90019", "90020", "90021", "90022", "90023", "90024", "90025", "90027", "90028", "90029", "90030", "90031", "90032", "90033", "90034", "90035", "90036", "90037", "90038", "90039", "90040", "90041", "90042", "90043", "90044", "90045", "90046", "90047", "90048", "90049", "90050", "90087", "90088", "90089", "90090", "90091"]}, "1012": {"datafields": ["90016", "90017", "90159", "90160", "90161", "90162", "90163", "90164", "90165", "90166", "90167", "90168", "90169", "90170", "90171", "90172", "90173", "90174", "90175", "90176", "90177"]}, "1010": {"datafields": ["90092", "90093", "90094", "90095", "90096", "90097", "90098", "90099", "90100", "90101", "90102", "90103", "90104", "90105", "90106", "90107", "90108", "90109", "90110", "90111", "90112", "90113", "90114", "90115", "90116", "90117", "90118", "90119", "90120", "90121", "90122", "90123", "90124", "90125", "90126", "90127", "90128", "90129", "90130", "90131", "90132", "90133", "90134", "90135", "90136", "90137", "90138", "90139", "90140", "90141", "90142", "90143", "90144", "90145", "90146", "90147", "90148", "90149", "90150", "90151", "90152", "90153", "90154", "90155", "90156", "90157", "90158"]}, "100098": {"datafields": ["100001", "100002", "100003", "100004", "100005", "100006", "100007", "100008", "100009", "100011", "100012", "100013", "100014", "100015", "100016", "100017", "100018", "100019", "100021", "100022", "100023", "100024", "100025"]}, "100104": {"datafields": ["100150", "100160", "100170", "100180", "100190", "100200", "100210", "100220", "100230", "100240", "100250", "100260", "100270", "100280", "100290", "100300", "100310", "100320", "100330", "100350", "100360", "100370", "100380", "100390", "100400", "100410", "100420", "100430", "100440", "100460", "100470", "100480", "100490", "100500", "100510", "100520", "100530", "100540", "100550", "100560"]}, "100107": {"datafields": ["100920", "102800", "102810", "102820", "102830", "102840", "102850", "102860", "102870", "102880", "102890", "102900", "102910", "102930", "102940", "102950", "102960", "102970", "102980"]}, "100106": {"datafields": ["103000", "103010", "103020", "103030", "103040", "103050", "103060", "103070", "103080", "103090", "103100", "103120", "103130", "103140", "103150", "103160", "103170", "103180", "103190", "103200", "103210", "103220", "103230"]}, "100111": {"datafields": ["103250", "103260", "103270", "103280", "103290"]}, "100103": {"datafields": ["103990", "104000", "104010", "104020", "104030", "104040", "104050", "104060", "104070", "104080", "104090", "104100", "104110", "104120", "104130", "104140", "104150", "104160", "104170", "104180", "104190", "104200", "104210", "104220", "104230", "104240", "104250", "104260", "104270", "104280", "104290", "104300", "104310", "104320", "104330", "104340", "104350", "104360", "104370", "104380", "104400", "104410", "104420", "104430", "104440", "104450", "104460", "104470", "104480", "104490", "104500", "104510", "104520", "104530", "104540", "104550", "104560", "104570", "104580", "104590"]}, "100108": {"datafields": ["104900", "104910", "104920"]}, "154": {"datafields": ["120000", "120001", "120002", "120003", "120004", "120005", "120006", "120007", "120008", "120009", "120010", "120011", "120012", "120013", "120014", "120015", "120016", "120017", "120018", "120019", "120020", "120021", "120022", "120023", "120024", "120025", "120026", "120027", "120028", "120029", "120030", "120031", "120032", "120033", "120034", "120035", "120036", "120037", "120038", "120039", "120040", "120041", "120042", "120043", "120044", "120045", "120046", "120047", "120048", "120049", "120050", "120051", "120052", "120053", "120054", "120055", "120056", "120057", "120058", "120059", "120060", "120061", "120062", "120063", "120064", "120065", "120066", "120067", "120068", "120069", "120070", "120071", "120072", "120073", "120074", "120075", "120076", "120077", "120078", "120079", "120080", "120081", "120082", "120083", "120084", "120085", "120086", "120087", "120088", "120089", "120090", "120091", "120092", "120093", "120094", "120095", "120096", "120097", "120098", "120099", "120100", "120101", "120102", "120103", "120104", "120105", "120106", "120107", "120108", "120109", "120110", "120111", "120112", "120113", "120114", "120115", "120116", "120117", "120118", "120119", "120120", "120121", "120122", "120123", "120124", "120125", "120126", "120127", "120128"]}, "2401": {"datafields": ["130000", "130001", "130002", "130003", "130004", "130005", "130006", "130007", "130008", "130009", "130010", "130011", "130012", "130013", "130014", "130015", "130016", "130017", "130018", "130019", "130020", "130021", "130022", "130023", "130024", "130025", "130026", "130027", "130028", "130029", "130030", "130031", "130032", "130033", "130034", "130035", "130036", "130037", "130038", "130039", "130040", "130041", "130042", "130043", "130044", "130045", "130046", "130047", "130048", "130049", "130050", "130051", "130052", "130053", "130054", "130055", "130056", "130057", "130058", "130059", "130060", "130061", "130062", "130063", "130064", "130065", "130066", "130067", "130068", "130069", "130070", "130071", "130072", "130073", "130074", "130075", "130076", "130077", "130078", "130079", "130080", "130081", "130082", "130083", "130084", "130085", "130086", "130087", "130088", "130089", "130090", "130091", "130092", "130093", "130094", "130095", "130096", "130097", "130098", "130099", "130100", "130101", "130102", "130103", "130104", "130105", "130106", "130107", "130108", "130109", "130110", "130111", "130112", "130113", "130114", "130115", "130116", "130117", "130118", "130119", "130120", "130121", "130122", "130123", "130124", "130125", "130126", "130127", "130128", "130129", "130130", "130131", "130132", "130133", "130134", "130135", "130136", "130137", "130138", "130139", "130140", "130141", "130142", "130143", "130144", "130145", "130146", "130147", "130148", "130149", "130150", "130151", "130152", "130153", "130154", "130155", "130156", "130157", "130158", "130159", "130160", "130161", "130162", "130163", "130164", "130165", "130166", "130167", "130168", "130169", "130170", "130171", "130172", "130173", "130174", "130175", "130176", "130177", "130178", "130179", "130180", "130181", "130182", "130183", "130184", "130185", "130186", "130187", "130188", "130189", "130190", "130191", "130192", "130193", "130194", "130195", "130196", "130197", "130198", "130199", "130200", "130201", "130202", "130203", "130204", "130205", "130206", "130207", "130208", "130209", "130210", "130211", "130212", "130213", "130214", "130215", "130216", "130217", "130218", "130219", "130220", "130221", "130222", "130223", "130224", "130225", "130226", "130227", "130228", "130229", "130230", "130231", "130232", "130233", "130234", "130235", "130236", "130237", "130238", "130239", "130240", "130241", "130242", "130243", "130244", "130245", "130246", "130247", "130248", "130249", "130250", "130251", "130252", "130253", "130254", "130255", "130256", "130257", "130258", "130259", "130260", "130261", "130262", "130263", "130264", "130265", "130266", "130267", "130268", "130269", "130270", "130271", "130272", "130273", "130274", "130275", "130276", "130277", "130278", "130279", "130280", "130281", "130282", "130283", "130284", "130285", "130286", "130287", "130288", "130289", "130290", "130291", "130292", "130293", "130294", "130295", "130296", "130297", "130298", "130299", "130300", "130301", "130302", "130303", "130304", "130305", "130306", "130307", "130308", "130309", "130310", "130311", "130312", "130313", "130314", "130315", "130316", "130317", "130318", "130319", "130320", "130321", "130322", "130323", "130324", "130325", "130326", "130327", "130328", "130329", "130330", "130331", "130332", "130333", "130334", "130335", "130336", "130337", "130338", "130339", "130340", "130341", "130342", "130343", "130344", "130345"]}, "2403": {"datafields": ["130622", "130623", "130624", "130625", "130626", "130627", "130628", "130629", "130630", "130631", "130632", "130633", "130634", "130635", "130636", "130637", "130638", "130639", "130640", "130641", "130642", "130643", "130644", "130645", "130646", "130647", "130648", "130649", "130650", "130651", "130652", "130653", "130654", "130655", "130656", "130657", "130658", "130659", "130660", "130661", "130662", "130663", "130664", "130665", "130666", "130667", "130668", "130669", "130670", "130671", "130672", "130673", "130674", "130675", "130676", "130677", "130678", "130679", "130680", "130681", "130682", "130683", "130684", "130685", "130686", "130687", "130688", "130689"]}, "2404": {"datafields": ["130690", "130691", "130692", "130693", "130694", "130695", "130696", "130697", "130698", "130699", "130700", "130701", "130702", "130703", "130704", "130705", "130706", "130707", "130708", "130709", "130710", "130711", "130712", "130713", "130714", "130715", "130716", "130717", "130718", "130719", "130720", "130721", "130722", "130723", "130724", "130725", "130726", "130727", "130728", "130729", "130730", "130731", "130732", "130733", "130734", "130735", "130736", "130737", "130738", "130739", "130740", "130741", "130742", "130743", "130744", "130745", "130746", "130747", "130748", "130749", "130750", "130751", "130752", "130753", "130754", "130755", "130756", "130757", "130758", "130759", "130760", "130761", "130762", "130763", "130764", "130765", "130766", "130767", "130768", "130769", "130770", "130771", "130772", "130773", "130774", "130775", "130776", "130777", "130778", "130779", "130780", "130781", "130782", "130783", "130784", "130785", "130786", "130787", "130788", "130789", "130790", "130791", "130792", "130793", "130794", "130795", "130796", "130797", "130798", "130799", "130800", "130801", "130802", "130803", "130804", "130805", "130806", "130807", "130808", "130809", "130810", "130811", "130812", "130813", "130814", "130815", "130816", "130817", "130818", "130819", "130820", "130821", "130822", "130823", "130824", "130825", "130826", "130827", "130828", "130829", "130830", "130831", "130832", "130833", "130834", "130835"]}, "2405": {"datafields": ["130836", "130837", "130838", "130839", "130840", "130841", "130842", "130843", "130844", "130845", "130846", "130847", "130848", "130849", "130850", "130851", "130852", "130853", "130854", "130855", "130856", "130857", "130858", "130859", "130860", "130861", "130862", "130863", "130864", "130865", "130866", "130867", "130868", "130869", "130870", "130871", "130872", "130873", "130874", "130875", "130876", "130877", "130878", "130879", "130880", "130881", "130882", "130883", "130884", "130885", "130886", "130887", "130888", "130889", "130890", "130891", "130892", "130893", "130894", "130895", "130896", "130897", "130898", "130899", "130900", "130901", "130902", "130903", "130904", "130905", "130906", "130907", "130908", "130909", "130910", "130911", "130912", "130913", "130914", "130915", "130916", "130917", "130918", "130919", "130920", "130921", "130922", "130923", "130924", "130925", "130926", "130927", "130928", "130929", "130930", "130931", "130932", "130933", "130934", "130935", "130936", "130937", "130938", "130939", "130940", "130941", "130942", "130943", "130944", "130945", "130946", "130947", "130948", "130949", "130950", "130951", "130952", "130953", "130954", "130955", "130956", "130957", "130958", "130959", "130960", "130961", "130962", "130963", "130964", "130965", "130966", "130967", "130968", "130969", "130970", "130971", "130972", "130973", "130974", "130975", "130976", "130977", "130978", "130979", "130980", "130981", "130982", "130983", "130984", "130985", "130986", "130987", "130988", "130989", "130990", "130991"]}, "2406": {"datafields": ["130992", "130993", "130994", "130995", "130996", "130997", "130998", "130999", "131000", "131001", "131002", "131003", "131004", "131005", "131006", "131007", "131008", "131009", "131010", "131011", "131012", "131013", "131014", "131015", "131016", "131017", "131018", "131019", "131020", "131021", "131022", "131023", "131024", "131025", "131026", "131027", "131028", "131029", "131030", "131031", "131032", "131033", "131034", "131035", "131036", "131037", "131038", "131039", "131040", "131041", "131042", "131043", "131044", "131045", "131046", "131047", "131048", "131049", "131050", "131051", "131052", "131053", "131054", "131055", "131056", "131057", "131058", "131059", "131060", "131061", "131062", "131063", "131064", "131065", "131066", "131067", "131068", "131069", "131070", "131071", "131072", "131073", "131074", "131075", "131076", "131077", "131078", "131079", "131080", "131081", "131082", "131083", "131084", "131085", "131086", "131087", "131088", "131089", "131090", "131091", "131092", "131093", "131094", "131095", "131096", "131097", "131098", "131099", "131100", "131101", "131102", "131103", "131104", "131105", "131106", "131107", "131108", "131109", "131110", "131111", "131112", "131113", "131114", "131115", "131116", "131117", "131118", "131119", "131120", "131121", "131122", "131123", "131124", "131125", "131126", "131127"]}, "2407": {"datafields": ["131128", "131129", "131130", "131131", "131132", "131133", "131134", "131135", "131136", "131137", "131138", "131139", "131140", "131141", "131142", "131143", "131144", "131145", "131146", "131147", "131148", "131149", "131150", "131151", "131152", "131153", "131154", "131155", "131156", "131157", "131158", "131159", "131160", "131161", "131162", "131163", "131164", "131165", "131166", "131167", "131168", "131169", "131170", "131171", "131172", "131173", "131174", "131175", "131176", "131177", "131178", "131179", "131180", "131181", "131182", "131183", "131184", "131185", "131186", "131187", "131188", "131189", "131190", "131191", "131192", "131193", "131194", "131195", "131196", "131197", "131198", "131199", "131200", "131201", "131202", "131203", "131204", "131205", "131206", "131207", "131208", "131209", "131210", "131211", "131212", "131213", "131214", "131215", "131216", "131217", "131218", "131219", "131220", "131221"]}, "2408": {"datafields": ["131222", "131223", "131224", "131225", "131226", "131227", "131228", "131229", "131230", "131231", "131232", "131233", "131234", "131235", "131236", "131237", "131238", "131239", "131240", "131241", "131242", "131243", "131244", "131245", "131246", "131247", "131248", "131249", "131250", "131251", "131252", "131253", "131254", "131255", "131256", "131257", "131258", "131259", "131260", "131261", "131262", "131263", "131264", "131265", "131266", "131267", "131268", "131269"]}, "2409": {"datafields": ["131270", "131271", "131272", "131273", "131274", "131275", "131276", "131277", "131278", "131279", "131280", "131281", "131282", "131283", "131284", "131285", "131286", "131287", "131288", "131289", "131290", "131291", "131292", "131293", "131294", "131295", "131296", "131297", "131298", "131299", "131300", "131301", "131302", "131303", "131304", "131305", "131306", "131307", "131308", "131309", "131310", "131311", "131312", "131313", "131314", "131315", "131316", "131317", "131318", "131319", "131320", "131321", "131322", "131323", "131324", "131325", "131326", "131327", "131328", "131329", "131330", "131331", "131332", "131333", "131334", "131335", "131336", "131337", "131338", "131339", "131340", "131341", "131342", "131343", "131344", "131345", "131346", "131347", "131348", "131349", "131350", "131351", "131352", "131353", "131354", "131355", "131356", "131357", "131358", "131359", "131360", "131361", "131362", "131363", "131364", "131365", "131366", "131367", "131368", "131369", "131370", "131371", "131372", "131373", "131374", "131375", "131376", "131377", "131378", "131379", "131380", "131381", "131382", "131383", "131384", "131385", "131386", "131387", "131388", "131389", "131390", "131391", "131392", "131393", "131394", "131395", "131396", "131397", "131398", "131399", "131400", "131401", "131402", "131403", "131404", "131405", "131406", "131407", "131408", "131409", "131410", "131411", "131412", "131413", "131414", "131415", "131416", "131417", "131418", "131419", "131420", "131421", "131422", "131423"]}, "2410": {"datafields": ["131424", "131425", "131426", "131427", "131428", "131429", "131430", "131431", "131432", "131433", "131434", "131435", "131436", "131437", "131438", "131439", "131440", "131441", "131442", "131443", "131444", "131445", "131446", "131447", "131448", "131449", "131450", "131451", "131452", "131453", "131454", "131455", "131456", "131457", "131458", "131459", "131460", "131461", "131462", "131463", "131464", "131465", "131466", "131467", "131468", "131469", "131470", "131471", "131472", "131473", "131474", "131475", "131476", "131477", "131478", "131479", "131480", "131481", "131482", "131483", "131484", "131485", "131486", "131487", "131488", "131489", "131490", "131491", "131492", "131493", "131494", "131495", "131496", "131497", "131498", "131499", "131500", "131501", "131502", "131503", "131504", "131505", "131506", "131507", "131508", "131509", "131510", "131511", "131512", "131513", "131514", "131515", "131516", "131517", "131518", "131519", "131520", "131521", "131522", "131523", "131524", "131525", "131526", "131527", "131528", "131529", "131530", "131531", "131532", "131533", "131534", "131535", "131536", "131537", "131538", "131539", "131540", "131541", "131542", "131543", "131544", "131545", "131546", "131547", "131548", "131549", "131550", "131551"]}, "2411": {"datafields": ["131552", "131553", "131554", "131555", "131556", "131557", "131558", "131559", "131560", "131561", "131562", "131563", "131564", "131565", "131566", "131567", "131568", "131569", "131570", "131571", "131572", "131573", "131574", "131575", "131576", "131577", "131578", "131579", "131580", "131581", "131582", "131583", "131584", "131585", "131586", "131587", "131588", "131589", "131590", "131591", "131592", "131593", "131594", "131595", "131596", "131597", "131598", "131599", "131600", "131601", "131602", "131603", "131604", "131605", "131606", "131607", "131608", "131609", "131610", "131611", "131612", "131613", "131614", "131615", "131616", "131617", "131618", "131619", "131620", "131621", "131622", "131623", "131624", "131625", "131626", "131627", "131628", "131629", "131630", "131631", "131632", "131633", "131634", "131635", "131636", "131637", "131638", "131639", "131640", "131641", "131642", "131643", "131644", "131645", "131646", "131647", "131648", "131649", "131650", "131651", "131652", "131653", "131654", "131655", "131656", "131657", "131658", "131659", "131660", "131661", "131662", "131663", "131664", "131665", "131666", "131667", "131668", "131669", "131670", "131671", "131672", "131673", "131674", "131675", "131676", "131677", "131678", "131679", "131680", "131681", "131682", "131683", "131684", "131685", "131686", "131687", "131688", "131689", "131690", "131691", "131692", "131693", "131694", "131695"]}, "2412": {"datafields": ["131696", "131697", "131698", "131699", "131700", "131701", "131702", "131703", "131704", "131705", "131706", "131707", "131708", "131709", "131710", "131711", "131712", "131713", "131714", "131715", "131716", "131717", "131718", "131719", "131720", "131721", "131722", "131723", "131724", "131725", "131726", "131727", "131728", "131729", "131730", "131731", "131732", "131733", "131734", "131735", "131736", "131737", "131738", "131739", "131740", "131741", "131742", "131743", "131744", "131745", "131746", "131747", "131748", "131749", "131750", "131751", "131752", "131753", "131754", "131755", "131756", "131757", "131758", "131759", "131760", "131761", "131762", "131763", "131764", "131765", "131766", "131767", "131768", "131769", "131770", "131771", "131772", "131773", "131774", "131775", "131776", "131777", "131778", "131779", "131780", "131781", "131782", "131783", "131784", "131785", "131786", "131787", "131788", "131789", "131790", "131791", "131792", "131793", "131794", "131795", "131796", "131797", "131798", "131799", "131800", "131801", "131802", "131803", "131804", "131805", "131806", "131807", "131808", "131809", "131810", "131811", "131812", "131813", "131814", "131815", "131816", "131817", "131818", "131819", "131820", "131821", "131822", "131823", "131824", "131825", "131826", "131827", "131828", "131829", "131830", "131831", "131832", "131833", "131834", "131835", "131836", "131837", "131838", "131839"]}, "2413": {"datafields": ["131840", "131841", "131842", "131843", "131844", "131845", "131846", "131847", "131848", "131849", "131850", "131851", "131852", "131853", "131854", "131855", "131856", "131857", "131858", "131859", "131860", "131861", "131862", "131863", "131864", "131865", "131866", "131867", "131868", "131869", "131870", "131871", "131872", "131873", "131874", "131875", "131876", "131877", "131878", "131879", "131880", "131881", "131882", "131883", "131884", "131885", "131886", "131887", "131888", "131889", "131890", "131891", "131892", "131893", "131894", "131895", "131896", "131897", "131898", "131899", "131900", "131901", "131902", "131903", "131904", "131905", "131906", "131907", "131908", "131909", "131910", "131911", "131912", "131913", "131914", "131915", "131916", "131917", "131918", "131919", "131920", "131921", "131922", "131923", "131924", "131925", "131926", "131927", "131928", "131929", "131930", "131931", "131932", "131933", "131934", "131935", "131936", "131937", "131938", "131939", "131940", "131941", "131942", "131943", "131944", "131945", "131946", "131947", "131948", "131949", "131950", "131951", "131952", "131953", "131954", "131955", "131956", "131957", "131958", "131959", "131960", "131961", "131962", "131963", "131964", "131965", "131966", "131967", "131968", "131969", "131970", "131971", "131972", "131973", "131974", "131975", "131976", "131977", "131978", "131979", "131980", "131981", "131982", "131983", "131984", "131985", "131986", "131987", "131988", "131989", "131990", "131991", "131992", "131993", "131994", "131995", "131996", "131997"]}, "2414": {"datafields": ["131998", "131999", "132000", "132001", "132002", "132003", "132004", "132005", "132006", "132007", "132008", "132009", "132010", "132011", "132012", "132013", "132014", "132015", "132016", "132017", "132018", "132019", "132020", "132021", "132022", "132023", "132024", "132025", "132026", "132027", "132028", "132029", "132030", "132031", "132032", "132033", "132034", "132035", "132036", "132037", "132038", "132039", "132040", "132041", "132042", "132043", "132044", "132045", "132046", "132047", "132048", "132049", "132050", "132051", "132052", "132053", "132054", "132055", "132056", "132057", "132058", "132059", "132060", "132061", "132062", "132063", "132064", "132065", "132066", "132067", "132068", "132069", "132070", "132071", "132072", "132073", "132074", "132075", "132076", "132077", "132078", "132079", "132080", "132081", "132082", "132083", "132084", "132085", "132086", "132087", "132088", "132089", "132090", "132091", "132092", "132093", "132094", "132095", "132096", "132097", "132098", "132099", "132100", "132101", "132102", "132103", "132104", "132105", "132106", "132107", "132108", "132109", "132110", "132111", "132112", "132113", "132114", "132115", "132116", "132117", "132118", "132119", "132120", "132121", "132122", "132123", "132124", "132125", "132126", "132127", "132128", "132129", "132130", "132131", "132132", "132133", "132134", "132135", "132136", "132137", "132138", "132139", "132140", "132141", "132142", "132143", "132144", "132145", "132146", "132147", "132148", "132149", "132150", "132151", "132152", "132153", "132154", "132155", "132156", "132157", "132158", "132159", "132160", "132161"]}, "2415": {"datafields": ["132162", "132163", "132164", "132165", "132166", "132167", "132168", "132169", "132170", "132171", "132172", "132173", "132174", "132175", "132176", "132177", "132178", "132179", "132180", "132181", "132182", "132183", "132184", "132185", "132186", "132187", "132188", "132189", "132190", "132191", "132192", "132193", "132194", "132195", "132196", "132197", "132198", "132199", "132200", "132201", "132202", "132203", "132204", "132205", "132206", "132207", "132208", "132209", "132210", "132211", "132212", "132213", "132214", "132215", "132216", "132217", "132218", "132219", "132220", "132221", "132222", "132223", "132224", "132225", "132226", "132227", "132228", "132229", "132230", "132231", "132232", "132233", "132234", "132235", "132236", "132237", "132238", "132239", "132240", "132241", "132242", "132243", "132244", "132245", "132246", "132247", "132248", "132249", "132250", "132251", "132252", "132253", "132254", "132255", "132256", "132257", "132258", "132259", "132260", "132261", "132262", "132263", "132264", "132265", "132266", "132267", "132268", "132269", "132270", "132271", "132272", "132273", "132274", "132275", "132276", "132277", "132278", "132279", "132280", "132281", "132282", "132283", "132284", "132285", "132286", "132287", "132288", "132289", "132290", "132291", "132292", "132293", "132294", "132295", "132296", "132297", "132298", "132299", "132300", "132301", "132302", "132303", "132304", "132305", "132306", "132307", "132308", "132309", "132310", "132311", "132312", "132313"]}, "2416": {"datafields": ["132314", "132315", "132316", "132317", "132318", "132319", "132320", "132321", "132322", "132323", "132324", "132325", "132326", "132327", "132328", "132329", "132330", "132331", "132332", "132333", "132334", "132335", "132336", "132337", "132338", "132339", "132340", "132341", "132342", "132343", "132344", "132345", "132346", "132347", "132348", "132349", "132350", "132351", "132352", "132353", "132354", "132355", "132356", "132357", "132358", "132359", "132360", "132361", "132362", "132363", "132364", "132365", "132366", "132367", "132368", "132369", "132370", "132371", "132372", "132373", "132374", "132375", "132376", "132377", "132378", "132379", "132380", "132381", "132382", "132383", "132384", "132385", "132386", "132387", "132388", "132389", "132390", "132391", "132392", "132393", "132394", "132395", "132396", "132397", "132398", "132399", "132400", "132401", "132402", "132403", "132404", "132405", "132406", "132407", "132408", "132409", "132410", "132411", "132412", "132413", "132414", "132415", "132416", "132417", "132418", "132419", "132420", "132421", "132422", "132423", "132424", "132425", "132426", "132427", "132428", "132429", "132430", "132431"]}, "2417": {"datafields": ["132432", "132433", "132434", "132435", "132436", "132437", "132438", "132439", "132440", "132441", "132442", "132443", "132444", "132445", "132446", "132447", "132448", "132449", "132450", "132451", "132452", "132453", "132454", "132455", "132456", "132457", "132458", "132459", "132460", "132461", "132462", "132463", "132464", "132465", "132466", "132467", "132468", "132469", "132470", "132471", "132472", "132473", "132474", "132475", "132476", "132477", "132478", "132479", "132480", "132481", "132482", "132483", "132484", "132485", "132486", "132487", "132488", "132489", "132490", "132491", "132492", "132493", "132494", "132495", "132496", "132497", "132498", "132499", "132500", "132501", "132502", "132503", "132504", "132505", "132506", "132507", "132508", "132509", "132510", "132511", "132512", "132513", "132514", "132515", "132516", "132517", "132518", "132519", "132520", "132521", "132522", "132523", "132524", "132525", "132526", "132527", "132528", "132529", "132530", "132531", "132532", "132533", "132534", "132535", "132536", "132537", "132538", "132539", "132540", "132541", "132542", "132543", "132544", "132545", "132546", "132547", "132548", "132549", "132550", "132551", "132552", "132553", "132554", "132555", "132556", "132557", "132558", "132559", "132560", "132561", "132562", "132563", "132564", "132565", "132566", "132567", "132568", "132569", "132570", "132571", "132572", "132573", "132574", "132575", "132576", "132577", "132578", "132579", "132580", "132581", "132582", "132583", "132584", "132585", "132586", "132587", "132588", "132589", "132590", "132591", "132592", "132593", "132594", "132595", "132596", "132597", "132598", "132599", "132600", "132601", "132602", "132603", "132604", "132605"]}, "1000": {"datafields": ["1020158", "1020201", "1020202", "1020203", "1020204", "1020206", "1020207", "1020208", "1020209", "1020210", "1020211", "1020212", "1020213", "1020214", "1020215", "1020216", "1020217", "1020218", "1020219", "1020220", "1020221", "1020222", "1020223", "1020224", "1020225", "1020226", "1020227", "1020228", "1020235", "1020241", "1020243", "1020249", "1020250", "1020251", "1020252", "1020253", "1020254", "1020259", "1020260", "1020263", "1020264", "1020265", "1020266", "1020267", "1025747", "1025748", "1025749", "1025750", "1025751", "1025752", "1025753", "1025754", "1025755", "1026300", "1026301", "2020158", "2020201", "2020202", "2020203", "2020204", "2020206", "2020207", "2020208", "2020209", "2020210", "2020211", "2020212", "2020213", "2020214", "2020215", "2020216", "2020217", "2020218", "2020219", "2020220", "2020221", "2020222", "2020223", "2020224", "2020225", "2020226", "2020227", "2020228", "2020235", "2020241", "2020243", "2020249", "2020250", "2020251", "2020252", "2020253", "2020254", "2020259", "2020260", "2020263", "2020264", "2020265", "2020266", "2020267", "2025747", "2025748", "2025749", "2025750", "2025751", "2025752", "2025753", "2025754", "2025755", "2026300", "2026301"]}}} -------------------------------------------------------------------------------- /ukbb_parser/scripts/data/self_report_level_map.csv: -------------------------------------------------------------------------------- 1 | coding,meaning,node_id,parent_id,selectable,Level 2 | -1,tropical & travel-related infections,1762,1495,N,1.0 3 | -1,dermatology,1510,1078,N,1.0 4 | -1,gynaecology/breast,1079,0,N,0.0 5 | -1,cardiovascular,1071,0,N,0.0 6 | -1,musculoskeletal/trauma,1077,0,N,0.0 7 | -1,renal/urology,1074,0,N,0.0 8 | -1,bowel problem,1155,1073,N,1.0 9 | -1,haematology/dermatology,1078,0,N,0.0 10 | -1,gastrointestinal/abdominal,1073,0,N,0.0 11 | -1,bacterial infection,1758,1495,N,1.0 12 | -1,other fractures,1713,1695,N,2.0 13 | -1,fracture,1695,1077,N,1.0 14 | -1,neurology,1265,1076,N,1.0 15 | -1,other urological problem (male),1231,1074,N,1.0 16 | -1,immunological/systemic disorders,1080,0,N,0.0 17 | -1,obstetric problem,1621,1079,N,1.0 18 | -1,cerebrovascular disease,1083,1071,N,1.0 19 | -1,haematology,1502,1078,N,1.0 20 | -1,neurology/eye/psychiatry,1076,0,N,0.0 21 | -1,fracture upper limb & shoulder,1702,1695,N,2.0 22 | -1,respiratory/ent,1072,0,N,0.0 23 | -1,fracture head & neck,1696,1695,N,2.0 24 | -1,viral infection,1747,1495,N,1.0 25 | -1,fracture pelvis & lower limb,1720,1695,N,2.0 26 | -1,infections,1495,0,N,0.0 27 | -1,endocrine/diabetes,1075,0,N,0.0 28 | -1,ent disorder/not cancer,1466,1072,N,1.0 29 | -1,substance abuse/dependency,1460,1269,N,2.0 30 | 1065,hypertension,1081,1071,Y,1.0 31 | 1066,heart/cardiac problem,1082,1071,Y,1.0 32 | 1067,peripheral vascular disease,1084,1071,Y,1.0 33 | 1068,venous thromboembolic disease,1085,1071,Y,1.0 34 | 1072,essential hypertension,1089,1081,Y,2.0 35 | 1073,gestational hypertension/pre-eclampsia,1090,1081,Y,2.0 36 | 1074,angina,1091,1082,Y,2.0 37 | 1075,heart attack/myocardial infarction,1092,1082,Y,2.0 38 | 1076,heart failure/pulmonary odema,1093,1082,Y,2.0 39 | 1077,heart arrhythmia,1094,1082,Y,2.0 40 | 1078,heart valve problem/heart murmur,1095,1082,Y,2.0 41 | 1079,cardiomyopathy,1096,1082,Y,2.0 42 | 1080,pericardial problem,1097,1082,Y,2.0 43 | 1081,stroke,1098,1083,Y,2.0 44 | 1082,transient ischaemic attack (tia),1099,1083,Y,2.0 45 | 1083,subdural haemorrhage/haematoma,1100,1083,Y,2.0 46 | 1086,subarachnoid haemorrhage,1103,1098,Y,3.0 47 | 1087,leg claudication/ intermittent claudication,1104,1084,Y,2.0 48 | 1088,arterial embolism,1105,1084,Y,2.0 49 | 1093,pulmonary embolism +/- dvt,1111,1085,Y,2.0 50 | 1094,deep venous thrombosis (dvt),1112,1085,Y,2.0 51 | 1111,asthma,1130,1072,Y,1.0 52 | 1112,chronic obstructive airways disease/copd,1131,1072,Y,1.0 53 | 1113,emphysema/chronic bronchitis,1132,1072,Y,1.0 54 | 1114,bronchiectasis,1133,1072,Y,1.0 55 | 1115,interstitial lung disease,1134,1072,Y,1.0 56 | 1117,other respiratory problems,1136,1072,Y,1.0 57 | 1120,asbestosis,1139,1134,Y,2.0 58 | 1121,pulmonary fibrosis,1140,1134,Y,2.0 59 | 1122,fibrosing alveolitis/unspecified alveolitis,1141,1134,Y,2.0 60 | 1123,sleep apnoea,1142,1136,Y,2.0 61 | 1124,respiratory failure,1143,1136,Y,2.0 62 | 1125,pleurisy,1144,1136,Y,2.0 63 | 1126,spontaneous pneumothorax/recurrent pneumothorax,1145,1560,Y,3.0 64 | 1134,oesophageal disorder,1153,1073,Y,1.0 65 | 1135,stomach disorder,1154,1073,Y,1.0 66 | 1136,liver/biliary/pancreas problem,1156,1073,Y,1.0 67 | 1137,other abdominal problem,1158,1073,Y,1.0 68 | 1138,gastro-oesophageal reflux (gord) / gastric reflux,1159,1153,Y,2.0 69 | 1139,oesophagitis/barretts oesophagus,1160,1153,Y,2.0 70 | 1140,oesophageal stricture,1161,1153,Y,2.0 71 | 1141,oesophageal varicies,1162,1153,Y,2.0 72 | 1142,gastric/stomach ulcers,1163,1154,Y,2.0 73 | 1143,gastritis/gastric erosions,1164,1154,Y,2.0 74 | 1154,irritable bowel syndrome,1175,1155,Y,2.0 75 | 1155,hepatitis,1176,1156,Y,2.0 76 | 1156,infective/viral hepatitis,1177,1176,Y,3.0 77 | 1157,non-infective hepatitis,1178,1176,Y,3.0 78 | 1158,liver failure/cirrhosis,1179,1156,Y,2.0 79 | 1159,bile duct disease,1180,1156,Y,2.0 80 | 1160,bile duct obstruction/ascending cholangitis,1181,1180,Y,3.0 81 | 1161,gall bladder disease,1182,1156,Y,2.0 82 | 1162,cholelithiasis/gall stones,1183,1182,Y,3.0 83 | 1163,cholecystitis,1184,1182,Y,3.0 84 | 1164,pancreatic disease,1185,1156,Y,2.0 85 | 1165,pancreatitis,1186,1185,Y,3.0 86 | 1190,peritonitis,1211,1158,Y,2.0 87 | 1191,gastrointestinal bleeding,1212,1158,Y,2.0 88 | 1192,renal/kidney failure,1213,1074,Y,1.0 89 | 1193,renal failure requiring dialysis,1214,1213,Y,2.0 90 | 1194,renal failure not requiring dialysis,1215,1213,Y,2.0 91 | 1196,urinary tract infection/kidney infection,1217,1074,Y,1.0 92 | 1197,kidney stone/ureter stone/bladder stone,1218,1074,Y,1.0 93 | 1200,ureteric obstruction/hydronephrosis,1221,1074,Y,1.0 94 | 1201,bladder problem (not cancer),1222,1074,Y,1.0 95 | 1202,urinary frequency / incontinence,1223,1222,Y,2.0 96 | 1207,prostate problem (not cancer),1232,1231,Y,2.0 97 | 1210,scrotal problem (not cancer),1235,1231,Y,2.0 98 | 1214,testicular problems (not cancer),1239,1231,Y,2.0 99 | 1220,diabetes,1245,1075,Y,1.0 100 | 1221,gestational diabetes,1246,1245,Y,2.0 101 | 1222,type 1 diabetes,1247,1245,Y,2.0 102 | 1223,type 2 diabetes,1248,1245,Y,2.0 103 | 1224,thyroid problem (not cancer),1249,1075,Y,1.0 104 | 1225,hyperthyroidism/thyrotoxicosis,1250,1249,Y,2.0 105 | 1226,hypothyroidism/myxoedema,1251,1249,Y,2.0 106 | 1228,thyroid radioablation therapy,1253,1249,Y,2.0 107 | 1229,parathyroid gland problem (not cancer),1254,1075,Y,1.0 108 | 1230,parathyroid hyperplasia/adenoma,1255,1254,Y,2.0 109 | 1232,disorder of adrenal gland,1257,1075,Y,1.0 110 | 1233,adrenal tumour,1258,1257,Y,2.0 111 | 1234,adrenocortical insufficiency/addison's disease,1259,1257,Y,2.0 112 | 1235,hyperaldosteronism/conn's syndrome,1260,1257,Y,2.0 113 | 1236,phaeochromocytoma,1261,1257,Y,2.0 114 | 1237,disorder or pituitary gland,1262,1075,Y,1.0 115 | 1238,pituitary adenoma/tumour,1263,1262,Y,2.0 116 | 1239,cushings syndrome,1264,1075,Y,1.0 117 | 1240,neurological injury/trauma,1266,1076,Y,1.0 118 | 1242,eye/eyelid problem,1268,1076,Y,1.0 119 | 1243,psychological/psychiatric problem,1269,1076,Y,1.0 120 | 1244,infection of nervous system,1270,1265,Y,2.0 121 | 1245,brain abscess/intracranial abscess,1271,1270,Y,3.0 122 | 1246,encephalitis,1272,1270,Y,3.0 123 | 1247,meningitis,1273,1270,Y,3.0 124 | 1248,spinal abscess,1274,1270,Y,3.0 125 | 1249,cranial nerve problem/palsy,1275,1265,Y,2.0 126 | 1250,bell's palsy/facial nerve palsy,1276,1275,Y,3.0 127 | 1251,spinal cord disorder,1277,1265,Y,2.0 128 | 1252,paraplegia,1278,1277,Y,3.0 129 | 1254,peripheral nerve disorder,1280,1265,Y,2.0 130 | 1255,peripheral neuropathy,1281,1280,Y,3.0 131 | 1256,acute infective polyneuritis/guillain-barre syndrome,1282,1280,Y,3.0 132 | 1257,trapped nerve/compressed nerve,1283,1280,Y,3.0 133 | 1258,chronic/degenerative neurological problem,1284,1265,Y,2.0 134 | 1259,motor neurone disease,1285,1284,Y,3.0 135 | 1260,myasthenia gravis,1286,1284,Y,3.0 136 | 1261,multiple sclerosis,1287,1284,Y,3.0 137 | 1262,parkinsons disease,1288,1284,Y,3.0 138 | 1263,dementia/alzheimers/cognitive impairment,1289,1284,Y,3.0 139 | 1264,epilepsy,1290,1265,Y,2.0 140 | 1265,migraine,1291,1265,Y,2.0 141 | 1266,head injury,1292,1266,Y,2.0 142 | 1267,spinal injury,1293,1266,Y,2.0 143 | 1274,eye infection,1300,1268,Y,2.0 144 | 1275,retinal problem,1301,1268,Y,2.0 145 | 1276,diabetic eye disease,1302,1268,Y,2.0 146 | 1277,glaucoma,1303,1268,Y,2.0 147 | 1278,cataract,1304,1268,Y,2.0 148 | 1279,eye trauma,1305,1268,Y,2.0 149 | 1281,retinal detachment,1307,1301,Y,3.0 150 | 1282,retinal artery/vein occlusion,1308,1301,Y,3.0 151 | 1286,depression,1312,1269,Y,2.0 152 | 1287,anxiety/panic attacks,1313,1269,Y,2.0 153 | 1288,nervous breakdown,1314,1269,Y,2.0 154 | 1289,schizophrenia,1315,1269,Y,2.0 155 | 1290,deliberate self-harm/suicide attempt,1316,1269,Y,2.0 156 | 1291,mania/bipolar disorder/manic depression,1317,1269,Y,2.0 157 | 1293,bone disorder,1319,1077,Y,1.0 158 | 1294,back problem,1320,1077,Y,1.0 159 | 1295,joint disorder,1321,1077,Y,1.0 160 | 1297,muscle/soft tissue problem,1323,1077,Y,1.0 161 | 1308,osteomyelitis,1347,1319,Y,2.0 162 | 1309,osteoporosis,1348,1319,Y,2.0 163 | 1310,paget's disease,1349,1319,Y,2.0 164 | 1311,spine arthritis/spondylitis,1350,1320,Y,2.0 165 | 1312,prolapsed disc/slipped disc,1351,1595,Y,3.0 166 | 1313,ankylosing spondylitis,1352,1320,Y,2.0 167 | 1322,myositis/myopathy,1361,1323,Y,2.0 168 | 1327,low platelets/platelet disorder,1366,1503,Y,3.0 169 | 1328,haemophilia,1367,1503,Y,3.0 170 | 1330,iron deficiency anaemia,1369,1504,Y,3.0 171 | 1331,pernicious anaemia,1370,1504,Y,3.0 172 | 1332,aplastic anaemia,1371,1504,Y,3.0 173 | 1339,sickle cell disease,1378,1509,Y,3.0 174 | 1340,thalassaemia,1379,1509,Y,3.0 175 | 1344,stevens johnson syndrome,1386,1513,Y,3.0 176 | 1345,pemphigoid/pemphigus,1387,1513,Y,3.0 177 | 1348,gynaecological disorder (not cancer),1390,1079,Y,1.0 178 | 1349,ovarian cyst or cysts,1391,1614,Y,3.0 179 | 1350,polycystic ovaries/polycystic ovarian syndrome,1392,1614,Y,3.0 180 | 1351,uterine fibroids,1393,1615,Y,3.0 181 | 1352,uterine polyps,1394,1615,Y,3.0 182 | 1353,vaginal prolapse/uterine prolapse,1395,1615,Y,3.0 183 | 1364,breast disease (not cancer),1407,1079,Y,1.0 184 | 1366,fibrocystic disease,1409,1407,Y,2.0 185 | 1367,breast cysts,1410,1407,Y,2.0 186 | 1371,sarcoidosis,1414,1080,Y,1.0 187 | 1372,vasculitis,1415,1416,Y,2.0 188 | 1373,connective tissue disorder,1416,1080,Y,1.0 189 | 1374,allergy/hypersensitivity/anaphylaxis,1417,1080,Y,1.0 190 | 1376,giant cell/temporal arteritis,1419,1415,Y,3.0 191 | 1377,polymyalgia rheumatica,1420,1415,Y,3.0 192 | 1378,wegners granulmatosis,1421,1415,Y,3.0 193 | 1379,microscopic polyarteritis,1422,1415,Y,3.0 194 | 1380,polyartertis nodosa,1423,1415,Y,3.0 195 | 1381,systemic lupus erythematosis/sle,1424,1416,Y,2.0 196 | 1382,sjogren's syndrome/sicca syndrome,1425,1416,Y,2.0 197 | 1383,dermatopolymyositis,1426,1416,Y,2.0 198 | 1384,scleroderma/systemic sclerosis,1427,1416,Y,2.0 199 | 1385,allergy or anaphylactic reaction to food,1428,1417,Y,2.0 200 | 1386,allergy or anaphylactic reaction to drug,1429,1417,Y,2.0 201 | 1387,hayfever/allergic rhinitis,1430,1417,Y,2.0 202 | 1394,peripheral nerve injury,1438,1266,Y,2.0 203 | 1396,enlarged prostate,1441,1232,Y,3.0 204 | 1397,other demyelinating disease (not multiple sclerosis),1442,1284,Y,3.0 205 | 1398,pneumonia,1443,1660,Y,2.0 206 | 1400,peptic ulcer,1445,1158,Y,2.0 207 | 1402,endometriosis,1451,1615,Y,3.0 208 | 1403,female infertility,1452,1390,Y,2.0 209 | 1404,male infertility,1453,1231,Y,2.0 210 | 1405,other renal/kidney problem,1454,1074,Y,1.0 211 | 1406,muscle or soft tissue injuries,1457,1323,Y,2.0 212 | 1407,burns,1458,1077,Y,1.0 213 | 1408,alcohol dependency,1461,1460,Y,3.0 214 | 1409,opioid dependency,1462,1460,Y,3.0 215 | 1410,other substance abuse/dependency,1463,1460,Y,3.0 216 | 1411,lung abscess,1464,1660,Y,2.0 217 | 1412,bronchitis,1465,1132,Y,2.0 218 | 1413,nasal/sinus disorder,1467,1466,Y,2.0 219 | 1414,throat or larynx disorder,1468,1466,Y,2.0 220 | 1415,ear/vestibular disorder,1469,1466,Y,2.0 221 | 1416,chronic sinusitis,1470,1467,Y,3.0 222 | 1417,nasal polyps,1471,1467,Y,3.0 223 | 1418,chronic laryngitis,1472,1468,Y,3.0 224 | 1419,vocal cord polyp,1473,1468,Y,3.0 225 | 1420,otosclerosis,1474,1469,Y,3.0 226 | 1421,meniere's disease,1475,1469,Y,3.0 227 | 1425,cerebral aneurysm,1480,1083,Y,2.0 228 | 1426,myocarditis,1481,1082,Y,2.0 229 | 1427,polycystic kidney,1482,1454,Y,2.0 230 | 1428,thyroiditis,1483,1249,Y,2.0 231 | 1429,acromegaly,1484,1262,Y,2.0 232 | 1430,hypopituitarism,1485,1262,Y,2.0 233 | 1431,hyperprolactinaemia,1486,1262,Y,2.0 234 | 1432,carcinoid syndrome/tumour,1487,1075,Y,1.0 235 | 1433,cerebral palsy,1488,1265,Y,2.0 236 | 1434,other neurological problem,1489,1265,Y,2.0 237 | 1435,optic neuritis,1490,1268,Y,2.0 238 | 1436,headaches (not migraine),1491,1489,Y,3.0 239 | 1437,myasthenia gravis,1492,1489,Y,3.0 240 | 1438,polycythaemia vera,1493,1507,Y,3.0 241 | 1439,hiv/aids,1748,1747,Y,2.0 242 | 1440,tuberculosis (tb),1761,1758,Y,2.0 243 | 1441,malaria,1763,1762,Y,2.0 244 | 1442,helicobacter pylori,1760,1758,Y,2.0 245 | 1443,schistosomiasis/bilharzia,1764,1762,Y,2.0 246 | 1445,clotting disorder/excessive bleeding,1503,1502,Y,2.0 247 | 1446,anaemia,1504,1502,Y,2.0 248 | 1447,pancytopenia,1505,1502,Y,2.0 249 | 1448,neutropenia/lymphopenia,1506,1502,Y,2.0 250 | 1449,myeloproliferative disorder,1507,1502,Y,2.0 251 | 1450,monoclonal gammopathy/not myeloma,1508,1502,Y,2.0 252 | 1451,hereditary/genetic haematological disorder,1509,1502,Y,2.0 253 | 1452,eczema/dermatitis,1511,1510,Y,2.0 254 | 1453,psoriasis,1512,1510,Y,2.0 255 | 1454,blistering/desquamating skin disorder,1513,1510,Y,2.0 256 | 1455,chronic skin ulcers,1514,1510,Y,2.0 257 | 1456,malabsorption/coeliac disease,1516,1155,Y,2.0 258 | 1457,duodenal ulcer,1517,1155,Y,2.0 259 | 1458,diverticular disease/diverticulitis,1518,1155,Y,2.0 260 | 1459,colitis/not crohns or ulcerative colitis,1519,1155,Y,2.0 261 | 1460,rectal or colon adenoma/polyps,1520,1155,Y,2.0 262 | 1461,inflammatory bowel disease,1521,1155,Y,2.0 263 | 1462,crohns disease,1522,1521,Y,3.0 264 | 1463,ulcerative colitis,1523,1521,Y,3.0 265 | 1464,rheumatoid arthritis,1524,1321,Y,2.0 266 | 1465,osteoarthritis,1525,1321,Y,2.0 267 | 1466,gout,1526,1321,Y,2.0 268 | 1467,other joint disorder,1527,1321,Y,2.0 269 | 1468,diabetic neuropathy/ulcers,1528,1280,Y,3.0 270 | 1469,post-traumatic stress disorder,1530,1269,Y,2.0 271 | 1470,anorexia/bulimia/other eating disorder,1531,1269,Y,2.0 272 | 1471,atrial fibrillation,1532,1094,Y,3.0 273 | 1472,emphysema,1534,1132,Y,2.0 274 | 1473,high cholesterol,1536,1071,Y,1.0 275 | 1474,hiatus hernia,1537,1153,Y,2.0 276 | 1475,sclerosing cholangitis,1538,1180,Y,3.0 277 | 1476,sciatica,1539,1320,Y,2.0 278 | 1477,psoriatic arthropathy,1540,1321,Y,2.0 279 | 1478,cervical spondylosis,1541,1608,Y,2.0 280 | 1479,rheumatic fever,1542,1082,Y,2.0 281 | 1480,dermatomyositis,1543,1426,Y,3.0 282 | 1481,polymyositis,1544,1426,Y,3.0 283 | 1482,chronic fatigue syndrome,1545,1080,Y,1.0 284 | 1483,atrial flutter,1546,1094,Y,3.0 285 | 1484,wolff parkinson white / wpw syndrome,1547,1094,Y,3.0 286 | 1485,irregular heart beat,1548,1094,Y,3.0 287 | 1486,sick sinus syndrome,1549,1094,Y,3.0 288 | 1487,svt / supraventricular tachycardia,1550,1094,Y,3.0 289 | 1488,mitral valve prolapse,1551,1650,Y,4.0 290 | 1489,mitral stenosis,1552,1650,Y,4.0 291 | 1490,aortic stenosis,1553,1652,Y,4.0 292 | 1491,brain haemorrhage,1554,1098,Y,3.0 293 | 1492,aortic aneurysm,1555,1084,Y,2.0 294 | 1493,other venous/lymphatic disease,1556,1071,Y,1.0 295 | 1494,varicose veins,1557,1556,Y,2.0 296 | 1495,lymphoedema,1558,1556,Y,2.0 297 | 1496,alpha-1 antitrypsin deficiency,1559,1132,Y,2.0 298 | 1497,pneumothorax,1560,1136,Y,2.0 299 | 1498,empyema,1561,1660,Y,2.0 300 | 1499,labyrinthitis,1562,1469,Y,3.0 301 | 1500,vertigo,1563,1469,Y,3.0 302 | 1501,pyloric stenosis,1564,1154,Y,2.0 303 | 1502,appendicitis,1565,1155,Y,2.0 304 | 1503,anal problem,1566,1155,Y,2.0 305 | 1504,anal fissure,1567,1566,Y,3.0 306 | 1505,haemorrhoids / piles,1568,1566,Y,3.0 307 | 1506,primary biliary cirrhosis,1569,1179,Y,3.0 308 | 1507,haemochromatosis,1570,1156,Y,2.0 309 | 1508,jaundice (unknown cause),1571,1156,Y,2.0 310 | 1509,gastroenteritis/dysentry,1572,1158,Y,2.0 311 | 1510,dyspepsia / indigestion,1573,1158,Y,2.0 312 | 1511,abdominal hernia,1574,1158,Y,2.0 313 | 1512,umbilical hernia,1575,1574,Y,3.0 314 | 1513,inguinal hernia,1576,1574,Y,3.0 315 | 1514,cystitis,1577,1217,Y,2.0 316 | 1515,pyelonephritis,1578,1217,Y,2.0 317 | 1516,bph / benign prostatic hypertrophy,1579,1232,Y,3.0 318 | 1517,prostatitis,1580,1232,Y,3.0 319 | 1518,erectile dysfunction / impotence,1581,1231,Y,2.0 320 | 1519,kidney nephropathy,1582,1454,Y,2.0 321 | 1520,iga nephropathy,1583,1582,Y,3.0 322 | 1521,diabetes insipidus,1584,1245,Y,2.0 323 | 1522,grave's disease,1585,1249,Y,2.0 324 | 1523,trigemminal neuralgia,1586,1275,Y,3.0 325 | 1524,spina bifida,1587,1277,Y,3.0 326 | 1525,benign / essential tremor,1588,1489,Y,3.0 327 | 1526,polio / poliomyelitis,1589,1489,Y,3.0 328 | 1527,retinitis pigmentosa,1590,1301,Y,3.0 329 | 1528,macular degeneration,1591,1301,Y,3.0 330 | 1529,dry eyes,1592,1268,Y,2.0 331 | 1530,iritis,1593,1268,Y,2.0 332 | 1531,post-natal depression,1594,1312,Y,3.0 333 | 1532,disc problem,1595,1320,Y,2.0 334 | 1533,disc degeneration,1596,1595,Y,3.0 335 | 1534,back pain,1597,1320,Y,2.0 336 | 1535,scoliosis,1598,1320,Y,2.0 337 | 1536,spinal stenosis,1599,1320,Y,2.0 338 | 1537,joint pain,1600,1527,Y,3.0 339 | 1538,arthritis (nos),1601,1321,Y,2.0 340 | 1540,plantar fascitis,1603,1323,Y,2.0 341 | 1541,carpal tunnel syndrome,1604,1323,Y,2.0 342 | 1542,fibromyalgia,1605,1323,Y,2.0 343 | 1544,dupuytren's contracture,1607,1323,Y,2.0 344 | 1545,neck problem/injury,1608,1077,Y,1.0 345 | 1546,essential thrombocytosis,1609,1503,Y,3.0 346 | 1548,acne/acne vulgaris,1611,1510,Y,2.0 347 | 1549,lichen planus,1612,1510,Y,2.0 348 | 1550,lichen sclerosis,1613,1510,Y,2.0 349 | 1551,ovarian problem,1614,1390,Y,2.0 350 | 1552,uterine problem,1615,1390,Y,2.0 351 | 1553,cervical problem,1616,1390,Y,2.0 352 | 1554,cervical intra-epithelial neoplasia (cin) / precancerous cells cervix,1617,1616,Y,3.0 353 | 1555,cervical polyps,1618,1616,Y,3.0 354 | 1556,menorrhagia (unknown cause),1619,1390,Y,2.0 355 | 1557,pelvic inflammatory disease/ pid,1620,1390,Y,2.0 356 | 1558,ectopic pregnancy,1622,1621,Y,2.0 357 | 1559,miscarriage,1623,1621,Y,2.0 358 | 1560,breast fibroadenoma,1624,1407,Y,2.0 359 | 1561,raynaud's phenomenon/disease,1625,1416,Y,2.0 360 | 1562,food intolerance,1626,1417,Y,2.0 361 | 1563,urticaria,1627,1417,Y,2.0 362 | 1564,antiphospholipid syndrome,1628,1080,Y,1.0 363 | 1566,mrsa / methicillin resistant staphylococcus aureus,1759,1758,Y,2.0 364 | 1567,infectious mononucleosis / glandular fever / epstein barr virus (ebv),1756,1747,Y,2.0 365 | 1568,measles / morbillivirus,1750,1747,Y,2.0 366 | 1569,mumps / epidemic parotitis,1757,1747,Y,2.0 367 | 1570,rubella / german measles,1751,1747,Y,2.0 368 | 1571,chickenpox,1753,1752,Y,3.0 369 | 1572,whooping cough / pertussis,1769,1758,Y,2.0 370 | 1573,shingles,1754,1752,Y,3.0 371 | 1574,diphtheria,1768,1758,Y,2.0 372 | 1575,herpes simplex,1755,1747,Y,2.0 373 | 1576,dengue fever,1765,1762,Y,2.0 374 | 1577,typhoid fever,1766,1762,Y,2.0 375 | 1578,hepatitis a,1642,1177,Y,4.0 376 | 1579,hepatitis b,1643,1177,Y,4.0 377 | 1580,hepatitis c,1644,1177,Y,4.0 378 | 1581,hepatitis d,1645,1177,Y,4.0 379 | 1582,hepatitis e,1646,1177,Y,4.0 380 | 1583,ischaemic stroke,1649,1098,Y,3.0 381 | 1584,mitral valve disease,1650,1095,Y,3.0 382 | 1585,mitral regurgitation / incompetence,1651,1650,Y,4.0 383 | 1586,aortic valve disease,1652,1095,Y,3.0 384 | 1587,aortic regurgitation / incompetence,1653,1652,Y,4.0 385 | 1588,hypertrophic cardiomyopathy (hcm / hocm),1654,1096,Y,3.0 386 | 1589,pericarditis,1655,1097,Y,3.0 387 | 1590,pericardial effusion,1656,1097,Y,3.0 388 | 1591,aortic aneurysm rupture,1657,1555,Y,3.0 389 | 1592,aortic dissection,1658,1555,Y,3.0 390 | 1593,varicose ulcer,1659,1556,Y,2.0 391 | 1594,respiratory infection,1660,1072,Y,1.0 392 | 1595,pleural plaques (not known asbestosis),1662,1136,Y,2.0 393 | 1596,pleural effusion,1663,1136,Y,2.0 394 | 1597,tinnitus / tiniitis,1664,1469,Y,3.0 395 | 1598,tonsiltis,1665,1468,Y,3.0 396 | 1599,constipation,1666,1155,Y,2.0 397 | 1600,bowel / intestinal perforation,1667,1155,Y,2.0 398 | 1601,bowel / intestinal infarction,1668,1155,Y,2.0 399 | 1602,bowel / intestinal obstruction,1670,1155,Y,2.0 400 | 1603,rectal prolapse,1671,1155,Y,2.0 401 | 1604,alcoholic liver disease / alcoholic cirrhosis,1672,1179,Y,3.0 402 | 1605,femoral hernia,1673,1574,Y,3.0 403 | 1606,incisional hernia,1674,1574,Y,3.0 404 | 1607,diabetic nephropathy,1675,1582,Y,3.0 405 | 1608,nephritis,1676,1454,Y,2.0 406 | 1609,glomerulnephritis,1677,1676,Y,3.0 407 | 1610,thyroid goitre,1678,1249,Y,2.0 408 | 1611,hyperparathyroidism,1679,1254,Y,2.0 409 | 1613,blepharitis / eyelid infection,1681,1268,Y,2.0 410 | 1614,stress,1682,1269,Y,2.0 411 | 1615,obsessive compulsive disorder (ocd),1683,1269,Y,2.0 412 | 1616,insomnia,1684,1269,Y,2.0 413 | 1617,osteopenia,1685,1319,Y,2.0 414 | 1618,soft tissue inflammation,1686,1323,Y,2.0 415 | 1619,tendonitis / tendinitis / tenosynovitis,1687,1686,Y,3.0 416 | 1620,bursitis,1688,1686,Y,3.0 417 | 1621,synovitis,1689,1686,Y,3.0 418 | 1622,epicondylitis,1690,1686,Y,3.0 419 | 1623,tennis elbow / lateral epicondylitis,1691,1690,Y,4.0 420 | 1624,housemaid's knee (prepatellar bursitis),1692,1688,Y,4.0 421 | 1625,cellulitis,1693,1510,Y,2.0 422 | 1626,fracture skull / head,1697,1696,Y,3.0 423 | 1627,fracture jaw,1698,1696,Y,3.0 424 | 1628,fracture nose,1699,1696,Y,3.0 425 | 1629,fracture face / orbit / eye socket,1700,1696,Y,3.0 426 | 1630,fracture neck / cervical fracture,1701,1696,Y,3.0 427 | 1631,fracture clavicle / collar bone,1703,1702,Y,3.0 428 | 1632,fracture shoulder / scapula,1704,1702,Y,3.0 429 | 1633,fracture upper arm / humerus / elbow,1705,1702,Y,3.0 430 | 1634,fracture forearm / wrist,1706,1702,Y,3.0 431 | 1635,fracture radius,1707,1706,Y,4.0 432 | 1636,fracture ulna,1708,1706,Y,4.0 433 | 1637,fracture wrist / colles fracture,1709,1706,Y,4.0 434 | 1638,fracture hand,1710,1702,Y,3.0 435 | 1639,fracture finger,1711,1702,Y,3.0 436 | 1640,fracture thumb,1712,1702,Y,3.0 437 | 1644,fracture rib,1717,1713,Y,3.0 438 | 1645,fracture sternum,1718,1713,Y,3.0 439 | 1646,fracture vertebra / crush fracture / vertebral collapse,1719,1713,Y,3.0 440 | 1647,fracture pelvis,1721,1720,Y,3.0 441 | 1648,fracture neck of femur / hip,1722,1720,Y,3.0 442 | 1649,fracture shaft of femur,1723,1720,Y,3.0 443 | 1650,fracture patella / knee,1724,1720,Y,3.0 444 | 1651,fracture lower leg / ankle,1725,1720,Y,3.0 445 | 1652,fracture tibia,1726,1725,Y,4.0 446 | 1653,fracture fibula,1727,1725,Y,4.0 447 | 1654,fracture foot,1728,1720,Y,3.0 448 | 1655,fracture metatarsal,1729,1728,Y,4.0 449 | 1656,fracture toe,1730,1728,Y,4.0 450 | 1657,septicaemia / sepsis,1731,1495,Y,1.0 451 | 1658,myelofibrosis,1732,1507,Y,3.0 452 | 1659,meningioma / benign meningeal tumour,1733,1489,Y,3.0 453 | 1660,rosacea,1734,1510,Y,2.0 454 | 1661,vitiligo,1735,1510,Y,2.0 455 | 1662,cervical erosion,1736,1616,Y,3.0 456 | 1663,abnormal smear (cervix),1737,1617,Y,4.0 457 | 1664,dysmenorrhoea / dysmenorrhea,1738,1390,Y,2.0 458 | 1665,menopausal symptoms / menopause,1739,1390,Y,2.0 459 | 1666,benign breast lump,1740,1407,Y,2.0 460 | 1667,alopecia / hair loss,1741,1510,Y,2.0 461 | 1668,allergy to house dust mite,1742,1417,Y,2.0 462 | 1669,contact dermatitis,1743,1417,Y,2.0 463 | 1670,allergy to elastoplast,1744,1743,Y,3.0 464 | 1671,allergy to nickel,1745,1743,Y,3.0 465 | 1674,varicella zoster virus,1752,1747,Y,2.0 466 | 1675,giardia / giardiasis,1767,1762,Y,2.0 467 | 1676,yellow fever,1770,1762,Y,2.0 468 | 1677,scarlet fever / scarlatina,1771,1758,Y,2.0 469 | 1678,chlamydia,1772,1758,Y,2.0 470 | 1679,undescended testicle,1773,1231,Y,2.0 471 | 1680,bowen's disease,1776,1510,Y,2.0 472 | 1681,hydatiform mole,1777,1391,Y,4.0 473 | 1682,benign insulinoma,1778,1075,Y,1.0 474 | 1683,benign neuroma,1779,1489,Y,3.0 475 | 99999,unclassifiable,99999,0,N,0.0 476 | -------------------------------------------------------------------------------- /ukbb_parser/scripts/demo_conversion.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import numpy as np 3 | 4 | def calculate_float_ages(dataFrame): 5 | convert = True 6 | if "Age1stVisit" not in dataFrame.columns: 7 | for col in ["34-0.0", "52-0.0", "53-0.0", "53-1.0", "53-2.0", "53-3.0"]: 8 | if col not in dataFrame.columns: 9 | convert = False 10 | 11 | if convert is True: 12 | for i, v in enumerate(["Age1stVisit", "AgeRepVisit", "AgeAtScan", "AgeAt2ndScan"]): 13 | try: 14 | y = [int(d.split('-')[0]) for d in dataFrame["53-%i.0" %i]] 15 | m = [int(d.split('-')[1]) for d in dataFrame["53-%i.0" %i]] 16 | dataFrame[v] = y - dataFrame['34-0.0'] + (m-dataFrame["52-0.0"])/12. 17 | except AttributeError: 18 | ages = [] 19 | date_ix = dataFrame["53-%i.0" %i].index.tolist() 20 | for j, d in enumerate(dataFrame["53-%i.0" %i]): 21 | if isinstance(d, str): 22 | y = int(d.split('-')[0]) 23 | m = int(d.split('-')[1]) 24 | ages.append(y - dataFrame['34-0.0'][date_ix[j]] + (m-dataFrame["52-0.0"][date_ix[j]])/12.) 25 | else: 26 | ages.append(np.nan) 27 | dataFrame[v] = ages 28 | 29 | return dataFrame, convert 30 | 31 | def convert_ethnicity(val): 32 | if np.isnan(val): 33 | new_val = np.nan 34 | elif len(str(int(val))) == 4: 35 | new_val = int(str(int(val))[0]) 36 | elif val == 5: 37 | new_val = 3 38 | elif val < 0: 39 | new_val = np.nan 40 | else: 41 | new_val = val 42 | return new_val 43 | 44 | def add_ethnicity_columns(dataFrame): 45 | 46 | convert = True 47 | if "Race" not in dataFrame.columns: 48 | 49 | ethn = [] 50 | 51 | for c in dataFrame.columns: 52 | if c.startswith('21000'): 53 | ethn.append(c) 54 | 55 | if len(ethn) > 0: 56 | converted_ethn = dataFrame[ethn] 57 | 58 | for c in ethn: 59 | converted_ethn[c] = converted_ethn[c].apply(convert_ethnicity) 60 | 61 | single_ethn = [] 62 | 63 | for i, row in converted_ethn.iterrows(): 64 | if len(converted_ethn.loc[i, ethn].value_counts()) > 1: 65 | possibilities = list(converted_ethn.loc[i, ethn].value_counts().keys()) 66 | if 2 in possibilities: 67 | single_ethn.append(2) 68 | elif 6 in possibilities: 69 | possibilities.remove(6) 70 | single_ethn.append(possibilities[0]) 71 | else: 72 | single_ethn.append(possibilities[0]) 73 | elif len(converted_ethn.loc[i, ethn].value_counts()) == 0: 74 | single_ethn.append(np.nan) 75 | else: 76 | single_ethn.append(converted_ethn.loc[i, ethn].value_counts().keys()[0]) 77 | 78 | dataFrame['Race'] = single_ethn 79 | else: 80 | convert = False 81 | 82 | return dataFrame, convert 83 | 84 | def add_education_columns(dataFrame): 85 | 86 | ukbb2eduyrs = {1: 19, 87 | 2: 13, 88 | 3: 10, 89 | 4: 10, 90 | 5: 19, 91 | 6: 15, 92 | -7: 7, 93 | -3: np.nan} 94 | 95 | eduyrs2isced = {19: 5, 96 | 15: 6, 97 | 13: 3, 98 | 10: 2, 99 | 7: 1} 100 | 101 | convert = True 102 | 103 | if "ISCED" not in dataFrame.columns: 104 | quals = [] 105 | 106 | for c in dataFrame.columns: 107 | if c.startswith('6138-'): 108 | quals.append(c) 109 | 110 | if len(quals) > 0: 111 | converted_quals = dataFrame[['eid'] + quals] 112 | for c in quals: 113 | converted_quals = converted_quals.replace({c: ukbb2eduyrs}) 114 | 115 | dataFrame['YearsOfEducation'] = np.nanmax(converted_quals[quals], axis=1) 116 | dataFrame['ISCED'] = dataFrame['YearsOfEducation'].replace(eduyrs2isced) 117 | else: 118 | convert = False 119 | 120 | return dataFrame, convert 121 | -------------------------------------------------------------------------------- /ukbb_parser/scripts/level_processing.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import pandas as pd 3 | from ukbb_parser.scripts.utils import find_icd10_ix_range 4 | 5 | def level_processing(dataFrame, datatype, datafields, level_map, code, level, sublevels): 6 | # Parse Input 7 | if level != "S": 8 | level = int(level) 9 | 10 | if datatype == "icd10": 11 | code_column = "Coding" 12 | parent_column = "Parent" 13 | node_column = "Node" 14 | selectable_column = "Selectable" 15 | col_pref = "icd10_" 16 | level_map_df = pd.read_csv(level_map, index_col=code_column) 17 | elif datatype in ["self_report", "careers"]: 18 | code_column = "coding" 19 | parent_column = "parent_id" 20 | node_column = "node_id" 21 | selectable_column = "selectable" 22 | if datatype == "self_report": 23 | col_pref = "sr_" 24 | elif datatype == "careers": 25 | col_pref = "job_" 26 | level_map_df = pd.read_csv(level_map, index_col=code_column) 27 | 28 | # Breakdown Input Codes and Inventory 29 | if level == "S": 30 | selectable = level_map_df.loc[level_map_df[selectable_column].isin(["Y", "Yes"])] 31 | codes_to_inventory = selectable.index.tolist() 32 | for s in codes_to_inventory: 33 | dataFrame[col_pref+str(s).replace(" ", "_")] = dataFrame[datafields].isin([s]).any(axis=1).astype(int) 34 | return dataFrame 35 | else: 36 | codes_dict = {} 37 | codes_to_inventory = get_level_codes(datatype, level_map_df, level, code) 38 | for cti in codes_to_inventory: 39 | codes_dict[cti] = {"branches": [cti], 40 | "leaves": []} 41 | inventory_codes_dict = get_sublevel_data(codes_dict, level_map_df, parent_column, node_column, selectable_column, level) 42 | for k,v in inventory_codes_dict.items(): 43 | dataFrame[col_pref+str(k).replace(" ", "_")] = dataFrame[datafields].isin(v['leaves']).any(axis=1).astype(int) 44 | if sublevels == True: 45 | for l in v['leaves']: 46 | dataFrame[col_pref+str(l).replace(" ", "_")] = dataFrame[datafields].isin([l]).any(axis=1).astype(int) 47 | return dataFrame 48 | 49 | def get_sublevel_data(codes_dict, level_map, parent_column, node_column, selectable_column, level): 50 | while level < 5: 51 | for k, v in codes_dict.items(): 52 | branches = [] 53 | for c in v['branches']: 54 | try: 55 | node_id = level_map.loc[c, node_column] 56 | if level_map.loc[c, selectable_column] in ["Y", "Yes"]: 57 | codes_dict[k]['leaves'].append(c) 58 | sr_neg_1 = False 59 | except KeyError: 60 | node_id = level_map.loc[level_map.meaning == c, node_column].values[0] 61 | sr_neg_1 = True 62 | if node_id in level_map[parent_column].tolist(): 63 | children = level_map.loc[level_map[parent_column] == node_id] 64 | elif str(node_id) in level_map[parent_column].tolist(): 65 | children = level_map.loc[level_map[parent_column] == str(node_id)] 66 | else: 67 | continue 68 | if sr_neg_1: 69 | branches += children.loc[children[selectable_column].isin(["N", "No"]), "meaning"].tolist() 70 | else: 71 | branches += children.loc[children[selectable_column].isin(["N", "No"])].index.tolist() 72 | codes_dict[k]['leaves'] += children.loc[children[selectable_column].isin(["Y", "Yes"])].index.tolist() 73 | codes_dict[k]['branches'] = branches 74 | level += 1 75 | 76 | return codes_dict 77 | 78 | 79 | def get_level_codes(datatype, level_map, level, code): 80 | codes_to_inventory = [] 81 | 82 | if datatype == "icd10": 83 | if code == "all": 84 | codes_to_inventory = level_map.loc[level_map.Level == level].index.tolist() 85 | elif ("-" in code) and ("Block" not in code): 86 | start_loc = code.split("-")[0] 87 | end_loc = code.split("-")[1] 88 | level_df = level_map.loc[level_map.Level == level] 89 | start_loc, end_loc = find_icd10_ix_range(level_df, start_loc, end_loc) 90 | codes_to_inventory += level_df.loc[start_loc: end_loc].index.tolist() 91 | else: 92 | codes_to_inventory.append(code) 93 | elif datatype == "self_report": 94 | if code == "all": 95 | level_codes = level_map.loc[level_map.Level == level] 96 | codes_to_inventory = [] 97 | for i, row in level_codes.iterrows(): 98 | if i == -1: 99 | codes_to_inventory.append(row["meaning"]) 100 | else: 101 | codes_to_inventory.append(int(i)) 102 | elif "-" in code: 103 | level_df = level_map.loc[level_map.Level == level] 104 | code_range += list(range(int(code.split("-")[0]), int(code.split("-")[1]) + 1)) 105 | codes_to_inventory += level_df.loc[level_df.index.isin(code_range)].index.tolist() 106 | elif not code[0].isdigit(): 107 | codes_to_inventory.append(code) 108 | else: 109 | codes_to_inventory.append(int(code)) 110 | elif datatype == "careers": 111 | if code == "all": 112 | codes_to_inventory = level_map.loc[level_map.Level == level].index.tolist() 113 | elif "-" in code: 114 | level_df = level_map.loc[level_map.Level == level] 115 | code_range += list(range(int(code.split("-")[0]), int(code.split("-")[1]) + 1)) 116 | codes_to_inventory += level_df.loc[level_df.index.isin(code_range)].index.tolist() 117 | else: 118 | codes_to_inventory.append(int(code)) 119 | return codes_to_inventory 120 | 121 | -------------------------------------------------------------------------------- /ukbb_parser/scripts/utils.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import pandas as pd 3 | import os 4 | 5 | def read_spreadsheet(datafile, filetype='csv'): 6 | delimiter = ',' 7 | file_reader = pd.read_csv 8 | if filetype=='unknown': 9 | if datafile.endswith(".csv"): 10 | file_reader = pd.read_csv 11 | elif datafile.endswith(".xls") or datafile.endswith(".xlsx"): 12 | file_reader = pd.read_excel 13 | delimiter = None 14 | elif datafile.endswith(".txt") or datafile.endswith(".tsv"): 15 | file_reader = pd.read_csv 16 | delimiter = "\t" 17 | else: 18 | file_reader = pd.read_csv 19 | delimiter='\s+' 20 | file_size = os.stat(datafile).st_size # This is in bytes 21 | if file_size > 500000000: 22 | chunk_size=10000 23 | if not datafile.endswith(".xls") or not datafile.endswith(".xlsx"): 24 | with open(datafile, "r") as f: 25 | first_line = f.readline().strip() 26 | num_columns = len(first_line.split(delimiter)) 27 | chunk_size = int(chunk_size*6000/num_columns) 28 | print("Number of Columns:", num_columns, "\nChunk size:", chunk_size) 29 | if delimiter: 30 | reader_object = file_reader(datafile, chunksize=chunk_size, encoding='ISO-8859-1', sep=delimiter) 31 | else: 32 | reader_object = file_reader(datafile, chunksize=chunk_size, encoding='ISO-8859-1') 33 | chunk_list = [] 34 | counter = 1 35 | for chunk in reader_object: 36 | print("Loading chunk {}...".format(counter)) 37 | counter += 1 38 | chunk_list.append(chunk) 39 | dataFrame = pd.concat(chunk_list, ignore_index=True) 40 | del chunk, chunk_list 41 | else: 42 | if delimiter: 43 | dataFrame = file_reader(datafile, encoding='ISO-8859-1', sep=delimiter) 44 | else: 45 | dataFrame = file_reader(datafile, encoding='ISO-8859-1') 46 | 47 | return dataFrame 48 | 49 | def find_icd10_ix_range(dataFrame, start_loc, end_loc): 50 | 51 | if start_loc not in dataFrame.index.tolist(): 52 | need_start = True 53 | else: 54 | need_start = False 55 | 56 | if end_loc not in dataFrame.index.tolist(): 57 | need_end = True 58 | else: 59 | need_end = False 60 | 61 | if need_start or need_end: 62 | for ix in dataFrame.index.tolist(): 63 | if ix.startswith(start_loc) and ix[1].isdigit() and need_start: 64 | start_loc = ix 65 | need_start = False 66 | if ix.startswith(end_loc) and ix[1].isdigit() and need_end: 67 | end_loc = ix 68 | need_end = False 69 | if not need_start and not need_end: 70 | break 71 | 72 | return start_loc, end_loc 73 | 74 | def find_icd10_letter_ixs(dataFrame, letter): 75 | ixs = [] 76 | 77 | for ix in dataFrame.index.tolist(): 78 | if ix.startswith(letter) and ix[1].isdigit(): 79 | ixs.append(ix) 80 | 81 | return ixs 82 | 83 | def parse_cat_tree(category, cattree): 84 | class Namespace(object): 85 | pass 86 | ns = Namespace() 87 | ns.results = [] 88 | 89 | def inner(data): 90 | if isinstance(data, dict): 91 | for k, v in data.items(): 92 | if k == "datafields": 93 | ns.results += v 94 | else: 95 | for item in v: 96 | if item not in cattree.keys(): 97 | print("Warning: Category {} not found in mapped tree".format(item)) 98 | else: 99 | inner(cattree[item]) 100 | 101 | if category not in cattree.keys(): 102 | print("Warning: Category {} not found in mapped tree".format(category)) 103 | else: 104 | inner(cattree[category]) 105 | 106 | return ns.results 107 | 108 | def create_long_bois(column_names, ref_df): 109 | 110 | name_key = {} 111 | 112 | for cn in column_names: 113 | try: 114 | datafield = int(cn.split("-")[0]) 115 | if datafield in ref_df.index.tolist(): 116 | name_key[cn] = ref_df.loc[datafield, "title"].replace(" ", "_") + "-" + cn.split("-")[1] 117 | except ValueError: 118 | pass 119 | 120 | return name_key 121 | -------------------------------------------------------------------------------- /ukbb_parser/tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/USC-IGC/ukbb_parser/0aafa84bd427253c202ce8a03d9a3d3abd898501/ukbb_parser/tests/__init__.py -------------------------------------------------------------------------------- /ukbb_parser/tests/test_installation.py: -------------------------------------------------------------------------------- 1 | def main(): 2 | try: 3 | import ukbb_parser.scripts.demo_conversion as dc 4 | print("Looking good so far") 5 | except ImportError: 6 | from ukbb_parser.ukbb_parser import scripts 7 | print(scripts.__file__) 8 | 9 | if __name__ == "__main__": 10 | main() 11 | --------------------------------------------------------------------------------