├── .DS_Store ├── .github └── workflows │ └── clinical_metadata.yml ├── .gitignore ├── LICENSE ├── README.md ├── current-release.md ├── enterprise ├── README.md ├── model_metadata.csv └── model_metrics.csv ├── metadata ├── .DS_Store ├── jsl_metadata.json └── pipeline │ ├── .DS_Store │ ├── release.sh │ └── upload_file_to_s3.py ├── python ├── 0 Merge Class Datasets.ipynb ├── 1 Model Hub Descriptions Read.ipynb ├── 2 Dataset Merging Internal.ipynb ├── 2 Dataset Merging.ipynb ├── 3 Model Hub Descriptions.ipynb ├── 4 Automated Code and ex generation.ipynb └── docs_module │ ├── __init__.py │ ├── langs.py │ ├── metadata │ ├── all_models_metadata.csv │ ├── class_metadata.csv │ ├── class_metadata_all.csv │ ├── class_metadata_licensed.csv │ ├── full_models_metadata.csv │ ├── model_metadata.csv │ ├── model_metadata_existing.csv │ ├── model_metadata_licensed.csv │ ├── models_metadata_all.csv │ ├── models_metadata_all_bu.csv │ └── params_metadata_licensed.csv │ ├── output │ ├── 2019-04-30-pos_clinical_en.md │ ├── 2019-06-04-deidentify_rb_en.md │ ├── 2020-01-22-onto_100_en.md │ ├── 2020-01-22-onto_300_en.md │ ├── 2020-01-22-wikiner_840B_300_de.md │ ├── 2020-01-22-wikiner_840B_300_fr.md │ ├── 2020-01-22-wikiner_840B_300_it.md │ ├── 2020-01-28-assertion_dl_en.md │ ├── 2020-01-28-assertion_ml_en.md │ ├── 2020-01-28-embeddings_clinical_en.md │ ├── 2020-01-28-ner_bionlp_en.md │ ├── 2020-01-28-ner_clinical_en.md │ ├── 2020-02-17-wikiner_6B_100_es.md │ ├── 2020-02-17-wikiner_6B_300_es.md │ ├── 2020-02-17-wikiner_840B_300_es.md │ ├── 2020-03-12-wikiner_6B_100_ru.md │ ├── 2020-03-12-wikiner_6B_300_ru.md │ ├── 2020-03-12-wikiner_840B_300_ru.md │ ├── 2020-03-17-ner_diseases_en.md │ ├── 2020-03-17-ner_drugs_en.md │ ├── 2020-03-17-ner_posology_en.md │ ├── 2020-03-19-ner_dl_en.md │ ├── 2020-03-26-embeddings_healthcare_en.md │ ├── 2020-04-17-spellcheck_clinical_en.md │ ├── 2020-04-21-chunkresolve_cpt_clinical_en.md │ ├── 2020-04-21-chunkresolve_icd10cm_clinical_en.md │ ├── 2020-04-21-chunkresolve_icd10pcs_clinical_en.md │ ├── 2020-04-21-chunkresolve_icdo_clinical_en.md │ ├── 2020-04-21-ner_anatomy_en.md │ ├── 2020-04-21-ner_cellular_en.md │ ├── 2020-04-21-ner_jsl_en.md │ ├── 2020-04-21-ner_jsl_enriched_en.md │ ├── 2020-04-21-ner_posology_large_en.md │ ├── 2020-04-21-ner_posology_small_en.md │ ├── 2020-04-21-ner_risk_factors_en.md │ ├── 2020-04-22-ner_cancer_genetics_en.md │ ├── 2020-04-28-chunkresolve_icd10cm_diseases_clinical_en.md │ ├── 2020-04-28-chunkresolve_icd10cm_injuries_clinical_en.md │ ├── 2020-04-28-chunkresolve_icd10cm_musculoskeletal_clinical_en.md │ ├── 2020-04-28-chunkresolve_icd10cm_neoplasms_clinical_en.md │ ├── 2020-04-28-chunkresolve_icd10cm_poison_ext_clinical_en.md │ ├── 2020-04-28-chunkresolve_icd10cm_puerile_clinical_en.md │ ├── 2020-05-03-wikiner_6B_100_nl.md │ ├── 2020-05-03-wikiner_6B_100_pl.md │ ├── 2020-05-03-wikiner_6B_100_pt.md │ ├── 2020-05-03-wikiner_6B_300_nl.md │ ├── 2020-05-03-wikiner_6B_300_pl.md │ ├── 2020-05-03-wikiner_6B_300_pt.md │ ├── 2020-05-03-wikiner_840B_300_nl.md │ ├── 2020-05-03-wikiner_840B_300_pl.md │ ├── 2020-05-03-wikiner_840B_300_pt.md │ ├── 2020-05-06-norne_6B_100_no.md │ ├── 2020-05-06-norne_6B_300_no.md │ ├── 2020-05-06-norne_840B_300_no.md │ ├── 2020-05-07-assertion_i2b2_en.md │ ├── 2020-05-16-chunkresolve_loinc_clinical_en.md │ ├── 2020-05-19-deidentify_rb_no_regex_en.md │ ├── 2020-05-21-assertion_dl_large_en.md │ ├── 2020-05-21-ner_clinical_large_en.md │ ├── 2020-05-21-ner_large_clinical_en.md │ ├── 2020-05-26-embeddings_scielo_150d_es.md │ ├── 2020-05-26-embeddings_scielo_300d_es.md │ ├── 2020-05-26-embeddings_scielo_50d_es.md │ ├── 2020-05-26-embeddings_scielowiki_150d_es.md │ ├── 2020-05-26-embeddings_scielowiki_300d_es.md │ ├── 2020-05-26-embeddings_scielowiki_50d_es.md │ ├── 2020-05-27-embeddings_sciwiki_150d_es.md │ ├── 2020-05-27-embeddings_sciwiki_300d_es.md │ ├── 2020-05-27-embeddings_sciwiki_50d_es.md │ ├── 2020-05-29-embeddings_healthcare_100d_en.md │ ├── 2020-06-02-embeddings_biovec_en.md │ ├── 2020-06-20-chunkresolve_snomed_findings_clinical_en.md │ ├── 2020-06-24-chunkresolve_rxnorm_xsmall_clinical_en.md │ ├── 2020-07-08-ner_deid_enriched_en.md │ ├── 2020-07-08-ner_diag_proc_es.md │ ├── 2020-07-08-ner_neoplasms_es.md │ ├── 2020-07-19-deidentify_large_en.md │ ├── 2020-07-22-ner_deid_large_en.md │ ├── 2020-07-27-chunkresolve_rxnorm_cd_clinical_en.md │ ├── 2020-07-27-chunkresolve_rxnorm_sbd_clinical_en.md │ ├── 2020-07-27-chunkresolve_rxnorm_scd_clinical_en.md │ ├── 2020-08-18-ner_events_clinical_en.md │ ├── 2020-08-18-re_temporal_events_clinical_en.md │ ├── 2020-08-18-re_temporal_events_enriched_clinical_en.md │ ├── 2020-08-19-explain_clinical_doc_carp_en.md │ ├── 2020-08-19-explain_clinical_doc_cra_en.md │ ├── 2020-08-19-explain_clinical_doc_era_en.md │ ├── 2020-08-27-ner_human_phenotype_gene_clinical_en.md │ ├── 2020-08-27-ner_human_phenotype_go_clinical_en.md │ ├── 2020-08-27-re_human_phenotype_gene_clinical_en.md │ ├── 2020-08-30-dane_ner_6B_100_da.md │ ├── 2020-08-30-dane_ner_6B_300_da.md │ ├── 2020-08-30-dane_ner_840B_100_da.md │ ├── 2020-08-30-swedish_ner_6B_100_sv.md │ ├── 2020-08-30-swedish_ner_6B_300_sv.md │ ├── 2020-08-30-swedish_ner_840B_300_sv.md │ ├── 2020-08-30-wikiner_6B_100_fi.md │ ├── 2020-08-30-wikiner_6B_300_fi.md │ ├── 2020-08-30-wikiner_840B_300_fi.md │ ├── 2020-09-03-re_drug_drug_interaction_clinical_en.md │ ├── 2020-09-06-chunkresolve_ICD10GM_de.md │ ├── 2020-09-06-ner_healthcare_de.md │ ├── 2020-09-06-w2v_cc_300d_de.md │ ├── 2020-09-07-ner_legal_de.md │ ├── 2020-09-08-ner_dl_bert_en.md │ ├── 2020-09-16-chunkresolve_athena_conditions_healthcare_en.md │ ├── 2020-09-23-assertion_dl_healthcare_en.md │ ├── 2020-09-24-re_clinical_en.md │ ├── 2020-10-06-ner_ade_biobert_en.md │ ├── 2020-10-06-ner_ade_clinical_en.md │ ├── 2020-10-06-ner_ade_clinicalbert_en.md │ └── 2020-10-06-ner_ade_healthcare_en.md │ └── templates │ └── model.md ├── release-template.md ├── training ├── README.md ├── lemmatizer │ └── README.md ├── ner_dl │ └── README.md └── part_of_speech │ └── README.md └── utils └── ModelsOperations.ipynb /.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/JohnSnowLabs/spark-nlp-models/377d53ad61700f131f456373b637527a71ace0bf/.DS_Store -------------------------------------------------------------------------------- /.github/workflows/clinical_metadata.yml: -------------------------------------------------------------------------------- 1 | name: metadataSync 2 | on: 3 | push: 4 | branches: 5 | - master 6 | 7 | jobs: 8 | build: 9 | runs-on: ubuntu-latest 10 | steps: 11 | - uses: actions/checkout@v1 12 | - name: Release to S3 13 | run: bash metadata/pipeline/release.sh ${{secrets.AWS_Bucket_Name}} "clinical/models/metadata.json" ${{secrets.AWS_Access_Key}} ${{secrets.AWS_Access_Secret}} "metadata/jsl_metadata.json" 14 | if: "contains(github.event.head_commit.message, '[metadata]')" 15 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.class 2 | *.log 3 | 4 | # Byte-compiled / optimized / DLL files 5 | __pycache__/ 6 | *.py[cod] 7 | *$py.class 8 | 9 | # C extensions 10 | *.so 11 | 12 | # Distribution / packaging 13 | .Python 14 | build/ 15 | develop-eggs/ 16 | dist/ 17 | downloads/ 18 | eggs/ 19 | .eggs/ 20 | lib64/ 21 | parts/ 22 | sdist/ 23 | var/ 24 | wheels/ 25 | pip-wheel-metadata/ 26 | share/python-wheels/ 27 | *.egg-info/ 28 | .installed.cfg 29 | *.egg 30 | MANIFEST 31 | 32 | # PyInstaller 33 | # Usually these files are written by a python script from a template 34 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 35 | *.manifest 36 | *.spec 37 | 38 | # Installer logs 39 | pip-log.txt 40 | pip-delete-this-directory.txt 41 | 42 | # Unit test / coverage reports 43 | htmlcov/ 44 | .tox/ 45 | .nox/ 46 | .coverage 47 | .coverage.* 48 | .cache 49 | nosetests.xml 50 | coverage.xml 51 | *.cover 52 | .hypothesis/ 53 | .pytest_cache/ 54 | 55 | # Translations 56 | *.mo 57 | *.pot 58 | 59 | # Django stuff: 60 | *.log 61 | local_settings.py 62 | db.sqlite3 63 | db.sqlite3-journal 64 | 65 | # Flask stuff: 66 | instance/ 67 | .webassets-cache 68 | 69 | # Scrapy stuff: 70 | .scrapy 71 | 72 | # Sphinx documentation 73 | docs/_build/ 74 | 75 | # PyBuilder 76 | target/ 77 | 78 | # Jupyter Notebook 79 | .ipynb_checkpoints 80 | 81 | # IPython 82 | profile_default/ 83 | ipython_config.py 84 | 85 | # pyenv 86 | .python-version 87 | 88 | # pipenv 89 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 90 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 91 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 92 | # install all needed dependencies. 93 | #Pipfile.lock 94 | 95 | # celery beat schedule file 96 | celerybeat-schedule 97 | 98 | # SageMath parsed files 99 | *.sage.py 100 | 101 | # Environments 102 | .env 103 | .venv 104 | env/ 105 | venv/ 106 | ENV/ 107 | env.bak/ 108 | venv.bak/ 109 | 110 | # Spyder project settings 111 | .spyderproject 112 | .spyproject 113 | 114 | # Rope project settings 115 | .ropeproject 116 | 117 | # mkdocs documentation 118 | /site 119 | 120 | # mypy 121 | .mypy_cache/ 122 | .dmypy.json 123 | dmypy.json 124 | 125 | # Pyre type checker 126 | .pyre/ 127 | 128 | 129 | 130 | # Spark NLP 131 | **/embeddings_index-*/** 132 | glove.* 133 | *.hdf 134 | .idea 135 | **/graphs_log/** 136 | **/wikiner/** 137 | 138 | 139 | # docs 140 | _site/ 141 | .sass-cache/ 142 | .jekyll-cache/ 143 | .jekyll-metadata 144 | 145 | 146 | .DS_Store -------------------------------------------------------------------------------- /current-release.md: -------------------------------------------------------------------------------- 1 | ## Model or model pack description: 2 | 3 | ### BioBERT models pack: 4 | 5 | We are very excited to share these 5 new BioBERT models with our enterprise users! 6 | 7 | | Model | name | language | loc | 8 | |----------------------------------------|---------------|---------------|---------------| 9 | |BertEmbeddingsModel | `biobert_pubmed_cased`|en|clinical/models| 10 | |BertEmbeddingsModel | `biobert_pmc_cased`|en|clinical/models| 11 | |BertEmbeddingsModel | `biobert_pubmed_pmc_cased`|en|clinical/models| 12 | |BertEmbeddingsModel | `biobert_clinical_cased`|en|clinical/models| 13 | |BertEmbeddingsModel | `biobert_discharge_cased`|en|clinical/models| 14 | 15 | The first 3 models `biobert_pubmed_cased`, `biobert_pmc_cased`, and `biobert_pubmed_pmc_cased` are thanks to [BioBERT](https://github.com/naver/biobert-pretrained) pretrained models from their paper: https://arxiv.org/abs/1901.08746 16 | And the last two models `biobert_clinical_cased` and `biobert_discharge_cased` are from another amazing release called [clinicalBERT](https://github.com/EmilyAlsentzer/clinicalBERT) from their paper: https://www.aclweb.org/anthology/W19-1909/ 17 | 18 | #### Spark NLP Version: 19 | - [x] HEALTHCARE 20 | - [ ] PUBLIC 21 | 22 | ### Last update 23 | -- DATE 24 | ### Last update 25 | -- NOTES 26 | ### WORKS WITH: 27 | -- 2.3.x and above 28 | ### Link 29 | -- to workshop example 30 | -------------------------------------------------------------------------------- /enterprise/model_metrics.csv: -------------------------------------------------------------------------------- 1 | "Name","Tag","Tp","Fp","Fn","Precision","Recall","F1" 2 | "ner_clinical","I-TREATMENT",3390,278,419,0.92420936,0.88999736,0.90678084 3 | "ner_clinical","I-PROBLEM",7890,749,656,0.91330016,0.92323893,0.9182427 4 | "ner_clinical","B-PROBLEM",5492,453,421,0.92380154,0.92880094,0.9262945 5 | "ner_clinical","I-TEST",3355,215,315,0.9397759,0.91416895,0.9267956 6 | "ner_clinical","B-TEST",3378,219,299,0.93911594,0.9186837,0.92878747 7 | "ner_clinical","B-TREATMENT",3812,314,378,0.92389727,0.9097852,0.9167869 8 | -------------------------------------------------------------------------------- /metadata/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/JohnSnowLabs/spark-nlp-models/377d53ad61700f131f456373b637527a71ace0bf/metadata/.DS_Store -------------------------------------------------------------------------------- /metadata/pipeline/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/JohnSnowLabs/spark-nlp-models/377d53ad61700f131f456373b637527a71ace0bf/metadata/pipeline/.DS_Store -------------------------------------------------------------------------------- /metadata/pipeline/release.sh: -------------------------------------------------------------------------------- 1 | bucket_name=$1 2 | clinical_aws_key=$2 3 | aws_access_key=$3 4 | aws_access_secret=$4 5 | clinical_local_path=$5 6 | 7 | # Remove any existing versions of a ZIP 8 | #rm -rf $local_path 9 | 10 | # Create a zip of the current directory. 11 | #zip -r $local_path . -x .git/ .git/*** .github/workflows/release.yml scripts/pipeline/release.sh scripts/pipeline/upload_file_to_s3.py .DS_Store 12 | 13 | #apt-get install python3-setuptools 14 | pip3 install -U pip setuptools 15 | # Install required dependencies for Python script. 16 | pip3 install boto3 17 | #git clone git://github.com/boto/boto.git && cd boto && python3 setup.py install 18 | #cd .. 19 | 20 | # Run upload script 21 | python3 metadata/pipeline/upload_file_to_s3.py $bucket_name $clinical_aws_key $aws_access_key $aws_access_secret $clinical_local_path -------------------------------------------------------------------------------- /metadata/pipeline/upload_file_to_s3.py: -------------------------------------------------------------------------------- 1 | import boto3 2 | import sys 3 | 4 | def main(): 5 | 6 | print (sys.argv) 7 | bucket_name=sys.argv[1] 8 | clinical_aws_key=sys.argv[2] 9 | aws_access_key=sys.argv[3] 10 | aws_access_secret=sys.argv[4] 11 | clinical_local_path=sys.argv[5] 12 | 13 | 14 | ''' 15 | session = boto3.Session( 16 | aws_access_key_id=aws_access_key, 17 | aws_secret_access_key=aws_access_secret, 18 | ) 19 | client = session.client('s3') 20 | 21 | response = client.upload_file( 22 | Filename=local_path, 23 | Bucket=bucket_name, 24 | Key=aws_key 25 | ) 26 | ''' 27 | 28 | s3 = boto3.resource('s3', aws_access_key_id=aws_access_key, aws_secret_access_key=aws_access_secret) 29 | 30 | s3.meta.client.upload_file(clinical_local_path, bucket_name, clinical_aws_key) 31 | print (clinical_aws_key,' done') 32 | 33 | print ('Done uploading') 34 | 35 | 36 | main() -------------------------------------------------------------------------------- /python/0 Merge Class Datasets.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import pandas as pd, boto3, re, os\n", 10 | "pd.set_option(\"display.max_rows\",1000)\n", 11 | "pd.set_option(\"display.max_colwidth\",1000)" 12 | ] 13 | }, 14 | { 15 | "cell_type": "code", 16 | "execution_count": 2, 17 | "metadata": {}, 18 | "outputs": [], 19 | "source": [ 20 | "public_classes = pd.read_csv(\"docs_module/metadata/class_metadata.csv\")\n", 21 | "licensed_classes = pd.read_csv(\"docs_module/metadata/class_metadata_licensed.csv\")" 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": 3, 27 | "metadata": {}, 28 | "outputs": [], 29 | "source": [ 30 | "class_metadata = pd.concat([licensed_classes, public_classes], sort=False).reset_index(drop=True)" 31 | ] 32 | }, 33 | { 34 | "cell_type": "code", 35 | "execution_count": 6, 36 | "metadata": {}, 37 | "outputs": [], 38 | "source": [ 39 | "array_to_text = [\"inputs\",\"output\",\"tags\"]\n", 40 | "for c in array_to_text:\n", 41 | " class_metadata[c] = class_metadata[c].str.replace(\"[\\[\\]'‘]\",\"\")" 42 | ] 43 | }, 44 | { 45 | "cell_type": "code", 46 | "execution_count": 5, 47 | "metadata": {}, 48 | "outputs": [], 49 | "source": [ 50 | "class_metadata.to_csv(\"docs_module/metadata/class_metadata_all.csv\", index=False)" 51 | ] 52 | }, 53 | { 54 | "cell_type": "code", 55 | "execution_count": null, 56 | "metadata": {}, 57 | "outputs": [], 58 | "source": [] 59 | } 60 | ], 61 | "metadata": { 62 | "kernelspec": { 63 | "display_name": "jsl368", 64 | "language": "python", 65 | "name": "jsl368" 66 | }, 67 | "language_info": { 68 | "codemirror_mode": { 69 | "name": "ipython", 70 | "version": 3 71 | }, 72 | "file_extension": ".py", 73 | "mimetype": "text/x-python", 74 | "name": "python", 75 | "nbconvert_exporter": "python", 76 | "pygments_lexer": "ipython3", 77 | "version": "3.6.8" 78 | } 79 | }, 80 | "nbformat": 4, 81 | "nbformat_minor": 2 82 | } 83 | -------------------------------------------------------------------------------- /python/docs_module/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/JohnSnowLabs/spark-nlp-models/377d53ad61700f131f456373b637527a71ace0bf/python/docs_module/__init__.py -------------------------------------------------------------------------------- /python/docs_module/metadata/class_metadata_licensed.csv: -------------------------------------------------------------------------------- 1 | type,approach_class,model_class,class_description,inputs,output,class_license,dataset_schema,class_annotation_sample,tags,class_license,dataset_schema,class_annotation_sample 2 | assertion_dl,AssertionDLApproach,AssertionDLModel,Assertion of Clinical Entities based on Deep Learning,"document, chunk, word_embeddings",assertion,,,,"clinical,assertion,dl",licensed,TODO,TODO 3 | assertion_logreg,AssertionLogRegApproach,AssertionLogRegModel,Assertion of Clinical Entities based on Logistic Regression,"document, chunk, word_embeddings",assertion,,,,"clinical,assertion,ml,logreg",licensed,TODO,TODO 4 | chunk_entity_resolver,ChunkEntityResolverApproach,ChunkEntityResolverModel,Entity Resolution model Based on KNN using Word Embeddings + Word Movers Distance,"token, chunk_embeddings",entity,,,,"clinical,entity,resolution",licensed,TODO,TODO 5 | deidentification,DeIdentification,DeIdentificationModel,Anonymization and DeIdentification model based on outputs from DeId NERs and Replacement Dictionaries,"document, token, chunk",document,,,,"clinical,deidentification,rule based",licensed,TODO,TODO 6 | relation_extraction,RelationExtractionApproach,RelationExtractionModel,Relation Extraction model based on syntactic features using deep learning,"word_embeddings, chunk, pos, dependency",category,,,,"clinical,relation,extraction",licensed,TODO,TODO 7 | -------------------------------------------------------------------------------- /python/docs_module/metadata/model_metadata.csv: -------------------------------------------------------------------------------- 1 | model_name,model_lang,upstream_deps,model_class,model_dataset,labels,reference_url,model_author,model_repo 2 | -------------------------------------------------------------------------------- /python/docs_module/metadata/models_metadata_all.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/JohnSnowLabs/spark-nlp-models/377d53ad61700f131f456373b637527a71ace0bf/python/docs_module/metadata/models_metadata_all.csv -------------------------------------------------------------------------------- /python/docs_module/metadata/params_metadata_licensed.csv: -------------------------------------------------------------------------------- 1 | type,subtype,names,defaults 2 | AssertionDL,Approach,, 3 | AssertionDL,Model,, 4 | AssertionDL,Both,, 5 | AssertionLogReg,Approach,, 6 | AssertionLogReg,Model,, 7 | AssertionLogReg,Both,, 8 | ChunkEntityresolver,Approach,, 9 | ChunkEntityresolver,Model,, 10 | ChunkEntityresolver,Both,, 11 | DeIdentification,Approach,, 12 | DeIdentification,Model,, 13 | DeIdentification,Both,, 14 | RelationExtraction,Approach,, 15 | RelationExtraction,Model,, 16 | RelationExtraction,Both,, 17 | -------------------------------------------------------------------------------- /python/docs_module/output/2019-04-30-pos_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: POS Tagger Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2019-04-30 10 | tags: [clinical,pos,medpost,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | 21 | 22 | {:.btn-box} 23 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/pos_clinical_en_2.0.2_2.4_1556660550177.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
24 | 25 | ## How to use 26 |
27 | 28 | {% include programmingLanguageSelectScalaPython.html %} 29 | 30 | ```python 31 | 32 | ``` 33 | 34 | ```scala 35 | 36 | ``` 37 |
38 | 39 | 40 | 41 | {:.model-param} 42 | ## Model Information 43 | {:.table-model} 44 | |-------------------------|---------------------| 45 | | Model Name | pos_clinical | 46 | | Model Class | PerceptronModel | 47 | | Spark Compatibility | 2.0.2 | 48 | | Spark NLP Compatibility | 2.4 | 49 | | License | Licensed | 50 | | Edition | Official | 51 | | Input Labels | token, sentence | 52 | | Output Labels | pos | 53 | | Language | en | 54 | | Upstream Dependencies | embeddings_clinical | 55 | 56 | 57 | 58 | 59 | 60 | {:.h2_title} 61 | ## Data Source 62 | Trained with MedPost dataset. 63 | 64 | -------------------------------------------------------------------------------- /python/docs_module/output/2019-06-04-deidentify_rb_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Deidentify RB 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2019-06-04 10 | tags: [clinical,deidentify,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Personal Information in order to deidentify 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/deidentify_rb_en_2.0.2_2.4_1559672122511.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|------------------------| 47 | | Model Name | deidentify_rb | 48 | | Model Class | DeIdentificationModel | 49 | | Spark Compatibility | 2.0.2 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, token, chunk | 54 | | Output Labels | document | 55 | | Language | en | 56 | | Upstream Dependencies | ner_deid | 57 | 58 | 59 | 60 | 61 | 62 | {:.h2_title} 63 | ## Data Source 64 | Rule based DeIdentifier based on `ner_deid`. 65 | 66 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-01-22-onto_300_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Onto 300 4 | author: John Snow Labs 5 | name: onto_300 6 | class: NerDLModel 7 | language: en 8 | repository: public/models 9 | date: 22/01/2020 10 | tags: [ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Onto is a Named Entity Recognition (or NER) model, meaning it annotates text to find features like the names of people, places, and organizations. Onto was trained on the OntoNotes text corpus. This NER model does not read words directly but instead reads word embeddings, which represent words as points such that more semantically similar words are closer together. Onto 300 is trained with GloVe 840B 300 word embeddings, so be sure to use the same embeddings in the pipeline. 19 | 20 | 21 | 22 | {:.btn-box} 23 | [Live Demo](https://demo.johnsnowlabs.com/public/NER_EN_18){:.button.button-orange}
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/streamlit_notebooks/NER_EN.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/models/onto_300_en_2.1.0_2.4_1579729071854.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
24 | 25 | ## How to use 26 |
27 | 28 | {% include programmingLanguageSelectScalaPython.html %} 29 | 30 | ```python 31 | 32 | ``` 33 | 34 | ```scala 35 | 36 | ``` 37 |
38 | 39 | {:.h2_title} 40 | ## Results 41 | ```bash 42 | 43 | ``` 44 | 45 | {:.model-param} 46 | ## Model Information 47 | 48 | {:.table-model} 49 | |-------------------------|---------------------------| 50 | | Model Name | onto_300 | 51 | | Model Class | NerDLModel | 52 | | Spark Compatibility | 2.1.0 | 53 | | Spark NLP Compatibility | 2.4 | 54 | | License | open source | 55 | | Edition | public | 56 | | Input Labels | | 57 | | Output Labels | | 58 | | Language | en | 59 | | Dimension | | 60 | | Case Sensitive | 0.0 | 61 | | Upstream Dependencies | OntoNotes with GloVe 300d | 62 | 63 | 64 | 65 | 66 | {:.h2_title} 67 | ## Data Source 68 | 69 | The model is trained based on data from[https://catalog.ldc.upenn.edu/LDC2013T19](https://catalog.ldc.upenn.edu/LDC2013T19) 70 | 71 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-01-28-assertion_dl_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Assertion DL Clinical Embeddings 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-01-28 10 | tags: [clinical,assertion] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Deep learning named entity recognition model for assertions. 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | hypothetical, present, absent, possible, conditional, associated_with_someone_else 23 | 24 | {:.btn-box} 25 |
[Open in Colab](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/2.Clinical_Assertion_Model.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/assertion_dl_en_2.4.0_2.4_1580237286004.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|----------------------------------| 47 | | Model Name | assertion_dl | 48 | | Model Class | AssertionDLModel | 49 | | Spark Compatibility | 2.4.0 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, chunk, word_embeddings | 54 | | Output Labels | assertion | 55 | | Language | en | 56 | | Case Sensitive | False | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text with `embeddings_clinical`. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-01-28-assertion_ml_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Assertion ML 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-01-28 10 | tags: [clinical,assertion] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Deep learning named entity recognition model for assertions. 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | hypothetical, present, absent, possible, conditional, associated_with_someone_else 23 | 24 | {:.btn-box} 25 |
[Open in Colab](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/2.Clinical_Assertion_Model.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/assertion_ml_en_2.4.0_2.4_1580237286004.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|----------------------------------| 47 | | Model Name | assertion_ml | 48 | | Model Class | AssertionLogRegModel | 49 | | Spark Compatibility | 2.4.0 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, chunk, word_embeddings | 54 | | Output Labels | assertion | 55 | | Language | en | 56 | | Case Sensitive | False | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text with `embeddings_clinical`. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-01-28-embeddings_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Embeddings Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-01-28 10 | tags: [clinical,embeddings,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Word2Vec feature vectors based on embeddings_clinical 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/embeddings_clinical_en_2.4.0_2.4_1580237286004.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|---------------------| 47 | | Model Name | embeddings_clinical | 48 | | Model Class | WordEmbeddingsModel | 49 | | Spark Compatibility | 2.4.0 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, token | 54 | | Output Labels | word_embeddings | 55 | | Language | en | 56 | | Dimension | 200.0 | 57 | 58 | 59 | 60 | 61 | 62 | {:.h2_title} 63 | ## Data Source 64 | Trained on PubMed corpora. 65 | 66 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-01-28-ner_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Ner DL Model Clinical 4 | author: John Snow Labs 5 | name: ner_clinical 6 | class: NerDLModel 7 | language: en 8 | repository: clinical/models 9 | date: 28/01/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Pretrained named entity recognition deep learning model for clinical terms. 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Problem, Test, Treatment 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/NER_DIAG_PROC/){:.button.button-orange}
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/1.Clinical_Named_Entity_Recognition_Model.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_clinical_en_2.4.0_2.4_1580237286004.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|--------------------------| 52 | | Model Name | ner_clinical | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.4.0 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | Problem, Test, Treatment | 60 | | Language | en | 61 | | Dimension | | 62 | | Case Sensitive | 0.0 | 63 | | Upstream Dependencies | embeddings_clinical | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | Trained on 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text with `embeddings_clinical` 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-03-17-ner_diseases_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Ner DL Model Diseases 4 | author: John Snow Labs 5 | name: ner_diseases 6 | class: NerDLModel 7 | language: en 8 | repository: clinical/models 9 | date: 17/03/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Pretrained named entity recognition deep learning model for diseases. 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Disease 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/NER_DIAG_PROC/){:.button.button-orange}
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/1.Clinical_Named_Entity_Recognition_Model.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_diseases_en_2.4.4_2.4_1584452534235.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|---------------------| 52 | | Model Name | ner_diseases | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.4.4 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | Disease | 60 | | Language | en | 61 | | Dimension | | 62 | | Case Sensitive | 0.0 | 63 | | Upstream Dependencies | embeddings_clinical | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | Trained on i2b2 with `embeddings_clinical` 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-03-17-ner_drugs_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Ner DL Model Drugs 4 | author: John Snow Labs 5 | name: ner_drugs 6 | class: NerDLModel 7 | language: en 8 | repository: clinical/models 9 | date: 17/03/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Pretrained named entity recognition deep learning model for Drugs. 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | DrugChem (Drug and Chemicals) 23 | 24 | {:.btn-box} 25 |
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/1.Clinical_Named_Entity_Recognition_Model.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_drugs_en_2.4.4_2.4_1584452534235.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|-------------------------------| 52 | | Model Name | ner_drugs | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.4.4 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | DrugChem (Drug and Chemicals) | 60 | | Language | en | 61 | | Dimension | | 62 | | Case Sensitive | 0.0 | 63 | | Upstream Dependencies | embeddings_clinical | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | Trained on i2b2_med7 + FDA with `embeddings_clinical`. 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-03-17-ner_posology_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Ner DL Model Posology 4 | author: John Snow Labs 5 | name: ner_posology 6 | class: NerDLModel 7 | language: en 8 | repository: clinical/models 9 | date: 17/03/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Pretrained named entity recognition deep learning model for posology, this NER is trained with the 'embeddings_clinical' word embeddings model, so be sure to use the same embeddings in the pipeline 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | DOSAGE,DRUG,DURATION,FORM,FREQUENCY,ROUTE,STRENGTH 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/NER_POSOLOGY/){:.button.button-orange}
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/1.Clinical_Named_Entity_Recognition_Model.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_posology_en_2.4.4_2.4_1584452534235.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|----------------------------------------------------| 52 | | Model Name | ner_posology | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.4.2 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | DOSAGE,DRUG,DURATION,FORM,FREQUENCY,ROUTE,STRENGTH | 60 | | Language | en | 61 | | Dimension | | 62 | | Case Sensitive | 0.0 | 63 | | Upstream Dependencies | embeddings_clinical | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | Trained on the 2018 i2b2 dataset and FDA Drug datasets with `embeddings_clinical`. 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-03-19-ner_dl_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: 4 | author: John Snow Labs 5 | name: ner_dl 6 | class: NerDLModel 7 | language: en 8 | repository: public/models 9 | date: 19/03/2020 10 | tags: [ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | 21 | 22 | {:.btn-box} 23 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/models/ner_dl_en_2.0.2_2.4_1584624950746.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
24 | 25 | ## How to use 26 |
27 | 28 | {% include programmingLanguageSelectScalaPython.html %} 29 | 30 | ```python 31 | 32 | embeddings = WordEmbeddingsModel.pretrained(glove_100d, lang=en) 33 | embeddings.setInputCols(["sentence", 'token']) 34 | embeddings.setOutputCol("embeddings") 35 | 36 | ner = NerDLModel.pretrained(ner_dl, lang=en) 37 | ner.setInputCols(["sentence", "token", "embeddings"]) 38 | ner.setOutputCol("ner") 39 | 40 | ner_converter = NerConverter() 41 | ner_converter.setInputCols(["sentence", "token", "ner"]) 42 | ner_converter.setOutputCol("ner_chunk") 43 | 44 | pipeline = Pipeline(stages=[ documentAssembler, 45 | sentenceDetector, 46 | tokenizer, 47 | embeddings, 48 | ner, 49 | ner_converter 50 | ]) 51 | 52 | pipeline_model = pipeline.fit(spark.createDataFrame([['']]).toDF("text")) 53 | lmodel = LightPipeline(pipeline_model) 54 | 55 | result = lmodel.fullAnnotate("The Mona Lisa is a 16th century oil painting created by Leonardo. It's held at the Louvre in Paris.")[0] 56 | 57 | ``` 58 | 59 | ```scala 60 | 61 | ``` 62 |
63 | 64 | {:.h2_title} 65 | ## Results 66 | ```bash 67 | +-------------+---------+-------+----------+ 68 | | ner_chunk | begin | end | entity | 69 | +=============+=========+=======+==========+ 70 | | Mona Lisa | 4 | 12 | PER | 71 | +-------------+---------+-------+----------+ 72 | | Leonardo | 56 | 63 | PER | 73 | +-------------+---------+-------+----------+ 74 | | Louvre | 83 | 88 | LOC | 75 | +-------------+---------+-------+----------+ 76 | | Paris | 93 | 97 | LOC | 77 | +-------------+---------+-------+----------+ 78 | ``` 79 | 80 | {:.model-param} 81 | ## Model Information 82 | 83 | {:.table-model} 84 | |-------------------------|----------------| 85 | | Model Name | ner_dl | 86 | | Model Class | NerDLModel | 87 | | Spark Compatibility | 2.0.2 | 88 | | Spark NLP Compatibility | 2.4 | 89 | | License | open source | 90 | | Edition | public | 91 | | Input Labels | | 92 | | Output Labels | | 93 | | Language | en | 94 | | Dimension | | 95 | | Case Sensitive | | 96 | | Upstream Dependencies | NER with GloVe | 97 | 98 | 99 | 100 | 101 | {:.h2_title} 102 | ## Data Source 103 | 104 | 105 | 106 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-03-26-embeddings_healthcare_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Embeddings Healthcare 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-03-26 10 | tags: [clinical,embeddings,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Word2Vec feature vectors based on embeddings_healthcare 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/embeddings_healthcare_en_2.4.4_2.4_1585188313964.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|-----------------------| 47 | | Model Name | embeddings_healthcare | 48 | | Model Class | WordEmbeddingsModel | 49 | | Spark Compatibility | 2.4.4 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, token | 54 | | Output Labels | word_embeddings | 55 | | Language | en | 56 | | Dimension | 400.0 | 57 | 58 | 59 | 60 | 61 | 62 | {:.h2_title} 63 | ## Data Source 64 | Trained on PubMed + ICD10 + UMLS + MIMIC III corpora. 65 | 66 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-04-17-spellcheck_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Contextual Spellchecker Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-04-17 10 | tags: [clinical,spellcheck,dl,contextual,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | 21 | 22 | {:.btn-box} 23 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/CONTEXTUAL_SPELL_CHECKER/){:.button.button-orange}
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/6.Clinical_Context_Spell_Checker.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/spellcheck_clinical_en_2.4.2_2.4_1587146727460.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
24 | 25 | ## How to use 26 |
27 | 28 | {% include programmingLanguageSelectScalaPython.html %} 29 | 30 | ```python 31 | 32 | ``` 33 | 34 | ```scala 35 | 36 | ``` 37 |
38 | 39 | 40 | 41 | {:.model-param} 42 | ## Model Information 43 | {:.table-model} 44 | |-------------------------|--------------------------| 45 | | Model Name | spellcheck_clinical | 46 | | Model Class | ContextSpellCheckerModel | 47 | | Spark Compatibility | 2.4.2 | 48 | | Spark NLP Compatibility | 2.4 | 49 | | License | Licensed | 50 | | Edition | Official | 51 | | Input Labels | token | 52 | | Output Labels | spell | 53 | | Language | en | 54 | | Upstream Dependencies | embeddings_clinical | 55 | 56 | 57 | 58 | 59 | 60 | {:.h2_title} 61 | ## Data Source 62 | Trained with PubMed and i2b2 datasets. 63 | 64 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-04-21-chunkresolve_cpt_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: ChunkResolver Cpt Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-04-21 10 | tags: [clinical,entity_resolution,cpt,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | chunkresolve_cpt_clinical Codes and their normalized definition 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/chunkresolve_cpt_clinical_en_2.4.5_2.4_1587491373378.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|---------------------------| 47 | | Model Name | chunkresolve_cpt_clinical | 48 | | Model Class | ChunkEntityResolverModel | 49 | | Spark Compatibility | 2.4.2 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | token, chunk_embeddings | 54 | | Output Labels | entity | 55 | | Language | en | 56 | | Case Sensitive | True | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on Current Procedural Terminology dataset. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-04-21-chunkresolve_icd10cm_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: ChunkResolver Icd10cm Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-04-21 10 | tags: [clinical,entity_resolution,icd10,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | ICD10-CM Codes and their normalized definition with `clinical_embeddings` 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/ER_ICD10_CM/){:.button.button-orange}
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/enterprise/healthcare/EntityResolution_ICD10_RxNorm_Detailed.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/chunkresolve_icd10cm_clinical_en_2.4.5_2.4_1587491222166.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|-------------------------------| 47 | | Model Name | chunkresolve_icd10cm_clinical | 48 | | Model Class | ChunkEntityResolverModel | 49 | | Spark Compatibility | 2.4.2 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | token, chunk_embeddings | 54 | | Output Labels | entity | 55 | | Language | en | 56 | | Case Sensitive | True | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on ICD10 Clinical Modification datasetwith tenths of variations per code. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-04-21-chunkresolve_icd10pcs_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: ChunkResolver Icd10pcs Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-04-21 10 | tags: [clinical,entity_resolution,icd10,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | ICD10-PCS Codes and their normalized definition with `clinical_embeddings` 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/chunkresolve_icd10pcs_clinical_en_2.4.5_2.4_1587491320087.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|--------------------------------| 47 | | Model Name | chunkresolve_icd10pcs_clinical | 48 | | Model Class | ChunkEntityResolverModel | 49 | | Spark Compatibility | 2.4.2 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | token, chunk_embeddings | 54 | | Output Labels | entity | 55 | | Language | en | 56 | | Case Sensitive | True | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on ICD10 Procedure Coding System dataset. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-04-21-chunkresolve_icdo_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: ChunkResolver Icdo Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-04-21 10 | tags: [clinical,entity_resolution,icd10,icdo,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | ICD-O Codes and their normalized definition with `clinical_embeddings` 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/ER_ICDO/){:.button.button-orange}

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/chunkresolve_icdo_clinical_en_2.4.5_2.4_1587491354644.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|----------------------------| 47 | | Model Name | chunkresolve_icdo_clinical | 48 | | Model Class | ChunkEntityResolverModel | 49 | | Spark Compatibility | 2.4.2 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | token, chunk_embeddings | 54 | | Output Labels | entity | 55 | | Language | en | 56 | | Case Sensitive | True | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on ICD-O Histology Behaviour dataset. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-04-21-ner_cellular_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Ner DL Model Cellular 4 | author: John Snow Labs 5 | name: ner_cellular 6 | class: NerDLModel 7 | language: en 8 | repository: clinical/models 9 | date: 21/04/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Pretrained named entity recognition deep learning model for molecular biology related terms. 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | DNA,RNA,cell_line,cell_type,protein 23 | 24 | {:.btn-box} 25 |
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/1.Clinical_Named_Entity_Recognition_Model.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_cellular_en_2.4.2_2.4_1587513308751.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|-------------------------------------| 52 | | Model Name | ner_cellular | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.4.2 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | DNA,RNA,cell_line,cell_type,protein | 60 | | Language | en | 61 | | Dimension | | 62 | | Case Sensitive | 0.0 | 63 | | Upstream Dependencies | embeddings_clinical | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | Trained on the JNLPBA corpus containing more than 2.404 publication abstracts with `embeddings_clinical` 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-04-21-ner_posology_large_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Ner DL Model Posology Large 4 | author: John Snow Labs 5 | name: ner_posology_large 6 | class: NerDLModel 7 | language: en 8 | repository: clinical/models 9 | date: 21/04/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Pretrained named entity recognition deep learning model for posology, this NER is trained with the 'embeddings_clinical' word embeddings model, so be sure to use the same embeddings in the pipeline 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | DOSAGE,DRUG,DURATION,FORM,FREQUENCY,ROUTE,STRENGTH 23 | 24 | {:.btn-box} 25 |
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/1.Clinical_Named_Entity_Recognition_Model.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_posology_large_en_2.4.2_2.4_1587513302751.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|----------------------------------------------------| 52 | | Model Name | ner_posology_large | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.4.2 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | DOSAGE,DRUG,DURATION,FORM,FREQUENCY,ROUTE,STRENGTH | 60 | | Language | en | 61 | | Dimension | | 62 | | Case Sensitive | 0.0 | 63 | | Upstream Dependencies | embeddings_clinical | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | Trained on the 2018 i2b2 dataset and FDA Drug datasets with `embeddings_clinical`. 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-04-21-ner_posology_small_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Ner DL Model Posology Small 4 | author: John Snow Labs 5 | name: ner_posology_small 6 | class: NerDLModel 7 | language: en 8 | repository: clinical/models 9 | date: 21/04/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Pretrained named entity recognition deep learning model for posology, this NER is trained with the 'embeddings_clinical' word embeddings model, so be sure to use the same embeddings in the pipeline 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | DOSAGE,DRUG,DURATION,FORM,FREQUENCY,ROUTE,STRENGTH 23 | 24 | {:.btn-box} 25 |
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/1.Clinical_Named_Entity_Recognition_Model.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_posology_small_en_2.4.2_2.4_1587513301751.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|----------------------------------------------------| 52 | | Model Name | ner_posology_small | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.4.2 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | DOSAGE,DRUG,DURATION,FORM,FREQUENCY,ROUTE,STRENGTH | 60 | | Language | en | 61 | | Dimension | | 62 | | Case Sensitive | 0.0 | 63 | | Upstream Dependencies | embeddings_clinical | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | Trained on the 2018 i2b2 dataset (no FDA) with `embeddings_clinical`. 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-04-21-ner_risk_factors_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Ner DL Model Risk Factors 4 | author: John Snow Labs 5 | name: ner_risk_factors 6 | class: NerDLModel 7 | language: en 8 | repository: clinical/models 9 | date: 21/04/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Pretrained named entity recognition deep learning model for Heart Disease Risk Factors and Personal Health Information. 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | CAD,DIABETES,FAMILY_HIST,HYPERLIPIDEMIA,HYPERTENSION,MEDICATION,OBESE,PHI,SMOKER 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/NER_RISK_FACTORS/){:.button.button-orange}
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/1.Clinical_Named_Entity_Recognition_Model.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_risk_factors_en_2.4.2_2.4_1587513300751.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|----------------------------------------------------------------------------------| 52 | | Model Name | ner_risk_factors | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.4.2 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | CAD,DIABETES,FAMILY_HIST,HYPERLIPIDEMIA,HYPERTENSION,MEDICATION,OBESE,PHI,SMOKER | 60 | | Language | en | 61 | | Dimension | | 62 | | Case Sensitive | 0.0 | 63 | | Upstream Dependencies | embeddings_clinical | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | Trained on plain n2c2 2014: De-identification and Heart Disease Risk Factors Challenge datasets with `embeddings_clinical` 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-04-22-ner_cancer_genetics_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Ner DL Model Cancer Genetics 4 | author: John Snow Labs 5 | name: ner_cancer_genetics 6 | class: NerDLModel 7 | language: en 8 | repository: clinical/models 9 | date: 22/04/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Pretrained named entity recognition deep learning model for biology and genetics terms. 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | DNA,RNA,cell_line,cell_type,protein 23 | 24 | {:.btn-box} 25 |
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/1.Clinical_Named_Entity_Recognition_Model.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_cancer_genetics_en_2.4.2_2.4_1587567870408.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|-------------------------------------| 52 | | Model Name | ner_cancer_genetics | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.4.2 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | DNA,RNA,cell_line,cell_type,protein | 60 | | Language | en | 61 | | Dimension | | 62 | | Case Sensitive | | 63 | | Upstream Dependencies | embeddings_clinical | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | Trained on Cancer Genetics (CG) task of the BioNLP Shared Task 2013 with `embeddings_clinical` 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-04-28-chunkresolve_icd10cm_diseases_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: ChunkResolver Icd10cm Diseases Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-04-28 10 | tags: [clinical,entity_resolution,icd10,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | ICD10-CM Codes and their normalized definition with `clinical_embeddings` 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/ER_ICD10_CM/){:.button.button-orange}

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/chunkresolve_icd10cm_diseases_clinical_en_2.4.5_2.4_1588105984876.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|----------------------------------------| 47 | | Model Name | chunkresolve_icd10cm_diseases_clinical | 48 | | Model Class | ChunkEntityResolverModel | 49 | | Spark Compatibility | 2.4.5 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | token, chunk_embeddings | 54 | | Output Labels | entity | 55 | | Language | en | 56 | | Case Sensitive | True | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on ICD10CM Dataset Range: A000-N989 Except Neoplasms and Musculoskeletal. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-04-28-chunkresolve_icd10cm_injuries_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: ChunkResolver Icd10cm Injuries Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-04-28 10 | tags: [clinical,entity_resolution,icd10,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | ICD10-CM Codes and their normalized definition with `clinical_embeddings` 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/ER_ICD10_CM/){:.button.button-orange}

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/chunkresolve_icd10cm_injuries_clinical_en_2.4.5_2.4_1588103825347.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|----------------------------------------| 47 | | Model Name | chunkresolve_icd10cm_injuries_clinical | 48 | | Model Class | ChunkEntityResolverModel | 49 | | Spark Compatibility | 2.4.5 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | token, chunk_embeddings | 54 | | Output Labels | entity | 55 | | Language | en | 56 | | Case Sensitive | True | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on ICD10CM Dataset Range: S0000XA-S98929S . 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-04-28-chunkresolve_icd10cm_musculoskeletal_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: ChunkResolver Icd10cm Musculoskeletal Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-04-28 10 | tags: [clinical,entity_resolution,icd10,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | ICD10-CM Codes and their normalized definition with `clinical_embeddings` 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/ER_ICD10_CM/){:.button.button-orange}

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/chunkresolve_icd10cm_musculoskeletal_clinical_en_2.4.5_2.4_1588103998999.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|-----------------------------------------------| 47 | | Model Name | chunkresolve_icd10cm_musculoskeletal_clinical | 48 | | Model Class | ChunkEntityResolverModel | 49 | | Spark Compatibility | 2.4.5 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | token, chunk_embeddings | 54 | | Output Labels | entity | 55 | | Language | en | 56 | | Case Sensitive | True | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on ICD10CM Dataset Range: M0000-M9979XXS. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-04-28-chunkresolve_icd10cm_neoplasms_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: ChunkResolver Icd10cm Neoplasms Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-04-28 10 | tags: [clinical,entity_resolution,icd10,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | ICD10-CM Codes and their normalized definition with `clinical_embeddings` 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/ER_ICD10_CM/){:.button.button-orange}

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/chunkresolve_icd10cm_neoplasms_clinical_en_2.4.5_2.4_1588108205630.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|-----------------------------------------| 47 | | Model Name | chunkresolve_icd10cm_neoplasms_clinical | 48 | | Model Class | ChunkEntityResolverModel | 49 | | Spark Compatibility | 2.4.5 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | token, chunk_embeddings | 54 | | Output Labels | entity | 55 | | Language | en | 56 | | Case Sensitive | True | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on ICD10CM Dataset Ranges: C000-D489, R590-R599. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-04-28-chunkresolve_icd10cm_poison_ext_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: ChunkResolver Icd10cm Poison Ext Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-04-28 10 | tags: [clinical,entity_resolution,icd10, en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | ICD10-CM Codes and their normalized definition with `clinical_embeddings` 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/chunkresolve_icd10cm_poison_ext_clinical_en_2.4.5_2.4_1588106053455.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|------------------------------------------| 47 | | Model Name | chunkresolve_icd10cm_poison_ext_clinical | 48 | | Model Class | ChunkEntityResolverModel | 49 | | Spark Compatibility | 2.4.5 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | token, chunk_embeddings | 54 | | Output Labels | entity | 55 | | Language | en | 56 | | Case Sensitive | True | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on ICD10CM Dataset Range: T1500XA-T879. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-04-28-chunkresolve_icd10cm_puerile_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: ChunkResolver Icd10cm Puerile Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-04-28 10 | tags: [clinical,entity_resolution,icd10,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | ICD10-CM Codes and their normalized definition with `clinical_embeddings` 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/ER_ICD10_CM/){:.button.button-orange}

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/chunkresolve_icd10cm_puerile_clinical_en_2.4.5_2.4_1588103916781.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|---------------------------------------| 47 | | Model Name | chunkresolve_icd10cm_puerile_clinical | 48 | | Model Class | ChunkEntityResolverModel | 49 | | Spark Compatibility | 2.4.5 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | token, chunk_embeddings | 54 | | Output Labels | entity | 55 | | Language | en | 56 | | Case Sensitive | True | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on ICD10CM Dataset Range: O0000-O9989. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-05-06-norne_6B_100_no.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: 4 | author: John Snow Labs 5 | name: norne_6B_100 6 | class: NerDLModel 7 | language: no 8 | repository: public/models 9 | date: 06/05/2020 10 | tags: [ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | 21 | 22 | {:.btn-box} 23 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/models/norne_6B_100_no_2.5.0_2.4_1588781289907.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
24 | 25 | ## How to use 26 |
27 | 28 | {% include programmingLanguageSelectScalaPython.html %} 29 | 30 | ```python 31 | 32 | embeddings = WordEmbeddingsModel.pretrained(glove_100d, lang=en) 33 | embeddings.setInputCols(["sentence", 'token']) 34 | embeddings.setOutputCol("embeddings") 35 | 36 | ner = NerDLModel.pretrained(norne_6B_100, lang=no) 37 | ner.setInputCols(["sentence", "token", "embeddings"]) 38 | ner.setOutputCol("ner") 39 | 40 | ner_converter = NerConverter() 41 | ner_converter.setInputCols(["sentence", "token", "ner"]) 42 | ner_converter.setOutputCol("ner_chunk") 43 | 44 | pipeline = Pipeline(stages=[ documentAssembler, 45 | sentenceDetector, 46 | tokenizer, 47 | embeddings, 48 | ner, 49 | ner_converter 50 | ]) 51 | 52 | pipeline_model = pipeline.fit(spark.createDataFrame([['']]).toDF("text")) 53 | lmodel = LightPipeline(pipeline_model) 54 | 55 | result = lmodel.fullAnnotate("Mona Lisa er et oljemaleri fra 1500-tallet skapt av Leonardo. Den holdes på Louvre i Paris.")[0] 56 | 57 | ``` 58 | 59 | ```scala 60 | 61 | ``` 62 |
63 | 64 | {:.h2_title} 65 | ## Results 66 | ```bash 67 | +-------------+---------+-------+----------+ 68 | | ner_chunk | begin | end | entity | 69 | +=============+=========+=======+==========+ 70 | | Mona Lisa | 0 | 8 | PER | 71 | +-------------+---------+-------+----------+ 72 | | Leonardo | 52 | 59 | PER | 73 | +-------------+---------+-------+----------+ 74 | | Louvre | 76 | 81 | LOC | 75 | +-------------+---------+-------+----------+ 76 | | Paris | 85 | 89 | GPE_LOC | 77 | +-------------+---------+-------+----------+ 78 | ``` 79 | 80 | {:.model-param} 81 | ## Model Information 82 | 83 | {:.table-model} 84 | |-------------------------|--------------| 85 | | Model Name | norne_6B_100 | 86 | | Model Class | NerDLModel | 87 | | Spark Compatibility | 2.5.0 | 88 | | Spark NLP Compatibility | 2.4 | 89 | | License | open source | 90 | | Edition | public | 91 | | Input Labels | | 92 | | Output Labels | | 93 | | Language | no | 94 | | Dimension | | 95 | | Case Sensitive | | 96 | | Upstream Dependencies | glove_100d | 97 | 98 | 99 | 100 | 101 | {:.h2_title} 102 | ## Data Source 103 | 104 | 105 | 106 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-05-06-norne_6B_300_no.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: 4 | author: John Snow Labs 5 | name: norne_6B_300 6 | class: NerDLModel 7 | language: no 8 | repository: public/models 9 | date: 06/05/2020 10 | tags: [ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | 21 | 22 | {:.btn-box} 23 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/models/norne_6B_300_no_2.5.0_2.4_1588781290264.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
24 | 25 | ## How to use 26 |
27 | 28 | {% include programmingLanguageSelectScalaPython.html %} 29 | 30 | ```python 31 | 32 | embeddings = WordEmbeddingsModel.pretrained(glove_6B_300, lang=xx) 33 | embeddings.setInputCols(["sentence", 'token']) 34 | embeddings.setOutputCol("embeddings") 35 | 36 | ner = NerDLModel.pretrained(norne_6B_300, lang=no) 37 | ner.setInputCols(["sentence", "token", "embeddings"]) 38 | ner.setOutputCol("ner") 39 | 40 | ner_converter = NerConverter() 41 | ner_converter.setInputCols(["sentence", "token", "ner"]) 42 | ner_converter.setOutputCol("ner_chunk") 43 | 44 | pipeline = Pipeline(stages=[ documentAssembler, 45 | sentenceDetector, 46 | tokenizer, 47 | embeddings, 48 | ner, 49 | ner_converter 50 | ]) 51 | 52 | pipeline_model = pipeline.fit(spark.createDataFrame([['']]).toDF("text")) 53 | lmodel = LightPipeline(pipeline_model) 54 | 55 | result = lmodel.fullAnnotate("Mona Lisa er et oljemaleri fra 1500-tallet skapt av Leonardo. Den holdes på Louvre i Paris.")[0] 56 | 57 | ``` 58 | 59 | ```scala 60 | 61 | ``` 62 |
63 | 64 | {:.h2_title} 65 | ## Results 66 | ```bash 67 | +-------------+---------+-------+----------+ 68 | | ner_chunk | begin | end | entity | 69 | +=============+=========+=======+==========+ 70 | | Mona Lisa | 0 | 8 | PER | 71 | +-------------+---------+-------+----------+ 72 | | Leonardo | 52 | 59 | PER | 73 | +-------------+---------+-------+----------+ 74 | | Louvre | 76 | 81 | LOC | 75 | +-------------+---------+-------+----------+ 76 | | Paris | 85 | 89 | GPE_LOC | 77 | +-------------+---------+-------+----------+ 78 | ``` 79 | 80 | {:.model-param} 81 | ## Model Information 82 | 83 | {:.table-model} 84 | |-------------------------|--------------| 85 | | Model Name | norne_6B_300 | 86 | | Model Class | NerDLModel | 87 | | Spark Compatibility | 2.5.0 | 88 | | Spark NLP Compatibility | 2.4 | 89 | | License | open source | 90 | | Edition | public | 91 | | Input Labels | | 92 | | Output Labels | | 93 | | Language | no | 94 | | Dimension | | 95 | | Case Sensitive | | 96 | | Upstream Dependencies | glove_6B_300 | 97 | 98 | 99 | 100 | 101 | {:.h2_title} 102 | ## Data Source 103 | 104 | 105 | 106 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-05-06-norne_840B_300_no.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: 4 | author: John Snow Labs 5 | name: norne_840B_300 6 | class: NerDLModel 7 | language: no 8 | repository: public/models 9 | date: 06/05/2020 10 | tags: [ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | 21 | 22 | {:.btn-box} 23 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/models/norne_840B_300_no_2.5.0_2.4_1588781290267.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
24 | 25 | ## How to use 26 |
27 | 28 | {% include programmingLanguageSelectScalaPython.html %} 29 | 30 | ```python 31 | 32 | embeddings = WordEmbeddingsModel.pretrained(glove_840B_300, lang=xx) 33 | embeddings.setInputCols(["sentence", 'token']) 34 | embeddings.setOutputCol("embeddings") 35 | 36 | ner = NerDLModel.pretrained(norne_840B_300, lang=no) 37 | ner.setInputCols(["sentence", "token", "embeddings"]) 38 | ner.setOutputCol("ner") 39 | 40 | ner_converter = NerConverter() 41 | ner_converter.setInputCols(["sentence", "token", "ner"]) 42 | ner_converter.setOutputCol("ner_chunk") 43 | 44 | pipeline = Pipeline(stages=[ documentAssembler, 45 | sentenceDetector, 46 | tokenizer, 47 | embeddings, 48 | ner, 49 | ner_converter 50 | ]) 51 | 52 | pipeline_model = pipeline.fit(spark.createDataFrame([['']]).toDF("text")) 53 | lmodel = LightPipeline(pipeline_model) 54 | 55 | result = lmodel.fullAnnotate("Mona Lisa er et oljemaleri fra 1500-tallet skapt av Leonardo. Den holdes på Louvre i Paris.")[0] 56 | 57 | ``` 58 | 59 | ```scala 60 | 61 | ``` 62 |
63 | 64 | {:.h2_title} 65 | ## Results 66 | ```bash 67 | +-------------+---------+-------+----------+ 68 | | ner_chunk | begin | end | entity | 69 | +=============+=========+=======+==========+ 70 | | Mona Lisa | 0 | 8 | PER | 71 | +-------------+---------+-------+----------+ 72 | | Leonardo | 52 | 59 | PER | 73 | +-------------+---------+-------+----------+ 74 | | Louvre | 76 | 81 | LOC | 75 | +-------------+---------+-------+----------+ 76 | | Paris | 85 | 89 | GPE_LOC | 77 | +-------------+---------+-------+----------+ 78 | ``` 79 | 80 | {:.model-param} 81 | ## Model Information 82 | 83 | {:.table-model} 84 | |-------------------------|----------------| 85 | | Model Name | norne_840B_300 | 86 | | Model Class | NerDLModel | 87 | | Spark Compatibility | 2.5.0 | 88 | | Spark NLP Compatibility | 2.4 | 89 | | License | open source | 90 | | Edition | public | 91 | | Input Labels | | 92 | | Output Labels | | 93 | | Language | no | 94 | | Dimension | | 95 | | Case Sensitive | | 96 | | Upstream Dependencies | glove_840B_300 | 97 | 98 | 99 | 100 | 101 | {:.h2_title} 102 | ## Data Source 103 | 104 | 105 | 106 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-05-07-assertion_i2b2_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Assertion DL I2B2 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-05-07 10 | tags: [clinical,assertion] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Deep learning named entity recognition model for assertions. 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | hypothetical, present, absent, possible, conditional, associated_with_someone_else 23 | 24 | {:.btn-box} 25 |
[Open in Colab](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/2.Clinical_Assertion_Model.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/assertion_i2b2_en_2.4.2_2.4_1588811895962.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|----------------------------------| 47 | | Model Name | assertion_i2b2 | 48 | | Model Class | AssertionDLModel | 49 | | Spark Compatibility | 2.4.2 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, chunk, word_embeddings | 54 | | Output Labels | assertion | 55 | | Language | en | 56 | | Case Sensitive | False | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text with `embeddings_clinical`. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-05-16-chunkresolve_loinc_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: ChunkResolver Loinc Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-05-16 10 | tags: [clinical,entity_resolution,loinc,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | LOINC Codes and ther Standard Name with `clinical_embeddings` 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/chunkresolve_loinc_clinical_en_2.5.0_2.4_1589599195201.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|-----------------------------| 47 | | Model Name | chunkresolve_loinc_clinical | 48 | | Model Class | ChunkEntityResolverModel | 49 | | Spark Compatibility | 2.5.0 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | token, chunk_embeddings | 54 | | Output Labels | entity | 55 | | Language | en | 56 | | Case Sensitive | True | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on LOINC dataset with `embeddings_clinical`. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-05-19-deidentify_rb_no_regex_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Deidentify RB No Regex 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-05-19 10 | tags: [clinical,deidentify,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Personal Information in order to deidentify 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/deidentify_rb_no_regex_en_2.5.0_2.4_1589924063833.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|------------------------| 47 | | Model Name | deidentify_rb_no_regex | 48 | | Model Class | DeIdentificationModel | 49 | | Spark Compatibility | 2.4.5 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, token, chunk | 54 | | Output Labels | document | 55 | | Language | en | 56 | | Upstream Dependencies | ner_deid | 57 | 58 | 59 | 60 | 61 | 62 | {:.h2_title} 63 | ## Data Source 64 | Rule based DeIdentifier based on `ner_deid`. 65 | 66 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-05-21-assertion_dl_large_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Assertion DL Large 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-05-21 10 | tags: [clinical,assertion] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Deep learning named entity recognition model for assertions. 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | hypothetical, present, absent, possible, conditional, associated_with_someone_else 23 | 24 | {:.btn-box} 25 |
[Open in Colab](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/2.Clinical_Assertion_Model.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/assertion_dl_large_en_2.5.0_2.4_1590022282256.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|----------------------------------| 47 | | Model Name | assertion_dl_large | 48 | | Model Class | AssertionDLModel | 49 | | Spark Compatibility | 2.4.2 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, chunk, word_embeddings | 54 | | Output Labels | assertion | 55 | | Language | en | 56 | | Case Sensitive | False | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text with `embeddings_clinical`. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-05-21-ner_clinical_large_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Ner DL Model Clinical (Large) 4 | author: John Snow Labs 5 | name: ner_clinical_large 6 | class: NerDLModel 7 | language: en 8 | repository: clinical/models 9 | date: 21/05/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Clinical NER (Large) is a Named Entity Recognition model that annotates text to find references to clinical events. The entities it annotates are Problem, Treatment, and Test. Clinical NER is trained with the 'embeddings_clinical' word embeddings model, so be sure to use the same embeddings in the pipeline. 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Problem, Test, Treatment 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/NER_EVENTS_CLINICAL/){:.button.button-orange}
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/streamlit_notebooks/healthcare/NER_EVENTS_CLINICAL.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_clinical_large_en_2.5.0_2.4_1590021302624.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|--------------------------| 52 | | Model Name | ner_clinical_large | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.5.0 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | Problem, Test, Treatment | 60 | | Language | en | 61 | | Dimension | | 62 | | Case Sensitive | 0.0 | 63 | | Upstream Dependencies | embeddings_clinical | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | Trained on i2b2 augmented data with `clinical_embeddings` 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-05-21-ner_large_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Ner DL Model Clinical (Large) 4 | author: John Snow Labs 5 | name: ner_large_clinical 6 | class: NerDLModel 7 | language: en 8 | repository: clinical/models 9 | date: 21/05/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | PROBLEM,TEST,TREATMENT 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_large_clinical_en_2.5.0_2.4_1590021302624.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|------------------------| 52 | | Model Name | ner_large_clinical | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.5.0 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | PROBLEM,TEST,TREATMENT | 60 | | Language | en | 61 | | Dimension | | 62 | | Case Sensitive | | 63 | | Upstream Dependencies | embeddings_clinical | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | Trained on data gathered and manually annotated by John Snow Labs 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-05-26-embeddings_scielo_150d_es.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Embeddings Scielo 150 dims 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-05-26 10 | tags: [clinical,embeddings,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Word2Vec feature vectors based on embeddings_scielo_150d 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/embeddings_scielo_150d_es_2.5.0_2.4_1590467082526.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|------------------------| 47 | | Model Name | embeddings_scielo_150d | 48 | | Model Class | WordEmbeddingsModel | 49 | | Spark Compatibility | 2.5.0 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, token | 54 | | Output Labels | word_embeddings | 55 | | Language | es | 56 | | Dimension | 150.0 | 57 | 58 | 59 | 60 | 61 | 62 | {:.h2_title} 63 | ## Data Source 64 | Trained on Scielo Articles. 65 | 66 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-05-26-embeddings_scielo_300d_es.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Embeddings Scielo 300 dims 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-05-26 10 | tags: [clinical,embeddings,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Word2Vec feature vectors based on embeddings_scielo_300d 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/embeddings_scielo_300d_es_2.5.0_2.4_1590467138742.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|------------------------| 47 | | Model Name | embeddings_scielo_300d | 48 | | Model Class | WordEmbeddingsModel | 49 | | Spark Compatibility | 2.5.0 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, token | 54 | | Output Labels | word_embeddings | 55 | | Language | es | 56 | | Dimension | 300.0 | 57 | 58 | 59 | 60 | 61 | 62 | {:.h2_title} 63 | ## Data Source 64 | Trained on Scielo Articles. 65 | 66 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-05-26-embeddings_scielo_50d_es.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Embeddings Scielo 50 dims 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-05-26 10 | tags: [clinical,embeddings,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Word2Vec feature vectors based on embeddings_scielo_50d 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/embeddings_scielo_50d_es_2.5.0_2.4_1590467114993.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|-----------------------| 47 | | Model Name | embeddings_scielo_50d | 48 | | Model Class | WordEmbeddingsModel | 49 | | Spark Compatibility | 2.5.0 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, token | 54 | | Output Labels | word_embeddings | 55 | | Language | es | 56 | | Dimension | 50.0 | 57 | 58 | 59 | 60 | 61 | 62 | {:.h2_title} 63 | ## Data Source 64 | Trained on Scielo Articles. 65 | 66 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-05-26-embeddings_scielowiki_150d_es.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Embeddings Scielowiki 150 dims 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-05-26 10 | tags: [clinical,embeddings,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Word2Vec feature vectors based on embeddings_scielowiki_150d 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/embeddings_scielowiki_150d_es_2.5.0_2.4_1590467545910.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|----------------------------| 47 | | Model Name | embeddings_scielowiki_150d | 48 | | Model Class | WordEmbeddingsModel | 49 | | Spark Compatibility | 2.5.0 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, token | 54 | | Output Labels | word_embeddings | 55 | | Language | es | 56 | | Dimension | 150.0 | 57 | 58 | 59 | 60 | 61 | 62 | {:.h2_title} 63 | ## Data Source 64 | Trained on Scielo Articles + Clinical Wikipedia Articles. 65 | 66 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-05-26-embeddings_scielowiki_300d_es.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Embeddings Scielowiki 300 dims 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-05-26 10 | tags: [clinical,embeddings,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Word2Vec feature vectors based on embeddings_scielowiki_300d 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/embeddings_scielowiki_300d_es_2.5.0_2.4_1590467643391.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|----------------------------| 47 | | Model Name | embeddings_scielowiki_300d | 48 | | Model Class | WordEmbeddingsModel | 49 | | Spark Compatibility | 2.5.0 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, token | 54 | | Output Labels | word_embeddings | 55 | | Language | es | 56 | | Dimension | 300.0 | 57 | 58 | 59 | 60 | 61 | 62 | {:.h2_title} 63 | ## Data Source 64 | Trained on Scielo Articles + Clinical Wikipedia Articles. 65 | 66 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-05-26-embeddings_scielowiki_50d_es.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Embeddings Scielowiki 50 dims 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-05-26 10 | tags: [clinical,embeddings,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Word2Vec feature vectors based on embeddings_scielowiki_50d 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/embeddings_scielowiki_50d_es_2.5.0_2.4_1590467602230.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|---------------------------| 47 | | Model Name | embeddings_scielowiki_50d | 48 | | Model Class | WordEmbeddingsModel | 49 | | Spark Compatibility | 2.5.0 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, token | 54 | | Output Labels | word_embeddings | 55 | | Language | es | 56 | | Dimension | 50.0 | 57 | 58 | 59 | 60 | 61 | 62 | {:.h2_title} 63 | ## Data Source 64 | Trained on Scielo Articles + Clinical Wikipedia Articles. 65 | 66 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-05-27-embeddings_sciwiki_150d_es.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Embeddings Sciwiki 150 dims 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-05-27 10 | tags: [clinical,embeddings,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Word2Vec feature vectors based on embeddings_sciwiki_150d 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/embeddings_sciwiki_150d_es_2.5.0_2.4_1590609340084.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|-------------------------| 47 | | Model Name | embeddings_sciwiki_150d | 48 | | Model Class | WordEmbeddingsModel | 49 | | Spark Compatibility | 2.5.0 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, token | 54 | | Output Labels | word_embeddings | 55 | | Language | es | 56 | | Dimension | 150.0 | 57 | 58 | 59 | 60 | 61 | 62 | {:.h2_title} 63 | ## Data Source 64 | Trained on Clinical Wikipedia Articles. 65 | 66 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-05-27-embeddings_sciwiki_300d_es.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Embeddings Sciwiki 300 dims 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-05-27 10 | tags: [clinical,embeddings,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Word2Vec feature vectors based on embeddings_sciwiki_300d 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/embeddings_sciwiki_300d_es_2.5.0_2.4_1590609454054.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|-------------------------| 47 | | Model Name | embeddings_sciwiki_300d | 48 | | Model Class | WordEmbeddingsModel | 49 | | Spark Compatibility | 2.5.0 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, token | 54 | | Output Labels | word_embeddings | 55 | | Language | es | 56 | | Dimension | 300.0 | 57 | 58 | 59 | 60 | 61 | 62 | {:.h2_title} 63 | ## Data Source 64 | Trained on Clinical Wikipedia Articles. 65 | 66 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-05-27-embeddings_sciwiki_50d_es.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Embeddings Sciwiki 50 dims 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-05-27 10 | tags: [clinical,embeddings,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Word2Vec feature vectors based on embeddings_sciwiki_50d 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/embeddings_sciwiki_50d_es_2.5.0_2.4_1590609287349.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|------------------------| 47 | | Model Name | embeddings_sciwiki_50d | 48 | | Model Class | WordEmbeddingsModel | 49 | | Spark Compatibility | 2.5.0 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, token | 54 | | Output Labels | word_embeddings | 55 | | Language | es | 56 | | Dimension | 50.0 | 57 | 58 | 59 | 60 | 61 | 62 | {:.h2_title} 63 | ## Data Source 64 | Trained on Clinical Wikipedia Articles. 65 | 66 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-05-29-embeddings_healthcare_100d_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Embeddings Healthcare 100 dims 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-05-29 10 | tags: [clinical,embeddings,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Word2Vec feature vectors based on embeddings_healthcare_100d 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/embeddings_healthcare_100d_en_2.5.0_2.4_1590794626292.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|----------------------------| 47 | | Model Name | embeddings_healthcare_100d | 48 | | Model Class | WordEmbeddingsModel | 49 | | Spark Compatibility | 2.5.0 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, token | 54 | | Output Labels | word_embeddings | 55 | | Language | en | 56 | | Dimension | 100.0 | 57 | 58 | 59 | 60 | 61 | 62 | {:.h2_title} 63 | ## Data Source 64 | Trained on PubMed + ICD10 + UMLS + MIMIC III corpora. 65 | 66 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-06-02-embeddings_biovec_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Embeddings BioVec 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-06-02 10 | tags: [clinical,embeddings,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Word2Vec feature vectors based on embeddings_biovec 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/embeddings_biovec_en_2.5.0_2.4_1591068211397.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|---------------------| 47 | | Model Name | embeddings_biovec | 48 | | Model Class | WordEmbeddingsModel | 49 | | Spark Compatibility | 2.5.0 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, token | 54 | | Output Labels | word_embeddings | 55 | | Language | en | 56 | | Dimension | 300.0 | 57 | 58 | 59 | 60 | 61 | 62 | {:.h2_title} 63 | ## Data Source 64 | Trained on PubMed corpora. 65 | 66 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-06-20-chunkresolve_snomed_findings_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: ChunkResolver Snomed Findings Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-06-20 10 | tags: [clinical,entity_resolution,snomed,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Snomed Codes and their normalized definition with `clinical_embeddings` 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/ER_SNOMED/){:.button.button-orange}
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/13.Snomed_Entity_Resolver_Model_Training.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/chunkresolve_snomed_findings_clinical_en_2.5.1_2.4_1592617161564.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|---------------------------------------| 47 | | Model Name | chunkresolve_snomed_findings_clinical | 48 | | Model Class | ChunkEntityResolverModel | 49 | | Spark Compatibility | 2.5.1 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | token, chunk_embeddings | 54 | | Output Labels | entity | 55 | | Language | en | 56 | | Case Sensitive | True | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on SNOMED CT Findings. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-06-24-chunkresolve_rxnorm_xsmall_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: ChunkResolver Rxnorm Xsmall Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-06-24 10 | tags: [clinical,entity_resolution,rxnorm,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Snomed Codes and their normalized definition with `clinical_embeddings` 23 | 24 | {:.btn-box} 25 |
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/13.Snomed_Entity_Resolver_Model_Training.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/chunkresolve_rxnorm_xsmall_clinical_en_2.5.2_2.4_1592959394598.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|-------------------------------------| 47 | | Model Name | chunkresolve_rxnorm_xsmall_clinical | 48 | | Model Class | ChunkEntityResolverModel | 49 | | Spark Compatibility | 2.5.2 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | token, chunk_embeddings | 54 | | Output Labels | entity | 55 | | Language | en | 56 | | Case Sensitive | True | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on December 2019 RxNorm Subset. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-07-08-ner_diag_proc_es.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Ner DL Model Clinical 4 | author: John Snow Labs 5 | name: ner_diag_proc 6 | class: NerDLModel 7 | language: es 8 | repository: clinical/models 9 | date: 08/07/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Pretrained named entity recognition deep learning model for diagnostics and procedures in spanish 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Diagnostico, Procedimiento 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_diag_proc_es_2.5.3_2.4_1594168623415.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|----------------------------| 52 | | Model Name | ner_diag_proc | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.5.3 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | Diagnostico, Procedimiento | 60 | | Language | es | 61 | | Dimension | | 62 | | Case Sensitive | | 63 | | Upstream Dependencies | embeddings_scielowiki_300d | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | Trained on CodiEsp Challenge dataset trained with `embeddings_scielowiki_300d` 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-07-08-ner_neoplasms_es.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Neoplasms NER 4 | author: John Snow Labs 5 | name: ner_neoplasms 6 | class: NerDLModel 7 | language: es 8 | repository: clinical/models 9 | date: 08/07/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Neoplasms NER is a Named Entity Recognition model that annotates text to find references to tumors. The only entity it annotates is MalignantNeoplasm. Neoplasms NER is trained with the 'embeddings_scielowiki_300d' word embeddings model, so be sure to use the same embeddings in the pipeline. 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | MORFOLOGIA_NEOPLASIA 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_neoplasms_es_2.5.3_2.4_1594168624415.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|----------------------------| 52 | | Model Name | ner_neoplasms | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.5.3 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | MORFOLOGIA_NEOPLASIA | 60 | | Language | es | 61 | | Dimension | | 62 | | Case Sensitive | 0.0 | 63 | | Upstream Dependencies | embeddings_scielowiki_300d | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | Named Entity Recognition model for Neoplasic Morphology 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-07-19-deidentify_large_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Deidentify Large 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-07-19 10 | tags: [clinical,deidentify,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Deidentify (Large) is a deidentification model. It identifies instances of protected health information in text documents, and it can either obfuscate them (e.g., replacing names with different, fake names) or mask them (e.g., replacing "2020,06,04" with ""). This model is useful for maintaining HIPAA compliance when dealing with text documents that contain protected health information. 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Contact, Location, Name, Profession 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/DEID_PHI_TEXT){:.button.button-orange}
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/4.Clinical_DeIdentificiation.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/deidentify_large_en_2.5.1_2.4_1595199111307.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|------------------------| 47 | | Model Name | deidentify_large | 48 | | Model Class | DeIdentificationModel | 49 | | Spark Compatibility | 2.5.1 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, token, chunk | 54 | | Output Labels | document | 55 | | Language | en | 56 | | Upstream Dependencies | ner_deid_large | 57 | 58 | 59 | 60 | 61 | 62 | {:.h2_title} 63 | ## Data Source 64 | Trained on 10.000 Contact, Location, Name and Profession random replacements. 65 | 66 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-07-22-ner_deid_large_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Deidentification NER (Large) 4 | author: John Snow Labs 5 | name: ner_deid_large 6 | class: NerDLModel 7 | language: en 8 | repository: clinical/models 9 | date: 22/07/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Deidentification NER (Large) is a Named Entity Recognition model that annotates text to find protected health information that may need to be deidentified. The entities it annotates are Age, Contact, Date, Id, Location, Name, and Profession. Clinical NER is trained with the 'embeddings_clinical' word embeddings model, so be sure to use the same embeddings in the pipeline. 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Age, Contact, Date, Id, Location, Name, Profession 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/NER_DEMOGRAPHICS/){:.button.button-orange}
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/streamlit_notebooks/healthcare/NER_DEMOGRAPHICS.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_deid_large_en_2.5.3_2.4_1595427435246.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|----------------------------------------------------| 52 | | Model Name | ner_deid_large | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.4.2 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | Age, Contact, Date, Id, Location, Name, Profession | 60 | | Language | en | 61 | | Dimension | | 62 | | Case Sensitive | 0.0 | 63 | | Upstream Dependencies | embeddings_clinical | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | Trained on plain n2c2 2014: De-identification and Heart Disease Risk Factors Challenge datasets with `embeddings_clinical` 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-07-27-chunkresolve_rxnorm_cd_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: ChunkResolver Rxnorm Cd Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-07-27 10 | tags: [clinical,entity_resolution,rxnorm,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | RxNorm Codes and their normalized definition with `clinical_embeddings` 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/ER_RXNORM/){:.button.button-orange}

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/chunkresolve_rxnorm_cd_clinical_en_2.5.1_2.4_1595813950836.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|---------------------------------| 47 | | Model Name | chunkresolve_rxnorm_cd_clinical | 48 | | Model Class | ChunkEntityResolverModel | 49 | | Spark Compatibility | 2.5.1 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | token, chunk_embeddings | 54 | | Output Labels | entity | 55 | | Language | en | 56 | | Case Sensitive | True | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on December 2019 RxNorm Clinical Drugs (TTY=CD) ontology graph with `embeddings_clinical`. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-07-27-chunkresolve_rxnorm_sbd_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: ChunkResolver Rxnorm Sbd Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-07-27 10 | tags: [clinical,entity_resolution,rxnorm,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | RxNorm Codes and their normalized definition with `clinical_embeddings` 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/ER_RXNORM/){:.button.button-orange}

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/chunkresolve_rxnorm_sbd_clinical_en_2.5.1_2.4_1595813912622.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|----------------------------------| 47 | | Model Name | chunkresolve_rxnorm_sbd_clinical | 48 | | Model Class | ChunkEntityResolverModel | 49 | | Spark Compatibility | 2.5.1 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | token, chunk_embeddings | 54 | | Output Labels | entity | 55 | | Language | en | 56 | | Case Sensitive | True | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on December 2019 RxNorm Clinical Drugs (TTY=SBD) ontology graph with `embeddings_clinical`. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-07-27-chunkresolve_rxnorm_scd_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: ChunkResolver Rxnorm Scd Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-07-27 10 | tags: [clinical,entity_resolution,rxnorm,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | RxNorm Codes and their normalized definition with `clinical_embeddings` 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/chunkresolve_rxnorm_scd_clinical_en_2.5.1_2.4_1595813884363.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|----------------------------------| 47 | | Model Name | chunkresolve_rxnorm_scd_clinical | 48 | | Model Class | ChunkEntityResolverModel | 49 | | Spark Compatibility | 2.5.1 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | token, chunk_embeddings | 54 | | Output Labels | entity | 55 | | Language | en | 56 | | Case Sensitive | True | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on December 2019 RxNorm Clinical Drugs (TTY=SCD) ontology graph with `embeddings_clinical`. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-08-18-ner_events_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Ner DL Model Events `embeddings_clinical` 4 | author: John Snow Labs 5 | name: ner_events_clinical 6 | class: NerDLModel 7 | language: en 8 | repository: clinical/models 9 | date: 18/08/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Pretrained named entity recognition deep learning model for clinical events. 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | CLINICAL_DEPT,DATE,DURATION,EVIDENTIAL,FREQUENCY,OCCURRENCE,PROBLEM,TEST,TIME,TREATMENT 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/NER_EVENTS_CLINICAL/){:.button.button-orange}
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/1.Clinical_Named_Entity_Recognition_Model.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_events_clinical_en_2.5.5_2.4_1597775531760.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|-----------------------------------------------------------------------------------------| 52 | | Model Name | ner_events_clinical | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.5.0 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | CLINICAL_DEPT,DATE,DURATION,EVIDENTIAL,FREQUENCY,OCCURRENCE,PROBLEM,TEST,TIME,TREATMENT | 60 | | Language | en | 61 | | Dimension | | 62 | | Case Sensitive | 0.0 | 63 | | Upstream Dependencies | embeddings_clinical | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | Trained on i2b2 events data with `clinical_embeddings` 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-08-18-re_temporal_events_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Relation Extraction Model Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-08-18 10 | tags: [clinical,events,relation,extraction,temporal,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | TrIP (improved), TrWP (worsened), TrCP (caused problem), TrAP (administered), TrNAP (avoided), TeRP (revealed problem), TeCP (investigate problem), PIP (problems related) 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/RE_CLINICAL_EVENTS/){:.button.button-orange}
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/10.Clinical_Relation_Extraction.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/re_temporal_events_clinical_en_2.5.5_2.4_1597774124917.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|-----------------------------------------| 47 | | Model Name | re_temporal_events_clinical | 48 | | Model Class | RelationExtractionModel | 49 | | Spark Compatibility | 2.5.5 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | word_embeddings, chunk, pos, dependency | 54 | | Output Labels | category | 55 | | Language | en | 56 | | Case Sensitive | False | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on data gathered and manually annotated by John Snow Labs. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-08-18-re_temporal_events_enriched_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Relation Extraction Model Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-08-18 10 | tags: [clinical,events,relation,extraction,temporal,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Extracts: Temporal relations (BEFORE, AFTER, SIMULTANEOUS, BEGUN_BY, ENDED_BY, DURING, BEFORE_OVERLAP) between clinical events (`ner_events_clinical`) 23 | 24 | {:.btn-box} 25 |
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/10.Clinical_Relation_Extraction.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/re_temporal_events_enriched_clinical_en_2.5.5_2.4_1597775105767.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|-----------------------------------------| 47 | | Model Name | re_temporal_events_enriched_clinical | 48 | | Model Class | RelationExtractionModel | 49 | | Spark Compatibility | 2.5.5 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | word_embeddings, chunk, pos, dependency | 54 | | Output Labels | category | 55 | | Language | en | 56 | | Case Sensitive | False | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on data gathered and manually annotated by John Snow Labs. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-08-19-explain_clinical_doc_carp_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Explain Clinical Doc Clinical Assertion Relation Posology 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-08-19 10 | tags: [clinical,pipeline,ner,assertion,relation,posology,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | A pretrained pipeline with ner_clinical, assertion_dl, re_clinical and ner_posology. It will extract clinical and medication entities, assign assertion status and find relationships between clinical entities. 19 | 20 | 21 | 22 | {:.btn-box} 23 |
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/11.Pretrained_Clinical_Pipelines.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/explain_clinical_doc_carp_en_2.5.5_2.4_1597841630062.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
24 | 25 | ## How to use 26 |
27 | 28 | {% include programmingLanguageSelectScalaPython.html %} 29 | 30 | ```python 31 | 32 | ``` 33 | 34 | ```scala 35 | 36 | ``` 37 |
38 | 39 | 40 | 41 | {:.model-param} 42 | ## Model Information 43 | {:.table-model} 44 | |-------------------------|---------------------------| 45 | | Model Name | explain_clinical_doc_carp | 46 | | Model Class | PipelineModel | 47 | | Spark Compatibility | 2.5.5 | 48 | | Spark NLP Compatibility | 2.4 | 49 | | License | Licensed | 50 | | Edition | Official | 51 | | Language | en | 52 | 53 | 54 | 55 | {:.h2_title} 56 | ## Included Models 57 | - ner_clinical 58 | - assertion_dl 59 | - re_clinical 60 | - ner_posology 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-08-19-explain_clinical_doc_cra_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Explain Clinical Doc Clinical Relation Assertion 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-08-19 10 | tags: [clinical,pipeline,ner,assertion,relation,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | A pretrained pipeline with ner_clinical, assertion_dl, re_clinical. It will extract clinical entities, assign assertion status and find relationships between clinical entities. 19 | 20 | 21 | 22 | {:.btn-box} 23 |
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/11.Pretrained_Clinical_Pipelines.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/explain_clinical_doc_cra_en_2.5.5_2.4_1597846145640.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
24 | 25 | ## How to use 26 |
27 | 28 | {% include programmingLanguageSelectScalaPython.html %} 29 | 30 | ```python 31 | 32 | ``` 33 | 34 | ```scala 35 | 36 | ``` 37 |
38 | 39 | 40 | 41 | {:.model-param} 42 | ## Model Information 43 | {:.table-model} 44 | |-------------------------|--------------------------| 45 | | Model Name | explain_clinical_doc_cra | 46 | | Model Class | PipelineModel | 47 | | Spark Compatibility | 2.5.5 | 48 | | Spark NLP Compatibility | 2.4 | 49 | | License | Licensed | 50 | | Edition | Official | 51 | | Language | en | 52 | 53 | 54 | 55 | {:.h2_title} 56 | ## Included Models 57 | - ner_clinical 58 | - assertion_dl 59 | - re_clinical 60 | 61 | 62 | {:.h2_title} 63 | ## Data Source 64 | 65 | 66 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-08-19-explain_clinical_doc_era_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Explain Clinical Doc Events Relation Assertion 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-08-19 10 | tags: [clinical,pipeline,ner,events,assertion,relation,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | A pretrained pipeline with ner_clinical_events, assertion_dl and re_temporal_events_clinical. It will extract clinical entities, assign assertion status and find temporal relationships between clinical entities 19 | 20 | 21 | 22 | {:.btn-box} 23 |
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/11.Pretrained_Clinical_Pipelines.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/explain_clinical_doc_era_en_2.5.5_2.4_1597845753750.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
24 | 25 | ## How to use 26 |
27 | 28 | {% include programmingLanguageSelectScalaPython.html %} 29 | 30 | ```python 31 | 32 | ``` 33 | 34 | ```scala 35 | 36 | ``` 37 |
38 | 39 | 40 | 41 | {:.model-param} 42 | ## Model Information 43 | {:.table-model} 44 | |-------------------------|--------------------------| 45 | | Model Name | explain_clinical_doc_era | 46 | | Model Class | PipelineModel | 47 | | Spark Compatibility | 2.5.5 | 48 | | Spark NLP Compatibility | 2.4 | 49 | | License | Licensed | 50 | | Edition | Official | 51 | | Language | en | 52 | 53 | 54 | 55 | {:.h2_title} 56 | ## Included Models 57 | - ner_clinical_events 58 | - assertion_dl 59 | - re_temporal_events_clinical 60 | 61 | 62 | {:.h2_title} 63 | ## Data Source 64 | 65 | 66 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-08-27-ner_human_phenotype_gene_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Ner DL Model Phenotype / Gene 4 | author: John Snow Labs 5 | name: ner_human_phenotype_gene_clinical 6 | class: NerDLModel 7 | language: en 8 | repository: clinical/models 9 | date: 27/08/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | GENE,HP 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/NER_HUMAN_PHENOTYPE_GENE_CLINICAL/){:.button.button-orange}

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_human_phenotype_gene_clinical_en_2.5.5_2.4_1598558253840.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|-----------------------------------| 52 | | Model Name | ner_human_phenotype_gene_clinical | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.5.5 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | GENE,HP | 60 | | Language | en | 61 | | Dimension | | 62 | | Case Sensitive | | 63 | | Upstream Dependencies | embeddings_clinical | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-08-27-ner_human_phenotype_go_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Ner DL Model Healthcare 4 | author: John Snow Labs 5 | name: ner_human_phenotype_go_clinical 6 | class: NerDLModel 7 | language: en 8 | repository: clinical/models 9 | date: 27/08/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | GO,HP 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/NER_HUMAN_PHENOTYPE_GO_CLINICAL/){:.button.button-orange}

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_human_phenotype_go_clinical_en_2.5.5_2.4_1598558398770.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|---------------------------------| 52 | | Model Name | ner_human_phenotype_go_clinical | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.5.5 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | GO,HP | 60 | | Language | en | 61 | | Dimension | | 62 | | Case Sensitive | | 63 | | Upstream Dependencies | embeddings_clinical | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-08-27-re_human_phenotype_gene_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Relation Extraction Model Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-08-27 10 | tags: [clinical,relation,extraction,phenotype,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | 21 | 22 | {:.btn-box} 23 |
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/10.Clinical_Relation_Extraction.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/re_human_phenotype_gene_clinical_en_2.5.5_2.4_1598560152543.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
24 | 25 | ## How to use 26 |
27 | 28 | {% include programmingLanguageSelectScalaPython.html %} 29 | 30 | ```python 31 | 32 | ``` 33 | 34 | ```scala 35 | 36 | ``` 37 |
38 | 39 | 40 | 41 | {:.model-param} 42 | ## Model Information 43 | {:.table-model} 44 | |-------------------------|-----------------------------------------| 45 | | Model Name | re_human_phenotype_gene_clinical | 46 | | Model Class | RelationExtractionModel | 47 | | Spark Compatibility | 2.5.5 | 48 | | Spark NLP Compatibility | 2.4 | 49 | | License | Licensed | 50 | | Edition | Official | 51 | | Input Labels | word_embeddings, chunk, pos, dependency | 52 | | Output Labels | category | 53 | | Language | en | 54 | | Case Sensitive | False | 55 | | Upstream Dependencies | embeddings_clinical | 56 | 57 | 58 | 59 | 60 | 61 | {:.h2_title} 62 | ## Data Source 63 | Trained on data gathered and manually annotated by John Snow Labs. 64 | 65 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-08-30-dane_ner_6B_100_da.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: 4 | author: John Snow Labs 5 | name: dane_ner_6B_100 6 | class: NerDLModel 7 | language: da 8 | repository: public/models 9 | date: 30/08/2020 10 | tags: [ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | 21 | 22 | {:.btn-box} 23 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/models/dane_ner_6B_100_da_2.6.0_2.4_1598810267725.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
24 | 25 | ## How to use 26 |
27 | 28 | {% include programmingLanguageSelectScalaPython.html %} 29 | 30 | ```python 31 | 32 | embeddings = WordEmbeddingsModel.pretrained(glove_100d, lang=en) 33 | embeddings.setInputCols(["sentence", 'token']) 34 | embeddings.setOutputCol("embeddings") 35 | 36 | ner = NerDLModel.pretrained(dane_ner_6B_100, lang=da) 37 | ner.setInputCols(["sentence", "token", "embeddings"]) 38 | ner.setOutputCol("ner") 39 | 40 | ner_converter = NerConverter() 41 | ner_converter.setInputCols(["sentence", "token", "ner"]) 42 | ner_converter.setOutputCol("ner_chunk") 43 | 44 | pipeline = Pipeline(stages=[ documentAssembler, 45 | sentenceDetector, 46 | tokenizer, 47 | embeddings, 48 | ner, 49 | ner_converter 50 | ]) 51 | 52 | pipeline_model = pipeline.fit(spark.createDataFrame([['']]).toDF("text")) 53 | lmodel = LightPipeline(pipeline_model) 54 | 55 | result = lmodel.fullAnnotate("Mona Lisa er et oliemaleri fra det 16. århundrede skabt af Leonardo. Det afholdes på Louvre i Paris.")[0] 56 | 57 | ``` 58 | 59 | ```scala 60 | 61 | ``` 62 |
63 | 64 | {:.h2_title} 65 | ## Results 66 | ```bash 67 | +-------------+---------+-------+----------+ 68 | | ner_chunk | begin | end | entity | 69 | +=============+=========+=======+==========+ 70 | | Mona Lisa | 0 | 8 | PER | 71 | +-------------+---------+-------+----------+ 72 | | Leonardo | 59 | 66 | PER | 73 | +-------------+---------+-------+----------+ 74 | | Louvre | 85 | 90 | LOC | 75 | +-------------+---------+-------+----------+ 76 | | Paris | 94 | 98 | LOC | 77 | +-------------+---------+-------+----------+ 78 | ``` 79 | 80 | {:.model-param} 81 | ## Model Information 82 | 83 | {:.table-model} 84 | |-------------------------|-----------------| 85 | | Model Name | dane_ner_6B_100 | 86 | | Model Class | NerDLModel | 87 | | Spark Compatibility | 2.6.0 | 88 | | Spark NLP Compatibility | 2.4 | 89 | | License | open source | 90 | | Edition | public | 91 | | Input Labels | | 92 | | Output Labels | | 93 | | Language | da | 94 | | Dimension | | 95 | | Case Sensitive | | 96 | | Upstream Dependencies | glove_100d | 97 | 98 | 99 | 100 | 101 | {:.h2_title} 102 | ## Data Source 103 | 104 | 105 | 106 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-08-30-dane_ner_6B_300_da.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: 4 | author: John Snow Labs 5 | name: dane_ner_6B_300 6 | class: NerDLModel 7 | language: da 8 | repository: public/models 9 | date: 30/08/2020 10 | tags: [ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | 21 | 22 | {:.btn-box} 23 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/models/dane_ner_6B_300_da_2.6.0_2.4_1598810268069.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
24 | 25 | ## How to use 26 |
27 | 28 | {% include programmingLanguageSelectScalaPython.html %} 29 | 30 | ```python 31 | 32 | embeddings = WordEmbeddingsModel.pretrained(glove_6B_300, lang=xx) 33 | embeddings.setInputCols(["sentence", 'token']) 34 | embeddings.setOutputCol("embeddings") 35 | 36 | ner = NerDLModel.pretrained(dane_ner_6B_300, lang=da) 37 | ner.setInputCols(["sentence", "token", "embeddings"]) 38 | ner.setOutputCol("ner") 39 | 40 | ner_converter = NerConverter() 41 | ner_converter.setInputCols(["sentence", "token", "ner"]) 42 | ner_converter.setOutputCol("ner_chunk") 43 | 44 | pipeline = Pipeline(stages=[ documentAssembler, 45 | sentenceDetector, 46 | tokenizer, 47 | embeddings, 48 | ner, 49 | ner_converter 50 | ]) 51 | 52 | pipeline_model = pipeline.fit(spark.createDataFrame([['']]).toDF("text")) 53 | lmodel = LightPipeline(pipeline_model) 54 | 55 | result = lmodel.fullAnnotate("Mona Lisa er et oliemaleri fra det 16. århundrede skabt af Leonardo. Det afholdes på Louvre i Paris.")[0] 56 | 57 | ``` 58 | 59 | ```scala 60 | 61 | ``` 62 |
63 | 64 | {:.h2_title} 65 | ## Results 66 | ```bash 67 | +-------------+---------+-------+----------+ 68 | | ner_chunk | begin | end | entity | 69 | +=============+=========+=======+==========+ 70 | | Mona Lisa | 0 | 8 | PER | 71 | +-------------+---------+-------+----------+ 72 | | Leonardo | 59 | 66 | PER | 73 | +-------------+---------+-------+----------+ 74 | | Louvre | 85 | 90 | LOC | 75 | +-------------+---------+-------+----------+ 76 | | Paris | 94 | 98 | LOC | 77 | +-------------+---------+-------+----------+ 78 | ``` 79 | 80 | {:.model-param} 81 | ## Model Information 82 | 83 | {:.table-model} 84 | |-------------------------|-----------------| 85 | | Model Name | dane_ner_6B_300 | 86 | | Model Class | NerDLModel | 87 | | Spark Compatibility | 2.6.0 | 88 | | Spark NLP Compatibility | 2.4 | 89 | | License | open source | 90 | | Edition | public | 91 | | Input Labels | | 92 | | Output Labels | | 93 | | Language | da | 94 | | Dimension | | 95 | | Case Sensitive | | 96 | | Upstream Dependencies | glove_6B_300 | 97 | 98 | 99 | 100 | 101 | {:.h2_title} 102 | ## Data Source 103 | 104 | 105 | 106 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-08-30-dane_ner_840B_100_da.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: 4 | author: John Snow Labs 5 | name: dane_ner_840B_100 6 | class: NerDLModel 7 | language: da 8 | repository: public/models 9 | date: 30/08/2020 10 | tags: [ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | 21 | 22 | {:.btn-box} 23 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/models/dane_ner_840B_300_da_2.6.0_2.4_1598810268070.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
24 | 25 | ## How to use 26 |
27 | 28 | {% include programmingLanguageSelectScalaPython.html %} 29 | 30 | ```python 31 | 32 | ``` 33 | 34 | ```scala 35 | 36 | ``` 37 |
38 | 39 | {:.h2_title} 40 | ## Results 41 | ```bash 42 | 43 | ``` 44 | 45 | {:.model-param} 46 | ## Model Information 47 | 48 | {:.table-model} 49 | |-------------------------|-------------------| 50 | | Model Name | dane_ner_840B_100 | 51 | | Model Class | NerDLModel | 52 | | Spark Compatibility | 2.6.0 | 53 | | Spark NLP Compatibility | 2.4 | 54 | | License | open source | 55 | | Edition | public | 56 | | Input Labels | | 57 | | Output Labels | | 58 | | Language | da | 59 | | Dimension | | 60 | | Case Sensitive | | 61 | | Upstream Dependencies | glove_840B_300 | 62 | 63 | 64 | 65 | 66 | {:.h2_title} 67 | ## Data Source 68 | 69 | 70 | 71 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-08-30-swedish_ner_840B_300_sv.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: 4 | author: John Snow Labs 5 | name: swedish_ner_840B_300 6 | class: NerDLModel 7 | language: sv 8 | repository: public/models 9 | date: 30/08/2020 10 | tags: [ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | 21 | 22 | {:.btn-box} 23 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/models/swedish_ner_840B_300_sv_2.6.0_2.4_1598810268072.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
24 | 25 | ## How to use 26 |
27 | 28 | {% include programmingLanguageSelectScalaPython.html %} 29 | 30 | ```python 31 | 32 | embeddings = WordEmbeddingsModel.pretrained(glove_840B_300, lang=xx) 33 | embeddings.setInputCols(["sentence", 'token']) 34 | embeddings.setOutputCol("embeddings") 35 | 36 | ner = NerDLModel.pretrained(swedish_ner_840B_300, lang=sv) 37 | ner.setInputCols(["sentence", "token", "embeddings"]) 38 | ner.setOutputCol("ner") 39 | 40 | ner_converter = NerConverter() 41 | ner_converter.setInputCols(["sentence", "token", "ner"]) 42 | ner_converter.setOutputCol("ner_chunk") 43 | 44 | pipeline = Pipeline(stages=[ documentAssembler, 45 | sentenceDetector, 46 | tokenizer, 47 | embeddings, 48 | ner, 49 | ner_converter 50 | ]) 51 | 52 | pipeline_model = pipeline.fit(spark.createDataFrame([['']]).toDF("text")) 53 | lmodel = LightPipeline(pipeline_model) 54 | 55 | result = lmodel.fullAnnotate("Mona Lisa är en oljemålning från 1500-talet skapad av Leonardo. Det hålls på Louvren i Paris.")[0] 56 | 57 | ``` 58 | 59 | ```scala 60 | 61 | ``` 62 |
63 | 64 | {:.h2_title} 65 | ## Results 66 | ```bash 67 | +-----------------+---------+-------+----------+ 68 | | ner_chunk | begin | end | entity | 69 | +=================+=========+=======+==========+ 70 | | Mona Lisa | 0 | 8 | PER | 71 | +-----------------+---------+-------+----------+ 72 | | Leonardo | 54 | 61 | PER | 73 | +-----------------+---------+-------+----------+ 74 | | Louvren i Paris | 77 | 91 | MISC | 75 | +-----------------+---------+-------+----------+ 76 | ``` 77 | 78 | {:.model-param} 79 | ## Model Information 80 | 81 | {:.table-model} 82 | |-------------------------|----------------------| 83 | | Model Name | swedish_ner_840B_300 | 84 | | Model Class | NerDLModel | 85 | | Spark Compatibility | 2.6.0 | 86 | | Spark NLP Compatibility | 2.4 | 87 | | License | open source | 88 | | Edition | public | 89 | | Input Labels | | 90 | | Output Labels | | 91 | | Language | sv | 92 | | Dimension | | 93 | | Case Sensitive | | 94 | | Upstream Dependencies | glove_840B_300 | 95 | 96 | 97 | 98 | 99 | {:.h2_title} 100 | ## Data Source 101 | 102 | 103 | 104 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-08-30-wikiner_6B_100_fi.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: 4 | author: John Snow Labs 5 | name: wikiner_6B_100 6 | class: NerDLModel 7 | language: fi 8 | repository: public/models 9 | date: 30/08/2020 10 | tags: [ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | 21 | 22 | {:.btn-box} 23 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/models/finnish_ner_6B_100_fi_2.6.0_2.4_1598965807300.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
24 | 25 | ## How to use 26 |
27 | 28 | {% include programmingLanguageSelectScalaPython.html %} 29 | 30 | ```python 31 | 32 | ``` 33 | 34 | ```scala 35 | 36 | ``` 37 |
38 | 39 | {:.h2_title} 40 | ## Results 41 | ```bash 42 | 43 | ``` 44 | 45 | {:.model-param} 46 | ## Model Information 47 | 48 | {:.table-model} 49 | |-------------------------|----------------| 50 | | Model Name | wikiner_6B_100 | 51 | | Model Class | NerDLModel | 52 | | Spark Compatibility | 2.6.0 | 53 | | Spark NLP Compatibility | 2.4 | 54 | | License | open source | 55 | | Edition | public | 56 | | Input Labels | | 57 | | Output Labels | | 58 | | Language | fi | 59 | | Dimension | | 60 | | Case Sensitive | | 61 | | Upstream Dependencies | glove_100d | 62 | 63 | 64 | 65 | 66 | {:.h2_title} 67 | ## Data Source 68 | 69 | 70 | 71 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-08-30-wikiner_6B_300_fi.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: 4 | author: John Snow Labs 5 | name: wikiner_6B_300 6 | class: NerDLModel 7 | language: fi 8 | repository: public/models 9 | date: 30/08/2020 10 | tags: [ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | 21 | 22 | {:.btn-box} 23 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/models/finnish_ner_6B_300_fi_2.6.0_2.4_1598965807718.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
24 | 25 | ## How to use 26 |
27 | 28 | {% include programmingLanguageSelectScalaPython.html %} 29 | 30 | ```python 31 | 32 | ``` 33 | 34 | ```scala 35 | 36 | ``` 37 |
38 | 39 | {:.h2_title} 40 | ## Results 41 | ```bash 42 | 43 | ``` 44 | 45 | {:.model-param} 46 | ## Model Information 47 | 48 | {:.table-model} 49 | |-------------------------|----------------| 50 | | Model Name | wikiner_6B_300 | 51 | | Model Class | NerDLModel | 52 | | Spark Compatibility | 2.6.0 | 53 | | Spark NLP Compatibility | 2.4 | 54 | | License | open source | 55 | | Edition | public | 56 | | Input Labels | | 57 | | Output Labels | | 58 | | Language | fi | 59 | | Dimension | | 60 | | Case Sensitive | | 61 | | Upstream Dependencies | glove_6B_300 | 62 | 63 | 64 | 65 | 66 | {:.h2_title} 67 | ## Data Source 68 | 69 | 70 | 71 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-08-30-wikiner_840B_300_fi.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: 4 | author: John Snow Labs 5 | name: wikiner_840B_300 6 | class: NerDLModel 7 | language: fi 8 | repository: public/models 9 | date: 30/08/2020 10 | tags: [ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | 21 | 22 | {:.btn-box} 23 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/models/finnish_ner_840B_300_fi_2.6.0_2.4_1598965807720.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
24 | 25 | ## How to use 26 |
27 | 28 | {% include programmingLanguageSelectScalaPython.html %} 29 | 30 | ```python 31 | 32 | ``` 33 | 34 | ```scala 35 | 36 | ``` 37 |
38 | 39 | {:.h2_title} 40 | ## Results 41 | ```bash 42 | 43 | ``` 44 | 45 | {:.model-param} 46 | ## Model Information 47 | 48 | {:.table-model} 49 | |-------------------------|------------------| 50 | | Model Name | wikiner_840B_300 | 51 | | Model Class | NerDLModel | 52 | | Spark Compatibility | 2.6.0 | 53 | | Spark NLP Compatibility | 2.4 | 54 | | License | open source | 55 | | Edition | public | 56 | | Input Labels | | 57 | | Output Labels | | 58 | | Language | fi | 59 | | Dimension | | 60 | | Case Sensitive | | 61 | | Upstream Dependencies | glove_840B_300 | 62 | 63 | 64 | 65 | 66 | {:.h2_title} 67 | ## Data Source 68 | 69 | 70 | 71 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-09-03-re_drug_drug_interaction_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Relation Extraction Model Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-09-03 10 | tags: [clinical,relation,extraction,drug,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | 21 | 22 | {:.btn-box} 23 |
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/10.Clinical_Relation_Extraction.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/re_drug_drug_interaction_clinical_en_2.5.5_2.4_1599156924424.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
24 | 25 | ## How to use 26 |
27 | 28 | {% include programmingLanguageSelectScalaPython.html %} 29 | 30 | ```python 31 | 32 | ``` 33 | 34 | ```scala 35 | 36 | ``` 37 |
38 | 39 | 40 | 41 | {:.model-param} 42 | ## Model Information 43 | {:.table-model} 44 | |-------------------------|-----------------------------------------| 45 | | Model Name | re_drug_drug_interaction_clinical | 46 | | Model Class | RelationExtractionModel | 47 | | Spark Compatibility | 2.5.5 | 48 | | Spark NLP Compatibility | 2.4 | 49 | | License | Licensed | 50 | | Edition | Official | 51 | | Input Labels | word_embeddings, chunk, pos, dependency | 52 | | Output Labels | category | 53 | | Language | en | 54 | | Case Sensitive | False | 55 | | Upstream Dependencies | embeddings_clinical | 56 | 57 | 58 | 59 | 60 | 61 | {:.h2_title} 62 | ## Data Source 63 | Trained on data gathered and manually annotated by John Snow Labs. 64 | 65 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-09-06-chunkresolve_ICD10GM_de.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: ChunkResolver ICD10GM 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-09-06 10 | tags: [clinical,entity_resolution,icd10,de] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Codes and their normalized definition with `clinical_embeddings` 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/ER_ICD10_GM_DE/){:.button.button-orange}

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/chunkresolve_ICD10GM_de_2.5.5_2.4_1599431635423.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|--------------------------| 47 | | Model Name | chunkresolve_ICD10GM | 48 | | Model Class | ChunkEntityResolverModel | 49 | | Spark Compatibility | 2.5.5 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | token, chunk_embeddings | 54 | | Output Labels | entity | 55 | | Language | de | 56 | | Case Sensitive | True | 57 | | Upstream Dependencies | w2v_cc_300d | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | FILLUP. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-09-06-w2v_cc_300d_de.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Fastext Word Embeddings in German 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-09-06 10 | tags: [] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Word2Vec feature vectors based on w2v_cc_300d 23 | 24 | {:.btn-box} 25 |
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/14.German_Healthcare_Models.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/w2v_cc_300d_de_2.5.5_2.4_1599428063692.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|---------------------| 47 | | Model Name | w2v_cc_300d | 48 | | Model Class | WordEmbeddingsModel | 49 | | Spark Compatibility | 2.5.5 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, token | 54 | | Output Labels | word_embeddings | 55 | | Language | de | 56 | | Dimension | 300.0 | 57 | 58 | 59 | 60 | 61 | 62 | {:.h2_title} 63 | ## Data Source 64 | FastText common crawl word embeddings for Germany. 65 | 66 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-09-07-ner_legal_de.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: NER DL Model Legal 4 | author: John Snow Labs 5 | name: ner_legal 6 | class: NerDLModel 7 | language: de 8 | repository: clinical/models 9 | date: 07/09/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | AN,EUN,GRT,GS,INN,LD,LDS,LIT,MRK,ORG,PER,RR,RS,ST,STR,UN,VO,VS,VT 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/NER_LEGAL_DE/){:.button.button-orange}
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/15.German_Legal_Model.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_legal_de_2.5.5_2.4_1599471454959.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|-------------------------------------------------------------------| 52 | | Model Name | ner_legal | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.5.5 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Legal | 58 | | Input Labels | | 59 | | Output Labels | AN,EUN,GRT,GS,INN,LD,LDS,LIT,MRK,ORG,PER,RR,RS,ST,STR,UN,VO,VS,VT | 60 | | Language | de | 61 | | Dimension | | 62 | | Case Sensitive | | 63 | | Upstream Dependencies | embeddings_clinical | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-09-08-ner_dl_bert_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: 4 | author: John Snow Labs 5 | name: ner_dl_bert 6 | class: NerDLModel 7 | language: en 8 | repository: public/models 9 | date: 08/09/2020 10 | tags: [ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | 21 | 22 | {:.btn-box} 23 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/models/ner_dl_bert_en_2.0.2_2.4_1599550979101.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
24 | 25 | ## How to use 26 |
27 | 28 | {% include programmingLanguageSelectScalaPython.html %} 29 | 30 | ```python 31 | 32 | ``` 33 | 34 | ```scala 35 | 36 | ``` 37 |
38 | 39 | {:.h2_title} 40 | ## Results 41 | ```bash 42 | 43 | ``` 44 | 45 | {:.model-param} 46 | ## Model Information 47 | 48 | {:.table-model} 49 | |-------------------------|---------------| 50 | | Model Name | ner_dl_bert | 51 | | Model Class | NerDLModel | 52 | | Spark Compatibility | 2.0.2 | 53 | | Spark NLP Compatibility | 2.4 | 54 | | License | open source | 55 | | Edition | public | 56 | | Input Labels | | 57 | | Output Labels | | 58 | | Language | en | 59 | | Dimension | | 60 | | Case Sensitive | | 61 | | Upstream Dependencies | NER with BERT | 62 | 63 | 64 | 65 | 66 | {:.h2_title} 67 | ## Data Source 68 | 69 | 70 | 71 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-09-16-chunkresolve_athena_conditions_healthcare_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: ChunkResolver Athena Conditions Healthcare 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-09-16 10 | tags: [clinical,entity_resolution,athena,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | Athena Codes and their normalized definition 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/chunkresolve_athena_conditions_healthcare_en_2.6.0_2.4_1600265258887.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|-------------------------------------------| 47 | | Model Name | chunkresolve_athena_conditions_healthcare | 48 | | Model Class | ChunkEntityResolverModel | 49 | | Spark Compatibility | 2.6.0 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | token, chunk_embeddings | 54 | | Output Labels | entity | 55 | | Language | en | 56 | | Case Sensitive | True | 57 | | Upstream Dependencies | embeddings_healthcare_100d | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on Athena dataset. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-09-23-assertion_dl_healthcare_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Assertion DL Healthcare Embeddings 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-09-23 10 | tags: [clinical,assertion] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Deep learning named entity recognition model for assertions. 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | hypothetical, present, absent, possible, conditional, associated_with_someone_else 23 | 24 | {:.btn-box} 25 |
[Open in Colab](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/2.Clinical_Assertion_Model.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/assertion_dl_healthcare_en_2.6.0_2.4_1600849811713.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|----------------------------------| 47 | | Model Name | assertion_dl_healthcare | 48 | | Model Class | AssertionDLModel | 49 | | Spark Compatibility | 2.6.0 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | document, chunk, word_embeddings | 54 | | Output Labels | assertion | 55 | | Language | en | 56 | | Case Sensitive | False | 57 | | Upstream Dependencies | embeddings_healthcare_100d | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text with `embeddings_clinical`. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-09-24-re_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: Relation Extraction Model Clinical 4 | author: John Snow Labs 5 | name: 6 | class: 7 | language: 8 | repository: clinical/models 9 | date: 2020-09-24 10 | tags: [clinical,relation,extraction,en] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Models the set of clinical relations defined in the 2010 i2b2 relation challenge. 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | TrIP (improved), TrWP (worsened), TrCP (caused problem), TrAP (administered), TrNAP (avoided), TeRP (revealed problem), TeCP (investigate problem), PIP (problems related) 23 | 24 | {:.btn-box} 25 | [Live Demo](https://demo.johnsnowlabs.com/healthcare/RE_CLINICAL/){:.button.button-orange}
[Open in Colab](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/10.Clinical_Relation_Extraction.ipynb){:.button.button-orange.button-orange-trans.co.button-icon}
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/re_clinical_en_2.5.5_2.4_1600987935304.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | 42 | 43 | {:.model-param} 44 | ## Model Information 45 | {:.table-model} 46 | |-------------------------|-----------------------------------------| 47 | | Model Name | re_clinical | 48 | | Model Class | RelationExtractionModel | 49 | | Spark Compatibility | 2.5.5 | 50 | | Spark NLP Compatibility | 2.4 | 51 | | License | Licensed | 52 | | Edition | Official | 53 | | Input Labels | word_embeddings, chunk, pos, dependency | 54 | | Output Labels | category | 55 | | Language | en | 56 | | Case Sensitive | False | 57 | | Upstream Dependencies | embeddings_clinical | 58 | 59 | 60 | 61 | 62 | 63 | {:.h2_title} 64 | ## Data Source 65 | Trained on data gathered and manually annotated by John Snow Labs. 66 | 67 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-10-06-ner_ade_biobert_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: NER Adverse Drug Events 4 | author: John Snow Labs 5 | name: ner_ade_biobert 6 | class: NerDLModel 7 | language: en 8 | repository: clinical/models 9 | date: 06/10/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Extract adverse drug reaction events and drug entites from text 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | ADE, DRUG 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_ade_biobert_en_2.6.0_2.4_1601594787264.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|---------------------------| 52 | | Model Name | ner_ade_biobert | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.6.2 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | ADE, DRUG | 60 | | Language | en | 61 | | Dimension | | 62 | | Case Sensitive | | 63 | | Upstream Dependencies | biobert_pubmed_base_cased | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | Trained on DRUG-AE, 2018 i2b2, CADEC, and twitter ADE dataset 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-10-06-ner_ade_clinical_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: NER Adverse Drug Events 4 | author: John Snow Labs 5 | name: ner_ade_clinical 6 | class: NerDLModel 7 | language: en 8 | repository: clinical/models 9 | date: 06/10/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Extract adverse drug reaction events and drug entites from text 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | ADE, DRUG 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_ade_clinical_en_2.6.0_2.4_1601368505818.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|---------------------| 52 | | Model Name | ner_ade_clinical | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.6.2 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | ADE, DRUG | 60 | | Language | en | 61 | | Dimension | | 62 | | Case Sensitive | | 63 | | Upstream Dependencies | embeddings_clinical | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | Trained on DRUG-AE, 2018 i2b2, CADEC, and twitter ADE dataset 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-10-06-ner_ade_clinicalbert_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: NER Adverse Drug Events 4 | author: John Snow Labs 5 | name: ner_ade_clinicalbert 6 | class: NerDLModel 7 | language: en 8 | repository: clinical/models 9 | date: 06/10/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Extract adverse drug reaction events and drug entites from text 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | ADE, DRUG 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_ade_clinicalbert_en_2.6.0_2.4_1601594831715.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|-----------------------------| 52 | | Model Name | ner_ade_clinicalbert | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.6.2 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | ADE, DRUG | 60 | | Language | en | 61 | | Dimension | | 62 | | Case Sensitive | | 63 | | Upstream Dependencies | biobert_clinical_base_cased | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | Trained on DRUG-AE, 2018 i2b2, CADEC, and twitter ADE dataset 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/output/2020-10-06-ner_ade_healthcare_en.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: NER Adverse Drug Events 4 | author: John Snow Labs 5 | name: ner_ade_healthcare 6 | class: NerDLModel 7 | language: en 8 | repository: clinical/models 9 | date: 06/10/2020 10 | tags: [clinical,ner] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | Extract adverse drug reaction events and drug entites from text 19 | 20 | {:.h2_title} 21 | ## Predicted Entities 22 | ADE, DRUG 23 | 24 | {:.btn-box} 25 |

[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/ner_ade_healthcare_en_2.6.0_2.4_1601450601043.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
26 | 27 | ## How to use 28 |
29 | 30 | {% include programmingLanguageSelectScalaPython.html %} 31 | 32 | ```python 33 | 34 | ``` 35 | 36 | ```scala 37 | 38 | ``` 39 |
40 | 41 | {:.h2_title} 42 | ## Results 43 | ```bash 44 | 45 | ``` 46 | 47 | {:.model-param} 48 | ## Model Information 49 | 50 | {:.table-model} 51 | |-------------------------|----------------------------| 52 | | Model Name | ner_ade_healthcare | 53 | | Model Class | NerDLModel | 54 | | Spark Compatibility | 2.6.2 | 55 | | Spark NLP Compatibility | 2.4 | 56 | | License | Licensed | 57 | | Edition | Healthcare | 58 | | Input Labels | | 59 | | Output Labels | ADE, DRUG | 60 | | Language | en | 61 | | Dimension | | 62 | | Case Sensitive | | 63 | | Upstream Dependencies | embeddings_healthcare_100d | 64 | 65 | 66 | 67 | 68 | {:.h2_title} 69 | ## Data Source 70 | 71 | Trained on DRUG-AE, 2018 i2b2, CADEC, and twitter ADE dataset 72 | 73 | -------------------------------------------------------------------------------- /python/docs_module/templates/model.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: model 3 | title: {{title}} 4 | author: {{author}} 5 | name: {{name}} 6 | class: {{model_class}} 7 | language: {{language}} 8 | repository: {{repo}} 9 | date: {{latest_date}} 10 | tags: [{{tags}}] 11 | article_header: 12 | type: cover 13 | use_language_switcher: "Python-Scala-Java" 14 | --- 15 | 16 | {:.h2_title} 17 | ## Description 18 | {{description}} 19 | 20 | {% if labels %} {:.h2_title} 21 | ## Predicted Entities 22 | {{labels}} {% endif %} 23 | 24 | {{buttons}} 25 | 26 | ## How to use 27 |
28 | {% raw %} 29 | {% include programmingLanguageSelectScalaPython.html %} 30 | {% endraw %} 31 | ```python 32 | {{python_sample}} 33 | ``` 34 | 35 | ```scala 36 | {{scala_sample}} 37 | ``` 38 |
39 | 40 | {# 41 | {:.h2_title} 42 | ## Results 43 | {{class_annotation_sample}} 44 | 45 | {:.result_box} 46 | ```python 47 | {{model_output_schema}} 48 | ``` 49 | #} 50 | 51 | {:.model-param} 52 | ## Model Information 53 | 54 | {:.table-model} 55 | {{table}} 56 | 57 | {% if included_models %} 58 | {:.h2_title} 59 | ## Included Models 60 | {{included_models}} 61 | {% else %} 62 | {%endif%} 63 | 64 | {:.h2_title} 65 | ## Data Source 66 | {{dataset_info}} 67 | Visit [this]({{reference_url}}) link to get more information 68 | 69 | {% if model_benchmarks %} 70 | {:.h2_title} 71 | ## Benchmarking 72 | {{model_benchmarks}} 73 | {%endif%} 74 | -------------------------------------------------------------------------------- /release-template.md: -------------------------------------------------------------------------------- 1 | ## Model or model pack description: 2 | 3 | ### BioBERT models pack: 4 | 5 | We are very excited to share these 5 new BioBERT models with our enterprise users! 6 | 7 | | Model | name | language | loc | 8 | |----------------------------------------|---------------|---------------|---------------| 9 | |BertEmbeddingsModel | `biobert_pubmed_cased`|en|clinical/models| 10 | |BertEmbeddingsModel | `biobert_pmc_cased`|en|clinical/models| 11 | |BertEmbeddingsModel | `biobert_pubmed_pmc_cased`|en|clinical/models| 12 | |BertEmbeddingsModel | `biobert_clinical_cased`|en|clinical/models| 13 | |BertEmbeddingsModel | `biobert_discharge_cased`|en|clinical/models| 14 | 15 | The first 3 models `biobert_pubmed_cased`, `biobert_pmc_cased`, and `biobert_pubmed_pmc_cased` are thanks to [BioBERT](https://github.com/naver/biobert-pretrained) pretrained models from their paper: https://arxiv.org/abs/1901.08746 16 | And the last two models `biobert_clinical_cased` and `biobert_discharge_cased` are from another amazing release called [clinicalBERT](https://github.com/EmilyAlsentzer/clinicalBERT) from their paper: https://www.aclweb.org/anthology/W19-1909/ 17 | 18 | #### Spark NLP Version: 19 | - [x] HEALTHCARE 20 | - [ ] PUBLIC 21 | 22 | ### Last update 23 | -- DATE 24 | ### Last update 25 | -- NOTES 26 | ### WORKS WITH: 27 | -- 2.3.x and above 28 | ### Link 29 | -- to workshop example 30 | -------------------------------------------------------------------------------- /training/lemmatizer/README.md: -------------------------------------------------------------------------------- 1 | # Lemmatizer 2 | 3 | How to train Spark NLP `Lemmatizer` annotator: 4 | 5 | ```scala 6 | val lemmatizer = new Lemmatizer() 7 | .setInputCols(Array("token")) 8 | .setOutputCol("lemma") 9 | .setDictionary("AntBNC_lemmas_ver_001.txt", "->", "\t") 10 | ``` 11 | 12 | The file must have the following format where the `keyDelimiter` in this case is `->` and the `valueDelimiter` is `\t`: 13 | 14 | ```bash 15 | abnormal -> abnormal abnormals 16 | abode -> abode abodes 17 | abolish -> abolishing abolished abolish abolishes 18 | abolitionist -> abolitionist abolitionists 19 | abominate -> abominate abominated abominates 20 | abomination -> abomination abominations 21 | aboriginal -> aboriginal aboriginals 22 | aborigine -> aborigines aborigine 23 | abort -> aborted abort aborts aborting 24 | abortifacient -> abortifacients abortifacient 25 | abortionist -> abortionist abortionists 26 | abortion -> abortion abortions 27 | abo -> abo abos 28 | abotrite -> abotrites abotrite 29 | abound -> abound abounds abounding abounded 30 | ``` 31 | 32 | > NOTE: For now, the `Lemmatizer` uses path to a file instead of a DataFrame. So any DataFrame iside `.fit()` will be ignored for this annotator. 33 | -------------------------------------------------------------------------------- /training/ner_dl/README.md: -------------------------------------------------------------------------------- 1 | # NerDLApproach 2 | 3 | To train Named Entity Recognition (NER) model by Spark NLP we use `NerDLApproach` annotator. 4 | 5 | ## Data prepration 6 | 7 | To prepare our training dataset and test dataset (optional), we use a class called `CoNLL()` to transform our CoNLL files (IOB and IOB2). 8 | 9 | Here is an example for CoNLL 2003 `eng.train`: 10 | 11 | ```scala 12 | import com.johnsnowlabs.nlp.training._ 13 | import com.johnsnowlabs.nlp.annotator._ 14 | import com.johnsnowlabs.nlp.base._ 15 | 16 | val conll = CoNLL() 17 | val training_data = conll.readDataset(spark, "/conll2003/eng.train") 18 | 19 | training_data.show(2) 20 | 21 | +--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+ 22 | | text| document| sentence| token| pos| label| 23 | +--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+ 24 | |EU rejects German...|[[document, 0, 27...|[[document, 0, 47...|[[token, 0, 1, EU...|[[pos, 0, 1, NNP,...|[[named_entity, 0...| 25 | |Rare Hendrix song...|[[document, 0, 96...|[[document, 0, 50...|[[token, 0, 3, Ra...|[[pos, 0, 3, NNP,...|[[named_entity, 0...| 26 | +--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+ 27 | ``` 28 | 29 | Now that we have our training dataset with all the required columns, we can transform it by using `WordEmbeddingsModel` or `BertEmbeddings` to another DataFrame that has an extra column for word embeddings. 30 | 31 | Her we use pre-trained WordEmbeddingsModel `GloVe 100d`: 32 | 33 | ```Scala 34 | val embeddings = WordEmbeddingsModel.pretrained() 35 | .setInputCols("sentence", "token") 36 | .setOutputCol("embeddings") 37 | .setCaseSensitive(false) 38 | 39 | val readyTrainingData = embeddings.transform(training_data) 40 | 41 | 42 | // Optional: You can save the result on disk if the DataFrame is too large. 43 | 44 | readyTrainingData.write.mode("Overwrite").parquet("/tmp/conll2003/GloVeCoNLL2003_6B_100_train") 45 | 46 | ``` 47 | 48 | Now we can start training our `NerDLModel`: 49 | 50 | ```scala 51 | 52 | // In case you saved it on disk, let's read it back first 53 | val readyTrainingData = spark.read.parquet("/tmp/conll2003/GloVeCoNLL2003_6B_100_train") 54 | 55 | val ner = new NerDLApproach() 56 | .setInputCols("sentence", "token", "embeddings") 57 | .setOutputCol("ner") 58 | .setLabelColumn("label") 59 | .setOutputCol("ner") 60 | .setLr(1e-3f) //0.001 61 | .setPo(5e-3f) //0.005 62 | .setDropout(5e-1f) //0.5 63 | .setBatchSize(128) 64 | .setMaxEpochs(50) 65 | .setRandomSeed(0) 66 | .setVerbose(0) 67 | .setEvaluationLogExtended(true) 68 | 69 | val myNerModel = ner.fit(readyTrainingData) 70 | 71 | myNerModel.write.save("/tmp/NerDLModel_conll2003") 72 | 73 | ``` 74 | 75 | You can later on use your `NerDLModel` inside any pipeline by simply loading it: 76 | 77 | ```scala 78 | 79 | val ner = NerDLModel.load("/tmp/NerDLModel_conll2003") 80 | 81 | ``` 82 | -------------------------------------------------------------------------------- /training/part_of_speech/README.md: -------------------------------------------------------------------------------- 1 | # PerceptronApproach 2 | 3 | To train Part of Speech model by Spark NLP we use `PerceptronApproach` annotator. 4 | 5 | ```scala 6 | val posTagger = new PerceptronApproach() 7 | .setNIterations(6) 8 | .setInputCols(Array("sentence", "token")) 9 | .setOutputCol("pos") 10 | .setPosColumn("tags") 11 | ``` 12 | 13 | The importan part in training POS model is `tags` column which must be generated by the following class: 14 | 15 | ```scala 16 | val trainingDataset = POS().readDataset(ResourceHelper.spark, "pos-corpus/anc", "|", "tags") 17 | ``` 18 | 19 | > NOTE: You can pass an existing SparkSession if you are using Zeppelin or Jupyter by just mentioning `spark` or use `ResourceHelper.spark` to create a new SparkSession 20 | 21 | Where inside `pos-corpus/anc` there are text files with `token|tag` formating: 22 | 23 | ```bash 24 | To|TO help|VB you|PRP see|VB how|WRB much|JJ your|PRP$ contribution|NN means|VBZ ,|, I|PRP 'm|VBP sharing|VBG with|IN you|PRP The|DT words|NNS of|IN people|NNS who|WP have|VBP lived|VBN Goodwill|NNP 's|POS mission|NN .|. 25 | We|PRP want|VBP you|PRP to|TO Know|VBP why|WRB your|PRP$ support|NN of|IN Goodwill|NNP is|VBZ so|RB important|JJ .|. 26 | ``` 27 | 28 | Now we can train the POS tagger model: 29 | 30 | ```scala 31 | val posTaggerModel = posTagger.fit(trainingDataset) 32 | ``` 33 | --------------------------------------------------------------------------------