├── LanguageModelling ├── README.md ├── codemix_data │ ├── lan_model_1.py │ ├── preprocess_1.py │ ├── preprocess_2.py │ ├── preprocess_3.py │ ├── test_1.txt │ ├── test_2.txt │ ├── test_3.txt │ ├── test_data.json │ ├── train_1.txt │ ├── train_2.txt │ ├── train_3.txt │ └── train_data.json ├── lan_model.py ├── lan_model_1.py ├── monolingual_data │ ├── lan_model_1.py │ ├── perplexity.png │ ├── preprocess.py │ └── preprocess1.py ├── perplexity.png ├── preprocess.py └── preprocess1.py ├── Perplexity_CMI ├── 1_train_data.json ├── 2_1_preprocess.py ├── 2_2_CMI.py ├── 2_3_dataToChunks.py ├── 3_CMI_values.txt ├── 3_lang_tagged_text.txt ├── 3_text.txt ├── CMIv2programme │ ├── cmi_corpus_reader.py │ ├── cmi_stats_v2.py │ └── cmi_tagsets.py ├── CMUToolkit.tar.gz ├── Chunks │ ├── 0_10.txt │ ├── 10_20.txt │ ├── 20_30.txt │ ├── 30_40.txt │ ├── 40_50.txt │ ├── 50_60.txt │ ├── 60_70.txt │ └── 70_80.txt ├── Commands_Readme ├── Observation ├── README.md ├── a.json └── codemix_train.txt └── README.md /LanguageModelling/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/README.md -------------------------------------------------------------------------------- /LanguageModelling/codemix_data/lan_model_1.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/codemix_data/lan_model_1.py -------------------------------------------------------------------------------- /LanguageModelling/codemix_data/preprocess_1.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/codemix_data/preprocess_1.py -------------------------------------------------------------------------------- /LanguageModelling/codemix_data/preprocess_2.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/codemix_data/preprocess_2.py -------------------------------------------------------------------------------- /LanguageModelling/codemix_data/preprocess_3.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/codemix_data/preprocess_3.py -------------------------------------------------------------------------------- /LanguageModelling/codemix_data/test_1.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/codemix_data/test_1.txt -------------------------------------------------------------------------------- /LanguageModelling/codemix_data/test_2.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/codemix_data/test_2.txt -------------------------------------------------------------------------------- /LanguageModelling/codemix_data/test_3.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/codemix_data/test_3.txt -------------------------------------------------------------------------------- /LanguageModelling/codemix_data/test_data.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/codemix_data/test_data.json -------------------------------------------------------------------------------- /LanguageModelling/codemix_data/train_1.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/codemix_data/train_1.txt -------------------------------------------------------------------------------- /LanguageModelling/codemix_data/train_2.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/codemix_data/train_2.txt -------------------------------------------------------------------------------- /LanguageModelling/codemix_data/train_3.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/codemix_data/train_3.txt -------------------------------------------------------------------------------- /LanguageModelling/codemix_data/train_data.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/codemix_data/train_data.json -------------------------------------------------------------------------------- /LanguageModelling/lan_model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/lan_model.py -------------------------------------------------------------------------------- /LanguageModelling/lan_model_1.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/lan_model_1.py -------------------------------------------------------------------------------- /LanguageModelling/monolingual_data/lan_model_1.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/monolingual_data/lan_model_1.py -------------------------------------------------------------------------------- /LanguageModelling/monolingual_data/perplexity.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/monolingual_data/perplexity.png -------------------------------------------------------------------------------- /LanguageModelling/monolingual_data/preprocess.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/monolingual_data/preprocess.py -------------------------------------------------------------------------------- /LanguageModelling/monolingual_data/preprocess1.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/monolingual_data/preprocess1.py -------------------------------------------------------------------------------- /LanguageModelling/perplexity.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/perplexity.png -------------------------------------------------------------------------------- /LanguageModelling/preprocess.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/preprocess.py -------------------------------------------------------------------------------- /LanguageModelling/preprocess1.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/LanguageModelling/preprocess1.py -------------------------------------------------------------------------------- /Perplexity_CMI/1_train_data.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/1_train_data.json -------------------------------------------------------------------------------- /Perplexity_CMI/2_1_preprocess.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/2_1_preprocess.py -------------------------------------------------------------------------------- /Perplexity_CMI/2_2_CMI.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/2_2_CMI.py -------------------------------------------------------------------------------- /Perplexity_CMI/2_3_dataToChunks.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/2_3_dataToChunks.py -------------------------------------------------------------------------------- /Perplexity_CMI/3_CMI_values.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/3_CMI_values.txt -------------------------------------------------------------------------------- /Perplexity_CMI/3_lang_tagged_text.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/3_lang_tagged_text.txt -------------------------------------------------------------------------------- /Perplexity_CMI/3_text.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/3_text.txt -------------------------------------------------------------------------------- /Perplexity_CMI/CMIv2programme/cmi_corpus_reader.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/CMIv2programme/cmi_corpus_reader.py -------------------------------------------------------------------------------- /Perplexity_CMI/CMIv2programme/cmi_stats_v2.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/CMIv2programme/cmi_stats_v2.py -------------------------------------------------------------------------------- /Perplexity_CMI/CMIv2programme/cmi_tagsets.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/CMIv2programme/cmi_tagsets.py -------------------------------------------------------------------------------- /Perplexity_CMI/CMUToolkit.tar.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/CMUToolkit.tar.gz -------------------------------------------------------------------------------- /Perplexity_CMI/Chunks/0_10.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/Chunks/0_10.txt -------------------------------------------------------------------------------- /Perplexity_CMI/Chunks/10_20.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/Chunks/10_20.txt -------------------------------------------------------------------------------- /Perplexity_CMI/Chunks/20_30.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/Chunks/20_30.txt -------------------------------------------------------------------------------- /Perplexity_CMI/Chunks/30_40.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/Chunks/30_40.txt -------------------------------------------------------------------------------- /Perplexity_CMI/Chunks/40_50.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/Chunks/40_50.txt -------------------------------------------------------------------------------- /Perplexity_CMI/Chunks/50_60.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/Chunks/50_60.txt -------------------------------------------------------------------------------- /Perplexity_CMI/Chunks/60_70.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/Chunks/60_70.txt -------------------------------------------------------------------------------- /Perplexity_CMI/Chunks/70_80.txt: -------------------------------------------------------------------------------- 1 | 1 no bhau 2 | -------------------------------------------------------------------------------- /Perplexity_CMI/Commands_Readme: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/Commands_Readme -------------------------------------------------------------------------------- /Perplexity_CMI/Observation: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/Observation -------------------------------------------------------------------------------- /Perplexity_CMI/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/README.md -------------------------------------------------------------------------------- /Perplexity_CMI/a.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/a.json -------------------------------------------------------------------------------- /Perplexity_CMI/codemix_train.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/Perplexity_CMI/codemix_train.txt -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Abhishekmamidi123/Natural-Language-Processing/HEAD/README.md --------------------------------------------------------------------------------