├── .DS_Store ├── .gitattributes ├── .gitignore ├── Idiom_Classifier ├── 00_Data │ ├── README.md │ └── TroFi_dataset.pkl ├── 01_English_Idiom_Classifier │ ├── BERT_Idiom_pytorch.ipynb │ ├── README.md │ └── TroFi-EDA.ipynb ├── 02_Korean_Idiom_Classifier │ ├── BERT(multi_lingual)_Idiom_Classifier_Final.ipynb │ ├── KoBERT_idiom_classifier.ipynb │ └── README.md ├── 03_Final_Model │ ├── KoBERT_idiom_classifier_by_oversampling.ipynb │ ├── KoGPT2_Idiom_classifier.py │ ├── README.md │ ├── final_Idiom_Classifier_for_KoGPT2.ipynb │ ├── final_Idiom_Classifier_for_KoGPT2_with_oversampling.ipynb │ ├── final_idiom_classifier_for_kogpt2.py │ └── final_idiom_classifier_for_kogpt2_with_oversampling.py ├── 04_etc │ ├── 20210902_BERT_Idiom_Classifier_kor01.ipynb │ ├── 20211016_korpora_3_KoGPT2_(oversampling)idiom_classification.ipynb │ ├── 20211016_korpora_3_kogpt2_(oversampling)idiom_classification.py │ └── README.md ├── 20210902_BERT_Idiom_Classifier_kor01.ipynb └── test │ └── test.txt ├── Open_NMT ├── .gitattributes ├── openNMT │ ├── kiss_bpe_2layer_lstm.yaml │ ├── kiss_bpe_4layer_lstm.yaml │ ├── kiss_spm_6layer_transformer.yaml │ ├── make_spm_subword.py │ ├── openNMT_prepare_and_train.ipynb │ └── openNMT_translate_and_evaluation.ipynb └── preprocess │ ├── K940_idmtokening_preprocessing.ipynb │ ├── combine_and_train_test_val_split.ipynb │ ├── making_handmade(val_train).ipynb │ └── re_idm(space_deleated).ipynb ├── README.md ├── requirements.txt └── tree_project ├── __pycache__ └── translate.cpython-38.pyc ├── main.py ├── static ├── css │ └── style.css ├── img │ ├── Tree.gif │ ├── loading.gif │ └── loading.png ├── js │ └── script.js └── model │ ├── model_step_200000.pt │ ├── subword_eng.model │ └── subword_kor.model ├── templates └── index.html └── translate.py /.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/.DS_Store -------------------------------------------------------------------------------- /.gitattributes: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/.gitattributes -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/.gitignore -------------------------------------------------------------------------------- /Idiom_Classifier/00_Data/README.md: -------------------------------------------------------------------------------- 1 | # 2021_NLP_Project 2 | -------------------------------------------------------------------------------- /Idiom_Classifier/00_Data/TroFi_dataset.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Idiom_Classifier/00_Data/TroFi_dataset.pkl -------------------------------------------------------------------------------- /Idiom_Classifier/01_English_Idiom_Classifier/BERT_Idiom_pytorch.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Idiom_Classifier/01_English_Idiom_Classifier/BERT_Idiom_pytorch.ipynb -------------------------------------------------------------------------------- /Idiom_Classifier/01_English_Idiom_Classifier/README.md: -------------------------------------------------------------------------------- 1 | # 2021_NLP_Project 2 | -------------------------------------------------------------------------------- /Idiom_Classifier/01_English_Idiom_Classifier/TroFi-EDA.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Idiom_Classifier/01_English_Idiom_Classifier/TroFi-EDA.ipynb -------------------------------------------------------------------------------- /Idiom_Classifier/02_Korean_Idiom_Classifier/BERT(multi_lingual)_Idiom_Classifier_Final.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Idiom_Classifier/02_Korean_Idiom_Classifier/BERT(multi_lingual)_Idiom_Classifier_Final.ipynb -------------------------------------------------------------------------------- /Idiom_Classifier/02_Korean_Idiom_Classifier/KoBERT_idiom_classifier.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Idiom_Classifier/02_Korean_Idiom_Classifier/KoBERT_idiom_classifier.ipynb -------------------------------------------------------------------------------- /Idiom_Classifier/02_Korean_Idiom_Classifier/README.md: -------------------------------------------------------------------------------- 1 | # 2021_NLP_Project 2 | -------------------------------------------------------------------------------- /Idiom_Classifier/03_Final_Model/KoBERT_idiom_classifier_by_oversampling.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Idiom_Classifier/03_Final_Model/KoBERT_idiom_classifier_by_oversampling.ipynb -------------------------------------------------------------------------------- /Idiom_Classifier/03_Final_Model/KoGPT2_Idiom_classifier.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Idiom_Classifier/03_Final_Model/KoGPT2_Idiom_classifier.py -------------------------------------------------------------------------------- /Idiom_Classifier/03_Final_Model/README.md: -------------------------------------------------------------------------------- 1 | # 2021_NLP_Project 2 | -------------------------------------------------------------------------------- /Idiom_Classifier/03_Final_Model/final_Idiom_Classifier_for_KoGPT2.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Idiom_Classifier/03_Final_Model/final_Idiom_Classifier_for_KoGPT2.ipynb -------------------------------------------------------------------------------- /Idiom_Classifier/03_Final_Model/final_Idiom_Classifier_for_KoGPT2_with_oversampling.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Idiom_Classifier/03_Final_Model/final_Idiom_Classifier_for_KoGPT2_with_oversampling.ipynb -------------------------------------------------------------------------------- /Idiom_Classifier/03_Final_Model/final_idiom_classifier_for_kogpt2.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Idiom_Classifier/03_Final_Model/final_idiom_classifier_for_kogpt2.py -------------------------------------------------------------------------------- /Idiom_Classifier/03_Final_Model/final_idiom_classifier_for_kogpt2_with_oversampling.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Idiom_Classifier/03_Final_Model/final_idiom_classifier_for_kogpt2_with_oversampling.py -------------------------------------------------------------------------------- /Idiom_Classifier/04_etc/20210902_BERT_Idiom_Classifier_kor01.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Idiom_Classifier/04_etc/20210902_BERT_Idiom_Classifier_kor01.ipynb -------------------------------------------------------------------------------- /Idiom_Classifier/04_etc/20211016_korpora_3_KoGPT2_(oversampling)idiom_classification.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Idiom_Classifier/04_etc/20211016_korpora_3_KoGPT2_(oversampling)idiom_classification.ipynb -------------------------------------------------------------------------------- /Idiom_Classifier/04_etc/20211016_korpora_3_kogpt2_(oversampling)idiom_classification.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Idiom_Classifier/04_etc/20211016_korpora_3_kogpt2_(oversampling)idiom_classification.py -------------------------------------------------------------------------------- /Idiom_Classifier/04_etc/README.md: -------------------------------------------------------------------------------- 1 | # 2021_NLP_Project 2 | -------------------------------------------------------------------------------- /Idiom_Classifier/20210902_BERT_Idiom_Classifier_kor01.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Idiom_Classifier/20210902_BERT_Idiom_Classifier_kor01.ipynb -------------------------------------------------------------------------------- /Idiom_Classifier/test/test.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Idiom_Classifier/test/test.txt -------------------------------------------------------------------------------- /Open_NMT/.gitattributes: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Open_NMT/.gitattributes -------------------------------------------------------------------------------- /Open_NMT/openNMT/kiss_bpe_2layer_lstm.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Open_NMT/openNMT/kiss_bpe_2layer_lstm.yaml -------------------------------------------------------------------------------- /Open_NMT/openNMT/kiss_bpe_4layer_lstm.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Open_NMT/openNMT/kiss_bpe_4layer_lstm.yaml -------------------------------------------------------------------------------- /Open_NMT/openNMT/kiss_spm_6layer_transformer.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Open_NMT/openNMT/kiss_spm_6layer_transformer.yaml -------------------------------------------------------------------------------- /Open_NMT/openNMT/make_spm_subword.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Open_NMT/openNMT/make_spm_subword.py -------------------------------------------------------------------------------- /Open_NMT/openNMT/openNMT_prepare_and_train.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Open_NMT/openNMT/openNMT_prepare_and_train.ipynb -------------------------------------------------------------------------------- /Open_NMT/openNMT/openNMT_translate_and_evaluation.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Open_NMT/openNMT/openNMT_translate_and_evaluation.ipynb -------------------------------------------------------------------------------- /Open_NMT/preprocess/K940_idmtokening_preprocessing.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Open_NMT/preprocess/K940_idmtokening_preprocessing.ipynb -------------------------------------------------------------------------------- /Open_NMT/preprocess/combine_and_train_test_val_split.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Open_NMT/preprocess/combine_and_train_test_val_split.ipynb -------------------------------------------------------------------------------- /Open_NMT/preprocess/making_handmade(val_train).ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Open_NMT/preprocess/making_handmade(val_train).ipynb -------------------------------------------------------------------------------- /Open_NMT/preprocess/re_idm(space_deleated).ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/Open_NMT/preprocess/re_idm(space_deleated).ipynb -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/README.md -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/requirements.txt -------------------------------------------------------------------------------- /tree_project/__pycache__/translate.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/tree_project/__pycache__/translate.cpython-38.pyc -------------------------------------------------------------------------------- /tree_project/main.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/tree_project/main.py -------------------------------------------------------------------------------- /tree_project/static/css/style.css: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/tree_project/static/css/style.css -------------------------------------------------------------------------------- /tree_project/static/img/Tree.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/tree_project/static/img/Tree.gif -------------------------------------------------------------------------------- /tree_project/static/img/loading.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/tree_project/static/img/loading.gif -------------------------------------------------------------------------------- /tree_project/static/img/loading.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/tree_project/static/img/loading.png -------------------------------------------------------------------------------- /tree_project/static/js/script.js: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/tree_project/static/js/script.js -------------------------------------------------------------------------------- /tree_project/static/model/model_step_200000.pt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/tree_project/static/model/model_step_200000.pt -------------------------------------------------------------------------------- /tree_project/static/model/subword_eng.model: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/tree_project/static/model/subword_eng.model -------------------------------------------------------------------------------- /tree_project/static/model/subword_kor.model: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/tree_project/static/model/subword_kor.model -------------------------------------------------------------------------------- /tree_project/templates/index.html: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/tree_project/templates/index.html -------------------------------------------------------------------------------- /tree_project/translate.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/itisused/2021_NLP_Project/HEAD/tree_project/translate.py --------------------------------------------------------------------------------