├── .docs ├── attention_heads.png ├── bert_embeddings.png ├── bert_fine_tune_pre_train.png ├── encoder.png ├── gelu.png ├── graph.png ├── model-pyTorch_BERT_model.svg ├── model.puml ├── res │ ├── mlm_loss.png │ ├── mlm_train_accuracy.png │ ├── nsp_loss.png │ ├── nsp_loss_smoothed.png │ └── nsp_train_accuracy.png └── the_process.png ├── .gitignore ├── .wheels └── torchtext-0.12.0a0+2683aa8-cp39-cp39-macosx_12_0_universal2.whl ├── README.md ├── bert ├── __init__.py ├── dataset.py ├── model.py └── trainer.py ├── graph.py ├── main.py └── requirements.txt /.docs/attention_heads.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/.docs/attention_heads.png -------------------------------------------------------------------------------- /.docs/bert_embeddings.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/.docs/bert_embeddings.png -------------------------------------------------------------------------------- /.docs/bert_fine_tune_pre_train.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/.docs/bert_fine_tune_pre_train.png -------------------------------------------------------------------------------- /.docs/encoder.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/.docs/encoder.png -------------------------------------------------------------------------------- /.docs/gelu.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/.docs/gelu.png -------------------------------------------------------------------------------- /.docs/graph.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/.docs/graph.png -------------------------------------------------------------------------------- /.docs/model-pyTorch_BERT_model.svg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/.docs/model-pyTorch_BERT_model.svg -------------------------------------------------------------------------------- /.docs/model.puml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/.docs/model.puml -------------------------------------------------------------------------------- /.docs/res/mlm_loss.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/.docs/res/mlm_loss.png -------------------------------------------------------------------------------- /.docs/res/mlm_train_accuracy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/.docs/res/mlm_train_accuracy.png -------------------------------------------------------------------------------- /.docs/res/nsp_loss.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/.docs/res/nsp_loss.png -------------------------------------------------------------------------------- /.docs/res/nsp_loss_smoothed.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/.docs/res/nsp_loss_smoothed.png -------------------------------------------------------------------------------- /.docs/res/nsp_train_accuracy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/.docs/res/nsp_train_accuracy.png -------------------------------------------------------------------------------- /.docs/the_process.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/.docs/the_process.png -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/.gitignore -------------------------------------------------------------------------------- /.wheels/torchtext-0.12.0a0+2683aa8-cp39-cp39-macosx_12_0_universal2.whl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/.wheels/torchtext-0.12.0a0+2683aa8-cp39-cp39-macosx_12_0_universal2.whl -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/README.md -------------------------------------------------------------------------------- /bert/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/bert/__init__.py -------------------------------------------------------------------------------- /bert/dataset.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/bert/dataset.py -------------------------------------------------------------------------------- /bert/model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/bert/model.py -------------------------------------------------------------------------------- /bert/trainer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/bert/trainer.py -------------------------------------------------------------------------------- /graph.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/graph.py -------------------------------------------------------------------------------- /main.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/main.py -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coaxsoft/pytorch_bert/HEAD/requirements.txt --------------------------------------------------------------------------------