├── LICENSE ├── README.md ├── attention.py ├── bert_model.py ├── bert_trainer.py ├── config.py ├── dataset.py ├── datasets └── dummy_data │ └── test │ └── neg │ ├── text1.txt │ ├── text2.txt │ ├── text3.txt │ ├── text4.txt │ ├── text5.txt │ ├── text6.txt │ └── text7.txt ├── demo ├── DEMO.md └── bert_demo.ipynb ├── embeddings.py ├── encoder.py ├── feed_forward.py ├── positional_embeddings.py ├── requirements.txt ├── run.py ├── scheduled_optim.py ├── tmp └── README.md ├── tokenizer.py ├── tokenizer └── README.md └── word_piece_trainer.py /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AliHaiderAhmad001/BERT-from-Scratch-with-PyTorch/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AliHaiderAhmad001/BERT-from-Scratch-with-PyTorch/HEAD/README.md -------------------------------------------------------------------------------- /attention.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AliHaiderAhmad001/BERT-from-Scratch-with-PyTorch/HEAD/attention.py -------------------------------------------------------------------------------- /bert_model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AliHaiderAhmad001/BERT-from-Scratch-with-PyTorch/HEAD/bert_model.py -------------------------------------------------------------------------------- /bert_trainer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AliHaiderAhmad001/BERT-from-Scratch-with-PyTorch/HEAD/bert_trainer.py -------------------------------------------------------------------------------- /config.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AliHaiderAhmad001/BERT-from-Scratch-with-PyTorch/HEAD/config.py -------------------------------------------------------------------------------- /dataset.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AliHaiderAhmad001/BERT-from-Scratch-with-PyTorch/HEAD/dataset.py -------------------------------------------------------------------------------- /datasets/dummy_data/test/neg/text1.txt: -------------------------------------------------------------------------------- 1 | t1 2 | -------------------------------------------------------------------------------- /datasets/dummy_data/test/neg/text2.txt: -------------------------------------------------------------------------------- 1 | t2 2 | -------------------------------------------------------------------------------- /datasets/dummy_data/test/neg/text3.txt: -------------------------------------------------------------------------------- 1 | t3 2 | -------------------------------------------------------------------------------- /datasets/dummy_data/test/neg/text4.txt: -------------------------------------------------------------------------------- 1 | t4 2 | -------------------------------------------------------------------------------- /datasets/dummy_data/test/neg/text5.txt: -------------------------------------------------------------------------------- 1 | t5 2 | -------------------------------------------------------------------------------- /datasets/dummy_data/test/neg/text6.txt: -------------------------------------------------------------------------------- 1 | t6 2 | -------------------------------------------------------------------------------- /datasets/dummy_data/test/neg/text7.txt: -------------------------------------------------------------------------------- 1 | t7 2 | -------------------------------------------------------------------------------- /demo/DEMO.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AliHaiderAhmad001/BERT-from-Scratch-with-PyTorch/HEAD/demo/DEMO.md -------------------------------------------------------------------------------- /demo/bert_demo.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AliHaiderAhmad001/BERT-from-Scratch-with-PyTorch/HEAD/demo/bert_demo.ipynb -------------------------------------------------------------------------------- /embeddings.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AliHaiderAhmad001/BERT-from-Scratch-with-PyTorch/HEAD/embeddings.py -------------------------------------------------------------------------------- /encoder.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AliHaiderAhmad001/BERT-from-Scratch-with-PyTorch/HEAD/encoder.py -------------------------------------------------------------------------------- /feed_forward.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AliHaiderAhmad001/BERT-from-Scratch-with-PyTorch/HEAD/feed_forward.py -------------------------------------------------------------------------------- /positional_embeddings.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AliHaiderAhmad001/BERT-from-Scratch-with-PyTorch/HEAD/positional_embeddings.py -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | pip install transformers 2 | -------------------------------------------------------------------------------- /run.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AliHaiderAhmad001/BERT-from-Scratch-with-PyTorch/HEAD/run.py -------------------------------------------------------------------------------- /scheduled_optim.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AliHaiderAhmad001/BERT-from-Scratch-with-PyTorch/HEAD/scheduled_optim.py -------------------------------------------------------------------------------- /tmp/README.md: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /tokenizer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AliHaiderAhmad001/BERT-from-Scratch-with-PyTorch/HEAD/tokenizer.py -------------------------------------------------------------------------------- /tokenizer/README.md: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /word_piece_trainer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AliHaiderAhmad001/BERT-from-Scratch-with-PyTorch/HEAD/word_piece_trainer.py --------------------------------------------------------------------------------