├── .gitattributes ├── .gitignore ├── README.md ├── demo.ipynb └── wordpiecemodel ├── __init__.py └── bpe.py /.gitattributes: -------------------------------------------------------------------------------- 1 | *.ipynb linguist-vendored 2 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lovit/WordPieceModel/HEAD/.gitignore -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lovit/WordPieceModel/HEAD/README.md -------------------------------------------------------------------------------- /demo.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lovit/WordPieceModel/HEAD/demo.ipynb -------------------------------------------------------------------------------- /wordpiecemodel/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lovit/WordPieceModel/HEAD/wordpiecemodel/__init__.py -------------------------------------------------------------------------------- /wordpiecemodel/bpe.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lovit/WordPieceModel/HEAD/wordpiecemodel/bpe.py --------------------------------------------------------------------------------