discourse-ai/tokenizers
Rafael dos Santos Silva 9783e3b025
FEATURE: Add a basic tokenizer API (#37)
* FEATURE: Add a basic tokenizer API

* Add tests

* lint
2023-04-19 11:55:59 -03:00
..
LICENSE FEATURE: Add a basic tokenizer API (#37) 2023-04-19 11:55:59 -03:00
bert-base-uncased.json FEATURE: Add a basic tokenizer API (#37) 2023-04-19 11:55:59 -03:00