Add tokenizer configurations

This commit is contained in:
Thomas Wolf 2020-11-24 18:37:56 +01:00
parent 8197097fe8
commit 1a7dd4986e

3
tokenizer_config.json Normal file

@ -0,0 +1,3 @@
{
"do_lower_case": true
}