default_tokenize {lime} | R Documentation |
This tokenizer uses stringi::stri_split_boundaries()
to tokenize a
character
vector. To be used with [explain.character()'.
default_tokenize(text)
text |
text to tokenize as a |
a character
vector.
data('train_sentences')
default_tokenize(train_sentences$text[1])