tokenizer

Tokenizer module for cross-encoder models.

This module contains the tokenizer class cross-encoder models.

Classes

CrossEncoderTokenizer(*args[, query_length, ...])