TextVectorizationΒΆ

A preprocessing layer which maps text features to integer sequences.

Abstract Signature:

TextVectorization(max_tokens: int, standardize: str = lower_and_strip_punctuation, split: str = whitespace, ngrams: int, output_mode: str = int, output_sequence_length: int, pad_to_max_tokens: bool = False, vocabulary: List[str])

PyTorch

API: β€”
Strategy: Plugin (text_vectorization_shim)

Keras

API: keras.layers.TextVectorization
Strategy: Direct Mapping

TensorFlow

API: tf.keras.layers.TextVectorization
Strategy: Direct Mapping