HFST - Helsinki Finite-State Transducer Technology - Python API  version 3.12.3 (under development)
HfstTokenizer Member List

This is the complete list of members for HfstTokenizer, including all inherited members.

__init__(self)HfstTokenizer
add_multichar_symbol(self, symbol)HfstTokenizer
add_skip_symbol(self, symbol)HfstTokenizer
check_utf8_correctness(input_string)HfstTokenizer
tokenize(self, input_string)HfstTokenizer
tokenize(self, input_string, output_string)HfstTokenizer
tokenize_one_level(self, input_string)HfstTokenizer
tokenize_space_separated(self, str)HfstTokenizer