|
HFST - Helsinki Finite-State Transducer Technology - Python API
version 3.12.3 (under development)
|
|
HFST - Helsinki Finite-State Transducer Technology - Python API
version 3.12.3 (under development)
|
This is the complete list of members for HfstTokenizer, including all inherited members.
| __init__(self) | HfstTokenizer | |
| add_multichar_symbol(self, symbol) | HfstTokenizer | |
| add_skip_symbol(self, symbol) | HfstTokenizer | |
| check_utf8_correctness(input_string) | HfstTokenizer | |
| tokenize(self, input_string) | HfstTokenizer | |
| tokenize(self, input_string, output_string) | HfstTokenizer | |
| tokenize_one_level(self, input_string) | HfstTokenizer | |
| tokenize_space_separated(self, str) | HfstTokenizer |
1.8.11
