HFST - Helsinki Finite-State Transducer Technology - C++ API: /home/eaxelson/hfst-git/hfst/libhfst/src/HfstTokenizer.h File Reference

HFST - Helsinki Finite-State Transducer Technology - C++ API version 3.9.1

All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages

Declaration of class hfst::HfstTokenizer. More...

#include "HfstSymbolDefs.h"
#include "HfstExceptionDefs.h"
#include <iostream>
#include <climits>
#include <string>
#include "hfstdll.h"

Go to the source code of this file.

Classes
class	HfstTokenizer
	A tokenizer for creating transducers from UTF-8 strings. More...

Namespaces
	hfst
	A namespace for HFST functions and datatypes.

Detailed Description

Declaration of class hfst::HfstTokenizer.