HfstBasicTransducer Class Reference

A simple transducer class with tropical weights. More...

Public Member Functions
def	__enumerate__
	Return an enumeration of the states and transitions of the transducer. More...
def	__init__
	Create a transducer with one initial state that has state number zero and is not a final state, i.e. More...
def	__init__
	Create a transducer equivalent to transducer. More...
def	__str__
	Return a string representation of the transducer. More...
def	add_state
	Add a new state to this transducer and return its number. More...
def	add_state
	Add a state s to this graph. More...
def	add_symbol_to_alphabet
	Explicitly add symbol to the alphabet of the graph. More...
def	add_symbols_to_alphabet
	Explicitly add symbols to the alphabet of the graph. More...
def	add_transition
	Add a transition transition to state state, add_symbols_to_alphabet defines whether the transition symbols are added to the alphabet. More...
def	add_transition
	Add a transition from state source to state target with input symbol input, output symbol output and weight weight. More...
def	disjunct
	Disjunct this transducer with a one-path transducer defined by consecutive string pairs in spv that has weight weight. More...
def	get_alphabet
	The symbols in the alphabet of the transducer. More...
def	get_final_weight
	Get the final weight of state state in this transducer. More...
def	get_max_state
	Get the biggest state number in use. More...
def	get_transition_pairs
	Get a list of all input/output symbol pairs used in the transitions of this transducer. More...
def	harmonize
	Harmonize this transducer and another. More...
def	insert_freely
	Insert freely any number of symbol_pair in the transducer with weight weight. More...
def	insert_freely
	Insert freely any number of transducer in this transducer. More...
def	is_final_state
	Whether state state is final. More...
def	is_infinitely_ambiguous
	Whether the transducer is infinitely ambiguous. More...
def	is_lookup_infinitely_ambiguous
	Whether the transducer is infinitely ambiguous with input str. More...
def	longest_path_size
	The length of the longest path in transducer. More...
def	lookup_fd
	Lookup tokenized input input in the transducer minding flag diacritics. More...
def	prune_alphabet
	Remove all symbols that do not occur in transitions of the transducer from its alphabet. More...
def	read_att
	Read a transducer in AT&T format from file f. More...
def	read_prolog
	Read a transducer from prolog file f. More...
def	remove_symbol_from_alphabet
	Remove symbol symbol from the alphabet of the graph. More...
def	remove_symbols_from_alphabet
	Remove symbols symbols from the alphabet of the graph. More...
def	remove_transition
	Remove transition transition from state s. More...
def	set_final_weight
	Set the final weight of state state in this transducer to weight. More...
def	sort_arcs
	Sort the arcs of this transducer according to input and output symbols. More...
def	states
	The states of the transducer. More...
def	states_and_transitions
	The states and transitions of the transducer. More...
def	substitute
	Substitute symbols or transitions in the transducer. More...
def	symbols_used
	Get a list of all symbols used in the transitions of this transducer. More...
def	transitions
	Get the transitions of state state in this transducer. More...
def	write_att
	Write this transducer in AT&T format to file f, write_weights defines whether weights are written. More...
def	write_prolog
	Write the transducer in prolog format to file f. More...
def	write_xfst
	Write the transducer in xfst format to file f. More...

Detailed Description

A simple transducer class with tropical weights.

An example of creating an HfstBasicTransducer [foo:bar baz:baz] with weight 0.4 from scratch:

  # Create an empty transducer
  # The transducer has initially one start state (number zero) 
  # that is not final
  fsm = hfst.HfstBasicTransducer()
  # Add two states to the transducer
  fsm.add_state(1)
  fsm.add_state(2)
  # Create a transition [foo:bar] leading to state 1 with weight 0.1
  tr = hfst.HfstBasicTransition(1, 'foo', 'bar', 0.1)
  # and add it to state zero
  fsm.add_transition(0, tr)
  # Add a transition [baz:baz] with weight 0 from state 1 to state 2 
  fsm.add_transition(1, hfst.HfstBasicTransition(2, 'baz', 'baz', 0.0))
  # Set state 2 as final with weight 0.3
  fsm.set_final_weight(2, 0.3)

An example of iterating through the states and transitions of the above transducer when printing them in AT&T format to standard output:

  # Go through all states
  for state, arcs in enumerate(fsm):
    for arc in arcs:
      print('%i ' % (state), end='')
      print(arc)
    if fsm.is_final_state(state):
      print('%i %f' % (state, fsm.get_final_weight(state)) )

See also

hfst.HfstBasicTransition

Constructor & Destructor Documentation

def __init__

(

self

)

Create a transducer with one initial state that has state number zero and is not a final state, i.e.

create an empty transducer.

 tr = hfst.HfstBasicTransducer()

def __init__	(		self,
			transducer
	)

Create a transducer equivalent to transducer.

Parameters

transducer

The transducer to be copied, hfst.HfstBasicTransducer or hfst.HfstTransducer.

 tr = hfst.regex('foo') # creates an HfstTransducer
 TR = hfst.HfstBasicTransducer(tr)
 TR2 = hfst.HfstBasicTransducer(TR)

Member Function Documentation

def __enumerate__

(

self

)

Return an enumeration of the states and transitions of the transducer.

 for state, arcs in enumerate(fsm):
   for arc in arcs:
     print('%i ' % (state), end='')
     print(arc)
   if fsm.is_final_state(state):
     print('%i %f' % (state, fsm.get_final_weight(state)) )

def __str__

(

self

)

Return a string representation of the transducer.

 print(fsm)

def add_state

(

self

)

Add a new state to this transducer and return its number.

Returns

The next (smallest) free state number.

def add_state	(		self,
			state
	)

Add a state s to this graph.

Parameters

state

The number of the state to be added.

Returns

state

If the state already exists, it is not added again. All states with state number smaller than s are also added to the transducer if they did not exist before.

def add_symbol_to_alphabet	(		self,
			symbol
	)

Explicitly add symbol to the alphabet of the graph.

Note

Usually the user does not have to take care of the alphabet of a graph. This function can be useful in some special cases. @ param symbol The string to be added.

def add_symbols_to_alphabet	(		self,
			symbols
	)

Explicitly add symbols to the alphabet of the graph.

Note

Usually the user does not have to take care of the alphabet of a graph. This function can be useful in some special cases.

Parameters

symbols

A tuple of strings to be added.

def add_transition	(		self,
			state,
			transition,
			add_symbols_to_alphabet = True
	)

Add a transition transition to state state, add_symbols_to_alphabet defines whether the transition symbols are added to the alphabet.

Parameters

state	The number of the state where the transition is added. If it does not exist, it is created.
transition	A hfst.HfstBasicTransition that is added to state.
add_symbols_to_alphabet	Whether the transition symbols are added to the alphabet of the transducer. (In special cases this is not wanted.)

def add_transition	(		self,
			source,
			target,
			input,
			output,
			weight = 0
	)

Add a transition from state source to state target with input symbol input, output symbol output and weight weight.

Parameters

source	The number of the state where the transition is added. If it does not exist, it is created.
target	The number of the state where the transition leads. If it does not exist, it is created. (?)
input	The input symbol of the transition.
output	The output symbol of the transition.
weight	The weight of the transition.

def disjunct	(		self,
			stringpairpath,
			weight
	)

Disjunct this transducer with a one-path transducer defined by consecutive string pairs in spv that has weight weight.

Precondition

This graph must be a trie where all weights are in final states, i.e. all transitions have a zero weight.

There is no way to test whether a graph is a trie, so the use of this function is probably limited to fast construction of a lexicon. Here is an example:

 lexicon = hfst.HfstBasicTransducer()
 tok = hfst.HfstTokenizer()
 lexicon.disjunct(tok.tokenize('dog'), 0.3)
 lexicon.disjunct(tok.tokenize('cat'), 0.5)
 lexicon.disjunct(tok.tokenize('elephant'), 1.6)

def get_alphabet

(

self

)

The symbols in the alphabet of the transducer.

The symbols do not necessarily occur in any transitions of the transducer. Epsilon, unknown and identity symbols are always included in the alphabet.

Returns

A tuple of strings.

def get_final_weight	(		self,
			state
	)

Get the final weight of state state in this transducer.

Parameters

state

The number of the state. If it does not exist, a StateIsNotFinalException is thrown.

Exceptions

hfst.exceptions.StateIsNotFinalException.

def get_max_state

(

self

)

Get the biggest state number in use.

Returns

The biggest state number in use.

def get_transition_pairs

(

self

)

Get a list of all input/output symbol pairs used in the transitions of this transducer.

def harmonize	(		self,
			another
	)

Harmonize this transducer and another.

In harmonization the unknown and identity symbols in transitions of both graphs are expanded according to the symbols that are previously unknown to the graph.

For example the graphs

 [a:b ?:?]
 [c:d ? ?:c]

are expanded to

 [ a:b [?:? | ?:c | ?:d | c:d | d:c | c:? | d:?] ]
 [ c:d [? | a | b] [?:c| a:c | b:?] ]

when harmonized.

The symbol '?' means hfst.UNKNOWN in either or both sides of a transition (transitions of type [?:x], [x:?] and [?:?]). The transition [?] means hfst.IDENTITY.

Note

This function is always called for all transducer arguments of functions that take two or more graphs as their arguments, unless otherwise said.

def insert_freely	(		self,
			symbol_pair,
			weight
	)

Insert freely any number of symbol_pair in the transducer with weight weight.

Parameters

symbol_pair	A string pair to be inserted.
weight	The weight of the inserted symbol pair.

def insert_freely	(		self,
			transducer
	)

Insert freely any number of transducer in this transducer.

param transducer An HfstBasicTransducer to be inserted.

def is_final_state	(		self,
			state
	)

Whether state state is final.

Parameters

state

The state whose finality is returned.

def is_infinitely_ambiguous

(

self

)

Whether the transducer is infinitely ambiguous.

A transducer is infinitely ambiguous if there exists an input that will yield infinitely many results, i.e. there are input epsilon loops that are traversed with that input.

def is_lookup_infinitely_ambiguous	(		self,
			str
	)

Whether the transducer is infinitely ambiguous with input str.

Parameters

str	The input.

A transducer is infinitely ambiguous with a given input if the input yields infinitely many results, i.e. there are input epsilon loops that are traversed with the input.

def longest_path_size

(

self

)

The length of the longest path in transducer.

Length of a path means number of arcs on that path.

def lookup_fd	(		self,
			input,
			kvargs
	)

Lookup tokenized input input in the transducer minding flag diacritics.

Parameters

str	A list/tuple of strings to look up.
kvargs	infinite_cutoff=-1, max_weight=None
infinite_cutoff	Defaults to -1, i.e. infinite.
max_weight	Defaults to None, i.e. infinity.

def prune_alphabet

(

self

)

Remove all symbols that do not occur in transitions of the transducer from its alphabet.

Epsilon, unknown and identity symbols are always included in the alphabet.

def read_att	(		f,
			epsilon_symbol,
			linecount
	)

Read a transducer in AT&T format from file f.

epsilon_symbol defines the symbol used for epsilon, linecount is incremented as lines are read.

Returns

A transducer constructed by reading from file file. This function is a static one.

def read_prolog	(		f,
			linecount
	)

Read a transducer from prolog file f.

linecount is incremented as lines are read (is it in python?).

Returns

A transducer constructed by reading from file file. This function is a static one.

def remove_symbol_from_alphabet	(		self,
			symbol
	)

Remove symbol symbol from the alphabet of the graph.

Note

Use with care, removing symbols that occur in the transitions of the graph can have unexpected results.

Parameters

symbol

The string to be removed.

def remove_symbols_from_alphabet	(		self,
			symbols
	)

Remove symbols symbols from the alphabet of the graph.

Note

Use with care, removing symbols that occur in the transitions of the graph can have unexpected results.

Parameters

symbols

A tuple of strings to be removed.

def remove_transition	(		self,
			s,
			transition,
			remove_symbols_from_alphabet = False
	)

Remove transition transition from state s.

Parameters

s	The state which transition belongs to.
transition	The transition to be removed.
remove_symbols_from_alphabet	(?)

def set_final_weight	(		self,
			state,
			weight
	)

Set the final weight of state state in this transducer to weight.

If the state does not exist, it is created.

def sort_arcs

(

self

)

Sort the arcs of this transducer according to input and output symbols.

Returns

This transducer.

def states

(

self

)

The states of the transducer.

Returns

A tuple of state numbers.

An example: /verbatim for state in fsm.states(): for arc in fsm.transitions(state): print('i ' % (state), end='') print(arc) if fsm.is_final_state(state): print('i f' % (state, fsm.get_final_weight(state)) ) /endverbatim

def states_and_transitions

(

self

)

The states and transitions of the transducer.

Returns

A tuple of tuples of HfstBasicTransitions.

See also

hfst.HfstBasicTransducer.__enumerate__

def substitute	(		self,
			s,
			S = None,
			kvargs
	)

Substitute symbols or transitions in the transducer.

Parameters

s	The symbol or transition to be substituted. Can also be a dictionary of substitutions, if S == None.
S	The symbol, transition, a tuple of transitions or a transducer (hfst.HfstBasicTransducer) that substitutes s.
kvargs	Arguments recognized are 'input' and 'output', their values can be False or True, True being the default. These arguments are valid only if s and S are strings, else they are ignored.
input	Whether substitution is performed on input side, defaults to True. Valid only if s and S are strings.
output	Whether substitution is performed on output side, defaults to True. Valid only if s and S are strings.

Possible combinations of arguments and their types are:

(1) substitute(str, str, input=bool, output=bool): substitute symbol with symbol on input, output or both sides of each transition in the transducer. (2) substitute(strpair, strpair): substitute transition with transition (3) substitute(strpair, strpairtuple): substitute transition with several transitions (4) substitute(strpair, transducer): substitute transition with a transducer (5) substitute(dict): perform several symbol-to-symbol substitutions (6) substitute(dict): perform several transition-to-transition substitutions

Examples:

(1) tr.substitute('a', 'A', input=True, output=False): substitute lowercase a:s with uppercase ones (2) tr.substitute(('a','b'),('A','B')): substitute transitions that map lowercase a into lowercase b with transitions that map uppercase a into uppercase b (3) tr.substitute(('a','b'), (('A','B'),('a','B'),('A','b'))): change either or both sides of a transition [a:b] to uppercase (4) tr.substitute(('a','b'), hfst.regex('[a:b]+')) change [a:b] transition into one or more consecutive [a:b] transitions (5) tr.substitute({'a':'A', 'b':'B', 'c':'C'}) change lowercase a, b and c into their uppercase variants (6) tr.substitute( {('a','a'):('A','A'), ('b','b'):('B','B'), ('c','c'):('C','C')} ): change lowercase a, b and c into their uppercase variants

In case (4), epsilon transitions are used to attach copies of transducer S between the SOURCE and TARGET state of each transition that is substituted. The transition itself is deleted, but its weight is copied to the epsilon transition leading from SOURCE to the initial state of S. Each final state of S is made non-final and an epsilon transition leading to TARGET is attached to it. The final weight is copied to the epsilon transition.

def symbols_used

(

self

)

Get a list of all symbols used in the transitions of this transducer.

def transitions	(		self,
			state
	)

Get the transitions of state state in this transducer.

If the state does not exist, a StateIndexOutOfBoundsException is thrown.

Returns

A tuple of HfstBasicTransitions.

 for state in fsm.states():
 for arc in fsm.transitions(state):
     print('%i ' % (state), end='')
     print(arc)
 if fsm.is_final_state(state):
    print('%i %f' % (state, fsm.get_final_weight(state)) )

def write_att	(		f,
			bool,
			write_weights = True
	)

Write this transducer in AT&T format to file f, write_weights defines whether weights are written.

def write_prolog	(		self,
			f,
			name,
			write_weights = True
	)

Write the transducer in prolog format to file f.

Name the transducer name.

def write_xfst	(		self,
			f,
			write_weights = True
	)

Write the transducer in xfst format to file f.

---

The documentation for this class was generated from the following file:

• hfst/__init__.py