Class AddLexer

source code

This is required when using Tokens.  It does the following:
- Find all tokens in the matcher graph
- Construct a lexer from the tokens
- Connect the lexer to the matcher
- Check that all children have a token parent 
  (and optionally add a default token)
Although possibly not in that order. 

alphabet is the alphabet for which the regular expressions are defined.

discard is a regular expression that is used to match space (typically)
if no token can be matched (and which is then discarded)