“Maximal-Munch” Tokenization in Linear Time THOMAS REPS University of Wisconsin The lexical-analysis (or scanning) phase of a compiler attempts to partition an input string

Source URL: www.csc.lsu.edu

• Computing
• Automata theory
• Parsing
• Models of computation
• Lexical analysis
• Maximal munch
• Tokenization
• Finite-state machine
• Algorithm
• Programming language implementation
• Compiler construction
• Software engineering