aboutsummaryrefslogtreecommitdiff
path: root/compiler/test_util_test.go (unfollow)
Commit message (Collapse)AuthorFilesLines
2021-02-25RefactoringRyo Nihei5-502/+351
* Remove token field from symbolNode * Simplify notation of nested nodes * Simplify arguments of newSymbolNode()
2021-02-24Add range expressionRyo Nihei4-9/+977
[a-z] matches any one character from a to z. The order of the characters depends on Unicode code points.
2021-02-20Add + and ? operatorsRyo Nihei6-21/+117
* a+ matches 'a' one or more times. This is equivalent to aa*. * a? matches 'a' zero or one time.
2021-02-17Fix computation of last positionsRyo Nihei2-0/+122
2021-02-16Add logging to lex commandRyo Nihei3-5/+126
lex command writes logs out to the maleeni-lex.log file. When you generate a lexer using driver.NewLexer(), you can choose whether the lexer writes logs or not.
2021-02-16Add CLIRyo Nihei6-0/+433
2021-02-16Add types of lexical specificationsRyo Nihei5-90/+133
APIs of compiler and driver packages use these types. Because CompiledLexSpec struct a lexer takes has kind names of lexical specification entries, the lexer sets them to tokens.
2021-02-14Add bracket expression matching specified characterRyo Nihei4-9/+127
The bracket expression matches any single character specified in it. In the bracket expression, the special characters like ., *, and so on are also handled as normal characters.
2021-02-14Add dot symbol matching any single characterRyo Nihei7-21/+201
The dot symbol matches any single character. When the dot symbol appears, the parser generates an AST matching all of the well-formed UTF-8 byte sequences. Refelences: * https://www.unicode.org/versions/Unicode13.0.0/ch03.pdf#G7404 * Table 3-6. UTF-8 Bit Distribution * Table 3-7. Well-Formed UTF-8 Byte Sequences
2021-02-14Add driverRyo Nihei2-0/+309
The driver takes a DFA and an input text and generates a lexer. The lexer tokenizes the input text according to the lexical specification that the DFA expresses.
2021-02-14Add compilerRyo Nihei9-0/+1268
The compiler takes a lexical specification expressed by regular expressions and generates a DFA accepting the tokens. Operators that you can use in the regular expressions are concatenation, alternation, repeat, and grouping.