aboutsummaryrefslogtreecommitdiff
path: root/driver/lexer/lexer.go (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Namespace packages with "urubu/"EuAndreh2024-12-101-335/+0
|
* Add byte position to tokensRyo Nihei2022-11-131-2/+17
|
* Set token positions for the EOF symbolRyo Nihei2022-11-121-2/+2
| | | | | | vartan intentionally had not set positional information for the EOF symbol in a token because the corresponding character does not exist in the source code. However, to be able to include the positional information in a syntax error message when a syntax error occurs due to the detection of the EOF symbol during parsing, this commit sets the position next to the last character in the source code as the position of the EOF symbol.
* Fix counting of token positionsRyo Nihei2022-11-121-37/+46
| | | | The name `unread` gives the impression of operating just the opposite of `read`. However, this commit renames `unread` function to `revert` because it was a function to revert the state to the last acceptance.
* Import source code of lexer generatorRyo Nihei2022-11-101-0/+311
From: https://github.com/nihei9/maleeni