aboutsummaryrefslogtreecommitdiff
path: root/compiler/parser.go (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Use new parser and DFA compilerRyo Nihei2021-12-101-862/+0
|
* Move UTF8-related processes to utf8 packageRyo Nihei2021-12-011-591/+17
|
* Make contributory properties unavailable except internal useRyo Nihei2021-11-281-1/+30
| | | | | | | | | | | | This change follows [UAX #44 5.13 Property APIs]. > The following subtypes of Unicode character properties should generally not be exposed in APIs, > except in limited circumstances. They may not be useful, particularly in public API collections, > and may instead prove misleading to the users of such API collections. > > * Contributory properties are not recommended for public APIs. > ... https://unicode.org/reports/tr44/#Property_APIs
* Move all UCD-related processes to ucd packageRyo Nihei2021-11-271-4/+5
|
* Make character properties available in an inverse expression (Make ↵Ryo Nihei2021-11-251-0/+4
| | | | [^\p{...}] available)
* Support Lowercase and Uppercase property (Meet RL1.2 of UTS #18 partially)Ryo Nihei2021-11-251-21/+34
|
* Support White_Space property (Meet RL1.2 of UTS #18 partially)Ryo Nihei2021-11-241-6/+24
|
* Keep the order of AST nodes constantRyo Nihei2021-09-221-13/+23
|
* Change APIsRyo Nihei2021-08-011-5/+7
| | | | | | | | | | | | | | | | | | | | | | Change fields of tokens, results of lexical analysis, as follows: - Rename: mode -> mode_id - Rename: kind_id -> mode_kind_id - Add: kind_id The kind ID is unique across all modes, but the mode kind ID is unique only within a mode. Change fields of a transition table as follows: - Rename: initial_mode -> initial_mode_id - Rename: modes -> mode_names - Rename: kinds -> kind_names - Rename: specs[].kinds -> specs[].kind_names - Rename: specs[].dfa.initial_state -> specs[].dfa.initial_state_id Change public types defined in the spec package as follows: - Rename: LexModeNum -> LexModeID - Rename: LexKind -> LexKindName - Add: LexKindID - Add: StateID
* Add fragment expressionRyo Nihei2021-05-251-29/+158
| | | | A fragment entry is defined by an entry whose `fragment` field is `true`, and is referenced by a fragment expression (`\f{...}`).
* Fix parser to recognize property expressions in bracket expressionsRyo Nihei2021-05-021-0/+3
|
* Add character property expression (Meet RL1.2 of UTS #18 partially)Ryo Nihei2021-04-301-1/+54
| | | | | | | | | | \p{property name=property value} matches a character has the property. When the property name is General_Category, it can be omitted. That is, \p{Letter} equals \p{General_Category=Letter}. Currently, only General_Category is supported. This feature meets RL1.2 of UTS #18 partially. RL1.2 Properties: https://unicode.org/reports/tr18/#RL1.2
* Add code point expression (Meet RL1.1 of UTS #18)Ryo Nihei2021-04-241-0/+55
| | | | | | | | \u{hex string} matches a character has the code point represented by the hex string. For instance, \u{3042} matches hiragana あ (U+3042). The hex string must have 4 or 6 digits. This feature meets RL1.1 of UTS #18. RL1.1 Hex Notation: https://unicode.org/reports/tr18/#RL1.1
* Change the lexical specs of regexp and define concrete syntax error valuesRyo Nihei2021-04-171-44/+142
| | | | | * Make the lexer treat ']' as an ordinary character in default mode * Define values of the syntax error type that represents error information concretely
* Increase the maximum number of symbol positions per patternRyo Nihei2021-04-121-1/+4
| | | | | This commit increases the maximum number of symbol positions per pattern to 2^15 (= 32,768). When the limit is exceeded, the parse method returns an error.
* Fix grammar the parser acceptsRyo Nihei2021-04-111-35/+68
| | | | | * Add cases test the parse method. * Fix the parser to pass the cases.
* Add logical inverse expressionRyo Nihei2021-04-011-6/+91
| | | | [^a-z] matches any character that is not in the range a-z.
* Pass values in error type to panic()Ryo Nihei2021-03-071-2/+2
| | | | Because parser.parse() expects that recover() returns a value in error type, apply this change.
* RefactoringRyo Nihei2021-02-251-449/+301
| | | | | | * Remove token field from symbolNode * Simplify notation of nested nodes * Simplify arguments of newSymbolNode()
* Add range expressionRyo Nihei2021-02-241-4/+705
| | | | [a-z] matches any one character from a to z. The order of the characters depends on Unicode code points.
* Add + and ? operatorsRyo Nihei2021-02-201-3/+9
| | | | | * a+ matches 'a' one or more times. This is equivalent to aa*. * a? matches 'a' zero or one time.
* Add bracket expression matching specified characterRyo Nihei2021-02-141-6/+25
| | | | The bracket expression matches any single character specified in it. In the bracket expression, the special characters like ., *, and so on are also handled as normal characters.
* Add dot symbol matching any single characterRyo Nihei2021-02-141-3/+104
| | | | | | | | | The dot symbol matches any single character. When the dot symbol appears, the parser generates an AST matching all of the well-formed UTF-8 byte sequences. Refelences: * https://www.unicode.org/versions/Unicode13.0.0/ch03.pdf#G7404 * Table 3-6. UTF-8 Bit Distribution * Table 3-7. Well-Formed UTF-8 Byte Sequences
* Add compilerRyo Nihei2021-02-141-0/+221
The compiler takes a lexical specification expressed by regular expressions and generates a DFA accepting the tokens. Operators that you can use in the regular expressions are concatenation, alternation, repeat, and grouping.
generated by cgit v1.2.3 (git 2.49.0) at 2025-07-13 11:52:32 +0000