Skip to content

Commit 71b8806

Browse files
authored
Merge pull request #71 from tunnelvisionlabs/update-ref
Update code to the 4.9 release of ANTLR 4
2 parents 0f058a6 + cf3c560 commit 71b8806

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

43 files changed

+592
-139
lines changed

antlr4-maven-plugin/pom.xml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
<parent>
1414
<groupId>com.tunnelvisionlabs</groupId>
1515
<artifactId>antlr4-master</artifactId>
16-
<version>4.7.5-SNAPSHOT</version>
16+
<version>4.9.0-SNAPSHOT</version>
1717
</parent>
1818

1919
<artifactId>antlr4-maven-plugin</artifactId>
@@ -184,7 +184,7 @@
184184
<plugin>
185185
<groupId>org.apache.maven.plugins</groupId>
186186
<artifactId>maven-plugin-plugin</artifactId>
187-
<version>3.3</version>
187+
<version>3.6.0</version>
188188
<configuration>
189189
<!-- see http://jira.codehaus.org/browse/MNG-5346 -->
190190
<skipErrorNoDescriptorsFound>true</skipErrorNoDescriptorsFound>

antlr4-testgen-maven-plugin/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
<parent>
1010
<groupId>com.tunnelvisionlabs</groupId>
1111
<artifactId>antlr4-master</artifactId>
12-
<version>4.7.5-SNAPSHOT</version>
12+
<version>4.9.0-SNAPSHOT</version>
1313
</parent>
1414

1515
<artifactId>antlr4-testgen-maven-plugin</artifactId>

contributors.txt

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,9 @@
11
ANTLR Project Contributors Certification of Origin and Rights
22

3+
NOTE: This tool is mature and Terence is mostly occupied elsewhere. We
4+
can't accept any changes that could have widespread effects on thousands
5+
of existing projects. Sorry!
6+
37
All contributors to ANTLR v4 must formally agree to abide by this
48
certificate of origin by signing on the bottom with their github
59
userid, full name, email address (you can obscure your e-mail, but it
@@ -204,6 +208,9 @@ YYYY/MM/DD, github id, Full name, email
204208
2018/06/16, EternalPhane, Zongyuan Zuo, eternalphane@gmail.com
205209
2018/07/03, jgoppert, James Goppert, james.goppert@gmail.com
206210
2018/07/27, Maksim Novikov, mnovikov.work@gmail.com
211+
2018/08/03, ENDOH takanao, djmchl@gmail.com
212+
2018/10/08, xsIceman, Andreas Skaar, andreas.skaar@gmail.com
213+
2018/10/18, edirgarcia, Edir García Lazo, edirgl@hotmail.com
207214
2018/07/31, Lucas Henrqiue, lucashenrique580@gmail.com
208215
2018/08/03, ENDOH takanao, djmchl@gmail.com
209216
2018/10/29, chrisaycock, Christopher Aycock, chris[at]chrisaycock[dot]com
@@ -213,3 +220,64 @@ YYYY/MM/DD, github id, Full name, email
213220
2018/11/29, hannemann-tamas, Ralf Hannemann-Tamas, ralf.ht@gmail.com
214221
2018/12/20, WalterCouto, Walter Couto, WalterCouto@users.noreply.github.com
215222
2018/12/23, youkaichao, Kaichao You, youkaichao@gmail.com
223+
2019/01/02, wkhemir, Wail Khemir, khemirwail@gmail.com
224+
2019/01/16, kuegi, Markus Zancolo, markus.zancolo@roomle.com
225+
2019/02/06, ralucado, Cristina Raluca Vijulie, ralucris.v[at]gmail[dot]com
226+
2019/02/23, gedimitr, Gerasimos Dimitriadis, gedimitr@gmail.com
227+
2019/03/13, base698, Justin Thomas, justin.thomas1@gmail.com
228+
2019/03/18, carlodri, Carlo Dri, carlo.dri@gmail.com
229+
2019/05/02, askingalot, Andy Collins, askingalot@gmail.com
230+
2019/05/13, mapio, Massimo Santini, massimo.santini@gmail.com
231+
2019/07/11, olowo726, Olof Wolgast, olof@baah.se
232+
2019/07/16, abhijithneilabraham, Abhijith Neil Abraham, abhijithneilabrahampk@gmail.com
233+
2019/07/26, Braavos96, Eric Hettiaratchi, erichettiaratchi@gmail.com
234+
2019/08/02, thron7, Thomas Herchenroeder, thron7[at]users[dot]sourceforge[dot]net
235+
2019/08/12, easonlin404, Eason Lin, easonlin404@gmail.com
236+
2019/08/23, akaJes, Oleksandr Mamchyts, akaJes@gmail.com
237+
2019/08/27, wurzelpeter, Markus Franke, markus[hyphen]franke[at]web[dot]de
238+
2019/09/10, ImanHosseini, Iman Hosseini, hosseini.iman@yahoo.com
239+
2019/09/03, João Henrique, johnnyonflame@hotmail.com
240+
2019/09/10, neko1235, Ihar Mokharau, igor.mohorev@gmail.com
241+
2019/09/10, yar3333, Yaroslav Sivakov, yar3333@gmail.com
242+
2019/09/10, marcospassos, Marcos Passos, marcospassos.com@gmail.com
243+
2019/09/10, amorimjuliana, Juliana Amorim, juu.amorim@gmail.com
244+
2019/09/15, sullis, Sean Sullivan, github@seansullivan.com
245+
2019/09/17, kaz, Kazuki Sawada, kazuki@6715.jp
246+
2019/09/28, lmy269, Mingyang Liu, lmy040758@gmail.com
247+
2019/10/29, tehbone, Tabari Alexander, tehbone@gmail.com
248+
2019/10/31, a-square, Alexei Averchenko, lex.aver@gmail.com
249+
2019/11/05, listba, Ben List, ben.list89@gmail.com
250+
2019/11/11, foxeverl, Liu Xinfeng, liuxf1986[at]gmail[dot]com
251+
2019/11/17, felixn, Felix Nieuwenhuizhen, felix@tdlrali.com
252+
2019/11/18, mlilback, Mark Lilback, mark@lilback.com
253+
2020/01/19, lingyv-li, Lingyu Li, lingyv.li@gmail.com
254+
2020/02/02, carocad, Camilo Roca, carocad@unal.edu.co
255+
2020/02/10, julibert, Julián Bermúdez Ortega, julibert.dev@gmail.com
256+
2020/02/17, quantumsheep, Nathanael Demacon, nathanael.dmc@outlook.fr
257+
2020/02/21, StochasticTinkr, Daniel Pitts, github@coloraura.com
258+
2020/03/17, XsongyangX, Song Yang, songyang1218@gmail.com
259+
2020/04/07, deniskyashif, Denis Kyashif, denis.kyashif@gmail.com
260+
2020/04/08, lwehmeier, Leon Wehmeier, wehmeier@st.ovgu.de
261+
2020/04/10, agrabski, Adam Grabski, adam.gr@outlook.com
262+
2020/04/23, martinvw, Martin van Wingerden, martin@martinvw.nl
263+
2020/04/30, TristonianJones, Tristan Swadell, tswadell@google.com
264+
2020/05/06, iammosespaulr, Moses Paul R, iammosespaulr@gmail.com
265+
2020/05/10, gomerser, Erik Gomersbach, gomerser@gomersba.ch
266+
2020/05/22, keywan-ghadami-oxid, Keywan Ghadami, keywan.ghadami@oxid-esales.com
267+
2020/05/25, graknol, Sindre van der Linden, graknol@gmail.com
268+
2020/05/31, d-markey, David Markey, dmarkey@free.fr
269+
2020/06/02, cohomology, Kilian Kilger, kkilger AT gmail.com
270+
2020/06/04, IohannRabeson, Iohann Rabeson, iotaka6@gmail.com
271+
2020/06/04, sigmasoldi3r, Pablo Blanco, pablobc.1995@gmail.com
272+
2020/07/01, sha-N, Shan M Mathews, admin@bluestarqatar.com
273+
2020/08/22, stevenjohnstone, Steven Johnstone, steven.james.johnstone@gmail.com
274+
2020/09/06, ArthurSonzogni, Sonzogni Arthur, arthursonzogni@gmail.com
275+
2020/09/10, Khailian, Arunav Sanyal, arunav.sanyal91@gmail.com
276+
2020/09/12, Clcanny, Charles Ruan, a837940593@gmail.com
277+
2020/09/15, rmcgregor1990, Robert McGregor, rmcgregor1990@gmail.com
278+
2020/09/16, trenki2, Markus Trenkwalder, trenki2[at]gmx[dot]net
279+
2020/10/08, Marti2203, Martin Mirchev, mirchevmartin2203@gmail.com
280+
2020/10/16, adarshbhat, Adarsh Bhat, adarshbhat@users.noreply.github.com
281+
2020/10/20, adamwojs, Adam Wójs, adam[at]wojs.pl
282+
2020/10/24, cliid, Jiwu Jang, jiwujang@naver.com
283+
2020/11/05, MichelHartmann, Michel Hartmann, MichelHartmann@users.noreply.github.com

doc/actions.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ Using a rule label looks like this:
8686
returnStat : 'return' e=expr {System.out.println("matched "+e.text);} ;
8787
```
8888

89-
You can also use `$ followed by the name of the attribute to access the value associated with the currently executing rule. For example, `$start` is the starting token of the current rule.
89+
You can also use `$` followed by the name of the attribute to access the value associated with the currently executing rule. For example, `$start` is the starting token of the current rule.
9090

9191
```
9292
returnStat : 'return' expr {System.out.println("first token "+$start.getText());} ;

doc/case-insensitive-lexing.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ For the 4.7.1 release, we discussed both approaches in [detail](https://github.c
1616

1717
## Case-insensitive grammars
1818

19-
As a prime example of a grammar that specifically describes case insensitive keywords, see the
19+
As a prime example of a grammar that specifically describes case insensitive keywords, see the
2020
[SQLite grammar](https://github.com/antlr/grammars-v4/blob/master/sqlite/SQLite.g4). To match a case insensitive keyword, there are rules such as
2121

2222
```
@@ -74,4 +74,8 @@ Lexer lexer = new SomeSQLLexer(upper);
7474

7575
Here are implementations of `CaseChangingCharStream` in various target languages:
7676

77-
* [Java](https://github.com/parrt/antlr4/blob/case-insensitivity-doc/doc/resources/CaseChangingCharStream.java)
77+
* [C#](https://github.com/antlr/antlr4/blob/master/doc/resources/CaseChangingCharStream.cs)
78+
* [Go](https://github.com/antlr/antlr4/blob/master/doc/resources/case_changing_stream.go)
79+
* [Java](https://github.com/antlr/antlr4/blob/master/doc/resources/CaseChangingCharStream.java)
80+
* [JavaScript](https://github.com/antlr/antlr4/blob/master/doc/resources/CaseChangingStream.js)
81+
* [Python2/3](https://github.com/antlr/antlr4/blob/master/doc/resources/CaseChangingStream.py)

doc/creating-a-language-target.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,12 +11,14 @@ Creating a new target involves the following key elements:
1111
1. For the tool, create class *X*Target as a subclass of class `Target` in package `org.antlr.v4.codegen.target`. This class describes language specific details about escape characters and strings and so on. There is very little to do here typically.
1212
1. Create *X*.stg in directory tool/resources/org/antlr/v4/tool/templates/codegen/*X*/*X*.stg. This is a [StringTemplate](http://www.stringtemplate.org/) group file (`.stg`) that tells ANTLR how to express all of the parsing elements needed to generate code. You will see templates called `ParserFile`, `Parser`, `Lexer`, `CodeBlockForAlt`, `AltBlock`, etc... Each of these must be described how to build the indicated chunk of code. Your best bet is to find the closest existing target, copy that template file, and tweak to suit.
1313
1. Create a runtime library to support the parsers generated by ANTLR. Under directory runtime/*X*, you are in complete control of the directory structure as dictated by common usage of that target language. For example, Java has: `runtime/Java/lib` and `runtime/Java/src` directories. Under `src`, you will find a directory structure for package `org.antlr.v4.runtime` and below.
14-
1. Create a template file for runtime tests. All you have to do is provide a few simple templates that indicate how to print values and declare variables. Our runtime test mechanism in dir `runtime-testsuite` will automatically generate code in a new target and check the results. All it needs to know is how to generate a test rig (i.e., a `main` program), how to define various class fields, compare members and so on. You must create a *X* directory underneath `runtime-testsuite/resources/org/antlr/v4/test/runtime`. Again, your best bet is to copy the templates from the closest language to your target and tweak it to suit.
14+
1. Create a template file for runtime tests. All you have to do is provide a few templates that indicate how to print values and declare variables. Our runtime test mechanism in dir `runtime-testsuite` will automatically generate code using these templates for each target and check the test results. It needs to know how to define various class fields, compare members and so on. You must create a *X*.test.stg file underneath [runtime-testsuite/resources/org/antlr/v4/test/runtime](https://github.com/antlr/antlr4/tree/master/runtime-testsuite/resources/org/antlr/v4/test/runtime). Again, your best bet is to copy the templates from the closest language to your target and tweak it to suit.
15+
1. Create test files under [/runtime-testsuite/test/org/antlr/v4/test/runtime](https://github.com/antlr/antlr4/tree/master/runtime-testsuite/test/org/antlr/v4/test/runtime). They will load defined test cases in each test descriptor. Also add the `/runtime-testsuite/test/org/antlr/v4/test/runtime/X/BaseXTest.java` which defines how test cases will execute and output.
16+
1. Create/edit shell scripts in [/.travis](https://github.com/antlr/antlr4/blob/master/.travis) and [/appveyor.yml](https://github.com/antlr/antlr4/blob/master/appveyor.yml) to run tests in CI pipelines.
1517

1618
## Getting started
1719

1820
1. Fork the `antlr/antlr4` repository at github to your own user so that you have repository `username/antlr4`.
19-
2. Clone `username/antlr4`, forked repository, to your local disk. Your remote `origin` will be the forked repository on GitHub. Add a remote `upstream` to the original `antlr/antlr4` repository (URL `https://github.com/antlr/antlr4.git`). Changes that you would like to contribute back to the project are done with [pull requests](https://help.github.com/articles/using-pull-requests/).
21+
2. Clone `username/antlr4`, the forked repository, to your local disk. Your remote `origin` will be the forked repository on GitHub. Add a remote `upstream` to the original `antlr/antlr4` repository (URL `https://github.com/antlr/antlr4.git`). Changes that you would like to contribute back to the project are done with [pull requests](https://help.github.com/articles/using-pull-requests/).
2022
3. Try to build it before doing anything
2123
```bash
2224
$ mvn compile

doc/faq/index.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,8 +38,8 @@ This is the main landing page for the ANTLR 4 FAQ. The links below will take you
3838

3939
## Translation
4040

41-
* [ASTs vs parse trees](parse-trees.md)
42-
* [Decoupling input walking from output generation](parse-trees.md)
41+
* [ASTs vs parse trees](translation.md)
42+
* [Decoupling input walking from output generation](translation.md)
4343

4444
## Actions and semantic predicates
4545

doc/faq/parse-trees.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ For writing a compiler, either generate [LLVM-type static-single-assignment](htt
5050

5151
### XPath
5252

53-
XPath works great when you need to find specific nodes, possibly in certain contexts. The context is limited to the parents on the way to the root of the tree. For example, if you want to find all ID nodes, use path `//ID`. If you want all variable declarations, you might use path `//vardecl`. If you only want fields declarations, then you can use some context information via path `/classdef/vardecl`, which would only find vardecls that our children of class definitions. You can merge the results of multiple XPath `findAll()`s simulating a set union for XPath. The only caveat is that the order from the original tree is not preserved when you union multiple `findAll()` sets.
53+
XPath works great when you need to find specific nodes, possibly in certain contexts. The context is limited to the parents on the way to the root of the tree. For example, if you want to find all ID nodes, use path `//ID`. If you want all variable declarations, you might use path `//vardecl`. If you only want fields declarations, then you can use some context information via path `/classdef/vardecl`, which would only find vardecls that are children of class definitions. You can merge the results of multiple XPath `findAll()`s simulating a set union for XPath. The only caveat is that the order from the original tree is not preserved when you union multiple `findAll()` sets.
5454

5555
### Tree pattern matching
5656

@@ -70,4 +70,4 @@ scopeStack.peek().define(new VariableSymbol("foo"))
7070

7171
That way each listener function does not have to compute its appropriate scope.
7272

73-
Examples: [DefScopesAndSymbols.java](https://github.com/mantra/compiler/blob/master/src/java/mantra/semantics/DefScopesAndSymbols.java) and [SetScopeListener.java](https://github.com/mantra/compiler/blob/master/src/java/mantra/semantics/SetScopeListener.java) and [VerifyListener.java](https://github.com/mantra/compiler/blob/master/src/java/mantra/semantics/VerifyListener.java)
73+
Examples: [DefScopesAndSymbols.java](https://github.com/mantra/compiler/blob/master/src/java/mantra/semantics/DefScopesAndSymbols.java) and [SetScopeListener.java](https://github.com/mantra/compiler/blob/master/src/java/mantra/semantics/SetScopeListener.java) and [VerifyListener.java](https://github.com/mantra/compiler/blob/master/src/java/mantra/semantics/VerifyListener.java)

doc/getting-started.md

Lines changed: 19 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -8,46 +8,46 @@ Hi and welcome to the version 4 release of ANTLR! It's named after the fearless
88

99
ANTLR is really two things: a tool that translates your grammar to a parser/lexer in Java (or other target language) and the runtime needed by the generated parsers/lexers. Even if you are using the ANTLR Intellij plug-in or ANTLRWorks to run the ANTLR tool, the generated code will still need the runtime library.
1010

11-
The first thing you should do is probably download and install a development tool plug-in. Even if you only use such tools for editing, they are great. Then, follow the instructions below to get the runtime environment available to your system to run generated parsers/lexers. In what follows, I talk about antlr-4.7.1-complete.jar, which has the tool and the runtime and any other support libraries (e.g., ANTLR v4 is written in v3).
11+
The first thing you should do is probably download and install a development tool plug-in. Even if you only use such tools for editing, they are great. Then, follow the instructions below to get the runtime environment available to your system to run generated parsers/lexers. In what follows, I talk about antlr-4.9-complete.jar, which has the tool and the runtime and any other support libraries (e.g., ANTLR v4 is written in v3).
1212

13-
If you are going to integrate ANTLR into your existing build system using mvn, ant, or want to get ANTLR into your IDE such as eclipse or intellij, see Integrating ANTLR into Development Systems.
13+
If you are going to integrate ANTLR into your existing build system using mvn, ant, or want to get ANTLR into your IDE such as eclipse or intellij, see [Integrating ANTLR into Development Systems](https://github.com/antlr/antlr4/blob/master/doc/IDEs.md).
1414

1515
### UNIX
1616

17-
0. Install Java (version 1.6 or higher)
17+
0. Install Java (version 1.7 or higher)
1818
1. Download
1919
```
2020
$ cd /usr/local/lib
21-
$ curl -O https://www.antlr.org/download/antlr-4.7.1-complete.jar
21+
$ curl -O https://www.antlr.org/download/antlr-4.9-complete.jar
2222
```
2323
Or just download in browser from website:
2424
[https://www.antlr.org/download.html](https://www.antlr.org/download.html)
2525
and put it somewhere rational like `/usr/local/lib`.
2626

27-
2. Add `antlr-4.7.1-complete.jar` to your `CLASSPATH`:
27+
2. Add `antlr-4.9-complete.jar` to your `CLASSPATH`:
2828
```
29-
$ export CLASSPATH=".:/usr/local/lib/antlr-4.7.1-complete.jar:$CLASSPATH"
29+
$ export CLASSPATH=".:/usr/local/lib/antlr-4.9-complete.jar.jar:$CLASSPATH"
3030
```
3131
It's also a good idea to put this in your `.bash_profile` or whatever your startup script is.
3232

3333
3. Create aliases for the ANTLR Tool, and `TestRig`.
3434
```
35-
$ alias antlr4='java -Xmx500M -cp "/usr/local/lib/antlr-4.7.1-complete.jar:$CLASSPATH" org.antlr.v4.Tool'
36-
$ alias grun='java -Xmx500M -cp "/usr/local/lib/antlr-4.7.1-complete.jar:$CLASSPATH" org.antlr.v4.gui.TestRig'
35+
$ alias antlr4='java -Xmx500M -cp "/usr/local/lib/antlr-4.9-complete.jar.jar:$CLASSPATH" org.antlr.v4.Tool'
36+
$ alias grun='java -Xmx500M -cp "/usr/local/lib/antlr-4.9-complete.jar.jar:$CLASSPATH" org.antlr.v4.gui.TestRig'
3737
```
3838

3939
### WINDOWS
4040

4141
(*Thanks to Graham Wideman*)
4242

43-
0. Install Java (version 1.6 or higher)
44-
1. Download antlr-4.7.1-complete.jar (or whatever version) from [https://www.antlr.org/download/](https://www.antlr.org/download/)
43+
0. Install Java (version 1.7 or higher)
44+
1. Download antlr-4.9-complete.jar (or whatever version) from [https://www.antlr.org/download/](https://www.antlr.org/download/)
4545
Save to your directory for 3rd party Java libraries, say `C:\Javalib`
46-
2. Add `antlr-4.7.1-complete.jar` to CLASSPATH, either:
46+
2. Add `antlr-4.9-complete.jar.jar` to CLASSPATH, either:
4747
* Permanently: Using System Properties dialog > Environment variables > Create or append to `CLASSPATH` variable
4848
* Temporarily, at command line:
4949
```
50-
SET CLASSPATH=.;C:\Javalib\antlr-4.7.1-complete.jar;%CLASSPATH%
50+
SET CLASSPATH=.;C:\Javalib\antlr-4.9-complete.jar;%CLASSPATH%
5151
```
5252
3. Create short convenient commands for the ANTLR Tool, and TestRig, using batch files or doskey commands:
5353
* Batch files (in directory in system PATH) antlr4.bat and grun.bat
@@ -73,7 +73,7 @@ Either launch org.antlr.v4.Tool directly:
7373

7474
```
7575
$ java org.antlr.v4.Tool
76-
ANTLR Parser Generator Version 4.7.1
76+
ANTLR Parser Generator Version 4.9
7777
-o ___ specify output directory where all output is generated
7878
-lib ___ specify location of .tokens files
7979
...
@@ -82,8 +82,8 @@ ANTLR Parser Generator Version 4.7.1
8282
or use -jar option on java:
8383

8484
```
85-
$ java -jar /usr/local/lib/antlr-4.7.1-complete.jar
86-
ANTLR Parser Generator Version 4.7.1
85+
$ java -jar /usr/local/lib/antlr-4.9-complete.jar
86+
ANTLR Parser Generator Version 4.9
8787
-o ___ specify output directory where all output is generated
8888
-lib ___ specify location of .tokens files
8989
...
@@ -114,8 +114,11 @@ Now test it:
114114

115115
```
116116
$ grun Hello r -tree
117+
(Now enter something like the string below)
117118
hello parrt
119+
(now,do:)
118120
^D
121+
(The output:)
119122
(r hello parrt)
120123
(That ^D means EOF on unix; it's ^Z in Windows.) The -tree option prints the parse tree in LISP notation.
121124
It's nicer to look at parse trees visually.
@@ -137,3 +140,4 @@ The book has lots and lots of examples that should be useful too. You can downlo
137140
Also, there is a large collection of grammars for v4 at github:
138141

139142
[https://github.com/antlr/grammars-v4](https://github.com/antlr/grammars-v4)
143+
/

0 commit comments

Comments
 (0)