-
Notifications
You must be signed in to change notification settings - Fork 39
HaskForce Documentation
This document collects bits and pieces of information that might be relevant for HaskForce developers.
-
Spotting intermittent stalls when they occur is easy but reproducing and tracking them down can be messy.
-
Input robustness means that a bug in the parser causes a local problem but the remainder of the file looks just fine. Defining what "reasonable" means is on the soft side, but testing whether syntax highlighting works for a particular scenario is easy.
-
There is a list of popular projects on Hackage that can be used for sampling, and there are probably some prime suspects among the less widely used packages that use almost every extension available.
Compilers take a well-defined input and transform to a well-defined output. The IDE is more of a grey area since input should be treated liberally and the user should get feedback on the parts that don't make sense. The typical scenario for working programmers is probably that they start out with a well-defined and working program and then change it to something that does not parse/type check for a while and then gradually work their way back to a well-defined program with the new functionality incorporated. The exception to this scenario is when there is a refactoring tool that does all the changes in one quick swoop and the program effectively works in every time instant.
There are three levels of syntax and error highlighting in IntelliJ (http://confluence.jetbrains.com/display/IDEADEV/Developing+Custom+Language+Plugins+for+IntelliJ+IDEA, under "Syntax Highlighting and Error Highlighting"), one from the lexer, one from the parser and one from a slow annotator.
Syntax highlighting with the lexer happens in the first pass. The scope of syntax highlighting is limited: it's the colours you get for imports/keywords/constructors. The lexer needs to keep going even when input is bad, typically returning BAD_CHARACTER. The lexer must return everything, including spaces, comments, and other things that a traditional lexer typically drops quietly.
The red marks on lines that had unexpected content is put there by a second pass with the parser. If you write the following in a file:
module Hello where
module HelloAgain where
main = print "Hello, World!"there will be a red mark to the left side on the second line since the module is already declared on the first line. The rest of the module will be syntax highlighted just fine despite the parse error though since that's the job of the lexer.
Annotators run as a third pass and allow us to change the information or annotate extra information no psi nodes. We have an annotator in HaskellAnnotator that creates text attributes to enforce highlighting, but that might be unnecessary with the handwritten lexer. More information about annotators here: https://theantlrguy.atlassian.net/wiki/display/~admin/Intellij+plugin+development+notes#Intellijplugindevelopmentnotes-Annotators
Note: we are currently working on a new parser which will obsolete this. See HaskellParser2 for details.
We use Grammar-kit to define the language grammar in Haskell.bnf and then generate a parser from that. Define keywords as their own tokens. Be generous in accepting things: define string literals as something starting and ending with a double quote, and anything that isn't an unquoted quote in between. This means the user will get syntax highlighting for invalid programs which is precisely what we want.
The parser is essentially generated from everything that is not declared as a token in Haskell.bnf.
Case sensitive languages need to override a method isCaseSensitive() so that it returns true in their Language class implementation.
The parser of the Erlang plugin (http://ignatov.github.io/intellij-erlang/, https://github.com/ignatov/intellij-erlang/blob/master/grammars/erlang.bnf) might serve as a partial guide, but it's a bad starting point since it uses many of the bells and whistles provided by Grammar-kit.
There is a "debug" feature described in the Grammar-kit tutorial (https://github.com/JetBrains/Grammar-Kit/blob/master/TUTORIAL.md) that lets you try the grammar and inspect the psi-nodes without having to compile anything. It's not perfect, but it's a good tool to start with if you're making changes.
Open issues:
- How do we handle new extensions without smattering the file with red underlines? Can the parser recover automatically? Is there a possible fallback mode?
The lexer and parser has some tests in the test suite that will react to changes.
Help! I get messages about local history corruption.
Try to invalidate your IntelliJ cache and restart. If that doesn't help, quit IntelliJ, rename ~/Library/Caches/IdeaIC13 to ~/Library/Caches/IdeaIC13.old and start IntelliJ again.
(OSX) Help! I have 2Gb of memory assigned to IntelliJ and I still run out of memory on HaskellParser2.java.
Use Java 1.6 for running IntelliJ.
Failed to download 'https://github.com/JetBrains/intellij-community/raw/master/tools/lexer/jflex-1.4/lib/JFlex.jar': handshake alert: unrecognized_name
Add -Djsse.enableSNIExtension=false to your idea.vmoptions or idea.exe.vmoptions file in the IntelliJ directory.
[Deprecated] Grammar-Kit does not update the flex file/Grammar-kit generates a 0 byte flex file.
We do not use Grammar-Kit for generating the lexer any longer, but keeping the information here for reference.
Workaround: restart IntelliJ and generate the flex file right away.
Help! All my top level menus except IntelliJ IDEA disappeared!
Update to the latest Java 6 SDK.
Temporary workaround: switch to another desktop and select some application there (Safari works for me). Switch back to the IntelliJ desktop. If that does not work you can try to change to having a Java file active for editing while doing the switching.
Help! My file-tree view in the project panel got stuck!
Update to the latest Java 6 SDK. Temporary workaround: restart IntelliJ.
Running the test suite produces java.lang.NoClassDefFoundError: io/netty/channel/EventLoopGroup
Check your IntelliJ build version and ensure that your IntelliJ SDK version matches as well as the idea community sources are checked out at that version. You can follow the instructions in the README.