Use gumtree API in IntelliJ on Windows

[ztwater]

I have added tree-sitter-parser.py to my system path and it could run from cmd. But when I called the API (e.g. JavaTreeSitterTreeGenerator().generateFrom().file(file); ) from IDE or from commands like "gumtree..." I got this error:

java.io.IOException: Cannot run program "tree-sitter-parser.py" (in directory "C:\Users\24426\AppData\Local\Temp"): CreateProcess error=193, %1 is not a valid Win32 application
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
at com.github.gumtreediff.gen.ExternalProcessTreeGenerator.readStandardOutput(ExternalProcessTreeGenerator.java:35)
at com.github.gumtreediff.gen.treesitter.AbstractTreeSitterGenerator.generate(AbstractTreeSitterGenerator.java:37)
at com.github.gumtreediff.gen.TreeGenerator.generateTree(TreeGenerator.java:41)
at com.github.gumtreediff.gen.TreeGenerator$ReaderConfigurator.reader(TreeGenerator.java:119)
at com.github.gumtreediff.gen.TreeGenerator$ReaderConfigurator.file(TreeGenerator.java:90)
at com.github.gumtreediff.gen.TreeGenerator$ReaderConfigurator.file(TreeGenerator.java:100)

Could you please tell me how to solve this error? I guess processbuilder has something to do with the OS...

[jrfaller]

Hi!

I guess the shebang #!/usr/bin/env python does not work on Windows 😄

However I don't know how to get a Python file running, but if you manage to make a bat file don't hesitate to make a PR!

Cheers.

[ztwater]

Thanks a lot! I found that ProcessBuilder can not direct run python scripts, so I changed the arguments of getCommandLine() in AbstractTreeSitterGenerator.java (see here). It is running now but a parse error occurs in TreeIoUtils.java.

while (r.hasNext()) {
    XMLEvent e = r.nextEvent();
    if (e instanceof StartElement) {

javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1,1]
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.next(XMLStreamReaderImpl.java:652)
	at java.xml/com.sun.xml.internal.stream.XMLEventReaderImpl.nextEvent(XMLEventReaderImpl.java:83)
	at com.github.gumtreediff.io.TreeIoUtils$XmlInternalGenerator.generate(TreeIoUtils.java:784)
	at com.github.gumtreediff.gen.TreeGenerator.generateTree(TreeGenerator.java:41)
        ...

[jrfaller]

Can you print the XML being parser to see what the problem is?

[ztwater]

This is the XML tree (from a java code snippet on GitHub) that generated by TreeGenerator:

<?xml version="1.0" ?>
<tree type="program" pos="0" length="598">
	<tree type="type_declaration" pos="0" length="598">
		<tree type="modifiers" pos="0" length="6">
			<tree type="visibility" pos="0" length="6" label="public"/>
		</tree>
		<tree type="type_keyword" pos="7" length="5" label="class"/>
		<tree type="identifier" pos="13" length="3" label="foo"/>
		<tree type="type_body" pos="16" length="582">
			<tree type="method_declaration" pos="18" length="578">
				<tree type="modifiers" pos="18" length="19">
					<tree type="visibility" pos="18" length="6" label="public"/>
					<tree type="final" pos="25" length="5" label="final"/>
					<tree type="static" pos="31" length="6" label="static"/>
				</tree>
				<tree type="type" pos="38" length="6" label="String"/>
				<tree type="identifier" pos="45" length="7" label="uniplus"/>
				<tree type="formal_parameters" pos="52" length="10">
					<tree type="formal_parameter" pos="53" length="8">
						<tree type="type" pos="53" length="6" label="String"/>
						<tree type="identifier" pos="60" length="1" label="s"/>
					</tree>
				</tree>
				<tree type="block" pos="63" length="533">
					<tree type="if_statement" pos="69" length="47">
						<tree type="if" pos="69" length="2" label="if"/>
						<tree type="parenthesized_expression" pos="72" length="17">
							<tree type="binary_expression" pos="73" length="15">
								<tree type="method_invocation" pos="73" length="10">
									<tree type="identifier" pos="73" length="1" label="s"/>
									<tree type="identifier" pos="75" length="6" label="length"/>
									<tree type="argument_list" pos="81" length="2"/>
								</tree>
								<tree type="comparison_operator" pos="84" length="2" label="=="/>
								<tree type="decimal_integer_literal" pos="87" length="1" label="0"/>
							</tree>
						</tree>
						<tree type="block" pos="90" length="26">
							<tree type="return_statement" pos="100" length="10">
								<tree type="string_literal" pos="107" length="2" label="&quot;&quot;"/>
							</tree>
						</tree>
					</tree>
					<tree type="block_comment" pos="121" length="55" label="/* This is just the minimum; sb will grow as needed. */"/>
					<tree type="local_variable_declaration" pos="181" length="55">
						<tree type="type" pos="181" length="12" label="StringBuffer"/>
						<tree type="variable_declarator" pos="194" length="41">
							<tree type="identifier" pos="194" length="2" label="sb"/>
							<tree type="affectation_operator" pos="197" length="1" label="="/>
							<tree type="object_creation_expression" pos="199" length="36">
								<tree type="new" pos="199" length="3" label="new"/>
								<tree type="type" pos="203" length="12" label="StringBuffer"/>
								<tree type="argument_list" pos="215" length="20">
									<tree type="binary_expression" pos="216" length="18">
										<tree type="decimal_integer_literal" pos="216" length="1" label="2"/>
										<tree type="arithmetic_operator" pos="218" length="1" label="+"/>
										<tree type="binary_expression" pos="220" length="14">
											<tree type="decimal_integer_literal" pos="220" length="1" label="3"/>
											<tree type="arithmetic_operator" pos="222" length="1" label="*"/>
											<tree type="method_invocation" pos="224" length="10">
												<tree type="identifier" pos="224" length="1" label="s"/>
												<tree type="identifier" pos="226" length="6" label="length"/>
												<tree type="argument_list" pos="232" length="2"/>
											</tree>
										</tree>
									</tree>
								</tree>
							</tree>
						</tree>
					</tree>
					<tree type="expression_statement" pos="241" length="16">
						<tree type="method_invocation" pos="241" length="15">
							<tree type="identifier" pos="241" length="2" label="sb"/>
							<tree type="identifier" pos="244" length="6" label="append"/>
							<tree type="argument_list" pos="250" length="6">
								<tree type="string_literal" pos="251" length="4" label="&quot;U+&quot;"/>
							</tree>
						</tree>
					</tree>
					<tree type="for_statement" pos="262" length="306">
						<tree type="local_variable_declaration" pos="267" length="10">
							<tree type="type" pos="267" length="3" label="int"/>
							<tree type="variable_declarator" pos="271" length="5">
								<tree type="identifier" pos="271" length="1" label="i"/>
								<tree type="affectation_operator" pos="273" length="1" label="="/>
								<tree type="decimal_integer_literal" pos="275" length="1" label="0"/>
							</tree>
						</tree>
						<tree type="binary_expression" pos="278" length="14">
							<tree type="identifier" pos="278" length="1" label="i"/>
							<tree type="comparison_operator" pos="280" length="1" label="&lt;"/>
							<tree type="method_invocation" pos="282" length="10">
								<tree type="identifier" pos="282" length="1" label="s"/>
								<tree type="identifier" pos="284" length="6" label="length"/>
								<tree type="argument_list" pos="290" length="2"/>
							</tree>
						</tree>
						<tree type="update_expression" pos="294" length="3">
							<tree type="identifier" pos="294" length="1" label="i"/>
							<tree type="increment_operator" pos="295" length="2" label="++"/>
						</tree>
						<tree type="block" pos="299" length="269">
							<tree type="expression_statement" pos="309" length="49">
								<tree type="method_invocation" pos="309" length="48">
									<tree type="identifier" pos="309" length="2" label="sb"/>
									<tree type="identifier" pos="312" length="6" label="append"/>
									<tree type="argument_list" pos="318" length="39">
										<tree type="method_invocation" pos="319" length="37">
											<tree type="identifier" pos="319" length="6" label="String"/>
											<tree type="identifier" pos="326" length="6" label="format"/>
											<tree type="argument_list" pos="332" length="24">
												<tree type="string_literal" pos="333" length="4" label="&quot;%X&quot;"/>
												<tree type="," pos="337" length="1" label=","/>
												<tree type="method_invocation" pos="339" length="16">
													<tree type="identifier" pos="339" length="1" label="s"/>
													<tree type="identifier" pos="341" length="11" label="codePointAt"/>
													<tree type="argument_list" pos="352" length="3">
														<tree type="identifier" pos="353" length="1" label="i"/>
													</tree>
												</tree>
											</tree>
										</tree>
									</tree>
								</tree>
							</tree>
							<tree type="if_statement" pos="367" length="125">
								<tree type="if" pos="367" length="2" label="if"/>
								<tree type="parenthesized_expression" pos="370" length="40">
									<tree type="binary_expression" pos="371" length="38">
										<tree type="method_invocation" pos="371" length="16">
											<tree type="identifier" pos="371" length="1" label="s"/>
											<tree type="identifier" pos="373" length="11" label="codePointAt"/>
											<tree type="argument_list" pos="384" length="3">
												<tree type="identifier" pos="385" length="1" label="i"/>
											</tree>
										</tree>
										<tree type="comparison_operator" pos="388" length="1" label="&gt;"/>
										<tree type="field_access" pos="390" length="19">
											<tree type="identifier" pos="390" length="9" label="Character"/>
											<tree type="identifier" pos="400" length="9" label="MAX_VALUE"/>
										</tree>
									</tree>
								</tree>
								<tree type="block" pos="411" length="81">
									<tree type="expression_statement" pos="425" length="4">
										<tree type="update_expression" pos="425" length="3">
											<tree type="identifier" pos="425" length="1" label="i"/>
											<tree type="increment_operator" pos="426" length="2" label="++"/>
										</tree>
									</tree>
									<tree type="block_comment" pos="430" length="52" label="/****WE HATES UTF-16! WE HATES IT FOREVERSES!!!****/"/>
								</tree>
							</tree>
							<tree type="if_statement" pos="501" length="61">
								<tree type="if" pos="501" length="2" label="if"/>
								<tree type="parenthesized_expression" pos="504" length="18">
									<tree type="binary_expression" pos="505" length="16">
										<tree type="binary_expression" pos="505" length="3">
											<tree type="identifier" pos="505" length="1" label="i"/>
											<tree type="arithmetic_operator" pos="506" length="1" label="+"/>
											<tree type="decimal_integer_literal" pos="507" length="1" label="1"/>
										</tree>
										<tree type="comparison_operator" pos="509" length="1" label="&lt;"/>
										<tree type="method_invocation" pos="511" length="10">
											<tree type="identifier" pos="511" length="1" label="s"/>
											<tree type="identifier" pos="513" length="6" label="length"/>
											<tree type="argument_list" pos="519" length="2"/>
										</tree>
									</tree>
								</tree>
								<tree type="block" pos="523" length="39">
									<tree type="expression_statement" pos="537" length="15">
										<tree type="method_invocation" pos="537" length="14">
											<tree type="identifier" pos="537" length="2" label="sb"/>
											<tree type="identifier" pos="540" length="6" label="append"/>
											<tree type="argument_list" pos="546" length="5">
												<tree type="string_literal" pos="547" length="3" label="&quot;.&quot;"/>
											</tree>
										</tree>
									</tree>
								</tree>
							</tree>
						</tree>
					</tree>
					<tree type="return_statement" pos="573" length="21">
						<tree type="method_invocation" pos="580" length="13">
							<tree type="identifier" pos="580" length="2" label="sb"/>
							<tree type="identifier" pos="583" length="8" label="toString"/>
							<tree type="argument_list" pos="591" length="2"/>
						</tree>
					</tree>
				</tree>
			</tree>
		</tree>
	</tree>
</tree>

When I looked into the XMLEventReader in TreeIoUtils.java, I got <?xml version="null" encoding='null' standalone='no'?> as the first element. Strangely, r.hasNext() returns true while I got parse error when I run nextEvent().

[jrfaller]

OK strange, the XML your parsed seems correct