Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

More XML formatting and GH Actions fix #61

Merged
merged 16 commits into from
Jun 23, 2023
Merged

More XML formatting and GH Actions fix #61

Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
0529374
Fix XML formatting configuration.
khatchad Jun 22, 2023
6539ab7
Format XML.
khatchad Jun 22, 2023
e1037d5
Fix directory handling in GitHub Actions config.
khatchad Jun 22, 2023
d74b32b
Ignore pom.xml in XML formatting.
khatchad Jun 22, 2023
b27e98a
Use /tmp in GH Actions build.
khatchad Jun 22, 2023
39ac40e
Use ${{ runner.temp }} instead of /tmp.
khatchad Jun 22, 2023
015ac51
Revert "Use ${{ runner.temp }} instead of /tmp."
khatchad Jun 22, 2023
99d41bb
Revert "Use /tmp in GH Actions build."
khatchad Jun 22, 2023
aa5261b
Don't format jython3 stuff.
khatchad Jun 22, 2023
c7e9e96
Try to exclude the specific file on the CI.
khatchad Jun 22, 2023
2b624c0
Revert "Try to exclude the specific file on the CI."
khatchad Jun 22, 2023
565c84f
Revert "Don't format jython3 stuff."
khatchad Jun 22, 2023
7988b7a
Use git clone directly.
khatchad Jun 22, 2023
7135a16
More XML formatting.
khatchad Jun 22, 2023
88cab00
Also recurse .gitgnore and markdown files.
khatchad Jun 22, 2023
2be0356
Try depth 1 for Git cloning.
khatchad Jun 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 8 additions & 12 deletions .github/workflows/continuous-integration.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,27 +17,23 @@ jobs:
distribution: 'temurin'
cache: maven
- name: Checkout wala/IDE sources.
uses: actions/checkout@v3.5.2
with:
repository: wala/IDE
# fetch-depth: 50
path: ./IDE
run: git clone --depth=50 https://github.com/wala/IDE ${{ runner.temp }}/IDE
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the purpose of these changes to use an explicit git clone? I don't quite get what is going on here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the lack of context. The XML format checker is going into the Jython3 code that is being cloned inside the project directory and causing a failure. The idea then is to move the cloned external dependencies outside the project directory, as was done in the original Travis CI configuration. However, the GitHub Actions only allows cloning in the project directory. Thus, we reverted to the explicit clone, as was also done in the original Travis CI configuration.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Relevant lines from Travis CI config:

ML/.travis.yml

Lines 6 to 7 in 6c10952

- git clone --depth=50 https://github.com/wala/IDE /tmp/IDE
- git clone https://github.com/juliandolby/jython3.git /tmp/jython3

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it not work to keep using the checkout action but pass path: ${{ runner.temp }}/IDE instead of path: ./IDE?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not, unfortunately. That variable returns a path outside the project directory but in the working space.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

39ac40e

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, yes, this is indeed a limitation, sigh actions/checkout#197

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not --depth=1?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see why not. This was also in the original Travis CI config. Travis CI clones a history of 50 the get the project sources. But, I found this post. Looks like we should stick with 50?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the issue on that post is a concern for PR CI jobs. If it comes up we can switch to 50, but for now I say depth 1.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure? https://stackoverflow.com/questions/31278233/disadvantages-of-shallow-cloning-on-travis-and-other-ci-services#answer-53857834:~:text=Do%20not%20use%20a%20depth%3D1%20on%20your%20project%20for%20CI!

Do not use a depth=1 on your project for CI!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I guess it was 1 before! https://github.com/actions/checkout#:~:text=%23%20Default%3A%201-,fetch%2Ddepth%3A%20%27%27,-%23%20Whether%20to

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like there could be a problem with tags: actions/checkout#217 (comment).

We can try 1. I don't foresee having a bunch of queued up builds.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also really like using the concurrency setting for GitHub Actions:

https://github.com/wala/WALA/blob/89de874efbc3369097e03b85493f4a3beb8e1a44/.github/workflows/continuous-integration.yml#L7-L9

This ensures there is only one build in progress for any PR or on the master branch. If you push a new commit to the PR branch, it cancels the ongoing build before starting a new one. If we want additional safety we can go with depth=1 and also add this concurrency setting.

Copy link
Collaborator Author

@khatchad khatchad Jun 23, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we have this kind of thing on Travis CI (there, it's specified on the web and not the YAML). The problem we've had with it is for builds emanating from tags. If you have a deploy action that only works on tagged commits, then you bump the version to the SNAPSHOT version, the CI stops building the tagged version (that was supposed to be deployed) and starts building the new SNAPSHOT version. The result is that the deployment gets skipped. Otherwise, I also like the feature :).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we ever need to do some kind of continuous deployment on GitHub Actions from tags, I'm sure we can configure it with a completely different .yml file, and the concurrency setting from one file shouldn't affect the other one.

- name: Checkout juliandolby/jython3 sources.
uses: actions/checkout@v3.5.2
with:
repository: juliandolby/jython3
path: ./jython3
run: git clone https://github.com/juliandolby/jython3.git ${{ runner.temp }}/jython3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could do --depth=1 here too?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #61 (comment)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #61 (comment) :-)

- name: Install Jython3.
run: |
cd ./jython3
pushd ${{ runner.temp }}/jython3
ant
cd dist
pushd dist
mvn install:install-file -Dfile=./jython-dev.jar -DgroupId="org.python" -DartifactId="jython3" -Dversion="0.0.1-SNAPSHOT" -Dpackaging="jar" -DgeneratePom=true
popd
popd
shell: bash
- name: Install IDE.
run: |
cd ./IDE/com.ibm.wala.cast.lsp
pushd ${{ runner.temp }}/IDE/com.ibm.wala.cast.lsp
mvn clean install -B -q -DskipTests
popd
- name: Check formatting with spotless.
run: mvn spotless:check -B
- name: Build with Maven
Expand Down
42 changes: 14 additions & 28 deletions com.ibm.wala.cast.python.jython/data/functools.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,43 +4,29 @@
<summary-spec>
<classloader name="PythonLoader">
<class name="functools" allocatable="true">
<method name="import"
static="true"
descriptor="()Lfunctools;">
<new def="x" class="Lfunctools"/>
<method name="import" static="true" descriptor="()Lfunctools;">
<new def="x" class="Lfunctools" />

<new def="reduce" class="Lfunctools/functions/reduce"/>
<putfield class="LRoot"
field="reduce"
fieldType="LRoot"
ref="x"
value="reduce"/>
<new def="reduce" class="Lfunctools/functions/reduce" />
<putfield class="LRoot" field="reduce" fieldType="LRoot" ref="x" value="reduce" />

<return value="x"/>
<return value="x" />
</method>
</class>

<package name="functools/functions">

<class name="reduce" allocatable="true">
<method name="do" descriptor="()LRoot;" numArgs="3" paramNames="self lambda data">
<constant name="l" type="int" value="0"/>
<aaload ref="data" def="v1" type="LRoot" index="l"/>
<constant name="r" type="int" value="1"/>
<aaload ref="data" def="v2" type="LRoot" index="r"/>
<call class="LRoot"
name="do"
descriptor="()LRoot;"
type="virtual"
arg0="lambda"
arg1="v1"
arg2="v2"
numArgs="3"
def="v"/>
<return value="v"/>
</method>
<method name="do" descriptor="()LRoot;" numArgs="3" paramNames="self lambda data">
<constant name="l" type="int" value="0" />
<aaload ref="data" def="v1" type="LRoot" index="l" />
<constant name="r" type="int" value="1" />
<aaload ref="data" def="v2" type="LRoot" index="r" />
<call class="LRoot" name="do" descriptor="()LRoot;" type="virtual" arg0="lambda" arg1="v1" arg2="v2" numArgs="3" def="v" />
<return value="v" />
</method>
</class>

</package>
</classloader>
</summary-spec>
14 changes: 4 additions & 10 deletions com.ibm.wala.cast.python.jython/data/numpy_turtle.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,10 @@
<summary-spec>
<classloader name="PythonLoader">
<class name="numpy" allocatable="true">
<method name="import"
static="true"
descriptor="()Lnumpy;">
<new def="x" class="Lturtle"/>
<putfield class="Lturtle"
field="any"
fieldType="LRoot"
ref="x"
value="x"/>
<return value="x"/>
<method name="import" static="true" descriptor="()Lnumpy;">
<new def="x" class="Lturtle" />
<putfield class="Lturtle" field="any" fieldType="LRoot" ref="x" value="x" />
<return value="x" />
</method>
</class>
</classloader>
Expand Down
41 changes: 15 additions & 26 deletions com.ibm.wala.cast.python.jython/data/pandas.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,44 +4,33 @@
<summary-spec>
<classloader name="PythonLoader">
<class name="pandas" allocatable="true">
<method name="import"
static="true"
descriptor="()Lpandas;">
<new def="x" class="Lpandas"/>
<method name="import" static="true" descriptor="()Lpandas;">
<new def="x" class="Lpandas" />

<new def="read_excel" class="Lpandas/functions/read_excel"/>
<putfield class="LRoot"
field="read_excel"
fieldType="LRoot"
ref="x"
value="read_excel"/>
<new def="read_excel" class="Lpandas/functions/read_excel" />
<putfield class="LRoot" field="read_excel" fieldType="LRoot" ref="x" value="read_excel" />

<new def="merge" class="Lpandas/functions/merge"/>
<putfield class="LRoot"
field="merge"
fieldType="LRoot"
ref="x"
value="merge"/>
<new def="merge" class="Lpandas/functions/merge" />
<putfield class="LRoot" field="merge" fieldType="LRoot" ref="x" value="merge" />

<return value="x"/>
<return value="x" />
</method>
</class>

<package name="pandas/functions">

<class name="read_excel" allocatable="true">
<method name="do" descriptor="()LRoot;" numArgs="3" paramNames="self file sheet">
<new def="v" class="Lobject"/>
<return value="v"/>
</method>
<method name="do" descriptor="()LRoot;" numArgs="3" paramNames="self file sheet">
<new def="v" class="Lobject" />
<return value="v" />
</method>
</class>

<class name="merge" allocatable="true">
<method name="do" descriptor="()LRoot;" numArgs="5"
paramNames="self left right on how">
<return value="left"/>
<return value="right"/>
</method>
<method name="do" descriptor="()LRoot;" numArgs="5" paramNames="self left right on how">
<return value="left" />
<return value="right" />
</method>
</class>
</package>
</classloader>
Expand Down
162 changes: 48 additions & 114 deletions com.ibm.wala.cast.python.jython/data/turtles.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,146 +4,80 @@
<summary-spec>
<classloader name="PythonLoader">
<class name="numpy" allocatable="true">
<method name="import"
static="true"
descriptor="()Lnumpy;">
<new def="x" class="Lturtle"/>
<putfield class="Lturtle"
field="any"
fieldType="LRoot"
ref="x"
value="x"/>
<return value="x"/>
<method name="import" static="true" descriptor="()Lnumpy;">
<new def="x" class="Lturtle" />
<putfield class="Lturtle" field="any" fieldType="LRoot" ref="x" value="x" />
<return value="x" />
</method>
</class>
<class name="sklearn" allocatable="true">
<method name="import"
static="true"
descriptor="()Lsklearn;">
<new def="x" class="Lturtle"/>
<putfield class="Lturtle"
field="any"
fieldType="LRoot"
ref="x"
value="x"/>
<return value="x"/>
<method name="import" static="true" descriptor="()Lsklearn;">
<new def="x" class="Lturtle" />
<putfield class="Lturtle" field="any" fieldType="LRoot" ref="x" value="x" />
<return value="x" />
</method>
</class>
<class name="pandas" allocatable="true">
<method name="import"
static="true"
descriptor="()Lpandas;">
<new def="x" class="Lturtle"/>
<putfield class="Lturtle"
field="any"
fieldType="LRoot"
ref="x"
value="x"/>
<return value="x"/>
<method name="import" static="true" descriptor="()Lpandas;">
<new def="x" class="Lturtle" />
<putfield class="Lturtle" field="any" fieldType="LRoot" ref="x" value="x" />
<return value="x" />
</method>
</class>
<class name="patsy" allocatable="true">
<method name="import"
static="true"
descriptor="()Lpatsy;">
<new def="x" class="Lturtle"/>
<putfield class="Lturtle"
field="any"
fieldType="LRoot"
ref="x"
value="x"/>
<return value="x"/>
<method name="import" static="true" descriptor="()Lpatsy;">
<new def="x" class="Lturtle" />
<putfield class="Lturtle" field="any" fieldType="LRoot" ref="x" value="x" />
<return value="x" />
</method>
</class>
<class name="seaborn" allocatable="true">
<method name="import"
static="true"
descriptor="()Lseaborn;">
<new def="x" class="Lturtle"/>
<putfield class="Lturtle"
field="any"
fieldType="LRoot"
ref="x"
value="x"/>
<return value="x"/>
<method name="import" static="true" descriptor="()Lseaborn;">
<new def="x" class="Lturtle" />
<putfield class="Lturtle" field="any" fieldType="LRoot" ref="x" value="x" />
<return value="x" />
</method>
</class>
<class name="statsmodels" allocatable="true">
<method name="import"
static="true"
descriptor="()Lstatsmodels;">
<new def="x" class="Lturtle"/>
<putfield class="Lturtle"
field="any"
fieldType="LRoot"
ref="x"
value="x"/>
<return value="x"/>
<class name="statsmodels" allocatable="true">
<method name="import" static="true" descriptor="()Lstatsmodels;">
<new def="x" class="Lturtle" />
<putfield class="Lturtle" field="any" fieldType="LRoot" ref="x" value="x" />
<return value="x" />
</method>
</class>
<class name="scipy" allocatable="true">
<method name="import"
static="true"
descriptor="()Lscipy;">
<new def="x" class="Lturtle"/>
<putfield class="Lturtle"
field="any"
fieldType="LRoot"
ref="x"
value="x"/>
<return value="x"/>
<class name="scipy" allocatable="true">
<method name="import" static="true" descriptor="()Lscipy;">
<new def="x" class="Lturtle" />
<putfield class="Lturtle" field="any" fieldType="LRoot" ref="x" value="x" />
<return value="x" />
</method>
</class>
<class name="matplotlib" allocatable="true">
<method name="import"
static="true"
descriptor="()Lmatplotlib;">
<new def="x" class="Lturtle"/>
<putfield class="Lturtle"
field="any"
fieldType="LRoot"
ref="x"
value="x"/>
<return value="x"/>
<class name="matplotlib" allocatable="true">
<method name="import" static="true" descriptor="()Lmatplotlib;">
<new def="x" class="Lturtle" />
<putfield class="Lturtle" field="any" fieldType="LRoot" ref="x" value="x" />
<return value="x" />
</method>
</class>
<class name="IPython" allocatable="true">
<method name="import"
static="true"
descriptor="()LIPython;">
<new def="x" class="Lturtle"/>
<putfield class="Lturtle"
field="any"
fieldType="LRoot"
ref="x"
value="x"/>
<return value="x"/>
<method name="import" static="true" descriptor="()LIPython;">
<new def="x" class="Lturtle" />
<putfield class="Lturtle" field="any" fieldType="LRoot" ref="x" value="x" />
<return value="x" />
</method>
</class>
<class name="mpl_toolkits" allocatable="true">
<method name="import"
static="true"
descriptor="()Lmpl_toolkits;">
<new def="x" class="Lturtle"/>
<putfield class="Lturtle"
field="any"
fieldType="LRoot"
ref="x"
value="x"/>
<return value="x"/>
<class name="mpl_toolkits" allocatable="true">
<method name="import" static="true" descriptor="()Lmpl_toolkits;">
<new def="x" class="Lturtle" />
<putfield class="Lturtle" field="any" fieldType="LRoot" ref="x" value="x" />
<return value="x" />
</method>
</class>
<class name="tensorflow" allocatable="true">
<method name="import"
static="true"
descriptor="()Ltensorflow;">
<new def="x" class="Lturtle"/>
<putfield class="Lturtle"
field="any"
fieldType="LRoot"
ref="x"
value="x"/>
<return value="x"/>
<method name="import" static="true" descriptor="()Ltensorflow;">
<new def="x" class="Lturtle" />
<putfield class="Lturtle" field="any" fieldType="LRoot" ref="x" value="x" />
<return value="x" />
</method>
</class>
</classloader>
Expand Down
Loading