Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move Spark-indpendent Table debug to cudf Java #13783

Merged
merged 10 commits into from
Aug 1, 2023

Conversation

gerashegalov
Copy link
Contributor

@gerashegalov gerashegalov commented Jul 28, 2023

Description

spark-rapids has code for debugging JNI Tables/Columns that is useful for debugging during dev work in cudf

This PR proposes to start moving it to cudf/java. spark-rapids will be updated to call into the cudf in a follow-up PR.

Sample Usage with JShell

(rapids) rapids@compose:~/cudf/java$ mvn dependency:build-classpath -Dmdep.outputFile=cudf-java-cp.txt
(rapids) rapids@compose:~/cudf/java$ jshell --class-path target/cudf-23.10.0-SNAPSHOT-cuda12.jar:$(< cudf-java-cp.txt) \
    -R -Dai.rapids.cudf.debug.output=log_error
|  Welcome to JShell -- Version 11.0.20
|  For an introduction type: /help intro

jshell> import ai.rapids.cudf.*;

jshell> Table tbl = new Table.TestBuilder().column(1,2,3,4,5,6).build()
tbl ==> Table{columns=[ColumnVector{rows=6, type=INT32, n ... e=140381937458144, rows=6}

jshell> TableDebug.get().debug("gera", tbl)
[main] ERROR ai.rapids.cudf.TableDebug - DEBUG gera Table{columns=[ColumnVector{rows=6, type=INT32, nullCount=Optional[0], offHeap=(ID: 4 7fad371d1a30)}], cudfTable=140381937458144, rows=6}
[main] ERROR ai.rapids.cudf.TableDebug - GPU COLUMN 0 - NC: 0 DATA: DeviceMemoryBufferView{address=0x7fad3be00000, length=24, id=-1} VAL: null
[main] ERROR ai.rapids.cudf.TableDebug - COLUMN 0 - INT32
[main] ERROR ai.rapids.cudf.TableDebug - 0 1
[main] ERROR ai.rapids.cudf.TableDebug - 1 2
[main] ERROR ai.rapids.cudf.TableDebug - 2 3
[main] ERROR ai.rapids.cudf.TableDebug - 3 4
[main] ERROR ai.rapids.cudf.TableDebug - 4 5
[main] ERROR ai.rapids.cudf.TableDebug - 5 6

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@gerashegalov gerashegalov requested a review from a team as a code owner July 28, 2023 18:11
@gerashegalov gerashegalov added the tests Unit testing for project label Jul 28, 2023
@github-actions github-actions bot added Java Affects Java cuDF API. and removed tests Unit testing for project labels Jul 28, 2023
@gerashegalov gerashegalov added tests Unit testing for project improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Jul 28, 2023
Copy link
Contributor

@revans2 revans2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fine. Just curious if we want to adjust/improve anything while we have the patient on the table?

Like passing in an output stream instead of always using stderr? Not that it is really needed.

@gerashegalov
Copy link
Contributor Author

Just curious if we want to adjust/improve anything while we have the patient on the table?

I am open to these and further suggestions

Signed-off-by: Gera Shegalov <gera@apache.org>
@gerashegalov gerashegalov self-assigned this Jul 31, 2023
Copy link
Contributor

@ttnghia ttnghia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are many nit formatting issues. Please check carefully before merging.

@gerashegalov
Copy link
Contributor Author

There are many nit formatting issues. Please check carefully before merging.

I am going to hit reformat with dev/cudf_java_styles.xml

@gerashegalov
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit 3254934 into rapidsai:branch-23.10 Aug 1, 2023
54 checks passed
@gerashegalov gerashegalov deleted the TableDebug branch August 1, 2023 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improvement / enhancement to an existing function Java Affects Java cuDF API. non-breaking Non-breaking change tests Unit testing for project
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants