Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 264: Update qviz for multiblock files #437

Merged
Merged
Changes from 1 commit
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
42e11d6
initial commit
jorgeMarin1 Sep 6, 2024
b0fda5d
Merge branch 'Qbeast-io:main' into qviz-bug
jorgeMarin1 Sep 10, 2024
0fe8b08
initial commit
jorgeMarin1 Sep 6, 2024
5fb0956
added process table function using delta tables
jorgeMarin1 Sep 10, 2024
5f60426
create qviz using Delta tables
jorgeMarin1 Sep 10, 2024
0ee8232
added custom table
jorgeMarin1 Sep 10, 2024
30256f8
added custom table
jorgeMarin1 Sep 10, 2024
cf9cfc3
removed ecommerce300k_2019
jorgeMarin1 Sep 10, 2024
cbcbaed
fixed visualization
jorgeMarin1 Sep 12, 2024
92cf732
deleted parquet files and added a new folder for table test
jorgeMarin1 Sep 12, 2024
e9559f7
added comments on the code
jorgeMarin1 Sep 12, 2024
ee95b87
added comments to the code
jorgeMarin1 Sep 12, 2024
2f0aea8
Merge branch 'main' into qviz-bug
jorgeMarin1 Sep 12, 2024
a267f87
deleted code and files that won't be used
jorgeMarin1 Sep 12, 2024
032077c
added unit test
jorgeMarin1 Sep 13, 2024
398494d
addressed changes
jorgeMarin1 Sep 17, 2024
353011f
addressed changes
jorgeMarin1 Sep 18, 2024
13e95a7
snake case
jorgeMarin1 Sep 18, 2024
31ac8e1
edited example of json log file
jorgeMarin1 Sep 19, 2024
8ecfdae
raised exception when failed to create delta table
jorgeMarin1 Sep 19, 2024
6f824d4
edited name of the method that extracted cubes from delta tables
jorgeMarin1 Sep 19, 2024
cb0f1a5
made sure blocks_str is indeed a string
jorgeMarin1 Sep 19, 2024
5f06c45
put definition of cube_string before if statement
jorgeMarin1 Sep 19, 2024
d565fff
added else statement for block_string
jorgeMarin1 Sep 25, 2024
0178d1e
deleted non-necessary comments
jorgeMarin1 Sep 25, 2024
a1d75a7
changed path to test table in README
jorgeMarin1 Sep 25, 2024
512dd67
default value of sampling fraction is 0.02 and that value can be modi…
jorgeMarin1 Sep 25, 2024
22ff6b5
deleted unnecessary files
jorgeMarin1 Sep 25, 2024
1faa7c2
switched to version 3.2.0 of Delta Lake, which is compatible with 3.5…
jorgeMarin1 Sep 25, 2024
cda8c6d
update size of cubes using path of blocks
jorgeMarin1 Sep 26, 2024
b407ada
WIP
Jiaweihu08 Oct 14, 2024
1b5ee6b
Remove large file
Jiaweihu08 Oct 14, 2024
aa486ca
WPI, first working version
Jiaweihu08 Oct 16, 2024
4bce299
Organize code, add tests
Jiaweihu08 Oct 16, 2024
b6855b0
Update README
Jiaweihu08 Oct 17, 2024
db3ebe9
Add comments
Jiaweihu08 Oct 17, 2024
a6b88a7
Update poetry lock and toml
Jiaweihu08 Oct 17, 2024
bc4ef3c
Update README
Jiaweihu08 Oct 17, 2024
2727e64
Correct stats computation
Jiaweihu08 Oct 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
deleted non-necessary comments
  • Loading branch information
jorgeMarin1 committed Sep 25, 2024
commit 0178d1e7224288b412cd07cde6bf78a7217c7794
9 changes: 0 additions & 9 deletions utils/visualizer/qviz/qviz.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,15 +31,6 @@ def show_tree(path: str, revision_id: int) -> None:
revision_id > 0
), f"Invalid value for revision_id: {revision_id}, it must be > 0."

# Gather revision AddFiles from the table _delta_log/
# add_files, metadata = process_table_delta_log(path, str(revision_id))
# Process build Cubes from AddFiles
# cubes = process_add_files(add_files, metadata)
# Populate the index by creating parent-child references
# _ = populate_tree(cubes)
# Gather drawing elements from populated cubes
# elements = get_nodes_and_edges(cubes)

metadata, cubes, _, elements = process_table(path, revision_id)

# Callback that modifies the drawing elements
Expand Down