Skip to content

[WIP] Omnisci on ray #1638

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ienkovich wants to merge 253 commits into from
Closed

[WIP] Omnisci on ray #1638

ienkovich wants to merge 253 commits into from

Conversation

@ienkovich
Copy link
Collaborator

@ienkovich ienkovich commented Jun 18, 2020

What do these changes do?

Add prototype backend and engine to use OmniSci as a computation engine

  • passes flake8 modin
  • passes black --check modin
  • signed commit with git commit -s
  • Resolves #?
  • tests added and passing

ienkovich and others added 30 commits May 19, 2020 17:50
@ienkovich
Concept draft which can process projections using OmniSci
764cf5d
@ienkovich
get OmniSci server path from env
62626de
@ienkovich
support view by row numeric idx in omnisci engine
e5d38fa
@ienkovich
basic groupby support with sum aggregate
6264146
@ienkovich
Merge pull request #3 from intel-go/ienkovich/omnisci/groupby-sum
5597f38 
basic groupby support with sum aggregate
@ienkovich
support simple fillna cases
b605f49
@ienkovich
formatting
2f84e31
@ienkovich
Merge pull request #6 from intel-go/ienkovich/omnisci/fillna
b41b864 
support simple fillna cases
Implement calcite serializer
f1bd121
Better type checking
bbfdcdc
@ienkovich
Merge pull request #8 from intel-go/fexolm/calcite_serializer
8db9814 
Implement calcite serializer
@ienkovich
fix serializer
533d837
@ienkovich
Merge pull request #9 from intel-go/ienkovich/fix-serialization
c890e55 
fix serializer
@Garra1980
Initial implementation of merge operation in omnisci engine - inner m…
db87402 
…erge is supported
@Garra1980
Make result joined DF correct with correct columns list
b1cfa1e
@Garra1980
Relax constraints on way of merging
96832b0
@Garra1980
Support for sort argument in a merge
06593bd
@Garra1980
Hack to make default_to_pandas merge work for pandas backend
541d65e
@Garra1980
Adjust JoinNode handling according to new expr interfaces
1c94cd7
@ienkovich
add union all (concat) support
01ccbfe
@ienkovich
Merge pull request #10 from intel-go/ienkovich/omnisci/concat
c4fbce1 
add union all (concat) support
@ienkovich
fix groupby aggregate type
22efe48
@ienkovich
Merge pull request #11 from intel-go/ienkovich/omnisci/fix-groupby-ag…
2b6b5fb 
…g-type

fix groupby aggregate type
@ienkovich
fix column index for groupby
fa15af5
@ienkovich
Merge pull request #13 from intel-go/ienkovich/omnisci/fix-groupby
846c490 
fix column index for groupby
@Garra1980
Merge branch 'omnisci-on-ray' into izamyati/omnisci-merge
36fe7d9
@Garra1980
Review comments addressed - some refactoring, rowid dropping, proper …
e6ebf98 
…filed for columns used
@ienkovich
support copy method for omnisci frame
f83df24
@ienkovich
Merge pull request #14 from intel-go/ienkovich/omnisci/frame-copy
9c459eb 
support copy method for omnisci frame
@Garra1980
Fix JoinNode printing and columns suffixing
c03ce9f
@ienkovich ienkovich marked this pull request as ready for review August 3, 2020 15:58
ienkovich added 21 commits August 3, 2020 11:50
@ienkovich
remove function duplicate
c1cb48a
@ienkovich
add missing licenses
ad6c120
@ienkovich
Merge pull request #77 from intel-go/ienkovich/fix-flake8
1a320ee 
remove function duplicate
@ienkovich
Merge pull request #78 from intel-go/ienkovich/add-licenses
0478b52 
add missing licenses
@ienkovich
use range index when possible
a66e39b
@ienkovich
Merge pull request #79 from intel-go/ienkovich/range-index
fed6ec2 
use range index when possible
@ienkovich
fix item access for series
4902ec2
@ienkovich
remove dup
e6c5672
@ienkovich
Merge pull request #81 from intel-go/ienkovich/gitignore
023b28d 
remove dup
@ienkovich
Merge pull request #80 from intel-go/ienkovich/fix-series-item-access
f0e866c 
fix item access for series
@ienkovich
fix/suppress flake8 errors
51e4592
@ienkovich
Merge pull request #82 from intel-go/ienkovich/fix-flake8
9ec3f77 
fix/suppress flake8 errors
@ienkovich
move shape_hint to omnisci backend
a69a266
@ienkovich
Merge pull request #83 from intel-go/ienkovich/shape_hint
4b3b3cd 
move shape_hint to omnisci backend
@ienkovich
remove merge artifact
96a5646
@ienkovich
Merge pull request #84 from intel-go/ienkovich/fix-factory
094547a 
remove merge artifact
@ienkovich
check for lazy execution in some tests
1a2f543
@ienkovich
add more checks for lazy execution
27e290e
@ienkovich
Merge pull request #85 from intel-go/ienkovich/force-lazy
da44cdf 
check for lazy execution in some tests
@ienkovich
enable from_arrow for pandas_on_ray
5108f89
@ienkovich
Merge pull request #86 from intel-go/ienkovich/from-arrow-fix
fd98cdc 
enable from_arrow for pandas_on_ray
@ienkovich ienkovich closed this Aug 12, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@devin-petersohn devin-petersohn Awaiting requested review from devin-petersohn devin-petersohn is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@ienkovich @modin-bot @Garra1980 @devin-petersohn @anton-malakhov @aregm