feat: enable ray by default

[xzdandy]

This pr intends to enable ray by default in EVA.

• Ray is enabled by default in eva server, test cases, notebooks
• Remove experimental code directory
• Introduce a new UDF decorator parralizable and uses it in the optimizer to filter out inexpensive functions.
  • This is not possible now. We need UDF decorator to be associated with the class instead of the setup function, so we can get the UDF properties without initializing the UDF.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[xzdandy]

Currently, the optimizer rule for ray does not work for UDFs in the projection's target list. So only the following two notebooks uses ray. Their results look expected.

• tutorials/03-emotion-analysis.ipynb
• tutorials/11-similarity-search-for-motif-mining.ipynb

The reason is that LogicalGetToSeqScan for ray does not make sense, since function expressions are in the LogicalProject instead of Logical Get. We need to create a LogicalProjectToPhysical for ray.
https://github.com/georgia-tech-db/eva/blob/e26c36dd71242de005e57baa3c4a6c2ab818d2d2/eva/experimental/parallel/optimizer/rules/rules.py#L90

[xzdandy]

@gaurav274 @jiashenC Any idea, why we have the ray skip tag for chatgpt test cases? Also, this seems to be unit test cases instead of integration test cases.

[gaurav274]

@gaurav274 @jiashenC Any idea, why we have the ray skip tag for chatgpt test cases? Also, this seems to be unit test cases instead of integration test cases.

It is a unit test because of the missing openai_api_key. Not sure about ray.

[gaurav274]

@jiashenC testcases are failing because of the coverage issue, right?

[jiashenC]

@jiashenC testcases are failing because of the coverage issue, right?

Yes, we probably still need to have a separate ray testing without generating coverage reports. Otherwise, it will cause timeout issue because of too many coverage trace to parse which are generated by many ray processes.

[jiashenC]

Just double check, so we won't get any functional expression in the SeqScan operator anymore?

[xzdandy]

Yes. It seems that the current statement to operator generation always creates a project operator.

[jarulraj]

We can get rid of experimental folder?

[xzdandy]

@jiashenC testcases are failing because of the coverage issue, right?

Yes, we probably still need to have a separate ray testing without generating coverage reports. Otherwise, it will cause timeout issue because of too many coverage trace to parse which are generated by many ray processes.

It is pending on local also, after all testcases are completed before the coverage report is generated. It seems that there may be some ray related resources are not correctly garbage cleaned.

[xzdandy]

We can get rid of experimental folder?

Make sense. We should get rid of it. Will do.

[xzdandy]

Update:

• Rollback the UDF decorator changes for ray. The current UDF decorator design can not support this feature in a proper manner.
• After removing the experimental directory, the coverage is dropping, given we are collecting coverage with ray disabled. Need to update .coveragerc and add some unit ray testcases even when ray is disabled.