Skip to content

[FEATURE][Lineage] Support Lateral view explode statement #4430

Closed
@iodone

Description

@iodone

Code of Conduct

Search before asking

  • I have searched in the issues and found no similar issues.

Describe the feature

Test case:

test("test lateral view explode") {
  withTable("t1", "t2") { _ =>
    spark.sql("CREATE TABLE t1 (a string, b string, c string) USING hive")
    spark.sql("CREATE TABLE t2 (a string, b string, c string) USING hive")
    val ret0 = exectractLineage("insert into t1 select 1,2,a.action " +
        "from t2 lateral view explode(split(c,'\\},\\{')) a as action")
    assert(ret0 == Lineage(
      List("default.t2"),
      List("default.t1"),
      List(
        ("default.t1.a", Set()),
        ("default.t1.b", Set()),
        ("default.t1.c", Set("default.t2.c"))
      )))
  }
}

Motivation

No response

Describe the solution

No response

Additional context

No response

Are you willing to submit PR?

  • Yes. I would be willing to submit a PR with guidance from the Kyuubi community to improve.
  • No. I cannot submit a PR at this time.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions