Skip to content

vdk-oracle: ORA-01002: fetch out of sequence error in _cache_tables when some rows fail to ingest #2932

@DeltaMichael

Description

@DeltaMichael

Overview

While doing load testing, this error was happening sporadically and failing for 1% of ingestion rows. It might be due to unnecessary commit calls.

https://github.com/vmware/versatile-data-kit/blob/main/projects/vdk-plugins/vdk-oracle/src/vdk/plugin/oracle/ingest_to_oracle.py#L164

We probably don't need this commit call, because of commit being called on connection close anyway.

https://github.com/vmware/versatile-data-kit/blob/main/projects/vdk-plugins/vdk-oracle/src/vdk/plugin/oracle/oracle_connection.py#L56

This has to be tested, however.

How to reproduce

  1. Remove this try-except
  2. Ingest 1 000 000 records using a functional test where you generate the table id in python code, e.g.
for i in range(1000000):
    p = payload.copy()
    p["id"] = i
    job_input.send_object_for_ingestion(
        payload=p, destination_table="test_table"
    )
  1. Check result logs for ORA-01002

Acceptance criteria

  1. Reproduce the bug and fix it
  2. Remove the extra commit call and test that everything else works correctly

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions