Unity Catalog Export: Reduce likelihood for action failure, due to race condition against databricks on table re-registration #9937

ItamarYuran · 2025-12-30T15:51:14Z

Closes https://github.com/treeverse/product/issues/1030

This PR tries to resolve two unwanted behaviors;

For each export to an existing table, two create table calls were made (first fails as the table exists) resulting in excessive unwanted error logs in databricks.

We try to overcome this be checking whether the table exists first and remove it if so.

The second issue is inconsistency in case of re-registration of a table - after successfully deleting a table and trying to create a new one, databricks returns an answer that the table already exists.
lakeFS action logs show
Error: action failed: runtime error: [string "lua"]:60: external table "intervaldata" creation failed: FAILED: BAD_REQUEST [TABLE_OR_VIEW_ALREADY_EXISTS] Cannot create table or view `prod_datahub`.`ami`.`intervaldata` because it already exists.

which means the client had to fail here after already deleting the table.

arielshaqed

Unfortunately I do not believe this resolves the race. Specifically, if multiple concurrent Lua hooks try to create-or-replace, then it is possible for one to succeed, one to fail, and no table exist. It might work if you have more retries than concurrent hooks running... but that of course is not a bounded number (and the higher the load, the more concurrent hooks we expect).

Thread A	Thread B
CREATE TABLE T	-
table exists	-
-	CREATE TABLE T
-	table exists
dropIfExists T	dropIfExists T
CREATE TABLE T	CREATE TABLE T
succeeds!	table exists
-	dropIfExists T
-	table T no longer exists
-	CREATE TABLE T
-	some random error

Result: A succeeded, B failed, table does not exist. This is not a possible result for two concurrent CREATE OR REPLACE TABLE T calls.

arielshaqed · 2025-12-30T16:22:07Z

pkg/actions/lua/databricks/client.go

+	bo.MaxInterval = 3 * time.Second
+	bo.MaxElapsedTime = 10 * time.Second
+
+	deleteAndCreate := func() (string, error) {


createOrDelete?

arielshaqed · 2025-12-30T16:32:50Z

If there is no direct programmatic access to the CREATE OR REPLACE DDL command, we could use ExecuteStatement. DataBricks SQL lets you parametrize table names - this is how to parametrize an identifier.

ItamarYuran · 2025-12-30T16:38:01Z

@arielshaqed This indeed does not solve the race between two running actions (though I do think it can decrease the amount of errors caused by such race). This tries to resolve timing issues where Databricks delete operations on large tables may not complete immediately, causing the retry to still see the table as existing.

arielshaqed · 2025-12-30T16:43:42Z

@arielshaqed This indeed does not solve the race between two running actions (though I do think it can decrease the amount of errors caused by such race). This tries to resolve timing issues where Databricks delete operations on large tables may not complete immediately, causing the retry to still see the table as existing.

Then please change the title "Fix: Export unity catalog hook race condition" and the part of the description which says "[t]he second issue is a race condition between the deletion of an existing table and a creation of a new one."
Please see my second comment. Fixing a race by retrying often does not help: the race occurs when load is high, and retrying increases that load. I think that comment gives a way to fix the race.

ItamarYuran · 2025-12-30T16:50:40Z

Thank you @arielshaqed, Using CREATE OR REPLACE was the option I considered at first, but unfortunately this will not work well, because there are strict limitation on the replaced scheme's structure.

"For Delta tables, the table inherits its configuration from the LOCATION if data already exists at that path. As a result, any specified TBLPROPERTIES, table_specification, or PARTITIONED BY clauses must exactly match the existing data at the Delta location." see docs

nopcoder · 2025-12-30T19:52:46Z

pkg/actions/lua/databricks/client.go

+	if err != nil {
+		return err
+	}
+	return nil


I see dead code

nopcoder · 2025-12-30T19:54:15Z

pkg/actions/lua/databricks/client.go

+func (client *Client) deleteTableIfExists(catalogName, schemaName, tableName string) error {
+	err := client.deleteTable(catalogName, schemaName, tableName)
+	if errors.Is(err, databricks.ErrResourceDoesNotExist) {
+		return nil
+	}


Don't see reason for this function as it just ignore 'not exists' while we add a check if table exists in this flow.

If we check existance and we delete the table and ignore the result, we will fail in the creation - so I don't see a reason to ignore the error.

nopcoder · 2025-12-30T19:56:50Z

pkg/actions/lua/databricks/client.go

+func (client *Client) checkTableExists(catalogName, schemaName, tableName string) bool {
+	table, err := client.workspaceClient.Tables.GetByFullName(client.ctx, tableFullName(catalogName, schemaName, tableName))
+	if err == nil && table != nil {
+		return true
+	}
+	return false


we ignore the error here

when you return a boolean value you can return the expression result.

Thanks, indeed looking better this way. still ignoring the err as if there is an unrelated err it will get caught when creating a table, is that ok?

no we should not ignore the error as it will not help us understand if something is not working as expected.
in this case we are checking that the table exists and delete the table - if we fail we need to know why, no matter the reason.

nopcoder · 2025-12-30T20:00:39Z

pkg/actions/lua/databricks/client.go

-				panic("unreachable")
-			}
-		} else {
+	// try and delete the table first if exists, to prevent "table already exists" errors in databricks


Without "try". Delete the table if exists.

nopcoder · 2025-12-30T20:02:24Z

pkg/actions/lua/databricks/client.go

 	return 1
 }

+func (client *Client) createExternalTableWithBackoff(warehouseID, catalogName, schemaName, tableName, location string, metadata map[string]any) (string, error) {


Don't need this function to use retry

nopcoder · 2025-12-30T20:05:11Z

pkg/actions/lua/databricks/client.go

+			if deletionErr != nil {
+				return "", backoff.Permanent(deletionErr)
+			}
+			return "", err


if we think the system is eventual consist, and we create+delete in a loop it may fail as we can delete the table we just created.

we like to try only the table creation as discussed.

True, Thank you.

nopcoder · 2025-12-31T06:08:58Z

pkg/actions/lua/databricks/client.go

+	status, err := backoff.RetryWithData(createTableBO, bo)
+
+	if err != nil {


remove whitespace

Suggested change

status, err := backoff.RetryWithData(createTableBO, bo)

if err != nil {

status, err := backoff.RetryWithData(createTableBO, bo)

if err != nil {

nopcoder · 2025-12-31T06:15:19Z

pkg/actions/lua/databricks/client.go


+func (client *Client) checkTableExists(catalogName, schemaName, tableName string) bool {
+	_, err := client.workspaceClient.Tables.GetByFullName(client.ctx, tableFullName(catalogName, schemaName, tableName))
+	return err == nil


If there is additional code this function just calls a function, so I'm not sure we need it.

We ignore errors here, so there is no difference between errors access the service and table not exists

nopcoder · 2025-12-31T06:21:49Z

pkg/actions/lua/databricks/client.go

+	bo := backoff.NewExponentialBackOff()
+	bo.InitialInterval = 100 * time.Millisecond
+	bo.MaxInterval = 3 * time.Second
+	bo.MaxElapsedTime = 10 * time.Second


need to also add the context

nopcoder · 2025-12-31T06:24:58Z

pkg/actions/lua/databricks/client.go

+		status, err := client.createExternalTable(warehouseID, catalogName, schemaName, tableName, location, metadataMap)
+		if err == nil {
+			return status, nil
+		}
+		if alreadyExists(err) {
+			return "", err
+		}
+		return "", backoff.Permanent(err)
+	}


prefer to check error first

Suggested change

status, err := client.createExternalTable(warehouseID, catalogName, schemaName, tableName, location, metadataMap)

if err == nil {

return status, nil

}

if alreadyExists(err) {

return "", err

}

return "", backoff.Permanent(err)

}

status, err := client.createExternalTable(warehouseID, catalogName, schemaName, tableName, location, metadataMap)

if err != nil {

if alreadyExists(err) {

return "", err

}

return "", backoff.Permanent(err)

}

return status, nil

}

ItamarYuran · 2025-12-31T08:04:05Z

@nopcoder , thanks, now panicing if we get non related err when checkin its existence. Is that ok?

nopcoder

minor comments; note that you call panic in the code to mark this code as a return as it will not get to that point, the errorf will panic.

nopcoder · 2025-12-31T09:10:17Z

pkg/actions/lua/databricks/client.go

-			}
-		} else {
+
+	// delete the table first if exists, to prevent "table already exists" errors in databricks


this comment is for the delete request.
here you need to comment that this call is done to check if table exists.

nopcoder · 2025-12-31T09:23:37Z

pkg/actions/lua/databricks/client.go

+	_, err := client.workspaceClient.Tables.GetByFullName(client.ctx, tableFullName(catalogName, schemaName, tableName))
+	if err != nil && !errors.Is(err, databricks.ErrResourceDoesNotExist) {
+		lua.Errorf(l, "%s", err.Error())
+		panic("unreachable")
+	}
+	if err == nil {


comment on the condition at the end that if no error, table exists and we need to delete it before creation.

note: the previous version had it in a function that just was a call, you can keep the function if you like to include the "is exists" logic, but you need to report the error.

ItamarYuran · 2025-12-31T10:50:34Z

Thanks again @nopcoder, kept the table checking without the func and adjusted the comments.
Do the backoff params seem reasonable to you?

nopcoder

added code style comment
about the backoff parameters - check the logs to see if it is ok to perform ~6 calls in 2 seconds. this pr is assumption about eventual consist; so I suggest to reproduce first.
if there are any pending comments from Ariel please review before merge.

nopcoder · 2025-12-31T10:54:13Z

pkg/actions/lua/databricks/client.go

+	var bo backoff.BackOff = backoff.NewExponentialBackOff(
+		backoff.WithInitialInterval(100*time.Millisecond), backoff.WithMaxInterval(3*time.Second), backoff.WithMaxElapsedTime(10*time.Second))
+	bo = backoff.WithContext(bo, client.ctx)
+
+	createTableBO := func() (string, error) {
+		status, err := client.createExternalTable(warehouseID, catalogName, schemaName, tableName, location, metadataMap)
+		if err != nil {
+			if alreadyExists(err) {
+				return "", err
+			}
+			return "", backoff.Permanent(err)
+		}
+		return status, nil
+	}
+
+	status, err := backoff.RetryWithData(createTableBO, bo)
+	if err != nil {
+		lua.Errorf(l, "%s", err.Error())
+		panic("unreachable")
+	}


style

Suggested change

var bo backoff.BackOff = backoff.NewExponentialBackOff(

backoff.WithInitialInterval(100*time.Millisecond), backoff.WithMaxInterval(3*time.Second), backoff.WithMaxElapsedTime(10*time.Second))

bo = backoff.WithContext(bo, client.ctx)

createTableBO := func() (string, error) {

status, err := client.createExternalTable(warehouseID, catalogName, schemaName, tableName, location, metadataMap)

if err != nil {

if alreadyExists(err) {

return "", err

}

return "", backoff.Permanent(err)

}

return status, nil

}

status, err := backoff.RetryWithData(createTableBO, bo)

if err != nil {

lua.Errorf(l, "%s", err.Error())

panic("unreachable")

}

bo := backoff.NewExponentialBackOff(

backoff.WithInitialInterval(100*time.Millisecond),

backoff.WithMaxInterval(3*time.Second),

backoff.WithMaxElapsedTime(10*time.Second),

)

status, err := backoff.RetryWithData(func() (string, error) {

status, err := client.createExternalTable(warehouseID, catalogName, schemaName, tableName, location, metadataMap)

if err != nil {

if alreadyExists(err) {

return "", err

}

return "", backoff.Permanent(err)

}

return status, nil

}, backoff.WithContext(bo, client.ctx))

if err != nil {

lua.Errorf(l, "%s", err.Error())

panic("unreachable")

}

arielshaqed · 2025-12-31T11:35:26Z

Thank you @arielshaqed, Using CREATE OR REPLACE was the option I considered at first, but unfortunately this will not work well, because there are strict limitation on the replaced scheme's structure.

"For Delta tables, the table inherits its configuration from the LOCATION if data already exists at that path. As a result, any specified TBLPROPERTIES, table_specification, or PARTITIONED BY clauses must exactly match the existing data at the Delta location." see docs

This appears under "LOCATION", so I don't understand how this affects CREATE OR REPLACE but not CREATE. If I create a new table with an existing location, it will still "inherit[s] its configuration from the LOCATION if data already exists at that path".
Assuming I am wrong and this is a limitation of CREATE OR REPLACE: Why are we implementing an additional SQL feature for DataBricks? I understand supporting anything DataBricks support; I do not think we should add something. What's the customer requirement here?

ItamarYuran · 2025-12-31T14:45:27Z

Thank you @arielshaqed, Using CREATE OR REPLACE was the option I considered at first, but unfortunately this will not work well, because there are strict limitation on the replaced scheme's structure.
"For Delta tables, the table inherits its configuration from the LOCATION if data already exists at that path. As a result, any specified TBLPROPERTIES, table_specification, or PARTITIONED BY clauses must exactly match the existing data at the Delta location." see docs

This appears under "LOCATION", so I don't understand how this affects CREATE OR REPLACE but not CREATE. If I create a new table with an existing location, it will still "inherit[s] its configuration from the LOCATION if data already exists at that path".

Assuming I am wrong and this is a limitation of CREATE OR REPLACE: Why are we implementing an additional SQL feature for DataBricks? I understand supporting anything DataBricks support; I do not think we should add something. What's the customer requirement here?

Tested and verified, databricks sql does not support CREATE OR REPLACE EXTERNAL TABLE:

2025/12/31 15:33:28 Uncaught Lua error: runtime error: external table "source_table" creation failed: FAILED: BAD_REQUEST 
[PARSE_SYNTAX_ERROR] Syntax error at or near 'EXTERNAL'. SQLSTATE: 42601 (line 1, pos 18)

costumers complained about two thing

actions occasionally fail - this is due to a table that was supposed to be deleted according to our logics - but wasn't. I am convinced that this is ours to correct
secondly, due to our create - delete - create logic, clients suffered from excessive logs of failing table creations.

This PR does not add support to any new feature, only fixes somewhat broken ones

arielshaqed

New code has less risk of a really bad outcome to the race that the previous version on this PR. But it will (almost?) always give the same result as before: the race still exists, the winner will always create the table, the loser will never create the table, it will just take longer for the loser to lose.

If we want to do this, you might perform the change I suggest about deletions and also get rid of the retries when creating the table: they do nothing.

arielshaqed · 2026-01-01T12:19:58Z

pkg/actions/lua/databricks/client.go

+	// check whether the the table already exists
+	_, err := client.workspaceClient.Tables.GetByFullName(client.ctx, tableFullName(catalogName, schemaName, tableName))
+	if err != nil && !errors.Is(err, databricks.ErrResourceDoesNotExist) && !strings.Contains(err.Error(), "does not exist") {
+		lua.Errorf(l, "%s", err.Error())
+		panic("unreachable")
+	}
+	// in case there is no error, table exists and we will delete it before creating a new one
+	if err == nil {
+		err = client.deleteTable(catalogName, schemaName, tableName)
+		if err != nil {


You anyway want to delete the table. Checking if it exists first is just another race. Instead, delete the table, keep going if DataBricks say it does not exist.

ItamarYuran · 2026-01-01T15:16:52Z

@arielshaqed @nopcoder @Isan-Rivkin

Running a simple script to push external table to databricks (from canadacentral) both with the original client and the one in this pr. Getting weird, inconsistent behaviors.

An important observation is that no failures occurred when running the test locally, only when running in canada central.

for the original client, I did manage to reproduce the original problem:

registering table iteration 0
2026/01/01 14:16:04 Uncaught Lua error: runtime error: failed deleting an existing table "source_table": Table 'main.test_schema.source_table' does not exist.
panic: Uncaught Lua error: runtime error: failed deleting an existing table "source_table": Table 'main.test_schema.source_table' does not exist.

This happens though only it the initial iteration (after first success it continues registering successfully)

for the client in this pr I got for example

2026/01/01 13:46:21 Uncaught Lua error: runtime error: failed deleting an existing table "source_table": Table 'main.test_schema.source_table' does not exist.

meaning table's existence was checked, then upon deletion, the table was absent (added a check that the error is not does not exists and it did not fail for that anymore )

It also sometimes failed after a couple of iterations over

registering table iteration 10
2026/01/01 14:22:16 Uncaught Lua error: runtime error: external table "source_table" creation failed: FAILED: BAD_REQUEST [TABLE_OR_VIEW_ALREADY_EXISTS] Cannot create table or view `main`.`test_schema`.`source_table` because it already exists.
Choose a different name, drop the existing object, add the IF NOT EXISTS clause to tolerate pre-existing objects, add the OR REPLACE clause to replace the existing materialized view, or add the OR REFRESH clause to refresh the existing streaming table. SQLSTATE: 42P07

also failed like this in the first iteration but less times than the original client

All in all, results were very inconsistent. failures appeared for both clients. Failures also tend to group over few consecutive runs. the original client did fail more.

UPDATE
Played with the backoff params and made elapsedtime = 100s so far haven't witnessed any failures. Tables sometime register after 40-50 seconds

ItamarYuran · 2026-01-01T15:53:11Z

The test func:

package databricks

import (
	"context"
	"fmt"
	"math/rand"
	"os"
	"strings"
	"time"

	"github.com/Shopify/go-lua"
	"github.com/databricks/databricks-sdk-go"
	"github.com/databricks/databricks-sdk-go/config"
)

// TestLuaRegisterExternalTable tests the actual RegisterExternalTable Lua function
func TestLuaRegisterExternalTable() {
	host := 
	token := 
	warehouseID := 
	catalog := "main"
	schema := "test_schema"
	tableName := "source_table"

	ctx := context.Background()
	w, err := databricks.NewWorkspaceClient(&databricks.Config{
		Host:        host,
		Token:       token,
		Credentials: config.PatCredentials{},
	})
	if err != nil {
		fmt.Printf("Failed: %v\n", err)
		os.Exit(1)
	}

	client := &Client{workspaceClient: w, ctx: ctx}

	// Create schema
	_, err = client.createSchema(catalog, schema, true)
	if err != nil {
		fmt.Printf("Failed to create schema: %v\n", err)
		os.Exit(1)
	}

	// Generate unique location for each test run
	timestamp := time.Now().Format("20060102_150405")
	sourceLocation := fmt.Sprintf("source locatoin")
	targetTableName := tableName

	fmt.Println("\n=== Step 1: Creating Delta files on storage with random schema ===")

	// Generate random schema
	rand.Seed(time.Now().UnixNano())
	numColumns := rand.Intn(5) + 3 // 3-7 columns
	columnTypes := []string{"INT", "STRING", "DOUBLE", "BOOLEAN", "DATE"}
	columnDefs := make([]string, numColumns)

	fmt.Println("Random Schema:")
	for i := 0; i < numColumns; i++ {
		colType := columnTypes[rand.Intn(len(columnTypes))]
		colName := fmt.Sprintf("col_%d_%s", i, strings.ToLower(colType))
		columnDefs[i] = fmt.Sprintf("%s %s", colName, colType)
		fmt.Printf("  %s\n", columnDefs[i])
	}
	fmt.Println()

	// Create a temporary table to populate Delta files on storage
	tempTableName := "temp_source_creator"
	dropTempSQL := fmt.Sprintf("DROP TABLE IF EXISTS %s.%s.%s", catalog, schema, tempTableName)
	_, _ = client.executeStatement(warehouseID, catalog, schema, dropTempSQL)

	createTempSQL := fmt.Sprintf(`
		CREATE TABLE %s.%s.%s (
			%s
		) USING DELTA LOCATION '%s'
	`, catalog, schema, tempTableName, strings.Join(columnDefs, ", "), sourceLocation)

	_, err = client.executeStatement(warehouseID, catalog, schema, createTempSQL)
	if err != nil {
		fmt.Printf("Failed to create Delta files: %v\n", err)
		os.Exit(1)
	}

	// Generate random data
	insertValues := make([]string, 5) // 5 rows
	for i := 0; i < 5; i++ {
		values := make([]string, numColumns)
		for j := 0; j < numColumns; j++ {
			colType := strings.ToUpper(strings.Split(columnDefs[j], " ")[1])
			switch colType {
			case "INT":
				values[j] = fmt.Sprintf("%d", rand.Intn(1000))
			case "STRING":
				values[j] = fmt.Sprintf("'value_%d'", rand.Intn(100))
			case "DOUBLE":
				values[j] = fmt.Sprintf("%.2f", rand.Float64()*100)
			case "BOOLEAN":
				values[j] = fmt.Sprintf("%t", rand.Intn(2) == 1)
			case "DATE":
				values[j] = fmt.Sprintf("'2024-%02d-%02d'", rand.Intn(12)+1, rand.Intn(28)+1)
			}
		}
		insertValues[i] = fmt.Sprintf("(%s)", strings.Join(values, ", "))
	}

	insertSQL := fmt.Sprintf(`
		INSERT INTO %s.%s.%s VALUES
		%s
	`, catalog, schema, tempTableName, strings.Join(insertValues, ", "))

	_, err = client.executeStatement(warehouseID, catalog, schema, insertSQL)
	if err != nil {
		fmt.Printf("Failed to insert data: %v\n", err)
		os.Exit(1)
	}

	// Drop the temporary table (Delta files remain on storage!)
	_, _ = client.executeStatement(warehouseID, catalog, schema, dropTempSQL)

	fmt.Printf("✓ Delta files created at: %s\n", sourceLocation)
	fmt.Printf("✓ Now will register '%s' as external table pointing to this location\n", targetTableName)

	// Create Lua state
	l := lua.NewState()
	lua.OpenLibraries(l)

	fmt.Println("\n=== Step 2: Testing Lua RegisterExternalTable ===")
	fmt.Printf("Registering '%s' as EXTERNAL TABLE pointing to source location\n\n", targetTableName)

	// Push parameters onto Lua stack as RegisterExternalTable expects
	l.PushString(targetTableName)
	l.PushString(sourceLocation)
	l.PushString(warehouseID)
	l.PushString(catalog)
	l.PushString(schema)

	// Push metadata table
	l.NewTable()
	l.PushString("External table registered from existing Delta files")
	l.SetField(-2, "description")

	fmt.Printf("Target Table: %s\nSource Location: %s\n\n", targetTableName, sourceLocation)
	// Call RegisterExternalTable
	for i := range 1000 {
		fmt.Printf("registering table iteration %d \n", i)

		numResults := client.RegisterExternalTable(l)

		if numResults > 0 && l.IsString(-1) {
			status := lua.CheckString(l, -1)
			fmt.Printf("✅ Success!\n")
			fmt.Printf("External Table: %s.%s.%s\n", catalog, schema, targetTableName)
			fmt.Printf("Status: %s\n", status)
			fmt.Printf("Points to: %s\n\n", sourceLocation)
			fmt.Println("You can now query the table in Databricks:")
			fmt.Printf("  SELECT * FROM %s.%s.%s;\n", catalog, schema, targetTableName)
		} else {
			fmt.Println("❌ Failed: No result returned")
			os.Exit(1)
		}
	}
}

ItamarYuran · 2026-01-03T00:14:15Z

@arielshaqed Using only databricks's api to delete and create external table. Running into the same errors:

Iteration 23: Dropping... Creating... ✓ Success: SUCCEEDED
Iteration 24: Dropping... Drop failed: Table 'main.test_schema.my_external_table' does not exist. | Creating... ❌ Failed: FAILED: BAD_REQUEST [TABLE_OR_VIEW_ALREADY_EXISTS] Cannot create table or view `main`.`test_schema`.`my_external_table` because it already exists.
Choose a different name, drop the existing object, add the IF NOT EXISTS clause to tolerate pre-existing objects, add the OR REPLACE clause to replace the existing materialized view, or add the OR REFRESH clause to refresh the existing streaming table. SQLSTATE: 42P07
Iteration 25: Dropping... Creating... ❌ Failed: FAILED: BAD_REQUEST [TABLE_OR_VIEW_ALREADY_EXISTS] Cannot create table or view `main`.`test_schema`.`my_external_table` because it already exists.
Choose a different name, drop the existing object, add the IF NOT EXISTS clause to tolerate pre-existing objects, add the OR REPLACE clause to replace the existing materialized view, or add the OR REFRESH clause to refresh the existing streaming table. SQLSTATE: 42P07
I

arielshaqed · 2026-01-04T07:49:42Z

Thanks! This latest certainly indicates dropping a table on DataBricks only applies eventually. Unfortunately, having to retry over 100 seconds (and just in a single trial - a bad day will be much worse) makes me wonder how this fix can help! It is one thing to tell users not to perform concurrent commits and/or merges. But post-commit hooks (where we can perform this action) are asynchronous. Now we will need users to space their commits out 2 minutes or more apart from each other.
I would like to step back and understand what we're really trying to do here. Documentation suggests that the way to update an external DataBricks table is to MSCK REPAIR TABLE table_name SYNC METADATA. If the table exists, can we do this instead?

Isan-Rivkin · 2026-01-22T07:40:29Z

Removed myself as a reviewer because I keep getting notification reminders 🙈

using backoff

445efb0

ItamarYuran added include-changelog PR description should be included in next release changelog minor-change Used for PRs that don't require issue attached labels Dec 30, 2025

github-actions bot added the area/hooks improvements or additions to the hooks subsystem label Dec 30, 2025

clean

fb53d8e

arielshaqed requested changes Dec 30, 2025

View reviewed changes

ItamarYuran changed the title ~~Fix: Export unity catalog hook race condition~~ Fix: Unity Catalog export hook fails sporadically on table re-registration Dec 30, 2025

createOrDelete

7ee3c55

ItamarYuran requested review from Isan-Rivkin, arielshaqed and nopcoder December 30, 2025 17:09

nopcoder requested changes Dec 30, 2025

View reviewed changes

adjustments

694a1e5

ItamarYuran requested a review from nopcoder December 30, 2025 21:26

nopcoder requested changes Dec 31, 2025

View reviewed changes

adjustments

999f261

ItamarYuran requested a review from nopcoder December 31, 2025 08:04

nopcoder requested changes Dec 31, 2025

View reviewed changes

comments

ef4af77

ItamarYuran requested a review from nopcoder December 31, 2025 10:50

nopcoder approved these changes Dec 31, 2025

View reviewed changes

code style

2041079

add string check for err

5244884

ItamarYuran changed the title ~~Fix: Unity Catalog export hook fails sporadically on table re-registration~~ Unity Catalog Export: Reduce likelihood for action failure, due to race condition against databricks on table re-registration Jan 1, 2026

arielshaqed reviewed Jan 1, 2026

View reviewed changes

backoff elapsed 100s, check err "do not exist"

01f862a

Isan-Rivkin removed their request for review January 22, 2026 07:40

		status, err := backoff.RetryWithData(createTableBO, bo)

		if err != nil {

Unity Catalog Export: Reduce likelihood for action failure, due to race condition against databricks on table re-registration #9937

Are you sure you want to change the base?

Unity Catalog Export: Reduce likelihood for action failure, due to race condition against databricks on table re-registration #9937

Uh oh!

Conversation

ItamarYuran commented Dec 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

arielshaqed left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arielshaqed commented Dec 30, 2025

Uh oh!

ItamarYuran commented Dec 30, 2025

Uh oh!

arielshaqed commented Dec 30, 2025

Uh oh!

ItamarYuran commented Dec 30, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ItamarYuran commented Dec 31, 2025

Uh oh!

nopcoder left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ItamarYuran commented Dec 31, 2025

Uh oh!

nopcoder left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arielshaqed commented Dec 31, 2025

Uh oh!

ItamarYuran commented Dec 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

arielshaqed left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ItamarYuran commented Jan 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ItamarYuran commented Jan 1, 2026

Uh oh!

ItamarYuran commented Jan 3, 2026

Uh oh!

ItamarYuran commented Dec 30, 2025 •

edited

Loading

ItamarYuran commented Dec 31, 2025 •

edited

Loading

ItamarYuran commented Jan 1, 2026 •

edited

Loading