Test failure: `test_runtime_backend_errors_handled[\nfrom databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unknown\nbackend = RuntimeBackend()\ntry:\n grants = backend.fetch("SHOW GRANTS ON METASTORE")\n print("FAILED")\nexcept Unknown:\n print("PASSED")\n]`

[github-actions]

❌ test_runtime_backend_errors_handled[\nfrom databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unknown\nbackend = RuntimeBackend()\ntry:\n grants = backend.fetch("SHOW GRANTS ON METASTORE")\n print("FAILED")\nexcept Unknown:\n print("PASSED")\n]: AssertionError: assert 'FAILED' == 'PASSED' (25.084s)

AssertionError: assert 'FAILED' == 'PASSED'
  
  - PASSED
  + FAILED
05:17 DEBUG [databricks.sdk] Loaded from environment
05:17 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:17 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:17 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:17 INFO [databricks.sdk] Using Databricks Metadata Service authentication
[gw1] linux -- Python 3.10.19 /home/runner/work/lsql/lsql/.venv/bin/python
05:17 DEBUG [databricks.sdk] Loaded from environment
05:17 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:17 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:17 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:17 INFO [databricks.sdk] Using Databricks Metadata Service authentication
05:17 DEBUG [databricks.sdk] GET /api/2.0/preview/scim/v2/Me
< 200 OK
< {
<   "active": true,
<   "displayName": "labs-runtime-identity",
<   "emails": [
<     {
<       "primary": true,
<       "type": "work",
<       "value": "**REDACTED**"
<     }
<   ],
<   "externalId": "d0f9bd2c-5651-45fd-b648-12a3fc6375c4",
<   "groups": [
<     {
<       "$ref": "Groups/153383108335587",
<       "display": "users",
<       "type": "direct",
<       "value": "**REDACTED**"
<     },
<     "... (1 additional elements)"
<   ],
<   "id": "4643477475987733",
<   "name": {
<     "givenName": "labs-runtime-identity"
<   },
<   "schemas": [
<     "urn:ietf:params:scim:schemas:core:2.0:User",
<     "... (1 additional elements)"
<   ],
<   "userName": "4106dc97-a963-48f0-a079-a578238959a6"
< }
05:17 DEBUG [databricks.labs.blueprint.wheels] Building wheel for /tmp/tmp0lxnye0g/working-copy in /tmp/tmp0lxnye0g
05:17 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.OVoy/wheels/databricks_labs_lsql-0.16.1+320260219051747-py3-none-any.whl
05:17 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:17 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 404 Not Found
< {
<   "error_code": "RESOURCE_DOES_NOT_EXIST",
<   "message": "The parent folder (/Users/4106dc97-a963-48f0-a079-a578238959a6/.OVoy/wheels) does not exist."
< }
05:17 DEBUG [databricks.labs.blueprint.installation] Creating missing folders: /Users/4106dc97-a963-48f0-a079-a578238959a6/.OVoy/wheels
05:17 DEBUG [databricks.sdk] POST /api/2.0/workspace/mkdirs
> {
>   "path": "/Users/4106dc97-a963-48f0-a079-a578238959a6/.OVoy/wheels"
> }
< 200 OK
< {}
05:17 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:17 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 3768182285747818
< }
05:17 DEBUG [databricks.labs.blueprint.installation] Converting Version into JSON format
05:17 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.OVoy/version.json
05:17 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:17 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 3768182285747820
< }
05:17 DEBUG [databricks.sdk] GET /api/2.1/clusters/get?cluster_id=DATABRICKS_CLUSTER_ID
< 200 OK
< {
<   "autotermination_minutes": 60,
<   "CLOUD_ENV_attributes": {
<     "availability": "ON_DEMAND_AZURE",
<     "first_on_demand": 2147483647,
<     "spot_bid_max_price": -1.0
<   },
<   "cluster_cores": 8.0,
<   "cluster_id": "DATABRICKS_CLUSTER_ID",
<   "cluster_memory_mb": 32768,
<   "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<   "cluster_source": "UI",
<   "creator_user_name": "liran.bareket@databricks.com",
<   "custom_tags": {
<     "ResourceClass": "SingleNode"
<   },
<   "data_security_mode": "SINGLE_USER",
<   "TEST_SCHEMA_tags": {
<     "Budget": "opex.sales.labs",
<     "ClusterId": "DATABRICKS_CLUSTER_ID",
<     "ClusterName": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "Creator": "liran.bareket@databricks.com",
<     "DatabricksInstanceGroupId": "-7571316921879686317",
<     "DatabricksInstancePoolCreatorId": "6779888502363704",
<     "DatabricksInstancePoolId": "TEST_INSTANCE_POOL_ID",
<     "Owner": "labs-oss@databricks.com",
<     "Vendor": "Databricks"
<   },
<   "disk_spec": {},
<   "driver": {
<     "host_private_ip": "10.179.12.11",
<     "instance_id": "3c075f30c7ac461aac3bed23b75df53d",
<     "ngrok_endpoint_base_domain": "green.mux.ngrok-dataplane.wildcard",
<     "node_attributes": {
<       "is_spot": false
<     },
<     "node_id": "32e8f19b9eac40f08e3ae86351bf3f4a",
<     "node_type_id": "Standard_D8ads_v6",
<     "private_ip": "10.179.14.11",
<     "public_dns": "",
<     "start_timestamp": 1771478184840
<   },
<   "driver_healthy": true,
<   "driver_instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "driver_instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "driver_node_type_id": "Standard_D8ads_v6",
<   "effective_spark_version": "16.4.x-scala2.12",
<   "enable_elastic_disk": true,
<   "enable_local_disk_encryption": false,
<   "init_scripts_safe_mode": false,
<   "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "jdbc_port": 10000,
<   "last_activity_time": 1771478207227,
<   "last_restarted_time": 1771478235069,
<   "last_state_loss_time": 1771478235025,
<   "node_type_id": "Standard_D8ads_v6",
<   "num_workers": 0,
<   "pinned_by_user_name": "6779888502363704",
<   "release_version": "16.4.17",
<   "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<   "spark_conf": {
<     "spark.databricks.cluster.profile": "singleNode",
<     "spark.master": "local[*]"
<   },
<   "spark_context_id": 7456092220667130223,
<   "spark_version": "16.4.x-scala2.12",
<   "spec": {
<     "autotermination_minutes": 60,
<     "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "custom_tags": {
<       "ResourceClass": "SingleNode"
<     },
<     "data_security_mode": "SINGLE_USER",
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<     "num_workers": 0,
<     "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<     "spark_conf": {
<       "spark.databricks.cluster.profile": "singleNode",
<       "spark.master": "local[*]"
<     },
<     "spark_version": "16.4.x-scala2.12"
<   },
<   "start_time": 1759339672984,
<   "state": "RUNNING",
<   "state_message": ""
< }
05:17 DEBUG [databricks.sdk] POST /api/1.2/contexts/create
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "2423109679674500450"
< }
05:17 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=2423109679674500450
< 200 OK
< {
<   "id": "2423109679674500450",
<   "status": "Pending"
< }
05:17 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=2423109679674500450: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~1s)
05:17 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=2423109679674500450
< 200 OK
< {
<   "id": "2423109679674500450",
<   "status": "Pending"
< }
05:17 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=2423109679674500450: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~2s)
05:17 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=2423109679674500450
< 200 OK
< {
<   "id": "2423109679674500450",
<   "status": "Running"
< }
05:17 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "get_ipython().run_line_magic('pip', 'install /Workspace/Users/4106dc97-a963-48f0-a079-a578238959... (110 more bytes)",
>   "contextId": "2423109679674500450",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "17df7a397b89499a9901aa54dd8b297e"
< }
05:17 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=17df7a397b89499a9901aa54dd8b297e&contextId=2423109679674500450
< 200 OK
< {
<   "id": "17df7a397b89499a9901aa54dd8b297e",
<   "results": null,
<   "status": "Running"
< }
05:17 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=17df7a397b89499a9901aa54dd8b297e, context_id=2423109679674500450: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~1s)
05:17 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=17df7a397b89499a9901aa54dd8b297e&contextId=2423109679674500450
< 200 OK
< {
<   "id": "17df7a397b89499a9901aa54dd8b297e",
<   "results": null,
<   "status": "Running"
< }
05:17 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=17df7a397b89499a9901aa54dd8b297e, context_id=2423109679674500450: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~2s)
05:18 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=17df7a397b89499a9901aa54dd8b297e&contextId=2423109679674500450
< 200 OK
< {
<   "id": "17df7a397b89499a9901aa54dd8b297e",
<   "results": null,
<   "status": "Running"
< }
05:18 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=17df7a397b89499a9901aa54dd8b297e, context_id=2423109679674500450: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~3s)
05:18 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=17df7a397b89499a9901aa54dd8b297e&contextId=2423109679674500450
< 200 OK
< {
<   "id": "17df7a397b89499a9901aa54dd8b297e",
<   "results": null,
<   "status": "Running"
< }
05:18 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=17df7a397b89499a9901aa54dd8b297e, context_id=2423109679674500450: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~4s)
05:18 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=17df7a397b89499a9901aa54dd8b297e&contextId=2423109679674500450
< 200 OK
< {
<   "id": "17df7a397b89499a9901aa54dd8b297e",
<   "results": {
<     "data": "Processing /Workspace/Users/4106dc97-a963-48f0-a079-a578238959a6/.OVoy/wheels/databricks_labs_ls... (5292 more bytes)",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:18 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "from databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unkno... (145 more bytes)",
>   "contextId": "2423109679674500450",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "656f72a7d5a14681ab2aaee5bb0a9105"
< }
05:18 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=656f72a7d5a14681ab2aaee5bb0a9105&contextId=2423109679674500450
< 200 OK
< {
<   "id": "656f72a7d5a14681ab2aaee5bb0a9105",
<   "results": {
<     "data": "FAILED",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:17 DEBUG [databricks.sdk] Loaded from environment
05:17 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:17 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:17 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:17 INFO [databricks.sdk] Using Databricks Metadata Service authentication
05:17 DEBUG [databricks.sdk] GET /api/2.0/preview/scim/v2/Me
< 200 OK
< {
<   "active": true,
<   "displayName": "labs-runtime-identity",
<   "emails": [
<     {
<       "primary": true,
<       "type": "work",
<       "value": "**REDACTED**"
<     }
<   ],
<   "externalId": "d0f9bd2c-5651-45fd-b648-12a3fc6375c4",
<   "groups": [
<     {
<       "$ref": "Groups/153383108335587",
<       "display": "users",
<       "type": "direct",
<       "value": "**REDACTED**"
<     },
<     "... (1 additional elements)"
<   ],
<   "id": "4643477475987733",
<   "name": {
<     "givenName": "labs-runtime-identity"
<   },
<   "schemas": [
<     "urn:ietf:params:scim:schemas:core:2.0:User",
<     "... (1 additional elements)"
<   ],
<   "userName": "4106dc97-a963-48f0-a079-a578238959a6"
< }
05:17 DEBUG [databricks.labs.blueprint.wheels] Building wheel for /tmp/tmp0lxnye0g/working-copy in /tmp/tmp0lxnye0g
05:17 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.OVoy/wheels/databricks_labs_lsql-0.16.1+320260219051747-py3-none-any.whl
05:17 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:17 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 404 Not Found
< {
<   "error_code": "RESOURCE_DOES_NOT_EXIST",
<   "message": "The parent folder (/Users/4106dc97-a963-48f0-a079-a578238959a6/.OVoy/wheels) does not exist."
< }
05:17 DEBUG [databricks.labs.blueprint.installation] Creating missing folders: /Users/4106dc97-a963-48f0-a079-a578238959a6/.OVoy/wheels
05:17 DEBUG [databricks.sdk] POST /api/2.0/workspace/mkdirs
> {
>   "path": "/Users/4106dc97-a963-48f0-a079-a578238959a6/.OVoy/wheels"
> }
< 200 OK
< {}
05:17 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:17 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 3768182285747818
< }
05:17 DEBUG [databricks.labs.blueprint.installation] Converting Version into JSON format
05:17 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.OVoy/version.json
05:17 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:17 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 3768182285747820
< }
05:17 DEBUG [databricks.sdk] GET /api/2.1/clusters/get?cluster_id=DATABRICKS_CLUSTER_ID
< 200 OK
< {
<   "autotermination_minutes": 60,
<   "CLOUD_ENV_attributes": {
<     "availability": "ON_DEMAND_AZURE",
<     "first_on_demand": 2147483647,
<     "spot_bid_max_price": -1.0
<   },
<   "cluster_cores": 8.0,
<   "cluster_id": "DATABRICKS_CLUSTER_ID",
<   "cluster_memory_mb": 32768,
<   "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<   "cluster_source": "UI",
<   "creator_user_name": "liran.bareket@databricks.com",
<   "custom_tags": {
<     "ResourceClass": "SingleNode"
<   },
<   "data_security_mode": "SINGLE_USER",
<   "TEST_SCHEMA_tags": {
<     "Budget": "opex.sales.labs",
<     "ClusterId": "DATABRICKS_CLUSTER_ID",
<     "ClusterName": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "Creator": "liran.bareket@databricks.com",
<     "DatabricksInstanceGroupId": "-7571316921879686317",
<     "DatabricksInstancePoolCreatorId": "6779888502363704",
<     "DatabricksInstancePoolId": "TEST_INSTANCE_POOL_ID",
<     "Owner": "labs-oss@databricks.com",
<     "Vendor": "Databricks"
<   },
<   "disk_spec": {},
<   "driver": {
<     "host_private_ip": "10.179.12.11",
<     "instance_id": "3c075f30c7ac461aac3bed23b75df53d",
<     "ngrok_endpoint_base_domain": "green.mux.ngrok-dataplane.wildcard",
<     "node_attributes": {
<       "is_spot": false
<     },
<     "node_id": "32e8f19b9eac40f08e3ae86351bf3f4a",
<     "node_type_id": "Standard_D8ads_v6",
<     "private_ip": "10.179.14.11",
<     "public_dns": "",
<     "start_timestamp": 1771478184840
<   },
<   "driver_healthy": true,
<   "driver_instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "driver_instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "driver_node_type_id": "Standard_D8ads_v6",
<   "effective_spark_version": "16.4.x-scala2.12",
<   "enable_elastic_disk": true,
<   "enable_local_disk_encryption": false,
<   "init_scripts_safe_mode": false,
<   "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "jdbc_port": 10000,
<   "last_activity_time": 1771478207227,
<   "last_restarted_time": 1771478235069,
<   "last_state_loss_time": 1771478235025,
<   "node_type_id": "Standard_D8ads_v6",
<   "num_workers": 0,
<   "pinned_by_user_name": "6779888502363704",
<   "release_version": "16.4.17",
<   "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<   "spark_conf": {
<     "spark.databricks.cluster.profile": "singleNode",
<     "spark.master": "local[*]"
<   },
<   "spark_context_id": 7456092220667130223,
<   "spark_version": "16.4.x-scala2.12",
<   "spec": {
<     "autotermination_minutes": 60,
<     "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "custom_tags": {
<       "ResourceClass": "SingleNode"
<     },
<     "data_security_mode": "SINGLE_USER",
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<     "num_workers": 0,
<     "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<     "spark_conf": {
<       "spark.databricks.cluster.profile": "singleNode",
<       "spark.master": "local[*]"
<     },
<     "spark_version": "16.4.x-scala2.12"
<   },
<   "start_time": 1759339672984,
<   "state": "RUNNING",
<   "state_message": ""
< }
05:17 DEBUG [databricks.sdk] POST /api/1.2/contexts/create
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "2423109679674500450"
< }
05:17 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=2423109679674500450
< 200 OK
< {
<   "id": "2423109679674500450",
<   "status": "Pending"
< }
05:17 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=2423109679674500450: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~1s)
05:17 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=2423109679674500450
< 200 OK
< {
<   "id": "2423109679674500450",
<   "status": "Pending"
< }
05:17 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=2423109679674500450: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~2s)
05:17 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=2423109679674500450
< 200 OK
< {
<   "id": "2423109679674500450",
<   "status": "Running"
< }
05:17 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "get_ipython().run_line_magic('pip', 'install /Workspace/Users/4106dc97-a963-48f0-a079-a578238959... (110 more bytes)",
>   "contextId": "2423109679674500450",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "17df7a397b89499a9901aa54dd8b297e"
< }
05:17 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=17df7a397b89499a9901aa54dd8b297e&contextId=2423109679674500450
< 200 OK
< {
<   "id": "17df7a397b89499a9901aa54dd8b297e",
<   "results": null,
<   "status": "Running"
< }
05:17 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=17df7a397b89499a9901aa54dd8b297e, context_id=2423109679674500450: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~1s)
05:17 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=17df7a397b89499a9901aa54dd8b297e&contextId=2423109679674500450
< 200 OK
< {
<   "id": "17df7a397b89499a9901aa54dd8b297e",
<   "results": null,
<   "status": "Running"
< }
05:17 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=17df7a397b89499a9901aa54dd8b297e, context_id=2423109679674500450: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~2s)
05:18 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=17df7a397b89499a9901aa54dd8b297e&contextId=2423109679674500450
< 200 OK
< {
<   "id": "17df7a397b89499a9901aa54dd8b297e",
<   "results": null,
<   "status": "Running"
< }
05:18 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=17df7a397b89499a9901aa54dd8b297e, context_id=2423109679674500450: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~3s)
05:18 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=17df7a397b89499a9901aa54dd8b297e&contextId=2423109679674500450
< 200 OK
< {
<   "id": "17df7a397b89499a9901aa54dd8b297e",
<   "results": null,
<   "status": "Running"
< }
05:18 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=17df7a397b89499a9901aa54dd8b297e, context_id=2423109679674500450: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~4s)
05:18 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=17df7a397b89499a9901aa54dd8b297e&contextId=2423109679674500450
< 200 OK
< {
<   "id": "17df7a397b89499a9901aa54dd8b297e",
<   "results": {
<     "data": "Processing /Workspace/Users/4106dc97-a963-48f0-a079-a578238959a6/.OVoy/wheels/databricks_labs_ls... (5292 more bytes)",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:18 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "from databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unkno... (145 more bytes)",
>   "contextId": "2423109679674500450",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "656f72a7d5a14681ab2aaee5bb0a9105"
< }
05:18 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=656f72a7d5a14681ab2aaee5bb0a9105&contextId=2423109679674500450
< 200 OK
< {
<   "id": "656f72a7d5a14681ab2aaee5bb0a9105",
<   "results": {
<     "data": "FAILED",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
[gw1] linux -- Python 3.10.19 /home/runner/work/lsql/lsql/.venv/bin/python

_{Running from nightly #464}

[github-actions]

❌ test_runtime_backend_errors_handled[\nfrom databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unknown\nbackend = RuntimeBackend()\ntry:\n grants = backend.fetch("SHOW GRANTS ON METASTORE")\n print("FAILED")\nexcept Unknown:\n print("PASSED")\n]: AssertionError: assert 'FAILED' == 'PASSED' (23.365s)

AssertionError: assert 'FAILED' == 'PASSED'
  
  - PASSED
  + FAILED
05:14 DEBUG [databricks.sdk] Loaded from environment
05:14 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:14 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:14 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:14 INFO [databricks.sdk] Using Databricks Metadata Service authentication
[gw3] linux -- Python 3.10.19 /home/runner/work/lsql/lsql/.venv/bin/python
05:14 DEBUG [databricks.sdk] Loaded from environment
05:14 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:14 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:14 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:14 INFO [databricks.sdk] Using Databricks Metadata Service authentication
05:14 DEBUG [databricks.sdk] GET /api/2.0/preview/scim/v2/Me
< 200 OK
< {
<   "active": true,
<   "displayName": "labs-runtime-identity",
<   "emails": [
<     {
<       "primary": true,
<       "type": "work",
<       "value": "**REDACTED**"
<     }
<   ],
<   "externalId": "d0f9bd2c-5651-45fd-b648-12a3fc6375c4",
<   "groups": [
<     {
<       "$ref": "Groups/153383108335587",
<       "display": "users",
<       "type": "direct",
<       "value": "**REDACTED**"
<     },
<     "... (1 additional elements)"
<   ],
<   "id": "4643477475987733",
<   "name": {
<     "givenName": "labs-runtime-identity"
<   },
<   "schemas": [
<     "urn:ietf:params:scim:schemas:core:2.0:User",
<     "... (1 additional elements)"
<   ],
<   "userName": "4106dc97-a963-48f0-a079-a578238959a6"
< }
05:14 DEBUG [databricks.labs.blueprint.wheels] Building wheel for /tmp/tmpdky7b525/working-copy in /tmp/tmpdky7b525
05:14 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.WDGK/wheels/databricks_labs_lsql-0.16.1+320260220051441-py3-none-any.whl
05:14 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:14 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 404 Not Found
< {
<   "error_code": "RESOURCE_DOES_NOT_EXIST",
<   "message": "The parent folder (/Users/4106dc97-a963-48f0-a079-a578238959a6/.WDGK/wheels) does not exist."
< }
05:14 DEBUG [databricks.labs.blueprint.installation] Creating missing folders: /Users/4106dc97-a963-48f0-a079-a578238959a6/.WDGK/wheels
05:14 DEBUG [databricks.sdk] POST /api/2.0/workspace/mkdirs
> {
>   "path": "/Users/4106dc97-a963-48f0-a079-a578238959a6/.WDGK/wheels"
> }
< 200 OK
< {}
05:14 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:14 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 2259303437370813
< }
05:14 DEBUG [databricks.labs.blueprint.installation] Converting Version into JSON format
05:14 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.WDGK/version.json
05:14 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:14 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 2259303437370814
< }
05:14 DEBUG [databricks.sdk] GET /api/2.1/clusters/get?cluster_id=DATABRICKS_CLUSTER_ID
< 200 OK
< {
<   "autotermination_minutes": 60,
<   "CLOUD_ENV_attributes": {
<     "availability": "ON_DEMAND_AZURE",
<     "first_on_demand": 2147483647,
<     "spot_bid_max_price": -1.0
<   },
<   "cluster_cores": 8.0,
<   "cluster_id": "DATABRICKS_CLUSTER_ID",
<   "cluster_memory_mb": 32768,
<   "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<   "cluster_source": "UI",
<   "creator_user_name": "liran.bareket@databricks.com",
<   "custom_tags": {
<     "ResourceClass": "SingleNode"
<   },
<   "data_security_mode": "SINGLE_USER",
<   "TEST_SCHEMA_tags": {
<     "Budget": "opex.sales.labs",
<     "ClusterId": "DATABRICKS_CLUSTER_ID",
<     "ClusterName": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "Creator": "liran.bareket@databricks.com",
<     "DatabricksInstanceGroupId": "-7571316921879686317",
<     "DatabricksInstancePoolCreatorId": "6779888502363704",
<     "DatabricksInstancePoolId": "TEST_INSTANCE_POOL_ID",
<     "Owner": "labs-oss@databricks.com",
<     "Vendor": "Databricks"
<   },
<   "disk_spec": {},
<   "driver": {
<     "host_private_ip": "10.179.12.14",
<     "instance_id": "9c515e10d5dc4fbc80fb085048b55929",
<     "ngrok_endpoint_base_domain": "green.mux.ngrok-dataplane.wildcard",
<     "node_attributes": {
<       "is_spot": false
<     },
<     "node_id": "553817a7779d47888125f2149b5b07ea",
<     "node_type_id": "Standard_D8ads_v6",
<     "private_ip": "10.179.14.14",
<     "public_dns": "",
<     "start_timestamp": 1771564397924
<   },
<   "driver_healthy": true,
<   "driver_instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "driver_instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "driver_node_type_id": "Standard_D8ads_v6",
<   "effective_spark_version": "16.4.x-scala2.12",
<   "enable_elastic_disk": true,
<   "enable_local_disk_encryption": false,
<   "init_scripts_safe_mode": false,
<   "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "jdbc_port": 10000,
<   "last_activity_time": 1771564473059,
<   "last_restarted_time": 1771564448871,
<   "last_state_loss_time": 1771564448822,
<   "node_type_id": "Standard_D8ads_v6",
<   "num_workers": 0,
<   "pinned_by_user_name": "6779888502363704",
<   "release_version": "16.4.18",
<   "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<   "spark_conf": {
<     "spark.databricks.cluster.profile": "singleNode",
<     "spark.master": "local[*]"
<   },
<   "spark_context_id": 6118799050751085440,
<   "spark_version": "16.4.x-scala2.12",
<   "spec": {
<     "autotermination_minutes": 60,
<     "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "custom_tags": {
<       "ResourceClass": "SingleNode"
<     },
<     "data_security_mode": "SINGLE_USER",
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<     "num_workers": 0,
<     "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<     "spark_conf": {
<       "spark.databricks.cluster.profile": "singleNode",
<       "spark.master": "local[*]"
<     },
<     "spark_version": "16.4.x-scala2.12"
<   },
<   "start_time": 1759339672984,
<   "state": "RUNNING",
<   "state_message": ""
< }
05:14 DEBUG [databricks.sdk] POST /api/1.2/contexts/create
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "2946499615187716046"
< }
05:14 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=2946499615187716046
< 200 OK
< {
<   "id": "2946499615187716046",
<   "status": "Pending"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=2946499615187716046: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~1s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=2946499615187716046
< 200 OK
< {
<   "id": "2946499615187716046",
<   "status": "Pending"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=2946499615187716046: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~2s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=2946499615187716046
< 200 OK
< {
<   "id": "2946499615187716046",
<   "status": "Running"
< }
05:14 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "get_ipython().run_line_magic('pip', 'install /Workspace/Users/4106dc97-a963-48f0-a079-a578238959... (110 more bytes)",
>   "contextId": "2946499615187716046",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "28ca8752933d40589013dcd3bad2c794"
< }
05:14 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=28ca8752933d40589013dcd3bad2c794&contextId=2946499615187716046
< 200 OK
< {
<   "id": "28ca8752933d40589013dcd3bad2c794",
<   "results": null,
<   "status": "Running"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=28ca8752933d40589013dcd3bad2c794, context_id=2946499615187716046: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~1s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=28ca8752933d40589013dcd3bad2c794&contextId=2946499615187716046
< 200 OK
< {
<   "id": "28ca8752933d40589013dcd3bad2c794",
<   "results": null,
<   "status": "Running"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=28ca8752933d40589013dcd3bad2c794, context_id=2946499615187716046: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~2s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=28ca8752933d40589013dcd3bad2c794&contextId=2946499615187716046
< 200 OK
< {
<   "id": "28ca8752933d40589013dcd3bad2c794",
<   "results": null,
<   "status": "Running"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=28ca8752933d40589013dcd3bad2c794, context_id=2946499615187716046: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~3s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=28ca8752933d40589013dcd3bad2c794&contextId=2946499615187716046
< 200 OK
< {
<   "id": "28ca8752933d40589013dcd3bad2c794",
<   "results": null,
<   "status": "Running"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=28ca8752933d40589013dcd3bad2c794, context_id=2946499615187716046: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~4s)
05:15 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=28ca8752933d40589013dcd3bad2c794&contextId=2946499615187716046
< 200 OK
< {
<   "id": "28ca8752933d40589013dcd3bad2c794",
<   "results": {
<     "data": "Processing /Workspace/Users/4106dc97-a963-48f0-a079-a578238959a6/.WDGK/wheels/databricks_labs_ls... (5292 more bytes)",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:15 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "from databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unkno... (145 more bytes)",
>   "contextId": "2946499615187716046",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "2cc50dbdc3ba48ac88d784fb706a080e"
< }
05:15 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=2cc50dbdc3ba48ac88d784fb706a080e&contextId=2946499615187716046
< 200 OK
< {
<   "id": "2cc50dbdc3ba48ac88d784fb706a080e",
<   "results": {
<     "data": "FAILED",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:14 DEBUG [databricks.sdk] Loaded from environment
05:14 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:14 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:14 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:14 INFO [databricks.sdk] Using Databricks Metadata Service authentication
05:14 DEBUG [databricks.sdk] GET /api/2.0/preview/scim/v2/Me
< 200 OK
< {
<   "active": true,
<   "displayName": "labs-runtime-identity",
<   "emails": [
<     {
<       "primary": true,
<       "type": "work",
<       "value": "**REDACTED**"
<     }
<   ],
<   "externalId": "d0f9bd2c-5651-45fd-b648-12a3fc6375c4",
<   "groups": [
<     {
<       "$ref": "Groups/153383108335587",
<       "display": "users",
<       "type": "direct",
<       "value": "**REDACTED**"
<     },
<     "... (1 additional elements)"
<   ],
<   "id": "4643477475987733",
<   "name": {
<     "givenName": "labs-runtime-identity"
<   },
<   "schemas": [
<     "urn:ietf:params:scim:schemas:core:2.0:User",
<     "... (1 additional elements)"
<   ],
<   "userName": "4106dc97-a963-48f0-a079-a578238959a6"
< }
05:14 DEBUG [databricks.labs.blueprint.wheels] Building wheel for /tmp/tmpdky7b525/working-copy in /tmp/tmpdky7b525
05:14 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.WDGK/wheels/databricks_labs_lsql-0.16.1+320260220051441-py3-none-any.whl
05:14 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:14 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 404 Not Found
< {
<   "error_code": "RESOURCE_DOES_NOT_EXIST",
<   "message": "The parent folder (/Users/4106dc97-a963-48f0-a079-a578238959a6/.WDGK/wheels) does not exist."
< }
05:14 DEBUG [databricks.labs.blueprint.installation] Creating missing folders: /Users/4106dc97-a963-48f0-a079-a578238959a6/.WDGK/wheels
05:14 DEBUG [databricks.sdk] POST /api/2.0/workspace/mkdirs
> {
>   "path": "/Users/4106dc97-a963-48f0-a079-a578238959a6/.WDGK/wheels"
> }
< 200 OK
< {}
05:14 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:14 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 2259303437370813
< }
05:14 DEBUG [databricks.labs.blueprint.installation] Converting Version into JSON format
05:14 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.WDGK/version.json
05:14 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:14 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 2259303437370814
< }
05:14 DEBUG [databricks.sdk] GET /api/2.1/clusters/get?cluster_id=DATABRICKS_CLUSTER_ID
< 200 OK
< {
<   "autotermination_minutes": 60,
<   "CLOUD_ENV_attributes": {
<     "availability": "ON_DEMAND_AZURE",
<     "first_on_demand": 2147483647,
<     "spot_bid_max_price": -1.0
<   },
<   "cluster_cores": 8.0,
<   "cluster_id": "DATABRICKS_CLUSTER_ID",
<   "cluster_memory_mb": 32768,
<   "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<   "cluster_source": "UI",
<   "creator_user_name": "liran.bareket@databricks.com",
<   "custom_tags": {
<     "ResourceClass": "SingleNode"
<   },
<   "data_security_mode": "SINGLE_USER",
<   "TEST_SCHEMA_tags": {
<     "Budget": "opex.sales.labs",
<     "ClusterId": "DATABRICKS_CLUSTER_ID",
<     "ClusterName": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "Creator": "liran.bareket@databricks.com",
<     "DatabricksInstanceGroupId": "-7571316921879686317",
<     "DatabricksInstancePoolCreatorId": "6779888502363704",
<     "DatabricksInstancePoolId": "TEST_INSTANCE_POOL_ID",
<     "Owner": "labs-oss@databricks.com",
<     "Vendor": "Databricks"
<   },
<   "disk_spec": {},
<   "driver": {
<     "host_private_ip": "10.179.12.14",
<     "instance_id": "9c515e10d5dc4fbc80fb085048b55929",
<     "ngrok_endpoint_base_domain": "green.mux.ngrok-dataplane.wildcard",
<     "node_attributes": {
<       "is_spot": false
<     },
<     "node_id": "553817a7779d47888125f2149b5b07ea",
<     "node_type_id": "Standard_D8ads_v6",
<     "private_ip": "10.179.14.14",
<     "public_dns": "",
<     "start_timestamp": 1771564397924
<   },
<   "driver_healthy": true,
<   "driver_instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "driver_instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "driver_node_type_id": "Standard_D8ads_v6",
<   "effective_spark_version": "16.4.x-scala2.12",
<   "enable_elastic_disk": true,
<   "enable_local_disk_encryption": false,
<   "init_scripts_safe_mode": false,
<   "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "jdbc_port": 10000,
<   "last_activity_time": 1771564473059,
<   "last_restarted_time": 1771564448871,
<   "last_state_loss_time": 1771564448822,
<   "node_type_id": "Standard_D8ads_v6",
<   "num_workers": 0,
<   "pinned_by_user_name": "6779888502363704",
<   "release_version": "16.4.18",
<   "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<   "spark_conf": {
<     "spark.databricks.cluster.profile": "singleNode",
<     "spark.master": "local[*]"
<   },
<   "spark_context_id": 6118799050751085440,
<   "spark_version": "16.4.x-scala2.12",
<   "spec": {
<     "autotermination_minutes": 60,
<     "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "custom_tags": {
<       "ResourceClass": "SingleNode"
<     },
<     "data_security_mode": "SINGLE_USER",
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<     "num_workers": 0,
<     "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<     "spark_conf": {
<       "spark.databricks.cluster.profile": "singleNode",
<       "spark.master": "local[*]"
<     },
<     "spark_version": "16.4.x-scala2.12"
<   },
<   "start_time": 1759339672984,
<   "state": "RUNNING",
<   "state_message": ""
< }
05:14 DEBUG [databricks.sdk] POST /api/1.2/contexts/create
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "2946499615187716046"
< }
05:14 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=2946499615187716046
< 200 OK
< {
<   "id": "2946499615187716046",
<   "status": "Pending"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=2946499615187716046: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~1s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=2946499615187716046
< 200 OK
< {
<   "id": "2946499615187716046",
<   "status": "Pending"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=2946499615187716046: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~2s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=2946499615187716046
< 200 OK
< {
<   "id": "2946499615187716046",
<   "status": "Running"
< }
05:14 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "get_ipython().run_line_magic('pip', 'install /Workspace/Users/4106dc97-a963-48f0-a079-a578238959... (110 more bytes)",
>   "contextId": "2946499615187716046",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "28ca8752933d40589013dcd3bad2c794"
< }
05:14 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=28ca8752933d40589013dcd3bad2c794&contextId=2946499615187716046
< 200 OK
< {
<   "id": "28ca8752933d40589013dcd3bad2c794",
<   "results": null,
<   "status": "Running"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=28ca8752933d40589013dcd3bad2c794, context_id=2946499615187716046: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~1s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=28ca8752933d40589013dcd3bad2c794&contextId=2946499615187716046
< 200 OK
< {
<   "id": "28ca8752933d40589013dcd3bad2c794",
<   "results": null,
<   "status": "Running"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=28ca8752933d40589013dcd3bad2c794, context_id=2946499615187716046: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~2s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=28ca8752933d40589013dcd3bad2c794&contextId=2946499615187716046
< 200 OK
< {
<   "id": "28ca8752933d40589013dcd3bad2c794",
<   "results": null,
<   "status": "Running"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=28ca8752933d40589013dcd3bad2c794, context_id=2946499615187716046: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~3s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=28ca8752933d40589013dcd3bad2c794&contextId=2946499615187716046
< 200 OK
< {
<   "id": "28ca8752933d40589013dcd3bad2c794",
<   "results": null,
<   "status": "Running"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=28ca8752933d40589013dcd3bad2c794, context_id=2946499615187716046: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~4s)
05:15 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=28ca8752933d40589013dcd3bad2c794&contextId=2946499615187716046
< 200 OK
< {
<   "id": "28ca8752933d40589013dcd3bad2c794",
<   "results": {
<     "data": "Processing /Workspace/Users/4106dc97-a963-48f0-a079-a578238959a6/.WDGK/wheels/databricks_labs_ls... (5292 more bytes)",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:15 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "from databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unkno... (145 more bytes)",
>   "contextId": "2946499615187716046",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "2cc50dbdc3ba48ac88d784fb706a080e"
< }
05:15 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=2cc50dbdc3ba48ac88d784fb706a080e&contextId=2946499615187716046
< 200 OK
< {
<   "id": "2cc50dbdc3ba48ac88d784fb706a080e",
<   "results": {
<     "data": "FAILED",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
[gw3] linux -- Python 3.10.19 /home/runner/work/lsql/lsql/.venv/bin/python

_{Running from nightly #465}

[github-actions]

❌ test_runtime_backend_errors_handled[\nfrom databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unknown\nbackend = RuntimeBackend()\ntry:\n grants = backend.fetch("SHOW GRANTS ON METASTORE")\n print("FAILED")\nexcept Unknown:\n print("PASSED")\n]: AssertionError: assert 'FAILED' == 'PASSED' (24.529s)

AssertionError: assert 'FAILED' == 'PASSED'
  
  - PASSED
  + FAILED
05:12 DEBUG [databricks.sdk] Loaded from environment
05:12 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:12 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:12 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:12 INFO [databricks.sdk] Using Databricks Metadata Service authentication
[gw0] linux -- Python 3.10.19 /home/runner/work/lsql/lsql/.venv/bin/python
05:12 DEBUG [databricks.sdk] Loaded from environment
05:12 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:12 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:12 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:12 INFO [databricks.sdk] Using Databricks Metadata Service authentication
05:12 DEBUG [databricks.sdk] GET /api/2.0/preview/scim/v2/Me
< 200 OK
< {
<   "active": true,
<   "displayName": "labs-runtime-identity",
<   "emails": [
<     {
<       "primary": true,
<       "type": "work",
<       "value": "**REDACTED**"
<     }
<   ],
<   "externalId": "d0f9bd2c-5651-45fd-b648-12a3fc6375c4",
<   "groups": [
<     {
<       "$ref": "Groups/153383108335587",
<       "display": "users",
<       "type": "direct",
<       "value": "**REDACTED**"
<     },
<     "... (1 additional elements)"
<   ],
<   "id": "4643477475987733",
<   "name": {
<     "givenName": "labs-runtime-identity"
<   },
<   "schemas": [
<     "urn:ietf:params:scim:schemas:core:2.0:User",
<     "... (1 additional elements)"
<   ],
<   "userName": "4106dc97-a963-48f0-a079-a578238959a6"
< }
05:12 DEBUG [databricks.labs.blueprint.wheels] Building wheel for /tmp/tmpmhg8za31/working-copy in /tmp/tmpmhg8za31
05:12 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.lub3/wheels/databricks_labs_lsql-0.16.1+320260221051215-py3-none-any.whl
05:12 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:12 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 404 Not Found
< {
<   "error_code": "RESOURCE_DOES_NOT_EXIST",
<   "message": "The parent folder (/Users/4106dc97-a963-48f0-a079-a578238959a6/.lub3/wheels) does not exist."
< }
05:12 DEBUG [databricks.labs.blueprint.installation] Creating missing folders: /Users/4106dc97-a963-48f0-a079-a578238959a6/.lub3/wheels
05:12 DEBUG [databricks.sdk] POST /api/2.0/workspace/mkdirs
> {
>   "path": "/Users/4106dc97-a963-48f0-a079-a578238959a6/.lub3/wheels"
> }
< 200 OK
< {}
05:12 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:12 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 2269762301694365
< }
05:12 DEBUG [databricks.labs.blueprint.installation] Converting Version into JSON format
05:12 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.lub3/version.json
05:12 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:12 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 2269762301694366
< }
05:12 DEBUG [databricks.sdk] GET /api/2.1/clusters/get?cluster_id=DATABRICKS_CLUSTER_ID
< 200 OK
< {
<   "autotermination_minutes": 60,
<   "CLOUD_ENV_attributes": {
<     "availability": "ON_DEMAND_AZURE",
<     "first_on_demand": 2147483647,
<     "spot_bid_max_price": -1.0
<   },
<   "cluster_cores": 8.0,
<   "cluster_id": "DATABRICKS_CLUSTER_ID",
<   "cluster_memory_mb": 32768,
<   "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<   "cluster_source": "UI",
<   "creator_user_name": "liran.bareket@databricks.com",
<   "custom_tags": {
<     "ResourceClass": "SingleNode"
<   },
<   "data_security_mode": "SINGLE_USER",
<   "TEST_SCHEMA_tags": {
<     "Budget": "opex.sales.labs",
<     "ClusterId": "DATABRICKS_CLUSTER_ID",
<     "ClusterName": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "Creator": "liran.bareket@databricks.com",
<     "DatabricksInstanceGroupId": "-7571316921879686317",
<     "DatabricksInstancePoolCreatorId": "6779888502363704",
<     "DatabricksInstancePoolId": "TEST_INSTANCE_POOL_ID",
<     "Owner": "labs-oss@databricks.com",
<     "Vendor": "Databricks"
<   },
<   "disk_spec": {},
<   "driver": {
<     "host_private_ip": "10.179.12.11",
<     "instance_id": "203e46ddda10498e8b81ade820bf805c",
<     "ngrok_endpoint_base_domain": "green.mux.ngrok-dataplane.wildcard",
<     "node_attributes": {
<       "is_spot": false
<     },
<     "node_id": "df1ca9159b784f72a68d9f4e20a92ec8",
<     "node_type_id": "Standard_D8ads_v6",
<     "private_ip": "10.179.14.11",
<     "public_dns": "",
<     "start_timestamp": 1771650660388
<   },
<   "driver_healthy": true,
<   "driver_instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "driver_instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "driver_node_type_id": "Standard_D8ads_v6",
<   "effective_spark_version": "16.4.x-scala2.12",
<   "enable_elastic_disk": true,
<   "enable_local_disk_encryption": false,
<   "init_scripts_safe_mode": false,
<   "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "jdbc_port": 10000,
<   "last_activity_time": 1771650681885,
<   "last_restarted_time": 1771650706345,
<   "last_state_loss_time": 1771650706302,
<   "node_type_id": "Standard_D8ads_v6",
<   "num_workers": 0,
<   "pinned_by_user_name": "6779888502363704",
<   "release_version": "16.4.18",
<   "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<   "spark_conf": {
<     "spark.databricks.cluster.profile": "singleNode",
<     "spark.master": "local[*]"
<   },
<   "spark_context_id": 6829547792133273073,
<   "spark_version": "16.4.x-scala2.12",
<   "spec": {
<     "autotermination_minutes": 60,
<     "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "custom_tags": {
<       "ResourceClass": "SingleNode"
<     },
<     "data_security_mode": "SINGLE_USER",
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<     "num_workers": 0,
<     "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<     "spark_conf": {
<       "spark.databricks.cluster.profile": "singleNode",
<       "spark.master": "local[*]"
<     },
<     "spark_version": "16.4.x-scala2.12"
<   },
<   "start_time": 1759339672984,
<   "state": "RUNNING",
<   "state_message": ""
< }
05:12 DEBUG [databricks.sdk] POST /api/1.2/contexts/create
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "8540582676242715699"
< }
05:12 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=8540582676242715699
< 200 OK
< {
<   "id": "8540582676242715699",
<   "status": "Pending"
< }
05:12 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=8540582676242715699: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~1s)
05:12 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=8540582676242715699
< 200 OK
< {
<   "id": "8540582676242715699",
<   "status": "Pending"
< }
05:12 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=8540582676242715699: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~2s)
05:12 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=8540582676242715699
< 200 OK
< {
<   "id": "8540582676242715699",
<   "status": "Running"
< }
05:12 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "get_ipython().run_line_magic('pip', 'install /Workspace/Users/4106dc97-a963-48f0-a079-a578238959... (110 more bytes)",
>   "contextId": "8540582676242715699",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "41783686454c47679e627fb0b4c246cb"
< }
05:12 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=41783686454c47679e627fb0b4c246cb&contextId=8540582676242715699
< 200 OK
< {
<   "id": "41783686454c47679e627fb0b4c246cb",
<   "results": null,
<   "status": "Running"
< }
05:12 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=41783686454c47679e627fb0b4c246cb, context_id=8540582676242715699: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~1s)
05:12 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=41783686454c47679e627fb0b4c246cb&contextId=8540582676242715699
< 200 OK
< {
<   "id": "41783686454c47679e627fb0b4c246cb",
<   "results": null,
<   "status": "Running"
< }
05:12 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=41783686454c47679e627fb0b4c246cb, context_id=8540582676242715699: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~2s)
05:12 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=41783686454c47679e627fb0b4c246cb&contextId=8540582676242715699
< 200 OK
< {
<   "id": "41783686454c47679e627fb0b4c246cb",
<   "results": null,
<   "status": "Running"
< }
05:12 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=41783686454c47679e627fb0b4c246cb, context_id=8540582676242715699: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~3s)
05:12 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=41783686454c47679e627fb0b4c246cb&contextId=8540582676242715699
< 200 OK
< {
<   "id": "41783686454c47679e627fb0b4c246cb",
<   "results": null,
<   "status": "Running"
< }
05:12 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=41783686454c47679e627fb0b4c246cb, context_id=8540582676242715699: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~4s)
05:12 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=41783686454c47679e627fb0b4c246cb&contextId=8540582676242715699
< 200 OK
< {
<   "id": "41783686454c47679e627fb0b4c246cb",
<   "results": {
<     "data": "Processing /Workspace/Users/4106dc97-a963-48f0-a079-a578238959a6/.lub3/wheels/databricks_labs_ls... (5292 more bytes)",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:12 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "from databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unkno... (145 more bytes)",
>   "contextId": "8540582676242715699",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "bb2b86c5eff74fe7bff7e74f0b57f5c6"
< }
05:12 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=bb2b86c5eff74fe7bff7e74f0b57f5c6&contextId=8540582676242715699
< 200 OK
< {
<   "id": "bb2b86c5eff74fe7bff7e74f0b57f5c6",
<   "results": {
<     "data": "FAILED",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:12 DEBUG [databricks.sdk] Loaded from environment
05:12 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:12 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:12 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:12 INFO [databricks.sdk] Using Databricks Metadata Service authentication
05:12 DEBUG [databricks.sdk] GET /api/2.0/preview/scim/v2/Me
< 200 OK
< {
<   "active": true,
<   "displayName": "labs-runtime-identity",
<   "emails": [
<     {
<       "primary": true,
<       "type": "work",
<       "value": "**REDACTED**"
<     }
<   ],
<   "externalId": "d0f9bd2c-5651-45fd-b648-12a3fc6375c4",
<   "groups": [
<     {
<       "$ref": "Groups/153383108335587",
<       "display": "users",
<       "type": "direct",
<       "value": "**REDACTED**"
<     },
<     "... (1 additional elements)"
<   ],
<   "id": "4643477475987733",
<   "name": {
<     "givenName": "labs-runtime-identity"
<   },
<   "schemas": [
<     "urn:ietf:params:scim:schemas:core:2.0:User",
<     "... (1 additional elements)"
<   ],
<   "userName": "4106dc97-a963-48f0-a079-a578238959a6"
< }
05:12 DEBUG [databricks.labs.blueprint.wheels] Building wheel for /tmp/tmpmhg8za31/working-copy in /tmp/tmpmhg8za31
05:12 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.lub3/wheels/databricks_labs_lsql-0.16.1+320260221051215-py3-none-any.whl
05:12 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:12 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 404 Not Found
< {
<   "error_code": "RESOURCE_DOES_NOT_EXIST",
<   "message": "The parent folder (/Users/4106dc97-a963-48f0-a079-a578238959a6/.lub3/wheels) does not exist."
< }
05:12 DEBUG [databricks.labs.blueprint.installation] Creating missing folders: /Users/4106dc97-a963-48f0-a079-a578238959a6/.lub3/wheels
05:12 DEBUG [databricks.sdk] POST /api/2.0/workspace/mkdirs
> {
>   "path": "/Users/4106dc97-a963-48f0-a079-a578238959a6/.lub3/wheels"
> }
< 200 OK
< {}
05:12 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:12 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 2269762301694365
< }
05:12 DEBUG [databricks.labs.blueprint.installation] Converting Version into JSON format
05:12 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.lub3/version.json
05:12 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:12 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 2269762301694366
< }
05:12 DEBUG [databricks.sdk] GET /api/2.1/clusters/get?cluster_id=DATABRICKS_CLUSTER_ID
< 200 OK
< {
<   "autotermination_minutes": 60,
<   "CLOUD_ENV_attributes": {
<     "availability": "ON_DEMAND_AZURE",
<     "first_on_demand": 2147483647,
<     "spot_bid_max_price": -1.0
<   },
<   "cluster_cores": 8.0,
<   "cluster_id": "DATABRICKS_CLUSTER_ID",
<   "cluster_memory_mb": 32768,
<   "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<   "cluster_source": "UI",
<   "creator_user_name": "liran.bareket@databricks.com",
<   "custom_tags": {
<     "ResourceClass": "SingleNode"
<   },
<   "data_security_mode": "SINGLE_USER",
<   "TEST_SCHEMA_tags": {
<     "Budget": "opex.sales.labs",
<     "ClusterId": "DATABRICKS_CLUSTER_ID",
<     "ClusterName": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "Creator": "liran.bareket@databricks.com",
<     "DatabricksInstanceGroupId": "-7571316921879686317",
<     "DatabricksInstancePoolCreatorId": "6779888502363704",
<     "DatabricksInstancePoolId": "TEST_INSTANCE_POOL_ID",
<     "Owner": "labs-oss@databricks.com",
<     "Vendor": "Databricks"
<   },
<   "disk_spec": {},
<   "driver": {
<     "host_private_ip": "10.179.12.11",
<     "instance_id": "203e46ddda10498e8b81ade820bf805c",
<     "ngrok_endpoint_base_domain": "green.mux.ngrok-dataplane.wildcard",
<     "node_attributes": {
<       "is_spot": false
<     },
<     "node_id": "df1ca9159b784f72a68d9f4e20a92ec8",
<     "node_type_id": "Standard_D8ads_v6",
<     "private_ip": "10.179.14.11",
<     "public_dns": "",
<     "start_timestamp": 1771650660388
<   },
<   "driver_healthy": true,
<   "driver_instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "driver_instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "driver_node_type_id": "Standard_D8ads_v6",
<   "effective_spark_version": "16.4.x-scala2.12",
<   "enable_elastic_disk": true,
<   "enable_local_disk_encryption": false,
<   "init_scripts_safe_mode": false,
<   "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "jdbc_port": 10000,
<   "last_activity_time": 1771650681885,
<   "last_restarted_time": 1771650706345,
<   "last_state_loss_time": 1771650706302,
<   "node_type_id": "Standard_D8ads_v6",
<   "num_workers": 0,
<   "pinned_by_user_name": "6779888502363704",
<   "release_version": "16.4.18",
<   "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<   "spark_conf": {
<     "spark.databricks.cluster.profile": "singleNode",
<     "spark.master": "local[*]"
<   },
<   "spark_context_id": 6829547792133273073,
<   "spark_version": "16.4.x-scala2.12",
<   "spec": {
<     "autotermination_minutes": 60,
<     "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "custom_tags": {
<       "ResourceClass": "SingleNode"
<     },
<     "data_security_mode": "SINGLE_USER",
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<     "num_workers": 0,
<     "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<     "spark_conf": {
<       "spark.databricks.cluster.profile": "singleNode",
<       "spark.master": "local[*]"
<     },
<     "spark_version": "16.4.x-scala2.12"
<   },
<   "start_time": 1759339672984,
<   "state": "RUNNING",
<   "state_message": ""
< }
05:12 DEBUG [databricks.sdk] POST /api/1.2/contexts/create
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "8540582676242715699"
< }
05:12 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=8540582676242715699
< 200 OK
< {
<   "id": "8540582676242715699",
<   "status": "Pending"
< }
05:12 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=8540582676242715699: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~1s)
05:12 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=8540582676242715699
< 200 OK
< {
<   "id": "8540582676242715699",
<   "status": "Pending"
< }
05:12 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=8540582676242715699: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~2s)
05:12 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=8540582676242715699
< 200 OK
< {
<   "id": "8540582676242715699",
<   "status": "Running"
< }
05:12 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "get_ipython().run_line_magic('pip', 'install /Workspace/Users/4106dc97-a963-48f0-a079-a578238959... (110 more bytes)",
>   "contextId": "8540582676242715699",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "41783686454c47679e627fb0b4c246cb"
< }
05:12 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=41783686454c47679e627fb0b4c246cb&contextId=8540582676242715699
< 200 OK
< {
<   "id": "41783686454c47679e627fb0b4c246cb",
<   "results": null,
<   "status": "Running"
< }
05:12 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=41783686454c47679e627fb0b4c246cb, context_id=8540582676242715699: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~1s)
05:12 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=41783686454c47679e627fb0b4c246cb&contextId=8540582676242715699
< 200 OK
< {
<   "id": "41783686454c47679e627fb0b4c246cb",
<   "results": null,
<   "status": "Running"
< }
05:12 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=41783686454c47679e627fb0b4c246cb, context_id=8540582676242715699: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~2s)
05:12 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=41783686454c47679e627fb0b4c246cb&contextId=8540582676242715699
< 200 OK
< {
<   "id": "41783686454c47679e627fb0b4c246cb",
<   "results": null,
<   "status": "Running"
< }
05:12 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=41783686454c47679e627fb0b4c246cb, context_id=8540582676242715699: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~3s)
05:12 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=41783686454c47679e627fb0b4c246cb&contextId=8540582676242715699
< 200 OK
< {
<   "id": "41783686454c47679e627fb0b4c246cb",
<   "results": null,
<   "status": "Running"
< }
05:12 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=41783686454c47679e627fb0b4c246cb, context_id=8540582676242715699: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~4s)
05:12 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=41783686454c47679e627fb0b4c246cb&contextId=8540582676242715699
< 200 OK
< {
<   "id": "41783686454c47679e627fb0b4c246cb",
<   "results": {
<     "data": "Processing /Workspace/Users/4106dc97-a963-48f0-a079-a578238959a6/.lub3/wheels/databricks_labs_ls... (5292 more bytes)",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:12 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "from databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unkno... (145 more bytes)",
>   "contextId": "8540582676242715699",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "bb2b86c5eff74fe7bff7e74f0b57f5c6"
< }
05:12 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=bb2b86c5eff74fe7bff7e74f0b57f5c6&contextId=8540582676242715699
< 200 OK
< {
<   "id": "bb2b86c5eff74fe7bff7e74f0b57f5c6",
<   "results": {
<     "data": "FAILED",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
[gw0] linux -- Python 3.10.19 /home/runner/work/lsql/lsql/.venv/bin/python

_{Running from nightly #466}

[github-actions]

❌ test_runtime_backend_errors_handled[\nfrom databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unknown\nbackend = RuntimeBackend()\ntry:\n grants = backend.fetch("SHOW GRANTS ON METASTORE")\n print("FAILED")\nexcept Unknown:\n print("PASSED")\n]: AssertionError: assert 'FAILED' == 'PASSED' (22.411s)

AssertionError: assert 'FAILED' == 'PASSED'
  
  - PASSED
  + FAILED
05:13 DEBUG [databricks.sdk] Loaded from environment
05:13 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:13 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:13 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:13 INFO [databricks.sdk] Using Databricks Metadata Service authentication
[gw3] linux -- Python 3.10.19 /home/runner/work/lsql/lsql/.venv/bin/python
05:13 DEBUG [databricks.sdk] Loaded from environment
05:13 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:13 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:13 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:13 INFO [databricks.sdk] Using Databricks Metadata Service authentication
05:13 DEBUG [databricks.sdk] GET /api/2.0/preview/scim/v2/Me
< 200 OK
< {
<   "active": true,
<   "displayName": "labs-runtime-identity",
<   "emails": [
<     {
<       "primary": true,
<       "type": "work",
<       "value": "**REDACTED**"
<     }
<   ],
<   "externalId": "d0f9bd2c-5651-45fd-b648-12a3fc6375c4",
<   "groups": [
<     {
<       "$ref": "Groups/153383108335587",
<       "display": "users",
<       "type": "direct",
<       "value": "**REDACTED**"
<     },
<     "... (1 additional elements)"
<   ],
<   "id": "4643477475987733",
<   "name": {
<     "givenName": "labs-runtime-identity"
<   },
<   "schemas": [
<     "urn:ietf:params:scim:schemas:core:2.0:User",
<     "... (1 additional elements)"
<   ],
<   "userName": "4106dc97-a963-48f0-a079-a578238959a6"
< }
05:13 DEBUG [databricks.labs.blueprint.wheels] Building wheel for /tmp/tmp_vzrx54z/working-copy in /tmp/tmp_vzrx54z
05:14 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.HeqQ/wheels/databricks_labs_lsql-0.16.1+320260222051359-py3-none-any.whl
05:14 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:14 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 404 Not Found
< {
<   "error_code": "RESOURCE_DOES_NOT_EXIST",
<   "message": "The parent folder (/Users/4106dc97-a963-48f0-a079-a578238959a6/.HeqQ/wheels) does not exist."
< }
05:14 DEBUG [databricks.labs.blueprint.installation] Creating missing folders: /Users/4106dc97-a963-48f0-a079-a578238959a6/.HeqQ/wheels
05:14 DEBUG [databricks.sdk] POST /api/2.0/workspace/mkdirs
> {
>   "path": "/Users/4106dc97-a963-48f0-a079-a578238959a6/.HeqQ/wheels"
> }
< 200 OK
< {}
05:14 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:14 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 4220239999802227
< }
05:14 DEBUG [databricks.labs.blueprint.installation] Converting Version into JSON format
05:14 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.HeqQ/version.json
05:14 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:14 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 4220239999802229
< }
05:14 DEBUG [databricks.sdk] GET /api/2.1/clusters/get?cluster_id=DATABRICKS_CLUSTER_ID
< 200 OK
< {
<   "autotermination_minutes": 60,
<   "CLOUD_ENV_attributes": {
<     "availability": "ON_DEMAND_AZURE",
<     "first_on_demand": 2147483647,
<     "spot_bid_max_price": -1.0
<   },
<   "cluster_cores": 8.0,
<   "cluster_id": "DATABRICKS_CLUSTER_ID",
<   "cluster_memory_mb": 32768,
<   "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<   "cluster_source": "UI",
<   "creator_user_name": "liran.bareket@databricks.com",
<   "custom_tags": {
<     "ResourceClass": "SingleNode"
<   },
<   "data_security_mode": "SINGLE_USER",
<   "TEST_SCHEMA_tags": {
<     "Budget": "opex.sales.labs",
<     "ClusterId": "DATABRICKS_CLUSTER_ID",
<     "ClusterName": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "Creator": "liran.bareket@databricks.com",
<     "DatabricksInstanceGroupId": "-7571316921879686317",
<     "DatabricksInstancePoolCreatorId": "6779888502363704",
<     "DatabricksInstancePoolId": "TEST_INSTANCE_POOL_ID",
<     "Owner": "labs-oss@databricks.com",
<     "Vendor": "Databricks"
<   },
<   "disk_spec": {},
<   "driver": {
<     "host_private_ip": "10.179.12.13",
<     "instance_id": "1041a978f67c49e4bdb7a56077d2bfd9",
<     "ngrok_endpoint_base_domain": "green.mux.ngrok-dataplane.wildcard",
<     "node_attributes": {
<       "is_spot": false
<     },
<     "node_id": "86c441a1a6ee40b68a262a4ac9195af1",
<     "node_type_id": "Standard_D8ads_v6",
<     "private_ip": "10.179.14.13",
<     "public_dns": "",
<     "start_timestamp": 1771737158037
<   },
<   "driver_healthy": true,
<   "driver_instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "driver_instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "driver_node_type_id": "Standard_D8ads_v6",
<   "effective_spark_version": "16.4.x-scala2.12",
<   "enable_elastic_disk": true,
<   "enable_local_disk_encryption": false,
<   "init_scripts_safe_mode": false,
<   "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "jdbc_port": 10000,
<   "last_activity_time": 1771737180422,
<   "last_restarted_time": 1771737206849,
<   "last_state_loss_time": 1771737206781,
<   "node_type_id": "Standard_D8ads_v6",
<   "num_workers": 0,
<   "pinned_by_user_name": "6779888502363704",
<   "release_version": "16.4.18",
<   "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<   "spark_conf": {
<     "spark.databricks.cluster.profile": "singleNode",
<     "spark.master": "local[*]"
<   },
<   "spark_context_id": 4307623356946638632,
<   "spark_version": "16.4.x-scala2.12",
<   "spec": {
<     "autotermination_minutes": 60,
<     "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "custom_tags": {
<       "ResourceClass": "SingleNode"
<     },
<     "data_security_mode": "SINGLE_USER",
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<     "num_workers": 0,
<     "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<     "spark_conf": {
<       "spark.databricks.cluster.profile": "singleNode",
<       "spark.master": "local[*]"
<     },
<     "spark_version": "16.4.x-scala2.12"
<   },
<   "start_time": 1759339672984,
<   "state": "RUNNING",
<   "state_message": ""
< }
05:14 DEBUG [databricks.sdk] POST /api/1.2/contexts/create
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "7349873641904391663"
< }
05:14 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=7349873641904391663
< 200 OK
< {
<   "id": "7349873641904391663",
<   "status": "Pending"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=7349873641904391663: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~1s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=7349873641904391663
< 200 OK
< {
<   "id": "7349873641904391663",
<   "status": "Pending"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=7349873641904391663: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~2s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=7349873641904391663
< 200 OK
< {
<   "id": "7349873641904391663",
<   "status": "Running"
< }
05:14 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "get_ipython().run_line_magic('pip', 'install /Workspace/Users/4106dc97-a963-48f0-a079-a578238959... (110 more bytes)",
>   "contextId": "7349873641904391663",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "4b89495fdbec42f88b0acde6afdcc63f"
< }
05:14 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=4b89495fdbec42f88b0acde6afdcc63f&contextId=7349873641904391663
< 200 OK
< {
<   "id": "4b89495fdbec42f88b0acde6afdcc63f",
<   "results": null,
<   "status": "Running"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=4b89495fdbec42f88b0acde6afdcc63f, context_id=7349873641904391663: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~1s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=4b89495fdbec42f88b0acde6afdcc63f&contextId=7349873641904391663
< 200 OK
< {
<   "id": "4b89495fdbec42f88b0acde6afdcc63f",
<   "results": null,
<   "status": "Running"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=4b89495fdbec42f88b0acde6afdcc63f, context_id=7349873641904391663: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~2s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=4b89495fdbec42f88b0acde6afdcc63f&contextId=7349873641904391663
< 200 OK
< {
<   "id": "4b89495fdbec42f88b0acde6afdcc63f",
<   "results": null,
<   "status": "Running"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=4b89495fdbec42f88b0acde6afdcc63f, context_id=7349873641904391663: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~3s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=4b89495fdbec42f88b0acde6afdcc63f&contextId=7349873641904391663
< 200 OK
< {
<   "id": "4b89495fdbec42f88b0acde6afdcc63f",
<   "results": null,
<   "status": "Running"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=4b89495fdbec42f88b0acde6afdcc63f, context_id=7349873641904391663: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~4s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=4b89495fdbec42f88b0acde6afdcc63f&contextId=7349873641904391663
< 200 OK
< {
<   "id": "4b89495fdbec42f88b0acde6afdcc63f",
<   "results": {
<     "data": "Processing /Workspace/Users/4106dc97-a963-48f0-a079-a578238959a6/.HeqQ/wheels/databricks_labs_ls... (5292 more bytes)",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:14 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "from databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unkno... (145 more bytes)",
>   "contextId": "7349873641904391663",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "26b1900f04fd4b9c9e6102daf00e77af"
< }
05:14 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=26b1900f04fd4b9c9e6102daf00e77af&contextId=7349873641904391663
< 200 OK
< {
<   "id": "26b1900f04fd4b9c9e6102daf00e77af",
<   "results": {
<     "data": "FAILED",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:13 DEBUG [databricks.sdk] Loaded from environment
05:13 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:13 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:13 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:13 INFO [databricks.sdk] Using Databricks Metadata Service authentication
05:13 DEBUG [databricks.sdk] GET /api/2.0/preview/scim/v2/Me
< 200 OK
< {
<   "active": true,
<   "displayName": "labs-runtime-identity",
<   "emails": [
<     {
<       "primary": true,
<       "type": "work",
<       "value": "**REDACTED**"
<     }
<   ],
<   "externalId": "d0f9bd2c-5651-45fd-b648-12a3fc6375c4",
<   "groups": [
<     {
<       "$ref": "Groups/153383108335587",
<       "display": "users",
<       "type": "direct",
<       "value": "**REDACTED**"
<     },
<     "... (1 additional elements)"
<   ],
<   "id": "4643477475987733",
<   "name": {
<     "givenName": "labs-runtime-identity"
<   },
<   "schemas": [
<     "urn:ietf:params:scim:schemas:core:2.0:User",
<     "... (1 additional elements)"
<   ],
<   "userName": "4106dc97-a963-48f0-a079-a578238959a6"
< }
05:13 DEBUG [databricks.labs.blueprint.wheels] Building wheel for /tmp/tmp_vzrx54z/working-copy in /tmp/tmp_vzrx54z
05:14 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.HeqQ/wheels/databricks_labs_lsql-0.16.1+320260222051359-py3-none-any.whl
05:14 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:14 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 404 Not Found
< {
<   "error_code": "RESOURCE_DOES_NOT_EXIST",
<   "message": "The parent folder (/Users/4106dc97-a963-48f0-a079-a578238959a6/.HeqQ/wheels) does not exist."
< }
05:14 DEBUG [databricks.labs.blueprint.installation] Creating missing folders: /Users/4106dc97-a963-48f0-a079-a578238959a6/.HeqQ/wheels
05:14 DEBUG [databricks.sdk] POST /api/2.0/workspace/mkdirs
> {
>   "path": "/Users/4106dc97-a963-48f0-a079-a578238959a6/.HeqQ/wheels"
> }
< 200 OK
< {}
05:14 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:14 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 4220239999802227
< }
05:14 DEBUG [databricks.labs.blueprint.installation] Converting Version into JSON format
05:14 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.HeqQ/version.json
05:14 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:14 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 4220239999802229
< }
05:14 DEBUG [databricks.sdk] GET /api/2.1/clusters/get?cluster_id=DATABRICKS_CLUSTER_ID
< 200 OK
< {
<   "autotermination_minutes": 60,
<   "CLOUD_ENV_attributes": {
<     "availability": "ON_DEMAND_AZURE",
<     "first_on_demand": 2147483647,
<     "spot_bid_max_price": -1.0
<   },
<   "cluster_cores": 8.0,
<   "cluster_id": "DATABRICKS_CLUSTER_ID",
<   "cluster_memory_mb": 32768,
<   "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<   "cluster_source": "UI",
<   "creator_user_name": "liran.bareket@databricks.com",
<   "custom_tags": {
<     "ResourceClass": "SingleNode"
<   },
<   "data_security_mode": "SINGLE_USER",
<   "TEST_SCHEMA_tags": {
<     "Budget": "opex.sales.labs",
<     "ClusterId": "DATABRICKS_CLUSTER_ID",
<     "ClusterName": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "Creator": "liran.bareket@databricks.com",
<     "DatabricksInstanceGroupId": "-7571316921879686317",
<     "DatabricksInstancePoolCreatorId": "6779888502363704",
<     "DatabricksInstancePoolId": "TEST_INSTANCE_POOL_ID",
<     "Owner": "labs-oss@databricks.com",
<     "Vendor": "Databricks"
<   },
<   "disk_spec": {},
<   "driver": {
<     "host_private_ip": "10.179.12.13",
<     "instance_id": "1041a978f67c49e4bdb7a56077d2bfd9",
<     "ngrok_endpoint_base_domain": "green.mux.ngrok-dataplane.wildcard",
<     "node_attributes": {
<       "is_spot": false
<     },
<     "node_id": "86c441a1a6ee40b68a262a4ac9195af1",
<     "node_type_id": "Standard_D8ads_v6",
<     "private_ip": "10.179.14.13",
<     "public_dns": "",
<     "start_timestamp": 1771737158037
<   },
<   "driver_healthy": true,
<   "driver_instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "driver_instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "driver_node_type_id": "Standard_D8ads_v6",
<   "effective_spark_version": "16.4.x-scala2.12",
<   "enable_elastic_disk": true,
<   "enable_local_disk_encryption": false,
<   "init_scripts_safe_mode": false,
<   "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "jdbc_port": 10000,
<   "last_activity_time": 1771737180422,
<   "last_restarted_time": 1771737206849,
<   "last_state_loss_time": 1771737206781,
<   "node_type_id": "Standard_D8ads_v6",
<   "num_workers": 0,
<   "pinned_by_user_name": "6779888502363704",
<   "release_version": "16.4.18",
<   "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<   "spark_conf": {
<     "spark.databricks.cluster.profile": "singleNode",
<     "spark.master": "local[*]"
<   },
<   "spark_context_id": 4307623356946638632,
<   "spark_version": "16.4.x-scala2.12",
<   "spec": {
<     "autotermination_minutes": 60,
<     "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "custom_tags": {
<       "ResourceClass": "SingleNode"
<     },
<     "data_security_mode": "SINGLE_USER",
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<     "num_workers": 0,
<     "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<     "spark_conf": {
<       "spark.databricks.cluster.profile": "singleNode",
<       "spark.master": "local[*]"
<     },
<     "spark_version": "16.4.x-scala2.12"
<   },
<   "start_time": 1759339672984,
<   "state": "RUNNING",
<   "state_message": ""
< }
05:14 DEBUG [databricks.sdk] POST /api/1.2/contexts/create
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "7349873641904391663"
< }
05:14 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=7349873641904391663
< 200 OK
< {
<   "id": "7349873641904391663",
<   "status": "Pending"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=7349873641904391663: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~1s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=7349873641904391663
< 200 OK
< {
<   "id": "7349873641904391663",
<   "status": "Pending"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=7349873641904391663: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~2s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=7349873641904391663
< 200 OK
< {
<   "id": "7349873641904391663",
<   "status": "Running"
< }
05:14 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "get_ipython().run_line_magic('pip', 'install /Workspace/Users/4106dc97-a963-48f0-a079-a578238959... (110 more bytes)",
>   "contextId": "7349873641904391663",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "4b89495fdbec42f88b0acde6afdcc63f"
< }
05:14 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=4b89495fdbec42f88b0acde6afdcc63f&contextId=7349873641904391663
< 200 OK
< {
<   "id": "4b89495fdbec42f88b0acde6afdcc63f",
<   "results": null,
<   "status": "Running"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=4b89495fdbec42f88b0acde6afdcc63f, context_id=7349873641904391663: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~1s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=4b89495fdbec42f88b0acde6afdcc63f&contextId=7349873641904391663
< 200 OK
< {
<   "id": "4b89495fdbec42f88b0acde6afdcc63f",
<   "results": null,
<   "status": "Running"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=4b89495fdbec42f88b0acde6afdcc63f, context_id=7349873641904391663: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~2s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=4b89495fdbec42f88b0acde6afdcc63f&contextId=7349873641904391663
< 200 OK
< {
<   "id": "4b89495fdbec42f88b0acde6afdcc63f",
<   "results": null,
<   "status": "Running"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=4b89495fdbec42f88b0acde6afdcc63f, context_id=7349873641904391663: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~3s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=4b89495fdbec42f88b0acde6afdcc63f&contextId=7349873641904391663
< 200 OK
< {
<   "id": "4b89495fdbec42f88b0acde6afdcc63f",
<   "results": null,
<   "status": "Running"
< }
05:14 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=4b89495fdbec42f88b0acde6afdcc63f, context_id=7349873641904391663: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~4s)
05:14 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=4b89495fdbec42f88b0acde6afdcc63f&contextId=7349873641904391663
< 200 OK
< {
<   "id": "4b89495fdbec42f88b0acde6afdcc63f",
<   "results": {
<     "data": "Processing /Workspace/Users/4106dc97-a963-48f0-a079-a578238959a6/.HeqQ/wheels/databricks_labs_ls... (5292 more bytes)",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:14 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "from databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unkno... (145 more bytes)",
>   "contextId": "7349873641904391663",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "26b1900f04fd4b9c9e6102daf00e77af"
< }
05:14 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=26b1900f04fd4b9c9e6102daf00e77af&contextId=7349873641904391663
< 200 OK
< {
<   "id": "26b1900f04fd4b9c9e6102daf00e77af",
<   "results": {
<     "data": "FAILED",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
[gw3] linux -- Python 3.10.19 /home/runner/work/lsql/lsql/.venv/bin/python

_{Running from nightly #467}

[github-actions]

❌ test_runtime_backend_errors_handled[\nfrom databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unknown\nbackend = RuntimeBackend()\ntry:\n grants = backend.fetch("SHOW GRANTS ON METASTORE")\n print("FAILED")\nexcept Unknown:\n print("PASSED")\n]: AssertionError: assert 'FAILED' == 'PASSED' (23.307s)

AssertionError: assert 'FAILED' == 'PASSED'
  
  - PASSED
  + FAILED
05:20 DEBUG [databricks.sdk] Loaded from environment
05:20 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:20 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:20 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:20 INFO [databricks.sdk] Using Databricks Metadata Service authentication
[gw1] linux -- Python 3.10.19 /home/runner/work/lsql/lsql/.venv/bin/python
05:20 DEBUG [databricks.sdk] Loaded from environment
05:20 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:20 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:20 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:20 INFO [databricks.sdk] Using Databricks Metadata Service authentication
05:20 DEBUG [databricks.sdk] GET /api/2.0/preview/scim/v2/Me
< 200 OK
< {
<   "active": true,
<   "displayName": "labs-runtime-identity",
<   "emails": [
<     {
<       "primary": true,
<       "type": "work",
<       "value": "**REDACTED**"
<     }
<   ],
<   "externalId": "d0f9bd2c-5651-45fd-b648-12a3fc6375c4",
<   "groups": [
<     {
<       "$ref": "Groups/153383108335587",
<       "display": "users",
<       "type": "direct",
<       "value": "**REDACTED**"
<     },
<     "... (1 additional elements)"
<   ],
<   "id": "4643477475987733",
<   "name": {
<     "givenName": "labs-runtime-identity"
<   },
<   "schemas": [
<     "urn:ietf:params:scim:schemas:core:2.0:User",
<     "... (1 additional elements)"
<   ],
<   "userName": "4106dc97-a963-48f0-a079-a578238959a6"
< }
05:20 DEBUG [databricks.labs.blueprint.wheels] Building wheel for /tmp/tmph_3ckst1/working-copy in /tmp/tmph_3ckst1
05:20 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.zBmA/wheels/databricks_labs_lsql-0.16.1+320260223052039-py3-none-any.whl
05:20 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:20 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 404 Not Found
< {
<   "error_code": "RESOURCE_DOES_NOT_EXIST",
<   "message": "The parent folder (/Users/4106dc97-a963-48f0-a079-a578238959a6/.zBmA/wheels) does not exist."
< }
05:20 DEBUG [databricks.labs.blueprint.installation] Creating missing folders: /Users/4106dc97-a963-48f0-a079-a578238959a6/.zBmA/wheels
05:20 DEBUG [databricks.sdk] POST /api/2.0/workspace/mkdirs
> {
>   "path": "/Users/4106dc97-a963-48f0-a079-a578238959a6/.zBmA/wheels"
> }
< 200 OK
< {}
05:20 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:20 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 4277213951769641
< }
05:20 DEBUG [databricks.labs.blueprint.installation] Converting Version into JSON format
05:20 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.zBmA/version.json
05:20 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:20 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 4277213951769642
< }
05:20 DEBUG [databricks.sdk] GET /api/2.1/clusters/get?cluster_id=DATABRICKS_CLUSTER_ID
< 200 OK
< {
<   "autotermination_minutes": 60,
<   "CLOUD_ENV_attributes": {
<     "availability": "ON_DEMAND_AZURE",
<     "first_on_demand": 2147483647,
<     "spot_bid_max_price": -1.0
<   },
<   "cluster_cores": 8.0,
<   "cluster_id": "DATABRICKS_CLUSTER_ID",
<   "cluster_memory_mb": 32768,
<   "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<   "cluster_source": "UI",
<   "creator_user_name": "liran.bareket@databricks.com",
<   "custom_tags": {
<     "ResourceClass": "SingleNode"
<   },
<   "data_security_mode": "SINGLE_USER",
<   "TEST_SCHEMA_tags": {
<     "Budget": "opex.sales.labs",
<     "ClusterId": "DATABRICKS_CLUSTER_ID",
<     "ClusterName": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "Creator": "liran.bareket@databricks.com",
<     "DatabricksInstanceGroupId": "-7571316921879686317",
<     "DatabricksInstancePoolCreatorId": "6779888502363704",
<     "DatabricksInstancePoolId": "TEST_INSTANCE_POOL_ID",
<     "Owner": "labs-oss@databricks.com",
<     "Vendor": "Databricks"
<   },
<   "disk_spec": {},
<   "driver": {
<     "host_private_ip": "10.179.12.13",
<     "instance_id": "46a7bcce5af44447af28a6bfe7ed5a49",
<     "ngrok_endpoint_base_domain": "green.mux.ngrok-dataplane.wildcard",
<     "node_attributes": {
<       "is_spot": false
<     },
<     "node_id": "d0fef0c8f2ab4adfac9c89fdab173e27",
<     "node_type_id": "Standard_D8ads_v6",
<     "private_ip": "10.179.14.13",
<     "public_dns": "",
<     "start_timestamp": 1771823957991
<   },
<   "driver_healthy": true,
<   "driver_instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "driver_instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "driver_node_type_id": "Standard_D8ads_v6",
<   "effective_spark_version": "16.4.x-scala2.12",
<   "enable_elastic_disk": true,
<   "enable_local_disk_encryption": false,
<   "init_scripts_safe_mode": false,
<   "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "jdbc_port": 10000,
<   "last_activity_time": 1771824037118,
<   "last_restarted_time": 1771824008103,
<   "last_state_loss_time": 1771824008040,
<   "node_type_id": "Standard_D8ads_v6",
<   "num_workers": 0,
<   "pinned_by_user_name": "6779888502363704",
<   "release_version": "16.4.18",
<   "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<   "spark_conf": {
<     "spark.databricks.cluster.profile": "singleNode",
<     "spark.master": "local[*]"
<   },
<   "spark_context_id": 3103662224390108264,
<   "spark_version": "16.4.x-scala2.12",
<   "spec": {
<     "autotermination_minutes": 60,
<     "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "custom_tags": {
<       "ResourceClass": "SingleNode"
<     },
<     "data_security_mode": "SINGLE_USER",
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<     "num_workers": 0,
<     "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<     "spark_conf": {
<       "spark.databricks.cluster.profile": "singleNode",
<       "spark.master": "local[*]"
<     },
<     "spark_version": "16.4.x-scala2.12"
<   },
<   "start_time": 1759339672984,
<   "state": "RUNNING",
<   "state_message": ""
< }
05:20 DEBUG [databricks.sdk] POST /api/1.2/contexts/create
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "1971467173894024699"
< }
05:20 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=1971467173894024699
< 200 OK
< {
<   "id": "1971467173894024699",
<   "status": "Pending"
< }
05:20 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=1971467173894024699: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~1s)
05:20 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=1971467173894024699
< 200 OK
< {
<   "id": "1971467173894024699",
<   "status": "Pending"
< }
05:20 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=1971467173894024699: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~2s)
05:20 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=1971467173894024699
< 200 OK
< {
<   "id": "1971467173894024699",
<   "status": "Running"
< }
05:20 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "get_ipython().run_line_magic('pip', 'install /Workspace/Users/4106dc97-a963-48f0-a079-a578238959... (110 more bytes)",
>   "contextId": "1971467173894024699",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "9f116fc8747c45f2b39e7e156f6edd5c"
< }
05:20 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=9f116fc8747c45f2b39e7e156f6edd5c&contextId=1971467173894024699
< 200 OK
< {
<   "id": "9f116fc8747c45f2b39e7e156f6edd5c",
<   "results": null,
<   "status": "Running"
< }
05:20 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=9f116fc8747c45f2b39e7e156f6edd5c, context_id=1971467173894024699: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~1s)
05:20 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=9f116fc8747c45f2b39e7e156f6edd5c&contextId=1971467173894024699
< 200 OK
< {
<   "id": "9f116fc8747c45f2b39e7e156f6edd5c",
<   "results": null,
<   "status": "Running"
< }
05:20 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=9f116fc8747c45f2b39e7e156f6edd5c, context_id=1971467173894024699: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~2s)
05:20 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=9f116fc8747c45f2b39e7e156f6edd5c&contextId=1971467173894024699
< 200 OK
< {
<   "id": "9f116fc8747c45f2b39e7e156f6edd5c",
<   "results": null,
<   "status": "Running"
< }
05:20 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=9f116fc8747c45f2b39e7e156f6edd5c, context_id=1971467173894024699: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~3s)
05:20 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=9f116fc8747c45f2b39e7e156f6edd5c&contextId=1971467173894024699
< 200 OK
< {
<   "id": "9f116fc8747c45f2b39e7e156f6edd5c",
<   "results": null,
<   "status": "Running"
< }
05:20 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=9f116fc8747c45f2b39e7e156f6edd5c, context_id=1971467173894024699: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~4s)
05:21 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=9f116fc8747c45f2b39e7e156f6edd5c&contextId=1971467173894024699
< 200 OK
< {
<   "id": "9f116fc8747c45f2b39e7e156f6edd5c",
<   "results": {
<     "data": "Processing /Workspace/Users/4106dc97-a963-48f0-a079-a578238959a6/.zBmA/wheels/databricks_labs_ls... (5292 more bytes)",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:21 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "from databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unkno... (145 more bytes)",
>   "contextId": "1971467173894024699",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "096688be9c1f4233b53078e2f2074983"
< }
05:21 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=096688be9c1f4233b53078e2f2074983&contextId=1971467173894024699
< 200 OK
< {
<   "id": "096688be9c1f4233b53078e2f2074983",
<   "results": {
<     "data": "FAILED",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:20 DEBUG [databricks.sdk] Loaded from environment
05:20 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:20 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:20 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:20 INFO [databricks.sdk] Using Databricks Metadata Service authentication
05:20 DEBUG [databricks.sdk] GET /api/2.0/preview/scim/v2/Me
< 200 OK
< {
<   "active": true,
<   "displayName": "labs-runtime-identity",
<   "emails": [
<     {
<       "primary": true,
<       "type": "work",
<       "value": "**REDACTED**"
<     }
<   ],
<   "externalId": "d0f9bd2c-5651-45fd-b648-12a3fc6375c4",
<   "groups": [
<     {
<       "$ref": "Groups/153383108335587",
<       "display": "users",
<       "type": "direct",
<       "value": "**REDACTED**"
<     },
<     "... (1 additional elements)"
<   ],
<   "id": "4643477475987733",
<   "name": {
<     "givenName": "labs-runtime-identity"
<   },
<   "schemas": [
<     "urn:ietf:params:scim:schemas:core:2.0:User",
<     "... (1 additional elements)"
<   ],
<   "userName": "4106dc97-a963-48f0-a079-a578238959a6"
< }
05:20 DEBUG [databricks.labs.blueprint.wheels] Building wheel for /tmp/tmph_3ckst1/working-copy in /tmp/tmph_3ckst1
05:20 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.zBmA/wheels/databricks_labs_lsql-0.16.1+320260223052039-py3-none-any.whl
05:20 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:20 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 404 Not Found
< {
<   "error_code": "RESOURCE_DOES_NOT_EXIST",
<   "message": "The parent folder (/Users/4106dc97-a963-48f0-a079-a578238959a6/.zBmA/wheels) does not exist."
< }
05:20 DEBUG [databricks.labs.blueprint.installation] Creating missing folders: /Users/4106dc97-a963-48f0-a079-a578238959a6/.zBmA/wheels
05:20 DEBUG [databricks.sdk] POST /api/2.0/workspace/mkdirs
> {
>   "path": "/Users/4106dc97-a963-48f0-a079-a578238959a6/.zBmA/wheels"
> }
< 200 OK
< {}
05:20 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:20 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 4277213951769641
< }
05:20 DEBUG [databricks.labs.blueprint.installation] Converting Version into JSON format
05:20 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.zBmA/version.json
05:20 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:20 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 4277213951769642
< }
05:20 DEBUG [databricks.sdk] GET /api/2.1/clusters/get?cluster_id=DATABRICKS_CLUSTER_ID
< 200 OK
< {
<   "autotermination_minutes": 60,
<   "CLOUD_ENV_attributes": {
<     "availability": "ON_DEMAND_AZURE",
<     "first_on_demand": 2147483647,
<     "spot_bid_max_price": -1.0
<   },
<   "cluster_cores": 8.0,
<   "cluster_id": "DATABRICKS_CLUSTER_ID",
<   "cluster_memory_mb": 32768,
<   "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<   "cluster_source": "UI",
<   "creator_user_name": "liran.bareket@databricks.com",
<   "custom_tags": {
<     "ResourceClass": "SingleNode"
<   },
<   "data_security_mode": "SINGLE_USER",
<   "TEST_SCHEMA_tags": {
<     "Budget": "opex.sales.labs",
<     "ClusterId": "DATABRICKS_CLUSTER_ID",
<     "ClusterName": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "Creator": "liran.bareket@databricks.com",
<     "DatabricksInstanceGroupId": "-7571316921879686317",
<     "DatabricksInstancePoolCreatorId": "6779888502363704",
<     "DatabricksInstancePoolId": "TEST_INSTANCE_POOL_ID",
<     "Owner": "labs-oss@databricks.com",
<     "Vendor": "Databricks"
<   },
<   "disk_spec": {},
<   "driver": {
<     "host_private_ip": "10.179.12.13",
<     "instance_id": "46a7bcce5af44447af28a6bfe7ed5a49",
<     "ngrok_endpoint_base_domain": "green.mux.ngrok-dataplane.wildcard",
<     "node_attributes": {
<       "is_spot": false
<     },
<     "node_id": "d0fef0c8f2ab4adfac9c89fdab173e27",
<     "node_type_id": "Standard_D8ads_v6",
<     "private_ip": "10.179.14.13",
<     "public_dns": "",
<     "start_timestamp": 1771823957991
<   },
<   "driver_healthy": true,
<   "driver_instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "driver_instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "driver_node_type_id": "Standard_D8ads_v6",
<   "effective_spark_version": "16.4.x-scala2.12",
<   "enable_elastic_disk": true,
<   "enable_local_disk_encryption": false,
<   "init_scripts_safe_mode": false,
<   "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "jdbc_port": 10000,
<   "last_activity_time": 1771824037118,
<   "last_restarted_time": 1771824008103,
<   "last_state_loss_time": 1771824008040,
<   "node_type_id": "Standard_D8ads_v6",
<   "num_workers": 0,
<   "pinned_by_user_name": "6779888502363704",
<   "release_version": "16.4.18",
<   "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<   "spark_conf": {
<     "spark.databricks.cluster.profile": "singleNode",
<     "spark.master": "local[*]"
<   },
<   "spark_context_id": 3103662224390108264,
<   "spark_version": "16.4.x-scala2.12",
<   "spec": {
<     "autotermination_minutes": 60,
<     "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "custom_tags": {
<       "ResourceClass": "SingleNode"
<     },
<     "data_security_mode": "SINGLE_USER",
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<     "num_workers": 0,
<     "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<     "spark_conf": {
<       "spark.databricks.cluster.profile": "singleNode",
<       "spark.master": "local[*]"
<     },
<     "spark_version": "16.4.x-scala2.12"
<   },
<   "start_time": 1759339672984,
<   "state": "RUNNING",
<   "state_message": ""
< }
05:20 DEBUG [databricks.sdk] POST /api/1.2/contexts/create
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "1971467173894024699"
< }
05:20 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=1971467173894024699
< 200 OK
< {
<   "id": "1971467173894024699",
<   "status": "Pending"
< }
05:20 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=1971467173894024699: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~1s)
05:20 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=1971467173894024699
< 200 OK
< {
<   "id": "1971467173894024699",
<   "status": "Pending"
< }
05:20 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=1971467173894024699: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~2s)
05:20 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=1971467173894024699
< 200 OK
< {
<   "id": "1971467173894024699",
<   "status": "Running"
< }
05:20 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "get_ipython().run_line_magic('pip', 'install /Workspace/Users/4106dc97-a963-48f0-a079-a578238959... (110 more bytes)",
>   "contextId": "1971467173894024699",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "9f116fc8747c45f2b39e7e156f6edd5c"
< }
05:20 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=9f116fc8747c45f2b39e7e156f6edd5c&contextId=1971467173894024699
< 200 OK
< {
<   "id": "9f116fc8747c45f2b39e7e156f6edd5c",
<   "results": null,
<   "status": "Running"
< }
05:20 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=9f116fc8747c45f2b39e7e156f6edd5c, context_id=1971467173894024699: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~1s)
05:20 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=9f116fc8747c45f2b39e7e156f6edd5c&contextId=1971467173894024699
< 200 OK
< {
<   "id": "9f116fc8747c45f2b39e7e156f6edd5c",
<   "results": null,
<   "status": "Running"
< }
05:20 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=9f116fc8747c45f2b39e7e156f6edd5c, context_id=1971467173894024699: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~2s)
05:20 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=9f116fc8747c45f2b39e7e156f6edd5c&contextId=1971467173894024699
< 200 OK
< {
<   "id": "9f116fc8747c45f2b39e7e156f6edd5c",
<   "results": null,
<   "status": "Running"
< }
05:20 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=9f116fc8747c45f2b39e7e156f6edd5c, context_id=1971467173894024699: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~3s)
05:20 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=9f116fc8747c45f2b39e7e156f6edd5c&contextId=1971467173894024699
< 200 OK
< {
<   "id": "9f116fc8747c45f2b39e7e156f6edd5c",
<   "results": null,
<   "status": "Running"
< }
05:20 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=9f116fc8747c45f2b39e7e156f6edd5c, context_id=1971467173894024699: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~4s)
05:21 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=9f116fc8747c45f2b39e7e156f6edd5c&contextId=1971467173894024699
< 200 OK
< {
<   "id": "9f116fc8747c45f2b39e7e156f6edd5c",
<   "results": {
<     "data": "Processing /Workspace/Users/4106dc97-a963-48f0-a079-a578238959a6/.zBmA/wheels/databricks_labs_ls... (5292 more bytes)",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:21 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "from databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unkno... (145 more bytes)",
>   "contextId": "1971467173894024699",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "096688be9c1f4233b53078e2f2074983"
< }
05:21 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=096688be9c1f4233b53078e2f2074983&contextId=1971467173894024699
< 200 OK
< {
<   "id": "096688be9c1f4233b53078e2f2074983",
<   "results": {
<     "data": "FAILED",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
[gw1] linux -- Python 3.10.19 /home/runner/work/lsql/lsql/.venv/bin/python

_{Running from nightly #468}

[github-actions]

❌ test_runtime_backend_errors_handled[\nfrom databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unknown\nbackend = RuntimeBackend()\ntry:\n grants = backend.fetch("SHOW GRANTS ON METASTORE")\n print("FAILED")\nexcept Unknown:\n print("PASSED")\n]: AssertionError: assert 'FAILED' == 'PASSED' (19.928s)

AssertionError: assert 'FAILED' == 'PASSED'
  
  - PASSED
  + FAILED
05:17 DEBUG [databricks.sdk] Loaded from environment
05:17 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:17 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:17 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:17 INFO [databricks.sdk] Using Databricks Metadata Service authentication
[gw3] linux -- Python 3.10.19 /home/runner/work/lsql/lsql/.venv/bin/python
05:17 DEBUG [databricks.sdk] Loaded from environment
05:17 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:17 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:17 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:17 INFO [databricks.sdk] Using Databricks Metadata Service authentication
05:17 DEBUG [databricks.sdk] GET /api/2.0/preview/scim/v2/Me
< 200 OK
< {
<   "active": true,
<   "displayName": "labs-runtime-identity",
<   "emails": [
<     {
<       "primary": true,
<       "type": "work",
<       "value": "**REDACTED**"
<     }
<   ],
<   "externalId": "d0f9bd2c-5651-45fd-b648-12a3fc6375c4",
<   "groups": [
<     {
<       "$ref": "Groups/153383108335587",
<       "display": "users",
<       "type": "direct",
<       "value": "**REDACTED**"
<     },
<     "... (1 additional elements)"
<   ],
<   "id": "4643477475987733",
<   "name": {
<     "givenName": "labs-runtime-identity"
<   },
<   "schemas": [
<     "urn:ietf:params:scim:schemas:core:2.0:User",
<     "... (1 additional elements)"
<   ],
<   "userName": "4106dc97-a963-48f0-a079-a578238959a6"
< }
05:17 DEBUG [databricks.labs.blueprint.wheels] Building wheel for /tmp/tmpa2hfakwv/working-copy in /tmp/tmpa2hfakwv
05:17 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.GvGQ/wheels/databricks_labs_lsql-0.16.1+320260224051757-py3-none-any.whl
05:17 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:17 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 404 Not Found
< {
<   "error_code": "RESOURCE_DOES_NOT_EXIST",
<   "message": "The parent folder (/Users/4106dc97-a963-48f0-a079-a578238959a6/.GvGQ/wheels) does not exist."
< }
05:17 DEBUG [databricks.labs.blueprint.installation] Creating missing folders: /Users/4106dc97-a963-48f0-a079-a578238959a6/.GvGQ/wheels
05:17 DEBUG [databricks.sdk] POST /api/2.0/workspace/mkdirs
> {
>   "path": "/Users/4106dc97-a963-48f0-a079-a578238959a6/.GvGQ/wheels"
> }
< 200 OK
< {}
05:17 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:17 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 3081735736544106
< }
05:17 DEBUG [databricks.labs.blueprint.installation] Converting Version into JSON format
05:17 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.GvGQ/version.json
05:17 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:17 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 3081735736544107
< }
05:17 DEBUG [databricks.sdk] GET /api/2.1/clusters/get?cluster_id=DATABRICKS_CLUSTER_ID
< 200 OK
< {
<   "autotermination_minutes": 60,
<   "CLOUD_ENV_attributes": {
<     "availability": "ON_DEMAND_AZURE",
<     "first_on_demand": 2147483647,
<     "spot_bid_max_price": -1.0
<   },
<   "cluster_cores": 8.0,
<   "cluster_id": "DATABRICKS_CLUSTER_ID",
<   "cluster_memory_mb": 32768,
<   "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<   "cluster_source": "UI",
<   "creator_user_name": "liran.bareket@databricks.com",
<   "custom_tags": {
<     "ResourceClass": "SingleNode"
<   },
<   "data_security_mode": "SINGLE_USER",
<   "TEST_SCHEMA_tags": {
<     "Budget": "opex.sales.labs",
<     "ClusterId": "DATABRICKS_CLUSTER_ID",
<     "ClusterName": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "Creator": "liran.bareket@databricks.com",
<     "DatabricksInstanceGroupId": "-7571316921879686317",
<     "DatabricksInstancePoolCreatorId": "6779888502363704",
<     "DatabricksInstancePoolId": "TEST_INSTANCE_POOL_ID",
<     "Owner": "labs-oss@databricks.com",
<     "Vendor": "Databricks"
<   },
<   "disk_spec": {},
<   "driver": {
<     "host_private_ip": "10.179.12.33",
<     "instance_id": "6dab17782ecc4e92876fa6853ecde565",
<     "ngrok_endpoint_base_domain": "green.mux.ngrok-dataplane.wildcard",
<     "node_attributes": {
<       "is_spot": false
<     },
<     "node_id": "70fdf0821566473ca1c918bfe76f4eb4",
<     "node_type_id": "Standard_D8ads_v6",
<     "private_ip": "10.179.14.33",
<     "public_dns": "",
<     "start_timestamp": 1771910202140
<   },
<   "driver_healthy": true,
<   "driver_instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "driver_instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "driver_node_type_id": "Standard_D8ads_v6",
<   "effective_spark_version": "16.4.x-scala2.12",
<   "enable_elastic_disk": true,
<   "enable_local_disk_encryption": false,
<   "init_scripts_safe_mode": false,
<   "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "jdbc_port": 10000,
<   "last_activity_time": 1771910222736,
<   "last_restarted_time": 1771910246998,
<   "last_state_loss_time": 1771910246942,
<   "node_type_id": "Standard_D8ads_v6",
<   "num_workers": 0,
<   "pinned_by_user_name": "6779888502363704",
<   "release_version": "16.4.18",
<   "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<   "spark_conf": {
<     "spark.databricks.cluster.profile": "singleNode",
<     "spark.master": "local[*]"
<   },
<   "spark_context_id": 3493511403551292915,
<   "spark_version": "16.4.x-scala2.12",
<   "spec": {
<     "autotermination_minutes": 60,
<     "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "custom_tags": {
<       "ResourceClass": "SingleNode"
<     },
<     "data_security_mode": "SINGLE_USER",
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<     "num_workers": 0,
<     "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<     "spark_conf": {
<       "spark.databricks.cluster.profile": "singleNode",
<       "spark.master": "local[*]"
<     },
<     "spark_version": "16.4.x-scala2.12"
<   },
<   "start_time": 1759339672984,
<   "state": "RUNNING",
<   "state_message": ""
< }
05:17 DEBUG [databricks.sdk] POST /api/1.2/contexts/create
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "2250003573544634086"
< }
05:17 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=2250003573544634086
< 200 OK
< {
<   "id": "2250003573544634086",
<   "status": "Pending"
< }
05:17 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=2250003573544634086: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~1s)
05:18 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=2250003573544634086
< 200 OK
< {
<   "id": "2250003573544634086",
<   "status": "Pending"
< }
05:18 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=2250003573544634086: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~2s)
05:18 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=2250003573544634086
< 200 OK
< {
<   "id": "2250003573544634086",
<   "status": "Running"
< }
05:18 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "get_ipython().run_line_magic('pip', 'install /Workspace/Users/4106dc97-a963-48f0-a079-a578238959... (110 more bytes)",
>   "contextId": "2250003573544634086",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "3967a96a41e5491190c434f29353c042"
< }
05:18 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=3967a96a41e5491190c434f29353c042&contextId=2250003573544634086
< 200 OK
< {
<   "id": "3967a96a41e5491190c434f29353c042",
<   "results": null,
<   "status": "Running"
< }
05:18 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=3967a96a41e5491190c434f29353c042, context_id=2250003573544634086: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~1s)
05:18 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=3967a96a41e5491190c434f29353c042&contextId=2250003573544634086
< 200 OK
< {
<   "id": "3967a96a41e5491190c434f29353c042",
<   "results": null,
<   "status": "Running"
< }
05:18 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=3967a96a41e5491190c434f29353c042, context_id=2250003573544634086: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~2s)
05:18 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=3967a96a41e5491190c434f29353c042&contextId=2250003573544634086
< 200 OK
< {
<   "id": "3967a96a41e5491190c434f29353c042",
<   "results": null,
<   "status": "Running"
< }
05:18 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=3967a96a41e5491190c434f29353c042, context_id=2250003573544634086: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~3s)
05:18 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=3967a96a41e5491190c434f29353c042&contextId=2250003573544634086
< 200 OK
< {
<   "id": "3967a96a41e5491190c434f29353c042",
<   "results": {
<     "data": "Processing /Workspace/Users/4106dc97-a963-48f0-a079-a578238959a6/.GvGQ/wheels/databricks_labs_ls... (5292 more bytes)",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:18 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "from databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unkno... (145 more bytes)",
>   "contextId": "2250003573544634086",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "9ac9884589ae4e09b39684c016423b2e"
< }
05:18 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=9ac9884589ae4e09b39684c016423b2e&contextId=2250003573544634086
< 200 OK
< {
<   "id": "9ac9884589ae4e09b39684c016423b2e",
<   "results": {
<     "data": "FAILED",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:17 DEBUG [databricks.sdk] Loaded from environment
05:17 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:17 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:17 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:17 INFO [databricks.sdk] Using Databricks Metadata Service authentication
05:17 DEBUG [databricks.sdk] GET /api/2.0/preview/scim/v2/Me
< 200 OK
< {
<   "active": true,
<   "displayName": "labs-runtime-identity",
<   "emails": [
<     {
<       "primary": true,
<       "type": "work",
<       "value": "**REDACTED**"
<     }
<   ],
<   "externalId": "d0f9bd2c-5651-45fd-b648-12a3fc6375c4",
<   "groups": [
<     {
<       "$ref": "Groups/153383108335587",
<       "display": "users",
<       "type": "direct",
<       "value": "**REDACTED**"
<     },
<     "... (1 additional elements)"
<   ],
<   "id": "4643477475987733",
<   "name": {
<     "givenName": "labs-runtime-identity"
<   },
<   "schemas": [
<     "urn:ietf:params:scim:schemas:core:2.0:User",
<     "... (1 additional elements)"
<   ],
<   "userName": "4106dc97-a963-48f0-a079-a578238959a6"
< }
05:17 DEBUG [databricks.labs.blueprint.wheels] Building wheel for /tmp/tmpa2hfakwv/working-copy in /tmp/tmpa2hfakwv
05:17 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.GvGQ/wheels/databricks_labs_lsql-0.16.1+320260224051757-py3-none-any.whl
05:17 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:17 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 404 Not Found
< {
<   "error_code": "RESOURCE_DOES_NOT_EXIST",
<   "message": "The parent folder (/Users/4106dc97-a963-48f0-a079-a578238959a6/.GvGQ/wheels) does not exist."
< }
05:17 DEBUG [databricks.labs.blueprint.installation] Creating missing folders: /Users/4106dc97-a963-48f0-a079-a578238959a6/.GvGQ/wheels
05:17 DEBUG [databricks.sdk] POST /api/2.0/workspace/mkdirs
> {
>   "path": "/Users/4106dc97-a963-48f0-a079-a578238959a6/.GvGQ/wheels"
> }
< 200 OK
< {}
05:17 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:17 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 3081735736544106
< }
05:17 DEBUG [databricks.labs.blueprint.installation] Converting Version into JSON format
05:17 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.GvGQ/version.json
05:17 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:17 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 3081735736544107
< }
05:17 DEBUG [databricks.sdk] GET /api/2.1/clusters/get?cluster_id=DATABRICKS_CLUSTER_ID
< 200 OK
< {
<   "autotermination_minutes": 60,
<   "CLOUD_ENV_attributes": {
<     "availability": "ON_DEMAND_AZURE",
<     "first_on_demand": 2147483647,
<     "spot_bid_max_price": -1.0
<   },
<   "cluster_cores": 8.0,
<   "cluster_id": "DATABRICKS_CLUSTER_ID",
<   "cluster_memory_mb": 32768,
<   "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<   "cluster_source": "UI",
<   "creator_user_name": "liran.bareket@databricks.com",
<   "custom_tags": {
<     "ResourceClass": "SingleNode"
<   },
<   "data_security_mode": "SINGLE_USER",
<   "TEST_SCHEMA_tags": {
<     "Budget": "opex.sales.labs",
<     "ClusterId": "DATABRICKS_CLUSTER_ID",
<     "ClusterName": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "Creator": "liran.bareket@databricks.com",
<     "DatabricksInstanceGroupId": "-7571316921879686317",
<     "DatabricksInstancePoolCreatorId": "6779888502363704",
<     "DatabricksInstancePoolId": "TEST_INSTANCE_POOL_ID",
<     "Owner": "labs-oss@databricks.com",
<     "Vendor": "Databricks"
<   },
<   "disk_spec": {},
<   "driver": {
<     "host_private_ip": "10.179.12.33",
<     "instance_id": "6dab17782ecc4e92876fa6853ecde565",
<     "ngrok_endpoint_base_domain": "green.mux.ngrok-dataplane.wildcard",
<     "node_attributes": {
<       "is_spot": false
<     },
<     "node_id": "70fdf0821566473ca1c918bfe76f4eb4",
<     "node_type_id": "Standard_D8ads_v6",
<     "private_ip": "10.179.14.33",
<     "public_dns": "",
<     "start_timestamp": 1771910202140
<   },
<   "driver_healthy": true,
<   "driver_instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "driver_instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "driver_node_type_id": "Standard_D8ads_v6",
<   "effective_spark_version": "16.4.x-scala2.12",
<   "enable_elastic_disk": true,
<   "enable_local_disk_encryption": false,
<   "init_scripts_safe_mode": false,
<   "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "jdbc_port": 10000,
<   "last_activity_time": 1771910222736,
<   "last_restarted_time": 1771910246998,
<   "last_state_loss_time": 1771910246942,
<   "node_type_id": "Standard_D8ads_v6",
<   "num_workers": 0,
<   "pinned_by_user_name": "6779888502363704",
<   "release_version": "16.4.18",
<   "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<   "spark_conf": {
<     "spark.databricks.cluster.profile": "singleNode",
<     "spark.master": "local[*]"
<   },
<   "spark_context_id": 3493511403551292915,
<   "spark_version": "16.4.x-scala2.12",
<   "spec": {
<     "autotermination_minutes": 60,
<     "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "custom_tags": {
<       "ResourceClass": "SingleNode"
<     },
<     "data_security_mode": "SINGLE_USER",
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<     "num_workers": 0,
<     "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<     "spark_conf": {
<       "spark.databricks.cluster.profile": "singleNode",
<       "spark.master": "local[*]"
<     },
<     "spark_version": "16.4.x-scala2.12"
<   },
<   "start_time": 1759339672984,
<   "state": "RUNNING",
<   "state_message": ""
< }
05:17 DEBUG [databricks.sdk] POST /api/1.2/contexts/create
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "2250003573544634086"
< }
05:17 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=2250003573544634086
< 200 OK
< {
<   "id": "2250003573544634086",
<   "status": "Pending"
< }
05:17 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=2250003573544634086: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~1s)
05:18 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=2250003573544634086
< 200 OK
< {
<   "id": "2250003573544634086",
<   "status": "Pending"
< }
05:18 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=2250003573544634086: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~2s)
05:18 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=2250003573544634086
< 200 OK
< {
<   "id": "2250003573544634086",
<   "status": "Running"
< }
05:18 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "get_ipython().run_line_magic('pip', 'install /Workspace/Users/4106dc97-a963-48f0-a079-a578238959... (110 more bytes)",
>   "contextId": "2250003573544634086",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "3967a96a41e5491190c434f29353c042"
< }
05:18 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=3967a96a41e5491190c434f29353c042&contextId=2250003573544634086
< 200 OK
< {
<   "id": "3967a96a41e5491190c434f29353c042",
<   "results": null,
<   "status": "Running"
< }
05:18 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=3967a96a41e5491190c434f29353c042, context_id=2250003573544634086: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~1s)
05:18 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=3967a96a41e5491190c434f29353c042&contextId=2250003573544634086
< 200 OK
< {
<   "id": "3967a96a41e5491190c434f29353c042",
<   "results": null,
<   "status": "Running"
< }
05:18 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=3967a96a41e5491190c434f29353c042, context_id=2250003573544634086: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~2s)
05:18 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=3967a96a41e5491190c434f29353c042&contextId=2250003573544634086
< 200 OK
< {
<   "id": "3967a96a41e5491190c434f29353c042",
<   "results": null,
<   "status": "Running"
< }
05:18 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=3967a96a41e5491190c434f29353c042, context_id=2250003573544634086: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~3s)
05:18 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=3967a96a41e5491190c434f29353c042&contextId=2250003573544634086
< 200 OK
< {
<   "id": "3967a96a41e5491190c434f29353c042",
<   "results": {
<     "data": "Processing /Workspace/Users/4106dc97-a963-48f0-a079-a578238959a6/.GvGQ/wheels/databricks_labs_ls... (5292 more bytes)",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:18 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "from databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unkno... (145 more bytes)",
>   "contextId": "2250003573544634086",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "9ac9884589ae4e09b39684c016423b2e"
< }
05:18 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=9ac9884589ae4e09b39684c016423b2e&contextId=2250003573544634086
< 200 OK
< {
<   "id": "9ac9884589ae4e09b39684c016423b2e",
<   "results": {
<     "data": "FAILED",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
[gw3] linux -- Python 3.10.19 /home/runner/work/lsql/lsql/.venv/bin/python

_{Running from nightly #469}

[github-actions]

❌ test_runtime_backend_errors_handled[\nfrom databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unknown\nbackend = RuntimeBackend()\ntry:\n grants = backend.fetch("SHOW GRANTS ON METASTORE")\n print("FAILED")\nexcept Unknown:\n print("PASSED")\n]: AssertionError: assert 'FAILED' == 'PASSED' (21.332s)

AssertionError: assert 'FAILED' == 'PASSED'
  
  - PASSED
  + FAILED
05:16 DEBUG [databricks.sdk] Loaded from environment
05:16 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:16 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:16 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:16 INFO [databricks.sdk] Using Databricks Metadata Service authentication
[gw2] linux -- Python 3.10.19 /home/runner/work/lsql/lsql/.venv/bin/python
05:16 DEBUG [databricks.sdk] Loaded from environment
05:16 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:16 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:16 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:16 INFO [databricks.sdk] Using Databricks Metadata Service authentication
05:16 DEBUG [databricks.sdk] GET /api/2.0/preview/scim/v2/Me
< 200 OK
< {
<   "active": true,
<   "displayName": "labs-runtime-identity",
<   "emails": [
<     {
<       "primary": true,
<       "type": "work",
<       "value": "**REDACTED**"
<     }
<   ],
<   "externalId": "d0f9bd2c-5651-45fd-b648-12a3fc6375c4",
<   "groups": [
<     {
<       "$ref": "Groups/153383108335587",
<       "display": "users",
<       "type": "direct",
<       "value": "**REDACTED**"
<     },
<     "... (1 additional elements)"
<   ],
<   "id": "4643477475987733",
<   "name": {
<     "givenName": "labs-runtime-identity"
<   },
<   "schemas": [
<     "urn:ietf:params:scim:schemas:core:2.0:User",
<     "... (1 additional elements)"
<   ],
<   "userName": "4106dc97-a963-48f0-a079-a578238959a6"
< }
05:16 DEBUG [databricks.labs.blueprint.wheels] Building wheel for /tmp/tmpbb4pmu25/working-copy in /tmp/tmpbb4pmu25
05:16 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.xMzP/wheels/databricks_labs_lsql-0.16.1+320260225051625-py3-none-any.whl
05:16 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:16 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 404 Not Found
< {
<   "error_code": "RESOURCE_DOES_NOT_EXIST",
<   "message": "The parent folder (/Users/4106dc97-a963-48f0-a079-a578238959a6/.xMzP/wheels) does not exist."
< }
05:16 DEBUG [databricks.labs.blueprint.installation] Creating missing folders: /Users/4106dc97-a963-48f0-a079-a578238959a6/.xMzP/wheels
05:16 DEBUG [databricks.sdk] POST /api/2.0/workspace/mkdirs
> {
>   "path": "/Users/4106dc97-a963-48f0-a079-a578238959a6/.xMzP/wheels"
> }
< 200 OK
< {}
05:16 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:16 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 1757016900061813
< }
05:16 DEBUG [databricks.labs.blueprint.installation] Converting Version into JSON format
05:16 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.xMzP/version.json
05:16 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:16 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 1757016900061819
< }
05:16 DEBUG [databricks.sdk] GET /api/2.1/clusters/get?cluster_id=DATABRICKS_CLUSTER_ID
< 200 OK
< {
<   "autotermination_minutes": 60,
<   "CLOUD_ENV_attributes": {
<     "availability": "ON_DEMAND_AZURE",
<     "first_on_demand": 2147483647,
<     "spot_bid_max_price": -1.0
<   },
<   "cluster_cores": 8.0,
<   "cluster_id": "DATABRICKS_CLUSTER_ID",
<   "cluster_memory_mb": 32768,
<   "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<   "cluster_source": "UI",
<   "creator_user_name": "liran.bareket@databricks.com",
<   "custom_tags": {
<     "ResourceClass": "SingleNode"
<   },
<   "data_security_mode": "SINGLE_USER",
<   "TEST_SCHEMA_tags": {
<     "Budget": "opex.sales.labs",
<     "ClusterId": "DATABRICKS_CLUSTER_ID",
<     "ClusterName": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "Creator": "liran.bareket@databricks.com",
<     "DatabricksInstanceGroupId": "-7571316921879686317",
<     "DatabricksInstancePoolCreatorId": "6779888502363704",
<     "DatabricksInstancePoolId": "TEST_INSTANCE_POOL_ID",
<     "Owner": "labs-oss@databricks.com",
<     "Vendor": "Databricks"
<   },
<   "disk_spec": {},
<   "driver": {
<     "host_private_ip": "10.179.12.12",
<     "instance_id": "bfb52877bb8b42ed899c54fb5acd9e8a",
<     "ngrok_endpoint_base_domain": "green.mux.ngrok-dataplane.wildcard",
<     "node_attributes": {
<       "is_spot": false
<     },
<     "node_id": "1f41903ebd33410d9346ea20a97a20fd",
<     "node_type_id": "Standard_D8ads_v6",
<     "private_ip": "10.179.14.12",
<     "public_dns": "",
<     "start_timestamp": 1771996515571
<   },
<   "driver_healthy": true,
<   "driver_instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "driver_instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "driver_node_type_id": "Standard_D8ads_v6",
<   "effective_spark_version": "16.4.x-scala2.12",
<   "enable_elastic_disk": true,
<   "enable_local_disk_encryption": false,
<   "init_scripts_safe_mode": false,
<   "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "jdbc_port": 10000,
<   "last_activity_time": 1771996532648,
<   "last_restarted_time": 1771996555325,
<   "last_state_loss_time": 1771996555289,
<   "node_type_id": "Standard_D8ads_v6",
<   "num_workers": 0,
<   "pinned_by_user_name": "6779888502363704",
<   "release_version": "16.4.18",
<   "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<   "spark_conf": {
<     "spark.databricks.cluster.profile": "singleNode",
<     "spark.master": "local[*]"
<   },
<   "spark_context_id": 1667503728518500372,
<   "spark_version": "16.4.x-scala2.12",
<   "spec": {
<     "autotermination_minutes": 60,
<     "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "custom_tags": {
<       "ResourceClass": "SingleNode"
<     },
<     "data_security_mode": "SINGLE_USER",
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<     "num_workers": 0,
<     "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<     "spark_conf": {
<       "spark.databricks.cluster.profile": "singleNode",
<       "spark.master": "local[*]"
<     },
<     "spark_version": "16.4.x-scala2.12"
<   },
<   "start_time": 1759339672984,
<   "state": "RUNNING",
<   "state_message": ""
< }
05:16 DEBUG [databricks.sdk] POST /api/1.2/contexts/create
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "5820422975766907989"
< }
05:16 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=5820422975766907989
< 200 OK
< {
<   "id": "5820422975766907989",
<   "status": "Pending"
< }
05:16 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=5820422975766907989: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~1s)
05:16 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=5820422975766907989
< 200 OK
< {
<   "id": "5820422975766907989",
<   "status": "Pending"
< }
05:16 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=5820422975766907989: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~2s)
05:16 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=5820422975766907989
< 200 OK
< {
<   "id": "5820422975766907989",
<   "status": "Pending"
< }
05:16 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=5820422975766907989: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~3s)
05:16 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=5820422975766907989
< 200 OK
< {
<   "id": "5820422975766907989",
<   "status": "Running"
< }
05:16 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "get_ipython().run_line_magic('pip', 'install /Workspace/Users/4106dc97-a963-48f0-a079-a578238959... (110 more bytes)",
>   "contextId": "5820422975766907989",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "ab6888b9d2f04c07994369661d5c2b66"
< }
05:16 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=ab6888b9d2f04c07994369661d5c2b66&contextId=5820422975766907989
< 200 OK
< {
<   "id": "ab6888b9d2f04c07994369661d5c2b66",
<   "results": null,
<   "status": "Running"
< }
05:16 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=ab6888b9d2f04c07994369661d5c2b66, context_id=5820422975766907989: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~1s)
05:16 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=ab6888b9d2f04c07994369661d5c2b66&contextId=5820422975766907989
< 200 OK
< {
<   "id": "ab6888b9d2f04c07994369661d5c2b66",
<   "results": null,
<   "status": "Running"
< }
05:16 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=ab6888b9d2f04c07994369661d5c2b66, context_id=5820422975766907989: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~2s)
05:16 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=ab6888b9d2f04c07994369661d5c2b66&contextId=5820422975766907989
< 200 OK
< {
<   "id": "ab6888b9d2f04c07994369661d5c2b66",
<   "results": null,
<   "status": "Running"
< }
05:16 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=ab6888b9d2f04c07994369661d5c2b66, context_id=5820422975766907989: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~3s)
05:16 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=ab6888b9d2f04c07994369661d5c2b66&contextId=5820422975766907989
< 200 OK
< {
<   "id": "ab6888b9d2f04c07994369661d5c2b66",
<   "results": {
<     "data": "Processing /Workspace/Users/4106dc97-a963-48f0-a079-a578238959a6/.xMzP/wheels/databricks_labs_ls... (5292 more bytes)",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:16 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "from databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unkno... (145 more bytes)",
>   "contextId": "5820422975766907989",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "7c03b0b87c99465ea22c16b35c5246da"
< }
05:16 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=7c03b0b87c99465ea22c16b35c5246da&contextId=5820422975766907989
< 200 OK
< {
<   "id": "7c03b0b87c99465ea22c16b35c5246da",
<   "results": {
<     "data": "FAILED",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:16 DEBUG [databricks.sdk] Loaded from environment
05:16 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred
05:16 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred
05:16 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service
05:16 INFO [databricks.sdk] Using Databricks Metadata Service authentication
05:16 DEBUG [databricks.sdk] GET /api/2.0/preview/scim/v2/Me
< 200 OK
< {
<   "active": true,
<   "displayName": "labs-runtime-identity",
<   "emails": [
<     {
<       "primary": true,
<       "type": "work",
<       "value": "**REDACTED**"
<     }
<   ],
<   "externalId": "d0f9bd2c-5651-45fd-b648-12a3fc6375c4",
<   "groups": [
<     {
<       "$ref": "Groups/153383108335587",
<       "display": "users",
<       "type": "direct",
<       "value": "**REDACTED**"
<     },
<     "... (1 additional elements)"
<   ],
<   "id": "4643477475987733",
<   "name": {
<     "givenName": "labs-runtime-identity"
<   },
<   "schemas": [
<     "urn:ietf:params:scim:schemas:core:2.0:User",
<     "... (1 additional elements)"
<   ],
<   "userName": "4106dc97-a963-48f0-a079-a578238959a6"
< }
05:16 DEBUG [databricks.labs.blueprint.wheels] Building wheel for /tmp/tmpbb4pmu25/working-copy in /tmp/tmpbb4pmu25
05:16 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.xMzP/wheels/databricks_labs_lsql-0.16.1+320260225051625-py3-none-any.whl
05:16 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:16 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 404 Not Found
< {
<   "error_code": "RESOURCE_DOES_NOT_EXIST",
<   "message": "The parent folder (/Users/4106dc97-a963-48f0-a079-a578238959a6/.xMzP/wheels) does not exist."
< }
05:16 DEBUG [databricks.labs.blueprint.installation] Creating missing folders: /Users/4106dc97-a963-48f0-a079-a578238959a6/.xMzP/wheels
05:16 DEBUG [databricks.sdk] POST /api/2.0/workspace/mkdirs
> {
>   "path": "/Users/4106dc97-a963-48f0-a079-a578238959a6/.xMzP/wheels"
> }
< 200 OK
< {}
05:16 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:16 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 1757016900061813
< }
05:16 DEBUG [databricks.labs.blueprint.installation] Converting Version into JSON format
05:16 DEBUG [databricks.labs.blueprint.installation] Uploading: /Users/4106dc97-a963-48f0-a079-a578238959a6/.xMzP/version.json
05:16 DEBUG [databricks.sdk] Retry disabled for non-seekable stream: type=<class 'dict'>
05:16 DEBUG [databricks.sdk] POST /api/2.0/workspace/import
> [raw stream]
< 200 OK
< {
<   "object_id": 1757016900061819
< }
05:16 DEBUG [databricks.sdk] GET /api/2.1/clusters/get?cluster_id=DATABRICKS_CLUSTER_ID
< 200 OK
< {
<   "autotermination_minutes": 60,
<   "CLOUD_ENV_attributes": {
<     "availability": "ON_DEMAND_AZURE",
<     "first_on_demand": 2147483647,
<     "spot_bid_max_price": -1.0
<   },
<   "cluster_cores": 8.0,
<   "cluster_id": "DATABRICKS_CLUSTER_ID",
<   "cluster_memory_mb": 32768,
<   "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<   "cluster_source": "UI",
<   "creator_user_name": "liran.bareket@databricks.com",
<   "custom_tags": {
<     "ResourceClass": "SingleNode"
<   },
<   "data_security_mode": "SINGLE_USER",
<   "TEST_SCHEMA_tags": {
<     "Budget": "opex.sales.labs",
<     "ClusterId": "DATABRICKS_CLUSTER_ID",
<     "ClusterName": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "Creator": "liran.bareket@databricks.com",
<     "DatabricksInstanceGroupId": "-7571316921879686317",
<     "DatabricksInstancePoolCreatorId": "6779888502363704",
<     "DatabricksInstancePoolId": "TEST_INSTANCE_POOL_ID",
<     "Owner": "labs-oss@databricks.com",
<     "Vendor": "Databricks"
<   },
<   "disk_spec": {},
<   "driver": {
<     "host_private_ip": "10.179.12.12",
<     "instance_id": "bfb52877bb8b42ed899c54fb5acd9e8a",
<     "ngrok_endpoint_base_domain": "green.mux.ngrok-dataplane.wildcard",
<     "node_attributes": {
<       "is_spot": false
<     },
<     "node_id": "1f41903ebd33410d9346ea20a97a20fd",
<     "node_type_id": "Standard_D8ads_v6",
<     "private_ip": "10.179.14.12",
<     "public_dns": "",
<     "start_timestamp": 1771996515571
<   },
<   "driver_healthy": true,
<   "driver_instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "driver_instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "driver_node_type_id": "Standard_D8ads_v6",
<   "effective_spark_version": "16.4.x-scala2.12",
<   "enable_elastic_disk": true,
<   "enable_local_disk_encryption": false,
<   "init_scripts_safe_mode": false,
<   "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<   "instance_source": {
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID"
<   },
<   "jdbc_port": 10000,
<   "last_activity_time": 1771996532648,
<   "last_restarted_time": 1771996555325,
<   "last_state_loss_time": 1771996555289,
<   "node_type_id": "Standard_D8ads_v6",
<   "num_workers": 0,
<   "pinned_by_user_name": "6779888502363704",
<   "release_version": "16.4.18",
<   "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<   "spark_conf": {
<     "spark.databricks.cluster.profile": "singleNode",
<     "spark.master": "local[*]"
<   },
<   "spark_context_id": 1667503728518500372,
<   "spark_version": "16.4.x-scala2.12",
<   "spec": {
<     "autotermination_minutes": 60,
<     "cluster_name": "Scoped MSI Cluster: runtime (Single Node, Single User)",
<     "custom_tags": {
<       "ResourceClass": "SingleNode"
<     },
<     "data_security_mode": "SINGLE_USER",
<     "instance_pool_id": "TEST_INSTANCE_POOL_ID",
<     "num_workers": 0,
<     "single_user_name": "4106dc97-a963-48f0-a079-a578238959a6",
<     "spark_conf": {
<       "spark.databricks.cluster.profile": "singleNode",
<       "spark.master": "local[*]"
<     },
<     "spark_version": "16.4.x-scala2.12"
<   },
<   "start_time": 1759339672984,
<   "state": "RUNNING",
<   "state_message": ""
< }
05:16 DEBUG [databricks.sdk] POST /api/1.2/contexts/create
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "5820422975766907989"
< }
05:16 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=5820422975766907989
< 200 OK
< {
<   "id": "5820422975766907989",
<   "status": "Pending"
< }
05:16 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=5820422975766907989: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~1s)
05:16 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=5820422975766907989
< 200 OK
< {
<   "id": "5820422975766907989",
<   "status": "Pending"
< }
05:16 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=5820422975766907989: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~2s)
05:16 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=5820422975766907989
< 200 OK
< {
<   "id": "5820422975766907989",
<   "status": "Pending"
< }
05:16 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, context_id=5820422975766907989: (ContextStatus.PENDING) current status: ContextStatus.PENDING (sleeping ~3s)
05:16 DEBUG [databricks.sdk] GET /api/1.2/contexts/status?clusterId=DATABRICKS_CLUSTER_ID&contextId=5820422975766907989
< 200 OK
< {
<   "id": "5820422975766907989",
<   "status": "Running"
< }
05:16 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "get_ipython().run_line_magic('pip', 'install /Workspace/Users/4106dc97-a963-48f0-a079-a578238959... (110 more bytes)",
>   "contextId": "5820422975766907989",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "ab6888b9d2f04c07994369661d5c2b66"
< }
05:16 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=ab6888b9d2f04c07994369661d5c2b66&contextId=5820422975766907989
< 200 OK
< {
<   "id": "ab6888b9d2f04c07994369661d5c2b66",
<   "results": null,
<   "status": "Running"
< }
05:16 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=ab6888b9d2f04c07994369661d5c2b66, context_id=5820422975766907989: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~1s)
05:16 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=ab6888b9d2f04c07994369661d5c2b66&contextId=5820422975766907989
< 200 OK
< {
<   "id": "ab6888b9d2f04c07994369661d5c2b66",
<   "results": null,
<   "status": "Running"
< }
05:16 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=ab6888b9d2f04c07994369661d5c2b66, context_id=5820422975766907989: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~2s)
05:16 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=ab6888b9d2f04c07994369661d5c2b66&contextId=5820422975766907989
< 200 OK
< {
<   "id": "ab6888b9d2f04c07994369661d5c2b66",
<   "results": null,
<   "status": "Running"
< }
05:16 DEBUG [databricks.sdk] cluster_id=DATABRICKS_CLUSTER_ID, command_id=ab6888b9d2f04c07994369661d5c2b66, context_id=5820422975766907989: (CommandStatus.RUNNING) current status: CommandStatus.RUNNING (sleeping ~3s)
05:16 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=ab6888b9d2f04c07994369661d5c2b66&contextId=5820422975766907989
< 200 OK
< {
<   "id": "ab6888b9d2f04c07994369661d5c2b66",
<   "results": {
<     "data": "Processing /Workspace/Users/4106dc97-a963-48f0-a079-a578238959a6/.xMzP/wheels/databricks_labs_ls... (5292 more bytes)",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
05:16 DEBUG [databricks.sdk] POST /api/1.2/commands/execute
> {
>   "clusterId": "DATABRICKS_CLUSTER_ID",
>   "command": "from databricks.labs.lsql.backends import RuntimeBackend\nfrom databricks.sdk.errors import Unkno... (145 more bytes)",
>   "contextId": "5820422975766907989",
>   "language": "python"
> }
< 200 OK
< {
<   "id": "7c03b0b87c99465ea22c16b35c5246da"
< }
05:16 DEBUG [databricks.sdk] GET /api/1.2/commands/status?clusterId=DATABRICKS_CLUSTER_ID&commandId=7c03b0b87c99465ea22c16b35c5246da&contextId=5820422975766907989
< 200 OK
< {
<   "id": "7c03b0b87c99465ea22c16b35c5246da",
<   "results": {
<     "data": "FAILED",
<     "resultType": "text"
<   },
<   "status": "Finished"
< }
[gw2] linux -- Python 3.10.19 /home/runner/work/lsql/lsql/.venv/bin/python

_{Running from nightly #470}