Persist the spec of the computing_unit entity and its relationship with workflow_executions into DB

[bobbai00]

This PR persists the computing unit information to the Database and uniforms the underlying implementation of single mode(with local computing master as the computing unit) and cluster mode(with kubernetes pod as the computing unit).

For Developers

Please do the following steps to incorporate with new changes:

1. Apply core/scripts/sql/updates/07.sql to your local postgres instance
2. Always starts ComputingUnitManagingService when you launch the Texera's backend micro-services.

Database Schema Changes

Modification of computing_unit Table

The following fields have been added:

• type: Type of computing unit (e.g., "local" or "kubernetes").
• resource: A json string contains:
  • cpu_limit: CPU resource limit allocated to the computing unit.
  • memory_limit: Memory resource limit allocated to the computing unit.
  • gpu_limit: GPU resource limit allocated to the computing unit.
  • jvm_memory_size: JVM memory size allocated to the computing unit.
  • node_addresses: List of node addresses associated with the computing unit.

Modification of workflow_execution Table

A new column cuid has been added to the existing workflow_execution table to establish a relationship between workflow executions and computing units. This column serves as a foreign key referencing the cuid in the computing_unit table.

Configuration File Changes

computing-unit.conf has been added

A variable computing-unit.local.enabled is added. The default value is true. In cluster mode, this should be set to false.

kubernetes.conf has been changed

A variable kubernetes.enabled is added. The default value is false. In cluster mode, this should be set to true.

User Experience

1. Cluster Mode

• Run & interact with the workflow on different computing units at the same time
  

• Check which computing unit is used in the execution via the execution history table
  

• If a workflow is running in some computing units, the workflow CANNOT be modified until there are no RUNNING executions
  

2. Single Node Mode (Local Development)

Single node Mode needs a little extra step of connecting to a computing unit: in the pop-up window, users can input an address that points to the computing unit master service (the default value is http://localhost:8085)

Users can also disconnect from the local computing unit by clicking on the delete button in the drop-down item:

Related Changes

1. Helm Chart

• KUBERNETES_COMPUTING_UNIT_ENABLED and COMPUTING_UNIT_LOCAL_ENABLED have been added
• request to the endpoint /api/executions/{wid}/stats/{eid} will be routed to envoy

2. Execution Lifecycle

• The latest execution of a workflow is now identified by BOTH wid and cuid, i.e. the latest execution of a workflow on a computing unit.
• Every time a new execution starts, the previous execution on the same computing unit will be cleared. The latest executions on other computing units will NOT be cleared.

3. Execution & Result Lifecycle Binding

• Currently, the lifecycle of the execution results and computing units are bundled.
• As a consequence, querying the result, including port result tables, stats, and console message, need to carry the cuid