Move the timing of FaaS object creation to the Infra API

[jianzs]

Problem Statement

During the Proof of Concept (PoC) phase, the approach to adapting to cloud platforms involved using a centralized runtime.ts file to analyze event types and then encapsulate them. Specifically, when a FaaS resource instance receives a resource request, it first enters the processing flow of runtime.ts. This file determines the type of request based on certain attributes of the event object, extracts and encapsulates the event according to the request type into parameters needed by the Handler function, and then calls the Handler to execute. This method mingles the adaptation code for various components within a single file, including the HTTP adaptation code for Routers, the Event adaptation code for Queues, etc., which leads to several issues:

1. Contextual separation. SDK resource component developers create platform-specific Infra implementation classes for their resource types, but the code for adapting to the platform is in runtime.ts, separated from the Infra class; developers need to know where to add adaptation code. For example, Router resource types use AWS's ApiGateway, and while the relationship between Lambda and ApiGateway is built within the Router's implementation class, the code to adapt Lambda specifications and invoke the user's Handler is in runtime.ts.
2. Resource component conflict. Differentiating request types based on event object property values is not always accurate, and when two components rely on the same property values, it necessitates adjustments to the logic in runtime.ts, resulting in high coupling.

Solution Approach

The early PoC method involved the Deducer stage outputting an Arch Ref that directly included FaaS resource instances. This approach has been modified so that the Deducer only deduces the compute closures contained within the application, without directly representing these closures as corresponding FaaS resource objects.

FaaS resource objects will be created by specific implementations of the BaaS Infra API. Within these implementations, adaptation code will be added on top of the compute closures based on their own functions and cloud platform specifications, forming a higher-level closure. Subsequently, a FaaS resource object is created to complete the FaaS resource instance's creation.

Implementation Approach

Basic Setup

Each exported computational closure will be saved in its own directory, containing an index.ts file that by default exports the closure's entry point. For example:

export default (a: number, b: number): number => {
	return a + b;
};

Under the core subdirectory of @plutolang/base, define the ComputeClosure interface:

interface Dependency {
	resourceObject: Resource;
	resourceType: string; // Format: package.type e.g. "@plutolang/pluto.Queue"
	type: "access" | "props";
}

interface ComputeClosure {
	filepath: string;  // Path to the directory of compute closure 
	dependencies?: Dependency[]; // The resource objects that the compute closure relies on, which includes two types of dependencies: Client API calls and Property accesses.
}

Generator Output

When generating IaC code, the generator imports the closure and configures its properties based on the arch ref. For example:

const router = new Router("foo");

import { default as closure } from "./path/to/closure";
const closure_1 = closure as ComputeClosure & (typeof closure);
closure_1.filepath = "./path/to/closure";

closure_1.dependencies = []
closure_1.dependencies.push({ 
  resourceObject: router, 
  resourceType: "@plutolang/pluto.Router", 
  type: "props", 
  method: "url" 
})

router.get("/", closure_1);

Infra SDK Development

When implementing the Infra API for a resource type in the Infra SDK, it's necessary to check if the Handler is of ComputeClosure type. If it is, the dependencies must be passed to the FaaS resource object during its construction. For example:

class Router {
  public get(path: string, handler: RouterEndpointHandler) {
    const dependencies = isComputeClosure(handler) ? handler.dependencies : undefined;
    const fn = new Function(wrapper, { dependencies });
    // do something...

    function wrapper(event, context) {
      // adapt the platform specification.
      // And then, call the handler function.
      handler(/* arguments */);
    }
  }
}

In the constructor of the FaaS resource type Function, it's also necessary to determine whether the incoming handler is of ComputeClosure type. If it is, this indicates that the user has created a Function resource object directly in the code, with ComputeClosure attributes filled in by the generator. Otherwise, it signifies that a Function resource object was constructed in the SDK, as shown in the code above. For example:

class Function {
  constructor(fn: FunctionHandler, opts: FunctionOptions) {
    const dependencies = opts.dependencies ?? [];
    if (isComputeClosure(fn) && fn.dependencies) {
      // The user creates this function directly in the code.
      dependencies.push(...fn.dependencies)
    }


    // extract envrionment variables list from dependencies
    const lambda = new aws.lambda.Function(/* ... envs */);

    // construct permission policies list from dependencies
    lambda.addPermission(/* policies */)
  }
}

Benefits

This approach addresses the issues mentioned earlier: developers need only focus on adaptation code within the Infra API of the resource type, and the adaptation code for different resource types is unrelated.

Moreover, the timing of FaaS resource object creation is decided by the Infra class of the BaaS resource type, allowing the Infra class to dictate the granularity of the code executed in the FaaS resource instance. For instance, multiple compute closures could be combined, and then only one FaaS resource object is created and deployed to a FaaS resource instance.

Existing Challenges

SDK developers need to understand the concept of compute closures, and when developing the SDK, they need to recognize that the object passed to them could be of ComputeClosure type. Upon confirmation, developers need to actively add dependencies to the Function's Options parameter.