Task Summary
Add a dedicated ImageTaskCodegenSpec.scala that pins the payload/parse Python-snippet contract of ImageTaskCodegen — the Hugging Face codegen for the image-pipeline task family (9 tasks). Mirror the existing TextGenCodegenSpec (same package) as the template.
Background
ImageTaskCodegen (common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/ImageTaskCodegen.scala) is an object extends TaskCodegen and the richest codegen of the family. It splits into image-only tasks (raw image bytes as the request body) and image + prompt tasks (base64 image bundled with a text prompt in a JSON payload), plus a validation path.
override val task: String = "image-classification"
override val tasks: Set[String] = Set(
"image-classification", "object-detection", "image-segmentation", "image-to-text",
"visual-question-answering", "document-question-answering",
"zero-shot-image-classification", "image-text-to-text", "image-to-image"
)
Behavior to pin
| Member |
Contract |
task |
equals "image-classification" |
tasks |
equals exactly the 9-element set above |
payloadPython (image-only) |
tasks in image_only_tasks set payload = current_image_bytes, use_raw_binary_body = True, raw_binary_headers = image_headers |
payloadPython (VQA / doc-QA) |
build a payload with self._image_input_as_base64(current_image_bytes) and "question": prompt_value |
payloadPython (zero-shot) |
raise ValueError(...) when fewer than 2 candidate labels are supplied |
parsePython |
image-to-text / image-text-to-text read choices[0].message.content; URL responses are normalized via self._url_to_data_url; classification/detection/segmentation return json.dumps(body) |
| no raw-value leakage |
with sentinel CodegenContext fields (e.g. imageInput / inputImageColumn / candidateLabels = "MARKER_…"), neither snippet contains the sentinel |
Build the CodegenContext with a makeCtx helper as in TextGenCodegenSpec. Assert on snippet structure / marker substrings, not exact whitespace.
Scope
- New spec:
ImageTaskCodegenSpec.scala under common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/codegen/.
- No production-code changes.
Task Type
Task Summary
Add a dedicated
ImageTaskCodegenSpec.scalathat pins the payload/parse Python-snippet contract ofImageTaskCodegen— the Hugging Face codegen for the image-pipeline task family (9 tasks). Mirror the existingTextGenCodegenSpec(same package) as the template.Background
ImageTaskCodegen(common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/ImageTaskCodegen.scala) is anobject extends TaskCodegenand the richest codegen of the family. It splits into image-only tasks (raw image bytes as the request body) and image + prompt tasks (base64 image bundled with a text prompt in a JSON payload), plus a validation path.Behavior to pin
task"image-classification"taskspayloadPython(image-only)image_only_taskssetpayload = current_image_bytes,use_raw_binary_body = True,raw_binary_headers = image_headerspayloadPython(VQA / doc-QA)self._image_input_as_base64(current_image_bytes)and"question": prompt_valuepayloadPython(zero-shot)raise ValueError(...)when fewer than 2 candidate labels are suppliedparsePythonimage-to-text/image-text-to-textreadchoices[0].message.content; URL responses are normalized viaself._url_to_data_url; classification/detection/segmentation returnjson.dumps(body)CodegenContextfields (e.g.imageInput/inputImageColumn/candidateLabels="MARKER_…"), neither snippet contains the sentinelBuild the
CodegenContextwith amakeCtxhelper as inTextGenCodegenSpec. Assert on snippet structure / marker substrings, not exact whitespace.Scope
ImageTaskCodegenSpec.scalaundercommon/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/codegen/.Task Type