Create new service diarizer for diarization

[thainguyensunya]

Changes:

• Create new diarizer service using pyannote/speaker-diarization-community-1 model
• Create helm chart for diarizer service
• New pipeline for diarizer service

[gemini-code-assist]

Code Review

This pull request introduces a new diarizer service and its corresponding Helm chart. The overall structure is good, but there are several high and critical severity issues that should be addressed. My review focuses on improving the reliability and performance of the FastAPI service by making it non-blocking, reducing memory usage during file uploads, and enhancing the Kubernetes configurations for better robustness and reusability. I've also included suggestions to correct and simplify the Dockerfile.

[gemini-code-assist]

The diarization endpoint is synchronous (def), but it calls a long-running, blocking function (diarization_endpoint). This will block the Uvicorn event loop, making the entire service unresponsive to any other requests, including health checks from Kubernetes. This can lead to Kubernetes killing the pod while it's processing a valid request. To fix this, the endpoint should be async and the blocking call should be wrapped in fastapi.concurrency.run_in_threadpool to run it in a separate thread.

You'll need to import run_in_threadpool from fastapi.concurrency.

Suggested change

	def diarization(file: UploadFile = File(...)):
	print('diarization')
	return diarization_endpoint(file)
	async def diarization(file: UploadFile = File(...)):
	print('diarization')
	return await run_in_threadpool(diarization_endpoint, file)

References

1. In an async application, avoid blocking the event loop with synchronous operations to prevent performance degradation and unresponsiveness. This principle applies to long-running blocking functions as well as synchronous HTTP clients.

[gemini-code-assist]

The liveness, readiness, and startup probes are currently using tcpSocket, which only checks if the port is open. This doesn't guarantee the application is healthy and ready to serve traffic. Since a /health endpoint is available, it's better to use an httpGet probe to get a more accurate health status of the application. This change should be applied to livenessProbe, readinessProbe, and startupProbe for a more reliable service.

livenessProbe:
  httpGet:
    path: /health
    port: 8080
  failureThreshold: 30
  periodSeconds: 10

[gemini-code-assist]

The liveness, readiness, and startup probes are currently using tcpSocket, which only checks if the port is open. This doesn't guarantee the application is healthy and ready to serve traffic. Since a /health endpoint is available, it's better to use an httpGet probe to get a more accurate health status of the application. This change should be applied to livenessProbe, readinessProbe, and startupProbe for a more reliable service.

livenessProbe:
  httpGet:
    path: /health
    port: 8080
  failureThreshold: 30
  periodSeconds: 10

[gemini-code-assist]

The BackendConfig resource name is hardcoded as diarizer-backend-config. This will cause naming conflicts if you try to install this Helm chart more than once in the same namespace. It's a best practice to template resource names using {{ include "diarizer.fullname" . }} to ensure they are unique per release. The annotation in the Service resource that refers to it should also be updated accordingly.

apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
  name: {{ include "diarizer.fullname" . }}-backend-config
spec:
  healthCheck:
    checkIntervalSec: 10
    timeoutSec: 5
    port: {{ .Values.service.port }}
    type: HTTP
    requestPath: /health  # Ensure this path returns 200 OK
---
apiVersion: v1
kind: Service
metadata:
  name: {{ include "diarizer.fullname" . }}
  labels:
    {{- include "diarizer.labels" . | nindent 4 }}
  annotations:
    cloud.google.com/neg: '{"ingress": true}'
    cloud.google.com/backend-config: '{"default": "{{ include "diarizer.fullname" . }}-backend-config"}'  # Link to BackendConfig
spec:
  type: {{ .Values.service.type }}
  ports:
    - port: {{ .Values.service.port }}
      targetPort: {{ .Values.service.port }}
      protocol: TCP
      name: http
  selector:
    {{- include "diarizer.selectorLabels" . | nindent 4 }}

[gemini-code-assist]

Manually installing the CUDA toolkit inside a generic Python image is complex and can be brittle. A better practice is to use an official NVIDIA base image that comes with CUDA pre-installed. This simplifies the Dockerfile, reduces the chance of errors, and often results in a more optimized image. Consider using a base image from nvidia/cuda on Docker Hub, such as nvidia/cuda:<version>-cudnn-runtime-ubuntu<version>, and then installing Python and your dependencies on top of it.

[gemini-code-assist]

There's a typo in the name of the .deb file being removed. It says debian11 but the downloaded file is for debian12. This causes the downloaded package file to not be cleaned up, which unnecessarily increases the final Docker image size.

rm -rf /var/lib/apt/lists/* cuda-repo-debian12-12-6-local_12.6.3-560.35.05-1_amd64.deb

[gemini-code-assist]

The code f.write(file.file.read()) reads the entire uploaded file into memory before writing it to disk. This can lead to high memory consumption and potential out-of-memory errors for large audio files. It's better to stream the file content directly to disk in chunks. You can use shutil.copyfileobj for this.

Don't forget to add import shutil at the top of the file.

Suggested change

	f.write(file.file.read())
	shutil.copyfileobj(file.file, f)

[beastoin]

lgtm @thainguyensunya