What we want the API to do

[ajkiessl]

Summary

Build an API that allows clients to upload a file, store it in our S3 input bucket, process it via PDF_Accessibility (https://github.com/psu-libraries/PDF_Accessibility), and provide a presigned URL for downloading the processed file from our output bucket.

Acceptance Criteria

• The API accepts a presigned URL, generates a unique job_id, and returns the job_id to the client
• The API asynchronously fetches the file from the presigned URL
• The API uploads the file to our S3 input bucket
• The service monitors the status of the pdf remediation process
• The client can poll the API for job_id status: processing, completed, failed
• When status is completed, the client can request a presigned URL to download the processed file
• Presigned download URLs expire after a configurable time (default: 1 hour)

[AndrewGearhart]

🧾 PDF Accessibility Remediation API – MVP Implementation

🎯 Objective

Design and implement an internal API for trusted Penn State systems (e.g., ETDA, Scholarsphere, Admissions) to submit PDF files for accessibility remediation. The API will manage the lifecycle of each remediation request, invoke the PDF_Accessibility tool, and return remediated files via a presigned URL. The system uses a unified endpoint for job status and result retrieval.

---

🧩 Architecture Overview

🅰️ Source System Responsibilities

• Identify PDFs that need remediation
• Submit jobs via POST /jobs with a presigned input URL
• Optionally include metadata (e.g., submitted_by)
• Poll GET /jobs/{id} for job status and final result
• Log remediation outcomes locally for auditing

---

🅱️ API Responsibilities

🔧 Core Endpoints

Method	Endpoint	Description
POST	/jobs	Submit a new remediation job
GET	/jobs/{id}	Unified endpoint to retrieve job status and result

🧠 Processing Pipeline

• Validate and store the incoming request
• Asynchronously:
  • Download the input PDF from source_url
  • Upload to S3 input bucket (e.g., jobs/<job_id>/input.pdf)
  • Process the file using PDF_Accessibility tool (tags, OCR, metadata)
  • Upload the remediated file to output bucket (e.g., jobs/<job_id>/output.pdf)
  • Generate presigned download_url
  • Update job metadata: status, timestamps, file hashes, and result info

---

🔐 Security

• Require a pre-shared bearer token for all endpoints
  Authorization: Bearer <token>
• Configure the token via environment variable
• Enforce HTTPS-only access
• Use presigned URLs for file input/output with expiration (default: 1 hour)
• Avoid exposing raw storage credentials or internal file paths

---

🛠️ Task Breakdown

🔐 Authentication

• Add middleware to enforce bearer token authentication
• Token must be configurable via .env
• Return 401 Unauthorized on missing or invalid token

📥 Job Submission

• Implement POST /jobs
• Accept payload:

  {
    "source_url": "https://...",
    "submitted_by": "etda"
  }

• Generate unique job_id
• Store initial job metadata with status: "queued"
• Enqueue job for background processing
• Return:

{ "job_id": "abc-123" }

⚙️ Background Job Worker

• Download PDF from source_url
• Upload to input bucket at jobs/<job_id>/input.pdf
• Run PDF_Accessibility remediation tools
• Upload result to output bucket at jobs/<job_id>/output.pdf
• Generate presigned download_url with expiration
• Update job metadata: status, result, timestamps
• Record errors and update status: failed if needed

📊 Unified Status + Result Endpoint

• Implement GET /jobs/{job_id}
• Return payload when job is completed:\

  {
    "job_id": "abc-123",
    "status": "completed",
    "submitted_by": "etda",
    "source_url": "https://...",
    "created_at": "2025-06-18T12:00:00Z",
    "updated_at": "2025-06-18T12:10:00Z",
    "error": null,
    "result": {
      "download_url": "[https://s3.../output.pdf](https://s3.../output.pdf)?...",
      "expires_at": "2025-06-18T13:10:00Z"
    }
  }

• if job is incomplete:

  {
    "job_id": "abc-123",
    "status": "processing",
    "error": null,
    "result": null
  }

🧾 Logging & Metadata

• Log all job status changes and timestamps
• Store file hashes for input/output (e.g., SHA256)
• Track processing duration
• Include structured error logging

---

📁 Deliverables

• Dockerized API service
• docker-compose.yml for local development
• .env support for configuration (token, buckets, expiration, etc.)
• Swagger/OpenAPI specification
• Setup documentation (README)
• Sample client usage (e.g., curl or Python script)

---

🔮 Future Enhancements (Not MVP)

• Webhook support for job completion
• Job cancellation and retry endpoints
• AI-generated alt text
• Accessibility scoring/reporting
• Admin dashboard
• Batch submissions
• OAuth (e.g., Entra) client authentication
• Support for other file types (Word, PowerPoint, etc.)

---

📘 Notes on Presigned URLs

A presigned URL is a secure, temporary link that allows upload or download access to a private file in object storage (e.g., S3, MinIO) without needing credentials.

Usage:

• source_url: presigned GET URL to retrieve original file
• download_url: presigned GET URL returned on job completion

Benefits:

• Avoids exposing file storage credentials
• Keeps files secure and ephemeral
• Simplifies client interactions with object storage

[AndrewGearhart]

Another aspect that should probably be built in is a mechanism to administer the tokens that would be used for the source applications.

[ajkiessl]

Per standup: We've decided to implement webhook support for MVP. So clients won't have to poll the API.

[ajkiessl]

MVP Checklist

• Project Initialization #2
• Base API Controller #3
• Add Job model #4
• POST /jobs endpoint #5
• Background remediation process #6
• S3 handler #7
• Webhook client #8
• Dockerize #9
• Swagger docs #10
• Continuous Integration #11
• Continuous Delivery #12