Add AWS S3 and MQTT setup documentation for A1 Mini camera with automated setup tools, validation, and authentication

scripts/run_capture_orchestrator.py

@@ -0,0 +1,36 @@
+"""Async orchestrator that triggers a deployment and returns its result.
+
+This uses `run_deployment` (async) to create and execute the flow run, then
+reads the flow run via the Prefect client and calls `aresult()` on the final
+state to retrieve the flow return value.
+
+Usage:
+    python scripts/run_capture_orchestrator.py [deployment-name]
+
+Defaults to `capture-image/capture-image`.
+"""
+
+import asyncio
+import sys
+from prefect.deployments import run_deployment
+from prefect.client.orchestration import get_client
+
+# WARNING: These have not been working, just simply trying to get the returned result without making an S3 bucket just for passing a string around. See src/ac_training_lab/a1_cam/_scripts/prefect/copilot-logs.md
+
+
+async def run_and_get(deployment_name: str = "capture-image/capture-image") -> object:
+    # Trigger the deployment and wait (the function awaits completion by default)
+    flow_run = await run_deployment(name=deployment_name)
+
+    # Read the flow run and get its final state/result
+    async with get_client() as client:
+        fr = await client.read_flow_run(flow_run.id)
+        state = fr.state
+        result = await state.aresult(raise_on_failure=True)
+        print("Flow result:", result)
+        return result
+
+
+if __name__ == "__main__":
+    dep = sys.argv[1] if len(sys.argv) > 1 else "capture-image/capture-image"
+    asyncio.run(run_and_get(dep))

src/ac_training_lab/a1_cam/README.md

@@ -6,6 +6,246 @@ Module 3 running RPi OS Lite (bookworm).
 ```{include} ../../docs/_snippets/network-setup-note.md
 ```
 
+## Architecture Overview
+
+The A1 Mini camera system uses a hybrid approach for image transfer:
+
+1. **MQTT** - Used to send capture commands to the device and receive image URIs back
+2. **AWS S3** - Used to store the actual image files (avoids MQTT payload size limitations)
+
+This architecture follows the recommendations from [ac-microcourses data logging tutorial](https://ac-microcourses.readthedocs.io/en/latest/courses/hello-world/1.5-data-logging.html#additional-resources) which suggests uploading files to cloud storage and storing URIs in your database rather than embedding large binary data.
+
+## AWS S3 Setup
+
+### 1. Create an AWS Account
+
+If you don't have an AWS account, create one at [aws.amazon.com](https://aws.amazon.com/). AWS offers a free tier that includes 5 GB of S3 storage for 12 months.
+
+### 2. Create an S3 Bucket
+
+Follow the official AWS documentation to create an S3 bucket:
+- [Creating a bucket (AWS Console)](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html)
+- [Tutorial: Creating your first S3 bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/creating-bucket.html)
+
+**Recommended bucket configuration settings** (based on [Issue #159](https://github.com/AccelerationConsortium/ac-dev-lab/issues/159#issuecomment-2725422824)):
+
+![S3 Bucket Settings](https://github.com/user-attachments/assets/f155f371-9c1c-4702-9530-89615026da80)
+
+Key considerations:
+- **Region**: Choose a region close to your devices for lower latency (e.g., `us-east-2`)
+- **Bucket name**: Must be globally unique (e.g., `rpi-zero2w-toolhead-camera`)
+- **Object Ownership**: ACLs disabled (recommended)
+- **Block Public Access settings**: 
+  - For enhanced security, keep "Block all public access" enabled (recommended)
+  - If you need to access images from external systems, use IAM-based access controls or generate signed URLs rather than making the bucket public
+  - Only uncheck public access if you fully understand the security implications and need publicly accessible URLs (as shown in the screenshot)
+
+![S3 Public Access Settings](https://github.com/user-attachments/assets/fb694a7f-4dc0-4baf-a603-01bfa74d3165)
+
+- **Bucket Versioning**: Can be left disabled by default. Enable if you want to keep multiple versions of files (less applicable when uploading timestamped images as this camera does)
+- **Default encryption**: Enable Server-side encryption with Amazon S3 managed keys (SSE-S3)
+
+**Optional: Enable Public Read Access**
+
+> **⚠️ Security Warning**: Enabling public read access means anyone with knowledge of your object URLs can download images from your bucket. This may lead to unintended data exposure and unexpected AWS data transfer charges. Only enable this if you specifically need publicly accessible URLs and understand these implications.
+
+If you unchecked "Block all public access" and want to allow public read access to images (useful for accessing images directly via URL without authentication), add the following bucket policy:
+
+1. Go to your bucket → **Permissions** tab → **Bucket policy**
+2. Click **Edit** and paste the following policy (replace `your-bucket-name` with your actual bucket name):
+
+```json
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Sid": "PublicRead",
+            "Effect": "Allow",
+            "Principal": "*",
+            "Action": "s3:GetObject",
+            "Resource": "arn:aws:s3:::your-bucket-name/*"
+        }
+    ]
+}
+```
+
+3. Click **Save changes**
+
+This policy allows anyone to read (download) objects from your bucket via their public URLs. The IAM user credentials are still required for uploading and deleting objects. **Note**: Anyone who knows or guesses your object URLs can access them without authentication.
+
+### 3. Create IAM Credentials
+
+Create AWS IAM credentials with S3 access permissions. Follow the official guide:
+- [Creating an IAM user in your AWS account](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html)
+- [Managing access keys for IAM users](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html)
+
+**Detailed step-by-step instructions**:
+
+1. **Navigate to IAM**:
+   - In the AWS Console, search for "IAM" in the top search bar or go to the Services menu → Security, Identity, & Compliance → IAM
+
+2. **Create a new user**:
+   - In the left sidebar, click **Users**
+   - Click the **Create user** button (orange button in top right)
+   - Enter a user name (e.g., `a1-cam-user`)
+   - Click **Next**
+
+3. **Set permissions**:
+   - Select **Attach policies directly**
+   - **Do not** select any AWS managed policies (we'll add a custom policy next)
+   - Click **Next**
+   - Review and click **Create user**
+
+4. **Add custom inline policy**:
+   - After creating the user, click on the user name to open the user details
+   - Click on the **Add permissions** dropdown → **Create inline policy**
+   - Click on the **JSON** tab
+   - Replace the default policy with the following (based on [Issue #159](https://github.com/AccelerationConsortium/ac-dev-lab/issues/159#issuecomment-2725490350)):
+
+```json
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Sid": "ListObjectsInBucket",
+            "Effect": "Allow",
+            "Action": [
+                "s3:ListBucket"
+            ],
+            "Resource": [
+                "arn:aws:s3:::your-bucket-name"
+            ]
+        },
+        {
+            "Sid": "AllObjectActions",
+            "Effect": "Allow",
+            "Action": [
+                "s3:GetObject",
+                "s3:PutObject",
+                "s3:DeleteObject"
+            ],
+            "Resource": [
+                "arn:aws:s3:::your-bucket-name/*"
+            ]
+        }
+    ]
+}
+```
+
+   - Replace `your-bucket-name` with your actual bucket name (e.g., `rpi-zero2w-toolhead-camera`)
+   - Click **Next**
+   - Enter a policy name (e.g., `a1-cam-s3-access`)
+   - Click **Create policy**
+
+5. **Create access keys**:
+   - Still on the user details page, click the **Security credentials** tab
+   - Scroll down to **Access keys** section
+   - Click **Create access key**
+   - Select **Application running outside AWS** as the use case
+   - Click **Next**
+   - (Optional) Add a description tag (e.g., "A1 Mini Camera Raspberry Pi")
+   - Click **Create access key**
+   - **IMPORTANT**: You'll see your `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` displayed
+   - Click **Download .csv file** or copy both values immediately - they will only be shown once!
+   - Save these credentials securely in a password manager
+   - Click **Done**
+
+**Security best practices**: 
+- Revoke credentials immediately if compromised
+- Never commit credentials to version control
+
+The a1_cam device generates URLs like:
+```
+https://{BUCKET_NAME}.s3.{AWS_REGION}.amazonaws.com/{object_name}
+```
+
+If you need to configure additional bucket policies, see:
+- [Using bucket policies](https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-policies.html)
+- [Bucket policy examples](https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-bucket-policies.html)
+
+## boto3 Setup
+
+The device uses [boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html), the AWS SDK for Python, to upload images to S3.
+
+### Installation
+
+boto3 is included in the device `requirements.txt` and will be installed when you follow the dependency installation instructions below.
+
+### Configuration
+
+The device code explicitly passes AWS credentials to boto3 from the `my_secrets.py` file. This approach keeps all credentials in one place and avoids the need to configure `~/.aws/credentials` on the Raspberry Pi.
+
+Add the following to your `my_secrets.py` file (see Secrets section below for creating this file):
+
+```python
+AWS_ACCESS_KEY_ID = "your-aws-access-key-id"
+AWS_SECRET_ACCESS_KEY = "your-aws-secret-access-key"
+AWS_REGION = "us-east-2"  # or your chosen region
+BUCKET_NAME = "rpi-zero2w-toolhead-camera"  # or your bucket name
+IMAGE_QUALITY = 85  # JPEG quality (1-100). Lower = smaller file size. 85 gives ~2-3 MB images
+```
+
+**Image quality settings**:
+- `IMAGE_QUALITY` controls JPEG compression (1-100 scale)
+- Default of 85 produces ~2-3 MB images (down from ~35 MB at quality 90)
+- Lower values reduce file size but may decrease image clarity
+- Recommended range: 75-90 depending on your needs
+
+The device.py code passes these credentials directly to boto3.client():
+```python
+s3 = boto3.client(
+    "s3",
+    aws_access_key_id=AWS_ACCESS_KEY_ID,
+    aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
+    region_name=AWS_REGION,
+)
+```
+
+**Security considerations**:
+- The `my_secrets.py` file stores credentials in plaintext. Ensure proper file permissions: `chmod 600 my_secrets.py`
+- Keep your Raspberry Pi login credentials secure and use SSH key authentication
+- Consider restricting SSH access and using fail2ban or similar tools
+- While boto3 also supports reading credentials from `~/.aws/credentials` or environment variables, this implementation explicitly passes them to keep all device secrets centralized in `my_secrets.py`
+
+### Additional Resources
+
+- [boto3 Documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html)
+- [boto3 S3 Guide](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3.html)
+- [boto3 S3 Examples](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3-examples.html)
+
+## MQTT Setup
+
+The A1 Mini camera uses MQTT for hardware-software communication. For comprehensive MQTT setup instructions, refer to the AC Microcourses documentation:
+
+- [Hardware-Software Communication](https://ac-microcourses.readthedocs.io/en/latest/courses/hello-world/1.4-hardware-software-communication.html)
+- [Onboard LED & Temperature Tutorial](https://ac-microcourses.readthedocs.io/en/latest/courses/hello-world/1.4.1-onboard-led-temp.html)
+
+### MQTT Credentials
+
+The device requires MQTT connection details in the `my_secrets.py` file (see Secrets section below for setup):
+
+- `MQTT_HOST` - Your MQTT broker host (e.g., HiveMQ Cloud)
+- `MQTT_PORT` - Usually 8883 for TLS-encrypted connections
+- `MQTT_USERNAME` - Your MQTT username
+- `MQTT_PASSWORD` - Your MQTT password
+- `DEVICE_SERIAL` - A unique identifier for this camera device
+- `CAMERA_READ_TOPIC` - Topic for receiving capture commands (e.g., `bambu_a1_mini/request/{DEVICE_SERIAL}`)
+- `CAMERA_WRITE_TOPIC` - Topic for publishing image URIs (e.g., `bambu_a1_mini/response/{DEVICE_SERIAL}`)
+
+### MQTT Library
+
+The device uses [paho-mqtt](https://pypi.org/project/paho-mqtt/), the standard Python MQTT client library. It's included in the device `requirements.txt`.
+
+## Quick Setup (Automated)
+
+For a streamlined setup process, use the provided `setup.sh` script:
+
+```bash
+bash setup.sh
+```
+
+This script automates all the steps described in the sections below. **Note**: If this script and README diverge, README is authoritative. See the [Codebase](#codebase), [Secrets](#secrets), and [Dependencies](#dependencies) sections for manual step-by-step instructions.
+
 ## Codebase
 
 Optionally, update the system packages to the latest versions (`-y` flag is used to automatically answer "yes" to any installation prompts):
@@ -83,7 +323,37 @@ To start the device manually and ensure that it's functioning normally, run:
 python3 device.py
 ```
 
-To verify quickly that this script works, you can run `_scripts/client.py` locally (e.g., on your PC), ensuring that you have the same credentials in a `my_secrets.py` located in the `_scripts` directory as you do on the RPi. This script will request the latest image from the device and save it to your local machine.
+### Testing the Camera
+
+To verify the camera is working, you have two options:
+
+**Option 1: Jupyter Notebook (Recommended)**
+
+Use the provided `test_camera.ipynb` notebook to interactively test the camera:
+
+[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/AccelerationConsortium/ac-training-lab/blob/main/src/ac_training_lab/a1_cam/test_camera.ipynb)
+
+The notebook allows you to:
+- Configure your MQTT credentials
+- Send a capture command
+- Download and display the captured image
+
+**Option 2: Python Script**
+
+Run `_scripts/client.py` locally (e.g., on your PC), ensuring you have the same credentials in a `my_secrets.py` located in the `_scripts` directory as you do on the RPi. This script will request the latest image from the device and save it to your local machine.
+
+## Workflow Example
+
+Here's how the complete image capture workflow operates:
+
+1. **Setup**: Configure AWS S3 bucket and credentials, set up MQTT broker
+2. **Image capture request**: Orchestrator sends `{"command": "capture_image"}` via MQTT to `CAMERA_READ_TOPIC`
+3. **Device captures**: Raspberry Pi takes a photo using the camera
+4. **Upload to S3**: Device uploads image to S3 bucket using boto3
+5. **Respond with URI**: Device publishes S3 URI back to orchestrator via MQTT on `CAMERA_WRITE_TOPIC`
+6. **Access image**: Orchestrator can download image from S3 or store URI in database
+
+For implementation details, see [Issue #159](https://github.com/AccelerationConsortium/ac-dev-lab/issues/159).
 
 ## Automatic startup