Add unit 5

units/en/_toctree.yml

@@ -54,6 +54,18 @@
         title: Mesh generation
       - local: unit4/hands-on
         title: Hands-on
+- title: Unit 5. Capstone
+  sections:
+      - local: unit5/introduction
+        title: Introduction
+      - local: unit5/walkthrough
+        title: Walkthrough
+      - local: unit5/run-in-notebook
+        title: Run in notebook
+      - local: unit5/run-locally
+        title: Run locally
+      - local: unit5/run-via-api
+        title: Run via API
 - title: Conclusion
   sections:
       - local: conclusion/conclusion

units/en/conclusion/conclusion.mdx

@@ -1,5 +1,14 @@
 # Conclusion
 
-This course is in-progress and has not yet concluded!
+Thank you for participating in the Machine Learning for 3D Course!
 
-To receive updates as the course releases, sign up for the course mailing list [here](https://mailchi.mp/911880bcff7d/ml-for-3d-course).
+This has been a high-level overview of what's going on at the intersection of Machine Learning and 3D. For further exploration:
+
+- Keep up with [3D Arena](https://huggingface.co/spaces/dylanebert/3d-arena) for the latest projects.
+- Explore open-source projects like [InstantMesh](https://github.com/TencentARC/InstantMesh).
+
+![3D Arena](https://huggingface.co/datasets/dylanebert/ml-for-3d-course/resolve/main/3d-arena.png)
+
+As the field evolves rapidly, it's easy to get overwhelmed. Stay tuned for more accessible tools and resources as these projects mature.
+
+To receive updates about future course releases, sign up for the course mailing list [here](https://mailchi.mp/911880bcff7d/ml-for-3d-course).

units/en/unit5/introduction.mdx

@@ -0,0 +1,45 @@
+# Introduction
+
+[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/dylanebert/ml-for-3d-course-notebooks/blob/main/capstone.ipynb)
+
+<iframe width="560" height="315" src="https://www.youtube.com/embed/l-KcMssIhI4?si=bAsM1wYWWenJjoAV" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
+
+Welcome to the final capstone project of the course.
+
+To complete the course and receive a certificate, you will create and host your own Generative 3D model and demo.
+
+## Requirements
+
+- **Model**: A model hosted under your username, e.g. `hf.co/{username}/{model_name}`.
+  - Ensure there is a [model card](https://huggingface.co/docs/hub/en/model-cards).
+  - Tag the model with the `image-to-3d` pipeline tag.
+  - Apply the correct license if applicable.
+
+- **Space**: A space hosted under your username, e.g. `hf.co/spaces/{username}/{space_name}`.
+  - Ensure the space converts images (`.png`, `.jpg`) to 3D (`.glb`, `.obj`, `.ply`, `.splat`).
+  - Reference your model URL in the space README.
+
+When completed, fill out this [form](https://forms.gle/rQedXFktHPYeikrt6) to receive your certificate.
+
+## What's expected
+
+This capstone project is very open-ended. You can:
+
+- Build something from scratch.
+- Train or fine-tune an existing open-source model.
+- Clone and existing open-source model and change the demo experience.
+- Directly clone an open-source model and demo.
+
+Check out [3D Arena](https://huggingface.co/spaces/dylanebert/3d-arena) for the latest image-to-3D demos to use as starting points.
+
+## Step-by-step
+
+If you aren't sure how to get started, the following sections will guide you through the process.
+
+Otherwise, you can start right away. When you're finished, fill out the [form](https://forms.gle/rQedXFktHPYeikrt6) to receive your certificate.
+
+## Join the community
+
+Want to ask questions? Share your work? Chat with the community? 
+
+[Join the Discord](https://hf.co/join/discord)!

units/en/unit5/run-in-notebook.mdx

@@ -0,0 +1,156 @@
+# Run in notebook
+
+[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/dylanebert/ml-for-3d-course-notebooks/blob/main/capstone.ipynb)
+
+The first open is to run in a Colab notebook. This is the easiest way to validate the code quickly.
+
+## Setup
+
+1. Click the "Open in Colab" button above.
+2. Change the runtime type to GPU.
+3. Scroll down to the `Run in this notebook section`.
+
+## Run the demo
+
+Start by installing dependencies.
+
+```bash
+!pip install -r https://huggingface.co/spaces/dylanebert/LGM-tiny/raw/main/requirements.txt
+```
+
+Then, run the demo code. This is exactly the same as in the space `app.py`. To ensure your model is working as expected, replace both instances of `dylanebert/LGM-full` with your `{username}/{model_name}`. Then, run the code.
+
+```python
+import shlex
+import subprocess
+
+import gradio as gr
+import numpy as np
+import spaces
+import torch
+from diffusers import DiffusionPipeline
+
+subprocess.run(
+    shlex.split(
+        "pip install https://huggingface.co/spaces/dylanebert/LGM-mini/resolve/main/wheel/diff_gaussian_rasterization-0.0.0-cp310-cp310-linux_x86_64.whl"
+    )
+)
+
+pipeline = DiffusionPipeline.from_pretrained(
+    "dylanebert/LGM-full",
+    custom_pipeline="dylanebert/LGM-full",
+    torch_dtype=torch.float16,
+    trust_remote_code=True,
+).to("cuda")
+
+
+@spaces.GPU
+def run(image):
+    input_image = np.array(image, dtype=np.float32) / 255.0
+    splat = pipeline(
+        "", input_image, guidance_scale=5, num_inference_steps=30, elevation=0
+    )
+    splat_file = "/tmp/output.ply"
+    pipeline.save_ply(splat, splat_file)
+    return splat_file
+
+
+demo = gr.Interface(
+    fn=run,
+    title="LGM Tiny",
+    description="An extremely simplified version of [LGM](https://huggingface.co/ashawkey/LGM). Intended as resource for the [ML for 3D Course](https://huggingface.co/learn/ml-for-3d-course/unit0/introduction).",
+    inputs="image",
+    outputs=gr.Model3D(),
+    examples=[
+        "https://huggingface.co/datasets/dylanebert/iso3d/resolve/main/jpg@512/a_cat_statue.jpg"
+    ],
+    cache_examples=True,
+    allow_duplication=True,
+)
+demo.queue().launch()
+```
+
+## Demo breakdown
+
+Let's break down the demo code.
+
+### Import dependencies
+
+Import the required libraries.
+
+```python
+import shlex
+import subprocess
+
+import gradio as gr
+import numpy as np
+import spaces
+import torch
+from diffusers import DiffusionPipeline
+```
+
+### Install diff-gaussian-rasterization
+
+For the gaussian splatting step of LGM, we need to install a custom wheel. This is a workaround for the space to run on [ZeroGPU](https://huggingface.co/zero-gpu-explorers).
+
+```python
+subprocess.run(
+    shlex.split(
+        "pip install https://huggingface.co/spaces/dylanebert/LGM-mini/resolve/main/wheel/diff_gaussian_rasterization-0.0.0-cp310-cp310-linux_x86_64.whl"
+    )
+)
+```
+
+### Construct the pipeline
+
+Construct the [LGM](https://huggingface.co/dylanebert/LGM-full) pipeline. Replace `dylanebert/LGM-full` with your `{username}/{model_name}`.
+
+```python
+pipeline = DiffusionPipeline.from_pretrained(
+    "dylanebert/LGM-full",
+    custom_pipeline="dylanebert/LGM-full",
+    torch_dtype=torch.float16,
+    trust_remote_code=True,
+).to("cuda")
+```
+
+### Define the run function
+
+Define the run function that takes an image and returns a ply file.
+
+1. Convert the image to a numpy array and normalize it to [0, 1].
+2. Run the pipeline with the default parameters.
+3. Save the ply file to `/tmp/output.ply`.
+4. Return the ply file.
+
+```python
+@spaces.GPU
+def run(image):
+    input_image = np.array(image, dtype=np.float32) / 255.0
+    splat = pipeline(
+        "", input_image, guidance_scale=5, num_inference_steps=30, elevation=0
+    )
+    splat_file = "/tmp/output.ply"
+    pipeline.save_ply(splat, splat_file)
+    return splat_file
+```
+
+### Create the demo
+
+Create the demo using [Gradio](https://www.gradio.app/guides/quickstart), which handles the UI for us.
+
+```python
+demo = gr.Interface(
+    fn=run,
+    title="LGM Tiny",
+    description="An extremely simplified version of [LGM](https://huggingface.co/ashawkey/LGM). Intended as resource for the [ML for 3D Course](https://huggingface.co/learn/ml-for-3d-course/unit0/introduction).",
+    inputs="image",
+    outputs=gr.Model3D(),
+    examples=[
+        "https://huggingface.co/datasets/dylanebert/iso3d/resolve/main/jpg@512/a_cat_statue.jpg"
+    ],
+    cache_examples=True,
+    allow_duplication=True,
+)
+demo.queue().launch()
+```

units/en/unit5/run-locally.mdx

@@ -0,0 +1,33 @@
+# Run locally
+
+[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/dylanebert/ml-for-3d-course-notebooks/blob/main/capstone.ipynb)
+
+The instructions below are tested on an RTX 4090 on WSL2 Ubuntu 22.04. Instructions will differ and may not work, depending on your setup.
+
+1. Install `git`, `python 3.10`, and `cuda` if not already installed.
+2. Open your terminal.
+3. Clone your space repository, replacing the URL below with your space URL.
+
+```bash
+git clone https://huggingface.co/spaces/dylanebert/LGM-tiny
+```
+
+4. Navigate into the space folder.
+
+```bash
+cd LGM-tiny
+```
+
+5. Create a virtual environment and install necessary dependencies.
+
+```bash
+python -m venv venv
+source venv/bin/activate
+pip install -r requirements.txt
+```
+
+6. Run the demo.
+
+```bash
+python app.py
+```

units/en/unit5/run-via-api.mdx

@@ -0,0 +1,35 @@
+# Run via API
+
+[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/dylanebert/ml-for-3d-course-notebooks/blob/main/capstone.ipynb)
+
+<iframe src="https://dylanebert-LGM-tiny-api.hf.space" frameborder="0" width="850" height="450"></iframe>
+
+To run via API, instead of duplicating the [LGM-tiny](https://huggingface.co/spaces/dylanebert/LGM-tiny) space, duplicate the [LGM-tiny-api](https://huggingface.co/spaces/dylanebert/LGM-tiny-api) space. This contains the following `app.py`.
+
+```python
+import gradio as gr
+from gradio_client import Client, file
+
+
+def run(image_url):
+    client = Client("dylanebert/LGM-tiny")
+    image = file(image_url)
+    result = client.predict(image, api_name="/predict")
+    return result
+
+
+demo = gr.Interface(
+    fn=run,
+    title="LGM Tiny API",
+    description="An API wrapper for [LGM Tiny](https://huggingface.co/spaces/dylanebert/LGM-tiny). Intended as a resource for the [ML for 3D Course](https://huggingface.co/learn/ml-for-3d-course).",
+    inputs=gr.Textbox(label="Image URL", placeholder="Enter image URL, e.g. https://huggingface.co/datasets/dylanebert/iso3d/resolve/main/jpg@512/a_cat_statue.jpg"),
+    outputs=gr.Model3D(),
+    examples=[
+        "https://huggingface.co/datasets/dylanebert/iso3d/resolve/main/jpg@512/a_cat_statue.jpg"
+    ],
+    allow_duplication=True,
+)
+demo.queue().launch()
+```
+
+This will work on CPU, but relies on the original LGM-tiny, instead of your custom model. However, is your focus is on UI/UX or downstream tasks, this may be acceptable.

units/en/unit5/walkthrough.mdx

@@ -0,0 +1,49 @@
+# Walkthrough
+
+[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/dylanebert/ml-for-3d-course-notebooks/blob/main/capstone.ipynb)
+
+This section will walk you step-by-step through the easiest way to set up a demo based on [LGM](https://huggingface.co/spaces/ashawkey/LGM), as covered in previous units.
+
+## 1. Duplicate the model
+
+Start by duplicating the [LGM-full](https://huggingface.co/dylanebert/LGM-full) model.
+
+1. Go to the [Repo duplicator](https://huggingface.co/spaces/huggingface-projects/repo_duplicator).
+2. Generate a `write token` in your [account settings](https://hf.co/settings/tokens). Copy it into the `token` field.
+3. Enter `dylanebert/LGM-full` in the `source_repo` field.
+4. Enter `{username}/{model_name}` in the `dst_repo` field, replacing `{username}` with your Hugging Face account username, and `{model_name}` with any name you like, such as `LGM`.
+5. Click Submit.
+
+You will see a link labeled "find your repo **here**", which leads to `http://huggingface.co/{username}/{model_name}`.
+
+![Repo duplicator](https://huggingface.co/datasets/dylanebert/ml-for-3d-course/resolve/main/duplicate.png)
+
+Congratulations! You can now use this model with Diffusers, and it will appear in [hf.co/models](https://huggingface.co/models?pipeline_tag=image-to-3d&sort=trending).
+
+## 2. Duplicate the space
+
+Go to the [LGM Tiny](https://huggingface.co/spaces/dylanebert/LGM-tiny) space, which provides a simplified image-to-3D demo.
+
+1. Click the `Duplicate Space` button.
+2. Choose free hardware, or ZeroGPU Nvidia A100 if available.
+3. Click `Files` in the top right.
+4. Click `app.py` to view the demo source code.
+5. Click `edit` to change the code.
+6. Replace the two instances of `dylanebert/LGM-full` with your model path, e.g. `{username}/{model_name}`.
+7. Click `Commit changes`.
+
+<iframe src="https://dylanebert-LGM-tiny.hf.space" frameborder="0" width="850" height="450"></iframe>
+
+Congratulations! You've created a demo and met the minimum requirements for this capstone project.
+
+## So how do I run it?
+
+The demo requires a GPU, so it won't work on free hardware. However, there are many free options:
+
+1. **Run in this notebook**: Validate the code quickly.
+2. **Run locally**: Clone your space and run it locally.
+3. **Community grant**: Building something cool? Apply for a community GPU grant in your space settings.
+4. **Run via API**: Less flexible, but runs on free hardware.
+5. **Just don't**: Alternatively, skip all this and go straight to the [form](https://forms.gle/rQedXFktHPYeikrt6).
+
+The following sections will walk you through each of these options.