Adding skeletonization notebooks to the repo. #59
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,305 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "id": "51813ca7", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Ultraliser Neuron Skeletonization Service Notebook\n", | ||
| "\n", | ||
| "Copyright (c) 2025 Open Brain Institute\n", | ||
| "\n", | ||
| "+ Author(s): \n", | ||
| " - Authors: Michael W. Reimann < michael.reimann@openbraininstitute.org >\n", | ||
| " - Marwan Abdellah < marwan.abdellah@openbraininstitute.org >\n", | ||
| "\n", | ||
| "Last modified: 10.2025\n" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "id": "e6687c74", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Imports and setting up platform authentication\n", | ||
| "\n", | ||
| "Please follow the displayed instructions to authenticate." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "id": "7cd5dc38", | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "# All essential imports needed for the notebook\n", | ||
| "import httpx\n", | ||
| "import os\n", | ||
| "import time \n", | ||
| "import json \n", | ||
| "import pathlib\n", | ||
| "from uuid import UUID\n", | ||
| "from urllib.parse import urlparse\n", | ||
| "from IPython.display import display, HTML\n", | ||
| "from obi_auth import get_token\n", | ||
| "from obi_notebook import get_projects, get_entities\n", | ||
| "from entitysdk.client import Client\n", | ||
| "from entitysdk.models import CellMorphology\n", | ||
| "from entitysdk.models.asset import AssetLabel\n", | ||
| "\n", | ||
| "# Ultraliser imports and version check\n", | ||
| "import ultraliser\n", | ||
| "ultraliser.copyrights()\n", | ||
| "ultraliser.version()\n", | ||
| "ultraliser.build()" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "id": "edf94fbf", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Selection of meshes and download\n", | ||
| "\n", | ||
| "The meshes will be downloaded, then loaded and processed by Ultraliser.\n", | ||
| "\n", | ||
| "### Project selection\n", | ||
| "As a first step we select one of the projects we have access to that the meshes are associated with. " | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "id": "67c0e91c", | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "environment = \"staging\"\n", | ||
| "token = get_token(environment=environment, auth_mode=\"daf\")\n", | ||
| "project_context = get_projects.get_projects(token, env=\"staging\")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "id": "1a98051d", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Set up clients\n", | ||
| "\n", | ||
| "Next, we select the meshes. A widget with all the meshes with their IDs will appear. Simply select the mesh you are interested to skeletonize and proceed. Note that a single mesh can only be selected." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "id": "5702b0a9", | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "# Initialize the client and search for EMCellMesh entities\n", | ||
| "client = Client(environment=environment, token_manager=token, project_context=project_context)\n", | ||
| "entitycore_api_url = urlparse(client.api_url)\n", | ||
| "platform_base_url = f\"{entitycore_api_url.scheme}://{entitycore_api_url.netloc}\"\n", | ||
| "mesh_api_base_url = f\"{platform_base_url}/api/small-scale-simulator/mesh/skeletonization\"\n", | ||
| "\n", | ||
| "http_client = httpx.Client()\n", | ||
| "\n", | ||
| "mesh_api_headers = httpx.Headers({\n", | ||
| " \"Authorization\": f\"Bearer {token}\",\n", | ||
| " \"virtual-lab-id\": str(project_context.virtual_lab_id),\n", | ||
| " \"project-id\": str(project_context.project_id)\n", | ||
| "})" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "id": "ed65fc07", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Display EMCellMesh table" | ||
|
There was a problem hiding this comment. Please add some instructions here. Explain what is being displayed and what a user is expected to do. And how they can understand what each mesh means. |
||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "id": "9b833a25", | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "mesh_ids = []\n", | ||
| "mesh_ids = get_entities.get_entities(\n", | ||
| " 'em-cell-mesh', token, [], env=environment, project_context=project_context, \n", | ||
| " multi_select=False, page_size=20)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "id": "f1e61ca1", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Check the selection " | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "id": "b8872782", | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "if len(mesh_ids) == 0:\n", | ||
| " display(HTML(\"<h3 style='color:#EDB95E'>⚠️ No mesh selected</h3>\"))\n", | ||
| " raise SystemExit()\n", | ||
| "mesh_id = mesh_ids[0]" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "id": "a58a8716", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Prepare task params" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "id": "160775d6", | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "input_params = {\n", | ||
| " \"name\": mesh_id,\n", | ||
| " \"description\": \"Reconstructed morphology from an EM surface mesh\"\n", | ||
| "}\n", | ||
| "\n", | ||
| "skeletonization_params = {\n", | ||
|
There was a problem hiding this comment. For the skeletonization params: Also, with some callback functions the behavior can be such that the spine resolution is always better than the other resolution. |
||
| " \"em_cell_mesh_id\": mesh_id,\n", | ||
| " \"neuron_voxel_size\": 0.1, # in microns\n", | ||
| " \"spines_voxel_size\": 0.05, # in microns\n", | ||
| " \"segment_spines\": True\n", | ||
| "}" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "id": "820fbf5a", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Submit mesh skeletonization task" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "id": "b3b4348f", | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "start_res = http_client.post(\n", | ||
| " f\"{mesh_api_base_url}/run\",\n", | ||
| " params=skeletonization_params,\n", | ||
| " headers=mesh_api_headers,\n", | ||
| " json=input_params\n", | ||
| ")\n", | ||
| "\n", | ||
| "job_id = None\n", | ||
| "\n", | ||
| "if start_res.is_success:\n", | ||
| " job_id = start_res.json().get(\"id\")\n", | ||
| "else:\n", | ||
| " print(start_res.text)\n", | ||
| " raise RuntimeError(\"Failed to submit mesh skeletonization task\")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "id": "9fbfb97d", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Wait for the job to complete" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "id": "574b35f2", | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "output_morphology_id = None\n", | ||
| "prev_status = None\n", | ||
| "\n", | ||
| "while True:\n", | ||
| " status_res = http_client.get(\n", | ||
| " f\"{mesh_api_base_url}/jobs/{job_id}\",\n", | ||
| " headers=mesh_api_headers\n", | ||
| " )\n", | ||
| "\n", | ||
| " if not status_res.is_success:\n", | ||
| " print(status_res.text)\n", | ||
| " raise RuntimeError(\"Failed to get job status\")\n", | ||
| "\n", | ||
| " job = status_res.json()\n", | ||
| " status = job.get('status')\n", | ||
| "\n", | ||
| " if status != prev_status:\n", | ||
| " print(f\"{time.strftime(\"%H:%M:%S\", time.localtime())} Status: {status}\")\n", | ||
| " prev_status = status\n", | ||
| "\n", | ||
| " if status == 'finished':\n", | ||
| " output_morphology_id = UUID(job.get('output').get('morphology').get('id'))\n", | ||
| " break\n", | ||
| " elif status == 'failed':\n", | ||
| " print(json.dumps(job, indent=2))\n", | ||
|
There was a problem hiding this comment. This can be formatted better than json.dumps |
||
| " raise RuntimeError(\"Skeletonization failed\")\n", | ||
| "\n", | ||
| " time.sleep(15)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "id": "eabd5eaf", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Get the resulting registered morphology " | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "id": "5034ac2f", | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "cell_morphology = client.get_entity(output_morphology_id, entity_type=CellMorphology)\n", | ||
| "asset = next((asset for asset in cell_morphology.assets if asset.label == AssetLabel.morphology_with_spines), None)\n", | ||
| "\n", | ||
| "# Download the file\n", | ||
| "client.download_assets(cell_morphology,selection={\"label\": AssetLabel.morphology_with_spines}, output_path=pathlib.Path(os.getcwd())).one()" | ||
| ] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": "Python 3", | ||
| "language": "python", | ||
| "name": "python3" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "name": "ipython", | ||
| "version": 3 | ||
| }, | ||
| "file_extension": ".py", | ||
| "mimetype": "text/x-python", | ||
| "name": "python", | ||
| "nbconvert_exporter": "python", | ||
| "pygments_lexer": "ipython3", | ||
| "version": "3.12.3" | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 5 | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use
env=environmentinstead ofenv="staging"