Skip to content
This repository was archived by the owner on Dec 8, 2023. It is now read-only.

Updates for codebook deployment #78

Merged
merged 12 commits into from
Sep 19, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .github/workflows/u24_workflow_before_release.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: u24_workflow_before_release
name: u24_workflow_before_release_0.0.1
on:
pull_request:
push:
Expand All @@ -16,4 +16,3 @@ jobs:
jhub_ver: 1.4.2
py_ver: 3.9
dist: debian
codebook_base_hash: 561b765
5 changes: 2 additions & 3 deletions .github/workflows/u24_workflow_release_call.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name: u24_workflow_release_call
name: u24_workflow_release_call_0.0.1
on:
workflow_run:
workflows: ["u24_workflow_tag_to_release"]
workflows: ["u24_workflow_tag_to_release_0.0.1"]
types:
- completed
jobs:
Expand All @@ -15,7 +15,6 @@ jobs:
jhub_ver: 1.4.2
py_ver: 3.9
dist: debian
codebook_base_hash: 561b765
secrets:
REGISTRY_USERNAME: ${{secrets.DOCKER_USERNAME}}
REGISTRY_PASSWORD: ${{secrets.DOCKER_PASSWORD}}
3 changes: 1 addition & 2 deletions .github/workflows/u24_workflow_tag_to_release.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: u24_workflow_tag_to_release
name: u24_workflow_tag_to_release_0.0.1
on:
push:
tags:
Expand All @@ -13,4 +13,3 @@ jobs:
jhub_ver: 1.4.2
py_ver: 3.9
dist: debian
codebook_base_hash: 561b765
7 changes: 6 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,11 @@
Observes [Semantic Versioning](https://semver.org/spec/v2.0.0.html) standard and
[Keep a Changelog](https://keepachangelog.com/en/1.0.0/) convention.

## [0.2.2] - 2022-09-18
+ Update - config ephys roots in notebooks for codebook deployment
+ Update - requirements.txt to add element electrode localization
+ Update - add lab.User insert to ingest.py for users with empty element-lab schemas

## [0.2.1] - 2022-07-22
+ Add - Mention CodeBook data directory in notebooks
+ Add - Mention of config items in notebook 09
Expand Down Expand Up @@ -55,4 +60,4 @@ specific forks for tests.

[0.2.1]: https://github.com/datajoint/workflow-array-ephys/compare/0.2.1...0.2.0
[0.2.0]: https://github.com/datajoint/workflow-array-ephys/compare/0.2.0...0.1.0
[0.1.0]: https://github.com/datajoint/workflow-array-ephys/releases/tag/0.1.0
[0.1.0]: https://github.com/datajoint/workflow-array-ephys/releases/tag/0.1.0
74 changes: 23 additions & 51 deletions notebooks/01-configure.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -11,30 +11,19 @@
"\n",
"## Setup - Working Directory\n",
"\n",
"To run the array ephys workflow, we need to properly set up the DataJoint configuration. The configuration will be saved in a file called `dj_local_conf.json` on each machine and this notebook walks you through the process.\n",
"To run the array ephys workflow, we need to properly set up the DataJoint configuration. The configuration can be saved in a local directory as `dj_local_conf.json` or at your root directory as a hidden file. This notebook walks you through the setup process.\n",
"\n",
"**The configuration only needs to be set up once**, if you have gone through the configuration before, directly go to [02-workflow-structure](02-workflow-structure-optional.ipynb).\n",
"\n",
"As a convention, we set the configuration up in the root directory of the workflow package and always starts importing datajoint and pipeline modules from there."
"**The configuration only needs to be set up once**, if you have gone through the configuration before, directly go to [02-workflow-structure](02-workflow-structure-optional.ipynb)."
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 1,
"id": "dc2dc935",
"metadata": {},
"outputs": [],
"source": [
"import os"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# change to the upper level folder\n",
"import os\n",
"if os.path.basename(os.getcwd()) == \"notebooks\": os.chdir(\"..\")\n",
"import datajoint as dj"
]
Expand Down Expand Up @@ -119,7 +108,7 @@
"The root path typically **do not** contain information of subjects or sessions, all data from subjects/sessions should be subdirectories in the root path.\n",
"\n",
"- In the example dataset downloaded with [these instructions](00-data-download-optional.ipynb), `/tmp/test_data` will be the root. \n",
"- For CodeBook users, the root is `/home/inbox/`\n",
"- For CodeBook users, the root is `/home/inbox/0.1.0a4/`\n",
"\n",
"```\n",
"- subject6\n",
Expand All @@ -139,7 +128,14 @@
"# If there is only one root path.\n",
"dj.config[\"custom\"][\"ephys_root_data_dir\"] = \"/tmp/test_data\"\n",
"# If there are multiple possible root paths:\n",
"dj.config[\"custom\"][\"ephys_root_data_dir\"] = [\"/tmp/test_data\", \"/home/inbox/\"]"
"dj.config[\"custom\"][\"ephys_root_data_dir\"] = [\n",
" \"/tmp/test_data/workflow_ephys_data1/\",\n",
" \"/tmp/test_data/workflow_ephys_data2/\",\n",
" \"/tmp/test_data/workflow_localization/\", \n",
" \"/home/inbox/0.1.0a4/workflow_ephys_data1/\",\n",
" \"/home/inbox/0.1.0a4/workflow_ephys_data2/\",\n",
" \"/home/inbox/0.1.0a4/workflow_localization/\"\n",
"]"
]
},
{
Expand Down Expand Up @@ -188,46 +184,22 @@
"source": [
"## Save configuration\n",
"\n",
"With the proper configurations, we could save this as a file, either as a local json file, or a global file."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dcc8b17b",
"metadata": {},
"outputs": [],
"source": [
"dj.config.save_local()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9b95596f",
"metadata": {},
"outputs": [],
"source": [
"ls"
]
},
{
"cell_type": "markdown",
"id": "d76904b4",
"metadata": {},
"source": [
"Local configuration file is saved as `dj_local_conf.json` in the root directory of this package `workflow-array-ephys`. Next time if you change your directory to `workflow-array-ephys` before importing datajoint and the pipeline packages, the configurations will get properly loaded.\n",
"With the proper configurations, we could save this as a file, either as a local json file, or a global file.\n",
"\n",
"If saved globally, there will be a hidden configuration file saved in your root directory. The configuration will be loaded no matter where the directory is."
"Local configuration file is saved as `dj_local_conf.json` in current directory, which is great for project-specific settings.\n",
"\n",
"For first-time and CodeBook users, we recommend saving globally. This will create a hidden configuration file saved in your root directory, loaded whenever there is no local version to override it."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dcc8b17b",
"metadata": {},
"outputs": [],
"source": [
"# dj.config.save_global()"
"# dj.config.save_local()\n",
"dj.config.save_global()"
]
},
{
Expand All @@ -250,7 +222,7 @@
"formats": "ipynb,py_scripts//py"
},
"kernelspec": {
"display_name": "Python 3.9.12 ('ele')",
"display_name": "Python 3.9.13 ('ele')",
"language": "python",
"name": "python3"
},
Expand All @@ -264,11 +236,11 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.12"
"version": "3.9.13"
},
"vscode": {
"interpreter": {
"hash": "61456c693db5d9aa6731701ec9a9b08ab88a172bee0780139a3679beb166da16"
"hash": "d00c4ad21a7027bf1726d6ae3a9a6ef39c8838928eca5a3d5f51f3eb68720410"
}
}
},
Expand Down
194 changes: 145 additions & 49 deletions notebooks/02-workflow-structure-optional.ipynb

Large diffs are not rendered by default.

Loading