diff --git a/docs/api-tools/matlab-api-tool.md b/docs/api-tools/matlab-api-tool.md index 9651b2c..567c31d 100644 --- a/docs/api-tools/matlab-api-tool.md +++ b/docs/api-tools/matlab-api-tool.md @@ -1,79 +1,162 @@ --- layout: default -title: Matlab API tool +title: MATLAB API tool parent: API tools nav_order: 1 --- -# Matlab API tool +# MATLAB API tool {: .no_toc} -You can download the BrainSTEM Matlab API tools repository from GitHub at [github.com/brainstem-org/brainstem_matlab_api_tools](https://github.com/brainstem-org/brainstem_matlab_api_tools) and add it to your Matlab setpath. +## Table of contents +{: .no_toc .text-delta } -Please see the dedicated [tutorial]({{"/tutorials/matlab-api-tool/"|absolute_url}}) with examples on usage. The main functions are described below. +1. TOC +{:toc} -| Function | Description | -|:-------------|:-------------| -| `get_token` | Get and save authentication token | -| `load_model` | Load data from any model | -| `save_model` | Save data to any model | -| `load_settings` | Local settings: API token, url to the server, and local storage. | -| `load_project` | Load project(s). Convenience function for handling projects. Extra parameters: `id`,`name`,`description`,`sessions`,`subjects`,`tags`,`is_public`. Included relational data: `sessions`,`subjects`,`collections`. | -| `load_subject` | Load subject(s). Convenience function for handling subjects. Extra parameters: `id`,`name`,`description`,`projects`,`strain`,`sex`,`tags`. Included relational data: `procedures`,`subjectlogs`. | -| `load_session` | Load session(s). Convenience function for handling sessions. Extra parameters: `id`,`name`,`description`,`projects`,`datastorage`,`tags`. Included relational data: `dataacquisition`,`behaviors`,`manipulations`,`epochs`. | -| `brainstem_api_tutorial` | Tutorial script. | +## Overview -### Filters -You can use filters, using fields and relationships by providing cell array with paired filters. Below example will just load the session with the id: +The BrainSTEM MATLAB API tool (`brainstem_matlab_api_tools`) provides a thin wrapper around the REST endpoints documented in the [STEM API reference]({{ "/api/stem/" | absolute_url }}). This page mirrors the structure of those docs while expanding on hands-on usage. Every snippet below has been tested against the public API; replace placeholders (for example, IDs) with values available to your account. -```m -output = load_session('filter', {'id', 'ee57e766-fc0c-42e1-9277-7d40d6e9353a'}); +Helper functions such as `get_token`, `load_model`, `load_project`, `load_session`, and `save_model` streamline typical MATLAB workflows while mapping their arguments directly onto the underlying HTTP parameters. + +> **Repository & tutorial:** Source code and examples live in [brainstem-org/brainstem_matlab_api_tools](https://github.com/brainstem-org/brainstem_matlab_api_tools). The repository includes the `brainstem_api_tutorial.m` script that mirrors this guide with runnable sections. + +## Installation + +Clone the MATLAB helper package and add it to your MATLAB path: + +```matlab +git clone https://github.com/brainstem-org/brainstem_matlab_api_tools.git +addpath(genpath('brainstem_matlab_api_tools')) ``` -### Change sorting -Loaded models can be sorted by different criteria applying to their fields. In below example, sessions will be sorted in descending order according to their name. +## Authentication + +Run `get_token` once to create and store an authentication token. Subsequent API calls reuse the cached credentials. -```m -output = load_model('model', 'session', 'sort', {'-name'}); +```matlab +get_token ``` -### Include related models +Tokens expire periodically; re-run `get_token` if you see HTTP 401 responses. -In some cases models contain relations with other models, and they can be also loaded with the models if requested. +## Loading data -In below example, all the projects, data acquisition, behaviors and manipulations related to each session will be included. +`load_model` mirrors the REST endpoints documented under [STEM API]({{ "/api/stem/" | absolute_url }}). All arguments map directly onto their HTTP counterparts, and the helper infers the correct `app` when you omit it. -```m -output = load_model('model', 'session', 'include', {'projects', 'dataacquisition', 'behaviors', 'manipulations'}); +```matlab +sessions = load_model('model', 'session'); +disp(numel(sessions.sessions)) ``` -### Convenience functions -For the three primary data models (projects, subjects and sessions), you can use convenience functions to ease commonly performed tasks. The convenience functions have the fields of the models accessible as input parameters providing convenient filters and the calls include relational data as well: +Convenience wrappers include the correct `app` and default `include` lists for common models: -__load_session__ -```m -output = load_session('name','mysession'); +- `load_project`: includes `sessions`, `subjects`, `collections`, `cohorts` +- `load_subject`: includes `procedures`, `subjectlogs` +- `load_session`: includes `dataacquisition`, `behaviors`, `manipulations`, `epochs` + +They expose a small set of shorthand arguments (for example, `name`, `projects`, `tags`) that expand into the appropriate filter entries internally. + +```matlab +project = load_project('name', 'Allen Institute: Visual Coding – Neuropixels'); +subject = load_subject('name', 'Mouse'); +session = load_session('name', 'Example Session'); ``` -Performs the same API call as: -```m -output = load_model('app', 'stem', 'model', 'session', 'filter', {'name', 'mysession'}, 'include', {'dataacquisition', 'behaviors', 'manipulations', 'epochs'}); + +## Filtering + +Filters accept cell arrays of key–value pairs. The modifier syntax (`.icontains`, `.iexact`, etc.) matches the API documentation. + +```matlab +filtered = load_model( + 'model', 'project', ... + 'filter', {'name.icontains', 'Allen'}); + +disp(numel(filtered.projects)) +``` + +Multiple filter conditions can be specified as pairs in the same cell array: + +```matlab +filtered = load_model( + 'model', 'session', ... + 'filter', {'name.icontains', 'Rat', 'description.icontains', 'demo'}); +``` + +The convenience functions also support multiple filter conditions through their shorthand parameters: + +```matlab +% Filter projects by name and tags +project = load_project('name', 'Allen', 'tags', '1'); + +% Filter subjects by name and sex +subject = load_subject('name', 'Mouse', 'sex', 'M'); ``` -__load_project__ +## Sorting + +Pass a cell array of field names in `sort`. Prefix with `-` for descending order. -```m -output = load_project('name','myproject'); +```matlab +descending_sessions = load_model( + 'model', 'session', ... + 'sort', {'-start_time'}); ``` -Performs the same API call as: -```m -output = load_model('app', 'stem', 'model', 'project', 'filter', {'name', 'myproject'}, 'include', {'sessions', 'subjects', 'collections'}); + +## Including related records + +Request related models (for example, `projects`, `datastorage`, `dataacquisition`) in a single call by supplying `include`. Each entry automatically expands to `.*`, mirroring the REST include syntax. + +```matlab +with_related = load_model( + 'model', 'session', ... + 'include', {'projects', 'datastorage', 'dataacquisition'}); + +project_names = arrayfun(@(p) p.name, with_related.projects, 'UniformOutput', false); ``` -__load_subject__ +## Creating records + +Use `save_model` with the required fields from the STEM API reference. The example below assumes you have contributor access to the project ID provided. Note that `tags` is a required field and can be set to an empty array if no tags are needed. -```m -output = load_subject('name','mysubject'); +```matlab +payload = struct(); +payload.name = 'Example Session'; +payload.description = 'Created via MATLAB API tool'; +payload.projects = {'your-project-uuid'}; +payload.tags = []; + +created = save_model('model', 'session', 'data', payload); +session_id = created.session.id; ``` -Performs the same API call as: -```m -output = load_model('app', 'stem', 'model', 'subject', 'filter', {'name','mysubject'}, 'include', {'procedures', 'subjectlogs'}); + +## Updating records + +Update existing records by placing the record ID inside the payload struct. `save_model` detects the `id` field, switches to `PUT`, and replaces the stored record with the data you supply. Remember to include the `tags` field even when updating. + +```matlab +update_data = struct( + 'id', 'your-session-uuid', ... + 'description', 'Updated via MATLAB API tool', ... + 'tags', []); + +updated = save_model( + 'model', 'session', ... + 'data', update_data); ``` + +## Troubleshooting + +- **401 Unauthorized**: Regenerate your token with `get_token`. +- **403 Forbidden**: Check that your user or group has the required permissions listed in the STEM API documentation. +- **404 Not Found**: Confirm the record exists in the selected portal (public/private). +- **Validation errors (400)**: Ensure your payload matches the field definitions in the API reference. Remember that `tags` is a required field for sessions (can be an empty array). +- **Delete operations**: The MATLAB helpers focus on read and write actions. Use the Python client or direct REST calls if you need to delete records. + +The repository ships with `brainstem_api_tutorial.m`, a tutorial script demonstrating common workflows. + +## Next Steps + +- **Complete Experimental Documentation**: Review the [Electrophysiology Workflow]({{site.baseurl}}/tutorials/electrophysiology-workflow) or [Two-Photon Imaging Workflow]({{site.baseurl}}/tutorials/two-photon-imaging-workflow) tutorials to understand how API data access fits into complete experimental workflows +- **Cross-Platform Integration**: If you work in mixed programming environments, explore the [Python API tool tutorial]({{site.baseurl}}/api-tools/python-api-tool) for complementary examples +- **Data Management**: Learn about [Managing Data Storage]({{site.baseurl}}/tutorials/managing-data-storage) to keep file locations synchronized with metadata returned by the API diff --git a/docs/api-tools/python-api-tool.md b/docs/api-tools/python-api-tool.md index 3207074..9e064f1 100644 --- a/docs/api-tools/python-api-tool.md +++ b/docs/api-tools/python-api-tool.md @@ -7,48 +7,204 @@ nav_order: 2 # Python API tool {: .no_toc} -### Installation +## Table of contents +{: .no_toc .text-delta } -Download the BrainSTEM Python API tools repository from GitHub at [github.com/brainstem-org/brainstem_python_api_tools](https://github.com/brainstem-org/brainstem_python_api_tools). +1. TOC +{:toc} -You can also install the package using `pip`: +## Overview - pip install brainstem_python_api_tools +The BrainSTEM Python API tool (`brainstem_python_api_tools`) provides a thin wrapper around the REST endpoints documented in the [STEM API reference]({{ "/api/stem/" | absolute_url }}). This page mirrors the structure of those docs while expanding on hands-on usage. Every snippet below has been tested against the public API; replace placeholders (for example, IDs) with values you can access in your own account. -### Tutorial -Please see the dedicated [tutorial]({{"/tutorials/python-api-tool/"|absolute_url}}) with examples on usage. The main functions are described below. +> **Repository & notebooks:** Source code and examples live in [brainstem-org/brainstem_python_api_tools](https://github.com/brainstem-org/brainstem_python_api_tools). The repository includes the `brainstem_api_tutorial.ipynb` notebook that mirrors this guide with runnable cells. -| Function | Description | -|:-------------|:-------------| -| `BrainstemClient` | The Brainstem API client | -| `brainstem_api_tutorial` | Tutorial script | +## Installation +Install the published package from PyPI (recommended) or clone the GitHub repository. -### Filters -You can use filters, using fields and relationships by providing cell array with paired filters. Below example will just load the session with the name: +```bash +pip install brainstem_python_api_tools +``` +```bash +git clone https://github.com/brainstem-org/brainstem_python_api_tools.git ``` -from brainstem_api_client import BrainstemClient +## Authentication + +Initializing `BrainstemClient` prompts for your BrainSTEM email and password unless you supply a saved token. Tokens are stored locally in the API tool cache and reused automatically. + +```python +from brainstem_api_tools import BrainstemClient + +# Prompts for credentials the first time and caches a token client = BrainstemClient() +``` + +```python +# Use a saved token instead of prompting for credentials +token = "YOUR_TOKEN" +client = BrainstemClient(token=token) -output1 = client.load_model('session', filters={'name': 'yeah'}).json() +# Or load a token from a .env file (keep .env gitignored) +from dotenv import load_dotenv +import os +from brainstem_api_tools import BrainstemClient + +load_dotenv() +client = BrainstemClient(token=os.getenv("BRAINSTEM_API_TOKEN")) ``` -### Change sorting -Loaded models can be sorted by different criteria applying to their fields. In below example, sessions will be sorted in descending order according to their name. +If you receive a `401 Unauthorized` response, run `BrainstemClient()` again to refresh the cached token or pass a newly generated token explicitly. + +## Quick start + +Load projects using the private portal (default) or specify the public portal for open datasets. Response payloads follow the schema described in `docs/api/stem/project.md`. +```python +public_projects = client.load_model("project", portal="public").json() + +projects = public_projects.get('projects', []) +print(f"Loaded {len(projects)} public projects") +if projects: + print(projects[0]["name"]) ``` -output1 = client.load_model('session', sort=['-name']).json() + +Private data uses the default portal. Ensure the authenticated user has the necessary permissions, as outlined in the API docs. + +```python +private_sessions = client.load_model("session").json() + +sessions = private_sessions.get('sessions', []) +if sessions: + print(sessions[0]["name"]) ``` -### Include related models +## Filtering requests -In some cases models contain relations with other models, and they can be also loaded with the models if requested. +Filters map directly onto the query parameters documented in the STEM API. Common case-insensitive modifiers include `.icontains`, `.iexact`, `.istartswith`, and `.iendswith`. -In below example, all the projects, data acquisition, behaviors and manipulations related to each session will be included. +```python +filtered = client.load_model( + "project", + portal="public", + filters={"name.icontains": "visual"} +).json()["projects"] +print(f"Matches: {len(filtered)}") ``` -output1 = client.load_model('session', include=['projects', 'dataacquisition', 'behaviors', 'manipulations']).json() +Multiple filters apply AND logic. + +```python +filtered = client.load_model( + "project", + portal="public", + filters={ + "name.icontains": "allen", + "description.icontains": "neuropixels" + } +).json()["projects"] + +print(f"Matches: {len(filtered)}") ``` + +## Sorting results + +Provide field names in `sort`. Prefix a field with `-` for descending order, mirroring the API convention. + +```python +# Sort projects alphabetically by name +alpha_projects = client.load_model( + "project", + portal="public", + sort=['name'] # Ascending alphabetical order +).json() + +print("Alphabetically sorted projects:") +for i, project in enumerate(alpha_projects.get('projects', [])[:3]): + print(f"{i+1}. {project.get('name')}") + +# Sort projects reverse-alphabetically by name +reverse_alpha = client.load_model( + "project", + portal="public", + sort=['-name'] # Descending alphabetical order +).json() + +print("Reverse alphabetically sorted projects:") +for i, project in enumerate(reverse_alpha.get('projects', [])[:3]): + print(f"{i+1}. {project.get('name')}") +``` + +## Creating records + +Write operations require contributor permissions on the target project or resource. Use `save_model` with the required fields documented in the relevant API page (for example, `docs/api/stem/session.md`). + +**Important**: The `projects` field must be a list (array) of project IDs, not a single string. + +```python +payload = { + "name": "Example Session", + "projects": ["your-project-uuid"], # Must be a list, not a string! + "description": "Created via API tool" +} + +created = client.save_model("session", data=payload).json() + +if 'session' in created: + session_id = created["session"]["id"] + print(f"Created session with ID: {session_id}") +else: + print(f"Error creating session: {created}") +``` + +## Updating records + +Fetch the target record, modify the fields you want to change, and resubmit the partial payload. + +```python +# First load a session you have permission to modify +filtered_session = client.load_model( + 'session', + filters={'name.iexact': 'your session'} +).json() + +# Update the description +filtered_session['description'] = 'Updated via API' + +# Pass the whole filtered_session object to save_model +updated_session = client.save_model( + 'session', + id=filtered_session['sessions'][0]['id'], + data=filtered_session +).json() + +print("Session updated successfully") +``` + +## Deleting records + +Use `delete_model` with the record ID. A successful delete returns status code `204 No Content`. + +```python +response = client.delete_model("session", id="your-session-uuid") +if response.status_code == 204: + print("Session deleted") +``` + +## Troubleshooting + +- **401 Unauthorized**: Re-run `BrainstemClient()` to generate a fresh token or confirm credentials. +- **403 Forbidden**: Verify you have the necessary permissions (see the permission tables in the STEM API docs). +- **404 Not Found**: Make sure the ID exists and is within the selected portal. +- **Validation errors (400)**: Match the required fields and value formats listed in the model-specific documentation. + +For a notebook-style walk-through, open `brainstem_api_tutorial.ipynb` in the API tools repository. + +## Next Steps + +- **Complete Experimental Workflows**: Review the [Electrophysiology Workflow]({{site.baseurl}}/tutorials/electrophysiology-workflow) or [Two-Photon Imaging Workflow]({{site.baseurl}}/tutorials/two-photon-imaging-workflow) tutorials to see how API data fits into complete experimental documentation +- **Data Storage Integration**: Learn about [Managing Data Storage]({{site.baseurl}}/tutorials/managing-data-storage) to optimize your API-based analysis workflows with proper data organization +- **MATLAB Integration**: If you work in mixed programming environments, check out the [MATLAB API tool tutorial]({{site.baseurl}}/api-tools/matlab-api-tool) for cross-platform data access patterns diff --git a/docs/api-tools/web-api-tool.md b/docs/api-tools/web-api-tool.md index b858176..6f90cf7 100644 --- a/docs/api-tools/web-api-tool.md +++ b/docs/api-tools/web-api-tool.md @@ -17,7 +17,7 @@ The Web API Tool provides a browser-based interface for exploring, testing, and ## Getting Started -Access the Web API Tool by navigating to any BrainSTEM API endpoint in your web browser. You'll need to be logged into your BrainSTEM account to access private data. +Access the Web API Tool by navigating to any BrainSTEM API endpoint in your web browser. You'll need to be logged into your BrainSTEM account to access private data. The web interface uses your existing browser session for authentication, so no additional API keys or tokens are required for basic exploration. ### Basic Usage {: .no_toc} @@ -53,7 +53,7 @@ The interface shows the actual API response structure, including: - **Metadata**: Pagination information (`"meta"` object with page details) - **HTTP status**: Response codes and headers -![web_api_screenshot_v2](/assets/images/web_api_screenshot_v2.png) +![web_api_screenshot_v3](/assets/images/web_api_screenshot_v3.png) ### Raw JSON Output {: .no_toc} diff --git a/docs/api/personal_attributes/datastorage.md b/docs/api/personal_attributes/datastorage.md index 122eafd..3775abc 100644 --- a/docs/api/personal_attributes/datastorage.md +++ b/docs/api/personal_attributes/datastorage.md @@ -234,7 +234,7 @@ resp = client.save_model("datastorage", id="9f322057-cf48-4ec7-ab19-d0d7175cffe2 'name': 'MyNewRepo', 'description': 'new text', 'is_public': False, - 'data_organization_json': [ + 'data_organization': [ { "elements": "Sessions" }, @@ -242,7 +242,7 @@ resp = client.save_model("datastorage", id="9f322057-cf48-4ec7-ab19-d0d7175cffe2 "elements": "Subjects" } ], - 'data_protocols_json': [ + 'data_protocols': [ { "protocol": "Dropbox (Cloud solution)", "path": "data/myproject", diff --git a/docs/api/personal_attributes/setup.md b/docs/api/personal_attributes/setup.md index a9ffd49..2ae38be 100644 --- a/docs/api/personal_attributes/setup.md +++ b/docs/api/personal_attributes/setup.md @@ -25,7 +25,7 @@ nav_order: 4 | `location` | string describing the location of the setup | | `description` | string [max length: 500]| | `setup_type` | related environment type ID formatted as a string **[required]** | -| `specifications` | JSON dictionary describing setup specifications | +| `specifications` | JSON array describing setup specifications | | `is_public` | boolean | @@ -65,7 +65,7 @@ resp = client.load_model('setup') 'location': 'Lab Room 102', 'description': '', 'setup_type': 'e1f14b91-e507-48c1-bfec-c68d7db9c166', - 'specifications': [], + 'specifications': {}, 'is_public': True } ]} @@ -91,9 +91,10 @@ resp = client.save_model("setup", data= 'location': 'Lab Room 103', 'description': '', 'setup_type': '78dc6c02-dcb0-4a31-a035-a358c7ee9e79', - 'specifications': [ - {'name': 'Length', 'value': 100, 'description': 'yards'} - ], + 'specifications': { + 'Length': 100, + 'Width': '30 cm' + }, 'is_public': False } ) @@ -109,9 +110,10 @@ resp = client.save_model("setup", data= 'location': 'Lab Room 103', 'description': '', 'setup_type': '78dc6c02-dcb0-4a31-a035-a358c7ee9e79', - 'specifications': [ - {'name': 'Length', 'value': 100, 'description': 'yards'} - ], + 'specifications': { + 'Length': 100, + 'Width': '30 cm' + }, 'is_public': False} } ``` @@ -142,9 +144,10 @@ resp = client.load_model('setup', id='d0ada97d-8607-48da-817b-bdd54bc9077b') 'location': 'Lab Room 103', 'description': '', 'setup_type': '78dc6c02-dcb0-4a31-a035-a358c7ee9e79', - 'specifications': [ - {'name': 'Length', 'value': 100, 'description': 'yards'} - ], + 'specifications': { + 'Length': 100, + 'Width': '30 cm' + }, 'is_public': False} } ``` @@ -174,9 +177,10 @@ resp = client.save_model("setup", id="d0ada97d-8607-48da-817b-bdd54bc9077b", dat 'name': 'MyNewEnv', 'description': 'new text', 'setup_type': '78dc6c02-dcb0-4a31-a035-a358c7ee9e79', - 'physical_dimensions_json': [ - {'name': 'Length', 'value': 100, 'description': 'yards'} - ], + 'specifications': { + 'Length': 100, + 'Width': '30 cm' + }, 'is_public': False} } ``` diff --git a/docs/api/stem/session.md b/docs/api/stem/session.md index e53a532..c2be320 100644 --- a/docs/api/stem/session.md +++ b/docs/api/stem/session.md @@ -72,12 +72,14 @@ Use the Session → Data Storage endpoint (`/api/private/stem/sessiondatastorage - **Responses:** `200` OK; `403` Not allowed; `404` Not found ### Example request (list, Python API) +{: .no_toc} ```python resp = client.load_model("session") ``` ### Example response (list) +{: .no_toc} ```json { @@ -125,6 +127,7 @@ resp = client.load_model("session") - **Responses:** `201` OK; `400` Bad request; `403` Not allowed; `404` Not found ### Example request (create, Python API) +{: .no_toc} ```python resp = client.save_model( @@ -138,6 +141,7 @@ resp = client.save_model( ``` ### Example response (create) +{: .no_toc} ```json { @@ -168,12 +172,14 @@ resp = client.save_model( - **Responses:** `200` OK; `403` Not allowed; `404` Not found ### Example request (detail, Python API) +{: .no_toc} ```python resp = client.load_model("session", id="13bdd793-86d4-428e-9708-167bbc26f6d2") ``` ### Example response (detail) +{: .no_toc} ```json { @@ -210,6 +216,7 @@ resp = client.load_model("session", id="13bdd793-86d4-428e-9708-167bbc26f6d2") - **Responses:** `200` OK; `400` Bad request; `403` Not allowed; `404` Not found ### Example request (update, Python API) +{: .no_toc} ```python resp = client.save_model( @@ -220,6 +227,7 @@ resp = client.save_model( ``` ### Example response (update) +{: .no_toc} ```json { @@ -250,6 +258,7 @@ resp = client.save_model( - **Responses:** `204` OK; `403` Not allowed; `404` Not found ### Example request (delete, Python API) +{: .no_toc} ```python resp = client.delete_model("session", id="13bdd793-86d4-428e-9708-167bbc26f6d2") diff --git a/docs/assets/images/tutorials/ephys/add_cohort.png b/docs/assets/images/tutorials/ephys/add_cohort.png new file mode 100644 index 0000000..58b44a0 Binary files /dev/null and b/docs/assets/images/tutorials/ephys/add_cohort.png differ diff --git a/docs/assets/images/tutorials/ephys/add_project.png b/docs/assets/images/tutorials/ephys/add_project.png new file mode 100644 index 0000000..5175c4a Binary files /dev/null and b/docs/assets/images/tutorials/ephys/add_project.png differ diff --git a/docs/assets/images/tutorials/ephys/add_subject.png b/docs/assets/images/tutorials/ephys/add_subject.png new file mode 100644 index 0000000..5ad889d Binary files /dev/null and b/docs/assets/images/tutorials/ephys/add_subject.png differ diff --git a/docs/assets/images/tutorials/ephys/equipment_list.png b/docs/assets/images/tutorials/ephys/equipment_list.png new file mode 100644 index 0000000..0cce107 Binary files /dev/null and b/docs/assets/images/tutorials/ephys/equipment_list.png differ diff --git a/docs/assets/images/tutorials/ephys/subject_procedure_form.png b/docs/assets/images/tutorials/ephys/subject_procedure_form.png new file mode 100644 index 0000000..b8a7b29 Binary files /dev/null and b/docs/assets/images/tutorials/ephys/subject_procedure_form.png differ diff --git a/docs/assets/images/tutorials/ephys/subject_procedure_view.png b/docs/assets/images/tutorials/ephys/subject_procedure_view.png new file mode 100644 index 0000000..f0a8e9f Binary files /dev/null and b/docs/assets/images/tutorials/ephys/subject_procedure_view.png differ diff --git a/docs/assets/images/web_api_screenshot_v3.png b/docs/assets/images/web_api_screenshot_v3.png new file mode 100644 index 0000000..8294b38 Binary files /dev/null and b/docs/assets/images/web_api_screenshot_v3.png differ diff --git a/docs/assets/images/webinterface/import_tool.png b/docs/assets/images/webinterface/import_tool.png new file mode 100644 index 0000000..9f6a51a Binary files /dev/null and b/docs/assets/images/webinterface/import_tool.png differ diff --git a/docs/datamodel/personal_attributes/setup.md b/docs/datamodel/personal_attributes/setup.md index f174d72..9a94af8 100644 --- a/docs/datamodel/personal_attributes/setup.md +++ b/docs/datamodel/personal_attributes/setup.md @@ -18,7 +18,7 @@ has_toc: false ## Introduction -Setups describe where experiments, procedures, or manipulations takes place. It also describes the environment of a recorded subject. Setups has physical dimensions and equipment associate with it. +Setups describe where experiments, procedures, or manipulations takes place. It also describes the environment of a recorded subject. Setups has specifications and equipment associate with it. ## Fields @@ -32,7 +32,7 @@ Setups describe where experiments, procedures, or manipulations takes place. It | ``Description`` | Rich text description of the setup (optional). Can include uploaded images. Example: "Linear track setup with two reward ports and tracking cameras..." | | ``Image`` | Image of the setup (optional). Uploaded images remain completely private. Example: "setup1_overview.jpg" | | ``Public access`` | Determines if the setup is public or private. Example: "False" for private access | -| ``Specifications`` | Physical specifications of the setup in key-value format (optional). Example: {"length": "200 cm", "width": "10 cm", "height": "15 cm"} | +| ``Specifications`` | Specifications to the setup in key-value format (optional). Example: {"length": "200 cm", "width": "10 cm", "height": "15 cm"} | ## Permissions diff --git a/docs/tutorials.md b/docs/tutorials.md index 3ca5977..c886f52 100644 --- a/docs/tutorials.md +++ b/docs/tutorials.md @@ -12,9 +12,14 @@ has_toc: false BrainSTEM provides comprehensive tutorials to help users get started and make the most of the platform's features. These step-by-step guides cover everything from basic setup to advanced functionality. - [**Get started!**]({{site.baseurl}}/tutorials/get_started): A beginner's guide to help new users create accounts, join groups, and create their first project and experiments in BrainSTEM. -- [**Share data publicly**]({{site.baseurl}}/tutorials/sharing-project-publicly): Learn how to make your research data publicly available and accessible through BrainSTEM's public sharing features. +- [**Setting Up Lab Infrastructure**]({{site.baseurl}}/tutorials/setting-up-lab-infrastructure): Comprehensive guide to establishing your lab's foundational elements: setups, equipment, inventories, and consumable stocks. - [**Manage Groups**]({{site.baseurl}}/tutorials/managing-groups): Detailed instructions for creating and managing research groups, including setting permissions and organizing team members. - [**Manage Projects**]({{site.baseurl}}/tutorials/managing-projects): Guidelines for creating and organizing research projects, including best practices for data organization and collaboration. -- [**Matlab API tool**]({{site.baseurl}}/tutorials/matlab-api-tool): Documentation for using BrainSTEM's Matlab API tools to programmatically access and manage your research data. -- [**Python API tool**]({{site.baseurl}}/tutorials/python-api-tool): Guide to utilizing BrainSTEM's Python API interface for automated data management and analysis. -- [**Submit Resources & Taxonomies**]({{site.baseurl}}/tutorials/submit-resource-and-taxonomies): Instructions for contributing to BrainSTEM's shared resources and taxonomies to enhance the platform's capabilities. \ No newline at end of file +- [**Managing Data Storage**]({{site.baseurl}}/tutorials/managing-data-storage): Comprehensive guide to setting up data storage locations, linking them to sessions, and enabling dynamic data access for analysis workflows. +- [**Share data publicly**]({{site.baseurl}}/tutorials/sharing-project-publicly): Learn how to make your research data publicly available and accessible through BrainSTEM's public sharing features. +- [**Behavioral Paradigms**]({{site.baseurl}}/tutorials/behavioral-paradigms): Complete guide to creating and managing behavioral paradigms for experiments, including environment types, stimulus presentations, and reward systems. +- [**Electrophysiology Workflow**]({{site.baseurl}}/tutorials/electrophysiology-workflow): Example experimental workflow for electrophysiology studies, from project setup and probe implantation to data collection and spatial analysis. +- [**Two-Photon Imaging Workflow**]({{site.baseurl}}/tutorials/two-photon-imaging-workflow): Example workflow for two-photon calcium imaging experiments, covering viral injections, surgical procedures, imaging sessions, and visual stimulus analysis. +- [**Matlab API tool**]({{site.baseurl}}/api-tools/matlab-api-tool): Documentation for using BrainSTEM's Matlab API tools to programmatically access and manage your research data. +- [**Python API tool**]({{site.baseurl}}/api-tools/python-api-tool): Guide to utilizing BrainSTEM's Python API interface for automated data management and analysis. +- [**Submit Resources & Taxonomies**]({{site.baseurl}}/tutorials/submit-resource-and-taxonomies): Instructions for contributing to BrainSTEM's shared resources and taxonomies to enhance the platform's capabilities. diff --git a/docs/tutorials/behavioral-paradigms.md b/docs/tutorials/behavioral-paradigms.md new file mode 100644 index 0000000..076ccb9 --- /dev/null +++ b/docs/tutorials/behavioral-paradigms.md @@ -0,0 +1,63 @@ +--- +layout: default +title: Behavioral Paradigms +parent: Tutorials +nav_order: 5 +--- +# Behavioral Paradigms in BrainSTEM +{: .no_toc} + +## Introduction + +Behavioral paradigms are standardized experimental protocols or tasks (such as T-maze alternation, open field exploration, or Morris water maze) that are performed within a specific setup environment. Defining paradigms in BrainSTEM ensures consistency, reproducibility, and clear documentation of behavioral experiments across your lab. + +## What is a Behavioral Paradigm? + +A behavioral paradigm defines: +- The type of behavioral task or protocol (e.g., spatial memory, anxiety, locomotion) +- The required environment type (must match the setup's environment type) +- Key parameters or variables (e.g., trial structure, cues, reward schedules) + +## How Behavioral Paradigms Relate to Setups + +- Each behavioral paradigm is associated with a specific environment type (e.g., T-maze, Open field) +- Only setups with a matching environment type can be used for a given paradigm +- This ensures that experimental sessions are created with the correct physical and procedural context + +## Creating a Behavioral Paradigm + +1. Go to *Personal Attributes* → *Behavioral Paradigms* +2. Click *Add behavioral paradigm* +3. Fill in the required fields: + - **Name**: Descriptive name (e.g., "T-maze Alternation Task") + - **Setup Type**: Select the matching setup type (e.g., T-maze) + - **Description**: Briefly describe the protocol, goals, and any special requirements +4. Save the paradigm. It will now be available when creating sessions in compatible setups. + +### Example: Defining a T-maze Alternation Paradigm +```json +{ + "name": "T-maze Alternation Task", + "setup_type": "T-maze", + "description": "A spatial working memory task where the subject must alternate between left and right arms for reward." +} +``` + +## Best Practices +- Use clear, standardized names for paradigms +- Include enough detail in the description for reproducibility +- Link paradigms to the correct environment type to avoid confusion +- Update paradigms as protocols evolve, but keep a record of changes + +{: .note } +> Behavioral paradigms are critical for organizing and analyzing behavioral data. Defining them up front ensures your lab's experiments are well-documented and comparable across projects. + +## Next Steps + +Now that you have behavioral paradigms defined, you can begin documenting complete experiments: + +- **Document complete experiments**: Follow the [Electrophysiology Workflow]({{site.baseurl}}/tutorials/electrophysiology-workflow) tutorial to see how behavioral paradigms integrate with full experimental documentation +- **Set up your infrastructure**: Configure the physical setups and equipment using [Setting Up Lab Infrastructure]({{site.baseurl}}/tutorials/setting-up-lab-infrastructure) to create the experimental environments where your behavioral paradigms will be executed +- **Set up data storage**: Configure [Managing Data Storage]({{site.baseurl}}/tutorials/managing-data-storage) to link your behavioral sessions to actual data files for analysis +- **Expand your resources**: Visit [Submit Resources & Taxonomies]({{site.baseurl}}/tutorials/submit-resource-and-taxonomies) to contribute new behavioral equipment or paradigm types to the BrainSTEM platform +- **Enable data sharing**: Make your behavioral paradigms publicly available through [Sharing Project Publicly]({{site.baseurl}}/tutorials/sharing-project-publicly) to promote reproducible research protocols diff --git a/docs/tutorials/electrophysiology-workflow.md b/docs/tutorials/electrophysiology-workflow.md new file mode 100644 index 0000000..cb3e192 --- /dev/null +++ b/docs/tutorials/electrophysiology-workflow.md @@ -0,0 +1,469 @@ +--- +layout: default +title: Ephys Workflow +parent: Tutorials +nav_order: 8 +--- + +# Ephys Workflow: Theta Maze with Sleep Sessions +{: .no_toc} + +## Introduction + +This tutorial walks through a complete electrophysiology experiment in BrainSTEM, from subject preparation to data analysis. We'll document a multi-day theta maze learning study with sleep sessions, demonstrating how to integrate subjects, procedures, behavioral paradigms, data acquisition, manipulations, and collections in a realistic research workflow. + +**Experimental Design Overview:** +- **Subjects**: Rat cohort for spatial learning study +- **Setup**: Theta maze with ceiling-mounted camera and OptiTrack 3D tracking +- **Sessions**: Pre-behavior sleep → Theta maze behavior → Post-behavior sleep +- **Data**: Neuropixels recordings, behavioral video, 3D position tracking +- **Analysis**: Learning curves, replay detection, place cell analysis + +{: .note } +> This tutorial assumes that the lab infrastructure is already set up (setups, equipment, inventories) as described in the [Setting Up Lab Infrastructure tutorial]({{site.baseurl}}/tutorials/setting-up-lab-infrastructure). + +## Part A: Project and Subject Setup + +### Step 1: Creating the Research Project + +First, we'll create a project to organize and provide context for our experimental work. The project submission is split in two steps. First, you fill in required details, after which you can add other details. Required fields are highlighted with an asterix (*) in the web interface and this tutorial. + +1. **Navigate to Projects**: + - Go to *Projects* + - Click *Add project* + +![Project creation interface](/assets/images/tutorials/ephys/add_project.png) + +2. **Configure Project Details**: + +| Field | Value | +|-------|-------| +| **Name** * | Theta Maze Spatial Learning Study | +| **Description** | Investigating hippocampal place cell dynamics and memory consolidation during spatial learning using theta maze behavioral paradigm with simultaneous Neuropixels recordings in freely moving rats. | +| **Authenticated Groups** | Select groups that should have access to this project | +| **Public Access** | No (keep project private to lab) | + +{: .note } +> Once submitted other details can be added. Additional project details like PI name, funding source, IACUC protocol numbers, and timeline information can be included in the project description or managed through your lab's separate project management systems. Authenticated groups will get contribute permissions. You can add additional groups or change permissions in the manage page: Go to *Project detail page* → *Manage* + +### Step 2: Adding Individual Subjects + +Now we'll add individual rats that will participate in our study. + +1. **Navigate to Add Subject**: + - Go to *Subjects* + - Click *Add subject* + +![Subject creation form](/assets/images/tutorials/ephys/add_subject.png) + +2. **Subject Configuration Example**: + +| Field | Value | +|-------|-------| +| **Name** * | TM_R001 | +| **Projects** * | Select: Theta Maze Spatial Learning Study | +| **Sex** * | Male | +| **Species** * | Select: Rattus norvegicus | +| **Strain** * | Select: Long-Evans | +| **Description** | Male Long-Evans rat for theta maze spatial learning study with Neuropixels recordings | +| **Genetic Line** | Wild type | +| **Birth Date** | 2024-05-01 | +| **Subject Identifier** | Red-001 (ear tag number) | + +**Repeat this process to create additional subjects:** +- TM_R002, TM_R003, TM_R004 (for a cohort of 4 rats) +- Use similar configurations but update IDs, ear tags + +{: .note } +> The additional subjects can be duplicated from the first subject. Go to the *Subject detail page of the first subject* → *Duplicate* +> Fill in the new subject name and click Duplicate. After this you can alter other fields if necessary. + +### Step 2b: Document Initial Housing and Weight + +After creating subjects, document their housing conditions and initial weights using subject logs. + +1. **Create Housing Log**: + - Go to *Subject logs* (below Subjects) + - Click *Add subject log* + +**Housing Log Configuration:** + +| Field | Value | +|-------|-------| +| **Type** | Housing log | +| **Subject** | Select: TM_R001 | +| **Description** | Initial housing assignment for experimental cohort | + +**Type-specific Details:** + +| Field | Value | +|-------|-------| +| **Start and end time** | 2024-08-01 08:00:00 to 2024-12-31 18:00:00 | +| **Notes** | Pair-housed with TM_R002. Standard rat cages with enrichment. | +| **Location** | Animal Facility Room 204 | +| **Cage ID** | Cage_A1_pair | +| **Cage type** | NexGen Rat 1800 | +| **Light cycle** | 12:12 light-dark cycle (lights on 07:00) | +| **Enrichment** | Paper tubes, wooden chew blocks | + +{: .note } +> Only the start time is required, so you can leave the end time blank. + +2. **Create Initial Weighing Log**: + +**Weighing Log Configuration:** + +| Field | Value | +|-------|-------| +| **Type** | Weighing log | +| **Subject** | Select: TM_R001 | + +**Type-specific Details:** + +| Field | Value | +|-------|-------| +| **Weight (grams)** | 385 | +| **Date and time** | 2024-08-01 09:00:00 | +| **Notes** | Pre-experiment baseline weight. Animal healthy and active. | + +### Step 3: Creating the Experimental Cohort + +After creating individual subjects, we can group them into a cohort for experimental organization. + +1. **Navigate to Cohorts**: + - Go to *Cohorts* (below Subjects) + - Click *Add cohort* + +![Cohort creation form](/assets/images/tutorials/ephys/add_cohort.png) + +2. **Configure Cohort Details**: + +| Field | Value | +|-------|-------| +| **Name** | TM_Learning_Cohort_1 | +| **Project** | Select: Theta Maze Spatial Learning Study | +| **Subjects** | Select: TM_R001, TM_R002, TM_R003, TM_R004 | +| **Description** | First experimental cohort for theta maze spatial learning study. 4 male Long-Evans rats, age-matched littermates, trained on alternation task with Neuropixels recordings. Pair housed, 12:12 light cycle, food restricted to 85% body weight during training. | +| **Tags** | theta maze, spatial learning, neuropixels, male rats | + + + +### Step 4: Equipment and Setup Verification + +Verify your theta maze setup includes all necessary equipment: + +![Equipment list view filtered by setup](/assets/images/tutorials/ephys/equipment_list.png) + +**Required Equipment**: +- Neuropixels data acquisition system (IMEC - Neuropixels OneBox) +- Ceiling-mounted camera for behavioral tracking (Basler - acA640-90uc) +- OptiTrack motion capture system for 3D head tracking (6+ cameras OptiTrack - Flex 13) + +### Step 5: Neuropixels Probe Implantation Procedure + +Document the surgical procedure for Neuropixels probe implantation. + +1. **Navigate to Procedures**: + - Go to *Subjects* (below Subjects) + - Click Edit for subject *TM_R001* + +![Procedure view](/assets/images/tutorials/ephys/subject_procedure_view.png) + +2. **Procedure Configuration**: + +**Basic Information:** + +| Field | Value | +|-------|-------| +| **Type** | Silicon probe implant | +| **Subject** | Select: TM_R001 | +| **Date and Time** | 2024-08-15 10:00:00 | +| **Setup** | Select: Surgical Station | +| **Equipment** | Select: Stereotaxic Frame | +| **Inventory** | Select: Neural Electrode Inventory | +| **Consumable Stock** | Select: Neuropixels 2.0 probe | +| **Atlas** | Allen Mouse Brain Atlas | +| **Brain Region** | HIP: Hippocampal region | +| **Notes** | Chronic Neuropixels probe implantation targeting CA1/CA3 hippocampus. | +| **Coordinates System** | Stereotaxic Bregma-Based Surface Coordinates with Depth | + +**Coordinate Details:** + +| Field | Value | +|-------|-------| +| **AP coordinate (mm)** | -3.5 | +| **ML coordinate (mm)** | +2.5 | +| **DV coordinate (mm)** | 2.0 (initial insertion depth) | +| **AP angle** | 0° | +| **ML angle** | 0° | +| **Rotation** | 0° | + +{: .note } +> Probe specifications (Neuropixels 2.0: 960 total sites, 384 recording sites, 20 μm spacing, etc.) are documented in the consumable stock entry, not in the procedure record. You can also document the Isoflurane anesthesia, Head fixation, Craniotomy, and the Headcap, but we will leave that out for brevity. + + + +## Part B: Multi-Session Experimental Workflow + +### Step 6: Pre-Behavior Sleep Session + +After recovery, we begin with a baseline sleep recording. + +1. **Navigate to Sessions**: + - Go to *Sessions* + - Click *Add session* + + + +2. **Sleep Session Configuration**: + +**Basic Session Information:** + +| Field | Value | +|-------|-------| +| **Name** | TM_R001_Day1_PreSleep_Baseline | +| **Projects** | Select: Theta Maze Spatial Learning Study | +| **Description** | Baseline sleep recording before theta maze training. 4-hour session to capture natural sleep-wake cycles and establish baseline neural activity patterns. | +| **Date_time_onset** | 2024-08-22 09:00:00 | +| **Tags** | baseline, sleep, pre-training, neuropixels | + +{: .note } +> Sessions don't have subject, setup, duration, or behavioral paradigm fields directly. These are specified in the Behavior, Data acquisition and Epoch tabs within the session. + +**Data Acquisition - Extracellular Recording:** + +| Setting | Value | +|---------|-------| +| **Type** | Extracellular Electrophysiology | +| **Subject procedures** | Select: TM_R001 : Silicon probe implant : Neuropixels Implantation | +| **Setup** | Select: Sleep Recording Box | +| **Equipment** | Select: Neuropixels System | +| **File name** | TM_R001_Day1_PreSleep_Baseline.dat | +| **Format** | Binary | +| **Data-type** | int16 | +| **Number of channels** | 384 | +| **Sampling rate (Hz)** | 30000 | +| **Least significant bit (μV/bit)** | 0.195 | + +**Data Acquisition - Video:** + +| Setting | Value | +|---------|-------| +| **Session** | Select: TM_R001_Day1_PreSleep | +| **Type** | Video Tracking | +| **Procedures** | Select: TM_R001 : Silicon probe implant : Neuropixels Implantation | +| **Setup** | Select: Sleep Recording Box | +| **Equipment** | Select: Sleep Box Camera | +| **Notes** | Sleep state monitoring | + +**Session Epochs:** + +| Epoch Name | Start (min) | Duration (min) | Description | +|------------|-------------|----------------|-------------| +| **Habituation** | 0 | 30 | Adaptation to recording environment | +| **Baseline_recording** | 30 | 210 | Continuous sleep/wake recording (3.5 hours) | + + + +### Step 7: Theta Maze Behavioral Session + +The main experimental session combining behavior with neural recording. + + + +1. **Behavioral Paradigm Setup**: + +| Field | Value | +|-------|-------| +| **Name** | Theta Maze Alternation Task | +| **Environment Type** | Theta maze | +| **Authenticated Groups** | Select your lab groups | +| **Description** | Forced alternation task in theta maze. Rats navigate alternating left/right arms to receive food rewards. Trial duration 180s, inter-trial interval 45s, total 40 trials per session. Performance criteria: 80% correct, minimum 15 trials for learning threshold. | +| **Public Access** | No | + +2. **Complete Session Configuration**: + +**Basic Session Information:** + +| Field | Value | +|-------|-------| +| **Name** | TM_R001_Day1_ThetaMaze_Training1 | +| **Projects** | Select: Theta Maze Spatial Learning Study | +| **Description** | First day theta maze behavioral training with Neuropixels recording | +| **Date_time_onset** | 2024-08-22 14:00:00 | +| **Tags** | behavior, theta-maze, training, day1 | +| **Data_storage** | /data/neuropixels/tm_r001/ | + +**Data Acquisition - Extracellular Recording:** + +| Setting | Value | +|---------|-------| +| **Session** | Select: TM_R001_Day1_Behavior | +| **Type** | Extracellular Electrophysiology | +| **Procedures** | Select: Neuropixels Implantation | +| **Setup** | Select: Theta Maze Rig | +| **Equipment** | Select: Neuropixels System | +| **Format** | Binary | +| **Data-type** | int16 | +| **Number of channels** | 384 | +| **Sampling rate (Hz)** | 30000 | +| **Number of samples** | 216000000 | +| **Least significant bit (μV/bit)** | 0.195 | + +**Data Acquisition - Video:** + +| Setting | Value | +|---------|-------| +| **Session** | Select: TM_R001_Day1_Behavior | +| **Type** | Video | +| **Procedures** | Select: Neuropixels Implantation | +| **Setup** | Select: Theta Maze Rig | +| **Equipment** | Select: Ceiling Camera | +| **Notes** | Behavioral video recording with LED tracking | + +**Data Acquisition - Position Tracking:** + +| Setting | Value | +|---------|-------| +| **Session** | Select: TM_R001_Day1_Behavior | +| **Type** | Behavioral Tracking | +| **Procedures** | Select: Neuropixels Implantation | +| **Setup** | Select: Theta Maze Rig | +| **Equipment** | Select: OptiTrack System | +| **Notes** | Head and body position tracking with LED markers | + +**Session Epochs:** + +| Epoch Name | Start (min) | Duration (min) | Description | +|------------|-------------|----------------|-------------| +| **Pre_task_baseline** | 0 | 10 | Open field exploration before task | +| **Training_trials** | 10 | 60 | Forced alternation training trials (longer for rats) | +| **Free_choice_trials** | 70 | 40 | Free choice alternation test (longer for rats) | +| **Post_task_rest** | 110 | 10 | Rest period after task completion | + + + +### Step 8: Adding Manipulations (Optional) + +For experiments with optogenetic or pharmacological manipulations. + + + +**Manipulation Configuration:** + +**Basic Information:** + +| Field | Value | +|-------|-------| +| **Session** | Select: TM_R001_Day1_Behavior | +| **Type** | Optogenetic stimulation | +| **Procedures** | Select: Neuropixels Implantation | +| **Setup** | Select: Theta Maze Rig | +| **Equipment** | Select: Yellow Laser System | +| **Notes** | PV interneuron silencing during choice trials | + +**Type-specific Details:** + +| Parameter | Value | +|-----------|-------| +| **Power (mW)** | 10.0 | +| **Amplitude (A)** | 0.5 | +| **Stimulation profile** | Continuous | +| **Duration (s)** | 0.5 | +| **Duty cycle** | 1.0 | +| **Repetitions** | 40 | +| **Wavelength (nm)** | 589 | +| **Closed loop** | false | + +### Step 9: Post-Behavior Sleep Session + +Capture sleep-dependent replay and consolidation. + +**Post-Sleep Session Configuration:** + +**Basic Session Information:** + +| Field | Value | +|-------|-------| +| **Name** | TM_R001_Day1_PostSleep_Consolidation | +| **Projects** | Select: Theta Maze Spatial Learning Study | +| **Description** | Post-behavior sleep session for memory consolidation recording | +| **Date_time_onset** | 2024-08-22 16:00:00 | +| **Tags** | postsleep, consolidation, replay | +| **Data_storage** | /data/neuropixels/tm_r001/ | + +**Data Acquisition - Extracellular Recording:** + +| Setting | Value | +|---------|-------| +| **Session** | Select: TM_R001_Day1_PostSleep_Consolidation | +| **Type** | Extracellular Electrophysiology | +| **Procedures** | Select: Neuropixels Implantation | +| **Setup** | Select: Sleep Recording Box | +| **Equipment** | Select: Neuropixels System | +| **Format** | Binary | +| **Data-type** | int16 | +| **Number of channels** | 384 | +| **Sampling rate (Hz)** | 30000 | +| **Number of samples** | 648000000 | + +**Session Epochs:** + +| Epoch Name | Start (min) | Duration (min) | Description | +|------------|-------------|----------------|-------------| +| **Post_behavior_rest** | 0 | 60 | Immediate post-behavior period | +| **Sleep_recording** | 60 | 300 | Extended sleep/replay recording (5 hours) | + +## Part C: Data Organization and API access + +### Step 10: Creating Collections + +Group related sessions for analysis and organization. + + + +**Collection Configuration:** + +| Field | Value | +|-------|-------| +| **Name** | TM_R001_Learning_Week1 | +| **Project** | Select: Theta Maze Spatial Learning Study | +| **Sessions** | Select: TM_R001_Day1_PreSleep_Baseline, TM_R001_Day1_ThetaMaze_Training1, TM_R001_Day1_PostSleep_Consolidation | +| **Description** | First week of theta maze training for subject TM_R001. Collection includes presleep baseline, behavioral training session, and postsleep consolidation recording. | +| **Tags** | learning, spatial_memory, hippocampus, week1, place_cells | + +### Step 11: API Data Access for Analysis + +Access your experimental data programmatically for analysis. + +```python +from brainstem_api_client import BrainstemClient + +client = BrainstemClient() + +# Get all sessions for a subject +subject_sessions = client.load_model('session', + filters={'subject__name': 'TM_R001'}).json() + +# Get behavioral performance data +behavior_data = client.load_model('behavioralparadigm', + filters={'session__subject__name': 'TM_R001'}).json() + +# Get neural data file paths +neural_files = client.load_model('dataacquisition', + filters={'session__subject__name': 'TM_R001', + 'type': 'Extracellular'}).json() + +# Download specific session data +session_data = client.load_model('session', filters={'name': 'tm_r001_day1_behavior_id'}) +``` + +## Next Steps + +After completing this comprehensive electrophysiology workflow, consider these logical progressions: + + +- **Access data programmatically**: Master the [Python API tool]({{site.baseurl}}/api-tools/python-api-tool) or [MATLAB API tool]({{site.baseurl}}/api-tools/matlab-api-tool) to automate analysis workflows for your large-scale electrophysiology datasets +- **Understand behavioral documentation**: Review [Behavioral Paradigms]({{site.baseurl}}/tutorials/behavioral-paradigms) to understand how behavioral protocols are documented and standardized across experiments +- **Optimize data management**: Learn about [Managing Data Storage]({{site.baseurl}}/tutorials/managing-data-storage) to efficiently organize and access your large electrophysiology datasets with proper file linking +- **Enable data sharing**: Use [Sharing Project Publicly]({{site.baseurl}}/tutorials/sharing-project-publicly) to make your electrophysiology datasets available to the research community for collaborative analysis and meta-studies diff --git a/docs/tutorials/get_started.md b/docs/tutorials/get_started.md index 31830b9..a8e7424 100644 --- a/docs/tutorials/get_started.md +++ b/docs/tutorials/get_started.md @@ -85,23 +85,48 @@ After joining or creating a group, you can start creating projects. Projects in ## Add your personal attributes -Next, create personal attributes relevant to your experiments. +Before creating subjects and sessions, set up personal attributes that describe your setups, inventories and behavioral paradigms. These provide standardized options for your experiments. ### Define behavioral paradigms to describe the behavioral conditions of your subjects {: .no_toc} +1. Navigate to *Personal Attributes* → *Behavioral paradigms* in the left menu +2. Click *Add behavioral paradigm* +3. Fill in the paradigm details (e.g., "Open Field Test", "Novel Object Recognition") +4. Add a description of the experimental protocol + ### Set up data storage locations to link to your data files {: .no_toc} +1. Go to *Personal Attributes* → *Data storage* +2. Click *Add data storage* +3. Define storage locations where your raw data files are kept (e.g., "Lab Server", "External Drive A") +4. Include relevant path information and access details + ### Create setups to define your physical experimental environments {: .no_toc} - - Configure the equipment in your setups +1. Navigate to *Personal Attributes* → *Setups* +2. Click *Add setup* +3. Describe your experimental setups (e.g., "Theta behavior maze", "Surgical table", "Head-fixed 2P Virtual environment", "Home cage") +4. **Configure the equipment in your setups:** + - Go to *Modules* → *Equipment* + - Add devices like "Intan RHD2000", "Behavior Camera", "LED Arrays" + - Associate equipment with appropriate setups ### Set up inventories to track your lab resources {: .no_toc} - - Add consumable stocks to your inventories +1. Go to *Personal Attributes* → *Inventories* +2. Click *Add inventory* +3. Create inventory categories (e.g., "Silicon Probes", "Virus stocks") +4. **Add consumable stocks to your inventories:** + - Navigate to *Modules* → *Consumable stocks* + - Add specific items like probe models, virus batches, drug concentrations + - Link them to appropriate inventories + +{: .note } +> These personal attributes will appear as selectable options when creating sessions, subjects, and experiments. Setting them up first streamlines your data entry process. Now that all your personal attributes are in place, you can continue and add your first subject. @@ -137,6 +162,24 @@ After adding subjects, you can create sessions to record your experimental data. {: .important } > Sessions inherit permissions from their associated projects. Make sure to select the correct project to enable proper access for your team members. +## Already Have Data? Import It Quickly! + +**Don't want to enter everything manually?** If you already have your research data in spreadsheets, our powerful import tool can save you hours of manual data entry! + +**Why use the import tool?** + +- **Save time** - Import dozens or hundreds of records at once +- **Easy to use** - Download a template, fill in your data, and upload +- **Comprehensive** - Supports all major data types (subjects, sessions, procedures, and more!) + +**Get started in 3 simple steps:** + +1. **Download a template** for your data type +2. **Fill in your data** following the provided examples +3. **Upload and review** - we'll check for errors and show you the results + +**Ready to import?** Check out our complete [Import Tool Guide]({{site.baseurl}}/webinterface/import-tool/) for detailed instructions and templates! + ## Build Your Own Experiment BrainSTEM uses Personal Attributes and Modules to document your experimental procedures. Below is a recommended order for setting up your experiment components. @@ -191,3 +234,13 @@ Personal Attributes: {: .important } > This order ensures dependent components are created first. For example, behaviors need setups and behavioral paradigms to be defined first. > Data acquisition are linked to subjects through procedures, and to setups through equipment, which should be created first. + +## Next Steps + +Now that you've completed the basic setup, here are the recommended next steps: + +- **Set up your lab infrastructure**: Follow the [Setting Up Lab Infrastructure tutorial]({{site.baseurl}}/tutorials/setting-up-lab-infrastructure) to properly configure your lab's setups, equipment, inventories, and consumable stocks +- **Learn project management**: Read [Managing Projects]({{site.baseurl}}/tutorials/managing-projects) to understand how to effectively organize and collaborate on research projects +- **Configure team access**: Review [Managing Groups]({{site.baseurl}}/tutorials/managing-groups) to set up proper permissions and team collaboration +- **Plan your experiments**: Explore [Behavioral Paradigms]({{site.baseurl}}/tutorials/behavioral-paradigms) to define standardized experimental protocols for your studies +- **Set up data storage**: Follow [Managing Data Storage]({{site.baseurl}}/tutorials/managing-data-storage) to link your metadata to actual data files for seamless analysis workflows diff --git a/docs/tutorials/managing-data-storage.md b/docs/tutorials/managing-data-storage.md new file mode 100644 index 0000000..2e686fc --- /dev/null +++ b/docs/tutorials/managing-data-storage.md @@ -0,0 +1,379 @@ +--- +layout: default +title: Managing Data Storage +parent: Tutorials +nav_order: 6 +--- + +# Managing Data Storage +{: .no_toc} + +## Table of contents +{: .no_toc .text-delta } + +1. TOC +{:toc} + +## Introduction + +Data storage in BrainSTEM provides a flexible way to link your metadata to actual data files stored on various platforms. This tutorial covers how to set up data storage locations, associate them with sessions, and configure dynamic data linking for seamless access to your raw data. + +## Understanding Data Storage Concepts + +BrainSTEM's data storage system consists of three key components: + +1. **Data Storage Locations**: Define where your data is physically stored (servers, cloud, local drives) +2. **Data Organization**: How your files are structured within the storage location +3. **Data Protocols**: How to access the data (paths, URLs, access methods) + +## Creating a Data Storage Location + +### Step 1: Navigate to Data Storage +{: .no_toc} + +1. From the dashboard, go to *Personal Attributes* → *Data storage* in the left navigation menu +2. Click the *Add data storage* button in the top right corner + +### Step 2: Configure Basic Information +{: .no_toc} + +Fill in the basic details: + +| Field | Description | +|-------|-------------| +| **Name** | Descriptive name for your data storage (e.g., "Lab Server 01", "Cloud Storage Main") | +| **Authenticated groups** | Groups that can access this data storage **(required)** | +| **Description** | Details about stored data types and access requirements | +| **Public access** | Whether this storage should be publicly accessible | + +{: .important } +> Permissions are directly set on data storages. Data storage has four permission levels: membership (read access), contributors, managers, and owners. + +{: .note } +> Choose names that clearly identify the storage location and its purpose. Other lab members will see these names when creating sessions. + +### Step 3: Define Data Organization +{: .no_toc} + +Configure how your data is organized within the storage location. This defines the hierarchy of folders/directories: + +**Available organization elements:** +- Projects +- Collections +- Cohorts +- Subjects +- Sessions +- Years + +**Example organization structures:** + +``` +Subjects → Sessions +``` + +``` +Projects → Subjects → Sessions +``` + +Add organization elements that match your lab's file structure conventions using the available element types. + +### Step 4: Configure Data Storage Protocols +{: .no_toc} + +Set up how to access your data storage. You can configure multiple protocols for the same storage location, e.g.: + +| Storage Type | Protocol | Path Example | Public | +|--------------|----------|--------------|--------| +| **Local Drive** | Local Storage | `/Users/researcher/data/` | No | +| **Network/Server** | SMB/CIFS | `smb://uni.domain/group/data/` | No | +| **Cloud Storage** | Dropbox | `data/myproject` | No | +| **Public Repository** | HTTPS/Web | `https://dandiarchive.org/dandiset/123456/` | Yes | + +{: .important } +> You can configure multiple protocols for the same data storage. This allows flexibility in how different users or systems access the same data. + +## Assigning Data Storage to Sessions + +### During Session Creation +{: .no_toc} + +1. When creating a new session, locate the **Data storage** field +2. Select from your configured data storage locations in the dropdown +3. Optionally, specify a **Name used in storage** - this is the folder/file name used in your actual storage system + +### For Existing Sessions +{: .no_toc} + +1. Navigate to the session you want to update +2. Click the *Edit* button +3. Select or change the **Data storage** field +4. Update the **Name used in storage** if needed +5. Save your changes + +{: .note } +> The "Name used in storage" field helps maintain consistent naming between BrainSTEM and your actual file system, making it easier to locate files programmatically. This is a session-level field, not a data storage field. + +## Dynamic Data Linking + +BrainSTEM's data storage system enables dynamic construction of file paths based on your metadata and organization structure. + +### How Dynamic Linking Works +{: .no_toc} + +When you associate a data storage with a session, BrainSTEM can automatically construct file paths based on: + +1. **Data storage base path**: The root location defined in your data storage protocols +2. **Organization structure**: How you've defined data should be organized +3. **Session metadata**: Project names, subject names, session names, dates + +### Example Dynamic Path Construction +{: .no_toc} + +**Data Storage Configuration:** + +| Field | Value | +|-------|-------| +| **Name** | Lab Server Main | +| **Base path** | `/Volumes/StorageName/Data/` | +| **Organization** | Projects → Subjects → Sessions | + +**Dynamic Path:** +``` +/Volumes/StorageName/Data/{projects}/{subjects}/{sessions}/ +``` + +{: .note } +> Data files should be stored in the session folder based on this organization structure. + +**Session Information:** + +| Field | Value | +|-------|-------| +| **Project** | Memory_Study_2024 | +| **Subject** | Mouse_001 | +| **Session** | Baseline_Recording | +| **Name used in storage** | ses01_baseline | + +**Resulting Path:** +``` +/Volumes/StorageName/Data/Memory_Study_2024/Mouse_001/ses01_baseline/ +``` + +### API Access to Dynamic Paths +{: .no_toc} + +Use the Python or MATLAB API to programmatically access your data paths: + +```python +from brainstem_api_client import BrainstemClient + +client = BrainstemClient() + +# Load session with data storage information +session_response = client.load_model('session', + id='your-session-id', + include=['datastorage']) + +session_data = session_response.json()['sessions'][0] +storage_info = session_data['datastorage'] + +# Access configured protocols and paths +for protocol in storage_info['data_protocols']: + print(f"Protocol: {protocol['protocol']}") + print(f"Base path: {protocol['path']}") + print(f"Public access: {protocol['is_public']}") +``` + +### Constructing Full Paths in Your Analysis Code +{: .no_toc} + +```python +def construct_data_path(session_data): + """ + Construct full path to session data based on BrainSTEM metadata + """ + storage = session_data['datastorage'] + + # Use the first configured protocol by default. + # Update this selection if your storage relies on a specific protocol. + base_path = storage['data_protocols'][0]['path'] + + # Extract organization elements + organization = storage['data_organization'] + + # Build path based on organization structure + path_components = [base_path] + + for element in organization: + if element['elements'] == 'Projects': + path_components.append(session_data['projects'][0]['name']) + elif element['elements'] == 'Subjects': + path_components.append(session_data['subject']['name']) + elif element['elements'] == 'Sessions': + storage_name = session_data.get('name_used_in_storage', + session_data['name']) + path_components.append(storage_name) + + return '/'.join(path_components) + +# Usage +full_path = construct_data_path(session_data) +print(f"Data location: {full_path}") +``` + +## Best Practices + +### Naming Conventions +{: .no_toc} + +- **Data Storage Names**: Use descriptive names that indicate location and purpose +- **Storage Names**: Keep file/folder names consistent with your lab's conventions +- **Avoid Special Characters**: Use underscores or hyphens instead of spaces + +### Organization Strategies +{: .no_toc} + +- **Standardize Early**: Establish organization patterns before accumulating lots of data +- **Document Conventions**: Create lab protocols for file naming and organization +- **Plan for Growth**: Design structures that scale with increasing data volumes + +### Security and Access +{: .no_toc} + +- **Authenticated Groups**: Required - only users in these groups can access the data storage +- **Public/Private Settings**: Configure based on data sensitivity needs +- **Group Matching**: Data storage can only be added to sessions in projects that share at least one authenticated group + +## Troubleshooting Common Issues + +### Cannot Access Data Storage in Sessions +{: .no_toc} + +**Problem**: Data storage doesn't appear in the session dropdown + +**Solutions**: +- Verify you're a member of at least one authenticated group for the data storage + +### Path Construction Errors +{: .no_toc} + +**Problem**: Generated paths don't match actual file locations + +**Solutions**: +- Review your data organization structure configuration +- Verify the base paths in your protocols are correct +- Check that "Name used in storage" matches your actual file/folder names + +### API Access Issues +{: .no_toc} + +**Problem**: Cannot access data storage information via API + +**Solutions**: +- Ensure you're including 'datastorage' in your API include parameters +- Verify your API authentication is working +- Check that you have proper permissions to access the data storage +- Confirm you're using the correct session ID + +## Integration with Analysis Workflows + +### Loading Data in Python +{: .no_toc} + +```python +import os +from brainstem_api_client import BrainstemClient + +def load_session_data(session_id): + """ + Load session metadata and construct data paths + """ + client = BrainstemClient() + + # Get session with data storage info + response = client.load_model('session', + id=session_id, + include=['datastorage', 'dataacquisition']) + + session = response.json()['sessions'][0] + + # Construct data path + data_path = construct_data_path(session) + + # Load actual data files + data_files = [] + if os.path.exists(data_path): + data_files = [f for f in os.listdir(data_path) + if f.endswith(('.dat', '.bin', '.h5'))] + + return { + 'session_metadata': session, + 'data_path': data_path, + 'data_files': data_files + } +``` + +### MATLAB Integration +{: .no_toc} + +```matlab +function data_info = load_session_data(session_id) + % Load session metadata and construct data paths + + % Get session with data storage info + session_data = load_model('model', 'session', ... + 'id', session_id, ... + 'include', {'datastorage', 'dataacquisition'}); + + % Extract data storage information + storage = session_data.datastorage; + + % Use the first configured protocol; adjust if a specific protocol is required. + base_path = storage.data_protocols{1}.path; + + % Construct full path (simplified example) + project_name = session_data.projects{1}.name; + subject_name = session_data.subject.name; + storage_name = session_data.name_used_in_storage; + + full_path = fullfile(base_path, project_name, subject_name, storage_name); + + data_info.session_metadata = session_data; + data_info.data_path = full_path; + data_info.exists = exist(full_path, 'dir') == 7; +end +``` + +This allows users to access data via local mount, cloud backup, or public repository depending on their needs. + +### Custom Organization Patterns +{: .no_toc} + +Configure your organization structure to match your lab's file hierarchy: + +```json +{ + "data_organization": [ + "Projects", + "Subjects", + "Sessions" + ], +} +``` + +**Example resulting path:** +``` +/Volumes/StorageName/Data/Memory_Study/Mouse_001/Baseline_Recording/ +``` + +By following this tutorial, you can effectively manage data storage in BrainSTEM, creating seamless links between your metadata and actual data files for efficient analysis workflows. + +## Next Steps + +With data storage configured, you can now effectively manage and analyze your research data: + +- **Start experimental workflows**: Follow complete experimental documentation with [Electrophysiology Workflow]({{site.baseurl}}/tutorials/electrophysiology-workflow) tutorial that demonstrates data storage integration +- **Use API tools**: Access your data programmatically with [Python API tool]({{site.baseurl}}/api-tools/python-api-tool) or [MATLAB API tool]({{site.baseurl}}/api-tools/matlab-api-tool) for seamless integration with analysis workflows +- **Organize experiments**: Create structured [Behavioral Paradigms]({{site.baseurl}}/tutorials/behavioral-paradigms) that integrate with your data storage system for organized behavioral data +- **Enable collaboration**: Set up proper [Managing Projects]({{site.baseurl}}/tutorials/managing-projects) to share your organized data storage with lab members and collaborators +- **Enable open data**: Make your data publicly accessible through [Sharing Project Publicly]({{site.baseurl}}/tutorials/sharing-project-publicly) to promote open science and collaboration diff --git a/docs/tutorials/managing-groups.md b/docs/tutorials/managing-groups.md index 2e0d0bd..f138c41 100644 --- a/docs/tutorials/managing-groups.md +++ b/docs/tutorials/managing-groups.md @@ -2,95 +2,135 @@ layout: default title: Manage Groups parent: Tutorials -nav_order: 3 +nav_order: 2 --- -# Managing Groups in BrainSTEM +# Managing Groups {: .no_toc} -## Table of contents -{: .no_toc .text-delta } - -1. TOC -{:toc} - ## Introduction Groups in BrainSTEM are essential for organizing lab members and managing permissions. This tutorial will guide you through the process of managing groups effectively, including understanding different permission levels and performing common management tasks. ## Group Permission Levels -BrainSTEM groups have three levels of membership: - -1. **Members (Basic)** - - Can view group resources - - Inherit project permissions assigned to the group - - Can leave the group voluntarily - - Cannot modify group settings or membership - -2. **Managers** - - All Member permissions - - Can add new members - - Can remove existing members - - Can create member invitations - - Cannot modify group settings +BrainSTEM groups have three levels of membership, each with increasing levels of control: -3. **Owners** - - All Manager permissions - - Can associate group with laboratory - - Can rename the group - - Can add/remove managers - - Can modify group settings - - Can grant manager/owner privileges to members +| Permission Level | Capabilities | +|-----------------|-------------| +| **Members** | View group resources
Inherit project permissions assigned to the group
Leave the group voluntarily
**Cannot** modify group settings or membership | +| **Managers** | All Member permissions
Add new members to the group
Remove existing members from the group
Create and manage member invitations
**Cannot** modify group settings or grant privileges | +| **Owners** | All Manager permissions
Associate group with laboratory information
Rename the group
Promote members to Manager or Owner roles
Modify all group settings
Full administrative control | -Visit the [permissions page]({{"datamodel/permissions/"|absolute_url}}) to learn more. +Visit the [permissions page]({{"datamodel/permissions/"|absolute_url}}) to learn more about the BrainSTEM permission system. ## Accessing Group Management +**To access your group's management page:** + 1. Navigate to the Groups page: - Visit [https://www.brainstem.org/private/auth/group/](https://www.brainstem.org/private/auth/group/) - - Or click the Users icon in the top navigation bar and select "Groups" + - Or click the **Users** icon in the top navigation bar and select **Groups** 2. Find your group in the list and click on it to access management options ## Manager Capabilities +As a group manager, you can add and remove members, as well as manage invitations. + ### Adding New Members +**To invite a new member to your group:** + 1. Go to your group's management page 2. Click the **Invite user** button -3. Fill in the required information for the invitation -4. Submit the invitation +3. Select the user from the dropdown list. +4. Click **Submit** to send the invitation + +{: .note } +> The invited user will receive an email notification and must accept the invitation to join the group. ### Checking Invitation Status +**To monitor pending invitations:** + 1. Navigate to [https://www.brainstem.org/private/users/groupmembershipinvitation/](https://www.brainstem.org/private/users/groupmembershipinvitation/) 2. View the status of all pending invitations 3. Monitor which invitations have been accepted or are still pending ### Removing Members +**To remove members from your group:** + 1. Access your group's member list 2. Locate the user(s) you want to remove 3. Check the checkbox next to their name(s) -4. Click the Save button to confirm the removal +4. Click the **Save** button to confirm the removal + +{: .important } +> Removing a member will revoke their access to all group resources. ## Owner Capabilities -### Managing Privileges +As a group owner, you have full administrative control including the ability to promote members, and modify group settings. + +### Managing Member Privileges + +**To promote members to Manager or Owner roles:** 1. Access your group's member list 2. Locate the member(s) you want to modify 3. Check/uncheck the appropriate boxes: - - "Managers" checkbox for manager privileges - - "Owners" checkbox for owner privileges -4. Click Save to apply the changes + - **Managers** checkbox: Grants manager privileges (can add/remove members) + - **Owners** checkbox: Grants owner privileges (full administrative control) +4. Click **Save** to apply the changes + +{: .important } +> Owner privileges should be granted carefully as they provide complete control over the group, including the ability to modify all settings and member roles. ### Associating with Laboratory +**To link your group to laboratory information:** + +Group owners can associate their group with institutional and Principal investigators for better organization and identification. + +1. Access your group's settings page +2. Click the **Add Laboratory** button +3. Fill in the laboratory details: + +| Field | Description | +|-------|-------------| +| **Principal investigators** | Select from group members | +| **Website** | Lab website URL (optional) | +| **Department** | Department name | +| **Institution** | University or research institution name | +| **City** | Laboratory location city | +| **Country** | Laboratory location country | + +4. Click **Save** to apply the association + +{: .note } +> Laboratory associations help other users identify your research group and can be useful for collaboration and data sharing. + ### Renaming the Group -1. Access group settings -2. Locate the group name field -3. Enter the new name -4. Save the changes +**To change your group's name:** + +1. Go to your group's main page +2. Click the **Edit** button +3. Locate the **Name** field at the top of the form +4. Enter the new group name +5. Click **Save** to confirm the changes + +{: .important } +> Choose group names carefully as they should clearly identify your lab or research team. Other users will see this name when collaborating on projects. + +## Next Steps + +After setting up your research groups, you're ready to organize your research: + +- **Create collaborative projects**: Use [Managing Projects]({{site.baseurl}}/tutorials/managing-projects) to organize your research and enable team collaboration +- **Set up your lab infrastructure**: Follow [Setting Up Lab Infrastructure]({{site.baseurl}}/tutorials/setting-up-lab-infrastructure) to configure equipment, setups, and inventories that your group can access +- **Configure data storage**: Set up [Managing Data Storage]({{site.baseurl}}/tutorials/managing-data-storage) locations that your team can use for data management +- **Start documenting experiments**: Begin with [Get Started]({{site.baseurl}}/tutorials/get_started) to create your first subjects and sessions with proper group permissions +- **Plan for data sharing**: Learn about [Share Data Publicly]({{site.baseurl}}/tutorials/sharing-project-publicly) when you're ready to make research openly accessible diff --git a/docs/tutorials/managing-projects.md b/docs/tutorials/managing-projects.md index 51944b5..79eeda0 100644 --- a/docs/tutorials/managing-projects.md +++ b/docs/tutorials/managing-projects.md @@ -2,88 +2,131 @@ layout: default title: Manage Projects parent: Tutorials -nav_order: 4 +nav_order: 3 --- -# Managing Projects in BrainSTEM +# Managing Projects {: .no_toc} -## Table of contents -{: .no_toc .text-delta } - -1. TOC -{:toc} - ## Project Permission Levels -BrainSTEM projects implement four distinct permission levels that can be assigned to both individual users and groups: - -1. **Members (Basic)** - - Read access to project-related subjects, sessions, and modules - - Can view but not modify project content - - Can leave the project voluntarily - -2. **Contributors** - - All Member permissions - - Can create new project-related content - - Can edit existing project content - - Can delete project-related models +BrainSTEM projects implement four distinct permission levels that can be assigned to both individual users and groups. Each level builds on the previous one with additional capabilities: -3. **Managers** - - All Contributor permissions - - Can add new project members - - Can remove project members - - Can add and remove project groups +| Permission Level | Capabilities | +|-----------------|-------------| +| **Members** | View project-related subjects, sessions, and modules
Read-only access to all project content
Leave the project voluntarily
**Cannot** create, edit, or delete content | +| **Contributors** | All Member permissions
Create new project-related content (subjects, sessions, etc.)
Edit existing project content
Delete project-related models
**Cannot** manage membership or project settings | +| **Managers** | All Contributor permissions
Add new individual members to the project
Remove individual members from the project
Add and remove project groups
Manage project invitations
**Cannot** modify core project settings or grant Owner privileges | +| **Owners** | All Manager permissions
Modify project details and metadata
Promote members to Manager or Owner roles
Change project visibility and access settings
Full administrative control over the project | -4. **Owners** - - All Manager permissions - - Can modify project details - - Can add/remove managers - - Full control over project settings - -Visit the [permissions page]({{"datamodel/permissions/"|absolute_url}}) to learn more. +Visit the [permissions page]({{"datamodel/permissions/"|absolute_url}}) to learn more about the BrainSTEM permission system. ## Accessing Project Management +**To access your project's management page:** + 1. Navigate to the Projects page: - Visit [https://www.brainstem.org/private/stem/project/](https://www.brainstem.org/private/stem/project/) - - Select the project you want to manage - - Click the "Manage" button in the top right corner + - Or go to **Stem** → **Projects** in the navigation menu +2. Select the project you want to manage from the list +3. Click the **Manage** button in the top right corner + +{: .note } +> You must have Manager or Owner permissions to access the project management interface. ## Manager Capabilities +As a project manager, you can add and remove members and groups, as well as manage invitations. + ### Adding New Members +**To invite users or groups to your project:** + 1. Go to your project's management page -2. Click the **Invite user** or **Invite group** button -3. Complete the invitation form -4. Submit the invitation +2. Click either: + - **Invite user** button to invite an individual researcher + - **Invite group** button to invite an entire research group +3. Complete the invitation form: + - **Email** (for users): The recipient's email address + - **Group** (for groups): Select the group from the dropdown + - **Permission Level**: Select Member, Contributor, or Manager + - **Message** (optional): Personal note explaining the invitation +4. Click **Submit** to send the invitation + +{: .note } +> When inviting a group, all current and future members of that group will inherit the assigned permission level for this project. ### Tracking Invitations +**To monitor pending project invitations:** + 1. Visit [https://www.brainstem.org/private/users/projectmembershipinvitation/](https://www.brainstem.org/private/users/projectmembershipinvitation/) -2. Monitor pending and accepted invitations +2. Review invitation status: + - **Pending**: Invitation sent but not yet accepted ### Managing Membership -1. Access the project's user/group list +**To remove members or groups from your project:** + +1. Access the project's user/group list in the management page 2. To remove members or groups: - - Select the checkbox next to their name - - Click Save to confirm removal + - Check the checkbox next to their name + - Click **Save** to confirm removal + +{: .important } +> Removing a member or group will immediately revoke their access to the project. ## Owner Capabilities +As a project owner, you have full administrative control including the ability to promote members, modify project settings, and manage all aspects of the project. + ### Setting Permission Levels -1. Access the project member list -2. For each member, you can assign: - - Contributor permissions (Change permissions checkbox) - - Manager permissions (Managers checkbox) - - Owner permissions (Owners checkbox) -3. Click Save to apply changes +**To modify member permission levels:** + +1. Access the project member list in the management page +2. For each member or group, you can assign elevated permissions by checking the appropriate boxes: + +| Checkbox | Permission Granted | Capabilities | +|----------|-------------------|-------------| +| **Change permissions** | Contributor | Create, edit, and delete project content | +| **Managers** | Manager | All Contributor permissions plus member management | +| **Owners** | Owner | Full administrative control | + +3. Click **Save** to apply changes + +{: .important } +> Owner permissions should be granted carefully as they provide complete control over the project, including the ability to modify settings and remove other owners. + +### Managing Project Settings + +**To modify core project details:** + +Project owners can update fundamental project information and configuration: + +1. Go to the project's main page +2. Click the **Edit** button +3. Modify project fields as needed: + +| Field | Description | +|-------|-------------| +| **Name** | Project title visible to all members | +| **Description** | Detailed project overview and goals | +| **Keywords/Tags** | Search terms for project discovery | +| **Public Access** | Toggle project visibility (private/public) | +| **Authenticated Groups** | Default groups with project access | + +4. Click **Save** to apply changes + +{: .note } +> Changes to public access settings will affect project visibility across BrainSTEM. See [Sharing Project Publicly]({{site.baseurl}}/tutorials/sharing-project-publicly) for more details on making projects public. + +## Next Steps -### Project Settings Management +With your projects properly configured, you can start documenting your research: -1. Edit project details -2. Modify project configuration -3. Update project metadata +- **Set up experimental protocols**: Define [Behavioral Paradigms]({{site.baseurl}}/tutorials/behavioral-paradigms) for standardized experimental procedures within your projects +- **Configure data management**: Set up [Managing Data Storage]({{site.baseurl}}/tutorials/managing-data-storage) to link project data to actual files for analysis workflows +- **Start documenting experiments**: Follow complete experimental workflows like [Electrophysiology Workflow]({{site.baseurl}}/tutorials/electrophysiology-workflow) +- **Enable programmatic access**: Master the [Python API tool]({{site.baseurl}}/api-tools/python-api-tool) or [MATLAB API tool]({{site.baseurl}}/api-tools/matlab-api-tool) to programmatically access your project data +- **Enable open science**: Use [Sharing Project Publicly]({{site.baseurl}}/tutorials/sharing-project-publicly) to make your project data accessible to the research community \ No newline at end of file diff --git a/docs/tutorials/matlab-api-tool.md b/docs/tutorials/matlab-api-tool.md deleted file mode 100644 index 2e72547..0000000 --- a/docs/tutorials/matlab-api-tool.md +++ /dev/null @@ -1,116 +0,0 @@ ---- -layout: default -title: Matlab API tool -parent: Tutorials -nav_order: 5 ---- -# Tutorial for the Matlab API tool -{: .no_toc} - -### Installation - -Download the BrainSTEM Matlab API tools repository from GitHub at [github.com/brainstem-org/brainstem_matlab_api_tools](https://github.com/brainstem-org/brainstem_matlab_api_tools) and add it to your Matlab setpath. - -### Setup credentials/token: - -Email and password will be requested - -```m -get_token -``` - -The token is saved to a mat file (`brainstem_authentication.mat`) in the Matlab API tool folder. - -### Loading sessions - -`load_model` can be used to load any model: We just need to set the name of the model. - -```m -output1 = load_model('model','session'); -``` - -We can fetch a single session entry from the loaded models. - -```m -session = output1.sessions(1); -``` - -We can also filter the models by providing cell array with paired filters In this example, it will just load sessions whose name is "yeah". - -```m -output1_1 = load_model('model','session','filter',{'name','yeah'}); -``` - -Loaded models can be sorted by different criteria applying to their fields. In this example, sessions will be sorted in descending ordering according to their name. - -```m -output1_2 = load_model('model','session','sort',{'-name'}); -``` - -In some cases models contain relations with other models, and they can be also loaded with the models if requested. In this example, all the projects, data acquisition, behaviors and manipulations related to each session will be included. - -```m -output1_3 = load_model('model','session','include',{'projects','dataacquisition','behaviors','manipulations'}); -``` - -The list of related data acquisition can be retrieved from the returned dictionary. - -```m -dataacquisition = output1_3.dataacquisition; -``` - -Get all subjects with related procedures and subject state changes - -```m -output1_4 = load_model('model','subject','include',{'procedures'}); -``` - -Get all projects with related subjects and sessions - -```m -output1_5 = load_model('model','project','include',{'sessions','subjects'}); -``` - -All these options can be combined to suit the requirements of the users. For example, we can get only the session that contain the word "Rat" in their name, sorted in descending order by their name and including the related projects. - -```m -output1_6 = load_model('model','session', 'filter',{'name.icontains', 'Rat'}, 'sort',{'-name'}, 'include',{'projects'}); -``` - -### Updating a session - -We can make changes to a model and update it in the database. In this case, we change the description of one of the previously loaded sessions. - -```m -session = output1.sessions(1); -session.description = 'new description'; -output2 = save_model('data',session,'model','session'); -``` - -### Creating a new session - -We can submit a new entry by defining a struct with the required fields. - -```m -session = {}; -session.name = 'New session85'; -session.description = 'new session description'; -session.projects = {'0c894095-2d16-4bde-ad50-c33b7680417d'}; -``` - -Submitting session - -```m -output3 = save_model('data',session,'model','session'); -``` - -### Load public projects - -Request the public data by defining the portal to be public - -```m -output4 = load_model('model','project','portal','public'); -``` - -This tutorial in also available in the [Matlab API tool Github repository](https://github.com/brainstem-org/brainstem_matlab_api_tools). - diff --git a/docs/tutorials/python-api-tool.md b/docs/tutorials/python-api-tool.md deleted file mode 100644 index b3456c2..0000000 --- a/docs/tutorials/python-api-tool.md +++ /dev/null @@ -1,102 +0,0 @@ ---- -layout: default -title: Python API tool -parent: Tutorials -nav_order: 6 ---- -# Tutorial for the Python API tool -{: .no_toc} - -### Installation - -Download the BrainSTEM Python API tools repository from GitHub at [github.com/brainstem-org/brainstem_python_api_tools](https://github.com/brainstem-org/brainstem_python_api_tools). - -You can also install the package using `pip`: - - pip install brainstem_python_api_tools - -### Load the client. User email and password will be requested. -``` -from brainstem_api_client import BrainstemClient - -client = BrainstemClient() -``` - -### Loading sessions. - -load_model can be used to load any model: We just need to pass our settings and the name of the model. - -``` -output1 = client.load_model('session').json() -``` - -We can fetch a single session entry from the loaded models. - -``` -session = output1["sessions"][0] -``` - -We can also filter the models by providing a dictionary where the keys are the fields and the values the possible contents. In this example, it will just load sessions whose name is "yeah". - -``` -output1 = client.load_model('session', filters={'name': 'yeah'}).json() -``` - -Loaded models can be sorted by different criteria applying to their fields. In this example, sessions will be sorted in descending ordering according to their name. - -``` -output1 = client.load_model('session', sort=['-name']).json() -``` - -In some cases models contain relations with other models, and they can be also loaded with the models if requested. In this example, all the projects, data acquisition, behaviors and manipulations related to each session will be included. - -``` -output1 = client.load_model('session', include=['projects', 'dataacquisition', 'behaviors', 'manipulations']).json() -``` - -The list of related data acquisition can be retrieved from the returned dictionary. - -``` -dataacquisition = output1["sessions"][0]["dataacquisition"] -``` - -All these options can be combined to suit the requirements of the users. For example, we can get only the session that contain the word "Rat" in their name, sorted in descending order by their name and including the related projects. - -``` -output1 = client.load_model('session', filters={'name.icontains': 'Rat'}, sort=["-name"], include=['projects']).json() -``` - -### Updating a session - -We can make changes to a model and update it in the database. In this case, we changed the description of one of the previously loaded sessions. - -``` -session = {} -session["description"] = 'new description' -output2 = client.save_model("session", id="0e39c1fd-f413-4142-95f7-f50185e81fa4", data=session).json() -``` - -### Creating a new session - -We can submit a new entry by defining a dictionary with the required fields. - -``` -session = {} -session["name"] = "New Session" -session["projects"] = ['e7475834-7733-48cf-9e3b-f4f2d2d0305a'] -session["description"] = 'description' -``` - -Submitting session -``` -output3 = client.save_model("session", data=session).json() -``` - -### Load public projects - -Request the public data by defining the portal to be public -``` -output4 = client.load_model("project", portal="public").json() -``` - -See the tutorial in the Python Github repository. diff --git a/docs/tutorials/setting-up-lab-infrastructure.md b/docs/tutorials/setting-up-lab-infrastructure.md new file mode 100644 index 0000000..47f7ffe --- /dev/null +++ b/docs/tutorials/setting-up-lab-infrastructure.md @@ -0,0 +1,328 @@ +--- +layout: default +title: Lab Infrastructure +parent: Tutorials +nav_order: 4 +--- + +# Setting Up Lab Infrastructure in BrainSTEM +{: .no_toc} + +## Table of contents +{: .no_toc .text-delta } + +1. TOC +{:toc} + +## Introduction + +Before conducting experiments in BrainSTEM, we recommend you establish your lab's infrastructure. This tutorial covers creating and configuring the foundational elements that support your research workflows: **setups** with **equipment** and **inventories** with **consumable stocks**. Proper infrastructure setup ensures smooth experiment creation and maintains consistency across your lab. + +## Understanding Lab Infrastructure Components + +BrainSTEM's lab infrastructure consists of four interconnected components: + +1. **Setups**: Physical experimental environments (behavioral setup, recording rigs, surgical stations) +2. **Equipment**: Hardware devices used in your setups +3. **Inventories**: Organizational systems for tracking lab resources +4. **Consumable Stocks**: Specific items and materials in your inventories + +These components work together to provide the foundation for documenting procedures, behaviors, data acquisition, and manipulations. + +## Behavioral Paradigms + +Behavioral paradigms are standardized experimental protocols (such as T-maze alternation, open field exploration, etc.) that are performed within a specific setup environment. In BrainSTEM, these are managed under *Personal Attributes → Behavioral Paradigms*. + +For detailed instructions on creating, managing, and best practices for behavioral paradigms, see the dedicated tutorial: [Behavioral Paradigms]({{site.baseurl}}/tutorials/behavioral-paradigms). + +{: .note } +> When planning your infrastructure, ensure that your setups have the correct environment type for the behavioral paradigms you intend to use. + +## Planning Your Infrastructure + +### Before You Begin +{: .no_toc} + +1. **Audit your physical spaces**: List experimental environments in your lab +2. **Inventory your equipment**: Document devices, instruments, and tools +3. **Catalog your supplies**: Identify consumables, probes, drugs, and other materials +4. **Define access permissions**: Determine which group members need access to what resources + +### Infrastructure Hierarchy +{: .no_toc} + +Understanding the relationships between components: + +``` +Setup (Physical Environment) +├── Equipment (Devices in that environment) +│ ├── Hardware devices (specific models/brands) +│ └── Coordinates systems (for positioning) +└── Associated with → Procedures, Behaviors, Data acquisition + +Inventory (Storage/Organization System) +└── Consumable Stocks (Specific items) + └── Associated with → Procedures +``` + +## Creating Setups + +Setups represent your physical experimental environments. Each setup corresponds to a specific location or configuration where experiments are conducted. + +### Step 1: Navigate to Setups +{: .no_toc} + +1. Go to **Personal Attributes** → **Setups** in the left navigation menu +2. Click the **Add setup** button + +### Step 2: Configure Basic Setup Information +{: .no_toc} + +**Fill in the essential details:** + +| Field | Instructions | +|-------|-------------| +| **Name** | Use descriptive, standardized names (e.g., "Behavior Room A", "Surgery Station 1", "2P Imaging Rig", "Head-fixed Linear Track") | +| **Environment Type** | Select appropriate category: Behavioral environments (T-maze, Open field, Barnes maze, Linear track), Recording environments (Head-fixed disc, Homecage, Custom rigs), or Surgical environments (Surgical table) | +| **Authenticated Groups** | Select groups that should have access to this setup | +| **Description** | Provide comprehensive details including: Physical location (room number, building), Setup purpose and capabilities, Special requirements or restrictions, Contact person for technical issues | + +### Step 3: Define Specifications +{: .no_toc} + +Add relevant specifications for your setup. Specifications can be a string or a number. Common examples: + +**Behavioral mazes**: Arm lengths, corridor widths, platform sizes +**Recording rigs**: Stage dimensions, working distances +**Surgical stations**: Table dimensions, reach distances + +**Example specification:** + +| Dimension | Value | +|-----------|-------| +| **Length** | "20 cm" | +| **Width** | "50 cm" | + +### Step 4: Setup Example +{: .no_toc} + +#### Behavioral Setup: "T-maze Behavior Room" +{: .no_toc} + +| Field | Value | +|-------|-------| +| **Name** | T-maze Behavior Room | +| **Setup Type** | T-maze | +| **Description** | Behavioral testing room with automated T-maze system for spatial memory tasks. Located in Room 302B, Neuroscience Building | +| **Public Access** | No | +| **Specifications** | ArmLength: "45 cm", StemLength: "50 cm", CorridorWidth: "10 cm" | + +{: .note } +> Use consistent naming conventions across your lab. This makes setups easier to find and select during experiment creation. + +## Adding Equipment to Setups + +Equipment represents the devices and instruments within your setups. Each piece of equipment belongs to a specific setup and can be used in procedures, data acquisition, and manipulations. + +### Step 1: Navigate to Equipment +{: .no_toc} + +1. Go to **Modules** → **Equipment** +2. Click the **Add equipment** button + +### Step 2: Configure Equipment Basics +{: .no_toc} + +| Field | Instructions | +|-------|-------------| +| **Name** | Use specific, identifiable names (Good: "Intan RHD2000 #1", "Behavior Camera Main", "LED Driver Ch1-4"; Avoid: "Recording device", "Camera", "Light") | +| **Setup** | Select the setup where this equipment is located | +| **Hardware Device** | Choose from the resources database or submit new devices | +| **Coordinates System** | Define how this equipment's position is measured (Common: "External_XYZ_Absolute") | + +### Step 3: Common Equipment Types +{: .no_toc} + +**Data Acquisition**: Recording systems (Intan, Plexon), imaging systems (two-photon microscopes, miniscopes), behavioral cameras, motion tracking systems + +**Stimulation**: Optogenetic systems (LED drivers, lasers), electrical stimulation equipment (stimulus isolators, function generators) + +**Surgical**: Stereotaxic systems, micromanipulators, injection pumps, drill systems + +### Step 4: Equipment Configuration Example +{: .no_toc} + +| Field | Value | +|-------|-------| +| **Name** | Intan RHD2000 System #1 | +| **Setup** | Select: Open Field Arena | +| **Hardware Device** | Intan RHD2000 USB interface board | +| **Type** | DataAcquisitionSystem | + +{: .important } +> Link equipment to appropriate hardware devices in the resources database. If your specific model isn't available, submit it for approval first. + +## Setting Up Inventories + +Inventories help organize and track consumable resources in your lab. They provide structure for managing supplies and ensure proper accounting of materials used in experiments. + +### Step 1: Create Inventory Categories +{: .no_toc} + +Navigate to **Personal Attributes** → **Inventories** and create logical groupings: + +#### Common Inventory Categories +{: .no_toc} + +Organize inventories by material type (Neural Probes, Viral Vectors, Pharmacological Agents), storage location (Freezer -80°C, Refrigerator 4°C), or experiment type (Electrophysiology Supplies, Optogenetics Materials). + +### Step 2: Configure Inventory Details +{: .no_toc} + +**For each inventory:** + +| Field | Instructions | +|-------|-------------| +| **Name** | Clear, descriptive identification (Examples: "Silicon Probe Storage", "AAV Virus Stocks", "Behavioral Supplies RT") | +| **Description** | Include important details: Physical location and access requirements, Storage conditions and handling notes, Responsible person or contact, Special safety considerations | +| **Authenticated Groups** | Groups that can access and modify this inventory | +| **Public Access** | Usually False for lab inventories | + +### Step 3: Inventory Setup Example +{: .no_toc} + +| Field | Value | +|-------|-------| +| **Name** | Neural Electrode Inventory | +| **Description** | Storage for all neural recording electrodes and probes. Located in Room 302, Cabinet B. Handle with anti-static precautions. Contact: Lab Manager for restocking. | +| **Authenticated Groups** | Electrophysiology Team, Lab Managers | + +## Adding Consumable Stocks + +Consumable stocks are the specific items within your inventories. These are the actual materials that get used in procedures and need to be tracked for experimental documentation. + +### Step 1: Navigate to Consumable Stocks +{: .no_toc} + +1. Go to **Modules** → **Consumable stocks** +2. Click **Add consumable stock** + +### Step 2: Configure Stock Information +{: .no_toc} + +| Field Category | Field | Description | +|----------------|--------|-------------| +| **Basic Information** | Name | Specific, identifiable descriptions | +| | Inventory | Select the appropriate inventory category | +| | Consumable | Choose from resources database or submit new items | +| **Tracking Details** | Lot number | Manufacturer's batch identifier | +| | Supplier | Where the item was purchased | +| | Cost | For budget tracking and planning | +| **Storage Information** | Storage location | Specific location within the inventory | +| | Storage conditions | Temperature, humidity, special requirements | +| | Expiration date | If applicable | +| **Usage Information** | Intended use | Specific experimental applications | +| | Notes | Special handling, preparation, or usage instructions | + +### Step 3: Consumable Stock Example +{: .no_toc} + +| Field | Value | +|-------|-------| +| **Type** | SiliconProbe | +| **Inventory** | Select: Neural Electrode Inventory | +| **Consumable** | NeuroNexus A1x32-Poly2 | +| **Acquisition Date** | 2024-01-15 | +| **Storage Location** | Cabinet B, Slot 15, Anti-static tube #A1x32-15 | +| **Storage Conditions** | Room temperature, anti-static protection | +| **Intended Use** | Chronic hippocampal recordings in behaving mice | +| **Cost** | $1,200 | +| **Notes** | Pre-cleaned with 70% ethanol. Handle with non-magnetic forceps only. Sterilize with EtO gas before implantation. | + +## Permission Management + +Proper permission management ensures that the right people have access to the right resources while maintaining security and organization. + +BrainSTEM infrastructure uses four permission levels for setups and inventories and therby inherited to equipment and consumable stocks: + +| Permission Level | Capabilities | +|-----------------|-------------| +| **Members** | View details
Read-only access
**Cannot** modify or create infrastructure | +| **Contributors** | All Member permissions
Create and modify associated equipment and consumable stocks.
Select setups and equipment when creating experiments
Select inventory and consumable stocks when creating experiments
**Cannot** modify setup settings or manage permissions | +| **Managers** | All Contributor permissions
Manage permissions for others
Configure setup specifications
**Cannot** delete infrastructure or transfer ownership | +| **Owners** | All Manager permissions
Full control over setups and equipment
Delete infrastructure components
Transfer ownership to other users
Complete administrative control | + +**Best Practices for Assigning Permissions**: +- **Owners**: PI, lab manager, or senior lab members with equipment responsibility +- **Managers**: Postdocs or experienced graduate students who primarily use specific rigs +- **Contributors**: Lab members who regularly use the equipment for their research +- **Members**: All lab members who might occasionally need read access + +### Group-Based Access +{: .no_toc} + +**Recommended Group Structure**: +``` +Lab Members (Basic access to most lab resources) +├── Graduate Students (Access to setups and equipment for their projects) +├── Postdocs (Broader access, can manage equipment in their research areas) +├── Lab Manager (Administrative access, inventory management) +├── PI and Senior Staff (Full ownership and administrative control) +└── Technique-Specific Groups (Optional, for specialized equipment) + ├── Electrophysiology Users (Access to recording equipment and probes) + ├── Imaging Users (Access to microscopes and imaging supplies) + └── Behavior Users (Access to behavioral setups and paradigms) +``` + +## Best Practices + +### Naming Conventions +{: .no_toc} + +- **Setups**: Include location and be specific (e.g., "Room302_TmazeBehavior", "HeadFixed_2P_VR") +- **Equipment**: Include model and serial number (e.g., "IntanRHD2000_SN1234") +- **Inventories**: Include storage info (e.g., "Probes_RoomTemp", "Viruses_Minus80") +- **Consumable Stocks**: Include lot numbers and concentrations where applicable + +### Documentation and Maintenance +{: .no_toc} + +- Include technical specifications, calibration status, and safety considerations in descriptions +- Specify physical locations, storage conditions, and responsible contacts +- Update consumable stock levels monthly, review equipment quarterly, audit permissions annually +- Document modifications, update permissions when roles change, and archive rather than delete old equipment + +## Integration with Experimental Workflows + +### Using Infrastructure in Procedures +{: .no_toc} + +When creating procedures, your infrastructure components become available: +- **Setups** provide the location context +- **Equipment** specifies tools used (stereotaxic frames, injection systems) +- **Consumable stocks** track materials consumed (probes, viruses, drugs) + +### Using Infrastructure in Sessions +{: .no_toc} + +Your setup determines available: +- **Equipment** for data acquisition modules +- **Behavioral paradigms** compatible with the environment type + +## Troubleshooting + +**Setup not available**: Verify authentication permissions and environment type compatibility + +**Equipment missing from dropdowns**: Confirm correct setup assignment, equipment type alignment, and permission matching + +**Consumable stock access issues**: Check group membership and owner/manager permissions + +## Next Steps + +With your lab infrastructure in place, you're ready to move on to experimental workflows: + +- **Start with experimental design**: Learn about [Behavioral Paradigms]({{site.baseurl}}/tutorials/behavioral-paradigms) to define standardized experimental protocols that use your setups +- **Set up data management**: Configure [Managing Data Storage]({{site.baseurl}}/tutorials/managing-data-storage) to link your metadata to actual data files stored on your systems +- **Organize team access**: Set up [Managing Groups]({{site.baseurl}}/tutorials/managing-groups) and [Managing Projects]({{site.baseurl}}/tutorials/managing-projects) to properly organize team access to your infrastructure +- **Explore complete workflows**: Follow the [Electrophysiology Workflow]({{site.baseurl}}/tutorials/electrophysiology-workflow) tutorial for end-to-end experimental documentation +- **Contribute to the platform**: Use [Submit Resources & Taxonomies]({{site.baseurl}}/tutorials/submit-resource-and-taxonomies) to add any missing equipment or consumable types to the BrainSTEM database diff --git a/docs/tutorials/sharing-project-publicly.md b/docs/tutorials/sharing-project-publicly.md index 70dfa6f..9ae72df 100644 --- a/docs/tutorials/sharing-project-publicly.md +++ b/docs/tutorials/sharing-project-publicly.md @@ -2,10 +2,10 @@ layout: default title: Share data publicly parent: Tutorials -nav_order: 2 +nav_order: 10 --- -# Share Data Publicly +# Sharing Data Publicly in BrainSTEM {: .no_toc} ## Table of contents @@ -16,95 +16,120 @@ nav_order: 2 ## Introduction -BrainSTEM allows you to make your research data publicly accessible to promote open science and collaboration. Public sharing is managed at the project level, with personal attributes requiring individual public sharing settings. +BrainSTEM enables open science by allowing you to make your research data publicly accessible to the scientific community. This tutorial explains how to configure public sharing for projects and their associated personal attributes. -The diagram below illustrates how public permissions are inherited through the BrainSTEM data model: +**Key Concepts:** +- **Projects** are the primary container for public data sharing +- **Personal Attributes** (behavioral paradigms, setups, data storage, inventories) have independent public settings +- Making a project public does NOT automatically make its personal attributes public + +The diagram below illustrates how public permissions flow through the BrainSTEM data model: ![permissions]({{site.baseurl}}/assets/images/permission_public.png) -## Project Public Sharing +{: .important } +> Making a project public does NOT automatically share its associated personal attributes. Behavioral paradigms, setups, data storage, and inventories must be made public separately to ensure complete data accessibility. -{: .warning } -> Important: Making a project public does NOT automatically make its associated personal attributes public. Personal attributes (behavioral paradigms, data storage, setups) must be made public separately. +## Understanding Public Project Sharing + +When you make a project public, the following components become publicly accessible: + +| Component | What Becomes Public | +|-----------|-------------------| +| **Subjects** | All subject details and associated procedures | +| **Sessions** | All session metadata, epoch information and related behavior, data acquisition, and manipulation modules | +| **Collections** | Session collections and groupings | +| **Cohorts** | Subject cohorts and group assignments | -When a project is made public, all associated components become publicly accessible, including: -- Subjects and their procedures -- Sessions and associated modules -- Collections and cohorts -- Related modules (behaviors, data acquisition, manipulations) +{: .note } +> Project-level data becomes public, but the **personal attributes** referenced in sessions (behavioral paradigms, setups, data storage) remain private unless separately shared. -### Making a Project Public +## Making a Project Public {: .important } -> Only project owners can change the public sharing status of a project. +> **Permission Required:** Only project **Owners** can change the public sharing status of a project. -To make a project public: +**To make a project public:** -1. Navigate to the project you want to make public -2. Click the "Edit" button to access the project profile -3. Locate the "Public" checkbox at the bottom of the form -4. When you check the box, a confirmation dialog will appear: -``` -Are you sure you want to make this project public? This will make all related content visible to -the public (subjects, sessions, and modules associated with the project). -``` -5. Click "Confirm" to proceed with making the project public, or "Cancel" to keep it private -6. Click "Save" to apply the changes to your project +1. Navigate to **Stem** → **Projects** +2. Select your project from the list +3. Click the **Edit** button to access the project settings +4. Scroll to the bottom of the form and locate the **Public** checkbox +5. Check the **Public** checkbox +6. A confirmation dialog will appear: + ``` + Are you sure you want to make this project public? This will make all related + content visible to the public (subjects, sessions, and modules associated with + the project). + ``` +7. Click **Confirm** to proceed, or **Cancel** to keep it private +8. Click **Save** to apply your changes -{: .warning } -> Once a project is made public, anyone can access: -> - All subjects in the project -> - All sessions in the project -> - All modules (behaviors, data acquisition, manipulations) associated with these sessions -> - All collections and cohorts within the project - -## Personal Attributes Public Sharing - -When using personal attributes in public projects, certain components must also be individually shared: -- Behavioral paradigms -- Data storage -- Setups -- Inventories - -### Example Scenario -Let's say you have: -1. A project called "Mouse Behavior Study" -2. Sessions in this project involving behavioral experiments -3. A behavioral paradigm called "Open Field Test" used in the behavior module of these sessions -4. A setup called "Behavior Room A" also used in the behavior module of these sessions - -If you make the project public but forget to make the behavioral paradigm and setup public: -- Users can see your project and its components (subjects, sessions, collections) -- When they open a session and try to view its behavior module: - - They can see that there is a behavior module - - But they'll get access errors when trying to view the behavioral paradigm details - - They'll also get access errors when trying to view the setup details -- This creates an incomplete view of your experimental methods, as users can't access the critical details about how the behavior was conducted +**What happens after making a project public:** +- The project appears in the public dashboard at [www.brainstem.org/public](https://www.brainstem.org/public) +- Anyone can browse subjects, sessions, and modules +- Users can access the project via the API without authentication +- All project metadata becomes searchable -{: .important } -> This is why it's essential to make both your project AND its associated personal attributes public. Remember: -> - Behaviors in sessions require both behavioral paradigms and setups to be public -> - Data acquisition in sessions require data storage to be public -> - The relationships between sessions and personal attributes determine which attributes need to be made public +## Sharing Related Personal Attributes + +Personal attributes referenced in your sessions must be shared separately to provide complete experimental context. Without this, users can see that modules exist but cannot access the methodological details. + +### Which Personal Attributes to Share +| Personal Attribute | When to Share Publicly | Why It's Needed | +|-------------------|----------------------|----------------| +| **Behavioral Paradigms** | If your sessions include behavior modules | Users need to understand experimental protocols and task parameters | +| **Setups** | If your sessions reference specific experimental environments | Users need to know equipment configuration and environmental conditions | +| **Data Storage** | If you want users to access raw data files | Users need file paths and access protocols to retrieve actual data | +| **Inventories** | If procedures reference specific consumables | Users need to know exact materials used (probes, drugs, etc.) | -### Making Personal Attributes Public +## Making Personal Attributes Public {: .important } -> Only owners of personal attributes can change their public status. +> **Permission Required:** Only **Owners** of personal attributes can change their public status. -To make a personal attribute public: +**To make any personal attribute public:** -1. Navigate to the personal attribute you want to make public -2. Click the "Edit" button -3. Find the "Public access" checkbox at the bottom of the form -4. Check the box to make the attribute public -5. Click "Save" to apply the changes +1. Navigate to the appropriate section: + - **Personal Attributes** → **Behavioral Paradigms** + - **Personal Attributes** → **Setups** + - **Personal Attributes** → **Data storage** + - **Personal Attributes** → **Inventories** +2. Select the item used in your public sessions +3. Click **Edit** +4. Scroll to the bottom and check the **Public access** checkbox +5. Click **Save** + +{: .note } +> Repeat this process for each personal attribute referenced in your public project's sessions. + +## Best Practices for Public Data Sharing + +### Before Making Data Public + +1. **Review sensitive information**: Ensure no personally identifiable information or proprietary data is included +2. **Complete documentation**: Fill in descriptions for projects, sessions, and personal attributes +3. **Verify data quality**: Check that all metadata is accurate and complete +4. **Test internally**: Have lab members verify access to ensure nothing is missing + +## Revoking Public Access + +If you need to make data private again: + +1. Navigate to the public project or personal attribute +2. Click **Edit** +3. **Uncheck** the **Public** or **Public access** checkbox +4. Click **Save** + +{: .warning } +> Making a project private will immediately remove it from the public dashboard. However, users who have already downloaded or accessed the data will retain their copies. -## Verifying Public Access +## Next Steps -After making items public, verify their accessibility: +After making your data publicly available, consider these enhancements: -1. Visit the public dashboard at [www.brainstem.org/public](https://www.brainstem.org/public) -2. Search for your shared project or attributes -3. Verify that all components are accessible as expected +- **Improve discoverability**: Use [Submit Resources & Taxonomies]({{site.baseurl}}/tutorials/submit-resource-and-taxonomies) to contribute standardized resources that make your data more findable +- **Document protocols thoroughly**: Ensure your [Behavioral Paradigms]({{site.baseurl}}/tutorials/behavioral-paradigms) are well-documented for protocol reproducibility +- **Optimize data access**: Review [Managing Data Storage]({{site.baseurl}}/tutorials/managing-data-storage) to ensure public data has appropriate file access protocols +- **Enable programmatic access**: Share example API scripts in the [Python API tool]({{site.baseurl}}/api-tools/python-api-tool) or [MATLAB API tool]({{site.baseurl}}/api-tools/matlab-api-tool) tutorials diff --git a/docs/tutorials/submit-resource-and-taxonomies.md b/docs/tutorials/submit-resource-and-taxonomies.md index d3541fa..9c332b7 100644 --- a/docs/tutorials/submit-resource-and-taxonomies.md +++ b/docs/tutorials/submit-resource-and-taxonomies.md @@ -1,11 +1,11 @@ --- layout: default -title: Submit Resources & Taxonomies +title: Resources & Taxonomies parent: Tutorials -nav_order: 8 +nav_order: 7 --- -# Submitting Resources & Taxonomies +# Resources & Taxonomies {: .no_toc} ## Table of contents @@ -16,40 +16,153 @@ nav_order: 8 ## Introduction -BrainSTEM maintains controlled lists of resources and taxonomies that can be expanded through user submissions. All submissions go through an approval process to maintain data quality and consistency. +BrainSTEM maintains standardized, controlled vocabularies for resources and taxonomies to ensure data consistency across the platform. As a user, you can contribute new entries when you encounter equipment, consumables, or taxonomic classifications that aren't yet in the system. All submissions undergo an administrative review process to maintain data quality and prevent duplicates. + +This tutorial guides you through submitting new resources and taxonomies, tracking their approval status, and using approved entries in your research documentation. ## Available Resources & Taxonomies -Resources that can be submitted: -- [Consumables]({{"datamodel/resources/consumable/"|absolute_url}}) -- [Hardware devices]({{"datamodel/resources/hardwaredevice/"|absolute_url}}) -- [Suppliers]({{"datamodel/resources/supplier/"|absolute_url}}) +### Resources +{: .no_toc} + +Laboratory materials and equipment that can be submitted: -Taxonomies that can be submitted: -- [Brain regions]({{"datamodel/taxonomies/brainregion/"|absolute_url}}) -- [Setup types]({{"datamodel/taxonomies/setuptype/"|absolute_url}}) -- [Species]({{"datamodel/taxonomies/species/"|absolute_url}}) -- [Strains]({{"datamodel/taxonomies/strain/"|absolute_url}}) +| Resource Type | Description | Examples | +|--------------|-------------|----------| +| **[Consumables]({{"datamodel/resources/consumable/"|absolute_url}})** | Materials used in experiments | Silicon probes, viral vectors, pharmaceutical agents, electrodes | +| **[Hardware devices]({{"datamodel/resources/hardwaredevice/"|absolute_url}})** | Electronic equipment and instruments | Recording systems, cameras, stimulation devices, microscopes | +| **[Suppliers]({{"datamodel/resources/supplier/"|absolute_url}})** | Vendors and manufacturers | Equipment manufacturers, chemical suppliers, research distributors | + +### Taxonomies +{: .no_toc} + +Standardized classifications for biological and experimental categories: + +| Taxonomy Type | Description | Examples | +|--------------|-------------|----------| +| **[Brain regions]({{"datamodel/taxonomies/brainregion/"|absolute_url}})** | Anatomical brain structures | Hippocampus CA1, Primary visual cortex, Basal ganglia | +| **[Setup types]({{"datamodel/taxonomies/setuptype/"|absolute_url}})** | Experimental environment categories | T-maze, Open field, Head-fixed disc, Surgical table | +| **[Species]({{"datamodel/taxonomies/species/"|absolute_url}})** | Organism species | Mus musculus, Rattus norvegicus, Macaca mulatta | +| **[Strains]({{"datamodel/taxonomies/strain/"|absolute_url}})** | Genetic strains within species | C57BL/6J, Long-Evans, Sprague-Dawley | ## Submission Process -1. Navigate to the appropriate section in BrainSTEM -2. Click the *Add* button for the resource or taxonomy type -3. Fill out the submission form with all required fields -4. Submit the form for review +### Before Submitting + +**Essential pre-submission checklist:** + +1. **Search for existing entries:** Check if the resource or taxonomy already exists to avoid duplicates +2. **Gather complete information:** Collect all necessary details: + - For consumables: Product name, catalog number, supplier, specifications + - For hardware devices: Manufacturer, model number, technical specifications, interface details + - For taxonomies: Scientific names, standard references, descriptions +3. **Verify accuracy:** Double-check names, spellings, and technical specifications +4. **Prepare documentation:** Have links to manufacturer pages, datasheets, or scientific references ready + +### Submitting New Entries + +**To submit a new resource or taxonomy:** + +1. Navigate to the appropriate section: + - **Resources**: Go to **Resources** → Select type (**Consumables**, **Hardware devices**, or **Suppliers**) + - **Taxonomies**: Go to **Taxonomies** → Select type (**Brain regions**, **Setup types**, **Species**, or **Strains**) + +2. Click the **Add** button for the resource or taxonomy type + +3. Fill out the submission form with required information: + +| Resource/Taxonomy | Required Fields | Best Practices | +|------------------|----------------|----------------| +| **Consumables** | Product name, consumable type, supplier | Include catalog numbers, specifications, and intended use | +| **Hardware devices** | Manufacturer, model, device type | Provide technical specs, connectivity details, and compatible systems | +| **Suppliers** | Name, website, contact information | Include geographic regions served and product specialties | +| **Setup types** | Environment type name, category | Describe typical experimental use cases | +| **Species** | Scientific name, common name | Follow NCBI Taxonomy database conventions | +| **Strains** | Strain name, species, description | Include genetic background and supplier sources | + +4. Add detailed **Description** field content to help other users understand when to use this entry + +5. Click **Submit** for admin review {: .note } -> Review existing entries before submitting to avoid duplicates. Each resource/taxonomy type has specific required fields detailed in their respective documentation pages. +> Detailed descriptions speed up admin review and help other users select the correct entries. Include specifications, use cases, and any relevant technical details. + +### Submission Guidelines + +**Follow these standards for high-quality submissions:** + +- **Use standard nomenclature:** Follow established scientific naming conventions (e.g., NCBI for species) +- **Be specific and descriptive:** + - Good: "NeuroNexus A1x32-Poly2 silicon probe, 32-channel, 25μm spacing" + - Avoid: "Silicon probe" +- **Include references:** Cite manufacturer datasheets, or scientific classifications +- **Check spelling carefully:** Typos can cause rejection or create confusion in the database +- **Avoid abbreviations without context:** Define acronyms and technical terms in descriptions +- **Provide manufacturer links:** Include URLs to product pages or technical documentation where applicable ## Tracking Submissions -1. Go to [https://www.brainstem.org/private/approvals/](https://www.brainstem.org/private/approvals/) -2. View the status of your submissions -3. Respond to any admin feedback if needed +### Monitoring Approval Status + +**To check the status of your submissions:** + +1. Navigate to [www.brainstem.org/private/approvals/](https://www.brainstem.org/private/approvals/) +2. Review your submissions and their current status: + +| Status | Description | Action Required | +|--------|-------------|----------------| +| **Pending** | Under administrative review | Wait for admin feedback; no action needed | +| **Approved** | Accepted and now available to all users | Entry is live; you can use it in your experiments | +| **Rejected** | Declined with feedback | Review comments and revise submission | +| **Revision Requested** | Needs modifications before approval | Address feedback and resubmit | + +3. Check the **Comments** field for detailed admin feedback or revision requests + +### Responding to Feedback + +**If your submission requires revisions:** + +1. **Read admin comments carefully** to understand specific concerns or needed improvements +2. **Gather additional information** if specifications, references, or details are requested +3. **Make suggested changes** to address each point of feedback +4. **Resubmit** following the same submission process +5. **Add notes** in the submission referencing the original entry and explaining your revisions + +**Common reasons for revision requests:** +- Missing technical specifications or catalog numbers +- Unclear or ambiguous naming +- Duplicate entries (similar item already exists) +- Need for additional references or documentation +- Inconsistent formatting or nomenclature + +{: .tip } +> Responding promptly to admin feedback speeds up the approval process. Most revisions are simple clarifications that can be addressed quickly. ## After Approval -Once approved: -- The entry becomes available in dropdown menus across BrainSTEM -- All users can select and use the approved entry -- The entry can be referenced in new submissions +Once your submission is approved, it becomes part of the BrainSTEM platform: + +**Immediate effects:** +- ✅ The entry appears in dropdown menus across BrainSTEM +- ✅ All users can select and reference the approved entry +- ✅ The entry becomes searchable in the resources/taxonomies database +- ✅ You can use it in your experimental documentation (setups, procedures, sessions) + +**Long-term impact:** +- Your contribution helps standardize data across the research community +- Other labs can use your submissions, improving data interoperability +- The entry can be referenced in future resource submissions +- It becomes part of the permanent BrainSTEM vocabulary + +{: .note } +> Approved entries cannot be deleted but can be marked as deprecated if they become obsolete. Contact administrators if an entry needs updating. + +## Next Steps + +After successfully contributing resources and taxonomies to BrainSTEM, consider these logical progressions: + +- **Set Up Your Lab Infrastructure**: Use your newly approved resources in the [Setting Up Lab Infrastructure tutorial]({{site.baseurl}}/tutorials/setting-up-lab-infrastructure) to configure your own experimental setups with the equipment and suppliers you've contributed +- **Begin Experimental Documentation**: Start documenting your research using the [Get Started tutorial]({{site.baseurl}}/tutorials/get_started) and incorporate your approved resources into real experimental workflows +- **Explore Specific Workflows**: Depending on your research focus, dive into detailed experimental tutorials like [Electrophysiology Workflow]({{site.baseurl}}/tutorials/electrophysiology-workflow) that will use your contributed resources +- **Organize Your Team**: Use [Managing Projects]({{site.baseurl}}/tutorials/managing-projects) and [Managing Groups]({{site.baseurl}}/tutorials/managing-groups) to organize access to your contributed resources across your research team +- **Continue Contributing**: As you discover new equipment, suppliers, or taxonomic categories in your research, continue expanding the BrainSTEM resource database by submitting additional entries diff --git a/docs/tutorials/two-photon-imaging-workflow.md b/docs/tutorials/two-photon-imaging-workflow.md new file mode 100644 index 0000000..7ba1fd2 --- /dev/null +++ b/docs/tutorials/two-photon-imaging-workflow.md @@ -0,0 +1,605 @@ +--- +layout: default +title: 2P Imaging Workflow +parent: Tutorials +nav_order: 9 +nav_exclude: true +--- + +# [Draft] Two-Photon Workflow: Visual Cortex Population Dynamics +{: .no_toc} + +## Introduction - Note: this tutorial is still in a drafting state + +This comprehensive tutorial walks through a complete two-photon calcium imaging experiment in BrainSTEM, from subject preparation to data analysis. We'll document a visual cortex population dynamics study with head-fixed mice, demonstrating how to integrate subjects, procedures, behavioral paradigms, data acquisition, manipulations, and collections in a realistic research workflow. + +**Experimental Design Overview:** +- **Subjects**: Mouse cohort for visual cortex population imaging +- **Setup**: Head-fixed two-photon microscope with visual stimulus presentation +- **Sessions**: Cranial window surgery → Habituation → Visual stimulus sessions +- **Data**: Two-photon calcium imaging, visual stimuli, eye tracking +- **Analysis**: Population dynamics, stimulus response, retinotopy mapping + +{: .note } +> This tutorial assumes you have already set up your lab infrastructure (setups, equipment, inventories) as described in the [Setting Up Lab Infrastructure tutorial]({{site.baseurl}}/tutorials/setting-up-lab-infrastructure). + +## Part A: Project and Subject Setup + +### Step 1: Creating the Research Project + +First, we'll create a project to organize our visual cortex imaging study. + +1. **Navigate to Projects**: + - Go to *Projects* + - Click *Add project* + +**[SCREENSHOT NEEDED: Project creation interface]** + +2. **Configure Project Details**: + +| Field | Value | +|-------|-------| +| **Name** | Visual Cortex Population Dynamics Study | +| **Description** | Investigating population-level neural dynamics in mouse visual cortex using two-photon calcium imaging during visual stimulus presentation. Study focuses on orientation selectivity, population correlations, and stimulus-response relationships in layer 2/3 neurons. | +| **Publications** | Related to ongoing visual cortex research | +| **Tags** | two-photon, visual-cortex, calcium-imaging, mice | +| **Authenticated Groups** | Select groups that should have access to this project | +| **Public Access** | No (keep project private to lab) | + +{: .note } +> For two-photon imaging studies, include information about the calcium indicator, imaging parameters, and visual stimulus paradigms in the project description. Authenticated groups will get contribute permissions. You can add additional groups or change permissions in the manage page: Go to *Project detail page* → *Manage* + +### Step 2: Adding Individual Subjects + +Now we'll add individual mice that will participate in our imaging study. + +1. **Navigate to Add Subject**: + - Go to *Subjects* + - Click *Add subject* + +**[SCREENSHOT NEEDED: Subject creation interface]** + +2. **Subject Configuration Example**: + +| Field | Value | +|-------|-------| +| **Name** | VC_M001 | +| **Projects** | Select: Visual Cortex Population Dynamics Study | +| **Sex** | Male | +| **Species** | Select: Mus musculus | +| **Strain** | Select: C57BL/6J | +| **Description** | Male C57BL/6J mouse for visual cortex two-photon calcium imaging study. Express GCaMP6f via viral injection. | +| **Genetic Line** | Wild type | +| **Birth Date** | 2024-06-15 | +| **Subject Identifier** | Ear punch: Left ear, position 2 | + +**Repeat this process to create additional subjects:** +- VC_M002, VC_M003, VC_M004 (for a cohort of 4 mice) +- Use similar configurations but update IDs and ear punch patterns + +{: .note } +> The additional subjects can be duplicated from the first subject. Go to the *Subject detail page of the first subject* → *Duplicate* +> Fill in the new subject name and click Duplicate. After this you can alter other fields if necessary. + +### Step 2b: Document Initial Housing and Weight + +After creating subjects, document their housing conditions and initial weights using subject logs. + +1. **Create Housing Log**: + - Go to *Modules* → *Subject logs* + - Click *Add subject log* + +**Housing Log Configuration:** +| Field | Value | +|-------|-------| +| **Type** | Housing log | +| **Subject** | Select: VC_M001 | +| **Description** | Standard housing for imaging cohort mice | +| **Start and end time** | 2024-07-01 08:00:00 to 2024-12-31 18:00:00 | +| **Notes** | Single housed post-surgery. Standard mouse cages with enrichment. | + +**Type-specific Details:** +| Field | Value | +|-------|-------| +| **Location** | Animal Facility Room 105 | +| **Cage ID** | Cage_B3_single | +| **Cage type** | Standard mouse cage (32×18×14 cm) | +| **Light cycle** | 12:12 light-dark cycle (lights on 06:00) | +| **Enrichment** | Paper nesting material, plastic igloo | + +2. **Create Initial Weighing Log**: + +**Weighing Log Configuration:** +| Field | Value | +|-------|-------| +| **Type** | Weighing log | +| **Subject** | Select: VC_M001 | +| **Description** | Baseline weight before surgical procedures | +| **Date and time** | 2024-07-01 09:00:00 | +| **Notes** | Pre-surgery baseline weight. Animal healthy and active. | + +**Type-specific Details:** +| Field | Value | +|-------|-------| +| **Weight (grams)** | 24.5 | + +### Step 3: Creating the Experimental Cohort + +After creating individual subjects, group them into a cohort for experimental organization. + +1. **Navigate to Cohorts**: + - Go to *Personal Attributes* → *Cohorts* + - Click *Add cohort* + +**[SCREENSHOT NEEDED: Cohort creation interface]** + +2. **Configure Cohort Details**: + +| Field | Value | +|-------|-------| +| **Name** | VC_Imaging_Cohort_1 | +| **Project** | Select: Visual Cortex Population Dynamics Study | +| **Subjects** | Select: VC_M001, VC_M002, VC_M003, VC_M004 | +| **Description** | First experimental cohort for visual cortex two-photon imaging study. 4 male C57BL/6J mice, age-matched, with viral GCaMP6f expression. Single housed post-surgery, 12:12 light cycle, standard chow ad libitum. | +| **Tags** | visual-cortex, two-photon, gcamp6f, male-mice | + +**[ILLUSTRATION NEEDED: Flowchart showing project → subjects → cohort → experiment relationship]** + +### Step 4: Equipment and Setup Verification + +Verify your two-photon imaging setup includes all necessary equipment: + +**[SCREENSHOT NEEDED: Equipment list view filtered by imaging setup]** + +**Required Equipment**: +- Two-photon microscope system (Bruker/Thorlabs) +- Ti:Sapphire laser (Mai Tai, Chameleon) +- High-speed scanning system (resonant/galvo) +- Photo-multiplier tubes (PMTs) +- Head-fixation apparatus +- Visual stimulus presentation system +- Eye tracking camera +- Temperature control system + +## Part B: Surgical Procedures and Preparation + +### Step 5: Viral Injection Procedure + +Document the viral injection for calcium indicator expression. + +1. **Navigate to Procedures**: + - Go to *Modules* → *Procedures* + - Click *Add procedure* + +**[SCREENSHOT NEEDED: Procedure creation interface]** + +2. **Viral Injection Configuration**: + +**Basic Information:** +| Field | Value | +|-------|-------| +| **Type** | Virus injection | +| **Subject** | Select: VC_M001 | +| **Date and time** | 2024-07-15 10:00:00 | +| **Setup** | Select: Surgical Station | +| **Equipment** | Select: Stereotaxic Frame | +| **Inventory** | Select: Viral Vectors | +| **Consumable Stock** | Select: AAV1-syn-GCaMP6f | +| **Atlas** | Allen Mouse Brain Atlas | +| **Brain Region** | Visual cortex V1 | +| **Coordinate System** | Stereotaxic Bregma-Based Absolute Coordinates | +| **Notes** | Bilateral AAV injection for GCaMP6f expression in V1. Anesthesia: Isoflurane 2.0% for 45 minutes. Recovery: 3 weeks for expression. | + +**Coordinate Details (Type-specific fields):** +| Field | Value | +|-------|-------| +| **AP (mm)** | -3.0 | +| **ML (mm)** | ±2.5 (bilateral) | +| **DV (mm)** | -0.3 (layer 2/3) | +| **AP Angle** | 0° | +| **ML Angle** | 0° | + +**Injection Details:** +| Parameter | Value | +|-----------|-------| +| **Injection volume (nL)** | 500 | +| **Injection rate (nL/min)** | 100 | +| **Titer (units/mL)** | 1×10^13 | +| **Titer unit** | vg/mL | +| **Injection profile** | Bolus Injection | + +### Step 6: Cranial Window Surgery + +Document the cranial window implantation procedure. + +**Cranial Window Procedure Configuration:** + +**Basic Information:** +| Field | Value | +|-------|-------| +| **Type** | Cranial window | +| **Subject** | Select: VC_M001 | +| **Date and time** | 2024-08-05 09:00:00 | +| **Setup** | Select: Surgical Station | +| **Equipment** | Select: Dental Drill, Stereotaxic Frame | +| **Inventory** | Select: Surgical Supplies | +| **Consumable Stock** | Select: Glass coverslip 3mm, Dental cement | +| **Atlas** | Allen Mouse Brain Atlas | +| **Brain Region** | Visual cortex V1 | +| **Notes** | 3mm diameter cranial window over binocular V1. Dura removed, sealed with 3mm glass coverslip. Dental cement head-plate attachment for head-fixation. | +| **Coordinate System** | Stereotaxic Bregma-Based Absolute Coordinates | + +**Window Specifications:** +| Parameter | Value | +|-----------|-------| +| **Method of cranial window** | Surgical drill | +| **Shape of cranial window** | Circular | +| **Length of cranial window (µm)** | 3000 | +| **Width of cranial window (µm)** | 3000 | +| **Thickness of cranial window (µm)** | 150 | +| **Orientation of cranial window** | 0° relative to midline | + +{: .note } +> Allow 1-2 weeks recovery before starting imaging sessions. Monitor for inflammation or coverslip clarity issues. + +**[ILLUSTRATION NEEDED: Cranial window placement diagram showing V1 location]** + +## Part C: Two-Photon Imaging Sessions + +### Step 7: Habituation Session + +Start with habituation to head-fixation and imaging setup. + +1. **Navigate to Sessions**: + - Go to *Modules* → *Sessions* + - Click *Add session* + +**[SCREENSHOT NEEDED: Session creation form for imaging]** + +2. **Habituation Session Configuration**: + +**Basic Session Information:** +| Field | Value | +|-------|-------| +| **Name** | VC_M001_Day1_Habituation | +| **Projects** | Select: Visual Cortex Population Dynamics Study | +| **Description** | Head-fixation habituation session. No imaging, just acclimatization to setup and handling procedures. | +| **Date_time_onset** | 2024-08-19 14:00:00 | +| **Tags** | habituation, head-fixation, baseline | +| **Data_storage** | /data/two-photon/vc_m001/ | + +**Behavioral Paradigm - Head Fixation:** +| Field | Value | +|-------|-------| +| **Name** | Head Fixation Habituation | +| **Environment Type** | Head-fixed setup | +| **Authenticated Groups** | Select your lab groups | +| **Description** | Habituation protocol for head-fixed two-photon imaging. 30-minute sessions with gradual increase in duration. Neutral gray screen, no visual stimuli. Water rewards for calm behavior. | +| **Public Access** | No | + +{: .note } +> Habituation sessions are crucial for reducing stress and motion artifacts in subsequent imaging sessions. + +### Step 8: Visual Stimulus Imaging Session + +The main experimental session combining visual stimulation with two-photon imaging. + +**[SCREENSHOT NEEDED: Two-photon imaging session interface]** + +1. **Visual Paradigm Setup**: + +| Field | Value | +|-------|-------| +| **Name** | Drifting Gratings Visual Stimuli | +| **Environment Type** | Head-fixed visual display | +| **Authenticated Groups** | Select your lab groups | +| **Description** | Drifting sine-wave gratings presented at 8 orientations (0°-315°, 45° steps). Each stimulus 2s duration, 4s inter-stimulus interval. 10 repeats per orientation, randomized presentation order. Screen distance 25cm, stimulus size 20° visual angle. | +| **Public Access** | No | + +2. **Complete Session Configuration**: + +**Basic Session Information:** +| Field | Value | +|-------|-------| +| **Name** | VC_M001_Day1_DriftingGratings | +| **Projects** | Select: Visual Cortex Population Dynamics Study | +| **Description** | Two-photon calcium imaging during drifting grating visual stimuli presentation | +| **Date_time_onset** | 2024-08-20 15:00:00 | +| **Tags** | imaging, visual-stimuli, drifting-gratings | +| **Data_storage** | /data/two-photon/vc_m001/ | + +**Data Acquisition - Two-Photon Imaging:** +| Setting | Value | +|---------|-------| +| **Session** | Select: VC_M001_Day1_DriftingGratings | +| **Type** | Two-Photon Microscopy | +| **Procedures** | Select: Cranial Window, Viral Injection | +| **Setup** | Select: 2P Imaging Rig | +| **Equipment** | Select: Two-Photon Microscope | +| **Notes** | Single plane imaging at 150μm depth in layer 2/3. GCaMP6f expression, 920nm excitation optimal for this indicator. | + +**Type-specific Details:** +| Field | Value | +|---------|-------| +| **Format** | TIFF | +| **Compression** | LZW | +| **Vertical resolution** | 512 | +| **Horizontal resolution** | 512 | +| **Frame rate (Hz)** | 15.5 | +| **Number of frames** | 46500 | +| **Laser power (mW)** | 25 | +| **Excitation wavelength (nm)** | 920 | +| **Imaging depth (μm)** | 150 | +| **Voxel size (X dimension)** | 0.65 | +| **Voxel size (Y dimension)** | 0.65 | +| **Voxel size (Z dimension)** | N/A | + +**Data Acquisition - Visual Stimuli:** +| Setting | Value | +|---------|-------| +| **Session** | Select: VC_M001_Day1_DriftingGratings | +| **Type** | General Time-Series | +| **Procedures** | Select: Cranial Window | +| **Setup** | Select: 2P Imaging Rig | +| **Equipment** | Select: Stimulus Computer | +| **Notes** | Visual stimulus timing and parameters. Drifting gratings, 8 orientations, 10 repeats each. | + +**Type-specific Details:** +| Field | Value | +|---------|-------| +| **Format** | CSV | +| **Data-type** | float32 | +| **Number of channels** | 3 | +| **Sampling rate (Hz)** | 60 | +| **Number of samples** | 108000 | + +**Data Acquisition - Eye Tracking:** +| Setting | Value | +|---------|-------| +| **Session** | Select: VC_M001_Day1_DriftingGratings | +| **Type** | Behavioral Tracking | +| **Procedures** | Select: Cranial Window | +| **Setup** | Select: 2P Imaging Rig | +| **Equipment** | Select: Eye Tracking Camera | +| **Notes** | Pupil tracking during visual stimulation. 60fps capture for pupil dilation analysis. | + +**Type-specific Details:** +| Field | Value | +|---------|-------| +| **Format** | AVI | +| **Compression** | H.264 | +| **Frame rate (Hz)** | 60 | +| **Number of frames** | 108000 | +| **Vertical resolution** | 480 | +| **Horizontal resolution** | 640 | + +**Session Epochs:** +| Epoch Name | Start (min) | Duration (min) | Description | +|------------|-------------|----------------|-------------| +| **Baseline** | 0 | 5 | Gray screen baseline recording | +| **Stimulus_presentation** | 5 | 25 | Drifting grating stimuli (8 orientations × 10 repeats) | +| **Post_stimulus** | 30 | 5 | Post-stimulus baseline recording | + +**[SCREENSHOT NEEDED: Two-photon imaging interface showing real-time calcium signals]** + +### Step 9: Retinotopy Mapping Session + +Create a session for visual field mapping. + +**Retinotopy Session Configuration:** + +**Basic Session Information:** +| Field | Value | +|-------|-------| +| **Name** | VC_M001_Day2_RetinotopyMapping | +| **Projects** | Select: Visual Cortex Population Dynamics Study | +| **Description** | Retinotopic mapping using moving bar stimuli to determine visual field organization | +| **Date_time_onset** | 2024-08-21 15:30:00 | +| **Tags** | retinotopy, mapping, visual-field | +| **Data_storage** | /data/two-photon/vc_m001/ | + +**Visual Paradigm:** +| Field | Value | +|-------|-------| +| **Name** | Retinotopic Bar Mapping | +| **Environment Type** | Head-fixed visual display | +| **Description** | Moving bar stimuli for retinotopic mapping. White bars on black background, 4 directions (horizontal/vertical), 2° width, 0.5°/s speed. Full screen sweep in each direction. | +| **Public Access** | No | + +**Data Acquisition - Two-Photon Imaging:** +| Setting | Value | +|---------|-------| +| **Session** | Select: VC_M001_Day2_RetinotopyMapping | +| **Type** | Two-Photon Microscopy | +| **Procedures** | Select: Cranial Window, Viral Injection | +| **Setup** | Select: 2P Imaging Rig | +| **Equipment** | Select: Two-Photon Microscope | +| **Notes** | Same imaging parameters as orientation session for consistency | + +**Type-specific Details:** +| Field | Value | +|---------|-------| +| **Format** | TIFF | +| **Compression** | LZW | +| **Vertical resolution** | 512 | +| **Horizontal resolution** | 512 | +| **Frame rate (Hz)** | 15.5 | +| **Number of frames** | 31000 | +| **Laser power (mW)** | 28 | +| **Excitation wavelength (nm)** | 920 | +| **Imaging depth (μm)** | 150 | +| **Voxel size (X dimension)** | 0.65 | +| **Voxel size (Y dimension)** | 0.65 | +| **Voxel size (Z dimension)** | N/A | + +## Part D: Advanced Imaging Features + +### Step 10: Multi-Depth Imaging Session + +Document sessions with z-stack or multiple imaging planes. + +**Multi-Depth Session Configuration:** + +**Basic Session Information:** +| Field | Value | +|-------|-------| +| **Name** | VC_M001_Day3_MultiDepth_Orientation | +| **Projects** | Select: Visual Cortex Population Dynamics Study | +| **Description** | Multi-depth two-photon imaging across cortical layers during orientation stimuli | +| **Date_time_onset** | 2024-08-22 14:00:00 | +| **Tags** | multi-depth, layers, orientation | +| **Data_storage** | /data/two-photon/vc_m001/ | + +**Data Acquisition - Layer 2/3:** +| Setting | Value | +|---------|-------| +| **Session** | Select: VC_M001_Day3_MultiDepth_Orientation | +| **Type** | Two-Photon Microscopy | +| **Procedures** | Select: Cranial Window, Viral Injection | +| **Setup** | Select: 2P Imaging Rig | +| **Equipment** | Select: Two-Photon Microscope | +| **Notes** | Layer 2/3 imaging plane | + +**Type-specific Details (Layer 2/3):** +| Field | Value | +|---------|-------| +| **Format** | TIFF | +| **Compression** | LZW | +| **Vertical resolution** | 512 | +| **Horizontal resolution** | 512 | +| **Frame rate (Hz)** | 15.5 | +| **Number of frames** | 46500 | +| **Laser power (mW)** | 25 | +| **Excitation wavelength (nm)** | 920 | +| **Imaging depth (μm)** | 150 | +| **Voxel size (X dimension)** | 0.65 | +| **Voxel size (Y dimension)** | 0.65 | +| **Voxel size (Z dimension)** | N/A | + +**Data Acquisition - Layer 4:** +| Setting | Value | +|---------|-------| +| **Session** | Select: VC_M001_Day3_MultiDepth_Orientation | +| **Type** | Two-Photon Microscopy | +| **Procedures** | Select: Cranial Window, Viral Injection | +| **Setup** | Select: 2P Imaging Rig | +| **Equipment** | Select: Two-Photon Microscope | +| **Notes** | Layer 4 imaging plane - increased laser power for deeper imaging | + +**Type-specific Details (Layer 4):** +| Field | Value | +|---------|-------| +| **Format** | TIFF | +| **Compression** | LZW | +| **Vertical resolution** | 512 | +| **Horizontal resolution** | 512 | +| **Frame rate (Hz)** | 15.5 | +| **Number of frames** | 46500 | +| **Laser power (mW)** | 35 | +| **Excitation wavelength (nm)** | 920 | +| **Imaging depth (μm)** | 250 | +| **Voxel size (X dimension)** | 0.65 | +| **Voxel size (Y dimension)** | 0.65 | +| **Voxel size (Z dimension)** | N/A | + +### Step 11: Adding Optogenetic Manipulations (Optional) + +For experiments combining imaging with optogenetic perturbations. + +**[SCREENSHOT NEEDED: Manipulation configuration for optogenetics]** + +**Optogenetic Manipulation Configuration:** + +**Basic Information:** +| Field | Value | +|-------|-------| +| **Session** | Select: VC_M001_Day4_OptoStim | +| **Type** | Optogenetic stimulation | +| **Procedures** | Select: Viral Injection (ChR2), Cranial Window | +| **Setup** | Select: 2P Imaging Rig | +| **Equipment** | Select: Blue LED System | +| **Notes** | ChR2 activation during visual stimuli presentation | + +**Type-specific Details:** +| Parameter | Value | +|-----------|-------| +| **Power (mW)** | 5.0 | +| **Amplitude (A)** | 0.2 | +| **Stimulation profile** | Biphasic | +| **Duration (s)** | 2.0 | +| **Duty cycle** | 0.5 | +| **Repetitions** | 20 | +| **Wavelength (nm)** | 473 | +| **Closed loop** | false | + +## Part E: Data Organization and Analysis Integration + +### Step 12: Subject Health Monitoring + +Track post-surgical recovery and imaging session performance. + +**[SCREENSHOT NEEDED: Subject log for imaging studies]** + +**Wellness Log Configuration:** +| Field | Value | +|-------|-------| +| **Type** | Wellness log | +| **Subject** | Select: VC_M001 | +| **Description** | Post-surgical recovery and imaging session monitoring | +| **Date and time** | 2024-08-20 18:00:00 | +| **Notes** | Excellent imaging session. Clear cranial window, strong GCaMP signals, minimal motion artifacts. Animal alert and responsive. | + +**Type-specific Details:** +| Field | Value | +|-------|-------| +| **Wellness** | Excellent - active, responsive, clear window | + +### Step 13: Creating Collections for Analysis + +Group related imaging sessions for comprehensive analysis. + +**[SCREENSHOT NEEDED: Collection creation for imaging data]** + +**Collection Configuration:** + +| Field | Value | +|-------|-------| +| **Name** | VC_M001_Orientation_Tuning_Week1 | +| **Project** | Select: Visual Cortex Population Dynamics Study | +| **Sessions** | Select: VC_M001_Day1_DriftingGratings, VC_M001_Day2_RetinotopyMapping, VC_M001_Day3_MultiDepth_Orientation | +| **Description** | First week orientation tuning and retinotopy sessions for VC_M001. Includes single-plane and multi-depth imaging with drifting gratings and retinotopic mapping stimuli. | +| **Tags** | orientation-tuning, retinotopy, calcium-imaging, week1 | + +### Step 14: API Data Access for Analysis + +Access your imaging data programmatically for analysis. + +```python +from brainstem_api_client import BrainstemClient +import numpy as np +import matplotlib.pyplot as plt + +client = BrainstemClient() + +# Get all imaging sessions for a subject +imaging_sessions = client.load_model('session', + filters={'projects__name': 'Visual Cortex Population Dynamics Study'}).json() + +# Get two-photon data acquisition details +imaging_data = client.load_model('dataacquisition', + filters={'session__projects__name': 'Visual Cortex Population Dynamics Study', + 'type': 'Two-Photon Microscopy'}).json() + +# Download specific imaging session data +session_data = client.download_session_data('vc_m001_day1_gratings_id') +``` + +## Next Steps + +After completing this comprehensive two-photon imaging workflow, consider these logical progressions: + +- **Cross-Modal Experiments**: Review the [Electrophysiology Workflow tutorial]({{site.baseurl}}/tutorials/electrophysiology-workflow) to learn how to combine two-photon imaging with simultaneous electrophysiology recordings for comprehensive neural circuit analysis +- **Data Storage Optimization**: Learn about [Managing Data Storage]({{site.baseurl}}/tutorials/managing-data-storage) to efficiently organize and access your large imaging datasets +- **API Integration**: Master the [Python API tool]({{site.baseurl}}/api-tools/python-api-tool) or [MATLAB API tool]({{site.baseurl}}/api-tools/matlab-api-tool) tutorials to programmatically access your imaging data for automated analysis workflows +- **Behavioral Integration**: Explore [Behavioral Paradigms]({{site.baseurl}}/tutorials/behavioral-paradigms) to understand how to combine imaging with more complex behavioral tasks +- **Data Sharing and Collaboration**: Learn about [Sharing Project Publicly]({{site.baseurl}}/tutorials/sharing-project-publicly) to make your imaging datasets available to the research community and enable collaborative analysis diff --git a/docs/webinterface/import-tool.md b/docs/webinterface/import-tool.md new file mode 100644 index 0000000..4582041 --- /dev/null +++ b/docs/webinterface/import-tool.md @@ -0,0 +1,257 @@ +--- +layout: default +title: Import tool +parent: Web interface +nav_order: 6 +--- +# Import tool +{: .no_toc} + +## Table of contents +{: .no_toc .text-delta } + +1. TOC +{:toc} + +## Overview + +The BrainSTEM import tool helps you add data to BrainSTEM by uploading CSV files. Instead of creating CSV files from scratch, you download pre-built templates for each data type, edit them with your data, and upload them. + +**How it works**: Download template → Edit with your data → Upload → Review results + +## Interface Overview + +The import tool interface is organized into three main tabs: **Projects**, **Subjects**, and **Sessions**. Each tab provides templates and import functionality for related data types. + +![Import tool web interface](/assets/images/webinterface/import_tool.png) + +### Main Navigation Tabs +{: .no_toc} + +- **Projects** - Import projects and project-related data +- **Subjects** - Import subjects, procedures, cohorts, and subject logs +- **Sessions** - Import sessions, behaviors, data acquisition, manipulations, and collections + +### Template Downloads +{: .no_toc} + +Each tab displays relevant template download buttons: + +- **Blue template buttons** - Download pre-configured CSV templates for each data type +- Templates include example data and proper column formatting +- Multiple templates available per tab (e.g., Subjects tab has Subjects, Procedures, Subject Logs, and Cohorts templates) + +### Import Process +{: .no_toc} + +1. **Choose CSV File** - Drag and drop or click to select your completed CSV file +2. **Select Import Type** - Click the appropriate purple import button for your data type +3. **Review Results** - Check the import results for any errors or warnings + + +## Getting Started + +### Quick Start +{: .no_toc} + +1. Go to the import tool page ([www.brainstem.org/private/import-tool/](https://www.brainstem.org/private/import-tool/)) +2. Select your data type (e.g., "subjects", "procedures") +3. Click "Download CSV Template" +4. Open the downloaded file and replace the example data with your data +5. Upload the modified CSV file +6. Check the results for any errors + +### Why Use Templates? +{: .no_toc} + +**No mistakes** - Templates have correct column names and formatting + +**Faster** - All required fields are already included + +**Better examples** - Shows you exactly how to format your data + +### Finding Your UUIDs +{: .no_toc} + +Templates use `YOUR_*_ID` placeholders for existing BrainSTEM data. To find these IDs: + +1. Navigate to the relevant admin section (Procedures, Equipment, etc.) +2. Click edit/view on your record +3. Copy the UUID from the URL (long string of letters and numbers) + +## Input conventions +- Column headers must follow the pattern `__` and match exactly; unrecognised columns are ignored. +- CSVs are read with `pandas.read_csv`, so values may be wrapped in double quotes or left bare—both are accepted. +- Trimming happens automatically, but lookups remain case-sensitive. Keep names ≤200 characters to stay within model limits. +- Boolean fields accept `true/false`, `yes/no`, `1/0`, `on/off`, or `enabled/disabled` in any casing (for example `TRUE`, `0`). Blank values resolve to `false`. +- Datetimes accept ISO-8601 (`2024-05-01T14:30`), spaced formats (`2024-05-01 14:30[:45]`), and regional variants (`05/01/2024`, `01/05/2024 14:30`). Dates accept the same patterns without a time component. +- List-style columns must use comma-separated values with no brackets (for example `behavior, imaging, ephys`). The importer splits on commas and trims whitespace. +- JSON columns must contain valid JSON objects; malformed JSON raises a warning and the field is skipped. Example: `{"laserPower": 5, "units": "mW"}`. +- Rich-text fields (project, subject, session, cohort, collection descriptions) are backed by CKEditor and therefore accept HTML. Because inline scripts and event handlers will render, keep inputs to simple formatting (paragraphs, bold, italics, links) to avoid introducing interactive content. + +## Data Type Details + +### Projects +{: .no_toc} + +| Field | Accepted Input | Notes | +| --- | --- | --- | +| `project__name` **\*** | Text (e.g. `Auditory Cortex Project`) | Must be unique across BrainSTEM; importer creates the project when it does not exist. | +| `project__description` | Plain text or limited HTML (e.g. `

Two-photon recordings.

`) | Stored in the rich-text description field; avoid interactive markup. | +| `project__is_public` | Boolean keywords (e.g. `true`, `no`, `0`) | Parsed via `parse_boolean`; blank values become `false`. | +| `project__tags` | Comma-separated tags (e.g. `vision, cortex`) | Converted into an array of individual tag names. | + +### Subjects +{: .no_toc} + +| Field | Accepted Input | Notes | +| --- | --- | --- | +| `project__name` **\*** | Existing project name (e.g. `Auditory Cortex Project`) | Projects must exist before importing subjects. | +| `subject__name` **\*** | Text (e.g. `Mouse_001`) | Subject names are globally unique. | +| `subject__sex` | `M`, `F`, `U` | Defaults to `U` (Unknown) when omitted. | +| `subject__description` | Plain text or limited HTML (e.g. `

WT male.

`) | Stored in the subject's rich-text description; avoid interactive markup. | +| `subject__strain` | Strain name (e.g. `C57BL/6J`) | Looks up the strain taxonomy; missing strains trigger warnings. | +| `subject__genetic_line` | Text (e.g. `Thy1-GCaMP6s`) | Free-form genetic line entry. | +| `subject__subject_identifier` | Text (e.g. `EarTag-123`) | External identifier. | +| `subject__supplier` | Supplier name (e.g. `Charles River Laboratories`) | Looks up existing suppliers; missing suppliers trigger warnings. | +| `subject__birth_date` | Date (e.g. `2023-02-14`) | Parsed and serialised to ISO date. | +| `subject__death_date` | Date (e.g. `2024/03/01`) | Parsed and serialised to ISO date. | +| `subject__tags` | Comma-separated tags (e.g. `training, cohortA`) | Split into a tag list. | +| `subject__name_used_in_storage` | Text (e.g. `mouse_001_data`) | Optional alternate storage label. | + + +### Procedures +{: .no_toc} + +| Field | Accepted Input | Notes | +| --- | --- | --- | +| `subject__name` **\*** | Existing subject name (e.g. `Mouse_001`) | Subjects must exist before importing procedures. | +| `procedure__type` **\*** | Procedure type code (e.g. `VirusInjection`) | Must match an API choice. | +| `procedure__notes` | Plain text (e.g. `Injected 200 nL AAV.`) | Stored as free-form notes (500 characters). | +| `procedure__date_time` | Datetime (e.g. `2024-04-12 09:30`) | Parsed and stored as ISO-8601. | +| `procedure__atlas` | Atlas identifier (e.g. `AllenCCFv3`) | Used to scope brain-region lookups. | +| `procedure__brain_region` | Brain-region UUID or name (e.g. `VISp`) | Resolves within the supplied atlas first; warns and drops the field if unresolved. | +| `procedure__brain_region_acronym` | Atlas acronym (e.g. `VISp`) | Fallback lookup when the name is absent or ambiguous. | +| `procedure__coordinates` | Coordinate system key (e.g. `Stereotaxic_BregmaBrainSurface`) | Defaults when omitted. | +| `procedure__coordinates_data` | JSON object (e.g. `{"apCoordinate": -3.2}`) | Parsed into `coordinates_json`. | +| `procedure__consumablestock` | Consumable stock UUID (e.g. `6ac4...`) | Must reference an existing consumable stock record. | +| `procedure__equipment` | Comma-separated equipment UUIDs (e.g. `7f1b...,d2c9...`) | Each UUID is validated individually; invalid IDs raise warnings. | +| `procedure__details` | JSON object (e.g. `{"virus": "AAV9"}`) | Parsed into `type_json`; schema defaults are loaded if omitted. | + +### Subject Logs +{: .no_toc} + +| Field | Accepted Input | Notes | +| --- | --- | --- | +| `subject__name` **\*** | Existing subject name (e.g. `Mouse_001`) | Subjects must exist before logging entries. | +| `subject_log__type` **\*** | Subject-log type code (e.g. `TrainingSession`) | Example values: `Housing`, `WaterDeprivation`, `Handling`. | +| `subject_log__description` | Plain text (e.g. `Habituation to head-fix.`) | Stored on the log record (500 characters). | +| `subject_log__entry_datetime` | Datetime (e.g. `2024-05-02 10:00`) | Required for single-point logs (anything not in the start/end list below). | +| `subject_log__entry_start_datetime` | Datetime (e.g. `2024-05-02 10:00`) | Required for `Housing`, `FoodDeprivation`, `WaterDeprivation`, `Habituation`, `Handling`, `TrainingSession`. | +| `subject_log__entry_end_datetime` | Datetime (e.g. `2024-05-02 11:00`) | Optional for the start/end list; defaults to one hour after the start when omitted. | +| `subject_log__entry_notes` | Plain text (e.g. `Handled by J.D.`) | Saved with the log entry. | +| `subject_log__entry_data` | JSON object (e.g. `{"task": "Go/No-Go"}`) | Parsed into the entry `details` field. | + +The importer finds or creates a `SubjectLog` for the subject/type pair, then adds a log entry when any entry timestamps are present. Duplicate entries for the same timestamp receive warnings and are counted as existing records. + +### Sessions +{: .no_toc} + +| Field | Accepted Input | Notes | +| --- | --- | --- | +| `project__name` **\*** | Existing project name (e.g. `Auditory Cortex Project`) | Projects must exist before importing sessions. | +| `session__name` **\*** | Text (e.g. `Session_001`) | Session names are globally unique. | +| `session__description` | Plain text or limited HTML (e.g. `

Baseline recording.

`) | Stored in the session's rich-text description; avoid interactive markup. | +| `session__date_time` | Datetime (e.g. `2024-05-10T13:45`) | Parsed and stored as ISO-8601. | +| `session__tags` | Comma-separated tags (e.g. `baseline, imaging`) | Converted into a tag list. | +| `session__data_storage` | Data-storage name (e.g. `NAS Array 01`) | Looks up a matching Personal Attribute › Data Storage record; warnings are emitted when not found. | +| `session_name_in_data_storage` | Text (e.g. `2024-05-10_baseline`) | Optional alias for storage systems. | + + +### Behaviors +{: .no_toc} + +| Field | Accepted Input | Notes | +| --- | --- | --- | +| `session__name` **\*** | Existing session name (e.g. `Session_001`) | Sessions must exist before importing behaviors. | +| `subject__name` **\*** | Existing subject name (e.g. `Mouse_001`) | Subjects must exist before importing behaviors. | +| `behavior__setup` **\*** | Setup UUID (e.g. `d3d0...`) | Validated via the Setup endpoint. | +| `behavior__behavioral_paradigm` **\*** | Behavioral paradigm UUID (e.g. `f6d4...`) | Validated via the Behavioral Paradigm endpoint. | + +### Data Acquisition +{: .no_toc} + +| Field | Accepted Input | Notes | +| --- | --- | --- | +| `session__name` **\*** | Existing session name (e.g. `Session_001`) | Sessions must exist before importing data acquisition records. | +| `data_acquisition__type` **\*** | Data-acquisition type code (e.g. `FiberPhotometry`) | Must match an API choice. | +| `data_acquisition__procedures` **\*** | Comma-separated procedure UUIDs (e.g. `9f50...,aa21...`) | At least one procedure is required; each UUID is validated before linking. | +| `data_acquisition__equipment` | Comma-separated equipment UUIDs (e.g. `7f1b...,d2c9...`) | Each UUID is validated and attached when found. | +| `data_acquisition__notes` | Plain text (e.g. `Laser at 10 mW.`) | Stored as free-form notes (500 characters). | +| `data_acquisition__details` | JSON object (e.g. `{"samplingRate": 30000}`) | Parsed into the `details` field; defaults are pulled from the schema when omitted. | + +### Manipulations +{: .no_toc} + +| Field | Accepted Input | Notes | +| --- | --- | --- | +| `session__name` **\*** | Existing session name (e.g. `Session_001`) | Sessions must exist before importing manipulations. | +| `manipulation__type` **\*** | Manipulation type code (e.g. `OptogeneticStimulation`) | Must match an API choice. | +| `manipulation__procedures` **\*** | Comma-separated procedure UUIDs (e.g. `9f50...,aa21...`) | At least one procedure is required; each UUID is validated. | +| `manipulation__equipment` | Comma-separated equipment UUIDs (e.g. `7f1b...,d2c9...`) | Each UUID is validated before linking. | +| `manipulation__notes` | Plain text (e.g. `Blue light, 20 Hz.`) | Stored as free-form notes (500 characters). | +| `manipulation__details` | JSON object (e.g. `{"pulseWidthMs": 10}`) | Parsed into `type_json`; schema defaults apply when omitted. | + +### Cohorts +{: .no_toc} + +| Field | Accepted Input | Notes | +| --- | --- | --- | +| `project__name` **\*** | Existing project name (e.g. `Auditory Cortex Project`) | Projects must exist before importing cohorts. | +| `cohort__name` **\*** | Text (e.g. `Training Cohort A`) | Names are enforced per project by the API. | +| `cohort__subjects` **\*** | Comma-separated subject names (e.g. `Mouse_001, Mouse_002`) | Each subject must already exist; missing subjects trigger warnings. | +| `cohort__description` | Plain text or limited HTML (e.g. `

For pilot sessions.

`) | Stored in the cohort's rich-text description; avoid interactive markup. | +| `cohort__tags` | Comma-separated tags (e.g. `pilot, cohortA`) | Split into a tag list. | + +### Collections +{: .no_toc} + +| Field | Accepted Input | Notes | +| --- | --- | --- | +| `project__name` **\*** | Existing project name (e.g. `Auditory Cortex Project`) | Projects must exist before importing collections. | +| `collection__name` **\*** | Text (e.g. `Baseline Series`) | Names are enforced per project by the API. | +| `collection__sessions` **\*** | Comma-separated session names (e.g. `Session_001, Session_002`) | Each session must already exist; missing sessions trigger warnings. | +| `collection__description` | Plain text or limited HTML (e.g. `

Includes pilot days.

`) | Stored in the collection's rich-text description; avoid interactive markup. | +| `collection__tags` | Comma-separated tags (e.g. `baseline, imaging`) | Split into a tag list. | + +## Error handling + +- Missing required fields stop the current row and records an error; later rows are still evaluated. +- Lookup failures on required relationships raise errors; missing optional lookups generate warnings and skip only the unresolved relationship. +- Invalid JSON, date, or datetime values raise warnings and the affected field is removed before the API call. +- Duplicate Subject Log entries and other unique-constraint hits are reported as warnings and counted as existing records. + +## Tips for Success + +### Before You Import +{: .no_toc} + +1. **Download the right template** for your data type +2. **Check existing data** to avoid duplicates (especially for projects, subjects, sessions) +3. **Find your UUIDs** if the template needs them (see guide above) +4. **Back up your data** before bulk imports + +### Common Issues +{: .no_toc} + +- **Missing required fields** - The template will show you which fields are required +- **Wrong date format** - Use `YYYY-MM-DD` format +- **Invalid JSON** - Copy the exact format from template examples +- **UUID not found** - Double-check that the ID exists in your system and that you have permissions to use it. + +### After Import +{: .no_toc} + +- Review the results for any errors or warnings +- Fix any issues in your CSV and re-upload +- Check that your data appears correctly in BrainSTEM diff --git a/docs/webinterface/two-factor-authentication.md b/docs/webinterface/two-factor-authentication.md new file mode 100644 index 0000000..07a39b8 --- /dev/null +++ b/docs/webinterface/two-factor-authentication.md @@ -0,0 +1,63 @@ +--- +layout: default +title: Two-Factor Authentication +parent: Web interface +nav_order: 7 +--- + +# Two-Factor Authentication (2FA) + +2FA adds an extra layer of security by requiring a second verification step beyond your password. BrainSTEM supports email verification and authenticator apps. + +## Setup & Methods + +### Quick Setup +1. **Avatar → Two-factor authentication** ([www.brainstem.org/account/two_factor/](https://www.brainstem.org/account/two_factor/)) +2. **Choose method:** Email (instant) or Authenticator app (scan QR code) +3. **Generate backup tokens** after completing your first 2FA login + +### Email Verification +- **Setup:** Click **Activate Email Authentication** +- **Use:** Enter 6-digit code received from `noreply@mg.brainstem.org` (expires in 5 minutes) + +### Authenticator Apps +- **Setup:** Click **Add Authentication App** → Scan QR code → Enter verification code +- **Use:** Enter current 6-digit code from app (refreshes every 30 seconds) +- **Apps:** Google Authenticator, Duo Mobile, Authy, or Microsoft Authenticator + +### Backup Tokens +- **Generate:** After first successful 2FA login → **Generate Tokens** → Save securely +- **Use:** Click **Use Backup Token** when locked out +- **Note:** Single-use only, auto-deleted when all primary methods removed + +## Usage & Management + +### Login Process +1. Enter username/password +2. Enter code from default method (or click alternate method/backup token) + +### Managing Methods +- **Add/Remove:** Visit [www.brainstem.org/account/two_factor/](https://www.brainstem.org/account/two_factor/) +- **Multiple apps:** Add new method before removing old +- **Default:** First enabled method (green badge) +- **Organization restrictions:** Some domains require at least one 2FA method and limit email disable + +### Troubleshooting +- **Expired codes:** Wait 1 minute before requesting new email code +- **Locked out:** Try backup tokens or alternate methods first +- **Need help:** Use [contact form](https://brainstem.org/#Get-in-touch) + +## Security & Best Practices + +### Security Features +- **Rate limiting:** Prevents brute-force attacks +- **Time limits:** Email codes expire in 5 minutes, 1-minute cooldown between requests +- **Auto-cleanup:** Backup tokens are deleted when all primary methods are removed + +### Organization Policies +Some organizations policy required 2FA and prevent complete disabling. You can still switch between email and authenticator methods. + +### Best Practices +- **Recommended:** Use authenticator apps for enhanced security +- **Backup:** Set up multiple methods and save backup tokens offline +- **Safety:** Test new methods before removing old ones, never share backup tokens