Conversation
Remove unused data and scripts Roberts param files exist in ibs_projects, and is also not really used in ISF beyond the one ongoing
… top of module, update docs
getattr and setattr work fine tested reproducibility with a single synapse activation simulation for identical results
- not dependant on barrel cortex - not dependent on old cluster paths - still can do the C2-relative redirect Todo: improve implementation, adapt tutorial Todo: finalize refactoring source data files etc
Added explicit default values for ongoing activity rates. useful for future
Network params can now be used in a builder pattern This makes it easy to add synapse dynamics, ongoing activity, evoked activity, and network embeddings to network parameter files in a modular, model system-agnostic way Todo: test if this gives the results we want and deprecate the runfiles Todo: ensure backwards compatible
Gives same results as old method :) Must however be invoked differently Todo: compatibility fix
Instead of having this specific reader for reading in whisker-evoked activity data, I simply pre-process the data here. The reader then only needs to worry about fetching the correct file, which I believe is the most configurable and general way it should work. I kept a notebook outlining how exactly I "augmented" the data. This was not very well documented. Added readme to explain the data as well.
scim.scim has been renamed to scim.udvary2022 to highlight the possibility of adding other methods Hardcoded data paths also moved to user config Top-level maper is a strategy pattern invoking the correct pipeline depending on configuration. Useful for future extensions Todo: to which extent should we pre-process and explore the barrel cortex data? to which extent do we want this specific pipeline to be generally applicable? Should we suggest other researchers to start from syn and con files?
We now just use either `simrun` or a `Simulator` object
Merged
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #467 +/- ##
===========================================
+ Coverage 47.18% 47.28% +0.10%
===========================================
Files 243 246 +3
Lines 22476 22124 -352
===========================================
- Hits 10605 10462 -143
+ Misses 11871 11662 -209 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Merged
bgmeulem
added a commit
that referenced
this pull request
Mar 23, 2026
Should be merged after #467 This PR properly separates L5PT specific code from the rest of ISF `hay` was already implemented as a separate package. Now, it is clearer this is an example implementation of `biophysics_fitting` etc. Additionally moved all code that is L5PT specific to this package. Only generic code is allowed to live in the top-level `biophysics_fitting` package. Removed `BAC_injection` as a `cell_modify_function`. We only ever use this for our own use cases in the form of a stimulus protocol configured with a `Simulator`. Not as a `cell_modify_function` for `simrun`. Removed `scale_apical_morph_86` as a `cell_modify_function` as it is _highly_ specific to one of our morphologies. However, I kept it in the test suite for reproducibility tests
bgmeulem
added a commit
that referenced
this pull request
Apr 2, 2026
…#479) The last parts of ISF that are (highly) specific to barrel cortex after the connectivity refactors (#465 #466) and removing hardcoded structure labels (#467 #469 #478 ) are some older writers for writing out simulation data. These writers hardcode values that are highly specific to barrel cortex or L5PTs for e.g. exportation to Amira, or to define a `cell_modify_function`. We don't use these writers anymore. Data output is handled by `simrun` and `data_base` and generally does not involve AMIRA anymore. In addition Visualization is now handled by the `visualize` module, or exporters to non-proprietary `VTK` or `Blender`. I figured it made the most sense to simply remove these writers. They are for properietary and expensive formats, highly specific to our use case, and even then remain unused.
This was referenced Apr 15, 2026
Closed
bgmeulem
added a commit
that referenced
this pull request
Apr 15, 2026
…c` support, and cell type agnosticism (#487) This PR adds all code implementations to address our revision. # swc support We now support `swc`. ISF pipelines are agnostic to whether a morphology file is `hoc` or `swc`. Reading in and building the morphology is simply handled by `single_cell_parser.io.morphology.read_morphology`. Big thanks to @su-saka and @fkkastrinakis for writing the initial converters, that save me a bunch of time for further integration. We have converters from `swc` to `hoc` and `hoc` to `swc`. Converting `hoc` to `swc` can slightly change connectivity of some sections to the soma, but this is of no impact in ISF as we connect them by default at `x=0.5`. Simulations on both yield almost identical results. I implemented a test for this. variable time step simulations are not fully identical, fixed timesteps simulations are. I am not sure yet why this is the case. Variable time step simulations for `hoc` vs `swc` are virtually indistinguishable, but the time steps are ever so slightly different. See #482 and #476 # Network mapping generalizibility I changed the network mapping procedure so it does not depend anymore on any particular label (e.g. "apical"). Instead, labels are inferred from the morphology file. Additional empirical data should then match these labels (e.g. connections spreadsheet) case-insensitively #478 #467 I extended the network parameters with a `NetworkParamBuilder` class that makes it way more convenient to build network parameters, and makes it explicit which data you add, when, and where. This is explored extensively in the tutorials. #465 Network mapping does not rely anymore on a `NrOfConnections.csv`. This used to be default output of our network mapper. However, the same information is contained in a `.con` file. We now parse it from a `.con` file. This way, ISF is fully independent of any particular network mapping procedure, and simply relies on a `syn` and `con` file. I augmented our dataset for activity data. The correct activity data file + key used to be inferred at runtime, i.e. depending on your recording site and stimulus, it would redirect you to the correct activity data file on the fly. This was very barrel cortex specific. I augmented the activity data by simply generating datasets for all stimuli. The augmentation method is saved as a notebook in `getting_started/functional_constraints/evoked_activity/empirical_recordings`. Now, the key you pass under "whisker" is instead interpreted as a globstring that fetches the correct data file in a preconfigured data directory. The `NetworkParamBuilder` of course also allows you to directly add activity data, independent of any directory configuration. #465 A tutorial now explores all the data requirements for Udvary 2022 #480 The network mapper used in ISF is Udvary 2022, but it should be clear that ISF does not rely on this particular network mapping. I refactored the top-level pipeline so it infers a specific network mapping procedure, and refactored Udvary 2022 as a separate subpackage in `singlecell_input_mapper`. #466 # L5PT mentions I removed all mentions of "L5PT" in the code, except for those methods that are actually specific to L5PTs. In the latter case, it should be clear that this methods provides an example, and should not live in top-level pipelines or modules #469 # Tutorials Updated the intro tutorial so be more targeted at exactly what ISF does. Removed configuration setup. Added more headings, intro and explanation on what we do, how, and why. #484 Updated network mapping tutorial to add API for the SynapseMapper and visualizations. Added auxiliary tutorials. They are hidden from direct view in the webpage, but reflinks work. This is e.g. used in tutorial 2.1 #485 # Config Added config for morphology labels. You can extend a map so labels identified in morphology files are renamed to something else. This is useful for having a consistent internal representation. I had to do this to rename "BasalDendrite" (in network mapping methods) to "Dendrite" Added mechanisms to user config, since the mechanisms are user-specific, technically. # Docs Added documentation on `swc` and `mod` files. Wrote a custom lexer for NMODL :)
bgmeulem
added a commit
that referenced
this pull request
Apr 22, 2026
This is a relatively large refactor to address many reviewer comments, and to make ISF more generally applicable. # swc support We now support `swc`. ISF pipelines are agnostic to whether a morphology file is `hoc` or `swc`. Reading in and building the morphology is simply handled by `single_cell_parser.io.morphology.read_morphology`. Big thanks to @su-saka and @fkkastrinakis for writing the initial converters, that save me a bunch of time for further integration. We have converters from `swc` to `hoc` and `hoc` to `swc`. Converting `hoc` to `swc` can slightly change connectivity of some sections to the soma, but this is of no impact in ISF as we connect them by default at `x=0.5`. Simulations on both yield almost identical results. I implemented a test for this. variable time step simulations are not fully identical, fixed timesteps simulations are. I am not sure yet why this is the case. Variable time step simulations for `hoc` vs `swc` are virtually indistinguishable, but the time steps are ever so slightly different. #482 and #476 # Network mapping generalizibility I changed the network mapping procedure so it does not depend anymore on any particular label (e.g. "apical"). Instead, labels are inferred from the morphology file. Additional empirical data should then match these labels (e.g. connections spreadsheet) case-insensitively #478 #467 `singlecell_input_mapper` has been refactored extensively. It now uses a builder pattern to add network parameters. this makes it way easier to modularize and understand which data is being added, and how (see #465). In addition, the default network embedding strategy can now be configured on a user-level (see #466 ) I extended the network parameters with a `NetworkParamBuilder` class that makes it way more convenient to build network parameters, and makes it explicit which data you add, when, and where. This is explored extensively in the tutorials. #465 Network mapping does not rely anymore on a `NrOfConnections.csv`. This used to be default output of our network mapper. However, the same information is contained in a `.con` file. We now parse it from a `.con` file. This way, ISF is fully independent of any particular network mapping procedure, and simply relies on a `syn` and `con` file. I augmented our dataset for activity data. The correct activity data file + key used to be inferred at runtime, i.e. depending on your recording site and stimulus, it would redirect you to the correct activity data file on the fly. This was very barrel cortex specific. I augmented the activity data by simply generating datasets for all stimuli. The augmentation method is saved as a notebook in `getting_started/functional_constraints/evoked_activity/empirical_recordings`. Now, the key you pass under "whisker" is instead interpreted as a globstring that fetches the correct data file in a preconfigured data directory. The `NetworkParamBuilder` of course also allows you to directly add activity data, independent of any directory configuration. #465 A tutorial now explores all the data requirements for Udvary 2022 #480 The network mapper used in ISF is Udvary 2022, but it should be clear that ISF does not rely on this particular network mapping. I refactored the top-level pipeline so it infers a specific network mapping procedure, and refactored Udvary 2022 as a separate subpackage in `singlecell_input_mapper`. #466 # L5PT mentions I removed all mentions of "L5PT" in the code, except for those methods that are actually specific to L5PTs. In the latter case, it should be clear that this methods provides an example, and should not live in top-level pipelines or modules #469 # Tutorials Updated the intro tutorial so be more targeted at exactly what ISF does. Removed configuration setup. Added more headings, intro and explanation on what we do, how, and why. #484 Updated network mapping tutorial to add API for the SynapseMapper and visualizations. Added auxiliary tutorials. They are hidden from direct view in the webpage, but reflinks work. This is e.g. used in tutorial 2.1 #485 # Config Added config for morphology labels. You can extend a map so labels identified in morphology files are renamed to something else. This is useful for having a consistent internal representation. I had to do this to rename "BasalDendrite" (in network mapping methods) to "Dendrite" Added mechanisms to user config, since the mechanisms are user-specific, technically. # Docs Added documentation on `swc` and `mod` files. Wrote a custom lexer for NMODL :) Extended installation instructions for more info on how ISF handles environments and env variables. Extended documentation on file formats, added data sources to paramfile overview Extended tutorials
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pr should be merged after #465 and #466
This PR removes all mentions of "ApicalDendrite" in those situations where it is overly specific.
This means that it is still ok to use it for examples, some default values, L5PT specific functions, and those functions that specifically target "ApicalDendrite"
Todo: refactor scim (again) to also make it more generic in the network embedding procedure