Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 8 additions & 11 deletions .github/workflows/python-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,21 +18,18 @@ on:
types: [created]

jobs:
deploy:
pypi:
name: Publish sdist to Pypi
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v5
with:
python-version: '3.11'
- run: uv build
# Check that basic features work and we didn't miss to include crucial files
- name: Smoke test (wheel)
run: uv run --isolated --no-project -p 3.11 --with dist/*.whl datastock/tests
- name: Smoke test (source distribution)
run: uv run --isolated --no-project -p 3.11 --with dist/*.tar.gz datastock/tests
- run: uv publish --trusted-publishing always
with:
user: __token__
password: ${{ secrets.PYPI_API_TOKEN }}
- run: uv build
# Check that basic features work and we didn't miss to include crucial files
- name: import test (wheel)
run: uv run --isolated --no-project -p 3.11 --with dist/*.whl datastock/tests/prepublish.py
- name: import test (source distribution)
run: uv run --isolated --no-project -p 3.11 --with dist/*.tar.gz datastock/tests/prepublish.py
- run: uv publish -t ${{ secrets.PYPI_API_TOKEN }}
21 changes: 10 additions & 11 deletions CLASSIFIERS.txt
Original file line number Diff line number Diff line change
@@ -1,11 +1,10 @@
"Development Status :: 5 - Production/Stable"
"Intended Audience :: Science/Research"
"Programming Language :: Python :: 3"
"Programming Language :: Python :: 3.6"
"Programming Language :: Python :: 3.7"
"Programming Language :: Python :: 3.8"
"Programming Language :: Python :: 3.9"
"Programming Language :: Python :: 3.10"
"Programming Language :: Python :: 3.11"
"Natural Language :: English"
"License :: OSI Approved :: MIT License"
Development Status :: 5 - Production/Stable
Intended Audience :: Science/Research
Programming Language :: Python :: 3
Programming Language :: Python :: 3.6
Programming Language :: Python :: 3.7
Programming Language :: Python :: 3.8
Programming Language :: Python :: 3.9
Programming Language :: Python :: 3.10
Programming Language :: Python :: 3.11
Natural Language :: English
68 changes: 34 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,15 +41,15 @@ Examples:
Straightforward array visualization:
------------------------------------

```
``
import datastock as ds

# any 1d, 2d or 3d array
aa = np.np.random.random((100, 100, 100))
aa = np.random((100, 100, 100))

# plot interactive figure using shortcut to method
dax = ds.plot_as_array(aa)
```
``

Now do **shift + left clic** on any axes, the rest of the interactive commands are automatically printed in your python console

Expand All @@ -75,7 +75,7 @@ Thanks to dref, the class knows the relationaships between all numpy arrays.
In particular it knows which arrays share the same references / dimensions


```
```python
import numpy as np
import datastock as ds

Expand All @@ -96,24 +96,24 @@ lprof = [(1 + np.cos(t)[:, None]) * x[None, :] for t in lt]
# Populate DataStock

# instanciate
st = ds.DataStock()
coll = ds.DataStock()

# add references (i.e.: store size of each dimension under a unique key)
st.add_ref(key='nc', size=nc)
st.add_ref(key='nx', size=nx)
coll.add_ref(key='nc', size=nc)
coll.add_ref(key='nx', size=nx)
for ii, nt in enumerate(lnt):
st.add_ref(key=f'nt{ii}', size=nt)
coll.add_ref(key=f'nt{ii}', size=nt)

# add data dependening on these references
# you can, optionally, specify units, physical dimensionality (ex: distance, time...), quantity (ex: radius, height, ...) and name (to your liking)

st.add_data(key='x', data=x, dimension='distance', quant='radius', units='m', ref='nx')
coll.add_data(key='x', data=x, dimension='distance', quant='radius', units='m', ref='nx')
for ii, nt in enumerate(lnt):
st.add_data(key=f't{ii}', data=lt[ii], dimension='time', units='s', ref=f'nt{ii}')
st.add_data(key=f'prof{ii}', data=lprof[ii], dimension='velocity', units='m/s', ref=(f'nt{ii}', 'x'))
coll.add_data(key=f't{ii}', data=lt[ii], dimension='time', units='s', ref=f'nt{ii}')
coll.add_data(key=f'prof{ii}', data=lprof[ii], dimension='velocity', units='m/s', ref=(f'nt{ii}', 'x'))

# print in the console the content of st
st
coll
```

<p align="center">
Expand All @@ -124,22 +124,22 @@ You can see that DataStock stores the relationships between each array and each
Specifying explicitly the references is only necessary if there is an ambiguity (i.e.: several references have the same size, like nx and nt2 in our case)


```
``
# plot any array interactively
dax = st.plot_as_array('x')
dax = st.plot_as_array('t0')
dax = st.plot_as_array('prof0')
dax = st.plot_as_array('prof0', keyX='t0', keyY='x', aspect='auto')
```
dax = coll.plot_as_array('x')
dax = coll.plot_as_array('t0')
dax = coll.plot_as_array('prof0')
dax = coll.plot_as_array('prof0', keyX='t0', keyY='x', aspect='auto')
``

You can then decide to store any object category
Let's create a 'campaign' category to store the characteristics of each measurements campaign
and let's add a 'campaign' parameter to each profile data

```
``
# add arbitrary object category as sub-dict of self.dobj
for ii in range(nc):
st.add_obj(
coll.add_obj(
which='campaign',
key=f'c{ii}',
start_date=f'{ii}.04.2022',
Expand All @@ -150,16 +150,16 @@ for ii in range(nc):
)

# create new 'campaign' parameter for data arrays
st.add_param('campaign', which='data')
coll.add_param('campaign', which='data')

# tag each data with its campaign
for ii in range(nc):
st.set_param(which='data', key=f't{ii}', param='campaign', value=f'c{ii}')
st.set_param(which='data', key=f'prof{ii}', param='campaign', value=f'c{ii}')
coll.set_param(which='data', key=f't{ii}', param='campaign', value=f'c{ii}')
coll.set_param(which='data', key=f'prof{ii}', param='campaign', value=f'c{ii}')

# print in the console the content of st
st
```
coll
``

<p align="center">
<img align="middle" src="https://github.com/ToFuProject/datastock/blob/devel/README_figures/DataStock_Obj.png" width="600" alt="Direct 3d array visualization"/>
Expand All @@ -168,31 +168,31 @@ st
DataStock also provides built-in object selection method to allow return all
objects matching a criterion, as lits of int indices, bool indices or keys.

```
In [9]: st.select(which='campaign', index=2, returnas=int)
``
In [9]: coll.select(which='campaign', index=2, returnas=int)
Out[9]: array([2])

# list of 2 => return all matches inside the interval
In [10]: st.select(which='campaign', index=[2, 4], returnas=int)
In [10]: coll.select(which='campaign', index=[2, 4], returnas=int)
Out[10]: array([2, 3, 4])

# tuple of 2 => return all matches outside the interval
In [11]: st.select(which='campaign', index=(2, 4), returnas=int)
In [11]: coll.select(which='campaign', index=(2, 4), returnas=int)
Out[11]: array([0, 1])

# return as keys
In [12]: st.select(which='campaign', index=(2, 4), returnas=str)
In [12]: coll.select(which='campaign', index=(2, 4), returnas=str)
Out[12]: array(['c0', 'c1'], dtype='<U2')

# return as bool indices
In [13]: st.select(which='campaign', index=(2, 4), returnas=bool)
In [13]: coll.select(which='campaign', index=(2, 4), returnas=bool)
Out[13]: array([ True, True, False, False, False])

# You can combine as many constraints as needed
In [17]: st.select(which='campaign', index=[2, 4], operator='Barnaby', returnas=str)
In [17]: coll.select(which='campaign', index=[2, 4], operator='Barnaby', returnas=str)
Out[17]: array(['c3', 'c4'], dtype='<U2')

```
``

You can also decide to sub-class DataStock to implement methods and visualizations specific to your needs

Expand All @@ -205,6 +205,6 @@ DataStock provides built-in methods like:
- size is the total size of all data stored in the instance in bytes
- dsize is a dict with the detail (size for each item in each sub-dict of the instance)
* `save()`: will save the instance
* `ds.load()`: will load a saved instance
* `coll.load()`: will load a saved instance


3 changes: 3 additions & 0 deletions datastock/tests/prepublish.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
print('test import datastock')
import datastock as ds
print('import datastock ok')
16 changes: 9 additions & 7 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,25 +3,28 @@ requires = ["setuptools", "setuptools_scm"]
build-backend = "setuptools.build_meta"


[tool.setuptools.packages.find]
where = ["datastock"]
include = ["datastock*"]
namespaces = false
#[tool.setuptools.packages.find]
#where = ["datastock"]
#include = ["datastock*"]
#namespaces = false

[tool.setuptools]
packages = ["datastock", "datastock.tests"]


[tool.setuptools_scm]
version_file = "datastock/_version.py"


[tool.setuptools.dynamic]
readme = {file = ["README.md"]}
classifiers = {file = ["CLASSIFIERS.txt"]}


[project]
name = "datastock"
readme = "README.md"
license = {text = "MIT"}
dynamic = ["version", "readme", "classifiers"]
dynamic = ["version", "classifiers"]
description = "Generic handler for multiple heterogenous numpy arrays and subclasses"
authors = [
{name = "Didier VEZINET", email = "didier.vezinet@gmail.com"},
Expand All @@ -38,7 +41,6 @@ dependencies = [
"scipy",
"matplotlib",
"PyQt5 ; platform_system != 'Windows'",
# "PySide2; platform_system == 'Windows'",
"astropy",
]

Expand Down
Loading