feat: add envidat data connectors support #1119
Conversation
|
You can access the deployment of this PR at https://renku-ci-ds-1119.dev.renku.ch |
olevski
force-pushed
the
feat-add-envidat-scicat
branch
from
7302f39 to
43d6a9b
Compare
leafty
left a comment
leafty
left a comment
There was a problem hiding this comment.
If this is an "initial support", should it be merged to a feature branch instead?
|
Sorry I meant to convert this to draft right away. But I forgot. Sorry @eikek |
components/renku_data_services/data_connectors/doi/schema_org.py
Outdated
Show resolved
Hide resolved
olevski
force-pushed
the
feat-add-envidat-scicat
branch
2 times, most recently
from
a8217cd to
8b2a0d0
Compare
olevski
force-pushed
the
feat-add-envidat-scicat
branch
6 times, most recently
from
abd7384 to
5deac52
Compare
olevski
force-pushed
the
feat-add-envidat-scicat
branch
4 times, most recently
from
50a5d8a to
d10793b
Compare
olevski
force-pushed
the
feat-add-envidat-scicat
branch
from
afc6c61 to
dc0802d
Compare
olevski
force-pushed
the
feat-add-envidat-scicat
branch
from
dc0802d to
b2adaf7
Compare
|
If you want to test this:
|
| match provider: | ||
| case DatasetProvider.envidat: | ||
| return __get_rclone_s3_config_envidat(dataset) | ||
| # TODO: Add scicat here |
| with contextlib.suppress(errors.ValidationError): | ||
| name_slug = base_models.Slug.from_name(name).value | ||
| target_path = base_models.Slug.from_name(f"{name_slug[:30]}-{target_path}").value | ||
| if metadata: |
There was a problem hiding this comment.
I'm not sure if I overlook something: the check for metadata seems unecessary, that symbol is not used in the body? did you maybe mean if not metadata?
| new_config = payload.model_copy(deep=True) | ||
| new_config.configuration = {} | ||
|
|
||
| envidat_url = "https://envidat.ch/converters-api/internal-dataset/convert/jsonld" |
There was a problem hiding this comment.
This occurs at another place, could it be useful to have a constant/central places for it?
|
|
||
| @classmethod | ||
| def doi_is(cls, doi: str, *args: str) -> Segment: | ||
| """Return slug-is query segment.""" |
There was a problem hiding this comment.
The comment is revealing the copy source :-) (below the same)
| @classmethod | ||
| def from_dict(cls, d: dict[str, Any]) -> DataConnector: | ||
| """Create a Project from a dictionary.""" | ||
| """Create a data connector from a dictionary.""" |
| """Process slug-is segment.""" | ||
| self.__append(st.field_is_any(Fields.slug, ft.values.map(st.from_str))) | ||
|
|
||
| async def visit_doi_is(self, ft: DoiIs) -> None: |
There was a problem hiding this comment.
There are some unit tests - perhaps they could be amended by a few cases now. The test_field_term() in test_user_query_parser could be such a place.
Then there is also a manual that gets included in the api spec description. New fields are automatically listed. You can view it by running test_query_manual.py manually opening the created manual.html. I guess for these two fields, nothing needs to be done, though.
/deploy renku-ui=olevski/feat-modify-global-dc-display
Initial support for envidat datasets.
The following changes are made: