The extract and load tests fail when upgrading s3fs past 2026.1.0, e.g
> cd infra/local
> docker compose up -d
> cd warehouses/accelerator_ops/extract_load/opralogweb/
> pytest --catalog-type=rest tests/
File "/home/ubuntu/code/analytics-data-platform/.venv/lib/python3.13/site-packages/s3fs/core.py", line 2587, in commit
write_result = self._call_s3("put_object", **kw, **match)
File "/home/ubuntu/code/analytics-data-platform/.venv/lib/python3.13/site-packages/s3fs/core.py", line 2421, in _call_s3
return self.fs.call_s3(method, self.s3_additional_kwargs, *kwarglist, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/code/analytics-data-platform/.venv/lib/python3.13/site-packages/fsspec/asyn.py", line 118, in wrapper
return sync(self.loop, func, *args, **kwargs)
File "/home/ubuntu/code/analytics-data-platform/.venv/lib/python3.13/site-packages/fsspec/asyn.py", line 103, in sync
raise return_result
File "/home/ubuntu/code/analytics-data-platform/.venv/lib/python3.13/site-packages/fsspec/asyn.py", line 56, in _runner
result[0] = await coro
^^^^^^^^^^
File "/home/ubuntu/code/analytics-data-platform/.venv/lib/python3.13/site-packages/s3fs/core.py", line 451, in _call_s3
return await _error_wrapper(
^^^^^^^^^^^^^^^^^^^^^
method, kwargs=additional_kwargs, retries=self.retries
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/home/ubuntu/code/analytics-data-platform/.venv/lib/python3.13/site-packages/s3fs/core.py", line 214, in _error_wrapper
raise err
PermissionError: Access Denied.
The issues seems to have arisen when fsspec/s3fs#1002 was merged. Reversing the changes locally recovers the original behaviour.
The extract and load tests fail when upgrading s3fs past 2026.1.0, e.g
The issues seems to have arisen when fsspec/s3fs#1002 was merged. Reversing the changes locally recovers the original behaviour.