`irods_client_http_api` crashes with SIGKILL under load during concurrent `parallel_write_init` requests

[mstfdkmn]

HTTP API Version:0.6.0
iRODS Server:5.0.2

In one of our use cases, we found that the API crashes when too many concurrent connections are made using a single account. I reproduced this issue. When I send 40 concurrent parallel_write_init requests to the HTTP API, some succeed but most crash the server.
The daemon hits a memory fault during cleanup (ssl_client_stop / rcDisconnect) and the OS kills it with SIGKILL.
This causes a cascade of 502 errors from the proxy.

Expected Behavior: Error responses, not a dead server. We think the API should never crash. Worst-case, the requests should be rejected, queued, or ignored but it should not lead to an unresponsive service.
Observed Behavior: Server dies. 502 Bad Gateway errors

Steps to reproduce:
POST to /irods-http-api/0.6.0/data-objects
Use op=parallel_write_init, stream-count=2, unique lpath each time

I used the following python script to reproduce:

import asyncio
import aiohttp
import os
import time

URL = "https://icts.irods.t.icts.kuleuven.be/irods-http-api/0.6.0/data-objects"
TOKEN = token
BASE_PATH = "/icts/home/u0137480/test_parallel.bin"

async def send_request(session, idx, headers):
    data = aiohttp.FormData(quote_fields=False)
    data.add_field("op", "parallel_write_init")
    data.add_field("lpath", f"{BASE_PATH}_{idx}_{int(time.time())}")
    data.add_field("stream-count", "2")
    try:
        async with session.post(URL, data=data, headers=headers) as resp:
            return resp.status
    except:
        return "Network Failure"

async def main():
    if not TOKEN:
        print("No ACCESS_TOKEN set")
        return

    headers = {"Authorization": f"Bearer {TOKEN}", "Accept": "application/json"}
    connector = aiohttp.TCPConnector(limit=None, force_close=True)

    async with aiohttp.ClientSession(connector=connector) as session:
        tasks = [send_request(session, i, headers) for i in range(40)] # FYI: during my test changing this number to 33 worked normal
        results = await asyncio.gather(*tasks)

        counts = {r: results.count(r) for r in set(results)}
        for status, n in counts.items():
            print(f"{status}: {n}")

if __name__ == "__main__":
    asyncio.run(main())

Log Traces:

[debug] op_parallel_write_init: replica token=[48de90c5-294a-49b7-8f54-188bee041f97], leaf resource name=[netapp]
[error] op_parallel_write_init: INVALID_HANDLE: Could not open output stream to [/icts/home/u0137480/test_parallel.bin_37].
[stderr] [-] /irods_source/plugins/network/src/ssl.cpp:522:irods::error ssl_client_stop(...) : status [SSL_SHUTDOWN_ERROR] -- message [error shutting down the SSL connection]
[stderr] remote addresses: 172.17.0.18 ERROR: [-] /irods_source/lib/core/src/rcConnect.cpp:280:int rcDisconnect(rcComm_t *) : status [SSL_SHUTDOWN_ERROR]
WARN exited: http-api (terminated by SIGKILL; not expected)
web: proxy error: EOF

some part of actual logs:

[172.17.0.18] op_parallel_write_init: replica token=[5cf7ad44-0392-4e59-b69f-7c613214110d], replica number=[0], leaf resource name=[netapp]"}
[2026-05-15T14:08:33.616Z][icts-t-cloud-rdm-hev-1] {"processname": "http-api", "groupname": "http-api", "pid": "258122", "channel": "stdout", "msg": "[2026-05-15 16:08:33.612] [P:258122] [debug] [T:258160] [172.17.0.18] op_parallel_write_init: (init) Parallel Write Handle = [595484fa-7ad9-435f-8ad3-c3898925189c]."}
[2026-05-15T14:08:33.630Z][icts-t-cloud-rdm-hev-1] {"processname": "http-api", "groupname": "http-api", "pid": "258122", "channel": "stdout", "msg": "[2026-05-15 16:08:33.621] [P:258122] [debug] [T:258155] [172.17.0.18] op_parallel_write_init: replica token=[e69c62bb-7200-4a9d-948b-f18ab0601e22], replica number=[0], leaf resource name=[netapp]"}
[2026-05-15T14:08:33.652Z][icts-t-cloud-rdm-hev-1] {"processname": "web", "groupname": "web", "pid": "231", "channel": "stdout", "web": {"time": "2026-05-15T16:08:33.648661699+02:00", "id": "", "remote_ip": "2a02:2c40:0:451:8a::5", "host": "icts.irods.t.icts.kuleuven.be", "method": "POST", "uri": "/irods-http-api/0.6.0/data-objects", "user_agent": "Python/3.12 aiohttp/3.13.5", "status": 200, "error": "", "latency": 3994196893, "latency_human": "3.994196893s", "bytes_in": 103, "bytes_out": 99}}
[2026-05-15T14:08:33.653Z][icts-t-cloud-rdm-hev-1] {"processname": "http-api", "groupname": "http-api", "pid": "258122", "channel": "stdout", "msg": "[2026-05-15 16:08:33.646] [P:258122] [debug] [T:258164] [172.17.0.18] op_parallel_write_init: (init) Parallel Write Handle = [cea8ff47-f35b-49c4-9bc4-0e8558bcfb03]."}
[2026-05-15T14:08:33.712Z][icts-t-cloud-rdm-hev-1] {"processname": "http-api", "groupname": "http-api", "pid": "258122", "channel": "stdout", "msg": "[2026-05-15 16:08:33.711] [P:258122] [debug] [T:258170] [172.17.0.18] op_parallel_write_init: replica token=[d293e12e-9554-4668-b159-18e208bf8e05], replica number=[0], leaf resource name=[netapp]"}
[2026-05-15T14:08:33.807Z][icts-t-cloud-rdm-hev-1] {"processname": "http-api", "groupname": "http-api", "pid": "258122", "channel": "stdout", "msg": "[2026-05-15 16:08:33.806] [P:258122] [debug] [T:258158] [172.17.0.18] op_parallel_write_init: replica token=[29ee30c1-17bb-46ab-9543-14dd81bce1b7], replica number=[0], leaf resource name=[netapp]"}
[2026-05-15T14:08:33.880Z][icts-t-cloud-rdm-hev-1] {"processname": "http-api", "groupname": "http-api", "pid": "258122", "channel": "stdout", "msg": "[2026-05-15 16:08:33.879] [P:258122] [debug] [T:258157] [172.17.0.18] op_parallel_write_init: replica token=[155f1d16-c02b-455b-9874-531b92293be9], replica number=[0], leaf resource name=[netapp]"}
[2026-05-15T14:08:33.900Z][icts-t-cloud-rdm-hev-1] {"processname": "http-api", "groupname": "http-api", "pid": "258122", "channel": "stdout", "msg": "[2026-05-15 16:08:33.899] [P:258122] [debug] [T:258162] [172.17.0.18] op_parallel_write_init: replica token=[fd7595b7-3ac2-4c65-9992-bfc790d76510], replica number=[0], leaf resource name=[netapp]"}
[2026-05-15T14:08:33.906Z][icts-t-cloud-rdm-hev-1] {"processname": "proxy", "groupname": "proxy", "pid": "257538", "channel": "stderr", "msg": "2026/05/15 16:08:33 error forwarding connection from [2a02:2c40:0:451:8a::5]:40000 at port 1247: out side closed a second ago"}
[2026-05-15T14:08:33.911Z][icts-t-cloud-rdm-hev-1] {"processname": "http-api", "groupname": "http-api", "pid": "258122", "channel": "stdout", "msg": "[2026-05-15 16:08:33.903] [P:258122] [debug] [T:258169] [172.17.0.18] op_parallel_write_init: (init) Parallel Write Handle = [e94def32-768b-4ad6-94d7-6bd0fd779010]."}
[2026-05-15T14:08:33.916Z][icts-t-cloud-rdm-hev-1] {"processname": "web", "groupname": "web", "pid": "231", "channel": "stdout", "web": {"time": "2026-05-15T16:08:33.904396969+02:00", "id": "", "remote_ip": "2a02:2c40:0:451:89::2", "host": "icts.irods.t.icts.kuleuven.be", "method": "POST", "uri": "/irods-http-api/0.6.0/data-objects", "user_agent": "Python/3.12 aiohttp/3.13.5", "status": 200, "error": "", "latency": 4249352169, "latency_human": "4.249352169s", "bytes_in": 103, "bytes_out": 99}}
[2026-05-15T14:08:34.026Z][icts-t-cloud-rdm-hev-1] {"processname": "http-api", "groupname": "http-api", "pid": "258122", "channel": "stdout", "msg": "[2026-05-15 16:08:34.025] [P:258122] [debug] [T:258165] [172.17.0.18] op_parallel_write_init: replica token=[6c46b47f-dfad-461a-bea2-cfeba5a8bfe3], replica number=[0], leaf resource name=[netapp]"}
[2026-05-15T14:08:34.032Z][icts-t-cloud-rdm-hev-1] {"processname": "proxy", "groupname": "proxy", "pid": "257538", "channel": "stderr", "msg": "2026/05/15 16:08:34 error forwarding connection from [2a02:2c40:0:451:8a::5]:33110 at port 1247: out side closed a second ago"}
[2026-05-15T14:08:34.093Z][icts-t-cloud-rdm-hev-1] {"processname": "proxy", "groupname": "proxy", "pid": "257538", "channel": "stderr", "msg": "2026/05/15 16:08:34 error forwarding connection from [2a02:2c40:0:451:8a::5]:35218 at port 1247: out side closed a second ago"}
[2026-05-15T14:08:34.163Z][icts-t-cloud-rdm-hev-1] {"processname": "proxy", "groupname": "proxy", "pid": "257538", "channel": "stderr", "msg": "2026/05/15 16:08:34 error forwarding connection from [2a02:2c40:0:451:8a::5]:34220 at port 1247: out side closed a second ago"}
[2026-05-15T14:08:34.240Z][icts-t-cloud-rdm-hev-1] {"processname": "http-api", "groupname": "http-api", "pid": "258122", "channel": "stdout", "msg": "[2026-05-15 16:08:34.239] [P:258122] [debug] [T:258161] [172.17.0.18] op_parallel_write_init: replica token=[c450934d-bbfd-4416-b0b8-71e77f2be779], replica number=[0], leaf resource name=[netapp]"}
[2026-05-15T14:08:34.253Z][icts-t-cloud-rdm-hev-1] {"processname": "proxy", "groupname": "proxy", "pid": "257538", "channel": "stderr", "msg": "2026/05/15 16:08:34 error forwarding connection from [2a02:2c40:0:451:8a::5]:44836 at port 1247: out side closed a second ago"}
[2026-05-15T14:08:34.316Z][icts-t-cloud-rdm-hev-1] {"processname": "proxy", "groupname": "proxy", "pid": "257538", "channel": "stderr", "msg": "2026/05/15 16:08:34 error forwarding connection from [2a02:2c40:0:451:8a::5]:59868 at port 1247: out side closed a second ago"}
[2026-05-15T14:08:34.367Z][icts-t-cloud-rdm-hev-1] {"processname": "proxy", "groupname": "proxy", "pid": "257538", "channel": "stderr", "msg": "2026/05/15 16:08:34 error forwarding connection from [2a02:2c40:0:451:89::2]:50812 at port 1247: out side closed a second ago"}
[2026-05-15T14:08:34.447Z][icts-t-cloud-rdm-hev-1] {"processname": "proxy", "groupname": "proxy", "pid": "257538", "channel": "stderr", "msg": "2026/05/15 16:08:34 received connection from [2a02:2c40:0:451:89::1]:33374 at port 1247"}
[2026-05-15T14:08:34.515Z][icts-t-cloud-rdm-hev-1] {"processname": "proxy", "groupname": "proxy", "pid": "257538", "channel": "stderr", "msg": "2026/05/15 16:08:34 error forwarding connection from [2a02:2c40:0:451:8a::5]:35000 at port 1247: out side closed a second ago"}
[2026-05-15T14:08:34.757Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -1 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259300, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.757Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259295, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.757Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259270, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.757Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259296, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.757Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259297, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.757Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259308, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.757Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "[-]\t/irods_source/server/core/src/rsApiHandler.cpp:595:int readAndProcClientMsg(rsComm_t *, int) :  status [Unknown iRODS error]  errno [Operation not permitted] -- message [failed to call 'read header']\n\t[-]\t/irods_source/lib/core/src/sockComm.cpp:219:irods::error readMsgHeader(irods::network_object_ptr, msgHeader_t *, struct timeval *) :  status [Unknown iRODS error]  errno [Operation not permitted] -- message [failed to call 'read header']\n\t\t[-]\t/irods_source/plugins/network/src/ssl.cpp:464:irods::error ssl_read_msg_header(irods::plugin_context &, void *, struct timeval *) :  status [Unknown iRODS error]  errno [Operation not permitted] -- message [read -1 expected 4]\n\n", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259314, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.757Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259302, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.757Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -1 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259319, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.757Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259300, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.757Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -1 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259314, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.757Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259318, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.757Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259301, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.757Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -1 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259319, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.757Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259312, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.757Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -1 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259314, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.757Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259313, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.757Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259319, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.757Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259314, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.762Z][icts-t-cloud-rdm-hev-1] 2026-05-15 16:08:34,761 INFO spawned: 'http-api' with pid 259341
[2026-05-15T14:08:34.762Z][icts-t-cloud-rdm-hev-1] 2026-05-15 16:08:34,758 WARN exited: http-api (terminated by SIGKILL; not expected)
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -1 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259266, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -1 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259266, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -1 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259248, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -1 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259294, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "[-]\t/irods_source/server/core/src/rsApiHandler.cpp:595:int readAndProcClientMsg(rsComm_t *, int) :  status [Unknown iRODS error]  errno [Operation not permitted] -- message [failed to call 'read header']\n\t[-]\t/irods_source/lib/core/src/sockComm.cpp:219:irods::error readMsgHeader(irods::network_object_ptr, msgHeader_t *, struct timeval *) :  status [Unknown iRODS error]  errno [Operation not permitted] -- message [failed to call 'read header']\n\t\t[-]\t/irods_source/plugins/network/src/ssl.cpp:464:irods::error ssl_read_msg_header(irods::plugin_context &, void *, struct timeval *) :  status [Unknown iRODS error]  errno [Operation not permitted] -- message [read -1 expected 4]\n\n", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259270, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -1 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259248, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -1 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259296, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -1 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259270, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259266, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -1 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259294, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -1 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259270, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259248, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -1 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259267, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -1 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259253, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -1 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259304, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -1 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259309, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -1 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259253, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259317, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259259, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259267, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259309, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259253, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259304, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.768Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259294, "server_timestamp": "2026-05-15T14:08:34.753Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.770Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "agent", "log_level": "error", "log_message": "[-]\t/irods_source/server/main_server/src/agent_main.cpp:965:int (anonymous namespace)::handle_client_request(int, std::time_t) :  status [SSL_SHUTDOWN_ERROR]  errno [] -- message [failed to call 'agent stop']\n\t[-]\t/irods_source/lib/core/src/sockComm.cpp:181:irods::error sockAgentStop(irods::network_object_ptr) :  status [SSL_SHUTDOWN_ERROR]  errno [] -- message [failed to call 'agent stop']\n\t\t[-]\t/irods_source/plugins/network/src/ssl.cpp:787:irods::error ssl_agent_stop(irods::plugin_context &) :  status [SSL_SHUTDOWN_ERROR]  errno [] -- message [error completing shutdown of SSL connection | error:0A000126:SSL routines::unexpected eof while reading | error:0A000197:SSL routines::shutdown while in init]\n\n", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259266, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.770Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "[-]\t/irods_source/server/core/src/rsApiHandler.cpp:595:int readAndProcClientMsg(rsComm_t *, int) :  status [Unknown iRODS error]  errno [Operation not permitted] -- message [failed to call 'read header']\n\t[-]\t/irods_source/lib/core/src/sockComm.cpp:219:irods::error readMsgHeader(irods::network_object_ptr, msgHeader_t *, struct timeval *) :  status [Unknown iRODS error]  errno [Operation not permitted] -- message [failed to call 'read header']\n\t\t[-]\t/irods_source/plugins/network/src/ssl.cpp:464:irods::error ssl_read_msg_header(irods::plugin_context &, void *, struct timeval *) :  status [Unknown iRODS error]  errno [Operation not permitted] -- message [read -1 expected 4]\n\n", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259296, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.770Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "agent", "log_level": "error", "log_message": "[-]\t/irods_source/server/main_server/src/agent_main.cpp:965:int (anonymous namespace)::handle_client_request(int, std::time_t) :  status [SSL_SHUTDOWN_ERROR]  errno [] -- message [failed to call 'agent stop']\n\t[-]\t/irods_source/lib/core/src/sockComm.cpp:181:irods::error sockAgentStop(irods::network_object_ptr) :  status [SSL_SHUTDOWN_ERROR]  errno [] -- message [failed to call 'agent stop']\n\t\t[-]\t/irods_source/plugins/network/src/ssl.cpp:787:irods::error ssl_agent_stop(irods::plugin_context &) :  status [SSL_SHUTDOWN_ERROR]  errno [] -- message [error completing shutdown of SSL connection | error:0A000126:SSL routines::unexpected eof while reading | error:0A000197:SSL routines::shutdown while in init]\n\n", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259294, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.770Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "agent", "log_level": "error", "log_message": "[-]\t/irods_source/server/main_server/src/agent_main.cpp:965:int (anonymous namespace)::handle_client_request(int, std::time_t) :  status [SSL_SHUTDOWN_ERROR]  errno [] -- message [failed to call 'agent stop']\n\t[-]\t/irods_source/lib/core/src/sockComm.cpp:181:irods::error sockAgentStop(irods::network_object_ptr) :  status [SSL_SHUTDOWN_ERROR]  errno [] -- message [failed to call 'agent stop']\n\t\t[-]\t/irods_source/plugins/network/src/ssl.cpp:787:irods::error ssl_agent_stop(irods::plugin_context &) :  status [SSL_SHUTDOWN_ERROR]  errno [] -- message [error completing shutdown of SSL connection | error:0A000126:SSL routines::unexpected eof while reading | error:0A000197:SSL routines::shutdown while in init]\n\n", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259248, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.770Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "[-]\t/irods_source/server/core/src/rsApiHandler.cpp:595:int readAndProcClientMsg(rsComm_t *, int) :  status [Unknown iRODS error]  errno [Operation not permitted] -- message [failed to call 'read header']\n\t[-]\t/irods_source/lib/core/src/sockComm.cpp:219:irods::error readMsgHeader(irods::network_object_ptr, msgHeader_t *, struct timeval *) :  status [Unknown iRODS error]  errno [Operation not permitted] -- message [failed to call 'read header']\n\t\t[-]\t/irods_source/plugins/network/src/ssl.cpp:464:irods::error ssl_read_msg_header(irods::plugin_context &, void *, struct timeval *) :  status [Unknown iRODS error]  errno [Operation not permitted] -- message [read -1 expected 4]\n\n", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259266, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.770Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "agent", "log_level": "error", "log_message": "[-]\t/irods_source/server/main_server/src/agent_main.cpp:965:int (anonymous namespace)::handle_client_request(int, std::time_t) :  status [SSL_SHUTDOWN_ERROR]  errno [] -- message [failed to call 'agent stop']\n\t[-]\t/irods_source/lib/core/src/sockComm.cpp:181:irods::error sockAgentStop(irods::network_object_ptr) :  status [SSL_SHUTDOWN_ERROR]  errno [] -- message [failed to call 'agent stop']\n\t\t[-]\t/irods_source/plugins/network/src/ssl.cpp:787:irods::error ssl_agent_stop(irods::plugin_context &) :  status [SSL_SHUTDOWN_ERROR]  errno [] -- message [error completing shutdown of SSL connection | error:0A000126:SSL routines::unexpected eof while reading | error:0A000197:SSL routines::shutdown while in init]\n\n", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259270, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.770Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -1 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259296, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.770Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "readAndProcClientMsg:readMsgHeader error. status = -4000 status = 0 Unknown iRODS error", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259239, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}
[2026-05-15T14:08:34.770Z][icts-t-cloud-rdm-hev-1] {"processname": "rsyslogd", "groupname": "rsyslogd", "pid": "119", "channel": "stdout", "rsyslogd": {"log_category": "legacy", "log_level": "error", "log_message": "[-]\t/irods_source/server/core/src/rsApiHandler.cpp:595:int readAndProcClientMsg(rsComm_t *, int) :  status [Unknown iRODS error]  errno [Operation not permitted] -- message [failed to call 'read header']\n\t[-]\t/irods_source/lib/core/src/sockComm.cpp:219:irods::error readMsgHeader(irods::network_object_ptr, msgHeader_t *, struct timeval *) :  status [Unknown iRODS error]  errno [Operation not permitted] -- message [failed to call 'read header']\n\t\t[-]\t/irods_source/plugins/network/src/ssl.cpp:464:irods::error ssl_read_msg_header(irods::plugin_context &, void *, struct timeval *) :  status [Unknown iRODS error]  errno [Operation not permitted] -- message [read -1 expected 4]\n\n", "request_api_name": "", "request_api_number": 20003, "request_api_version": "d", "request_client_user": "u0137480", "request_host": "172.17.0.18", "request_proxy_user": "rods", "request_release_version": "rods5.0.2", "server_host": "icts.irods.t.icts.kuleuven.be", "server_pid": 259294, "server_timestamp": "2026-05-15T14:08:34.754Z", "server_type": "agent", "server_zone": "icts"}}

[korydraughn]

What happens when SSL/TLS is not enabled?
What version of Python are you using?
What version of aiohttp are you using?

Can you post your http api config file?

[mstfdkmn]

What happens when SSL/TLS is not enabled?

We haven't tried that.

What version of Python are you using?

root@26715a9fb27c:/# python3 --version
Python 3.12.3

What version of aiohttp are you using?

It was the latest version out when I tried it, so probably v3.13.5.

Can you post your http api config file?

sh-5.1# cat http-api.config

    {
    "http_server": {
        "host": "0.0.0.0",
        "port": 9000,

        "log_level": "debug",

        "authentication": {
            "eviction_check_interval_in_seconds": 60,

            "basic": {
                "timeout_in_seconds": 3600
            },

            "openid_connect": {
                "timeout_in_seconds": 14400,

                "provider_url": "custom.provider",

                "client_id": "irods.zone",

                "client_secret": "none",

                "require_aud_member_from_introspection_endpoint": true,

                "access_token_validation_method": "introspection",

                "state_timeout_in_seconds": 60,

                "user_mapping": {
                    "plugin_path": "/usr/lib64/irods_http_api/plugins/user_mapping/libirods_http_api_plugin-user_claim.so",

                    "configuration": {
                        "irods_user_claim": "preferred_username"
                    }
                }
            }
        },

        "requests": {
            "threads": 16,
            "max_rbuffer_size_in_bytes": 33554432,
            "max_size_of_request_body_in_bytes": 33554432,
            "timeout_in_seconds": 120
        },

        "background_io": {
            "threads": 16
        }
    },

    "irods_client": {
        "host": "irods.zone",
        "port": 1247,
        "zone": "icts",

        "tls": {
            "client_server_negotiation": "request_server_negotiation",
            "encryption_algorithm": "AES-256-CBC",
            "encryption_key_size": 32,
            "encryption_hash_rounds": 16,
            "encryption_salt_size": 8,
            "client_server_policy": "CS_NEG_REQUIRE",
            "ca_certificate_file": "/etc/ssl/certs/ca-bundle.crt",
            "certificate_chain_file": "/etc/irods/ssl/server.crt",
            "dh_params_file": "/etc/irods/ssl/dhparams.pem",
            "verify_server": "cert"
        },

        "enable_4_2_compatibility": false,

        "proxy_admin_account": {
            "username": "rods",
            "password": "password"
        },

        "connection_pool": {
            "size": 16,
            "refresh_timeout_in_seconds": 600,
            "max_retrievals_before_refresh": 16,
            "refresh_when_resource_changes_detected": true
        },

        "max_number_of_parallel_write_streams": 10,

        "max_number_of_bytes_per_read_operation": 8192,
        "max_number_of_bytes_per_write_operation": 8192,
        "buffer_size_in_bytes_for_write_operations": 8192,

        "max_number_of_rows_per_catalog_query": 15
    }
}