rpc-load-balancer service is broken

[dboreham]

The explorer cluster of services is arranged such that the api service needs a working RPC service in order for the container to start (this itself is a bug -- container should start even if its RPC is down). But it gets worse : the configured RPC service URL points at another service in the same cluster (rpc-load-balancer). Fun fact -- that service loads its set of "upstream" proxy origin servers (the real RPC service endpoints) from a "node watcher" service in...guess where? -- the api service. This chicken/egg arrangement can lead to the entire contraption failing to start because the api service won't start without an RPC service but the RPC service it is using won't work unless it can get updated upstream nodes from the api service that isn't started yet.

[dboreham]

Also we don't have the source for the rpc-load-balancer container image we're running (although it could be fairly easily reverse engineered from the image contents).

[dboreham]

This is a higher priority because it hits us every time the cluster is restarted.

[sirouk]

Hi David,

I originally architected the https://github.com/0LNetworkCommunity/rpc-load-balancer and I maintain the live instance available at:

https://rpc.openlibra.space:8080/v1

Do you remember the update I posted about on our 0L server on 5/23?

[dboreham]

Yes, plan is to migrate to yours.