Custom kube api qps and kube api burst values.

[nomis-rzp]

We have observed client-side throttling when a large number of pods are running on a spot node, as not all pods get drained within the 2-minute window. Currently, the kubernetes API QPS and burst limits are set to 5 and 10, respectively, and NTH does not provide an option to override these values. Would it make sense for NTH to support configurable overrides for these settings?

https://github.com/kubernetes/client-go/blob/master/rest/config.go#L46-L49
Throttling logs

INF Waited for 1.160043617s due to client-side throttling, not priority and fairness, request: GET:https://172.20.0.1:443/api/v1/namespaces/test/pods/test-6475fbcfcb-sxd4d
INF Waited for 1.05353185s due to client-side throttling, not priority and fairness, request: GET:https://172.20.0.1:443/api/v1/namespaces/test/pods/test-6475fbcfcb-sgdgd