Skip to content

Converting ingress service type as deployment var and adding a infra CLI command to show deployment vars#406

Merged
geekodour merged 7 commits intoprometheus:masterfrom
imrajdas:provider-env
Aug 2, 2020
Merged

Converting ingress service type as deployment var and adding a infra CLI command to show deployment vars#406
geekodour merged 7 commits intoprometheus:masterfrom
imrajdas:provider-env

Conversation

@imrajdas
Copy link
Contributor

@imrajdas imrajdas commented Jul 6, 2020

Motivation:

  • As KIND doesn't support Loadbalance type service, That's why it converts ingress service type as deployment var (PROVIDER_SERVICE) and assigning that deployment var in NewGKEClient function of gke.go

Changes:

  • Adding NGINX_SERVICE variable for the load balancer service type
  • Adding gke info deploymentvars command to show all default deployment variables.

Signed-off-by: Raj Babu Das mail.rajdas@gmail.com

@imrajdas
Copy link
Contributor Author

imrajdas commented Jul 6, 2020

cc @geekodour

@imrajdas
Copy link
Contributor Author

imrajdas commented Jul 6, 2020

Logs-
log.txt

@imrajdas imrajdas force-pushed the provider-env branch 4 times, most recently from b2067a1 to efff23a Compare July 6, 2020 18:28
…ar in gke.go

Signed-off-by: Raj Babu Das <mail.rajdas@gmail.com>
Copy link
Contributor

@geekodour geekodour left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I was suggesting was something like

c.DeploymentVars["PROVIDER_SERVICE"] = "GKE" 

and use something like

{{ if (eq .PROVIDER_SERVICE "GKE") }}

in the manifest file, but the approach in this PR actually looks better i think :) . I've commented some ways this can be improved.

cc @krasi-georgiev @nevill

func (c *GKE) NewGKEClient(*kingpin.ParseContext) error {

// Set PROVIDER_SERVICE Deployment variable to LoadBalancer
c.DeploymentVars["PROVIDER_SERVICE"] = "LoadBalancer"
Copy link
Contributor

@geekodour geekodour Jul 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
c.DeploymentVars["PROVIDER_SERVICE"] = "LoadBalancer"
c.DeploymentVars["NGINX_SERVICE_TYPE"] = "LoadBalancer"

In the provider specific docs, we could just add a section about what are the deployment variables that are automatically passed for this specific provider.

It'll be good if we could have these deployment variables overridable when uses passes the same using the -v flag. Maybe also have a provider specific flag to list out all the set deployment variables. (This way we won't have to update readme manually and we could just use embedmd #353)

What I think will be good is to have a default DeploymentVars and have it set to the default value, like for eg. in this example only nginx controller k8s service type needs to be changed when running in KIND (to CLusterIP).

But for EKS and GKE, the type LoadBalancer works, so we could have some defaults and only modify them when creating the provider that needs the change(in our case KIND)

@geekodour
Copy link
Contributor

@rajdas98 Also ingress in kind uses
https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/kind/deploy.yaml

And GKE currently uses
https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/cloud/deploy.yaml

Difference being :

λ diff gke_deploy.yml kind_deply.yml 
287,288c287
<   type: LoadBalancer
<   externalTrafficPolicy: Local
---
>   type: NodePort
322a322,325
>   strategy:
>     rollingUpdate:
>       maxUnavailable: 1
>     type: RollingUpdate
343d345
<             - --publish-service=ingress-nginx/ingress-nginx-controller
349a352
>             - --publish-status-address=localhost
390a394
>               hostPort: 80
393a398
>               hostPort: 443
404a410,415
>       nodeSelector:
>         ingress-ready: 'true'
>       tolerations:
>         - effect: NoSchedule
>           key: node-role.kubernetes.io/master
>           operator: Equal
406c417
<       terminationGracePeriodSeconds: 300
---
>       terminationGracePeriodSeconds: 0

So, is just changing the service type enough?

@geekodour
Copy link
Contributor

So as far as i can see there should be three levels to this:

  • default deployment vars (which get set provider agnostic)

  • provider deployment vars which has a different value than default deployment var Eg. (Loadbalancer in gke and Nodeport in kind) Here the default could be Loadbalancer as it's used by both gke and eks and we need to replace it only for KIND

  • the normal deployment vars at use, this should be able to override both default and provider specific deployment vars.

@imrajdas imrajdas force-pushed the provider-env branch 3 times, most recently from e3acd7b to 42a7e28 Compare July 18, 2020 19:50
Signed-off-by: Raj Babu Das <mail.rajdas@gmail.com>
Copy link
Contributor

@geekodour geekodour left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! 👍 Thanks!

Some suggestions and improvements.

@imrajdas imrajdas force-pushed the provider-env branch 9 times, most recently from d827e86 to aa7e113 Compare July 23, 2020 16:16
@imrajdas imrajdas changed the title Converting ingress service type as deployment var Converting ingress service type as deployment var and adding a infra CLI command to show deployment vars Jul 23, 2020
@imrajdas imrajdas requested a review from geekodour July 23, 2020 16:19
Signed-off-by: Raj Babu Das <mail.rajdas@gmail.com>
Copy link
Contributor

@geekodour geekodour left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some questions and suggestions. :)

imrajdas added 3 commits July 29, 2020 22:39
Signed-off-by: Raj Babu Das <mail.rajdas@gmail.com>
Signed-off-by: Raj Babu Das <mail.rajdas@gmail.com>
Signed-off-by: Raj Babu Das <mail.rajdas@gmail.com>
Copy link
Contributor

@geekodour geekodour left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some questions and suggestions :)

I think once this is merged, we don't have any other blockers for local?

@imrajdas imrajdas force-pushed the provider-env branch 4 times, most recently from 07bb3c8 to 6a5175b Compare August 2, 2020 11:47
@imrajdas imrajdas requested a review from geekodour August 2, 2020 11:51
Copy link
Contributor

@geekodour geekodour left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks good 🥳 just small nits.

Will merge after changes are resolved :)

Signed-off-by: Raj Babu Das <mail.rajdas@gmail.com>
@geekodour geekodour merged commit 8c4c50c into prometheus:master Aug 2, 2020
@imrajdas imrajdas deleted the provider-env branch August 2, 2020 13:50
kushalShukla-web pushed a commit to kushalShukla-web/test-infra that referenced this pull request Feb 8, 2025
* Add NGINX_SERVICE_TYPE to DefaultDeploymentVars
* Add info command to infra to show provider specific DeploymentVars

Signed-off-by: Raj Babu Das <mail.rajdas@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants