Bug: The External LB is getting stuck into "pending" state on AWS & few other clouds when protocol is TCP

[narmidm]

📜 Description

while testing the SGE on private clusters, we found some issues with AWS specific annotations that worker-operator is adding.

// Note: Special treatment for AWS EKS clusters. The LB is not provisioned unless we add AWS specific annotations // to the service. This is needed only for EKS. if clusterProvider, _ := getClusterProviderID(ctx, r.Client); clusterProvider == "aws" { if svc.ObjectMeta.Annotations == nil { svc.ObjectMeta.Annotations = make(map[string]string) } svc.ObjectMeta.Annotations["service.beta.kubernetes.io/aws-load-balancer-type"] = "external" svc.ObjectMeta.Annotations["service.beta.kubernetes.io/aws-load-balancer-nlb-target-type"] = "ip" svc.ObjectMeta.Annotations["service.beta.kubernetes.io/aws-load-balancer-scheme"] = "internet-facing" }

https://github.com/kubeslice/worker-operator/blob/master/controllers/slice/slice_gw_edge.go#L128C1-L137C3
the LB is getting stuck into "pending" state on AWS when protocol is TCP. The issues were resolved simply by removing the annotations.

👟 Reproduction steps

while testing the SGE on private clusters, we found some issues with AWS specific annotations that worker-operator is adding.

👍 Expected behavior

external lb start running without any issue.

👎 Actual Behavior

The External LB is getting stuck into "pending" state.

🐚 Relevant log output

No response

Version

1.2.0

🖥️ What operating system are you seeing the problem on?

Windows

✅ Proposed Solution

We can introduce a new field in the Cluster Custom Resource Definitions (CRDs) for a set of annotations for the External Load Balancer (Ext LB). If any cloud provider needs specific annotations for their Network Load Balancer (Network LB), these can be passed as input during the creation of the Cluster CRD. The Slice Gateway Edge (SGE) will then take these annotations and apply them to the External Load Balancer during its creation.

👀 Have you spent some time to check if this issue has been raised before?

• I checked and didn't find any similar issue

Code of Conduct

• I agree to follow this project's Code of Conduct

[Bhargav-InfraCloud]

Hi team! Is this being worked on? If not, I want to give it a try.
/assign

[narmidm]

@Bhargav-InfraCloud, Thanks you for addressing the issue. You will likely need more details, so please join this channel if you are not here yet - https://kubernetes.slack.com/archives/C03Q64HNEJF, and contact Mridul Gain or Bharath for a detailed explanation.

[Bhargav-InfraCloud]

Sure @narmidm. Will check with them. Thanks!

[narmidm]

@Bhargav-InfraCloud any update. did you get clarification from @mridulgain?

[Bhargav-InfraCloud]

@narmidm Yes, he has shared a few details on Slack. Unfortunately, I got caught up with some other work around the same time and it is still going on. If anyone is up for working on this, please feel free to pick. Otherwise, I'll try again sometime.