¶ What is Ingress?
- A Kubernetes Ingress Controller Deployment is a setup that manages external access to services within a Kubernetes cluster.
- It routes HTTP and HTTPS traffic to different services based on rules defined in Ingress resources.
- The deployment ensures secure, scalable, and efficient traffic management, often supporting TLS termination, load balancing, and path-based routing.
¶ Internal Ingress
Internal ingress handles incoming traffic between services within the cluster using an Ingress resource that routes traffic internally rather than exposing it externally.
Sview supports letsencrypt encrypted internal HTTPS ingress endpoints via internal ingress controller. Internal ingress controller might be disabled on your environment. Please contact CTO2B support ([email protected]) to get it enabled.
¶ Requirements before start using
- create role with name
dns-managerthat contain below policy, whereZXXXXXXXXXXXis your internal zone ID in Route53
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "route53:GetChange",
"Resource": "arn:aws:route53:::change/*"
},
{
"Effect": "Allow",
"Action": [
"route53:ChangeResourceRecordSets",
"route53:ListResourceRecordSets"
],
"Resource": "arn:aws:route53:::hostedzone/ZXXXXXXXXXXX"
},
{
"Effect": "Allow",
"Action": "route53:ListHostedZonesByName",
"Resource": "*"
}
]
}
- add trust relationship to above role, where
idxxxxxxxxxxis AWS account ID for env where you deploy your applications
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::idxxxxxxxxxx:root"
},
"Action": "sts:AssumeRole",
"Condition": {
"ArnLike": {
"aws:PrincipalArn": "arn:aws:iam::idxxxxxxxxxx:role/env-cert-manager-sa-*"
}
}
}
]
}
- Add DNS record in your domain zone
domain.com
*.internal.domain.co CNAME internal-a633333333333333333.eu-west-1.elb.amazonaws.com
where internal-a633333333333333333.eu-west-1.elb.amazonaws.com is from traefik-int SVC
- by deploy application with internal ingress make sure you update
cluster-issuerandingress.classannotation
ingress:
enabled: true
annotations:
cert-manager.io/cluster-issuer: route53-dns
kubernetes.io/ingress.class: traefik-int
traefik.ingress.kubernetes.io/router.entrypoints: websecure
traefik.ingress.kubernetes.io/router.tls: 'true'
¶ gRPC load balancer
A gRPC load balancer is a mechanism that distributes gRPC traffic across multiple backend services to ensure high availability, scalability, efficient load distribution and retries for gRPC services.
gRPC load balancer is exposed Through Internal Ingress:
-
The internal ingress acts as an entry point for gRPC clients within the network.
-
External traffic cannot directly reach the gRPC load balancer—it must go through controlled network paths.
app-k8s-generic deployment support grpc endpoints.
Expose gRPC LB host via 3 settings in app generic deployment
- make sure port is set as example
50051 - make sure service has
targetPort: - provide FQDN endpoints and DNS resolver for TLS certificate provision in grpchosts
ports:
- name: grpc
containerPort: 50051
protocol: TCP
service:
type: ClusterIP
targetPort: 50051
name: grpc
port: 8080
grpchosts:
- host: grpc.internal.demo.cto2b.eu
port: 50051
dnsResolver: cto2b-dns # route53-dns, etc
¶ test you gRPC load balancer
/ # ./grpcurl -d '{"name": "Andrius"}' grpc.internal.demo.cto2b.eu:443 helloworld.Greeter.SayHello
{
"message": "Hello Andrius, from my-grpc-app-64d87b7854-zn8c2"
}
¶ External Ingress
An External Ingress is a network entry point that allows traffic from the public internet (or external networks) to access services within a private infrastructure, such as a Kubernetes cluster or a cloud-based service.
How External Ingress Handles Routing & Security:
-
The ingress component (e.g., Kubernetes Ingress Controller, Cloud Load Balancer) routes traffic to the appropriate backend service.
-
It may also perform TLS termination, authentication, and security checks.
¶ Traefik with NLB
Supported clouds: aws, gcp
Sview by default uses external ingress traefik ingress controller with public NLB. This option support not only HTTP and HTTPS endpoints but any TCP protocol supported via TCPRoute
¶ Traefik with ALB
Supported clouds: aws, gcp
In some use cases where Cloud Armor or AWS Shield is part of your architecture as front gate entrypoint for EKS or GKE ingress, you can use CTO2B traefik-web app.
¶ AWS
¶ Prerequisites
traefik-web for AWS requires wildcard ACM certificate for all domains and subdomains. This certificate will be attached to ALB (Figure 1.1)
Figure 1.1 AWS ACM certificate
¶ Setup
To enable traefik-web edit cluster and add 2 parameters shown below. Don't forget to replace traefik_web_ingress_cert_arn with the right ARN.
terraform.variables.traefik-web: true
terraform.variables.traefik_web_ingress_cert_arn: arn:aws:acm:ap-southeast-1:xxxxxxxx:certificate/7fr4567-dcfb-5g667-6789-34fg6782d
¶ GCP
To provision the traefik-web for GCP you need to follow steps below:
¶ Prerequisites
- Security policy (optional) https://cloud.google.com/armor/docs/configure-security-policies
- Access to domain management console (i.e. dash.cloudflare.com)
¶ Setup
- Enable traefik-web by editing cluster and adding 3 parameters (“dummy” value will be overwritten later)
terraform.variables.traefik-web: true
terraform.variables.traefik-web_managedCertificates: "dummy"
terraform.variables.traefik-web_securityPolicyName: "security-policy-name"
- Open ArgoCD and go to traefik-web application
- Copy dynamically assigned IP address from ingress resource and note it down
- Open domain management console and create A records for each domain which you want to use with ingress routes with IP address from previous step
- Open SVIEW and create new ingress route application with desired routes
- Open ArgoCD and find Ingress routes application
- Copy name of managedCertificate resource and replace the “dummy” value in step 1
- Your certificate will be provisioned and deployed after all domains passed the verification (be patient, it can take over an hour!)
¶ Ingress Routes
In this option only HTTPS protocol is supported and Traefik Ingress Routes catalog item is used to deploy/manage ingress routes to services. As per example below (Figure 1.2)
Figure 1.2 Traefik Ingress Routes deployment
You can provide list of ingress routes.
routes:
- host: myapp.demo.cto2b.eu
service: my-app-svc
prefix: /
namespace: app
port: 8080
as per one route :
host - public FQDN
service - name of application service in kubernetes
namespace - namespace where application service is deployed
port - application service port
If you manage your application ingress via Ingress Routes then make sure default ingress is set :
false(Figure 1.3)
Figure 1.3. Default ingres sin application deployment disabled
¶ CloudFlare example
If you use WAF on CloudFlare(CF) you need to add your wildcard cert to Edge Certificates to let CF to renew and manage
¶ Middlewares
It is possible to benefit from using Traefik middlewares for ingresses.
ingress:
middlewares:
ipWhiteList:
sourceRange:
- 127.0.0.1/32
- 192.168.1.7
Middlewares section also supports adding the X-Request-ID header with a generated UUIDv4 value to HTTP requests and responses, allowing downstream services to identify requests via plugin.
ingress:
middlewares:
plugin:
traefik-plugin-request-id:
Enabled: "true"
HeaderName: X-Request-ID
Before adding plugin configuration to ingress ensure plugin itself is enabled on Traefik side.
Adding plugin to Traefik have to be done by adding following variable to terarorm environment configuration
terraform:
variables:
traefik:
plugins:
traefik-plug:
moduleName: github.com/mdklapwijk/traefik-plugin-request-id
version: v0.1.1