HTTP Traffic Routing and TLS Certificates
Introduction
Manhattan Active® Platform is deployed as a distributed application using Google Kubernetes Engine on Google Cloud Platform. MA products exposes two HTTPS endpoints by default,and they are the Authentication endpoint and Application endpoint.These endpoints are exposed to external using NGINX ingress controller. NGINX Ingress Controller is an Ingress controller that manages external access to HTTP services in a Kubernetes cluster using NGINX. All the configurable traffic routing using the ingress resource,and TLS termination for Manhattan Application are done on the ingress controller level. Currently, the NGINX ingress controller exposes the MA application to the outside world using the Cloud provider TCP load balancer.
All the application and authentication urls are mapped to the cloud load balancer with a publicly resolvable DNS names.
TLS Routing
All the traffic to, from and within Manhattan Active® Platform is encrypted with TLS v1.2 by default. The inbound HTTPS load balancer listens to port 443. The HTTPS endpoints accept traffic from Manhattan Active® Platform web user interface, mobile applications, and REST clients. The ingress controller has configured to serve the application in HTTPS. It works by offloading this functionality from the application. We have configured the SSL certificate in the ingress controller for all the ingress resource rules created for the MA application.
Also, We have automated the lifecycle of the TLS certificates with Cert-manager and Let’s Encrypt as a certification authority. Cert-manager automates the provisioning of HTTPS certificates within the Kubernetes cluster. It provides custom resources to simplify the provisioning, renewal, and use of those certificates.
End-to-End Encryption
With NGINX, we can achieve end‑to‑end encryption of all requests in addition to it making Layer-7 routing decisions. In this case the clients communicate with NGINX over HTTPS, which then decrypts the requests and then re‑encrypts them before sending them downstream to the application gateway, or the Zuul Server.
SSL / TLS Client Certificates
NGINX can handle SSL/TLS client certificates and can be configured to make them optional or required. Client certificates are a way of restricting access to the application to only the authorized clients without requiring a password. We can control the certificates by adding revoked certificates to a certificate revocation list (CRL), which NGINX checks to determine whether a client certificate is still valid.
Cert Manager
The Cert Manager adds certificates and certificate issuers as resource types in a Kubernetes clusters, and simplifies the process of provisioning, renewing and using these certificates. It can issue certificates from a variety of supported sources, including Let’s Encrypt, HashiCorp Vault, and private PKI. Cert Managers ensure the validity of the certificates, and automatically renews the certificate at a configurable time before the expiration.
Manhattan Active® Platform uses Let’s Encrypt, a global Certificate Authority (CA). The Cert Manager ecosystem has API based automation built with Let’s Encrypt to renew and provision the certificates, which then are distributed to many deployment stacks via a Kubernetes cronjob that detects the need for renewal and requests a renewed certificate from the central certificate governing system. Let’s Encrypt serves as a platform for advancing TLS security best practices. All certificates issued or revoked will be publicly recorded and available for anyone to inspect.
How Cert-Manager works in Manhattan Active Platform
Cert-manager runs within in a dedicated tools Kubernetes clusters as a series of deployment resources. It utilizes CustomResourceDefinitions to configure Certificate Authorities and request certificates.Along with cert-manager we have configured Let’s Encrypt as the ClusterIssuer resources which represent a certificate authority. For every customer we are creating a wildcard domain certificate which is injected as a kubernetes secret used by the ingress resources in the cluster.Also, the Let’s Encrypt CA is using the DNS-01 challenge to validate the domain name of the certificate before issuing the certificate.
How certificates are automatically renewed in Manhattan Active® Platform
Cert-manager will automatically renew Certificates. It will calculate when to renew a Certificate based on the issued X. 509 certificate’s duration, and a ‘renewBefore’ value which specifies how long before expiry a certificate should be renewed. Default duration configured in Manhattan Active® Platform is 90 days. There is a cronjob running on the manhattan tools cluster to periodically runs the helper scripts found with in the cert manager in order to generate/renew SSL certificates using the cert manager from LetsEncrypt.
All the generated, and the renewed certificates are stored on a shared storage used by the sidekick application. Each custom environment will download the ssl certificate from the sidekick application and will expose the certificate via a Kubernetes secret. There is a dedicated cronjob running on each custom environment to validate and fetch the certificate before it expires.
Zuul Routing
Zuul is an API gateway in the Manhattan Active® Platform that provides dynamic routing,monitoring and resiliency,security and more. The NGINX ingress controller traffic matching the ingress rules are sent to Zuul server first and then Zuul delegates the authenticated user traffic request to the other application components running in the microservice deployment. Zuul will also run as another microservice application running on the kubernetes cluster. Authentication service, called as oauth server is another Manhattan platform component responsible for both authentication and authorization. All the client initiated traffic which are not authenticated are redirected to auth server by zuul server for completing the authentication by generating the valid JWT tokens.
Zuul and Auth service are exposed to outside world by using the NGINX ingress controller. Ingress resource routing rules are created on the NGINX controller for both zuul and auth service. When the client traffic hit the NGINX ingress controller via the cloud load balancer, NGINX will compare the host header of the user traffic, and the ingress rules on the controller, and if the header match then it will forward the traffic to the Zuul server. For the first time login, Zuul will redirect the call to auth server and that will be another NGINX ingress routing call to auth server for authenticating.
Learn More
Authors
- Giri Prasad Jayaraman: Technical Director, Manhattan Active® Platform, R&D
- Akhilesh Narayanan : Sr Principal Software Engineer, Manhattan Active® Platform, R&D