Skip to main content

CodeTogether HQ On-Premises Installation via Kubernetes

Using Kubernetes (k8s) is the recommended approach for larger scale deployments in an Enterprise as it is easier to provide seamless upgrades and high availability via replicas. During upgrades of the HQ container, k8s will automatically handle spinning up the new instance while keeping the old around.

Step 1. Set Up Cassandra​

To start with Cassandra, you are first required to activate the Bitnami chart repository from which the core Cassandra configuration will come from:

helm repo add bitnami https://charts.bitnami.com/bitnami

Once the repository is added, you will then create the values file to start Cassandra. By default, this can be quite a simple file similar to the following. Adjust the password as desired.

cassandra-values.yaml​

global:
  storageClass: do-block-storage
dbUser:
  user: cassandra
  password: cassandra

Once the file is ready, start up Cassandra in k8s. 

helm install codetogether-cassandra bitnami/cassandra -f cassandra-values.yaml

To monitor the status of the pod coming up, you can check status using this command.

kubectl exec -it --namespace default $(kubectl get pods --namespace default -l app.kubernetes.io/name=cassandra,app.kubernetes.io/instance=codetogether-cassandra -o jsonpath='{.items[0].metadata.name}') -- nodetool status

Once the pod is up and running, you need to connect in to enable the keyspace that CodeTogether HQ will be using on the server. To connect into Cassandra, you will use a one-time use client to Cassandra that allows you to run the cli to create the keyspaces. The pod will go away once you're done with it. 

kubectl run --namespace default codetogether-cassandra-client --rm --tty -i --restart='Never' --image docker.io/bitnami/cassandra:4.1.3-debian-11-r71 -- bash

Once you are in the Cassandra cli, you connect in using a command similar to the following and create the keyspace using a regular replication strategy. 

I have no name!@codetogether-cassandra-client:/$ cqlsh -u cassandra -p cassandra codetogether-cassandra.default.svc.cluster.local

Warning: Using a password on the command line interface can be insecure.
Recommendation: use the credentials file to securely provide the password.


Connected to cassandra at hq-cassandra.default.svc.cluster.local:9042
[cqlsh 6.1.0 | Cassandra 4.1.3 | CQL spec 3.4.6 | Native protocol v5]
Use HELP for help.
cassandra@cqlsh> CREATE KEYSPACE hq WITH replication = {'class':'SimpleStrategy', 'replication_factor' : 1};
cassandra@cqlsh> exit

Step 2. Set Up External DNS (Optional)​

Depending on your Kubernetes configuration, you may want to activate External DNS updating to your DNS service to facilitate automatic provisioning of DNS names. If you use a different system or prefer to map IP addresses manually, this step can be skipped.

provider: <provider>

digitalocean:
  apiToken: <api-token>

interval: "1m"

policy: sync

To start External DNS which is also included in the Bitnami repository, simply run the following command:

helm install external-dns bitnami/external-dns -f externaldns-values.yaml

To monitor the startup of the External DNS pod you can run this command:

kubectl --namespace=default get pods -l "app.kubernetes.io/name=external-dns,app.kubernetes.io/instance=external-dns"

Step 3. Enable Nginx Ingress (Optional)​

Depending on your Kubernetes configuration, you can decide what sort of Ingress controller you use. The example values file shared below assumes using the Nginx ingress, which if not active can be enabled via first adding the chart repository:

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx

Next, install the Nginx ingress in the Kubernetes pod.

helm install nginx-ingress ingress-nginx/ingress-nginx --set controller.publishService.enabled=true

Step 4. Register the SSL Certificate​

To prepare for the CodeTogether container’s ingress, you’ll need to register the SSL configuration files as secrets in Kubernetes.

kubectl create secret tls codetogether-hq-tls --key ssl.key --cert ssl.bundle.crt
IMPORTANT

Configuring the SSL certificate requires a trusted certificate that is fully rooted on the Internet including providing the intermediate certificates in the ssl-bundle.crt file. Given the variety of clients and IDEs used with CodeTogether HQ, this is important to simplify any client-side configuration required. Use of an internal certificate or a certificate that is not fully rooted will require additional complex configuration within IDE clients to trust the certificate.

Step 5. Configure the Values for CodeTogether HQ​

To access the full values.yaml template file, download the latest version from GitHub. The following section will highlight section by section the values that should be configured as part of running via Kubernetes.

Download the latest version of the file from:

https://github.com/CodeTogether-Inc/CodeTogether-Deployment/blob/main/charts/hq/values.yaml

Set Up Registry Access​

Provide your credentials to the CodeTogether HQ docker registry:

imageCredentials:
  enabled: true
  registry: hub.edge.codetogether.com
  username: "my-customer-username"
  password: "my-customer-password"
  email: unused

Configure Server URL​

Set the URL that will be used to connect to the server:

codetogether:
  url: https://<server-fqdn>

Configure HQ Services​

Provide the CodeTogether HQ container configuration properties. The following are the initial values that typically need to be configured with the exception of single sign-on values covered below.

cthq.properties​

hqproperties:
  hq.sso... (see below)
  hq.db.type: CASSANDRA
  hq.secret: SECRET1
  hq.encryption.secret: SECRET2
  hq.base.url: https://<server-fqdn>
  hq.cassandra.db.name: hq
  hq.cassandra.db.port: 9042
  hq.cassandra.db.host: codetogether-cassandra.default.svc.cluster.local
  hq.sso.redirect.uri: https://<server-fqdn>/api/v1/auth/sso/success/insights
  hq.cassandra.db.password: cassandra
  hq.cassandra.db.username: cassandra

If you followed the instructions above, the Cassandra database will be under the codetogether-cassandra name in the k8s Pod. If the name was changed, make sure to update it above.

Regarding the SSO configuration (hq.sso…), the following are the different keys that should be present to connect to your sign-on service.

hqproperties: (continues from above)
  # Values that must be present for configuring SSO
  hq.sso.client.id: <clientid>
  hq.sso.client.secret: <clientsecret>
  hq.sso.redirect.uri: https://<server-fqdn>/api/v1/auth/sso/success/insights
  # Generic single sign-on configuration --
  hq.sso.client.issuer.url: https://<sso-fqdn>/<path>/.well-known/openid-configuration
  # hq.sso.name.attr: [preferred_username|name]
  # Example for connecting with Keycloak --
  # hq.sso.client.issuer.url: https://<keycloak-fqdn>/realms/<realm>
  # Example for connecting to GitHub --
  # hq.sso.provider: github
  # hq.sso.auth.uri: https://github.com/login/oauth/authorize
  # hq.sso.token.uri: https://github.com/login/oauth/access_token
  # hq.sso.info.uri: https://api.github.com/user
  # hq.sso.jwt.set.uri: https://token.actions.githubusercontent.com/.well-known/jwks
  # hq.sso.logout.uri: https://github.com/logout
  # hq.sso.name.attr: name

The majority of SSO providers require only a few settings to get connected. The issuer URL tells CodeTogether how to talk to the server, the client ID and secret are configured within the single sign-on service and authorize our backend to talk to the server. CodeTogether normally works well out of the box, but you can also set the primary name attribute to use for names in the system.

For GitHub, it can be configured using a different integration which provides added value. Most importantly, it allows CodeTogether to get all committer IDs for the users that are logging in, which eliminates extra steps requiring each end user to register their additional emails after login.

When in doubt, please contact your CodeTogether technical support representative for guidance on configuration of your specific SSO connection.

Configure Ingress​

Configure the desired Ingress configuration. There are three key options, using className to determine which to use. In a default configuration, the already available ingress is likely the right option to choose and would look like the following. Note that you need to register the SSL certificate earlier.

ingress:
  enabled: true
  tls:
    secretName: codetogether-hq-tls

The other values in the file likely do not require modification but can be changed based on your specific use cases or tuning as required.

Step 6. Start the CodeTogether HQ Backend​

Before actually launching the CodeTogether backend, you first need to register the Helm chart repository for CodeTogether. Run the following command:

helm repo add codetogether https://helm.codetogether.io

Once the repository is set up, you can now start up the CodeTogether HQ Pod by running the following command:

helm install codetogether-hq codetogether/codetogether-hq -f values.yaml

Finally, you can monitor the pod to confirm it is up and running by using the following command:

kubectl --namespace=default get pods -l "app.kubernetes.io/name=codetogether-hq,app.kubernetes.io/instance=codetogether-hq"

With that, you can now connect to your server!

Simply open your browser to https://<server-fqdn> and you’ll have an opportunity to log in via your single sign-on service and then proceed to provide your license file.

Continue next with the Getting Started Guide which walks through setting up projects and the flow for end-users of CodeTogether HQ.

Step 7. Applying Updates​

To apply updates to CodeTogether, you can use the following command. Kubernetes will take into account starting up the new instance before spinning down the old, minimizing any downtime for users.

helm upgrade codetogether-hq codetogether/codetogether-hq -f values.yaml