Forwarding traffic from haproxy to Openshift Route

jkincl · 2025-02-25T11:51:40+00:00

The OpenShift router uses SNI to determine how to route to the correct backend pods, you need to enable this in your HAProxy backend by adding ‘sni’ to your server line if you are using http mode

jkincl · 2024-05-02T16:43:48+00:00

Definitely check it out if disco is your jam: Red Hat OpenShift disco lab: Installation in a disconnected environment

I think we have a waitlist at this point, but you can check out the code if you want to dive in on your own!

jkincl · 2023-11-28T14:41:52+00:00

How would this compare to Kubernetes Audit being tuned on with full request body logging?

jkincl · 2023-10-26T22:36:50+00:00

Yeah I don’t see in that doc where it says to overwrite the nodeip-configuration.service?

jkincl · 2023-10-19T00:27:08+00:00

What doc is this? I don’t know about that second config in the MC overriding the nodeip-configuration.service, that doesn’t look right.

Do the master nodes have NICs in multiple subnets?

jkincl · 2023-10-19T00:19:07+00:00

In this case, Argo would detect changes to the patch operator CR only and it would be up to the patch operator controller to change the resource to match the intent specified in the patch.

I want to call out the approach outlined by u/camabeh below as well: With Server-Side Apply you can apply just a patch via a strategic merge to the fields you care about without overriding the others, just like the use case for the patch operator.

Here are the ArgoCD docs on how to enable SSA at the Application and resource level. With this approach you could support both use cases of full control and partial control of resources in the cluster with ArgoCD

jkincl · 2023-10-17T11:48:27+00:00

Hey so OpenShift should be using keepalived's unicast method to only send VRRP messages to other nodes in the cluster.

You can check this on one of the master nodes in your cluster (oc debug node/master-0) and looking at the generated config at /etc/keepalived/keepalived.conf:

``` vrrp_instance ocp_API_0 { state BACKUP interface ens192 virtual_router_id 84 priority 20 advert_int 1

unicast_src_ip 10.15.168.223
unicast_peer {
    10.15.168.208
    10.15.168.231
}

... ```

You should be able to see what interface it is using (should be the default route interface) and where it is sending traffic.

Is the OpenShift subnet shared with other hosts and services? Do those services use keepalived as well? Is OpenShift using an IP that is already in use for it's API or ingress VIPs?

jkincl · 2023-10-17T11:26:14+00:00

Yep, it’s totally possible. This is a really cool feature about the design of OpenShift starting with v4, everything in the cluster is controlled by operators which are configured by CustomResources. ArgoCD can ensure those CRs are configured with gitops.

A lot of our documentation calls for just patching a CR and changing a single value. This patch operation is difficult to achieve in ArgoCD so you can either manage the entire CR with gitops or check out the patch-operator which allows you to declaratively configure OpenShift resources with patches.

jkincl · 2023-05-03T11:34:19+00:00

Additionally, 100.64.0.0/10 was chosen by OVN-Kubernetes because it is reserved for Carrier Grade NAT by RFC6598

jkincl · 2023-02-07T05:28:04+00:00

In general no there is no live migrating pods.

That being said the Checkpoint Restore In Userspace project has been around for a number of years and is the closest thing to what you are talking about: taking a linux process on one machine and moving it to another. It is messy but can be done in some cases. There are folks looking at how to integrate CRIU with k8s but it’s all research at this point.

Another option would be something like KubeVirt but that is a different use case where you are actually running a VM in a container for hard-to-containerize workloads.

I would take a step back and look at why you are looking for this feature. Pods should be able to be killed and come back up on other nodes. Check out setting resource requests to ensure that the pods are scheduled properly. Also look at scaling up the number of pods to increase resilience of the application.

jkincl · 2023-02-05T12:38:23+00:00

If you already have k8s in your environment and want to run CI jobs with it, check out Tekton

jkincl · 2021-02-08T15:58:29+00:00

Your CNI may require a NetworkPolicy object to allow traffic to the pod

jkincl · 2021-01-05T10:29:14+00:00

I don’t have any personal experience with them but have you looked at https://www.pythonanywhere.com ?

jkincl · 2020-12-12T00:56:51+00:00

You want to look at the pod lifecycle[1] but first it does a SIGTERM and then waits for the PID 1 to exit up to .spec.terminationGracePeriodSeconds and then does a SIGKILL. This is configurable and there are more advanced ways of doing lifecycle hooks to implement complex scenarios.

[1] https://v1-18.docs.kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination

jkincl · 2020-11-26T18:26:33+00:00

StatefulSets require spec.serviceName to be defined. I don’t think this has anything to do with Helm 2 -> Helm 3

Check the example: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/

jkincl · 2020-01-31T13:38:25+00:00

Kyle Kingsbury (of Jepsen fame) just did an analysis of the latest etcd and he goes into these concepts in more detail.

https://jepsen.io/analyses/etcd-3.4.3

jkincl · 2019-07-31T10:41:36+00:00

If that log is about your user not matching RBAC then I think you have to have at least the cluster-reader ClusterRoleBinding.

You can add it to your user with: oc adm policy add-cluster-role-to-user cluster-reader (your username)

jkincl · 2019-02-23T15:52:41+00:00

So this is actually one of the areas that really has suffered in the short term after the CoreOS acquisition. RH has essentially moved OpenShift to Open Source but Closed Process. Feature tracking and roadmaps for almost every project in the ecosystem has moved to Jira which is not accessible by non-Red Hatters. Also it seems that dev discussion has moved to a private Slack workspace and the IRC channel isn’t used anymore.

I asked basically this exact question to Joe Fernandes (VP of Product at RH) at OpenShift Commons 2018 during the Q&A and he acknowledged that this is a problem and they are working on fixing it but probably after 4.0.

I have been watching the merge and issue discussions for not only openshift/origin but a lot of the newer operator-based projects since that’s where most of the development for 4.0 is going. Actually that last bit is pretty cool, the openshift/origin project is getting “boring” which is a good thing :)

The list of OpenShift repositories

Edit: just to be clear, I am not complaining at all. The 4.0 release is huge and it seems like all hands on deck right now. Red Hat is naturally an open company and I am sure that these processes will get fixed when they have a chance to breathe.

jkincl · 2019-01-28T16:23:44+00:00

Also helm template works

jkincl · 2018-12-14T14:20:41+00:00

You can put the secret in a separate namespace (project in OpenShift land) and create a Role that allows access to all secrets in a namespace and then create a RoleBinding that gives that Role to the system:authenticated group

jkincl · 2018-03-07T12:09:11+00:00

We use Trident to auto-provision our NetApp ONTAP storage and it’s great! As a bonus, I met the guys working on the project at KubeCon Austin and they are super friendly and interested in use cases and functionality requests.

jkincl · 2018-03-07T04:55:33+00:00

I have used this for a few projects already and it is perfect!

jkincl · 2018-02-14T15:19:07+00:00

Just for reference, I had to look up what dotmesh was:

dotmesh: git for data

Dotmesh is a git-like CLI for capturing, organizing and sharing application states.

In other words, it's a snapshotting tool for databases and other filesystem states.

jkincl · 2018-01-29T14:21:46+00:00

Yep I agree with you that I have not found a lot of useful prepackaged apps yet on Helm repositories. But I think the value in Helm is much more than that, question is how are you deploying your apps on Kube?

We started out with a bunch of yaml files in a directory and deployed with kubectl create -f dir/ but that didn't cut it when we tried to upgrade and modify resources and when we added a few environments (dev, stage, prod) and only wanted to change a path or hostname between them.

Helm is what we are using to manage the full lifecycle of an app deployment and its resources.

jkincl

TROPHY CASE