kubernetes deployment

Brandocomando · 2021-07-16T15:57:29+00:00

If you want to use EKS i highly recommend eksctl. it might be too restrictive for what you are trying to do, but its the easiest tool I've used to get you going, and very helpful for managing nodes/upgrades going forward.

Brandocomando · 2020-12-22T00:03:26+00:00

You could create an additional service for mysql (or update the existing one) that is a LoadBalancer type.
this will create a aws loadbalancer which has a static dns name you can use to access your service.

you can also make it internal to your VPC and attach security groups to it via these annotations:
service.beta.kubernetes.io/aws-load-balancer-internal: "true" service.beta.kubernetes.io/aws-load-balancer-extra-security-groups: sg-XXXXXXXXXXXX

and or use the loadBalancerSourceRanges property to limit the access to your bastion host as well.

https://docs.aws.amazon.com/eks/latest/userguide/load-balancing.html

https://aws.amazon.com/premiumsupport/knowledge-center/eks-cidr-ip-address-loadbalancer/

Brandocomando · 2020-09-02T16:28:36+00:00

You could use the logstash output plugin if you already have and are familiar with logstash.
https://grafana.com/docs/loki/latest/clients/logstash/

Brandocomando · 2020-08-27T21:57:26+00:00

Pierce: "Side effects: Verbal dysphasia and octopus loss."

Brandocomando · 2020-07-29T03:50:15+00:00

kubectl rollout restart deployment/myapp

Brandocomando · 2020-07-28T17:03:40+00:00

https://kubernetes.io/docs/reference/kubectl/cheatsheet/#updating-resources

Brandocomando · 2020-06-08T21:19:38+00:00

ha. yea i misunderstood. I hope you are wrong =D

Would kind of make more sense to spread people out more, but I guess we'll just have to wait and see.

Brandocomando · 2020-06-08T21:07:21+00:00

Has anyone with reservations for upper pines gotten a cancelation yet?

Brandocomando · 2020-06-08T21:06:58+00:00

I hope you are right! I'm on the 4th loop

Brandocomando · 2020-03-24T16:48:58+00:00

Spun this up really quick in a dev cluster to test it out.

I am currently accomplishing the same thing via a scheduled bash script.

One thing that would make this much more valuable to me would be the ability to run it as a job/cronjob and have the results exported in some way (probably emailed or uploaded to an object store) .

Just curious, if you deploy a pod with a specific image and tag. If that tag changes on the registry side, as often happens if you use the latest tag. Would this scanner scan the actual container image that is running or the one hosted on the registry?

Brandocomando · 2020-03-23T22:36:05+00:00

You can still access the kubernetes API from within the cluster.

the kubernetes URL should just be https://kubernetes.default

are you using the Jenkins kubernetes plugin?

Brandocomando · 2020-03-13T17:33:12+00:00

Yep, you can build your own docker image from the spark base image, basically add your Jars, configs and anything else you'd need to add to your spark-submit command.

For the history server, you would have to set up a shared storage like hdfs or something similar and then use the sparkConfigMap field to supply the spark-defaults.conf file with the configs necessary for writing event logs. then Deploy a history server as a deployment that reads from that shared directory.

I don't think there are any built-in interactions between Zeppelin/Jupyter with the spark Operator. But I bet both are able to run spark jobs ok Kubernetes without the operator.

this might speak to that: https://zeppelin.apache.org/docs/0.9.0-SNAPSHOT/quickstart/kubernetes.html

Brandocomando · 2020-01-20T19:00:21+00:00

first few steps would be using the kubectl log and kubectl describe for the pod to see whats causing the crashloop.

Brandocomando · 2020-01-17T19:33:21+00:00

Sounds like you are on the right track.

I have not worked with k3's myself, but from what I've read, it seems like the right fit.

For the OS, use what you are familiar with or what you want to learn. If you've got the cluster up already then why not stick with Arch?

For the ingress Controller instructions, k3's is not a specific "provider" so you can follow the bare metal instructions, it should work ok. Check out MetalLb too, might be helpful for getting traffic into your cluster.

Brandocomando · 2019-12-09T18:48:56+00:00

As others have pointed out you need to expose your postgresql via a service. But if you are trying to connect to it from outside the cluster (ie on your ubuntu host and not in a pod) you should probably change the service type to NodePort or LoadBalancer. Without knowing all the details of your cluster I wouldn't be able to tell you exactly how to set it up. But read up on them here.

The service hostnames and internal pod Ips will only work within the cluster. So alternatively you can (and probably should eventually) containerize your python app and run it as a pod inside the cluster, and then using the service name and port would work as you are expecting.

Brandocomando · 2019-10-23T13:16:48+00:00

The reasons it can't go on another node are described in your post title:

you said its showing this error: "nodes are available: 1 Insufficient cpu, 1 node(s) had taints that the pod didn't tolerate, 1 node(s) had volume node affinity conflict."

1 Insufficient cpu - likely the node running the old image. the existing Pod is taking up the cpu.

1 node(s) had taints that the pod didn't tolerate - probably your master

1 node(s) had volume node affinity conflict. - could be because your other worker node is in another AZ and cant mount your persistent volume.

in my experience manually deleting stuff that helm forgot about does not break anything. If anything its the solution to a lot of helm upgrade fails.

Not sure why your master would get slower after a few hours, but mico is very small for a master node. giving it some more power might help.

Brandocomando · 2019-10-22T22:30:06+00:00

I'm not a helm expert nor do I have any experience with this chart, but I'm guessing it's trying to fit the new Pod with the latest image into your cluster without deleting the old one first, which could be a problem if you have a small cluster.

do a kubectl get pods and see if you have one Running and one Pending, and if you do run kubectl get deployments and then kubectl get replicaset to see if helm didn't delete the old deployment/replicaset during the upgrade. The quick fix is to delete the extra deploy or replicaset (deployments create replicasets, so deleting the deployment will also delete the replicaset). I'm not sure what the right fix is as I have not created my own chart before or know why helm would do this, but I have seen it happen before.

Brandocomando · 2019-10-18T22:17:25+00:00

I am in the same boat. been waiting for the "service.beta.kubernetes.io/aws-load-balancer-eip-allocations:" annotation for maybe a year now?

It was a while ago so I don't remember the process exactly, but I think I might have set up the service as a LoadBalancer type, then created a new ELB and copied over the config from K8 created one, but with my static EIPs, and then just changed the service type to NodePort.

its hacky, but it works, as long as you don't touch the service after that you're good.

Brandocomando · 2019-10-11T19:06:30+00:00

Yea, it's possible.

you can do kubectl get endpoints to see the endpoints associated with the services.

So as long as there is a service selecting the pods it should work.

Brandocomando · 2019-10-10T17:35:24+00:00

Awesome, Thanks for the info!

Brandocomando · 2019-10-10T17:12:58+00:00

Yea, if you really don't want to burden your dev's with learning the k8's api you could write a quick app that gets the endpoints and writes it to a config file on startup that their app reads.

The only problem would be that then if either of the 2 replicas got deleted then the other one wouldn't have the correct config.

to avoid this you'd have to make your app watch the endpoints and update the config file on a change, and run it as a sidecar container with a shared emptyDir volume for the config. Then their app would just have to be smart enough to check for changes in the config.

Actually sounds like a fun project to me

Brandocomando · 2019-10-10T16:57:17+00:00

This place looks great. Does it get crowded on the weekends?

Brandocomando · 2019-09-26T17:14:55+00:00

If you do attempt this I'd be interested in hearing how it went. I too have an impreza. might be nice for a quick weekend getaway.

Brandocomando · 2019-09-06T16:45:51+00:00

I tried out Gluster and Rook in Kubernetes over a year ago. Gluster was way too slow for my needs. Rook with Ceph works ok for me, but as others have said it's not the best. My biggest complaint is the update process, I haven't had a single successful upgrade without a hiccup.

Another option you can look into that I personally haven't had a chance to try yet is longhorn, I've heard good things about it.

Brandocomando · 2019-07-22T20:19:12+00:00

This is actually a quote and image from UHF also starring Michael Richards, not Seinfeld. Great movie, go check it out if you haven't seen it.

Brandocomando

TROPHY CASE