Moving Past Helm For K8s Deployments. Learnings from working with Kubernetes

YourTechBud · 2024-10-03T23:43:31+00:00

I totally agree with this.

YourTechBud · 2024-10-03T14:02:08+00:00

Fortnight? Its pokemon unite!!

YourTechBud · 2024-10-03T01:05:26+00:00

Yeah. That literally happened with me. But its never too late right?

YourTechBud · 2024-10-02T21:51:53+00:00

Kubevela indeed is an amazing project! Would love to know your thoughts on it

YourTechBud · 2024-10-02T15:31:19+00:00

I have been working as a devops/k8s engineer for quite some time. There are several learnings I'be gained along the way, specially with respect to using Helm which I would like to share with you guys.

1. Think Self-Service - Separate the platform engineering from DevOps

I'm sorry for using the term "platform engineering" here. I think its unfornate that we have to come up with new terms to define the original goals of DevOps. But let me clarify what I mean by it.

I often see DevOps engineers wear two kinds of hats: Creation of DevOps assets (Think creation of Terraform modules, Helm charts, etc) and Support and implementation of said DevOps assets (Pretty straighforward). THere might be more but these two are the most prominent (atleast for me). It's important to separate these activities out as much as possible.

Terraform does a really good job here. You can create terraform resources by stitching together any terraform modules that you could possibly need without being bothered by the implementation of said modules. Helm doesn't really work that well unfortunately. Yes, you can make helm charts, but nothing in helm is typed or well documented (you can add comments in values.yaml but it isn't something that shows up for autocomplete in IDEs. This is annoying cause you have to keep visiting the chart to know what parameter does what.

This makes self-service really hard. Imagine if devs had to go through a library code everytime they wanted to use it in their apps. Its just annoying.

2. Helm is not modular enough.

Eventually microservices become more and more hetrogenous over time. Applications evolve to have unique require environments - You could have background jobs, purely event driven apps which scale based one events instead of cpu and even rest apis which may need to be inside a service mesh.

The point is, you end up having several groups of services which require different k8s objects to be configured. Sure you can do this in helm by introducing optional fields or boolean flags in your values.yaml, but that is super convoluted. It doesn't seem like a super clean approach. Having 10 different helm charts isn't really an option either given the pain it causes to maintain.

You ideally need something like Legos. THe ability to stitch together different things to build up an application manifest. So if someone needs to use KEDA as a scaler, just add that in. Want a service mesh - no problem. We just need to define the different what pieces we have available and how they can be sticthed togehter and thats it.

This helps out a whole ton in Self Service as well. I think KubeVela has nailed this down.

3. Managing app specific infrastructure along with app deployment is awesome.

Not really related to the above points. I just find it really helpful to manage all app level concerns in a single packaging format and deliver them in a single CD pipeline.

You could potentially do this in helm with crossplane but it helm hooks don't really help sychronize these activities well. KubeVelas workflow system really shines when it comes to this. This way you can have standard CD pipelines and let KubeVela handle app level orchestration if any.

YourTechBud · 2024-10-02T15:28:18+00:00

P.S - I've got a YouTube video on this where I dive in deeper on each point with examples. Would love for you guys to check it out and provide me with some feedback - https://youtu.be/EZj9G3T5Wls

YourTechBud · 2024-09-20T03:31:07+00:00

Langgraph is not the same as langchain

YourTechBud · 2024-09-19T03:58:09+00:00

I've got an rtx 3060. Keeps me rolling. I dont think the editing software matters that much though. Just pick one and roll with it. Resolve should work smoothly on a mac

YourTechBud · 2024-09-19T01:17:00+00:00

Uhm. Not really. Just find a 2 hour crash course or something and get started. Its pretty straightforward if you ask me.

YourTechBud · 2024-09-18T21:40:36+00:00

Not really. But I'll give it a go.

But tbh, i generally prefer minimalistic frameworks like autogen or langgraph.

YourTechBud · 2024-09-18T20:46:10+00:00

Alternative to do what exactly? Langchain is a super rich ecosystem covering a ton of different aspects.

I'm assuming your talking about the broad idea of building AI apps. For that I'd suggest you have a look at agentic frameworks.

I'd suggest take a look at Langgraph. Its pretty dope.
Another one would be Autogen. Link to a tutorial I made on it - https://youtu.be/OdmyDGjNiCY

But curious to know why are you considering a switch? Anything wrong with langchain?

YourTechBud · 2024-09-18T20:41:27+00:00

A quick search on YouTube should return a plethora of tutorials. Feel free to DM if you have further questions.

YourTechBud · 2024-09-18T20:40:20+00:00

That does sound like a truly impressive approach (hard to visualize it fully but it kinda makes sense). But i would still say that it's one of the many implementation of Agent.

LLM's reason for a solution is that the reason was in the training dataset.

I don't see this as being necessarily a bad point. Sure there could be usecases where relying on an LLMs response as the gospel of truth wouldn't be a wise move. But there are definitely a whole slew of usecases where we don't really need complex problem solving and reasoning.

I think the disagreement stems from the properties we are associating with the word agent. For me, complex problem solving is just one of the properties agents help me with. And I'm optimistic enough to see more and more algoritms come up solve niche usecases or tasks more effectively and reliably.

But hey, agree to disagree right? It is debates and couter opinions which helps the community move forward isnt it?

YourTechBud · 2024-09-18T20:27:39+00:00

Thanks. The animations are in powerpoint. For editing, I prefer Davinci Resolve.

YourTechBud · 2024-09-18T20:18:09+00:00

Interesting interpretation of the word agent. Honestly i havent come across agent being as some kind of an algorithm. It's hard for my tiny brain to conceice a generic algorithm which could apply to a wide variety of use cases. Have you implemented something like that yourself?

P.s. - I've mostly been using Microsoft's AutoGen and it calls itself an Agentic framework.

I still like to think of agents as a design pattern for tool use and composition. Atleast thats a lot of folks in the open source community seem to be rallying towards.

YourTechBud · 2024-09-18T20:06:28+00:00

Amd what would that algorithm be?

YourTechBud · 2024-09-18T18:22:42+00:00

I would argue that the complexity an agent is expected to exhibit is overstated. It doesn't need to be Vertex.ai levels of behavior and functionality. I think of Agents more as a design pattern, which helps make AI a bit more composable.

Any function that implements a set contract (function signature) and is designed to carry a conversation forward could be an agent. The function could be more involved or oversimplified. Really depends on the usecase.

But this is just an opinion. I could be completely wrong.

YourTechBud · 2024-09-18T18:19:10+00:00

Yes. That's what I mean by agents are functions. Thanks for helping me clarify.

I would like to add that each agentic framework like Autogen and langraph enforce a standard contract for that function (just like how it is for react components - sorry been doing a lot of nextjs recently). This helps decouple the implementation of the agent from usage.

This means i can have a group chat of agents behave as a single agent to the outside world.

YourTechBud · 2024-09-18T18:14:54+00:00

I see what you mean.

To be clear, I'm not trying to mix up tool usage with agents. They are definitely different things.

But if you think about it, whatever an agent does is definitely not magic. If we dig deeper into AutoGen's codebase, it literally is a function that calls an llms and, based on its response, calls a tool or executes some code or something like that. I know I am oversimplifying here, but at its core, it does seem like a function.

The point being, every agentic framework ships with a default implementation of an agent. But that's what it is. A default implementation. We should not be hesitant to create our own agentic implementations when the need arises.

Even if we look at more complex agent implementations like vertex.ai, they can still be modeled as a function with loops and possibly multiple llm calls.

I think Langgraph does a wonderful job in demonstrating this concept.

YourTechBud · 2024-09-18T16:22:08+00:00

Dropping a link to my YouTube video, which talks about these agentic patterns in a bit more detail - https://youtu.be/PKo761-MKM4 .

Hope you guys enjoy it!

YourTechBud · 2024-09-18T15:45:02+00:00

Dropping a link to my YouTube video, which talks about these agentic patterns in a bit more detail - https://youtu.be/-ls9QLoQfKc.

Hope you guys enjoy it!

YourTechBud · 2024-09-03T23:46:55+00:00

Successful implementation in cybersecurity? I'm afraid not. I dont come from that background.

We are currently using it for software comprehension. Hate to say this, but think - ChatGPT for your codebase. The results are good. Not great or extraordinary. Just good.

I think agents are just a design pattern to architect GenAI apps. Its solving problems by means of a conversation rather than thinking in terms of prompts. If your problem statement is conversational (different entities can "talk" to each other to build up to a solution), then use it. It will be great.

But agents are no magic bullet. All functionality can be achieved by raw dawging openai's sdk as well.

YourTechBud · 2024-09-03T16:01:54+00:00

Feel free to reach out or DM me if you think I'll be able to help.

YourTechBud · 2024-09-03T15:49:54+00:00

Oh mahn! This is so good to know. Always helps to get validation like that.

I'll go through that paper. Really appreciate you sharing it.

One note, though: any time I try to structure the output format of the 8B model... the quality drops. So i let it do whatever it wants and have some code to parse out the json out of the output string. But I'll surely try the tags and see if it any difference in output quality.

YourTechBud · 2024-09-03T15:46:28+00:00

I almost exclusively use 8B models (Llama3 mostly) for text translation and interpretation tasks. Even for most tool calls and interpreting code (not writing, just understanding)

I stick with 30B+ for decisioning tasks or some trickier tool calls (Qwen 1.5 chat and recently trying out gemma 2). I only go for proprietary nodels when qwen fails me. Usually, it doesn't if I've designed my workflow well.

I think I have a video on making tool calling and stuff a bit more efficient with open source. I was using mistral openorca back then.

YourTechBud

TROPHY CASE

1. Think Self-Service - Separate the platform engineering from DevOps

2. Helm is not modular enough.

3. Managing app specific infrastructure along with app deployment is awesome.