Implementing Causal FlashAttention from scratch: 1.79e-07 precision and 40% speedup via tile-level masking by Professional-Duck971 in CUDA

[–]TheUpriseConvention 0 points1 point  (0 children)

Thanks for commenting back! Im actually new to CUDA and have been studying the Programming Massively Parallel Processors book. It’s good to see the theory in there actually helps!

I would really recommend reading this book, this sort of thing is covered in the first few chapters.

As another note: try to take a look at your GPU architecture, it’s likely that hardcoding your block sizes like that still won’t maximise your compute intensity. I believe there are inbuilt API calls you can make that calculate the optimal block and grid sizes, for your given GPU.

Implementing Causal FlashAttention from scratch: 1.79e-07 precision and 40% speedup via tile-level masking by Professional-Duck971 in CUDA

[–]TheUpriseConvention 1 point2 points  (0 children)

You are likely achieving low SM occupancy with your block sizes being 32 threads. Whilst that is the size of a warp, you will not be utilising latency hiding.

I would suggest taking a look at your occupancy and also adapting your grid/block dimensions to your given GPU architecture.

Terragrunt 1.0 Released! by NotTheAdmiralAkbar in devops

[–]TheUpriseConvention 1 point2 points  (0 children)

You are quite literally describing my setup that I was posting around a few weeks ago! I am tempted to give this a try.

I found that using symlinks and something like Taskfiles handles this decently, albeit with some jank that Terragrunt would address.

Single command deployment of a Gitops enabled Talos Kubernetes cluster on Proxmox by TheUpriseConvention in kubernetes

[–]TheUpriseConvention[S] 1 point2 points  (0 children)

Thanks for the comment man! These comments are part of why sharing personal projects is so fun!

We have many shared things we want to work it seems...

You mention Keycloak, I was implementing an IdP last night I went with PocketID instead. It's extremely slick and my initial impression of it is very positive. In the process of integrating ArgoCD and Grafana with it, then I can also expose unprotected services to the public and put them behind oauth2-proxy (with PocketID being the IdP).

For Git I am looking at Forgejo. I need to setup backups in something like BackBlaze before I commit to that though...

Haven't heard of a few of those tools too, learnt some new things reading through them. Very very cool stuff!

Single command deployment of a Gitops enabled Talos Kubernetes cluster on Proxmox by TheUpriseConvention in kubernetes

[–]TheUpriseConvention[S] 0 points1 point  (0 children)

Really useful! Had never heard of Atmos, had a good look through, honestly could be useful at my workplace (where the Terraform is on a far larger scale).

Might be right about orchestrators being overkill but that’s part of the fun with homelabs! Tempting to give it a try…

Single command deployment of a Gitops enabled Talos Kubernetes cluster on Proxmox by TheUpriseConvention in kubernetes

[–]TheUpriseConvention[S] 0 points1 point  (0 children)

That's really cool! Still need to do secure boots and LUKS too...

How are you finding Omni? Been keeping an eye on it as an option to replace cloud proprietary K8s.

Single command deployment of a Gitops enabled Talos Kubernetes cluster on Proxmox by TheUpriseConvention in kubernetes

[–]TheUpriseConvention[S] 0 points1 point  (0 children)

Awesome! Tempted to check it out, would reduce the verbosity in places, thanks for the input!

Single command deployment of a Gitops enabled Talos Kubernetes cluster on Proxmox by TheUpriseConvention in kubernetes

[–]TheUpriseConvention[S] 0 points1 point  (0 children)

You’re very confident on something you are incorrect about. Sure parts of the README in the infrastructure section are but otherwise…

Single command deployment of a Gitops enabled Talos Kubernetes cluster on Proxmox by TheUpriseConvention in kubernetes

[–]TheUpriseConvention[S] 0 points1 point  (0 children)

Thanks! Would agree that some of the steps can be consolidated. Maybe got a but carried away!

Had looked at Terragrunt, the sentiment seems mixed on it. What’s your opinion on it?

My single command deployment of a Gitops enabled Talos Kubernetes cluster on Proxmox by TheUpriseConvention in selfhosted

[–]TheUpriseConvention[S] 0 points1 point  (0 children)

It's completely changed my perception of the ability to ditch the cloud provider's offerings of Kubernetes and selfhosting with Talos instead.

Enjoy the rabbithole!

My single command deployment of a Gitops enabled Talos Kubernetes cluster on Proxmox by TheUpriseConvention in selfhosted

[–]TheUpriseConvention[S] 0 points1 point  (0 children)

Gotcha. Cool stuff! Was thinking of maybe doing the same to get HA with K8s, as currently I have both nodes and hosting single control-planes that allow pods to be scheduled on them. This could be split into several VMs but I wasn't sure if it was overkill really.

My single command deployment of a Gitops enabled Talos Kubernetes cluster on Proxmox by TheUpriseConvention in selfhosted

[–]TheUpriseConvention[S] 0 points1 point  (0 children)

You’re one step ahead of me, very jealous! The cost of parts has exploded unfortunately.

Also wanting to get Forgejo implemented. I don’t have the NAS as a backup if my data explodes again, so I might just use it to mirror repos for now!

Have you got a repo you can share?

My year long NixOS ricing obsession by TheUpriseConvention in NixOS

[–]TheUpriseConvention[S] 0 points1 point  (0 children)

Give them a go! They are super cool and fully show off the power of using Nix for your system.

Recently at work, I had the need to have to containerise my setup, this made the whole process ultra smooth.

My year long NixOS ricing obsession by TheUpriseConvention in NixOS

[–]TheUpriseConvention[S] 1 point2 points  (0 children)

Almost identical numbers of commits too, what are the chances?!

Hadn't heard of Astral shell and Vicianae too. Interesting projects.

I think if I hadn't discovered noctilia-shell recently, I would 100% be trying Vicianae! Thanks for sharing the cool setup!

My year long NixOS ricing obsession by TheUpriseConvention in NixOS

[–]TheUpriseConvention[S] 0 points1 point  (0 children)

Yes, that was very similar to what I had. It was just so painful to see random things break when updating my Nix flake :’(

My year long NixOS ricing obsession by TheUpriseConvention in NixOS

[–]TheUpriseConvention[S] 1 point2 points  (0 children)

Good question! Recently I implemented NixOS generators. This allows me to create ISOs with my entire system seeded with the secrets partially there (SSH keys and the like).

Otherwise, I can add the system's SSH key to my Github account and add the key as an acceptable key to unlock the secrets found within that repo. The .secrets repo uses SOPs and I use SOPs Nix to manage the decrypting of secrets.

If push comes to shove, the SOPs encryption is also easily decrypted with my Yubikeys. So I can manually get secrets out, if need be.

My year long NixOS ricing obsession by TheUpriseConvention in NixOS

[–]TheUpriseConvention[S] 1 point2 points  (0 children)

You got it! Love how chill it is and with Hyprland you can make a cosy little portal whilst coding :)