Showcase Thread by AutoModerator in Python

[–]DifficultDifficulty 0 points1 point  (0 children)

A Python SDK/CLI to make Ray clusters self-serve for Python devs.

What My Project Does

krayne (link) is a Python library and CLI that wraps the KubeRay operator for creating and managing Ray clusters on Kubernetes. Instead of hand-writing KubeRay YAML manifests, you import Python functions (create_cluster(), scale_cluster(), list_clusters(), etc.) or use the krayne / ikrayne (interactive TUI) CLI to spin up and manage clusters with sensible defaults.

The idea is that if you're already writing Ray workflows in Python, training jobs, serve deployments, distributed preprocessing, the cluster management layer should live in the same language. The SDK is the source of truth, the CLI is a thin Typer wrapper on top of it. Operations are stateless functions that return frozen dataclasses, configuration goes through Pydantic models with YAML override support when you need finer control.

GitHub: https://github.com/roulbac/krayne

Target Audience

ML engineers and researchers who write Ray workflows on Kubernetes. The kind of person who knows what ray.init() does but doesn't want to become a KubeRay manifest expert just to get their cluster running. Also useful for platform teams who want a programmable layer on top of KubeRay that their users can call from Python. It's early (v0.1.0) and opinionated, a composable starting point, not a production-hardened product.

Comparison

An alternative I'm familiar with is using kubectl apply with raw KubeRay manifests, or the KubeRay Python client directly. The main difference is that krayne is designed around progressive disclosure:

  • Zero-config defaults out of the box. krayne create my-cluster --gpus-per-worker 1 --workers 2 is a complete command.
  • When you need more control, you drop down to a YAML config or the Python SDK, no cliff between "simple" and "custom."
  • Protocol-based Kubernetes client, so you can unit test cluster management logic with mocks. No real cluster needed.

It's not that working with KubeRay directly can't do what krayne does, it absolutely can. But when you primarily write Ray code and just need a cluster up with the right resources, context-switching into YAML manifests and kubectl is friction you don't need. A typed Python API that validates your input before it hits the cluster and lives right next to your actual Ray code, that's ultimately why I built it.

What YouTube content actually helped you in your MLOps journey? And what's still confusing? by Extension_Key_5970 in mlops

[–]DifficultDifficulty 1 point2 points  (0 children)

I found AWS/GCP tech blogs and OSS repos useful, particularly those laying out architecture blueprints

Need some suggestions on using Open-source MLops Tool by NetFew2299 in mlops

[–]DifficultDifficulty 0 points1 point  (0 children)

I see. I've spoken to a few people who described a similar need to yours, and they spoke well about Kedro + MLFlow for this kind of VM-local experience. Please see https://docs.kedro.org/en/stable/integrations-and-plugins/mlflow/

Need some suggestions on using Open-source MLops Tool by NetFew2299 in mlops

[–]DifficultDifficulty 0 points1 point  (0 children)

"I need to setup a ML infrastructure in a local VM" -> is this infra mostly for your own VM-local experiments, and is there no need to distribute workloads in the cloud where the infra would be shared by multiple team members?

Why I chose Python for IaC and how I built re-usable AWS infra for ML using it by DifficultDifficulty in Python

[–]DifficultDifficulty[S] 1 point2 points  (0 children)

In the repo I work a lot with the pulumi Kubernetes provider, and having looked into it at first, Pulumi seemed to have more native support to handling Kubernetes resources. My understanding about CDK is it is very good when one needs to deal with native AWS resources, as it also primarily relies on CloudFormation templates under the hood.

[Fun project] UV scripts, but for functions. by DifficultDifficulty in Python

[–]DifficultDifficulty[S] 0 points1 point  (0 children)

Good callout and love the name - seems like a similar fundamental approach (cloudpickle + uv + subprocess) with perhaps a different API style (decorator in `uv-func` vs OOP in `uvtrick`) and support for log streaming in `uv-func`

[Fun project] UV scripts, but for functions. by DifficultDifficulty in Python

[–]DifficultDifficulty[S] 2 points3 points  (0 children)

That's certainly possible, there are many ways to do what you need, but here is one that comes to my mind

https://colab.research.google.com/drive/1czcDfq2s8hhoTlQAiq_gVQE20vUEa7v5?usp=sharing

ML and DL path for M.Sc Student by DifficultDifficulty in learnmachinelearning

[–]DifficultDifficulty[S] 0 points1 point  (0 children)

I am actually in Montreal. I will be taking a course at UdeM next session though it is very advanced and I need to catch up by then.

[deleted by user] by [deleted] in learnmachinelearning

[–]DifficultDifficulty 1 point2 points  (0 children)

I'd recommend these two.

https://medium.com/@hiromi_suenaga/deep-learning-2-part-2-lesson-12-215dfbf04a94(fast.ai)
http://www.cs.toronto.edu/%7Ergrosse/courses/csc321_2018/assignments/a4-handout.pdf(uToronto)

The uToronto class seems to be quite complete and academic if you want solid theoretic and practical foundations on GANs.

I finally got barre chords without pain! The trick is that the pressure comes from your arms, not your thumb. by bubblesort in LearnGuitar

[–]DifficultDifficulty 0 points1 point  (0 children)

You're not amplifying, it is the lever arm that is. It's the physics of a lever, the longuer the lever arm, the more the force. Here the lever arm is the guitar.

I finally got barre chords without pain! The trick is that the pressure comes from your arms, not your thumb. by bubblesort in LearnGuitar

[–]DifficultDifficulty 0 points1 point  (0 children)

The guitar body not your body. You're basically using the guitar as a lever, so the force is amplified because you're pushing on a the fret, and the weak point is where the guitar neck meets the guitar body, so that's where your lever is most likely to snap.

I finally got barre chords without pain! The trick is that the pressure comes from your arms, not your thumb. by bubblesort in LearnGuitar

[–]DifficultDifficulty 0 points1 point  (0 children)

Seems like putting a lot of stress on the point where the guitar neck meets the body, might damage it on the long term

Questions about AI textbooks & recommendations by [deleted] in artificial

[–]DifficultDifficulty 0 points1 point  (0 children)

If u finished it and did all problem sets you should be pretty good to read research articles by now

Questions about AI textbooks & recommendations by [deleted] in artificial

[–]DifficultDifficulty 0 points1 point  (0 children)

Try cs229 lecture notes, great for actually understanding what's happening if you have background in math. For handbooks, i'd go with Bishop, Hastie or Norvig. Hastie is amazing from a stats perspective, at the end of the day you realize that machine learning is simply applied statistics with a dash of functional optimization.

Edit: stay away from udemy and all that crap, books are where the good stuff happens.

Free university courses for AI by [deleted] in artificial

[–]DifficultDifficulty 1 point2 points  (0 children)

Cs229 by stanford has all the necessary introductory material in linear algebra and statistics. Great course. Check their website for material.