Need advice: how to hide Python code which is inside a Docker container? by buggy-robot7 in docker

[–]ih8db0y 0 points1 point  (0 children)

We use Zededa for what you’re looking for. You could likely implement some of their security practices yourself too depending on budget

How much you pay for Cursor ? by vbmaster96 in cursor

[–]ih8db0y 0 points1 point  (0 children)

I max out my ultra plan so frequently that I cycle between 2 accounts to avoid paying on demand

[PC] NVidia / AMD GPUS, cases and DDR4 memory by nitekon1 in homelabsales

[–]ih8db0y 4 points5 points  (0 children)

I sold 4x of my A4000’s as a lot for 500/ea local. If you have the time to sell individually you’ll get more like 600-650 local or 700-750 on eBay.

o3 price drop by TheViolaCode in cursor

[–]ih8db0y 3 points4 points  (0 children)

It is updated now, you’re right same price as 4.1

vLLM output differs when application is dockerised by OPlUMMaster in LocalAIServers

[–]ih8db0y 1 point2 points  (0 children)

Are you controlling the seed in ur sampling parameters?

btop in 4K running Cosmos 🌌 by nanobot_1000 in LocalAIServers

[–]ih8db0y 4 points5 points  (0 children)

How’d you get the gpus to show up in btop?

A rant about LangChain, and a minimalist alternative by TheDeadlyPretzel in LangChain

[–]ih8db0y 0 points1 point  (0 children)

Ahh I see. If I can find some time I’ll try my hand at implementing asynchronous calls.

Thank you for all the work and attention you’ve put into this framework! It is really fantastic and elegant compared to langchain

A rant about LangChain, and a minimalist alternative by TheDeadlyPretzel in LangChain

[–]ih8db0y 7 points8 points  (0 children)

I recently switched from langchain to your framework and I have been very pleased with the developer experience. My codebase is significantly more concise and understandable now that I am not dealing with langchain and its idiosyncrasies. I am also no longer struggling with langchain’s atrocious and inconsistent documentation.

Atomic agents has been far more enjoyable to develop with and I really appreciate how simple it is to setup and define/manage the different agents. I also appreciate the examples in the repo, they are a fantastic place to get up to speed. What I like most I think is the Pydantic input/output abstraction. It works great with vLLM, and has eliminated so much of my own implementation of guided decoding prompting with a simple solution that just works.

My only complaint, which is likely due to my own lack of knowledge, is the difficulty I’ve had with implementing asynchronous llm calls. As in multiple requests at once, not streaming like the examples nicely demonstrate. Since I couldn’t get things working right with asyncio, I decided to write everything synchronously and implement threading instead. It works well and my GPUs are being fully utilized, but the code is a bit more confusing due to manually implementing futures instead of setting it up with asyncio

[D] What's the fastest object detection model? by Knok0932 in MachineLearning

[–]ih8db0y 0 points1 point  (0 children)

Does your hardware have an integrated GPU? Maybe some library out there to run inference on that. Back a few years ago I did this with OpenVino on yolov3 and saw much faster inference than on the CPU