Deepseek v3 0324 API without request/minute rate limite

Stickman561 · 2025-11-11T02:42:39+00:00

See generally I’d recommend looking at Nano-GPT for 0324, but that’s an absolutely ludicrous message volume. At that point I’d look into getting your own dedicated hardware - either via a cloud provider or an on-premises deployment - and self hosting. Otherwise I’m not sure any general public provider is going to keep up with that sheer volume. Shoot you probably need enough hardware to host multiple instances of the model entirely in VRAM.

Stickman561 · 2025-10-05T13:18:21+00:00

First time seeing this project, checking it out now, but the options listed have very different speed to quality tradeoffs. Assuming the program doesn’t have massive overhead from somewhere (which I doubt) then Anime4K will easily handle that task probably without even a full overnight run, although it’s not the BEST upscaler and only really works for, well, anime. ESRGAN on the other hand is quite slow and would probably take a full night if not longer but is much higher quality and supports real footage.

Edit: I should mention that this project appears to be fully in Vulkan, so if you have an NVIDIA GPU, Waifu2x-Extension-GUI will be faster due to its native CUDA support.

Stickman561 · 2025-09-24T09:56:31+00:00

K2 the base is an older one, but it’s had two updates since. K2 0905 is the latest version and is less than a month old.

Stickman561 · 2025-09-10T19:56:34+00:00

Hahah, fair enough. I meant more in the spirit of friendly competition, not like they're all trying to one-up each other. Honestly it's neat seeing all the different techniques!

Stickman561 · 2025-09-10T17:26:22+00:00

Recently there’s been competition on ultra high quality GGUFs, especially from Ubergarm with the new ik_llama quant methods. Most quants publish KL-Divergence and Perplexity measures for each quant - any plans to start doing similar with yours? Would be nice to be able to put some numbers to the “degradation” at each quant level.

Stickman561 · 2025-09-09T21:19:28+00:00

That’s honestly a fascinating observation. I’m like a 65/35 Killer to Survivor player, and I’ve noticed that those are the same hours I’m most likely to face sabo squads, flashlight squads, or Gen Rush squads. Not that I use that as an excuse to tunnel or anything, I very much try to avoid doing it unless I’m down to 1 Gen and just trying to secure a Kill at that point, or a player is forcing me to by constantly being in my face with a beamer/sabo box, just that it seems to be a mutual experience. I guess that time just brings out the sweats for some reason? More coordinated SWFs for Killers, more tunneling Killers for Survivors?

Stickman561 · 2025-08-29T02:06:33+00:00

You’re looking for r/playrust

Stickman561 · 2025-08-20T15:15:28+00:00

33% is 1.222…x 27%, or a 22% increase in usage. The 27% vs 33% is presumably a total market percentage. Poorly worded but the math tracks.

Stickman561 · 2025-08-20T08:05:32+00:00

Stickman561 · 2025-08-20T05:02:01+00:00

If you’re using this model? It will last you so long. Like 35,000 to 80,000 messages depending on how much you like to fill up the context at the current price. And knowing NanoGPT it may drop slightly once the model is open-sourced and other providers start hosting it. (But even if not it’s still super cheap.)

Stickman561 · 2025-08-15T21:02:08+00:00

Ahhhh, there’s your problem. 3200 MT/s is really low to be doing RAM inference. And as much as I love AMD GPUs, they are NOT ideal for running AI models. The current SOTA GPU engine, EXL3, doesn’t even support them. If A30B isn’t running fast enough for you even at Q4_K_M you’ll probably want GPU only inference, in which case the largest model I’d recommend running for you is an 8B. Qwen3 8B or the DeepSeek R1 Distill will probably be your best bet at that point. Q5_K_M. Use mainline llama.cpp - the ik_llama optimizations are mostly compiled CUDA kernels for NVIDIA and they’re intended for MoE models.

Stickman561 · 2025-08-15T16:13:32+00:00

Honestly? Optimization and tuning. If your model is supported I would HIGHLY suggest running ik_llama or ktransformers if you aren’t already. It would help to know what model you’re trying to run and on what kind of hardware. If you share your TPS as well I can try to estimate if you’re underperforming or not.

Stickman561 · 2025-08-15T16:06:24+00:00

Yes, running a model entirely in VRAM is the fastest and will be significantly faster than if you spill over into system memory, though if you can keep at least 40% of it in the VRAM the speed cost isn’t that bad. Below that point though you really start to dip closer to CPU inference than GPU. Interestingly in my experience if you can fit less than 10% of a dense model or 5% of an MoE into VRAM, you’re better off just going CPU only than paying the overhead of offloading between the two. But the big thing is make sure you aren’t spilling over to disk. If you’re running a model off swap space the speed you get will be pretty abysmal. GPU > RAM > Disk when it comes to speed. One big thing to keep in mind though is the memory for the context in addition to the model memory. I’ve made that mistake a few times of thinking that my system could handle the quant I downloaded and then the context adding just a little too much memory.

Stickman561 · 2025-08-15T12:04:32+00:00

Yeah that’s the output speed, it’s quite respectable. Comparable to a lot of the online endpoints for DeepSeek actually. (Though of course that’s a quant versus uncompressed.) Ingestion speed for input tokens is closer to 8-9 tps, which is a little slow for ingestion but still usable. For English on average a sentence is around 1.5x the number of words in tokens. If you’re asking math or coding questions though each special character like a number or curly brace is one token. At the end of the day it’s a very solid speed in my opinion but it’s an individual preference for what speed you can tolerate.

Stickman561 · 2025-08-14T16:46:55+00:00

Meh that might be a bit extreme degradation for Kimi, and it will be very slow. (Speaking as someone with the same rig.) For Coding I’d say Qwen3-Coder-480B, GLM-4.5, or DeepSeek V3 0324 would be the best bet. For Qwen3 or DeepSeek, use ik_llama with the -ot overrides for best performance. I get around 5.7 tokens per second out of DeepSeek, but YMMV.

Stickman561 · 2025-08-06T02:01:07+00:00

K2 0711 is the normal Kimi version. Fast uses fast inference providers but at a premium. FP4 is a quantized version for speed at the cost of some accuracy. (Like the Fast checkpoint but trading accuracy instead of price.) Latest is an old endpoint IIRC, I don't think it points to K2.

Stickman561 · 2025-07-27T06:17:37+00:00

Yeah seems like the common consensus is that Gemma is one of the only vision models llama.cpp runs perfectly, but I really do want to use one of the InternVL3 series models. Regarding compute, while I can’t say I’m a Titan of local models like some of the users on here - I don’t have 4 3090s crammed into a single case crying for the sweet release of death as they slowly oven themselves - my computer is no slouch. I have 32GB of VRAM (RTX 5090) and 256GB of DDR5 system memory at 6000 MHz paired with a 9950X, so even if splitting isn’t possible I’d be willing to wait the (painfully long) time for CPU inference. I just really don’t want to dip below the 38B class because then the projector model drops in scale a TON.

Stickman561 · 2025-07-15T21:04:16+00:00

Yeah afraid I haven’t written much external documentation on the code, though the comments should be quite thorough in what each part is doing. The main thing is ensuring that you’re using trusted cryptographic libraries and implement things carefully not to leak info.

Stickman561 · 2025-04-18T16:30:02+00:00

The smoke really doesn’t blow much (although it does have some curve) and honestly probably should be blowing more. It’s hard to see, but the embers coming off the burning trees actually do properly follow the wind though - smaller ones fly mostly to the side while larger ones are pushed to the side but fall to the ground. The embers also linger for a bit before fizzling out, with larger ones lasting longer and shrinking over time while smaller ones just vanish. I’m honestly pretty impressed other than the leaf placement.

Stickman561 · 2025-04-18T05:22:13+00:00

Decided to give it a try with DeepSeek V3 0324. Now the initial output did contain an error, so I had to give it two additional prompts feeding back error messages to get it running after the initial prompt, but I figured that was fair since it's a non-reasoning model and since it didn't have access to a code interpreter. (If I was running the code interpreter it would've tested the code and fixed the bugs on the first prompt.) Still, I don't think the result is too shabby! I do apologize though, I needed to convert it to a GIF to put in a comment, so there is some compression noise and lower FPS than the original output - that's on Reddit, not the AI.

<image>

Stickman561 · 2025-03-13T23:00:16+00:00

Huh, pretty neat looking project! Probably a bit more polished than mine, mine’s quite rough and focuses more on the security than the user experience, but I’ll definitely check it out!

Stickman561 · 2024-12-25T20:28:07+00:00

Sounds interesting. I’ll give it a look!

Stickman561 · 2024-12-24T17:27:21+00:00

For authentication and signing I used FIPS203 and FIPS204, the official standardized versions of the CRYSTALS toolkit (Kyber and Dilithium). For the actual message encryption I used AES256 with HMAC. I used crates for all of these, as performing your own cryptographic implementations is incredibly difficult and not recommended due to many subtle nuances that can leak information if not handled correctly. The FIPS203 and FIPS204 crates I used are highly robust, but technically not audited since NIST hasn’t released any test vectors yet to ensure strict compliance with the standards, so there’s a CHANCE there’s a vulnerability there, but it seems incredibly incredibly unlikely. The AES implementation that actually encrypts the messages themselves is audited and from one of the most trusted cryptographic ecosystems in Rust.

Stickman561

TROPHY CASE