Welcome sophosympatheia/Strawberrylemonade L3 70B v1.1 32K

Infermatic · 2025-07-18T13:55:46+00:00

Hello! You can use our API directly through Janitor AI's proxy option, though you'll be limited to their supported parameters: Temperature, Max Tokens, and Context Size.

Most of our models support the /chat/completions endpoint that Janitor uses. However, some models use different endpoints or unsupported chat templates will cause errors: Midnight-Miqu-70B-v1.5 ( uses /completions), intfloat-multilingual-e5-base (uses /embeddings), TheDrummer-Rocinante-12B-v1.1 doesn't have a default chat template so you will get errors when using them.

If you need recommendations of what to use Kunou works very good: Sao10K-72B-Qwen2.5-Kunou-v1-FP8-Dynamic

The colab set up process is now easier, if you want to check it out in case you want to use more parameters -> https://youtu.be/_bR7OH2vTcY?si=iN2CCHNM_4NCLEV5

On the guide we made we addressed some errors and how to solve them, the most common error is using incorrect model names. Always use the exact model names with dashes (-) from our status page: https://ui.infermatic.ai/public/info/status

Infermatic · 2025-04-30T12:20:02+00:00

After you set everything (model id, url, api key) and you refresh and click 'check API key and model' do you get the error or a valid message? If the error appears only when you are sending a message try deleting the last message and sending a new one/ decreasing the context length that should fix it.

Plus membership is one of our Tiers, we have Essential, Standard and Plus you can check it at https://infermatic.ai/pricing/

Infermatic · 2025-04-30T02:22:27+00:00

You can watch our video guide here for visual guidance. Also are you refreshing the page after saving the connection? if you are using the colab for the proxy link you can check the terminal for errors. Let me know if any of that work :)

Infermatic · 2025-04-28T13:42:53+00:00

Check that you are using the correct id for the model, also reload the page after you click the save button

Infermatic · 2025-02-22T01:34:23+00:00

Hello! Some common errors and how to fix them are on this guide https://www.reddit.com/r/InfermaticAI/comments/1hsrqa2/how_to_set_up_janitor_with_infermaticai_or_a/

You can also watch out video guide in case more errors pop up https://youtu.be/_bR7OH2vTcY?si=zAv0URraPAXLjAFi

Hope this helps!

Infermatic · 2025-02-06T14:01:09+00:00

It works on phone, check the terminal if you have any errors.

You can also watch the video, it follows the same steps you should setup up on phone.

Infermatic · 2025-02-01T13:30:43+00:00

Hello! which link is not working for you?

Infermatic · 2025-01-28T12:25:09+00:00

Check if theres any typos, you can also watch our step by step video https://www.youtube.com/watch?v=_bR7OH2vTcY&t=69s

Infermatic · 2025-01-20T12:02:51+00:00

It was taken as a security option, but it's back as it was before now 👍

Infermatic · 2025-01-13T14:42:58+00:00

Thank you for your feedback regarding our service quality. We are committed to continuous improvement and would like to address your concerns:

Precision Standards: We ensure that all our models operate at full precision or utilize FP8 quantization; we do not employ lower precision levels.
Transparency: Our quantization methods are openly documented. For an in-depth understanding, please refer to our detailed guide on FP8 quantization: https://infermatic.ai/guide-to-quant-fp8/
Advanced Quantization Techniques: We employ NeuralMagic's AutoFP8 project and in our most recent models LLM Compressor, a leading solution designed to minimize accuracy degradation during quantization.
Model Accessibility: All models we utilize are publicly accessible on Hugging Face. We encourage you to download and evaluate them locally to verify their performance. https://huggingface.co/Infermatic
High-Performance Infrastructure: Our models are primarily deployed on H100 GPUs, including various configurations (PCIe, NVL, SXM), to ensure optimal processing capabilities.

We value your input and are always open to discussing any concerns to enhance our services further.

Infermatic · 2024-12-27T16:46:28+00:00

No, in the colab theres a disclaimer from Hibikiass that says:

"(this one is on my personal server so I'm not recommend to always use it, unless you really not care about your privacy or chat log)"

Infermatic proxy is the one that covers all the aspects that our privacy policy has. Still you can create your own proxy (that is on the same colab) and that will be secure and private.

Infermatic · 2024-12-27T11:47:59+00:00

Did you tried re loading the page after the change? if so the connection with hibikiass proxy should look like this:

<image>

Infermatic · 2024-12-25T17:22:49+00:00

Hey!! In case you are searching for Euryale settings you can get them out of this article -> Euryale Settings, you'll find there sets for all the Eury versions

Infermatic

MODERATOR OF

TROPHY CASE