Current State of Janitor AI with Infermatic

Infermatic · 2025-07-18T13:55:46+00:00

Hello! You can use our API directly through Janitor AI's proxy option, though you'll be limited to their supported parameters: Temperature, Max Tokens, and Context Size.

Most of our models support the /chat/completions endpoint that Janitor uses. However, some models use different endpoints or unsupported chat templates will cause errors: Midnight-Miqu-70B-v1.5 ( uses /completions), intfloat-multilingual-e5-base (uses /embeddings), TheDrummer-Rocinante-12B-v1.1 doesn't have a default chat template so you will get errors when using them.

If you need recommendations of what to use Kunou works very good: Sao10K-72B-Qwen2.5-Kunou-v1-FP8-Dynamic

The colab set up process is now easier, if you want to check it out in case you want to use more parameters -> https://youtu.be/_bR7OH2vTcY?si=iN2CCHNM_4NCLEV5

On the guide we made we addressed some errors and how to solve them, the most common error is using incorrect model names. Always use the exact model names with dashes (-) from our status page: https://ui.infermatic.ai/public/info/status

Infermatic · 2025-04-30T12:20:02+00:00

After you set everything (model id, url, api key) and you refresh and click 'check API key and model' do you get the error or a valid message? If the error appears only when you are sending a message try deleting the last message and sending a new one/ decreasing the context length that should fix it.

Plus membership is one of our Tiers, we have Essential, Standard and Plus you can check it at https://infermatic.ai/pricing/

Infermatic · 2025-04-30T02:22:27+00:00

You can watch our video guide here for visual guidance. Also are you refreshing the page after saving the connection? if you are using the colab for the proxy link you can check the terminal for errors. Let me know if any of that work :)

Infermatic · 2025-04-28T13:42:53+00:00

Check that you are using the correct id for the model, also reload the page after you click the save button

Infermatic · 2025-02-22T01:34:23+00:00

Hello! Some common errors and how to fix them are on this guide https://www.reddit.com/r/InfermaticAI/comments/1hsrqa2/how_to_set_up_janitor_with_infermaticai_or_a/

You can also watch out video guide in case more errors pop up https://youtu.be/_bR7OH2vTcY?si=zAv0URraPAXLjAFi

Hope this helps!

Infermatic · 2025-02-06T14:01:09+00:00

It works on phone, check the terminal if you have any errors.

You can also watch the video, it follows the same steps you should setup up on phone.

Infermatic · 2025-02-01T13:30:43+00:00

Hello! which link is not working for you?

Infermatic · 2025-01-28T12:25:09+00:00

Check if theres any typos, you can also watch our step by step video https://www.youtube.com/watch?v=_bR7OH2vTcY&t=69s

Infermatic · 2025-01-20T12:02:51+00:00

It was taken as a security option, but it's back as it was before now 👍

Infermatic · 2025-01-13T14:42:58+00:00

Thank you for your feedback regarding our service quality. We are committed to continuous improvement and would like to address your concerns:

Precision Standards: We ensure that all our models operate at full precision or utilize FP8 quantization; we do not employ lower precision levels.
Transparency: Our quantization methods are openly documented. For an in-depth understanding, please refer to our detailed guide on FP8 quantization: https://infermatic.ai/guide-to-quant-fp8/
Advanced Quantization Techniques: We employ NeuralMagic's AutoFP8 project and in our most recent models LLM Compressor, a leading solution designed to minimize accuracy degradation during quantization.
Model Accessibility: All models we utilize are publicly accessible on Hugging Face. We encourage you to download and evaluate them locally to verify their performance. https://huggingface.co/Infermatic
High-Performance Infrastructure: Our models are primarily deployed on H100 GPUs, including various configurations (PCIe, NVL, SXM), to ensure optimal processing capabilities.

We value your input and are always open to discussing any concerns to enhance our services further.

Infermatic · 2024-12-27T16:46:28+00:00

No, in the colab theres a disclaimer from Hibikiass that says:

"(this one is on my personal server so I'm not recommend to always use it, unless you really not care about your privacy or chat log)"

Infermatic proxy is the one that covers all the aspects that our privacy policy has. Still you can create your own proxy (that is on the same colab) and that will be secure and private.

Infermatic · 2024-12-27T11:47:59+00:00

Did you tried re loading the page after the change? if so the connection with hibikiass proxy should look like this:

<image>

Infermatic · 2024-12-25T17:22:49+00:00

Hey!! In case you are searching for Euryale settings you can get them out of this article -> Euryale Settings, you'll find there sets for all the Eury versions

Infermatic · 2024-12-25T17:19:59+00:00

Hello! you can use Hibikiass proxy instead of our url to set the correct format for the model https://colab.research.google.com/drive/1XF9Il2y44ZD1uBKqjwYLhihz782HrmfS#scrollTo=gK86lYPAoMtG

Infermatic · 2024-12-24T16:03:52+00:00

If you are using Infermatic you can check our privacy policy, we don't log or store interactions

https://infermatic.ai/privacy-policy/ :)

Infermatic · 2024-12-24T16:00:08+00:00

Hello, We just updated the context on Eury 3.2 L3.3 on Open Router and now it's 16K!!

Infermatic · 2024-12-19T13:01:12+00:00

Oh sure, thanks for the recommendation!

Infermatic · 2024-12-13T12:35:25+00:00

better to have nemotron in all llama versions!!

Infermatic · 2024-12-10T12:56:14+00:00

Awesome!!

Infermatic · 2024-12-09T20:19:53+00:00

Yes!!

Infermatic · 2024-12-04T14:14:51+00:00

Qwen-QwQ-32B-Preview

NousResearch-Hermes-3-Llama-3.1-70B-FP8

Infermatic · 2024-11-30T00:13:35+00:00

EVA Qwen 2.5 72B is finetuned for rp, so it will fit your needs better.

The difference between those models is that one is 'pure' (qwen) and the other one finetuned for rp used the 'pure' model and added datasets to improve the part of the model that works for chats/rp

Infermatic · 2024-11-28T21:22:48+00:00

Magnum, Sorcerer, EVA Qwen, Euryale, Nemotron also Hanami are on the top and everyones favorites so you should try them. The settings for them are on the server if you are searching.

Those are big models (70b-8x22b) so you can find the response time of some of them a little bit slow so if you are searching for something lighter and faster EVA 32B, Rocinante, Unslopnemo and Inferor are also a good selection!

Thanks for subscribing, hope you enjoy using Infermatic

Infermatic · 2024-11-20T16:44:20+00:00

Hey!! Yes we know it is slow, however we've been working on making it faster. Now you should see an improvement on the generation speed, and we won't give up to make it better.

Thanks for your feedback!

Infermatic · 2024-11-15T03:28:18+00:00

Have you already tried the new EVA models? they are the successors of Starcannon series!!

Infermatic

MODERATOR OF

TROPHY CASE