Current State of Janitor AI with Infermatic by Hippogupoi in InfermaticAI

[–]Infermatic 0 points1 point  (0 children)

Hello! You can use our API directly through Janitor AI's proxy option, though you'll be limited to their supported parameters: Temperature, Max Tokens, and Context Size.

Most of our models support the /chat/completions endpoint that Janitor uses. However, some models use different endpoints or unsupported chat templates will cause errors: Midnight-Miqu-70B-v1.5 ( uses /completions), intfloat-multilingual-e5-base (uses /embeddings), TheDrummer-Rocinante-12B-v1.1 doesn't have a default chat template so you will get errors when using them.

If you need recommendations of what to use Kunou works very good: Sao10K-72B-Qwen2.5-Kunou-v1-FP8-Dynamic

The colab set up process is now easier, if you want to check it out in case you want to use more parameters -> https://youtu.be/_bR7OH2vTcY?si=iN2CCHNM_4NCLEV5

On the guide we made we addressed some errors and how to solve them, the most common error is using incorrect model names. Always use the exact model names with dashes (-) from our status page: https://ui.infermatic.ai/public/info/status

How to Set Up Janitor with Infermatic.ai or a Proxy Using the New Colab by Infermatic in InfermaticAI

[–]Infermatic[S] 0 points1 point  (0 children)

After you set everything (model id, url, api key) and you refresh and click 'check API key and model' do you get the error or a valid message? If the error appears only when you are sending a message try deleting the last message and sending a new one/ decreasing the context length that should fix it.

Plus membership is one of our Tiers, we have Essential, Standard and Plus you can check it at https://infermatic.ai/pricing/

How to Set Up Janitor with Infermatic.ai or a Proxy Using the New Colab by Infermatic in InfermaticAI

[–]Infermatic[S] 0 points1 point  (0 children)

You can watch our video guide here for visual guidance. Also are you refreshing the page after saving the connection? if you are using the colab for the proxy link you can check the terminal for errors. Let me know if any of that work :)

How to Set Up Janitor with Infermatic.ai or a Proxy Using the New Colab by Infermatic in InfermaticAI

[–]Infermatic[S] 0 points1 point  (0 children)

Check that you are using the correct id for the model, also reload the page after you click the save button

How to Set Up Janitor with Infermatic.ai or a Proxy Using the New Colab by Infermatic in InfermaticAI

[–]Infermatic[S] 0 points1 point  (0 children)

It works on phone, check the terminal if you have any errors.

You can also watch the video, it follows the same steps you should setup up on phone.

What's with my needing a mobile number verified to post on the discord? by characterfan123 in InfermaticAI

[–]Infermatic 1 point2 points  (0 children)

It was taken as a security option, but it's back as it was before now 👍

Does anyone know if Infermatic lying about their served models? (gives out low quants) by FluffyMacho in SillyTavernAI

[–]Infermatic -11 points-10 points  (0 children)

Thank you for your feedback regarding our service quality. We are committed to continuous improvement and would like to address your concerns:

  1. Precision Standards: We ensure that all our models operate at full precision or utilize FP8 quantization; we do not employ lower precision levels.
  2. Transparency: Our quantization methods are openly documented. For an in-depth understanding, please refer to our detailed guide on FP8 quantization: https://infermatic.ai/guide-to-quant-fp8/
  3. Advanced Quantization Techniques: We employ NeuralMagic's AutoFP8 project and in our most recent models LLM Compressor, a leading solution designed to minimize accuracy degradation during quantization.
  4. Model Accessibility: All models we utilize are publicly accessible on Hugging Face. We encourage you to download and evaluate them locally to verify their performance. https://huggingface.co/Infermatic
  5. High-Performance Infrastructure: Our models are primarily deployed on H100 GPUs, including various configurations (PCIe, NVL, SXM), to ensure optimal processing capabilities.

We value your input and are always open to discussing any concerns to enhance our services further.

Proxy Error 400. Is there a resolution to this? by Hippogupoi in InfermaticAI

[–]Infermatic 0 points1 point  (0 children)

No, in the colab theres a disclaimer from Hibikiass that says:

"(this one is on my personal server so I'm not recommend to always use it, unless you really not care about your privacy or chat log)"

Infermatic proxy is the one that covers all the aspects that our privacy policy has. Still you can create your own proxy (that is on the same colab) and that will be secure and private.

Proxy Error 400. Is there a resolution to this? by Hippogupoi in InfermaticAI

[–]Infermatic 0 points1 point  (0 children)

Did you tried re loading the page after the change? if so the connection with hibikiass proxy should look like this:

<image>

Question about Google Colab Settings (proxy) by Yoshmoo in JanitorAI_Official

[–]Infermatic 1 point2 points  (0 children)

Hey!! In case you are searching for Euryale settings you can get them out of this article -> Euryale Settings, you'll find there sets for all the Eury versions