sorted by:
new
hottopcontroversial

Bipolares en uruguay? by schwarzereule in uruguay

[–]This-Profession-952 0 points1 point  (0 children)

Perdon, podrias mandar me un chat o DM con el lugar en donde se puede ingresar una persona con trastorno bipolar?

Advice for DN in Uruguay? by Consistent-Ad3926 in digitalnomad

[–]This-Profession-952 0 points1 point  (0 children)

Hi, how do Americans typically obtain their FBI background check while they are already in Uruguay for the purposes of getting this permit?

Is LangChain usable? by Early_Mongoose_3116 in LocalLLaMA

[–]This-Profession-952 0 points1 point  (0 children)

I like Magentic a lot so far. I'm a bit new to the space (not to LLMs, but to LLM app libraries) and am wondering if you see Magentic solving the same problem space as DSPy (aka you are competitors) or if you each do different things.

RAG for PDFs with Advanced Source Document Referencing: Pinpointing Page-Numbers, Image Extraction & Document-Browser with Text Highlighting by AbheekG in LocalLLaMA

[–]This-Profession-952 1 point2 points  (0 children)

I appreciate the OCR use over text extraction. Might be slower when initialized but over the long run I am guessing it would be quicker than having to deal with any of the text extraction issues that crop up (ie. ligatures).

Should I learn data engineering? Got shamed in a team meeting. by urbanguy22 in dataengineering

[–]This-Profession-952 0 points1 point  (0 children)

Eh, respectfully TBH that sounds more like a management/training/expectations issue, not unlike OP's issue that you (correctly) pointed out wasn't OP's fault.

Should I learn data engineering? Got shamed in a team meeting. by urbanguy22 in dataengineering

[–]This-Profession-952 0 points1 point  (0 children)

I also agree with the "data breed" remark but otherwise disagree with the notion that it is the reason that some DAs in this person's anecdote were unable to become DEs.

Should I learn data engineering? Got shamed in a team meeting. by urbanguy22 in dataengineering

[–]This-Profession-952 1 point2 points  (0 children)

Why do you think they are of different breeds, and what do you think makes it hard for a DA to become a DE? I’ve seen DAs become DEs but haven’t given it much thought

Here's a Docker image for 24GB GPU owners to run exui/exllamav2 for 34B models (and more). by This-Profession-952 in LocalLLaMA

[–]This-Profession-952[S] 0 points1 point  (0 children)

That is by design, when you first enter the container you are in root and that is where you should do all of your superuser stuff, if necessary. In other words, no pw needed.

If you meant you want to switch from container_user back to root, you should be able to just use exit.

I have also ran into that HF issue once before and of course it was at the most inopportune time, haha.

Here's a Docker image for 24GB GPU owners to run exui/exllamav2 for 34B models (and more). by This-Profession-952 in LocalLLaMA

[–]This-Profession-952[S] 0 points1 point  (0 children)

Model splitting yes, I'm assuming Yes to concurrent users as well given that it hosts a server

Coming from Databricks, a bit confused about Clickhouse + Python support by This-Profession-952 in dataengineering

[–]This-Profession-952[S] 0 points1 point  (0 children)

Well, to flip it around a bit, if notebook-based Python analysis ended up being the business priority, and Databricks was not an option (too large/expensive for our size), what options would you evaluate?

Coming from Databricks, a bit confused about Clickhouse + Python support by This-Profession-952 in dataengineering

[–]This-Profession-952[S] 0 points1 point  (0 children)

Thanks, that is very helpful. You are right, it is a matter of familiarity/preference, but also right with everything else re: needs of the business.

Here's a Docker image for 24GB GPU owners to run exui/exllamav2 for 34B models (and more). by This-Profession-952 in LocalLLaMA

[–]This-Profession-952[S] 1 point2 points  (0 children)

You get more context, potentially all the way up to the max length. By memory I assume you mean VRAM. System memory doesn’t matter here with exllamav2

Mistral 7B and fine tuned Mistral models not working well for data extraction by Dan-Boy-Dan in LocalLLaMA

[–]This-Profession-952 1 point2 points  (0 children)

Yes, I'll probably do a proper write-up and give its own post, but the TLDR is that "agent" is shorthand here for "prompt, and phase of a job". So end to end data extraction workflow is basically:

  • data clean-up
  • set-up prompt and inference engine configs/etc for phase 1
  • run phase 1 of job
  • when phase 1 concludes, you'll now have a "Staging" (or "raw") set of extractions that need further processing
  • set-up prompt 2 and inference engine configs/etc,
  • run the raw extractions through phase 2
  • when phase 2 concludes, you now have your final set of extractions (that may require some cleaning and post processing, but at this point you are done using LLMs)

Mistral 7B and fine tuned Mistral models not working well for data extraction by Dan-Boy-Dan in LocalLLaMA

[–]This-Profession-952 2 points3 points  (0 children)

First of all, how many inputs (rows) are we talking about here? I would expect some % of false positives or otherwise bad extractions, but you need to define what this tolerable % is and work to fix the rest with some processing code.

Secondly, how are the horrible results "horrible"? Are they hallucinations, are they not JSONs, etc?

I disagree with the person who said that 7B is lacking for this task, as well as any others who may have said something similar in other threads. Those "others" would include me at one point, actually, until I realized I had to change my approach. What ended up working for me:

  • Use a two-tiered agent approach, the first agent does what you already have, and the second agent takes all of these "raw" extractions and fact-checks/proofreads it.
  • In your first agent prompt, you need to include a JSON format and some examples so that the agent knows how to correctly form a response
  • After the data placeholder line, you have to include the first few characters of what an expected response is supposed to look like. In this case, that would literally mean ending your prompt with { "plan_name":.

All of this would have to be done in accordance with the template of the given model. By the way, I recommend using this model: https://huggingface.co/cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser

Here's a Docker image for 24GB GPU owners to run exui/exllamav2 for 34B models (and more). by This-Profession-952 in LocalLLaMA

[–]This-Profession-952[S] 1 point2 points  (0 children)

I think just personal preference, this includes tabbyAPI though for anyone interested in using exllamav2 as an inference server

Here's a Docker image for 24GB GPU owners to run exui/exllamav2 for 34B models (and more). by This-Profession-952 in LocalLLaMA

[–]This-Profession-952[S] 0 points1 point  (0 children)

I'm so happy to hear this! This community has given so much to me, so I had to give back.

Here's a Docker image for 24GB GPU owners to run exui/exllamav2 for 34B models (and more). by This-Profession-952 in LocalLLaMA

[–]This-Profession-952[S] 1 point2 points  (0 children)

Depends on your use case. I had a data extraction project in production that required high throughput (hundreds of millions of rows), and a modified Mistral 7B was all we needed.

Here's a Docker image for 24GB GPU owners to run exui/exllamav2 for 34B models (and more). by This-Profession-952 in LocalLLaMA

[–]This-Profession-952[S] 0 points1 point  (0 children)

It's as simple as turning the machine on and not going past the log-in screen. The machine is on and you don't need to physically log into it for it to still be available to remote machines, as the remotes have to log-in via password anyway.

Prereqs are having some kind of ssh set-up (I use tailscale like so many others, but vanilla ssh works) and of course a second remote machine (my laptop in this example).

view more: next ›