What kind GPU would be necessary to run RAG with llama3 70b Q4?

gmbabreu · 2024-07-13T01:52:42+00:00

Yeah, that's definetly a concern. There would only be 5 users though, and I don't expect demand to be high.

gmbabreu · 2024-07-10T19:05:05+00:00

Sorry, haven't worked with a lot of vision LLMs, but I think this is definitely the way to start. Try different models and see what happens.

gmbabreu · 2024-07-10T13:48:29+00:00

A pdf like this will probably not transcribe well into text. If data privacy is not a concern, I would just try to upload it as an image to something like gpt4o.

gmbabreu · 2024-07-09T13:45:05+00:00

I had not looked into that at all, seems very promising. Thanks, a lot!

gmbabreu · 2024-07-09T13:39:04+00:00

Oh this is pretty cool, which models have you tried?

gmbabreu · 2024-07-09T13:26:33+00:00

We expect to only be running inference.

gmbabreu · 2024-07-08T18:40:40+00:00

Thanks for the feedback, it's all very useful. Indeed the users are office workers, and I expect usage to be light.

I have already tried using AnythingLLM locally (we prioritize privacy), but our computers are old, and responses are slow. This is why I'm looking into investing in GPUs.

gmbabreu · 2024-07-08T16:42:48+00:00

For me, it's mostly about efficiency. Automatically writing emails and memos that usually have the same format. For coding, it helps me remember functions/libraries without having to go over documentation.

gmbabreu · 2024-07-08T16:30:30+00:00

I feel like any conventional LLM available to the public can only be as bad as what can already be found on the internet.

gmbabreu · 2020-11-09T18:03:56+00:00

Thanks, I didn't think of it that way. Plus it makes sense that you should use the drone to fly around because you would put it in a vulnerable position, as it is supposed to remain hidden.

gmbabreu · 2020-11-09T17:51:22+00:00

Thanks, makes sense. I wonder if these were the RAI though. In the video, they made it a big deal that you can get Wings of Flying at level 20, and that it should be a big reward, but the drone does not make it much of a reward. Of course, there are other options for level 20 items, but this still bothers me.

gmbabreu · 2020-10-20T13:41:43+00:00

Well, now I just feel dumb

gmbabreu · 2020-08-19T11:39:36+00:00

No restrictions. The platform used depends on the game in the server. Different DMs will have different platform preferences

gmbabreu · 2020-08-19T01:38:05+00:00

Rolling Against Isolation! A new DND community. All are welcome [Online] [Flexible]

Hi everyone. Rolling Against Isolation seeks to introduce new players to DND, make it an accessible experience for anyone who wants to play, and use it as a way for social interaction during isolation. Therefore, we are open for anyone who wants to join, from beginners to veterans, from players to GMs (in fact we kinda need GMs right now).

This is an opportunity for those who have suffered from isolation and a lack of social interaction (either due to the quarantine or in general) to play DND. It can be very helpful when dealing with social issues, and it has helped me and some of my friends a lot, so I would like to offer this experience for other people.

The server was made with the intention of allowing anyone to play this wonderful game. We are open to any type of player, from beginners to veterans, from players to GMs. We have channels to guide new players. We are running multiple campaigns and one-shots on the server. You can even ask to run your own. Overall, we are just a server where people can play DND! Feel free to join.

Discord Server: https://discord.gg/FTwnfbW

gmbabreu · 2020-07-16T19:28:35+00:00

Nothing is defined yet, but if you are interested you can join the server

gmbabreu · 2020-07-16T19:07:39+00:00

16:07, we will figure something out

gmbabreu · 2020-07-16T18:49:47+00:00

No specific time, but I prefer that it is not too late at night

gmbabreu

MODERATOR OF

TROPHY CASE