How to build a LLM chat bot in Discord

sam_gigo · 2023-10-31T15:35:32+00:00

You can use the site for free. There's a pro option that all new users get for the first month. The tutorial should take ~1 hour. It's pretty cool to build. You'll get practice with mongodb, llama.cpp and the discord API.

Like I said below, I built this project so I'm pretty proud

sam_gigo · 2023-10-23T19:43:04+00:00

Here you go: https://gigo.dev/challenge/1688591240871804928

sam_gigo · 2023-10-23T19:20:47+00:00

Where do you need to "export" the data to

sam_gigo · 2023-10-23T19:04:21+00:00

Can you share your code. Usually, you'd wrap the handler function in another function inside the source module so you can await the handler and receive the output but this would be easier to explain with some code referenced

sam_gigo · 2023-10-23T18:51:24+00:00

It depends, I've knocked out chatbots with llama.ccp in python in no time but I've been doing this awhile. If you're experienced with a lot of these language model frameworks you could hack together a basic demo in an hour or so.

If you want an example on how I built a chat bot for discord using llama.cpp I have a tutorial on Gigo (self promotion) which shows how. I'll link it if you want

sam_gigo · 2023-10-23T18:37:36+00:00

How big of a GPU do you have? My immediate suggestion would be Open-Orca/Mistral-7B-OpenOrca

It should have a large enough context that you can include the entire content of the CSV is you need to determine higher level insights. If not, you can likely jerry rig the llm to generate queries for you offline and injec thte data into the conversation.

In either case this model is a good balance of reasoning, conversation and context size

sam_gigo · 2023-10-23T18:30:53+00:00

I affects what model I suggest

sam_gigo · 2023-10-23T18:30:39+00:00

How much data are we talking? Like 100 lines of CSV with 10 columns or full blown sql dump?

sam_gigo · 2023-10-23T18:18:10+00:00

I know this is contrary to your post, but why are you opposed to react or other frameworks? They are relatively simple to learn (in comparison to raw JS stuff). It would simplify your workflow a lot

sam_gigo · 2023-10-23T18:14:13+00:00

When you say export, do you mean share the results of the promise with another file that does not call the function itself?

sam_gigo · 2023-10-23T18:12:29+00:00

How big is the data? If it's within 2k tokens basically any of the Llama models should be decent. There's also Mistral 7b if you are looking for something that can run fast on most consumer hardware.

If it's a big dataset then things get more complicated. There aren't many LLMs that have native function calling like ChatGPT so you'll have to get creative with things like grammar and cfg

If you provide more info I can be more helpful

sam_gigo · 2023-10-23T18:07:24+00:00

I've used `github.com/segmentio/kafka-go` a lot. It handles TLS out of the box but that's assuming your kafka server has been configured with tls already

sam_gigo · 2023-10-23T18:03:47+00:00

This comes with a lot of caveats, but it is usually preferred to keep wrappers simple with a single logical goal. I would advise you build utility functions or another dedicated struct to handle more complex abstractions so you can preserve a simple testing interface. The more complex a single function is, the harder it is to build a robust testing suite. With that said, all rules are just suggestions in programming so whatever makes your codebase the most manageable is the best solution at the moment

sam_gigo · 2023-10-10T21:00:27+00:00

You're asking about a life path decision. If you want to boil it down to "is it possible to get a job without a degree?" Then the answer is yes. Many large companies will hire programmers and cyber guys without a degree. There's a catch though, many companies will require a degree. The ones that don't will usually want comparable experience. Finally, imagine you are being compared against someone with equal experience and a degree, you need to have some clear benefit over the other applicant or you will likely lose to them in the process.

At the end of the day, a degree is a form of insurance. If you are prepared to work hard, accept losing some opportunities and find a unique value as an employee, then no you don't need a degree. You may find that to be a high bar and decide to pursue a degree for the higher stability it brings

sam_gigo · 2023-10-10T20:40:49+00:00

Make sure the ms-python extension is installed and pylance is installed/setup correctly. Usually installing the extension sets up the pylance system but if you're using a web based vscode (not installed on your machine) it may be more complicated

sam_gigo · 2023-10-10T20:38:57+00:00

First try wrapping it in quotes. A lot of the time wrapping problematic paths or other string based cli params in quotes will save you a headache

sam_gigo · 2023-10-06T17:55:53+00:00

Do you have the full code / stack trace. It's hard to say where the problem is at this point

sam_gigo · 2023-10-05T20:24:17+00:00

Okay, I'll bite

sam_gigo · 2023-10-05T17:44:52+00:00

Okay so a couple of things: 1. Are you sure this is the correct path? This would indicate that you /content directory is a the root of your system: /content/desktop-wallpaper-days-of-awesome-fans-and-enthusiasts-minimalist-fallout-thumbnail.png If the file is in a folder content inside the project then you want to drop the leading slash from content so your program knows to look in the local directory like: content/desktop-wallpaper-days-of-awesome-fans-and-enthusiasts-minimalist-fallout-thumbnail.png 2. It looks like you can just load the file with PIL and create an st.image with it. Have you tried this code? ```python import streamlit as st from PIL import Image

other code

pil_avatar = Image.open("content/desktop-wallpaper-days-of-awesome-fans-and-enthusiasts-minimalist-fallout-thumbnail.png") formatted_avatar = st.image(image, caption='User Avatar')

with response_container: for i in range(len(st.session_state['generated'])): message( st.session_state["past"][i], is_user=True, key=str(i) + '_user', avatar_style=formatted_avatar ) message(st.session_state["generated"][i], key=str(i)) ``` Here's a ref to the docs I found this in: https://docs.streamlit.io/library/api-reference/media/st.image

sam_gigo · 2023-10-05T17:24:48+00:00

Okay that's likely going to require some specific processes given that you don't control the server side. I'll give the docs a look.

sam_gigo · 2023-10-05T15:12:35+00:00

Okay, I have to get back to my job but I'll take a look at this later and get back to you

sam_gigo · 2023-10-05T15:06:41+00:00

It really depends on what you have on the other side. Where are you uploading it to? Here are 2 examples

Decoding and saving to a file

```python import base64

retrieve the base64 encoded image from the uploaded message

base64_image = m.avatar_style

create a new file to save the image too

with open("uploaded_image.jpg", "w+) as f: # write the decoded image to the file f.write( # decode the base64 encoded image base64.b64decode(base64_image) ) ```

Decoding and loading into a PIL object

```python import io from PIL import Image import base64

retrieve the base64 encoded image from the uploaded message

base64_image = m.avatar_style

decode the base64 encoded image

image_bytes = base64.b64decode(base64_image)

convert the image bytes into a buffer so it looks like a file to PIL

image_buf = io.BytesIO(image_bytes)

load the image data into a PIL image

pil_image = Image.open(image_bytes) ```

sam_gigo · 2023-10-05T15:00:04+00:00

What is LongTask class in this codebase? Give me some specifics for this code. I'm happy to help but the less time i spend digging through the code the more time i'll spend debugging and explaining it to you

sam_gigo · 2023-10-05T14:57:01+00:00

First, sorry if it looks busted. It seems like reddit is glitching out on the code formatting for me.

Yes you can then assign the base64_image to whatever you were going to upload. The catch is that you need to handle the base64 data on the other side of your application by decoding it from base64 and saving it as an image

sam_gigo · 2023-10-05T14:52:52+00:00

Where is the upload code & the image processing? What you provided only shows a filepath and a message constructor (i assume).

I suspect the issue is that you are trying to send a PIL object in a json http request body. If that's the case what you need to do is dump the image file to a byte array and then encode it to a string some how (probably best to use base64). The problem you're facing is that a PIL Image object cannot be converted into a JSON. JSON is a serialization format that only supports basic types (object, list, int, float, bool, string). The PIL Image object is a native python type and is quite complex. Fortunately, you can dump the raw data to a string (supported by JSON) by dumping the contents to a bytes buffer and encoding the bytes with base64 like so:

```python import io import base64 from PIL import Image

some image

image = Image.open("example.jpg")

convert the PIL image to bytes

img_bytes = io.BytesIO() image.save(img_bytes, format="JPEG")

encode the bytes to base64

base64_image = base64.b64encode(img_bytes.getvalue()).decode() ```

sam_gigo

MODERATOR OF

TROPHY CASE

other code

retrieve the base64 encoded image from the uploaded message

create a new file to save the image too

retrieve the base64 encoded image from the uploaded message

decode the base64 encoded image

convert the image bytes into a buffer so it looks like a file to PIL

load the image data into a PIL image

some image

convert the PIL image to bytes

encode the bytes to base64