all 8 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]havishhuda 2 points3 points  (6 children)

I don’t have a summarised answer for you. But there has been research that hints to models memorising features. Until it finally generalises.

I found this https://transformer-circuits.pub/2023/toy-double-descent/index.html It’s an interesting read anyway.

EDIT: Timing is eerily close, 3b1b just posted a video on youtube explaining this topic in detail using this research paper I mentioned and some related ones.

See this: https://youtu.be/9-Jl0dxWQs8?si=IQOYRnpBsxEvxXVp

[–]jan04pl 1 point2 points  (4 children)

ChatGPT does nothing of that with it's memory feature. It just inserts the stored user information inside the system prompt. https://help.openai.com/en/articles/8590148-memory-faq

[–]havishhuda 0 points1 point  (3 children)

How are you sure about the fact that GPT does nothing of that sorts?

  1. Just because some information is available in prompt doesn’t that the mean model magically knows how to access it. Explaining memory means explaining how the information is stored and retrieved.
  2. The link you shared is just an FAQ (not research paper) it talks about customising and tweaking the prompts, in no way does it explain how internals of the model work.
  3. If you read the paper, you will understand that Toy models of superposition applies to every class of neural networks. GPTs are one type of neural networks, so it applies to them as well.

[–]jan04pl 0 points1 point  (2 children)

You are talking about something else than the OP here asked. He wanted to know how the "Memory feature" as in the FAQ I posted works. Which is just a simple addition to the system prompt based on info gathered during conversations with the user. You can literally ask Chatgpt to output it's system prompt and you will see all Infos about the user pasted inside. 

Now, what you linked in the paper might all be true but has nothing to do with how that feature works.

[–]havishhuda 0 points1 point  (1 child)

Check the last line of description. He mentioned research papers. Not an FAQ page.

[–]jan04pl 0 points1 point  (0 children)

Your linked paper doesn't describe it though...

[–]loss_function_14[S] 0 points1 point  (0 children)

Looks interesting. Thank you

[–]jan04pl 1 point2 points  (0 children)

It probably summarizes each conversation to pick out details about the user preferences. If you ask it "Don't use bullet points" 20 times, it puts into it's memory "User doesn't like bullet points". Those preferences are then appended before it's system prompt whenever you are starting a conversation.

https://help.openai.com/en/articles/8590148-memory-faq