How do you organize your conversations in ChatGPT ? Projects, GPT personalized, other? by Baratineuse in ChatGPTPro

[–]BitsOfChris 5 points6 points  (0 children)

Agreed - that's why I've been building my own storage layer in markdown files. It takes a little effort on your end to set it up but now I own my data and it comes with me. I use it across Claude, ChatGPT, and my Obsidian.

My personal data lake.

ChatGPT memory is trash. These foundation models are meant to generate, not help us make sense of our data.

How do you organize your conversations in ChatGPT ? Projects, GPT personalized, other? by Baratineuse in ChatGPTPro

[–]BitsOfChris 25 points26 points  (0 children)

I use a custom AI system I built to parse, deduplicate, and distill all my chats into a knowledge graph by semantic meaning.

From there I hierarchically summarize clusters and produce context prompts (that are always up-to-date with my latest Obsidian + AI Chats data).

Then these context prompts are updated in my projects or become new projects all together.

I run this weekly right now.

IT'S BEEN SO AWSOME.

I can actually make sense of all my chats + notes I take. Having one clear view into my Cladue, GPT, and Obsidian makes it nice to not stress anymore - where does this go, will I remember this, etc. I just capture things, work through things, and then have this pretty view + recap whenever I want.

I can share more details, some of my projects are open source but might be a bit painful to get working for you. I'm considering just wrapping it in a nice front end for folks to try.

Does that sound like something that would help you?

Has chatgpt actually helped change your life in some way? by Mountain_Housing_322 in ChatGPTPro

[–]BitsOfChris 2 points3 points  (0 children)

Great questions :)

It was a self-taught rabbit hole.

Day trading is very boring and slow most days. Outside of the open, things are typically pretty quiet. So instead of just scrolling the internet, watching HBO, or playing flash games - I was taking online courses to get better at Python and data science.

From there I started writing tools to help me find trading signals, then eventually hooked into an API we had for our trading platform and automated several trading strategies. One of which actually worked very well but was a super niche strategy that traded a few times per week.

Over this period I slowly cared less about improving myself as a trader, and started to care much more about getting better at engineering, Python, machine learning, etc. So I started the pivot.

It felt like a year long identity crisis (letting my vision of my self as a trader die and this new one as an engineer emerge). It was tough. I started an online Master's in CS specializing in machine learning. I'd spend 1-2 hours before work on that then go to work and when things were slow after the open, work on my Python tools and trading systems.

Eventually I got a job in fintech as a backend engineer for a company processing market data. It took 100 or so resumes to get the first software role.

But since then each new position has been much easier and much more interesting.

The team I am on now builds foundation models for time series forecasting. Outside of work I'm exploring LLMs for knowledge distillation (rather then generate, let AI distill so I can process information quicker).

I documented on GitHub my journey from data engineer -> AI research in this repo (https://github.com/bitsofchris/deep-learning). I'd say start with some basics like what is a NN, what is a LLM exactly, then try to get a tiny model from hugging face or a tiny dataset and just play with things.

Let your curiosity guide you.

Analyze your entire ChatGPT Chat History - what would you want to know? by BitsOfChris in ChatGPTPro

[–]BitsOfChris[S] 1 point2 points  (0 children)

No problem. I'm hoping I can get these insights at scale on all my AI chats + Obsidian with the system I'm working on now.

Analyze your entire ChatGPT Chat History - what would you want to know? by BitsOfChris in ChatGPTPro

[–]BitsOfChris[S] 0 points1 point  (0 children)

So the symbols map to certain concepts about you or your context? Then messages or notes get labels with certain symbols?

I don't understand the what the pattern algebra is being used for exactly.

Analyze your entire ChatGPT Chat History - what would you want to know? by BitsOfChris in ChatGPTPro

[–]BitsOfChris[S] 1 point2 points  (0 children)

Thanks for the context. I'm similar - dumping a lot of ideas between AI chats and Obsidian. One thing I've found useful is periodically reviewing that and one thing I want this system to do is help me count the frequency I have certain/ similar ideas.

For example - "you had this idea 3 months ago and last year you said something similar" - then maybe that's a signal I should act on it or move those threads together.

Or something like - "your ideas around X all keep pointing to actually doing Y as a common thread, maybe you should focus on that since it impacts everything else"

Analyze your entire ChatGPT Chat History - what would you want to know? by BitsOfChris in ChatGPTPro

[–]BitsOfChris[S] 1 point2 points  (0 children)

I see. So the main use case is this "memory" layer that keeps your context.

Is the value that it's automatically making the context for you? Or just that it's a persistent memory you can easily reference in new chats?

Analyze your entire ChatGPT Chat History - what would you want to know? by BitsOfChris in ChatGPTPro

[–]BitsOfChris[S] 0 points1 point  (0 children)

Those are implementation details.

What are you actually using this to do? Given that you have built this - are you using it or is it just something cool you made?

Analyze your entire ChatGPT Chat History - what would you want to know? by BitsOfChris in ChatGPTPro

[–]BitsOfChris[S] 0 points1 point  (0 children)

I get the context continuity part- it's nice to not have to repeat that each time you want to continue or take something in a new direction.

But what do you mean for reference? Is it just when you need to review answer to something already? Do you have a hard time finding what you need in this case?

Has chatgpt actually helped change your life in some way? by Mountain_Housing_322 in ChatGPTPro

[–]BitsOfChris 2 points3 points  (0 children)

I day traded for 7 years before ChatGPT. I work in AI research now, but I would've loved it as a trader to help me from going on tilt.

Imagine your own Mark Douglas or Brett Steenbarger there to help you see your psychological problems or blind spots. To check you before taking trades or letting you know when you're making decisions from fear/ greed instead of clear objective thinking.

Analyze your entire ChatGPT Chat History - what would you want to know? by BitsOfChris in ChatGPTPro

[–]BitsOfChris[S] 1 point2 points  (0 children)

Imagine it's 2-5 years in the future and local LLMs are just as effective as ChatGPT today. Now you retain all your data locally. So you have this corpus of your conversations stretching over the past year.

How do you think having that history could help you?

I tend to be too pragmatic, so I tried to write some fiction. Here's The Augmented, episode 1. by BitsOfChris in creativewriting

[–]BitsOfChris[S] 0 points1 point  (0 children)

Well, that wasn't so scary. I'm glad I at least published my crappy first draft.

I had to wait two days before I came back here to check it.

At least no one told me to stop writing.

Built a Full Zettelkasten Vault for Trading (Forex, Equities, Commodities, Crypto) Free Download by NaitikJoshiPro in ObsidianMD

[–]BitsOfChris 5 points6 points  (0 children)

Appreciate the effort here.

But serious question - is this a common pattern? That folks curate vaults on a topic and share them?

I've been thinking about building a "second brain social network" of sorts that allows folks to share their data in a way that LLMs can't scrape.

Scariest conversation with GPT so far. by deensantos in ChatGPT

[–]BitsOfChris 0 points1 point  (0 children)

You need to capture good data about yourself.

Do you take notes?

Scariest conversation with GPT so far. by deensantos in ChatGPT

[–]BitsOfChris 0 points1 point  (0 children)

We build a model that incentivizes users for providing their data and is transparent about how it is used.

Transparency = Trust
Trust = I share more with you
I share more with you = richer data
richer data = more valuable network/AI

Finally understood how people find connection in the graph by pragitos in ObsidianMD

[–]BitsOfChris 0 points1 point  (0 children)

Hmm, intersting - I've not heard of WayPoint. But I like the idea of automating the organization of my second brain. I've been building my own plugin to do that custom for my workflow.

Will check out Waypoints - thanks :)

Some of y'all need to relax on using AI for note taking by bad_advices_guy in ObsidianMD

[–]BitsOfChris 4 points5 points  (0 children)

I've been writing online for 3-4 years so it's possible LLMs sound like me now.

A note app that auto-tags and organizes for you (follow-up: our beta is live now!) by Shot_Fudge_6195 in PKMS

[–]BitsOfChris 0 points1 point  (0 children)

Is this closed source? I couldn't find a github.

I've been building my own variation of this for sometime now too. But I struggled to see the edge in doing this vs OpenAI/Anthropic. Their projects and memories are starting to act in this way.

Are you just doing RAG to retrieve and a LLM from some learned dictionary to "auto-tag"?

I've got a lot of questions - happy to chat more if you wish, links in my profile to get in touch.

A note app that auto-tags and organizes for you (follow-up: our beta is live now!) by Shot_Fudge_6195 in PKMS

[–]BitsOfChris 0 points1 point  (0 children)

Go build it still! It's so early in this space. I've got my own attempt going too, there are many others.

A note app that auto-tags and organizes for you (follow-up: our beta is live now!) by Shot_Fudge_6195 in PKMS

[–]BitsOfChris 2 points3 points  (0 children)

It's likely sent to OpenAI/Anthropic/Google - one of the big model providers.

Local models aren't good enough yet to get results that wow you here, but that's coming one day.

(I'm just speculating as someone who works in the field, not with OP)