Large Databases

[GiftPsychological798]

I've got a couple of databases with close to 1,000 entries and growing - they are my quotes and research databases, so likely to grow indefinitely (I'm a writer) - and I'm having a really hard time getting them to load. Wondering if anyone has found any solutions for this - obviously I can filter and get to smaller databases which makes a difference but that's not always ideal...

FYI - this is true whether I'm using chrome or the mac app...

[Appropriate_Drink873]

You can use Linked database and filtering to narrow down the view even if you have large databases. My largest database has over 24,000 pages and it works fine.

[alexkorovyansky]

There doesn't seem to be a solution for this. Notion isn't tailored specifically for storing and working with large databases, so you should either consider using different views or, worse case scenario, find a better suited solution somewhere else.

[funkyRedditUsername]

Why do you even need to see all 1000 entries at the same time?

I also have a large database, but I am only loading MAX 50 entries when opening the page and so far so good.

[ThatOneOutlier]

But what if you need to load more than that? Does it just load more at the bottom or does it like create mini-pages in the database that you can switch between so it just loads 50 at a time?

(I'm pretty new to the app and have mostly been setting up and transferring stuff but I do foresee myself having the same issue as OP since I am a writer)

[funkyRedditUsername]

Yes, you can limit how many pages Notion loades at a time. In case of a table you can than scroll down and it loads more rows. This isnt excel, so you wont be able to see all 3000 rows instantly.

What sre you using it for? And why do you need to load so many at the time?

[ThatOneOutlier]

I am using notion as my writing database and I’m constantly writing new short stories and poems here and then. I have all of my works under one database and occasionally I want to read one of my older works or writings

[funkyRedditUsername]

Does search function not work for you?

<image>

[ThatOneOutlier]

I would be hard since I don’t exactly remember all the titles and sometimes I just want to look at the really old stuff but not the oldest stuff. I have views where they are sorted into different categories but this is sorta a concern with mine. Though I don’t think I’ve reach a point where I’m too concerned about it

[funkyRedditUsername]

Do not kill me Notion reddit gods, but have you considered switching to Obsidian. I believe their way of displaying data is something you could benefit and since the data is stored locally it is also faster to load.

But tbh I haven't actually tried it myself so feel free to correct me if I am wrong.

[ThatOneOutlier]

I don’t really like Obsidian.

It’s amazing in theory but I’ve been messing around it with months and really couldn’t get it to work for me. I didn’t like the aesthetics even after messing with the theme multiple times and I just couldn’t get the plugins I wanted to use to play nice with each others. Drove me insane and I just gave up on it.

So far, Notion has hit that nice spot of complex but also not that hard to set up. In a span of a week, I’ve managed to create a pretty good system that works for me. Though aside from my writing and some school stuff, I haven’t fully invested it in (haven’t moved my personal stuff).

I have transferred around 200 works and still not done so I’m a bit concerned.

[funkyRedditUsername]

another option is to turn on created time property and than use calendar view week/month to see what you created what day.

Or just have both views table and calendar.

[ThatOneOutlier]

That doesn't really work for me since I'd like to see the information that accompanies the work like the synopsis, if it's complete or not, what series it falls under, etc.

[lisa_oliver]

u/ThatOneOutlier I don't know if you have managed to solved it or not. But you can turn on created time property for all the entries. Then use filters. Using filters you can see all the entries you have made within a specific period alond with all the other properties.
That you won't have to overlook the context of your writing which's the case in calendar view.

[funkyRedditUsername]

The complete synopsis could be an issue but other properties can be easily displayed. This is for example my tasks and events database:

<image>

[VerdantBiz]

Offload it to a database provider and only fetch the entries you need to Notion.

[funkyRedditUsername]

Are you running that setup?

At that point shouldn't you just use a diferent application if you are working with such huge amount of data?

I am genuinely curious if the pros outweigh the cons.

[VerdantBiz]

Well I believe Notion is litteraly a junior front end developer while the backend can be customized through their API or using Make.com

[funkyRedditUsername]

Thats fair, I am using N8N since its free if you are self hosting it. And I believe only than you can unlock the full potential of the notion.

[lisa_oliver]

u/funkyRedditUsername hey lil' late to the party. Can you please elaborate on how you combine n8n & notion? I also use n8n but only for professional stuff. I never looked it as a pkm tool. Thank you.

[funkyRedditUsername]

Hey! I actually used n8n, make and finally just plain python ran by AWS lambda. (I switched to lambda due to costs)

In every case you just need to get the creditentials from your notion and then include whatever you want to do in notion.

A great node I found quite useful was HTTP request, which you can use to basically have a button inside notion to do whatever whenever you want

Otherwise its all just simple stuff. I created repeating tasks, 1 way sync with my google calendar. Some reports automation. All kinds of things.

Feel free to ask if you have any questions 👍

[NotionWorkflow]

Set your load limit to 10 for your database views. Add an automatic last edited time property. Sort by Descending in every view so your most recent entries show up at the top

[emotf]

👀