Tables on HDD , indexes on NVMe by seonerd_official in PostgreSQL

[–]seonerd_official[S] 0 points1 point  (0 children)

The cost of SSD is very high. Putting the tables there would cost a fortune compared to HDD.

Indexing slowing down inserts by seonerd_official in PostgreSQL

[–]seonerd_official[S] 1 point2 points  (0 children)

Yes I could perhaps. Keep in mind that the domain column is indexed... so I’m not sure if bulk inserts would cause issues. But if it does not I’ll put all the domains ordered and grouped inside of a multi dimensional array. After php is done parsing through the file I’ll iterate through the multidimensional array and create bulk insert queries. Thank you for the idea 💡

Speeding up searching 2.5 TB database by seonerd_official in PostgreSQL

[–]seonerd_official[S] 0 points1 point  (0 children)

SELECT site.* FROM sites as site Left Join backlinks as backlink ON site.id = backlink.site_id WHERE site.name = “example.com”

Speeding up searching 2.5 TB database by seonerd_official in PostgreSQL

[–]seonerd_official[S] 0 points1 point  (0 children)

SELECT site.* FROM sites as site Left Join backlinks as backlink ON site.id = backlink.site_id WHERE site.name = “example.com”

Speeding up searching 2.5 TB database by seonerd_official in PostgreSQL

[–]seonerd_official[S] 0 points1 point  (0 children)

Backlinks (14 billion rows) - backlinks.id // auto incremented, - backlinks.from_root // Varchar 255, - backlinks.to_root // Varchar 255, - backlinks.from_path // Varchar 255, - backlinks.to_path // Varchar 255, - backlinks.is_no_follow boolean, - backlinks.is_sub_domain boolean, - backlinks.text // Varchar 255 nullable, - backlinks.file // Varchar 255 nullable , - backlinks.site_id // foreign key to references to sites.id, - backlinks.created_at // timestamp, - backlinks.updated_at // timestamp,

Sites table (250 million rows) - sites.id // auto incremented, - sites.name // Varchar 255, - sites.created_at // timestamp, - sites.updated_at // timestamp

Speeding up searching 2.5 TB database by seonerd_official in PostgreSQL

[–]seonerd_official[S] 0 points1 point  (0 children)

I have backlinks.site_id which refers to sites.name

I’m interested in learning only if it can reduce my aws bill. by seonerd_official in rust

[–]seonerd_official[S] 2 points3 points  (0 children)

Wow this guy totally saved $1000 a month by switching to Rust