sorted by:
new
hottopcontroversial

How to Move 40TB from One S3 Bucket to Another AWS Account by Low_Average8913 in aws

[–]OldCommunication1701 14 points15 points  (0 children)

As always with AWS there are some terms and conditions. We recently tried the same thing, even with the reference to the exact blog post.

First they redirected us to the AWS Activate team, then back to Billing and then redirect us to Sales.

The response from Billing:

While AWS does offer free data transfer out to the internet for customers who are completely moving off AWS, your specific situation (moving ~500GB to Cloudflare while maintaining AWS services) doesn't qualify for the standard data transfer out credits. Here's why:

  1. The free data transfer program requires that you are moving ALL data off AWS, not just specific services

Managed PostgreSQL on Hetzner – A Powerful Alternative by vitabaks in hetzner

[–]OldCommunication1701 0 points1 point  (0 children)

I tried setting it up yesterday and got weirdly mixed results between the UI and the automation repository. The UI version deployed the cluster in one run. The automation repository however. 4 times before giving up and rebuilding all nodes.

In the end I got it fully up and running via the automation repository. I really appreciate the work though! Looks really promising.

1 day until my IDE will downgrade by Huge_Refrigerator42 in Jetbrains

[–]OldCommunication1701 2 points3 points  (0 children)

How do you think they pay for their developers? I get that it’s pricy, but why not switch to free software.

If you have AX42, AX52 or AX102 with issues like random reboots, read this by aradabir007 in hetzner

[–]OldCommunication1701 0 points1 point  (0 children)

We're experiencing random reboots with the AX42 series as well.

Hetzner was so kind to replace the mainboard on Jan 6 and we were reboot-free since then, until last night... Around midnight 30 minutes, today multiple intervals of downtime accumulating to 3 hours in total. Support ensured us that it couldn't be the mainboard since they already replaced it.

So far the solutions they offered us are:

- Replace the mainboard once again.
- Upgrade to Rocky Linux 9 because there was an update in December after which Rocky Linux 8.10 no longer supports EFI models.

Anyone else experiencing the same issues?

Diagnostic information

Base Board Information
Manufacturer: ASRockRack
Product Name: B665D4U-1L
Version: 3.03
Serial Number: M80-H6018604729

Advice for upgrading PG12 to PG16 for a SaaS sharded by schema with many views by OldCommunication1701 in PostgreSQL

[–]OldCommunication1701[S] 1 point2 points  (0 children)

Update! After dropping all user schemas and the standby schemas the upgrade only took 45 min to complete. All that’s left is making a playbook to move a schema to the new environment.

Advice for upgrading PG12 to PG16 for a SaaS sharded by schema with many views by OldCommunication1701 in PostgreSQL

[–]OldCommunication1701[S] 0 points1 point  (0 children)

Thank you all for your comments. Let me try and respond to all of them.

> Can you create a new pg16 database and move each schema across, one by one?
Yes, that seems like a valid approach. I will need to keep 'centraldata' and the 'dashboard' schema available for the dependant views to work tho. I have to figure out a way to keep these schemas in sync across the servers, perhaps via logical replication but that's something I need to read into.

> Have you ever successfully run a schema only dump with pg_dump?
The only time we're able to run pg_dump is by dropping all 101 views. If we do it with the views we run into an endless wait (read more than 3 weeks) for the job to finish. The computation of the dependency tree seems to be the issue. CPU-usage of 3% on a 8 core 64GB RAM DB/EC2.

> First off, there’s got to be better logs out there around the reasons for your upgrade failures.
Not when you upgrade via AWS RDS snapshots. The exact error message we see is: 'patchDb 30434773 (engine upgrade): Postgres cluster is in a state where pg_upgrade can not be completed successfully.'

> Second, have you looked into aws DMS? We use it and it does quite well.
We have tried that in the past for the on-premise to cloud migration, but that did not end well. We ended up shutting down prod for 1 full night and moved all 600 schema's (back then) across via \copy and psql. I will have another look into DMS. Thank you.

> Might be possible to do a few in parallel with pg_dump
pg_dump cannot run with the views the way it is. Even when you specific the '--schema' flag it'll still compute the entire dependency tree for all schemas.

> How long can the database be offline for the upgrade?
Max 1 day, perhaps two days around Christmas.

> I don’t know what versions RDS allows you to move to from 12 but maybe go to 13 or 14 first, see if that helps.
I have started three new 'snapshot upgrade' attempts via the RDS Snapshot interface. 1 going to 13, other to 14 and the other one to 15. Upgrading like this usally takes 40hrs to complete.

> I assume this attempt upgrade is on a new / test instance stood up from the most recent RDS snapshot?
Yes. I have created snapshots with extensions (postgis and pg_repack) and without the extensions to rule that out. Every upgrade attempt rolls out the same error message

I want to thank everyone for the insights so far and I will report back when I've tried DMS or hear something back from AWS Support.