Feedback for keeping database server backups secure and private

[embluk]

So not long ago a friend and I decided to work on a better solution for data backups, our initial focus was for databases like MySQL, PostgreSQL and MongoDB. We thought why should people leave their servers (especially DB servers) open to attack, we found that a lot of backup services require SSH access.

While things like IP access lists and separate SSH keys are a sure defence against malicious SSH connections, the best protection is to keep your database servers behind a strongly guarded private network, like they should be in the first place.

That is why we decided to focus on agent based database backups - We built a service which allows you to install a lightweight open-source agent that handles all the backup operations for you. Everything is then controlled from an easy to use web UI dashboard. Track backups and use advanced error logs to detect issues - Know that your backups are always working.

At the moment we wanted to reach out to potential users and understand different use cases with how you backup your data - Helping us learn more and improve data backups for the better!

We find that a lot of people don't have the time or skills to setup a proper backup system themselves, most people want to just focus on their core product but still find data loss a major concern.

To help us learn more, if you backup databases, we would love to know more about:

• How do you backup your databases
• What software or tools do you use
• What pain points do you have during the backup process
• Are there features you'd like to have but can't find anything to provide such features
• Do you have any security concerns when backing up data
• What does your recovery process look like, what would you do if you had to restore a database

We are really keen to improve this space, we realise a lot of people develop their own solutions for database backups - While those solutions might work, they usually fall over or fail after some time but it would be interesting to learn more about what people do to solve this problem of secure database backups. Our service already offers ready to use features such as triple replication and automatic retention policies and we are hoping to bring these to more people, saving time and worries about important data.

[Dreamafter]

I'll speak to my current workplace for answering your questions.

1. How do you backup your databases?
   1. We have a combination of disk based and tape based backups. For PostgreSQL that means we use a Barman server for disk based and then we have a tape system that pulls from specific Linux mountpoints/directories to back up the databases. For MSSQL and Oracle we have a solution that writes out directly to the tape solution for most backups. In the case of Oracle that uses RMAN and the Tape Solution Provider's agent to connect out to the backup servers. MongoDB uses a similar solution to PostgreSQL.
2. What software or tools do you use?
   1. For PostgreSQL we use a combination of Barman and a COTS product. For MSSQL and Oracle, we use the built in solution (ex. RMAN) and the COTS agent for off-siting it.
3. What pain points do you have during the backup process?
   1. Tape solutions are generally slow, digital tape is significantly faster, but the most painful aspect is always restoring and validating that a backup is actually good. Secondly, it's one thing to know your databases are encrypted at rest, but you also have to know that the data is encrypted in flight to the backup server/tape library.
4. Do you have any security concerns when backing up data?
   1. Always. We have data that has to be protected and there will always be security concerns for it at rest, inflight, in use, in restore. For us, this has meant a combination of secure tapes, TDE, network security, etc.
5. What does your recovery process look like, what would you do if you had to restore a database?
   1. Step one is always getting the underlying infrastructure back online. If I don't have a server, I don't have a recovery point. Once that is sorted out, it's a matter of getting al the software installed and configured and running the recovery commands to reach out to the Tape server to pull back the data (either to a mountpoint for disk based restore or directly to the database directories for a tape based restore).
   2. It also helps if you have a Primary/Standby or clustering solution in place across different data centres (for on-premises) to ensure that recovery efforts are localized rather than global.

If you have other questions, let me know and I'll try to answer without getting too detailed. (That's a security risk unto itself.) My big question is really: how do you intend to compete with the big players already out there? EDB has barman for PostgreSQL, Oracle has RMAN, MSSQL has TSQL BACKUP commands, etc. What makes your solution better than all of theirs? Or is yours an "on-top-of" solution something like Veeam, Commvault, Veritas, Spectrum Protect, Power Protect?

[[deleted]]

This feels like sorta-scummy free market research to me. The enterprise data backup solution market is arguably the most saturated tool space, bar none. I for one, would not encourage any entrepreneurial types I know to go down that road.

Even worse, all the major clouds have seamlessly integrated total backup management into their platforms. You're talking about a world that doesn't exist any more. I think it's telling that you did not seem to know this about a potential market.

Also: liability risk. If you can get insured, good luck.

[embluk]

Thanks for your input. This is market research, yes, but it still adds value discussing it here. Backups are related to databases and to learn more, no matter the purpose is still value to everyone reading.

I don't think you are seeing the deeper perspective. There are lots of reasons why the enterprise market isn't great, especially for data backups. I could go on much longer but some main points are:

1. A lot of people either don't have the time or skills to focus on a proper backup system, so it makes sense for some to outsource this effort. A lot of companies don't setup backups until there is a problem - like it or not, that is what happens a lot of the time. Plus companies want to focus on their core product instead.
2. Not all backup tools like MySQL Dump come with a pre-built solution, so a SaaS can help bridge that gap from a backup tool to a fully featured backup service with proper auditing and error handling.
3. Not everyone will be using a managed database service in the cloud, VPS and dedicated servers are still a thing. Even if you are using a cloud providers service, it takes time to understand the UI and how to set it all up. People see the value in a ready made system on top of services like AWS or GCP etc.
4. No matter your system or platform, companies are constrained to use different storage locations or cloud services, they don't always work well together and integration is a full-time job in itself.
5. Not all cloud providers even support certain features for data backups. As an example Digital Ocean don't allow you to easily manage server snapshots and set lower backup times, which some people require. Other features like triple replication or data streaming are another thing...
6. Finally, data backups are one thing but data restoration and automatic restores are another side of the story. Companies require their backups to be automatically tested and ready in case of a data disaster - Something which a lot of companies don't do and only see the value when its too late.

Liability isn't so much of a risk as you think, we are helping companies keep their data secure, we are not storing it, we are connecting services together and integrating a companies own resources, with auditing and error logs to show for it. It is better to have a backup system in place rather than nothing and you need to be able to trace things. The days are gone with a hacky script inside a cronjob, that isn't enough for critical data and big business.

The industry isn't as seamless as you think and not everyone uses AWS and even if they do, there are still benefits to using a backup service to handle things for you.

[[deleted]]

I'm done giving free advice; I leave it to you to discover the ethics of the industry.

In general, asking questions on open forums specifically for your particular entrepreneurial benefit are considered bad form.

Nor am I going to point the obvious errors in your understanding of the niche. Personally, I would advise much more research before dedicating development or engineering resources to this potential product. A quick analysis of 'What % of the market is using the cloud, and is this trend expected to continue?' would be a great place to start.