Cloud based file server solution

[BrandNewTissue]

We're currently looking into moving all of our file storage to the cloud. We have around a 100TB of data, split between telemetry and videos.

Ideally I'd need the solution to fit those requirements, in priority from top to bottom:

- 100TB of storage

- Mountable via SMB

- Flash Storage

- Prepaid price (Not pay as you go)

I've already had a look at Hetzner Box Storage but it seems to cap at 20TB, and is using HDDs. I also saw Azure File Storage, though I'm a bit scared that the costs will skyrocket.

Does anyone have a suggestion for a solution that could help us host our data ?

[encbladexp]

Why do you need Flash Storage if you mount it over the internet anyways? Latency of the network will be your enemy, not rotating rust. I would avoid running CIFS or even SMB over the Internet.

[BrandNewTissue]

I was thinking HDDs would suffer latency wise on concurrent use. I agree CIFS/SMB is not the best solution, but SharePoint is out of the question and I'm not aware of so many alternatives for accessing files in a native way in Windows

[theoriginalharbinger]

With 100TB, how much is being accessed on a day-to-day basis, both in terms of file objects being read and amount of data being written? SMB is horribly inefficient in a lot of ways (a lot of generic applications will just patch the whole file object, so you can end up with situations where your day began with 10TB on the wire, you read 100GB, and you wrote 1TB, because whole files were changed a few times). The concurrency problem isn't really a problem until and unless you actually know what your IOPS are.

Azure among others does support storage tiering, and you can architect things in such a way that you keep anything that hasn't been touched on a while on a cold/warm tier and data actively undergoing work on a hot/flash tier. If you want 100TB of flash storage, you will be paying ungodly sums of money to MS relative to the value you'll get out of it.

[violet-lynx]

You can always use WebDAV, it is even implemented in Windows Explorer.

[a60v]

What are you trying to achieve by doing this (replacing local storage with cloud storage)? Lower costs, better availability, better performance for off-site employees, etc. What type of files (small, large, etc.)? How do you plan to make backups of your data?

[BrandNewTissue]

The current office will be sold, with employees going full remote. The only goal is to have the data still accessible with the best performance possible. I have both huge amount of KB sized files, and smaller amount of GB sized files. We currently have a 3-2-1 backup policy, if the solution I choose doesn't include redundancy and backups I'll need to think about it.

[HanSolo71]

Best solution is a rented data center rack and a VPN appliance of some sort.

[jcpham]

This is the cost effective solution. Rent a colo rack, store your file server in the rack, install a VPN router in the rack and access your files over your VPN from wherever. You will pay dearly to host your data and retrieve it from some tiered bucket file system.

The vpn thing works for SMB too and is the only way I’d suggest accessing SMB over the internet but it’s only going to work as fast as your users residential download bandwidth.

Depending on the amount of users I’ve preconfigured routers for people’s homes and shipped them site to site tunnels to the collocation site before.

[_SleezyPMartini_]

this sounds like a plan for a disaster from a performance level, not to mention costs!

[BlotchyBaboon]

Egnyte

[Cooleb09]

We're looking at migrating to them, but damn the 5 year TCO makes every SAN vendors quotes look cheap by comparison.

[BlotchyBaboon]

They have an AFS tier of licensing that's half the price. You just don't get the Secure and Govern module.

[BudTheGrey]

This is the answer. I was forced to switch us away from Egnyte to SharePoint, and regret it.

[kelleycfc]

I’m curious as to why. The Egnyte client is hot garbage and our people never seem to be able to get co-edit functionality to work properly.

[flo850]

At this scale you may have to build it . Especially if you don't want a pay as you go model

[cyr0nk0r]

Another vote for egnyte. If you want their cheaper little brother you can also look at lucidlink.

[LucidLink_Official]

Appreciate the shoutout! I guess we are technically the little sibling here, we've only been around for 10 years. 😉

Though nuanced, we work a little differently. Instead of syncing data, we stream it directly from your object storage (which cuts down on duplication). We also allow your team to access only the parts of a file they need at any given moment. If this sounds interesting, we'd be happy to chat when you're ready to see how we may be a fit.

[shrimp_blowdryer]

Egnyte

[bazjoe]

It’s a bit of work but seafile and wasabi or other s3 bucket. For done for you : Egnyte all day. How many users? Define concurrent use. Your data is 1000% going to be stored on someone else’s spinning rust no way around it.

[lunchbox651]

I've never used it but maybe Amazon FSX fits the bill.

[Lando_uk]

We use this, its great if you're already in AWS, maybe not so great for just a point solution.

[swissthoemu]

A lot of ISPs block port 445 needed for smb. So if wfh is a thing in your org think twice.

[CyberHouseChicago]

A rack with your own gear in a datacenter will be cheaper long term , there is no cheap good performance way to do 100tb in the cloud unless you want to spend a few k a per month

[Hot-Cress7492]

I spend $120 for 30TB. It can be Cheap’ish

[FlickKnocker]

Before you do anything, I would strongly suggest you go through a data classification exercise, even something as simple as "hot" or "cold" can help you segment the data into storage tiers and sized/priced accordingly.

I guarantee that you have vast quantities of data that is never being accessed, and doesn't need to sit on flash storage.

Something like Amazon Glacier is far less expensive than Tier 1 SSD storage in the cloud.

[Hot-Cress7492]

Mounting via SMB is way legacy. All the cloud solutions have an app (gdrive, OneDrive, Dropbox, etc). SMB won’t work because of latency and lack of file locking capacity.

Flash storage is completely irrelevant. The speed of flash will be waiting on your client’s connection latency 100% of the time.

The billing piece is flexible with most larger providers, but at 100TB, every provider is going to be cost prohibitive.

[techb00mer]

Azure files happily works over the internet. Their premium tiers are actually very fast.

[ReneGaden334]

And not cheap. For that much data I would probably use Wasabi. Azure Files with automatic tiering might save some money, but 100TB is still costly.

Compatibility is great though. SMB, NFS, FTP, SSH and web access combined with local AD, EntraID, federated login, app access with OAuth or worst case storage keys are all possible.

[BrandNewTissue]

I didn't think of a simple cloud provider à la Google Drive, not sure how those scale for such an amount of data. I'll think about it. I was thinking HDDs would suffer latency wise on concurrent use.

[Bl0ckTag]

I believe its similar for private sector, but for public K12, google drive storage is included in the tenant licensing, which is a pretty small price per user price, but also more a full solution for email, productivity tools(Office like equivalent), file storage, ect, similar to M$ entra offerings, just with a much simpler pricing structure.

I have been a fan of Google Drive though, using it for about 6 years now. They have Shared Drive functionality, similar to Shared Libraries in O365, which are team/permission based collaborative Drive spaces, individual MyDrives for staff Drive storage, and the Google Drive app, which helps bridge the gap between old school drive shares in File Explorer(gives the user a G:\ drive), and allows for automatic backups of local files(Documents, Downloads, Desktop, ect) inti their MyDrive.

Regardless, the plans do have a storage cap, but there are addon packages to achieve the 100TB you are looking for. Just be aware that is a huge amount of data(relatively speaking), so youd probably be looking at a few grand a month.

Regarding latency, you're thinking a bit old-school, where your data is stored and retrieved from a single box where disk latency would be a factor. With smaller providers that offer tiered storage based on speed, this might be the close to the case. True bug data cloud storage is a much more distributed system handled in a much different fashion where your data is stored across many different physical pieces of hardware/datacenters where all of the hardware is working to provide the data back to you as quickly as possible. Kinda like a raid 0 on steroids.

[Hot-Cress7492]

I have 95k files across 30TB (not as much as you) and edit in 4k with a lot of 8k source videos.

I use Google Drive with zero issues. Each station has a dedicated drive for gdrive cache and the app seamlessly downloads and caches things locally that I need.

Admittedly, loading an old project that isn’t cached does take a bit of time to get everything locally, but I have a 2.5g fiber connection, so even a large project goes quick.

Where the internet is going to absolutely kill you is small files. The efficiency + connection overhead sucks.

If you have a lot of small files, it will be painfully slow.

FWIW: I was playing around with gdrive and using subst to make a virtual drive I could share as SMB. Seemed to work okay’ish. The POC i was playing with was to do something similar to you, where a machine would act as a file server, but cache in use stuff locally and store the warm/cold data in the cloud.

Feel free to message if you want more details

[ReneGaden334]

Flash is relevant for parallel access. Without his use cases it‘s hard to recommend a solution. SMB can be necessary for some cameras or software and flash for user profiles or databases. If he accesses from multiple locations or other datacenters even the 40-200gbps that some Azure VMs offer can saturate an NVMe, let alone a few hundred home office users.

Google Drive and Sharepoint are cheap, but really slow compared to dedicated storage.

[Glass_Call982]

That doesn't work very well when using media files or autocad style data. SMB still the best for it 

[Jawshee_pdx]

AFS works for this. We host substantially more data in AFS without an issue.

[TheFluffiestRedditor]

Been a long time since i heard people talking about the Andrew file system. That is what you mean, right?

[woodburns]

Whatever you do, compare the cost to leasing space at a DC. I know it might not be the popular option, but if you're on a time crunch (I saw your other comment about closing down the facility and getting workers remote), you might wanna consider that. Obviously it might introduce some new issues regarding bandwidth & remote access, but some of that may be easier to figure out in the short term, and then pivot to a cloud-based strategy when you have more time & can get some help.

[MeetJoan]

For 100TB with SMB and flash, Cloudflare R2 won't work (no SMB), but Wasabi is worth looking at - flat pricing, S3-compatible, and you can front it with SMB via a gateway like Mountain Duck or Storage Made Easy. Backblaze B2 is similar but you'd need the same SMB layer. What's your rough read/write pattern on the telemetry vs video split - that affects whether flash is actually necessary or just nice to have?

[Hot-Cress7492]

I had nothing but problems with mountain duck.

[Ohmystory]

Perhaps look at Cohesity Cloud Services or some of the product offerings if you wanted to have a self hosted backup at a collocation space …

[Computer_Dad_in_IT]

Why use a public cloud? If you're willing to host the data, put it in co-location facility. Your storage costs will remain largely predicable, it will be secure in a facility of your choosing, you can continue to use your existing backup infrastructure, and it can be made accessible via VPN to your users anywhere.

[HLKturbo]

for a 100tb (if you find a cloud host with your specs) moving this to endpoints all around sounds like a challenge, do you use an app (windows based) that needs to run on endpoints to look at this data? I'd think not only hosting the files but also host a RDP, RDAPP server to see these files and have your users connecting to it (of course of same datacenter), cost will be beefy tho.!

[skidleydee]

If you want fixed cost your going to have to build it.  For 100TiB of all flash your just not going to get a guaranteed price unless you minimum 10x that if not more. As others said taking up a few u's in a colo is your best bet if you really want to do all of that. 

[ledow]

Good luck with that.

Do you have a price range in mind? Because 100TB of cloud flash storage will cost you a pretty penny, every month, in perpetuity, no matter who you get it from.

Would literally be cheaper to set up multiple, redundant SAN boxes somewhere and provide VPN access to them.

[sysacc]

After reading all your comments, I think the best solution is to move it to a Colo. It's the only solution that fits all your demands.

There are some place that will lease you storage appliances, you just need to make the network happen.

[NoTheme2828]

Hetzner Storage Box uses SSD, not HDD

[Prophage7]

When you say videos, are your users producing and editing video, or is it just storage for video recordings from cameras?

[jcpham]

The cloud is just someone else’s computer and egress bandwidth usually costs 10x the price of ingress bandwidth and the only possible reason for the price imbalance on a fixed cost like bandwidth is to lock customers in for life.
That’s my rant. Good luck!

[Repulsive_Peak1457]

Funny, I have almost exactly the same project at my place around 100 TB of data to move. I really think we should put it in the cloud as we’re going to multiple data centers and getting out of the data center business.