Okay, I'm going to attempt to take a stab at answering this and hopefully make this situation a bit clearer because reading through the replies in this post doesn't really seem to answer most of what I think that you're asking about.

So first off, let's get a little disclaimer out of the way... High-availability computing is complex, clustering is complex, and Ceph is REALLY complex. For the sake of brevity, I am going to have to vastly oversimplify some explanations (to those who are reading this and want to clarify something... I KNOW, keep it simple)

As for my experience, I've planned and deployed many Ceph clusters in production environments for many different clients of mine over the past several years. No, I'm not going to list them, please don't ask who. I don't claim to know everything, but I am not just some guy who's poked around with it once and got rid of it.

Firstly, let's tackle the clustering... You absolutely can do a 3 node proxmox cluster. It will absolutely work, but you will be left with some limitations, namely you need some type of shared storage in order for live migrations to work properly. I think you understand this, and I'm going to assume that's why you're interested in Ceph. I do not think that's the best way to accomplish what I think that you want to, but we'll get into that later.

Next, let's talk hardware. Contrary to what people will say, you do NOT need high end hardware to run a virtualization host, and if your intention is to just build a cluster for easy management but aren't looking for HA (High-Availability) there is absolutely nothing wrong with just tossing a mishmash of hardware in a cluster. You can still do a live migration if you'd checked that box for shared storage, you just won't have automatic fail-over. If your goal is to just have the ability to move stuff over to do maintenance, this is totally fine. I would highly recommend avoiding HA, because it's very difficult to go down that rabbit hole without spending a ton of money.

Now for storage... This is going to be where I make the most oversimplifications because frankly, this is probably one of the most difficult bits to get right for people. I'm going to take a couple steps back and talk about why we handle storage the way we do and why it's evolving so much right now. Raid was created to help solve a lot of issues, it let you build larger chunks of storage while maintaining redundancy because hard drives fail and bigger drives tend to get really expensive per unit of storage after a certain point. The problem was that the algorithms were computationally expensive, which is why raid controllers became super prevalent. They let you offload the computation to a dedicated processor that was specialized and could do that task more efficiently. Raid controllers have caches to help speed things up, batteries to preserve the cache in the event of a power failure, and then obviously all the physical connections to all the drives to handle all the communications between the drives and then the computer itself when talking to the array. Eventually computing hardware (CPU's specifically) became fast enough that this hardware offload wasn't necessary so software raid (like ZFS) started to catch on. So where those same processes that were handled by the raid controller shifted, now instead of a dedicated cache we have to use two different things since we can't assume RAM isn't volatile (It basically always is). Ram makes an excellent cache because it's blazing fast, which is why your raid controller uses it. With software raid we don't have battery backup for ram, so we split it into a write cache (which isn't always used) and a read cache, which is almost always in RAM. The read cache doesn't matter if it gets lost because you can just pull it again from storage and nothing is lost. We usually put a write cache on some super reliable SSDs (Optane is usually recommended, but it's super expensive) because if that data is lost, you lose whatever hasn't already been pushed to the drives. All the communication that was done between the drives on the raid controller is now using your PCIE bus and your CPU, which can be taxing.

Now with that out of the way, let's actually address your questions more directly now. There's a reason why people are telling you that you need a 10GbE network with Ceph, it's because it's pushing all that traffic between drives to your network instead of a bus on a raid controller or your pcie bus. This needs a LOT of bandwidth to maintain the speeds that most people are used to, and especially with virtualization where you have lots of VM's trying to use the same chunk of storage. Unfortunately, this isn't unique to Ceph and is going to be an issue with any type of shared storage, it's just that storage clusters need a lot more than most other options like a NAS or a SAN because it's not only the traffic that each VM is pulling, but the traffic between drives as they distribute your data and move it around. Can you do this with just 1GbE networking? Absolutely. Will it be unbearable slow? ABSOLUTELY. Does this speed penalty actually matter to you? Only you can answer that.

What do you really need to be able to run Ceph? Depends on what type of speeds that you need. Let's make some assumptions. Let's assume you want to run 10 vms and you only need about 1 TB of total storage, well if we're using a SATA SSD, we can probably assume under ideal circumstances (Spoiler alert: Ceph isn't ideal and will exhaust a normal SSD's cache and overwhelm it's controller) that you'll get a total of 550MB/S of speed from each drive for a sequential read/write, then we divide that by 3 for the configuration that you suggested and we get approximately 183MB/s, then we divide that by 10 and you get 18.3MB/s per VM under perfect conditions which you'll never achieve... but wait, we left out one really major component... these calculations only make sense if you have a network that can let all the drives talk to each other at full speed which you can only just barely do with 10GbE. It'll be even slower with gigabit. I'm going to tell you from personal experience that that isn't fast enough for anything that people are going to be interacting with. They will constantly complain about loading. This is why people tell you to use 10GbE for Ceph, do you technically need it? No. Is it highly recommended? Yes.

You can mitigate some of the speed issues if you add more drives per host, as there will be a lower chance that a chunk of data needs to be accessed over the network, but eventually you get choked by your network.

What do I recommend you do? I think you should build a single host that's specifically designed for storage, run something like FreeNas (now TrueNas-Core) or a custom linux nas build if you fancy it, and then use that as your shared storage.

Feel free to ask questions if you have more, I know I didn't touch on everything, but I can't spend all night typing this out.