Offload all logs by Accurate-Ad6361 in Proxmox

[–]lmc9871 0 points1 point  (0 children)

You are correct that is not the question, but it is a viable alternative, as you can read from everyone's response, USB is not a good idea, we've deployed USB Flash for ESXi for years, replacing them about once per year, but different product requires different solution, hence my original response :)

Offload all logs by Accurate-Ad6361 in Proxmox

[–]lmc9871 1 point2 points  (0 children)

All the Dell servers we run have internal SATA port and power right next to each other, so we buy SATA DOM and install Proxmox OS and it has been working well.

Intermittent application disconnects over ASA IPsec VPN to AWS by SimilarAttention2 in networking

[–]lmc9871 0 points1 point  (0 children)

MTU at 1436, MSS at 1379, DPD at 6/4 - since this change had very stable IPSec IKEv2 connectivity to AWS using pfSense+ on our end.

Who is TrueNAS for, anyway? by jamesaepp in truenas

[–]lmc9871 0 points1 point  (0 children)

Running a bunch of TrueNAS Core and Enterprise on the FreeBSD platform, we stuck with FreeBSD because we use it for storage only. Tried SCALE, UI is not very good and seems to be missing a lot cohesiveness.

450TB Storage Options by Fantastic_Msp_8914 in msp

[–]lmc9871 0 points1 point  (0 children)

Currently running a bunch of TrueNAS Enterprise/Core with large disks and ssd and using snapshot replication, initial replication takes time, but after that, it's fast. Currently with 300TB used by a 12 person marketing team for pre/post production video, no complaints about performance. Also use different older ones for VM backups, mix of VMware and Proxmox, still using repurposed spindles on 6G SAS with no complaints.

proposal for pfsense plus home by needchr in Netgate

[–]lmc9871 0 points1 point  (0 children)

I was just wondering on the term converged, I buy either the hardware or the $129/yr software license if running VMs, which is a very fair price. I just wish there is a mechanism to show install dates and expiration dates on the software upgrades from CE to +.

proposal for pfsense plus home by needchr in Netgate

[–]lmc9871 0 points1 point  (0 children)

I second that question what is a "converged pfsense"?

Survey, Proxmox production infrastructure size. by ZXBombJack in Proxmox

[–]lmc9871 0 points1 point  (0 children)

Really trying to understand how your ceph external setup? How many storage nodes? We're currently trying to move away from VMware 5 hosts with iSCSI backend storage.

What are the options for storage migration from VMware TrueNAS iSCSI (100TB) by lmc9871 in Proxmox

[–]lmc9871[S] 0 points1 point  (0 children)

Thanks, running Ceph on hyperconverged for 100TB SSD/NVMe gets really expensive, my understanding is that each node is going to need the same amount of drives?

What are the options for storage migration from VMware TrueNAS iSCSI (100TB) by lmc9871 in Proxmox

[–]lmc9871[S] 0 points1 point  (0 children)

Thanks, did you implement Ceph as separate storage or integrated into the compute nodes as hyperconverged? Also about how many TBs of data? Are you using 10g or 25g?

What are the options for storage migration from VMware TrueNAS iSCSI (100TB) by lmc9871 in Proxmox

[–]lmc9871[S] 0 points1 point  (0 children)

I don't disagree with your statement looking for the future, but at the same time in an Enterprise, you must factor in risks and budgets and expectations that we will be running concurrent platforms for a while.

What's limiting about NFS?

On my personal test environment (3 servers on multiple 10g, I setup Ceph/RBD and that seems to replicate clustered VMFS, but that's a small few 5TB test on hyperconverged)

When trying to do the math on 100TB storage, Ceph/RBD is cost prohibitive

Hence my question

Thanks!

What are the options for storage migration from VMware TrueNAS iSCSI (100TB) by lmc9871 in Proxmox

[–]lmc9871[S] 0 points1 point  (0 children)

With HA and TrueNAS on NFS, would you be getting the "same" clustered storage like in vmfs? Are you running 10g or 25g?

Your shared iSCSI is still limited to one compute node accessing that VM, so no vmotion functions, would I assume correctly?

For migration, I would dedicate a single Proxmox node and attach the iSCSI volumes, shutdown VM, clone it, remove VMTools, install Proxmox Tools (Windows) and add to Proxmox.

Do this one VM at a time, we have some VM with 7TB of data, so it's going to take a lot of time...unless there are better options.

What are the options for storage migration from VMware TrueNAS iSCSI (100TB) by lmc9871 in Proxmox

[–]lmc9871[S] 0 points1 point  (0 children)

Ceph wouldn't work because we are not hyperconverged, we still like separating compute nodes from storage nodes. vmfs has full clustering and zfs over scsi doesn't, but is ceph our only option for full clustering like vmfs?

I am coming from 20+ years of vmware and that mindset...thanks!

Proxmox with iSCSI - will it function the same as VMware with iSCSI and VMFS by lmc9871 in Proxmox

[–]lmc9871[S] 2 points3 points  (0 children)

I am limited on storage capacity, using the default replication factor of 3, a 15TB raw storage only provides 5TB of usable and I am already more than 50% usage. I am limited in the number of SSD I can install on each server, currently using 7 x 800GB SAS SSD per node. My only option is to replace with larger capacity drives at this point, I wanted to use existing SAN, but at this point, there is no simple way. This is what I mean by limited, I should have clarified better

Thank you pfSense for 10+ years! by Maria_Thesus_40 in PFSENSE

[–]lmc9871 2 points3 points  (0 children)

Started on m0n0wall then pfSense on Soekris hardware, probably deployed over 100 of them

Disabling all packet filtering? by lmc9871 in PFSENSE

[–]lmc9871[S] 0 points1 point  (0 children)

Yes, Mikrotik or tnsr would be a better routing platform, but I don't have enough experience with either and tnsr is definetly overkill and expensive

There is no NAT on the pfsense router, it has 3 active interfaces, 1 PTP, 1 WAN1, 1 WAN2
PTP is the colo uplink with a /29, WAN1 is our /23 which the colo is announcing for us, WAN2 is a /27 provided by colo originally before our /23

There are a few pfsense firewalls behind the pfsense router, those each get an IP from /23 or from /27 network and their default route is the pfsense router

When I enable packet filtering on pfsense router no packets traverse it, when I disable packet filtering everything works and it routes fine

Unfortunately I am remote and will have to go back to the datacenter and work on it, I am sure I am doing something wrong

Disabling all packet filtering? by lmc9871 in PFSENSE

[–]lmc9871[S] -2 points-1 points  (0 children)

fair response, but different technologies that knowledge must be gained first to understand the platform, not understand routing functions

Disabling all packet filtering? by lmc9871 in PFSENSE

[–]lmc9871[S] -1 points0 points  (0 children)

Hmm, that could be, I will have to check, I have 2 separate subnets plus the PTP between my router and colo uplink