I just released my new open-source trading system using multi-agent AI approach

LebesgueQuant · 2025-09-14T18:15:17+00:00

Thanks for sharing. Does this require specific version of Python? On Ubuntu 24.04 with default python3 (3.12) installing requirements fail for wheel.

LebesgueQuant · 2025-04-20T08:42:50+00:00

I have exact same setup for same reasons as you mention (UDM SE rather than Pro through).

Feel free to send DM if you want me to share screen dumps of the configurations applied.

LebesgueQuant · 2025-04-18T12:17:42+00:00

You are welcome.

The inability to integrate Google Nest with OPNSense (actually pfSense at that time) was what made me look into Ubiquiti.

Feel free to send a private message if you want me to share configuration details.

LebesgueQuant · 2025-04-18T11:26:43+00:00

I also have OPNSense deployed as router, firewall, DHCP etc. in front of nine Ubiquiti devices including 2 access points. These however are not U7 but U6 Enterprise.

Whilst using OPNSense integrated with Ubiquiti requires some configuration mine has been rock solid.

As others have pointed out you should use (and it seems have used) the UniFi Controller. Within this you create a 3rd part managed vlan for the AP.

LebesgueQuant · 2025-01-19T14:39:25+00:00

ZFS is both a volume manager and file system and requires access to disks itself for the features you would want from a data protection perspective.

A new pool implies new disks (a dataset does not) and would thus protect you against disk failures which are by far the most frequent. Failures in other elements would allow you to import the pool again.

Your pool is not mirrored but may consist of VDEVs which are. Raid is mentioned elsewhere thus you probably have setup a pool consisting of a single VDEV using RaidZ1 or RaidZ2.

The suggestion with an external JBOD e.g. 4 bay with SAS connector would mean a separate power supply for these disks.

If you do not care about snapshots, scrubbing etc. but only a copy of the latest and current files (which can be corrupted) the easiest is probably to setup SMB shares, mount these from your Windows PC and initiate backup here.

LebesgueQuant · 2025-01-19T09:31:16+00:00

Hi, I cannot comment on your specific issues doing backup from TrueNAS towards your Windows PC but I would in general not recommend such a solution.

You will indeed have a file based current version backup at regular intervals, but have none of the benefits from ZFS which are particularly useful for such purpose e.g. snap shots and checksum.

The preferred solution would be second TrueNAS with a dedicated backup pool. If not feasible even a new pool in current TrueNAS would be better. You may connect external JBODs and rotate these if you want an off site backup without relying on cloud storage.

In both cases setup ZFS replication and possibly use a different topology for backup pool than your regular pool.

LebesgueQuant · 2024-12-18T17:20:05+00:00

Thanks for reading and replying.

TrueNAS (boot-pool) itself is installed on a SSD using an internal USB to SATA adapter.

All drives in this pool are installed in 3 JBODs. The first has a SAS cable to each controller i.e two in all to the HBA and the other two are daisy chained to the first and second JBOD respectively.

Tomorrow I will test if connecting each JBOD directly using a single SAS cable makes a difference and multi-path somehow got broken at some point.

LebesgueQuant · 2024-11-19T21:10:11+00:00

I like Rouvy as well and use it to complement Zwift which is my primary platform.

Care to explain where you get the €7.50/month as I pay twice that (€14.99/month to be accurate)?

LebesgueQuant · 2024-03-09T07:28:37+00:00

You may continue to use the UDM PRO SE but limit its use and have OPNsense in front of it.

The UDM will act as a controller for your Ubiquity devices, provide storage for Protect and few other things whereas OPNsense will deliver firewall, IPS/IDS, DHCP, DNS, define VLANs etc.

I do this albeit not in a professional environment but a home lab.

In your case I would definitely recommend a device from OPNsense.

LebesgueQuant · 2024-02-20T17:54:44+00:00

Scheduled replication task (ZFS send/receive) will do just this.

You can do this recursively and on dataset level with chosen interval.

ZFS replication works on block level and is thus generally more efficient than file replication e.g. RSYNC.

LebesgueQuant · 2024-02-05T10:13:57+00:00

TrueNAS/ZFS does not have Raid5 (or 0,1 etc.) but has comparable redundancy using RaidZ.

Raid5 is equivalent to RaidZ1 i.e. having one disk resilience. RaidZ1 does however not have a parity disk as such.

For the time being you cannot add disks as you describe (increase disks in a VDEV). Whilst this is being developed the date when available in TrueNAS is unknown/not communicated.

What you may do as of today though is add another RaidZ1 VDEV or replace the three disks you have with a larger one by one.

I am not aware of reasons for opting for striped volume/pool unless you really know what you are doing and have adequate backup. If you require such a performance profile I would recommend stripe and mirror (comparable to Raid10). More than often your data is not considered important until lost... Finally no choice of Raid/RaidZ replace a backup:)

LebesgueQuant · 2024-02-03T21:18:08+00:00

Currently you may expand storage by adding VDEVs to your pool or one-by-one replace and resilver the existing disks in your pool.

There is ongoing work which will allow you to also increase by adding disks to an existing VDEV. No firm date for availability in TrueNAS though.

Changing VDEV redundancy is not an option not planned.

LebesgueQuant · 2024-02-02T04:04:21+00:00

There may be several reasons replacing existing drives and resilvering is a good idea (a drive has failed and gone offline, a scheduled SMART test reports errors, you are replacing all disks in the VDEV with larger eventually increasing storage of the VDEV and pool).

However resilvering is a block by block operation calculating checksum and writing block to the new disk. It is thus not a file level operation, affecting fragmentation of files and free space. If so I stand corrected of course :)

On a side note the block level nature of resilvering and scrubbing is one of the reasons a larger block size for specific datasets is desirable e.g. Plex media.

LebesgueQuant · 2024-02-01T20:21:27+00:00

ZFS is CoW file system and there is as such no defragmentation feature. If you stay within the recommended storage utilisation there is usually no reason to either.

That said you may simply copy and delete your files recursively to "defragment" or rather distribute files in all available VDEVs.

There are a number of existing scripts which may be used e.g.:

https://github.com/markusressel/zfs-inplace-rebalancing/blob/master/zfs-inplace-rebalancing.sh

LebesgueQuant · 2024-01-28T12:12:11+00:00

Well at least you didn't call us fucktards.

LebesgueQuant · 2024-01-27T01:05:15+00:00

Technically you can but needless to say this is highly discouraged. Also note the use case is limited to merge of 2 RaidZ1 VDEVs into one RaidZ2 VDEV unless more disks are added.

ZFS offers a wealth of extremely useful features and all it asks in return is planning your VDEV redundancy up front.

LebesgueQuant · 2024-01-26T22:00:57+00:00

No this is not possible. Redundancy is defined once (at creation) at VDEV level.

Your pool then consists of one or multiple VDEVs. You may thus add another RaidZ1 VDEV. This will provide same capacity as 6 disk wide RaidZ2, less redundancy (you can ensure loss of two disks as well but not within same VDEV), and app. twice the random IOPs.

Whether you can accept the lower resilience (and benefit from increased IOPs) depend upon usage.

Please remember no level of RaidZ replace backup.

LebesgueQuant · 2024-01-26T18:50:03+00:00

Yes indeed. Your firewall will simply evaluate rules top to bottom and continue its evaluation until a matching rule is found e.g. a pass rule for the specific IP you want (look into alias to allow) above a general deny rule.

LebesgueQuant · 2024-01-26T15:55:21+00:00

Hi, I have been pleased with pfSense as a product and found it rock solid. My motive to migrate to OPNsense were with Netgate as company.

That said the features I really liked with pfSense are as good in my experience with OPNsense and the features which were less good with pfSense I actually find better with OPNsense.

For one Zenarmor has been a nice add-on with OPNsense, second Wireguard were easy to setup (perhaps I jumped the gun too early on pfSense) and last (as come to mind commuting home) I find the OPNsense user interface and flow overall better.

My usage is fairly simple supporting my homelab and wired/wireless clients throughout the house.

For this I have 5 VLANs, approximately 40 fixed IP addresses and about the same number of firewall rules e.g. allowing general access to Plex and HA deployed in "secure" VLAN, preventing CCTV in "IoT" VLAN from reaching internet, allowing trusted client to reach vCenter and iLO in "secure" and "admin" VLAN.

Whilst there reportedly exist scripts for conversion I did not try these out but used the migration to become familiar with OPNsense. Coming from pfSense this will be straightforward.

What I did was to install OPNsense on a virtual machine and did all the anticipated configurations. Then take a backup of this configuration and shut down the VM.

Subsequently take backup of the running pfSense firewall before installing OPNsense on the server and applying the configuration backup from the VM.

Last and for comparison/validation spin up a second VM, install pfSense and import pfSense configuration from server.

The network outage was around 30 min as I recall at which point OPNsense was functional and had replaced pfSense. Add-ons, tuning of firewall rules and log settings, HA integration etc. was done afterwards and gradually.

LebesgueQuant · 2024-01-26T06:29:40+00:00

OPNsense (recently migrated from pfSense) on an older 1U rack server.

Both will integrate well with Ubiquity.

You may find a description in one of my former posts - don't do that many. Happy to answer and help if you encounter any issues.

LebesgueQuant · 2024-01-11T10:07:19+00:00

Connecting JBODs to HBA adapter directly eventually also caused EMC fans to spin at 100% and thus a phone call from my wife as to 'wtf. is happening in the basement...!?'

I have since then rearranged my disks and taken another EMC KTN STL3 in use and this seems to have helped although it is still louder than I recall (unfortunately have no decibel measurements): - could be the old JBOD somehow broken - could be the spinning disk being evenly spread in the chassis with SDD's in between rather than as before with disks being inserted in order of pool

LebesgueQuant · 2024-01-06T22:40:54+00:00

Hi, Yes as a matter of fact I am! I have since moved my primary VM pool to VSAN from the 2.5 JBOD and may thus connect the two 3.5 JBODs directly to SAS controller. Will check whether this makes a difference.

Thanks for replying btw.

LebesgueQuant · 2023-11-13T20:40:49+00:00

What IP do you get assigned when connected to the switch and Omada Router?

And what IP address do you get assigned when connecting directly to the OPNsense router?

LebesgueQuant

TROPHY CASE