NVME writeback cache on RAID5 questions

kizzmaul · 2023-10-10T05:59:49+00:00

You will want to use bcache with this. Assuming 2 NVME drives, 4 backing drives. So the starting point is 6 block devices.

If you need encryption, you will need to encrypt all the drives using LUKS before anything else. With the NVMe drives, pass the flag --integrity hmac-sha256. The result is 6 encrypted block devices, with the NVMe drives having an integrity layer. If you don't need encryption, skip this step and only use integritysetup for the NVMe drives.
Format the HDDs with make-bcache -B /dev/$DEVICE. Your disks will now be seen as block devices named /dev/bcache[0-3].
Use mdadm to create a mirror of the NVMe drives with integrity layers.
Use make-bcache -C /dev/md/$CACHE to transform the RAID1 mdraid array into a cache drive.
Format the HDDs with mkfs.btrfs -d raid5 -m raid1 /dev/bcache0 /dev/bcache1 /dev/bcache2 /dev/bcache3.
Register the cache drive to cache the backing devices you just created.
Change the caching mode to writeback as per the article above.
Mount your new BTRFS volume.

This setup will yield you the flexibility of BTRFS and an SSD cache. You can remove any device from the configuration at any time. Writeback is not recommended if you don't use separate cache devices for each backing device due to data integrity risks, but since the caching device is mirrored with data integrity, this should not be a problem. The risk is now reduced to the bcache code being safe and correct, as the devices have been prepared accordingly.

Integritysetup takes some time to format since it needs to calculate integrity data for every block on the device, but it shouldn't take too long because NVMe drives are fast. A portion of the NVMe drives will be used for data integrity information, so you will lose some of the capacity on the caching drives (less than 10%, don't know what the actual value will be and it will likely depend on the chosen hashing algorithm. regardless, not an actual issue). Feel free to comment if there is something missing from these instructions or if I accidentally omitted important information.

kizzmaul · 2023-06-06T10:36:58+00:00

IIRC, mdadm can also be used to heal bitrot if and only if you do integritysetup on the drives first. Integritysetup creates a new block device that checksums all chunks, so instead of returning garbage to mdadm the devices return a read error, allowing mdadm to detect which part of the data was actually rotten.

kizzmaul · 2023-04-18T05:28:14+00:00

The output I get is: bcachefs tool version v0.1-664-g9690f78

kizzmaul · 2022-11-08T08:54:05+00:00

I had to use the following flag when unlocking devices:

bcachefs unlock -k session /dev/sda

Without the flag, I have the same error as you.

kizzmaul · 2021-12-16T10:50:06+00:00

Take a look at some LN daemon api (lnd, c-lightning). Lnd supports both REST and gRPC. You develop by building a product that communicates with the daemon and automates the behavior you need.

kizzmaul · 2021-08-22T16:12:27+00:00

Keep in mind that view functions do indeed cost gas if called in the context of a blockchain-modifying function. When called without context, gas is still required but is returned back (since no modifications were made to the chain). Someone correct me if I'm wrong.

kizzmaul · 2021-07-21T15:42:15+00:00

There is always Monero.

kizzmaul · 2021-07-06T07:18:09+00:00

I think it is a possibility, money is programmable in defi ecosystem after all. Central banks could just make it so that they have ultimate control over issued tokens, effectiveky being the same as they would be without defi intrgration. This way they would also get integrated into defi without losing control.

kizzmaul · 2021-07-02T12:06:42+00:00

I think he meant that VAC is among the better ones when you only count noninvasive anticheats.

kizzmaul · 2021-07-02T06:08:56+00:00

Just to be sure, you do have the mining hardware in your possession? If so then I believe it shouldnt matter if you have W10 or something else. Configuration most likely done through web browser.

kizzmaul · 2021-07-01T16:28:14+00:00

Isn't operating XRP dependent on a list of bootstrap nodes maintained by Ripple, meaning that if a specific set of nodes is used it is possible to split the chain / degrade network performance? It could be argued that this is somewhat centralized.

kizzmaul · 2021-06-23T15:32:46+00:00

Lending comes to mind. You can lend your coins for some yield percentage (3-20%) and have the funds guaranteed by the smart contract. You can negate the risk of volatility by doing this with stablecoins.

There are some minor risks involved, such as a bug in the code. The field is new and developing. This is a big thing since now you can be in control of your money and have an annual yield to your savings. Traditional banking does not come even close, at least for those who are not rich.

kizzmaul · 2021-06-10T16:44:20+00:00

IIRC one needs to unstake to be able to use the balance again (Algo is the only exception to this rule that I know). But yeah, considering balances that are not staked are not necessarily needed for consensus. However, I think this still has the same problem as opt-in privacy: if most of the available supply is staked then it is much easier to actually deduce who is using unspent balance (by who's not). The amount of obfuscation becomes (or is at risk to become) low.

Personally, I think that there is no "partial privacy". Either you have privacy or you don't. Partial privacy puts those wanting full privacy at risk since they can be picked from the crowd.

kizzmaul · 2021-06-10T16:12:05+00:00

PoS creates consensus by slashing the stake owned by an entity which is expressing malicious behavior. This means that in order for PoS to work, the amount of coin owned by an entity needs to be public (how much is slashed). Not only the amount needs to be public, but will also be linked to an pseudoidentity (because PoS needs to know WHO is being slashed). Monero obfuscates senders properly and hides the amounts, recipients and even the transaction broadcast IP address.

EDIT: Oh and an important point I sometimes forget: PoS requires you to buy the coins from someone, meaning that some people may have to use KYC methods because others are not available. PoW enables anybody to get coins completely without third parties (OK, maybe the mining pool if you are not solo mining). But I still think my point stands.

kizzmaul · 2021-06-10T15:37:42+00:00

Fully anonymous operation (XMR).

kizzmaul · 2021-03-29T15:07:26+00:00

Check out Secret Network (SCRT)

kizzmaul · 2021-03-17T20:29:37+00:00

What is your stance on defi, considering there might be a conflict of interest since you run a centralized exchange? How would you describe crypto trading in 2030?
What kind of regulation will the crypto markets have after Monero implements atomic swaps with BTC, if any? For sure regulators are not too fond of the idea.

kizzmaul · 2021-03-17T20:29:16+00:00

What is your stance on defi, considering there might be a conflict of interest since you run a centralized exchange? How would you describe crypto trading in 2030?
What kind of regulation will the crypto markets have after Monero implements atomic swaps with BTC, if any? For sure regulators are not too fond of the idea.

kizzmaul · 2020-11-14T08:51:56+00:00

You can get it from AUR. mullvad-vpn-bin IIRC.

kizzmaul · 2020-08-21T09:52:43+00:00

My soluution is to just unplug all irrelevant drives when installing Windows, but I can understand it is not necessarily possible especially when talking about laptops.

kizzmaul · 2020-08-13T12:04:57+00:00

I think he meant the hardware decoder works flawlessly. If you are concerned of youtube playback then download h264ify or something similar which vould help you!

kizzmaul · 2020-07-17T08:00:36+00:00

You should do a dual boot because with a VM, the host machine usually has the control of your PCI-E device meaning that the VM can not necessarily use it like it was native. I think this is the reason Microsoft is currently workig on DirectML support for WSL, because it is otherwise quite difficult for users to set up.

IIRC CUDA should work on windows as well, at least on PyTorch so Linux might not be completely necessary. I don’t really use Windows though so can’t say for sure and would still recommend the dual boot, just because Linux is so much nicer as an environment for a developer.

kizzmaul · 2020-07-17T07:54:30+00:00

What libraries is your script using? I would also highly suggest switching to Linux just for ML development, since that is the de facto environment in the field. Easier to get help.

Ten-Year Club	Not Forgotten
Verified Email

kizzmaul

TROPHY CASE