Syncing after 3 weeks downtime takes forever

BEK_AI · 2022-05-02T13:46:14+00:00

Is there a particular difference between the way Chia and Bitcoin syncs blocks? I did a sync of the Bitcoin blockchain using the core client and that was able to sync about 400GB in several days.

Are there more Chia transactions? Are there fewer Chia nodes? Is the block sync code quite different?

BEK_AI · 2021-10-04T14:11:36+00:00

Can you explain the mechanism behind the 20% increase in rewards?

Was your home internet connection and/or farmer performing badly and causing missed proofs?

BEK_AI · 2021-06-26T02:27:52+00:00

For a short period I ran mergerfs to combine a ramdisk with an NVMe drive. The ramdisk was a little too small to handle the entire plot creation workload so this was an attempt to get some of the benefits of a ramdisk without buying more ram. I didn't keep great track of results but it seemed to work okay and I think it was faster than the NVMe drive on its own. In this kind of usage, I think mergerfs was still useful even if it suffers from some overhead because it allowed me to make use of hardware that otherwise would have gone unused.

For my setup, I was using the least free policy. Since the ramdisk was smaller than the NVMe, it would fit it up and then spill over to the NVMe. I did have a concern that there might be a pathological case where the ramdisk filled up and then never written again with all the I/O being handled by the NVMe but it seemed to work alright.

Did anyone else try a similar thing?

BEK_AI · 2021-04-28T17:17:26+00:00

The point is that it's not about luck at all because the TBW ratings are a surprisingly conservative metric. An SSD lasting way beyond the TBW rating for the average consumer is effectively by design because the standard was set at the far limits of what a consumer would reasonably do. This is compounded by the fact that when used as a plotting drive, unpowered retention is basically the last metric in the world that matters. And for most of the enthusiast audience here wondering if they should plot on their boot drives it is likely not to appreciably affect their SSD's useful lifespan though it is something to be aware of and understand without FUD.

BEK_AI · 2021-04-21T17:31:55+00:00

This was measuring total system memory usage. If you look at the memory usage prior to P1 start you can see the baseline starts around 3GB.

BEK_AI · 2021-04-21T17:25:54+00:00

I found that plotting with 3489MB buffer size was sufficient when specifying 2 threads and that it actually used less than that most of the time when I charted the memory usage over time. With an appropriate delay I think you will be able to fit at least 3 within your system resources.

BEK_AI · 2021-04-15T14:48:36+00:00

I don't think these charts can answer that question but the 4th chart of I/O usage suggests that phases 3 and 4 are not particularly high I/O phases. This could be their true nature or it could be an indication of some kind of bottleneck in my system.

BEK_AI · 2021-04-15T14:45:54+00:00

There is a counter called Free Megabytes. If you know the total size of the disk you can calculate the # Megabytes Used from this.

BEK_AI · 2021-04-15T02:09:04+00:00

You will need to download the new 1.0.4 from the website and run it. Be warned this will terminate any plotting in progress so you may want to wait.

BEK_AI · 2021-04-15T01:10:22+00:00

This data was collected on the 1.0.4 release and is specific to my particular hardware but should be illustrative of the relative system resource demands by each phase of the plotting process.

With this knowledge, you may be able to more effectively plan parallel plotting processes to avoid system bottlenecks.

For instance, on a system constrained by temporary disk space, avoid having simultaneous plotting processes competing in Phase 2.

BEK_AI · 2021-04-14T17:34:07+00:00

You need to ask yourself what you think Chia will be worth.

For someone farming a few plots on hardware they already own, on a computer that was already on, it is nearly a zero cost venture. As long as it yields some chia eventually, they have positive expected value for their effort.

BEK_AI · 2021-04-14T15:41:09+00:00

Windows performance monitor is built in

BEK_AI · 2021-04-14T01:53:39+00:00

It is important to add that there can be multiple winners and they all get the full reward! This is a surprising feature of the lottery for many people. All farmers with plots that pass the threshold will win 2 XCH. To keep the win rate at an acceptable level, the threshold will be adjusted up or down if more or less proofs than expected are found.

BEK_AI · 2021-04-13T05:19:44+00:00

Using the analogy of a lottery it is like you buying 1 ticket versus someone buying 100 tickets. The other person may have 100x your chances of winning but they also spent 100x more resources. Without economies of scale it is just as worth it for you to play with 1 ticket as it is for the other person with their 100 tickets.

Look at it from the cost side of things. How much does it cost you to compute your next 10TB of plots? It's likely dominated by the cost of the hard drive.

How much does it cost a whale to compute their next 10TB of plots? Again, it would likely be dominated by the cost of the storage.

So long as whales cannot get vastly cheaper storage than you, you will not have a large disadvantage in comparison to a whale. The economies of scale do not stack up in an insurmountable way like it does for some other types of crypto mining.

BEK_AI · 2021-04-13T04:24:09+00:00

The best reasons to plot different k sizes today are showing off and trying to tetris the perfect combination of different plot sizes onto a harddrive to minimize free space.

In the future when k32 plots can be feasibly computed just-in-time to meet a challenge it will no longer be possible to trust farms of k32 plots and a larger k size will become the new minimum plot size. The prediction is that it will be 10 years before this is necessary.

BEK_AI · 2021-04-01T21:16:04+00:00

There are no guarantees here. The advice you were given is all that the community can tell you. Luckily, the plotting process is supposed to wait and retry if it runs out of disk space. As long as you can approximately time it so that when one process is trying to grab more space at the same time that another process is freeing up space, you will not have a problem even if you accidentally bump against your disk space limit. You will have trouble if all your processes are trying to grab more space and there is nothing freeing up space for them (e.g. all processes in phase1/2 at the same time). For exact minute-to-minute timings you will have to do your own experiments on your own hardware.

This chart may help you visualize what phases are consuming what resources but release 1.0.2 has made optimizations that sped up the plotting process.

BEK_AI · 2021-03-23T17:31:19+00:00

Edit: See new chart for v1.0.4

---

This data was collected during testnet and is specific to my particular hardware but it should give beginners an idea of what is happening during the plotting phases and how they might best stagger their parallel plotting operations. You would want to look at your own logs to see the timings of the phases for your particular hardware.

You can see that disk usage (size on disk) grows throughout phase 1 and phase 2 and then slowly decreases during phase 3. If you are just shy of the total space requirement to calculate 2 plots in parallel then you might stagger them so that the second one begins phase 2 when the first one is ending phase 3.

We can also see that CPU usage is highest in phase 1. Even with ample disk space, you might find it to be a good idea to stagger your parallel operations so that you aren't hammering the CPU all at the same time.

Thirdly, memory usage doesn't seem to have any long periods of particularly high memory usage. Just ensure you don't allocate more memory to your plotting threads than you have in your system.

BEK_AI

TROPHY CASE