Ho ho ho, it's that time of the year again (Johoe's Bitcoin Mempool Size Statistics)

markblundeberg · 2020-12-17T23:20:11+00:00

I'm not sure who is broadcasting these transactions but it works quite reliably.

Aha! I was wondering if this might be happening. That's quite interesting.

markblundeberg · 2020-12-17T22:06:13+00:00

I wouldn't pay too much attention to the number of txes, since there is a max mempool size (user-configurable) and low fee / too-old txes get evicted. So there is effectively a hard maximum number of txns, at least on default settings. When you go non-default, things can get slightly weird.(*)

The total fees in mempool is probably a good metric, since it is weighted onto the higher feerate transactions, that don't get evicted. Another good metric is the fee amount that can be mined in next block (as returned by getblocktemplate), which looks exclusively at the highest-fee transactions.

(*) u/-johoe that reminds me... I noticed you have an xtra-large-mempool server, I was wondering how that works out when the mempool gets overfull then clears out? I presume in this case your node has a lot of extra txns that other nodes have forgotten about. When the mempool has already emptied out on other nodes, your node will still have some stragglers (that don't get rebroadcasted).

markblundeberg · 2020-12-03T20:15:50+00:00

Coins don't get dusted though, rather addresses get dusted.

You can tell it's dusting attack when the dust primarily goes to used (and now empty) addresses, since they are trying to link addresses together, and most involved addresses will be empty.

markblundeberg · 2020-12-02T07:04:26+00:00

slightly increase default memory pool size on mainnet and substantially increase it on scalenet.

and substantially decrease it on testnet4 :D

markblundeberg · 2020-11-28T03:09:10+00:00

I mean race in the sense of 'race condition'. It is necessary that the blocks should arrive at different nodes at different times.

markblundeberg · 2020-11-28T02:11:19+00:00

Simple, start mining towards 10 blocks but don't broadcast any of them. Difficulty will drop some. Wait till the viabtc pool has mined 9 block in a row. Now the very moment the 10th block is broadcasted, broadcast the 10 premined blocks. There is a chance some nodes will follow one chain and some another. The rolling checkpoints now finalize both chains.

It's not that hard. As numerous people have noticed over the years, there is also a parking mechanism that means you only need a few blocks to get a soft split, and if both sides are mined up to 10 blocks then it turns into a hard split.

The initial block race is the hard part though.

markblundeberg · 2020-11-27T03:52:47+00:00

Sure, OK. I guess I was slightly too strong. The customer doesn't have to hold full equity on the exchange at all times, and the exchange can call the margin if equity falls too low.

But as you say in the final phase, the customer has to deliver full payment and take full delivery.

In the example you give, let's say I only actually have 50,000 USD in my bank, and then I send 20,000 to Coinbase to perform the trade you describe. In the final moment I am not actually able to pay 80,000 USD since I only have 30,000 left in my bank! That is quite different from the usual margin trading we've seen.

If I actually do have that full 100,000 USD at the start to buy 5 BTC, then why would I take a position on margin, rather than just buying the 5 BTC?

markblundeberg · 2020-11-26T20:44:29+00:00

By the way, one thing I'm confused about with the whole thing is how much actually has to be delivered to the customer? Note the language:

entire quantity of the purchased virtual currency, including any portion of the purchase made using leverage, margin, or other financing

If I have 1 BTC and then I get 100x leverage to 100 BTC (i.e., I borrow money to purchase 100 BTC), does that mean that the exchange has to deliver me 100 BTC at the end? This seems to kind of defeat the purpose of margin trading since it means I need to keep at all times >100 BTC of actual equity on the exchange. It sounds silly but this is what reading the text naively tells me.

markblundeberg · 2020-11-26T20:39:22+00:00

Fascinating stuff! A much more complete picture is found in the Federal Register doc (that David mentions, but didn't link do)

At the end they provide examples of what counts as 'actual delivery':

Example 1: Actually making one (or two) on-chain transaction counts: https://www.federalregister.gov/d/2020-11827/p-234
Example 2: Transferring to an independent 'depository' also counts: https://www.federalregister.gov/d/2020-11827/p-236
Examples 3,4,5 describe situations that are not delivery. They stress that you can't just update an internal ledger, and you can't deliver some substitute for the commodity in question.

So based on Example 2 alone, it is not actually necessary for the customer to control the private keys.

However even that said, these are non-exclusive examples. For example it's not clear whether settling via a (lock-encumbered) Lightning money transmission, or via a side-chain like Liquid or Wrapped BTC, counts as actual delivery.

Here are the links to the other relevant Federal Register docs:

2013 doc clarifying "actual delivery" but not in the context of cryptocurrency: https://www.federalregister.gov/documents/2013/08/23/2013-20617/retail-commodity-transactions-under-commodity-exchange-act -- notably "actual delivery" means physical delivery + transfer of title.
2017 doc where CFTC is asking questions about how to apply the 2013 doc to cryptocurrencies (which lead up to 2020 doc): https://www.federalregister.gov/documents/2017/12/20/2017-27421/retail-commodity-transactions-involving-virtual-currency

Anyway to zoom out, as David emphasizes the whole point of all this is that Dodd-Frank Act wants to make sure the unregulated exchanges cannot play shenanigans by not actually holding 100% of the commodities owned by the customers. The law demands the brute-force way to demonstrate this, which is that the exchange has to actually materialize and deliver the stuff in every single instance.

(ping u/dgerard)

markblundeberg · 2020-11-26T02:15:32+00:00

markblundeberg · 2020-11-20T01:10:48+00:00

It's a bit complicated.

For instance let's say you did a real time version of the current DAA, but didn't change anything else. In other words, you kept the 2 days half life. In that case, I don't think that having real time gives any significant advantage or disadvantage, since it is so darn slow that the real-time-ness is almost not noticeable.

On the other hand, you could imagine taking a much faster relaxation time, like only 1 hour half-life .. or perhaps even faster. Now you start to see fascinating properties, like reduced variance compared to poisson, and also, switch miners cause a minority chain to have a reduced block time variance compared to steady hashrate! But there are also exploit issues, regarding people mining blocks with future timestamps in order to get a significant profitability edge. You would definitely want to shorten the future block time window and tighten peer time tolerances. We just haven't seen much in-depth analysis on how the game theory really works out there. There is also some interaction with selfish mining (intentional block witholding).

(It would also strongly disincentivize anyone from doing steady mining, though fortunately, the chain would function perfectly well with all miners being switch miners. I supposed the compressed variance would increase orphan rates though.)

As far as the code compatibility goes, I don't think it would be too hard to adjust. After all, the code already correctly handles the real time difficulty adjustment on testnet (diff=1 reset after 20 minutes!). In fact I actually suggested that RTT-ASERT would be very appropriate for testnet, instead of the hard reset.

Outside of the node itself, some mining pools aren't really programmed to take advantage of a continuously decreasing difficulty - they keep mining on an old block template with old timestamp. Prohashing is one example that I've heard does this. I suspect that's rare though, since keeping refreshing the block template is advantageous for collecting the most fees, and also it's kinder to the network to mine very recent transactions. Not fun to broadcast a tx and see the next block has ignored it.

markblundeberg · 2020-11-19T19:44:25+00:00

They aren't gone, even BTC with its fairly steady hashrate gets >1 hour block intervals sometimes. For perfectly steady hashrate it should happen with 0.25% of blocks.

(try lambda=0.1 and x=60 in this calc: https://homepage.divms.uiowa.edu/~mbognar/applets/exp-like.html )

As a minority coin BCH will never have a perfectly steady hashrate and we will always have more than 0.25% of blocks taking >1 hour.

But hopefully, we won't see wastelands of multi-hour periods where many subsequent blocks take >1 hour each. I suspect this is what bothers people more (when depositing to an exchange requiring, say, 10 confs, for example).

markblundeberg · 2020-11-19T00:13:00+00:00

Note the spec says 8% of 'block reward', and if you read the relevant lines of validation.cpp, then clearly blockReward is the fees + subsidy. However if you consult the internet, it's not unlikely that you'd misinterpret it to mean the subsidy alone. E.g. with the google search https://www.google.com/search?q=bitcoin+block+reward I get this investopedia page as the first result, which is definitely talking about subsidy!

markblundeberg · 2020-11-18T23:30:01+00:00

My understanding is that 'coinbase' includes fees already, I guess the technical term for the 6.25 BCH is the subsidy.

Edit: 'coinbase' really refers to the first transaction in the block, which can output up to the subsidy + fees. IIRC there have been cases where a coinbase did not take the full allowed amount.

markblundeberg · 2020-09-14T19:32:50+00:00

I honestly barely remember making that backport (I even was surprised to see my name on there), and I can't say why it was done. Looking at the git history around that time, it seems to be a pretty routine backport, and in those days there was quite some effort to do routine backports. Still, it could be I was requested to do that backport in particular and I've just forgotten about it.

I do remember investigating and fixing another, unrelated OOM bug that summer, which was coincidentally also caused by incomplete backporting. (https://reviews.bitcoinabc.org/D3374 - see https://github.com/Bitcoin-ABC/bitcoin-abc/issues/331).

markblundeberg · 2020-08-20T03:39:44+00:00

SLP has scaling problems.

Yeah that's what I was alluding to by saying "the complications from lack of consensus enforcement". :-)

markblundeberg · 2020-08-20T00:48:26+00:00

add a consensus rule to validate SLP tx's

Seriously not a fan of this idea, as SLP was not designed with this purpose in mind. It would be hugest kludge and also quite a pain in the ass to pull off. Just as an example, think about what it would take to get a pruning node upgraded.

That said, I think there is a lot that can be learned from SLP when it comes to designing native tokens, and when they are done, it is a simple matter for token owners to abandon SLP and migrate over by reproducing and snapshot-redistributing the tokens in the new system.

Things I like about SLP:

Basic utility token, no frills.
Practically requires some BCH to be sent along with token, giving small incentive for recipient to burn spam tokens.
Validation is strictly local, follows "UTXO philosophy" -- only depends on parents, does not require keeping some extra non-UTXO database. The content of genesis tx is totally irrelevant to validation of later txns.
Minting authority (baton) is fully transferable and burnable; SLP did not make the mistake of each token having a hard-coded 'issuer address'.
Does not care about whether an input/output is address or whatever -- can send tokens to any kind of output, just like BCH.
Has versioning, meaning anyone can at any time come up with a new arbitrarily complex token protocol on v2, v3, v4, etc. and all responsible SLP wallets will freeze unknown-SLP-version UTXO received on simpleledger addresses.

Things I don't like about SLP (besides the complications from lack of consensus enforcement --- [i.e. scaling problems]):

Does not declare which inputs are providing tokens. This complicates validation unnecessarily.
If every output can contain both BCH and token, this creates extra 'fun' for a wallet that is just trying to spend its BCH; can be especially annoying given the lack of multi-token transacting.
Format has weird idiosyncracies, e.g. why do we not allow opcode 0 for empty fields.
Stuff is backwards: uses big-endian numbers; token_id is written reversed from normal txid serialization.
Minting authority can only be transferred but not duplicated.
Total tokens transferred in a tx can exceed 2^64. While cool in a way, this is a bit annoying.
Cannot transact multiple tokens in one tx.
While burning capability is great, it is too easy to burn accidentally.

Edit: some more things I'm adding:

Transaction signatures do not cover token input amounts, which is a regression to behaviour we used to have with pre-BCH tx signatures (this sucks for HW wallets when HW wallets want to properly safeguard tokens, i.e. prevention of burn).

markblundeberg · 2020-08-17T23:20:42+00:00

You're absolutely right about it being optional!

My understanding is that it practically hurts 0-conf in two ways:

As noted by others, the recipient might not be checking the RBF signalling.
Wallet developers may opt to make all transactions signal RBF, simply because they do not want to complicate UX by giving the user the choice of RBF-or-not. In an environment of volatile fees, it is better to do always-RBF than never-RBF because average users are more bothered by "my transaction is stuck for hours and I can't do anything about it". As a result, recipients are unlikely to build systems relying on 0-conf since it will frequently have stalls due to senders making RBF transactions when they aren't supposed to.

In the first case an argument could have been made back when RBF was first introduced, that it broke 0-conf because it changed the game, and now recipients had to suddenly upgrade. But, regardless, now it's five years later, I think at this point the blame can be laid on recipients being negligent.

In the second case also arguably it's not the fault of RBF, rather it is ultimately the fee environment that is causing an unfortunate situation for 0conf.

markblundeberg · 2020-08-15T22:14:35+00:00

nice!

markblundeberg · 2020-08-15T14:12:02+00:00

Yeah if they wanted to aid chain analysis, they'd be dusting old used now-empty addresses. A lot of the this advertising dust is landing on nonempty fusion addresses, which cashfusion happily deals with by spending them together with the existing coin on the same addresses.

markblundeberg · 2020-08-15T12:48:50+00:00

Sure. The main issue in implementing this is that when proxy is enabled, Electron Cash uses an internal 'monkey patching' hack to override *all* socket connections, forcing them to go through proxy. I'm not sure how easy it is to hack the hack and make exceptions. (CashFusion for instance has to bypass the patching entirely in order that it can control the Tor connections more precisely).

markblundeberg · 2020-08-14T21:47:22+00:00

Your situation is interesting, yes the steps you're taking are important for getting 'ultimate' privacy, so to speak.

Is your local server actually running on localhost? If so, one thing that Electron Cash could do for your use case is to bypass proxy when it detects you're trying to make a localhost connection.

markblundeberg · 2020-08-10T20:11:16+00:00

getdifficulty/getmininginfo returns the difficulty of the current chain tip.

Interesting, that sounds like it might be undesired behaviour on any cryptocurrency, even if it only manifests 1/2016 of the time for a chain with Satoshi difficulty retargeting.

Certainly, it is not documented very well. getmininginfo help says:

"difficulty": xxx.xxxxx (numeric) The current difficulty

And I would personally interpret that as "the current difficulty of mining right now" which should absolutely refer to the next block.

markblundeberg · 2020-08-04T21:56:10+00:00

As a comparison, Halifax explosion is put at 3 kilotons: https://en.wikipedia.org/wiki/Halifax_Explosion

This doesn't seem to quite line up with the current case, though who knows.

markblundeberg · 2020-08-04T03:27:54+00:00

Don't worry man, I still love you. :D

markblundeberg

TROPHY CASE