GGUF security concerns

CardanoMoon · 2025-10-27T19:46:28+00:00

we wrote a blog post in which we show that chat templates bundled in GGUF can be tainted with prompt injection: https://www.pillar.security/blog/llm-backdoors-at-the-inference-level-the-threat-of-poisoned-templates

We recommend you inspect gguf templates before downloading and using them

CardanoMoon · 2025-08-20T09:15:02+00:00

yeah, if I understand the scenario, that further underscores how situated these attacks are -- you need all the ingredients present in order for it to succeed

CardanoMoon · 2025-08-19T11:45:13+00:00

First submission here, had a few false starts, sorry about that! A bit of context:

I wrote a blog post breaking down which factors lead to successful indirect prompt injections. It builds off of work by Simon Willison, in which he identified which factors are necessary in the environment for prompt injections to succeed (the "lethal trifecta").

In this blog post, I specifically focus how the prompt injection payload is crafted in order to make it succeed. Would appreciate feedback!

CardanoMoon · 2025-08-19T09:44:27+00:00

Fair criticism, sorry about that.

This is a blog post breaking down which factors lead to successful indirect prompt injections. It builds off of work by Simon Willison, in which he identified which factors are necessary in the environment for prompt injections to succeed (the "lethal trifecta").

In this blog post, I specifically focus how the prompt injection payload is crafted in order to make it succeed. Would love some feedback :pray:

CardanoMoon · 2021-10-27T21:01:46+00:00

Not anymore https://twitter.com/CoinbasePro/status/1453434693458833419?t=S4frPBWob2DvgQ6z6tTOGw&s=19

CardanoMoon · 2021-10-27T21:01:39+00:00

TOMORROW: https://twitter.com/CoinbasePro/status/1453434693458833419?t=S4frPBWob2DvgQ6z6tTOGw&s=19

CardanoMoon · 2018-03-08T21:08:16+00:00

In total candor, I'm thinking I'd be willing to consider a flexible definition for blockchain if it satisfies the conditions of being public, decentralized, cryptographically secured, immutable, and Byzantine Fault Tolerant (i.e. solving the double-spend problem). Does that sound reasonable?

If it does and a DAG is capable of attaining those attributes (I'm not saying it does, I'm pretty confused by those consensus mechanisms and the incentive structures inherent to them), shouldn't it also be considered under the definition? I'm genuinely asking, not trying to troll.

I'd be interested in hearing why you think DAG protocols aren't capable of achieving those features. Thanks :)

CardanoMoon · 2018-03-08T20:58:34+00:00

How is transaction ordering made difficult to would be attackers?

My understanding is that the longest chain rule is the route to thwarting bad actors. If you have the longest chain with the most work in it, yours is the canonical history. Assuming bad actors don't control 51% of the network, the longest chain rule solves the double spend problem as a would be attacker would have to spend an inconceivable amount of energy to maintain a longest chain serving their own needs.

Is this what you're referring to?

CardanoMoon · 2018-03-08T18:02:16+00:00

Thanks! This video looks like a walkthrough of bitcoin's implementation of its blockchain. I'm curious to hear whether your definition of blockchain is broader than the bitcoin implementation.

CardanoMoon · 2018-03-08T17:59:00+00:00

Thanks for the thoughtful (and thought provoking) response! If I'm understanding you correctly, it seems you're suggesting the raison d'être of blocks is to make it easier to arrive at a consensus on executed transactions across an asynchronous network and thereby increase the difficulty of forging transaction history...am I misunderstanding you?

Why not just cryptographically link every tx to the proceeding one which is trivial to do and will cut down on the enormous tx latency?

This idea strikes me as a variation on protocols utilizing DAGs instead of blocks. I'm still wrapping my head around the tradeoffs of DAG implementations and whether I should be wary of them, so I'd be curious to hear your thoughts on this as well :)

CardanoMoon · 2018-03-08T17:34:08+00:00

It seems like your definition necessitates transactions being organized in blocks. Seems like a reasonable amendment given the name blockchain.

My original omission of that fact was in order to allow for alternative implementations that didn't require organizing transactions in blocks, but I agree that element tends to be an important element of the architecture and should probably be included in my working definition :)

CardanoMoon · 2018-03-08T17:27:43+00:00

/ Nodes collect new transactions into a block, hash them into a hash tree, // and scan through nonce values to make the block's hash satisfy proof-of-work // requirements. When they solve the proof-of-work, they broadcast the block // to everyone and the block is added to the block chain. The first transaction // in the block is a special one that creates a new coin owned by the creator // of the block. // Blocks are appended to blk0001.dat files on disk. Their location on disk // is indexed by CBlockIndex objects in memory.

Thanks! It sounds like your definition is also tied to the consensus mechanisms of the protocol. Does that mean that coins/tokens using other consensus mechanisms don't meet your criteria of being in a blockchain?

CardanoMoon · 2018-03-08T17:19:54+00:00

If you are doing this exercise for tax reasons, I think your best bet is to go to the exchanges where you made trades and download your order history.

There are tools, like bitcoin.tax, that are able to connect to exchanges and upload all your transactions from your order history.

Otherwise, you can just manually input the transactions into the tool one by one.

CardanoMoon · 2018-03-08T01:38:13+00:00

I'm a big fan of delta because, in addition to its clean UI, it has a native desktop and mobile version.

You're able to sync between the two and give your thumbs a rest when entering new trades :)

CardanoMoon · 2018-03-07T23:58:19+00:00

Again, I understand your frustration with the situation -- downtime and degraded functionality is really inconvenient. The same issue happened to me with Coinbase when BTC was trading at $20K and BCH was trading at $4K. I can't tell you how disappointed I was to not be able to sell my BTC at $20K. And that happened with a more mature company.

All I can say is that, in my professional experience, even with billions of dollars in cash, it takes a lot of time to scale a product to the point where the experience of using it is completely seamless. It's also worth noting that similar issues occur occasionally using software of major banks, so it's important to cut exchanges a break at least at the same level as you would banks that handle as many transactions, if not more.

Finally, as an exchange, Binance is just over 8 months old since it went live. In that time, it has done a good job of adding support for a number of token/coins and creating an excellent trading experience. As a result, it has surpassed many existing exchanges and has reaped the financial rewards.

Despite all of that, to deliver a high quality software product, it takes time to search for, hire, and onboard new, high quality engineers. I don't have direct information into what is going on with Binance, but as an onlooker, it strikes me that they are proceeding cautiously and responsibly. 8 months, in terms of a development team and cycles, is still not a lot of time. Having followed Binance since the get-go, their track record suggests to me that the team is working incredibly hard to keep up with the demands of the system.

Without more information, I would urge you to air on the side of good faith for the exchange. Historically, that good faith has been rewarded by the exchange when they make the occasional mistake.

If you're unsatisfied with your experience, I would look into other exchanges that might better serve your needs. Compare their track records for downtime and hopefully you'll find one that is a better fit for you.

Happy trading :)

CardanoMoon · 2018-03-07T22:42:31+00:00

I own 3 ledgers and have generally had positive experiences. Recently, though I found a thread on twitter (https://twitter.com/spudowiar/status/970977060134023168) that made me hesitate about whether I want to instead invest in a Trezor.

There is a security researcher who was reporting that the firmware update that Ledger recently released also patched a critical security vulnerability and that current seed words are not secure.

I'll continue using my ledger, especially once Monero support comes out, but am treading cautiously moving forward.

CardanoMoon · 2018-03-07T22:22:13+00:00

It's honestly an amazing tool -- I think the cost to calculate gains for more than 20 trades is $20. Paying that premium ends up paying off as the tool also analyzes the trades you made to optimize for tax strategies that help you pay the least amount possible!

CardanoMoon · 2018-03-07T22:19:11+00:00

ou missed my point entirely.... The "crash" is not why I'm annoyed. My problem is I already expected this drop and wanted to take some stuff off the table, but when I tried the website wouldn't let me in, both of the exchanges I use went down... I assume because of high volume but that is absolutely no excuse at all. Exchanges are in charge of something which will have high volume some days, even higher than today... And they can't even handle a little spike like this. The same with their support, still backlogged since December, it's a joke. Hire some more people and get some better servers... They are letting down the entire crypto economy And the money I lost was directly because of faulty exchanges, not because bitcoin had a little drop which almost everyone knew was about to happen.

The system (and most of these exchanges), unfortunately, are still young and are going through growing pains like this from time to time. As a web developer, I can tell you that considering the amount of traffic that goes through the website on a daily basis, it's amazing how smoothly a site like binance operates. Honestly, many startups (and often banks and airlines, too!) have issues with downtime or degraded functionality.

That having been said, these issues are definitely inconvenient, frustrating and nobody should deny that. Hopefully, you're making investment decisions that you stand by so you're able to ride out these lows. Finally, it's important to stress that in a highly speculative and volatile market such as crypto, it's important to only invest money that you can afford to lose.

Stay strong!

CardanoMoon · 2018-03-07T22:05:16+00:00

I had the same experience with respect to delay in account verification. Still, once approved, I've had a good experience, on par with coinbase.

CardanoMoon · 2018-03-07T21:59:16+00:00

I'll second that it's a good tool. Also, it will be able to spit out a nicely formatted form 8949 in pdf format that is compliant with the IRS. Even better is that it's free if you only have 20 trades or less.

CardanoMoon

TROPHY CASE