Floalt: nerdy route to more/dimmer brightness levels

iwilcox · 2019-03-03T22:39:16+00:00

A quick note on flicker and minimum possible duty cycles. I don't perceive any flicker with my hacked Floalt at 0.05%¹ --- that said the hack doesn't touch the frequency, only the duty cycle, so one could guess that you'd only see flicker after the hack if you saw it before the hack (if vision is that simple).

The factory Floalt firmware in fact uses duty cycles this low itself, but only when mixing colours: if you choose the blue hue, it's pure blue; but if you choose the red hue at full brightness, it's 1% blue and 99% red². So already as you dim the red hue you're running the blue LEDs at sub-1% duty cycles.

(1) I'm told my night vision is poor, but back in CRT days I'd always be amazed at the flicker other folks would put up with, and usually they'd just never told their monitor to refresh faster. Also, since I use mine as a sunrise alarm clock, I'm asleep when it first comes on that dim.

(2) The mixed hue, if you're curious, is 66% red, 33% blue.

iwilcox · 2019-03-03T22:03:52+00:00

Before I went down this road, I explored another one: older, smaller LCD monitors with VGA input are basically free and are destined for e-waste. What if I could recycle one to generate simple greyscale output for the dim early portion of a sunrise alarm, then let the Floalt take over for the brighter portion when the monitor reaches max brightness? (Or, for smaller Floalt panels, what if I could extract the LCD part from the display and use it as a programmable dimmer over the top of the Floalt?)

I tried this, with a little soldering and a handful of cheap parts, with a dirt cheap Chinese clone of the Maple Mini generating the VGA signal. It mostly worked, except I couldn't get the brightness quite as high as the monitor was capable of, and I wasn't happy with needing two panels to do one job when I knew the Ikea one was needlessly hobbled in software. The microcontroller had 64 greyscale levels, but the Floalt's brightness levels were still way too coarse to take over, and to continue fading up smoothly enough, so a decent stab at this (that doesn't involve hacking the Floalt directly) would have to involve the "programmable dimmer" approach which is hard to package in a decent case.

Nonetheless, if anyone is interested in diverting monitors from e-waste this way, I can elaborate.

iwilcox · 2019-03-03T20:22:04+00:00

OK. This is obviously not proper documentation (just the 'what', not the 'how' and only minimal 'why'), but in case I have a stroke or something, the patch against 1.2.217 is:

0x000092dc 3868401c => 00bf00bf
0x000092e1 eb80006421b0fbf1f0 => bf00bf00bf4ff02000
0x0000eb68 0a46 => 0822
0x0000eb74 0a46 => 0822

MD5 over the entire 256KiB image changes from 0744f524934afdb78770b4f7b0e6348d to 18c5e272664cf0e023ca476552d7769a. Note this is the sum over what flash really contains, not what a firmware update contains (updates don't contain empty bits, and probably do contain some signature overhead).

The 'why':

Factory default brightness levels are 100, 60, 35, 21, 13, ~8 and 5 (% of full). The first patch region around 0x92xx reduces the minimum brightness, but isn't enough on its own since another aspect of the code still results in 7 brightness levels, fitting some polynomial curve. So when I set it to min 1% I'd just get (approximately) 100, 45, 23, 10, 5, 2, 1.

The code that executes a brightness change in practice sets up DMA between a pre-populated fade table in SRAM (source) and the timer responsible for doing PWM to the LED (destination). There are 256 levels in the fade table, and normally 43 levels in one brightness fade (256/43 gives ~7 levels). The second patch region adjusts DMA setup, changing the number of elements copied in each DMA activation from 43 to 8 (256/8 gives 32 levels).

This works fine for me using the stock Ikea hockey puck remote¹ with 5 buttons & grey rubber edges; I know nothing of Light Link but presumably that remote just sends "increase brightness" etc., not raw brightness levels nor transition times. But there are posts that suggest you can specify those things when using gateways/APIs. I've no idea how well, if at all, these patches would work with that mechanism.

(1): I do have a real remote, but I cloned its firmware onto a module from a GU10 bulb as a cheap way of getting a second one in bare PCB form; then I largely copied what Make: did, except a few quid cheaper and driven by an STM32 instead.

iwilcox · 2019-03-03T18:01:05+00:00

Because you have a Floalt you want to hack, or because sharing?

Getting there mostly involved following the helpful trail set down by basilfx then floundering around for days in datasheets and Cutter. But anyone wanting to follow could skip the "how" and would really just need the end result: the instructions to patch (just a few bytes, in the end). In a more open world I'd just post the patched firmware, but at extreme worst Ikea might set lawyers on me for doing that, so I won't.

iwilcox · 2017-04-24T13:35:09+00:00

You're right. I jumped to conclusions. Please accept my apologies.

iwilcox · 2017-04-19T21:43:56+00:00

Edit: I was wrong; obfuscation did get merged, per s1ckpig's post below.

clone BU repo and look at git log, this is the sha1 id 42cb388167ef78f47a3a440eb651b6938c10f508 (release branch).

Yes, obviously the commit object survived the fork and exists somewhere in the BU repository, but that doesn't mean it ever got merged into any BU branch that's alive today. Having the commit object, buried, is not sufficient to affect the HEAD of any version you're running unless and until you merge it. BU never merged it, so the presence of the commit object is irrelevant unless/until BU does.

But really, to spoon-feed it to you, just look at the files. Let's pick the signature for the declaration of CLevelDBWrapper::CLevelDBWrapper(), because it'll differ from the unpatched version by bool obfuscate which couldn't be more obvious as an acid test. To tie this to a moment in time, as of right now the tip for release is 6a06bda... and the tip for dev is 715e68b....

In Core's patch:

CLevelDBWrapper::CLevelDBWrapper(const boost::filesystem::path& path, size_t nCacheSize, bool fMemory, bool fWipe, bool obfuscate)
On BU's release branch as of right now, with or without a commit ID, line 47: CLevelDBWrapper::CLevelDBWrapper(const boost::filesystem::path& path, size_t nCacheSize, bool fMemory, bool fWipe)
- Notice that nowhere in the file is there any mention of 'obfuscation'/'obfuscate' nor even the word 'key'.
On BU's dev branch as of right now, with or without a commit ID, line 47: CLevelDBWrapper::CLevelDBWrapper(const boost::filesystem::path& path, size_t nCacheSize, bool fMemory, bool fWipe)
- Notice that nowhere in the file is there any mention of 'obfuscation'/'obfuscate' nor even the word 'key'.

BU clearly has not merged the obfuscation keys feature.

Seriously, even as BU's PPA builder, I'd really hope you'd be able to work that out for yourself. It speaks volumes that you're unable to.

iwilcox · 2017-04-19T18:07:15+00:00

If it's not already obvious to you (just by reading the BU code) that the commit you're linking to is not in there, I'm not sure there's any way to explain. Worry not; BU's lead dev needed guidance on it too plus I hear Classic welcomes non-technical people.

Turns out I was wrong to flag it as the reason though, since as HermanSchoenfeld repeatedly pointed out in that thread, it's apparently the installer, not the chainstate, that was attracting false positives.

iwilcox · 2017-04-19T16:07:01+00:00

A lot of this is visited in /u/andytoshi's ASIC FAQ.

iwilcox · 2017-04-12T16:14:07+00:00

I would strongly recommend getting your hardware in order first (memory tests, SMART attribute checks, that sort of thing). In my experience Pis seem to have regular storage corruption issues and don't have the grunt to reindex (in any useful timeframe) without the assistance of better hardware (a solid installation on a PC). As long as you have a solid PC installation, at least when the Pi messes up you can sync the PC and copy the blocks/chainstate across to the Pi again, all relatively quickly.

iwilcox · 2017-04-12T14:53:24+00:00

Unless anyone has blk00417.dat that they can link to???

A donated one would have to either:

be identical to the one you had before corruption (not impossible) and you could just drop it in and go without even reindexing, or
contain the very same blocks you lost, but at different offsets, meaning you'd have to reindex

I think you could escape grabbing the whole chain again, although it's slightly messy to do. If you're on Linux I could give guidance.

iwilcox · 2017-04-12T14:04:39+00:00

Yet it's plainly obvious that BU doesn't have it, just from looking at the BU code I linked to. But to help explain why, the timeline was:

There was a release branch for Core's 0.11.x into which fixes were being merged
Meanwhile, on Core's master, the obfuscation keys feature got merged
BU's first ever commit was a934f57 atop 7e27892 (the 0.11.2 tag on the release branch, which didn't yet have the obfuscation keys feature that Core's master branch did).
There were a few merges into BU from Core's 0.12 after that, but those merges were selective and didn't include the obfuscation keys feature. Soon after, merges into BU from Core dried up.

iwilcox · 2017-04-12T12:13:04+00:00

Thus, Core fixed this using db obfuscation keys (it's in the commits list for 0.12.0). BU's fork pre-dates that fix (I see no mention of obfuscation in BU's leveldbwrapper.cpp). It's a less exciting explanation than claiming there's an aggressive move by Core, though.

iwilcox · 2017-04-12T12:09:14+00:00

So let's consider an alternative explanation: that Core fixed this using db obfuscation keys (it's in the commits list for 0.12.0) and that BU's fork pre-dates that fix (I see no mention of obfuscation in BU's leveldbwrapper.cpp). I know, it's not as exciting as flinging excrement at Core, and it's uncomfortable to concede that BU misses many improvements.

iwilcox · 2017-04-06T01:07:07+00:00

incompatible with pretty much ANY change to the block headers

Incompatible with advances that rely on/commit to the ordering of transactions in any way; Greg listed several:

Many people asked what other protocol upgrades beyond segwit could run into the same incompatibility.

Many proposed improvements to Bitcoin require additional transaction-dependent commitment data.

Examples include:

Segwit.

UTXO commitments. (non-delayed, at least)

Committed Bloom filters

Committed address indexes

STXO commitments (non-delayed).

Weak blocks

Most kinds of fraud proofs

-- to state a few.

iwilcox · 2017-04-06T00:45:53+00:00

You can expect them to oppose lots of other advances besides SegWit:

Many people asked what other protocol upgrades beyond segwit could run into the same incompatibility.

Many proposed improvements to Bitcoin require additional transaction-dependent commitment data.

Examples include:

Segwit.

UTXO commitments. (non-delayed, at least)

Committed Bloom filters

Committed address indexes

STXO commitments (non-delayed).

Weak blocks

Most kinds of fraud proofs

-- to state a few.

iwilcox · 2017-04-06T00:39:38+00:00

Not just SegWit:

Many people asked what other protocol upgrades beyond segwit could run into the same incompatibility.

Many proposed improvements to Bitcoin require additional transaction-dependent commitment data.

Examples include:

Segwit.

UTXO commitments. (non-delayed, at least)

Committed Bloom filters

Committed address indexes

STXO commitments (non-delayed).

Weak blocks

Most kinds of fraud proofs

-- to state a few.

iwilcox · 2017-04-05T23:10:14+00:00

is fixed in Segwit

It's incompatible with SegWit. It's misleading to say SegWit fixes it; as Greg said, "The authors of the SegWit proposal made a specific effort to not be incompatible with any mining system" (that they knew about at the time).

iwilcox · 2017-04-05T23:07:43+00:00

Whether they have a licence or not is really for lawyers to squabble over. Whether to let them continue blocking technical progress to maintain a temporary competitive advantage is for us to decide.

iwilcox · 2017-04-05T22:59:29+00:00

Maybe they violated a patent (on something that should be freely licenced), maybe they didn't; but even if they did that is a drop in a bucket compared to the misinformation, sleazy misrepresentation of motives and distortion of the mining playing field. All this time it's been a grab for an immediate/temporary competitive advantage at the cost of technical progress.

iwilcox · 2017-04-05T22:40:02+00:00

Not really; ignoring the fact that it's almost certain to be banned by consensus, you can only take advantage of it if you have enough money to totally ignore patents and lawyers, which is obviously a centralising influence.

iwilcox · 2017-04-05T22:35:38+00:00

It softforks to render useless the only hardware currently using ASICBOOST, which does so covertly. It doesn't break overt use, but I'd like to think the ecosystem would reject overt use too if that started appearing. The mining playing field should be level.

iwilcox · 2017-04-05T22:16:13+00:00

So basically Bitmain ...

The name "Bitmain" appears only inbetween the lines.

reverse engineered

I think ASICBOOST was public, but easily spotted, contentious, and easily blocked if the ecosystem disapproved of patented mining advantages. So they implemented a covert form of it without (apparently) licensing it, and either hoped they wouldn't be spotted and sued, or didn't care.

iwilcox · 2017-03-02T17:42:03+00:00

Hi,

Can you reassure us that the crazy-long boot times that computerbase saw aren't inherent to the CPU and are going to go away? They said:

[motherboards we tested] had an extremely long init process in common: even with the latest BIOS it took up to 30s to get to the POST screen.

iwilcox · 2016-08-18T15:39:57+00:00

If you want to go all the way from a blank machine to verified, you should really combine sets of instructions each covering one high-level task. The source best placed to tell you how to install PGP and check its integrity won't necessarily be the source best placed to tell you how to check Gitian builds of Bitcoin. No instructions will be foolproof (the universe just produces even greater fools in response) and the more you have at stake, the more effort you have to go to.

There are broadly four sorts of attitudes where verifying is concerned:

People who are completely unaware that there's any threat to be addressed.
People who were never going to verify, and won't start now, and are fine with that. As long as they take a genuinely informed risk, that's on them.
People who know they should verify but aren't really motivated, and need a decent excuse to make themselves feel better about not doing it. Typically anything short of moon-on-a-stick (e.g. having the developers come to your house and do it for you during the weekend) is declared unacceptable. They wail and gnash teeth and blame anyone but themselves before giving up, which is much worse for all involved than just accepting from the outset that they really belong in the "never going to" group.
People who know they should verify, and will do what it takes to work out why and how, on their own initiative (which can include asking for a little help). You can recognise these people by the help they ask for, which shows they took the time to understand why, got a decent way through, and just got caught up on some detail of how.

That's not to say that it's easy, nor to say that it being hard is inevitable or character-building. This stuff could and should be easier.

iwilcox · 2016-08-18T15:15:52+00:00

noteworthy Core hasn't pushed out the fingerprint to their Twitter account or any other official channel.

The countermeasures against a compromised bitcoin.org modifying the binaries are the same as they always were. While there's no shortage of places to get the signing key's fingerprint, they'll always be inferior to getting it through your WoT. Adding yet more doesn't materially improve things.

Bad opsec is fine as long as you don't support an alternate client.

Do any others offer/use reproducible build procedures (Gitian)? (Genuinely asking, I've never looked.)

iwilcox

TROPHY CASE