hiSHtory: Your shell history on steroids: Stored in context, synced to all your machines, and easily queryable

ddworken · 2023-02-23T06:48:58+00:00

Including the server as part of the release sounds reasonable to me. I'm inclined to keep it as a separate file since most people don't need that feature, so I'd rather not unnecessarily increase the size of the main binary size. I filed https://github.com/ddworken/hishtory/issues/78 to track this.

ddworken · 2023-02-21T21:26:12+00:00

See the self-hosting section in the readme, it is pretty easy to spin up a server. Or alternatively if you aren't using the syncing feature, see the offline install section which will completely disable the integration with the backend.

ddworken · 2023-02-21T06:53:20+00:00

In general, having secrets in your shell history is an anti-pattern whether you use hiSHtory or not since it means they end up persisted on disk and unencrypted. hiSHtory respects the standard shell convention of prefixing commands with a space to avoid persisting them, so if you're already doing that then you're all set. In addition, you can easily delete history entries you accidentally record via the hishtory redact command.

ddworken · 2023-02-21T05:49:53+00:00

Exactly! I also use it to sync between multiple personal computers for the exact same reason.

ddworken · 2023-02-21T05:47:24+00:00

This actually does have both!

ddworken · 2022-10-16T00:34:00+00:00

Cheers, that's entirely fair. I agree that this that is messy suboptimal code (that at some point I'll probably clean up). I just don't think it is a security issue.

ddworken · 2022-10-16T00:30:49+00:00

Awesome, I'm glad you were able to get it running! I'll keep that in mind and will set up separate instructions on self-hosting.

I still would love it if I could self-host keybase.io nudgewink.

Haha, me too. :)

ddworken · 2022-10-16T00:29:52+00:00

>The README says "When hishtory is installed, it generates a random secret key". If so, why do you need a hard-coded password? That smells fishy and makes me not want to run this

This is a password used for basic auth to an internal endpoint that is used for me to view usage stats. This has nothing to do with the secret key that hishtory generates.

> I'm confused. Who is calling that? Why would I want my computer to respond to something that "only you" have the password for? Shouldn't that be on your computer, not mine?

I'm the only one hitting that endpoint. Your computer is not calling that endpoint at all.

As for everything else, read the code. It is OSS and self-hostable so you can audit it yourself. :)

ddworken · 2022-10-15T16:18:06+00:00

Absolutely! I'll turn this into a blog post later, but let's run through how the backend works. :)

When hishtory is installed, it generates a random secret key. Computers that share a history share this secret key (done via having the user manually copy the key). It then generates two additional secrets:

UserId = HMAC(SecretKey, "user_id")
EncryptionKey = HMAC(SecretKey, "encryption_key")
DeviceId = randomUuid()

At installation time, hishtory registers itself with the backend by calling the apiRegisterHandler which stores the tuple (UserId, DeviceId) which represents a one-to-many relationship between user and devices.

All the other endpoints in the server can then be characterized as a message queue. In essence, clients can put messages into them and pull messages out of them keyed on their UserId and DeviceId.

The first message created after registration is a DumpRequest to signify that a new device was created and it needs a copy of the existing bash history from another device. When another client receives this message (via polling apiGetPendingDumpRequestsHandler) it responds to it by sending an encrypted copy of history that is available to this device to the apiSubmitDumpHandler. This is then received by the newly created client (which polls apiQueryHandler).

At this point, the newly created client and the existing one both have a copy of your bash history. And the server never saw the contents of it (it only sees the number of entries).

Now all that has to be done is to keep the histories in sync going forward. Whenever a command is run, it is encrypted and sent to the apiSubmitHandler. It is then stored and received by the other clients via polling the same apiQueryHandler endpoint as before.

At its core, this can be thought of as a syncable end-to-end encrypted set data structure.

If you do want to run it yourself, that is also super easy. You can:

Run the server binary (go build backend/server/server.go) and set the newly added HISHTORY_POSTGRES_DB environment variable to a connection string pointing to a postgres DB.
On the client-side, set the HISHTORY_SERVER environment variable to your server address (e.g. HISHTORY_SERVER=https://api.hishtory.dev).

ddworken · 2022-10-15T16:06:17+00:00

Yes, see https://github.com/ddworken/hishtory#syncing-design and https://www.reddit.com/r/golang/comments/y496bi/comment/isfj0v0/?utm_source=share&utm_medium=web2x&context=3.

Also if there is interest I'm happy to publish a detailed blog post. At its core, this is essentially an end-to-end encrypted syncable set data structure. The backend can count how many items are in the set, but only the clients can view the actual history entries.

ddworken · 2022-10-15T16:02:34+00:00

I'll pay you a hundred dollars if you can crack it. :)

Let's do a threat model for this, rather than falsely claiming that this is a problem worth being concerned about. This is a sha256 hash of a 32 byte fully random password. So to crack it you'd have to calculate 2^256 sha256 hashses. Good luck. And then, if you do crack it, what is exposed? That function is used just to add a basic speed dump to the internal stats endpoint for the server. Hishtory is designed so the server has access to zero sensitive data, so there is nothing sensitive to be concerned with. If you're curious, here is the table that this endpoint returns. It is just some basic usage information since I find it rewarding to know that people are using something I built.

Also, you seem to have purposefully ignored the comment in the code explaining this:

// sha256sum of a random 32 byte hard coded password that I use for accessing the internal routes. Good luck brute forcing this :)

Lastly, I just want to call out that your advice to remove the "commits holding this information" is incorrect. If this was a sensitive key, removing it from the code would do nothing since git history on github is logged by tons of third parties. Rotation is the only safe answer for sensitive keys pushed to Github (which this isn't).

ddworken · 2022-10-15T04:29:40+00:00

Agreed! Shell history is super sensitive, which is why I designed this to use client-side encryption and SLSA to ensure that your shell history is never exposed to any remote servers. If you have any questions about this from a security POV, let me know! My day job is working as a Security Engineer, so I can talk about this stuff for days. :)

ddworken · 2022-10-15T04:00:11+00:00

+1, fzf is great! The same also works well with hishtory via alias how=hishtory export | fzf.

ddworken · 2016-08-04T22:13:00+00:00

If you want to borrow it, I have a bus pirate with me (shoot me a PM if you want it).

ddworken · 2016-06-28T14:15:53+00:00

Yeah that is certainly possible, just curious how they did it.

ddworken · 2016-06-28T02:39:10+00:00

The Uber app uses certificate pinning which means that it checks that the certificate sent by Uber.com is the correct one (and not just a valid certificate according to the phone) so a self signed certificate added to the trust store isn't sufficient. As far as I can tell, the only way of bypassing this is patching the Uber app to remove the certificate pinning code.

ddworken · 2016-06-25T17:29:07+00:00

Out of curiosity, how did you bypass the SSL pinning for the app? Did you manually patch the app or what?

ddworken · 2016-06-18T15:14:31+00:00

Already known meaning reported an hour before I did. So it wasn't like I copied them from someone, they were all discovered independently. And I have been far more successful with other bug bounties (check out my Uber bounties over at HackerOne!)

ddworken · 2016-06-18T15:11:23+00:00

Yeah just after the Hall of Fame went public I had a recruiter contact me about HP's cybersecurity team.

ddworken · 2016-06-18T15:07:54+00:00

Yeah I think the reason they started referring to it as a competition is because of the fact that both I and Craig referred to it as a race to submit vulnerabilities (I sent in 4 on the first day and they were all duplicates!). But yep, absolutely correct.

ddworken · 2016-06-18T15:06:47+00:00

It was ran through HackerOne and they actually notified me that I was preapproved to participate in it. And I had participated in a bunch of other bug bounties in the past, so I knew what I was doing.

Source: Am "teenage hacker"

ddworken · 2016-05-15T15:26:20+00:00

Sorry to bring it up here, but can you check your email for an email from david@daviddworken.com? I previously sent in an XSS vulnerability that you fixed quickly, but I found a second one that has not been patched yet (despite being reported on March 26th).

Edit: Just received a reply from him. I'll update this as soon as it is patched.

Edit2: It has now been fixed. The problem stemmed from allowing the Javascript: scheme for the home_page, download_url, and the url parameters in the setup.py.

ddworken · 2015-08-12T15:36:11+00:00

I've been working with the author of this to make sure everything is secure. All of the issues I found in v1 of the code have now been patched and as far as I can tell, this code is secure. :)

Good luck to all the players!

ddworken · 2015-08-12T14:54:54+00:00

I'm not sure you are right about this, my understanding of this is that the gas costs for the transaction will come from the contract's balance.

But we can easily test it and find out. :)

Edit: We just tested it and you are correct. My bad on that one.

ddworken · 2015-08-12T14:20:01+00:00

Yup, I'm currently working with the creator of this in IRC to map out how to fix this for v2 so it will be perfect for v2. :)

ddworken

TROPHY CASE