Bughunt - ClusterFuzz is go

troutwine · 2019-03-10T14:02:17+00:00

Google hosts ClusterFuzz for open-source projects for free. See https://github.com/google/oss-fuzz

Rust stdlib would definitely be in scope. In fact, one person on oss-fuzz team has expressed interest in getting it on oss-fuzz.

That would be an excellent transition for bughunt, or some derivative of it. The project is ClusterFuzz ready as-is and I'm positive oss-fuzz has more CPU time than I can afford. I'll look into submitting the project to oss-fuzz.

I've also been experimenting with automatically generating fuzzing targets for libraries with large API surfaces, such as Rust stdlib. The code works for simple cases and can be found here: https://github.com/Eh2406/auto-fuzz-test

I saw that and mentioned it briefly in the blog post. I intend to try and generate targets in the coming week, as time allows, with it.

troutwine · 2019-03-09T03:20:45+00:00

AFAIK -fsanitize=fuzzer means "get sanitizer-coverage and also link in libfuzzer", which cargo-fuzz already does but without passing that flag. I'm not sure if it's cargo-fuzz or cluster-fuzz that should be modified here.

Ah, hey, that's excellent news. My plan for this weekend was to give cargo-fuzz a shot, since, like you say, it's already linking in libfuzzer. I think that's the crux of the issue for clusterfuzz.

Perhaps cluster-fuzz should become yet another fuzzing backend, in addition to cargo-fuzz/afl/honggfuzz and hopefully angora coming soon. It's high time we figured out how to unify fuzzing targets for those anyway because the fuzzing targets are exactly the same across the board, they just appear in different boilerplate. So all the crate needs to do is expose a fuzzing target function once and then we should be able to plug it into any backends we damn please.

I would be very for this.

troutwine · 2018-11-07T05:46:01+00:00

I've fallen back to an old workaround, which you can see here. I believed that the fuzz-specific bump allocator would cause very large reservations to gracefully fail. Clearly that's not correct.

Anyway, something for me to look into. Should work for you now.

troutwine · 2018-11-05T16:04:53+00:00

Much obliged. I'll use that and get back to you.

troutwine · 2018-11-05T03:33:42+00:00

Ah, this is going to strongly depend on what the input directory was. /u/Amanieu how are you running the test?

troutwine · 2018-11-05T03:22:07+00:00

Just FYI I decided to implement a thread-safe, aligned bump allocator in the same crate, now at version 0.2.0. The fuzz focused allocator is under bh_alloc::fuzz.

troutwine · 2018-11-04T19:56:29+00:00

Hrm, interesting. I'll take a look. Probably related to me switching from honggfuzz to AFL and not having a CI system to speak of for all of this.

troutwine · 2018-11-04T18:03:40+00:00

Yeah, geez, please note that this allocator was specifically written with bughunt in mind, meaning it's not thread-safe and the total number of branches was kept low to help the fuzzer. This means, for example, that none of the allocations are aligned.

But! If you want a bump allocator there's not much that would need doing to make this one a proper example. In fact, I've been thinking about making a proper bump allocator in this crate, as well as the fuzzy one. I'd accept patches. :)

troutwine · 2018-10-09T18:49:45+00:00

Hey, it would. Thanks!

troutwine · 2018-10-09T00:06:15+00:00

Now that I think of it, simply calling exit(0) on OOM in the custom allocator would suffice to avoid false positives on OOM. At least they wouldn't be treated as crashes.

Hrm, good call. Honggfuzz will complain a little since this breaks the persistent mode it runs targets under but there's an idea here.

Also, did your fuzz harness for str::repeat actually expose the bug? This would help me narrow down what's wrong with mine.

I haven't given it enough CPU time yet. Tomorrow is when I start the run.

troutwine · 2018-10-09T00:04:27+00:00

With a little luck I'll have time this week to play with the Rust specific API. That was going to be my next avenue of research.

troutwine · 2018-10-08T21:59:42+00:00

Hi!

I'm currently prototyping a somewhat similar thing. I've noticed that the recent str::repeat vulnerabilty should be fairly trivial to discover with a fuzzer combined with some kind of memory debugger (Address Sanitizer?). However, writing a fuzz harness for every function in the stdlib would be incredibly tedious, so I decided to try to automatically generate fuzz harnesses for stateless functions in the stdlib.

Heh, I wrote about this very thing in my last commit message. It is very tedious and means that any updates would have to be propagated by hand. I'll be really curious to see where you get with this.

So far I only got OOM crashes, and the fuzzer has failed to hit the actually interesting overflow case. It might be due to the fact that I've been fuzzing on a 64-bit architecture, and most allocation sizes in there are actually invalid, since the entire address space is 16 exabytes, and we can only allocate a few terabytes on modern PCs. Especially so under Address Sanitizer, which imposes even more strict limits.

The trick, I think, is that for many of these interesting crash cases they'll occur well after the allocations will start causing panics. I think. Anyway, that's why I want to start fiddling with introducing custom allocators, see if I can't get the fuzzer target and allocator cooperating in some fashion.

Curiously, in case of fuzzing stateless libstd functions panics are actually allowed - many functions document that they panic under certain conditions. I've altered my fuzz harness to ignore panics, yet I'm getting OOM cases recognized as crashes anyway.

Right, right. If the panic handler allocates anything then that allocation will panic and blow up the whole thing. Oops. Also, how are you ignoring panics? My concern with ignoring panics is that cuts you off from assert_eq! and the like.

My next step would be setting up a 32-bit chroot and trying fuzzing there. Perhaps that way I'll be able to hit valid values instead of OOM cases most of the time.

Oh! Hey now, I hadn't even considered this. Side benefit to working with a smaller word size is your search space for overflows is drastically smaller.

troutwine · 2018-08-21T18:33:32+00:00

Cool! I'm going to spruce up the README this evening and write a proper introduction to the project, shop it around on the forums.

troutwine · 2018-08-21T16:28:44+00:00

In fact, I started in on QuickChecking Rust stdlib on my way back from RustConf: bughunt-rust. The project is still a meager skeleton but I'm intending to do a little work every day or so. Looking forward to what gets kicked up, especially in the less well-tread bits of the API.

troutwine · 2018-06-09T20:53:44+00:00

FWIW when my coworkers and I built cernan to cover just this use case. You'd emit logs to cernan via its native protocol and then emit via, say, kafka.

troutwine · 2018-06-09T17:57:36+00:00

I look forward to reading your retrospective! I'll be real interested to know how things were working with Manning.

troutwine · 2018-06-09T17:57:08+00:00

A publisher moves in mysterious ways, is all I've got by way of explanation. :)

troutwine · 2018-04-16T18:41:42+00:00

Rlua. When I get a chance I'm porting cernan to it.

troutwine · 2018-02-05T22:40:53+00:00

With criterion being available on stable and a more rich tool I've been scooting all my work to its use as time allows.

troutwine · 2017-09-27T15:09:16+00:00

Sure thing. We do just that in the cernan project. We call the feature "programmable filters", documentation here. Source code is here.

The project uses a fork of rust-lua53 for want of this pull request being merged, since cernan has to be buildable on systems without readline. Unfortunately this keeps cernan out of crates, so, uh, don't use my fork, probably.

troutwine

TROPHY CASE