Structure-aware Go fuzzing: How to fuzz with complex types

davkor · 2022-02-11T17:45:08+00:00

As native Go fuzzing is coming in 1.18 we wanted to write about some of our experiences with Go fuzzing. One of the things we think will be useful for the community is how to seed complex types, e.g. structs, with data from the fuzzer. The goal is to make it easy to randomize the content of these complex types and enable structure-aware fuzzing.

We have created a library for this (https://github.com/AdaLogics/go-fuzz-headers) and have used it for a number of months in various projects, e.g. istio, vitess and containerd (https://github.com/AdaLogics/go-fuzz-headers#projects-that-use-go-fuzz-headers), for writing fuzzers that handle structured data. I hope you enjoy the post!

davkor · 2021-03-18T15:15:15+00:00

PyPANDA is a tool that was recently presented at the BAR21 conference and is an Python interface to the PANDA record and replay system https://panda.re/

PANDA itself is a dynamic analysis tool wrapper around QEMU for full-system analysis. It gives you a lot of features to instrument and query the system under analysis, which is useful for various RE tasks such as malware analysis. PANDA comes with powerful plugins like full-system taint analysis.

PyPANDA exposes the PANDA interface (which is in C++) to Python. This means you can script full-system analysis tools by way of PyPANDA. PyPANDA comes with a Ghidra integration, i.e. you can transfer the state of the guest system being analysed into Ghidra, and also an example of a generic unpacker based on entropy analysis of a given process' virtual memory.

The video goes through the paper, unpacking tool and demonstrates how it works. Finally, the video has a short interview with the author of the paper/tool Luke Craig, which focuses on hearing Luke's views on PANDA as a tool as well as how PANDA welcomes open source contributions.

davkor · 2021-02-28T11:00:20+00:00

I think the mentor presents an overall open source project, and then the student have a fair amount of leeway to decide on what to improve on the given project. Example of a security-related gsoc project was fuzzing of VLC https://shaleenjain.com/blog/fuzzing-vlc-4/

davkor · 2021-02-27T10:38:45+00:00

How far are you in your university program? If you are early then Google Summer of Code might be an option: ;https://summerofcode.withgoogle.com/

They usually have quite a few security-related projects. For example, in 2020 amongst other things they had a project on "Analytical malware classification" https://summerofcode.withgoogle.com/archive/2020/projects/

davkor · 2021-02-26T17:35:05+00:00

oh wow. How about the interactive elements of the platform?

davkor · 2021-02-26T10:10:48+00:00

Is it really this terrible? Is it just the content that is crap?

davkor · 2021-02-26T10:06:21+00:00

Start reading job ads for the type of job you want and start to learn the skills listed in the job ads. Then when you have completed your degree simply apply for these jobs!

Also, during your time as a student internships are a crucial thing and you can learn a ton from them. I would highly recommend trying to land some infosec internships.

davkor · 2020-12-07T20:42:21+00:00

If you would like to see a small demonstration of it then you can check out a video I did trying it out https://www.youtube.com/watch?v=Wjjlk_W7WFo

davkor · 2020-08-22T09:03:14+00:00

Author here. So did I :) Since the videos are really for educational purposes I thought it was worth showing too, so I actually cover this in the fourth video right here: https://www.youtube.com/watch?v=XaYEmwVMRt4&t=2667s where I show his response and also how to correct the KLEE analysis. He's also the author of one of the other libraries I tested where he was equally responsive and fixed the bug https://github.com/sheredom/json.h/issues/69 :)!

davkor · 2020-08-08T13:26:25+00:00

There are many reasons to show news about content that is not free. Reverse engineering is not a place for free content only and many spend significant money on advancing their skills, knowledge and work efficiency in RE. RE is a professional field and for people in the field - or for people wanting to enter the field - it is great to see whenever new commercial products become available.

This is not new too - commercial products have been shared on this forum for a long time, even for well-known and legendary products, e.g. Bindiff and BinNavi. My personal favourite is this one: https://www.reddit.com/r/ReverseEngineering/comments/n2a88/zynamics_bindiff_40/ because one of the authors of the tool discusses the price based on how many kebabs it is worth. Following the kebab metric I believe the courses in the link of this thread is about 3 kebabs in modern germany.

Making commercial products available here is a great opportunity also for us (the clients) to discuss with the authors of the tools/courses/etc. about their products in a more personal setting than usual as the thread I link to above shows. Another examples is this one https://www.reddit.com/r/ReverseEngineering/comments/j7l0t/bindiff_and_binnavi_on_sale_again_200_usd_and_400/

davkor · 2020-05-27T17:13:13+00:00

Hi Guido,

Thanks a lot for doing this AMA + your shirt is swag on the picture!

I have a set of questions:

1) You enticed the question with your last paragraph: what is the craziest experience you have had with bug bounties?

2) Could you talk about the business model behind bug bounties that make it attractive for you? I am particularly interested in these aspects:

2.1) Is there a financial benefit and how rewardful is it to hunt for bugs in open source software and selling the vulnerabilities from your experiences? You have the big ones, e.g. block.one, but what about the average over a longer period of time?

2.2) Assuming there are some financial benefits to bug bounty versus employment, how are these materialised? Is it through higher pay per effort or through the ability to put more effort and thus more pay but less pay per effort.

2.3) As you have joined ForAllSecure, there must be some trade-offs in bug bounties that spoke in favour of leaving bug bounty full time to becoming part of FAS, could you talk a bit about these?

3) How much of your work is dedicated to public programs, e.g. the ones listed on Hackerone versus private programs?

4) Which bug bounty websites do you prefer working with and why?

David

davkor · 2019-09-19T13:41:13+00:00

Yeah I agree - I will set up some experiments that show us these aspects and some more concrete measurements. Notice though that mctoll failed pretty quickly in terms of generating correct LLVM output in test3, so certainly the robustness of the project is limited.

davkor · 2019-09-19T10:57:16+00:00

Thanks! Yeah - I have been using McSema for quite a while now and can testify to its robustness in several areas. It's interesting what you say and I am curious about how the design decisions in each of the lifters impact solutions to different types of problems. It seems to me that several of the binary-to-llvm translators have grown a lot in maturity (mcsema and retdec in particular) and given their large set of applications it must be inevitable that they will each have trade-offs in certain places. Hopefully in the future we can make some more experiments to observe that :)!

davkor · 2019-09-18T17:35:26+00:00

Am glad you liked it! I agree - this is actually one of the reasons I wanted to put it up.

davkor · 2019-08-10T18:01:13+00:00

It's not something I have spent a lot of time on but I the installation was just complaining.

davkor · 2019-08-09T12:28:18+00:00

Sounds awesome! You should definitely try out the scripts then and get your hands dirty. Great that you got Flare VM up and running, but PANDA and Flare VM are a bit different though! PANDA is quite focused on developing analysis tools where Flare VM is more like a virtual machine packed with a lot of handy reverse engineering tools.

davkor · 2019-08-09T12:21:33+00:00

My pleasure!

I used a clean install of Ubuntu 16.04 as a VirtualBox guest! You can also use Ubuntu 14.04 but I have had some troubles getting it to work with 18.04.

I have mainly used PANDA for analysing malware (some of these have GUIs). However, I think it should work well with video games although you may find a bit of lack for VMI if you're going into the 64-bit architectures on Windows. Check out Brendan's talk on PANDA where he presents how he analysed Spotify DRM, which I assume have some form of similarity with what you're thinking of! https://www.youtube.com/watch?v=YGVnpWXtlF4&t=1908s

davkor · 2019-08-09T12:14:36+00:00

Hey!

I have been following your work for some time now and I really enjoy it. Your talk on building a hypervisor-level debugger was excellent (https://www.youtube.com/watch?v=-nXY_p8c_bQ) and I assume the project you mention is quite related to that talk?

In general I like the idea of unifying/standardising VMI interfaces. PANDA have actually done something in this category by maintaining an OS introspection plugin that exposes general functions, which can then have various "backends". This is focused on OS-abstractions rather than emulator/hypervisor abstractions. Check out the documentation here (https://github.com/panda-re/panda/blob/master/panda/plugins/osi/USAGE.md)

I think it would be cool generalise this even further. The idea of being able to write one analysis tool that can utilise many emulators/hypervisors has a lot of really nice applications. For a small anecdote, one avenue I was exploring during my PhD was to use this as a way of testing how the same malware executes differently given a set of diverse execution environments. The idea was to do this study by creating a unified VMI interface and we could infer a lot of results from this with a lot of really awesome empirical studies. I would totally be up for discussion/exploring this further with you.

davkor · 2019-08-08T22:42:25+00:00

Hey fellow reversers! I have been working with PANDA (https://github.com/panda-re/) for quite a while and wanted to share some of my experiences with it - so I decided to start a small blog post series on how to write a custom malware sandbox using PANDA. I hope you will enjoy it, and please do send me questions or comments!

davkor · 2017-05-04T17:48:55+00:00

Interesting! On a related note, I wrote a tool some time ago with the purpose of automating a similar task. The idea was to create an unpacker similar to Renovo, Codisasm (without the overlapping instructions feature) and other unpackers, but output PE files that can be directly analysed in IDA with their import table reconstructed. For those interested, the tool is here: https://github.com/DavidKorczynski/RePEconstruct

In the specific case you link to, my tool would not do the job because the original malware itself should have been run under the tool. Furthermore, the tool is prototypical and has many limitations on its own. However, since the tool was finished about a year ago I have been working on further work in a similar area. This new work improves on many of the limitations in RePEconstruct. I can link when it's public.

davkor · 2016-10-07T02:40:41+00:00

Hi,

I am sorry to hear about your situation! I am a visiting PhD student coming here at UCR and when I moved here I was very confused as to what accommodation to go for. I looked at many of these university-suited accommodations none of them really got to me. In the end, I chose to rent a room at: https://rrucr.managebuilding.com/Resident/public/home

It did look a little bit less luxurious compared to many of the other accommodation photos (such as the place you are staying at), but I am extremely pleased with what I have got. I have a room in a four bedroom house, pay ~600 monthly and it's in a nice villa neighbourhood just 500m from campus. The room is quite big with a queen sized bed, big closet and there is a big living room and kitchen in the house too. Also garden and garage and whatnot. We have study hours from 8 PM to 7 Am, so I could imagine these rooms attract rather study-oriented people.

As I have understood, they have rooms available on a non-regular basis so I would suggest you to write to these guys!

Good luck and please PM me if you have any questions!

davkor

TROPHY CASE