Welcome to Defcon30

Many people are interested in learning about vulnerability discovery but dont know where to start, this workshop is aimed at providing hands on details on how to fuzz open source softwares and findvulnerabilities on linux OS using various fuzzers like AFL, libfuzzer, honggfuzz etc.

What you will be doing?

  1. You will be setting up your own environment, fuzzers and tools.
  2. Working on various excercises.
  3. Will be running various fuzzers to fuzz software.
  4. Debugging crashes to find root cause of vulnerabilities.
  5. Will also do crash triage, corpus minimization etc.

This workshop contains various excercises on these topics, i will be walking you though the solutions as well.

What is needed?

  1. A laptop/desktop with atleast 16gb RAM,4 core processor and 40GB storage.
  2. Vmware/Virtualbox installed.
  3. Internet connection.
  4. willingness to learn :)

What you will learn and key take away

  1. What are common types of vulnerabilities - Buffer overflow, heap overflow, integer overflow, use after free, Double free, out of bound read/Write etc., few real life CVE source code analysis.
  2. Different types of fuzzers - dumb fuzzer, mutation fuzzer, coverage guided fuzzer
  3. How does coverage guided fuzzer works - Binary instrumentation, code coverage, basic blocks
  4. Fuzzing Process - corpus collection, corpus minimization, running fuzzers, crash triage, root cause analysis
  5. How to fuzz software with various fuzzers

Feel Free to -

  1. Ask questions, I don't want to speak alone :)
  2. Ask for help if you are facing any issue.
  3. Take a short break, its 4 hour long workshop.
  4. Correct me if I make any mistake.

Current: Principal Threat Researcher @ Sophos, Bangalore, India

Past: Senior Security Researcher @ McAfee,Bangalore,India

Past: Security Researcher @ Symantec

Hardik Shah

What I do?

Official Blogs: https://news.sophos.com/en-us/author/hardik-shah/

Past Blogs: https://www.mcafee.com/blogs/author/hardik-shah/

Twitter: @hardik05

You can download workshop vm from here, it contains all the tools and softwares we will be using in our workshop:

Download VM from here:

Click here

Zip File password: infected

user name: kali

password: kali

What is a vulnerability?

Bug in the software which can be used to perform unwanted activities.

Example:

Vulnerabilities can be used for:

What is an Exploit?

Common types of vulnerabilities found through fuzzing

Integer overflow/underflow, stack/heap overflow, out of bound read/write, use after free, double free

1. Integer Overflow

Vulnerability in integer data types, they way they store data.

Example:

unsigned int j;
int i;

Size of integer = 4 bytes

Max Value = 11111111 11111111 11111111 11111111

Each bit can be either 0 or 1. so it can denote 2 ue it can store is = 2^32

Signed vs unsigned?

MSB is used for signedness.

1= 00000000000000000000000000000001

-1 = 10000000000000000000000000000001

Max value for signed int = 0x7FFFFFFF

Max value for unsigned int = 0xFFFFFFFF

What happens in this case?

int i;
unsigned int j;
j = 0xFFFFFFFF + 1

Result will become 0, carry 1 bit will be truncated.

i = 0x7FFFFFFFF + 1

Result will become -0x8000000 (negative number) Integer Overflow

2. Integer Underflow

Size of integer = 4 bytes

Signed vs unsigned?

Range for signed int= -0x80000000 to 0x7FFFFFFF

Range for unsigned int = 0 to 0xFFFFFFFF

What happens in this case?

int i;
i = -0x80000000 – 1 = 0x7FFFFFFF

i = highest possible positive number.

Integer Underflow

3. Stack Buffer Overflow

Overflowing the local variable stored inside the stack. They can corrput data on the stack like return address and other variables.

Stack buffer overflow

Image Source: https://www.techtarget.com/searchsecurity/definition/buffer-overflow

Example:

Function foo(){
char var1[8];
char var2[100];
memcpy(var1,var2,sizeof(var2)); 🡪 stack overflow
}

Following is a real life example of a stack overflow vulnerability in vlc media player: Stack Overflow reference: https://hackerone.com/reports/489102

4. Heap Buffer overflow

Heap are used when a process is not sure about the memory size. so they are allocated at run time by functions like malloc,calloc,realloc etc. Overflow in such memory allocation can corrupt various heap managment data.

Heap buffer overflow

Image Source: https://www.researchgate.net/figure/Example-of-Malloc-based-Heap-Overflow-Attack_fig2_4038604

Example:

function test(){
char *var1 = (char*) malloc(8);
char var2[100];
memcpy(var1,var2,sizeof(var2)); 🡪 heap overflow
}

5. Out of bound Read

Memory access or write operation at beyond the allowed limits of Stack memory. Can occur in both - Stack and Heap memory.

Example:

char a[10];
char b;
b=a[100];  🡪OOB Read
a[100] = ‘c'; 🡪OOB Write

Following is a real life out of bound read issue in libgd: Out of bound read reference: https://github.com/libgd/libgd

6. Out of bound Write

Memory access or write operation at beyond the allowed limits of heap memory. Can occur in both - Stack and Heap memory.

Example:

char* a = (char*)malloc(10);
char b;
b=a[100];  🡪 OOB read
a[100] =‘c'; 🡪OOB Write

7. Use after free

Using a memory after it has been freed.

Example:

char *buff = (char*)malloc(10);
free(buff);
buff[0]=‘c'; 🡪 use after free

Following is a real life example of an use after free vulnerability in libtiff: use after free

reference: https://www.asmail.be/msg0055359936.html

8. Double Free

Freeing allocated memory multiple time. Example:

char *buff = (char*)malloc(10);
free(buff);
free(buff); 🡪 double free

Quick Recap

Goal: To get an idea of how to find vulnerabilities by looking at the code.

Time: 10 minutes

Manually indentifying vulnerabilities in C program

//https://github.com/hardik05/Damn_Vulnerable_C_Program/blob/master/imgRead.c
#include<stdio.h>
#include<stdlib.h>
#include<string.h>

struct Image
{
	char header[4];
	int width;
	int height;
	char data[10];
};

int ProcessImage(char* filename){

	FILE *fp;
	char ch;
	struct Image img;

	fp = fopen(filename,"r"); 

	if(fp == NULL)
	{
		printf("\nCan't open file or file doesn't exist.");
		exit(0);
	}

	printf("\n\tHeader\twidth\theight\tdata\t\r\n");

	while(fread(&img,sizeof(img),1,fp)>0){
		printf("\n\t%s\t%d\t%d\t%s\r\n",img.header,img.width,img.height,img.data);
	
		int size1 = img.width + img.height;
		char* buff1=(char*)malloc(size1);

		memcpy(buff1,img.data,sizeof(img.data));
		free(buff1);
	
		if (size1/2==0){
			free(buff1);
		}
		else{
			if(size1 == 123456){
				buff1[0]='a';
			}
		}

		int size2 = img.width - img.height+100;
		//printf("Size1:%d",size1);
		char* buff2=(char*)malloc(size2);

		memcpy(buff2,img.data,sizeof(img.data));

		int size3= img.width/img.height;
		//printf("Size2:%d",size3);

		char buff3[10];
		char* buff4 =(char*)malloc(size3);
		memcpy(buff4,img.data,sizeof(img.data));

		char OOBR_stack = buff3[size3+100]
		char OOBR_heap = buff4[100];

		buff3[size3+100]='c';
		buff4[100]='c';

		if(size3>10){
				buff4=0;
		}
		else{
			free(buff4);
		}

		free(buff2);
	}
	fclose(fp);
}

int main(int argc,char **argv)
{
	ProcessImage(argv[1]);
}


//https://github.com/hardik05/Damn_Vulnerable_C_Program/blob/master/imgRead.c

#include<stdio.h>
#include<stdlib.h>
#include<string.h>

struct Image
{
	char header[4];
	int width;
	int height;
	char data[10];
};

int ProcessImage(char* filename){

	FILE *fp;
	char ch;
	struct Image img;

	fp = fopen(filename,"r"); 

	if(fp == NULL)
	{
		printf("\nCan't open file or file doesn't exist.");
		exit(0);
	}

	printf("\n\tHeader\twidth\theight\tdata\t\r\n");

	while(fread(&img,sizeof(img),1,fp)>0){
		printf("\n\t%s\t%d\t%d\t%s\r\n",img.header,img.width,img.height,img.data);
	
		int size1 = img.width + img.height; //Vulnerability: integer overflow
		char* buff1=(char*)malloc(size1);

		memcpy(buff1,img.data,sizeof(img.data)); //Vulnerability: no data buffer size/malloc success check?
		free(buff1);
	
		if (size1/2==0){
			free(buff1); //Vulnerability: double free
		}
		else{
			if(size1 == 123456){
				buff1[0]='a'; //Vulnerability: use after free
			}
		}

		int size2 = img.width - img.height+100; //Vulnerability: integer underflow
		//printf("Size1:%d",size1);
		char* buff2=(char*)malloc(size2);

		memcpy(buff2,img.data,sizeof(img.data));

		int size3= img.width/img.height;
		//printf("Size2:%d",size3);

		char buff3[10];
		char* buff4 =(char*)malloc(size3);
		memcpy(buff4,img.data,sizeof(img.data));

		char OOBR_stack = buff3[size3+100]; //Vulnerability: out of bound read (stack)
		char OOBR_heap = buff4[100];

		buff3[size3+100]='c'; //Vulnerability: out of bound write (Stack)
		buff4[100]='c'; //Vulnerability: out of bound write (Heap)

		if(size3>10){
				buff4=0; //memory leak?
		}
		else{
			free(buff4);
		}

		free(buff2);
	}
	fclose(fp);
}

int main(int argc,char **argv)
{
	ProcessImage(argv[1]);
}

Bug Hunting

Problem with the manual approach?

Automated

Fuzzing

Fuzzing is the process of automated bug finding. A fuzzer will generate or provide crafted input to a program and will monitor its behaviour. if its crashes then test case can be saved to further analysis and reproduction.

fuzzing

There are various types of fuzzers like:

1. Dumb fuzzers

Generate and process random inputs.

Example: radamsa

https://github.com/hardik05/Damn_Vulnerable_C_Program

You need to be lucky to find bugs in complex program with these fuzzers.

2. Mutation/Generation fuzzers

Generate and process input based on predefined templates.

Example: Peach, Sulley.

You can find bugs but need to do work on understanding and generating file templates or network protocol structure.

3. Coverage guideded fuzzers

Monitors program execution using compile time or runtime instrumentation and can generate new input files based on the paths taken.

Example: AFL,Honggfuzz,libfuzzer

Very useful and successful in finding bugs.

Interesting case study:

pulling jpeg out of thin air: https://lcamtuf.blogspot.com/2014/11/pulling-jpegs-out-of-thin-air.html

$ mkdir in_dir
$ echo 'hello' >in_dir/hello
$ ./afl-fuzz -i in_dir -o out_dir ./jpeg-9a/djpeg

Basic blocks, Instrumentation and Code Coverage

1. Basic block

A program is a collection of basic blocks preforming various operations.

Consider following code:

basic block

What it looks like in IDA: basic block IDA

2. Code Coverage

3. Instrumentation?

How to trace the program execution at runtime to check what code paths are being taken?

basic instrumentation

If source code is available.

compile time instrumentation

If source code is not available.

runtime instrumentation

Steps involved in the fuzzing:

fuzzing Steps

Corpus Collection

Example:

Corpus Mimization

Why do we need to Minimize input corpus?

How?

afl-cmin –i input –o mininput -- ./program @@

Root cause analysis

We found a crash by running fuzzer – now what?

Crash Triage

if we find 1-2 crashes then we can do manual analysis but what if we find gundred or thousands of crashes?

There are various tools availble like:

Crashwalk, atriage, afl-collect

What is AFL?

How does AFL works?

1. Program Flow tracking to monitor code paths

2. Fuzzing strategies - How AFL changes input?

Fuzzing Strategy

Details

Bit Flip

Flips a bit i.e. 1 becomes 0, 0 becomes 1 - this can be done in steps of 1⁄1,2⁄1,4⁄1,8⁄8 ....32⁄8 (same strategy will be used for byteflip as well)

Byte Flip

Flips a byte - this can be done in steps of 1⁄1,2⁄1,4⁄1,8⁄8 ....32⁄8

Arithmetic

Random arithmetic like addition/substraction of random values

Havoc

Random strategy - anythings from bit/bytes/interest/splice/addition/subtraction

Dictionary

User provided dictionary or auto discovered tokens.

Interest

Replace content in original file with interesting values like: 0xff,0x7f etc – 8⁄8,16⁄8..

Splice

Splits and combine two or more files to get a new file.

Reference:

https://github.com/google/AFL/blob/master/docs/technical_details.txt

3. Working Mode

Fork Server Vs Persistent Mode

Fork Server Mode

Persistent Mode

This is very useful when you want to fuzz a selected part of the program.

Step involves in fuzzing a program with AFL

  1. compile program with afl compiler wrappers like afl-gcc, afl-g++, afl-clang, afl-clang++, afl-clang-fast, afl-clang-fast++
  2. collect corpus
  3. minimize corpus using afl-cmin
  4. run the fuzzer - afl-fuzz

Reference:

https://lcamtuf.blogspot.com/2014/10/fuzzing-binaries-without-execve.html

https://lcamtuf.blogspot.com/2015/06/new-in-afl-persistent-mode.html]

santizers helps in finding bugs at the eary stages of program execution.

There are various types of sanitizers which can be enabled at compile time by passing required options. you need to add following flags at compile time if you want to enable any of the sanitizers below:

1. ASAN (-fsanitize=address)
2. MSAN (-fsanitize=memory)
3. UBSAN (-fsanitize=undefined)
4. TSAN (-fsanitize=thread)

sanitizers

AFL has various environment variables like AFL_HARDEN, AFL_USE_ASAN and AFL_USE_MSAN etc. which can be used to enable support for this sanitizers as well as to do various fuzzing related customization. check here: https://github.com/mirrorer/afl/blob/master/docs/env_variables.txt

Reference:

https://clang.llvm.org/docs/AddressSanitizer.html

https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html

https://clang.llvm.org/docs/MemorySanitizer.html

https://clang.llvm.org/docs/ThreadSanitizer.html

Quick Recap

 
$ git clone https://github.com/google/AFL.git
$ cd AFL
$ make
$ cd llvm_mode
$ make
$ cd ..
$ make install

This will install various compiler wrappers and utilities like:

you can install it by typing following command:

$ sudo apt install clang llvm

Goal: learning how to fuzz a program with AFL.

Time: 15-20 minutes

Fuzzing Damn vulnerable C program with AFL

Lets download it first:

$ git clone https://github.com/hardik05/Damn_Vulnerable_C_Program.git

Let's compile it using afl-gcc/afl-clang(when nothing works, use afl-gcc,afl-g++) :

$ afl-gcc -fsanitize=address,undefined -ggdb -O0 imgRead.c -o imgRead_afl

Generate some seed corpus

$ mkdir in
$ echo "IMG" > in/1.img

Fuzz it using following command (explain various command line options)

$ afl-fuzz -i in -o out -m none -- ./imgRead_afl @@

you should see a screen like below: afl run

It has various status messages.

not all are needed. you can read more about them here: https://github.com/google/AFL/blob/master/docs/status_screen.txt

In persistent mode, AFL only fuzzes part of the program and not the entire program. this is useful when you only want to fuzz a specific functionality in a complex software. this offer lot of speed improvements over the fork server mode. you can insert the code you want to fuzz inside a while loop like this:

while (__AFL_LOOP(1000)) {

<code you want to fuzz here>
}

Goal: Learning how to use persistent mode in AFL.

Time: 10-15 minutes

Change our imgRead.c to use persistent mode:

//https://github.com/hardik05/Damn_Vulnerable_C_Program/blob/master/imgRead.c

#include<stdio.h>
#include<stdlib.h>
#include<string.h>

struct Image
{
	char header[4];
	int width;
	int height;
	char data[10];
};

int ProcessImage(char* filename){

	FILE *fp;
	char ch;
	struct Image img;

	fp = fopen(filename,"r");

	if(fp == NULL)
	{
		printf("\nCan't open file or file doesn't exist.");
		exit(0);
	}

	printf("\n\tHeader\twidth\theight\tdata\t\r\n");

	while(fread(&img,sizeof(img),1,fp)>0){
		printf("\n\t%s\t%d\t%d\t%s\r\n",img.header,img.width,img.height,img.data);
		
		int size1 = img.width + img.height;
		char* buff1=(char*)malloc(size1);

		memcpy(buff1,img.data,sizeof(img.data));
		free(buff1);
	
		if (size1/2==0){
			free(buff1);
		}
		else{
			if(size1 == 123456){
				buff1[0]='a';
			}
		}

		int size2 = img.width - img.height+100;
		//printf("Size1:%d",size1);
		char* buff2=(char*)malloc(size2);

		memcpy(buff2,img.data,sizeof(img.data));

		int size3= img.width/img.height;
		//printf("Size2:%d",size3);

		char buff3[10];
		char* buff4 =(char*)malloc(size3);
		memcpy(buff4,img.data,sizeof(img.data));

		char OOBR = buff3[size3];

		buff3[size3]='c';

		if(size3>10){
				buff4=0;
		}
		else{
			free(buff4);
		}

		free(buff2);
	}
	fclose(fp);
}

int main(int argc,char **argv)
{
	ProcessImage(argv[1]);
}

Here is the solution:

/*
Author: Hardik Shah
Email: hardik05@gmail.com
Web: http://hardik05.wordpress.com
*/

//a vulnerable c program to explain common vulnerability types
//fuzz with AFL

#include<stdio.h>
#include<stdlib.h>
#include<string.h>

struct Image
{
	char header[4];
	int width;
	int height;
	char data[10];
};

int ProcessImage(char* filename){

	FILE *fp;
	char ch;
	struct Image img;
while (__AFL_LOOP(1000)) { //this is for persistent mode.
	fp = fopen(filename,"r"); 

	if(fp == NULL)
	{
		printf("\nCan't open file or file doesn't exist.");
		exit(0);
	}

	printf("\n\tHeader\twidth\theight\tdata\t\r\n");

	while(fread(&img,sizeof(img),1,fp)>0){
		printf("\n\t%s\t%d\t%d\t%s\r\n",img.header,img.width,img.height,img.data);

		//integer overflow 0x7FFFFFFF+1=0
		//0x7FFFFFFF+2 = 1
		//will cause very large/small memory allocation.
		int size1 = img.width + img.height;
		char* buff1=(char*)malloc(size1);

		//heap buffer overflow
		memcpy(buff1,img.data,sizeof(img.data));
		free(buff1);
		//double free	
		if (size1/2==0){
			free(buff1);
		}
		else{
			//use after free
			if(size1 == 123456){
				buff1[0]='a';
			}
		}


		//integer underflow 0-1=-1
		//negative so will cause very large memory allocation
		int size2 = img.width - img.height+100;
		//printf("Size1:%d",size1);
		char* buff2=(char*)malloc(size2);

		//heap buffer overflow
		memcpy(buff2,img.data,sizeof(img.data));

		//divide by zero
		int size3= img.width/img.height;
		//printf("Size2:%d",size3);

		char buff3[10];
		char* buff4 =(char*)malloc(size3);
		memcpy(buff4,img.data,sizeof(img.data));

		//OOBR read bytes past buffer
		char OOBR = buff3[size3];

		//OOBW write bytes past buffer
		buff3[size3]='c';

		if(size3>10){
			//memory leak here
			buff4=0;
		}
		else{
			free(buff4);
		}

		free(buff2);


	}
	fclose(fp);
	}//persistent mode loop ends here
}

int main(int argc,char **argv)
{
	ProcessImage(argv[1]);
}

Compile it:

$ afl-clang-fast -fsanitize=address,undefined -g imgRead.c -o imgRead_persistent
$ afl-fuzz -i in/ -o out_per/ -m none -- ./imgRead_persistent @@

Check the speed difference between fork mode and persistent mode.

you can run multiple instances of AFL on your system in Master and Slave mode. this will help in utilizing all the cpu cores and finding bugs and will improve fuzzing speed.

First lets see how many cores machine have, type this command:

$ afl-gotcpu
afl-gotcpu 2.57b by <lcamtuf@google.com>
[*] Measuring per-core preemption rate (this will take 1.00 sec)...
    Core #0: CAUTION (114%)
    Core #1: AVAILABLE (99%)
    Core #2: AVAILABLE (103%)
    Core #6: AVAILABLE (109%)
    Core #3: AVAILABLE (100%)
    Core #7: AVAILABLE (105%)
    Core #5: AVAILABLE (102%)
    Core #4: AVAILABLE (106%)

>>> PASS: You can run more processes on 7 to 8 cores. <<<

you can see that this machine has 8 cores so we can run 8 AFL instance. lets fuzz sample C program now. you need to add -M for master instance and -S to all the slave instance. they can sync up with each other.

so run following commands:

$ screen -S master
$ afl-fuzz -M master -i in/ -o out/ -m none -- ./imgRead_afl @@

Press ctrl+d to detach from screen session, then type following to create another screen session:

$screen -S slave1
$ afl-fuzz -S slave1 -i in/ -o out/ -m none -- ./imgRead_afl @@

Only 1 master should be there but multiple slaves can be run. try this out.

Following screenshot shows, how it looks:

afl multicore

Goal: Learning how to use afl in parallel mode to utilize available CPU cores.

Time: 10-15 minutes

If you don't have source code then you can use Qemu mode to fuzz the binaries. for this go to AFL/qemu_mode and run following commands:

$ sudo ./build_qemu_support.sh

it will complain about some missing dependency so we need to install following packages:

$ sudo apt install libtool-bin
$ sudo apt install bison
$ sudo apt install libglib2.0-dev

then run again:

$ sudo ./build_qemu_support.sh

but latest version of qemu has some bug so again build script will fail. open ./build_qemu_support.sh look for following:

tar xf "$ARCHIVE" || exit 1

change it to following to avoid unpacking failure:

tar xf "$ARCHIVE" --exclude qemu-2.10.0/roms/u-boot/scripts/Kconfig || exit 1

after that run again:

$ sudo ./build_qemu_support.sh

again build will fail with following error:

254 | _syscall0(int, gettid)
 |                ^~~~~~
 TOPDIR/tmp/work/x86_64-linux/qemu-native/3.1.0-r0/qemu-3.1.0/linux-user/syscall.c:185:13: note: in definition of macro ‘_syscall0'
 185 | static type name (void)   \
 |             ^~~~

for this we need to use a hack, open qemu/linux-user/syscall.c file and look for following at line 191 and 265 respectivly:

static type name (void)
static int gettid

just remove static and change it to following:

type name (void)
int gettid

after that again run build script:

$ sudo ./build_qemu_support.sh

it will take sometime to compile and compilation should succeed. After that go to AFL directory and install:

$ AFL/qemu_mode# cd ..
$ sudo make install

this will install afl-qemu-trace which is needed to use qemu mode. you are done installing it. Now lets compile our C program with gcc, type following commands:

gcc imgRead.c -o imgRead_qemu

After that lets fuzz it in Qemu mode, we need to give option -Q for it:

afl-fuzz -Q -i in/ -o out/ -- ./imgRead_qemu

you should be able to see afl fuzzing screen.

Quick Recap

Some helpful GDB commands:

help <command> - displays help

b <filename:linenumber> - breakpoint
disable b <breakpoint> - disable breakpoint
enable b <breakpoint> - enable breakpoint
info b - info on breakpoints

si - step one instruction
s - step till new source line

ni - next instruction (step over)
n - next line (step over)

r - run
c - continue

fi - execute till return

print <var> - prints value of variable
set <var> - change value of the variable

Open imgRead_afl in GDB and provide crash file path with r option as shown below:

$ gdb ./imgRead_afl
(gdb) r out/crashes/id:000000,sig:06,src:000000,op:havoc,rep:128

Example command:

afl-gcc -fsanitize=address,undefined imgread.c -ggdb -O0 -o imgread_afl

AFL_DONT_OPTIMIZE is a flag which disable optimization like "-ggdb -O0"

Goal: to learn root cause analysis using GDB.

Time: 15-20 minutes

  1. What is the condition in the program which is causing this crash?
  2. What are the fields and values in crash files which is causing this crash?

This will be needed when you analyze any real world vulnerability/exploit.

Installing crashwalk

First we will install exploitable gdb plugin as this is needed by crashwalk and the path is hardcoded, so run following commands:

mkdir ~/src
cd ~/src
git clone https://github.com/jfoote/exploitable.git

After we need to install golang, type this command:

$ sudo apt install golang

and then we can install crashwalk:

$ go get -u github.com/bnagy/crashwalk/cmd/...

it will install crashwalk binaries: cwtriage, cwdump, cwfind in ~/go/bin directory. Modify your system path so that these crashwalk binaries are available:

export PATH=$PATH:~/go/bin

Triaging Crashes with cwtriage

First lets see what all options it have:

$ cwtriage

  cwtriage runs crashfiles with instrumentation and outputs results in various 
  formats  
  Usage: cwtriage -root /path/to/afl-dir [-match pattern] -- /path/to/target -in @@ -out whatever
  ( @@ will be substituted for each crashfile )

  -afl
        Prefer the AFL recorded crashing command, if present
  -every int
        Run every n seconds (default -1)
  -f string
        Template filename to use while running crash
  -ignore string
        Directory skip pattern ( go regex syntax )
  -match string
        Match pattern for files ( go regex syntax )
  -mem int
        Memory limit for target processes (MB) (default -1)
  -output string
        Output format to use: [json pb text] (default "text")
  -root string
        Root directory to look for crashes
  -seen
        Include seen results from the DB in the output
  -seendb string
        Path to BoltDB (default "crashwalk.db")
  -strict
        Abort the whole run if any crashes fail to repro
  -t int
        Timeout for target processes (secs) (default 60)
  -tidy
        Move crashes that error under Run() to a tidy dir
  -workers int
        Number of concurrent workers (default 1)

It has built in afl support, so you can type following command:

$ cwtriage -afl -root out

you will see something like this:

2020/11/04 09:12:16 ------
Command: ./imgRead_afl out/crashes/id:000000,sig:06,src:000000,op:havoc,rep:128
File: out/crashes/id:000000,sig:06,src:000000,op:havoc,rep:128
Memory Limit: -1
Timeout: 60
Error: no crash detected
---------
---CRASH SUMMARY---
Filename: out/crashes/id:000001,sig:06,src:000000,op:havoc,rep:32
SHA1: d5c3cd9fe0c7e4d95f1a27d86e2ad34b496b1d67
Classification: PROBABLY_NOT_EXPLOITABLE
Hash: 50aaa03fc7675a1c8baaa05d808c21dd.50aaa03fc7675a1c8baaa05d808c21dd
Command: ./imgRead_afl out/crashes/id:000001,sig:06,src:000000,op:havoc,rep:32
Faulting Frame:
   ProcessImage @ 0x00000000004c6568: in /mnt/b/myworkwsl/Damn_Vulnerable_C_Program/imgRead_afl
Disassembly:
   0x00000000004c6553: mov rcx,QWORD PTR [rsp+0x8]
   0x00000000004c6558: mov rdx,QWORD PTR [rsp]
   0x00000000004c655c: lea rsp,[rsp+0x98]
   0x00000000004c6564: mov eax,r14d
   0x00000000004c6567: cdq
=> 0x00000000004c6568: idiv r15d
   0x00000000004c656b: mov r14d,eax
   0x00000000004c656e: mov r12,r13
   0x00000000004c6571: mov WORD PTR [r13+0x7fff800c],0x200
   0x00000000004c657b: movsxd r15,eax
Stack Head (2 entries):
   ProcessImage              @ 0x00000000004c6568: in /mnt/b/myworkwsl/Damn_Vulnerable_C_Program/imgRead_afl
   main                      @ 0x00000000004c76c8: in /mnt/b/myworkwsl/Damn_Vulnerable_C_Program/imgRead_afl
Registers:
rax=0x000000000000ffff rbx=0x00007ffffffedd00 rcx=0x0000000000000000 rdx=0x0000000000000000
rsi=0x0000000000000000 rdi=0x00007ffffffedb81 rbp=0x00007ffffffeddb0 rsp=0x00007ffffffedc80
 r8=0x00007ffffffecf50  r9=0x0000000000000002 r10=0x00000000004d5973 r11=0x00000000004d5973
r12=0x00007ffffffedca4 r13=0x00000fffffffdb90 r14=0x000000000000ffff r15=0x0000000000000000
rip=0x00000000004c6568 efl=0x0000000000010202  cs=0x0000000000000033  ss=0x000000000000002b
 ds=0x0000000000000000  es=0x0000000000000000  fs=0x0000000000000000  gs=0x0000000000000000
Extra Data:
   Description: Floating point exception signal
   Short description: FloatingPointException (17/22)
   Explanation: The target crashed on a floating point exception. This may indicate a division by zero or a number of other floating point errors. It is generally difficult to leverage these types of errors to gain control of the processor.
---END SUMMARY---
2020/11/04 09:12:18 ------
Command: ./imgRead_afl out/crashes/id:000002,sig:06,src:000000,op:havoc,rep:128
File: out/crashes/id:000002,sig:06,src:000000,op:havoc,rep:128
Memory Limit: -1
Timeout: 60
Error: no crash detected

You will notice that not all the inputs are causing crash. you will see a message like this:

Error: no crash detected

this is because we have not given ASAN options to cwtriage. so run it again with following:

$ ASAN_OPTIONS="abort_on_error=1:symbolize=0" cwtriage -afl -seen -root out

This will create a filenamed crashwalk.db. you can then use cwdump to dump the crash info:

$ cwdump crashwalk.db

you can either redirect this output to a text file or use |more to go through them.

you can also use cwfind to find all the files with the given hash. use following command:

$ cwfind -db crashwalk.db b9bdc301f6ec7b1f38a58796cac7369e.dcde4daab363919b57d3903b9ffcea8c

you can see all the files which causes same crash.

Quick Recap

Get the source code of tcpdump and libpcap.

git clone https://github.com/the-tcpdump-group/tcpdump.git
cd tcpdump
git clone https://github.com/the-tcpdump-group/libpcap.git
cd libpcap

Compile it using AFL

CC=afl-gcc CFLAGS="-g -fsanitize=address -fno-omit-frame-pointer" LDFLAGS="-g -fsanitize=address -fno-omit-frame-pointer" ./configure
sudo make && make install

it will complain about some missing dependencies, run following commands:

sudo apt install flex bison

your compilation should be successfull.

How to get test corpus?

How to minimize it?

use afl-cmin, run following command:

afl-cmin –i tests –o mincorpus –m none -- ./tcpdump –vv –ee –nnr @@

You should see a screenshot something like below: afl-cmin

Lets, Fuzz it!

afl-fuzz –i mincorpus –o fuzzoutput –m none -- ./tcpdump –vv –ee –nnr @@

you should see something like this: tcpdump-fuzz

Fuzzing libtiff

git clone https://gitlab.com/libtiff/libtiff.git

Quick Recap

Installing honggfuzz

git clone https://github.com/google/honggfuzz.git
make
sudo apt install binutils-dev libunwind-dev
sudo make install

How to compile program with HonggFuzz?

hfuzz-clang -fsanitize=address imgRead.c -g –O0 -o imgRead_hfuzz

Fuzzing the program

honggfuzz -i input -–workspace output -- ./imgRead ___FILE___

What is libfuzzer

What is the difference?

LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size)

your harness should call this function and not the standard main() function. when you compile your program with clang and libfuzzer support, it will call its main function, and then it will call LLVMFuzzerTestOneInput function with mutated data.

this function expects 2 arguments:

Data - this is the actual fuzzed or mutated data which will be passed to API you want to fuzz.
Size - size of the Data.

Installing libfuzzer

Latest version of clang have libfuzzer in built. you just need to install clang.

sudo apt install clang

how to compile program with libfuzzer?

clang -g -O1 -fsanitize=fuzzer mytarget.c # Builds the fuzz target w/o sanitizers
clang -g -O1 -fsanitize=fuzzer,address mytarget.c # Builds the fuzz target with ASAN
clang -g -O1 -fsanitize=fuzzer,signed-integer-overflow mytarget.c # Builds the fuzz target with a part of UBSAN
clang -g -O1 -fsanitize=fuzzer,memory mytarget.c # Builds the fuzz target with MSAN

Compiling Damn Vulnerable C program with clang and libfuzzer support

use following command to compile it:

clang -fsanitize=fuzzer,address,undefined -g imgRead_libfuzzer.c -o imgRead_libfuzzer

Fuzzing the program

./imgRead_libfuzzer

Replicating crashes

./imgRead_libfuzzer <crash file>

source & reference: https://llvm.org/docs/LibFuzzer.html

Goal: How to fuzz a program with libfuzzer.

Time:15-20 minutes

Hints:

  1. you get mutated data and size of data, so you dont need to open any file.
  2. you just need to treat data as file contents i.e. it is a buffer which you just read from the file.
  3. you may need to do some typecasting to change it to structure Image.

Solution is in the next page :)

First we need to change our program and need to add LLVMFuzzerTestOneInput function. so lets modify it:

/*
Author: Hardik Shah
Email: hardik05@gmail.com
Web: http://hardik05.wordpress.com
*/

//a vulnerable C program to explain common vulnerability types
//fuzz with libfuzzer

#include<stdio.h>
#include<stdlib.h>
#include<string.h>
#include<stdint.h>
struct Image
{
	char header[4];
	int width;
	int height;
	char data[10];
};

int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size){
if(size<=12){
return 0;
}
	struct Image *img;
	img = (struct Image*)data;
		//integer overflow 0x7FFFFFFF+1=0
		//0x7FFFFFFF+2 = 1
		//will cause very large/small memory allocation.
		int size1 = img->width + img->height;
		char* buff1=(char*)malloc(size1);

		//heap buffer overflow
		memcpy(buff1,img->data,sizeof(img->data));
		free(buff1);
		//double free	
		if (size1/3==0){
			free(buff1);
		}
		else{
			//use after free
			if(size1/20 == 0){
				buff1[0]='a';
			}
		}


		//integer underflow 0-1=-1
		//negative so will cause very large memory allocation
		int size2 = img->width - img->height+100;
		//printf("Size1:%d",size1);
		char* buff2=(char*)malloc(size2);

		//heap buffer overflow
		memcpy(buff2,img->data,sizeof(img->data));

		//divide by zero
		int size3= img->width/img->height;
		//printf("Size2:%d",size3);

		char buff3[10];
		char* buff4 =(char*)malloc(size3);
		memcpy(buff4,img->data,sizeof(img->data));

		//stack OOBR read bytes past buffer
		char OOBR_stack = buff3[size3];

		char OOBR_heap = buff4[size1];

		//stack OOBW write bytes past buffer
		buff3[size3]='c';
		buff4[size1]='c';

		if(size3/5==0){
			//memory leak here
			buff4=0;
		}
		else{
			free(buff4);
		}

		free(buff2);


}

Compiling with clang and libfuzzer support

use following command to compile it:

clang -fsanitize=fuzzer,address,undefined -g imgRead_libfuzzer.c -o imgRead_libfuzzer

Fuzzing the program

./imgRead_libfuzzer

but it stops at first crash. how can we run it continously?

try this commands:

./imgRead_libfuzzer -jobs=10
./imgRead_libfuzzer -fork=1 -ignore_crashes=1

Press Ctrl+c to cancel.

Quick recap

✅ Always report to vendor first.generally vendor have security@vendor.com email id. you can also check their website for security contact.

✅ Dont disclose anything until vendor releases the patch.

✅ Vendors may reward you with bug bounty for your work!

My youtube channel on fuzzing: Click here

Twitter: @hardik05

email: hardik05[AT]gmail[DOT]com

Website: fuzzing.in

Thanks everyone, hope you have enjoyed this workshop as much as I have enjoyed delivering it and creating contents and various excericses for it.