Fuzzing the FreeBSD Kernel with Syzkaller and Nested Virtualization on a Linux Host
During my internal research time I decided to dive into kernel fuzzing. In particular, my goal was to set up Syzkaller in a first step and extend the system call descriptions in a second step to increase coverage and find new bugs. As Syzkaller is fuzzing the Linux kernel quite extensively, I hoped that fuzzing the FreeBSD kernel could maybe find some low hanging fruits.
However, as my main operating system is Ubuntu, I encountered some annoying issues. Therefore, I had the idea to run a FreeBSD host in a virtual machine and use this VM as a host to spawn nested FreeBSD machines which are fuzzed by Syzkaller. Unfortunately, this was not as easy as I thought at first. I tried different combinations of VirtualBox, VMWare, qemu and bhyve for the nested virtualization, many of which failed and took me a lot of time. Hence, I want to describe in this blog post how to set this up to give the community something back and hopefully make the life of someone else easier.
Setup
To distinguish commands, which are executed on the Linux host and the FreeBSD VM, commands on the Linux host and the FreeBSD VM start with $
and %
, respectively.
Setting Up the FreeBSD Host VM
Create a directory where you put the FreeBSD host image into, download and decompress it:
$ mkdir ~/fuzzing_freebsd
$ cd ~/fuzzing_freebsd
$ wget https://download.freebsd.org/releases/VM-IMAGES/14.0-RELEASE/amd64/Latest/FreeBSD-14.0-RELEASE-amd64-zfs.qcow2.xz
$ xz --decompress FreeBSD-14.0-RELEASE-amd64-zfs.qcow2.xz
Please make sure to download an image that is using zfs as a file system, as this is required at a later point to setup the bhyve machine.
Before you run the VM, increase its size:
$ qemu-img resize FreeBSD-14.0-RELEASE-amd64-zfs.qcow2 +25G
Now, start this VM with qemu and set everything up that is needed. First, run the VM as follows and adjust the number of cores and amount of RAM to your preferences:
$ qemu-system-x86_64 -m 16G -smp 12 -drive file=FreeBSD-14.0-RELEASE-amd64-zfs.qcow2,format=qcow2 -enable-kvm -net user,host=10.0.2.10,hostfwd=tcp::10022-:22 -net nic,model=e1000 -cpu host
Log in as root
(no password is required) and make sure that the zpool has grown by +25G
. Otherwise run the following commmands to make sure that the zpool grows:
% pkg install gpart
% gpart show
[in my case the drive I want to increase is: ada0]
% gpart recover ada0
% gpart resize -i 4 /dev/ada0 [take the index of freebsd-zfs]
% zpool set autoexpand=on zroot
% zpool online -e zroot ada0
If the zpool has not grown, please reboot.
SSH Configuration on The Host VM
To configure ssh, first edit /etc/ssh/sshd_config
and set
PermitRootLogin yes
Additionally, run
% sysrc sshd_enable=YES
% sysrc ifconfig_DEFAULT=DHCP
% /etc/rc.d/sshd start
Give the current user root
a password (I just chose root
as password):
% passwd
Now, check if you can connect to the VM via
$ ssh -p 10022 root@localhost
Checking Out and Building Syzkaller
Before you download and configure Syzkaller, install the following required dependencies on the host VM:
$ ssh -p 10022 root@localhost
Checking out and Building Syzkaller
Before you download and configure Syzkaller, install the following required dependencies on the host VM:
% pkg install bash gcc git gmake go golangci-lint llvm
Due to the nested VM setup, bhyve will be used as the VM backend. Hence, a DHCP server has to be installed, too:
% pkg install dnsmasq
Next, checkout the Syzkaller sources:
% mkdir /root/fuzzing
% git clone https://github.com/google/syzkaller /root/fuzzing/syzkaller
Check if you can build the binaries:
% cd /root/fuzzing/syzkaller
% gmake manager fuzzer execprog executor TARGETOS=freebsd
Setting up the Nested FreeBSD VM
To set up a nested FreeBSD VM inside our FreeBSD host VM, download an image of the latest FreeBSD version. This time download a raw image instead of a qcow image as the raw image will managed by bhyve:
% pkg install wget
% cd /root
% wget https://download.freebsd.org/snapshots/VM-IMAGES/15.0-CURRENT/amd64/Latest/FreeBSD-15.0-CURRENT-amd64.raw.xz
% xz --decompress FreeBSD-15.0-CURRENT-amd64.raw.xz
To use bhyve on the FreeBSD host VM, some additional steps are required. First execute:
% zfs create -o mountpoint=/syzkaller zroot/syzkaller
% mv FreeBSD-15.0-CURRENT-amd64.raw /syzkaller
Furthermore, configure networking and DHCP for the nested VM:
% ifconfig bridge create bridge0
% ifconfig bridge0 inet 169.254.0.1
% echo 'dhcp-range=169.254.0.2,169.254.0.254,255.255.255.0' > /usr/local/etc/dnsmasq.conf
% echo 'interface=bridge0' >> /usr/local/etc/dnsmasq.conf
% sysrc dnsmasq_enable=YES
% service dnsmasq start
% echo 'net.link.tap.up_on_open=1' >> /etc/sysctl.conf
% sysctl net.link.tap.up_on_open=1
Add the following lines to /etc/rc.conf
such that the configuration of bridged networking is enabled every time the system boots:
cloned_interfaces="bridge0 tap0"
ifconfig_bridge0="inet 169.254.0.1 addm tap0 up"
ifconfig_tap0="up"
Before the VM is started by bhyve, check that the bhyve kernel module is really loaded:
% kldload vmm
Next, start the nested VM from the host VM with bhyve. First, create a new tap interface:
% ifconfig tap create
In my case, tap0
was created. Now, add this interface to the bridge:
% ifconfig bridge0 addm tap0
Finally, run
% bhyveload -c stdio -m 512M -d /syzkaller/FreeBSD-15.0-CURRENT-amd64.raw -e autoboot_delay=0 testvm0
% bhyve -H -A -P -c 1 -m 512M -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s 3:0,virtio-blk,/syzkaller/FreeBSD-15.0-CURRENT-amd64.raw -l com1,stdio testvm0
and log in as root
(no password required).
SSH Configuration on the Nested VM
As Syzkaller has to be able to connect to the nested VM via SSH and a SSH key, configure SSH and first edit /etc/ssh/sshd_config
to set
PermitRootLogin yes
Additionally, run
% sysrc sshd_enable=YES
% sysrc ifconfig_DEFAULT=DHCP
% /etc/rc.d/sshd start
Give the current user root
a password (I just chose root
as password):
% passwd
Find out the ip address of the nested VM by running ifconfig
on the nested VM for example. In my case the ip address of the nested VM is 169.254.0.46
. Now, try to connect to the nested VM via SSH by running:
% ssh root@169.254.0.46
Next, create a ssh key pair on the host VM and add the public key to the running VM:
% mkdir -p /root/fuzzing/files
% cd /root/fuzzing/files
% ssh-keygen -f root_key_nested.id_rsa -t rsa -N ''
% ssh-copy-id -i ./root_key_nested.id_rsa.pub root@169.254.0.46
Now, you should be able to connect to the nested VM by executing the following command:
% ssh -i /root/fuzzing/files/root_key_nested.id_rsa root@169.254.0.46
Downloading, Building And Installing Kernel Sources
The next step is to check out the FreeBSD kernel sources:
% git clone --depth=1 --branch=main https://github.com/freebsd/freebsd-src /usr/src
Create a custom kernel config for Syzkaller and build the kernel:
% cd /usr/src/sys/amd64/conf
% cat <<__EOF__ > SYZKALLER
include "./GENERIC"
ident SYZKALLER
options COVERAGE
options KCOV
__EOF__
% cd /usr/src
% make -j $(sysctl -n hw.ncpu) KERNCONF=SYZKALLER buildkernel
To install the built kernel on the nested VM, run the following commands on the host VM (shut the nested VM down before installing the new kernel):
% mdconfig -a -f /syzkaller/FreeBSD-15.0-CURRENT-amd64.raw
% mount /dev/md0p4 /mnt
% cd /usr/src
% make KERNCONF=SYZKALLER installkernel DESTDIR=/mnt
% umount /mnt
% mdconfig -d -u 0
Start the nested VM and make sure that it is running correctly:
% bhyveload -c stdio -m 512M -d /syzkaller/FreeBSD-15.0-CURRENT-amd64.raw -e autoboot_delay=0 testvm0
% bhyve -H -A -P -c 1 -m 512M -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s 3:0,virtio-blk,/syzkaller/FreeBSD-15.0-CURRENT-amd64.raw -l com1,stdio testvm0
Running Syzkaller
Now that we have a working setup, it’s time for actually running Syzkaller.
Configuring Syzkaller
Create a config file for Syzkaller that determines how many VMs should be spawned, and that contains the information where to find the kernel sources and object files. Moreover, you can tell Syzkaller by this config file how to spawn the nested VMs with bhyve and how to connect via SSH using the private SSH key. Create the file /root/fuzzing/files/freebsd.cfg
with the following contents:
{
"name": "freebsd",
"target": "freebsd/amd64",
"http": "127.0.0.1:56741",
"workdir": "/root/fuzzing/syzkaller/workdir",
"syzkaller": "/root/fuzzing/syzkaller/",
"kernel_obj": "/usr/obj/usr/src/amd64.amd64/sys/SYZKALLER",
"kernel_src": "/",
"sshkey": "/root/fuzzing/files/root_key_nested.id_rsa",
"sandbox": "none",
"procs": 12,
"image": "/syzkaller/FreeBSD-15.0-CURRENT-amd64.raw",
"type": "bhyve",
"vm": {
"count": 4,
"cpu": 4,
"mem": "4048M",
"bridge": "bridge0",
"hostip": "169.254.0.1",
"dataset": "zroot/syzkaller"
}
}
Furthermore, this config file contains the path to the workdir
of Syzkaller which has to be created:
% mkdir /root/fuzzing/syzkaller/workdir
Running Syzkaller
Syzkaller presents the current fuzzing results really nicely by spawning a webserver. However, this webserver is running on our host VM and not on our host OS (Ubuntu). Hence, we forward the port 56741
on which the spawned webserver is running (see the entry http
in /root/fuzzing/files/freebsd.cfg
) by the following command on our main OS:
$ ssh -N -L56741:127.0.0.1:56741 -p 10022 root@localhost
Finally, everything is ready to go! On the nested VM, execute the following commands:
% cd /root/fuzzing/syzkaller
% bin/syz-manager -config ../files/freebsd.cfg
Yeahhh, enjoy viewing the results by visiting http://127.0.0.1:56741
with a browser on your Linux host.
Fuzzing the Bluetooth Stack
Now that the setup is ready to run Syzkaller to fuzz nested FreeBSD VMs on a FreeBSD host VM hosted by a Linux main OS, I could finally try to add some system call descriptions to find uncovered bugs. While analyzing which system calls have already been described I discovered that the description of the socket system calls for the bluetooth stack were missing. Hence, I decided to give this a try.
Adding System Call Descriptions for the Bluetooth Stack
I do not want to go into detail here how the syzlang works as this has already been done several time, for example here.
To include new system call descriptions for the bluetooth stack, I added the file syzkaller/freebsd/socket_bluetooth.txt
with the following contents:
include <sys/types.h>
include <sys/socket.h>
include <sys/sockio.h>
include <sys/bitstring.h>
include <netinet/in.h>
include <netgraph/bluetooth/include/ng_hci.h>
include <netgraph/bluetooth/include/ng_l2cap.h>
include <netgraph/bluetooth/include/ng_btsocket.h>
resource sock_bt[sock]
resource sock_bt_hci[sock_bt]
resource sock_bt_sco[sock_bt]
resource sock_bt_rfcomm[sock_bt]
resource sock_bt_l2cap[sock_bt]
socket$bt_hci(domain const[PF_BLUETOOTH], type const[SOCK_RAW], protocol const[BLUETOOTH_PROTO_HCI]) sock_bt_hci
bind$bt_hci(s sock_bt_hci, addr ptr[in, sockaddr_hci], addrlen len[addr])
ioctl$sock_bt_hci(fd sock_bt_hci, cmd flags[bt_hci_ioctl], arg buffer[inout])
setsockopt$bt_hci_HCI_FILTER(s sock_bt_hci, level const[SOL_HCI_RAW], optname const[SO_HCI_RAW_FILTER], optval ptr[in, ng_btsocket_hci_raw_filter], optlen len[optval])
setsockopt$bt_hci_HCI_DIRECTION(s sock_bt_hci, level const[SOL_HCI_RAW], optname const[SO_HCI_RAW_DIRECTION], optval ptr[in, int32], optlen len[optval])
getsockopt$bt_hci(s sock_bt_hci, level const[SOL_HCI_RAW], optname flags[bt_hci_sockopt], optval buffer[out], optlen ptr[inout, len[optval, int32]])
write$bt_hci(fd sock_bt_hci, buf buffer[in], nbytes bytesize[buf])
socket$bt_sco(domain const[PF_BLUETOOTH], type const[SOCK_SEQPACKET], protocol const[BLUETOOTH_PROTO_SCO]) sock_bt_sco
bind$bt_sco_sockaddr_sco(s sock_bt_sco, addr ptr[in, sockaddr_sco], addrlen len[addr])
connect$bt_sco_sockaddr_sco(s sock_bt_sco, name ptr[in, sockaddr_sco], namelen len[name])
getsockopt$bt_sco_SOL_SCO(s sock_bt_sco, level const[SOL_SCO], optname flags[bt_sol_sockopt], optval buffer[out], optlen ptr[inout, len[optval, int32]])
socket$bt_l2cap(domain const[PF_BLUETOOTH], type flags[bt_l2cap_type], protocol const[BLUETOOTH_PROTO_L2CAP]) sock_bt_l2cap
bind$bt_l2cap(fd sock_bt_l2cap, addr ptr[in, sockaddr_l2], addrlen len[addr])
connect$bt_l2cap(fd sock_bt_l2cap, addr ptr[in, sockaddr_l2], addrlen len[addr])
setsockopt$bt_l2cap(fd sock_bt_l2cap, level const[SOL_L2CAP], optname flags[bt_l2cap_sockopt], optval ptr[in, int32], optlen len[optval])
getsockopt$bt_l2cap(s sock_bt_l2cap, level const[SOL_L2CAP], optname flags[bt_l2cap_sockopt], optval buffer[out], optlen ptr[inout, len[optval, int32]])
socket$bt_rfcomm(domain const[PF_BLUETOOTH], type const[SOCK_STREAM], protocol const[BLUETOOTH_PROTO_RFCOMM]) sock_bt_rfcomm
bind$bt_rfcomm(fd sock_bt_rfcomm, addr ptr[in, sockaddr_rfcomm], addrlen len[addr])
connect$bt_rfcomm(fd sock_bt_rfcomm, addr ptr[in, sockaddr_rfcomm], addrlen len[addr])
setsockopt$bt_rfcomm(fd sock_bt_rfcomm, level const[SOL_RFCOMM], optname flags[bt_rfcomm_sockopt], optval ptr[in, int32], optlen len[optval])
getsockopt$bt_rfcomm(s sock_bt_rfcomm, level const[SOL_RFCOMM], optname flags[bt_rfcomm_sockopt], optval buffer[out], optlen ptr[inout, len[optval, int32]])
ng_btsocket_hci_raw_filter {
packet_mask int64[32]
event_mask int64[64]
}
sockaddr_hci {
hci_len int8
hci_family int8
hci_node int8[32]
}
sockaddr_sco {
sco_len int8
sco_family int8
sco_bdaddr int8[6]
}
sockaddr_l2 {
l2cap_len int8
l2cap_familty int8
l2cap_psm int16
l2cap_bdaddr int8[6]
l2cap_cid int16
l2cap_bdaddr_type int8
}
sockaddr_rfcomm {
rfcomm_len int8
rfcomm_family int8
rfcomm_bdaddr int8[6]
rfcomm_channel int8
}
bt_hci_ioctl = SIOC_HCI_RAW_NODE_GET_STATE, SIOC_HCI_RAW_NODE_INIT, SIOC_HCI_RAW_NODE_GET_DEBUG, SIOC_HCI_RAW_NODE_SET_DEBUG, SIOC_HCI_RAW_NODE_GET_BUFFER, SIOC_HCI_RAW_NODE_GET_BDADDR, SIOC_HCI_RAW_NODE_GET_FEATURES, SIOC_HCI_RAW_NODE_GET_STAT, SIOC_HCI_RAW_NODE_RESET_STAT, SIOC_HCI_RAW_NODE_FLUSH_NEIGHBOR_CACHE, SIOC_HCI_RAW_NODE_GET_NEIGHBOR_CACHE, SIOC_HCI_RAW_NODE_GET_CON_LIST, SIOC_HCI_RAW_NODE_GET_LINK_POLICY_MASK, SIOC_HCI_RAW_NODE_SET_LINK_POLICY_MASK, SIOC_HCI_RAW_NODE_GET_PACKET_MASK, SIOC_HCI_RAW_NODE_SET_PACKET_MASK, SIOC_HCI_RAW_NODE_GET_ROLE_SWITCH, SIOC_HCI_RAW_NODE_SET_ROLE_SWITCH
bt_hci_sockopt = SO_HCI_RAW_FILTER, SO_HCI_RAW_DIRECTION
bt_sol_sockopt = SO_SCO_MTU, SO_SCO_CONNINFO
bt_l2cap_sockopt = SO_L2CAP_IMTU, SO_L2CAP_OMTU, SO_L2CAP_IFLOW, SO_L2CAP_OFLOW, SO_L2CAP_FLUSH, SO_L2CAP_ENCRYPTED
bt_l2cap_type = SOCK_SEQPACKET, SOCK_RAW
bt_rfcomm_sockopt = SO_RFCOMM_MTU, SO_RFCOMM_FC_INFO
Extracting, Generating and Rebuilding Binaries
After adding the new system call descriptions, you have to first run a make extract
command on the host VM to extract information based on the new file socket_bluetooth.txt
:
% cd /root/fuzzing/syzkaller
% gmake extract ARCH=amd64 TARGETOS=freebsd SOURCEDIR=/usr/src/
The new file syzkaller/freebsd/sys/socket_bluetooth.txt.const
should have been created now. Next, execute a generate
command to update the generated code:
% cd /root/fuzzing/syzkaller
% gmake generate
To rebuild the binaries, run a gmake
command:
% cd /root/fuzzing/syzkaller
% gmake manager fuzzer execprog executor TARGETOS=freebsd
Rebuilding the Kernel
Unfortunately, our current kernel build does not support bluetooth. Therefore, edit the file /usr/src/sys/amd64/conf/SYZKALLER
and rebuild/install the kernel afterward:
include "./GENERIC"
ident SYZKALLER
options COVERAGE
options KCOV
options KASAN
options NETGRAPH
options NETGRAPH_DEBUG
options NETGRAPH_ASYNC
options NETGRAPH_BLUETOOTH
options NETGRAPH_BLUETOOTH_BT3C
options NETGRAPH_BLUETOOTH_H4
options NETGRAPH_BLUETOOTH_HCI
options NETGRAPH_BLUETOOTH_L2CAP
options NETGRAPH_BLUETOOTH_SOCKET
options NETGRAPH_BLUETOOTH_UBT
options NETGRAPH_BLUETOOTH_UBTBCMFW
options NETGRAPH_BPF
options NETGRAPH_BRIDGE
options NETGRAPH_CAR
options NETGRAPH_CHECKSUM
options NETGRAPH_CISCO
options NETGRAPH_DEFLATE
options NETGRAPH_DEVICE
options NETGRAPH_ECHO
options NETGRAPH_EIFACE
options NETGRAPH_ETHER
options NETGRAPH_ETHER_ECHO
options NETGRAPH_FEC
options NETGRAPH_FRAME_RELAY
options NETGRAPH_GIF
options NETGRAPH_GIF_DEMUX
options NETGRAPH_HOLE
options NETGRAPH_IFACE
options NETGRAPH_IP_INPUT
options NETGRAPH_IPFW
options NETGRAPH_KSOCKET
options NETGRAPH_L2TP
options NETGRAPH_LMI
options NETGRAPH_MPPC_COMPRESSION
options NETGRAPH_MPPC_ENCRYPTION
options NETGRAPH_NAT
options NETGRAPH_NETFLOW
options NETGRAPH_ONE2MANY
options NETGRAPH_PATCH
options NETGRAPH_PIPE
options NETGRAPH_PPP
options NETGRAPH_PPPOE
options NETGRAPH_PPTPGRE
options NETGRAPH_PRED1
options NETGRAPH_RFC1490
options NETGRAPH_SOCKET
options NETGRAPH_SPLIT
options NETGRAPH_SPPP
options NETGRAPH_TAG
options NETGRAPH_TCPMSS
options NETGRAPH_TEE
options NETGRAPH_TTY
options NETGRAPH_UI
options NETGRAPH_VJC
options NETGRAPH_VLAN
Now, rebuild the kernel on the host VM and install the kernel on the nested VMs (shut the nested VM down before the following steps):
% cd /usr/src
% make -j $(sysctl -n hw.ncpu) KERNCONF=SYZKALLER buildkernel
% mdconfig -a -f /syzkaller/FreeBSD-15.0-CURRENT-amd64.raw
% mount /dev/md0p4 /mnt
% cd /usr/src
% make KERNCONF=SYZKALLER installkernel DESTDIR=/mnt
% umount /mnt
% mdconfig -d -u 0
Check again if the nested VM is booting properly.
Finding New Bugs
Before you run Syzkaller, edit the config file /root/fuzzing/files/freebsd.cfg
and specify which system calls should be fuzzed by the field enable_syscalls
. For example, we can focus on socket$bt_hci
and sendto
(this is only a small subset of the new system call descriptions I added):
{
"name": "freebsd",
"target": "freebsd/amd64",
"http": "127.0.0.1:56741",
"workdir": "/root/fuzzing/syzkaller/workdir",
"syzkaller": "/root/fuzzing/syzkaller/",
"kernel_obj": "/usr/obj/usr/src/amd64.amd64/sys/SYZKALLER",
"kernel_src": "/",
"sshkey": "/root/fuzzing/files/root_key_nested.id_rsa",
"sandbox": "none",
"procs": 12,
"image": "/zroot/syzkaller/FreeBSD-15.0-CURRENT-amd64.raw",
"type": "bhyve",
"vm": {
"count": 4,
"cpu": 4,
"mem": "4048M",
"bridge": "bridge0",
"hostip": "169.254.0.1",
"dataset": "zroot/syzkaller"
},
"enable_syscalls": [ "socket$bt_hci", "sendto" ]
}
Finally, start Syzkaller and find new bugs!
% cd /root/syzkaller
% bin/syz-manager -config ../files/freebsd.cfg
After a short time Syzkaller finds a RedZonePartial
crash of the following form:
login: panic: ASan: Invalid access, 34-byte read at 0xfffffe0002044eb0, RedZonePartial(2)
After some more time Syzkaller was able to trigger a UMAUseAfterFree
crash in ng_btsocket_hci_raw_send()
which was caused by the same bug as the RedZonePartial
crash before:
login: panic: ASan: Invalid access, 34-byte read at 0xfffffe00835ad0f0, UMAUseAfterFree(fd)
cpuid = 2
time = 1711976404
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0xc6/frame 0xfffffe00abe76210
kdb_backtrace() at kdb_backtrace+0xd0/frame 0xfffffe00abe76370
vpanic() at vpanic+0x26a/frame 0xfffffe00abe76530
panic() at panic+0xb5/frame 0xfffffe00abe76600
kasan_code_name() at kasan_code_name/frame 0xfffffe00abe766d0
kasan_memmove() at kasan_memmove+0x1c9/frame 0xfffffe00abe76710
ng_btsocket_hci_raw_send() at ng_btsocket_hci_raw_send+0x498/frame 0xfffffe00abe767d0
sosend_generic() at sosend_generic+0xd4a/frame 0xfffffe00abe769a0
sousrsend() at sousrsend+0x116/frame 0xfffffe00abe76a30
kern_sendit() at kern_sendit+0x5a2/frame 0xfffffe00abe76ba0
sendit() at sendit+0x157/frame 0xfffffe00abe76bf0
sys_sendto() at sys_sendto+0x181/frame 0xfffffe00abe76d10
amd64_syscall() at amd64_syscall+0x47a/frame 0xfffffe00abe76f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00abe76f30
--- syscall (198, FreeBSD ELF64, __syscall), rip = 0x2ac88a, rsp = 0x826567ef8, rbp = 0x826567f80 ---
KDB: enter: panic
[ thread pid 1086 tid 100349 ]
Stopped at kdb_enter+0x6e: movq $0,0x23c69a7(%rip)
As we are fuzzing with a FreeBSD host, Syzkaller was even able to find a C-reproducer! This would have not been possible in an automatic way if we were fuzzing the FreeBSD Kernel with a Linux host.
Moreover, I have reported the UMAUserAfterFree bug to the security team of FreeBSD. The method ng_btsocket_hci_raw_send()
was not verifying that the sockaddr
, which has been allocated in sendit()
, is large enough. Therefore, copying the sockaddr
into an mbuf
in this line could trigger an out-of-bounds read.
Conclusion
It required a lot of time and effort to get this setup running which consists of a Linux host that runs a FreeBSD host VM which fuzzes nested FreeBSD VMs with Syzkaller. Therefore I hope that this blog post helps some people to get this setup working quickly. To make a contribution to the open source project Syzkaller I sent a pull request for my additions which have been merged, see here.