配置交叉编译工具链

先安装一些必要的软件

sudo apt install build-essential open-vm-tools open-vm-tools-desktop

然后安装交叉编译工具链

sudo apt install gcc-arm-linux-gnueabi
sudo apt install g++-arm-linux-gnueabi

安装QEMU

下载

下载9.2.4版本

wget https://download.qemu.org/qemu-9.2.4.tar.xz
tar -xf qemu-9.2.4.tar.xz

安装依赖

必要依赖:

sudo apt-get install git libglib2.0-dev libfdt-dev libpixman-1-dev zlib1g-dev ninja-build

推荐依赖:

sudo apt-get install git-email
sudo apt-get install libaio-dev libbluetooth-dev libcapstone-dev libbrlapi-dev libbz2-dev
sudo apt-get install libcap-ng-dev libcurl4-gnutls-dev libgtk-3-dev
sudo apt-get install libibverbs-dev libjpeg8-dev libncurses5-dev libnuma-dev
sudo apt-get install librbd-dev librdmacm-dev
sudo apt-get install libsasl2-dev libsdl2-dev libseccomp-dev libsnappy-dev libssh-dev
sudo apt-get install libvde-dev libvdeplug-dev libvte-2.91-dev libxen-dev liblzo2-dev
sudo apt-get install valgrind xfslibs-dev 
sudo apt-get install libnfs-dev libiscsi-dev

编译安装

首先配置目标为ARM架构

./configure --target-list=arm-softmmu

然后编译并安装

make -j$(nproc); sudo make install

编译Linux内核

下载

wget https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.12.35.tar.xz
tar -xf linux-6.12.35.tar.xz 

编译

首先设置ARCH和CROSS_COMPILE,修改Makefile中对应字段为:

ARCH            ?= arm
CROSS_COMPILE   ?= arm-linux-gnueabi-

安装依赖之前,先配置一下软件源,之前忘改了,修改/etc/apt/sources.list.d/ubuntu.sources

Types: deb
URIs: https://mirrors.tuna.tsinghua.edu.cn/ubuntu
Suites: noble noble-updates noble-backports
Components: main restricted universe multiverse
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg

Types: deb-src
URIs: https://mirrors.tuna.tsinghua.edu.cn/ubuntu
Suites: noble noble-updates noble-backports
Components: main restricted universe multiverse
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg

Types: deb
URIs: http://security.ubuntu.com/ubuntu/
Suites: noble-security
Components: main restricted universe multiverse
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg

Types: deb-src
URIs: http://security.ubuntu.com/ubuntu/
Suites: noble-security
Components: main restricted universe multiverse
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg

然后安装依赖

sudo apt update
sudo apt build-dep linux

在menuconfig中,勾选Device Drivers —>Network device support —> Universal TUN/TAP device driver support

make vexpress_defconfig;make menuconfig

之后开始编译

make zImage
make modules; make dtbs

现在可以通过下面的指令,直接启动内核

qemu-system-arm \
        -M vexpress-a9 \
        -m 512M \
        -kernel arch/arm/boot/zImage \
        -dtb arch/arm/boot/dts/arm/vexpress-v2p-ca9.dtb \
        -nographic \
        -append "console=ttyAMA0"

最终输出

Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.12.35 #1
Hardware name: ARM-Versatile Express
Call trace: 
 unwind_backtrace from show_stack+0x10/0x14
 show_stack from dump_stack_lvl+0x54/0x68
 dump_stack_lvl from panic+0x10c/0x350
 panic from mount_root_generic+0x238/0x2e8
 mount_root_generic from prepare_namespace+0x1fc/0x254
 prepare_namespace from kernel_init+0x1c/0x12c
 kernel_init from ret_from_fork+0x14/0x28
Exception stack(0xa0825fb0 to 0xa0825ff8)
5fa0:                                     00000000 00000000 00000000 00000000
5fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
5fe0: 00000000 00000000 00000000 00000000 00000013 00000000
---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ]---

PS: QEMU可以按Ctrl+a,再按c调出monitor,然后输入q既可退出QEMU

制作根文件系统

下载

wget https://busybox.net/downloads/busybox-1.37.0.tar.bz2
tar -xf busybox-1.37.0.tar.bz2

编译

首先还是设置CROSS_COMPILE和ARCH,然后开始配置

make defconfig; make menuconfig

勾选 Settings -> Build static binary (no shared libs)选项

image-20250703195148760

然后执行编译和安装

make; make install

出现了一个错误

libbb/hash_md5_sha.c: In function ‘sha1_end’:
libbb/hash_md5_sha.c:1316:35: error: ‘sha1_process_block64_shaNI’ undeclared (first use in this function); did you mean ‘sha1_process_block64’?
 1316 |          || ctx->process_block == sha1_process_block64_shaNI
      |                                   ^~~~~~~~~~~~~~~~~~~~~~~~~~
      |                                   sha1_process_block64
libbb/hash_md5_sha.c:1316:35: note: each undeclared identifier is reported only once for each function it appears in
make[1]: *** [scripts/Makefile.build:198: libbb/hash_md5_sha.o] Error 1
make: *** [Makefile:744: libbb] Error 2
  CC      libbb/hash_md5_sha.o
libbb/hash_md5_sha.c: In function ‘sha1_end’:
libbb/hash_md5_sha.c:1316:35: error: ‘sha1_process_block64_shaNI’ undeclared (first use in this function); did you mean ‘sha1_process_block64’?
 1316 |          || ctx->process_block == sha1_process_block64_shaNI
      |                                   ^~~~~~~~~~~~~~~~~~~~~~~~~~
      |                                   sha1_process_block64
libbb/hash_md5_sha.c:1316:35: note: each undeclared identifier is reported only once for each function it appears in
make[1]: *** [scripts/Makefile.build:198: libbb/hash_md5_sha.o] Error 1
make: *** [Makefile:744: libbb] Error 2

一个解决方法是修改配置,禁用对SHA1和SHA256的硬件加速选项,取消勾选Settings-> SHA1: Use hardware accelerated instructions if possible 以及 Settings-> SHA256: Use hardware accelerated instructions if possible

然后重新编译,又遇到了新问题,报错很长,截取了前半部分,看到是在编译tc.c啥的出错了,那就再把它取消勾选,在Networking Utilities中。

networking/tc.c:236:24: warning: unused variable ‘tb’ [-Wunused-variable]
  236 |         struct rtattr *tb[TCA_CBQ_MAX+1];
      |                        ^~
make[1]: *** [scripts/Makefile.build:198: networking/tc.o] Error 1
make: *** [Makefile:744: networking] Error 2
  CC      networking/tc.o
networking/tc.c: In function ‘cbq_print_opt’:
networking/tc.c:236:27: error: ‘TCA_CBQ_MAX’ undeclared (first use in this function); did you mean ‘TCA_CBS_MAX’?
  236 |         struct rtattr *tb[TCA_CBQ_MAX+1];
      |                           ^~~~~~~~~~~
      |                           TCA_CBS_MAX

重新编译后终于通过

制作根文件系统

在当前用户的家目录下新建一个文件夹作为虚拟机的根文件系统

mkdir ~/rootfs
cp -r _install/* ~/rootfs

然后创建一些基本的目录和设备节点

sudo mkdir -p dev etc lib proc sys tmp root
sudo mknod dev/console c 5 1
sudo mknod dev/null c 1 3

接下来,需要拷贝交叉编译工具的库文件到根文件系统

sudo cp /usr/arm-linux-gnueabi/lib/* ~/rootfs/lib/

最后编译以下文件,完善文件系统

  1. /etc/fstab 该文件用于配置文件系统挂载信息

    proc    /proc     proc     defaults         0 0
    tmpfs   /tmp      tmpfs    defaults         0 0
    sysfs   /sys      sysfs    defaults         0 0
    tmpfs   /dev      tmpfs    defaults         0 0
    var     /dev      tmpfs    defaults         0 0
    ramfs   /dev      ramfs    defaults         0 0
    
  2. /etc/inittab 该文件用于配置一些基本设置。第一行执行系统初始化脚本;第二行在控制台上启用一个交互式的shell;第三行定义当按下 Ctrl+Alt+Delete 组合键时的系统行为;第四行定义了在系统关闭过程中要执行的命令

    ::sysinit:/etc/init.d/rcS
    console::askfirst:-/bin/sh
    ::ctrlaltdel:/sbin/reboot
    ::shutdown:/bin/umount -a -r
    
  3. /etc/profile 定义了命令提示符

    PS1='szy@vexpress:\w # '
    export PS1
    
  4. /etc/init.d/ 定义了一些环境变量,以及负责挂载文件系统

    #!/bin/sh
    PATH=/sbin:/bin:/usr/sbin:/usr/bin
    LD_LIBRARY_PATH=/lib
    export PATH LD_LIBRARY_PATH
    
    mount -a
    mkdir -p /dev/pts
    mount -t devpts devpts /dev/pts
    mdev -s
    mkdir -p /var/lock
    
    echo "--------------------------------"
    echo "  Welcome to A9 vexpress board  "
    echo "--------------------------------"
    

编译U-Boot

下载

wget https://ftp.denx.de/pub/u-boot/u-boot-2025.04.tar.bz2
tar -xf u-boot-2025.04.tar.bz2

编译

分别编辑Makefile和config.mk,设置CROSS_COMPILE和ARCH字段

ARCH            ?= arm		# 在config.mk中
CROSS_COMPILE   ?= arm-linux-gnueabi-	# 在Makefile中

编译:

make vexpress_ca9x4_defconfig
make -j$(nproc)

配置NFS

这一步的目的是,让虚拟机通过NFS的方式挂载宿主机上的根文件系统,这样就可以实现在宿主机上开发,实时同步修改到虚拟机。

安装NFS

  1. 安装nfs服务器

     sudo apt install nfs-kernel-server
    
  2. 配置nfs,修改/etc/exports,在底下加上一行

    /home/szy/rootfs (rw,sync,no_root_squash,no_subtree_check)
    
  3. 重启nfs服务

    sudo /etc/init.d/rpcbind restart
    sudo /etc/init.d/nfs-kernel-server restart
    

网络配置

这一步的目的是连接QEMU虚拟机与宿主机的网络,采用桥接方式。大致思路是在宿主机创建一个虚拟网桥br0,接入网卡eth0与TAP设备tap0,在QEMU虚拟机处连接网卡eth0与宿主机tap0,实现互通。

/etc/netplan/目录下创建一个yaml文件,内容如下:

network:
  version: 2
  renderer: NetworkManager
  ethernets:
    ens33:  # 物理网卡
      dhcp4: no # 自身不获取IP

  bridges:
    br0:  # 创建的网桥
      interfaces: [ens33] # 将物理网卡桥接进来
      dhcp4: yes # 让网桥自动获取IP
      parameters:
        stp: false

保存后输入sudo netplan apply应用更改,然后在/etc/systemd/system目录下创建一个systemd服务:

[Unit]
Description=Persistent TAP device for QEMU
Wants=network-pre.target
Before=network-pre.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/sbin/ip tuntap add dev tap-qemu mode tap
ExecStop=/usr/sbin/ip tuntap del dev tap-qemu mode tap

[Install]
WantedBy=multi-user.target

最后启动新服务,这样就可以在开机时自动配置TAP设备

sudo systemctl enable qemu-tap.service 
sudo systemctl start qemu-tap.service 

然后让我们配置一下虚拟机,创建 /etc/qemu-ifup,写入如下内容:

#! /bin/sh
# Script to bring a network (tap) device for qemu up.
# The idea is to add the tap device to the same bridge
# as we have default routing to.

# in order to be able to find brctl
PATH=$PATH:/sbin:/usr/sbin
ip=$(which ip)

if [ -n "$ip" ]; then
   ip link set "$1" up
else
   brctl=$(which brctl)
   if [ ! "$ip" -o ! "$brctl" ]; then
     echo "W: $0: not doing any bridge processing: neither ip nor brctl utility not found" >&2
     exit 0
   fi
   ifconfig "$1" 0.0.0.0 up
fi

switch=$(ip route ls | \
    awk '/^default / {
          for(i=0;i<NF;i++) { if ($i == "dev") { print $(i+1); next; } }
         }'
        )

# only add the interface to default-route bridge if we
# have such interface (with default route) and if that
# interface is actually a bridge.
# It is possible to have several default routes too
for br in $switch; do
    if [ -d /sys/class/net/$br/bridge/. ]; then
        if [ -n "$ip" ]; then
          ip link set "$1" master "$br"
        else
          brctl addif $br "$1"
        fi
        exit	# exit with status of the previous command
    fi
done

echo "W: $0: no bridge for guest interface found" >&2

重新编译U-Boot

修改U-Boot目录下 /include/configs/vexpress_common.h,添加如下内容:

#define CONFIG_BOOTCOMMAND \
	"tftp 0x60003000 uImage;tftp 0x60500000 vexpress-v2p-ca9.dtb; \
	setenv bootargs 'root=/dev/nfs rw \
  nfsroot=192.168.110.136:/home/szy/rootfs \
  ip=192.168.110.137 console=ttyAMA0';\
	bootm 0x60003000 - 0x60500000; "

#define CONFIG_IPADDR 192.168.110.137
#define CONFIG_NETMASK 255.255.255.0
#define CONFIG_SERVERIP 192.168.110.136

然后修改QEMU启动脚本,启动时调用/etc/qemu-ifup脚本,这样我们的QEMU开发环境就完成了。在这里我的内核和设备树都通过tftp获取。

qemu-system-arm \
  -M vexpress-a9 \
  -m 512M \
  -kernel /home/szy/linux/tftpboot/u-boot \
  -nographic \
  -nic tap,script=/etc/qemu-ifup,downscript=no \
  -audiodev none,id=silent