User:No92/Random Bits

Warnings

Random info on how to get tools to run without warnings or annoying features.

OVMF

EDK has a very weird build system. After cloning, it requires you to make the Base Tools and set up a target.txt configuration. In order to disable serial output cluttering your terminal you have to comment out the TerminalDxe module:

make -C BaseTools
cp /path/to/target.txt Conf/target.txt
patch OvmfPkg/OvmfPkgX64.fdf < /path/to/OvmfPkgX64.fdf.patch
cd OvmfPkg
./build.sh

You'll find OVMF.fd at Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd.

target.txt may look something like this for a x86_64 4-threaded machine with GCC 5+ installed:

ACTIVE_PLATFORM       = OvmfPkg/OvmfPkgX64.dsc
TARGET_ARCH           = X64
TOOL_CHAIN_TAG        = GCC5
BUILD_TARGET          = RELEASE
MAX_CONCURRENT_THREAD_NUMBER = 9

OvmfPkgX64.fdf.patch looks like this for the EDK2 tree at c17956e0eedce299ac253ac40238ce90a5e623e0:

--- OvmfPkg/OvmfPkgX64.fdf	2017-03-26 17:01:47.324258000 +0200
+++ OvmfPkg/OvmfPkgX64.fdf	2017-03-26 17:02:22.321889038 +0200
@@ -239,7 +239,7 @@
 INF  MdeModulePkg/Universal/Console/ConPlatformDxe/ConPlatformDxe.inf
 INF  MdeModulePkg/Universal/Console/ConSplitterDxe/ConSplitterDxe.inf
 INF  MdeModulePkg/Universal/Console/GraphicsConsoleDxe/GraphicsConsoleDxe.inf
-INF  MdeModulePkg/Universal/Console/TerminalDxe/TerminalDxe.inf
+# INF  MdeModulePkg/Universal/Console/TerminalDxe/TerminalDxe.inf
 INF  MdeModulePkg/Universal/DriverHealthManagerDxe/DriverHealthManagerDxe.inf
 INF  MdeModulePkg/Universal/BdsDxe/BdsDxe.inf
 INF  MdeModulePkg/Application/UiApp/UiApp.inf

QEMU

There are still dumbasses out there abusing stderr for debug output (why is this allowed and included in release 2.8.0?), so we have to patch it out.

--- block/vvfat.c	2017-04-11 00:00:05.816645363 +0200
+++ block/vvfat.c	2017-04-11 00:02:40.890137141 +0200
@@ -1162,10 +1162,6 @@
     s->qcow_filename = NULL;
     s->fat2 = NULL;
     s->downcase_short_names = 1;
-
-    fprintf(stderr, "vvfat %s chs %d,%d,%d\n",
-            dirname, cyls, heads, secs);
-
     s->sector_count = cyls * heads * secs - (s->first_sectors_number - 1);
 
     if (qemu_opt_get_bool(opts, "rw", false)) {

Compile QEMU as always using configure and make.

Running QEMU properly™ requires some flags to be set:

qemu-system-x86_64 -M accel=kvm:tcg -net none -serial stdio -vga std

Also, it's quite convenient that QEMU can treat directories in your file system as FAT disks:

qemu-system-x86_64 -drive format=raw,file=fat:rw:hdd

In this case, hdd is the directory with your operating system's file system.

YASM

The latest release 1.3.0 has no flag to turn off the segment register ignored in 64-bit mode warning. Fix this by compiling from source. Nicely enough, it both adds the flag to turn it off and disables the warning by default (why did they bother creating the flag?)

The build system isn't documented (or the documentation is hidden in such a way that I can't find it), but it appears that it's based on cmake which does most of the work for you.

cmake -DCMAKE_INSTALL_PREFIX="$HOME/opt/yasm" -Wno-dev .
make
make install

Makefile

Filtering out flags for specific files is easy. This is usually required for Stack Smashing Protector:

kernel/boot/ssp.c.o: CFLAGS := $(filter-out -fstack-protector-all,$(CFLAGS)) -fno-stack-protector

This line filters out the -fstack-protector-all flags, replacing it with -fno-stack-protector.

VirtualBox

This is nothing but a huge mess, but still something you might want to test your OS on. While endusers will mostly use the GUI, it actually makes a lot of sense to configure it from a bash script.

Basic VM Management

You'll be using VBoxManage the entire time.

Creating a VM is simple: you just pass it the name:

VBoxManage createvm --name "MyOS" --register

Deleting should be just as simple, but there is no way to force deleting in the rm -f way. Therefore, we just suppress the output and run true on failure:

VBoxManage unregistervm "MyOS" --delete 2> /dev/null || true

Next, you'll want to set some basic properties of the VM.

VBoxManage modifyvm "MyOS" --memory 1024 --cpus 2

For an extensive list of all supported options, look here.

You need something to boot from; but first, you have to configure your IDE controller:

VBoxManage storagectl "MyOS" --add ide --name "IDE"

Next, you can attach your images. Attaching an ISO image is done like this:

VBoxManage storageattach "MyOS" --storagectl "IDE" --port 0 --device 0 --medium /path/to/your/image.iso --type dvddrive

Host Key Behavior

The default host key is Right Control, which one might want to change. Setting this requires knowing the VirtualBox-specific keycodes:

Key	VirtualBox Code
Left Ctrl	65507
Left Alt	65513
Right Ctrl	65508
Super/Meta	65515
Menu	65383

Note: these are known to change between versions; these were tested with VBoxManage 5.1.22r115126. Also, setting this affects all VMs; remember that when booting up your other VMs.

VBoxManage setextradata global GUI/Input/HostKeyCombination 65508

You can also change keybinds. For instance, you'd do this in order to unset the Close action and map Power Off to Host+X:

VBoxManage setextradata global GUI/Input/MachineShortcuts Close=,PowerOff=X

Managing the GUI

To disable the bottom Status Bar on a per-VM basis, run:

VBoxManage setextradata "MyOS" GUI/StatusBar/Enabled false

Reordering the indicators can be done as well:

VBoxManage setextradata "MyOS" GUI/StatusBar/IndicatorOrder HardDisks,OpticalDisks,FloppyDisks,Network,USB,SharedFolders,Display,VideoCapture,Features,Mouse,Keyboard

You can hide entire menus by their label:

VBoxManage setextradata "MyOS" GUI/RestrictedRuntimeMenus View,Input,Devices,Debug,Help

Or you can hide members of a menu. For example, if you want to hide Detach from the Machine menu, do:

VBoxManage setextradata "MyOS" GUI/RestrictedRuntimeMachineMenuActions Detach

Video Modes

If you use GRUB for video mode setting, you'll see that nothing happens. Apparently, VirtualBox requires you to define custom resolutions.

VBoxManage setextradata "MyOS" CustomVideoMode1 1920x1080x32

Please note that you can define multiple resolutions; just name them CustomVideoMode2, CustomVideoMode3, and so forth.

VM Shutdown

If you want your script to wait for the VM to shut down, we have to loop until our VM is not in the list of running VMs:

until [ -z "$(VBoxManage list runningvms | grep \"MyOS\")" ]
do
	sleep .5
done

Multiboot 2

While the tag structure is very useful, it's not perfect; but that's for another discussion. Sometimes, the spec is just blatantly wrong. According to the spec, the structure of the ELF-Symbols tag (type 9) is this:

struct mb2_elf {
	uint32_t	type;
	uint32_t	size;
	uint16_t	num;
	uint16_t	entsize;
	uint16_t	shndx;
	uint16_t	reserved;
	char		shdr[1];
};

However, when examining the section header table, it appears to be off by 4 bytes. Digging into the latest GRUB2 source reveals that either GRUB2 or the spec is wrong. In reality, the structure provided by GRUB2 looks like this:

struct mb2_elf {
	uint32_t	type;
	uint32_t	size;
	uint32_t	num;
	uint32_t	entsize;
	uint32_t	shndx;
	char		shdr[1];
};

Notice that the reserved member is gone and the 16-bit wide members are actually 32-bit. Sometimes, I do wonder why I'm still using broken software.

PXE

Unsurprisingly, this is a pain to set up if you start from zero.

Setting up proxyDHCP

I will assume that you have some sort of router that is serving DHCP to your network, which also happens to not support PXE. Unless that router is an old computer rocking pfSense, you'll have no way to configure PXE directly on the router. This is where proxyDHCP comes in: it allows you to keep the DHCP server on your network, while also serving PXE requests. For this, you'll have to know one IP address from the subnet you'll be serving. My local IPs, for instance, are in the format 192.168.1.xxx. Find your local IP (usually by running ifconfig or hostname -I) and install dnsmasq, which will be serving the PXE requests. The configuration at /etc/dnsmasq.conf looks like this:

port=0
log-dhcp
dhcp-range=192.168.1.0,proxy
pxe-service=x86PC,"PXE boot",boot/grub/i386-pc/core
enable-tftp
tftp-root=/home/osdev/OS/hdd

Note: replace 192.168.1.0 with your PXE server and replace the tftp-root with your own. Do not, however, change boot/grub/i386-pc/core; we will need this later.

Save and restart the dnsmasq service.

Setting up a working network directory

cd into the tftp root. Here, we will be running GRUB; please note that running anything lower than GRUB 2.02 has produced numerous issues for me. First, we will create a basic netdir:

mkdir -p boot/grub
grub-mknetdir --net-directory=boot --subdir grub

Fixing everything

... and stop here. This entire technology (or the implementation, to be more specific), is broken beyond belief. If you though this is a smooth ride, you'll be disappointed, because we have to redo 50% of the work GRUB just did for us.

For some reason, GRUB will require you to load every single module you use, even though most of them usually are pre-loaded. Even after 30 years of GNU they can't get their stuff straight, GRUB is looking in the wrong directory for the modules. As a workaround, symlink boot/grub/i386-pc to i386-pc. I'll leave this as an exercise to you.

At this point, we'll have to generate a two-line early config file.

set root=(tftp,192.168.1.xxx)
configfile boot/grub/grub.cfg

Replace 192.168.1.xxx with the local IP of the machine running dnsmasq. All this does is set up the correct root and jumping to your normal grub.cfg.

Finally, recreate a proper core image. Run something like this:

grub-mkimage -O i386-pc-pxe --output boot/grub/i386-pc/core.0 -c i386-pc/grub.cfg -p=boot/grub  tftp pxe net configfile

Note: make sure you do not drop the .0 at the end of the output file and the double space before specifying the modules.

Finally, make sure to load the module for every command you run. For instance, if you use multiboot2, you will have to add

insmod multiboot2

to your normal grub.cfg.

Score some bonus points ...

... and combine this with Wake-on-LAN. See this as a challenge to write your own script to do this.