Virtual Device Driver Basics

In this tutorial series, I assume you, the reader, are familiar with Intel 80x86's protected mode operations such as virtual 8086 mode, paging, GDT, LDT, IDT. If you don't know about them, read Intel documentations first at http://developer.intel.com/design/pentium/manuals/

Content:

Windows 95 is a multithreaded operating system running in the most privileged level, ring 0. All application programs run at ring 3, the least privileged level. As such, application programs are restricted in what they can do to the system. They cannot use privileged CPU instructions, they cannot access I/O port directly and so on. You're undoubtedly familiar with the big three system components: gdi32, kernel32 and user32. You would think that such important pieces of code should be running in ring 0. But in reality, they run in ring 3, like all other applications. Thus they don't have more privileged than, say, the Window calculator or the minesweeper game. The real power of the system is under control of the virtual machine manager (VMM) and virtual device drivers (VxD).
All this may not happen if DOS doesn't make the picture more complicated. During Windows 3.x era, there are lots of successful DOS programs in the market. Windows 3.x has to be able to run them alongside normal Windows programs else it will fail commercially.
This dilemma is not easy to solve. DOS and Windows programs are drastically different from each other. DOS programs are BAD in that they think they own everything in the system: keyboard, CPU, memory, disk etc. They don't know how to cooperate with other programs while Windows programs (at that time) rely on cooperative multitasking, i.e. every Windows program must yield control to other programs via GetMessage or PeekMessage.
The solution is to run each DOS program in a virtual 8086 machine while all other Windows programs run in another virtual machine called system virtual machine. Windows is responsible for giving CPU time to each virtual machine in a round-robin way. Thus under Windows 3.x, Windows programs use cooperative multitasking but virtual machines use preemptive multitasking.
What is a virtual machine? A virtual machine is a fiction created solely by software. A virtual machine reacts to programs running in it like a real machine. Thus a program doesn't know that it runs in a virtual machine and it doesn't care. So long as the virtual machine responds to the program exactly like a real machine, it can be treated like the real thing.
You can think of the interface between the real machine and its software as a kind of API. This unusual API consists of interrupts, BIOS calls, and I/O ports. If Windows can somehow emulate this API perfectly, the programs running in the virtual machine will behave exactly like they run in the real machine.
This is where VMM and VxDs come into the scene. To coordinate and supervise virtual machines (VMs), Windows needs a program dedicated to the task. That program is the Virtual Machine Manager.

Virtual Machine Manager

VMM is a 32-bit protected mode program. Its primary responsibility is to erect and maintain the framework that supports virtual machines. As such, it's responsible for creating, running, and terminating VMs. VMM is one of the many system VxDs that are stored in VMM32.VXD in your system folder. It's also a VxD but it can be considered the supervisor of other VxDs. Let's examine the boot sequence of Windows 95
  1. io.sys is loaded into memory
  2. config.sys and autoexec.bat are processed
  3. win.com is called
  4. win.com runs VMM32.VXD which is actually a simple DOS EXE file.
  5. VMM32.VXD loads VMM into extended memory using the XMS driver
  6. VMM initializes itself and other default virtual device drivers.
  7. VMM switches the machine into protected mode and creates the system virtual machine
  8. Virtual Shell Device, which is loaded last, starts Windows in the system VM by running krnl386.exe
  9. krnl386.exe loads all other files, culminating in the Windows 95 shell.
As you can see, VMM is the first VxD that is loaded into memory. It creates the system virtual machine and initializes other VxDs. It also provides numerous services to those VxDs.
VMM and VxDs' operation mode is different from that of the realprograms. They are, most of the time, dormant. While application programs are running in the system, those VxDs are not active. They will be awakened when some interrupts/faults/events occur that need their attention.
VMM is not reentrant. That means VxDs must synchronize their accesses to VMM services. There are some situations in which it's not safe to call VMM services such as when a hardware interrupt is being serviced. During that time, VMM cannot tolerate reentrancy. You as the VxD writer must be extremely careful about what you are doing. Remember that, there is no one to take care of your code's errors for you. You're absolutely on your own in ring 0.

Virtual Device Driver

Virtual Device Driver is abbreviated as VxD. x is the placeholder for a device name such as virtual keyboard driver, virtual mouse driver and so on. VxDs are the keys to successful hardware virtualization. Remember that DOS programs think they own everything in the system. When they run in virtual machines, Windows has to provide them with stand-ins for the real devices. VxDs are that stand-ins. VxDs usually virtualize some hardware devices. So, for example, when a dos program thinks it is interacting with the keyboard, it is actually the virtual keyboard device that works with it. A VxD usually takes control of the real hardware device and manages the sharing of the device between VMs.
However, there is no rule that a VxD MUST be associated with a hardware device. It's true that VxDs are designed to virtualize hardware devices but we can also treat VxDs like ring-0 DLLs. For example, if you want some features that can only be achieved in ring 0, you can code a VxD that performs the job for you. In this regard, you can view the VxD as your program's extension since it doesn't virtualize any hardware device.
Before plunging on and creating your own VxDs, let me point out something about them first. There are two types of VxD under Windows 95 Static VxDs are those VxDs that are loaded during system bootup and stay loaded until the system shutdown. This type of VxD dates back from the days of Windows 3.x. Dynamic VxDs are available under Windows 9x only. Dynamic VxDs can be loaded/unloaded when needed. Most of them are VxDs that controls Plug and Play devices which are loaded by Configuration Manager and Input Output Supervisor. You can also load/unload dynamic VxDs from your win32 applications.

Communication between VxDs

VxDs, including VMM, communicate with each other via three mechanisms: Control Messages: VMM sends system control messages to ALL loaded VxDs in the system when some interesting events occur. In this regards, control messages are like windows messages of the ring-3 Windows applications. Every VxD has a function that receives and deals with control messages called device control procedure. There are about 50 or so system control messages. The reason that there are not many control messages is that there are often many VxDs loaded in the system and each of them gets a crack at every control message, if there are too many control messages, the system will grind to a halt. Thus you would find only the really important messages related to VMs such as when a VM is created, destroyed and such. In addition to system control messages, a VxD can define its own custom control messages that it can use to communicate with other VxDs that understand them.

Service APIs: A VxD, including VMM, usually exports a set of public functions that can be called by other VxDs. Those functions are called VxD services. The mechanism of calling those VxD services is quite different from that of ring-3 applications. Every VxD that exports VxD services MUST have a unique ID number. You can obtain such IDs from Microsoft. The ID is a 16-bit number which uniquely identifies a VxD. For example,
 

You can see that VMM has the ID of 1, VPICD has ID of 3 and so on. VMM uses this unique ID to find the VxD that exports the requested VxD services. You also have to select the service you want to call by its index in the service branch table. When a VxD exports VxD services, it stores the addresses of the services in a table. VMM will use the supplied index to locate the address of the desired service from the service table. For example, if you want to call GetVersion which is the first service, you have to specify 0 ( the index is zero-based). The real mechanism of calling VxD services involves int 20h. Your code issues int 20h followed by a dword value that is composed of the device ID and the service index. For example, if you want to call service number 1 which is exported by a VxD that has the device ID of 000Dh, the code would like this: The high word of the dword that follows int 20h instruction contains the device ID. The low word is the zero-based index into the service table.
When int 20h is issued, VMM gains control and it examines the dword that immediately follows the interrupt instruction. Then it extracts the device ID and uses it to locate the VxD and then uses the service index to locate the address of the desired service in that VxD.
You can see that this operation is time-consuming. VMM must waste time in locating the VxD and the address of the desired service. So VMM cheats a little. After the first int 20h operation is successful, VMM snaps the link. By snapping the link, it means VMM replaces the int 20h and the followed dword with the direct call to the service. So the above int 20h snippet would be transformed to: This trick works because the int 20h+dword takes 6 bytes, which is exactly the same as call dword ptr instruction. So subsequent calls is efficient. This method has its pros and cons. On the good side, it reduces the workload of VMM and VxD Loader because they don't have to fix ALL service calls of VxDs at load time. The calls which are never executed will remain unmodified. On the not-so-good side, it makes VxD unloading impossible. Since VMM fixes the calls with the actual addresses of the VxD services, the VxDs which provide those services cannot unload. There is no mechanism to unsnap the links. The corollary of this is that dynamic VxDs are not suitable as VxD service providers.

Callbacks: callbacks or callback functions are functions in VxD that exist to be called by other VxDs. Don't confuse between callbacks and VxD services. Callbacks are not public like services. They are private functions that a VxD gives their addresses to other VxDs in specific situations. For example, when a VxD is servicing a hardware interrupt, since VMM is not reentrant, the VxD cannot use any VxD services that may cause page faults. The VxD may give the address of one of its own function (callback) to VMM so that VMM can call the function when VMM can tolerate page faults. The VxD can then proceed with its work when its callback function is called. The callback idea is not unique to VxD. Many Windows APIs use them as well. The best example is perhaps the window procedure. You specify the address of a window procedure in WNDCLASS or WNDCLASSEX structure and pass it to Windows with RegisterClass or RegisterClassEx call. Windows will call your window procedure when there are messages for the window. Another example is the Window hook procedure. Your application gives the address of the hook procedure to Windows so that Windows will call it when the events the application is interested in happen.
The above three methods are for communication between VxDs. There are also interfaces for V86, protected-mode and Win32 application.

[Iczelion's Win32 Assembly Homepage]