VxD Programming: Primer

We know how to build a VxD which does nothing. In this tutorial, we will make it more productive by adding control message handlers.

VxD Initialization and Termination

There are two types of VxD: static and dynamic. Each type has different load method. They also receive different initialization and termination control messages.

Static VxD:

VMM loads a static VxD when: During development period, I suggest you load the VxD from system.ini because if something goes wrong with your VxD to the extent that Windows cannot start, you can edit system.ini from DOS. You can't do that with registry.
When the VMM loads your static VxD, your VxD will receive three system control messages in the following order: If your VxD must clear the carry flag if the initialization is successful else you must set the carry flag in case of error before returning. You don't have to process any of the three initialization messages if your VxD doesn't need any initialization.
When it's time to terminate the static VxD, the VMM sends the following control messages: Most VxDs don't need to process those two messages, except when you want to prepare the system to enter real mode. You should know that when Windows 95 shuts down, it enters real mode. So if your VxD did something to the real-mode image that will make it unstable, it should restore the change during this time.
You may wonder why those two exit messages have "2" appended to them.Remember that when the VMM loads the static VxDs, it loads the VxDs with the lower initialization order first so that the VxDs can rely upon services of the VxDs that load before them. For example, if VxD2 relies on the services of VxD1, it must specify its initialization order to be larger than that of VxD1. The load order would be:
..... VxD1 ===>  VxD2 ===> VxD3 .....
Now during unloading, it stands to reason that the VxDs that initialize later should uninitialize first so that they may still call VxD services of the VxDs that were loaded before them. In the above example, the order should be:
.... VxD3 ===> VxD2 ===> VxD1.....
In the above example, if VxD2 called some VxD1's services during initialization, it may need to rely on VxD1's services again during unloading. System_Exit2 and Sys_Critical_Exit2 are sent in reverse initialization order. It means that, when VxD2 receives those messages, VxD1 hasn't done uninitialization yet and it can still call VxD1's services. System_Exit and Sys_Critical_Exit messages are not sent in reverse initialization order. It means that when you process those two messages, you can't be sure that you can still call VxD's services of the VxDs that were loaded before you. Those messages should not be used for newer VxDs.
There are two more exit messages: By now, you should be able to guess that there are Device_Reboot_Notify and Crit_Reboot_Notify messages but they are not sent in reverse initialization order as the "2" version.

Dynamic VxD:

Dynamic VxDs can be dynamically loaded and unloaded during Windows 95 session. This feature is not available under Windows 3.x. The primary goal of dynamic VxDs are to support dynamic hardware reconfiguration such as Plug and Play devices. However, you can load/unload them from your win32 applications as well, making them ideal as your applications' ring-0 extension.
The example in the previous tutorial is a static VxD. You can convert the example into a dynamic VxD by adding the keyword DYNAMIC to the VXD statement in .DEF file.
VXD   FIRSTVXD   DYNAMIC
That's all you have to do to convert a static VxD into a dynamic one.
Dynamic VxD can be loaded by: FILE_FLAG_DELETE_ON_CLOSE flag specifies that the VxD is unloaded when the handle returned by CreateFile is closed.
If you use CreateFile to load a dynamic VxD, the VxD must handle w32_DeviceIoControl message. VWIN32 sends this control message to your dynamic VxD when it is first loaded by CreateFile method. Your VxD must return 0 in eax in response to this message. w32_DeviceIoControl messages are also sent when the application calls DeviceIoControl API to communicate with the VxD. We will examine DeviceIoControl interface in later tutorial.
A dynamic VxD receives one message during initialization: And one control message during termination: A dynamic VxD won't receive Sys_Critical_Init, Device_Init and Init_Complete control messages because those messages are sent during system VM initialization. Other than that, dynamic VxDs receives all other control messages when it's in memory. It can do anything a static VxD can do. In short, a dynamic VxD is loaded with different mechanisms and receives different initialization/termination control messages, other than that, it can do anything a static VxD can do.

Other System Control Messages

During the time a VxD stays in memory, it will receive many control messages other than those related to initialization and termination. Some of them are related to virtual machine management and some to miscellaneous events. For example, VM-related control messages are: It's your responsibility to choose to respond to the control messages you're interested in.

Creating procedures inside VxD

You declare a procedure in VxD inside a segment. You should define a segment first and then put your procedure inside it. For example, if you want your function to be in a pageable segment, you should define a pageable segment first, like this:
VxD_PAGEABLE_CODE_SEG

[Your procedure here]

VxD_PAGEABLE_CODE_ENDS

You can put many procedures inside a segment. You as the VxD writer must decide in which segment you should put your procedures. If your procedures must be in memory at all time such as hardware interrupt handlers, put them in a locked segment. Otherwise you should put them in the pageable segment.
You define your procedure with BeginProc and EndProc macros.
BeginProcname

EndProcname

name is the name of your procedure. BeginProc macro can take several more parameters, you should consult Win95 DDK documentation for detail. But most of the time, you can get by with only the name of the procedure.
You should use BeginProc-EndProc macros instead of the normal proc-endp directives because BeginProc-EndProc macros provide more functionality than proc-endp.

VxD Coding Convention

Register Usage

Your VxD can use any general register, FS and GS. But you should beware about modifying segment registers. Especially, you should definitely not alter CS and SS unless you are quite positive you know what you're doing. You can use DS and ES so long as you remember to restore their values when you return. Two flags are especially important: direction and interrupt flags. You should not disable interrupts for an extended period of time and if you modify the direction flag, don't forget to restore its previous state before you return.

Parameter-passing Convention

There are two calling conventions for VxD services: register-based and stack-based. With register-based services, you pass parameters to the services via various registers and you can check the carry flag after calling the service to see if the operation is successful. You can not assume that the values in the general registers will be preserved after calling the services. With stack-based services, you push the parameters on the stack and you got the return value in eax. Stack-based services preserve ebx, esi, edi and ebp. Most of the register-based services originate from Windows 3.x days. Most of the time, you can differentiate between those two kinds of services by looking at the names. If the name of the service begins with an underscore like _HeapAllocate, it's a stack-based (C) service (except for a few services exported by VWIN32.VXD). If the service name doesn't begin with an underscore, it's a register-based service.

Calling VxD Services

You call VMM and VxD services by using VMMCall and VxDCall macros. Both macros have exactly the same syntax. You use VMMCall when you want to call VxD services exported by VMM and you use VxDCall when you call services exported by VxDs other than the VMM.
VMMCall service                ; for calling register-based service
VMMCall  _service, <argument list>    ; for calling stack-based service
VMMCall and VxDCall decompose to int 20h followed by a dword that I described in the previous tutorial but they are much more convenient to use. In the case of stack-based services, you must enclose the argument list with a pair of angle bracket.
VMMCall  _HeapAllocate, <<size mybuffer>, HeapLockedIfDP>
_HeapAllocate is a stack-based service. It accepts two parameters. We must enclose them inside an angle bracket. However, the first parameter is an expression that the macro may interpret incorrectly, so we put it inside another angle bracket.

Flat Addresses

In older tools, the assembler and linker generate incorrect addresses when you use offset operator. So VxD programmers use offset flat: instead of offset. vmm.inc contains a macro to make it easier. OFFSET32 expands to offset flat:. So if you want to use offset operator, you should use OFFSET32 instead.
Note: I experimented with offset operator when I wrote this tutorial. It generated correct addresses so I think the bug has been removed in MASM 6.14. But just to play safe, you should use OFFSET32 macro instead of the plain offset.
[Iczelion's Win32 Assembly Homepage]