DeviceIoControl Interface

We will learn about dynamic VxD in this tutorial. Specifically, we will learn how to create, load and use them.
Download the example here.

VxD Interfaces

There are a total of 4 interfaces that VxD provides.

VxD services
V86 Interface
Protected-mode (PM) Interface
Win32 DeviceIoControl Interface

We already know about VxD services. V86 and PM interfaces are functions that are callable from V86 and PM applications respectively. Since V86 and PM applications are 16-bit, we cannot use those two interfaces from a win32 application. With Windows 95, Microsoft adds an additional interface for win32 applications to call on the services of VxDs: DeviceIoControl Interface.

DeviceIoControl Interface

To put it simply, the DeviceIoControl interface is a way for win32 applications to call functions inside VxDs. Don't confuse functions called via DeviceIoControl with VxD services: they are not the same. For example, DeviceIoControl function 1 may not be the same as VxD service 1. You should think of DeviceIoControl functions as a separate group of functions, provided for use by win32 applications only.
Since it's an interface, there are two sides:

On the win32 application side:

It has to call CreateFile to open/load the VxD first. If the call is successful, the VxD will be in memory and CreateFile returns the handle to the VxD in eax.
Next you call DeviceIoControl API function to select the function to execute. DeviceIoControl has the following syntax:

DeviceIoControl PROTO hDevice:DWORD,\

dwIoControlCode:DWORD,\

lpInBuffer:DWORD,\

nInBufferSize:DWORD,\

lpOutBuffer:DWORD,\

nOutBufferSize:DWORD,\

lpBytesReturned:DWORD,\

lpOverlapped:DWORD

hDevice is the handle of the VxD returned from CreateFile
dwIoControlCode is the value that specifies the operation the VxD should perform. You must somehow obtain the list of possible dwIoControlCode values for a VxD before you know what operation you should select. But most of the time, you're the one who codes the VxD so you knowthe list of all possible dwIoControlCode values.
lpInBuffer is the address of the buffer that contains the data the VxD needs to use to perform the operation specified in dwIoControlCode. If the operation doesn't need data, you can pass NULL.
nInBufferSize is the size, in bytes, of the data in the buffer pointed to by lpInBuffer.
lpOutBuffer is the address of the buffer that the VxD will fill with the output data when the operation is successful. If the operation doesn't have any data to return, this field can be NULL.
nOutBufferSize is the size, in bytes, of the buffer pointed to by lpOutBuffer
lpBytesReturned is the address of a dword variable that will receive the size of data the VxD filled into lpOutBuffer.
lpOverlapped is the address of the OVERLAPPED structure if you want the operation to be asynchronous. If you want to wait until the operation is finished, put NULL in this field.

On the VxD side:

It must process w32_deviceIoControl message. When the VxD receives w32_deviceIoControl message, its registers have the following values:

ebx contains the VM handle.
esi is the pointer to DIOCParams structure which contains information passed from the win32 application.

DIOCParams is defined as follows:

DIOCParams STRUC

Internal1 DD ?

VMHandle DD ?

Internal2 DD ?

dwIoControlCode DD ?

lpvInBuffer DD ?

cbInBuffer DD ?

lpvOutBuffer DD ?

cbOutBuffer DD ?

lpcbBytesReturned DD ?

lpoOverlapped DD ?

hDevice DD ?

tagProcess DD ?

DIOCParams ENDS

Internal1 is the pointer to the client register structure of the win32 application
VMHandle no explanation needed
Internal2 is the pointer to device descriptor block (DDB)
dwIoControlCode, lpvInBuffer, cbInBuffer, lpvOutBuffer, cbOutBuffer, lpcbBytesReturned, lpOverlapped are the parameters that were passed to DeviceIoControl API call.
hDevice is the ring-3 device handle
tagProces is the process tag

From DIOCParams structure, you have all the information the win32 application passed to your VxD.
Your VxD must at least process DIOC_Open (a value passed in dwIoControlCode) which VWIN32 will send to your VxD when a win32 application calls CreateFile to open your VxD. If your VxD is ready, it must return 0 in eax and CreateFile call will be successful. If your VxD is not ready, it must return a nonzero value in eax and CreateFile will fail. Other than DIOC_Open, your VxD will receive DIOC_Closehandle code from VWIN32 when the win32 application closes the device handle.

Minimum dynamic VxD skeleton that is loadable by CreateFile

.386p
include vmm.inc
include vwin32.inc
DECLARE_VIRTUAL_DEVICE DYNAVXD,1,0, DYNAVXD_Control,\
     UNDEFINED_DEVICE_ID, UNDEFINED_INIT_ORDER
Begin_control_dispatch DYNAVXD
    Control_Dispatch w32_DeviceIoControl, OnDeviceIoControl
End_control_dispatch DYNAVXD
VxD_PAGEABLE_CODE_SEG
BeginProc OnDeviceIoControl
    assume esi:ptr DIOCParams
    .if [esi].dwIoControlCode==DIOC_Open
        xor eax,eax
    .endif
    ret
EndProc OnDeviceIoControl
VxD_PAGEABLE_CODE_ENDS
end

;--------------------------------------------------------------------------------------------------------------------------------
; Module Definition File
;---------------------------------------------------------------------------------------------------------------------------------

VXD DYNAVXD DYNAMIC
SEGMENTS
    _LPTEXT    CLASS 'LCODE'    PRELOAD NONDISCARDABLE
    _LTEXT      CLASS 'LCODE'    PRELOAD NONDISCARDABLE
    _LDATA      CLASS 'LCODE'    PRELOAD NONDISCARDABLE
    _TEXT        CLASS 'LCODE'    PRELOAD NONDISCARDABLE
    _DATA        CLASS 'LCODE'    PRELOAD NONDISCARDABLE
    CONST       CLASS 'LCODE'    PRELOAD NONDISCARDABLE
    _TLS          CLASS 'LCODE'    PRELOAD NONDISCARDABLE
    _BSS          CLASS 'LCODE'    PRELOAD NONDISCARDABLE
    _LMGTABLE    CLASS 'MCODE'    PRELOAD NONDISCARDABLE IOPL
    _LMSGDATA    CLASS 'MCODE'    PRELOAD NONDISCARDABLE IOPL
    _IMSGTABLE   CLASS 'MCODE'    PRELOAD DISCARDABLE IOPL
    _IMSGDATA     CLASS 'MCODE'    PRELOAD DISCARDABLE IOPL
    _ITEXT             CLASS 'ICODE'     DISCARDABLE
    _IDATA             CLASS 'ICODE'    DISCARDABLE
    _PTEXT            CLASS 'PCODE'    NONDISCARDABLE
    _PMSGTABLE    CLASS 'MCODE'    NONDISCARDABLE IOPL
    _PMSGDATA    CLASS 'MCODE'    NONDISCARDABLE IOPL
    _PDATA            CLASS 'PDATA'    NONDISCARDABLE SHARED
    _STEXT            CLASS 'SCODE'    RESIDENT
    _SDATA            CLASS 'SCODE'    RESIDENT
    _DBOSTART    CLASS 'DBOCODE'    PRELOAD NONDISCARDABLE CONFORMING
    _DBOCODE       CLASS 'DBOCODE'    PRELOAD NONDISCARDABLE CONFORMING
    _DBODATA        CLASS 'DBOCODE'    PRELOAD NONDISCARDABLE CONFORMING
    _16ICODE          CLASS '16ICODE'    PRELOAD DISCARDABLE
    _RCODE             CLASS 'RCODE'
EXPORTS
    DYNAVXD_DDB @1

Full Example

Below are the source code of the win32 application that loads a dynamic VxD and calls a function in the VxD via DeviceIoControl API.

; VxDLoader.asm

.386
.model flat,stdcall
include windows.inc
include kernel32.inc
includelib kernel32.lib
include user32.inc
includelib user32.lib
.data
    AppName db "DeviceIoControl",0
    VxDName db "\\.\shellmsg.vxd",0
    Success db "The VxD is successfully loaded!",0
    Failure db "The VxD is not loaded!",0
    Unload db "The VxD is now unloaded!",0
    MsgTitle db "DeviceIoControl Example",0
    MsgText db "I'm called from a VxD!",0
    InBuffer dd offset MsgTitle
                  dd offset MsgText
.data?
    hVxD dd ?
.code
start:
    invoke CreateFile,addr VxDName,0,0,0,0,FILE_FLAG_DELETE_ON_CLOSE,0
    .if eax!=INVALID_HANDLE_VALUE
        mov hVxD,eax
        invoke MessageBox,NULL,addr Success,addr AppName,MB_OK+MB_ICONINFORMATION
        invoke DeviceIoControl,hVxD,1,addr InBuffer,8,NULL,NULL,NULL,NULL
        invoke CloseHandle,hVxD
        invoke MessageBox,NULL,addr Unload,addr AppName,MB_OK+MB_ICONINFORMATION
    .else
        invoke MessageBox,NULL,addr Failure,NULL,MB_OK+MB_ICONERROR
    .endif
    invoke ExitProcess,NULL
end start

Below is the source code of the dynamic VxD that is called by vxdloader.asm
; ShellMsg.asm

.386p
include vmm.inc
include vwin32.inc
include shell.inc
DECLARE_VIRTUAL_DEVICE SHELLMSG,1,0, SHELLMSG_Control,\
     UNDEFINED_DEVICE_ID, UNDEFINED_INIT_ORDER
Begin_control_dispatch SHELLMSG
    Control_Dispatch w32_DeviceIoControl, OnDeviceIoControl
End_control_dispatch SHELLMSG
VxD_PAGEABLE_DATA_SEG
    pTitle dd ?
    pMessage dd ?
VxD_PAGEABLE_DATA_ENDS
VxD_PAGEABLE_CODE_SEG
BeginProc OnDeviceIoControl
    assume esi:ptr DIOCParams
    .if [esi].dwIoControlCode==DIOC_Open
        xor eax,eax
    .elseif [esi].dwIoControlCode==1
        mov edi,[esi].lpvInBuffer
        ;-----------------------------------
        ; copy the message title to buffer
        ;-----------------------------------
        VMMCall _lstrlen, <[edi]>
        inc eax
        push eax
        VMMCall _HeapAllocate,<eax,HEAPZEROINIT>
        mov pTitle,eax
        pop eax
        VMMCall _lstrcpyn,<pTitle,[edi],eax>
        ;-----------------------------------
        ; copy the message text to buffer
        ;-----------------------------------
        VMMCall _lstrlen, <[edi+4]>
        inc eax
        push eax
        VMMCall _HeapAllocate,<eax,HEAPZEROINIT>
        mov pMessage,eax
        pop eax
        VMMCall _lstrcpyn,<pMessage,[edi+4],eax>
        mov edi,pTitle
        mov ecx,pMessage
        mov eax,MB_OK
        VMMCall Get_Sys_VM_Handle
        VxDCall SHELL_sysmodal_Message
        VMMCall _HeapFree,pTitle,0
        VMMCall _HeapFree,pMessage,0
        xor eax,eax
    .endif
    ret
EndProc OnDeviceIoControl
VxD_PAGEABLE_CODE_ENDS
end

Analysis:

We start from VxDLoader.asm.

    invoke CreateFile,addr VxDName,0,0,0,0,FILE_FLAG_DELETE_ON_CLOSE,0
    .if eax!=INVALID_HANDLE_VALUE
        mov hVxD,eax
        ....
    .else
        invoke MessageBox,NULL,addr Failure,NULL,MB_OK+MB_ICONERROR
    .endif

We call CreateFile to load the dynamic VxD. Note FILE_FLAG_DELETE_ON_CLOSE flag. This flag instructs Windows to unload the VxD when the VxD handle returned from CreateFile is closed. If CreateFile is successful, we store the VxD handle for future use.

        invoke MessageBox,NULL,addr Success,addr AppName,MB_OK+MB_ICONINFORMATION
        invoke DeviceIoControl,hVxD,1,addr InBuffer,8,NULL,NULL,NULL,NULL
        invoke CloseHandle,hVxD
        invoke MessageBox,NULL,addr Unload,addr AppName,MB_OK+MB_ICONINFORMATION

The program displays a message box when the VxD is loaded/unloaded. It calls DeviceIoControl with dwIoControlCode 1 and passes the address of InBuffer in lpInBuffer parameter, and the size of InBuffer (8) in nInBufferSize. InBuffer is a dword array of two elements: each element is the address of a text string.

    MsgTitle db "DeviceIoControl Example",0
    MsgText db "I'm called from a VxD!",0
    InBuffer dd offset MsgTitle
                  dd offset MsgText

Now we turn our attention to the VxD.
It processes only w32_deviceIoControl message. When w32_deviceIoControl message is sent, OnDeviceIoControl procedure is called.

BeginProc OnDeviceIoControl
    assume esi:ptr DIOCParams
    .if [esi].dwIoControlCode==DIOC_Open
        xor eax,eax

OnDeviceIoControl processes DIOC_Open code by returning 0 in eax.

.elseif [esi].dwIoControlCode==1
mov edi,[esi].lpvInBuffer

It also processes control code 1. The first thing it does is to extract the data in lpvInBuffer which are the two dwords passed in lpInBuffer of DeviceIoControl API. It puts the address to the dword array into edi for extraction. The first dword is the address of the text to be used as the message box title. The second dword is the address of the text to be used as the message box text.

        ;-----------------------------------
        ; copy the message title to buffer
        ;-----------------------------------
        VMMCall _lstrlen, <[edi]>
        inc eax
        push eax
        VMMCall _HeapAllocate,<eax,HEAPZEROINIT>
        mov pTitle,eax
        pop eax
        VMMCall _lstrcpyn,<pTitle,[edi],eax>

It calculates the length of the message box title by calling VMM service _lstrlen. The value in eax returned by _lstrlen is the length of the string. We increase the length by 1 to take into account the terminating NULL. Next we allocate a block of memory large enough to hold the string with its terminating NULL by calling _HeapAllocate. HEAPZEROINIT flag instructs _HeapAllocate to zero out the memory block. _HeapAllocate returns the address of the memory block in eax. We then copy the string from the address space of the win32 app into the memory block we allocated. We do the same operation on the text string that we will use as the message box text.

        mov edi,pTitle
        mov ecx,pMessage
        mov eax,MB_OK
        VMMCall Get_Sys_VM_Handle
        VxDCall SHELL_sysmodal_Message

We store the addresses of the title and message into edi and ecx, respectively. Put the desired flag into eax, obtain the VM handle of the system VM by calling Get_Sys_VM_handle and then call SHELL_Sysmodal_Message. SHELL_SysModal_Message is the system modal version of SHELL_Message. It freezes the system until the user responds to the message box.

VMMCall _HeapFree,pTitle,0
VMMCall _HeapFree,pMessage,0

When SHELL_Sysmodal_Message returns, we can free the memory blocks by calling _HeapFree.

Conclusion

DeviceIoControl interface makes it ideal to use a dynamic VxD as a ring-0 DLL extension to your win32 application.

[Iczelion's Win32 Assembly Homepage]