SETUP INSTRUCTIONS FOR WINDOWS OS INTERNALS LABS

Send questions to daves@solsem.com

BASIC SETUP:
  • Any 32-bit or 64-bit Windows client or server installation (Vista or later preferred - if you use XP there will be several experiments that won't work)
  • One computer can be shared by 2 students, but 1 per student is better
  • The account that you will log into must be a member of the local administrators group and have the Debug Programs privilege

OPTIONAL: RUNNING THE EXPERIMENTS IN A VIRTUAL MACHINE

NOTE: Labs are nondestructive (except one lab in the crash dump analysis section, which you can choose to skip during class), but can be done in a virtual machine (even on a Mac) if you prefer.

You can use one of Microsoft's free Virtual PC products (can only run 32-bit OS's):

Or you can use one of these VMWare products (allows installing 32-bit and 64-bit guests):

There are other virtual machine products as well (e.g. Virtualbox).

If you don't have a licensed copy of Windows to install in the virtual machine, you can download and use evaluation versions or ready to go VHDs (virtual hard disks) for XP, Server 2003, Windows 7, or Server 2008 R2 (note: since Server 2008 R2 is only available in 64-bit form, you cannot use Microsoft Virtual PC or Virtual Server - but you can run it under Hyper-V or VMWare).

TOOLS SETUP:

  1. Install the Debugging Tools for Windows:

    Public location: http://www.microsoft.com/whdc/devtools/debugging/default.mspx
    Microsoft internal location: http://dbg

    NOTE: the public Debugger Tools web page requires downloading the entire Windows Driver Kit (WDK), even though you will only install the debugging tools - do NOT install the older versions that are still available for separate download, or the old version that comes with the SDK.
     
  2. Download and unzip the Sysinternals tools suite (this is a single zip file with the majority of the Sysinternals tools). The class notes assume they are unzipped to c:\sysint, but you can put them anywhere you choose:

    http://www.microsoft.com/technet/sysinternals/utilities/sysinternalssuite.mspx
    (Microsoft employees can get the latest unreleased version at \\redmond\files\SYSINTERNALS\LBI\Latest\)
     
  3. Download and unzip to c:\sysint the Blue Screen Screen Saver (the one tool not included in the Sysinternals Tools Suite):

    http://www.microsoft.com/technet/sysinternals/Utilities/BlueScreen.mspx
     
  4. Download and unzip to c:\sysint the tools that are referenced by the book Windows Internals:

    http://download.sysinternals.com/Files/Notmyfault.zip
    http://download.sysinternals.com/Files/Testlimit.zip
    http://download.sysinternals.com/Files/Accvio.zip
    http://download.sysinternals.com/Files/Iopriority.zip
     
  5. Download and unzip http://www.solsem.com/solsem.zip into c:\solsem (these are additional files and tools used for various demonstrations and labs)
     
  6. (Only for IT Pro version of class) Install the Windows Performance Toolkit. The MSIs (Microsoft installer files) containing these tools are installed as part of the Win32 Development Tools portion of the Windows SDK. You can use the Web installer to install just the Win32 Development Tools portion of the Windows SDK. After you install this part of the SDK, go to the C:\Program Files\Microsoft SDKs\Windows\v7.0\Bin folder and run the appropriate WPT MSI file (e.g. wpt_x86.msi for 32-bit systems, wpt_x64.msi for x64 64-bit systems, etc.).

  7. If you are running Vista or later, download these tools by Alex Ionescu and unzip into c:\solsem (again, these tools do not run on XP or 2003):
      http://www.winsiderss.com/tools/winsiderss-tools.zip

  8. Download Dependency Walker from www.dependencywalker.com into c:\sysint (you may already have this tool on your system)
     
  9. Go to the Startup and Recovery Settings (right click on Computer, click Advanced system settings, on Advanced Tab, click "Settings" under Startup and Recovery). Uncheck "Automatically restart" and make sure dump type is "Kernel memory dump" (not minidump).

  10. Add a system-wide environment variable for the symbol path:

    1. Right click on My Computer->Properties, click on Advanced Tab
    2. Press Environment variables button
    3. Press "New" button under System Variables section and enter:
            Variable name: _NT_SYMBOL_PATH  (must be upper case)      Value:
                 Non-Microsoft employees:
                       srv*c:\symbols*http://msdl.microsoft.com/download/symbols
                 Microsoft employees:
                       srv*c:\symbols*\\symbols\symbols

    Or on Vista and later, you can use the new SETX command line command:

       Public symbol server:
    setx _NT_SYMBOL_PATH "srv*c:\symbols*http://msdl.microsoft.com/download/symbols" /M

       Microsoft-internal symbol server (for Microsoft employees):
           setx _NT_SYMBOL_PATH "srv*c:\symbols*\\symbols\symbols" /M

    For Microsoft internal Windows 7 systems, "srv*c:\symbols" is enough - it will figure out the symbol path internally.

    If you are running an interim build that does not have symbols on the symbol server, please add the symbol location to your symbol path. Find the build directory at \\winbuilds\release\winmain (or other directory, if it's taken from another build lab). From that directory, find the version you installed. Then open "symbols.pri" under that directory.

    For example, \\winbuilds\release\winmain\7071.0.090326-1750\x86fre\symbols.pri

    NOTE! It is not necessary to download and install the symbols - please use the symbol server (configured above) to automatically download symbols as needed. For more information on symbols, see http://www.microsoft.com/whdc/devtools/debugging/debugstart.mspx (or for MS internal, http://dbg)

  11. If XP 32-bit (5.1 kernel) or earlier, run Gflags (in the Debugging Tools folder) and enable pool tagging, press Apply, then OK, and reboot (not necessary for XP 64-bit, Server 2003, Vista, or Server 2008).

  12. Enabled crash from keyboard (doesn't work with USB keyboards).

    Run one of two .REG files in Solsem.zip:
    - crashon-rightctrl-scrlk-scrlk.reg (default keystroke sequence: hold down right control button and press Scroll Lock twice)
    - crashon-right-alt-space-space.reg (alternate choice: hold down right ALT button and press space bar twice)

    For details, see this KB article.
     
  13. (Optional) Run \Solsem\Logonhlp.reg and copy \Solsem\Logonhlp.exe to \Windows\System32 (or whatever %Systemroot% is on your system)
  14. NOTE FOR THOSE USING BITLOCKER ENCRYPTION
    Some class activities such as booting in debugging mode or changing data execution prevention (DEP) configuration settings will cause a system configured with Bitlocker to prompt you to enter the recovery key to boot. To avoid this, please either disable or suspend Bitlocker before class (see below).
    BUT JUST TO BE SAFE, please bring your recovery keys.
    For Vista systems, you can turn off Bitlocker and re-enable it after class. On Windows 7, you can temporarily suspend Bitlocker on the Bitlocker control panel page as shown below and then re-enable it after calss:


  15. Many experiments in the class use the kernel debugger to inspect internal system state. You can either use the built in ability to do local kernel debugging (which on Vista and later requires booting in debugging mode) or use the Sysinternals LiveKD tool. There are some commands that may fail in LiveKD, but for the most part it is fine for class use.

    Whichever you choose, please test in advance that you can perform kernel debugging (requires the Debug privilege, normally granted to administrators).

    USING LIVEKD:
    Open an elevated command prompt and set your current directory to the Debugging Tools: for 32-bit systems go to C:\Program Files\Debugging Tools for Windows, or on 64-bit systems, C:\Program Files\Debugging Tools for Windows (x64) (NOTE: Microsoft internal debugger installations go by default into C:\Debuggers).

    Then run Livekd from the Sysinternals folder with the -w switch, e.g. "\sysint\livekd -w". This should launch Windbg in a way that lets you use kernel debugger commands to inspect system state (note: LiveKD uses a trick such that the debugger thinks it is looking at a crash dujmp, when in reality it is reading actual system memory).

    To make sure things are working (e.g. symbols are configured properly), type "!process" at the debugger prompt and make sure you get output similar to what is shown below.

    USING LOCAL KERNEL DEBUGGING:

    NOTE FOR Vista and later: you must boot in Debugging Mode to perform local kernel debugging (fyi, this breaks MPEG2/DVD video playback - this is not a bug but is there to prevent getting around digital rights management by using the kernel debugger). To boot in debugging mode, you can either press F8 during the boot process and choose Debugging Mode from the list of advanced boot options or configure the system to boot automatically in debugging mode (either run MsConfig, click on the Boot tab, then click the Advanced Options button, then check Debug or from a command prompt type "bcdedit /debug ON").

    NOTE: There is an odd case where enabling debugging still does not permit kernel debugging - try the workaround on Alex's blog.

    NOTE     FOR SKYPE USERS: If you run a program that performs a user mode debug breakpoint (like Skype does) while booted in debugging mode, your system will freeze - to work around this, you need to disable user mode exceptions (or just disable Skype from running automatically before you reboot in Debugging Mode). To do this, boot in debugging mode, open an elevated command prompt, go to the Debugging Tools folder, and type "kdbgctrl -du".  This disables kernel debugger user mode exception handling. However, this takes effect only for the current boot. To configure the system to do this permanently, type:
         bcdedit /debug on
       bcdedit /dbgsettings debugport:1 baudrate:115200 /noumex
    NOTE TO MICROSOFT EMPLOYEES USING INTERNAL DEBUGGER BUILDS: If you are using an internal build of the debugger tools and are having problems getting local kernel debugging to work, please try using the latest released version and use the public symbols (vs the internal symbols).

    Steps to test local kernel debugging:
    1. Run Windbg from the Debugging Tools (on Vista and later, make sure you run it elevated with administrative credentials if you have UAC enabled)
    2. Click on File->Kernel Debug, Click on Local tab & click OK
    3. If symbols are configured properly, a command window should open up
    4. Type "!process" at the "lkd>" prompt to make sure symbols are loaded and configured.  This should show information for the Windbg process that starts like this:

       
    5. OPTIONAL: Force the download of other kernel symbol files to your local symbol cache (typically c:\symbols) by typing ".reload /f" (if you are sure that you will have network connectivity during the class, you can skip this step and let the symbols download on demand as they are referenced during the class).

      NOTE: It is normal to get symbol loading errors for third party device drivers, as their symbols are not available on the symbol server.
       
  16. Configure symbols for Process Explorer and Process Monitor

    Run Process Explorer and Process Monitor.  In both tools, click on Options->Configure Symbols.  Change the Dbghelp.dll path to reference the one in your Debugging Tools folder and make sure the symbol path is set (see step 10).  NOTE: you cannot use the Dbghelp.dll in \Windows\System32 as it does not support the symbol service; you must use the one in the Debugging Tools folder.

    An example configuration dialog (for a 64-bit system) using public symbols would be:


     
  17. In the Process Explorer's list of processes, double click on the process called "System" (usually 4th in the list) and click on the Threads tab (there may be a delay while symbols are downloaded).

    When the list of threads are displayed, to confirm symbols were downloaded properly, sort by the Start Address column and scroll down until you see threads with start addresses in the form "ntoskrnl.exe!xxx" or ""ntkrnlpa.exe!xxx" - make sure you do NOT see any "+0x" after any of these entries.  This is an example of a correct output:



    If you see entries like "ntoskrnl.exe!yyyyyy+0xnnn" for most of the Ntoskrnl/Ntkrnlpa lines, then your symbols are not configured correctly. For example, this kind of display indicates kernel symbols are NOT correctly configured:


     
  18. Finally, double click on several other processes to force the download of other user mode symbols: Explorer.exe, a few Svchost.exe processes, Csrss.exe, Winlogon.exe, etc.  The reason for doing this is to get a variety of other user mode .EXE symbol files cached on your machine for use during the class. After doing the above, you should see a number of subfolders under c:\symbols -- these folders contain symbols for the various images referenced.