In Simple Active Directory tricks: Event logs, we discussed a technique for using Microsoft Excel as a tool to organize and search data in event logs for troubleshooting purposes. This is a great way to sift through a lot of data to find clues that are of interest in resolving a problem. Of course, this assumes that you have sufficient data to give you the results you want. After all, if you don't have the right data, a tool to sort...
it won't help.
This article will discuss the benefits of verbose logging, popular verbose logging tools and some ways to turn up verbose logging to achieve maximum benefits.
What is verbose logging?
Verbose logging is a computer logging mode that records more information than the usual logging mode. Verbose means "using more words than necessary." Verbose logging options are usually enabled specifically for troubleshooting because they create large log files and can slow down performance.
There are several ways to turn up verbose logging for Active Directory-related errors. I've always believed that in most cases you can never have too much data for troubleshooting. You can always dismiss extraneous data, but you can't analyze something you don't have.
Popular verbose logging tools
Let's look at some popular diagnostic tools, the syntax for verbose mode, when to use the tool and how to identify the switch to enable verbosity. While these first few tools are pretty basic, keep reading -- I'll get to the good stuff. By the way, I never run these tools without enabling verbose logging.
DCDiag.exe is a great tool for getting domain-related information on a domain controller. It analyzes Flexible Single Master Operations (FSMO) roles, replication test, machine account viability, Service Principle Name (SPN) registration, DNS tests and more. DCDiag's output lists events that also occur in the event logs, but they are organized so that the replication errors are in the replication section, the DNS errors are in the DNS section and so on. It does save time as opposed to searching event logs, and it adds verbiage on errors that sometimes isn't found anywhere else.
One of the new cool switches is DCDiag/Test:DNS /e /s, which gives you an end-to-end analysis of the DNS infrastructure and tests each DNS server with seven tests. It is only available in Windows 2003 SP1 support tools on the Microsoft download site.
Enable Verbose logging: Dcdiag/v
It is included and run in verbose mode in MPS Reports. You can run it remotely and you can redirect output to a text file by appending the command with >file.txt.
Netdiag.exe is excellent for retrieving network-related data, which includes the IPConfig of the computer, Netbios name registration and so forth. This is handy because it verifies the DNS server that the DC is pointing to. Thus, with Netdiag output from each DC, you can construct the actual DNS structure of the domain easily.
Enable verbose logging: Netdiag/v
It is included and run in verbose mode in MPS Reports. It can only be run locally and you can redirect output to a text file by appending the command with >file.txt.
Repadmin doesn't have a verbose mode, but it has a number of very helpful switches. They include:
/replsum /bysrc /bydest /sort:delta -- Provides an end-to-end summary of replication for every DC in every domain in the forest -- when the last successful replication took place and which ones are out of the tombstoneLifetime. Not run in MPS Reports output.
/showrepl (in Windows 2000, this was /showreps) -- Very powerful, easy-to-use command that gives you an instant report of all replication on a DC, inbound and outbound. Included in MPS Reports output and you can run it remotely.
/showobjmeta -- Dumps all attributes of a given AD object. Use this very powerful tool to determine when an object was modified and can be used as a poor man's way to determine when an object has replicated to all DCs.
Articles I've written for SearchWinIT.com on Repadmin include: Use command line tools to monitor Active Directory, Quick fix for a non-replicating DC and Best practices for Active Directory replication topology design.
Directory Service event log
Turning up verbose NTDS logging for additional output to the DS event log isn't quite as simple as a /v option on a command. There are, in fact, about 25 different functions in which you can turn up verbosity in addition to defining five different levels of verbosity. Turning up verbosity will add additional events to the DS event log that normally won't be there.
I recall in one instance that replication was broken and we were getting an event indicating "Internal Error." We enabled the "Internal" verbose option and obtained an additional event that provided the globally unique identifier (GUID) of the problem DC. We were able to find the offending DC and repair the problem. Without the additional events, we wouldn't have found it so easily.
To enable verbose NTDS logging, open the Registry with regedt32 or regedit (they now perform the exact same function). Go to HKLM\System\CCS\Services\NTDS\Diagnostics. Click on this key and observe the values as shown in the text box: NTDS diagnostic settings. These values describe various NTDS functions, such as Name Resolution (DNS), Global Catalog, Knowledge Consistency Checker, Replication, etc. I have never seen any documentation on what kind of information you get in each of these areas, other than the name. The "Internal" value provides additional information on events that indicate internal errors. I usually just look through the list and see which functions match the problem. If I have a GC that isn't replicating, I might enable verbose logging on Global Catalog, Replication and Name Resolution. Since Name Resolution is at the core of replication, I usually enable verbosity on that as well.
Each of these values can accept a data value of 0 to 5. The default value is zero. The higher the value, the more verbose the logging will be. I usually set the value data at three. Occasionally I will crank it up to five if three doesn't give me what I want.
After setting the appropriate values, clear the DS event log and reproduce the problem. Examine the DS event log for the additional events.
Important: After you have completed your data gathering, be sure to go back and reset all the values you defined to zero. Leaving them at increased verbosity will fill your event logs with a lot of data that you won't need and may cause it to overwrite more important data if you have circular logging enabled.
This technique is extremely valuable for providing specific information to troubleshoot in replication-related problems. Now that you have as much data as you can gather, you can import it into an Excel spreadsheet and use the techniques I described in my previous article to analyze the data. Also refer to Microsoft KB 314980.
Gary Olsen is a systems software engineer for Hewlett-Packard Co. in Global Solutions Engineering. He authored Windows 2000: Active Directory Design and Deployment and co-authored Windows Server 2003 on HP ProLiant Servers.