Troubleshooting KDC 7 event errors when no one else can

Finding the cause of errors logged as Event ID: 7 Source: KDC can be tricky business, especially when the error leads you down the wrong path. These steps will help you find the real cause – and fix it.

This Content Component encountered an error

The Active Directory error Event ID: 7 Source: KDC -- or KDC 7, as we'll refer to it in this article -- can be difficult to troubleshoot. Not only does the problem appears as an event ID that doesn't show up on many Internet searches, but the event details point to the wrong source of the problem.

Fortunately, there are several steps you can take to determine the real cause. I recently encountered the error in my environment, where we had a large multi-domain forest, and several domain controllers (DCs) in one domain were logging Event ID: 7 Source: KDC, as shown below:

Event Type: Error
Event Source: KDC
Event Category: None
Event ID: 7
Date: 08/01/2009
Time: 16:05:22
User: N/A
Computer: Corp-DC3

Description: The Security Account Manager failed a KDC request in an unexpected way. The error is in the data field. The account name was
wilnop and lookup type 0x0.

Note that the event data in words is c0000017. This equates to:

# for hex 0xc0000017 / decimal -1073741801 : STATUS_NO_MEMORY ntstatus.h
# {Not Enough Quota}
# Not enough virtual memory or paging file quota is available
# to complete the specified operation.

These errors appeared inconsistently – at random times on random days -- and occasionally 50 or so would occur in one second. This happened for several months, with the frequency increasing as time went on.

The data in the event suggested the problem was memory, even though the DCs were running on 64-bit machines with 12 GB of RAM. Therefore, to determine the true cause I gathered Performance Monitor (PerfMon) data and event logs from some of the DCs with this error, collecting all the standard counters including:

  • Cache
  • LogicalDisk
  • Physical Disk
  • Memory
  • Network Interface
  • Processor
  • Server
  • ServerWorkQueues
  • System
  • NTDS
  • Process -- LSASS

Note that because this is a DC, I included the NTDS object counter and the process object for LSASS.

Since this was a random error and the events happened in only a few seconds, I configured PerfMon to save data in 52 MB files.

The observation was that although LSASS was using a lot of memory, the memory was not exhausted, and there was nothing unusual with the LDAP counters (LDAP client sessions, LDAP bind time, etc.).

Analysis of KDC 7 events

This type of event definitely means a resource is being depleted -- you just have to figure out which one. There are several causes of KDC 7 events and different ways to resolve them.

  1. If the KDC 7 event is logged when the DC is shut down, you can apply the hotfix in Microsoft Knowledge Base article 973667. This hotfix forces the correct error to be logged to prevent application failure.
  2. If the Systems Management Server (SMS) inventory agent -- or any similar third-party application with an inventory agent -- runs too frequently, it can cause KDC 7 events to be logged. In this case you should modify the schedule so that inventory is not taken as often.
  3. Third-party monitoring tools that put a load on the Active Directory to gather data could push the performance over the edge and cause KDC 7 events.

In my case, InTrust and DirectoryAnalyzer from Quest Software were installed. InTrust generates its own event log and writes events when the Active Directory is accessed. Looking at the InTrust event log, scores of the event existed:

11/13/2009 8:39 Warning  None  1    ITAD Directory Changes    Corp-DC24
       Corp\Test123$       Failed attempt to modify AD object.

There was an exact correlation between these events and the KDC 7 events.

Disabling InTrust didn't solve the issue -- but disabling the DirectoryAnalyzer agent did. Ironically, Quest's InTrust auditing tool gave me the proof I needed of the problem. Without it, we would have had to relegate to an ADPlus dump.

In this case, while it appeared that DirectoryAnalyzer caused the KDC 7 events, it wasn't the application itself that was at fault-- it was the overhead that was being added to an already busy Active Directory.

What to do if you encounter KDC 7 events

Follow these steps to determine the cause of the KDC 7 events in your environment:

  1. If the events are logged after a reboot, review Microsoft Knowledge Base article 973667. If it applies, install the hotfix.
  2. Look for running client inventory applications. These apps could be taking inventory too frequently, resulting in a heavy load being put on the DC. Decreasing this frequency should in turn lighten the load.
  3. Look for any running third-party AD monitoring tools. These tools may be putting an extra burden on the directory. Disable them to see if the KDC 7 events go away or if things improve.

If steps 1 through 3 don't work:

  1. Gather performance data with PerfMon:

a. Add All counters and All instances for each of the following objects:

    •  
      • Memory
      • Network Interface
      • Objects
      • Paging File
      • Physical Disk
      • Process (collect only _total and LSASS)
      • Processor
      • Redirector
      • Server
      • NTDS
      • System

b. Frequency: This setting depends on how frequently the KDC 7 events occur in your environment. Since in my case they occurred in about one second, I collected data in two second intervals.

c. Logs: Set the logs to 50 MB. This helps because you'll have smaller logs to run through tools like Performance Analysis of Logs, it minimizes the data that needs to be analyzed when the event happens, and it allows you to delete unnecessary log files so the disk doesn't fill up.

5. Gather a network trace. I like to collect without filters on the DC. By using the command line version of Network Monitor, you can specify timing parameters so only small files are captured.

Use the command below from a scheduled task. Run continuously and monitor the capture files, and delete as needed when no KDC 7s occur:

NMCAP.EXE /network "HP Network Team #1"
/DisableConversations /MinDiskQuota:50M
/Capture /File X:\path\DC24.chn:50M
    with P:\path being the path to whatever drive/path you want to use.

The MinDiskQuota can be whatever size you want. In this case, a small file was used since the KDC 7 events occurred over such a small time period. The chn:50M chains the logs together so analysis is easy to follow from one file to another.

There is no magic answer here, but collecting data is the best way to determine what is causing the KDC 7 events. As a last resort, gather an ADPlus dump. Just remember that in a case like the one described here, a dump like that would be challenging since the events were random and only lasted for a second or two.

ABOUT THE AUTHOR
Gary Olsen is a systems software engineer for Hewlett-Packard in Global Solutions Engineering. He authored Windows 2000: Active Directory Design and Deployment and co-authored Windows Server 2003 on HP ProLiant Servers. Gary is a Microsoft MVP for Directory Services and formerly for Windows File Systems.

This was first published in March 2010

Dig deeper on Microsoft Active Directory Tools and Troubleshooting

Pro+

Features

Enjoy the benefits of Pro+ membership, learn more and join.

0 comments

Oldest 

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

-ADS BY GOOGLE

SearchServerVirtualization

SearchCloudComputing

SearchExchange

SearchSQLServer

SearchWinIT

SearchEnterpriseDesktop

SearchVirtualDesktop

Close