News Stay informed about the latest enterprise technology news and product updates.

Berkeley National Lab wrestles with Windows patch management

The nation's oldest government laboratory is in the midst of testing two software applications to better manage thousands of Windows desktops, and to keep from drowning in a flood of patches.

There are 5,000 PCs spread throughout dozens of buildings on the 130-acre campus of the Ernest Orlando Lawrence Berkeley National Laboratory.

Berkeley Lab's bottom line
Though software vendors can grant government discounts of up to 40%, price is still a main concern for Berkeley Lab's Active Directory project rollout coordinator Douglas Spindler. Why? Because the lab is subject to the whim of government spending. Like other national laboratories, Berkeley Lab, founded in 1931, is primarily funded by the U.S. Department of Energy, which in turn pays the University of California to manage the lab. The 2002 IT budget was $73 million, and Spindler suspects the budget will decrease for this year.

One network admin is in charge of keeping the security of Windows PCs in each of the lab's 12 main departments up to date. Using Microsoft's Baseline Security Analyzer, Douglas Spindler knows which departments are not up to speed.

"Everybody's behind," said Spindler, Berkeley Lab's Active Directory project rollout coordinator. Some machines, like those used by administrative assistants, go untouched. Can't blame the admins. Microsoft releases security updates constantly, sometimes up to five in one week.

The elite laboratory, where scientists perform research in areas ranging from particle physics to life sciences (and have, over the years, won nine Nobel Prizes), is struggling with the same problem stumping IT pros the world over: how to best deploy security patches for Windows. Today, security is of paramount importance and, for the country's oldest national laboratory, that means plugging "leaks" in the network. Microsoft, in an effort to shore up its complex operating systems and its reputation, has been delivering a steady stream of security patches that is overwhelming IT departments.

"There are too many security holes in Windows, and we need to take care of them," Spindler said in a telephone interview, speaking over the hum in the lab's basketball court-sized server room in the Berkeley hills, above the grounds of the University of California campus. Six air-conditioning units cool 12 rows of servers; Spindler said he was wearing a parka.

David versus Goliath

He and colleagues are testing two software products that will allow admins to remotely and methodically install Windows security patches across the Berkeley Lab network. The first is Software Update Services (SUS), a free application from Microsoft. The second is ZENworks for Desktops 4 from Novell, which plays traffic cop but also can inventory network equipment. Spindler and colleagues began testing the two applications in December and likely will make a decision in February.

One reason Spindler is taking a closer look at SUS and ZENworks is because they allow pros to control the flow of security patches. Lab pros must be able to control patch deployment to avoid, for example, hefty patches clogging network bandwidth, which in turn interrupts the lab's day-to-day work and science experiments.

"A hotfix can hit a snag and delay a few thousand people until someone fixes it," Spindler said.

Not all patches need to be installed, said Karen Christian, founder of In Touch Systems, a consulting firm in San Marcos, Calif. Some Windows patches are more significant than others, and some are not thoroughly tested by Microsoft and can disrupt PCs, she said.

"When people sit down at work, they want to get to work," Christian said. "They don't want to reboot their PC."

Running the gamut
PCs at Berkeley Lab run a variety of operating systems, including Linux, Macintosh and DOS 6.22, which dates back to the early 1990s. However, most run Windows NT 4.0, Windows 98, Windows 2000 or Windows XP. The types of users also run the gamut, from administrative assistants using Windows PCs for e-mail and Internet browsing, to scientists relying on PCs for experiments. Though office assistants' PCs are not directly linked to experiments, their computers can be easy openings for hackers. Each computer at the lab must be up to date.


Spindler set up SUS on a Windows Server 2003, Release Candidate 2. The server connects to Microsoft's Web site once a day to download the available security patches. Berkeley Lab pros review and approve the patches, and a few dozen users are prompted to begin security updates that typically take less than a minute. Spindler intends to increase the SUS test pool to 50 PCs in February.

Though free, SUS comes with limitations. It only works with Windows 2000 and later versions, can only send out hotfixes -- no service packs or Microsoft Office fixes -- and has no reporting capabilities, Spindler said.

"It is extremely difficult to tell if a machine has been successfully updated," he said.

In addition, Microsoft's SUS instructions are 90 pages long and, at times, are more confusing and esoteric than helpful, he said.

On the other hand, ZENworks, which the lab is testing on 10 PCs, can handle all Windows operating systems back to NT 4.0 and can remotely install security patches without help from users. In addition, ZENworks can inventory software and hardware on the network, a handy feature that would allow the Berkeley Lab pros to track software, software licenses and hardware equipment on the network. However, the rollout of ZENworks would take a month, whereas SUS would take only a few hours.

Decisions, decisions

Though Spindler said ZENworks can "easily replace Microsoft SUS server," he suspects Berkeley Lab will use a combination of the two, with SUS perhaps playing a backup role.

Berkeley Lab must balance the risk of security holes with the cost of plugging them, he said. That ruled out Microsoft's Systems Management Server. It's too expensive, not to mention too complicated, said Spindler, who has viewed demonstrations of the application at Microsoft events.

Dig Deeper on Windows Server troubleshooting

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.