Q

Troubleshooting common Windows Server deduplication problems

Learn how to fix an application having problems with deduplicated data, as well as when you should turn off dedupe.

Q. What happens if an application has problems with deduplicated data? Can I turn Windows Server deduplication

off?

Storage needs have increased dramatically as new applications work with data-intensive file types ranging from customer databases, email archives and PDFs to image libraries and streaming media files. Data deduplication has emerged as a powerful technology for mitigating enterprise storage demands by identifying and removing redundant data elements, or chunks, without altering or compromising the data. Windows Server 2012 R2 provides expanded deduplication capabilities, but IT professionals must consider the proper use and suitability of deduplication to ensure that application performance and system performance benefit in a predictable manner.

Once enabled and configured on a Windows Server 2012 R2 platform, data deduplication is typically delivered as a background job periodically initiated by the server; this is slightly different from the inline deduplication capabilities native to some storage subsystems. Consequently, when Windows Server deduplication is applied, some applications or file types may not perform well, such as files that require continuous access, change often or require active I/O patterns. Examples include Hyper-V host systems, SQL database files, Exchange servers, Windows Server Update Services (WSUS) and files larger than 1 TB.

The underlying problem here is that files that change frequently cannot be efficiently deduplicated. The server must deliver significant computing resources to process a deduplication job -- and immediate changes to the application's files only undo the deduplication. This isn't an issue for dedicated storage subsystems such as EMC's that are designed to provide continuous deduplication, but it's a computing burden for conventional servers. In actual practice, deduplication is best applied to files that have reached some age (more than zero days) without changing. If you must select a short deduplication cycle, be certain to benchmark the application before and after deduplication is enabled in order to compare performance and ensure that the application's performance is not adversely affected.

There are several options available when an application or file access is adversely impacted by Windows Server deduplication. First, an IT administrator may elect to change the deduplication frequency by changing the schedule or opting for manual deduplication jobs. In addition, job options like StopWhenSystemBusy will halt deduplication if the job interferes with the server's workload, while the Priority options can be used to prioritize deduplication jobs. The Expand-DedupFile cmdlet can expand (or un-deduplicate) specific files if needed for compatibility or performance.

In most cases, scheduling and other options can improve performance without it being necessary to disable deduplication, but there are times when deduplication simply needs to be disabled. Deduplication can be removed on a volume by running a Start-DedupJob cmdlet with "Unoptimization" as the job type.

This was first published in February 2014

Dig deeper on Microsoft Windows Data Backup and Protection

Pro+

Features

Enjoy the benefits of Pro+ membership, learn more and join.

Have a question for an expert?

Please add a title for your question

Get answers from a TechTarget expert on whatever's puzzling you.

You will be able to add details on the next page.

0 comments

Oldest 

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

SearchServerVirtualization

SearchCloudComputing

SearchExchange

SearchSQLServer

SearchWinIT

SearchEnterpriseDesktop

SearchVirtualDesktop

Close