This article is an excerpt from the Deploying and Managing Exchange Server 2013 High Availability ebook.
A lagged database copy is a passive database copy in a database availability group that has a delayed log replay time configured.
Normally a passive database copy will replay the transaction log data into the database immediately, so that the passive database copy is as up to date as possible.
With a lagged database copy the administrator sets a delay on the log replay, so that the database copy “lags” behind the others in terms of the latest database changes. This lag interval specifies the amount of time between when a transaction log file is generated and when it is replayed into the passive database copy. The default lag interval is 0 and the maximum lag interval is 14 days.
The purpose of a lagged copy is to provide the ability to recover the database from an earlier point in time if some kind of database fault occurs, such as logical corruption.
Configuring Lagged Database Copies
The replay lag interval for a lagged database copy is configured using Set-MailboxDatabaseCopy, and the value is in the format “days.hours:minutes:seconds”. For example, a value of “7.0:0:0” means 7 days.
It is common to choose the least preferred database copy to be the lagged copy, so in an environment with four database copies the copy with activation preference of 4 would be set as the lagged copy.
To select all database copies that have an activation preference of 4 we can use Get-MailboxDatabaseCopyStatus.
[PS] C:\>Get-MailboxDatabaseCopyStatus * | where {$_.ActivationPreference -eq "4"}
Name Status CopyQueueLength ReplayQueueLength LastInspectedLogTime ContentIndexState
---- ------ --------------- ----------------- -------------------- -----------------
DB02\SYDEX1 Healthy 0 0 23/09/2014 10:53:37 PM Healthy
DB03\SYDEX2 Healthy 0 0 23/09/2014 10:45:59 PM Healthy
DB04\MELEX1 Healthy 0 0 23/09/2014 10:47:08 PM Healthy
DB01\MELEX2 Healthy 0 0 23/09/2014 10:45:02 PM Healthy
Piping that output to Set-MailboxDatabaseCopy will configure the replay lag interval.
[PS] C:\>Get-MailboxDatabaseCopyStatus * | where {$_.ActivationPreference -eq "4"} | Set-MailboxDatabaseCopy -ReplayLagTime 7.0:0:0 Real World: When you set the replay lag time you will see a warning about the SafetyNetHoldTime as well. It is always recommended to set Safety Net hold time to the same value or greater value than the replay lag time.
In addition to setting the replay lag time you can also prevent a lagged database copy from being automatically activated during a database failover scenario. To block automatic activation for a database copy use the Suspend-MailboxDatabaseCopy cmdlet.
[PS] C:\>Get-MailboxDatabaseCopyStatus * | where {$_.ActivationPreference -eq "4"} | Suspend-MailboxDatabaseCopy -ActivationOnly When automatic activation has been blocked in this manner the database copy can only become active after manual intervention by an administrator.
Real World: Blocking automatic activation for a database copy reduces the number of database copies that are available for Active Manager to attempt to mount during the Best Copy and Server Selection process. It is a decision for your organization whether the preservation of the lagged database copy by blocking automatic activation is more important than having the maximum number of available database copies available to mount in a failover scenario.
Using Lagged Database Copies in Recovery Scenarios
Lagged copies can be used for recovery in a variety of ways.
- Activate a lagged database copy by replaying all uncommitted log files. In this scenario a decision is made to replay all of the log files in the replay queue into the database and bring it online.
- Activate a lagged database copy to a specific point in time. In this scenario some of the log files are removed from the server to prevent them from being replayed into the database copy as it is brought online.
- Activate a lagged database copy and use Safety Net for recovering lost data. In this scenario the database is mounted without replaying transaction log from the replay queue, and messages are resubmitted to the database from Safety Net.
In each of these scenarios there is an optional (but recommended) step to make a copy of the lagged database files before activating it, so that you are still able to use the lagged copy at a later stage if you have a different recovery scenario arise.
Activating a Lagged Database Copy by Replaying All Log Files
A lagged database copy can be activated by replaying all of the uncommitted transaction log files (those that are in the replay queue) into the database and then mounting it. This effectively “catches up” the database copy to the others.
The amount of time it takes to commit all of the outstanding transaction log files to the database will depend on the length of the lag interval and the amount of transaction log data that has been generated in that space of time.
First, identify the lagged copy of the database. In this example of database DB01 the lagged copy is DB01\MELEX2.
[PS] C:\>Get-MailboxDatabaseCopyStatus DB01 | Where {$_.ReplayLagStatus.Enabled -eq $true}
Name Status CopyQueueLength ReplayQueueLength LastInspectedLogTime ContentIndexState
---- ------ --------------- ----------------- -------------------- -----------------
DB01\MELEX2 Healthy 0 361 26/09/2014 11:46:31 PM Healthy
Suspend the lagged database copy using Suspend-MailboxDatabaseCopy.
[PS] C:\>Suspend-MailboxDatabaseCopy DB01\MELEX2
Make a backup of the database and log files for this database copy. You can use a backup application or simply copy them to another location.
A suspended database copy can’t be made active, so we need to resume the lagged database copy again. It may take a few minutes afterwards for the content index to return to a healthy state as well.
[PS] C:\>Resume-MailboxDatabaseCopy DB01\MELEX2
Finally, activate the lagged database copy. Note that this uses the same Move-ActiveMailboxDatabase cmdlet as a normal database switchover, with the additional –SkipLagChecks switch.
[PS] C:\>Move-ActiveMailboxDatabase DB01 –ActivateOnServer MELEX2 -SkipLagChecks
When all of the log files have been replayed the database copy will mount and become the active database copy.
[PS] C:\>Get-MailboxDatabaseCopyStatus DB01\MELEX2 Name Status CopyQueueLength ReplayQueueLength LastInspectedLogTime ContentIndexState ---- ------ --------------- ----------------- -------------------- ----------------- DB01\MELEX2 Mounted 0 0 Healthy
The database copy still has the same replay lag interval configure, it is just not in effect as long as the database copy is active. If the active database copy is moved to another DAG member then the replay lag interval will come into effect again and the replay queue length will begin increasing for the lagged database copy.
Activating a Lagged Database Copy to a Point in Time
Activating a lagged database copy to a specific point in time follows a different process, and is often used as a way to make a copy of the database to mount as a recovery database, rather than activate the lagged copy itself.
First, identify the lagged copy of the database. In this example of database DB02 the lagged copy is DB02\SYDEX1, so this procedure should be performed on server SYDEX1.
[PS] C:\>Get-MailboxDatabaseCopyStatus DB02 | Where {$_.ReplayLagStatus.Enabled -eq $true} | ft -auto
Name Status CopyQueueLength ReplayQueueLength LastInspectedLogTime ContentIndexState
---- ------ --------------- ----------------- -------------------- -----------------
DB02\SYDEX1 Healthy 0 391 27/09/2014 12:16:08 AM Healthy
Suspend the lagged database copy using Suspend-MailboxDatabaseCopy.
[PS] C:\>Suspend-MailboxDatabaseCopy DB02\SYDEX1
Make a backup of the database and log files for this database copy. You can use a backup application or simply copy them to another location.
Look at the log file timestamps to determine which log files were created after the point in time you want to recover to.
Move the log files after the time that you’re recovering to into another folder. Don’t delete them, just in case you need them, but they do need to be moved to another folder.
Delete the .chk file from the transaction log folder.
Using ESEUtil in a CMD prompt we can see that the database file in a dirty shutdown state.
C:\>eseutil /mh C:\ExchangeDatabases\DB02\DB02.db\DB02.edb
Extensible Storage Engine Utilities for Microsoft(R) Exchange Server
Version 15.00
Copyright (C) Microsoft Corporation. All Rights Reserved.
Initiating FILE DUMP mode...
Database: C:\ExchangeDatabases\DB02\DB02.db\DB02.edb
DATABASE HEADER:
Checksum Information:
Expected Checksum: 0xe4311492
Actual Checksum: 0xe4311492
Fields:
State: Dirty Shutdown
So the next step is to perform a soft recovery using the log files that we’ve left in the log folder. Navigate to the log folder for the database and run ESEUtil with the /r switch and the log prefix (E00 in this example).
C:\>cd ExchangeDatabases\db02\DB02.log
C:\ExchangeDatabases\DB02\DB02.log>eseutil /r e00 /a
Extensible Storage Engine Utilities for Microsoft(R) Exchange Server
Version 15.00
Copyright (C) Microsoft Corporation. All Rights Reserved.
Initiating RECOVERY mode...
Logfile base name: e00
Log files:
System files:
Performing soft recovery...
Restore Status (% complete)
0 10 20 30 40 50 60 70 80 90 100
|----|----|----|----|----|----|----|----|----|----|
...................................................
Operation completed successfully in 31.859 seconds.
Check the database file again to verify it is in a clean shutdown state.
C:\>eseutil /mh C:\ExchangeDatabases\DB02\DB02.db\DB02.edb
Extensible Storage Engine Utilities for Microsoft(R) Exchange Server
Version 15.00
Copyright (C) Microsoft Corporation. All Rights Reserved.
Initiating FILE DUMP mode...
Database: C:\ExchangeDatabases\DB02\DB02.db\DB02.edb
DATABASE HEADER:
Checksum Information:
Expected Checksum: 0xf3f5ae3e
Actual Checksum: 0xf3f5ae3e
Fields:
State: Clean Shutdown
The database file can now be copied to another location and used as a recovery database for a restore operation.
The lagged database copy itself can be resumed as well. If you wanted to return it to the lagged state before the recovery process above was run you would need to restore the files you backed up at the start of this process before resuming the database copy.
[PS] C:\>Resume-MailboxDatabaseCopy DB02\SYDEX1
Activating a Lagged Database Copy Using Safety Net Recovery
Activating a lagged database copy using Safety Net recovery involves removing all unnecessary transaction log files from the log folder, so that only the bare minimum required for mounting the database still exist.
First, identify the lagged copy of the database. In this example of database DB03 the lagged copy is DB02\SYDEX2, so this procedure should be performed on the server SYDEX2.
[PS] C:\>Get-MailboxDatabaseCopyStatus DB03 | Where {$_.ReplayLagStatus.Enabled -eq $true} |
Name Status CopyQueueLength ReplayQueueLength LastInspectedLogTime ContentIndexState
---- ------ --------------- ----------------- -------------------- -----------------
DB03\SYDEX2 Healthy 0 379 27/09/2014 1:11:18 AM Healthy
Suspend the lagged database copy using Suspend-MailboxDatabaseCopy.
[PS] C:\>Suspend-MailboxDatabaseCopy DB03\SYDEX2
Make a backup of the database and log files for this database copy. You can use a backup application or simply copy them to another location.
Use ESEUtil to determine which log files are required by the database.
C:\>eseutil /mh C:\ExchangeDatabases\DB03\DB03.db\DB03.edb
Extensible Storage Engine Utilities for Microsoft(R) Exchange Server
Version 15.00
Copyright (C) Microsoft Corporation. All Rights Reserved.
Initiating FILE DUMP mode...
Database: C:\ExchangeDatabases\DB03\DB03.db\DB03.edb
DATABASE HEADER:
Checksum Information:
Expected Checksum: 0x62a178a2
Actual Checksum: 0x62a178a2
Fields:
Log Required: 5722-5728 (0x165a-0x1660)
There are two hexadecimal values displayed in the ESEUtil output. In this example they are “0x165a” and “0×1660”. It is the second value that is of interest, because this is the “High Generation” number.
Move all log files that have a sequence number that is higher than the “High Generation” number. For example, 0×1660 means log files E0100001661 and above should be removed from the server hosting the lagged copy, which is SYDEX2. Don’t delete the files, just move them to a temporary folder in case you need them again.
Note: Transaction log file sequence numbers use a hexadecimal numbering sequence. This means that, for example, log file E0100001699 is not followed by E0100001700, but rather E01000016A0 instead. The last log file in the E01000016xx range is E01000016FF, and then E0100001700. This is important to be aware of because sorting log files by name in Windows Explorer does not correctly sort them by log file sequence number. If you are not aware of this, and do not remove all of the correct log files, then the switchover to the lagged database copy will fail.
Next, identify the server hosting the active copy of the database. In this example it is the server SYDEX1.
[PS] C:\>Get-MailboxDatabaseCopyStatus DB03 -Active | ft -auto Name Status CopyQueueLength ReplayQueueLength LastInspectedLogTime ContentIndexState ---- ------ --------------- ----------------- -------------------- ----------------- DB03\SYDEX1 Mounted 0 0 Healthy
On the server hosting the active copy of the database stop the Microsoft Exchange Replication service.
Perform a database switchover to activate the lagged database copy, in this example the server is SYDEX2.
[PS] C:\>Move-ActiveMailboxDatabase DB03 –ActivateOnServer SYDEX2 –MountDialOverride BestEffort –SkipActiveCopyChecks –SkipClientExperienceChecks –SkipHealthChecks -SkipLagChecks
The database will mount and make a request to Safety Net for resubmission of the mail that is missing from the database.
After Lagged Copy Activation
Activating a lagged database copy is not the end of the exercise. There are still several things that you should do to ensure that your environment is back to a healthy state. After all, some event occurred to cause you to activate a lagged copy, so you most likely still have work ahead of you to restore your environment to its original healthy condition.
Here are some steps that you should consider:
- Use Get-MailboxDatabaseCopyStatus to review the health of your database copies
- Re-seed any database copies that are failed or otherwise considered to be unusable (eg, if the reason you activated a lagged copy was due to corruption in your database)
- Move the active database copy back to a more preferred server
- Re-apply or verify that your lagged copy configuration is in effect
This article is an excerpt from the Deploying and Managing Exchange Server 2013 High Availability ebook.
This article Exchange Server 2013 Lagged Database Copies In Action is © 2014 ExchangeServerPro.com
Get more Exchange Server tips at ExchangeServerPro.com
