File Share Witness and Datacenter Failback

This afternoon we ran across an issue with a fairly new Exchange 2010 Database Availability Group comprised of 3 nodes all running SP1 with Update Rollup 3. The primary datacenter had 2 nodes with a local file share witness while the 3rd node and alternate file share witness were in a DR site. We also had recently performed a successfull datacenter failover and failback test that went swimmingly so everything was back up and running in the primary datacenter.

What we noticed today was that the cluster quorum and file share witness settings persisted as a node and file share majority after the failback instead of reverting to a node majority model like a 3-node DAG should be using. The only time Exchange should be using this model is when we have an even number of servers in the DAG. So without reproducing this again I can only see this as a timing issue – when one of the primary datacenter nodes gets added back to the DAG the quorum settings are flipped, but once the 3rd and final node joins again the quorum settings are not adjusted. This leaves us with a node and file share majority, and the FSW being our alternate FSW.

You can see here if you open the Cluster MMC our DAG is operating as a node and file share majority model even though all 3 nodes are online:

The fix for the issue is really easy – just run the Set-DatabaseAvailabilityGroup with no parameters. This process does not take the databases or cluster offline, but you’ll see the DAG detect it is using the wrong model for an odd number of nodes and adjust itself accordingly:

After the change you can verify in the cluster MMC that the quorum settings have been corrected to be a node majority:

I’m sure there’s a rational reason behind this behavior, but I haven’t quite nailed down why this happens quite yet. In the meantime it’s just one more step to add to your DR documentation!

Here

Recent content I've written for you—just for you!— to enjoy while you're here.

There

Quick commentary and links to other sources you'll find interesting. I promise.

Everywhere

Some personal background, links to related projects, and other ways to connect.

Hi there. My name is Tom Pacyk and this is my small home on the web. I love the intersection of design, technology, and communication, which is a combination that led me to a career in sales and marketing roles at places like Zoom and ServiceNow. They're a bit old now, but I also had the opportunity to publish a couple of books along the way.

Portland, Oregon is home for me, my wife Beth, and our three kids, but I'm actually a Midwestern transplant—I grew up in the Chicago suburbs and went to school at Purdue and Illinois. When I find some free time I'm probably going to concerts, rooting for the Portland Timbers, or working on my Sunshine Burn Photography project.