Thursday, November 11, 2010

Configuring DTC on Windows Server 2008 for eConnect 9 or 10

This evening I received an e-mail from a client who was getting DTC error messages when they tried to run an eConnect 9 import.

In theory, that shouldn't be too difficult.  But there were a lot of red herrings.

First, the error they were receiving was "An error occurred while enlisting in a distributed transaction".  That is all I had to go on at first.  But then their GP partner told me that these errors seemed to start when they transferred their Dynamics GP databases to a new SQL Server.  Okay, that's good to know.

So I connect to their SQL Server, and I am reminded that they are running a Windows 2008 R2 Enterprise clustered server.  Okay....  I've never had to deploy an eConnect 9 or eConnect 10 integration on a cluster before--only eConnect 2010--so this should be interesting.  After researching DTC on clusters, I found several articles on cluster configuration that were well outside of my comfort zone, so I thought I might have to defer to a Windows cluster expert.

But then I thought I should just try and check the basics to see if there was anything else I can identify that might help to troubleshoot the issue.  So I start with the basics:  Check the MSDTC configuration.  On the SQL Server, I open Component Services.  When I right click on My Computer and view Properties, I'm greeted with an unfamiliar version of the window--there are no MSDTC options.

I forgot that on Server 2008, the DTC configuration is located in a different place in Component Services--it's under a separate folder called "Distributed Transaction Coordinator".  Hiding in plain sight, very sneaky.

When I checked the DTC security options, all seemed well.

Just to verify connectivity, I used osql.exe at the command line on the GP Terminal Server to verify that I could connect to the new SQL Server.  That worked fine, and GP was running fine, so the general network connectivity seemed okay.

Given that the error appeared to be related to DTC, I put a copy of dtcping on both the application server and database server.  When I tried to have the application server ping the SQL Server, dtcping would time out with an error.

Invoking RPC method on APP1
Problem:fail to invoke remote RPC method
Error(0x6D9) at dtcping.cpp @303
-->RPC pinging exception
-->1753(There are no more endpoints available from the endpoint mapper.)
RPC test failed

Based on excellent information from this fantastic Microsoft article on troubleshooting DTCPing errors, this error itself is misleading.  It implies that DTCPing was unable to retrieve a port, but it actually means that a firewall is blocking the DTC traffic.

I then switched directions, using the SQL Server to ping the application server.  When I did that, I received a response!

Received Bind call from SQL1
Trying Reverse Bind to SQL1
Error(0x6D9) at ServerManager.cpp @453
-->RPC reverse BIND failed
But when the application server tried to "reverse bind" back to the SQL Server, it failed.

So this told me that something was blocking the DTC traffic from the application server to the SQL Server, but the SQL Server could send DTC traffic to the application server.

I then decided to check the SQL Server to confirm that the Windows Firewall was disabled.  Alas, the firewall was enabled.  Aha!  When I turned off the firewall, sure enough, DTC Ping operated fine in both directions between the app and SQL server.  And I also found that the option to "Notify me when Windows Firewall blocks a new program" was unchecked, which is why I didn't receive a dialog when DTCPing was blocked.  When I enabled notifications, I received this message when I launched DTCPing.

So, now the fun hunt for the Windows Firewall Rule that would enable inbound DTC traffic on the SQL Server.

After some searching, I learned that DTC has pre-defined rules on the Windows Server 2008 firewall.  Under the Windows Firewall "Advanced Settings", if you click on Inbound Rules, there are dedicated Distributed Transaction Coordinator rules.  But there are 3 separate rules for different DTC protocols.

Remembering that the DTC properties window listed only TCP/IP, I guessed that the TCP firewall rule would do the trick.

Sure enough, once I enabled that inbound firewall rule, DTCPing worked fine.

I then fired up an eConnect integration, and transactions started flowing.

So, if you are configuring eConnect 9 or eConnect 10 on Windows Server 2008 (or any machine with the newer version of Windows Firewall), if the firewall is enabled, check to see if inbound DTC traffic is allowed.

Steve Endow is a Dynamics GP Certified Trainer and Dynamics GP Certified Professional.  He is also the owner of Precipio Services, which provides Dynamics GP integrations, customizations, and automation solutions.

No comments: