Wednesday, April 30, 2014

Windows Application Event Log fills with Information events: Event ID 1903 from source HHCTRL

By Steve Endow

UPDATE: A Dynamics GP partner found a way to disable the rampant HHCTRL events by adding some registry entries.  That solution is listed at the bottom.

I recently had a customer contact me regarding an issue he saw in his Windows Application event log.  The log was being flooded with events from source "HHCTRL".  Several events per second were being recorded, resulting in thousands of entries in the Application log.

The events were only "Information" messages, so they didn't appear to indicate an issue, but the volume was annoying, as it made it very difficult to identify any legitimate errors that might be logged.

The description for Event ID 1903 from source HHCTRL cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

I have seen this issue before, and did some preliminary research previously, but all of the threads I found either had no real information about the problem, or they offered solutions that didn't work.  Some threads mentioned that Microsoft issued a security patch for HTML Help, and it was that security patch that introduced this HHCTRL event log message issue.

Since the issue came up again, I finally researched it with a colleague and we discovered what we believe is the cause of the issue.

We knew that the messages appeared to be associated with an application that could be run as a standard Windows app, or as a Windows Service.  My colleague did some testing and discovered that when the application was run as a typical program in a Windows session, the messages did not appear.  But, when he started the service for the application, the HHCTRL messages did appear in Event Viewer.  I was able to verify and reproduce this on my server as well.

We then tried checking permission issues, but we were both testing with Administrator logins, so we doubted it was a permission issue.

Stumped, I decided to try a long shot and launched the SysInternals Process Monitor application.  I ran a trace in Process Monitor, then searched for HHCTRL.  I found that the application was trying to access a copy of the hhctrl.ocx file in the local application directory, but was unable to find it, so an error was generated.  I tried copying the hhctrl.ocx to the app directory, which resolved the error in Process Monitor, but did not eliminate the messages being logged to Event Viewer.  Hmmm.

I then tried unregistering and re-registering the hhctrl.ocx file in different directories, but that didn't eliminate the Event Viewer messages.

So then we pondered why the messages did not appear when the app was launched in a Windows session, but did appear when the app was launched as a service.  And then it dawned on us.

When a service is started, it runs in a security and user-profile "nether world".  The service can have permissions that are equivalent to a user, but the service will not have the exact same "amenities" as an application launched in a user session.

For instance, a service does not necessarily use the same system variables as a Windows user, so the TEMP directory may be different.  And certain operations are blocked for a service that are allowed for a user.  I have had to deal with this quirk over the last few years, and it has resulted in a fair amount of frustration and troubleshooting, but we at least know that such issues exist and know to look for them.

Since we know the HHCTRL messages only appear when the app is run as a service, and I found that the messages appear to be related to HHCTRL, we dug further into the HHCTRL component.  It is part of the Microsoft HTML Help system on Windows, and as we learned, there were a few security updates to HTML Help that restricted its ability to perform certain operations.  We speculated that when an app was run in a user session, it had sufficient permissions, but when run as a service, the Microsoft HTML Help security patch restricted its capabilities and caused the Event ID 1903.

To validate this, I performed a simple test.  I setup a new task in Windows Task Scheduler that would simply launch the Dynamics GP eConnect Programmers Guide, which is a .chm help file.  I did this because Task Scheduler allows you to run an application in your user session, or as a system task that does not require you to be logged in--this runs it in a "nether world" similar to running as a Windows Service.

When I chose the option to "Run only when a user is logged on", the HHCTRL message did not appear in Event Viewer.   But when I chose "Run whether a user is logged on or not", an HHCTRL message DID appear in Event Viewer.

We had found the cause of the problem:  If a Windows application that uses HTML Help attempts to run as a Windows Service or other type of background process (such as with Task Scheduler) that is NOT in a Windows user session, the security restrictions in HTML Help prevent it from working.  When this happens, Event 1903 is logged to the Application event log.  That, in concept, is not a problem.

The problem, which appears to be poor design, or a bug, is that the HTML Help appears to constantly try and run multiple times per second, which results in a flood of HHCTRL 1903 messages in the Application log.  This is annoying, to put it mildly.

So, now that we had found the cause of the problem, what can be done to "fix" it?  The ideal solution is for Microsoft to fix the problem, but since this issue appears to have existed for years, this seems unlikely.

My colleague and I then pondered how we might work around the issue.  Since Process Monitor told us that one component of this issue is the hhctrl.ocx file, what if we unregistered or removed the OCX file?  My colleague had tried that before, but because the file is locked by Windows, it can't be deleted or renamed.

And that is where Unlocker comes in.  Unlocker is an elegant Windows utility that lets you rename, move, or delete files that are locked.  I fired up Unlocker and renamed the two copies of the OCX files on my server.  That didn't fix the problem--we suspected that the OCX was still in memory, so I rebooted my server.  And then we tested again.

Eureka!  Renaming the hhctrl.ocx file in the Windows\System32 and Windows\SysWOW64 directories eliminated the HHCTRL messages going to Event Viewer.  We could launch the application as a service, and no messages were sent to the Application log.  So that solved the problem.

But, there is a minor downside.  By renaming the hhctrl.ocx files, we essentially disabled the HTML Help system on Windows, so we were unable to use the F1 key or open any help files.  On a server, I don't think this will be much of an issue, but on a workstation it may pose a problem.

So the options we offered the customer are:

1. Just ignore the HHCTRL messages
2. Disable the HTML Help system
3. Don't use the app service, and instead run the app in a user session

None of these are ideal, but we believe that #1 and #2 are the least inconvenient.

And with that, I am able to finally close the saga of the rampant HHCTRL event log messages.

UPDATE:  Thanks to Bill at JourneyTeam for finding a way to disable the HHCTRL messages with registry entries.

Copy this text into notepad and save the file as hhctrlfix-32.reg:

Windows Registry Editor Version 5.00


Copy this text into notepad and save the file as hhctrlfix-64.reg:

Windows Registry Editor Version 5.00


With those two reg files created, double click on them to import the settings into the registry.

Steve Endow is a Microsoft MVP for Dynamics GP and a Dynamics GP Certified IT Professional in Los Angeles.  He is the owner of Precipio Services, which provides Dynamics GP integrations, customizations, and automation solutions.

You can also find him on Google+ and Twitter

No comments: