Monday, July 2, 2018

Move Forward, Every Day. And Bring Chocolate Cake.

By Steve Endow


Change Is Hard

It's trite, I know.  But it doesn't make it less true.

I think it's worthwhile to pause and take a moment to consider how difficult change can be.  Actually acknowledge it.

Change can be uncomfortable, scary, and stressful.  Change can be costly and arduous and tiring.  Change can be inherently destructive.

It often feels much easier to ignore it, delay it, or pretend it isn't happening.  But you can't always avoid change.  Sometimes you have time to prepare for change, and sometimes you have no advance notice.

In the ERP world, consultants see the symptoms of change regularly when customers implement a new ERP system.

"My old system did X, why can't the new system do X?"

"I just don't have time to learn this new system. I have too much work to do!"

"I liked the old system better."



Turnabout is Fair Play

But occasionally, it seems that those pesky consultants get a taste of change.

Microsoft has changed quite a bit since it acquired Great Plains Software.  Great Plains was rebranded as Dynamics GP, and the GP team at Microsoft put a lot of work into the product, adding new features and expanding its capabilities.

But over the last several years, Microsoft has invested heavily in its 'cloud computing' strategy, with Azure and Software as a Service (SaaS) offerings.  As part of the cloud strategy, Microsoft has also invested heavily in ERP software as a service, hosted in Azure.

Dynamics AX has become Dynamics 365 Finance and Operations, and Dynamics NAV has now, finally, become Dynamics 365 Business Central.  Dynamics GP is notably not on the Microsoft Azure ERP SaaS menu.  It's our turn to change.


Free GP Transaction Search v2.0 now available with new RM and SOP search windows!

By Steve Endow

Version 2.0 of the FREE Dynamics GP Transaction Search is now available! 

A big thanks to Ian Grieve for assistance with the development and testing of this new version!


For more information and to download compiled releases:



Here is a brief video showing the simple installation process and an overview of the search windows:




The latest source code is available on GitHub:



This new release includes two new search windows:

  • RM Transaction Search
  • SOP Transaction Search

Monday, June 11, 2018

Opening Dynamics GP Windows using .NET and Visual Studio Tools

By Steve Endow

When you develop a Dynamics GP customization, you may want to automatically open Dynamics GP windows to add dramatic flair to your application.

I mean really, let's be honest, what's cooler than saying your customization lets the user "drill down" into Dynamics GP?  (Is that still a buzzword? I may be behind the times)

Take the GP Transaction Search tool as an example. When a user searches for an AP transaction, I thought it would be handy if the user could right click on a row and view both vendor information and transaction information directly in Dynamics GP.


Just because I developed the PM Transaction Search window doesn't mean that I want to (or need to) develop a new window just to view vendor info or transaction info. 

Why not just leverage Dynamics GP windows to view that information?

I want the user to be able to right click or double click on a row in the grid and view the details in an existing Dynamics GP window.

Here's a full video with a code review of opening GP windows from .NET using Visual Studio Tools:




Fortunately, adding this drill-down functionality to a VS Tools AddIn for Dynamic GP is relatively easy.  For some windows.

For other windows, it requires a slightly more involved process.

And, unfortunately, for some windows, it may be very difficult or impossible. 

Tuesday, June 5, 2018

Dynamics GP Transaction Search v1.0 is Available! For FREE!

By Steve Endow

A few weeks ago, a question was posted to the GPUG Open Forum asking why the AP Transaction Inquiry by Document was so slow.


The post explained that the company has so many different vendor accounts for FEDEX that they can't easily search based on vendor ID.  So they find it easier to search by vendor Invoice Number.

Unfortunately, because they have so much transaction history, and because the window pre-loads some data, it takes over 3 minutes for the inquiry window to even be displayed.

That. Is. Crazy.

I was...intrigued.

What would cause a seemingly simple Dynamics GP inquiry window to take 3 minutes just to open?  That's just weird.

So I decided to test the window and trace the SQL activity that occurs when the window performs a search.

It...was...bad.  The PM Transaction Inquiry Document window throws out SQL queries like they're going out of style.


Every single PM transaction record that is retrieved by the inquiry window results in at least 3 subsequent queries.  FOR.  EACH.  RECORD.

After seeing that, I gave up on attempting to "fix" or optimize the window.  It wasn't a SQL performance issue or a question of optimization.  It's just a terrible design.

So, what would an ideal PM Transaction Inquiry window look like?

How about this:



A window that allows you to search by date range AND vendor ID AND vendor name AND document number AND document amount.  All simultaneously.

Here is a video explaining the project and demonstrating the PM Transaction Search window that is included in version 1.0:




As you type each character, this magical search window calls a SQL stored procedure and updates the search results.  In real time.

To keep the window honest, a status bar at the bottom of the window shows how many records were retrieved, how many milliseconds it took to retrieve the data from SQL, as well as how many milliseconds it took to display the data in the window.

We're talking real-world accountability here, folks.

Want to see more info about a transaction?  Just double click on the row to drill into GP!  Or right click on a row to view the master record inquiry window.

Don't like the columns that are displayed in the search window data grid?  No problem.  You can modify the included stored procedure.  Want to add a column or two to the data?  No problem.  Just add a few fields to the procedure.

Are you so thrilled by the results of your search that you want to share them with your coworkers?  No problem.  You can click on the Copy button to copy all of the data in the grid and paste it into Excel.  Or you can click on the CSV button and save the data to a CSV text file.

And the GP Transaction Search tool is extremely simple to install.  Just run a SQL script to create a stored procedure in each of your GP company databases, then copy 3 files into the GP AddIns subdirectory.  It really is that easy.



You must be thinking: Wow, with all of those features, that fancy search window must cost a fortune!

But that's where you're wrong!  The Dynamics GP Transaction Search tool is available for FREE!


Here is the temporary link to download the search tool from OneDrive:

https://1drv.ms/f/s!Au567Fd0af9TogwHx8OKAarDM3Zm


UPDATE: I now have a dedicated page for the GP Transaction Search in the Free Products sections of my web site at:

https://precipioservices.com/free/gp-transaction-search/



But that's not all!  In addition to offering the tool for the rock bottom price of Free, I have published the code on GitHub as an open source project so that others can contribute to the project:

https://github.com/steveendow/gp-transaction-search


If you are a .NET developer or a Dexterity developer and are interested in helping enhance GP Transaction Search, please contact me!

If you have any questions about the search tool, or if you have used it in a real GP environment and have any suggestions for improving it, please contact me!


https://precipioservices.com/contact-us/




You can also find him on Twitter, YouTube, and Google+







Monday, June 4, 2018

"There's nothing wrong with our SQL Servers or network", says the IT department confidently

By Steve Endow

Let me share a story about an all-to-common situation in the life of the Dynamics GP consultant or developer.

You develop an integration, or customization, or implement some software that talks to a SQL Server.  We do this stuff every day.  Normal, routine, common projects.  Just software talkin' to a SQL Server.  It's usually so reliable you don't think twice about it.

Then, after your integration or software has been running just fine for months, without a single issue, you encounter this error on Monday at 1am:

A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: SQL Network Interfaces, error: 26 - Error Locating Server/Instance Specified)


Hmmm, that's odd, we've never had this issue before--the integration has literally never had an issue connecting to the SQL Server.  We've had two dozen other errors related to bad data or import errors, but never before have we seen an issue connecting to the SQL Server.


"Hey IT manager, we received this SQL Server connection error from our weekly on Monday at 1am. Can you look into what might be causing this?"

IT Manager, 5 minutes later:  "Nothing wrong on our end! No errors, no issues, all of our jobs are running fine. If there was a problem, I would have known about it."


Oooookay then.


One week later.  Monday.  1am.  Same error, second week in a row.  And this is occurring for two different integrations--one pointing to the production GP company, and the other pointing to the Test GP company. 


When both integrations are run manually at 10am, they both run fine.  So this is not a fluke, and it is occurring consistently now.


"Hey IT manager, for the second week in a row, we received the SQL Server connection error at 1am. It's occurring with two different integrations, so there is definitely something causing this. Can you please look into this again?"


IT Manager, 1 minute later:  "I was able to figure out the cause of the problem.  We are shutting down our GP SQL Server every night from 10pm to 5am."


Face.  Palm.


I actually encounter this on a regular basis.  Fortunately in this case, it turned out to be a blatantly obvious cause--the IT manager had just forgotten about the change to the server maintenance schedule.  But once he remembered, we had our explanation.

But usually, the problem is less obvious and much more difficult to track down.

It could be a bad switch or network card that causes intermittent SQL connection errors.  It could be custom code that only fires at certain times, locking or blocking SQL resources, causing seemingly random SQL command timeouts that get blamed on your software.  I've even had a situation where a Veeam backup of a completely different VM caused the host machine to drop network connections for other VMs. (a Veeam bug that I believe has been fixed)

I've seen all of my custom SQL objects literally disappear every week because a super security conscious corporate customer has a routine that deleted any unapproved SQL objects from the database.  And then there's the common case of anti-virus software blocking or deleting an EXE or DLL.

Modern networks are complex, and when you have dozens or hundreds of things going on, it's usually not easy to identify what might be causing an intermittent or infrequent problem.  When hardware and software is normally incredibly reliable, it seems that people are resistant to consider the possibility that something other than your integration or software is causing the problem.

Just because your software happens to be the one that is logging the error, there seems to be a strange bias that has people blame your software and deny that something else is preventing your software from communicating with the SQL Server.

I currently work with hundreds of customers, and as a result, I probably see this issue weekly.  Unfortunately, if the IT department claims that their systems are working perfectly, in many cases there isn't much that I can do except to add additional error handling, logging, and diagnostic information to my log files to present to the IT department repeatedly.  Sometimes it's enough to help a motivated tech do enough research to find the cause.  But many times an intermittent non-critical error just gets ignored.

If you encounter this issue with IT departments, do you have any suggestions?  Do you have a technique for making the IT department curious about researching the problem instead of getting defensive?  If so, I'm all ears.




You can also find him on Twitter, YouTube, and Google+




Tuesday, May 29, 2018

Dramatically Improve Dynamics GP eConnect Performance, But There's a Catch

By Steve Endow

If you've developed a Dynamics GP integration using eConnect and .NET, you should be familiar with the eConnect Serialization process, whereby the eConnect Serialization .NET assembly converts your transaction objects and data into an XML document.

Here's an example of the serialization of a GL transaction.


Notice that this code involves a single GL transaction object, glTrans, with the glTrxHeader and glTransLines assigned to the object properties.

That single GL transaction is then assigned as a single element to the glType array, which is then converted to XML.

Wait a minute.  Why is the GLTransactionType object, in this case named glType, an array?

Well, it's an array because eConnect Serialization supports the conversion of multiple transactions into a single XML document.

What?

So...why would you want to do that?  Why serialize several transactions instead of one transaction at a time?

Good question!

Here is a video where I discuss this feature and demonstrate the performance:



I asked that same question ages ago when I was developing with eConnect on Dynamics GP 7.5, probably just over 12 years ago.  I was using Visual Basic 6 and the eConnect COM components.

When I saw that eConnect serialization supported transaction arrays, I assumed there must be some reason why that feature existed, so I performed a test to compare the import performance.

I recall that with VB 6, when serializing and sending individual transactions, I was able to import an average of 20 AP invoice transactions per second, which is actually quite a bit better than what I see with .NET today.  And I recall that when I tested serializing and sending multiple transactions at the same time, I did not see a performance benefit. I was therefore puzzled about the functionality.

While I didn't observe any performance benefit, I did find a downside to sending multiple transactions to eConnect at once.  If eConnect encountered an error while importing any of the transactions in the 'batch', the entire batch would be rolled back and no transactions would be saved.  So if you send in 100 transactions, and 1 transaction fails to import, nothing is imported.

And another annoyance:  If your batch of 100 transactions had 5 transactions with errors, eConnect would only return 1 error.  As soon as the first transaction has an error, eConnect stops, rolls everything back, and returns a single error.  Once you fix error #1 and resubmit, eConnect will stop on error #2.  So you are forced to deal with the errors one at a time, and in the meantime, zero transactions have been imported.  It's not an ideal error handling process.

So, fast forward to the year 2018, when I was recently asked if eConnect could import multiple transactions in a single call.  I initially responded that it can, but that there was no performance benefit or value in doing so.

But then, for some unknown reason, I figured I should test single vs. multiple transaction submissions and verify my long-held 'belief'.

So I pulled up my eConnect load testing tool and modified it to support serializing multiple transactions, in addition to one transaction at a time.

When I saw the results of sending multiple transactions to eConnect in a single submission...

My...

Jaw...

Dropped.

100 GL JE transactions imported in 17 seconds when sent individually to eConnect.

But when I submitted a single XML document with 100 journal entries, eConnect imported them in 3 seconds.

WHAT???

I then imported 700 transactions, submitting them all to eConnect in a single import request.  They imported in 17 seconds, the same time it took to import just 100 transactions individually.


That's a 7 fold increase in performance, resulting from a minor change in the import code.  That is a staggering difference.

I repeated the tests several times and confirmed that it wasn't a fluke.

I then tested with AP Invoices and saw the same results.  In this test, I was able to import 750 AP Invoices in a single submission in the same amount of time as submitting 100 AP Invoices individually.  That is cra-zee.



So, based on my new tests with GL JEs and AP Invoices, submitting multiple transactions to eConnect can produce a huuuuuuge performance improvement with the eConnect .NET assemblies.

My guess is that something changed when the eConnect assemblies were converted to .NET, resulting in a slower performance when importing individual transactions, but faster performance when importing multiple transactions.

But....

There is a catch.

It's the same catch that I found when testing with VB 6 and eConnect COM objects.  If there is a single import error, the entire batch will fail to import, and no transactions will be saved.  So if you try to import 500 transactions, a single error will roll everything back and you'll need to fix the error before resubmitting.  And if there is a second or third error, the entire batch will fail a second and third time.  495 transactions will be held up because of 5 errors.  It's a hassle.

And if you assigned 500 journal entry numbers and the batch of transactions fails to import, you've just burned through 500 JEs.  When your auditors review the JEs in Dynamics GP, they'll see a gap of 500 JE numbers and start asking questions.  If such errors happen frequently, you'll quickly burn through thousands of transaction numbers.  It's not the end of the world, but it's not ideal.

So, given these drawbacks, what's the conclusion?

My personal recommendation is that most Dynamics GP users should stick with submitting single transactions to eConnect.  For most Dynamics GP users, eConnect imports are not a bottleneck for the business, and I find that it is uncommon for customers to be waiting around for eConnect integrations to complete.  Usually, customers are able to import hundreds or thousands of transactions relatively quickly, but are then having to wait for those transactions to post in Dynamics GP, as posting is far slower than importing.

However, there are some very high volume customers that import tens of thousands or hundreds of thousands of transactions every day, and are either importing or posting transactions non-stop throughout the day.  If you have Dynamics GP integrations that collectively take hours to complete, and this technique could save you hours of import time, then I would definitely recommend looking into it.  But only if you 1) have the ability to develop pre-validation routines for all of your transactions to minimize errors, and 2) have the ability to deal with errors and reimport the failed transactions, and 3) you can develop a way to manage transaction numbers so that they aren't wasted with each failed import.

I hope this was informative!



You can also find him on Twitter, YouTube, and Google+






Friday, May 18, 2018

Get Next Dynamics GP SOP Transaction Number Using SQL

By Steve Endow

As I mentioned in a recent post, some times it's better to use SQL to retrieve the next document number from Dynamics GP.

I am currently working on an eConnect SOP Order import and needed to retrieve the next order number from GP.

Here is my C# code for retrieving the next SOP number from GP using SQL.



The SQL stored procedure is called taGetSopNumber and you need to provide it with the SOP Type and the Doc Type ID of your desired transaction.


public string GetNextSOPNumber(int sopType, string docTypeID)
{
    //SOP Type:  
    //1 = Quote
    //2 = Order
    //3 = Invoice
    //4 = Return
    //5 = Back Order
    //6 = Fulfillment Order

    try
    {
        SqlConnection gpConn = ConnectionGP();

        string commandText = "taGetSopNumber";

        SqlParameter[] sqlParameters = new SqlParameter[5];
        sqlParameters[0] = new SqlParameter("@I_tSOPTYPE", System.Data.SqlDbType.TinyInt);
        sqlParameters[0].Value = sopType;
        sqlParameters[1] = new SqlParameter("@I_cDOCID", System.Data.SqlDbType.Char, 15);
        sqlParameters[1].Value = docTypeID;
        sqlParameters[2] = new SqlParameter("@I_tInc_Dec", System.Data.SqlDbType.TinyInt);
        sqlParameters[2].Value = 1;
        sqlParameters[3] = new SqlParameter("@O_vSopNumber", System.Data.SqlDbType.Char, 21);
        sqlParameters[3].Value = string.Empty;
        sqlParameters[3].Direction = ParameterDirection.Output;
        sqlParameters[4] = new SqlParameter("@O_iErrorState", System.Data.SqlDbType.Int);
        sqlParameters[4].Direction = ParameterDirection.Output;
        sqlParameters[4].Value = 0;

        string nextNum = string.Empty;
        int recordCount = DataAccess.ExecuteNonQuery(gpConn, Controller.Instance.Model.GPDatabase, CommandType.StoredProcedure, commandText, sqlParameters);
        nextNum = sqlParameters[3].Value.ToString().Trim();

        return nextNum;
    }
    catch (Exception ex)
    {
        Log.Write("An unexpected error occurred in GetNextSOPNumber: " + ex.Message, true);
        return string.Empty;
    }
}





You can also find him on Twitter, YouTube, and Google+