Wednesday, September 15, 2010

The Software Testing Disconnect

So you've taken some exams and become "certified"? So how much better prepared are you to handle those complex client requirements that invariably come up?

I've been in conference rooms with teams of very smart people, and all the certifications on the planet wouldn't have helped us with certain basic, fundamental requirements that were both challenging, and which pushed well past the limits of certain areas of Dynamics GP.

While I am a big proponent of certification, the limitations of the exam process are still frustrating to me.

I bring this up because I'm studying for my third exam to update my GP certifications.  GP 2010 Install & Config felt pretty relevant.  There are a lot of valuable real world topics and knowledge that aren't included in the exams, but there's only so much you can cover in 75 questions based on the instructor led training materials.

The GP 2010 Financials exam felt, well, a lot like a GP 10 Financials exam.  I wasn't able to tell if the exam had even been updated, and it felt surprisingly easy.  It did cover all of the basics, but it seemed a little too basic. 

And now, the reason why I am writing about this topic:  The SQL Server 2008 Implementation and Maintenance (70-432) exam.  This exam, like it's SQL 2005 equivalent, is a doozie.  For the average GP consultant, I think this exam is out in left field.  There are certainly sections that are very relevant to the GP world (configuration, backups, maintenance plans, security, etc.), but then there is a lot of content that is so irrelevant to the GP crowd that it's painful to waste brain capacity memorizing it just for the test.

The emphasis on enterprise features is predictable--those tend to be the fancier, cooler sounding, and newer features that allow Microsoft to compete against the big competitors.  But an 8 member SQL cluster just isn't a common occurrence for GP customers.  And honestly, if one of my clients had a server cluster for GP, I would want a dedicated SQL Server expert by my side to handle those complexities.  I don't want my family physician performing my brain surgery.

Here are a few examples of questions that make my eyes roll:

What is the command to extend the expiration period for a merge replication subscription?

What @sync_type value should you use for new peer-to-peer replication nodes?

When would you use an On change: prevent policy?  When would you use On Demand?

What is the exact syntax of the BACKUP DATABASE T-SQL command?

When do you want to configure the cost threshold for parallelism, and what is the syntax for the T-SQL to do so?

What is the exact syntax of the sp_add_notifications command and parameter values? 

If you know the answer to any of these, well, move to the front of the line--you can be my DBA any day.  Most of these are not applicable to GP, and for those that are, well, that is what Books On Line is for.  Memorizing the syntax of obscure or rarely used T-SQL commands that can be performed in SQL Management Studio isn't really the value that I bring to my clients.

For the rest of us that focus more on GP, and the SQL Server features that are relevant to GP, many of these questions seem like a pedantic waste of time.

I understand that there is no longer a "SQL for GP" exam, and I can understand the desire to point everyone to a single, uniform SQL exam track, but there used to be a SQL exam for GP consultants, and I think it did a great job of covering the topics that were relevant to GP folks without battering us with features that either aren't used, or can't be used, in GP environments.

Studying for the SQL 2005 exam, I did learn about some SQL Server features that I never knew existed, but since that last exam, there have been exactly zero times that I have needed to know about those features.  Kind of interesting, but completely irrelevant.

This is an age old question that has been deliberated for years, but having to study again for the SQL exam brought it up again in a very specific way.

For those other poor souls out there also having to take the SQL 2008 exam, I feel your pain...


Andy Nifong said...

Steve - I found the SQL 2008 exam to be much more focused than the SQL 2005 exam, which blended the administration topics with a lot of developer-oriented gibberish. Good luck to you!

Steve Endow said...

Thanks Andy, that is good to hear. Studying off of the sample test, the questions do seem to be more coherent--I can often guess which 1 or 2 answers make sense, even if I don't know the specific syntax or topic.

Mariano Gomez said...

Amen brother! I find these other non-GP exams to be extremely painful for the average Joe to go through. C'mon, how many GP consultants (even the long time folks like us) do you know that are actually Database Administrators? Better yet, how many client Database Admins do you know that are interested in learning GP.

Not saying they are not out there, but in 13 years of jogging these lanes, they have been very few and far between.

Mariano Gomez, MVP

Steve Endow said...

Hi Mariano,

Obviously I agree completely! To truly have all of the knowledge required to easily pass the SQL 2008 exam, you would have to be a walking SQL Server encyclopedia. I'll go so far to say that even a real enterprise SQL DBA wouldn't normally know about or use many of the items covered by the exam.

And what really brings it home is that the SQL 2008 exam is only 40 questions! So the exam takes 40 data points from thousands and thousands of possible incredibly diverse and deep concepts and is supposed to assess your competence with SQL Server? I just don't get it.

That's like having a 40 question test for Dynamics GP that covered everything from setup to GL to FA to Service to Manufacturing to FRx to development tools. It would be nonsensical.

It's unfortunate, since I think a focused SQL Server exam for GP folks could be a really valuable learning and training tool.