Don’t Get Caught in the COTS Trap! Let iLAB Drive Your COTS Testing!

Your public sector organization just invested a large chunk of change into a COTS system with all the bells and whistles.  It seems bright and shiny like a brand-new Cadillac just driven off the showroom floor.  Pricy, but worth it your leadership assumes.  A smart move.  One that will allow the organization to zoom their way to successful implementation.   

 Commercial Off-the-Shelf (COTS) systems are a popular choice amongst public sector entities.  Many turn to them after being burned by clunky and expensive custom-developed software. No one wants to be stuck with a lemon.  COTS systems are ready-made software products available in the commercial market.  They are commonly used by many industries.  COTS systems are advertised as ready to go, like that new Cadillac. And like that Cadillac, they come with some lofty expectations.  COTS systems are billed as being cost effective, serving up improved functionality and offering a smooth and fast implementation.  Quick and dirty.  Easy-peasy.  What could go wrong?  Well…..LOTS. 

COTS systems sound good.  They look good.  And they can be good.  HOWEVER, implementing a COTS system without well thought out planning and testing can be fraught with peril.  Picture running over a pothole you should’ve seen but didn’t, flattening the tire of what was once your nice Cadillac.  Not good.  And not smart.  

The largest oil slick in the road towards COTS implementation can be summarized in one word, “assumptions.”  (And you know what they say about assumptions.) 

For example: 

• COTS Packages can be implemented quickly. 
• The Development vendor has already deployed COTS Package ABC for another public sector organization, so COTS Package ABC will work well for our public sector organization, too. 
• COTS Package ABC will work perfectly for my organization and will not require customization. 
• Third-party systems integrate easily with COTS packages. 
• User Acceptance Testing is the only type of testing needs for COTS systems. 

Not so fast!  Don’t be an Edsel!  If you don’t want to be the person responsible for telling management your three-MONTH COTS implementation is going to change lanes into a three-YEAR implementation, keep reading. 

In reality: 

• Often what is being implemented is a highly customized version of a COTS system that has been updated to meet the needs of the client.  It is a sales and marketing ploy to make the customer believe they are receiving a true COTS system. 
• Each public sector organization has their own line of business.  Different customizations may be needed.  Different teams will likely be involved, as well as different underlying technology stacks. 
• COTS customization is very common and is often required. 
• Not every third-party system has been integrated with every COTS package.  There are lots of variables to be accounted for. 
• Complex integrations required more than User Acceptance Testing. 

There is no “one size fits all” approach for COTS testing.  Implementing a COTS system is a far more complex undertaking than is usually perceived.  COTS implementation testing is quite an involved process and many times not much different from custom-developed software testing.  A carefully planned and detailed approach is needed to define a comprehensive test strategy that ensures proper test coverage of customizations, integrations, and critical business workflows.  The project delivery cycle and technology being used should also be considered. 

When crafting a COTS implementation test strategy, at minimum, the following areas need to be covered as part of defining the test schedules:  

• Requirements gathering and analysis 
• Data migration testing 
• Security roles and access  
• Functional Testing 
• System Integration 
• End-to-End Testing 
• Performance testing  
• Regression Testing 
• User Acceptance Testing 

As part of the test strategy, a comprehensive test plan needs to be determined: 

• What areas will be covered. 
• How these areas will be tested. 
• When these areas will be tested. 
• How many iterations are needed. 
• Positive and negative testing approaches. 

As COTS implementations are usually multi-year efforts with incremental development schedules, iLAB highly recommends companies invest in a team of dedicated and independent professional testers.  iLAB Testers and Quality Assurance experts are experienced and have been down the COTS implementation road many times.  We are seasoned professionals, not student drivers.  

iLAB has helped several clients whose COTS testing had veered off course.  Common challenges experienced by these clients were schedule delays, missed deadlines, increased costs, and poor software quality.  For each client, iLAB performed a baseline review to determine what went wrong and mitigation recommended strategies.  iLAB played the role of the compass and was able to help each client finish their COTS testing and implementation journeys.   

It is often said that money does not buy happiness, it also does not buy a COTS system that is guaranteed to meet your business needs, straight of the box, no questions asked.  But with iLAB’s testing and Quality Assurance expertise, your organization can navigate its way past roadblocks to an efficient and on time COTS implementation.  

Contact iLAB today.