John Deere executive says thorough testing prevented an embarrassing, costly misfire in the company's DAM deployment.Enterprise-level DAM rollouts can meet disaster if they can't handle real-life user loads, but realistic load testing of a large digital asset management application is often difficult and time-consuming.
Kim Sievers, manager of visual and production services at Deere & Co., of Moline, Ill., spoke at the recent Henry Stewart Digital Asset Management Symposium and described what she found out when she took charge of the DAM initiative at her company.
Her first challenge was the fact that she was an application owner rather than an IT manager.
"To be honest with you, as an application owner and never [having] done this before, I really didn't understand what it was. And like all major projects, we're running six months behind, we're over budget, users are grumbling, 'Let's skip it.' I had a very wise deployment specialist who said, 'No, no, no, you'd better not do that.'"
Click here to read about the effectiveness of a limited DAM rollout.
She stayed with the plan for a 20-day load testing program and says she was glad she did.
"Had we skipped this process, the application would have closed within two hours. We would have had to regroup again. So really the number one reason why you want a performance test is you don't want those show-stoppers to be discovered by your customers."
Her in-house performance testing team ran the new application through its paces, using load-testing tools such as Mercury Interactive Corp.'s TestDirector (a tool for storing scripts), Mercury's Win Runner (for capturing and replaying user interactions), and Load Runner (to predict system behavior and performance, and for emulating thousands of simultaneous users with real-time monitoring).
The tests revealed that the system performed better with faster two-processor servers than with four-processor servers, so the team swapped hardware to get the most effective configuration, then ran the tests again, just to be sure.
Sievers points out that you can get along without an in-house testing team by hiring an independent testing firm.
The team performed the tests on two identical systems for safety's sake.
"We first built our system in the test environment to get the configuration, test our migration and make sure that that system was operating 100 percent, and then we went in and built the system in our production environment," Sievers said.
"What is really important is to involve your vendor during the load test process and share all of the results," she said. "Not only will this create a stronger product for your installation, but your installation will benefit from the lessons learned from the vendor's other customers.
"At John Deere we invite the vendor's developers to come up during the load test process. They can't always be there, but at major milestones during the process we have a review with everyone on the team, and then at the end of the load testing process we have a comprehensive review that we supply to everyone on the team. We also supply them all of our data in a written form."
Click here to read about what can happen when librarians and creative professionals collaborate on a DAM project.
Naturally, this all comes at a cost.
"Of course, it's billed to the business application owner. The rate is $750 per day. Our initial launch took us approximately 20 days, and our last upgrade took us about five days. So I budget every year for performance testing for the upgrades."
She insists on performance testing as an essential part of any asset management installation or upgrade.
"If your project is in the discovery phase, ask your vendors for their performance load test data. Use this data to determine your acceptance criteria and include it in your license and maintenance agreement.
"We didn't accept the product and start paying for the product until it met our acceptance criteria. If your project is in the planning stage, add the budget and the time. If your project is in the implementation stage, delay the deployment, add the budget and the time, and if your project is in the deployment and problems exist, performance testing will help you find where that contention is."
Web Design 
