EPAM Provides Agile Testing Within a Center of Excellence for a Case Management Solutions Vendor

Being a leader in business processes management solutions for years, the Company helps its clients across the world to cut down on paperwork, optimize business processes flow and assist individuals in their search for increased business performance with reduced costs. The Company specializes in delivering out-of-the-box solutions, empowering its users to achieve high service levels and bring their unique experiences to the marketplace.

In order to reduce time-to-market and quickly respond to changing market demands, the Company made a high-level decision to deploy their products on a quarterly basis instead of less effective yearly builds. In terms of IT, it meant applying new methodologies, adopting new processes, and most importantly choosing the right services partner to bring in the best practices and expertise.

After a rigorous selection process, the Company started to work with EPAM in several small teams, mostly for testing, to support the Company’s internal development forces highly distributed across the US. From the start, EPAM suggested moving to the Scrum methodology to leverage the best practices in testing, enabling Agility and thus ensuring flexibility and higher responsiveness to market needs.

One of the first challenges was to introduce, organize and adapt Scrum practices to projects launched initially using RUP. Since the majority of project members from the Company side came from RUP project backgrounds, their settled understanding how the test process should be organized, documented, signed, etc. was challenged by the shift to an agile approach. Scrum, which places less emphasis on formal organization and much more on continuous verbal communication, required everyone to take a very different approach to the testing process than they were used to.

Further challenges included:

• Reluctance for change from experienced developers on the Company side
• Difficulty for new project members to obtain knowledge in a short time - lack of documentation, absence of test cases, etc
• Dealing with the large number of small distributed development teams with their own on-site leads, POCs, plans, schedules, and scopes

With a view of developing a long-lasting, strong partnership, EPAM established a dedicated Center of Excellence for the Company that embraced the entire testing effort. From the establishment of the CoE in 2008, EPAM has accumulated extensive experience on working effectively with the Company and has implemented a number of projects for them centered around testing.

The range of services rendered by EPAM for each particular Customer product covered:

• new features testing
• test cases development
• integration testing
• localization testing
• regression testing (both manual and automated)
• new automated scripts development
• ongoing product maintenance
  

Relying on EPAM as a strategic partner, the Company entrusted it with the development and execution of a full-fledged testing methodology for all of the Company’s internal projects. The System Test Team, staffed with many of EPAM’s experts, is now working to bring a common, consistent testing methodology to all of the Company’s products.

So far, the System Test Team has reached some key milestones in this process:

• Developed the overall test architecture for all Scrum teams including standards, practices and systems
• Elaborated guidelines for naming, coding, projects structure, etc
• Indicated configurations for UI and API testing
• Created a reference resource which has all the necessary information to let people start development of automated testing scripts from scratch (installation guides, lists of already implemented re-usable functions, etc.)
• Held multiple presentations and training sessions to provide the Company’s in-house engineers with the needed information and practices to efficiently integrate the testing effort
  

The Team of 80+ testing engineers, featuring strong development backgrounds in C# and advanced skills in automated and manual testing, is currently responsible for the maintenance of the test architecture and operation of the continuous test systems.

They hold meetings every week with all of the Scrum teams to gather all possible questions and issues, and provide expert advice and insight into industry best practices.

• SilkTest; TestComplete
• MS SQL Server 2000/2005, Oracle 9i/10g; OLAP/Analysis Services and analytics data; SQL, SQL statements and maintenance; MDX
• All Windows operating systems. IIS, SharePoint, InfoPath
• Oracle WebLogic, IBM WebSphere
• C#

The Results

All the projects have become a good example of implementing the Agile methodology in a testing environment. The continuous integration system being used is proving to be extremely effective — each QA session is started daily with a "fresh" overnight build, helping to uncover any new problems quickly. Daily Scrum stand-ups have kept all the teams aware of the problems/activities related to each project. A two-week iterative planning meetings have provided additional flexibility for feature design and implementation. Thanks to using Agile, Russia and US development teams could effectively organize the development process and workflows and speed up delivery.

EPAM’s input helped the Company gain numerous benefits:                                            

• Improved time-to-market – product release schedule reduced from one year to three months
• Greater efficiency – Agile approach allowed quicker delivery of high-quality software
• Reduced TCO – Server Farm created with 170 VMs supporting 75% of total QA services
• Decreased testing costs – no need for expensive tool experts. Each month Test Automation Scripts do the same work as the team of 20 qualified testers would do:
  • Number of automated test cases created – 800+
  • Number of specialized data classes created – 200+
  • Number of functions created – 2,000+
  • LOC (lines of code) – 40,000+
  • Number of builds – 30+/month
  • Number of configurations per build – 8-10