The challenge: eliminate manual testing

The UPS and Engage ESM teams completed delivery of the UPS ServiceNow platform in Q1 of 2020. UPS’s goals in deploying the platform included not only supporting the organization's global footprint and scale but also minimizing customizations to enable agility in completing changes and upgrades as fast as possible.  The first upgrade cycle from Madrid to Orlando stretched over four weeks and required heavy manual testing.  This month-long upgrade process took focus away from continued development improvements for business processes and projects.  UPS needed a way to reduce that impact and still thoroughly test their platform.

The goal:  do it faster

Reduce upgrade time by 50% while maintaining test coverage, minimizing risk, and allowing for continued development.

The plan: release the machines

The first step was to outline a library of ServiceNow Automated Testing Framework (ATF) automated tests to reduce manual testing time.  A primary regression testing suite was created, along with secondary level suites for each of the applications in play.

Over the course of a month, this library was developed to validate configurations for Incident Management, Major Incident Management, Problem Management, Knowledge Management, Change Management, CMDB, Catalog and Request Management, Customer Service Management, Field Service Management, and several Integrations to other UPS applications.

In parallel to the construction of the regression test suite, a sub-production environment was upgraded to Paris so that process owners, administrators, and technical consultants could analyze the impact of the upgrade and formulate plans to address it. Skipped records, changes to functionality, and anticipated release functionality was vetted and reviewed to ensure updates and training plans were put in place where needed.

An upgrade schedule was outlined and closely maintained by UPS administrators.  This included milestones for upgrades, clones, testing activity, development fixes, and a critical fix path for issues that might arise in production while the upgrade was being worked and tested on sub-production environments.

After the upgrade was complete, a process was put in place that allowed development team members to maintain the integrity and success of the test cases as the process was modified. The Agile 2.0 application was modified to ensure developers checked the functionality of tests alongside their unit and peer testing activities.

Execution: let’s go to Paris

By the time UPS scheduled the upgrade of their development environment to Paris, the ATF regression testing suite was complete, the two-week plan was finalized, and preliminary reviews of the Paris functionality were completed in a sub-production environment.  UPS was ready!

Week 1
The Development environment was upgraded and skipped records were reviewed.  Skipped records were processed and automated tests were executed.  Adjustments to customizations were made, and tests scripts tweaked where necessary.  By the end of week one, the Development environment was fully functional on the Paris release.  Updates were batched and made ready for the next environment.

Week 2
The Test environment was upgraded, update sets were applied, and automated tests were made ready.  Process owners worked through their regression test suites and added supplemental manual testing where time allowed. By the end of the week, all process owners gave the green light to continue with the upgrade into Production.

Production was upgraded over the weekend, and by Monday morning UPS had upgraded Development → Test → Production to the new Paris release.

Summary: how did it go?

Some schematic challenges with clones and scheduled upgrades caused a bit of delay through this process.  Going forward those lessons will be applied in future upgrades.  Over time, automated testing will become more familiar to process owners and leveraged for major releases in addition to upgrades.  

Thanks to the combined efforts between the UPS and Engage ESM team, UPS has the framework and the template to upgrade quickly with little risk to deliver new business capabilities with speed, agility, and quality.

Get in touch with our ServiceNow Elite experts to learn more about the Now Platform releases and how we can use them to boost agility and resilience for your business.