IMLP Rotation Close-Up: Software Testing

During college many of us have probably had projects where we needed to run and document tests on an application we had created. Or we had to memorize different types of software testing to answer multiple-choice questions on an exam. In my current IMLP rotation, however, I have had the opportunity to lead testing for a new interface system, and begun to understand why this part of the software development life-cycle is so important.

The Project

The project I am working on is an interface between Oracle ERP and Proficy PMCS (Production Monitoring and Control System). The ERP keeps track of inventory and materials as well as production scheduling, while the PMCS communicates with the machines in the factory and tells them what to produce and when. Messages telling the PMCS what to produce need to come from

ERP, and PMCS needs to send information on what has been produced in return. Because the two systems can’t talk directly to each other, an interface has been created using Oracle Fusion Middleware. This interface runs transformations and basically “translates” the messages into the correct format for each system. The stakeholders for this project included the manufacturing team that runs the shop floor, as well as the supply chain team who schedules production and tracks materials. With such a direct impact on the manufacturing process, this project has played an essential in the start-up of the new Battery plant here in Schenectady.

Both the ERP and the PMCS had to make some changes in order to be able to send and receive messages from the interface. This meant that there were many places in the system that needed to be tested and a lot of possible failure modes. All three systems were tested individually first, and then we were ready for my part of the project: System Integration Testing (SIT) and User Acceptance Testing (UAT).

System Integration Testing (SIT)

The first time we ran the system all the way through from ERP to Fusion to PMCS and back was during SIT. During this week-long testing process, we conducted more than 70 tests to try out all the functionality of the system. The goal here was to identify as many bugs as possible, fix them, and run the same tests again to catch other errors. Our SIT was run on our development environment – a copy of the system that is used to develop changes before pushing them into the production environment. This allows you to use fake data, run as many tests as needed, and make changes along the way.

My role in our SIT was to facilitate brainstorming our required test cases beforehand, and ensure that we completed each one successfully during testing. During the week of SIT, I kept track of defects and issues as they were identified, and ensured that each was resolved. Finally, I sent a daily report to stakeholders of our progress and open issues.

User Acceptance Testing (UAT)

After we successfully passed all our tests in SIT, we began to prepare for our second round of tests: User Acceptance Testing. Our project was unique in the sense that the users would not actually interact with the interface itself, but only with the ERP and PMCS systems on either end. However, that didn’t change the fact that we had to ensure that they knew how the interface worked and how their actions could impact its ability to function correctly. During this phase of testing, we had the users run as much of the tests as possible, and got their sign-off that the system was functioning as expected.

UAT, as the second phase of testing, was run in our Quality Assurance (QA) environment. This is a step between development and production – an exact replica of production used to test changes that are developed in development before making them live.

My role in UAT was similar to what I did in SIT, and I also coordinated with our end-users to get their involvement in testing and training. At the end of UAT, we received approval from all of our users to go-live with our system in production on schedule.

Go-Live!

Successful testing wasn’t the only requirement before we were given the full approval to go live, but it was a significant portion of what needed to be accomplished. After tying up the other requirements (documentation, failure analysis, etc.) we were able to successfully deploy our interface to production on November 20^th. Thanks to a meticulous and thorough testing strategy, it was a relatively smooth and successful go-live!

Lessons Learned

Though I had learned about the software testing process as a part of various classes and training, I never understood how important it is until this project. After seeing how many tests we ran, how many issues we identified, and how much the system impacts this business, it was clear to me that this is an essential step that needs to be done carefully and thoroughly. The most challenging part of the process for me was how tedious it could be at times: running and re-running tests would sometimes take hours. My favorite part of the process, however, was being able to pull together the various teams involved with this project in order to work through issues that arose during testing.

Has anyone else been a part of testing for a software project? Can you share your experiences and advice?