Internet of Things (IoT) systems have become increasingly complex, with many including a vast array of apps, data systems and devices. This new hyper-connected world has created a diverse and ever-changing landscape for testing and DevOps teams, that many are struggling to navigate.
Testing and test-automation have not evolved much throughout the last 20 years and are still stuck in the dark ages. This is why most teams cannot keep up with the demands of users, DevOps, device fragmentation or technology evolution. So what do teams need to do in this new dynamic world? Below are five steps that DevOps and testing teams must take to deliver smart testing in an IoT-driven world.
1. Test the Product, Not the Code
Teams need to re-orientate from focusing on the code to the product and the actual user experience. With the IoT, products and services are composed of technologies from a range of vendors, and user experiences thus comprise products from an array of vendors, and the actual user experience is critical to adoption.
So if you only test the code you write, you’re probably only testing 5 percent of what’s impacting the user experience, you’re hoping that everything else just magically works together, and so things are going to go wrong. Therefore, you have to move from testing the code, as smart IoT only magnifies the gap between testing code and testing the product testing is no longer a compliance function.
2. Test Channel Consistency
There are now a range of interfaces for accessing products and services, such as a mobile, Web or voice interface, or an on-device screen. The product may even be interacting with other products via APIs. If two people are getting different information about where an item is located in a warehouse, this will create a great deal of confusion and errors. DevOps teams testing IoT systems need to ensure users receive a consistent view of the service, independent of the interface used. So it’s essential with smart IoT systems to test for channel consistency.
3. Automate Testing
Manual testing cannot be the foundation of test strategies with the IoT, due to the complexity and variation of products and services. Automation of test execution doesn’t suffice, as the entire testing process from creation through to analysis needs to be automated. This requires intelligent models to auto generate tests, with AI, ML, and analytics allowing DevOps teams to analyze data from testing, and to identify the patterns with bugs.
4. Converge Testing and Monitoring
By converging testing and monitoring, DevOps teams bring the user into the automated testing process, so testing is not just product-focused—it’s really user-focused. Teams observe what users really care about, what impacts their productivity, what impacts their effectiveness, and what impacts their sentiment. They can then use this to determine whether the test was a pass or failure for a smart IoT product. The IoT is often about bringing technology deeper into our lives, and user acceptance is critical. Therefore, bringing the user into testing is even more critical in the IoT than elsewhere.
5. Get Ready for Load Testing
As the IoT continues to gather steam, DevOps teams need to start thinking about load testing. Many companies add IoT solutions to their existing IT infrastructure, and very few are ensuring that their infrastructure can handle the resulting surge in data. Load testing is an essential preventative measure that DevOps and testing teams must undertake to ensure their network can cope with the explosion in data volumes without impacting the product or user experience.
So testing today is not ready for smart IoT, and if nothing changes it will lead to delays in deployment, updates and user acceptance. However, by following these five steps, DevOps teams will be able to deliver smart testing in an IoT world—and ensure that the digital experience delights.
Antony Edwards studied computer engineering at the University of South Wales, Australia. He worked as a developer in Sydney before joining IBM Research in New York. Moving to London, Antony joined mobile operating system builder Symbian, moving from system architecture to eventually become a VP and member of the executive team. Prior to joining Eggplant, he held the position of CTO with a major U.S. online entertainment company.