Applause helps to ensure that digital products work from the user’s point of view by testing its clients’ software with real users, on real devices and under real conditions. Critical bugs & issues are uncovered before they can impact the customer experience and business success of the clients. We talked to Jan Wolter, General Manager EU at Applause, about crowdtesting and the new opportunities and challenges that AI brings to both Applause and its clients.
Scout24: Crowdtesting is one way to make sure software works properly. What exactly are the advantages of using this approach?
Jan: By using crowdtesting, you can be sure that all testers are checking the product in an unbiased way and are able to find problems internal testers might not have thought about yet. Furthermore, it allows testing on a variety of different devices and operating systems under real-world circumstances. One of the most significant advantages is the fact that the testers can be chosen according to the real target group of the customer — in terms of location, language or interest, for example.
Scout24: How did the rise of AI affect your work?
Jan: In our own platform, we made use of machine learning and AI in order to automate the process of choosing the best candidates for specific projects. Not only does it make work more efficient, it also eliminates possible human bias that could affect the selection process. That way we make sure that the best candidates are being chosen to test clients’ digital products.
We also launched a new solution to train and test clients’ AI experiences. The solution leverages our global community of vetted testers to deliver the widest possible range of training inputs. The results are then tested across every possible device, location, and circumstance to identify issues and provide actionable user feedback in real-time. This enables today’s leading brands to identify issues of quality or bias earlier in the development process so that they are ultimately delivering top-quality AI experiences for their customers.
Scout24: There are still problems arising when companies want to make use of AI and Machine Learning. What are the main reasons behind these failed implementations of AI?
Jan: All types of AI — from virtual assistants learning how different users ask for the same thing, to nutrition apps identifying food from uploaded photos — have been hampered by the same challenge: sourcing enough data to teach the machine how to interpret and respond, and then testing the output at scale to ensure the results are accurate and human-like when necessary. Oftentimes, AI will work as expected in the lab, but when it is released into the real world, companies find the AI has unintended bias, meaning that it does not work for a portion of customers. AI needs a diverse group of testing and training data from real people in real locations in order to be fully inclusive of all users.
Scout24: Would testing AI algorithms through a company such as Applause help to minimise these problems?
Jan: Testing is definitely useful to improve the algorithm and to ensure it is using the data it has been provided with properly. In terms of finding the right data and creating an unbiased program, Applause gathers a huge amount of data from people across many different countries, ages, and cultures, among other criteria. The data provided is diverse, as are the people who test the AI output. Thanks to the testers providing instant feedback, the AI can be constantly adapted and improved.
Scout24: Could you name some of the most common engagements AI is being used for and how the testing procedure would work for those?
Jan: Applause’s AI training and testing solution operates across five unique types of AI engagements:
- Voice: Source utterances to train voice-enabled devices, and test those devices to ensure they understand and respond accurately.
- OCR (Optimized Character Recognition): Provide documents and corresponding text to train algorithms to recognise text, and compare printed docs and the recognised text for accuracy.
- Image Recognition: Deliver photos taken of predefined objects and locations, and ensure objects are being recognised and identified correctly.
- Biometrics: Source biometric inputs like faces and fingerprints, and test whether those inputs result in an experience that’s easy to use and actually works
- Chatbots: Give sample questions and varying intents for chatbots to answer, and interact with chatbots to ensure they understand and respond accurately in a human-like way.
Scout24: AI will probably become more and more important in the next few years and is already being implemented strongly. Does this mean that testing AI is already an established routine?
Jan: Contrary to the expectation that AI testing should have a high priority, there are still objections against it. First of all the budget question. A lot of companies don’t want to spend the money even though it is important to ensure the quality of the product. That is what the customer expects and what should be delivered to them. The pressure to deliver as soon as possible also causes companies to save at the wrong ends. Testing can be integrated into the development process without causing any delays whatsoever. It should definitely become a more established standard in the future.
Scout24: Looking at the future, do you think AI will be broadly implemented in most applications and is the fear of humans being less effective justified?
Jan: Companies across various industries are implementing AI at the moment, but of course, it won’t be suitable for every single application even if that is what everyone aims for. The fear of humans being less effective and efficient or the hope that AI might solve all problems in this regard is still not reality. Companies should be aware that there is no one-size-fits-all solution in terms of AI and machine learning.
