This post is part of the SATF work.

The section elaborates on one particular set of simulation auto acceptance tests – the Monte Carlo localization service tests. These tests are used as an example in the most of the following material, so they are worth of individual scrutiny. By the same token, the MC localization case is complex enough to illustrate the most important aspects of the testing framework including probabilistic testing. Although its implementation part looks Greek at the moment (further sections will clarify it), it is also presented, along with the general description of the tests and problem, to provide the taste of what the whole idea is worth in reality. Continue reading →

This post is part of the SATF work.

One particular testing issue requires some special measures, but could also be addressed within the approach proposed in the “Solution” section. The issue is testing of the nondeterministic aspects of robotics systems, which are inherent and abundant within the field. If the outcome of a given test is nondeterministic i.e., every single run of the test, the initial configuration being the same, could end up with either a failure or a success, then neither of them would imply the definitive answer to the real question behind the test – an attempt to assess some nondeterministic process. There are two distinctive flavours of this problem. Continue reading →