This work describes a framework for automated acceptance testing of robotic systems within Microsoft Robotics Developer Studio environment. The framework provides means for development and organization of a test suite and a tool for its automated execution. Each test proves correctness of some high-level system’s functionality by performing a sequence of actions in a physics environment and checking the result against a success criterion, which, generally, assesses the state of the software and the world. These actions take place inside of the MRDS simulation engine being in charge of physics and graphics simulation. Each test consists of three constituent parts describing physics environment, software configuration and the test’s scenario. The first two are xml, the last has to be written in C#. The framework also handles the case of tests with nondeterministic outcomes by rerunning each of them multiple times and assessing resultant success rate. The framework proved considerably beneficial through development of a mobile robot control system containing services like localization, path planning etc.

Disclaimer: MS has abandoned development and support of MRDS framework.

The project’s code resides on GitHub. Feel free to contact me if you are interested.

Table of contents

• Introduction
• Part I. Contemplating
  1. Problem – the challenge of a change
  2. Solution – automated acceptance testing for robotics
  3. Probabilistic testing – rinse and repeat
  4. Test example – Monte Carlo localization
• Part II. Designing
  5. MRDS review – DSS, CCR, simulator and friends
  6. Test flow – setup, execute, assert
  7. Test structure – stage, players and scenario
• Part III. Implementing
  8. Tests organization – for we are many
  9. Test runner service – a bundle of joy
  10. Time handling – real time faking
• Part IV. Analyzing
  11. The test suite
  12. Good things
  13. Bad things
• Conclusion