Due to the incompatibility with Blender 2.78 I updated the addon to a new version. Thanks to Julian Eisel I managed to fix the error in the add-on. Thank you Julian.
You will notice that the pop-up, when clicking on the "Render > Benchmark" link in Blender, is a bit different. The "Cancel" button is removed, because it wasn't doing anything anyway. A different kind of pup-up is used to work around the error the previous version with Blender 2.78. However the end-result remains the same: The blenchmark scene is opened. If you don't want it to be opened, just move your mouse away from the pop-up, or hit escape.