Assert.Inconclusive

re: Assert.Inconclusive

Pascal Craponne — Mon, 24 Jan 2011 21:28:56 GMT

Unfortunately, Assert.Inconclusive marks the build as partially failed when using TFS build and automated tests.

In this case, there are two options:

- make the tests more reliable (for example retry twice your ping).

- remove them from release build (in my job, I run the tests locally in debug configuration and TFS build runs them in release configuration, so a few differences can be made)

re: Assert.Inconclusive

Emil Gustafsson — Sat, 22 Jan 2011 18:41:23 GMT

@Matthew: Yes I agree it's a cop-out. And frequent inconclusive tests should be addressed and fixed. But when yo only get occasional failures due to environment glitches I think it's better to signal it as a known glitch than just failing. Naturally you can fail with a good message but my experience is that differentiating known glitches from actual failures leads to less stress in the team since it's easy to identify the "glitch failures".

re: Assert.Inconclusive

Matthew — Sat, 22 Jan 2011 15:12:04 GMT

I don't feel very comfortable with this idea. It feels like a bit of a cop-out for when we don't have the time to make sure our integration tests run in a reliable/repeatable manner.

re: Assert.Inconclusive

chrisgl — Sat, 22 Jan 2011 11:20:50 GMT

I think this is a way Assert.Inconclusive is meant to be used.

I think including an informative Message, so the Tester knows which Problem to Fix to get the Test running, is very important.