Testing a probabilistic prime-testing algorithm that may fail

Question

I have implemented Pollard's Rho algorithm in Java. Due to the nature of it there is a small chance for it to fail (have not seen it happening yet).

Since I do not want to see my unit tests failing due to this "too often", I set a retry, with which I am not totally satisfied, as you can see here:

@Test
void primeFactorsCalculatedWithPollardsRhoAreCorrect() {
    Operations op = new Operations();
    int remainingAttempts = 5;

    while (true) {
        try {
            assertEquals(5, op.primeFactorsPollardRho(13195));
            break;
        } catch (UnsupportedOperationException e) {
            remainingAttempts -= 1;

            if (remainingAttempts == 0) {
                throw e;
            }
        }
    }
}

I am using jUnit.

Is there a better, more pleasing to the eye, way of doing this?

Answer 1

Your test is missing the central point of unit testing which is to prove the code under test works in the specific circumstances in question, these should be deterministic. So for each parameter you can know in advance what the result should be. Create multiple unit tests for the known factors that should work and a number for the parameters expected to fail.

Secondly, I suggest you create your own Checked Exception for the failing circumstance rather than reusing the runtime UnsupportedOperationException, this will ensure a client using your function/library is forced to handled the exception. When the error can be handled use a checked exception, only use the unchecked runtime exceptions when the error cannot be handled by the code. If the program can, it should still fail gracefully with either. I can see that is what you are trying to achieve for your live code but visibility of the issue is more important in unit tests.

The client code handling exception could skip the value and proceed to the next value. Simply retrying with the same value should produce the same result, so the retry with the same value is actually pointless.

You can verify that the test produces the result you expect by catching the expected failure with the expected Exception clause of the @Test annotation.

@Test(expected = BirthdayParadoxException.class) 
public void testBirthdayParadox() {
    op.primeFactorsPollardRho(BIRTHDAY_PARADOX_VALUE); 
}

@Test 
public void testPassingValue() {
    assertEquals(EXPECTED_RESULT, op.primeFactorsPollardRho(FACTORABLE_VALUE); 
}

Create multiple tests for each specific situation you can predict and verify the expected behaviour, catching failures should be considered expected behaviour for unit testing purposes.

Also see : https://github.com/junit-team/junit4/wiki/Exception-testing