Some notes about the tests:

1. The first the method has a smell: [Conditional Test Logic][1]. I'd extract out the `do-while` loop to an `enumeratePrimesBelow(ulong upperLimit)` method. It would improve readability a lot.

2. 

        Assert.IsFalse(Primes.IsPrime(1));

 I'd create a [custom assertion method][2] for that:

        public void AssertIsNotPrime(ulong value) {
            Assert.IsFalse(Primes.IsPrime(value));
        }

 It would make the tests easier to read.

3. `_1IsNotPrimeTest`, `_2IsPrimeTest`, `_3IsPrimeTest`, `_42IsNotPrimeTest` etc. should be a [parametrized test][3]. I don't know whether C# unit testing frameworks supports it or not.


[1]: http://xunitpatterns.com/Conditional%20Test%20Logic.html
[2]: http://xunitpatterns.com/Custom%20Assertion.html
[3]: http://xunitpatterns.com/Parameterized%20Test.html