Is there a better way of doing this? If so, how?
Yes, there is. Your current algorithm is \$O \left( n + \left( n \times \dfrac{n}{2} \right) + \left( \dfrac{n}{2} \right) \right) \$, or \$ O(n^{2}) \$. (The first \$n\$ is the iterating of the numbers, the \$ \left( n \times \dfrac{n}{2} \right) + \left( \dfrac{n}{2} \right) \$ is the time complexity of a triangular number.)
If you use the Sieve of Eratosthenes, which works like this...
(image courtesy of linked Wikipedia article)
you get \$ O \left(n \left( \log n \right) \left( \log \log n \right) \right) \$, which is faster. See this Stack Overflow questionthis Stack Overflow question on how the time complexity was determined.