Is there a better way of doing this? If so, how?
Yes, there is. Your current algorithm is \$O(n + (n*n / 2) + (n/2))\$, or \$O(n^2)\$. (The first \$n\$ is the iterating of the numbers, the \$(n*n / 2) + (n/2)\$ is the time complexity of a triangular number.) If you use the Sieve of Eratosthenes, which works like this...
(image courtesy of linked Wikipedia article)
You get \$O(n(logn)(loglogn))\$, which is faster. See this stackoverflow question on how the time complexity was determined.