In practice it's not always easy to determine which prime factors make N, so one successively tries those on file.
Like when you asked about 15, 2 doesn't work so we know to try 3, but sometimes it's not obvious with which odd prime to start, so one plows through the ones on file in succession, up to sqrt(N), at which point one should have found one if there is one. If not, add to the stash and proceed.
Put another way, factoring is a non-trivial problem and once the integers get big enough, we run out of memory and/or time given present capacity.
Which is not to denigrate your algorithm. The algorithms giving us primes of many thousands of digits tend to shoot for "probable primes" which means very very unlikely to be composite. Composites sneak through the filters.
Like remember the Carmichael numbers: they pass all the Fermat tests suggesting they're prime, but they're not: