Why do they call it losing someones virginity?
Its not lile you don't know where it went.
They say that because you lost it, as in its not yours anymore. Just like saying you lost money in a bet. You lost it but you know where it is. ;)
You had something, and it has been taken, that is why losing.