At that age he should be ok to go into the men's room by himself, as long as your mom is standing right near the door to keep an eye on him. Personally I'm probably the wrong person to ask though, my youngest is severely autistic and I have to take him into the restroom with me all the time, since he can't go by himself even though he's 12. I quit caring what anyone had to say about it a long time ago, but oddly though no one's ever really given me any trouble about it. I think women are more understanding about that sort of thing. But for the most part if he is capable of taking care of everything by himself, then he's probably good to go, unless there is a specific reason to be worried that it's not safe.
I don't have kids but if I did it would be hard for me to let a kid be out of my sight. But I guess if I stood right outside the door it would be ok as long as the kid was ok with it too. Sephira I think said it best. What Lady is really going to say anything? They all know the dangers that can lurk behind closed doors.