Depends. On some people (like me) hand soap causes their face to break out and dry. so if you've got sensitive skin then don't use it. I'd recommend face wash anyway because hand soap works by literally killing your cells (which is also why if it doesn't have a moisturizer it dries your skin) do just use something meant for your face with a moisturizer in it.
I use ivory soap and my skin is pretty sensitive and I'm okay so I guess it's fine I just put on lotion a while before i wash my face to moisturize it.... and kinda protect my skin and if you wear makeup it helps remove eyemakeup