If they feel a lot warmer than usual, then yes, he may have a fever. I have an old kitty prone to bladder infections. I feel his ears on a regular basis to make sure his medications are doing the job. His ears feel significantly warmer, almost hot to the touch, when he is sick. If your cat stops eating or is lethargic, then you should probably take him to the vet.
Cats have normally higher body temps than humans (around 102) if you really want to check for fever, get a fast working (digital) thermometer (one you know works properly) lubricate it with petroleum jelly, scruff your cat, and take its temperature anally. It wont like it but it can be done. I've done it. They do it at Vets and then you'll know for sure. Good luck.