Federal tax is taken out immediately. You can file a w-4 and claim EXEMPT and the won't take any fed tax except FICA. If you claim exempt make sure you anticipate paying ZERO fed taxes. Usually students who earn a few thousand dollars a year will file EXEMPT.
Federal taxes are taken out from the time you start making any money at all. Also FICA or social security, and Medicaid. Not all states take state tax, that depends on the state that you are working in. Where I live and work, they do.