A potato helps power a light bulb by acting as both a bridge and a separator for two electrodes made of different metals. The two metals most frequently used are zinc and copper because of their ubiquity and difference in electron affinity. The acid in the potato starts the reaction, and the electrolytes in the potato allow a flow of electrons, an electric current, to flow between the electrodes.
It is important to note that the potato itself is not a source of energy, at least in this case. While it does provide chemicals necessary to get the reaction started, the reaction that produces energy is ultimately between the metals in the two electrodes. However, the metals have a potato between them, and so cannot undergo chemical reactions directly. While metal ions cannot flow through the potato, electrons can. The flow of electrons that results from the chemical reaction is literally an electric current. At least one of the electrodes is gradually worn down by this process, losing mass as solid metal compounds with the potato's chemicals.
The power output of potatoes is very limited, and most light bulbs can be powered only dimly or briefly. With LED lights and a cooked potato, however, scientists have actually achieved impressive results with potato batteries.