You need to find the nest and kill that. You can find organic pesticides. Two of them are Borax and Diatomaceous earth. Borax is a naturally occurring compound that kills insects. After an ant ingests borax, it destroys the ant's digestive system. Diatomaceous earth is a special kind of earth that dehydrates ants. It draws so much moisture from their bodies that they die.
Keep your place very clean.......and use just a little cooking oil on the areas that are getting ants. Spread it very thinly. I decided to do this at my home, and found it works just great! I thought of it, just thinking that they couldn't breath oil, even just walking in it. Try it.
Put a container of water by where the ants are. Add a drop of dish soap. The way it works it that when the ants go to investigate they break the surface tension of the water, when they try to get out, they can't because of the soap