If on an iPhone you will need to go to applications and double click the file to delete it. Another option is to open iTunes and delete the desired application. If using anything other than an iPhone you will need to either to go to Settings or your Applications folder. The folder will have an option to remove the application.
I have a 3GS. This is way simpler to do than the above answer. Press down on the app and it will start wiggling and will have an X in the upper left hand corner of the app. Touch the X. A delete/cancel box will appear. Touch the delete and your app is deleted----very simple.