Keep in mind that any hosting that costs less than $25 a month is "shared hosting". This means there will be several hundred other websites residing on the same server as yours. This helps keep costs down but depending on how much resources the other websites need and how much traffic they draw, it can start slowing your site down or start creating downtime. The cheaper web hosts typically pack more than 1,000 sites on a single server and offer them all "unlimited" bandwidth and disk space. This is what creates problems.
Some of the higher quality hosts that have been mentioned here keep a closer eye on their servers and don't overload the servers quite as much as the cheap guys. This results in more uptime and faster page load times when your customer loads your website. It's very hard to judge who is good and who is bad so you have to pretty much rely on word of mouth but most people don't understand this and just go with the cheapest provider.