I've been designing sites and working with various hosts for the past 10 years. I finally ended up with solely using Network Solutions. They're not the cheapest but they probably offer the best support, the best security, the best server management, and the best offering of free features.
The support is especially critical, I can reach a highly skilled troubleshooter 24/7, who not only identifies the problem but fixes it for me. Working with other cheap hosts has been a nightmare on the support side - long waits on support lines, incompetent support people, sometimes not even being able to reach a human. Poor support costs me many hours of wasted time. Support is especially critical if you're using a CMS system like Joomla or WordPress.
Another place where cheaper hosts fall flat is with customer support regarding billing and account management. Many horror stories to tell of customers getting overcharged, not being notified when bills come due, and difficulties in closing and transfering accounts.