Grassmaster Wilson said, If you are in the business then you at some point you need to take all of these. Once you get a good base and don't care to pick up ton more work you can start dictating what you do. You can slowly raise the price on accounts you don't want and say no to ones you don't care to touch.
That's pretty much what I was trying to agree with. As you grow you should tailor your services to best serve your own goals and to serve your clients. My demographics are probably different than yours. I serve a metropolitan area with @ 500k prospective clients. You may be in a rural area with less prospective clients. I have fired a few clients that I didn't want to keep; but of course always kept my word and kept the contract until it was at the end of the term. In my case I found that I couldn't be "all things to all men".