They are not YOUR customers, they are your former boss's. They hired HIS company to do the work and he put you in charge of making sure the work was done. Why would you DESERVE his customers when HE built his company, HE marketed and sold his services to those customers and then HE paid you to take care of them.
Get over yourself and quit being a little rat. If you think you are that much better than him, start your own company and find your own customers and if his clients are that upset with his service they will start looking for someone else and if YOU have spent YOUR money to market properly they may find YOUR company and hire YOU. Then you can rightfully claim them as yours.