Depends on your area and customer base. I work for a company that's 100% commercial. Very loyal customer base, good paying accounts. I mean, yeah, we're afraid of losing a PITA account next year, but it happens. And some places are cheap, but if they were so cheap, then we wouldn't have bid them in the first place.
You might run into problems with fast food joints, mom and pop stores, and little stuff like that. You might make a killing off HOA complexes and retirement communities like we do because you're equipped to do so and the property manager is willing to pay your company $25k more per year than Brickman.
You might not be able to handle the high-paying loyal commercial customers yet. Maybe residential is better for you. Sounds like you like resi accounts better, so maybe you'd better stick that route for a while.