Also: Even if I keep my finger over the hole in the primber bulb it works fine. I got some carb cleaner and sprayed the carb out but that didn't do anything.
However, there is a little fuzzy gasket that fits between the air filter and the carb. It is torn. Might that have an effect on it?