Are you a bank?
If not, then use their money.
I think folks like that want to be in the drivers seat. He can have you jumping through hoops. Let him make someone else's life miserable.
In the past I have told clients that I don't hire subs or buy the material until the money is in my bank. I never had an issue with that.
When I built larger projects - additions to houses - I used this method successfully...
I would get a deposit to cover initial costs to a certain stage - lets say footers poured.
Then I would meet with them, with invoice in hand for work completed, and ask for more money to get me through the next stage - lets say framing, then I would meet again and ask for more. The closer I got the end the end, the more frequent the meetings. I was never more that a few hundred in the hole right before the occupation inspection. And I always got that before the final inspection was scheduled. I always stayed ahead of them with the money. And I was able to sleep very good at night.
Getting behind on the money (using your own money) is not smart in my opinion.