Dude, a 50 foot tree is nothing to mess around with. I've done some light logging and selective thinning in ponderosa pine, a 60 foot tree is a little intimidating to fall, even when there's nothing but other trees for it to land on. Digging out the roots on your side of the house isn't smart even if your neighbor didn't mind, but I'm sure he would. If you weaken the root system on one side of a tree there's a good chance that even if the tree doesn't die it's obviously weaker on that side making it more prone to falling in that direction. Not good. My advice is to not mess with the tree at all
As far as renting a machine, just get the cheapest one and go to it, this isn't rocket science. Get a machine, enjoy the day, and pray that you get to do another job that requires the rental of a machine. This was how I got my start at 14, doing small landscape jobs with rented Bobcats and minis.
Don't stress the small stuff, score a machine, keep it simple.
Go hard, go fast, or go home