Before you adjust the idler pulley CHECK the transmission belt's tension
because if it's tight there's no reason to adjust it.
Once you loosen that dang bolt it can be a real PITA to get it back just right,
ONLY ever adjust that thing if the belt underneath driving the transmission is loose.
Invest $120 or so in two new wheel belts, put them on.
If that doesn't fix it report back.