Had a chance to actually look at it today. I checked, then released the relief valve and then re-tightened it.
When I tested it it actually did good. The filter housing had signs of seepage so I tightened it a bit more (wasn't really loose or anything but could have been tighter.)
Read in the manual I was supposed to put 'Lubrizol' in it when I changed it per the manual. Does this still hold true since I swapped it over to the 20w50 ?