I guess you should stomp down more?
You aren't taking away dirt are you?
Physically speaking if you insert a item into something ( Pipe into dirt) you are displacing the original compound no matter what.
with the mere fact that it's almost impossible to compact dirt the same as it was. Simply digging up the ground and putting it back together generally displaces it ( To an amount thats usually not noticed)
So generally you should have an overage of dirt. This isn't always the case. due to wash off .. pipe breaks.. hills ects..
Typically you should compact dirt every 2-4 inches to get a good solid compaction