Why do planets start off small, get the largest midway through, then drastically shrink?

I noticed this after looking at the diameter of 2018 vg18. 2018 ag37 is larger, but at times it does come into the Neptunian orbit. I think it is maybe different from vg18. But why is this trend (generally) here?