It matters since the relation of the body to the hub changes when lowering the car.

How much it matters? Not sure on that one. Stock preload spec is 8mm above hub center. Maybe increase it a few...