Upload & Sell: Off
Definitely makes a difference, same as with any other setup. Given a choice, I would pretty much always use a polarizer for any lens. Lenses like my 35 and 85mm f1.4's most of the time have the CPL on them. Lets me go wide open in broad daylight if I want and improves color, contrast, etc. They have the speed to shrug off the loss of light from the filter.
Where I lose the CPL is if my shutter speeds are going down due to dimmer light and I don't have solid support (dusk, dawn, night shooting).
With the 200/2 and other exotics, they're often used under these conditions (struggling to keep shutters up and iso's down) so the CPL gets left in the bag or isn't even purchased.