Coming from the Zf with the Nikkor 40mm f/2, the size and weight of the Voigt Lanthar 35 f/2 was ultimately a deal-breaker for me. It wasn't the size or weight in itself, but the front-heavy imbalance I felt when it was mounted on the Zf. I use a leather half-case rather than an attachable grip, which probably contributed to it.
In exchange, I've since “settled” for the “imperfections” of the Voigt 40mm f/1.2 Nokton. I never would've imagined allowing the size or weight of a lens to deter me, but alas, never say never or always.
It's also not lost on me that plenty of native Z glass exceeds the Lanthar in size and weight, so I may be excluding myself from some excellent lenses on the same basis. I suppose that, if I were truly committed to a particular lens, I'd adapt and find a way, perhaps even switch bodies.
But not in this case, at this point, or for my purposes. Just my $0.02.