The smoothness usually requires the lens with no outlining in the blur circle, that the blur circle near corners/edges not be too prominently optically vignetted, and in many cases, requires the lens to be well corrected, except for LoCA and spherical aberration (undercorrected), the single one factor to extend a smooth transition or DOF, when one is talking about a subject in the center with a background OOF. Human is equiv approx to f8 to f11 FF as seen by 40mm lens. So if using a 200mm lens, one wants to not be close to the subject, and keep a certain (smaller) distance between subject and background.
One way to see if one lens may be better suited to the impression of depth (and not distractions) is to shoot point light sources while focusing in front of those lights. If the blur circles (say you shoot Christmas lights) doesn't have bright edges, and if it has some slight higher intensity in the center that the outer parts, but is still sharp while focused, and when you shoot this lights near the edges of the frame they don't become too pronounced "cat eyes", chances are the lens will have rendering that makes subjects appear more like they really are there in 3D in the scene. If the subject doesn't have much depth, then one can use a very flat (no spherical aberration) lens too, and the idea is that the part that your retina focuses on do not have blur in the subject. It can transition to blur (important to be smooth) but things like one eye lash in focus and another not, or hair going forward start in focus and end in blurriness...this kill the 3D impression.