Upload & Sell: On
I'm not sure what you mean by "there's not a large area for the dirt to be distributed to".
I don't baby my lenses. Most of the time I just breathe on the lens and wipe it with my t-shirt. Only once in a while I use lenspen.
The little felt cleaning tip of a lens pen has a very small surface area. Because of that it doesn't take very long for it to become saturated with dirt/dust and once it does the dirt isn't going anywhere.
Microfibers were a revolution in the car detailing community where they have been the de-facto standard for some time now. The reason being is that traditional materials, namely cotton, are actually quite abrasive and can easily leave scratches in paint. Microfibers are more absorbent, softer, have much better mechanical cleaning properties (the thousands of micro fibers adhere themselves to individual pieces of dirt) and they do a better job of encapsulating the dirt so you're not just rubbing the dirt around on the surface. The only time I used cotton towels was when I was working with an abrasive polish and required more "cut" to correct defects. Now that I have an orbital buffer with foam polishing pads (which are also abrasive), I don't even use them for that.
The risk you run whenever you clean your lenses is putting small scratches/abrasion into the lens coating or glass, either due to the roughness of the cloth or the fact that you're rubbing dirt that's trapped between the cloth and the lens surface into the lens. This is why it's important to use a clean, good quality microfiber.
That all being said, is it going to matter? No. Tests have shown that the front element can have huge amounts of dirt, scratches, fungus, or even be outright broken without significantly affecting the overall image quality. But, it is going to affect your resale down the line.
I don't baby my lenses either: they're a tool to create images, not collectors items. But, at the same time, they're an investment so I do try and take care of them when I can. Mostly, I don't clean my lenses in the field at all unless it's something totally egregious. I rarely clean them in general, a little bit of dirt or smudges on the element isn't going to affect my pictures, but when I do I clean them carefully as not to induce any unnecessary wear on the glass.
Also, if anyone is looking for some good reasonably priced microfibers: