So I've always struggled a bit with getting my shots...first, in focus, and second - sharp. Usually I shoot sports, so OOF can always be put down to a range of factors.
Yesterday, however, I attended a portraiture workshop. Using a Canon 30D and Sigma 70-200 2.8, shooting mainly at 2.8 or 4.0, at a range of focal lengths, with a model anywhere between 10-20m away.
Quite often I'd focus lock and recompose just a little to reposition the model in the frame, and I've found a lot of the time this resulted in OOF shots (the eyes......you can always spot it in the eyes.....),
Having a quick look at a DoF calculator, and I don't believe that the amount I recomposed would have changed the subject distance significantly, so why did I wing up with around 15-25% of my shots being out of focus?
In one example, I had 2 shots straight after each other, without myself or the model moving. One was OOF, was was in focus - what would cause that?
My shutter speed was generally at 1/400 or faster.
Have you got a good, steady tripod available? If so, try shooting with the mirror locked up, and a remote shutter release (or timer delay on the shutter), to test focus on some static subjects. This will help to rule out user-error and/or out-of-adjustment equipment. With "around 15-25%" of your shots out of focus, the problem could be either, but testing on a tripod should yield consistent results. Test in good, natural light, and don't shoot some silly angled ruler as a test target.
A lot of variables as you say.
Sometimes a couple images with the settings you used are helpful.
I'd especially like to see images without focus and recompose.
Process of elimination can help too. Do you have another lens and body to mix?
Your camera-lens combination might be right on the edge of the depth of field such that a slight variation will put it oof.
Lenses do not focus in the same spot every time, there is a variation.
This is a situation where having a body with AFMA can really help. 10-20m seems like a long distance away. (33-66 ft) There should be a huge depth of field.
Af is off often. 1/400 on the slow side at 180+ I like 1/800 or 1/1000 but use 1/500 or slower. 1/500 or 1/640 maybe enough. I rarely use a tripod. You dont need one if ss is high enough and focus
is on.
Af servo will goof things up. 15-25% with that lens at 2.8 sounds quite good.
By the traditional rule of thumb you need a shutter speed of at least 1/320 to consistently get sharp images of a stationary subject when shooting a 30D hand-held. However, it could well be that your technique is not up to par and that you need to allow for faster shutter speeds. I know that I do, but that could be partly due to medications.
Your technique may result in a level of camera movement at shutter release that is more evident in the first shot of a group than in successive shots. Not uncommon because for the second and subsequent shots you probably won't be moving your finger as much nor changing your grip.
Also be aware that DOF tables are based on making small prints and having poor eyesight. Tell the DOF calculator that you have 20/20 vision and see the DOF plummet.
Another possibility is that you may be touching the focus ring of the lens and slightly altering the automatic focus setting. That could easily happen with the Canon USM lenses unless the camera is set to disable the full time manual focus.
It can be difficult to tell the difference beween inccurate focusing and mothion blur. The latter, where the shutter speeds is appropriate for the focal length, is most likely to be from stabbing at the released button, even when the camera is on a tripod.