I have the non AF-S version. Unless you are shooting birds etc and require the faster aperature, this thing is tack sharp my friend. If speed is an issue, go af-s, if not the ED-IF is perfect IMO.
The 300 f4 af-s is a very nice lens. Versatile. I miss mine. The 300 2.8 is a beast with the only advantage is it gets a little more ss in the dawn and dusk when birding. Both lens hunt in low light, low contrast situations. Definiton of low light is a little dimmer for the 2.8. Cost benefit goes to the 300f4. I also liked the f4 as a close up lens when the 500D is added.
I love the 300 f/4. Its light and like Jericho said, tack sharp.
I bought mine from KEH and got an awesome deal. It is small enough to enjoy taking with me on trips and hikes when the 300 f/2.8 is just too darn big and heavy to be carrying.