Upload & Sell: Off
Just because you might use the fast lenses more, that doesn't make the 40 a paperweight.
Perhaps you might use the lenses in proportion to what they cost as a fraction of the cost of the three.
Much depends on what you use your lenses for. If you use them at f/1.4 most frequently for low light purposes, then you'd miss that in the 40 mm f/2.8, though less so on modern high ISO cameras. If you use them most for shallow depth of field at f/1.4, then you'd miss that.
However, the 40 mm f/2.8 might cure some photographers of the razor thin Depth of Field fetish.
Besides, it is less risky and lighter and smaller and less obvious to carry around than the big honking lenses.
But only you would be able to say what you use the lenses for, though you haven't said so far.