When that happens, type Q to go into Quick Mask mode and tweak your selection manually. In Quick Mask, the red tint represents your unselected area. Type B for the Brush tool, then D to make sure black and white are in your color palette. Start with a brush at 90% hardness, then paint with white where you want to add to your selection, or paint with black to remove it. Zoomed in close, check all the edges, making sure you've selected only what you need. Type Q again to exit Quick Mask mode.