I'm preparing to order an Orvis shirt myself. After chatting with them, the "navy" is in fact more of a "grey." So that's probably what you'll want to order. I'm still debating between it and the sky blue. As far as the sizing goes, I'm similar to your with a slightly smaller chest. Since the Long Sleeved Open Air Caster is one of their Active Fit Shirts, they run a little bigger/smaller in specific areas. Here are the more detailed sizes for both a small and a medium:
SMALL: Chest 41.5 Bottom Circumference (Hem) 40.5 Neck 16.5 Sleeves 33.5
MEDIUM: Chest 45.5 Bottom Circumference (Hem) 44.5 Neck 17.5 Sleeves 36
Based on those measurements, you and I should both probably order a size small from Orvis. Hope that helps!