Divergent populations across different environments are exposed to critical sensory information related to locating a host or mate, as well as avoiding predators and pathogens. These sensory signals generate evolutionary changes in neuroanatomy and behavior; however, few studies have investigated patterns of neural architecture that occur between sensory systems, or that occur within large groups of closely-related organisms. Here we examine 62 species within the genus Drosophila and describe an inverse resource allocation between vision and olfaction, which we consistently observe at the periphery, within the brain, as well as during larval development. This sensory variation was noted across the entire genus and appears to represent repeated, independent evolutionary events, where one sensory modality is consistently selected for at the expense of the other. Moreover, we provide evidence of a developmental genetic constraint through the sharing of a single larval structure, the eye-antennal imaginal disc. In addition, we examine the ecological implications of visual or olfactory bias, including the potential impact on host-navigation and courtship.