A wide range of behaviors can be formalized as instances of probabilistic inferences. This includes odor recognition, navigation, motor control, decision making, visual search, simple arithmetic and causal reasoning, to name just a few. In all cases, the probabilistic inferences involve a type of operation known as marginalization. We will show that, given the variability reported in neural responses, marginalization can be implemented in neural circuits through a nonlinearity known as quadratic divisive normalization. This approach makes very specific predictions for any task involving marginalization. We have tested these predictions for the specific case of visual search and found that human subjects behave near optimally. Moreover, we will show that a network with quadratic divisive normalization provides a tight fit to the human data. A similar approach can be used to obtain an approximate solution to intractable marginalizations such as the ones involved in olfaction. This approximate solution maps very naturally onto the architecture of the olfactory system. This work suggests that seemingly unrelated behaviors, such as visual search in humans, or olfactory processing in rodents, could in fact rely on very similar neural mechanisms.