Most infants first encounter language through the words spoken in their environment. However, for a smaller number of deaf and hearing infants, language can be presented in different sensory modalities, including a visual-manual signed language (e.g. American Sign Language - ASL or British Sign Language - BSL) and an auditory-oral spoken language (e.g. English). Language acquisition trajectories for children exposed to both signed and spoken language are less understood and less recognised. For hearing children with deaf parents using sign language, recent research suggests that they develop a special case of bilingualism - bimodal bilingualism - which offers some advantages in early communication skills. In deaf children, it has now been clearly demonstrated that early exposure to sign language brings about gains in both the spoken and signed modalities, suggesting an amodal impact of language experience in infancy. The present review presents progress made in the last 25 years in understanding the impact of sign language experience in infancy. It will discuss potential neurocognitive mechanisms by which learning gains in one language modality can be transferred to the other language modality. The research data collected so far leave several questions unanswered and suggest many avenues for future research.