Deaf and hard-of-hearing (DHH) children are often reported to show deficits on working memory (WM) tasks. These deficits are often characterized as contributing to their struggles to acquire spoken language. Here we report a longitudinal study of a large (N = 103) sample of DHH children who acquired American Sign Language (ASL) as their first language. Using an n-back working memory task, we show significant growth in WM performance across the 7–13-year-old age range. Furthermore, we show that children with early access to ASL from their DHH parents demonstrate faster WM growth and that this group difference is mediated by ASL receptive skills. The data suggest the important role of early access to perceivable natural language in promoting typical WM growth during the middle school years. We conclude that the acquisition of a natural visual–gestural language is sufficient to support the development of WM in DHH children. Further re-search is required to determine how the timing and quality of ASL exposure may play a role, or whether the effects are driven by acquisition-related corollaries, such as parent–child interactions and maternal stress.