Understanding how the mammalian diversity responds to anthropogenic disturbances on local and landscape scales is an urgent task. The Atlantic Forest biome, with only 12% of its original area remaining, still harbors great diversity of small mammals (Rodentia and Didelphimorphia), a key group that responds quickly to disturbances. Here, using the largest dataset of Atlantic Forest small mammals, we evaluate how forest amount and bioclimatic variables affect the non-volant small mammal diversity. For this purpose, we use 214 small mammal assemblages across the Atlantic Forest domain. Our results show that forest amount, with a positive relation, was the most important predictor explaining the diversity of small mammals in Atlantic Forest remnants. We also found that the bioclimatic variables (temperature and precipitation) can positively and/or negatively affect small mammal biodiversity, depending on the region analyzed. This is the first study that has assessed diversity across the entire Atlantic Forest biome, showing the importance of large-scale assessment and of forest amount and bioclimatic variables in shaping the diversity of small mammals regardless of the biogeographic context.