A new segment of feed industry based on bakery by-products (BBP) has emerged. Yet, information is lacking regarding the effects of inclusion of BBP in ruminant diets on ruminal fermentation and microbiota. Therefore, the aim of this study was to evaluate the effect of the gradual replacement of grains by BBP on ruminal fermentation, nutrient degradation, and microbial community composition using the rumen-simulation technique. All diets consisted of hay and concentrate mixture with a ratio of 42:58 (dry matter basis), but differed in the concentrate composition with either 45% cereal grains or BBP, whereby 15, 30, or 45% of BBP were used in place of cereal grains. The inclusion of increasing levels of BBP in the diet linearly enhanced ruminal degradation of starch from 84% (control) to 96% (45% BBP), while decreasing degradation of crude protein and fiber. The formation of methane was lowered in the 45% BBP diet compared with all other diets. Whereas the ammonia concentration was similar in the control and 15% BBP, a significant decrease was found in 30% BBP (-23%) and 45% BBP (-33%). Also, BBP feeding shifted fermentation profile toward propionate at the expense of acetate. Moreover, isobutyrate linearly decreased with increasing BBP inclusion. Bacterial 16S rRNA Illumina MiSeq (Microsynth AG, Balach, Switzerland) sequencing revealed a decreased microbial diversity for the 45% BBP diet. Furthermore, the replacement of cereal grains with BBP went along with an increased abundance of the genera Prevotella, Roseburia, and Megasphaera, while decreasing Butyrivibrio and several OTU belonging to Ruminococcaceae. In conclusion, the inclusion of BBP at up to 30% of the dry matter had no detrimental effects on pH, fiber degradability, and microbial diversity, and enhanced propionate production. However, a higher replacement level (45%) impaired ruminal fermentation traits and fiber degradation and is not recommended.