The increased concentrate amounts in cow diets may initiate changes in both particle-associated (PaM) and epimural microbiota (EpM) with the potential for promoting the establishment of pathogens. Clay minerals have shown promising potentials in binding harmful microorganisms and metabolites due to their high adsorption capacity. This study evaluated the effects of a clay-mineral based product (CM) on PaM, EpM, fermentation parameters, and epithelial gene expression in cows fed a high-concentrate diet. Eight rumen-cannulated non-lactating Holstein cows received a concentrate mix supplemented with CM or not (CON) in a change-over design with an initial 100% roughage diet phase (RD, 1 week), followed by intermittent 65%-high-concentrate-diet phases (HC1, HC2; 1 and 2 week duration, respectively), interrupted by 1 week roughage only. Rumen samples for short-chain fatty acids, ammonia, and lactate quantification, as well as PaM, and epithelial biopsies for EpM examination and epithelial gene expression were collected via the cannula once during each feeding phase. Phylogenetic distance analysis of Illumina MiSeq sequencing of the 16S rRNA gene region V345 showed a clear clustering of RD microbiota compared to HC in PaM, showing the impact of the high-concentrate diet on the bacterial community. In the EpM this effect was less pronounced, due to higher variability in RD. In the PaM, a decrease (P < 0.01) of community diversity occurred with the onset of HC feeding, while in the EpM there was an increase in diversity (P < 0.05). In the PaM, CM increased the relative abundance of genus Butyrivibrio (P < 0.01), a commensal bacterium of the rumen, which was, with 6.4%, the second most abundant genus. There, the CM supplementation decreased the genera Lactobacillus, Fusobacterium, and Treponema (P = 0.05), which are potentially either lactate producing or opportunistic pathogens. In the EpM, CM decreased the relative abundance of Succiniclasticum genus (P < 0.01), a possible endotoxin producer, and increased bacteria that are associated with a normobiotic rumen, such as Campylobacter (P = 0.06). Barrier function genes were upregulated in HC2 and nutrient transport genes downregulated in HC1 (P < 0.05); however, there was little effect on pro-inflammatory genes at the epithelium. The CM showed a significant decreasing effect on the cellular metabolism genes HMGCS1 (P = 0.04). Our results suggest that CM supplementation can increase the relative abundance of commensal microbiota and decrease bacteria that could negatively impact the rumen milieu and health during high-concentrate feeding.