Depolarization of light can be measured by polarization-sensitive optical coherence tomography (PS-OCT) and has been used to improve tissue discrimination as well as segmentation of pigmented structures. Most approaches to depolarization assessment for PS-OCT - such as the degree of polarization uniformity (DOPU) - rely on measuring the uniformity of polarization states using spatial evaluation kernels. In this article, we present a different approach which exploits the spectral dimension. We introduce the spectral DOPU for the pixelwise analysis of polarization state variations between sub-bands of the broadband light source spectrum. Alongside a comparison with conventional spatial and temporal DOPU algorithms, we demonstrate imaging in the healthy human retina, and apply the technique for contrasting hard exudates in diabetic retinopathy and investigating the pigment epithelium of the rat iris.