The development of sustained, long-term endothermy was one of the major transitions in the evolution of vertebrates. Thermogenesis in endotherms does not only occur via shivering or activity, but also via non-shivering thermogenesis (NST). Mammalian NST is mediated by the uncoupling protein 1 in the brown adipose tissue (BAT) and possibly involves an additional mechanism of NST in skeletal muscle. This alternative mechanism is based on Ca2+-slippage by a sarcoplasmatic reticulum Ca2+-ATPase (SERCA) and is controlled by the protein sarcolipin. The existence of muscle based NST has been discussed for a long time and is likely present in all mammals. However, its importance for thermoregulation was demonstrated only recently in mice. Interestingly, birds, which have evolved from a different reptilian lineage than mammals and lack UCP1-mediated NST, also exhibit muscle based NST under the involvement of SERCA, though likely without the participation of sarcolipin. In this review we summarize the current knowledge on muscle NST and discuss the efficiency of muscle NST and BAT in the context of the hypothesis that muscle NST could have been the earliest mechanism of heat generation during cold exposure in vertebrates that ultimately enabled the evolution of endothermy. We suggest that the evolution of BAT in addition to muscle NST was related to heterothermy being predominant among early endothermic mammals. Furthermore, we argue that, in contrast to small mammals, muscle NST is sufficient to maintain high body temperature in birds, which have enhanced capacities to fuel muscle NST by high rates of fatty acid import.