Active matter, responsive (“smart”) materials and materials under time-dependent load are systems out of thermal equilibrium. To construct coarse-grained models for such systems, one needs to integrate out a distribution of microstates that evolves in time. This is a challenging task. As a preparation to the topic, we recall equilibrium coarse-graining methods, both theoretical and numerical, such as projection operator formalisms, united atom simulations, numerical reconstruction of memory and Markov State Modelling. Then we review recent developments in theoretical approaches to the non-equilibrium coarse-graining problem, in particular, time-dependent projection operator formalisms, dynamic density functional theory and power functional theory, as well as numerical schemes to construct explicitly time-dependent memory kernels.