Though computing power continues to grow quickly, our appetite to solve larger and larger problems grows just as fast. As a consequence, reduced-order modeling has become an essential technique in the computational scientist’s toolbox. By reducing the dimensionality of a system, we are able to obtain approximate solutions to otherwise intractable problems. And because the methodology we develop is sufficiently general, we may agnostically apply it to a plethora of problems, whether the high dimensionality arises due to the sheer size of the computational domain, the fine resolution we require, or stochasticity of the dynamics. In this thesis, we develop time integration schemes, called retractions, to efficiently evolve the dynamics of a system’s low-rank approximation. Through the study of differential geometry, we are able to analyze the error incurred at each time step. A novel, explicit, computationally inexpensive set of algorithms, which we call perturbative retractions, are proposed that converge to an ideal retraction that projects exactly to the manifold of fixed-rank matrices. Furthermore, each perturbative retraction itself exhibits high-order convergence to the best low-rank approximation of the full-rank solution. We show that these high-order retractions significantly reduce the numerical error incurred over time when compared to a naive Euler forward retraction. Through test cases, we demonstrate their efficacy in the cases of matrix addition, real-time data compression, and deterministic and stochastic differential equations.