We study the performance of sparse regression methods and propose new techniques to distill the governing equations of nonlinear dynamical systems from data. We start from the recently proposed generic methodology of learning interpretable equation forms from data, followed by performance of least absolute shrinkage and selection operator (LASSO) for this purpose. We first develop an algorithm that uses the dual of LASSO optimization for higher accuracy and stability. We then derive a second algorithm that learns the candidate function library in a dynamic data driven applications systems (DDDAS) manner to distill the governing equations of the dynamical system. This is achieved via sequentially thresholded ridge regression (STRidge) over a orthogonal polynomial space. The performance of the methods is illustrated using the Lorenz 63 system and a marine ecosystem model.