Machine learning techniques are applied to Lagrangian trajectory reconstructions, which are important in oceanography for providing guidance to search and rescue efforts, forecasting the spread of harmful algal blooms, and tracking pollutants and marine debris. This study evaluates the ability of two types of neural networks for learning ocean trajectories from nearly 250 surface drifters released during the Grand Lagrangian Deployment in the Gulf of Mexico from Jul-Oct 2012. First, simple fully connected neural networks were trained to predict an individual drifter’s trajectory over 24 h and 5 d time windows using only that drifter’s previous velocity time series. These networks, despite having successfully learned modeled trajectories in a previous study, failed to outperform common autoregressive models in any of the tests conducted. This was true even when drifters were pre-sorted into geospatial groups based on past trajectories and different networks were trained on each group to reduce the variability that each network had to learn. In contrast, a more sophisticated social spatio-temporal graph convolutional neural network (STN), originally developed for learning pedestrian trajectories, demonstrated greater potential due to two important features: learning spatial and temporal patterns simultaneously, and sharing information between similarly-behaving drifters to facilitate the prediction of any particular drifter. Position prediction errors averaged around 60 km at day 5, roughly 20 km lower than autoregression, and even better for certain subsets of drifters. The passage of Tropical Cyclone Isaac over the drifter array as a tropical storm and Category 1 hurricane provided a unique opportunity to also explore whether these models would benefit from adding wind as a predictor when making short 24 h predictions. The STNs were found to not benefit from wind on average, though certain subsets of drifters exhibited slightly lower reconstruction errors at hour 24 with the addition of wind.

Improving the predictive capability and computational cost of dynamical models is often at the heart of augmenting computational physics with machine learning (ML). However, most learning results are limited in interpretability and generalization over different computational grid resolutions, initial and boundary conditions, domain geometries, and physical or problem-specific parameters. In the present study, we simultaneously address all these challenges by developing the novel and versatile methodology of unified neural partial delay differential equations. We augment existing/low-fidelity dynamical models directly in their partial differential equation (PDE) forms with both Markovian and non-Markovian neural network (NN) closure parameterizations. The melding of the existing models with NNs in the continuous spatiotemporal space followed by numerical discretization automatically allows for the desired generalizability. The Markovian term is designed to enable extraction of its analytical form and thus provides interpretability. The non-Markovian terms allow accounting for inherently missing time delays needed to represent the real world. Our flexible modeling framework provides full autonomy for the design of the unknown closure terms such as using any linear-, shallow-, or deep-NN architectures, selecting the span of the input function libraries, and using either or both Markovian and non-Markovian closure terms, all in accord with prior knowledge. We obtain adjoint PDEs in the continuous form, thus enabling direct implementation across differentiable and non-differentiable computational physics codes, different ML frameworks, and treatment of nonuniformly-spaced spatiotemporal training data. We demonstrate the new generalized neural closure models (gnCMs) framework using four sets of experiments based on advecting nonlinear waves, shocks, and ocean acidification models. Our learned gnCMs discover missing physics, find leading numerical error terms, discriminate among candidate functional forms in an interpretable fashion, achieve generalization, and compensate for the lack of complexity in simpler models. Finally, we analyze the computational advantages of our new framework.

Predictive dynamical models for marine ecosystems are used for a variety of needs. Due to the sparse measurements and limited understanding of the myriad of ocean processes, there is however significant uncertainty. There is model uncertainty in the parameter values, functional forms with diverse parameterizations, and level of complexity needed, and thus in the state variable fields. We develop a Bayesian model learning methodology that allows interpolation in the space of candidate dynamical models and discovery of new models from noisy, sparse, and indirect observations, all while estimating state variable fields and parameter values, as well as the joint probability distributions of all learned quantities. We address the challenges of high-dimensional and multidisciplinary dynamics governed by partial differential equations (PDEs) by using state augmentation and the computationally efficient Gaussian Mixture Model – Dynamically Orthogonal filter. Our innovations include stochastic formulation parameters and stochastic complexity parameters to unify candidate models into a single general model as well as stochastic expansion parameters within piecewise function approximations to generate dense candidate model spaces. These innovations allow handling many compatible and embedded candidate models, possibly none of which are accurate, and learning elusive unknown functional forms that augment these models. Our new Bayesian methodology is generalizable and interpretable. It seamlessly and rigorously discriminates among existing models, but also extrapolates out of the space of models to discover new ones. We perform a series of twin experiments based on flows past a ridge coupled with three-to-five component ecosystem models, including flows with chaotic advection. We quantify the learning skill, and evaluate convergence and the sensitivity to hyper-parameters. Our PDE framework successfully discriminates among functional forms and model complexities, and learns in the absence of prior knowledge by searching in dense function spaces. The probabilities of known, uncertain, and unknown model formulations, and of biogeochemical-physical fields and parameters, are updated jointly using Bayes’ law. Non-Gaussian statistics, ambiguity, and biases are captured. The parameter values and the model formulations that best explain the noisy, sparse, and indirect data are identified. When observations are sufficiently informative, model complexity and model functions are discovered.

The state of the ocean evolves and its dynamics involves transitions occurring at multiple scales. For efficient and rapid interdisciplinary forecasting, ocean observing and prediction systems must have the same behavior and adapt to the ever-changing dynamics. This chapter sets the basis of a distributed system for real-time interdisciplinary ocean field and uncertainty forecasting with adaptive modeling and adaptive sampling. The scientific goal is to couple physical and biological oceanography with ocean acoustic measurements. The technical goal is to build a dynamic modeling and instrumentation system based on advanced infrastructures, distributed/grid computing, and efficient information retrieval and visualization interfaces, from which all these are incorporated into the Poseidon system. Importantly, the Poseidon system combines a suite of modern legacy physical models, acoustic models, and ocean current monitoring data assimilation schemes with innovative modeling and adaptive sampling methods. The legacy systems are encapsulated at the binary level using software component methodologies. Measurement models are utilized to link the observed data to the dynamical model variables and structures. With adaptive sampling, the data acquisition is dynamic and aims to minimize the predicted uncertainties, maximize the optimized sampling of key dynamics, and maintain overall coverage. With adaptive modeling, model improvements dynamically select the best model structures and parameters among different physical or biogeochemical parameterizations. The dynamic coupling of models and measurements discussed here, and embodied in the Poseidon system, represents a Dynamic Data-Driven Applications Systems (DDDAS). Technical and scientific progress is highlighted based on examples in Massachusetts Bay, Monterey Bay, and the California Current System.

Finite element discretizations of problems in computational physics often rely on adaptive mesh refinement (AMR) to preferentially resolve regions containing important features during simulation. However, these spatial refinement strategies are often heuristic and rely on domain-specific knowledge or trial-and-error. We treat the process of adaptive mesh refinement as a local, sequential decision-making problem under incomplete information, formulating AMR as a partially observable Markov decision process. Using a deep reinforcement learning approach, we train policy networks for AMR strategy directly from numerical simulation. The training process does not require an exact solution or a high-fidelity ground truth to the partial differential equation at hand, nor does it require a pre-computed training dataset. The local nature of our reinforcement learning formulation allows the policy network to be trained inexpensively on much smaller problems than those on which they are deployed. The methodology is not specific to any particular partial differential equation, problem dimension, or numerical discretization, and can flexibly incorporate diverse problem physics. To that end, we apply the approach to a diverse set of partial differential equations, using a variety of high-order discontinuous Galerkin and hybridizable discontinuous Galerkin finite element discretizations. We show that the resultant deep reinforcement learning policies are competitive with common AMR heuristics, generalize well across problem classes, and strike a favorable balance between accuracy and cost such that they often lead to a higher accuracy per problem degree of freedom.

A new methodology for rigorous Bayesian learning of high-dimensional stochastic dynamical models is developed. The methodology performs parallelized computation of marginal likelihoods for multiple candidate models, integrating over all state variable and parameter values, and enabling a principled Bayesian update of model distributions. This is accomplished by leveraging the dynamically orthogonal (DO) evolution equations for uncertainty prediction in a dynamic stochastic subspace and the Gaussian Mixture Model-DO filter for inference of nonlinear state variables and parameters, using reduced-dimension state augmentation to accommodate models featuring uncertain parameters. Overall, the joint Bayesian inference of the state, model equations, geometry, boundary conditions, and initial conditions is performed. Results are exemplified using two high-dimensional, nonlinear simulated fluid and ocean systems. For the first, limited measurements of fluid flow downstream of an obstacle are used to perform joint inference of the obstacle’s shape, the Reynolds number, and the O(10⁵) fluid velocity state variables. For the second, limited measurements of the concentration of a microorganism advected by an uncertain flow are used to perform joint inference of the microorganism’s reaction equation and the O(10⁵) microorganism concentration and ocean velocity state variables. When the observations are sufficiently informative about the learning objectives, we find that our posterior model probabilities correctly identify either the true model or the most plausible models, even in cases where a human would be challenged to do the same.

Complex dynamical systems are used for predictions in many domains. Because of computational costs, models are truncated, coarsened, or aggregated. As the neglected and unresolved terms become important, the utility of model predictions diminishes. We develop a novel, versatile, and rigorous methodology to learn non-Markovian closure parameterizations for known-physics/low-fidelity models using data from high-fidelity simulations. The new neural closure models augment low-fidelity models with neural delay differential equations (nDDEs), motivated by the Mori-Zwanzig formulation and the inherent delays in complex dynamical systems. We demonstrate that neural closures efficiently account for truncated modes in reduced-order-models, capture the effects of subgrid-scale processes in coarse models, and augment the simplification of complex biological and physical-biogeochemical models. We find that using non-Markovian over Markovian closures improves long-term prediction accuracy and requires smaller networks. We derive adjoint equations and network architectures needed to efficiently implement the new discrete and distributed nDDEs, with any time-integration scheme and allowing nonuniformly-spaced temporal training data. The performance of discrete over distributed delays in closure models is explained using information theory, and we find an optimal amount of past information for a specified architecture. Finally, we analyze computational complexity and explain the limited additional cost due to neural closure models.

Many aspects of geosciences pose novel problems for intelligent systems research. Geoscience data is challenging because it tends to be uncertain, intermittent, sparse, multiresolution, and multiscale. Geosciences processes and objects often have amorphous spatiotemporal boundaries. The lack of ground truth makes model evaluation, testing, and comparison difficult. Overcoming these challenges requires breakthroughs that would significantly transform intelligent systems, while greatly benefitting the geosciences in turn. Although there have been significant and beneficial interactions between the intelligent systems and geosciences communities, the potential for synergistic research in intelligent systems for geosciences is largely untapped. A recently launched Research Coordination Network on Intelligent Systems for Geosciences followed a workshop at the National Science Foundation on this topic. This expanding network builds on the momentum of the NSF EarthCube initiative for geosciences, and is driven by practical problems in Earth, ocean, atmospheric, polar, and geospace sciences. Based on discussions and activities within this network, this article presents a research agenda for intelligent systems inspired by geosciences challenges.

For efficient progress, model properties and measurement needs can adapt to oceanic events and interactions as they occur. The combination of models and data via data assimilation can also be adaptive. These adaptive concepts are discussed and exemplified within the context of comprehensive real-time ocean observing and prediction systems. Novel adaptive modeling approaches based on simplified maximum likelihood principles are developed and applied to physical and physical-biogeochemical dynamics. In the regional examples shown, they allow the joint calibration of parameter values and model structures. Adaptable components of the Error Subspace Statistical Estimation (ESSE) system are reviewed and illustrated. Results indicate that error estimates, ensemble sizes, error subspace ranks, covariance tapering parameters and stochastic error models can be calibrated by such quantitative adaptation. New adaptive sampling approaches and schemes are outlined. Illustrations suggest that these adaptive schemes can be used in real time with the potential for most efficient sampling.

Physical and biogeochemical ocean dynamics can be intermittent and highly variable, and involve interactions on multiple scales. In general, the oceanic fields, processes and interactions that matter thus vary in time and space. For efficient forecasting, the structures and parameters of models must evolve and respond dynamically to new data injected into the executing prediction system. The conceptual basis of this adaptive modeling and corresponding computational scheme is the subject of this presentation. Specifically, we discuss the process of adaptive modeling for coupled physical and biogeochemical ocean models. The adaptivity is introduced within an interdisciplinary prediction system. Model-data misfits and data assimilation schemes are used to provide feedback from measurements to applications and modify the runtime behavior of the prediction system. Illustrative examples in Massachusetts Bay and Monterey Bay are presented to highlight ongoing progress.

We present the high level architecture of a real-time interdisciplinary ocean forecasting system that employs adaptive elements in both modeling and sampling. We also discuss an important issue that arises in creating an integrated, web-accessible framework for such a system out of existing stand-alone components: transparent support for handling legacy binaries. Such binaries, that are most common in scientific applications, expect a standard input stream, maybe some command line options, a set of input files and generate a set of output files as well as standard output and error streams. Legacy applications of this form are encapsulated using XML. We present a method that uses XML documents to describe the parameters for executing a binary.