We propose a new delay neural operator applicable to both large-scale advective flow field prediction and corresponding subgrid-scale closure. Our operator optimizes computation by focusing solely on input fields and correcting for any unseen field influences due to model truncation, coarsening, or aggregation of the full-order model. Compressing input fields to a latent space efficiently enables arbitrary output resolution without storing a complete, discretized system state in memory. Additionally, unseen fields are never computed, unlike classic numerical and many deep-learning Markov process models.  

We construct the delay neural operator by extending neural delay differential equations to 2D and higher dimensions. Inspired by the Mori-Zwanzig formulation, neural delay differential equations and neural closure models perform temporal convolution or kernel integration to accumulate hidden processes (a distributed delay), approximating unseen field effects without storing additional variables. Linear, discretized versions of this distributed delay (discrete delays) have been used to develop effective reduced-order models. We extend these distributed and discrete delays to neural operators. In particular, we present discrete delayed RNNs as a superset of Picard iteration-performing neural operators. We explore multiscale and scale-invariant architectures, enabling arbitrary input and output resolution flow fields. We also investigate the origin and extension of physical representations—concepts of waves, eddies, and vortices—through network layers. Towards an efficient backpropagation with constant memory (which is independent of the number of layers), we simplify adjoint computation and explore integral-free alternatives, including Suzuki-Trotter operator-splitting and simple discretization. Tests are performed against simulated 2D viscous Burger’s equation with Smagorinsky closure, 2D homogeneous isotropic, quasi-geostrophic beta-plane turbulence (2D-HIT QG), and data-assimilated ocean surface velocity simulations.

Recent advances in deep learning have led to neural architectures effective for modeling fluid dynamics, with an emphasis on weather prediction and atmospheric modeling. In this work, we develop physics-inspired deep learning models for fluid and oceanic processes, integrating principles from physics and numerical modeling directly within the deep neural architecture to learn multi-scale features and train effectively from limited data — essential characteristics of ocean dynamics and data. Inspired by attention-based architectures, we adapt attention mechanisms based on physics and computational stencil concepts from numerical PDE solvers. Given that fluid dynamics depends on both spatial locality and temporal history, we modify attention mechanisms to capture the rich spatiotemporal dynamics of fluid flows efficiently. Our new physics-inspired attention mechanisms can handle complex bathymetry and coastal land, support learning multiscale features and multi-dynamics, and model the effects of external ocean forcing. We also investigate different choices of numerical integration schemes, error norms, and loss functions to ensure stable predictions over long temporal roll-outs. 

To evaluate and validate the utility of these models, we first showcase applications to predict idealized fluid flows such as eddy shedding past obstacles, vorticity dynamics, and bottom gravity currents for varied Reynolds and Grashof numbers. We then train our deep learning architectures for realistic high-resolution data-assimilative ocean simulations and real-time sea experiments, e.g., surface velocity fields from the Loop Current System (LCS) in the Gulf of Mexico. We illustrate both ensemble and deterministic deep learning forecasts under various scenarios and in recursive and non-recursive applications. We quantify the performance of the deep learning training and forecasts using comprehensive skill metrics.

Mesoscale and submesoscale features play a critical role in transporting heat and biogeochemical tracers from the surface ocean to depths below the mixed layer, by driving vertical motions across density gradients. In the winter of 2022, strong mesoscale and submesoscale features were observed in the Western Mediterranean Sea during the ONR CALYPSO oceanographic campaign. This multidisciplinary experiment combined multiplatform in-situ observations with high-resolution numerical simulations to observe and predict small-scale ocean variability. In particular, a mesoscale density front associated with a vortex dipole was observed using CALYPSO observations and satellite imagery. A 650m resolution model simulation is used here to understand the evolution of the front and the energy transfer to submesoscale cyclonic eddies. The simulation properly reproduces the intense, narrow, and elongated frontal convergence structure forming a dense cyclonic ridge linked to the vortex dipole. The evolution of the front is characterized by: i) an intensification through frontogenesis, and ii) a decay due to favorable conditions for overturning instabilities during a down-front wind event. These processes enhance vertical motion via an across-front ageostrophic secondary circulation and contribute to the restratifying effect. After a few days, the front decayed and cascaded into smaller-scale structures, forming submesoscale cyclonic eddies (SCEs) at the edges of the front. The formation of SCEs is associated with the frontal decay, as well as centrifugal and gravitational instabilities, which transfer energy from the mesoscale front to the SCEs. The SCE structure reveals a 3D helical-spiral recirculation pattern that transports parcels vertically. Observations of oxygen and chlorophyll confirm the enhancement of the vertical transport of tracers from the surface to the ocean interior. Submesoscale eddy-induced frontogenesis mechanism and instability processes drove subduction along outcropping isopycnals at the periphery of the SCE.