Authors: Ricardo Todling (1), D. Holdaway (1), B. Ménétrier (2), Y. Trémolet (3), A. Shlyaeva (3), F.R. Diniz (3), W. Gu (4), J. Jin (4), A. Sewnath (4), M. Sienkiewicz ( 4), Y. Zhu (1), and R. Gelaro (1)

1: NASA/GMAO, 2: Norwegian Meteorol. Institute, 3: UCAR, 4: SSAI/GMAO

Abstract

This brief report provides a summary of the status of experiments performed so far with GEOS-JEDI. This note is nearly a near word-by-word copy of a poster prepared by the first author at the 9th International Symposium on Data Assimilation, in Bologna, Italy, Oct. 16-20, 2023. The writeup assumes the reader to have reasonable familiarity with the transition of GSI-based systems to JEDI, and more specifically, to the GEOS atmospheric data assimilation system.

Prolegomenon

The first phase of transitioning the NASA GMAO GEOS atmospheric data assimilation capabilities to JEDI involves the replacement of the Grid-point Statistical Interpolation (GSI) with a corresponding JEDI analysis. This includes taking JEDI's Unified Observation Operator (UFO), its underlying dependencies, and the JEDI solver that enables a hybrid 4DEnVar strategy similar to what is used in the current GEOS-GSI system.

Variational analysis involves at least two main components associated with the observation and background cost function terms. The first is directly related to the UFO, which is being carefully validated in a joint collaboration between GMAO and NCEP to demonstrate consistency with corresponding observations usage in GSI by these groups. The second component is the background term, which in a hybrid system involves a climatologically-based term and an ensemble-based term. JEDI provides the means to implement both terms through its BUMP component. Use of BUMP, however, requires complete re-tune of both climatological and ensemble terms, which is a non-trivial exercise we prefer to avoid. As an alternative, the work here studies the results of interfacing the GSI-background error capability (GSIBEC) into JEDI through SABER. With this, the exact same background error covariance formulation used in GSI can be employed in JEDI without need for re-tuning.

This brief summary covers the work done so far to interface GSIBEC into JEDI and shows preliminary results when the background error covariances of the control (GEOS-GSI) and experiment (GEOS-JEDI) are identical in corresponding cycling experiments. Any cycling exercise is still preliminary and only so much can be expected from comparing GEOS-JEDI with GEOS-GSI. There are a number of features that need closer attention, including some aspects of the observing system and the background error covariance model itself. There are also features in JEDI that are ready for trying but have been chosen not to be exercised yet, e.g., VarBC is applied but not cycled, and inter-channel correlations for hyperspectral IR are not used. Other features are still pending implementation, one such example is the implementation of the Tangent Linear Normal Mode Constraint. Nonetheless, some of the results are quite encouraging as hopefully the discussion here illustrates.

GSI Background Error Covariance in JEDI

We have added capability in SABER to invoke the same background error covariance formulation as used in GSI -- indeed the covariance software of GSI has been made into an independent library which can be loaded to JEDI to allow for GSI-like background errors to be used in the variational procedures. Figure 1 provides a schematic illustration of this SABER feature.

A number of systematic stand-alone analysis tests and experiments have been performed comparing increments from GSI with those from JEDI in various configurations of the variational covariance and observing systems. From analyzing simple single observations of different types, to using conventional observations such as radiosondes (Fig. 2, to ozone sources, to radiance observations. Most tests show reasonable match of increments; some care to the moisture variable is still pending.

Figure 1. A Schematic showing added SABER capability to use GSI Background Error Covariance (BEC) formulation.

Figure 2. Comparing GSI and JEDI increments when both exercise the same background error covariance formulation in a 3DVar setting.

Cycling GEOS-JEDI

GMAO is adopting a phased transition to JEDI. The first phase preserves the present DA Workflow. This is illustrated in the schematic in Fig. 3. The diagram shows an enhanced version of the Workflow with components needed to run a variational analysis using JEDI along with GSI. The Workflow handles 3DVar, 3D/4D-EnVar, and Hybrid 3D/4D-EnVar, regardless of which analysis is ultimately used to update the model.

The first phase deals only with the deterministic part of the (operational) hybrid system, leaving the ensemble DA Workflow unchanged. A further revision of this first phase is being considered in which the present Workflow would be adapted to accommodate a JEDI-LETKF option.

It has been about two years that some cycling with JEDI using the enhanced Workflow has been first tried. The initial configuration used what was available then, namely 3DEnVar using the BUMP background error covariance capability and radiosonde observations only. Only a very minor effort was put into tuning some of the scales in BUMP. Since then, a few other configurations have been cycled. The various attempts, including some very recent attempts, are summarized in Table 1. The introduction of GSIBEC avoids (or postpones) need to properly tune a completely new background error covariance based on BUMP; use of GSIBEC allows for effort to concentrate on aspects of integration other than on work that's already been done. This capability allows for 3DVar to be readily cycled and tested, hybrid 3DVar is also readily available and, aside from some details being sorted out in the JEDI cost function, hybrid 4DEnVar is also available and near ready for testing in cycled mode.

Cycled attempts thus far:

Circa mid-2021: BUMP-BEC 3DEnVar, radiosondes only \& ozone.
Circa Fall-2022: GSI-BEC 3DVar, radiosondes \& ozone.
Fall 2023: GSI-BEC 3DVar; near full observing system (see Table \ref{tab:Exps}).
Fall 2023: GSI-BEC Hyb-4DEnVar; near full observing system (not quite going yet)

figure 3. Enhanced Workflow of Deterministic component of the Hybrid 4DEnVar of GEOS Atmospheric DAS. The option to have either the GSI or the JEDI analysis feedback to the GEOS model as IAU tendencies (red) allows for on-the-fly comparison of the two variational systems.

Table 1: Summary of cycling experiments performed with GEOS-JEDI thus far.

Figure 4: Percentage timing taken in the main components of four consecutive cycles in the enhanced GEOS ADAS Workflow. Left: 3DVar; Right: Hybrid 4DEnVar. Results are from the most recent experiments performed in the Fall of 2023.

Preliminary Results

Figure 4 displays percentage timings derived from a day of cycling 3DVar and Hybrid 4DEnVar. These experiments were preformed during a time when our computers had been going through an upgrade that affected disk access, thus affecting timings of components that rely on I/O (GSI more so than JEDI). Still the preliminary assessment serves as illustration for how the cost of running JEDI is comparable to that of running GSI. We should however note that the GSI integrations in these exercises use two middle loops and invoke one extra call to the observation operators, unlike the JEDI runs that involve a single middle loop and two calls to the observer.
Figure 5 provides illustration of the behavior of the initial and final cost functions for three cycling configurations of GEOS-JEDI and the corresponding control experiments. The initial 3DEnVar experiment (top-left) illustrate the poor attempt of mildly tuning BUMP-based ensemble covariances (blame it on first author only). The second and third experiments, using 3DVar with radiosondes and ozone only (top-right), and a near full observing system (bottom), show way more reasonable results, with Jo(b) and Jo(a) of JEDI being relatively close to the results of their corresponding control experiments. Although in the most recent 3DVar experiment the analysis fits the observations as in control, the fits to the background are not as good -- further experimentation is necessary. The two 3DVar experiments benefit from the well established GSI background errors, with no need for re-tuning; the same error covariances are used in the control and JEDI experiment in each case.

Figure 6 examines the most recent 3DVar experiment, when a near complete observing system is used. It provides observation-space diagnostics comparisons between GEOS-JEDI and the control GEOS-GSI. The observation counts (top-right) compare reasonably well, with slightly less hyperspectral IR data being taken in the JEDI case. When it comes to the contribution of individual instruments to reduce the cost function (top-right), aircrafts shows as a major player in both systems, however, radiosondes and satellite winds play a much larger role in GEOS-GSI than in GEOS-JEDI. Reversely, most of the MW instruments and hyrperspectral IR are seen to contribute more to GEOS-JEDI. Results for the latter must be interpreted with caution since GSI treats these data as correlated in channel-space, whereas JEDI (here) is not yet using this feature. Overall, the impact of observations in reducing errors in the analysis is much larger in the JEDI system (bottom), a clear consequence of the results displayed in the last panel of Fig. 5.

Finally, Fig. 7 shows day-one self-verified forecast RMS errors (expressed as global linearized total moist energy). The poorly tuned 3DEnVar experiment (top-right) shows "low errors"; in a self-verified measure, low errors are an indication of the model not listening to the analyses (also seen in Figs. 5 and 6). The situation is much improved in the 3DVar experiment carried out ca. Fall 2022 (top-right) which shows rather reasonable agreement in the day-one errors and their corresponding nonlinear impact (negative curves). In the more recent Fall 2023 experiment with a nearly complete observing system forecast errors from GEOS-JEDI seem to lag about 6-hours from those of GEOS-GSI. This is not bad for a first GEOS-JEDI experiment using nearly 4 million observations in each 6-hour cycle.

Figure 5. Time series of observation cost function divided by number of observations in the first three experiment described in Table 1. Left: Fall 2021 experiments. Right: Fall 2022 experiments. Bottom: Fall 2023 experiments.

Figure 6. Observation space evaluation for Fall 2023 3DVar experiments. Left: average observation count per cycle. Right: Cost reduction due to assimilation of various observation types. Bottom: time series of global cost reduction in control GEOS-GSI experiment (solid black); GEOS-JEDI (dashed black); and GSI running along JEDI within GEOS-JEDI workflow (green).

Figure 7. Day-one RMS forecast errors for the three main experiments in Table 1. Left: Fall 2021, BUMP-based 3DEnVar. Right: Fall 2022, GSIBEC-based 3DVar. Bottom: Fall 2023, GSIBEC 3DVar.

Closing Remarks

The implementation of the GSI background error capability in JEDI is now available for testing. The implementation supports both the climatological and hybrid formulations just as the GSI-based system does. Preliminary tests cycling with a 3DVar (climatological background error covariance) produce quite reasonable results, though some puzzles in the treatment of certain observing types must be resolved. We have experimented cycling the hybrid 4DEnVar configuration, but this has been done simply as an engineering test to exercise the Workflow; there are still pending issues with the 4D-Ens-Var cost function in JEDI when GSIBEC tis used -- this is a source of intense work being done at the time of this writing.

Additionally, there are still a number of details that need to be tackled in GSIBEC: (i) the moisture control variable is not being handled as in GSI; (ii) a (weak) reproducibility issue across different number of PEs has been found; and (iii) the need for some form of dynamical balance, e.g., the GSI Tangent Linear Normal Mode Constraint needs to be brought into JEDI.

Acknowledgement

The work performed here would not have been possible without the tremendous achievements of D. F. Parrish, J. C. Derber, and R. J. Purser, from NOAA/NCEP, in implementing the background error covariance machinery in GSI. The first author has also benefitted from discussions with R. Treadon, C. Thomas, and D. T. Kleist, also from NOAA/NCEP. Thanks are due to the NASA Center for Climate Simulation.