2019 IODA Workshop in Monterey, CA

The Interface for Observation Data Access (IODA) is a component of the Joint Effort for Data assimilation Integration (JEDI) software system.  From a user’s perspective, IODA handles the input of observation data and output of diagnostic data. Efforts to transform IODA from a prototype system into an operational system that can handle big data is under way, and the task of collecting requirements from the JCSDA partner organizations was initiated by the 2019 IODA Workshop. The workshop took place from February 11 through February 13, 2019 in Monterey, California, and was attended by representatives from the U.S. Navy, NOAA, NASA, ECMWF, UK Met Office, NCAR, UCAR and JCSDA. In total 41 people attended from 8 partner organizations.

The workshop opened on Monday (February 11th) with introductory remarks from two of the organizers Nancy Baker (NRL) and Yannick Trémolet (JCSDA). These talks were followed by speakers from the Navy, NOAA, NASA, ECMWF and the UK Met Office describing requirements for observation handling from their various perspectives. The opening presentations were very informative, and provided useful requirements information for IODA along with an effective foundation for the focused discussions that were to take place through the remainder of the workshop.

Three focused discussion sessions were conducted over the remainder of the workshop. Tuesday’s topics were “in-memory observation handling” and “File and I/O requirements for observation handling”. The third discussion session took place on Wednesday with the subject “pre- and post-processing diagnostics requirements”. Each discussion session was preceded by a set of talks given by representatives from the various partner organizations which served to prepare everyone for the associated discussion. Each set of talks were designed to allow the audience to hear from multiple perspectives including users of DA systems, scientist contributors to JEDI, and JEDI software developers. Material from the presentations was condensed, on the fly, into a list of prompts that were used to seed the subsequent discussion. All workshop participants attended all of the discussion sessions resulting in an effective means for communicating requirements and perspectives among the different partner organizations in the JEDI project.

Work is under way to compile the requirements gathered at the workshop and transform these into the necessary actions to satisfy them. The compilation step has yielded categories that represent different aspects of the IODA design. These categories include themes such as security (the need to handle classified and private data), reliability (an operational system needs to be running 24/7), ease of use, and portability (runs with a variety of hardware platforms, software compilers, and programming languages). We will be using agile software development practices, such as ZenHub tracking and code sprints, to implement solutions for the IODA requirements. The results of the compilation of requirements along with the tracking of the work will be posted where they can be viewed by the partner organizations.

Overall, the workshop was conducted in a highly positive and enthusiastic manner that solidly reflected the collaborative spirit of the JEDI project. People from different organizations showed up with the intention of shaping IODA into a world-class observation handling system. Many important details relevant to the IODA design and architecture surfaced during the time we spent together. We mention here only a few highlights. Although the Human Genome Project is famous for its dealing with big data, the amount of observation data that we are dealing with is roughly an order of magnitude larger. When you consider the model data that are also involved in DA flows, that difference becomes three times greater. The manner in which observation data is collected, organized and stored in files is incredibly diverse, and yet there appears to be real promise in creating a common method, or a very small set of methods, for organizing and storing those data within IODA. Attaining a common method of organizing and storing the vast amounts of observation data will go a long way in fostering the collaboration, exchange, and sharing of scientific results.

Dr. Yannick Trémolet presenting at the 2019 IODA Workshop in Monterey, CA from February 11 - 13th.

Dr. Yannick Trémolet presenting at the 2019 IODA Workshop in Monterey, CA from February 11 - 13th.

The persistence and resourcefulness of the workshop participants were put to the test Wednesday morning when a wind storm knocked out the power and internet at the meeting site and surrounding neighborhood. Within an hour, communication between on-site and off-site participants was restored using a cell-phone and bluetooth speaker, with local participants clustered around laptops, allowing the day’s presentations and discussion session to be completed on time. This is a further testament to the collaborative spirit and can-do attitude of the partner organizations.

Monterey proved to be a wonderful venue for the workshop. Many fabulous restaurants and sights to see were at our disposal, including the world famous Monterey Aquarium and the beautiful sights along the Pacific Coast Highway.

Thanks are due to the organizers of the workshop, Thomas Auligne (JCSDA), Nancy Baker (NRL) and Yannick Trémolet (JCSDA); moderators of the discussion session, Daryl Kleist (NOAA), Ron Gelaro (NASA) and Nancy Baker (NRL) and all of the participants who contributed presentations and commentary. Special thanks go to Katherine Shanahan (JCSDA) who handled all of the necessary logistics for the workshop (including a last-minute venue change required as a consequence of the government shutdown), as well as Ben Ruston (NRL) and Elizabeth Satterfield (NRL) who organized the social activities that built the commaradie of the group. The organizers and the JEDI team are grateful to all participants for making the 2019 IODA Workshop remarkably successful, productive, and enjoyable.

This summary was written by Stephen Herbener (JCSDA).

JCSDA Symposium at the 2019 Annual AMS Meeting in Phoenix, AZ

The JCSDA held its 7th Symposium at the 99th American Meteorological Society (AMS) Annual Meeting in Phoenix, AZ on Janurary 8, 2019. The symposium spanned the whole day on Tuesday, consisting of six topical sessions and one poster session. Presentations and posters were offered by staff and contractors of the JCSDA partner agencies, the broader academic community, and international representatives such as ECMWF and the UK Met Office. The symposium proves time and again to be an important outreach event for JCSDA, as there was strong attendance from the data assimilation community (including JCSDA partners) and lots of opportunities for stimulating discussion. Though attendance was generally affected by the shutdown, the event was still a great success.

The oral sessions had a total of 18 presentations dedicated to various aspects of the data assimilation study. The first session was chaired by Benjamin Johnson (JCSDA), and was devoted to the New and Improved Data Assimilation Tools and Methods. This was followed by the session of Land, Ocean, Cryosphere, Air Quality, and Coupled Earth System Data Assimilation, chaired by Benjamin Ruston (NRL). The third and fourth sessions, chaired by François Vandenberghe (JCSDA) and Thomas Auligné (JCSDA), were dedicated to the Next-Generation Satellites and Sensors Data Assimilation. Two more oral sessions were held in the afternoon and featured talks on all-sky radiance assimilation, chaired by Hui Shao (JCSDA), and the Joint Efforts for Data Assimilation Integration (JEDI), chaired by Guillaume Vernieres (JCSDA). The JCSDA Symposium also co-sponsored a joint session (invitation only), National and International Program Overviews for Environmental Satellites, on the morning of Tuesday, January 9.

Thank you to James Yoe (NWS) for organizing this JCSDA event and to all of our partners and dedicated team members who participated.

This text was adapted from an excerpt written by Dr. Hui Shao (JCSDA) for the winter edition of the JCSDA newsletter.

Dr. François Vandenberghe presenting his poster at AMS. Photo credit: Dawn Mullally | CPAESS | UCAR

Dr. François Vandenberghe presenting his poster at AMS. Photo credit: Dawn Mullally | CPAESS | UCAR