3rd JEDI Academy

The 3rd JEDI Academy was held this past week in Boulder, Colorado from June 10–13, 2019. The JEDI Academy is a tutorial held to act as a training session for those working on the Joint Effort for Data assimilation Integration (JEDI) utilized by JCSDA and affiliated partners focused on producing more accurate forecasting predictions. The JEDI coding framework utilizes standard interfaces between models and observations to create a unified data assimilation system. JEDI incorporates coupled model systems, unified observation (forward) operators (UFO), research to operation strategies (R2O/O2R), and a collaborative, innovative effort amongst team members and the scientific community.

 The academy offers attendees both lectures and practicums (hands-on practice) to familiarize employees and affiliates with the JEDI system, discuss updates, and work collaboratively on improving the framework. Several members of the JEDI team gave detailed lectures in the mornings, with practicum work following in the afternoons. About 40 participants attended from affiliated institutions; turnout and engagement were at an all-time high. The consensus from the crowd was that the academy has gotten progressively more refined from when it was first held in June of 2018.

 Dr. Yannick Trémolet, JEDI Project Lead, kicked off the academy with a talk introducing the importance of the Object-Oriented Prediction System (OOPS) and JEDI design for improved forecasting.

Some of the other lecturers and topics were as follows:

  • Steve Herbener reviewed features of Interface for Observation Data Access (IODA) which helps to isolate code from data storage.

  • Dr. Mark Miesch discussed his work on portability, containers (virtual environments to help run OOPS more efficiently on remote machines), and collaboration tools.

  • Dr. Anna Shlyaeva talked about the various aspects of integrating observations in OOPS to make a model-agnostic interface.

  • Dr. Dan Holdaway used the FV3 (GFS – NOAA, GEOS – NASA) forecasting model as an example to discuss how model interfacing works with OOPS.

  • Dr. Maryam Abdi-Oskouei talked about various modes of testing within JEDI.

  • Dr. Travis Sluka highlighted marine applications of OOPS models.

Dr. Anna Shlyaeva presenting her work.

Dr. Anna Shlyaeva presenting her work.

 In the afternoon, participants got some hands-on practice working with JEDI code by running the framework to assess outputs with small modifications to variables. Most coding modules tied directly back to concepts covered in the morning lectures for that day.

Participants working together on practicums in the afternoon.

Participants working together on practicums in the afternoon.

Ultimately, the goal for JCSDA and the JEDI project is to create usable, open-source JEDI code that can be utilized by academics and engineers alike. We are hopeful to refine the JEDI Academy to a point where these training sessions can be emulated within the community and at universities to provide students with an outlet to develop their practical coding skills. It’s safe to say that JCSDA is leading the charge for innovative forecasting, and the JEDI framework is a key component. 

‘Padawans’ graduating from the 3rd JEDI Academy, June 10-13, 2019 in Boulder, Colorado

‘Padawans’ graduating from the 3rd JEDI Academy, June 10-13, 2019 in Boulder, Colorado

17th JCSDA Technical Review Meeting and Science Workshop

The 17th Annual JCSDA Technical Review Meeting and Science Workshop was held from May 29 - 31, 2019 in Washington, D.C. at NASA Headquarters.

The 17th JCSDA Technical Review Meeting and Science Workshop is designed to be a showcase of what JCSDA employees and affiliates have worked on scientifically throughout the year. Participants present posters or give oral presentations on project findings.

This year, the workshop was a great success with over 100 participants from all different corners of JCSDA. Thank you to everyone who contributed, and we look forward to another successful workshop next year.

You can find copies of the presentation slides (under Agenda & Presentations) and a poster gallery on the meeting website.

Photo credit: Kat Shanahan | JCSDA

Photo credit: Kat Shanahan | JCSDA

NCAR/UCP Science & Discovery Day

On Friday, May 8, NCAR/UCP hosted a day for science and discovery with numerous guest speakers including experts on research, engineering, and technical advancements within the scientific community.

Each year NCAR, UCAR, and UCP staff are invited to attend this full day event to network and share recent scientific developments. This year, JCSDA director Tom Auligné gave a talk entitled, “A Fresh Look at O2R/R2O Transition.”

Photo credit: Dawn Mullally | CPAESS/UCAR Communications

Photo credit: Dawn Mullally | CPAESS/UCAR Communications

JCSDA Marine IODA & UFO Code Sprint in Boulder, CO.

From March 25, 2019 to April 5, 2019 members of the JCSDA SOCA team and affiliated partners are working on developing interfaces to databases (IODA), refining existing UFOs, and implementing new UFOs.

Code sprints are a large part of JCSDA culture. Project members come together to work collaboratively from a few days to a few weeks in order to improve existing infrastructure and come up with new solutions.

Photo courtesy of Dr. Thomas Auligné, JCSDA director. Pictured: Stylianos Flampouris, Shastri Paturi, Stephen Herbener, Hamideh Ebrahimi, Guillaume Vernieres, Rahul Mahajan, and Travis Sluka.

Photo courtesy of Dr. Thomas Auligné, JCSDA director. Pictured: Stylianos Flampouris, Shastri Paturi, Stephen Herbener, Hamideh Ebrahimi, Guillaume Vernieres, Rahul Mahajan, and Travis Sluka.

2019 IODA Workshop in Monterey, CA

The Interface for Observation Data Access (IODA) is a component of the Joint Effort for Data assimilation Integration (JEDI) software system.  From a user’s perspective, IODA handles the input of observation data and output of diagnostic data. Efforts to transform IODA from a prototype system into an operational system that can handle big data is under way, and the task of collecting requirements from the JCSDA partner organizations was initiated by the 2019 IODA Workshop. The workshop took place from February 11 through February 13, 2019 in Monterey, California, and was attended by representatives from the U.S. Navy, NOAA, NASA, ECMWF, UK Met Office, NCAR, UCAR and JCSDA. In total 41 people attended from 8 partner organizations.

The workshop opened on Monday (February 11th) with introductory remarks from two of the organizers Nancy Baker (NRL) and Yannick Trémolet (JCSDA). These talks were followed by speakers from the Navy, NOAA, NASA, ECMWF and the UK Met Office describing requirements for observation handling from their various perspectives. The opening presentations were very informative, and provided useful requirements information for IODA along with an effective foundation for the focused discussions that were to take place through the remainder of the workshop.

Three focused discussion sessions were conducted over the remainder of the workshop. Tuesday’s topics were “in-memory observation handling” and “File and I/O requirements for observation handling”. The third discussion session took place on Wednesday with the subject “pre- and post-processing diagnostics requirements”. Each discussion session was preceded by a set of talks given by representatives from the various partner organizations which served to prepare everyone for the associated discussion. Each set of talks were designed to allow the audience to hear from multiple perspectives including users of DA systems, scientist contributors to JEDI, and JEDI software developers. Material from the presentations was condensed, on the fly, into a list of prompts that were used to seed the subsequent discussion. All workshop participants attended all of the discussion sessions resulting in an effective means for communicating requirements and perspectives among the different partner organizations in the JEDI project.

Work is under way to compile the requirements gathered at the workshop and transform these into the necessary actions to satisfy them. The compilation step has yielded categories that represent different aspects of the IODA design. These categories include themes such as security (the need to handle classified and private data), reliability (an operational system needs to be running 24/7), ease of use, and portability (runs with a variety of hardware platforms, software compilers, and programming languages). We will be using agile software development practices, such as ZenHub tracking and code sprints, to implement solutions for the IODA requirements. The results of the compilation of requirements along with the tracking of the work will be posted where they can be viewed by the partner organizations.

Overall, the workshop was conducted in a highly positive and enthusiastic manner that solidly reflected the collaborative spirit of the JEDI project. People from different organizations showed up with the intention of shaping IODA into a world-class observation handling system. Many important details relevant to the IODA design and architecture surfaced during the time we spent together. We mention here only a few highlights. Although the Human Genome Project is famous for its dealing with big data, the amount of observation data that we are dealing with is roughly an order of magnitude larger. When you consider the model data that are also involved in DA flows, that difference becomes three times greater. The manner in which observation data is collected, organized and stored in files is incredibly diverse, and yet there appears to be real promise in creating a common method, or a very small set of methods, for organizing and storing those data within IODA. Attaining a common method of organizing and storing the vast amounts of observation data will go a long way in fostering the collaboration, exchange, and sharing of scientific results.

Dr. Yannick Trémolet presenting at the 2019 IODA Workshop in Monterey, CA from February 11 - 13th.

Dr. Yannick Trémolet presenting at the 2019 IODA Workshop in Monterey, CA from February 11 - 13th.

The persistence and resourcefulness of the workshop participants were put to the test Wednesday morning when a wind storm knocked out the power and internet at the meeting site and surrounding neighborhood. Within an hour, communication between on-site and off-site participants was restored using a cell-phone and bluetooth speaker, with local participants clustered around laptops, allowing the day’s presentations and discussion session to be completed on time. This is a further testament to the collaborative spirit and can-do attitude of the partner organizations.

Monterey proved to be a wonderful venue for the workshop. Many fabulous restaurants and sights to see were at our disposal, including the world famous Monterey Aquarium and the beautiful sights along the Pacific Coast Highway.

Thanks are due to the organizers of the workshop, Thomas Auligne (JCSDA), Nancy Baker (NRL) and Yannick Trémolet (JCSDA); moderators of the discussion session, Daryl Kleist (NOAA), Ron Gelaro (NASA) and Nancy Baker (NRL) and all of the participants who contributed presentations and commentary. Special thanks go to Katherine Shanahan (JCSDA) who handled all of the necessary logistics for the workshop (including a last-minute venue change required as a consequence of the government shutdown), as well as Ben Ruston (NRL) and Elizabeth Satterfield (NRL) who organized the social activities that built the commaradie of the group. The organizers and the JEDI team are grateful to all participants for making the 2019 IODA Workshop remarkably successful, productive, and enjoyable.

This summary was written by Stephen Herbener (JCSDA).

JCSDA Symposium at the 2019 Annual AMS Meeting in Phoenix, AZ

The JCSDA held its 7th Symposium at the 99th American Meteorological Society (AMS) Annual Meeting in Phoenix, AZ on Janurary 8, 2019. The symposium spanned the whole day on Tuesday, consisting of six topical sessions and one poster session. Presentations and posters were offered by staff and contractors of the JCSDA partner agencies, the broader academic community, and international representatives such as ECMWF and the UK Met Office. The symposium proves time and again to be an important outreach event for JCSDA, as there was strong attendance from the data assimilation community (including JCSDA partners) and lots of opportunities for stimulating discussion. Though attendance was generally affected by the shutdown, the event was still a great success.

The oral sessions had a total of 18 presentations dedicated to various aspects of the data assimilation study. The first session was chaired by Benjamin Johnson (JCSDA), and was devoted to the New and Improved Data Assimilation Tools and Methods. This was followed by the session of Land, Ocean, Cryosphere, Air Quality, and Coupled Earth System Data Assimilation, chaired by Benjamin Ruston (NRL). The third and fourth sessions, chaired by François Vandenberghe (JCSDA) and Thomas Auligné (JCSDA), were dedicated to the Next-Generation Satellites and Sensors Data Assimilation. Two more oral sessions were held in the afternoon and featured talks on all-sky radiance assimilation, chaired by Hui Shao (JCSDA), and the Joint Efforts for Data Assimilation Integration (JEDI), chaired by Guillaume Vernieres (JCSDA). The JCSDA Symposium also co-sponsored a joint session (invitation only), National and International Program Overviews for Environmental Satellites, on the morning of Tuesday, January 9.

Thank you to James Yoe (NWS) for organizing this JCSDA event and to all of our partners and dedicated team members who participated.

This text was adapted from an excerpt written by Dr. Hui Shao (JCSDA) for the winter edition of the JCSDA newsletter.

Dr. François Vandenberghe presenting his poster at AMS. Photo credit: Dawn Mullally | CPAESS | UCAR

Dr. François Vandenberghe presenting his poster at AMS. Photo credit: Dawn Mullally | CPAESS | UCAR