In quarter four Eric Lingerfelt, Evan Parker, and Dom Heinzeller from the JCSDA core team and Tariq Hamzey from NASA GMAO met in Boulder for a code sprint to redesign R2D2 into a scalable, flexible client/server system. R2D2, the Research Repository for Data and Diagnostics, is JEDI’s unique data management and retrieval system. It stores datasets and experiment results, manages model input and output, and uses a generic code that can be used by many organizations and models.
By splitting R2D2 into a lightweight, intuitive client application and an online REST API shaped to industry standards, the system achieves a separation of concerns. The advantages of this include scalability through stateless transactions and bottleneck reduction, flexibility by total client-server uncoupling, and independence of the underlying technology utilized. These attributes make for much greater sustainability and lower maintenance costs for deployment and future capability upgrades.
Led by Eric Lingerfelt, the code sprint focused on three core technical goals. The first goal was creating a standardized REST web interface comprising R2D2’s current codebase, the Flask micro web framework, and the OpenAPI 3.0 Specification. The second was to implement a new python client API, r2d2-client, which communicates with R2D2’s web service and manages data files locally and in the cloud. The final goal was to deploy this new service using EC2, AWS APIGateway, Docker, and Swagger support tools. All code sprint and future wrap-up tasks were planned and tracked using JCSDA’s Agile/SCRUM project management methodologies and techniques. The deliverables for all three technical goals were described in detail at the epic, issue, and sprint level.
Advanced planning, research, and development by the team allowed the code sprint to hit the ground running. Data server boilerplates and proof of principle draft pull requests were ready for full implementation, and the application details of the OpenAPI 3.0 Specification had already been thoroughly investigated.
The code sprint launched with an overview discussion on the importance of these central components and their integration into an end-to-end data service. Each day began with a daily “Spin Up” planning meeting and ended with a review and debrief meeting where the team evaluated the day’s outcomes and adjusted the next day’s goals. Despite a blizzard that dropped 18 inches of snow on Boulder partway through the week, closing UCAR offices, the team delivered an end-to-end prototype at the end of the sprint. The product of over a dozen pull requests, the new system enables secure HTTP Delete, Get, Patch, and Post operations on experiments and observations using the new client application with all other R2D2 data types to be implemented in the near future.