Building Incremental and Reproducible Data Pipelines - Patrick Ferris - FUN OCaml 2024

Patrick Ferris's FUN OCaml 2024 talk recording! Overview by Patrick: We present the good and the bad of building a dataflow engine in OCaml. The engine underpins a complex ecological analysis of avoided deforestation projects in tropical moist rainforests. We will discuss: Onboarding experienced developers who are new to OCaml. - Building an operating system in OCaml to run Python/R code.Developing geospatial libraries and how this benefited from Outreachy internships and the compiler's backwards compatibility. Managing a transition from monadic, asynchronous libraries to direct-style code. This work is part of a multi-year collaboration between the departments of Computer Science, Ecology, Zoology and Geography at the University of Cambridge. Full title: Building Incremental and Reproducible Data Pipelines for Tackling Climate Change Connect with us Website: https://fun-ocaml.com/ Twitter: https://x.com/FunOCaml Bluesky: https://bsky.app/profile/fun-ocaml.com #ocaml