Libraries for generic structural representation and serialisation of tree-shaped data

General
1

Hello,

Background/context: we have an in-house library doing automatic conversion of tree-shaped immutable data (no pointers, cycles, functions/closures, …) with automatic unwrapping of abstract types with representations [1] and serialisation thereof. So its doing two things simultaneously (representation and serialisation). We have to be able to do this live for arbitrarily defined data (see [1] for examples), because we use it for interactive data serialization in exploratory analysis in e.g. notebooks.

We have to update the library and I have the following questions:

  1. Is there a library a la Haskells Generic / Scala’s Shapeless in the F# ecosystem? I could use this as an intermediate step.

  2. Can somebody recommend a serialisation library (+ format) to serialize the following to binary: strings, some numeric datatypes, tuples, records, DUs, options, maps, sets, arrays and arbitrary combinations thereof? Maybe automatically. The libraries I found (FSPickler, Prime, Fleece) are mostly around explicit combinators and a much richer universe of types.

  3. We do the logic in [1] currently live via reflection and tree-walking. We want to lift it purely to the type level and do it once for a given composite type as in the examples in [1]. In some cases the types are known at compile time in projects and we think about generating the serialisation code at compile time. Which approach would you recommend?

    • Source code generation? Which lib? Seems hard with the white-space sensitive nature of F#.
    • What is the story around source generators in F# (my C# colleagues use them a lot …)
    • Type providers seem out, because the type might be defined inside the same project in which I want to add the code to.

  4. Is logic like in [1] already publicly available in F#? Or interesting to people in the community? Open-sourcing might be an option for us (it is a generic helper for us).

Thanks for any pointers!

[1]: The library can, given a registered transformation Abs1<'a> -> 'a * 'a and Abs2 -> array<int>, derive a representation such as and its partial inverse. 3 examples, one per line:

Abs1<int> -> int * int
array<Abs1<bool>> -> array<bool  * bool>
Set<Abs1<option<Abs2>>> -> Set<option<array<int>> * option<array<int>>>