How to avoid bitrot in fsx scripts?

Hello everybody,

Context: I have a codebase with about 40k LOC in dlls (core logic, infrastructure) and about 10k+ in fsx scripts (data analysis, bug hunts, “applications”). I am working with VS 22 and sometimes VS Code.

Problem: How do I prevent bitrot in the fsx scripts as I evolve the dlls?

Question: What are the known ways to programmatically typecheck only fsx scripts so that my CI pipeline/a Post-Build check can handle this?

Details:

  • VS 22 gives met Intellisense errors for open fsx files. As this is neither automatic nor complete, it only helps if you currently happen to work with certain scripts. Not with scripts you use once every few months.
  • We currently have special projects bundling functionality for a collection of scripts, with each scripts assigned a “backing module”. This reduces the problem, but does not eliminate it (in particular member names may change). And the script itself remains.
    * I considered signature files for the backing modules, but as F# forbids non-concrete abstract types in signatures, this would require lots of wrappers. Not scalable.
  • According to fs_comp_guide, fsi should take all fsc options. Well, I tried “dotnet fsi --typecheckonly test.fsx”, and it still executed the script. And it raises the warning “FS0075” (only usable for testing purposes) … well, the test did not even work out.
  • I could do something like [fs_tut_proj_analysis], but this seems
    a) quite involved in using the compiler dlls directly
    b) is Fsharp.Compiler.Service always available via the SDK? It is part of my installation, but all tutorials use the nuget packages … and I wonder about incompatibilities via my dotnet-sdk provided libs.
  • Maybe there are some Ionide libs already providing this? Again, the same question as with the compiler libs above arises.
    • I do not have much experience with Ionide - it takes about 1.5 min to load our solution (but has been getting faster!), so VS Code is not really used that much.
    • Ionide does not even show the fsx files in projects (with xml entry type …). So the VS 22 UI is more convenient than Ionide on this aspect.

Summa summarum:

I am grateful for any pointers to approaches/scripts/tools and would like to know if anybody has already solved this problem.

[fs_tut_proj_analysis]: Tutorial: Project analysis | F# Compiler Guide (fsharp.github.io)

Example content for test.fsx:

let a = 2
do printfn $"Here is a: %i{a}."

Running dotnet fsi --typecheckonly test.fsx executes the printfn. :frowning:

I added tests that run in GitHub actions. My code isn’t scripts but I could run scripts too. The tests use bats core which is a test framework for running command line tools and verifying the output. What I’ve done is I added options to not do everything all the way, i.e. —dry-run flags and similar stuff. This allows me to run most code even in GitHub actions runners. Maybe you can do the same.

Thanks @dlidstrom. Unluckily most scripts use hardcoded values and are used interactively as “durable scratchpads” … with code sent directly from the IDE to the the interpreter. We have dry-run like flags for 2-3 major scripts, but I see it as nearly impossible for all the persisted data-analysis scripts. And our census last week showed 100+ active scripts.

We are currently investing another route: Compiling all scripts into a dll into a separate namespace and using this to typecheck stuff. Scripts are still run only on demand interactively.
I’ll report back how this approach worked out.

That sounds like a better approach to me. Good luck!

Hello!
It sounds like you’re dealing with a complex codebase and trying to prevent bitrot in your FSX scripts as you evolve your DLLs. You’re already bundling functionality with backing modules, but it’s not a complete solution. You’ve considered signature files, but they’re not scalable due to F#'s restrictions. Using dotnet fsi --typecheckonly didn’t work as expected, and you’re unsure about using FSharp.Compiler.Service directly.

Have you tried using Ionide libraries or FsAutoComplete for type checking? They might offer a more integrated solution for your CI pipeline or post-build checks. If you haven’t already, it could be worth exploring these options further.