I get pretty irritated by slow feedback, so I would like to know if running tests from the REPL is possible? Rather than use dotnet test.
It is in Haskell and I find it very enjoyable because I can automate the reload/rerun the tests on file change and get instant feedback.
I’d be happy to use any other tool to get faster feedback too.
Basically I’d like to have this kind of workflow:
$ dotnet fsi ./MathService/Library.fs
> #load "./MathService/Library.fs";;MathService.Say.greet "World";;
// change the code, save the file, press up to re-run the tests (I'll automate this step)
> #load "./MathService/Library.fs";;MathService.Say.greet "World";;
I can’t just load a test file because some unknown (to me) setup steps are required:
$ dotnet fsi ./MathService.Tests/Tests.fs
.../MathService.Tests/Tests.fs(4,6): error FS0039: The namespace or module 'Xunit' is not define
My dev machine is a Linux host if it matters, and I’ve also got a Jetbrains Rider licence if it can help tooling wise.
I have never heard of anyone running unit tests from FSI.
The issue you’re seeing is essentially that you need to load all of the NuGet dependencies that your .NET project needs. FSI itself has no way to do that. This means loading all of the transitive dependencies of your dependencies too, resolving valid versions for each dependency in the graph, and loading them in the right order. This is something that package managers like NuGet or Paket can do for your .NET projects. If you’re using NuGet (built in to FSPROJ), then you’re out of luck. If you’re using Paket, there is a generate-load-scripts command which will generate scripts that load all the dependencies.
But even while using that Paket feature, over the years I’ve found that trying to load and run my application code into FSI often has subtle dependency. Sometimes I was able to work around the issue and sometimes I wasn’t able to understand it and just abandoned using FSI. Perhaps I’ve been unlucky.
I personally limit my use of FSI to exploring stand-alone F# code or exploring libraries with the #r directive. Within a script file you can quickly use a command like #r "nuget: FsToolkit.ErrorHandling to load a library and its dependencies automatically. This is distinct from the package management built into .NET projects.
I think it’s unlikely that FSI will allow you to get a faster test feedback loop, while staying reliable. Good luck!
I’ve used FSI for running unit tests, but it involves a bit of setup work, and it has plenty of caveats so the experience isn’t that great. But it does speed up my feedback time pretty significantly in some cases. Basically, I’ll have a Packages.fsx file in the root folder that just has a bunch of #r "path/to/whatever/package.dll" lines for each package (I don’t use the #r "nuget: ..." syntax here because it’s so much slower). You might be able to use paket’s “generate load scripts” feature to make this simpler.
Then for each project file I have a Proj.fsx file that will pretty much look like
so just include the project you’re in plus all the dependent projects (note these have to be loaded in order, or things will break).
Then finally you can setup the test file - let’s say I have a BunchOfLogic.fs file and a BunchOfLogicTest.fs file. In BunchOfLogicTest.fs, I’ll have
Then I can load it into an FSI session with dotnet fsi --load:BunchOfLogicTest.fsx, and any changes I make to BunchOfLogic or BunchOfLogicTest, I can just #q;; from my FSI window and rerun the dotnet fsi --load:BunchOfLogicTest.fs command again. Note that this will only pickup changes to the BunchOfLogic.fs and BunchOfLogicTest.fs - any other changes will require you to compile the project again, so it’s really only suitable for unit tests. Sometimes if only the test file changed, I’ll just highlight from the change down and hit alt+enter to rerun that section in FSI.
I’ve found dotnet fsi --load:file.fs to often have a much faster startup time than dotnet test, but as is probably obvious, it seems that FSI was never really designed for this sort of use, so it’s clunky, and sometimes it’s pretty easy to get confused. I can’t say I really recommend this approach. I’m hoping this feature request will get implemented at some point, which I think would make the situation a lot easier to work with.
You might also want to play around with stuff like dotnet build --no-dependencies && dotnet test --no-build to try and speed things up.
Thanks. That’s quite a shame. If I only changed one module containing only pure functions, it seems quite wasteful to “recompute the world” and have to wait.
I assumed dotnet fsi was similar to cabal repl in Haskell, which can load the whole project into memory. Instead, it seems to be just a “dumb” repl, similar to GHCi
Thanks. This looks promising. Could you give me a little more details please? Maybe point to a working repo?
Can this step be automated with a simple script or do I need to understand the inner workings of the tool chain to tweak things?
As a first step, I’ve done this:
# From project root
find */bin/Debug* -name "*.dll" | sed -E 's/(.+)/#r "\1"/' > Packages.fsx
But I get into the loading order problem you described. I’m also not quite sure which dlls I should track from the project root (in root/Packages.fsx) vs in the sub project’s folder (in sub/Proj.fsx)
F# users typically recommend using paket for managing dependencies in multi-project solutions, and paket pretty much follows the philosophy that all nuget dependencies should be managed at the solution level (you can end up getting yourself into a lot of trouble managing dependencies at the project level if two different projects call for different versions of the same dependent package). If you use paket for dependencies, you can put
generate_load_scripts: true
at the top of your paket.dependencies file like so, which will create a script file “.paket/load/<target_framework>/main.group.fsx” which loads all the dependencies of your solution, presumably in the correct order. Note it only does dependent packages from nuget, it doesn’t automatically load all your own project dlls. Those you have to get from the bin folder directly. I haven’t played around with paket load scripts too much, since I usually build the file by hand.
As a general rule, the packages you get from nuget (like FSharp.Core or NUnit or the like) should be managed at the .sln root folder, and then each .fsproj should have it’s own script to load that project’s referenced peer projects.
Keep in mind the speed of this technique comes from avoiding rebuilding the project with each code change. But that introduces a pitfall that if you change code outside the test file (and any file you add a #load directive for), those changes won’t be picked up until you do rebuild the project. You can easily lose all the time you’ve saved by debugging something while running on old code. So like I mentioned earlier, I use this to speed me up, but exclusively for unit tests where I’m only ever worrying about code changes in the test file and the file being tested. If you try to do anything else, you’re more likely to trip up somewhere and end up losing time instead.
@benjamin-thomas, please note that the generate-load-script feature now has a frontend which works as an extension exactly like the #r "nuget: ..." one.
Note that you can use FSI from VS Code or Rider rather than from command line, so you get the IDE feature to edit your script, and only reevaluate the files of your project that changed and those that are subsequent to it.
FSI is indeed more akin to GHCi rather than cabal repl, it remains valuable for workflows similar to what you are attempting, like calling into code that is in F# projects, despite it requires a bit of preparing the load directives accordingly.
Normally, to run tests from Rider is a smooth experience, despite it involves recompile cycle.
