Can I run Unquote unit tests via VS Code? [fixed]

[GarryP]

Edit: This problem seems to have been fixed with version 7.5.2 of Ionide.

I have a simple unit test project – using Unquote on xUnit – which works fine in VS but I can’t get the unit tests to complete in VS Code.

When I tell VS Code to run the tests via the Run Tests button in the Testing panel the tests run and run but they never complete – I have to cancel the run after about five minutes of watching the little circles spinning.

I don’t get any error messages, the tests just keep ‘running’.

The only VS Code extensions I have installed are:

• C# (I think that was a default extension)
• Ionide for F#

Do I need more? If so, which specific ones?

I’ve read quite a few articles relating to running tests in VS Code but they either don’t mention F#, or Unquote, or both, and they are all quite complicated to read – I just want to run some simple tests, like I did in VS.

The tests look like this:

namespace LuminosityTests

open System
open Xunit
open Swensen.Unquote
open MyStuff.Vector // Contains Luminosity type which is very simple.

module ``fromInt`` =

    module ``with minimum luminosity`` = 

        let luminosity = Luminosity.fromInt 0 

        [<Fact>]
        let ``Value should be same as input value`` () = test <@ luminosity.Value = 0 @> 

        [<Fact>]
        let ``Zero Padded Decimal should be as expected`` () = test <@ luminosity.ZeroPaddedDecimal = "000" @> 

        [<Fact>]
        let ``Decimal should be as expected`` () = test <@ luminosity.Decimal = "0" @> 

        [<Fact>]
        let ``Hex should be as expected`` () = test <@ luminosity.Hex = "00" @> 

        [<Fact>]
        let ``Percentage should be as expected`` () = test <@ luminosity.Percentage = "0%" @>

I’ve attached a screen-grab showing the package references of the UnitTests project (all packages were added via NuGet in VS, not VS Code, as I don’t know how to add stuff via NuGet in VS Code yet because there are too many options and I’m totally confused).

Note: I’d also like to run FsCheck tests too if the ‘fix’ will let me do both.

Note: I’m using VS Code 1.76.0 on Windows 10.

[ntwilson]

What I’ve done is in a fresh folder, run:

dotnet new xunit -lang f#
dotnet add package unquote

and then in the test file, add open Swensen.Unquote and change the Assert.IsTrue(true) line to test <@ true @>, and then ran the tests. Everything worked fine. Here’s the fsproj that it produced:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net7.0</TargetFramework>

    <IsPackable>false</IsPackable>
    <GenerateProgramFile>false</GenerateProgramFile>
  </PropertyGroup>

  <ItemGroup>
    <Compile Include="Tests.fs" />
    <Compile Include="Program.fs" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.2" />
    <PackageReference Include="unquote" Version="6.1.0" />
    <PackageReference Include="xunit" Version="2.4.2" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.5">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="coverlet.collector" Version="3.1.2">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
  </ItemGroup>

</Project>

I’m guessing if there’s an issue with your setup it’s likely in the fsproj file. Can you try running through the same steps that I did? That should tell us if the problem is with how your VSCode is setup vs if it’s how your particular project is setup.

[GarryP]

Thanks for looking into this.

I’ve never run any dotnet stuff in the command line before so that was a bit scary but I think I’ve done it.
I don’t understand what has happened, but it’s happened.
I made the changes you suggested to the Tests.fs file via VS Code, ran the tests and it worked.

The fsproj file for my original unit test project is:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>

    <IsPackable>false</IsPackable>
    <GenerateProgramFile>false</GenerateProgramFile>
    <AssemblyName>$(MSBuildProjectName)</AssemblyName>
  </PropertyGroup>

  <ItemGroup>
    <Compile Include="UnitTestHelpers.fs" />
    <Compile Include="LuminosityTests.fs" />
    <Compile Include="OpacityTests.fs" />
    <Compile Include="Program.fs" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.1.0" />
    <PackageReference Include="Unquote" Version="6.1.0" />
    <PackageReference Include="xunit" Version="2.4.1" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.3">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="coverlet.collector" Version="3.1.2">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\Vector\Vector.fsproj" />
  </ItemGroup>

</Project>

Apart from some extra files, a project reference, and some version number differences (I don’t know how to ‘fix’ those when not using VS) there doesn’t seem to be much different to yours that I can see.

Interestingly, I found a file called Ionide.trx in the TestResults folder in my original unit tests project which contains (along with some other things):

...
<Times creation="2023-03-07T09:44:09.4940914+00:00" queuing="2023-03-07T09:44:09.4940919+00:00" start="2023-03-07T09:44:07.3776245+00:00" finish="2023-03-07T09:44:09.5409736+00:00" />
...
<ResultSummary outcome="Completed">
    <Counters total="45" executed="45" passed="45" failed="0" error="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" completed="0" inProgress="0" pending="0" />
    <Output>
      <StdOut>[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.3+1b45f5407b (64-bit .NET 6.0.14)&#xD;
[xUnit.net 00:00:00.74]   Discovering: UnitTests&#xD;
[xUnit.net 00:00:00.79]   Discovered:  UnitTests&#xD;
[xUnit.net 00:00:00.79]   Starting:    UnitTests&#xD;
[xUnit.net 00:00:01.14]   Finished:    UnitTests&#xD;
</StdOut>
    </Output>
  </ResultSummary>

…which seems to suggest, to me anyway, that the tests ran successfully (timestamp reported is when I last ran the tests) but the result isn’t being reported back to the VS Code GUI somehow (or maybe I’m just not looking at it right).

Any further advice would be much appreciated.

[ntwilson]

I tried downgrading the versions in my copy to match the version numbers that you have, and it still works just fine .
To be clear I can hit the “Run Test” button next to a test or test module in the test explorer, then it shows a little spinning wheel for maybe 10 seconds

and then eventually, the spinning wheels change to a green checkmark

At this point, I don’t have any good recommendations for you besides to see if you can reproduce a working state by making a blank xunit project using the steps I posted above, and then only start adding code once you have the blank one working. I think we’d need to know if it’s some kind of problem with the code project you’re working with, or some kind of problem with how VS Code is setup on your computer.

[GarryP]

Thanks for sticking with this.

I think I might have found the problem (or part of it).

If I put the test in a module, such as in:

module Tests

open System
open Xunit
open Swensen.Unquote 

module ``with my module`` = 

    [<Fact>]
    let ``My test`` () =
        test <@ true @>

…the tests keep running and have to be cancelled.

As soon as I take the test out of the module, such as:

module Tests

open System
open Xunit
open Swensen.Unquote 

[<Fact>]
let ``My test`` () =
    test <@ true @>

…the test completes as expected.

I tried giving the module a name without the double-back-ticks but that made no difference and the tests did not complete as before.

However, If I put the code in a namespace, such as:

namespace MyUnitTests

open System
open Xunit
open Swensen.Unquote 

module ``with my module`` = 

    [<Fact>]
    let ``My test`` () =
        test <@ true @>

…the tests run and complete.

But, if I have a module within a module, such as:

namespace MyUnitTests

open System
open Xunit
open Swensen.Unquote 

module ``with my module`` = 

    module ``and this function`` = 

        [<Fact>]
        let ``My test`` () =
            test <@ true @>

…the tests don’t complete and have to be cancelled.

Am I doing something wrong?

I can rearrange and rename my tests so they’re not in a module hierarchy but that’s not a great solution for the long term.

[ntwilson]

Oh interesting! That’s a good find, and I can reproduce your issue now. I don’t think you’re doing anything wrong, I think you’ve found a bug. I notice that I can reproduce the problem with NUnit as well - it’s not just related to XUnit. I’m guessing that’s an Ionide bug, since dotnet test works just fine and we can experience it on multiple test frameworks. Do you want to make an issue in ionide? Otherwise I can.

[GarryP]

I found a bug issue which has similar symptoms but I’m not sure if the problem is the same:

Test Explorer doesn't support tests with `TestCaseAttribute`

Do you think it’s worth me adding a new issue or adding a comment to the existing one?

[ntwilson]

Yup, I think you found it. I can see in that issue:

where e.g. t.FullName is PoudriereC2.Tests.MediaType.MediaTypeHandlerTests.TestGoodValues and the actual map key is PoudriereC2.Tests.MediaType+MediaTypeHandlerTests.TestGoodValues .

Oh! regarding the + in the computed name - that’s what a type name for a type inside a module looks like - I believe the format is <NAMESPACE>+<MODULE>.<TYPE> . So we may have something off in our detection/calculation of module-embedded vs namespace-level tests?

And when I run dotnet test --list-tests, I get a + in the computed name (so on my repo: Tests+SubTests.My test).
So I think it’s the same issue. It looks like the reason that the issue was reopened 3 days ago. Nice digging!

[GarryP]

Thanks for checking.

I’ve added a comment to that issue; hopefully it will be of some use.

[GarryP]

This problem seems to have been fixed with version 7.5.2 of Ionide.
I haven’t done any extensive checking but my tests work now as they did in VS.