MTP documentation feedback

[Youssef1313]

Type of issue

Other (describe below)

Description

👋🏻 Hi @Youssef1313 - thank you very much for you prompt reply, really appreciated!

I've read a few docs about this and it's all very confusing.

REF 1: https://learn.microsoft.com/en-us/dotnet/core/testing/microsoft-testing-platform-intro?tabs=dotnetcli#use-dotnet-test

So this doc starts out by saying:

• dotnet run
• dotnet exec
• dotnet test

If i'm trying to explain this to team members or newbies, I'm sure I would get asked: "ok.. so we have 3 options - which one, and why?". I know that's exactly what I'm thinking.

As to why I thought dotnet test is going away is because of the order of operations, listed above. I incorrectly assumed that dotnet test is here right now is because of this (emphasis, mine):

Microsoft.Testing.Platform offers a compatibility layer with vstest.console.exe and dotnet test ensuring you can run your tests as before while enabling new execution scenario.

That's how I read that paragraph. It felt like (to me) that it was/is a temp stop-gap solution until we all move over to some "new execution scenario". Execution scenario (to me) is 'dotnet run'

Also, trying to make sure i'm using MTP instead of VSTest is a struggle. I'm on .NET 9. So I've got 2x properties in my csproj files:

<UseMicrosoftTestingPlatformRunner>true</UseMicrosoftTestingPlatformRunner>
<TestingPlatformDotnetTestSupport>true</TestingPlatformDotnetTestSupport>

and then i'm not 💯 sure what dotnet test is using behind the scenes. I still feel like dotnet test is setup to use vstest by default with a temp hack added into to switch over to MTP. But the real way should be dotnet run.

That's how i'm feeling, from reading docs and stuff.

How could the docs be updated? Glad you asked! I feel like they are ok right now but are just a dump of info: "Here's all the things. K-gl-have-fun-etc"

versus

Ok - here's the recommended way .. the Happy Path™️ . (aka. dotnet test)
and if you're still reading ...
here is all the advanced and hardcore ways to do stuff (aka. dotnet run, dotnet exec) etc. with reasons why i would be using this over that.

If dotnet test is just a wrapper for dotnet run with some other fancy pants things like test project discovery, etc... great! But .. the average jane/joe prolly doesn't need to know that in the Happy Path™️ .

Anyways, I hope this helps.

Oh. and other confusing things:

• settings files? can we say 'parallel all the things'?
• code coverage? (i saw coverlet is undersupported so we're stuck using ms coverage, I believe). Also, settings for this (exclude test projects, etc).
• merging of coverage files

I think I stumbled on the Coverage Samples. This is seriously, seriously awesome btw! 👏🏻

Originally posted in microsoft/testfx#5408 (reply in thread) by @PureKrome

Page URL

https://learn.microsoft.com/en-us/dotnet/core/testing/microsoft-testing-platform-intro?tabs=dotnetcli#use-dotnet-test

Content source URL

https://github.com/dotnet/docs/blob/main/docs/core/testing/microsoft-testing-platform-intro.md

Document Version Independent Id

e3466361-50b2-7399-3f1d-91ace5686996

Platform Id

5714f5f9-3384-187b-e8d8-2d810f1c0d7b

Article author

@Evangelink

Metadata

• ID: b8933d12-6ba6-8b9a-94b8-36fb3bf883c9
• PlatformId: 5714f5f9-3384-187b-e8d8-2d810f1c0d7b
• Service: dotnet-fundamentals

Related Issues

[Youssef1313]

@gewarren @BillWagner @IEvangelist Do you have ideas here on how to re-word the doc to address the points of confusion here? I don't think the original intent here is to indicate that dotnet test is going away.

[PureKrome]

Further to this / my confusion. Here's a really good starting point (as to, which does what and why?)

• dotnet test -t : runs all the tests in the solution 😢
• dotnet run --list-tests (in the test folder, not repo root) : lists all the tests ✅

So this makes me think: okay .. dotnet test isn't really working. maybe it's a temp wrapper for the time being .. but don't get used to it and move over to dotnet run as that is where we should be, for the future.

[Evangelink]

@PureKrome Thanks for the pointers!

For the new dotnet test experience, we specifically decided not to use short version of the options as we had many cases where the short name we used was later on picked up by other dotnet SDK commands for different actions causing us to be forced to do breaking changes. On a side note, I also don't like short name that much and I think a good CLI completion is a better solution.

[PureKrome]

@Evangelink completely agree! I always think "if a new person comes here and reads this code (or ci commands/steps/script) will they understand what is going on? -t == no idea. --list-tests == totally get it)

fwiw, dotnet test --list-tests also runs the tests 😢

[Youssef1313]

@PureKrome You need dotnet test -- --list-tests and not dotnet test --list-tests. This was intended to be clarified in Testing with dotnet test article which details the two modes of dotnet test for running MTP test projects. I also summarized the article in this StackOverflow answer

[PureKrome]

Ah yes - i remember trying that also, and ... it didn't display any tests :(

dotnet test -- --list-tests

vs

dotnet run --project "blah" --list-tests (of course this is just one project in the sln)

displays tests

Which was another reason why i though dotnet test wasn't doing the job right for MTP and will be replaced.

[Youssef1313]

Maybe try: dotnet test -v:n -- --list-tests (might be related to terminal logger)

[PureKrome]

Nope - exactly the same thing, as above :(

I think this is a great example about my anxiety with dotnet test + MTP.

I feel like i'm hitting it hard, in it's v1/beta stage :awkward:

Or .. I'm coal-face testing it for the rest of the world!

[Youssef1313]

The dotnet test experience will be much improved for .NET 10, by opting-in via dotnet.config.

For the current experience, you need the following combination:

• <TestingPlatformCaptureOutput>false</TestingPlatformCaptureOutput> in project file or Directory.Build.props
• dotnet test -v:detailed -- --list-tests

Example output of that:

Restore complete (0.6s)
    Determining projects to restore...
    All projects are up-to-date for restore.
You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy
  TestProject69 succeeded (1.0s) → bin\Debug\net8.0\TestProject69.dll
  TestProject69 test succeeded (1.4s)
    Run tests: 'C:\Users\ygerges\source\repos\TestProject69\TestProject69\bin\Debug\net8.0\TestProject69.dll' [net8.0|x64]
    MSTest v3.8.3 (UTC 17/03/2025) [win-x64 - .NET 8.0.14]
    The following Tests are available:
    TestMethod1
    Tests succeeded: 'C:\Users\ygerges\source\repos\TestProject69\TestProject69\bin\Debug\net8.0\TestProject69.dll' [net8.0|x64]

[Evangelink]

@PureKrome You are probably facing the issue linked to the .NET 9+ terminal logger that is basically just breaking discovery display. can you do dotnet test /tl:false -- --list-tests?

[PureKrome]

Sure can:

[Youssef1313]

@PureKrome Are you doing that in addition to <TestingPlatformCaptureOutput>false</TestingPlatformCaptureOutput> please?

[PureKrome]

blah.api.tests.csproj

<Project Sdk="Microsoft.NET.Sdk">

    <PropertyGroup>
        <OutputType>Exe</OutputType>
        <UseMicrosoftTestingPlatformRunner>true</UseMicrosoftTestingPlatformRunner>
        <TestingPlatformDotnetTestSupport>true</TestingPlatformDotnetTestSupport>
    </PropertyGroup>

    <ItemGroup>
        <PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" />
        <PackageReference Include="Microsoft.Extensions.Diagnostics.Testing" />
        <PackageReference Include="Microsoft.Testing.Extensions.CodeCoverage" />
        <PackageReference Include="Moq" />
        <PackageReference Include="Serilog.AspNetCore" />
        <PackageReference Include="Serilog.Extensions.Hosting" />
        <PackageReference Include="Shouldly" />
        <PackageReference Include="Testcontainers" />
        <PackageReference Include="Testcontainers.Azurite" />
        <PackageReference Include="WorldDomination.HttpClient.Helpers" />
        <PackageReference Include="xunit.v3" />
    </ItemGroup>

    <ItemGroup>
      <ProjectReference Include="<snip>" />
      <ProjectReference Include="<snip>" />
    </ItemGroup>

</Project>

[Youssef1313]

The project you pasted is missing <TestingPlatformCaptureOutput>false</TestingPlatformCaptureOutput>.
You need to add it and do one of:

• dotnet test /tl:false -- --list-tests
• dotnet test -v:detailed -- --list-tests