You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Full Microsoft.Testing.Platform support... Eventually #275
Right now YoloDev supports Microsoft.Testing.Platform (MTP) via Microsoft.Testing.Extensions.VSTestBridge. This allows a smooth transition.
However, this does not allow us to leverage new features in MTP, like better hierarchy support and more granular code locations.
I suggest we not move forward with direct MTP support.
Unfortunately, Visual Studio doesn't actually respect the new hierarchy system in MTP. In fact, it's handling the hierarchy worse and flattening all tests to one level, including the groupings which are shown but never executed.
Rider, for unclear reasons, does not work with the experiment repo even with MTP support enabled. It does work with the current YoloDev MTP support, but only on windows.
I still wanted to post the experiment as a starting point for whenever MTP support is sufficiently mature to pick this up again
I haven't implemented a replacement for runsettings to enable expecto-specific configuration. The options are testconfig.json or forwarding command line arguments. I'd lean toward forwarding command line arguments
There's not an easy way to test if the test node hierarchies are working correctly. I'm guessing i'll need to run the mtp-based test project in server mode to get the needed information
How we should integrate with YoloDev. I'd hoped I could make MTP integration a function users call in their test project's main, but MTP won't detect any extensions unless you let it generate the main method
Could the failing tests be used in detecting the hierarchy? I have some tests that are failing to check that we actually run tests, with some simple stdout validation (iirc). Also, I'm not entirely sure I understand the last point? Don't we already generate main?
Using failing tests is an interesting idea, but I don't think it'd work for this case.
I can't think of any command we give MTP that evaluates based on test hierarchy.
There's filter, but they just pass that back to us and we evaluate the filter consistently with hierarchy.
As to generating main.
My understanding is that the current Expecto project template creates a Main.fs that invokes the Expecto cli directly while VSTest discovers YoloDev.Expecto.TestSdk via convention and then YoloDev discovers tests by attribute. So, both the Expecto CLI and VSTest are supported simultaneously, with MTP partially supported using VSTestBridge.
My experiment supports MTP with it's own runner, not MPT running under VSTest.
For MTP's own runner, they expect the test project to be an exe that starts MTP. So MTP and the and the Expecto CLI can't both be supported at once, since they both expect to be what runs when the project is executed.
My thought was to make mtp an option similar to runTestsWithCLIArgs (maybe runTestsWithMtp). This way users could easily swap between cli experiences.
The problem is that MTP's extensibility model heavily prioritizes code generation.
Extensions, like the TrxReporter, are discovered at build time and included in a generated main method.
It's possible to register extensions manually, but it's not expected and so it's rarely documented very well and users coming from other libraries will expect that referencing an extension package will auto-register the extension.
This means expecto projects using the MTP runner cannot have an explicit main without turning off code generation and manually register every extension.
Circling all the way back, that means adding MTP runner support to YoloDev will be a breaking change to many projects, requiring them to delete their main method
You can opt-in to MTP since the last version. We should have a breaking release at some point where it turns to an opt-out (or just forced feature). I think tests are ran in both modes (if I remember correctly).
Hmm. It's true that the incompatibility is with the Expecto cli and the current expecto template, not with YoloDev.Test.Sdk.
I'll have to experiment with how EnableExpectoTestingPlatformIntegration (which in turn sets IsTestingPlatformApplication) handles having both test bridge and a direct MTP extension available.
Activity
Alxandr commentedon May 22, 2026
Maybe open issues on MTP repo about the issues you found? That way they can be tracked?
farlee2121 commentedon May 22, 2026
I know they're already trying to work with the VS test explorer team to fix the hierarchy issue.
I'll have to poke around the rider issue a bit more to characterize the issue. My guess is I'll need to open that one with Jetbrains
farlee2121 commentedon May 22, 2026
Ionide also doesn't support direct MTP yet. That's waiting on an official MTP client package
farlee2121 commentedon Nov 7, 2026
Visual Studio support for id-based hierarchy is in the works and Ionide/FSAC support for MTP is close.
So, I decided to come back to this.
The experiment repo is pretty close to ready.
Some outstanding issues/questions
Alxandr commentedon Nov 16, 2026
Could the failing tests be used in detecting the hierarchy? I have some tests that are failing to check that we actually run tests, with some simple stdout validation (iirc). Also, I'm not entirely sure I understand the last point? Don't we already generate main?
farlee2121 commentedon Nov 16, 2026
Using failing tests is an interesting idea, but I don't think it'd work for this case.
I can't think of any command we give MTP that evaluates based on test hierarchy.
There's filter, but they just pass that back to us and we evaluate the filter consistently with hierarchy.
As to generating main.
My understanding is that the current Expecto project template creates a Main.fs that invokes the Expecto cli directly while VSTest discovers YoloDev.Expecto.TestSdk via convention and then YoloDev discovers tests by attribute. So, both the Expecto CLI and VSTest are supported simultaneously, with MTP partially supported using VSTestBridge.
My experiment supports MTP with it's own runner, not MPT running under VSTest.
For MTP's own runner, they expect the test project to be an exe that starts MTP. So MTP and the and the Expecto CLI can't both be supported at once, since they both expect to be what runs when the project is executed.
My thought was to make mtp an option similar to
runTestsWithCLIArgs(mayberunTestsWithMtp). This way users could easily swap between cli experiences.The problem is that MTP's extensibility model heavily prioritizes code generation.
Extensions, like the TrxReporter, are discovered at build time and included in a generated main method.
It's possible to register extensions manually, but it's not expected and so it's rarely documented very well and users coming from other libraries will expect that referencing an extension package will auto-register the extension.
This means expecto projects using the MTP runner cannot have an explicit main without turning off code generation and manually register every extension.
Circling all the way back, that means adding MTP runner support to YoloDev will be a breaking change to many projects, requiring them to delete their main method
Alxandr commentedon Nov 17, 2026
https://github.com/YoloDev/YoloDev.Expecto.TestSdk/blob/main/src/YoloDev.Expecto.TestSdk/build/net8.0/YoloDev.Expecto.TestSdk.targets#L6
You can opt-in to MTP since the last version. We should have a breaking release at some point where it turns to an opt-out (or just forced feature). I think tests are ran in both modes (if I remember correctly).
The sample test-project does not have
main:https://github.com/YoloDev/YoloDev.Expecto.TestSdk/blob/main/test/Sample.Test/Sample.Test.fsproj
farlee2121 commentedon Nov 17, 2026
Hmm. It's true that the incompatibility is with the Expecto cli and the current expecto template, not with YoloDev.Test.Sdk.
I'll have to experiment with how EnableExpectoTestingPlatformIntegration (which in turn sets IsTestingPlatformApplication) handles having both test bridge and a direct MTP extension available.