Executable origin check gives incorrect results

[barometz]

As of v0.12.0, "the origin of test executables is verified before running them - if they have not been created on the local machine, they will be ignored with a warning"

I have several projects which are stored on a network drive, and after building them I invariably get the warning that the executables came from another computer. The same executables register fine when I make a locally stored checkout and build from there.

I imagine this is a result of an OS-provided mechanism to determine the origin, which not unreasonably seems to decide that anything on a network drive can't be trusted. Nevertheless, the feature does not work as stated - the origin of that executable absolutely is this computer.

A simple (if slightly distasteful) solution might be to make this check optional.

[csoltenborn]

Thanks for your report!

@nickuhlenhuth @LukaszMendakiewicz @spebl What's your take on this? I'd like to release a version within the next couple of days containing some project and item templates as well as the fix of #178, and I'm tempted to introduce a possibility to opt-out of that check (which due to GTA's settings can be done for single executables if desired)... Or is there a better way, i.e., a "real fix"?

[LukaszMendakiewicz]

I am not sure I fully understand the scenario with network drive. What would be the path Visual Studio sees to such executable? I think it might be what you suspect, such location is treated with lower trust, which actually means the feature is working as intended -- it prevents from automatically running executables that might be potentially replaced by bad actors.

@csoltenborn I think opt-out is fine, as long as we can guarantee that the opt-out is performed explicitly by the user. For example, reading the opt-out flag from .gta.runsettings file will be bad because the bad actor may control this file as well.

[barometz]

I was being slightly facetious, sorry about that. Yes, I imagine the feature works as intended, as the origin of the executable cannot be confirmed via this mechanism. Visual Studio itself gives a similar warning when I open a project from that location for the first time. I was hoping there might be another mechanism that could confirm the origin.

The path Visual Studio sees, as far as I can tell, does not indicate that it is on a network drive.

Debug logs, in case they're useful:

[2017-11-14 10:42:05 Informational] Visual Studio Version: VS2017
[2017-11-14 10:42:05 Informational] Google Test Adapter: Test discovery starting...
[2017-11-14 10:42:05 Informational] Solution settings: AdditionalTestExecutionParam: '', BatchForTestSetup: '', BatchForTestTeardown: '', BreakOnFailure: False, CatchExceptions: False, DebugMode: True, KillProcessesOnCancel: False, MaxNrOfThreads: 4, NrOfTestRepetitions: 1, ParallelTestExecution: False, ParseSymbolInformation: True, PathExtension: '', PrintTestOutput: False, RunDisabledTests: False, ShowReleaseNotes: True, ShuffleTests: False, ShuffleTestsSeed: 0, TestDiscoveryRegex: '', TestDiscoveryTimeoutInSeconds: 30, TestNameSeparator: '', TimestampOutput: False, TraitsRegexesAfter: {}, TraitsRegexesBefore: {}, UseNewTestExecutionFramework: True, WorkingDir: '$(ExecutableDir)'
[2017-11-14 10:42:05 Informational] No settings configured for test executable 'N:\(...).exe'; running with solution settings: AdditionalTestExecutionParam: '', BatchForTestSetup: '', BatchForTestTeardown: '', BreakOnFailure: False, CatchExceptions: False, DebugMode: True, KillProcessesOnCancel: False, MaxNrOfThreads: 4, NrOfTestRepetitions: 1, ParallelTestExecution: False, ParseSymbolInformation: True, PathExtension: '', PrintTestOutput: False, RunDisabledTests: False, ShowReleaseNotes: True, ShuffleTests: False, ShuffleTestsSeed: 0, TestDiscoveryRegex: '', TestDiscoveryTimeoutInSeconds: 30, TestNameSeparator: '', TimestampOutput: False, TraitsRegexesAfter: {}, TraitsRegexesBefore: {}, UseNewTestExecutionFramework: True, WorkingDir: '$(ExecutableDir)'
[2017-11-14 10:42:05 Error] ERROR: Executable N:\(...).exe came from another computer and was blocked to help protect this computer.
[2017-11-14 10:42:05 Informational] Test discovery completed, overall duration: 00:00:00.0032938
[2017-11-14 10:42:05 Error] ERROR: 
================
The following errors and warnings occured during test discovery:
ERROR: Executable N:\(...).exe came from another computer and was blocked to help protect this computer.
[2017-11-14 10:42:05 Informational] ========== Discover test finished: 0 found (0:00:00.5240524) 
==========

[csoltenborn]

@LukaszMendakiewicz Thanks for your feedback! Basically you are suggesting that we can offer to opt-out via VS/GTA's options, but not receive that particular option from the .gta.runsettings file? Is the assumption that it will be harder for an attacker to mess around with the settings as stored by VS than with a settings file located in some solution folder?

I'm fine with that approach - imho, the only drawback is that we loose the possibility to switch that option off only for the according executables (and still have that security check in place in general).

[LukaszMendakiewicz]

Yes. The scenario I have in mind is where you download a .zip with a solution from the Internet -- it might have some prebuilt binaries and .gta.runsettings both prepared by the bad actor. If we allow opt-out in .gta.runsettings, we lose. If we allow opt-out only manually in Options, that's a risk the end-user took willingly.

[csoltenborn]

@barometz This build contains an option to switch off origin verification - would you mind to give it a try?

[barometz]

Doesn't work for me. I installed the .vsix, opened VS, toggled the setting to True, tried, restarted VS and tried again. Same message, and the diagnostic logs indicate that the setting has been applied.

I also tried un/reinstalling the addon, with no change. How much trouble would it be for me to set it up so I can step through the addon's code?

[csoltenborn]

Stupid me - the (i.e., all) settings didn't make it to the discovery code due to some sanity checks... Please give this build a try.

Debugging the adapter in general isn't difficult, just a bit tricky since the adapter's code is executed within three different processes - Debugger.Launch() is your friend here. For more information, have a look at our wiki (and let me know if there's anything missing which prevents you from building/debugging the adapter).

[barometz]

That neatly explains why I wasn't seeing the same debug messages as before despite having debug info set to true. Probably should have mentioned that.

Yes, this build does work. I flipped the setting a few times to be sure, haven't seen any surprises. The additional warning seems entirely appropriate. Thank you, this adapter has been very helpful lately.

[csoltenborn]

You're welcome!