Environment Variables from IIS auto added to web.config cause 502 when .net core 2 MVC app deployed

[niico100]

Create a new ASP.net Core 2 MVC template app. Set it up to use full IIS locally, add an environment variable; (I need to do this for AWS), the app runs fine locally on full IIS.

Publish to a server with IIS (Win Server 2012 R2 in my case, the web.config contains said environment variables). You get a 502, no stdout log or Event in Event Viewer is created. No helpful error message anywhere.

To get it working you need to delete the environment variables from the web.config - if you need environment variables you need to add them in IIS manually (IIS / [click site] / management / configuration editor / system.webServer / aspNetCore / environmentVariables)

cc @guardrex

[shirhatti]

Which variable is breaking you?

[niico100]

ANY env variable will stop the project working on the sever. Try it.

[Tratcher]

is it inheriting a environmentVariables section from a parent app? There are some known issues with conflicts there.

[shirhatti]

Just so I understand correctly. If you set a system environment variable and then set the same environment variable in your web.config, then you're broken?

[niico100]

If I set an env variable in IIS on my dev machine (as detailed in my original comment), an environment variable is put in the web.config automatically. If you just then publish the site - the env variables are still in the published web config web.config.

Having env variables in the web.config breaks the site on the server. When you delete them - the site then works.

Visual Studio 2017 also puts other stuff in the web.config that also breaks it - you need to strip the web.config down to this to make it work:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <system.webServer>
    <handlers>
      <add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModule" resourceType="Unspecified" />
    </handlers>
    <aspNetCore processPath="dotnet" arguments=".\MyApp.dll" stdoutLogEnabled="true" stdoutLogFile=".\logs\stdout" />
  </system.webServer>
</configuration>
<!--ProjectGuid: 13188be9-ccd6-4411-bac8-d3cdad5d924a-->

[guardrex]

@niico100 Show them the web.config you showed me that was breaking you that had the two env vars in it with the values redacted.

[niico100]

Here is a web.config that was generated by Visual Studio 2017 Community when I published the project (after env variables added to IIS) - it causes a 502:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <system.webServer>
    <handlers>
      <add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModule" resourceType="Unspecified" />
    </handlers>
    <aspNetCore processPath="dotnet" arguments=".\MyApp.dll -argFile IISExeLauncherArgs.txt" forwardWindowsAuthToken="false" startupTimeLimit="3600" requestTimeout="23:00:00" stdoutLogEnabled="false" stdoutLogFile=".\logs\stdout">
      <environmentVariables>
        <environmentVariable name="AWS_ACCESS_KEY_ID" value="redacted" />
        <environmentVariable name="AWS_SECRET_ACCESS_KEY" value="redacted" />
      </environmentVariables>
    </aspNetCore>
  </system.webServer>
</configuration>
<!--ProjectGuid: 20b626d4-9f01-4a05-897e-3a8b1c820dfb-->

Having any env variables in the web.config has the same effect.

[guardrex]

... and IIRC during troubleshooting, we whittled down the aspNetCore element to just ...

<aspNetCore processPath="dotnet" arguments=".\MyApp.dll" 
            stdoutLogEnabled="false" stdoutLogFile=".\logs\stdout">

... and it was still throwing until @niico100 removed those env vars.

[shirhatti]

@niico100 There was an issue in Visual Studio where it modified your web.config. That has since been fixed in Visual Studio 2017 Update 15.7
https://developercommunity.visualstudio.com/content/problem/220371/webconfig-modified-startuptimelimit3600-requesttim.html

This should no longer happen.

[shirhatti]

I'm going to go close this issue. Please feel free to re-open if you continue to this behavior or if I misunderstood what you said. Thanks!

[niico100]

This addresses part of the issue - but the environment variables issue remains and isn't addressed by this update.

[niico100]

Also - we need to be clear about where the issue is here. Is the issue that the env variables are added to the web.config - or is the issue that having env variables in the web.config stops the app running on the server? (probably the latter?).

[jkotalik]

Let's try to repro this on the latest build ANCM. @niico100 what verison of the aspnetcore.dll is being used on the server? You can check this in the C:\Windows\System32\inetsrv folder for the dll. @pan-wang has there ever been an issue with ANCM having issues with environment variables?

[niico100]

I actually just tried that (in VS 15.7.1) - but I got a different error and have had to switch back to some other urgent work for now.

(for what it's worth here's the new error - vanilla .net core 2 mvc solution, though it's probably not very relevant to this issue and happens regardless of env variables):

Error:
An assembly specified in the application dependencies manifest (WebApplication1.deps.json) was not found:
package: 'Microsoft.AspNetCore.Antiforgery', version: '2.0.3'
path: 'lib/netstandard2.0/Microsoft.AspNetCore.Antiforgery.dll'

This assembly was expected to be in the local runtime store as the application was published using the following target manifest files:
aspnetcore-store-2.0.8.xml

[jkotalik]

Yeah that seems like an issue with your application rather than ANCM. Running in VS will use IISExpress with a different aspnetcore.dll than the one used by IIS. When you get the chance, can you check the versions of aspnetcore.dll in C:\Windows\System32\inetsrv and C:\Program Files\IIS Express\

[niico100]

So right click C:\Windows\System32\inetsrv\aspnetcore.dll - and go to details / file version?

7.1.1990.0

I'm not using IIS Express locally I'm using full IIS (with env variable set)

[niico100]

locally its 8.2.1991.0

[guardrex]

btw @niico100 ... WRT that error. If it's a 2.0 project using the All package, try rolling your All package reference back to 2.0.7 in the app. U might be hitting a snag with the All 2.0.8 package. Discussion is on a docs issue and an IIS Integration repo issue.

[shelland]

@shirhatti Sorry, but i'm still experiencing this problem with -argFile IISExeLauncherArgs.txt in my web.config file after publishing to the remote server (using MS Deploy) although I have Visual Studio v.15.7.1.

[shirhatti]

@shelland I don't believe what you're observing is related to the issue tracked here. Please file a new issue otherwise your concern going to be lost in the crowd here.

[muratg]

@BillHiebert I was told that you implemented a workaround for this on tooling. What's it?

[muratg]

cc @pan-wang please sync up with @BillHiebert on this.

[BillHiebert]

Which "this" are you referring to? The original issue where environment variables were being written into web.config is fixed. The -argfile IISExeLauncherArgs.txt is still an issue, as is any other ancm settings in web.config, unless publish fixes these settings when the application is published.

[pan-wang]

One issue is about IISExeLauncherArgs.txt. For environment variables, I believe @jhkimnew had test covering it automatically.

[muratg]

@shirhatti Do you know what's the latest on this?

[shirhatti]

We haven't been able to repro @niico100's original issue.

The only hypothesis I have on what could be going on is that thee version of ANCM on the server isn't new enough and hence it's failing at schema validation.
However, that hypothesis is at odds with the claim that deleting it and re-adding the environment variable via the configuration editor 😕

@niico100 If you're still observing this issue, can you get back to us? Perhaps a FREB log can indicate where the 502 is even originating from

[pakrym]

Closing. Please reopen if there is more information about the issue.