Mixing
Advice diagnosing Azure Reverse Proxy Gateway Timeouts
I've been using the ARR module in Azure App Service for a month or so (using the technique described here, using it as a reverse proxy on some routes to forward requests to a Wordpress server.
The site on the AppService is an ASP.NET MVC site, with some paths using the proxy, the rest handled by MVC.
Everything was working great.
Then all of a sudden, i started receving intermittent 502 errors, only for the reverse proxy paths (all other routes that aren't proxyed and get handled by ASP.NET MVC are fine). In the IIS logs, the sc-win32-status is 12002. ('internet timed out'. More info here)
Facts:
- The machine doesn't appear to be under stress (if it was, i'd expect all routes to be affected)
- Only the reverse proxy routes are timing out
- Downtime/slowness usually lasts 1-5 minutes
- I've tested the downstream server being proxy'ed to at the time of the issue, but it's always fine. Means it's the proxy, not the server.
The proxy rule looks like this:
<rule name="Wordpress Blog - Hubs" stopProcessing="true">
<match url="^(advice|lifestyle|news|wp-json|feed|about-us|contact-us|privacy-policy|terms-of-use)$" />
<conditions>
<add input="{HTTP_HOST}" pattern="www.mysite.com.au" />
</conditions>
<action type="Rewrite" url="https://mysite.wpengine.com/{R:1}" />
</rule>
Here's what "Collect .NET Profiler Trace" captured:
Clicking on 'Details' doesn't help much:
The failed requests look like this:
So it's definetely the ARR module timing out, but i can't figure out how to tell why. Any ideas on how i can diagnose this? I'm not sure how to debug since it's not 'code', therefore things like ApplicationInsights won't help. I'm trying to take memory dumps but since it only ever occurs for < 5 mins, and memory dumps take longer than that to capture, it's hard to capture it in time.
Any advice would be much appreciated.
Thanks
azure-web-sites reverse-proxy arr asked Jan 2 at 22:27
RPM1984RPM1984
55k45200308
add a comment |
I've been using the ARR module in Azure App Service for a month or so (using the technique described here, using it as a reverse proxy on some routes to forward requests to a Wordpress server.
The site on the AppService is an ASP.NET MVC site, with some paths using the proxy, the rest handled by MVC.
Everything was working great.
Then all of a sudden, i started receving intermittent 502 errors, only for the reverse proxy paths (all other routes that aren't proxyed and get handled by ASP.NET MVC are fine). In the IIS logs, the sc-win32-status is 12002. ('internet timed out'. More info here)
Facts:
- The machine doesn't appear to be under stress (if it was, i'd expect all routes to be affected)
- Only the reverse proxy routes are timing out
- Downtime/slowness usually lasts 1-5 minutes
- I've tested the downstream server being proxy'ed to at the time of the issue, but it's always fine. Means it's the proxy, not the server.
The proxy rule looks like this:
<rule name="Wordpress Blog - Hubs" stopProcessing="true">
<match url="^(advice|lifestyle|news|wp-json|feed|about-us|contact-us|privacy-policy|terms-of-use)$" />
<conditions>
<add input="{HTTP_HOST}" pattern="www.mysite.com.au" />
</conditions>
<action type="Rewrite" url="https://mysite.wpengine.com/{R:1}" />
</rule>
Here's what "Collect .NET Profiler Trace" captured:
Clicking on 'Details' doesn't help much:
The failed requests look like this:
So it's definetely the ARR module timing out, but i can't figure out how to tell why. Any ideas on how i can diagnose this? I'm not sure how to debug since it's not 'code', therefore things like ApplicationInsights won't help. I'm trying to take memory dumps but since it only ever occurs for < 5 mins, and memory dumps take longer than that to capture, it's hard to capture it in time.
Any advice would be much appreciated.
Thanks
azure-web-sites reverse-proxy arr asked Jan 2 at 22:27
RPM1984RPM1984
55k45200308
is all proxyed requests timed out? or you only get some response timeout? I have experienced similar symptoms, in my case main cause was ARR proxy timeout too short (some requests execution time are longer than 30 seconds - default timeout value).
– Geunho
Jan 3 at 3:45
@Geunho it's all of them. It's like the ARR module locks up altogether. I could however try adding a proxy rule to another site, just as a test - to see if it's isolated to just this particular downstream server, or the ARR module altogether.
– RPM1984
Jan 3 at 10:44
add a comment |
I've been using the ARR module in Azure App Service for a month or so (using the technique described here, using it as a reverse proxy on some routes to forward requests to a Wordpress server.
The site on the AppService is an ASP.NET MVC site, with some paths using the proxy, the rest handled by MVC.
Everything was working great.
Then all of a sudden, i started receving intermittent 502 errors, only for the reverse proxy paths (all other routes that aren't proxyed and get handled by ASP.NET MVC are fine). In the IIS logs, the sc-win32-status is 12002. ('internet timed out'. More info here)
Facts:
- The machine doesn't appear to be under stress (if it was, i'd expect all routes to be affected)
- Only the reverse proxy routes are timing out
- Downtime/slowness usually lasts 1-5 minutes
- I've tested the downstream server being proxy'ed to at the time of the issue, but it's always fine. Means it's the proxy, not the server.
The proxy rule looks like this:
<rule name="Wordpress Blog - Hubs" stopProcessing="true">
<match url="^(advice|lifestyle|news|wp-json|feed|about-us|contact-us|privacy-policy|terms-of-use)$" />
<conditions>
<add input="{HTTP_HOST}" pattern="www.mysite.com.au" />
</conditions>
<action type="Rewrite" url="https://mysite.wpengine.com/{R:1}" />
</rule>
Here's what "Collect .NET Profiler Trace" captured:
Clicking on 'Details' doesn't help much:
The failed requests look like this:
So it's definetely the ARR module timing out, but i can't figure out how to tell why. Any ideas on how i can diagnose this? I'm not sure how to debug since it's not 'code', therefore things like ApplicationInsights won't help. I'm trying to take memory dumps but since it only ever occurs for < 5 mins, and memory dumps take longer than that to capture, it's hard to capture it in time.
Any advice would be much appreciated.
Thanks
azure-web-sites reverse-proxy arr asked Jan 2 at 22:27
RPM1984RPM1984
55k45200308
I've been using the ARR module in Azure App Service for a month or so (using the technique described here, using it as a reverse proxy on some routes to forward requests to a Wordpress server.
The site on the AppService is an ASP.NET MVC site, with some paths using the proxy, the rest handled by MVC.
Everything was working great.
Then all of a sudden, i started receving intermittent 502 errors, only for the reverse proxy paths (all other routes that aren't proxyed and get handled by ASP.NET MVC are fine). In the IIS logs, the sc-win32-status is 12002. ('internet timed out'. More info here)
Facts:
- The machine doesn't appear to be under stress (if it was, i'd expect all routes to be affected)
- Only the reverse proxy routes are timing out
- Downtime/slowness usually lasts 1-5 minutes
- I've tested the downstream server being proxy'ed to at the time of the issue, but it's always fine. Means it's the proxy, not the server.
The proxy rule looks like this:
<rule name="Wordpress Blog - Hubs" stopProcessing="true">
<match url="^(advice|lifestyle|news|wp-json|feed|about-us|contact-us|privacy-policy|terms-of-use)$" />
<conditions>
<add input="{HTTP_HOST}" pattern="www.mysite.com.au" />
</conditions>
<action type="Rewrite" url="https://mysite.wpengine.com/{R:1}" />
</rule>
Here's what "Collect .NET Profiler Trace" captured:
Clicking on 'Details' doesn't help much:
The failed requests look like this:
So it's definetely the ARR module timing out, but i can't figure out how to tell why. Any ideas on how i can diagnose this? I'm not sure how to debug since it's not 'code', therefore things like ApplicationInsights won't help. I'm trying to take memory dumps but since it only ever occurs for < 5 mins, and memory dumps take longer than that to capture, it's hard to capture it in time.
Any advice would be much appreciated.
Thanks
azure-web-sites reverse-proxy arr azure-web-sites reverse-proxy arr asked Jan 2 at 22:27
RPM1984RPM1984
55k45200308
asked Jan 2 at 22:27
RPM1984RPM1984
55k45200308
asked Jan 2 at 22:27
RPM1984RPM1984
55k45200308
asked Jan 2 at 22:27
RPM1984RPM1984
55k45200308
asked Jan 2 at 22:27
RPM1984RPM1984
55k45200308
55k45200308
is all proxyed requests timed out? or you only get some response timeout? I have experienced similar symptoms, in my case main cause was ARR proxy timeout too short (some requests execution time are longer than 30 seconds - default timeout value).
– Geunho
Jan 3 at 3:45
@Geunho it's all of them. It's like the ARR module locks up altogether. I could however try adding a proxy rule to another site, just as a test - to see if it's isolated to just this particular downstream server, or the ARR module altogether.
– RPM1984
Jan 3 at 10:44
add a comment |
is all proxyed requests timed out? or you only get some response timeout? I have experienced similar symptoms, in my case main cause was ARR proxy timeout too short (some requests execution time are longer than 30 seconds - default timeout value).
– Geunho
Jan 3 at 3:45
@Geunho it's all of them. It's like the ARR module locks up altogether. I could however try adding a proxy rule to another site, just as a test - to see if it's isolated to just this particular downstream server, or the ARR module altogether.
– RPM1984
Jan 3 at 10:44
is all proxyed requests timed out? or you only get some response timeout? I have experienced similar symptoms, in my case main cause was ARR proxy timeout too short (some requests execution time are longer than 30 seconds - default timeout value).
– Geunho
Jan 3 at 3:45
is all proxyed requests timed out? or you only get some response timeout? I have experienced similar symptoms, in my case main cause was ARR proxy timeout too short (some requests execution time are longer than 30 seconds - default timeout value).
– Geunho
Jan 3 at 3:45
@Geunho it's all of them. It's like the ARR module locks up altogether. I could however try adding a proxy rule to another site, just as a test - to see if it's isolated to just this particular downstream server, or the ARR module altogether.
– RPM1984
Jan 3 at 10:44
@Geunho it's all of them. It's like the ARR module locks up altogether. I could however try adding a proxy rule to another site, just as a test - to see if it's isolated to just this particular downstream server, or the ARR module altogether.
– RPM1984
Jan 3 at 10:44
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54013989%2fadvice-diagnosing-azure-reverse-proxy-gateway-timeouts%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54013989%2fadvice-diagnosing-azure-reverse-proxy-gateway-timeouts%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
is all proxyed requests timed out? or you only get some response timeout? I have experienced similar symptoms, in my case main cause was ARR proxy timeout too short (some requests execution time are longer than 30 seconds - default timeout value).
– Geunho
Jan 3 at 3:45
@Geunho it's all of them. It's like the ARR module locks up altogether. I could however try adding a proxy rule to another site, just as a test - to see if it's isolated to just this particular downstream server, or the ARR module altogether.
– RPM1984
Jan 3 at 10:44