Mixing
![Should I join a company if its reviews on glassdoor etc are not very good? [closed]](https://lh4.googleusercontent.com/-T9tqk3BiQOg/AAAAAAAAAAI/AAAAAAAAAAA/ABtNlbA9F9zwGdr9-hownlNsE057kqtU9A/s72-c-mo/photo.jpg?sz=32)
Ok to use meta noindex on 404 pages?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}
I have a custom 404.html file on my website that gets served with a 404 status code when a URL is not found, standard stuff. But if you visit the exact path /404.html on the server, the server says "hey, there's actually a file there" and shows you the 404 page but without the 404 status code.
I wanted to make sure that the /404.html path doesn't get accidentally indexed in search, so I added a <meta name="robots" content="noindex"> tag to it.
Of course, that means that my nonexistant URLs also have the meta noindex tag as part of the 404 page. Is there any way that the presence of this meta noindex tag could cause an issue for search engines / crawlers, or is this setup totally fine?
seo web-crawlers 404 noindex
asked Jan 30 at 18:24
Maximillian LaumeisterMaximillian Laumeister
4,8161829
add a comment |
I have a custom 404.html file on my website that gets served with a 404 status code when a URL is not found, standard stuff. But if you visit the exact path /404.html on the server, the server says "hey, there's actually a file there" and shows you the 404 page but without the 404 status code.
I wanted to make sure that the /404.html path doesn't get accidentally indexed in search, so I added a <meta name="robots" content="noindex"> tag to it.
Of course, that means that my nonexistant URLs also have the meta noindex tag as part of the 404 page. Is there any way that the presence of this meta noindex tag could cause an issue for search engines / crawlers, or is this setup totally fine?
seo web-crawlers 404 noindex
asked Jan 30 at 18:24
Maximillian LaumeisterMaximillian Laumeister
4,8161829
Alternately you could name the file/404-3pAhD3tM.htmlwhich would be unlikely to ever to get crawled unless you linked to it or had directory indexes on.
– Stephen Ostermiller♦
Jan 30 at 22:32
I also suspect that if Google did crawl/404.htmlit would treat it as a "soft 404" because it presumably has "Page Not Found" in the title and H1.
– Stephen Ostermiller♦
Jan 30 at 22:34
add a comment |
I have a custom 404.html file on my website that gets served with a 404 status code when a URL is not found, standard stuff. But if you visit the exact path /404.html on the server, the server says "hey, there's actually a file there" and shows you the 404 page but without the 404 status code.
I wanted to make sure that the /404.html path doesn't get accidentally indexed in search, so I added a <meta name="robots" content="noindex"> tag to it.
Of course, that means that my nonexistant URLs also have the meta noindex tag as part of the 404 page. Is there any way that the presence of this meta noindex tag could cause an issue for search engines / crawlers, or is this setup totally fine?
seo web-crawlers 404 noindex
asked Jan 30 at 18:24
Maximillian LaumeisterMaximillian Laumeister
4,8161829
I have a custom 404.html file on my website that gets served with a 404 status code when a URL is not found, standard stuff. But if you visit the exact path /404.html on the server, the server says "hey, there's actually a file there" and shows you the 404 page but without the 404 status code.
I wanted to make sure that the /404.html path doesn't get accidentally indexed in search, so I added a <meta name="robots" content="noindex"> tag to it.
Of course, that means that my nonexistant URLs also have the meta noindex tag as part of the 404 page. Is there any way that the presence of this meta noindex tag could cause an issue for search engines / crawlers, or is this setup totally fine?
seo web-crawlers 404 noindex
seo web-crawlers 404 noindex
asked Jan 30 at 18:24
Maximillian LaumeisterMaximillian Laumeister
4,8161829
asked Jan 30 at 18:24
Maximillian LaumeisterMaximillian Laumeister
4,8161829
edited Jan 30 at 20:28
Maximillian Laumeister
asked Jan 30 at 18:24
Maximillian LaumeisterMaximillian Laumeister
4,8161829
asked Jan 30 at 18:24
Maximillian LaumeisterMaximillian Laumeister
4,8161829
asked Jan 30 at 18:24
Maximillian LaumeisterMaximillian Laumeister
4,8161829
4,8161829
Alternately you could name the file/404-3pAhD3tM.htmlwhich would be unlikely to ever to get crawled unless you linked to it or had directory indexes on.
– Stephen Ostermiller♦
Jan 30 at 22:32
I also suspect that if Google did crawl/404.htmlit would treat it as a "soft 404" because it presumably has "Page Not Found" in the title and H1.
– Stephen Ostermiller♦
Jan 30 at 22:34
add a comment |
Alternately you could name the file/404-3pAhD3tM.htmlwhich would be unlikely to ever to get crawled unless you linked to it or had directory indexes on.
– Stephen Ostermiller♦
Jan 30 at 22:32
I also suspect that if Google did crawl/404.htmlit would treat it as a "soft 404" because it presumably has "Page Not Found" in the title and H1.
– Stephen Ostermiller♦
Jan 30 at 22:34
Alternately you could name the file
/404-3pAhD3tM.html which would be unlikely to ever to get crawled unless you linked to it or had directory indexes on.– Stephen Ostermiller♦
Jan 30 at 22:32
Alternately you could name the file
/404-3pAhD3tM.html which would be unlikely to ever to get crawled unless you linked to it or had directory indexes on.– Stephen Ostermiller♦
Jan 30 at 22:32
I also suspect that if Google did crawl
/404.html it would treat it as a "soft 404" because it presumably has "Page Not Found" in the title and H1.– Stephen Ostermiller♦
Jan 30 at 22:34
I also suspect that if Google did crawl
/404.html it would treat it as a "soft 404" because it presumably has "Page Not Found" in the title and H1.– Stephen Ostermiller♦
Jan 30 at 22:34
add a comment |
1 Answer
1
active
oldest
votes
There's no problem having a meta noindex tag on the 404 page, to prevent 200 OK responses being indexed.
If this was a PHP page then you could obviously just send a 404 Not Found header as part of the standard response - to make sure that it always returns a 404.
You could also use .htaccess (mod_rewrite) to force any direct requests to 404.html to also trigger a 404:
RewriteEngine On
RewriteCond %{ENV:REDIRECT_STATUS} ^$
RewriteRule ^404.html$ - [R=404]
The RewriteCond directive that checks the REDIRECT_STATUS environment variable is required if you want your custom 404 to be served (so that it only applies to direct requests). Otherwise, there will likely be a rewrite loop (500 error) when trying to serve the custom error document and you'll see the standard server generated 404 response (but it's still a 404).
answered Jan 30 at 21:12
MrWhiteMrWhite
32k33367
1
Thank you for the good ideas. For my hosting setup it's easiest to just leave themeta noindexin place, so it's good to know it won't cause any issues if I do.
– Maximillian Laumeister
Jan 30 at 21:59
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "45"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fwebmasters.stackexchange.com%2fquestions%2f120620%2fok-to-use-meta-noindex-on-404-pages%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
There's no problem having a meta noindex tag on the 404 page, to prevent 200 OK responses being indexed.
If this was a PHP page then you could obviously just send a 404 Not Found header as part of the standard response - to make sure that it always returns a 404.
You could also use .htaccess (mod_rewrite) to force any direct requests to 404.html to also trigger a 404:
RewriteEngine On
RewriteCond %{ENV:REDIRECT_STATUS} ^$
RewriteRule ^404.html$ - [R=404]
The RewriteCond directive that checks the REDIRECT_STATUS environment variable is required if you want your custom 404 to be served (so that it only applies to direct requests). Otherwise, there will likely be a rewrite loop (500 error) when trying to serve the custom error document and you'll see the standard server generated 404 response (but it's still a 404).
answered Jan 30 at 21:12
MrWhiteMrWhite
32k33367
1
Thank you for the good ideas. For my hosting setup it's easiest to just leave themeta noindexin place, so it's good to know it won't cause any issues if I do.
– Maximillian Laumeister
Jan 30 at 21:59
add a comment |
There's no problem having a meta noindex tag on the 404 page, to prevent 200 OK responses being indexed.
If this was a PHP page then you could obviously just send a 404 Not Found header as part of the standard response - to make sure that it always returns a 404.
You could also use .htaccess (mod_rewrite) to force any direct requests to 404.html to also trigger a 404:
RewriteEngine On
RewriteCond %{ENV:REDIRECT_STATUS} ^$
RewriteRule ^404.html$ - [R=404]
The RewriteCond directive that checks the REDIRECT_STATUS environment variable is required if you want your custom 404 to be served (so that it only applies to direct requests). Otherwise, there will likely be a rewrite loop (500 error) when trying to serve the custom error document and you'll see the standard server generated 404 response (but it's still a 404).
answered Jan 30 at 21:12
MrWhiteMrWhite
32k33367
1
Thank you for the good ideas. For my hosting setup it's easiest to just leave themeta noindexin place, so it's good to know it won't cause any issues if I do.
– Maximillian Laumeister
Jan 30 at 21:59
add a comment |
There's no problem having a meta noindex tag on the 404 page, to prevent 200 OK responses being indexed.
If this was a PHP page then you could obviously just send a 404 Not Found header as part of the standard response - to make sure that it always returns a 404.
You could also use .htaccess (mod_rewrite) to force any direct requests to 404.html to also trigger a 404:
RewriteEngine On
RewriteCond %{ENV:REDIRECT_STATUS} ^$
RewriteRule ^404.html$ - [R=404]
The RewriteCond directive that checks the REDIRECT_STATUS environment variable is required if you want your custom 404 to be served (so that it only applies to direct requests). Otherwise, there will likely be a rewrite loop (500 error) when trying to serve the custom error document and you'll see the standard server generated 404 response (but it's still a 404).
answered Jan 30 at 21:12
MrWhiteMrWhite
32k33367
There's no problem having a meta noindex tag on the 404 page, to prevent 200 OK responses being indexed.
If this was a PHP page then you could obviously just send a 404 Not Found header as part of the standard response - to make sure that it always returns a 404.
You could also use .htaccess (mod_rewrite) to force any direct requests to 404.html to also trigger a 404:
RewriteEngine On
RewriteCond %{ENV:REDIRECT_STATUS} ^$
RewriteRule ^404.html$ - [R=404]
The RewriteCond directive that checks the REDIRECT_STATUS environment variable is required if you want your custom 404 to be served (so that it only applies to direct requests). Otherwise, there will likely be a rewrite loop (500 error) when trying to serve the custom error document and you'll see the standard server generated 404 response (but it's still a 404).
answered Jan 30 at 21:12
MrWhiteMrWhite
32k33367
edited Jan 30 at 22:00
answered Jan 30 at 21:12
MrWhiteMrWhite
32k33367
answered Jan 30 at 21:12
MrWhiteMrWhite
32k33367
answered Jan 30 at 21:12
MrWhiteMrWhite
32k33367
32k33367
1
Thank you for the good ideas. For my hosting setup it's easiest to just leave themeta noindexin place, so it's good to know it won't cause any issues if I do.
– Maximillian Laumeister
Jan 30 at 21:59
add a comment |
1
Thank you for the good ideas. For my hosting setup it's easiest to just leave themeta noindexin place, so it's good to know it won't cause any issues if I do.
– Maximillian Laumeister
Jan 30 at 21:59
1
1
Thank you for the good ideas. For my hosting setup it's easiest to just leave the
meta noindex in place, so it's good to know it won't cause any issues if I do.– Maximillian Laumeister
Jan 30 at 21:59
Thank you for the good ideas. For my hosting setup it's easiest to just leave the
meta noindex in place, so it's good to know it won't cause any issues if I do.– Maximillian Laumeister
Jan 30 at 21:59
add a comment |
Thanks for contributing an answer to Webmasters Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fwebmasters.stackexchange.com%2fquestions%2f120620%2fok-to-use-meta-noindex-on-404-pages%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Alternately you could name the file
/404-3pAhD3tM.htmlwhich would be unlikely to ever to get crawled unless you linked to it or had directory indexes on.– Stephen Ostermiller♦
Jan 30 at 22:32
I also suspect that if Google did crawl
/404.htmlit would treat it as a "soft 404" because it presumably has "Page Not Found" in the title and H1.– Stephen Ostermiller♦
Jan 30 at 22:34