data get requests from a website with unsupported browser error

i have basically tried everything but still i get the unsupported browser error. i use requests module. i have tried with headers, other modules but still the same.

import requests

url = "https://www.ratemyagent.com.au/"

response = requests.get(url)

html_icerigi = response.text

soup = BeautifulSoup(html_icerigi, "html.parser")

https://www.ratemyagent.com.au/ this is the adres. so please if you have any idea to get rid of this error let me know.

thanks a lot in advance

edit: this is what i have tried as headers :

user_agent = 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36' headers = { 'User-Agent' : user_agent } response = requests.get(url, headers=headers)

versions // python 3. BeautifulSoup4: 4.6.0 requests:2.20
this is the output with soup.text or reponse.text

****STILL COULD NOT FIND THE SOLUTION*** IF HAVE ANY IDEA PLEASE LET ME KNOW

edited Nov 20 '18 at 15:14

asked Nov 20 '18 at 1:12

uestadeus

Can't reproduce error, receiving HTTP 200 from the request

– kstullich
Nov 20 '18 at 1:15

but when you print the text just unsupported browser

– uestadeus
Nov 20 '18 at 1:16

What headers have you tried? Specifically, what User-Agent headers?

– John Gordon
Nov 20 '18 at 1:16

user_agent = 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36' headers = { 'User-Agent' : user_agent } response = requests.get(url, headers=headers)

– uestadeus
Nov 20 '18 at 1:17

this is what i have tried

– uestadeus
Nov 20 '18 at 1:18

|
show 11 more comments

i have basically tried everything but still i get the unsupported browser error. i use requests module. i have tried with headers, other modules but still the same.

import requests

url = "https://www.ratemyagent.com.au/"

response = requests.get(url)

html_icerigi = response.text

soup = BeautifulSoup(html_icerigi, "html.parser")

https://www.ratemyagent.com.au/ this is the adres. so please if you have any idea to get rid of this error let me know.

thanks a lot in advance

edit: this is what i have tried as headers :

versions // python 3. BeautifulSoup4: 4.6.0 requests:2.20
this is the output with soup.text or reponse.text

****STILL COULD NOT FIND THE SOLUTION*** IF HAVE ANY IDEA PLEASE LET ME KNOW

edited Nov 20 '18 at 15:14

asked Nov 20 '18 at 1:12

uestadeus

Can't reproduce error, receiving HTTP 200 from the request

– kstullich
Nov 20 '18 at 1:15

but when you print the text just unsupported browser

– uestadeus
Nov 20 '18 at 1:16

What headers have you tried? Specifically, what User-Agent headers?

– John Gordon
Nov 20 '18 at 1:16

user_agent = 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36' headers = { 'User-Agent' : user_agent } response = requests.get(url, headers=headers)

– uestadeus
Nov 20 '18 at 1:17

this is what i have tried

– uestadeus
Nov 20 '18 at 1:18

|
show 11 more comments

i have basically tried everything but still i get the unsupported browser error. i use requests module. i have tried with headers, other modules but still the same.

import requests

url = "https://www.ratemyagent.com.au/"

response = requests.get(url)

html_icerigi = response.text

soup = BeautifulSoup(html_icerigi, "html.parser")

https://www.ratemyagent.com.au/ this is the adres. so please if you have any idea to get rid of this error let me know.

thanks a lot in advance

edit: this is what i have tried as headers :

versions // python 3. BeautifulSoup4: 4.6.0 requests:2.20
this is the output with soup.text or reponse.text

****STILL COULD NOT FIND THE SOLUTION*** IF HAVE ANY IDEA PLEASE LET ME KNOW

edited Nov 20 '18 at 15:14

asked Nov 20 '18 at 1:12

uestadeus

i have basically tried everything but still i get the unsupported browser error. i use requests module. i have tried with headers, other modules but still the same.

import requests

url = "https://www.ratemyagent.com.au/"

response = requests.get(url)

html_icerigi = response.text

soup = BeautifulSoup(html_icerigi, "html.parser")

https://www.ratemyagent.com.au/ this is the adres. so please if you have any idea to get rid of this error let me know.

thanks a lot in advance

edit: this is what i have tried as headers :

versions // python 3. BeautifulSoup4: 4.6.0 requests:2.20
this is the output with soup.text or reponse.text

****STILL COULD NOT FIND THE SOLUTION*** IF HAVE ANY IDEA PLEASE LET ME KNOW

python web-scraping python-requests

edited Nov 20 '18 at 15:14

asked Nov 20 '18 at 1:12

uestadeus

edited Nov 20 '18 at 15:14

asked Nov 20 '18 at 1:12

uestadeus

edited Nov 20 '18 at 15:14

asked Nov 20 '18 at 1:12

uestadeus

asked Nov 20 '18 at 1:12

uestadeus

asked Nov 20 '18 at 1:12

uestadeus

Can't reproduce error, receiving HTTP 200 from the request

– kstullich
Nov 20 '18 at 1:15

but when you print the text just unsupported browser

– uestadeus
Nov 20 '18 at 1:16

What headers have you tried? Specifically, what User-Agent headers?

– John Gordon
Nov 20 '18 at 1:16

user_agent = 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36' headers = { 'User-Agent' : user_agent } response = requests.get(url, headers=headers)

– uestadeus
Nov 20 '18 at 1:17

this is what i have tried

– uestadeus
Nov 20 '18 at 1:18

|
show 11 more comments

Can't reproduce error, receiving HTTP 200 from the request

– kstullich
Nov 20 '18 at 1:15

but when you print the text just unsupported browser

– uestadeus
Nov 20 '18 at 1:16

What headers have you tried? Specifically, what User-Agent headers?

– John Gordon
Nov 20 '18 at 1:16

user_agent = 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36' headers = { 'User-Agent' : user_agent } response = requests.get(url, headers=headers)

– uestadeus
Nov 20 '18 at 1:17

this is what i have tried

– uestadeus
Nov 20 '18 at 1:18

Can't reproduce error, receiving HTTP 200 from the request

– kstullich
Nov 20 '18 at 1:15

but when you print the text just unsupported browser

– uestadeus
Nov 20 '18 at 1:16

What headers have you tried? Specifically, what User-Agent headers?

– John Gordon
Nov 20 '18 at 1:16

this is what i have tried

– uestadeus
Nov 20 '18 at 1:18

|
show 11 more comments

1 Answer
1

active

oldest

votes

it's `not a error`

when you normally search `https://www.ratemyagent.com.au/` by `chrome browser`, the page resource your get also has the info like you said:

enter image description here

while the resource your get by requests module is same as what you get from a web browser I don't think it is a error.

edited Jan 11 at 9:18

answered Jan 11 at 8:57

jia Jimmy

564

New contributor

add a comment |

Your Answer

StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53384855%2fdata-get-requests-from-a-website-with-unsupported-browser-error%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

1 Answer
1

active

oldest

votes

1 Answer
1

active

oldest

votes

it's `not a error`

when you normally search `https://www.ratemyagent.com.au/` by `chrome browser`, the page resource your get also has the info like you said:

enter image description here

while the resource your get by requests module is same as what you get from a web browser I don't think it is a error.

edited Jan 11 at 9:18

answered Jan 11 at 8:57

jia Jimmy

564

New contributor

add a comment |

it's `not a error`

when you normally search `https://www.ratemyagent.com.au/` by `chrome browser`, the page resource your get also has the info like you said:

enter image description here

while the resource your get by requests module is same as what you get from a web browser I don't think it is a error.

edited Jan 11 at 9:18

answered Jan 11 at 8:57

jia Jimmy

564

New contributor

add a comment |

it's `not a error`

when you normally search `https://www.ratemyagent.com.au/` by `chrome browser`, the page resource your get also has the info like you said:

enter image description here

while the resource your get by requests module is same as what you get from a web browser I don't think it is a error.

edited Jan 11 at 9:18

answered Jan 11 at 8:57

jia Jimmy

564

New contributor

it's `not a error`

when you normally search `https://www.ratemyagent.com.au/` by `chrome browser`, the page resource your get also has the info like you said:

enter image description here

while the resource your get by requests module is same as what you get from a web browser I don't think it is a error.

edited Jan 11 at 9:18

answered Jan 11 at 8:57

jia Jimmy

564

New contributor

edited Jan 11 at 9:18

answered Jan 11 at 8:57

jia Jimmy

564

New contributor

answered Jan 11 at 8:57

jia Jimmy

564

answered Jan 11 at 8:57

jia Jimmy

564

New contributor

jia Jimmy is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

add a comment |

draft saved

draft discarded

Thanks for contributing an answer to Stack Overflow!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

Search This Blog

Ufyukyu