i was trying earlier today to parse a web page fetched by python3 requests library with beautifulsoup, here is the link: https://azuremlsdktestpypi.azureedge.net/codegen
response = requests.get('https://azuremlsdktestpypi.azureedge.net/codegen')
i tried to get all the links, with:
ph = BeautifulSoup(response.text,"lxml")
ph.findAll('a')
but couldn't get any results, and what confuses me more is when i tried to search for a string in response.text, the string does exist in response.text but the interpreter couldn't find it. I think the problem is with encodings because type(response.text) returned "<type 'unicode'>", but response.encoding returned 'ISO-8859-1'. so I would be grateful if anyone can help me try to figure out how to deal with such responses.
Regards
