web scraping - Clicking link using beautifulsoup in python

Question

In mechanize we click links either by using follow_link or click_link. Is there a similar kind of thing in beautiful soup to click a link on a web page?

Answer 1

BeautifulSoup is an HTML parser.

Further discussion really depends on the concrete situation you are in and the complexity of the particular web page.

If you need to interact with a web-page: submit forms, click buttons, scroll etc - you need to use a tool that utilizes a real browser, like selenium.

In certain situations, for example, if there is no javascript involved in submitting a form, mechanize would also work for you.

And, sometimes you can handle it by simply following the link with urllib2 or requests.