I am new to web scraping, using beautiful soup and selenium. I am trying to scrape the following webpage
'''
https://www.amazon.in/gp/product/B08GG7W329/ref=ewc_pr_img_3?smid=A14CZOWI0VEHLG&psc=1
'''
I am trying to scrape the title and price of the product using this code.
When I used this code to scrape the title on first run it returend the title correctly
when I run the code again it return NONE object.
Why does this happen?
Below is the code I used.
''' python
from selenium import webdriver
from bs4 import BeautifulSoup
url = 'https://www.amazon.in/gp/product/B08GG7W329/ref=ewc_pr_img_3?smid=A14CZOWI0VEHLG&psc=1'
driver = webdriver.PhantomJS(executable_path=r'C:phantomjs-2.1.1-windowsphantomjs-2.1.1-
windowsinphantomjs.exe')
driver.get(url)
page = driver.page_source
soup = BeautifulSoup(page, 'lxml')
title = soup.find('span', id = 'productTitle')
print(title.text.strip())
price = soup.find('tr', id = 'priceblock_ourprice_row').find('span', id = 'priceblock_ourprice').text
print(price)
converted_price = price[0:5]
'''
This is the HTML code for title part:
''' html
<div class="a-section a-spacing-none" id="titleSection">
<h1 class="a-size-large a-spacing-none" id="title">
<span class="a-size-large product-title-word-break" id="productTitle">
Lenovo Legion 5 AMD Ryzen 7 4800H 15.6 inch Full HD Gaming Laptop (8GB/1TB HDD + 256GB
SSD/Windows 10/120 Hz/NVIDIA GTX 1650Ti 4GB GDDR6 Graphics/Phantom Black/2.3Kg), 82B500BMIN
</span>
</h1>
<div class="a-section a-spacing-none expand aok-hidden" id="expandTitleToggle">
</div>
</div>
'''
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…