How to Scrape eBay deals using Python and BeautifulSoup

eBay is one of the most popular e-commerce application which allows finding millions of products online. Even though it has millions of products everyone like to buy a product when it has some discount price. Tracking and checking the discount price manually every time is tedious. There should be an easy way to automate this process

In this article we are going to implement small python script to track the price and if the price goes down than the specific price we will trigger an email to alert about the deal.

What is web scraping

Web scraping is a way of extracting the relevant or required information from the website. Web scraping used in a wide range of places like news scraping, price and deal comparison and monitoring, product review scraping, website change detection, whether data scraping and other wide range of areas

Web scraping tools

Wide range of developers uses python for automating web scraping task. There are few libraries in python which allow making this scraping part easier. One is selenium web driver and another one is BeautifulSoup. Selenium used to automate browser activities and if you have done any web automation testing you may have used this. In another side, BeautifulSoup uses to parse the Html or XML document and you can extract those data more easily using this library.

In this one we are going to use BeautifulSoap and scrap the price of the processor in Ebay site.

Application flow

Before start the coding we need to think about what is the flow of the application.

  1. First we need a way to read the url and extract the price of the that page.
  2. Next we need to remove unwanted values like symbols from the price tag
  3. Next check whether that deal is in our expected price range
  4. Then if it below than expected price we can send the email with a message
  5. At last, This process need to be run continuously.

Implementation

First, we import requests and BeautifulSoap library to perform get request and extract Html values. Let’s define a getEbayOffer method which performs get request and parse that HTML content and find the text inside the ‘prcIsum’ id. This id can be found using inspect element. find_all return an array of elements and we just need the text of the first element

    
def getEbayOffer():
    request = requests.get("https://www.ebay.com/itm/AMD-Ryzen-5-3600XT-Unlocked-Processor-w-Wraith-Spire-Cooler-MSI-Gaming-Mouse/293797673558")
    html  = BeautifulSoup(request.content, 'html.parser')
    price = html.find_all("span", {'id':"prcIsum"})[0].text
    

This text contains a dollar symbol along with ‘US’ text which not needs to the comparison. So we can replace those values with empty and parse that values to float convert that to a floating number.

    
price = float(price.replace("US $",""))
    

Let’s assume our expected price is 245 and we check that in the next line using the ‘if’ condition and print that values if the condition met.

    
if price < 229:
        print(tags)
    

Sending Email

After identifying the deal, you could notify that through the email. For that, we can use smtplib in the python to send an email through your email account. For that, you need to pass the username and the password for the email account and specify the from email to email and email message which you needs to send. I get those details from the userlogin.json file and you can add those details directly inside the code. But it good practises to keeping those in the separate file because the username and password are sensitive data.

To send a email like this you must disable 2-step verification in your email and enable Less secure app access. To enable those go to https://myaccount.google.com/u/0/security and select security section from the left navigation and Turn on Less secure app access. This will does not show if your account has enabled 2-step verification.

    
import smtplib
import json

with open('userlogin.json') as f:
  data = json.load(f)

def sendEmail():
    try:
        server = smtplib.SMTP_SSL('smtp.gmail.com', 465)
        server.login(data['userName'], data['password'])
        server.sendmail(data['from'], data['to'], "Your Item price dropped");
        server.close()
        print("Offer email send")

    except Exception as ex:
        print('Something went wrong...')
        print(ex)
    

Finally, we need to run this code continuously and we can use while loop with ‘True’ condition to loop this process. You can sleep this process for 5 seconds after each execution to run this process.

    
while True:
    getEbayOffer()
    sleep(5)
    

Conclusion

you can slightly modify this code to add multiple product url to track different product based on your need

You Will Like Also