Need help scraping a website

Hello. I’m very new to app building, so bare with me. I’m trying to scrape data from the Home Depot website. I’m in one of the categories (Plywood). I’m trying to extract all of the item links on each page. When I start selecting the items, it will auto select only the top half of the items, but not the bottom half. For some reason, it wont select the items underneath the banner that is in the middle of the page. Even if I manually continue selecting the bottom half of the items so that every item on the page is selected, when I go to scrape and export to Excel, it still only shows the top half. I even tried just selecting the bottom half of the page and not the top, but no data shows up in Excel. It’s not recognizing any of the items below that banner for some reason, even if it shows that I selected them.

Below is a screenshot showing the end of the top half of items, the banner in-between, and the start of the bottom half where it’s not auto-selecting. I’ve also included the link I’m working with. Any thoughts on a workaround for this issue? Thanks.

Hey, I have never scraped anything other than with WordPress websites. Though, after a quick Google search I found that Home Depot may have an API ? If so that might be your best option, as this would allow you to grab and search etc. The Home Depot Product API - SerpApi

Apologies I can’t be of more help, I haven’t looked into this before, but I’d have thought API route would be best. Sorry for the limited response.

2 Likes

Hey thanks for the suggestion. Never done anything with API, but I’ll look into it. Thanks :slight_smile:

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.