The site seems very locked down lol, i guess they really want me to pay $300 for semi high resolution images, so i want to scrape the previews instead. it’s probably some sort of script since with noscript on the site doesn’t even load. It’s even beaten my ‘absolute enable right click’ Extension , and while i can still get the right click going and take a screenshot, i have no option to open the image itself in a new window.
Next up was a simple scraping extension, one i use regularly is webscraper but it’s a huge process to use and can snag super easily, so i tried this one called Download All images
That one didn’t grab anything besides headers and icons and seemed to have gotten me IP banned. Thankfully i have a vpn and they didn’t even revoke my gallery access and i’m back at it again.
I have tried commercial scraping software before but the problem is, afaik these are very big on following robots.txt and that makes a lot of sites unscrapable.
So you’ve all heard my dilemma, and i’m curious, cause at this point it’s a game. How would you all approach this? what software would you use?
Have a look at RobotFramework with the Selenium library. Anything you can manage manually, you can automate repetitively with Robot.
Also, have a look at the F12 Network tab, in case the real images are stored in a predictably named manner.