Bulk-Testing Structured Data Markup Using Screaming Frog
I published a post yesterday on Moz.com about how to use Screaming Frog and Google’s Structured Data Testing Tool together. The concept allows us to test large sets of webpages for structured markup, such as the following:
- E-commerce product reviews and pricing
- Google+ Authorship
- Rel Publisher
- Event Listings
- Review and price markup on App Listings
- Music Snippets
- Recipes
- Business Reviews
- Just about anything referencing schema.org, data-vocabulary.org, and similar markup.
You can read the full post here: Quick Guide to Scaling Your Authorship Testing with Screaming Frog
Here’s the YouTube video that describes the process:
Adam
I’ve set up a similar thing on SF but sadly Google is very good at spotting bot usage and, even when crawling as slowly as possible, is throwing captchas at me after about 7 – 8 URLs.
The only way around this I can find it to change IP address using a VPN each time the crawler is ‘blocked’, which makes this a manual process that you need to watch over still.. I’m not done yet!
Kane Jamison
You’ll need to use a proxy service to fix this – probably one with built in rotation. It will be much easier than manually switching VPNs. Search “proxies screaming frog” and you should find a few services.