Get data from massive web crawling system

Amazon owns Alexa, and today Alexa is announcing a web service that allows developers to make special requests for content during Alexa’s web crawls. The Wall Street Journal article gives the example of asking the crawler to help you identify new music files or images that have recently been posted to the web.

I’m interested in using the Alexa service to identify and track the number of links coming in to sites I am promoting (or competing with). The service is free for 10,000 requests a month and then there is a charge of $0.15 for each additional 1,000 requests, according to

Last month I met the evangelist from Amazon’s six other web services at a BYU event, including the amazing Mechanical Turk.

It’s becoming more difficult to keep track of all the web services and APIs coming out from the major internet companies.

Who knows the best blog or web site that tracks all of these?

429 total views, 1 views today


  1. jordy

    Here’s a wiki page that tries to do that:

    I bookmarked it some time ago, and it looks like it has increased in both volume of material and pagerank since the last time I gave it a visit. I hope that’s what you were looking for.

  2. » Alexa Web Search Platform - It’s a Problem - R

    […] Other Thoughts If Everyone Had A Foundry – Seth Godin Get data from massive web crawling system – Paul Allen Alexa Allows Personal Search Development Alexa’s Crawler is Yours – Marc Eisenstadt Alexa’s search platform – Rajesh Jain […]

  3. Russell Page

    Alexa Web Search Platform – It’s a Problem

    There’s just one problem with the new Alexa Web Search Platform, which was recently introduced. Just one. Alexa charges money for it. It’s cheap. $1. In fact, that’s really cheap. The only reason it could be a problem is because of a…

Leave a Reply

Your email address will not be published. Required fields are marked *