Using Amazon’s Web Services for Spidering the Web

Courtesy Ruby Inside  Wed, 02/13/2008 - 05:17

aws.png

Robert Dempsey has written a code-packed article for Amazon Web Services’ “Developer Connection ” site called Using Amazon S3, EC2, SQS, Lucene and Ruby for Web Spidering .

It’s a bit of an epic and covers using a multitude of Amazon Web Services together (namely the S3 storage system, the EC2 “Elastic Compute Cloud”, and the Simple Queue Service), with Ruby acting as the glue that holds them all together.

This could be of great interest...


 

More related items

Amazon Silk revisited: Is the “split” cloud...
When the Kindle Fire came out last month, one of the biggest disappointments users had was that Amazon’s promise of “ultra-fast web browsing” seemed to be all talk. In our original...

EUCALYPTUS: a Tree Growing in the Cloud
From the Linux (and Linux Journal) perspective, there's an issue with clouds—those back-end Web services that compose Utility Computing. They're proprietary. Amazon owns AWS (Amazon Web...

Amazon Slashes Cloud Pricing for Those Who Commit
Amazon Web Services has added a new option for its EC2 hosted cloud service that allows customers to sign long-term commitments in exchange for substantial discounts over its exiting offering.


 

Post new comment

The content of this field is kept private and will not be shown publicly.
computer-internet.marc8.com