Comments on: Creating A PHP Spider/Bot/Crawler

By: Aneesh Dogra

Aneesh Dogra — Sun, 06 Mar 2011 16:57:57 +0000

Nice for beginners …
Need something advanced for me..
Will be waiting for more articles…

By: Peter

Peter — Sun, 07 Nov 2010 15:12:08 +0000

Very good article. It is somehow possible to make shoes at the entrance to the site identified under any name (eg, Google bot will indetifikuje in statistics such as Googlebot). very thanks for advice on how to name the boot. For site owners knew they had been visited just my boot. very thanks for the advice.

By: Steve

Steve — Sun, 19 Sep 2010 06:06:04 +0000

good article. I use this same approach on a project i’m currently working on (http://www.quickscrape.com) and I would recommend using curl. how do you handle recursion?

By: Christopher McCulloh

Christopher McCulloh — Mon, 16 Aug 2010 18:21:36 +0000

So, I posted this article two years ago and I know I eventually figured out how to do it using cURL but I have absolutely no recollection as to how I actually accomplished this. So, @erick, no, sorry, I don’t have any idea… I really wish I had done a follow up post…

By: erick

erick — Thu, 03 Jun 2010 05:20:43 +0000

Hey, I was looking for something similar, but for this delivery company http://www.sendaexpress.com.mx/cotizador.asp to calculate shipping rate, any ideas? ..

Thanks

By: Austin Oblouk

Austin Oblouk — Sun, 08 Nov 2009 23:54:18 +0000

Thank you for this article. I also recommend str_replace characters that could harm your database if you are working with hostile sites and caching info. Also using the php GDLib is very nice when working with cURL, you can really make some nice gui improvements.

By: Web Designing Gurgaon

Web Designing Gurgaon — Mon, 26 Oct 2009 10:20:43 +0000

Thank you for this great article. I was looking for a resource that helps me with spider creation and your article simply has given me a starting point. Thank you again,
Sumit