The following tutorial is a simple example of how to create a site crawl robot with php. This can be used for search engines, creating caches of your sites at regular intervals etc. The crawler uses php’s CURL library to fetch the URL contents. It will then look for an anchor tag inside the page […]