From 238313fbb1503cb8e248c05ebd1401d25034221f Mon Sep 17 00:00:00 2001 From: Christopher Giffard Date: Fri, 15 Apr 2016 15:33:15 +0700 Subject: [PATCH] Add simplecrawler (#489) Simplecrawler is a simple but powerful event driven web crawler. Rationale: * Simplecrawler is well documented and tested. * Simplecrawler has more than thirty individual contributors and a core team of three. * Simplecrawler has some notoriety: with nearly 1000 stars on Github it is the most popular still-maintained web crawler library for node. * Articles and blog entries have been written about simplecrawler, (most notably [this one from WIRED](http://www.wired.com/2015/10/cyphon-wired-archive-migration/)) and it has seen continued use in many large organisations. Closes #489 --- readme.md | 1 + 1 file changed, 1 insertion(+) diff --git a/readme.md b/readme.md index 3ea4905..818bd1e 100644 --- a/readme.md +++ b/readme.md @@ -748,6 +748,7 @@ - [json-strictify](https://github.com/pigulla/json-strictify) - Safely serialize a value to JSON without data loss or going into an infinite loop. - [parent-module](https://github.com/sindresorhus/parent-module) - Get the path of the parent module. - [resolve-from](https://github.com/sindresorhus/resolve-from) - Resolve the path of a module like `require.resolve()` but from a given path. +- [simplecrawler](https://github.com/cgiffard/node-simplecrawler) - Event driven web crawler. ## Resources