romanmir/awesome-nodejs
1
0
Fork 0
mirror of https://github.com/sindresorhus/awesome-nodejs.git synced 2026-01-24 04:18:13 -08:00
Code Issues Packages Projects Releases Wiki Activity

Add simplecrawler (#489)

Browse Source 
Simplecrawler is a simple but powerful event driven web crawler.

Rationale:

* Simplecrawler is well documented and tested.
* Simplecrawler has more than thirty individual contributors and a core team of three.
* Simplecrawler has some notoriety: with nearly 1000 stars on Github it is the most popular still-maintained web crawler library for node.
* Articles and blog entries have been written about simplecrawler, (most notably [this one from WIRED](http://www.wired.com/2015/10/cyphon-wired-archive-migration/)) and it has seen continued use in many large organisations.

Closes #489
This commit is contained in:
Christopher Giffard
2016-04-15 15:33:15 +07:00
committed by Sindre Sorhus
parent 999998b52f
commit 238313fbb1
1 changed files with 1 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
readme.md
View File

@@ -748,6 +748,7 @@
- [json-strictify](https://github.com/pigulla/json-strictify) - Safely serialize a value to JSON without data loss or going into an infinite loop.
- [parent-module](https://github.com/sindresorhus/parent-module) - Get the path of the parent module.
- [resolve-from](https://github.com/sindresorhus/resolve-from) - Resolve the path of a module like `require.resolve()` but from a given path.
- [simplecrawler](https://github.com/cgiffard/node-simplecrawler) - Event driven web crawler.
## Resources