G2

Jaydip L.

Senior Software Engineer

Small-Business (50 or fewer emp.)

9/2/2020

"Very efficient, faster and open source tool for crawler"

4.5/5

What do you like best about Apache Nutch?

Open Source

Scalable

Parsing and indexing techniques.

Easy Integration with elastic search and solr.

Different plugins to parse various content types. Review collected by and hosted on G2.com.

What do you dislike about Apache Nutch?

Nothing much in my list of dislike because we really enjoyed it very much and it fulfilled our organization needs. But based on experience I can say some cons like it requires good infrastructure in place and consumes good amount of memory and cpu utilization. We also feel if nutch provide good dashboard and kind of admin panel then it would have very helpful to us. Review collected by and hosted on G2.com.

Recommendations to others considering Apache Nutch:

When we had requirement for crawling we went with different tools like StormCrawler, scrapy etc. But we found this tool as very much reliable and most importantly open source. It's various features like automatic crawling, finding inner links to crawl, parse different kind of contents, various integrations etc. made us to go for this tool and believe me we never felt regret after using it. Best crawling tool. Review collected by and hosted on G2.com.

What problems is Apache Nutch solving and how is that benefiting you?

Our business need is to develop search engine where we provide list of URLs to nutch and it will crawl all those URLs as well as find its inner URL and crawl them as well. We were storing these crawled data to cassandra db and then there was elastic search in place to fulfill our search query. These was actually working perfectly and nutch really helped us to provide crawling with their abilities to parse different content types and store them. Review collected by and hosted on G2.com.