Recommendations to others considering python celery:
The biggest advice that I'd like to give to other developers using Celery is to not compromise with recommended coding practices. Celery's coding implementation is basically task based. You assign each function that you've written to a particular task queue. There are high chances of code redundancy if a developer is not careful enough to modularise their code.
Moreover, wisely assign the number of workers for each task queue. You would not want to assign large number of workers to a task which is not invoked very frequently, as it would only lead to unnecessary wastage of processing. Review collected by and hosted on G2.com.
What problems is python celery solving and how is that benefiting you?
We are trying to collect publicly available tweets posted by various eminent personalities of several engineering schools, and understand the intersection in the topics for which the academia is collectively interested to solve. With Celery, we are able to collect massive amounts of data in lesser time duration, process it simultaneously and store the results in a NoSQL database. Review collected by and hosted on G2.com.