What programming language/server/frameworks would you use for a web crawler if you were bad at functional programming (no python)?
Java with some NIO-capable server
>>55237966
what about node js? doesnt NPM have tons of libraries for web automation?
>>55237750
Why the fuck would you grab a spider like that? A black widow at that
angularjs bro
>>55238041
That works too.
>>55238068
why not? are you some sort of pussy?
>make a service that crawls classifieds sites through TOR in Java
>tfw site owner cucks mad
>tfw can't get blacklisted, ever
>>55238243
OP here. I think I will go with Java too. I am more used to it than JS. Can you recommend some technologies to set up the basic server? It doesnt have to have an UI, but use SQL (no Mongo) and run in parallel on multiple VMs (with same DB), so I think I cant just start everything up with XAMPP.
Jsoup html parser for java.
Thank me later op.
>>55238068
black widows are pretty docile, you can actually handle them without them biting.
>>55238201
You don't need a lot of common sense to just know that spiders need to fucking die and extinct
