Hosting
Images are sometimes not shown due to bandwidth/network limitations. Refreshing the page usually helps.
You are currently reading a thread in /g/ - Technology
You are currently reading a thread in /g/ - Technology
Where do you guys host your non-website software? I would like to get a few VPS to host a web crawler (Java). I check DigitalOcean's ToS and its says every vps with such activity will be deleted.
>>53572192
https://www.feralhosting.com/pricing
>>53572192
OVH
>>53572661
>OVH
3.5Euro for 2GB RAM VPS??? wha
>>53572192
CloudAtCost
>>53573172
>CloudAtCost
This one time payment is fishy as hell.
>>53573522
Works for me.
I host my own server at home, on an old netbook. Works pretty well for my needs.
>>53572192
Why does everyone seem to be writing web crawlers these days? What's your use case for this?
>>53573604
I need to scrape all kinds of product data from their manufacturers. These sites I have to target have really poor performance, so some of them would take ages If I don't want to crash them, so I will go distributed and have a few vps handle the crawling.
>>53573729
Also I need to re-crawl after some time, so It is a continuous process.
>>53572192
Where in the ToS does it say that? I glanced through it and couldn't find anything applicable.
>>53573172
very shady
>>53574039
It works though.
>>53572661
Same here. Since they're my ISP, I got 60€ free to spend on my VPS rent, and a 10TB hubic account. No issues so far with either service.
>>53572361
Kek feralhosting is great, there is always someone seeding shit on public tracker using feralhosting, with speed over 20 MB/s
>>53573871
ToS, 3.3 You shall not: (i) take any action that imposes or may impose... (v) use manual or automated software, devices, or other processes to “crawl” or “spider” any page of the Website;
https://www.digitalocean.com/legal/terms/
>>53576199
Or they meant their own website? Maybe, but I read somewhere that If you allocate too much resource they shut your droplet off without any warning.
>>53572192
I have own servers, using for
> VPS hosting
> VPN
> webhosting
> cloud
> more shits
>>53576236
It is also my concern, that a web crawler allocated more resources (bandwidth, cpu) than a small website you would usually host on a 512-1024MB RAM VPS, you maybe they don't like that.
I host all my stuff on gethub
>>53576333
I also have a dedicated server, but with only one IP address. Some websites block your IP, even If you are being polite, like 50 request in a few minutes (3-10min), so I need more to be effective.
(Googlebot's maximum is 4 request per sec in Google Webmasters tool)
>>53576410
i have only 8 servers so i have one IP and i have hosting via ports...
