Comment Monetize? - How about pay for bandwidth (Score 1) 57
From personal experience, bots are repeatedly interrogating repositories, websites, and even bugzilla bug logs and collecting the data. This is tripling the volume of data moving a month. So much so it is causing denial of service failures for smaller web-servers and sites.
Further, many cloud hosted sites are charged by the gigabyte of outbound data usage. It is killing small projects with limited budgets.
And yes, they are ignoring the robots.txt file.
Further, many cloud hosted sites are charged by the gigabyte of outbound data usage. It is killing small projects with limited budgets.
And yes, they are ignoring the robots.txt file.