was working for a year or something
now it always asks to fill captcha
what i tried:
PS. i did similar scraper in Kantu (alternative version of SELENIUM IDE chrome automation) and it works. the reason why I hesitate using kantu is because creating easy things in it takes a lot of time and headache (multithreading is especially headache), often theres no way to create something that was simple to make in BAS.
PS2. from my understanding, google tends to lock requests that it thinks come from abusive identity (and it can tell if you scrape complex footprints queries many times in a row in short period of time). so either me or someone else triggered BAS to be locked from google scraping. because, i can still browse search result with firefox, with chrome, and with kantu, and probably with zennoposter as well (checked all of these except zennoposter but im sure it works). and with these devices im not prompted to fill captcha on the same IP or same set of proxies.
yet same native IP and proxies as well get instantly triggered for captcha when i scrape with BAS (since yesterday). So despite good IP (that works when use different software) and fresh fingerprint, google can still detect that my BAS instance belongs to some spammer, and shows captcha fill request instantly or after up to 5 searches.
Any help appreciated. it worked fine for up to year till yesterday.
@yourfriendkarol said in my long-time-working google scrapper stopped working yesterday but KANTU works:
was working for a year or something
now it always asks to fill captcha
what i tried:changing ip of my ISP
using proxies
changing user-agent
fingerprints
cookies
removing all footprints and only googling random english words
waiting 24h
combinations of abovePS. i did similar scraper in Kantu (alternative version of SELENIUM IDE chrome automation) and it works. the reason why I hesitate using kantu is because creating easy things in it takes a lot of time and headache (multithreading is especially headache), often theres no way to create something that was simple to make in BAS.
PS2. from my understanding, google tends to lock requests that it thinks come from abusive identity (and it can tell if you scrape complex footprints queries many times in a row in short period of time). so either me or someone else triggered BAS to be locked from google scraping. because, i can still browse search result with firefox, with chrome, and with kantu, and probably with zennoposter as well (checked all of these except zennoposter but im sure it works). and with these devices im not prompted to fill captcha on the same IP or same set of proxies.
yet same native IP and proxies as well get instantly triggered for captcha when i scrape with BAS (since yesterday). So despite good IP (that works when use different software) and fresh fingerprint, google can still detect that my BAS instance belongs to some spammer, and shows captcha fill request instantly or after up to 5 searches.
Any help appreciated. it worked fine for up to year till yesterday.
Most likely just google now defines a fake browser BAS ((
@UserTrue after more tests i think its either anti-scraper google update OR just my footprints were marked, or both.
i will wait one or more days till footprints get unlocked. lets hope they do.
Edit:
after even more tests i think its both footprints and my software identity that are "marked"
the problem is that my queries get asked for captcha instantly after i open BAS, while same query works in my chrome/firefox browsers. it somehow must recognize that its not normal chrome, or something
