@DuckDuck Thank you very much. Besides blocking libraries, what else can be done to optimize BAS and reduce RAM and CPU usage? Because the tool I'm working with is expected to simultaneously scrape 20-30 news pages.
Getting detected as bot when you even loading page
-
Hello , i had a crawler bot which was working pretty well and now website updated their bot security, its weird how they detect it.
When i try to load the url , its giving error with datamatrix which says unsual connection.
So i ofc tried fingerprint switcher didnt matter, changed ip adresses didnt matter.
I load the main page surfed a little bit and then go to a url by clicking from main page, didnt matter.When i try it on my normal browser incognito mode its loading the page successfully.
Bad it does not really load the page for besides mail url's.
this is the failed page.
i am open to any suggestions.