I think you could use another thread with the browser to accomplish this. And saving the necessary data to a resource / global variable, then using this data in the new thread, whether with the browser or whatever needs to be done.
Error after 100.000 runs
-
I am having an issue with running a script: Input are 500.000 URLs to be scraped.
After ~100k successes the script freezes.
I am running this in run mode. Till then the output (writing json) works fine. The urls are loaded as external source but with the freezing, also the input file is not updated.
Cant provide an error log as the script crashes :(
-
check all resources how much set in field - successful
