Bablosoft

Twitter endless scroll problem. How to fix?

Support

  • S

    Problem with Frames

    Support
    0 Votes
    1 Posts
    403 Views
    No one has replied
  • D

    Twitter Data Scraping Issue: Unable to Retrieve Likes Beyond the 21st User

    Support
    -1 Votes
    8 Posts
    981 Views
    sergerdnS

    @dayyangee said in Twitter Data Scraping Issue: Unable to Retrieve Likes Beyond the 21st User:

    As you can see in this picture, I have added loop and also scrolled, but still, I am getting less data.

    I cannot see any logic in your picture that shows me that you scrolled before that loop to get the new data loaded.

    If you scrolled inside a loop while scraping data and assumed that you would obtain that data within the loop, this logic is incorrect because the HTML DOM has changed, and you need to restart the loop to retrieve the newly loaded data.

    You can not parse all data within one loop. You need to have at minimum 2 loops.

    While true(loop first) <==

    Scrape data with the Start Loop(loop second) from the last point or from the beginning if you are getting it for the first time. Scroll down or click to load new data.<==

    Alternatively, you can follow another approach:

    Scroll down to the end of the page and wait for all the data to load. Scrape the data with the Start Loop
  • E

    how to fix failed to run database

    Support
    0 Votes
    5 Posts
    990 Views
    I

    Спасибо за ссылки на похожую тему, но к сожалению мне не помогло, возможно я, что то делаю не так. Удаление mongod.lock действительно убрало надпись вначале об ошибке, но базы не работают. Переустановка BAS тоже не решила проблему.

    @Fox said in Глюкнула БД прошу подсказать:

    @dobrok said in Глюкнула БД прошу подсказать:

    @UserTrue said in Глюкнула БД прошу подсказать:

    @dobrok я с таким не сталкивался

    Ни одна из БД не работает, даже новую создаю, после перезагрузки не коннект.

    Удалите файл C:\Users%USERNAME%\AppData\Local\BasDatabases\НАЗВАНИЕ БАЗЫ\current\mongod.lock Запустите mongod --dbpath C:/Users/%USERNAME%/AppData/Local/BasDatabases/НАЗВАНИЕ БАЗЫ/ --repair. mongod есть в папке с BAS.

    Данный метод сделал с помощью командной строки, получив вот это:

    PS C:\Users\ilyaa> C:\Users\ilyaa\AppData\Roaming\BrowserAutomationStudio\apps\24.7.1\mongod.exe --dbpath C:/Users/ilyaa/AppData/Local/BasDatabases/Database.nmarket/ --repair
    2022-03-14T17:45:25.775+0300 [initandlisten] MongoDB starting : pid=34756 port=27017 dbpath=C:/Users/ilyaa/AppData/Local/BasDatabases/Database.nmarket 64-bit host=DESKTOP-A70S6NA
    2022-03-14T17:45:25.775+0300 [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2
    2022-03-14T17:45:25.775+0300 [initandlisten] db version v2.6.6
    2022-03-14T17:45:25.775+0300 [initandlisten] git version: 608e8bc319627693b04cc7da29ecc300a5f45a1f
    2022-03-14T17:45:25.775+0300 [initandlisten] build info: windows sys.getwindowsversion(major=6, minor=1, build=7601, platform=2, service_pack='Service Pack 1') BOOST_LIB_VERSION=1_49
    2022-03-14T17:45:25.775+0300 [initandlisten] allocator: system
    2022-03-14T17:45:25.775+0300 [initandlisten] options: { repair: true, storage: { dbPath: "C:/Users/ilyaa/AppData/Local/BasDatabases/Database.nmarket/" } }
    2022-03-14T17:45:25.838+0300 [initandlisten] finished checking dbs
    2022-03-14T17:45:25.838+0300 [initandlisten] dbexit:
    2022-03-14T17:45:25.838+0300 [initandlisten] shutdown: going to close listening sockets...
    2022-03-14T17:45:25.838+0300 [initandlisten] shutdown: going to flush diaglog...
    2022-03-14T17:45:25.838+0300 [initandlisten] shutdown: going to close sockets...
    2022-03-14T17:45:25.838+0300 [initandlisten] shutdown: waiting for fs preallocator...
    2022-03-14T17:45:25.838+0300 [initandlisten] shutdown: closing all files...
    2022-03-14T17:45:25.838+0300 [initandlisten] closeAllFiles() finished
    2022-03-14T17:45:25.838+0300 [initandlisten] shutdown: removing fs lock...
    2022-03-14T17:45:25.838+0300 [initandlisten] dbexit: really exiting now
    PS C:\Users\ilyaa>

    Но базы не заработали. Если же просто запустить отдельно файл mongod.exe, то увидим:

    C:\Users\ilyaa\AppData\Roaming\BrowserAutomationStudio\apps\24.7.1\mongod.exe --help for help and startup options
    2022-03-14T18:02:16.828+0300 [initandlisten] MongoDB starting : pid=39768 port=27017 dbpath=\data\db\ 64-bit host=DESKTOP-A70S6NA
    2022-03-14T18:02:16.843+0300 [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2
    2022-03-14T18:02:16.843+0300 [initandlisten] db version v2.6.6
    2022-03-14T18:02:16.843+0300 [initandlisten] git version: 608e8bc319627693b04cc7da29ecc300a5f45a1f
    2022-03-14T18:02:16.843+0300 [initandlisten] build info: windows sys.getwindowsversion(major=6, minor=1, build=7601, platform=2, service_pack='Service Pack 1') BOOST_LIB_VERSION=1_49
    2022-03-14T18:02:16.854+0300 [initandlisten] allocator: system
    2022-03-14T18:02:16.863+0300 [initandlisten] options: {}
    2022-03-14T18:02:16.863+0300 [initandlisten] exception in initAndListen: 10296

    ERROR: dbpath (\data\db) does not exist.
    Create this directory or give existing directory in --dbpath.
    See http://dochub.mongodb.org/core/startingandstoppingmongo

    , terminating
    2022-03-14T18:02:16.863+0300 [initandlisten] dbexit:
    2022-03-14T18:02:16.863+0300 [initandlisten] shutdown: going to close listening sockets...
    2022-03-14T18:02:16.868+0300 [initandlisten] shutdown: going to flush diaglog...
    2022-03-14T18:02:16.870+0300 [initandlisten] shutdown: going to close sockets...
    2022-03-14T18:02:16.870+0300 [initandlisten] shutdown: waiting for fs preallocator...
    2022-03-14T18:02:16.870+0300 [initandlisten] shutdown: lock for final commit...
    2022-03-14T18:02:16.870+0300 [initandlisten] shutdown: final commit...
    2022-03-14T18:02:16.870+0300 [initandlisten] shutdown: closing all files...
    2022-03-14T18:02:16.870+0300 [initandlisten] closeAllFiles() finished
    2022-03-14T18:02:16.870+0300 [initandlisten] dbexit: really exiting now

    [процесс завершил работу с кодом 100 (0x00000064)]

  • X

    BAS and capmonster IP problem (post)

    Support
    0 Votes
    10 Posts
    2058 Views
    FoxF

    @hellomembersha said in BAS and capmonster IP problem (post):

    @Fox Does the user have to have their host file set for the captcha service they are using? Is there a way to set a npm package in bas itself and not require user to change their host file?

    Why change the host file at all? The module allows you to specify the ip of the service yourself.

  • A

    Unable to scroll in instagram follower frame

    Support
    0 Votes
    8 Posts
    2495 Views
    A

    @andy231 thanks i surely will try this...as always, you are the best andy