Bablosoft

Newbie Question: Correlated Data, Correlated Elements, Combined AddNewRow

Support
  • R

    I have successfully scraped the same site with Ubot Studio and then with Parsehub. I am now trying BAS on the same site to find my favorite program. The site has just two pages, Ajax on both pages. It is for a Boat Reservation site.

    Logic
    Get Element DOCK_LOCATIONS
    For Each Dock Location

    For Each Date from today to 90 Days From Now

    • Create Date From a DateAdd or Add Seconds (BAS uses Add Seconds with a Day Option) using a DaysToAdd variable set to 1
    1. Scrape the Date and Dock Location from Page One
    2. Click the "Next" Button - page loads to Page Two but URL remains the same - no different URL per dock location = dynamic site.
    3. Scrape the list of boat names
    4. Scrape the availability status next to the boat name
    5. Click the (I) button next to the boat name - Ajax modal window comes up with boat details
    6. Scrape the Boat Type, Boat Capacity
    7. Combine Date, Dock Location, Boat Name, Availability, Type, Capacity into a single row
    8. Loop through remaining Boat Names
    9. Increase DaystoAdd variable by one
      Loop Date
      Loop Dock

    I have successfully scraped this site using the other two programs, but I can't figure out how to keep the (I) button and relevant modal windows, and all of the data together to write it to a file or database. There doesn't seem to be a correlation feature or a way to cycle through lists using the same variable, or some better method. This is pretty basic scraping, so I know I am missing something fundamental.

    Can you provide a link to a tutorial that goes into more detail about how to scrape with sub-pages while keeping the data correlated for a final row append to a table? Alternatively, a link to how the database structure in BAS keeps a key that ties Dock Location, Date, Boat Name and (boat details) together?

    This program is great, but there is not a lot out there re: English tutorials that cover detailed scraping that I can find.

    Thank you!

    Rob

    0
  • O

    About Ajax modal window - not sure how to do that.
    About other question (write all data to file or database).

    There are few functions made for that purpose, inside Foreach function there is special function called [[FOREACH_DATA]] (look in the description of the function) and, you can manipuliate it for Filesystem -> Write to File (https://prnt.sc/17z1wsu), you will need to set new resource at the start of the script and use that resource as the file you are writing to, then you can use this file and manipulate it as you wish with variables to save to Database.

    If you are using Multi-threaded script, you will need to use Global Variable so other threads won't override your current Thread.

    0