The standard method of performing multi page scraping in these cases are explained at the following links :-
Pages with 'Load more content' or 'Display more data' link or button
Pages where more data is loaded automatically when you scroll down
In both the above cases, during mining stage, WebHarvy initially tries to load all pages (by scrolling down or by clicking the 'show more data' link repeatedly) before attempting scraping. This works for most web pages, but sometimes when the requested number of pages is very high, some websites stop serving data beyond a limit resulting in incomplete mining. This can be solved as follows.
The first step is to try again after increasing the 'AJAX Load Wait Time' value in Miner Settings. Increasing this value gives the page enough time to load data completely after each scroll or click.
Still, if you are unable to get all data, try the method below which involves manually loading the pages.
1. Load the page in WebHarvy
2. Manually scroll down the page or click the 'show more data' link repeatedly till the required amount of data is displayed
3. Scroll up to the start of the page
4. Start configuration
5. Select required data, When WebHarvy asks whether to generate complete data preview, select YES
6. Do not follow links from the page, this must be handled separately
7. Stop configuration
8. Right click the Preview area and select 'Copy Preview' option.
9. Open any spreadsheet software (Microsoft Excel) and directly paste the data in table format. This method does not involve the mining stage.
In case you need to follow each listing link and get resulting data in Step 6 above, click on the link and select 'Capture Target URL' option to get the URL of the details page. Alternatively, URLs can also be captured from the HTML source of the link as explained here. In Step 8 you will get the entire list of URLs from which you can extract data using a single configuration as explained at https://www.webharvy.com/tour3.html#AddURLs.