Can't place properly (a portion of code written for clicking next page) within my existing script

[shahin]

I've written a script in vba using IE to parse some information connected to some names from a webpage. The names only appears when a particular search is performed. However, my scraper is able to open that webpage, fills in the city field (from the search input), populates the search results by hitting the search button, clicks each results and parse document from a pop up box. That's it. Up to this point my scraper doing exactly I was expecting it to, The problem is the populated results may traverse several pages but my scraper can do this for the first page. I've already defined elements what can lead to the next page but can't think of putting it within my scraper in the right way. Moreover, the way I've written codes for next page can click once, not keep on clicking until the last page. I hope someone will help me as to how i can put this code (written for next page) within my scraper and twitch a little to make the next page code work until the last page. Thanks in advance.


Here is what I've written so far:

Sub Facdl_Data()

    Dim IE As New InternetExplorer, html As HTMLDocument
    Dim posts As Object, post As Object, elem As Object, search_list As Variant, search_item As Variant

    With IE
        .Visible = True
        .navigate "http://www.facdl.org/page/find-a-lawyer"
        Do Until .readyState = READYSTATE_COMPLETE: Loop
        Set html = .document
    End With

    search_list = Array("Naples", "Miami Beach", "Miami", "Ocala", "Orlando")

    For Each search_item In search_list

        For Each posts In html.getElementsByTagName("input"): If InStr(posts.Name, "city") > 0 Then posts.Value = search_item: Exit For:
        Next posts

        For Each post In html.getElementsByTagName("input"): If InStr(post.Value, "Search") > 0 Then post.Click: Exit For:
        Next post

        Application.Wait (Now + TimeValue("0:00:05"))

        For Each elem In html.getElementsByClassName("ajaxBtn")
            elem.Click
            Application.Wait (Now + TimeValue("0:00:02"))

            Row = Row + 1: Cells(Row, 1) = html.getElementById("popclick").getElementsByClassName("h4 modal-title")(0).innerText
            Cells(Row, 2) = html.getElementById("popclick").getElementsByClassName("table table-bordered")(0).getElementsByTagName("td")(1).innerText
            Cells(Row, 3) = html.getElementById("popclick").getElementsByClassName("table table-bordered")(0).getElementsByTagName("td")(3).innerText
            Cells(Row, 4) = html.getElementById("popclick").getElementsByClassName("table table-bordered")(0).getElementsByTagName("td")(5).innerText
            Cells(Row, 5) = html.getElementById("popclick").getElementsByClassName("table table-bordered")(0).getElementsByTagName("td")(7).innerText
        Next elem
    Next search_item

    IE.Quit

End Sub

Codes for clicking on the next page link:

For Each next_page In html.getElementsByClassName("pagination")(0).getElementsByTagName("a"): If InStr(next_page.innerText, ">") > 0 Then next_page.Click: Exit For:
Next next_page