• Hi All

    Please note that at the Chandoo.org Forums there is Zero Tolerance to Spam

    Post Spam and you Will Be Deleted as a User

    Hui...

  • When starting a new post, to receive a quicker and more targeted answer, Please include a sample file in the initial post.

Navigate between pages using option element

Using "GET" request you can do that in far more easier way:
Code:
Sub central_bank()

    Dim http As New XMLHTTP60, html As New HTMLDocument
    Dim tbl As Object, tRow As Object, tCel As Object
    Dim ArgumentStr As String

    ArgumentStr = "searchEntity=FundServiceProvider&searchType=Name&searchText=&registers=6%2c29%2c44%2c45"

    With http
        .Open "GET", "http://registers.centralbank.ie/FundSearchResultsPage.aspx?" & ArgumentStr, False
        .setRequestHeader "Content-type", "application/x-www-form-urlencoded"
        .send
        html.body.innerHTML = .responseText
    End With

    Set tbl = html.getElementsByTagName("table")(1)

    For Each tRow In tbl.getElementsByTagName("tr")
        For Each tCel In tRow.getElementsByTagName("td")
            If Not InStr(tCel.innerText, "Page") > 0 Then
                c = c + 1: Cells(x + 1, c) = tCel.innerText
            End If
        Next tCel
        c = 0
        x = x + 1
    Next tRow

End Sub

Same results!!! Ain't it?
 
Perhaps your problem is solved. I've found the pagination control. Give you an update in a while.
 
That's wonderful. This is the most important part and please tell me how you discover it to learn this technique
 
Your issue is clearly solved. Here is the code:
Code:
Sub central_bank()

    Dim http As New XMLHTTP60, html As New HTMLDocument
    Dim tbl As Object, tRow As Object, tCel As Object
    Dim ArgumentStr As String
   
    For y = 1 To 10  ''You know this is where you put the highest number of page you wanna crawl
        ArgumentStr = "__EVENTTARGET=ctl00%24cphRegistersMasterPage%24gvwSearchResults%24ctl18%24ddlPages&__EVENTARGUMENT=&__LASTFOCUS=&__VIEWSTATE=%2FwEPDwUKLTEwMjc3MzEyMw9kFgJmD2QWAgIDD2QWBAIRDw8WAh4EVGV4dAUHR2FlaWxnZRYIHgtvbm1vdXNlb3ZlcgUzdGhpcy5zdHlsZS5jb2xvcj0nIzk3MjAwYSc7dGhpcy5zdHlsZS5jdXJzb3I9J2hhbmQnHgdvbmZvY3VzBTB0aGlzLnN0eWxlLmNvbG9yPScjMDAwJzt0aGlzLnN0eWxlLmN1cnNvcj0naGFuZCceCm9ubW91c2VvdXQFF3RoaXMuc3R5bGUuY29sb3I9JyMwMDAnHgZvbmJsdXIFF3RoaXMuc3R5bGUuY29sb3I9JyMwMDAnZAITD2QWAgIZDxQrAA0PFggeE0F1dG9HZW5lcmF0ZUNvbHVtbnNoHgtfI" & _
        "URhdGFCb3VuZGceC18hSXRlbUNvdW50AtMGHgxBbGxvd1NvcnRpbmdnFgIeB3N1bW1hcnkFc0Rpc3BsYXlzIGEgbGlzdCBvZiBmdW5kcyBhbmQgbGlua3MgZWFjaCBmdW5kcyBkZXRhaWxzIGFuZCB0aGUgdHJ1c3RlZSBhbmQgbWFuYWdlbWVudCBjb21wYW5pZXMgZmlybSBhdXRob3Jpc2F0aW9ucy4QFgNmAgECAhYDFCsABRYKHghSZWFkT25seWceCkhlYWRlclRleHQFB1JlZiBOby4eCURhdGFGaWVsZAUPUmVmZXJlbmNlTnVtYmVyHg5Tb3J0RXhwcmVzc2lvbgUPUmVmZXJlbmNlTnVtYmVyHgpGb290ZXJUZXh0BQNBU0MWBB4IQ3NzQ2xhc3MFCWd2d0NvbHVtbh4EXyFTQgICZGRkFCsABRYMHg1EYXRhVGV4dEZpZWxkBQROYW1lHwsFCUZpcm0gTmFtZR4VRGF0YU5hdmlnYXRlVXJsR" & _
        "mllbGRzFQEPUmVmZXJlbmNlTnVtYmVyHhtEYXRhTmF2aWdhdGVVcmxGb3JtYXRTdHJpbmcFPUZpcm1EYXRhUGFnZS5hc3B4P2Zpcm1SZWZlcmVuY2VOdW1iZXI9ezB9JnJlZ2lzdGVyPTYsMjksNDQsNDUfDQUETmFtZR8OBQNBU0MWBB8PBRBlbnRpdHlOYW1lQ29sdW1uHxACAmRkZBQrAAUWCh8KZx8LBQxUcmFkaW5nIE5hbWUfDAULVHJhZGluZ05hbWUfDQULVHJhZGluZ05hbWUfDgUDQVNDFgQfDwUXZW50aXR5VHJhZGluZ05hbWVDb2x1bW4fEAICZGRkFgNmAgRmFgQfDwUSc2VhcmNocmVzdWx0c3BhZ2VyHxACAhYGHw8FE3NlYXJjaHJlc3VsdHNoZWFkZXIeBFdyYXBoHxACgoAQZGRkZGQWDB4IUG9zaXRpb24LKidTeXN0ZW0uV2ViLlVJLldlYkNvbnRyb2xzLlBhZ2VyUG9zaXRpb" & _
        "24AHglQYWdlck1vZGULKiZTeXN0ZW0uV2ViLlVJLldlYkNvbnRyb2xzLlBhZ2VyQnV0dG9ucwIeDE5leHRQYWdlVGV4dAUETmV4dB4QUHJldmlvdXNQYWdlVGV4dAUIUHJldmlvdXMeDUZpcnN0UGFnZVRleHQFBUZpcnN0HgxMYXN0UGFnZVRleHQFBExhc3RkFgYeCUdyaWRMaW5lcwsqI1N5c3RlbS5XZWIuVUkuV2ViQ29udHJvbHMuR3JpZExpbmVzAR8PBQ1zZWFyY2hyZXN1bHRzHxACgoAgZBYCZg9kFiICAQ9kFgZmDw8WAh8ABQdDMTMyNDUyZGQCAQ9kFg" & _
        "JmDw8WBB8ABShBdXJ1bSBJc2lzIEluc3RpdHV0aW9uYWwgRG9sbGFyIEZ1bmQgTHRkHgtOYXZpZ2F0ZVVybAVBRmlybURhdGFQYWdlLmFzcHg%2FZmlybVJlZmVyZW5jZU51bWJlcj1DMTMyNDUyJnJlZ2lzdGVyPTYsMjksNDQsNDVkZAICDw8WAh8ABQYmbmJzcDtkZAICD2QWBmYPDxYCHwAFB0MxMzI0NTNkZAIBD2QWAmYPDxYEHwAFJkF1cnVtIElzaXMgSW5zdGl0dXRpb25hbCBFdXJvIEZ1bmQgTHRkHxwFQUZpcm1EYXRhUGFnZS5hc3B4P2Zpcm1SZWZlcmVuY2VOdW1iZXI9QzEzMjQ1MyZyZWdpc3Rlcj02LDI5LDQ0LDQ1ZGQCAg8PFgIfAAUGJm5ic3A7ZGQCAw9kFgZmDw8WAh8ABQdDMTMyNDU0ZGQCAQ9kFgJmDw8WBB8ABSpBdXJ1bSBJc2lzIEluc3RpdHV0aW9uYWwg" & _
        "U3RlcmxpbmcgRnVuZCBMdGQfHAVBRmlybURhdGFQYWdlLmFzcHg%2FZmlybVJlZmVyZW5jZU51bWJlcj1DMTMyNDU0JnJlZ2lzdGVyPTYsMjksNDQsNDVkZAICDw8WAh8ABQYmbmJzcDtkZAIED2QWBmYPDxYCHwAFB0MxMzI0NTdkZAIBD2QWAmYPDxYEHwAFHEF1cnVtIElzaXMgU3RlcmxpbmcgRnVuZCBMdGQfHAVBRmlybURhdGFQYWdlLmFzcHg%2FZmlybVJlZmVyZW5jZU51bWJlcj1DMTMyNDU3JnJlZ2lzdGVyPTYsMjksNDQsNDVkZAICDw8WAh8ABQYmbmJzcDtkZAIFD2QWBmYPDxYCHwAFB0MxMzI0NThkZAIBD2QWAmYPDxYEHwAFH0F1cnVtIElzaXMgU3dpc3MgRnJhbmMgRnVuZCBMdGQfHAVBRmlybURhdGFQYWdlLmFzcHg%2FZmlybVJlZmVyZW5jZU51bWJlcj1DMTMyNDU4" & _
        "JnJlZ2lzdGVyPTYsMjksNDQsNDVkZAICDw8WAh8ABQYmbmJzcDtkZAIGD2QWBmYPDxYCHwAFB0MxMzI0NTlkZAIBD2QWAmYPDxYEHwAFJEF1cnVtIE11bHRpIFN0cmF0ZWd5IERvbGxhciBGdW5kIEx0ZB8cBUFGaXJtRGF0YVBhZ2UuYXNweD9maXJtUmVmZXJlbmNlTnVtYmVyPUMxMzI0NTkmcmVnaXN0ZXI9NiwyOSw0NCw0NWRkAgIPDxYCHwAFBiZuYnNwO2RkAgcPZBYGZg8PFgIfAAUHQzEzMjQ2MGRkAgEPZBYCZg8PFgQfAAUiQXVydW0gTXVsdGkgU3RyYXRlZ3kgRXVybyBGdW5kIEx0ZB8cBUFGaXJtRGF0YVBhZ2UuYXNweD9maXJtUmVmZXJlbmNlTnVtYmVyPUMxMzI0NjAmcmVnaXN0ZXI9NiwyOSw0NCw0NWRkAgIPDxYCHwAFBiZuYnNwO2RkAggPZBYGZg8PFgIfAAUHQzEzMj" & _
        "Q2M2RkAgEPZBYCZg8PFgQfAAUmQXVydW0gTXVsdGkgU3RyYXRlZ3kgU3RlcmxpbmcgRnVuZCBMdGQfHAVBRmlybURhdGFQYWdlLmFzcHg%2FZmlybVJlZmVyZW5jZU51bWJlcj1DMTMyNDYzJnJlZ2lzdGVyPTYsMjksNDQsNDVkZAICDw8WAh8ABQYmbmJzcDtkZAIJD2QWBmYPDxYCHwAFB0MxMzI0NjRkZAIBD2QWAmYPDxYEHwAFI0F1cnVtIFN5bmNocm9uaWNpdHkgRG9sbGFyIEZ1bmQgTHRkHxwFQUZpcm1EYXRhUGFnZS5hc3B4P2Zpcm1SZWZlcmVuY2VOdW1iZXI9QzEzM" & _
        "jQ2NCZyZWdpc3Rlcj02LDI5LDQ0LDQ1ZGQCAg8PFgIfAAUGJm5ic3A7ZGQCCg9kFgZmDw8WAh8ABQdDMTMyNDY1ZGQCAQ9kFgJmDw8WBB8ABSFBdXJ1bSBTeW5jaHJvbmljaXR5IEV1cm8gRnVuZCBMdGQfHAVBRmlybURhdGFQYWdlLmFzcHg%2FZmlybVJlZmVyZW5jZU51bWJlcj1DMTMyNDY1JnJlZ2lzdGVyPTYsMjksNDQsNDVkZAICDw8WAh8ABQYmbmJzcDtkZAILD2QWBmYPDxYCHwAFB0MxMzI0NjZkZAIBD2QWAmYPDxYEHwAFJUF1cnVtIFN5bmNocm9uaWNpdHkgU3RlcmxpbmcgRnVuZCBMdGQfHAVBRmlybURhdGFQYWdlLmFzcHg%2FZmlybVJlZmVyZW5jZU51bWJlcj1DMTMyNDY2JnJlZ2lzdGVyPTYsMjksNDQsNDVkZAICDw8WAh8ABQYmbmJzcDtkZAIMD2QWBmYPDxYCHwAF" & _
        "B0MxMzQ0NjBkZAIBD2QWAmYPDxYEHwAFJUF2ZW50aWN1bSBDYXBpdGFsIE1hbmFnZW1lbnQgKFVLKSBMTFAfHAVBRmlybURhdGFQYWdlLmFzcHg%2FZmlybVJlZmVyZW5jZU51bWJlcj1DMTM0NDYwJnJlZ2lzdGVyPTYsMjksNDQsNDVkZAICDw8WAh8ABQYmbmJzcDtkZAIND2QWBmYPDxYCHwAFB0MxMzI5MDBkZAIBD2QWAmYPDxYEHwAFNUF2aXZhIEludmVzdG9ycyBKZXJzZXkgVW5pdCBUcnVzdHMgTWFuYWdlbWVudCBMaW1pdGVkHxwFQUZpcm1EYXRhUGFnZS5hc3B4P2Zpcm1SZWZlcmVuY2VOdW1iZXI9QzEzMjkwMCZyZWdpc3Rlcj02LDI5LDQ0LDQ1ZGQCAg8PFgIfAAUGJm5ic3A7ZGQCDg9kFgZmDw8WAh8ABQdDMTM3MDMyZGQCAQ9kFgJmDw8WBB8ABR1Bdml2YSBJbnZlc3R" & _
        "vcnMgTHV4ZW1ib3VyZyBTQR8cBUFGaXJtRGF0YVBhZ2UuYXNweD9maXJtUmVmZXJlbmNlTnVtYmVyPUMxMzcwMzImcmVnaXN0ZXI9NiwyOSw0NCw0NWRkAgIPDxYCHwAFBiZuYnNwO2RkAg8PZBYGZg8PFgIfAAUGQzI4MTc0ZGQCAQ9kFgJmDw8WBB8ABSdBWEEgSW52ZXN0bWVudCBNYW5hZ2VycyBJcmVsYW5kIExpbWl0ZWQfHAVARmlybURhdGFQYWdlLmFzcHg%2FZmlybVJlZmVyZW5jZU51bWJlcj1DMjgxNzQmcmVnaXN0ZXI9NiwyOSw0NCw0NWRkAgIPDxYCHwAFBiZuYnNwO2RkAhAPDxYCHgdWaXNpYmxlaGRkAhEPZBYCZg9kFgQCBQ8QDxYCHwZnZA8WOWYCAQICAgMCBAIFAgYCBwIIAgkCCgILAgwCDQIOAg8CEAIRAhICEwIUAhUCFgIXAhgCGQIaAhsCHAIdAh4CHwIgAiECIgIjAi" & _
        "QCJQImAicCKAIpAioCKwIsAi0CLgIvAjACMQIyAjMCNAI1AjYCNwI4FjkQBQExBQExZxAFATIFATJnEAUBMwUBM2cQBQE0BQE0ZxAFATUFATVnEAUBNgUBNmcQBQE3BQE3ZxAFATgFAThnEAUBOQUBOWcQBQIxMAUCMTBnEAUCMTEFAjExZxAFAjEyBQIxMmcQBQIxMwUCMTNnEAUCMTQFAjE0ZxAFAjE1BQIxNWcQBQIxNgUCMTZnEAUCMTcFAjE3ZxAFAjE4BQIxOGcQBQIxOQUCMTlnEAUCMjAFAjIwZxAFAjIxBQIyMWcQBQIyMgUCMjJnEAUCMjMFAjIzZxAFAjI0BQ" & _
        "IyNGcQBQIyNQUCMjVnEAUCMjYFAjI2ZxAFAjI3BQIyN2cQBQIyOAUCMjhnEAUCMjkFAjI5ZxAFAjMwBQIzMGcQBQIzMQUCMzFnEAUCMzIFAjMyZxAFAjMzBQIzM2cQBQIzNAUCMzRnEAUCMzUFAjM1ZxAFAjM2BQIzNmcQBQIzNwUCMzdnEAUCMzgFAjM4ZxAFAjM5BQIzOWcQBQI0MAUCNDBnEAUCNDEFAjQxZxAFAjQyBQI0MmcQBQI0MwUCNDNnEAUCNDQFAjQ0ZxAFAjQ1BQI0NWcQBQI0NgUCNDZnEAUCNDcFAjQ3ZxAFAjQ4BQI0OGcQBQI0OQUCNDlnEAUCNTAFAjUwZxAFAj" & _
        "UxBQI1MWcQBQI1MgUCNTJnEAUCNTMFAjUzZxAFAjU0BQI1NGcQBQI1NQUCNTVnEAUCNTYFAjU2ZxAFAjU3BQI1N2cWAQIHZAIHDw8WAh8ABQI1N2RkGAIFHl9fQ29udHJvbHNSZXF1aXJlUG9zdEJhY2tLZXlfXxYEBTxjdGwwMCRjcGhSZWdpc3RlcnNNYXN0ZXJQYWdlJGd2d1NlYXJjaFJlc3VsdHMkY3RsMTgkYnRuRmlyc3QFO2N0bDAwJGNwaFJlZ2lzdGVyc01hc3RlclBhZ2UkZ3Z3U2VhcmNoUmVzdWx0cyRjdGwxOCRidG5QcmV2BTtjdGwwMCRjcGhSZWdpc3RlcnNNYXN0ZXJQYWdlJ" & _
        "Gd2d1NlYXJjaFJlc3VsdHMkY3RsMTgkYnRuTmV4dAU7Y3RsMDAkY3BoUmVnaXN0ZXJzTWFzdGVyUGFnZSRndndTZWFyY2hSZXN1bHRzJGN0bDE4JGJ0bkxhc3QFLWN0bDAwJGNwaFJlZ2lzdGVyc01hc3RlclBhZ2UkZ3Z3U2VhcmNoUmVzdWx0cw88KwAKAgICBwgCOWS1%2B1hxQDCzN1iO5VVM1R1LDsmTZw%3D%3D&__SCROLLPOSITIONX=0&__SCROLLPOSITIONY=687&__EVENTVALIDATION=%2FwEWQwKm9vbmBQKdmL%2BNBQLBvq75AwKEl773CgL%2B4cLqBQLR7%2FSRCgKrq7r4AgK" & _
        "nifiNCgKo5tLjBgKp5tLjBgKq5tLjBgKr5tLjBgKs5tLjBgKt5tLjBgKu5tLjBgK%2F5tLjBgKw5tLjBgKo5pLgBgKo5p7gBgKo5prgBgKo5qbgBgKo5qLgBgKo5q7gBgKo5qrgBgKo5rbgBgKo5vLjBgKo5v7jBgKp5pLgBgKp5p7gBgKp5prgBgKp5qbgBgKp5qLgBgKp5q7gBgKp5qrgBgKp5rbgBgKp5vLjBgKp5v7jBgKq5pLgBgKq5p7gBgKq5prgBgKq5qbgBgKq5qLgBgKq5q7gBgKq5qrgBgKq5rbgBgKq5vLjBgKq5v7jBgKr5pLgBgKr5p7gBgKr5prgBgKr5qbgBgKr5qLgBgKr5q7gBgKr5qr" & _
        "gBgKr5rbgBgKr5vLjBgKr5v7jBgKs5pLgBgKs5p7gBgKs5prgBgKs5qbgBgKs5qLgBgKs5q7gBgKs5qrgBgKs5rbgBgLg2MrQBwLk2LbQCK%2BoTEtn68HTsqsApIuECE2GK8p8&ctl00%24cphRegistersMasterPage%24gvwSearchResults%24ctl18%24ddlPages=" & y
   
        With http
            .Open "POST", "http://registers.centralbank.ie/FundSearchResultsPage.aspx?searchEntity=FundServiceProvider&searchType=Name&searchText=&registers=6%2c29%2c44%2c45&AspxAutoDetectCookieSupport=1", False
            .setRequestHeader "Content-type", "application/x-www-form-urlencoded"
            .send ArgumentStr
            html.body.innerHTML = .responseText
        End With
       
        Set tbl = html.getElementsByTagName("table")(1)
        For Each tRow In tbl.getElementsByTagName("tr")
            For Each tCel In tRow.getElementsByTagName("td")
                If Not InStr(tCel.innerText, "Page") > 0 Then
                    c = c + 1: Cells(x + 1, c) = tCel.innerText
                End If
            Next tCel
            c = 0
            x = x + 1
        Next tRow
   Next y
End Sub

Just let me know if you have any issues?
 
That's great and wonderful ... Thank you very very much for wonderful solution
Do you use specific tools that facilitates the prcoess of inspecting the elements or just use the Developer Tools?
 
In the second image google your eyes to the bottom, you may find that culprit, I meant pagination.
 
Thanks a lot
I have copied the monster string for more than a page to compare and found that there is a common part and there is different in some parts .. But as for the string you used it is working for all the pages no problem ...I wonder why ?
 
After testing the code again I found that the pages scrapped are the same ..
I change the loop from 1 to 5 and check the results and the results are the same
although I have tested it earlier and it worked well .. That's weird
 
May be this string is part of security issue that changes automatically ...
I haven't selenium installed .. I heard about it but I have no idea of using it
Will it be useful in that case?
 
That will be useful in every case. No site will be able to put a barrier from scraping their data. You can deal with javascript-heavy sites like regular one.
 
I've already written the script to get data from 57 pages. It will give you the desired results whenever you run it. Here is the script:

Code:
Sub centralbank_data()

    Dim driver As New ChromeDriver
    Dim tabl As Object, rdata As Object, cdata As Object
   
    With driver
        .Get "http://registers.centralbank.ie/FundSearchResultsPage.aspx?searchEntity=FundServiceProvider&searchType=Name&searchText=&registers=6%2c29%2c44%2c45&AspxAutoDetectCookieSupport=1"
           
        Do
            Set tabl = driver.FindElementsByTag("table")(2)
            For Each rdata In tabl.FindElementsByTag("tr")
                For Each cdata In rdata.FindElementsByTag("td")
                    If Not InStr(cdata.Text, "Page") > 0 Then
                        y = y + 1: Cells(x + 1, y) = cdata.Text
                    End If
                Next cdata
            x = x + 1
            y = 0
            Next rdata
           
            On Error Resume Next
            .FindElementById("ctl00_cphRegistersMasterPage_gvwSearchResults_ctl18_btnNext").Click
            If Err.Number <> 0 Then Exit Do
            On Error GoTo 0
        Loop
       
    End With
   
    driver.Quit
End Sub

Does it look like hard to understand? Btw, Marc L might scold me for this. He doesn't like selenium specially when it comes to work with vba.
 
Last edited:
With selenium, you can scrape what you can see. Btw, google it. I forgot the way. It should not take long, though.
 
Thanks a lot I have searched for it and install
And after putting the code I encountered this error (chrome launched and the error appears)
Error.png
 
Any idea about how to fix the addon ..? I have tried several versions and I got the same error. Is it working for specific version of Chrome?
 
Back
Top