Handling webpages with lazyload in an efficient way

[shahin]

I've created a script which can handle webpages with slowload and parse the data when there is no more scrolling left. It works on infinite scrolling webpages as well.

This is the script (give it a go):

Sub ControlSlowload()
    Const URL As String = "https://www.yellowpages.ca/search/si/1/coffee/all%20states"
    Dim IE As New InternetExplorer, HTML As HTMLDocument, R&
    Dim post As Object, elem As Object, prevlen&, curlen&

    With IE
        .Visible = True
        .navigate URL
        While .Busy = True Or .ReadyState < 4: DoEvents: Wend
        Set HTML = .document
    End With
  
    prevlen = HTML.getElementsByClassName("listing__name--link").Length
  
    Do
        prevlen = curlen
        HTML.parentWindow.scrollBy 0, 99999
        Application.Wait Now + TimeValue("00:00:02") ''increase the delay as per your internet speed
        Set post = HTML.getElementsByClassName("listing__name--link")
        curlen = post.Length
        If prevlen = curlen Then Exit Do
    Loop
  
    For Each elem In post
        R = R + 1: Cells(R, 1) = elem.innerText
    Next elem
    IE.Quit
End Sub

 

[klimbo13]

That’s something to consider as well – but I’d suggest using visibility:hidden instead of display:none. Visibility will keep space for the item so it doesn’t make content jump around when you finally show it. AppValley Vidmate TweakBox