Scraping caused IP banning

[YasserKhalil]

Hello everyone
I have tried this code for scraping Status

Sub Scrape_Status()
    Dim ws          As Worksheet
    Dim v          As Variant
    Dim s          As String
    Dim r          As Long

    Set ws = ThisWorkbook.Worksheets("Test Set")

    For r = 3 To ws.Cells(Rows.Count, 2).End(xlUp).Row
        With CreateObject("WinHttp.WinHttpRequest.5.1")
            .Open "GET", "https://register.epo.org/application?number=" & ws.Cells(r, 2).Text, False
            .setRequestHeader "DNT", "1"
            On Error Resume Next
                .send
                If .Status = 200 Then
                    v = Split(.responseText, "Status</td>")
                    s = Split(v(1), "<br/>")(0)
                    s = Trim(Mid(s, InStr(s, ">") + 1))
                    ws.Cells(r, 4).Value = s
                End If
            On Error GoTo 0
        End With
    Next r
End Sub

It worked well for me when testing with just one or two links
But when trying to deal with about 200 links in the same way I got nothing .. and when navigating manually I noticed that my IP has been banned
How can I deal with that issue?

Here's an example link
https://register.epo.org/application?number=EP10156846

 

[Marc L]

Hi !

When request detected and IP banned,
try by piloting IE after a while (a day or two or more) …
Same issue under Google after 200 requests for example …
​

 

[YasserKhalil]

Thanks a lot for reply
But I think using IE will make the process too slow ..
I heard about using proxies .. How can I implement that attitude ?

 

[Marc L]

As it's not an Excel VBA subject,
better is you ask on a specialized web forum …

And the question is what do you prefer ?
A fast request failing each time after a number of access
or a slow IE processing working whatever the number ?
​

 

[YasserKhalil]

I think in this case I would prefer using IE ..
Thanks a lot Mr. Marc

 

[Chihiro]

Using proxy is easy. But since most likely that you'd be using Public Proxy, you'll need to ensure it's working before you use it. Most Public Proxy won't be up/stable 24/7.

As well, Google has pretty much banned use of Public Proxy for scraping in 2016. Don't know if things have changed since.
https://www.my-proxy.com/blog/google-proxies-dead

Some quick tips on using Proxy can be found in link. Though info is bit old.
http://analystcave.com/web-scraping-proxy-http-request-vba/