• Hi All

    Please note that at the Chandoo.org Forums there is Zero Tolerance to Spam

    Post Spam and you Will Be Deleted as a User

    Hui...

  • When starting a new post, to receive a quicker and more targeted answer, Please include a sample file in the initial post.

Get data from web site

Hello SirJB7 and Marc L!
Now I tried again both codes at home and they worked perfectly.

Is it possible to use the same code to get data from another Ambito's page?
http://www.ambito.com/economia/mercados/riesgo-pais/info/?id=2

I tried to use the same code and changed the web address and date, but it gets everything, except the first line of each page...
Could you tell me why, please?

Thank you very much again for all your great cooperation
 

Instead to tell a code doesn't work as expected, it's far better to tell you modify it ‼ :rolleyes:

The build of the two web pages are very different …
If you are not comfortable with the study of a web page (HTML),
easier for you to stay with QueryTable method …

My next post will be for the faster alternative Let It Be way …
 
Hello MarcL and SirJB7! Thank you very much for all your help!

MarcL: how can I execute the code? I press the GO button but a message pops up: "No Web!".

Hi, MarcL!
Yesterday I tried to execute the code at home and it worked perfectly (Windows 8, Excel 2013) But now I am trying at work (Windows 7, Excel 2010) and the message "No Web!" appears. :eek:
 
Hi, MarcL!
Yesterday I tried to execute the code at home and it worked perfectly (Windows 8, Excel 2013) But now I am trying at work (Windows 7, Excel 2010) and the message "No Web!" appears. :eek:
Works on my side !

You could put in comment line #23 If WebOK(URL)
Procedure WebOK checks if computer is connected to web and site URL responds or not …
Useful for laptops Wi-Fi !

Excel version in 32 or 64 bits ?

Did you try it several or just once ?​
 
Hi, chronot!

Give a look at the uploaded file, it has the same name as the previous version but it's been changed to handle similar web queries of analogue structure (varying date from, date to and page).

Now there are this worksheets:
Hoja_Blue, for U$D real not fictitious
Hoja_RP, for Country Risk
Work, well, someone has to work

Each data worksheet has the following named ranges:
FromCell, date from, if empty starts at 01/01/1900
ToCell, date to, if empty ends today
URLCell, link to the actual web page (check below for wildcards)
TitleCell, no. of title rows (yeah, wanna kill Ambito's guys)

These are now ranges with worksheet scope instead of workbook scope as before, so you can copy any sheet and keep their definitions without having to change anything, just name the worksheet as "Hoja_xxx", where "xxx" is the new variable and you can even change "Hoja_" from the constant section of the code.

The URL has these wildcards for replacing date from, date to and page, which are replaced for each query with the proper related values:
@#@ for date from
#@# fro date to
## for page no.

The reason because it was missing the 1st entry of each page is that they funny Ambito IT guys placed 2 row titles for Blue tables (so actual data started at 3rd row) and at Country Risk tables the placed only 1 title. That's why it missed the 1st one since it stared at row 3 for copying data; now with the title parameter this issue is solved.

But, and my friend b(ut)ob(ut)hc (who uses to say "there's always a but...t") who knows more than most of us (for being a very old man, in part) would smile... but in this CR tables you'll have to copy the last page records manually into the data worksheet because there's an internal error with their data. Check the image file uploaded.

Well, I hope that this will be enough for this week, if any issue, send me a case of cases of six-pack of Carlsberg via Fly Emirates (don't use Aerolíneas Argentinas, please) or wait until next week, but not on Monday.

Regards!
 

Attachments

  • Get data from web site (for chronot at chandoo.org).xlsm
    274.1 KB · Views: 6
  • Get data from web site (for chronot at chandoo.org).png
    Get data from web site (for chronot at chandoo.org).png
    198.8 KB · Views: 4
Hello, SirJB7!
What a wonderful work, infinite thanks for all your help! I tried at home and it works flawlessly!!

I prefer not to modify the code because this is a weird world for me and I don't have the knowledge to understand what the code means. Don't worry about the last page that has an internal error, it'd be easy to manually copy after the huge increase of productivity with your awecome code :)

Thanks a lot for explaining me about the row titles, now that you told me it's easy to realize on screen.

I think it's more than enough with all your help, I really appreciate your effort.
Oh I am afraid that Fly Emirates is not operating in my country... there are few airlines, and one of them is Aerolineas Argentinas. Last year the big news was that American Airlines returned to Paraguay after many years of absence.

Best wishes!
 
To grab the 101 web pages data of chronot's example, my code needs 45 seconds
with standard MSXML2 way on the same computer.​
An average of 24/25 sec, vs. 1'15" of WebQuery version.

SirJB7, you're already at the level of my alternative way !​
To grab the 101 web pages data of chronot's example, my alternative
Let It Be way needs only 25 seconds on the same computer.​
I wanna know if you could have a better time …
But before to discover the alternative way code, let me tell a little story :

The queen of a colony asks to collect the daily harvest of a particular nectar
at one of her bees.
Seeing this bee exhausted by the back and forth between each flower and the nest,
the queen says :
« You are responsible to collect this divine nectar,
but you have the right to ask sisters to help you ! »

By analogy, a bee is a request procedure and a flower, a Web page,
as nectar is data and the nest, a worksheet of an Excel workbook ...

Alternative way principle:
a VBA procedure (mono-task) launches VBScript requests in Windows (multi-tasking) …

As the famous Tarantino's movie, this alternative way is in two parts :
the first ascending (LetItBee), after finding the last Web page
and generation of VBScript file, launches requests,
the second descending (Worksheet_Change event) counts down requests.
Do not worry if nothing seems to happen, monitor still the status bar,
the data will come bundled ! And notice the end identical to the movie !

Edit : little optimization …​
Code:
Const TITULO$ = "      Web Import"

Private BeeC%, BeeL$, BeeT!

Private Declare Function InternetCheckConnectionA Lib "Wininet" (ByVal SITE$, ByVal one&, _
                                                                 ByVal zero&) As Boolean

Function WebOK(Optional ByVal URL$ = "http://www.msn.com") As Boolean
            P& = InStr(9, URL, "/"):  If P Then URL = Left$(URL, P)
         WebOK = InternetCheckConnectionA(URL, 1, 0)
End Function


Sub WindUp(Optional BeeMIA As Boolean = True)
    Me.Shapes("GO!").Visible = True:  Application.StatusBar = False
    If BeeMIA Then MsgBox "Bee Missing In Action !", vbExclamation, TITULO
    On Error Resume Next
    Kill BeeL:      End
End Sub


Sub Been(SHEDULE As Boolean)
    Static TS
    If SHEDULE Then TS = Now + 0.0007
    Application.OnTime TS, Me.CodeName & ".WindUp", , SHEDULE
End Sub


Private Sub LetItBee()
    Dim TP%()
    BeeT! = Timer

    For Each D In [{"DESDE", "HASTA"}]
        If Range(D).Value = "" Then _
           Range(D).Select: MsgBox "No " & D & " !", vbExclamation, TITULO: End
    Next

    URL$ = "http://www.ambito.com/economia/mercados/monedas/dolar/info/?ric=ARSB=&desde=" & _
           [DESDE].Text & "&hasta=" & [HASTA].Text & "&pag="

    If WebOK(URL) = False Then MsgBox "URL no responde !", vbExclamation, TITULO: End

    With [FECHA].Offset(1)
        FC& = .Column:  FR& = .Row
    End With
                          C& = Me.UsedRange.Rows.Count
    Me.Shapes("GO!").Visible = False
    If C >= FR Then Cells(FR, FC).Resize(C - FR + 1, 3).Clear
    Application.ScreenUpdating = False
    Been True:  LP% = 1:  N% = -1:  RC% = 99

    Do
        P% = LP:  Application.StatusBar = Format(P, "Page @@@")

        With CreateObject("MSXML2.XMLHTTP")
            .Open "POST", URL & P, False
            .send
            If .Status = 200 Then T$ = .responseText Else T = ""
        End With

        If T > "" Then
            With CreateObject("HTMLFile")
                .Write T:  ReDim AR(1 To RC, 2):  C = 2:  R% = 0

                For Each D In .GetElementsByTagName("div")
                    If D.className = "numeros" Then
                        C = (C + 1) Mod 3
                        If C Then AR(R, C) = Replace(D.innerText, ",", ".") _
                             Else R = R + 1: AR(R, 0) = CLng(DateValue(D.innerText))
                    End If
                Next

                If R Then
                    Cells(FR + (P - 1) * RC, FC).Resize(R, 3).Value = AR
                    If P = 1 Then RC = R
                     D = Split(.GetElementById("boxPaginador").innerText, vbCrLf)
                    LP = CInt(D(UBound(D)))

                    If LP = P Then
                        If N > 0 Then ReDim Preserve TP(N - 1)
                    Else
                        N = N + 1:  ReDim Preserve TP(N):  TP(N) = LP
                    End If
                End If
            End With
        End If
    Loop Until T = "" Or P = LP

    If R Then
        With Range(Cells(FR, FC), Cells(Rows.Count, FC + 2).End(xlUp))
                .Columns(1).NumberFormat = "dd/mm/yyyy "
            .Columns("B:C").NumberFormat = "#,##0.000 "
        End With

        If LP > 2 Then
            With ThisWorkbook
                BeeL = .Path & "\Bee - " & Split(.Name, ".")(0) & " - " & Me.Name & " .vbs"

            SC = Array("Dim AR(" & RC & ",2)", "On Error Resume Next", _
                       "P=WScript.Arguments(0): If Err.Number Then WScript.Quit 1", _
                       "With CreateObject(""MSXML2.XMLHTTP"")", _
                       "If Err.Number Then WScript.Quit 2", _
                       ".open ""POST"",""" & URL & """ & P,False", _
                       "If Err.Number Then WScript.Quit 3", _
                       ".send: If .status=200 Then T=.responseText", "End With", "R=-1", _
                       "IF T>"""" Then", "With CreateObject(""HTMLFile"")", ".Write T: C=2", _
                       "For Each D In .GetElementsByTagName(""div"")", _
                       "If D.className=""numeros"" Then C=(C+1) Mod 3: " & _
                       "If C Then AR(R,C)=Replace(D.innerText,"","",""."") Else " & _
                       "R=R+1: AR(R,0)=CLng(DateValue(D.innerText))", "Next", _
                       "End With", "End If", "GetObject(,""Excel.Application"")" & _
                       ".Workbooks(""" & .Name & """).Worksheets(""" & Me.Name & _
                       """).Cells(3+(P-1)*" & RC & "," & FC & ").Resize(R+1,3).Value=AR")
            End With

            F% = FreeFile
            Open BeeL For Output As #F
            Print #F, Join(SC, vbNewLine)
            Close #F
            SC = """" & BeeL & """ "

            With CreateObject("WScript.Shell")
                For P = 2 To LP - 1
                    D = Application.Match(P, TP, 0)

                    If IsError(D) Then
                        .Run SC & P:     BeeC = BeeC + 1
                        Application.StatusBar = "Let it Bee : " & Format(BeeC, "@@@")
                    End If
                Next
            End With
        End If

    Else
        MsgBox "Compruebe las fechas !", vbExclamation, TITULO
    End If

    Application.ScreenUpdating = True:  If BeeC = 0 Then Been False: WindUp False
End Sub


Private Sub Worksheet_Change(ByVal Target As Range)
    If BeeC And Target.Columns.Count = 3 Then
        BeeC = BeeC - 1:  Application.StatusBar = "Let it Bee : " & Format(BeeC, "@@@")

        If BeeC = 0 Then
            S$ = Format$(Timer - BeeT, " (0.000s)"):  Been False:  Debug.Print "LetItBee" & S
            MsgBox "Operación completada …" & S, vbInformation, TITULO:  WindUp False
        End If
    End If
End Sub


Regards !

______________________________________________________________
This is the answer, Let It Bee !
 

Attachments

  • Let It Bee v2 Web Data .xls
    41 KB · Views: 9
Last edited:
Hi, Marc L!
This flew at 17.5 sec... wow... Will read it thoroughly later or tomorrow.
Good job, buddy.
ReCarlsGardsBerg!
 
Oh I am afraid that Fly Emirates is not operating in my country... there are few airlines, and one of them is Aerolineas Argentinas. Last year the big news was that American Airlines returned to Paraguay after many years of absence.
Hi, chronot!
Well, by LAN then :), but never by AA! :eek:
Regards!
PS: Glad you could test it and found everything Ok.
 
Back
Top