Unable to add an item to the already existing dictionary

[shahin]

I've written a macro using IE to scrape three fields from each container out of a webpage. They all are rightly coming through. However, the problem with the results is that it contains duplicate values. Few days back, with the help of sir chihiro I could write one using dictionary which can handle duplicates on the fly when I parse two fields. Now, I've added one more fields within my scraper but can't get any idea to add it to the existing dictionary so that I can scrape three fields kicking out the duplicates. Thanks in advance.

Here is the script:

Sub GetItems()
    Const URL As String = "https://www.inc.com/profile/sumup-payments-limited"
    Dim post As Object, container As Object, idic As Object, key As Variant
    Dim a$, b$, c$, R&
  
    Set idic = CreateObject("Scripting.Dictionary")
  
    With CreateObject("InternetExplorer.Application")
        .Visible = True
        .navigate URL
        While .Busy = True Or .readyState < 4: DoEvents: Wend

        For Each post In .document.getElementsByClassName("profile")
            With post.getElementsByTagName("h1")
                If .Length Then
                    a = .item(0).innerText
                End If
            End With
            With post.getElementsByClassName("ifi_industry")(0).getElementsByTagName("dd")
                If .Length Then
                    b = .item(0).innerText
                End If
            End With
            With post.getElementsByClassName("employees")(0).getElementsByTagName("dd")
                If .Length Then
                    c = .item(0).innerText      ''how to add this "c" to the existing dictionary like I did in the following line
                End If
            End With
            idic(a) = b    ''adding to the dictionary
        Next post
       For Each key In idic.Keys
            R = R + 1: Cells(R, 1) = key
            Cells(R, 2) = idic(key)
       Next key
        .Quit
    End With
End Sub

 

[NARAYANK991]

Hi ,

Before you get down to coding , first describe the condition for calling an entry a duplicate.

Do you have duplicates within the first field alone , the second field alone or the third field alone ?

Do you have duplicates in any combination of 2 or more of the 3 fields i.e. will there be a duplicate using fields 1 and 2 , or fields 2 and 3 or fields 1 and 3 or using fields 1 , 2 and 3 ?

The correct way to enter data into the dictionary is to concatenate all 3 fields and then enter that concatenated value into the dictionary.

Narayan

 

[shahin]

Thanks for your reply Narayan. The results I'm having like this for a single lead:

A                                B                     C
Sumup Payments Limited    Financial Services          500
Sumup Payments Limited    Financial Services          500
Sumup Payments Limited    Financial Services          500

 

[shahin]

Thanks again, Narayan for your invaluable tips. I tried and got success. However, I only need a better idea to concatenate and separate them later. Thanks again.

Sub GetItems()
    Const URL As String = "https://www.inc.com/profile/sumup-payments-limited"
    Dim post As Object, container As Object, idic As Object, key As Variant
    Dim a$, b$, c$, R&
   
    Set idic = CreateObject("Scripting.Dictionary")
   
    With CreateObject("InternetExplorer.Application")
        .Visible = True
        .navigate URL
        While .Busy = True Or .readyState < 4: DoEvents: Wend

        For Each post In .document.getElementsByClassName("profile")
            With post.getElementsByTagName("h1")
                If .Length Then
                    a = .item(0).innerText
                End If
            End With
            With post.getElementsByClassName("ifi_industry")(0).getElementsByTagName("dd")
                If .Length Then
                    b = .item(0).innerText
                End If
            End With
            With post.getElementsByClassName("employees")(0).getElementsByTagName("dd")
                If .Length Then
                    c = .item(0).innerText
                End If
            End With
            idic(a & "-" & b & "-" & c) = 1  ''I need to find a good way to separate them later to their individual cells. So I tried using this "-"
        Next post
        For Each key In idic.Keys
            R = R + 1: Cells(R, 1) = Split(key, "-")(0)
            Cells(R, 2) = Split(key, "-")(1)
            Cells(R, 3) = Split(key, "-")(2)
        Next key
        .Quit
    End With
End Sub

 

[Marc L]

and separate them later

You already know ways ! Like Split, TextToColumns, …
​

 

[shahin]

Yeah I know it. However, do you think the way i concatenated them like "-" is a good idea.

 

[shahin]

This is because when i come across this lead, the results gets messy (without the quotes):

"Interskol-Alabuga Power Tools"  "Consumer Products & Services"   "48"

 

[Marc L]

do you think the way i concatenated them like "-" is a good idea.

If it works it can't be nothing else than a good idea !
​

 

[NARAYANK991]

Hi Shahin ,

When concatenating always remember to use a character which will never occur within your data , as the separator ; one common character is the pipe character |

Narayan

 

[shahin]

Just love you man @Narayan. Yes, I experimented it just now. Thanks a trillion.

 

[Marc L]

Oh it was unclear but as it's so obvious
so yes the delimiter wasn't a good idea !
As you can see in many samples within this forum …​