Recently, I had a peculiar problem. I have a list of zip codes and I wanted to find out nearest zip codes for each of them.
Now, If I wanted to find out near by zip codes for one area, I could go and search in Google. But, how to do it for dozens of them?
Today, lets understand how you can use Excel (that’s right) to do this automatically. We will be using Excel 2013 for this.
A little background – Excel 2013 Web Formulas
In Excel 2013, Microsoft has introduced 3 powerful new formulas. These will help you fetch & parse XML / HTML data from web. The formulas are,
- ENCODEURL: to encode web URLs (replaces special characters in URLs with % codes like space becomes %20, / becomes %2F etc…)
- WEBSERVICE: connects to a webservice / website and fetches response as XML / HTML.
- FILTERXML: extracts a portion of XML/HTML using specified XPATH.
Using these formulas and web services, we can quickly fetch near by zipcodes for any input value.
Step 1: Find a web-service that can tell us near by zipcodes
I am sure there are many web sites that can offer a service like this. After searching a while, I came across a website called as geonames.org which has many webservices around address / zip code search. The service I have used is,
This service is available as XML & JSON. Since Excel 2013 formulas only process XML data, I went with XML service. The service API url is this:
http://api.geonames.org/findNearbyPostalCodes?postalcode=ZIPCODE&country=US&radius=15&username=UNAME&maxRows=10
ZIPCODE is where you enter the zipcode from which you want to find nearby zipcodes
UNAME is where you enter your user name for geonames.org. Click here to register with geonames.org.
Step 2: List all original Zip codes in a column
This is simple. Just paste all original zip codes in a column.
Step 3: Write WEBSERVICE Formula
First enter the API URL in a cell like B1. (Make sure your user name is included in the service url)
Now write WEBSERVICE formulas so that we can fetch XML listing for each of the zip codes. Assuming zip codes are in A3:Ax, in adjacent column write =WEBSERVICE(SUBSTITUTE($B$1,”ZIPCODE”,A3))
And drag it down to fill down the formula for all zipcodes.
Step 4: Write FILTERXML formulas
Now that we have full XML corresponding to each zip code, we need to parse this XML to extract the nearby zip code numbers. The original XML looks something like this:
To extract the zipcodes alone, we need to use FILTERXML formula.
FILTERXML takes 2 inputs – XML text, Xpath.
XML text is what WEBSERVICE has generated.
XPATH will tell Excel, which portion of XML to extract.
What is XPATH?
If you imagine XML as a tree, then XPATH is the language you use to tell how to navigate to a certain node in that tree. Since XPATH is a complex world, I think explaining all the syntax & nuances can be hard. So I will leave you with 2 useful links.
So what is the XPATH for nearby zip code.
As you can see in above image, the response from geonames has 10 code nodes, each containing one zip code (in the postalcode child node).
If we write =FILTERXML(b3,”/geonames/code/postalcode”) we will get all the postalcodes as an array.
Since Excel cannot show arrays in cells, it will show one of the 10 values.
So we need 10 cells to show these 10 zip codes. Once you have 10 cells, you can use either INDEX formula or alternative XPATH syntax (/geonames/code[1]/postalcode for first code, ../code[2]/.. for second code etc.) to extract all the 10 zip codes.
Things to keep in mind
Web formulas (WEBSERVICE formula to be specific) can be really slow depending on your net connection and webserver speeds. Since for most data, we do not need a live connection once the data is fetched, it would be good idea to replace WEBSERVICE formula with results once you have the XML.
Also, working with XPATH can be frustrating if the source XML is not correctly formatted or you are not familiar with right XPATH commands. In such cases, use SUBSTITUE or Text formulas to strip away un-necessary portions of webservice text before feeding it to FILTERXML.
Last but not least, Web formulas are compatible only with Excel 2013 or above. So you need to replace all formulas with results when emailing the workbooks to colleagues who are using older versions of Excel.
Download Example File – Finding Nearby Zipcodes
Click here to download the Excel workbook. Play with it to understand how the formulas are working. Please note that this file is protected as I do not want you to use my username for geonames.org.
Do you use Excel Web formulas?
Although Excel 2013 includes only 3 web formulas, they can let us do several interesting things. I am playing with them often to see what additional uses we can put them to.
What about you? Have you used Excel 2013 web formulas? What is your experience like? Please share using comments.
More on using Excel to get data from web
If you often need data from external websites for your Excel analysis work, check out below articles too:
24 Responses
I’d suggest simply using the subtotal function and filtering the data using the Win/Loss column. You get the same results and the formula is more comprehensible.
@John
That is one option.
There are times however when you want to see the whole data table or a filtered subset and still want to produce summary reports against an unfiltered field.
Is there a particular reason why you are using a comma and the unary (–) operator for the second array in the SUMPRODUCT formula? It seems to work the same if you were to string the arrays together using the asterisk (*). The advantage is that SUMPRODUCT treats the entire string of arrays as a single array.
@Mathew
Your correct, There is no difference.
I thought it may have been easier to explain this method.
Is there a way to do this on a large set of data? As in ~100,000 rows? When I try I get an error because the formula becomes too long. It says the max length of a formula is 8,192 characters. Excel 2010.
How do I incorporate a specific text within a cell for the second array. For instance, – -(C7:C13=”Apple”)
when I chose a specific text the formula does not work.
@RB
I am not sure what is the issue as if I use the sample data in the post the following work fine
Count:
=SUMPRODUCT(SUBTOTAL(3,OFFSET(C7:C13,ROW(C7:C13)-MIN(ROW(C7:C13)),,1)), –(C7:C13=”L”))
Sum:
=SUMPRODUCT(SUBTOTAL(3,OFFSET(C7:C13,ROW(C7:C13)-MIN(ROW(C7:C13)),,1)),(C7:C13=”L”)*(D7:D13))
You may want to check that there are no leading or trailing spaces in your list of Apples
I should have given a better explanation. Heres my situation. I have a column with cells filled with names like Column 1, Column 2, Pier 1, Pier 2, etc. If the cell just contained Pier and searched for that it works. But because it has other characters in the cell its not recognizing the pier. So how can I extract specific characters of a string of text in this formula?
Hopefully this was a better explanation
Hello-
This formula works pretty well for me except that it slow down excel and prevents some of my macros from working. I was wondering if there was a way to program this in VBA so that excel isn’t always trying to recalculate it. I would like to use a push of a button to get it to run then paste in a cell.
Thanks!
I am trying to sum filtered data in a column, but would want to ignore the negative values in the column. How to go about doing this?
@Akshay
Why not just add a filter to that column to only show the values greater than zero?
The negative values are required for reporting purposes, but their effect on the total is distorting the required output. Please advise.
@Akshay
I’d suggest making a post in the Chandoo.org Forums
http://forum.chandoo.org/
Attach a sample file to simplify the task
I have this working for counting and summing, however, I have a list and for the second array, I need a criteria. That is, I’m looking for b13:b200=”01.??.??” or =left((a1,2) or something like that. These types of criteria matches do not appear to work as I get a blank as a result.
Thanks!
@Bob
As your formula b13:b200=”01.??.??” looks like you are trying to check the first day of the month of the range
What about trying Day(B13:B200)=1
Hai Experts,
i understood this formula well and working fine in MS Excel 2013
but when the same am trying to place in google Spreadsheet it shows error as
“SUMPRODUCT has mismatched range sizes. Expected row count: 1. column count: 1. Actual row count: 2014, column count: 1.” and as a result #VALUE! Appears in cell.
Can anyone please help me how would i get it done in Google Spread sheet
or is there any other formula as a substitute for this.
Thank you very much.
thanks for providing this.. but why does excel keeps on prompting Circular referencing in cell D3?
@Vivek
I don’t know
I just downloaded the file and it is working fine and not showing that error
Goto the Formulas, Calculation Options Tab and check that Calculation is set to Automatic
What version of Excel and Windows are you using ?
I know that this forum is for MS Excel, but I am trying to help someone who is working in Google Sheets. The below formula works in Excel but Google Sheets returns:
“SUMPRODUCT has mismatched range sizes. Expected row count: 1. column count: 1. Actual row count: 39000, column count: 1.” and as a result #VALUE! Appears in cell.
This is the same problem asked by Srichirin above. Does anyone know if there is a formula for Google Sheets that will replicate what MS Excel does?
=SUMPRODUCT(SUBTOTAL(3,OFFSET($C$6:$C$39500,ROW($C$6:$C$39500)-MIN(ROW($C$6:$C$39500)),,1)),- -($C$6:$C$39500=H1),($D$6:$D$39500))
Trying to find a SUMPRODUCT formula that counts the word Closed by date for the last 7 days in a filtered list.
=COUNTIF(M:M,”>”&TODAY()-7) works ok for unfiltered count Column M contains Closure dates (blank if open) and Column L is Status Open or Closed
@ Terry
Please ask the question at the Chandoo.org Forums
https://chandoo.org/forum/
Please attach a sample file to ensure a quicker more accurate answer
I used this formula and worked like a charm! But, now I’ve been requested to use it but adding not one but two criteria in the same formula. For instance the sum I was doing added negative and positive numbers. I’ve been asked to use the exact same formula but adding that only positive numbers were considered… any idea on how to do this?
How exactly do you do sum filtered cells when two criteria are need not just one?
Thank you so much brother literally I have been struggling since morning to get the sum of the filtered category, however, after reading your blog attentively i got my solution, so thanks a lot once again.