Let’s take a whirlwind trip to coolest little capital – Wellington. It is a windy place, so hold on to your hats and spreadsheets.
Almost everyone who spends more than 2 days in Wellington would agree that it is a windy place. But how windy is Welly? In this two part series, we will use Power Query, Excel charts and coffee to answer that question.
But, first let’s start with a joke.
What happens when you throw a boomerang in Frank Kitts Park?
You will have to buy another one, coz you are not getting that one back.
Extracting the wind data
In order to understand how windy Wellington is, we need to get average wind speeds by day for last several days. Let’s get the data for last 2+ years (ie from 1 Jan 2016 to 21 Feb 2018).
There are many places where you can collect latest wind data. But when it comes to historical wind data, surprisingly few resources are available. We can use The National Climate database – CliFlo, to gather wind data. But the interface is confusing and I could only locate gust speeds, rather than average wind speeds over time.
We can use wunderground.com to fetch weather data for up to 13 months at a time.
But we need data for almost 26 months.
Very simple, we can query wunderground twice (or thrice), once per each year.
The historical data query URL looks like this:
https://www.wunderground.com/history/airport/NZWN/2016/1/1/CustomHistory.html?dayend=31&monthend=12&yearend=2016&req_city=&req_state=&req_statename=&reqdb.zip=&reqdb.magic=&reqdb.wmo=
All we had to do is, change 2016 to 2017 & 2018 to get respective data.
The actual data set will be a web page. But we can use power query to extract the portion of page that contains weather information.
On to Power Query – Building our Weather Data Extractor
Note: This is a slightly advanced tutorial on PQ. If you are a beginner, start with Introduction to Power Query and work thru examples on PQ tag page before reading any more.
Getting data from the web – building URL in parts
Open Excel and go to Data > New Query > From Other Sources > Web
For Power BI, this would be Edit Queries > New Source > Web
Switch to “Advanced” mode and enter the URL as parts like below. We will switch the 2016 part to parameters soon, so we could get data for any year easily.
In the navigation pane, select “Table 1” which is the weather table.
Set up a parameter for Year
How would we get data for 2017 or 2018? Simple, we use parameters. These are like variables which can be plugged in to any part of your Power Query process.
In Power Query Editor, go to Home > Manager Parameters > New Parameter and call it Year. Enter the default value as 2016.
Now, go back and edit the source settings for the query and replace 2016s with parameter Year.
Cleaning the weather table
Turns out the weather data table is not clean. Although there are 366 days in 2016 (leap year), Wunderground adds headers for each month. So we end up with 378 rows (excluding the header). Each header contains month name and repeat of all the column names. We can extract the month name & combine that with date and year parameter to create the date for each row.
Here is a quick illustration of what we need to do.
But first, rename the very first column
Notice the first column? It is called as 2016. This is ok if we are interested in just 1 year of data. But if we re-run this query with Parameter=2017, our column heading will change. If you have dabbled with Power Query a few times, you will quickly realize that PQ will get in to a nasty fit anytime column headers change and impact downstream steps.
Simple, we shall rename it as FirstCol.
When you apply the new name, PQ will write this M instruction.
#”Renamed Columns”= Table.RenameColumns(Data1,{{“2016”, “First col”}})
This is not a fool proof solution, as when we change parameter to 2017, there won’t be a 2016 column in that new table.
So, instead, we can ask PQ to rename first column of the table.
You can do this by:
- Note: You need “Formula Bar”. Enable “Formula Bar” by clicking View > Formula bar. This way you can actually see all the M code PQ is cranking up whenever you perform some actions on your data.
- Click on fx button on the formula bar to insert a step. Simply type = Table.RenameColumns(Data1,{{Table.ColumnNames(Data1){0}, “First col”}})
- Press Enter
- Bingo, you have renamed the first column of your query to “First col”. This has no reference to 2016 or any year, so it should work on any table you fetch from that weather data page.
Cleaning the weather data – steps
Just follow these steps to clean the weather data.
- Add a custom column called Month and write this formula = if Text.Length([First col]) > 2 then [First col] else null
- Select Month column and Fill Down (Transform > Fill >Down)
- Select First col and change its type to whole number. This will make all month names as Error
- Remove errors from First col (Right click on column header and choose Remove Errors)
- Add a custom column called Date with the formula = Text.From([First col])&”-“&[Month]&”-“&Year
- Change this column to date type.
- Keep only Temp. (°C)2, Wind (km/h), Wind (km/h)2, Wind (km/h)3, Events, Date columns and remove all other
- Rename first four columns to Avg. Temp, Wind Max, Avg. Wind, Wind Gust
At this stage we have one year of wind and temperature data for Wellington. Time to create getWeatherData() function.
Making getWeatherData function in Power Query
Now that we have a parameterized query, just right click on the query and choose “Convert to Function”
PQ will build the function that can take year as input and return a table of weather data for that year (provided Wunderground.com co-operates)
Now, we just need to run this function three times, once each for 2016, 2017 and 2018 to get all the data.
Go back to Excel
Save your queries, but don’t load them yet. If PQ prompts about data load, select “Connection only” and jump to Excel.
- Create a table with 3 rows and type 2016, 2017 and 2018 in that. Call this table Years.
- Load this table to Power Query (Data > From Table)
- Go to Add Column > Invoke Custom Function and invoke getWeatherData function for each year.
- Expand the weather data tables.
- Done!
At this stage, we have data for all 3 years. You can add some data clean up steps if you want. But all the wind & temperature data is here for us to analyze and visualize.
Download Example Workbook
Click here to download the Wellington Wind workbook. As you can see, I have added few more steps in PQ to clean up the data and include a “Is it windy?” conditional column.
Please note that this workbook is designed in Excel 2016. It may not work in older versions of Power Query. You can replicate most of the steps. Try doing it so that you will learn more about Power Query.
In the next part – Wind in Wellington – few visualizations
In the next part of this tutorial, we will build some visualizations to understand how windy Wellington gets and what is the best time to enjoy the beautiful outdoors.
Stay tuned.
How are you using Power Query? Please post about your power query escapades in the comments section. Also tell me how you went about re-creating the steps in this tutorial. I am all ears.
Why there are no undercover cops in Wellington? Their cover was always getting blown. That is why.
55 Responses to “Quickly Fill Blank Cells in a Table [Reader Tip]”
this can be done in 3 steps:
1. select the blank cells (as described above)
2. select the cell with the value you want to copy (CTRL-CLICK to add to the selection)
3. place cursor into formula bar and hit CTRL-ENTER
please ignore or delete my comment - it solves a different problem: copying a single value to all blank cells. apologies.
That is a great method and it saves me a lot of time! I first heard about it from Mr Excel in this video - http://www.youtube.com/watch?v=jHmh_viESuw. He has a neat way of doing the paste special values at the end of his video.
Hi!
I fill blank cells with an almost identical method; go to any the first blank cell in any column and place the equation and enter (=D2, for the same example above); then copy that cell, select the columns/range you want to fill (even if in different columns), Special, go to Blanks, Paste (default), copy all range and paste as values.
although the two methods are almost identical, what i use might be less hectic regarding entering formulas without clicking any of the cells (step 7)
ie:
1. fill an empty cell with using =D2(cell above)
2. copy D3 (the cell with the formula)
2. Select blank cells after selecting the range with empty cells (steps 1,2,3,4 and 5)
3. paste (normal)
4. copy then paste as values
BR
AQ
Great tip. I'll use it later today!
Martin -Thank You! This wonderful tip will save me a great deal of time each week.
Thanks Martin! Up to this point, I've always used a clumsy combination of filters and fill-down's. This is much cleaner.
Fantastic. Thanks for sharing.
None of these steps are necessary, Excel has this feature built into the ribbon.
Click on any row label in the table where there are blanks under it.
Click on the PivotTable Tools>Design tab on the ribbon
Click the Report Layout button in the Layout group at the far left
Select the option in the list
Done
To remove the duplication, use the feature right below that option.
There is a slightly simpler way and more flexible. Hihglight the required cells - which could be the column only in your table. Do the Ctrl-G, Alt-S, K, Enter (or Goto, Special, Blank Cells) so that they are highlighted and Type ={up arrow}, Ctl-Enter. This will make the cells equal the cell above - you do not have to enter any address at all. The technique can obviously be adapted to many situations. An example of the practical use for this is when you have saved an Inventory report from an accounting program that prints a heading (or something) on one line and prints details of that group (the heading) on subsequent lines (without the heading).
Hi Martin,
great trick! If only I had known it earlier, it would have saved me quite some time...
Not again, thanks!
I came across this in a class recently myself and posted a tutorial on my blog. The Special area of the Go To dialogue box is wicked. Some great options in there, hidden away waiting to be found.
Good work Martin.
Hi Martin,
Many thanks for sharing this powerful trick. Saves alot of time.
Gabriel
Please give credit where credit is due. Posted on June 30, 1998: http://www.mvps.org/dmcritchie/excel/fillempt.htm
Ahhh... Very neat trick. Thank you, Martin.
Ken, I tried to follow your post but could not get it to work. Could not find options
I have been using this trick for ages and would be lost without it.
Thank you very much!!! I had other tricks to deal with it, but this one is way faster and easier!!!
@BigG: Good resource there. Thanks for sharing the link with us. Please note that, this technique is not new. I am sure many Excel users would have discovered this already. We have not copied or inspired from David's article. It was just a happy coincidence.
@Ken: Your technique works only with Pivot Tables made in Excel 2010 or above.
Thanks Martin!! Nice post 🙂
@Chandoo: I also use the ASAP utilities add- in available in the link below:
http://www.asap-utilities.com/download-asap-utilities.php
This summarizes lot of hidden features in excel (like using Find function on entire workbook, password protecting all sheets at once, copying print setting of sheets etc.,) and is quite useful for beginners like me 😉
Thanks Martin and Ahmad Qadah. This is useful. I previously used to ask the senders to retrieve the data again so that I did not have the blanks.
Nice trick. I always use the specialcells method of the range object in code to access this powerful goto special dialog box in vba - a trick that Chandoo taught me in vba school - which is another reason you should join (a free bit of promotion for you Chandoo..!)
🙂
Yes I have seen this one before so credit may belong elsewhere. Never the less still especially useful where a legacy system report is sent to a text file which is subsequently re-imported to Excel but the original report is indented by groups. You can then recreate a complete data record for each report line
NB Different Ken to above
Thanks Martin - great post. I often work with data in this form and I usually fill in the blanks manually, by copying and dragging a cell value down - this way is much less prone to human error!
One challenge.. the last step where I change formula to constants. This replaces any formulas that I have as well. What If I want to change the formula to constants only where I replaced them with blank ?
Hi martin, thanks a million 🙂
Nicely explained Martin, thanks for sharing this tip. As Tanja says, this method is far less error-prone. When I first learned this method it saved me lots of time, so I decided to create a video on Youtube to share it with others. In my 3 minute video I compare side-by-side two methods of filling in blanks on 500 rows of data (1) using the fill handle, (2) using Go To > Special > Select Blanks
Just like in Mr Excel's video shared by Andrew in comment (3), I used the right mouse button to drag the selection border to do paste special values at the last step.
If you want to check out my video, visit this link: http://www.youtube.com/watch?v=9TDcVOKbm34&hd=1
I've came across this a month ago, and it really is a gem of a tip!
Thanks. Great tip and useful for a range of excel projects 🙂
Vishy,
When you Ctrl Enter the formula into all blank cells, Excel keeps the formerly blank cells highlighted, revealing the new values.
At this point you can choose to Copy and Paste Special them as constants. All other formulas remain untouched.
BigG,
I was not familiar with that link and I certainly didn't copy the article from it. As Chandoo commented this is not a new technique, and I am hardly the first to have written about it.
@Martin,
using office 2007; you can not copy multiple selection, what version are you using?
Thanks
Thanks, Really nice, really helpful.
wow, how cool is that! Thank you for this tipp!! GREAT!
I thought this was a great tip. I had never done such things with tables in Excel (having only converted to 2007 a couple of months ago, I soon discovered what a versatile tool they can be). So I decided to create my own copy and duplicate the process. Taking it a step further, I recorded the steps in VBA and used those as a guideline to create this simple macro which accomplishes the same function.
Caveat: this will only work when a cell in the table is selected and it will replace ALL formulas in the table with their values.
Sub FillTableBlanks()
' Macro created 20 October 2011 by Jason B White
'Declare Variable
Dim strTable As String
'Get Current Table Name
strTable = ActiveCell.ListObject.Name
'Select Current Table
Range(strTable).Select
'Fill Blank Cells With Formulas
Selection.SpecialCells(xlCellTypeBlanks).FormulaR1C1 = "=R[-1]C"
'Paste Values Of Formulas
Selection = Selection.Value
End Sub
I hope that submitting macros is sanctioned in this forum. My previous post was my first ever attempt at contributing to an Excel blog. And I'm unaware if there is a way to differentiate macro snippets by using tags as I've seen in other Excel VBA forums.
I just wanted to mention that I figured out a way to modify my macro so that it doesn't overwrite ALL formulas in the table, but only those which were filled in by the macro.
Modifying the fourth section (Fill Blank Cells With Formulas) as shown below accomplishes that:
'Fill Blank Cells With Formulas
Selection.SpecialCells(xlCellTypeBlanks).Select
Selection.FormulaR1C1 = "=R[-1]C"
Hi,
I face a similar situation in office and use the below macro after selecting the range of data across which I want to duplicate the data below.
Sub FillBlankCellsSelectionDown()
Dim rAcells As Range, rLoopCells As Range
Set rAcells = Selection
For Each rLoopCells In rAcells
If rLoopCells.Value = "" Then
rLoopCells.FillDown
End If
Next rLoopCells
End Sub
re: paste special -> values
Drag the Paste Values toolbutton on to the standard toolbar next to the Paste button and save a couple of clicks.
Hi everyone many thanks for sharing this solutions but do not work Excel 2003? right? Thanks
@Alejandra:
I know that the macro I created was in Excel 2007. I assume that it's probably specific to 2007 (or 2010), but can't be sure, as I no longer have access to a PC running Excel 2003.
I have to admit that I didn't even realize that tables existed when I was using 2003.
Filling blank cells (cleaning-up the pivot-table aftermath) is one of our "daily-ritual", to dealing with those, we've create a short-cut (one of the many) to very quickly fill-up those blanks.
Basically what we need to do is to select the whole area to be filled-up (with the value above), and click a button, VBA automatically deals with the rest.
We use VBA to handle this problem just as mentioned above by several other people, however, I think we'll also need to consider the extreme (well, actually not that extreme if you're dealing with lots of data on a day-to-day basis) case: that the "blank" cells are highly fragmented, e.g. the maximum "areas" that Excel 2003 can handle is around 6500 (sorry I couldn't find the exact spec).
Thus, in our function, there's another step to cut-off the number of cells going into the "specialcells" function, just to make sure that the function will run in every condition.
I just wanna give a solution to similar problem which i face regularly while copying the data from a pivot as it is. I apply the following solution which i think is the easiest one on earth. Select a cell F2 (considering that column E is the last column filled with data) and type the following formula =IF(ISBLANK(A2),F1,A2). Now just drag the formula equivalent to the length and breadth of the entire range of data which want to fill in this case drag it from F2:I21 , remember do not apply on the cost column.
Now just copy whole new range i.e: F2:I21 and paste special it over the former range A2:D21. That's it 🙂
If u find any problem related to this formula u r welcome to contact me.
thanks martin
This doesn't work in excel 2007. So request to Martin , if he can confirm which version he has used. Guess 2010.
@BK
my method (comment #4) which is almost the same as Martins works on excel 2007... i've been using it since 2007 came out actually.
Excelent trick, thanks Martin.
[...] Quickly Fill Blank Cells in a Table [...]
eXCEEELLTOOOOOOOOOOOOOOOO......!
Many thanks to Martin.
im getting an error no cells were found why is this
Very cool trick!
I'm facing a similar problem, but I'd like to use a formula to pick the first non-empty above the referenced cell, and keep the empty cells empty. Any solution?
Example case:
I've got 3 columns, 1) consecutive dates, 2) my current weight, 3) my BMI. The first data row would be like: A2) jan-1, B2) 70 (kg), C2) =70/1,75^2 (because my height, 175cm, is pretty constant)
Now of course I forget to write down my weight on jan-2nd, so the formula would return 0. If my weight is blank, I'd like to refer to the last 'non-blank' weight (up the list of course, so jan-1st).
The solution on this page would solve my problem partially, but every time I leave cells blank, I have to repeat these steps. A formula would prevent this, AND I can still see which days were actually not filled in.
[…] http://chandoo.org/wp/2011/10/17/fill-blank-cells-in-a-table/ […]
Thxs! Yes, "knew" you could do this with "one" col of data...never thought to try it with >>multiple<< cols...Cool!
Thanks a lot i was searching this thing for many days ,
Thanks a lot to martin
Thanks a lot to martin
Thanks a lot i was searching this thing for many days ,
Thanks a lot to martin
The north, on the contrary, is the land of mighty and sometimes creepy-looking pinetrees, often compared to monsters from fairy tales.
Pages 1 through 3 of the tentative budget are also printed in portrait format so
the writing on those pages is also sideways.
There are occasionally long discussions of the cost of nuclear relative to the cost of renewables in the technical literature.