Let’s take a whirlwind trip to coolest little capital – Wellington. It is a windy place, so hold on to your hats and spreadsheets.
Almost everyone who spends more than 2 days in Wellington would agree that it is a windy place. But how windy is Welly? In this two part series, we will use Power Query, Excel charts and coffee to answer that question.
But, first let’s start with a joke.
What happens when you throw a boomerang in Frank Kitts Park?
You will have to buy another one, coz you are not getting that one back.
Extracting the wind data
In order to understand how windy Wellington is, we need to get average wind speeds by day for last several days. Let’s get the data for last 2+ years (ie from 1 Jan 2016 to 21 Feb 2018).
There are many places where you can collect latest wind data. But when it comes to historical wind data, surprisingly few resources are available. We can use The National Climate database – CliFlo, to gather wind data. But the interface is confusing and I could only locate gust speeds, rather than average wind speeds over time.
We can use wunderground.com to fetch weather data for up to 13 months at a time.
But we need data for almost 26 months.
Very simple, we can query wunderground twice (or thrice), once per each year.
The historical data query URL looks like this:
https://www.wunderground.com/history/airport/NZWN/2016/1/1/CustomHistory.html?dayend=31&monthend=12&yearend=2016&req_city=&req_state=&req_statename=&reqdb.zip=&reqdb.magic=&reqdb.wmo=
All we had to do is, change 2016 to 2017 & 2018 to get respective data.
The actual data set will be a web page. But we can use power query to extract the portion of page that contains weather information.
On to Power Query – Building our Weather Data Extractor
Note: This is a slightly advanced tutorial on PQ. If you are a beginner, start with Introduction to Power Query and work thru examples on PQ tag page before reading any more.
Getting data from the web – building URL in parts
Open Excel and go to Data > New Query > From Other Sources > Web
For Power BI, this would be Edit Queries > New Source > Web
Switch to “Advanced” mode and enter the URL as parts like below. We will switch the 2016 part to parameters soon, so we could get data for any year easily.
In the navigation pane, select “Table 1” which is the weather table.
Set up a parameter for Year
How would we get data for 2017 or 2018? Simple, we use parameters. These are like variables which can be plugged in to any part of your Power Query process.
In Power Query Editor, go to Home > Manager Parameters > New Parameter and call it Year. Enter the default value as 2016.
Now, go back and edit the source settings for the query and replace 2016s with parameter Year.
Cleaning the weather table
Turns out the weather data table is not clean. Although there are 366 days in 2016 (leap year), Wunderground adds headers for each month. So we end up with 378 rows (excluding the header). Each header contains month name and repeat of all the column names. We can extract the month name & combine that with date and year parameter to create the date for each row.
Here is a quick illustration of what we need to do.
But first, rename the very first column
Notice the first column? It is called as 2016. This is ok if we are interested in just 1 year of data. But if we re-run this query with Parameter=2017, our column heading will change. If you have dabbled with Power Query a few times, you will quickly realize that PQ will get in to a nasty fit anytime column headers change and impact downstream steps.
Simple, we shall rename it as FirstCol.
When you apply the new name, PQ will write this M instruction.
#”Renamed Columns”= Table.RenameColumns(Data1,{{“2016”, “First col”}})
This is not a fool proof solution, as when we change parameter to 2017, there won’t be a 2016 column in that new table.
So, instead, we can ask PQ to rename first column of the table.
You can do this by:
- Note: You need “Formula Bar”. Enable “Formula Bar” by clicking View > Formula bar. This way you can actually see all the M code PQ is cranking up whenever you perform some actions on your data.
- Click on fx button on the formula bar to insert a step. Simply type = Table.RenameColumns(Data1,{{Table.ColumnNames(Data1){0}, “First col”}})
- Press Enter
- Bingo, you have renamed the first column of your query to “First col”. This has no reference to 2016 or any year, so it should work on any table you fetch from that weather data page.
Cleaning the weather data – steps
Just follow these steps to clean the weather data.
- Add a custom column called Month and write this formula = if Text.Length([First col]) > 2 then [First col] else null
- Select Month column and Fill Down (Transform > Fill >Down)
- Select First col and change its type to whole number. This will make all month names as Error
- Remove errors from First col (Right click on column header and choose Remove Errors)
- Add a custom column called Date with the formula = Text.From([First col])&”-“&[Month]&”-“&Year
- Change this column to date type.
- Keep only Temp. (°C)2, Wind (km/h), Wind (km/h)2, Wind (km/h)3, Events, Date columns and remove all other
- Rename first four columns to Avg. Temp, Wind Max, Avg. Wind, Wind Gust
At this stage we have one year of wind and temperature data for Wellington. Time to create getWeatherData() function.
Making getWeatherData function in Power Query
Now that we have a parameterized query, just right click on the query and choose “Convert to Function”
PQ will build the function that can take year as input and return a table of weather data for that year (provided Wunderground.com co-operates)
Now, we just need to run this function three times, once each for 2016, 2017 and 2018 to get all the data.
Go back to Excel
Save your queries, but don’t load them yet. If PQ prompts about data load, select “Connection only” and jump to Excel.
- Create a table with 3 rows and type 2016, 2017 and 2018 in that. Call this table Years.
- Load this table to Power Query (Data > From Table)
- Go to Add Column > Invoke Custom Function and invoke getWeatherData function for each year.
- Expand the weather data tables.
- Done!
At this stage, we have data for all 3 years. You can add some data clean up steps if you want. But all the wind & temperature data is here for us to analyze and visualize.
Download Example Workbook
Click here to download the Wellington Wind workbook. As you can see, I have added few more steps in PQ to clean up the data and include a “Is it windy?” conditional column.
Please note that this workbook is designed in Excel 2016. It may not work in older versions of Power Query. You can replicate most of the steps. Try doing it so that you will learn more about Power Query.
In the next part – Wind in Wellington – few visualizations
In the next part of this tutorial, we will build some visualizations to understand how windy Wellington gets and what is the best time to enjoy the beautiful outdoors.
Stay tuned.
How are you using Power Query? Please post about your power query escapades in the comments section. Also tell me how you went about re-creating the steps in this tutorial. I am all ears.
Why there are no undercover cops in Wellington? Their cover was always getting blown. That is why.
70 Responses to “10 Tips to Make Better and Boss-proof Excel Spreadsheets”
Proper print settings on each sheet helps your boss to print the reports quickly without hastling you after printing irrelevant stuff.
It is highly relevant that you print your reports once before circulating it to your boss or other people.
Knowing that what your boss actully look at in the entire report can be very usefull. You can build a good summary of what your boss wants and put that as separate tab in the form of dashbord report, so that your boss does not peep into rest of your work and start pocking you with irrelevant stuff.
You can also put that Dashboard into the email summary and not trouble your boss to open your workbook. This is ultimate boss proof tip and I have been using this for long time now.
Thank you Chandoo. Great checklist to follow before delivering an excel spreadsheet to someone else. Some points you mention are seemingly so simple that we might overlook them - like selecting cell#A1, but they make a difference to the impression the spreadsheet creates at the recipient's end.
Dear Chandoo,
Great tricks.
One trick I use (more and more) is to hide the sheet tabs and to hide the formulabar via the 'tools' 'options' and the 'view'-tab.
Another trick is to limiting the scrolling area to hide all columms (or rows) until the end of the sheet. Select the column, press CTRL+SHIFT+RIGHT, right-click on the column and hide (also possible via VBA).
I was wondering though if 'boss-proof' is related to 'excel-stupid-proof'?
Cheerio
Tom
Just wondering if the hiding formula bar really works when a recipient opens it whose "view-show-Formula Bar" is still checked...
It's saved to the sheet I believe.
Absolutely agree with this post !!!
on the past months, after reading this blog, PTS's and Debra's Contextures, one of the things I've beggining to do as a best practice is to create all my spreadsheets with 3 tabs: data, summary and control, and this last one generally xlveryhidden, and sometimes the data one hidden as well.
And this restrictions are also being applied as best practice, and with a lot of benefits as you well mentioned. Furthermore, if combined with dynamic named ranges, formulae is more readable to users, and the WOW effect is often achieved when the question "How did you do that?" arises.....
Keep on the good posts !!!
Rgds,
Martin
Would you mind sharing an example of this technique?
Is there a way to keep the data in a seperate file rather than the same excel. This way you could keep presentation and data separate. But not sure how you would link up the two excel files
Yes, there is a way but it is not prefered.
I used this a coulple of times, (You need to code).
mail me if you need assistance with some sort
It entirely is possible. The problem comes though, when you share the spreadsheet.
If the recipient doesn't have both files, or access to both, things break when the values try to refresh.
ey, why is the boss a she??
haha - welcome to the future. About time.
Chandoo, one more trick that we could use with the help of VBA, RT click on the View code of the particular sheet, in the properties table set the Visible status to 2-xlveryhidden, this ensures the sheet name does not show up even when the BOSS tries to unhide the sheet from the sheet >> unhide option. Dont forget to password protect the VBA (available under tools >> VBAProject properties.
Very good tips, although I have to say Chandoo, that your cats probably need to be spayed or neutered if they behave like that. =)
Good to see all these tips on a single "sheet", and giving the name *boss proof*, and Dilbert was a great welcome 😀
The best way to "Boss Proof" (and "Self Proof"!!) a spreadsheet is to keep back ups. I use a macro that saves the last 3 significant versions of the spreadsheet all with a date stamp included in the file name.
To quickly select cell A1 on all sheet, use CTRL-Page UP or CTRL-Page down to navigate between sheets and CTRL-Home to select cell A1 (if you have frozen pane, it will select the top left cell of the section below).
Great list. And I follow every single item... I also use a consistent background color for input cells in every report/dashboard. And I use a little VBA to identify the user and change the report accordingly (selecting the right market, for example).
Chandoo, Nice post. I like to use the hidden Paste Picture Link option. Keep the original report you want displayed on a hidden sheet and only show the boss the report picture. Also great to watch the confusion when boss trying to select cells is worth the effort!
I usually save as PDF if there's no interactivity in the report. That way nothing can go wrong 🙂
PDFs work a dream for me too and saves the boss's EA from telling me all the time that she can't print my work!!
@All.. thanks a ton for sharing your ideas. I am thinking of writing a part 2 of this post explaining some of your ideas in detail.
@Bazlina ... I will make sure the boss is a HE in the next post 🙂
"10 Tips to Make Better and Boss-proof Excel Spreadsheets"...
Unless of course your Boss reads PHD !
Great article with one glaring error.
If (like me) the majority of your spreadsheet errors are *caused* by cats, adding more cats is just going to increase the problem.
@Hui you always have a boss, even if you are boss. If you dont have a boss, then may be a cat or even a dog.
@Debra: hmm... Are you sure the cats are not after the mouse? Go learn some keyboard shortcuts.. now 😛
Great Web Site. I've done almost all the above in trying to build my application and it's taken me hours and hours reading my "dummies " book. Thank you for all this information.
Is there a formula I can use that will automatically return to "A1" cell should an associate use the 10 page spreadsheet I have?
Is there a way to set an expiration date on my workbook so that beynd that date no one will get beyond the cover page?
Paul, in all my "user facing" workbooks (those that I distribute) I create a named range called "Home" on the worksheet(s) that are most likely to be used. Then I write a little VBA that selects the Home range whenever that worksheet is activated or on other triggers depending on the context of the sheet. This is more appropriate for the dashboard tabs or summary tabs my job requires.
But I usually set this functionality up early on in the design process so I can take advantage of it as well. I will sometimes assign a keystroke to the GoHome macro.
I'm in the marketing department (aka the picture department) and have to say that the macros/Excel sheets from our controlling department are the worst! They come to me to sort out the mess!!
@Peter: You can try creating a table of contents and then place it on each and every sheet so that user can jump to anywhere from anywhere. Here is a tutorial to help you get started.
Also, You can prevent users from accessing the workbook after a certain date using macros. But users can certainly by pass it by disallowing macros on that workbook.
@Jimmy: Wow... (just kidding) Welcome 🙂
I was recently given a spreadsheet to improve upon.
One of the "boss-proof" actions that the previous author had used was to use data validation instead of protecting the sheet to ward off people changing formulas.
After entering a formula or value into a cell, use data validation to only allow, in this spreadsheet, whole numbers between 9999999 to 99999999.
It's a bit of a pain to actually correct stuff instead of just unprotecting a sheet, but for those that know how to unprotect a sheet, it's a definite way to keep them from fooling with formulas.
Puchu,
We would love to see "Print" in your links section.
It helps us taking prints as neat as your posts 🙂
Chandoo,
I've emailed you a couple of times looking for avenues I need to try to put my workbook on the Internet.
I notice you use PremiumThemes for your Web Site...You must feel good about their service. Do you think PremiumThemes might be an option for me?
Paul
Instead of :
Now Right click and select “Hide” option.
Shortcut can be used : Ctrl+0 (to hide)..
sir i wanted to know,how to hide cells or tab without hiding rows and columns? PLZ TELL ME
Hi Chandoo!
Great tips! Im researching on an excel project now that you can create to "lighten" the size without sacrificing the data inside..
We usually encounter problems with the data, excel file is shared, in a network folder.. and there are 11 people that enters their own productivity in each tab.. however, there comes a time (uncertain) where some of the data they enter either gets deleted or changes value.. could this be a file size problem? are there other ways to create this file that will decrease data inconsistencies?
thanks!
[...] Hide un-necessary rows to create clean looking workbooks (and 9 more tips) [...]
[...] Presentation format: all spreadsheets, should be designed so that it is easy to follow the process flow and result. Almost every spreadsheet should be presentable and understandable to senior management without additional formatting or explanation. (tips: how to design boss-proof excel sheets) [...]
[...] on Excel formatting here: How to make better excel sheets, Formatting [...]
[...] on Excel formatting here: How to make better excel sheets, Formatting [...]
[...] 10 Tips to make better & boss-proof spreadsheets in Excel [...]
You will find another quick and easy technique here:
http://www.onsitetrainingcourses.com.au/main/page_blog_hiding_most_excel_rows_and_columns.html
[...] tips: Learn how to make better Excel sheets Spread some love,It makes you awesome! [...]
Save what you want the boss to see as a PDF. Absolutely foolproof and no cats hurt in the process.
I really enjoyed allot of the tips on here, especially the one on comments on cells. That will come in handy on allot of our projects. I would also like to share on on my little tricks. I am constantly working on several different reports with several different systems and in doing so I am constantly running in problems and my way out of them is simply calling <a href"http://www.reportingguru.com/"> Reporting Guru </a> and telling exactly what I'm going through and they can tell me exactly how to get out.
One of the things I've found to boss proof my worksheets are a few simple VBA scripts to automatically protect the workbook/worksheets, and direct them to the "Quick Look" dashboard page, I hide all of the raw data sheets before saving. The script looks like this:
Private Sub Workbook_Open()
Sheets("Summary").Protect Password:="password"
Sheets("Labor Cost by Site").Protect Password:="password", AllowUsingPivotTables: =true
Sheets("Labor Cost by month").Protect Password:="password"
Sheets("Quick Look").Protect Password:="password"
Sheets("Quick look").Activate
ActiveWorkbook.Protect Password:="password", Structure:=True, Windows:=False
End Sub
I also have a pivot that contains labor cost data which cannot be refreshed while the worksheet is locked.
Private Sub Worksheet_Activate()
Sheets("labor cost by site").Unprotect Password = "password"
Set pvttable = Worksheets("labor cost by site").Range("a1").PivotTable
pvttable.RefreshTable
Sheets("labor cost by site").Protect Password = "password", AllowUsingPivotTables:=True
End Sub
OPPAN GANGAM STYLE!
good
Your post are always with something creative , thanks for sharing this information , your post are worth reading and implementing 🙂 great job
Hi,
I will try to learn every point slowly !
Shokran Chandoo.
Best boss Proofing of sheets is useing indirect(address 😛 this prevents most smartass bossess from doing any actual changes cus the formula will be long and hard to understand for any bystanders..
Also putting the actual calculations on a different sheet can make a sheet bulletproof from bosses.. especialy if you put them in the Very hidden so when the boss learns how to unhide sheets he wont simply find them.
One thing iv also learned is that most bosses is scared of macros that gives "virus" warnings before beeing run 😛 That include the default warning from Excel...
Long formulas or work arounds is best way to go.
What's the best way to amalgamate two existing excel spreadsheets into one?
Two teams use the same format spreadsheets with individual data split into calendar months and I want to make them one without manually entering the data.
Alt + D + D + N
Write a query and viola, Two sheets into one.
Changing the properties of the file to read-only . (While the file is closed, right click on the file and check the read-only box.)
This allows my boss(es) to access the file -- even change it -- without being able to save their changes. If a boss likes his 'new' version, he can save it with a different file name.
But now -- how to prevent the boss from deleting the file altogether? Or deleting the whole network?
Hey man.
Think you can go as easy as to make a shortcut that links to your read only document. Then the boss wont know of the root document. He can figure it out but lets face it. He is a boss and 70% if them wont know squat
Instead of "Hiding" rows & columns, I find "Grouping" works best as its very easy to quickly see if a worksheet has hidden rows/columns. Sometimes hiding a random row/column is not easily noticed and can create issues.
I have one xl sheet with different dates in many columns and one raw's. I want to send this data to another xl sheets for each date. if somebody can help me will be great.
Dear Samantha,
Check out the website of Ron de Bruin. He has a great set of macro's and free add-in that can help you with this issue.
http://rondebruin.nl/win/s3/win006.htm
Tom
Hello, I have just found out that I made a mistake in my spreadsheet: I had a column of negative numbers, but one of them was positive (while it should have been negative). Is there a formula/system to avoid this?
Thanks.
Mariateresa
Yes, data validation. Values you denote would be between -1 and -999,999,999.
Hi,
Hiding any worksheet can be unhidden and messed around easily. I change the visibility in visual basic from -xlSheetVisible to -xlSheetVeryHidden. By this, even if you right click on sheets, you will be unable to find the hidden sheets.
Cool? I think so...
Very informative, Thanks
Is there a way to lock cells in an already protected worksheet.
(Thus the entire worksheet is protected, then the entire office can open it as read only but only a few users have the password to edit the file)
I would like an additional password or prompt box so these few users don't accidentally change formulas.
Itss such as you learn my thoughts! You appear too understand
a lot abnout this, like you wrote thee e-book in it
or something. I fel that you just could do with some percent to presseure the message house a little bit,
but insatead off that, this iis wondeerful blog.
An excellent read. I'll definitely be back.
It is in reality a nice and helpful piece of info.
I am happy that you just shared this useful info with
us. Please keep us up to date like this. Thank
you for sharing.
I laughed out loud reading the 2nd solution about moving to marketing department and making ppts.
I've been using "technical" sheets for a long time already and depending on the audience it is hidden or not. I'm currently in my NO VBA mindset, so the very hidden option is no longer. Using sheets names like: TechnicalCodes; ExplicitVariables;SetUp; HeavyCalc seem to work to my experience as they send along a message "Don' t you mess-up here, you fool!". A "Read This" section or sheet however does not work!
Reading stuff on this site has helped me develop a good habit of using colors and themes to assist the end user in being well-behaved. In my book the best advise here, because it is about the user experience and not only about protection your own work.
For dashboards I get rid of tabs and scroll bars. Besides 2 exceptions, I need to come across a manager who can turn them on again without my help.
Seems that I forgot about protecting cells, sheets and workbooks altogether. Damn!
Thanks for the informative article Chandoo, I've been struggling with Excel lately. It's a powerful tool, but hard to learn for me.
Thanks Chandoo for sharing these excel sheet tips it helps me a lot to understand excel more.
Nice roundup, Chandoo! Here's one more I thought would be relevant:
For Excel 2013+, you can hide the ribbon, as shown in this animated gif: https://gridmaster.io/tips/hide-ribbon-excel-space
This will simplify the interface, making it less likely for people to accidentally make changes. 🙂
THANK YOU SIR
I'm better at Power BI thanks to you!