This post is from GuestBuster Jeff Weir in our Chart Busters series.
Note: The post slightly longer, but worth every word. Just get a cup of coffee and soak in to this visualization goodness. (Also, click on any image to see its full version)
Over at the FlowingData blog, they’ve been talking about this pretty slick looking Choropleth Map that shows how Medicare returns vary across the United States:
The above shows total Medicare reimbursements in 2006, either by Hospital Referring Region or by State, depending on the radio button. Using the dropdown box, you can change it to this:
…which is how the data looks if you overlay it on a Giraffe. Oops, I forgot to rotate it before saying that. Bear with me a moment…
There. See the Giraffe now? Good.
A picture is worth a thousand words, or so they say. But is a Choropleth worth the many line charts and clowns that you could squeeze into the same valuable screen real estate? Let’s find out, by evaluating what this particular chart does well, and what it does poorly, and whether other charting methods might better convey its information.
Words and music.
Right off the bat, there’s a simple way that the authors could improve this chart. While they include a description below the chart to point out what the data is, and where it came from, they miss something just as important…what they concluded from all of this. So before we consider adding – say – bullet graphs, let’s consider adding some bullet points. A few sentences can tell readers important stuff that would otherwise remain hidden in an undownloaded PDF report. Insights like:
- Care is often better in low-cost areas.
- Growth in returns are only partly explained by advancing technology, and
- Differences in growth rates across regions seem largely due to discretionary decisions by physicians that are influenced by the local availability of hospital beds, imaging centres and other resources-and a payment system that rewards growth and higher utilization.
Straight off the bat, this would make the graph a better graph…without even messing with its form.
But mess we must…
…because lurking below the chlorophyll green of this Choropleth Map are a few serious charting oversights. Ready? Let’s check ’em out.
Scale? Fail!
First, check out the legend.
Crikey…its bands are as discrete as Bruno. Its scale is about as even as my temperament. It varies about as much as =RANDBETWEEN(PaydayBankBalance, UsualOverdraft).
If you fire up Excel and look at the spread covered by each range, you see just how arbitrary the different price bands are:
Whoa…the spread of that $9k to $16k band is nearly 15 times larger than two of the other bands. That can’t be good, can it?
Nice profile
If you were to graph financial spread of each group against the aggregated number of Hospital Referral Regions that fall within each spread, you get something like a histogram. The difference between the sizes of these bands is about as different as the number of performers on stage at a Bob Dylan concert in 1964 compared with 1974. See for yourself:
Oops, wrong graphic. Try this:
Normally histograms have equal widths for each band, but here I want to highlight just how unequal the bands used are. Plus, this lets us regroup the data into evenly spread $1k bands, and overlay it on the first distribution, to see how it compares. Here’s one that I prepared earlier, with the red line as the regrouped data…
Vastly different picture isn’t it. The red is kinda like Data Pig’s heart rate before he eats chocolate covered bacon on Saturdays, and the blue is how his ECG would look when he’s in the ambulance, on the way to the hospital.
This makes it very hard to answer that important question “…compared to what?” With such different sized bands, how can we compare one to another? How can we be sure that the distributions within each band will even allow us to?
For instance, take the highest band spread of $9k to $16k: without any further information to go on, we might assume that the median (i.e. middle) value for districts in this category is midway between the $9k to $16k boundaries, like this:
But that’s like assuming that Simon and Chartjunkle (oops, Garfunkel) have equal talent. We’d be wrong. Verywrong. In actual fact, there’s only three data points to the right of our guessed median line. And as for the 55 hospital regions in Group Five that fall to the left of it…well, they all get tarred with the same brush those worst three performers. The actual median for this group is a lot further left, as shown below:
This means that over half the data in this 5th band actually falls much closer to the far left of the graph than to the far right of the same group it’s been placed in.
You can see this better if you add a one-dimensional strip plot above the graph, which gives an idea of where the 300 odd values fall within the entire range:
Whoa…looks like we’ve got a few outliers to contend with.
What a State we’re in…
This seemingly arbitrary ‘bucketing’ effect is exacerbated when aggregating the different hospital regions into State-wide totals. Except this time regions are being penalised by arbitrary geographical boundaries, as well as the arbitrary financial ones above.
Take Texas for example. Aggregating everything up to the State level, Texas appears in that highest band. Yet at the Hospital Referral Region level, one third of its 22 different hospital fall below the national average, and the median for the whole State is around $8,800. So we better be careful making assumptions from a State-wide view, because the Choropleth averages some very diverse costs over some very large chunks of real estate.
To see just how diverse, let’s rank the entire US values from smallest to largest, and highlight where the Texas readings fall within that range:
What can we tell from this? Firstly, nearly all regions nationwide fall between $5k and $10k. Secondly, there are a few outliers that really skew the picture at the high end. Thirdly, in the Texas case, the State average is boosted somewhat by 3 Texan districts that happen to be among the worst 10 culprits nationwide – one of which is clearly an outlier at $15k. Unfortunately for the lower cost Texan regions, they’re guilty by geographical association…kinda like being kidnapped and held for a zillion dollar ransom, just because you happen to live in the same State as Bill Gates.
So what do we get by aggregating to State boundaries? Probably more blurring than insight. After all, what good would a weather report be to Texans if it only reported the average weather they could expect as a State! Instead, it’s better to keep the aggregation at the Hospital Referral Region level. That way, we can look at this:
…and ask things like “Wow, why such a difference between Waco and the surrounding bits of Texas?” and “What the hell is Alaska doing there?”
Legends in the making…
What’s far worse that this though is that when looking at the State-wide map, the legend is now really, really wrong.
Here’s the legend next to the actual State-wide figures, for comparison:
Whoops…the graph title has changed to reflect we’re now looking at Medicare spending per beneficiary per State; i.e. State averages. The legend is still looking at Hospital Referral Region averages, which have a much greater spread. For instance, the Choropleth shows six States as being dark green regions, and the legend says they fall somewhere within $9k to $16k. But the actual data shows they fall in a $9.4k to $9.6k range. Oops! Slight misrepresentation, there.
How to fix it
Obviously this graph really should use a quantitative scale with equal increments; one that changes to reflect the selection that users make. What’s more, colors should have just enough variation so as to highlight any important differences, without being overwhelming or mistaken for camouflage.
But is a Choropleth Map the best way to present this data in the first place? If you want something for people to play with online, then maybe…but if you want to compare things very closely to other things, then maybe not.
For sure, a Choropleth Map looks cool, and it has what Tusha Metha calls “natural context”. But from an analytical perspective, a Choropleth only really reports how one thing changes with regards to geography. If geography is a major determinant – or if you want to show people how things look in their own back yard compared to others – then perhaps this is the piece of kit you need. But if there’s other factors that have much more sway on your data than geography, then perhaps not. For instance, we might want to see whether population density plays a significant part in Medicare returns, given the likely economies of scale from providing healthcare to densely populated regions vs. urban regions. Nows the time to break out a scatter plot:
Hmmm…looks promising. (Note: I’ve used State-wide data for the above…ran out of time to track down densities in the different Hospital Referral Regions, which is what I’d prefer to do.)
Or we might want to zoom in on the best or worst offenders, and see just how different they are to each other, and to the median value:
Conclusion
I think a better, fairer Choropleth Map at the Hospital Referral Region level would be interesting. But I don’t think it would be enough. To quote from Stephen Few’s latest book Now you see it: “Color is good at drawing your attention to something if used sparingly, but is one of the ‘pre-attentive attributes’ that is not quantitatively perceived in and of themselves”.
Whereas lines and 2D precision are very precise ways to encode quantitative values.
So when it comes to answering the ‘Compared to what’ question, I don’t think you can beat this:
Choropleth Maps in Excel
For information on the implementation of Choropleth Maps in Excel, check out Tushar Mehta’s excellent resources.
For more information on the pros and cons of Choropleth Maps, check out the Clearly and Simply blog, where Robert has built on Tushar’s excellent approach to produces some great downloadable templates. He also offers advice on potential drawbacks of Choropleth Maps, such as:
- No visualization of development over time
- No information on exact values (unless you are implementing tooltips including the data)
- Very limited direct comparability of the regions
- Possible perception problems with regards to the size of regions (e.g. Rhode Island on a US map)
- Possible misinterpretation because the size of a region may have a greater impact on the user’s visual perception than the intensity of the fill color
- Requirement of real estate on a dashboard
His recommendation: carefully consider whether or not a Choropleth Map is the best visualization for your purposes. Check out his dashboard of Lithuania at a glance to see how he mitigates some of the potential problems by incorporating other graphs into the display.
I used Robert’s template to produce this State-wide Choropleth Map of total Medicare spending per enrollee, 2006 using the same Medicare ranges as the Choropleth that’s the subject of this post:
…
…then I replotted the graph using data that had been regrouped $1k bands:
While I don’t advocate this approach, it’s interesting that even though this is aggregated to State-wide totals, you can see significant differences between the graphs.
Right, that’s it. I’m off to the Hospital to see someone about my writers cramp…
About the Author
Jeff is a Business Analyst from Wellington, New Zealand who has recently discovered a strong interest in Data Visualization. He swears by Edward Tufte and Stephen Few as much as he swears at Excel 2007. He’s so new to advanced Excel, that 2 years ago he had to ask a work friend what the dollar signs in $A$1 meant. Now that he knows that, he’s trying to find out what the dollar signs in $A$2 mean.
Note from PHD:
Thank you Jeff. Your passion and knowledge is truly outstanding. I have a whole pack of donuts waiting for you.
28 Responses to “CP024: Customize Excel to boost your productivity”
I think that the most useful customisation that I utilise frequently is some code I found that allows multiple items to be selected from a data validation list. For example one workbook we use lists the business drivers for a project. Another uses this technique to select the staff involved in a particular project. See http://blog.contextures.com/archives/2009/09/18/select-multiple-items-from-excel-data-validation-list/
Another terrific podcast Chandoo! Glad to hear that you, your family, friends and community are recovering well from HurHur.
You did a great job of covering so many of the customization techniques that I use but one that I really like is the addition of those top portion of the quick access tool bar with items that I use often but are not standard to any of the tool bar options.
Thank again Chandoo for the terrific website! It definitely is helping to make me awesome in Excel.
I customise excel toolbars by inserting an icon and dragging it to the appropriate area.
I prefer not to customize (too much).. I use Excel on a work pc, on two pc's at home, and sometimes on other PC's (even on different languages). I've found that if I keep the most "standard" look, I manage to get around on any pc, whatever the language pack.
The only thing I do is "pinning" the most used files on the Windows taskbar. And even then I sometimes forget what I placed where, and look it up manually anyway...
Hi, Chandoo. Thanks for the complete review of customizations. I use most of the items you mention, such as Options to set the default font to “old” Arial rather than Cambria. I’ve also adapted the QAT, adding a button to Repeat last action as well as ones to close the current workbook and the Excel application. Lastly, I developed a personal Add-in that contains macros and changes the ribbon (via Microsoft’s Custom UI editor) to rearrange buttons for functions I always use and adds a new ribbon tab with buttons for my personal macros. For example, one of my most frequently used macros deletes all rows below and columns to the right of a selected cell and resets the Excel used range value for the worksheet (helpful for when I press Ctrl-End).
Thank you again for another great podcast! I always learn something from your podcast/Blog/site. I can't believe I never bothered to look at setting the number of sheets excel creates when you open a New workbook. That will save me time everyday!
I usually create a Personal Macro set to use, as well as creating some custom ribbon and toolbar sets to make my most needed functions handy (I even include some personal macros in these toolbars.) I also spend some time creating some themes to match the Company reporting colors so that my work can easily be incorporated into company communications.
Thanks for helping me be awesome at Excel everyday!
Hi Chandoo - excellent podcast. At the moment I only use the quick access tool bar to add not standard icons. But now I have several techniques that I am going to try. Thanks
excellent podcast!! not enough users maximise their productivity by making Excel more comfortable to work with!!
I use many of the techniques already mentioned (personal macro book, customised QAT/ribbon, etc) but but, instead of adding lot's of buttons I find it's better learn and use the relevant keyboard shortcuts instead of adding buttons for all those functions you use all the time (avoid button overload!!).
Far better to add buttons for functions you don't use often and that are difficult to find/use in their regular guise
I also use a default 'dashboard' workbook that has hyperlinks to a handful of files I use regularly.
also maybe worth mentioning creating default workbook/worksheet 'styles' - to do this create a workbook with the formats you want (fonts, page setup, # of sheets, etc) and save it to your excel start folder with the name BOOK (use SHEET for the sheet template). From then on all new books/sheets will have the same basic formatting
Thanks for the podcast Chandoo, glad you are recovering from the typhoon experience.
Here at work we created 2 template files, a "simple" one and a "large" one. We often have to put together engineering calculations and having these templates makes it so much faster. The simple one has a cover page (used for tracking revisions) and then a sheet set out in our standard format. The large one has the same cover page, then a contents page that has macro buttons to jump to any of the 10 tabs set up in it. And yes, we often use 5-10 tabs when doing our calculations! This makes the navigation much easier and means we can get on with calcs and not have to spend time formatting.
Thanks and enjoy, Oxi
Hi Chandoo – I liked this episode. So far I only use quick access bar to add icons for Macros, special paste etc.. Looking forward to explore some of the techniques shared by you. Thanks
Thanks for the podcast. There's a couple of things I'll have to try.
I've set up a default workbook that's saved to my excel start directory that contains:
* corporate colours and fonts
* a cover sheet
* modified and expanded cell styles
* default table and pivot table styles
* custom page setup settings
Another thing I've tried is creating chart templates which saves a bit of time.
My Quick Access toolbar is jampacked with macros:
- Save all open spreadsheets
- Backup the active workbook to another folder. Useful if I'm making big changes to the spreadsheet that i might want to revert later.
- Copy & Paste special values. Why click two buttons when you can click one?
- Set my preferred number formatting
- Format graphs in company colours
- Apply red/green conditional formatting for growths data. 0% growth is white, 20% growth is green, -20% growth is red
- Format and apply settings to pivot tables as I like them. E.g. Tabular format, no retained items, no style, etc
- Sort items in a pivot filter. Standard Excel capability only lets you sort items if they are in rows or columns not in a filter
- Delete all custom styles. It amazing how many junk styles spreadsheets can accrue, especially when copying in data from other spreadsheets
- Resize and reposition all comments. I swear comments have a mind of their own, constantly moving to bizarre places.
- Print in a Userform all external links and whether the links are active. Including those in name manager and in pivot tables that don't show up in the 'Edit links' window.
- Print in a Userform all HIDDEN name manger names. As a company we need to be careful about what is exposed when we publish spreadsheets.
These (and more I've not mentioned) save me countless hours during a normal working week!
Thanks for a great podcast. Really useful tips on customisation. The tip on only one sheet instead of three - using that now myself.
Personally I use customisation for two main purposes. Quick access to functions I use regularly – to state the obvious – but also some functions I use so infrequently I may not be able to quickly find them when I need them. Forms, for example.
And if I ever need to use a function or command that I have recently learned or used in a new or different way, I add that to the Quick Access Ribbon.
Hi Chandoo,
On top all those customization you've mentioned, I also use the Custom UI Editor for Microsoft Office to create my own Ribbon menu buttons in Excel 2007 and hide the default ones. With the help of macros of course, this makes Excel looks and behave like an App.
Try that to impress your boss !
Chandoo,
Until recently, I had only created custom VBA to solve specific problems related to work. I usually placed a shortcut in a new ribbon. Now I am going to use some of the tips that you mentioned to customize one ribbon for all of the common items that I use (like filter, sort, number format, etc.)
Thank you.
Hi Chandoo,
I really like your podcast. I will appreciate if you continue to do podcast in the future.
Regarding my customize of excel, I usually keep lot of commands in Quick Access Toolbars (QAT), esp, insert/delete sheet row, insert/delete sheet column,copy, cut, format copying, paste, paste special, paste value, Font color, Fill color, camera tool and open recent file etc. Also I click on click on Developer tab so that it is visible on my tabs.
When I was studying MBA, I used to do add-ins of Analysis Toolpak but I have not used that Toolpak since I completed MBA.
Thanking you,
Sam
Hi Chandoo,
I really like your page and it's the first time I listen your podcast and it's great!!. I use a lot the Quick Access Toolbar and also use personal macros for rutine task and for look-and-feel in corporate colors, but after listen you I will try to costumize the ribbon and also I will work with themes and styles.
Thanks to inspire us!!.
Regards from Mexico.
Uriel
Thank you Chandoo, very useful.
I have done 3 things to customize my excel,
Created a new tab and put in it all my frequently used commands.
Recorded all my repeated activities as macros and I am running this everyday.
Added the commands I use very very frequently (including the recorded macros) to the QAT.
Thank you again
Hi Chandoo,
Great podcast. I usually do the following to customize my Excel:
1) I use custom themes and chart & slicer templates
2) Custom macros in QAT and Ribbon for repetitive tasks
3) Keep generic VBA routines in a module for use when needed for specific workbooks, I can just copy it into the target workbook.
Thanks,
Dave
Hi Chandoo,
I have been using the quick access toolbar and it is great for having frequently used features all in one place. As well as macros and those less commonly known features, such as select visable cells when working with hidden rows/columns. The list of all commands available to add to the toolbar is mind numbing.
I did not know about adding your own tab to the ribbon. I have already begun contructing my own.
Thank you for the wonderful information.
Chandoo,
I mainly use the personal macros. There are a few charts that I update frequently, this requires that I keep the old data and change the range of the chart. Changing the range causes Excel to pick all new colors for my chart. So, one macro reverses this issue.
Thanks for sharing your awesomeness.
My customization:
I have three add-ins: Macabacus light for shortcuts (free), XL Campus to work with lots of tabs and the Tableau reshaper add-in (free), to reshape tables in database form.
I have created and copied macros for:
Clean formats
Search and select constants
Format as constant (I used to do financial modeling)
Trace dependents simultaneously
Center across selection (not used anymore since it's in one of the Add-ins)
Search and select blanks cells
Select cell A1 in every sheet (before closing the file)
Protect and unprotect sheets and workbook
Almost all macros have shortcuts and a dedicated tab in the ribbon.
As for Excel options themselves:
Automatic calculations
Error tracking disabled
Cursor stays in same cell when Enter is pressed
And I used to have a Financial modeling template as a predefined sheet where different formats and styles were pre-established
Dear chandoo
Thanks for the excellent podcast. I regularly visit your website and have listened to all your podcast which are very informative. I customise my toolbar by using a quick access and inserting icons in toolbar.
i used all of you suggestions, but have refrained from pinning files to the Recent Workbook List. It bothered me that this pinned files also took over the keybord shortcuts for the last opened files. If I press ALT-F+1, i want to open the last used file, not the file i pinned first.
I missed a very powerful Excel customization option in your podcast, which i use every day and saves me hours: custom keyboard shortcuts. There are two of them:
1. Assign a Key combination to a macro (e.g. "Ctrl-Shift-V" for my VBA-procedure "PasteSpecial_Values")
-> Open the Macro-dialog (Alt-F8) and go to Options, set the key...
2. Assign a Key combination even to F-Keys (e.g. "F5" for "my_favorite_Macro")
-> Open VBA-Editor (Alt-F11), go to any Module in your Personal.xlsb and add this code:
Sub auto_open()
Application.OnKey "{F5}", "my_favorite_Macro"
End Sub
There is no quicker way to call your macro!
Hi Chandoo, very excellent podcast.
I use the quick access toolbar to link my VBA macro's and at the moment I'm experimenting on adding a personal ribbon tab with personal ribbon buttons. I use a Custom UI Editor to do this, it let me link macro's to the buttons used in the personal ribbontab.
This way for a project dashboard I can add the buttons necessary there and they are attached to the file. So other people opening the file have the same ribbon tab. Looking professional and to quote 'awesome' ;-).
Thanks for your prodcast.
I customize my excel in the ribbon. This allows to feature the items I need
Hi Chandoo,
Would you please link me to your Excel Customization Handbook? I am sure that this will help me in a huge way!
Hi Chandoo!
I would love an Excel Customization Handbook! Here is my tip:
To recreate the CTRL-HOME functionality on Excel for Mac copy and paste the following code into your personal macro workbook. I assigned my code to the hotkey Option-Cmd-G and now it's like I have a "Home" button on my Mac!
----------
Sub GoHome()
'
' GoHome Macro
' Simulates Ctrl-Home on a PC
'
' Keyboard Shortcut: Option+Cmd+g
If ActiveSheet.Type = xlWorksheet Then
With ActiveWindow
.ScrollRow = 1
.ScrollColumn = 1
.ActivePane.VisibleRange.Cells(1).Select
End With
End If
End Sub