Howdy folks. Jeff here, bringing you a Public Service Announcement: Thanks to the magic of VBA , Structured PivotTable References are coming to a PivotTable near you!
Structured References for PivotTables? So what? Well, because PivotTables are the best bit of ‘old’ Excel, and Tables are the best thing about ‘new’ Excel, and it’s about time their strengths were brought together:
- Tables magically expand to accommodate anything you put in them. Even better, because of the automated Dynamic Named Ranges built into Tables – called Structured Table References – any Formulas, Charts, Data Validation lists, or conditional formatting formulas that point to that table will instantly be updated with the latest data. And any PivotTables that point to that table will automatically include the new data whenever you refresh them. (Read more here).
- PivotTables allow you to do serious yet effortless number crunching without the need for a single formula. Just as well, because the kinds of formulas you need to replicate what a PivotTable can do easily are often mind-bogglingly complex, and very resource intensive. So using a PivotTable instead of formulas means that people that inherit your spreadsheet are less likely to struggle to follow what you’re doing, and the spreadsheet is less likely to suffer from slow recalculation issues. (Although yes, you will have to refresh that PivotTable from time to time. But that’s a small price to pay.) But there’s a problem with PivotTables: they don’t have any kind of inbuilt Dynamic Named Ranges like Tables do. And so because their structure is very likely to change whenever new data is added or a user decides to filter or rearrange the order or number of field displayed, then any formulas that point at PivotTable ranges will have to be changed manually. (With the exception of a single cell in the Data area referenced by the GETPIVOTDATA() function). So PivotTables are great for getting a result, but lousy for passing on those results to other parts of your spreadsheet.
Who knows why MS haven’t already implemeted Structured Referencing for PivotTables. But why wait for Microsoft to get around to it. Let’s just do it ourselves!
Download the sample file to see my hand-built Structured PivotTable Referencing in action: DynamicPivotRanges_20141019 unprotected
Open it, enable macros, and you’ll see a PivotTable like this:
Now, click on the arrow to the right of the name box, and you’ll see this:
As you can see, in my implementation of Structured PivotTable References, the automatic name that gets generated is prefixed with the Sheet name for uniqueness and uses a period to separate the Sheetname, PivotTable name, and FieldName. So it differs slightly from the notation that Tables use. But it’s every bit as handy.
For instance, check out what happens when I start typing a formula somewhere:
Awesome: That’s pretty much the same kind of thing I get when I want to reference a Table:
Let’s see if it handles changes in the structure of a PivotTable, shall we? Here, the Pivot is filtered in such a way that only 5 rows of data are returned. I’ve selected the entire State region, so that you can see that this corresponds with the automatically generated Structured PivotTable Reference shown in the Name box:
If I change the City filter to include additional cities, then the data returned grows by a few rows, as you can see below. Check out how the Structured PivotTable Reference automatically updated to accommodate the extra rows:
…and if I change the layout of the PivotTable by bringing in a new field – such as the BloodType field shown below – then as you can see, the Structured PivotTable Reference picks up the change too, and recognizes that the State field has shifted to the right:
If you change the Sheet name, then the SheetName part of the Structured PivotTable References syntax get updated next time the Pivot gets refreshed. And if you change the PivotName, then that part of the Structured PivotTable References syntax gets updated immediately. Unfortunately the same doesn’t occur for changes to PivotField names. So if you change the name of a field, any formulas pointing at the associated Structured Reference will need to be updated. This is shown below:
So there you have it: a proof-of-concept implementation of Structured PivotTable References. I’ve been using this to create complicated non-PivotCharts from Pivots, such as ScatterPlots (which are not supported in PivotCharts), or to serve up data labels to non-Pivot charts. And also to avoid having to have lots of extra formulas down the side of my PivotTable just to handle growth.
Take it for a spin, and let me know your thoughts and suggestions for improvements in the comments. Who knows…someone at Microsoft might even see this, and think “Now why didn’t WE think of that?”
What other functionality is missing from Excel that you’d like to see added?
While the things that Excel can do are cool, Excel often makes us jump through an awful lot of hoops – and click through an awful lot of dialog boxes – in order to actually do them. At the same time, there’s lots of things we routinely do that Excel simply doesn’t provide tools handy tools for. The end result is this: for every millisecond that Excel actually does some real work, we’ve probably spent hours ‘prepping’ it to do it.
Whenever we have to do lots of manual steps in order to leverage Excel’s cool inbuilt functionality, then Excel is programming us. It’s like some kind of epic experiment in behavioral psychology; and we’re the mice. It should be the other way around.
Fortunately, VBA (Macros) gives us the means to program Excel so that it behaves like we want it too. So if there’s something you would like to see added to Excel, let us know in the comments. We’ll see what we can collectively do to make Excel even greater than it already is!
—Edit—
My pal Doug Glancy actually wrote a post on how to do this back in 2012, on my birthday no less. I’d clean forgotten about that post. So be sure to check out Doug’s implementation of this too.
About the author
Yep, that’s me all right. Jeff Weird. Excel Madman.
If you liked this post, then you’ll love my upcoming book: Excel for Superheroes and Evil Geniuses. Keep a lookout for it in early 2015, and check out my posts on this blog or over at Daily Dose of Excel to get a feel for what kinds of things I’ll be covering. The book will give users an excellent overview of how Excel works under the covers, and what tools the interface puts at their fingers right out of the box. And it will ship with free code that will add amazing new features and functionality to Excel. You’ll be an Excel Evil Genius in no time!
Stay tuned…
44 Responses to “Introducing…Structured References for PivotTables”
Would love to see the code ... but it's PW protected.
Very slick, though.
Hi Jeff ,
Wonderful idea ; thanks for sharing.
Whoops, didn't mean to password protect. Have amended.
Very nice, Jeff! I agree - why didn't Microsoft think of that?
Very cool Jeff! I like the concept and similarity to structured reference behavior with the named ranges.
I think the CUBE functions are probably the closest thing you're going to get to a built-in functions in Excel. Of course the CUBE functions don't work with standard pivot tables. You will need a data model like PowerPivot. But the CUBESET and CUBESETCOUNT functions will pretty much do the same thing for the data in the row and column areas of your pivot. When combined with the CUBERANKEDMEMBER function you can really create some complex filters and interactive reports.
The CUBEVALUE function does similar calculations for the fields in the values area of the pivot.
I agree that it would be great if this functionality was extended to regular pivot tables. I believe this would be the easiest implementation for MS to use existing functions that integrate with data models. It's all there, but requires you to create a pivot table using PowerPivot. Once created you can use the CUBE functions anywhere in your workbook. This integration will probably become even easier in the future as PowerPivot becomes more baked into the application.
Thanks for sharing the code BTW!
No sweat. THanks for the comments.
Good points Jon. Excellent code Jeff.
Regarding CUBE functions, in Excel 2013, we can add any pivot table to data model (at the time of creation of the pivot table) and technically they will be then accessible thru CUBE functions. Also, don't forget the trusty GETPIVOTDATA().
Good idea Jeff. Here's a post with a downloadable workbook that I did on this topic a while back:http://yoursumbuddy.com/create-pivot-table-named-ranges/
Hi Doug. I'd completely forgotten about that post, despite your reminder! I love your line Programming pivot tables is fun. The extensive object model is a VBA wonderland with treats around every turn.
More like twists around every turn, I'd say.
I'll download your workbook and take another peek.
I like your method for regenerating the names on the fly, Doug. I use a similar method for creating summary tables and the like. It seems to simplify code and add a bit of automatic error correction.
@Martin, thanks. Yes, a while back I realized it's cleaner to start from scratch each time with some activities (populating a combobox leaps to mind). It's often way easier than trying to adjust a bunch of existing items.
Thanks Jeff. Happy birthday :).
Did you do the artwork on your book cover? http://www.amazon.com/Excel-Superheroes-Evil-Geniuses-Jeff/dp/1615470379
I didn't even know that was up yet! I better get cracking and finish it.
No I didn't draw that...it's actually a picture of me and the wife. Now stop ogling my wife, you.
One of the cover ideas I pitched to Bill Jelen on a bad day was "How to absolutely murder Excel":
Bill reckons only a Psychopath would buy it. I told Bill that some of my best friends are psychopaths.
When I try it the workbook downloads as corrupted.
Hi Jomili. What version of Excel do you have?
I'm on 2010. I downloaded three times, about 10 minutes between tries. All 3 were corrupt. Two hours later I tried again, and that time it worked. Must have been something here.
Holy sheet!!!! 🙂 Jeff, this is flippin' AWESOME!!! I have shied away from pivot tables of which I was a big fan of (ever since Tables came along) simply because of that very singular reason - pivot tables stink at passing values to other parts of the workbook.
I'd write more but I've got to go experiment with this! Seriously - thank you! You have no idea how much work this could save me and I really appreciate your posts!
Cheers,
Gino
Thanks Gino. Your kind words are my oxygen!
You asked what we would like to see in Excel. Here are two things that have come up lately.
1. Published Worksheet HideColumns and HideRows events or, at least a HideColumns events that triggers the Worksheet/Workbook Calculate events, just like the unpublished HideRows does.
2. Handling comments like they are in Word. In Word, a Comment can include an in-line image (e.g. a BMP file). In Excel the best you can do is make such an image the background.
Great ideas, Don. I'm working with a smart developer to add functionality to Excel. We'll put our collective thinking caps on re your suggestions.
Hi Jeff,
When comments are defined in these lines a strange effect occurs in my Excel 2010:
ActiveWorkbook.Names.Add Name:=strNewName, RefersTo:="=" & strAddress
ActiveWorkbook.Names(strNewName).Comment = pf.SourceName & "|" & pf.DataRange.Address
When "Centimeters" are added, last line changes the Structured PivotTable References syntax to R1C1 format: =Sheet1!'F9C4':'F16C4' when the comment is: Centimeters|$D$9:$D$16
If I use that named range in a formula a #NAME error is returned.
To fix it, I've added RefersTo another time at last to change to: =Sheet1!$D$9:$D$16
ActiveWorkbook.Names.Add Name:=strNewName, RefersTo:="=" & strAddress
ActiveWorkbook.Names(strNewName).Comment = pf.SourceName & "|" & pf.DataRange.Address
ActiveWorkbook.Names(strNewName).RefersTo = "=" & strAddress
Does it happen to anyone else?
Thanks Pedro. I'll take a look, assuming I still have Excel 2010 loaded on a machine somewhere. Appreciate your troubleshooting.
Jeff, I like troubleshooting and I also really like your awesome contribution to this subject.
One more question, when some range changes in the structure of a PivotTables. we also need to update the comments of the referenced structures.
' The PivotField range has changed, but the name remains the same
' So all we need to do is update the RefersTo address AND THE COMMENT
ActiveWorkbook.Names(strNewName).Comment = pf.SourceName & "|" & pf.DataRange.Address
ActiveWorkbook.Names(strNewName).RefersTo = "=" & strAddress
I put the comment before the Refersto address as a workaround of the strange Excel 2010 effect reported before.
This macro I keep it in my main inventory.
Pretty cool, Jeff. Pivot tables are lousy at passing on data. Being able to interrogate a pivot table with structured references would be awesome.
Hi Dave. You're not wrong. It's their Achilles Heel.
And I absolutely agree with your reply to my comment on your outstanding article that despite the fact they are so much simpler than actual formulas for basic number crunching, so few people use them.
What I feel really surprising is the amount of times on help forums where the OP asks for a formula solution is given some horrendously complicated formula that solves it, when a mere PivotTable would do it with ease without the complexity. In fact, I did a blog post recently over at Daily Dose of Excel called Formulas? Pah! on that very subject. There was some great debate at the original Stack Overflow thread I reference in that post as to whether one proposed formula solution is awesome, potentially obfuscating, or incomprehensibly mutant. And yet a PivotTable would solve the problem nicely.
Nice to see you back Jeff - and on my birthday no less.
One thing I've always thought would be useful is if the amounts in the Status Bar could be copied. That way, if you quickly wanted to paste the value of the sum of a range of cells into an email, you could highlight the range, right-click on the Sum area of the Status Bar and copy the value. This would be true of any of the calculation types in the Status Bar.
Hi Roan. Happy belated birthday. Hey...good idea. Dick over at Daily Dose of Excel did an article some time back about subtracting cells in the status bar that might be useful to you. But your idea is a great one. I'll put my thinking hat on.
[…] Jeff Weir is working on a system to create structured references for pivot tables. […]
It seems to me that Excel 2013 (and maybe 10 also) lost the ability for the dynamic pivot table input range that it had in Excel 2002 or 2003. Then you could add a $ before the column name of your beginning input field, a ":", and a $ before the last input column and you could add any number of rows to the input range and the pivot table would automatically pick them up. Am I wrong? If not, why did they change this useful feature?
Chuck,
You can still do that in 2010. I prefer to use Tables as sources. They're dynamic and have other nice features.
They’re dynamic and have other nice features..
Just like you, Doug.
Snort! Why thank you.
Jeff - I downloaded the file (a year too late, maybe), but it doesn't have the macro you referenced, and the layout is slightly different than the screenshots. The macros included are called "Cleanup" and "test," and I don't see the structured references for the pivot tables anywhere. I'm very interested in using this because I reference pivots tables a lot in my work, and this would be super handy. Is there somewhere else I would be able to get the file?
Yvonne: there are a lot more macros in the file than those two. Those just happen to be the only two macros that you can see from Excel. And they don't actually do anything...I just forgot to delete them from the file before I saved it.
The macros are triggerred automatically by event handlers saved in the 'ThisWorkbook' module you'll see if you open the Visual Basic Editor. You have to have a bit of understanding about how to program to reuse this code in your own projects.
That said, I'm developing a commercial addin that lets non-coders set up code like this just by pointing and clicking. It's not quite ready yet, but I'm getting close. It lets you do this plus a whole lot more, including letting you filter Pivots or Tables from external ranges, or invert filter selections, et cetera. Will probably retail in a 'Standard' version for USD $85 and a 'Premium' version for USD $100.
Jeff,
Not sure why I didn't spot this sooner, but I'm consistently getting an error whenever I move a field. For instance, if I move "Blood Type" from the "Row Labels" field to the "Report Filter" field I get a "Fix_Name = Error#40040, Application-defined or object-defined error". Not sure how to track it through the "Fix_Name" function to fix the issue.
weird, I don't get the same error. Can you save the file and flick it to me at weir.jeff@gmail.com and I'll take a look
Curiouser and curiouser: yesterday I was working with copy I downloaded months ago. When I noticed it was having an issue I saved a new copy from your link on this post, and it had the same issue. I have NOT rebooted, but this morning both files are working fine. So strange.
Add the missing
"Application.Calculation = xlCalculationManual"
at the beginning of the code and that´s all 😀
Thanks Jeff for your work!
Amazing stuff Jeff! One quick question.. I want to use this permanently and I feel that the only way to do that is via an addin (right?).. Can you please provide a code which I can directly put in my addin?
Hi Suril. Funny you should mention an Add-In, because I'm working on several commercial addins at the moment to do with PivotTables, and one of them is exclusively about this functionality. So I can't provide code publically, I'm afraid, as that undermines my business model. But the addin will be pretty cheap, and will let you do more stuff than the above simple code allows. For instance, it will handle more complicated PivotTables that have both Row AND Column fields, and a few other things.
It's not quite done yet (and I'm working on several other addins that are slightly higher up the priority list) but I can let you know when it's ready if you like.
Hi
I was wondering if this helps with column size auto resizing when you have only one row variable and value setting?
Thanks
@Jeff Weir, I know it's years later but here's to hoping you can do an article on how to write the vba and create the named ranges for the above article. I'm in dire need of your assistance.
Thank you
Hei Jeff,
this is really what I want to use, awesome pick. I copied the workbook code and module code, but the magic does not work i my XLSM file. What next is necessary for the implementation?
Thanks, Jiri