Howdy again, folks. Jeff Weir here, borrowing the keys to the blog off Chandoo again. (Hopefully I don’t scratch it again).
How remiss of me…jacob reminded me in the comments of my previous BLOATED post on good spreadsheet anti-bloat practices that I completely missed one of Excel’s newest and most efficient anti-bloat features: PowerPivot. So today’s post is going to rectify that. In less words, I promise.
Does my data look flat in this?
In Rob Collie’s excellent book DAX Formulas for PowerPivot – the Excel Pro’s Guide to Mastering DAX, Rob makes the point that before PowerPivot came along, Excel pros spent lots of their time ‘flattening’ data in order to feed their pivots. In fact, many Excel Pro’s had become Excel Pro’s largely on the back of those data flattening skills.
What does he mean by ‘flattening’? Well, PivotTables are such finicky eaters that they only like digital Pizza. That is, if you want a PivotTable to fully digest your data directly from the worksheet, then you need to lay that data out in a hierarchical structure that obfuscating geeks like to call a flat file. (You or I call a flat file a table. That’s why we’re not geeks.)
A PivotTable’s rather restrictive diet reminds me of this joke:
Question: What do you feed someone with ebola, SARS, and Swine flu?
Answer: Anything that fits under the door.
So your picky PivotTable will only eat flat, boring old Tables. In fact, it will only eat ONE table, and that table better have good labeling of all the ingredients (i.e. column headers) or your precious PivotTable will not even open it’s mouth.
Which is a problem, because the BOSS just ordered you to serve up some crazy concoction that isn’t even on your regular menu. The BOSS wants you to mix a little bit of this table with a tiny bit of that table, then add a sprinkling of some other table over the top as garnish. And the BOSS expects you to slam all this into your pre-heated PivotOven for a quick bake at 2.30GHz for no more than a few minutes, and then serve it up to the BOSS right away. Because the BOSS is hungry for data, and the BOSS is hungry NOW, DAMMIT!
So what did you do? You used as many VLOOKUPS as you have rows in your final flat data-set to join just one column of one of those additional tables onto the first table, didn’t you. And then you repeated this VLOOKUP frenzy for each and every other column that you ended up bringing into your steam-rolled mega-flat pivot-ready data-set. All of which resulted in one very bloated filesize, compared to the original footprint of the underlying tables.
And while you managed to serve up the order just in time, boy did you make a mess back in the kitchen. Formulas everywhere, and the whole joint is slowing down as a result. What’s worse, the BOSS liked the taste of what you just served up. So you’ll be working in the same messy kitchen next week to refresh it, unless you tidy up somehow.
Let’s face it…it’s such a complete mess, that you’re screwed.
Or are you?
PowerPivot….No fast data joint should be without it!
If PowerPivot was marketed on the Shopping Channel, then some obnoxiously loud voice would say something like this about it:
It slices. It dices. It joins. But wait, there’s more!
It cooks. It cleans. It washes up. It takes up practically no bench-space. But wait, there’s STILL more!
In fact there’s so much more, that that’s a subject for another post. Fortunately Chandoo already wrote it: What is Power Pivot – an Introduction. (Chandoo, that title is way too descriptive. You’ll never make Class 1 Geek unless you learn to obfuscate, my man).
Give that link a spin, because this product lives up to it’s hype. Indeed, no modern fast data joint should be without it. Emphasis on modern though, because you’ll need Excel 2010 or later in order to use this bloat-busting add-in.
But back to how it helps with bloat, the subject of this post. PowerPivot cuts through potential bloat, because it is a lot less fussy than Old-School-Pivots about what it eats:
- It allows you to create pivots on the fly from any mix of multiple data sources – Access, SQL, Excel Tables, Web Data, etc – and then effortlessly slice, dice, and navigate to your hearts content.
- You can incorporate/mash up additional data sources at any point.
- You can create very powerful calculated fields within PivotTables that simply are not possible to replicate with in traditional pivots.
- All without ship-loads of VLOOKUPS.
In fact, Rob Collie – master of both PowerPivot and understatement – has a great video showing how PowerPivot is the answer to “the dreaded VLOOKUP problem, among other things” in his post Be Gone, Scary VLOOKUP”.
So it does away with all those nasty VLOOKUPs. But that’s not the half of it…PowerPivot has some amazing data compression stuff going on under the hood too! (Check out Rob’s post Surprising Example of PowerPivot Compression for more on this.)
Okay, I’m convinced. But I’m a little scared, too…
If you want help to learn PowerPivot, then help is at hand: Check out Chandoo’s Advanced Excel & Power Pivot Training Classes. Rob Collie puts in a guest appearance in one of the modules, too. (And I think that you get a copy of his great book as part of the course fee.)
But before I return you to your regular schedule, be warned: Chandoo has the following public service message on his PowerPivot landing page that you might want to consider, if your boss is attractive as mine is:
Warning: Learning Excel and Powerpivot might suddenly make you boss fall in love with you.
Indeed, that is a good warning that I will heed, Chandoo. I’m burning Rob’s PowerPivot books as you read this.
About the Author.
Jeff Weir – a local of Galactic North up there in Windy Wellington, New Zealand – is more volatile than INDIRECT and more random than RAND. In fact, his state of mind can be pretty much summed up by this:
=NOT(EVEN(PROPER(OR(RIGHT(TODAY())))))
That’s right, pure #VALUE!
Find out more at http:www.heavydutydecisions.co.nz
12 Responses to “29 Excel Formula Tips for all Occasions [and proof that PHD readers truly rock]”
Some great contributions here.
Gotta love the Friday 13th formula 😀
Great tips from you all! Thanks a lot for sharing! bsamson, particularly you helped me on a terribly annoying task. 🙂
(BTW, Chandoo, it's not exactly "Find if a range is normally distributed" what my suggestion does. It checks if two proportions are statistically different. I probably gave you a bad explanation on twitter, but it'd be probably better if you fix it here... 🙂 )
Great compilation Chandoo
For the "Clean your text before you lookup"
=VLOOKUP(CLEAN(TRIM(E20)),F5:G18,2,0)
I would like to share a method to convert a number-stored-as-text before you lookup:
=VLOOKUP(E20+0,F5:G18,2,0)
@Peder, yeah, I loved that formula
@Aires: Sorry, I misunderstood your formula. Corrected the heading now.
@John.. that is a cool tip.
Hey Chandoo,
That p-value formula is really great for a statistics person like me.
What a p-value essentially is, is the probability that the results obtained from a statistical test aren't valid. So for example, if my p value is .05, there's a 5% probability that my results are wrong.
You can play with this if you install the Data Analysis Toolpak (which will perform some statistical tests for you AND provide the P Value.)
Let's say for example I've got two weeks of data (separated into columns) with the number of hours worked per day. I want to find out if the total number of hours I worked in week two were really all the different than week one.
Week1 Week2
10 11
12 9
9 10
7 8
5 8
Go to Data > Data Analysis > T-Test Assuming Unequal Variances > OK
In the Variable 1 Box, select the range of data for week 1.
In the Variable 2 Box, select the range of data for week 2.
Check "Labels"
In the Alpha box, select a value (in percentage terms) for how tolerant you are of error.
.05 is the general standard; that is to say I am willing to accept a 95% level of confidence that my result is accuarate.
Select a range output.
Excel calculates a number of results: Average (mean) for each week's data, etc.
You'll notice however that there are two P Values; one-tail and two-tail. (one tail tests are for > or .05), the number of hours I worked in week two is statistically equivalent to the number of hours I worked in week one.
So here’s a way you might want to use this. You put up a new entry on your blog. You think it’s the best entry ever! So you pull your webstats for this week and compare it to last week. You gather data for each week on the length of time a visitor spends on your website. The question you’re trying to prove statistically is whether there’s an average increase in the amount of time spent on your website this week as compared to last week (as a result of your fancy new blog post). You can run the same statistical test I illustrated above to find out. Incidentally, it matters very little to the stat test whether the quantity of visitors differs or not.
Anyhow, the Data Analysis toolpack doesn't perform a lot of stat tests that folks like me would like to have access to. In those cases I have to either use different software, or write some very complicated mathematical formulas. Having this p-value formula makes my life a LOT easier!
Thanks!
Eric~
Fantastic stuf..One line explanation is cool.
Thanks to all the contributors
OS
Take FirstName, MI, LastName in access (you can fix it to work in excel) capitalize first letter of each and lowercase the rest and add ". " if MI exists then same for last name:
Full Name: Format(Left([FirstName],1),">") & Format(Right([FirstName]),Len([FirstName])-1),"") & ". ","") & Format(Left([LastName],1),">") & Format(Right([LastName],Len([LastName])-1),"<")
I teach excel, access, etc etc for a living and i have my access students build this formula one step at a time from the inside out to show how formulas can be made even if it looks complicated. Yes I know I could just do IsNull([MI]) and reverse the order in the Iif() function but the point here is to nest as many functions as possible one by one (also I illustrate how it will fail without the Not() as it is)
Extract the month from a date
The easiest formula for this is =MONTH(a1)
It will return a 1 for January, 2 for February etc.
if in a column we write the value of total person for eg. 10 if we spent 1.33 paise each person then how we get total amount in next column and the result will in round form plzzzzz solve my problem sir................... thank u
@Anjali
If the value 10 is in B2 and 1.33 paise is in C2 the formula in D2 could be =B2*C2
If the values are a column of values you can copy the formula down by copy/paste or drag the small black handle at the bottom right corner of cell D2
kindly share with me new forumulas.
How to convert a figure like 870.70 into 870 but 871.70 into 880 using excel formula ? Please help.