In mid May, Anup47 asked a question in the Chandoo.org forums about the use of a VBA macro to run a number of iterations of a variable against two sets of X values, you can see the post here. It turns out that the number of iterations was 500 columns of data with each column having 27 values.
On examination of the problem, it was going to be a straight forward matter of setting up a statistical function Linest and then using the Data Table command to run each set of data through the function.
The Linest will take the input data and return the statistics that Anup wanted.
The Data Table function will feed in the source data and tabulate the Input and Output data.
This Post follows through a worked example which you can follow along, download the Sample file to suit Sample File 97/2003 or Sample File 2007/10 version. The Sample File contains a worked example of the completed model as well as a Practice Page of the original data. Download the Excel 95/2003 or 2007/10 version above.
Please note that the sample file only contains 14 sets of data as opposed to the 500 Anup47 wanted to process.
Setup
There are a few things that needed setting up before the work starts.
- Headers
- Linest Area
- Link Area
- Data Table Area
Once these areas are setup we simply use the Excel Data Table function.
Once the Data Table function has run, the results can be processed or analysed as required.
Headers
The original data was just that, a tabulation of raw data. The two X sets of Data were in Columns 1 & 2. Each Column from D onwards has a set of Y data that was to be processed.
The first thing that was required was some Headers for the Input Data.
This isn’t strictly required but it is good practice and makes it easier to tabulate and analyse results later.
Insert a Row above the first line
Put X1, X2 in A1, B1 and Y1 in D1 and then drag the lower right Black Handle across top to the right and Excel will autofill the remaining cells.
Linest Area
To get the statistics which Anup wanted we will use the Excel Linest function.
Linest is a Statistical Function that takes a set of data and compares it, in this case to two sets of X Values and produces a set of statistical measure relevant to the correlation between the data sets.
This post isn’t going to explain the intricacies of Linest and I refer you to the Links section at the end where you can read more about the Linest function at your leisure.
For our purposes we need to know that Linest is an Array Formula and requires a 5 Row x 5 Column area to be entered into. For now we will just Array Enter the function =Linest($D$2:$D$28,A2:B28,True, True) into B32:F36.
To do that select the range B32:F36, Press F2 and type/paste the equation in, then Array Enter with Ctrl Shift Enter.
Link Area
To Link the Linest equation to a Data Table we need a link cell, which we will put just above the Linest area.
For now just enter a 1 in it.
We can now go back to the Linest area and link the Linest equation to our link area using the equation, =LINEST(OFFSET($C$2:$C$28,,$B$30),A2:B28,TRUE, TRUE)
To do that select the range B32:F36, Press F2 and type/paste the equation in, then Array Enter with Ctrl Shift Enter.
What this does is allow the Linest formula to access different columns Y1 to Y500 depending on the value of the Link cell B30 which is now 1.
Data Table Area
To setup a Data Table area we need a column of Inputs which will be the Run Numbers and the Row Inputs will be links to the Input and Output Cells.
In a range J33:J46 put the values 1 to 14. These will be the Run Numbers. ie Run No 1, Run No 2 etc (Green in the example below).
Across the top of the Data Table area we can put a number of links and associated labels (Yellow and Blue)
In this case there are 4 Output links =B31, =C31, =B34 and =B33 and their associated labels above them, as well as 2 Input equations and there Labels. The Input equations are simple Offset function that retrieves a value from Rows 1 or 2 based on the value of the Link Cell B30.
These are technically not required but make data analysis and identification of individual results later on a lot simpler.
Run Data Table
We can now run the data Table by selecting the Data Table area: J32:P46
Noting that we will be using a Column Input cell and that it will link to $B$30, the Link cell for the Linest command.
What this does is takes the first value from the Column J32:J46 and puts it into B30, then the Linest command will be calculated and the results put into the Data Table area along with the Inputs.
This is repeated for each cell in J32:J46 automatically.
The final Data Table is now populated as below:
You can see by extending the Data Table input column from 14 to 500 that the full 500 columns of Input Data could easily be processed.
Results
You now have a set-off data that can be analyzed using normal statistics, Min, Max, Std Deviation etc, or can be fed into a Pivot Table/Chart for analysis etc.
References
Linest References
http://chandoo.org/wp/2011/01/26/trendlines-and-forecasting-in-excel-part-2/
http://newtonexcelbach.wordpress.com/2011/01/19/using-linest-for-non-linear-curve-fitting/
Data Table References
http://chandoo.org/wp/2010/05/06/data-tables-monte-carlo-simulations-in-excel-a-comprehensive-guide/
How can the Data Table command help you become a data processing super hero?
How can the Data Table command help you become a data processing super hero?
Let us know in the comments below:
12 Responses to “29 Excel Formula Tips for all Occasions [and proof that PHD readers truly rock]”
Some great contributions here.
Gotta love the Friday 13th formula 😀
Great tips from you all! Thanks a lot for sharing! bsamson, particularly you helped me on a terribly annoying task. 🙂
(BTW, Chandoo, it's not exactly "Find if a range is normally distributed" what my suggestion does. It checks if two proportions are statistically different. I probably gave you a bad explanation on twitter, but it'd be probably better if you fix it here... 🙂 )
Great compilation Chandoo
For the "Clean your text before you lookup"
=VLOOKUP(CLEAN(TRIM(E20)),F5:G18,2,0)
I would like to share a method to convert a number-stored-as-text before you lookup:
=VLOOKUP(E20+0,F5:G18,2,0)
@Peder, yeah, I loved that formula
@Aires: Sorry, I misunderstood your formula. Corrected the heading now.
@John.. that is a cool tip.
Hey Chandoo,
That p-value formula is really great for a statistics person like me.
What a p-value essentially is, is the probability that the results obtained from a statistical test aren't valid. So for example, if my p value is .05, there's a 5% probability that my results are wrong.
You can play with this if you install the Data Analysis Toolpak (which will perform some statistical tests for you AND provide the P Value.)
Let's say for example I've got two weeks of data (separated into columns) with the number of hours worked per day. I want to find out if the total number of hours I worked in week two were really all the different than week one.
Week1 Week2
10 11
12 9
9 10
7 8
5 8
Go to Data > Data Analysis > T-Test Assuming Unequal Variances > OK
In the Variable 1 Box, select the range of data for week 1.
In the Variable 2 Box, select the range of data for week 2.
Check "Labels"
In the Alpha box, select a value (in percentage terms) for how tolerant you are of error.
.05 is the general standard; that is to say I am willing to accept a 95% level of confidence that my result is accuarate.
Select a range output.
Excel calculates a number of results: Average (mean) for each week's data, etc.
You'll notice however that there are two P Values; one-tail and two-tail. (one tail tests are for > or .05), the number of hours I worked in week two is statistically equivalent to the number of hours I worked in week one.
So here’s a way you might want to use this. You put up a new entry on your blog. You think it’s the best entry ever! So you pull your webstats for this week and compare it to last week. You gather data for each week on the length of time a visitor spends on your website. The question you’re trying to prove statistically is whether there’s an average increase in the amount of time spent on your website this week as compared to last week (as a result of your fancy new blog post). You can run the same statistical test I illustrated above to find out. Incidentally, it matters very little to the stat test whether the quantity of visitors differs or not.
Anyhow, the Data Analysis toolpack doesn't perform a lot of stat tests that folks like me would like to have access to. In those cases I have to either use different software, or write some very complicated mathematical formulas. Having this p-value formula makes my life a LOT easier!
Thanks!
Eric~
Fantastic stuf..One line explanation is cool.
Thanks to all the contributors
OS
Take FirstName, MI, LastName in access (you can fix it to work in excel) capitalize first letter of each and lowercase the rest and add ". " if MI exists then same for last name:
Full Name: Format(Left([FirstName],1),">") & Format(Right([FirstName]),Len([FirstName])-1),"") & ". ","") & Format(Left([LastName],1),">") & Format(Right([LastName],Len([LastName])-1),"<")
I teach excel, access, etc etc for a living and i have my access students build this formula one step at a time from the inside out to show how formulas can be made even if it looks complicated. Yes I know I could just do IsNull([MI]) and reverse the order in the Iif() function but the point here is to nest as many functions as possible one by one (also I illustrate how it will fail without the Not() as it is)
Extract the month from a date
The easiest formula for this is =MONTH(a1)
It will return a 1 for January, 2 for February etc.
if in a column we write the value of total person for eg. 10 if we spent 1.33 paise each person then how we get total amount in next column and the result will in round form plzzzzz solve my problem sir................... thank u
@Anjali
If the value 10 is in B2 and 1.33 paise is in C2 the formula in D2 could be =B2*C2
If the values are a column of values you can copy the formula down by copy/paste or drag the small black handle at the bottom right corner of cell D2
kindly share with me new forumulas.
How to convert a figure like 870.70 into 870 but 871.70 into 880 using excel formula ? Please help.