So you’ve made it to part 3 of Are You Trendy, well done.
In todays final post of Are You Trendy? we will take a break from the maths and discuss techniques and tools that Excel provides to assist us with Trend Analysis.
We will look at 2 techniques, one built-in to Excel called Charts and another which is a User Defined Function (UDF) I developed.
Both techniques have uses in analysis of trends.
Once again all examples below are found on the Examples Workbook.
Excel Charts
Excel has a number of charting styles some of which have the ability to calculate and display Trend Lines.
The following tables shows which charts Do and Don’t support built-in trendlines.
| Supports Trendlines | Doesn’t Support Trendlines |
| Scatter (X Y) | Pie |
| Line | Donut |
| Area | Radar |
| Column | Stock |
| Bar | Surface |
| Bubble |
For those charts that don’t support trendlines, this doesn’t stop us, as Excel users, from adding other series or using other techniques to add trendlines to the charts, it just that we have to do the hard work ourselves and not use Excels built-in skills.
For this exercise we will be using a Scatter (X Y) Chart but the techniques apply equally to all charts that support trendlines.
So lets go: First we need some data.
Next add a scatter chart , Select the data and Insert Chart, X Y (Scatter Chart)
Adding a Trend Line to the chart is as simple as selecting the chart and right click on the Series you want to add the Trend Line to, see below.
You can change the trendlines properties to make it stand out as per the above.
Once you add a Trendline a Format Trendline dialog will appear.
This is the main window for setting Trendline properties and format options.
In this post I am not going to be dealing with the Tab Options of Line Color, Line Style, Shadow and Glow and Soft Edge options, as I am sure you can work out what they do and play with them at your leisure.
Trendline Properties
Once you have added a Trendline you can access the Trendlines properties at any time by:
Selecting the chart
Select the Trendline
Right Click and Format Trendline.
How Many Trendlines Can I Have ?
You can add a large number of Trendlines to each Chart Series in Excel by simply selecting the Right Clicking on the Chart Series and select Add Trendline.
Some of the regression types may not be available on second and subsequent regressions.
Trend/Regression Type
The main section of the Format Trendline dialog is the Trend/Regression Type:
This section lists 5 trend types we are familiar with from the Are You Trendy? (Part 2).
For more details on the individual Tren Types refer to Are You Trendy (Part 2).
The default trend type is Linear ( Y = mX + c ).
You can select each type in turn and see the estimated trend.
Exponential
Linear
Logarithmic
Polynomial
This shows the Trendline for a Polynomial Estimate of Power = 2. Excel supports up to Power = 6.
Power
Moving Average
Moving average isn’t a forecasting tool, but is more of a data manipulation tool, which is commonly used for elimination of short term trends or smoothing of the data source. Some industries use Moving Averages to look at buying and selling opportunities by comparing the current price against longer term trends.
It is possible to use moving average data for trend analysis, but this must be first done manually in Excel and then it can be used with the techniques discussed here.
As such we won’t be discussing its use here.
If you are interested in reading more about the use of moving averages have a look at Moving Average.
Trendline Name
The Trendline Name section by default shows you the Selected Trend Estimation type and your Data Series Name eg: Linear (My Data)
However you can type whatever Name you want by using the custom field:
eg: Huis Fruit Shop Sales Trend
Forecast
The Excel Chart Trendline function allows us the option to project our Trendline a number of periods or X Values into the future or past.
The above will forecast our trend line 50 X axis values past the limits of our data and 10 X axis values prior to the limits of our data and is shown below.
This is a great option to use for 2 reasons.
Firstly you can use this to see where the Trendline will cross the Y axis (X = 0) and is an important check to validate your selected Trendline.
You can see in the example above that the Trendline crosses the axis at -10. If this isn’t correct we can either try another estimation technique or Excel has a tool, which is discussed below, to deal with that.
Secondly, you can use it to look into the future by projecting the Trendline past your data limits.
Misc Parameters
The final 3 parameters
Set Intercept
By default Excel Chart Trendline has used a Const value of True and hence calculates the natural intercept value for the data. If you want to over-ride this with a 0 intercept value Tick the box and select a value
Set Intercept : Not selected
Set Intercept : Selected Value of 10.0
I would always suggest using the Forecast backwards option first to see what the natural intercept is before using this option (Both charts above show the Trendline projected back 10 periods) but the second chart has a Y intercept set manually to 10.
Display Equation on Chart
The Display Equation on Chart options use is self explanatory, it Display the equation to the current Trendline on the current Chart:
The equation is based on the select Trend type.
The equation is also live, in that if you change and selected another Trend Type or the data changes the equation will update, similarly if you select to intercept the Y axis at 0 or another value.
By selecting the Text Box which has the equation the equations Font properties can be changed eg: Font Style, Size, Italic, Bold etc as well as background color.
Display R2 on Chart
This display the R2 value on the chart in the equation Text Box as shown above.
The R2 value is based on the select Trend type.
The R2 value is also live, in that if you change and selected another Trend Type or the data changes the R2 will update, similarly if you select to intercept the Y axis at 0 or another value.
By selecting the Text Box which has the equation the R2 the Font properties can be changed eg: Font Style, Size, Italic, Bold etc as well as background color.
How Can We Look at Values Along a Trend Line?
In Are You Trendy? (Part 2) and in the section above we looked at a number of ways of establishing a Trendline.
So how can I predict values along a Trendline?
There are a few ways and we will discuss 2 of them.
1. Write an equation in excel
2. Use my Trendy User Defined function.
Write an Equation in Excel
In the previous section we had a chart shown below:
The equation for the line of best fit is Y=7.8984.e^(0.0256.X) which in Excel cell formula talk is =7.8984*EXP(0.0256*X Cell)
So we can setup a cell or range of cells in excel to show us either intermediate values or future values of X and the corresponding Y values.
The great thing about having an equation is that we can now use this to animate our chart.
I’m not talking about Avatar style animation, but simply adding a tool for the user to interactively select X values and the equation and chart will show us the new Y value. This even allows us to show future values past the end of our data because we are using the equation to our data.
How do we do this?
Refer to Sample Workbook.
Firstly lets add a cell which will contain the X value
E52 = 10
Secondly add a cell which has our equation
E53: =7.8984*EXP(0.0256*X Cell)
Third add a Scroll Bar Control from the Active X controls on the Developer toolbar and set a few properties
Minimum = 0
Maximum = 300
Linked Cell = E52
Large Change = 10
Fourth add a new series to our chart which will just have a single X and Y value
Series Name = “Our Trend”
X Series = E52
Y Series = E53
You can now interactively move the srcoll bar back and forth and the Chart will show you the new forecast value for our sales in Huis Fruit Shop.
The limitations with the above approach is that it is fixed to the equation you use for your Y value, which is based on your selected choice of Trendline type.
Can we link this technique to the equation from the Trendline ?
With Excel natively, No 🙁 .
TrendY – A User Defined Function for the Analysis of Past, Intermediate & Future Trendline Values
In a post at Chandoo.org user Trevian3969 asked the question “How can I view intermediate values along a Charts trend line?”
The answer was the development of the UDF, TrendY (Trendy).
Trendy was developed to do exactly that, take the equation from a Charts Trendline and evaluate it to determine intermediate, past or future values based on a given X input.
Because the UDF is taking the equation for the trend line it can be used to return a Y value for any X values, prior to, later than or within the supplied or known X Range of the chart.
An Example of the use of Trendy is shown in the attached Workbook
Form: = TrendY(X Value, [Chart No], [Series No], [Trendline No])
X Value: Is the X Value that you want to know the Y Value of the Trendline
Chart No: Chart No is optional and is the Chart No on the current sheet, Default or omitted = 1
Series No: Series No is optional and is the Series No on the chart, Default or omitted = 1. The series No is the number shown at the end of the Formula Bar when a series is selected
eg: =SERIES(“My Data”,Sheet1!$B$2:$B$41,Sheet1!$C$2:$C$41,1)
Trendline No: Trendline No is optional and is the Trendline No of the Trendline you wish to track. Default or omitted = 1.
[Option Parameters, Default = 1]
Eg: = TrendY(20) will place the value of X=20 into the Trendline Equation of the Trendline of Chart 1 Series 1 Trendline 1
= TrendY(40,1,3,2) will place the value of X=20 into the Trendline Equation of the Trendline of Chart 1 Series 3 Trendline 2
=TrendY(B43,,3) will place the value of cell B43 into the Trendline Equation of the Trendline of Chart 1 Series 3 Trendline 1. Chart and Trendline both default to 1 as they are omitted.
The function works with all the Trendline types and variants of those, except the Moving Average.
To use the Trendy UDF copy the following code to a VBA Module or alternatively import the Txt File as a new Module
The code is also available in the Example Workbook with a fully working m0del.
If you don’t have the Trendlines equation shown it will do that for you and it ignores the R^2 function if shown.
The Trendlines equation box is actually where the function gets the equation from in the first place.
Then on a worksheet simply use then =Trendy(X Value) as described above.
Trendy has only been tested on Excel 2007 and 2010, use on other versions at your own risk.
Limitations
The Trendy UDF is limited to use the available precision of the parameters, for each equation, as displayed in the Equation Text Box on the Chart.
In some cases especially when multiplying by powers of large numbers and especially in the Polynomial equations, there may be a large discrepancy between the calculated value by Trendy and the displayed value by the Trendline on the Chart. In these cases the Trendline is correct. This will show on the chart as the Plot Y value of Trendy not being on the Excel calculated Trendline.
Trendy has 2 constants at the top of the UDF that are used for the calculation of Intermediate Equations. These are equations that the user doesn’t see but are done at a much higher level of precision to avoid these errors. These constants are:
Const DataLabelNoFormat = “#,##0.0000”
Const PolynomialNoFormat = “#,##0.000000000000”
If you have rounding errors the easiest way to fix them is to increase the number of decimals by increasing the number of zeroes after the decimal in the appropriate line, remembering that Excel only carries 15 decimal places anyway and Trendy is already using 12 for Polynomial Trends already.
Trendy returns the displayed equations to 4 decimals after the intermediate calculations in any case.
Further Readings
All Examples from the Are You Trendy? Series in one Workbook
Final
I hope you have enjoyed this 3 part series on Trend Analysis and Forecasting using Excel.
It has been put together to Introduce you to, and whet your appetite to, some of the functions, tools and techniques that you may not have had a lot of exposure to in the area of Trend Analysis and Forecasting using Excel.
I’d like to thank Chandoo for the opportunity to look after Chandoo.org for the past week and the opportunity to put the 4 posts up.
I will now hand Chandoo back the keys to the blog, hopefully without any dents.
Let me know what you thought of the series in the comments below:
Keep us informed on How you go with real data once you start applying some of these techniques.
Hui…
ps: I still don’t know if Trevian3969 was happy with my response to his post as he never responded ?
12 Responses to “Speeding up & Optimizing Excel – Tips for Charting & Formatting [Speedy Spreadsheet Week]”
Usually when I dump data into my files to update values, the formatting sometimes go to all rows or columns. So what I typically will do is go to the last row and then the last column and use Ctrl + Shift + end and then delete the cells highlighted. this will remove all unknown formats in the worksheet. Also, after you have done this, you won't see the benefit until you save the document. Sometimes I even have to close and reopen. The direct sign that this has improved is the size of the scroll bar and range.
I have some comments on a couple of the points.
1. Camera objects
Tip: I use defined names in conjunction with camera tool objects.
Each camera object gets a name like so:
CameraItem01
Referring to: =IF(PicsOn=1,Sheet1!$C$2:$S$5,"")
By setting the PicsOn name to 1, the camera objects become "live", by setting the PicsOn name to 0, they become static. That improves performance enormously.
4: Conditional formatting
Lots of CF rules can slow down your workbook a lot. And it does not show the calc progress a "normal" recalc does on slow workbooks.
5. Format whole columns/rows
as far as I know, there is no problem with formatting entire columns/rows performance-wise, on the contrary, Excel is more efficient when you format an entire column than when you format a couple of 100 rows of a column.
6. Styles.
Here I wholeheartedly disagree. I say: Use styles. And use them religously.
I mean: if you have applied a (custom) style and you need to change a small piece of formatting to make that one cell look right, force yourself to create a new style just for that cell. It forces you to really think about your spreadsheet design and try and streamline it. It also makes it much, much easier to change your sheet's appearance later on. See http://www.jkp-ads.com/articles/styles00.asp
Very good insights Jan..
Camera objects: I often use similar technique to turn off images in my dashboards.
Formats: Thanks for clearing this. Do you think formatting larger ranges has any impact on macro speeds or it does not matter?
Styles: Thanks for telling us about this. As I mentioned, I am not sure about the styles, but I am under the impressions that excessive use of styles can bloat the file size.
@Chandoo:
If you stick to formatting entire rows/columns I don't expect macro speed is affected. Better: try it!
If you use styles properly AND as a replacement of ad-hoc cell formatting, I expect you'll see that the file actually is smaller in size.
This is because the cells now only have a reference to a single style instead of a reference to a custom cell formatting style.
Many cell formatting combinations get created if you format your cells in an ad-hoc manner, which was responsible for the dreaded "Too many different cell formats" error in Excel 2003 and older. Excel 2007 and 2010 have a higher limit there, but it does slow down your file with many of them.
Style bloat in my point of view is what you get by copying and pasting a lot from various other files and thus get Normal 1, Normal 1 1, Normal 1 1 1, ... I have seen workbooks with as many as 6000 styles, all caused by copying and pasting from various differently formatted workbooks.
Excel 2007 and 2010 have fixed a number of issues regarding copying of styles, but for workbooks with a long editing history, the trouble is already in the workbooks.
Cant emphasise the importance of reducing the amount of formatting in a workbook - this has a suprising impact on workbook size. I've always kept to one font, and no more than three colours - this has worked well for me. Keeping things clean and simple should be the motto when designing any type of report/dashboard that is going to be distributed around the organisation.
You can also save a few MB's by saving as an xlsb file.
Has anyone else mentioned that only the first item in the "more ..." section is hyperlinked.
Prem, have you confirmed by trial that XLSB file size is smaller than same XLSX file? Sorry, I just tried it with a small, simple XLSM file. I was surprised to see you are correct. File went from 40kb to 37kb. I thought that the compression of the new file would make the new file smaller.
@Ron
All Excel files have a minimum overhead that they have to include which is around 8KB, just to store a simple number or letter.
So with a small file of 40KB you will not see a huge improvement in file size
With files greater than 10MB you will see large improvements in size.
The compression gained also depends on what the contents of the file include. That is straight numbers, text and formulas can be greatly compressed whereas files that contain a lot of objects especially pictures gain very little from using *.xlsb files.
@Ron.. the other articles are yet to be published. All the links will be updated by Tuesday (27th March).
Hi,
I have a need for x,y scatter chart to have arround 30 data series.
like this:
http://i65.tinypic.com/jra8lc.jpg
Also I have multiple of such charts in one excel file.
Is there any way to make excel faster, because it is irritatingly slow?
(though my PC config. is quite on the level)
Thanks in advance!!!
@Mil
30 series won't be the issue
It is the number of points in the series
Also remove all fancy modifications, like shadows, fancy fills etc
I'd suggest asking the question in the Chandoo.org Forums http://forum.chandoo.org/
Attach a sample file with an example of what you are after
@Hui
I've already removed all fancy mod. The problem is there are also a lot of data points in one series.
Thanks for the advice!
@Mil
Do you really need every data point ?
Where is the chart being presented Screen or Report
On a screen you are unlikely to use more than 800 pixels for the chart area
So using any more than about 250 points is not adding values
On an A4 chart in landscape lets say the chart area is 6" long and at 300dpi that is 2000 pixels
Once again using more than 800-1000 points will not add any value
I have seen charts with 30,000+ points and when this is explained and a work around shown people appreciate the speed up
For a work around try setting up an area where you select say every x'th point using an Offset or Index Function
Then plot that data
I'd suggest asking the question in the Chandoo.org Forums http://forum.chandoo.org/
Attach a sample file with an example of what you are after