Are You Trendy ?

Share

Facebook
Twitter
LinkedIn

Are You Trendy ?

Chandoo is off Holidaying teaching excel in the Maldives and has lent me the keys to his Blog (Chandoo.org) and this week I plan to take it for a spin.

I will be posting 3 posts on Trend Analysis/Forecasting using Excel and a forth post on some Hidden Worksheet Properties which I stumbled onto last week !

Hopefully if I look after the Blog while Chandoo is gone, He will let me borrow the keys another day.

Forecasting

“Tomorrows weather will be fine and hot with a chance of showers in the morning.”

We have all seen this type of forecasting during the nightly news.

This week I am going to go through the basics of forecasting and trend analysis using Excel as a tool.

We will look at some simple trends and make predictions about future values.

In later posts we will look at more complex data and other methods of tackling these analysis.

Introduction

Often you may have a set of data and need to know what an intermediate or future value of that data may be.

This week we will investigate 3 methods of tackling this problem using Excel.

In this post we’ll look at doing forecasting manually

In the second post we’ll look at a few excel functions that assist us with forecasting

The third post will discuss a method of looking at any value along an Excel generated Trend Line and give you a tool to assist you in this.

Manual Forecasting

In all environments where numbers are collected and people make use of these numbers the ability to forecast or extrapolate data may be required.

In forecasting we are going to look at the trends that the data has and use these trends to help forecast future values or values outside the measured data. The trends can also be used to infill data where gaps may be missing in the collected data.

This post will look at doing this manually, albeit with some help from Excel.

We will examine a business that makes things and we will measure some measurement of those things every 5 days. In trend analysis it doesn’t matter what you measure or what your measuring it against.

We have collected some data which is tabulated

Day Measure
5 7
10 10
15 24
25 30
30 40

One of the easiest ways to visualise this relationship is to draw a quick chart of one measure vs a base or in our case a time line.

This can be shown graphically as a simple Excel Scatter chart

You can see that there is some level of variability in the measurement as the data doesn’t quite fit a straight line.

Manually we can make an estimate of a line of best fit and draw it on the chart by adding a new data series consisting of 2 points.

There are 3 quick methods of using this line of best fit

  • Manual Estimates
  • Equal Triangles
  • Equation for the line

 

Manual Estimates

If we want to know what the measurement would be for a location where no measurement was taken we can use the chart and 2 quick lines to show in this example that for 20 days we would expect a measurement of about 26 units.

This can also be used for extrapolation of our data past the limits of what was measured.

By extrapolating the Line of Best Fit beyond the data, the same technique can be applied to estimating what some future value maybe.

Equal Triangles

Equal Triangles is a technique where a simple ratios of 2 similarly shaped but different sized right angle triangles can be used to make estimates of missing or extrapolated data.

Using Equal Triangles the ratio of the height to the width of Triangle 1 (Red) is equal to the ratio of the height to the width of Triangle 2 (Blue).

So in the example above

Y1/X1 = Y2/X2

Y1 = 38 – 8 = 30

X1 = 30 – 5 = 25

Y1/X1 = 30/25 = 1.2

So for Triangle 2

Y2/X2 = 1.2

Y2 = ? – 8

X2 = 20 – 5 = 15

from Y2/X2 = 1.2

(? – 8 ) /15 – 1.2

We can rewrite this as

? = 8 + 1.2 x 15 = 26.0

Or

Unknown Y = Min Y + Ratio x (New X – Min x)

Once we have an equation we can setup a new series on out chart based on an equation in some cells and then directly plot the data onto our chart.

In this case we have used the equation =F105+1.2*(E111-E105)

 

Equation of the line of Best Fit

If we are using a straight line to model our line of best fit, we can also write an equation for the line in the form

Y = mX + c

Where: Y is the unknown measure

X is the X value for which we want to know the value of Y

m is the gradient of the line

c is the Y intercept of the line (or Y value when there is no X value or X =0 )

The gradient m is calculated as the Rise / Run or in our example 30/25 = 1.2

The Y Intercept is the value when x = 0. This can be back calculated from the first point (5,8)

C = 8 – (5 x 1.2) = 2.0

So the equation for our line of best fit is Y = 1.2 X + 2

We have used this in the next example =E136*1.2 + 2

The good thing about having an equation for the line is that we can use that to calculate any value of our measure.

So if we want to know the measure on a day outside the range we measured, say the 40th day

Downloads

You can download examples of all the above charts from the following link

https://chandoo.org/wp/wp-content/uploads/2011/01/Trends1.xls

Benefits of Manual Estimation

  • Applicable to simple models
  • Can be used without a computer or a calculator in the field
  • Gives the user a better feel for the data

Problems of Manual Estimation

  • Only applicable to simple models
  • Reliant on the accuracy of your estimate of the trend
  • No measure of how accurately your estimate fits the data

Next:

In the next post we will look at using excel functions to automatically estimate lines of best fit and other excel functions to aid in estimation of non-linear functions.

Further Readings

Are You Trendy (Part 2)

Are You Trendy (Part 3)

 

What have you measured trends of ? Let us know in the comments below

Have you used Excel to assist in analyzing trends ? Let us know about it in the comments below

Facebook
Twitter
LinkedIn

Share this tip with your colleagues

Excel and Power BI tips - Chandoo.org Newsletter

Get FREE Excel + Power BI Tips

Simple, fun and useful emails, once per week.

Learn & be awesome.

Welcome to Chandoo.org

Thank you so much for visiting. My aim is to make you awesome in Excel & Power BI. I do this by sharing videos, tips, examples and downloads on this website. There are more than 1,000 pages with all things Excel, Power BI, Dashboards & VBA here. Go ahead and spend few minutes to be AWESOME.

Read my storyFREE Excel tips book

Overall I learned a lot and I thought you did a great job of explaining how to do things. This will definitely elevate my reporting in the future.
Rebekah S
Reporting Analyst
Excel formula list - 100+ examples and howto guide for you

From simple to complex, there is a formula for every occasion. Check out the list now.

Calendars, invoices, trackers and much more. All free, fun and fantastic.

Advanced Pivot Table tricks

Power Query, Data model, DAX, Filters, Slicers, Conditional formats and beautiful charts. It's all here.

Still on fence about Power BI? In this getting started guide, learn what is Power BI, how to get it and how to create your first report from scratch.

8 Responses to “Pivot Tables from large data-sets – 5 examples”

  1. Ron S says:

    Do you have links to any sites that can provide free, large, test data sets. Both large in diversity and large in total number of rows.

    • Chandoo says:

      Good question Ron. I suggest checking out kaggle.com, data.world or create your own with randbetween(). You can also get a complex business data-set from Microsoft Power BI website. It is contoso retail data.

  2. Steve J says:

    Hi Chandoo,
    I work with large data sets all the time (80-200MB files with 100Ks of rows and 20-40 columns) and I've taken a few steps to reduce the size (20-60MB) so they can better shared and work more quickly. These steps include: creating custom calculations in the pivot instead of having additional data columns, deleting the data tab and saving as an xlsb. I've even tried indexmatch instead of vlookup--although I'm not sure that saved much. Are there any other tricks to further reduce the file size? thanks, Steve

    • Chandoo says:

      Hi Steve,

      Good tips on how to reduce the file size and / or process time. Another thing I would definitely try is to use Data Model to load the data rather than keep it in the file. You would be,
      1. connect to source data file thru Power Query
      2. filter away any columns / rows that are not needed
      3. load the data to model
      4. make pivots from it

      This would reduce the file size while providing all the answers you need.

      Give it a try. See this video for some help - https://www.youtube.com/watch?v=5u7bpysO3FQ

  3. John Price says:

    Normally when Excel processes data it utilizes all four cores on a processor. Is it true that Excel reduces to only using two cores When calculating tables? Same issue if there were two cores present, it would reduce to one in a table?
    I ask because, I have personally noticed when i use tables the data is much slower than if I would have filtered it. I like tables for obvious reasons when working with datasets. Is this true.

    • Ron MVP says:

      John:
      I don't know if it is true that Excel Table processing only uses 2 threads/cores, but it is entirely possible. The program has to be enabled to handle multiple parallel threads. Excel Lists/Tables were added long ago, at a time when 2 processes was a reasonable upper limit. And, it could be that there simply is no way to program table processing to use more than 2 threads at a time...

  4. Jen says:

    When I've got a large data set, I will set my Excel priority to High thru Task Manager to allow it to use more available processing. Never use RealTime priority or you're completely locked up until Excel finishes.

Leave a Reply