This article is written by Michael Hutchens from Best Practice Modelling.
In the first article on Modular Spreadsheet Development, we got a high level overview of Modular Spreadsheet Development principles. This article discusses the practical implementation of these principles in Excel.
A quick review
In my first article, I discussed the infinite potential provided by Modular Spreadsheet Development to improve the way spreadsheets are built, used, shared and communicated.
This is made possible by sub-dividing spreadsheets into modules, which can be re-used and linked like Lego® pieces, thereby reducing model build time and the risk of errors.
These concepts were well received by Chandoo’s readers, although some concerns were raised about their practical implementation in Microsoft Excel. This article aims to address those concerns.
Overview of implementation methods
From my experience using Modular Spreadsheet Development over the past decade, there are three increasingly-efficient methods of implementation in Microsoft Excel:
- Manual implementation;
- VBA automated implementation; and
- Commercial add-in implementation.
I will provide an overview of each of these methods and a summary of their advantages and disadvantages.
1. Manual Implementation
The key to implementing Modular Spreadsheet Development is standardization, because it is the robust consistency created by standardized spreadsheets that makes it possible to interchange the modules within a spreadsheet.
There are numerous approaches to spreadsheet standardization (my organization uses the Best Practice Spreadsheet Modeling Standards), but the key requirement of any standardized approach for Modular Spreadsheet Development purposes is consistency. Once this consistency is present, re-using and sharing modular content within spreadsheets becomes surprisingly easy.
Let’s consider an example ( ➡ download example files ) in which a model developer has built a dynamic 3-way financial statement model. This model contains various modules, ranging from revenues and expenses through to financial statements and a dashboard module that looks as follows:
After completing this model, the model developer decides to add an equity valuation. Creating the equity valuation assumptions and outputs from scratch is a big job, and a risky one given the complexity of the discounted cash flow (DCF) valuation formulas required.
Luckily, the model developer has included an equity valuation in a prior model, and had the foresight to keep a copy of this equity valuation module in a standalone workbook called Equity Valuation.xlsb. The composition of this module is shown below:
This equity valuation module contains four components; an assumptions component, a calculation outputs component, an outputs summary component and a lookups component (to hold drop down box control lookup data).
From an Excel perspective, this module is a workbook with these four components placed on three sheets; one assumptions sheet, one outputs sheet and one lookups sheet, as shown below (the lookups sheet is to the right of the image):
Each of these components is comprised of blocks of entire rows that contain Excel content (such as constants, formulas, controls, hyperlinks, etc.) that together undertake an equity valuation. For example, the top sections of the equity valuation assumptions component are shown below:
Rather than try to re-build all of this content into the financial model, the model developer decides to implement Modular Spreadsheet Development and insert this equity valuation module into the financial model.
Two steps are required to do this:
- Collectively copy the sheets containing the equity valuation module components into the financial model workbook from the workbook containing the equity valuation module; then
- Insert formulas into the equity module assumptions to link this module to the surrounding financial model outputs, and thereupon calculate the DCF equity valuation.
To copy the sheets containing the equity valuation module components into the financial model workbook, the following actions are required:
- Activate the Equity Valuation.xlsb workbook;
- Collectively select the three sheets containing the equity valuation module components (i.e. the sheets named Eq_Val_Ann_TA, Eq_Val_Ann_TO and Eq_Val_LU) and copy them to the end of the Financial Model.xlsb workbook using the Excel Move or Copy sheets command, as shown below:
- Click the OK button (or press/hold down the Enter key) each time you are asked to use the destination workbook version of a range name. This will happen quite a few times in this example, as range names have been used in the time series parts of the model to ensure consistency;
- Move the inserted sheets into their appropriate locations within the Financial Model.xlsb workbook – i.e. move the sheet containing the equity valuation assumptions into the assumptions section of the workbook, etc.; then
- Update any necessary surrounding content within the financial model workbook, such as the table of contents, to reflect the inclusion of these sheets.
After doing this, the newly-inserted equity valuation module needs to be linked to other modules within the financial model in order to correctly calculate the DCF equity valuation. This is a complex example, but for those familiar with DCF valuations, the following data must be linked into the equity valuation module:
- Cash flow available to equity;
- Tax paid;
- Earnings before interest, tax, depreciation and amortization (EBITDA); and
- Closing debt balances.
You can learn more about DCF valuation theory from the financial modelling resources on the Best Practice Modelling website.
After doing this, the financial model contains a DCF equity valuation, as shown below in the table of contents, which has been compacted to highlight the newly-added components:
Amazingly, this sophisticated equity valuation analysis was inserted into the financial model in minutes, with only a few formulas required to link it to the surrounding model outputs. As a result, assuming that the source equity valuation module has integrity, the model developer instantly has confidence that the new equity valuation in the financial model is also reliable and correct.
As demonstrated by these steps, the manual implementation of Modular Spreadsheet Development is somewhat fiddly, but it is possible as long as the content within all workbooks and module files is sufficiently standardized to support interchanging components.
The implementation steps used in this simple example can be used to insert any pre-existing module into an existing modular spreadsheet, thereby greatly reducing model development time, cost and risk.
2. VBA automated implementation
Excel users with intermediate to advanced VBA skills will probably have recognised while reading the manual implementation steps that VBA code can be written to automate to automate the majority of this process.
A full discussion and example VBA code is outside the scope of this article, but if you’re considering have a go at this here are some general tips:
- Use a user form containing list box control with its MultiSelect property set to fmMultiSelectMulti to allow users to select the sheets containing the module assumptions and outputs to be imported;
- Ensure that Application.DisplayAlerts is set to FALSE before running the code used to copy sheets to prevent prompts being displayed to users; and
- Use the VBA Range.Replace function to redirect formula links after moving the imported module assumptions and outputs onto existing sheets.
These steps were in fact the steps that I first took when developing an add-in for our organization to automate the insertion of modules. 10 years later, this add-in is called bpmModules, and I’ve provided an overview of it below.
3. Commercial add-in implementation
When I first started implementing Modular Spreadsheet Development, I did it manually. It was fiddly, but as my colleagues and I built more complex modules, it soon became much quicker than re-building content from scratch in each model.
In 2004 we started refining a basic Excel add-in to automate the insertion and deletion of modules, and this once again saved us a lot of time and reduced the risks involved in these processes. But three main issues still haunted us:
- Models often differed in term – e.g. one model might be 5 years long while the next might be 10 years long;
- Every model required a different number of categories – i.e. one might require 3 revenue categories while the next might require 20 revenue categories; and
- Manually entering and removing formula links between modules before deletion and after insertion was tedious and error-prone.
So we set out to build a comprehensive add-in to make Modular Spreadsheet Development quick and easy within Excel. We called it bpmModules, and thought it would take a couple of years to develop.
It ended up taking 10 years to develop, and only after we completed it did we realize that we’d effectively created a modular content creation, management and sharing system for Excel. It’s a mouthful, but that’s exactly what it is, and with it you can do things like:
- Create your own modules;
- Use and edit other people’s modules;
- Insert, delete and link modules;
- Automatically change the term of a model; and
- Automatically add and remove categories without manually editing formulas, etc.
bpmModules also allows you view and manage the modules within your spreadsheet via simple diagrammatic interfaces, such as the one below that shows all the modules in the underlying Excel workbook:
We’ve made the Lite version of this add-in free, and you can download thousands of free modules fromthe downloads section of BPM’s website. The software does become commercial once you start building larger models, but by this stage it is hopefully saving you enough time to justify the investment.
Comparison of approaches
Modular Spreadsheet Development is an awesome concept capable of revolutionising the way spreadsheets are created, managed and shared. Unfortunately, Excel is not by default modular, so to implement Modular Spreadsheet Development you will need to standardize your spreadsheets and then use one of the three implementation methods discussed in this article.
I’ve provided a summary of the advantages and disadvantages of each of these implementation methods below:
- Manual implementation is free but requires rigid standardization and is somewhat fiddly, thereby creating risks of errors when inserting modules into workbooks;
- VBA automated implementation is free and less fiddly then manual implementation, but still requires rigid standardization and at least intermediate VBA skills to develop a reliable Excel add-in; and
- Commercial add-in implementation is not free for larger models but provides an automated mechanism for creating, re-using and sharing modular content in Excel.
Each of these methods provides the core efficiency gains resulting from Modular Spreadsheet Development, so your choice depends largely on your existing Excel skills and the time, cost and risk savings you estimate you would achieve via automation.
Source files
Click here to download example workbooks [zip file].
The following workbooks can be used to replicate the manual Modular Spreadsheet Development example provided in this article, and consist of the financial model before and after the insertion of the equity valuation module, and the workbook containing this module:
- Financial Model.xlsb
- Equity Valuation Module.xlsb
- Financial Model (with Equity Valuation).xlsb
The following workbooks can be used to demonstrate the automation of this process using the bpmModules Excel add-in:
- Financial Model (bpmModules).xlsb
- Equity Valuation.bpm
- Financial Model (with Equity Valuation) (bpmModules).xlsb
Note that you will need to download and install a trial of bpmModules to insert the equity valuation module (with the file extension *.bpm) into the bpmModules-created financial model.
More information
You can watch a range of Modular Spreadsheet Development movie tutorials via the following link:
www.bestpracticemodelling.com/chandoo/msd
Download thousands of modular Excel workbook examples from:
www.bestpracticemodelling.com/downloads/excel_models
Download thousands of modules from:
www.bestpracticemodelling.com/downloads/modules
Watch bpmModules build a financial model in less than 1 minute
Added by Chandoo
Thanks Michael for writing these very detailed articles on Modular Spreadsheet Development to spread the awareness among our readers. With your help, I am sure many modeling professionals & analysts around the world can embark on the time-saving & fruitful journey of modular development.
If you enjoyed these articles, Please take a minute and say thanks to Micheal. Also please share your thoughts, implementation notes & experiences with us using comments.
41 Responses to “How to use Excel Data Model & Relationships”
Data is Excel 2013 behaves so much like a OLAP cube when using with PivotTables. And this is actually wow. Consider learning not just DAX but MDX too 🙂 Happy Excel
@Chandoo.. Have a nice and safe time in US. Best Wishes. And when they are publishing your interview in Entrepreneur 🙂
I have been using PowerPivot in Excel 2010. My understanding was (via PowerPivot Pro blog) that Power Pivot would NOT be available in Excel 2013 in all versions; my recollection is that it was only going to be available in certain enterprise subscription editions. Thus, for individual users, it will no longer be available? For that reason I have moved some of my projects to Tableau, and do not expect to upgrade to Excel 2013.
Can you confirm the availability of Power Pivot for all Excel 2013 users , or will it be restricted and unavailable for some users?
@Buzz
I believe that Power Pivot is included with all MS Office installs in 2013.
Have a read of:
http://office.microsoft.com/en-001/excel-help/version-compatibility-between-powerpivot-data-models-in-excel-2010-and-excel-2013-HA103929426.aspx
http://office.microsoft.com/en-001/excel-help/whats-new-in-powerpivot-in-excel-2013-HA102893837.aspx
http://en.wikipedia.org/wiki/Microsoft_Office_2013#Comparison (Note that Excel isn't listed as having Multiple editions)
This is something that Excel is doing very poorly: letting people know what's available with what. It also doesn't help that they have so many packages and versions.
Excel Professional Plus has Power Pivot and Power View. I went through this last week. Spent 4 days upgrading from 365 Home Premium ($99 for the 1-yr subscription) to an additional $7/month for Professional Plus.
Also, one page on Microsoft's site says that their Sharepoint Online Package 2 has Power View. NO! It's not true. Sharepoint Online Package 2 will INTERACT with a spreadsheet that has PP or PV, Package 1 will not.
Just this weekend I upgraded from Home Premium to Professional Plus and spent time with Power View and PowerPivot.
Up to that point I never saw myself in VLOOKUP Hell, and it may not be going away any time soon. I'm surprised to discover how many of my clients are still on Excel 2003. And then I have Mac users who don't have a lot of this great stuff available to them at all.
These are great features and I'm going to dive into the Data Models. Unfortunately, I suspect, for me, the practical use may be limited to blogposts because I can't teach Power View in my workshops or send a client a spreadsheet that has a Power View in it.
Hi OZ,
I think the Microsoft would only upgrade the excel to a certain level instead of making it so powerful that it might threat their BI product. You know these "powerful" stuff can be easily done with a entry level crystal reports version.
Glad to listen to ur opinion on it.
I spent quite some time and energy on Excel and used it a lot, but now I am focusing energy on BI software like crystal reports.
We both know that based on the technology today. All the time we spend on the Macro and advanced function of Excel can be done easily with other softwares which costs only hundreds of bucks.
@Thondom
I don't think Excel tries to be the solver of all problems
It is a generic tool
Which for about 95% of people will do what they want 95% of the time
There will always be specifics where specific custom software will do better than Excel
It is the commonness of Excel which means that I can send a model to you and it will work , most of the time, that is its strength, of course combined with its flexility in being able to be adapted to suit most needs
Hi Hui,
You are right.
But,
for the business and individual, who spend too much resource on Excel to meet their BI requirements and other processing requests.
Should they open their eyes to other ways to do it, in this age? Especially for many people try too much time to process stuff with thousands lines of macro programming.
It is just as when human being created gun fire, the martial arts would not be that effective.
Ppl need to be prodent when they choose their solution.
Hi guys, I just came across your conversation. I have an example of BI vs. Excel stuff. Here in Russia there is an ERP-system called "1C". It became a defacto standart for accounting, planning and BI / analytics. It is positioned as a flexible and powerful system and it really is.
But its reporting abilities aren't user-friendly (or maybe just not me-friendly).
Many reports require programming and all those SQL things, so that is common for a company to have a couple of programmers who develop and code those reports.
So the common solution is to export data to Excel and then process it to be more suitable for further analysis or reporting.
Well, it's obviously not a rule of thumb that special BI software can outperform Excel in day-to-day routine.
Hi Chandoo, thanks for publishing great Excel information. Pardon the ignorance as I havent used Data Model nor PowerPivot. But having seen your video clip on PowerPivot, how does Data Model differ from PowerPivot - the "process" seems familiar? Have a great day! And Excel to new heights! Regards,
@Tris, one main difference is that Data Model is part of Excel Home Premium. You've got to upgrade to ProPlus to get PowerPivot and Power View.
Also, keep in mind that Data model only lets you combine tables. If you want to create powerful measures (for example show % improvement over last year sales), then PowerPivot is the way to go. For a tour of PowerPivot, visit http://chandoo.org/wp/2013/01/21/introduction-to-power-pivot/
Excellent posting, some pride themselves for having sheets with thousands of formulas or complicated formulas, but in the end the important thing is to work as little as possible.
@Nolberto let's not gloat yet. Some people are forced to have thousands of complicated formulas when they don't have the fancy tools. I'm sad for the 2003 users who have to use SUMPRODUCT when the rest of us have SUMIFS available.
In the end, I think the important thing is clean, trustworthy data--however you arrive at it. People survived more than 300 years with slide rules and paper. No PowerPivot for the Wright Brothers.
hi chandoo,
i added 2 column into sales, 1st column vlookup customer ID to CUST sheet to get the male or female, then 2nd column vlookup Product ID to Product sheet to get the product name, then after that i make pivot table out of sales sheet.
but then the result is really different from yours
the purposes is just try to do the vlookup vs add to data model to see if they get same result
thanks
Hi Koi,
We are using gender vs. category in the report. Can you try with that? I am sure the results will match.
ups sorry, didnt see that you're filtering using slicer..then it is good now the result are same with less effort 🙂
thanks
Hi Chandoo, .I am interested to know whether we can build a star schema or snow flake data models through relations in Excel? (trying to correlate with Qlikview)
Hi there,
You can create a Star schema for sure. Snow-flake is possible too. As long as all relationships are one to many (or one to one) anything is possible.
What if customer.profession change its value after sometime?
Supposed we have monthly data for Sales. What if one customer is a doctor in Feb, then a pilot in October, for example?
How to build data model for such that situation?
Thank you.
[...] Introduction to Excel 2013 Data Model & Relationships [...]
Hello ,
I find this option similar to that of MS Access.
In MS Access as well we have relationship concept and once you create a relationship, you can start creating number of queries based on that.
But MS Access is not so user friendly and basically its database. Good that we are getting those options/functions in Excel.
Thanks for sharing this info.
Regards,
Raghavendra Shanbog
What is star schema and snow flake.??? Can we have next article on that if it is useful for us???
Hi there, can anyone help? I tried testing this out in Excel using two tables. When I go to the Data tab the Relationships button does not appear at all. I am using Microsoft version 14.0.4760.1000, Microsoft Office Professional Plus 2010. Does this version have this capability? Or is there an add-in required?
[…] even a layperson can perform if they have the almighty Excel 2010 and PowerPivot installed. Or Excel 2013′s Data Model, which lets you mash up data from Excel Tables and serve them up directly as PivotTables with not a […]
Chandoo/Hui,
The dates grouping feature does not seem to work in Data Model. Is that true or am I making a mistake somewhere?
I don't think this is really for "lookups"...
Try creating a pivot with sale ID and customer name in row fields. It will give you ALL customer names per sale ID.
You'd need to use RELATED function in a new column in powerpivot if you want something equivalent to "vlookup"
Please explain the difference between data model and power pivot, the functions of both of them are different and similar
thanks
[…] Handling large volumes of data in Excel—Since Excel 2013, the “Data Model” feature in Excel has provided support for larger volumes of data than the 1M row limit per worksheet. Data Model also embraces the Tables, Columns, Relationships representation as first-class objects, as well as delivering pre-built commonly used business scenarios like year-over-year growth or working with organizational hierarchies. For several customers, the headroom Data Model is sufficient for dealing with their own large data volumes. In addition to the product documentation, several of our MVPs have provided great content on Power Pivot and the Data Model. Here are a couple of articles from Rob Collie and Chandoo. […]
I need to use a slicer to allow a user to select vendor by name. In the background, I need to obtain the vendor ID to link to multiple datasets where the name may not be spelled consistently. Any advice?
@Bernadette
You can use the technique described here:
http://chandoo.org/wp/2016/05/11/apply-conditional-formatting-using-slicers/
I'd suggest try that and then if you have troubles ask the question in the Chandoo.org Forums http://forum.chandoo.org/
Attach a sample file to simplify the response
I've tried this in Excel 2016. It works great.
I can even create Cube Formulas on the Data model after I've inserted the pivot table.
Just for the fun of it, I tried to see if I could do Cube Formulas without creating the pivot table in advance. I can define Cube members, but it seems as if the measure part is playing tricks on me.
I can't get a Cube Value for Chocolates sold to Male customers.
With the Pivot created the formula looks like this (and works fine)
=CUBEVALUE("ThisWorkbookDataModel";"[Customer].[Gender].&[Male]";"[Product].[Category].&[Chocolates]";"[Measures].[Sum of Quantity]")
Does anyone know how I can solve this, or am I asking the impossible?
I want to see the video on this topic
What if customer.profession change its value after sometime?
Supposed we have monthly data for Sales. What if one customer is a doctor in Feb, then a pilot in October, for example?
How to build data model for such that situation?
Thank you.
In such case, you need to make relationships based on two columns. This kind of feature is not supported in Excel. You can use Power Query to merge tables based on multiple columns and return a consolidated giant table to Excel for reporting.
Is it able in MS Access?
I have never used access before.
thanks chandooo your article is very helpfull for troubling peoples' especially in office environment under boss pressure.
Here is an introduction to PowerPivot.
The link above is broken
Hi. This has really taken my interest.. I have huge data tables to work with...and I use vlookup to fetch certain data. I have different data in different sheets...
Like customer sales (customer code, product code,qty, piece rate, total amount, branch code) data in one sheet
Branch details in another (branch code, branch address, state , region)
Customer Geographical Data in third sheet (region, region name)
Product details in fourth sheet (product code, product description and related)
Now I use a vlookup to get branch name, state and product name respectively into my main sheet.
Now what I want is
customer code, product code,qty, piece rate, total amount, branch code) data in one sheet, branch address, state , region, region name, product description
Can't his be done thru data model... I tried but it's not working... Eitherway, I will gonthru thr session on e again and give a try... Any help, is appreciated. Thankyou
Dear All,
i am striving to do reverse relationship in Power pivot ,
example : -
1 - Data sheet
2. - Source data
step to stops - import first data sheet in power piovt and then source data , made relationship with both sheet , after created relationship i am able to do put related formula in source data sheet only (=releted('Source data'[Amount]), if i go to put formula in data sheet , parameter of Source data are not visible ,
could someone educate me how can i do , and utilize related formula in data sheet.