As part of our Excel Interview Questions series, today let’s look at another interesting challenge. How-to handle more than million rows in Excel?
You may know that Excel has a physical limit of 1 million rows (well, its 1,048,576 rows). But that doesn’t mean you can’t analyze more than a million rows in Excel.
The trick is to use Data Model.
Excel data model can hold any amount of data
Introduced in Excel 2013, Excel Data Model allows you to store and analyze data without having to look at it all the time. Think of Data Model as a black box where you can store data and Excel can quickly provide answers to you.
Because Data Model is held in your computer memory rather than spreadsheet cells, it doesn’t have one million row limitation. You can store any volume of data in the model. The speed and performance of this just depends on your computer processor and memory.
How-to load large data sets in to Model?
Let’s say you have a large data-set that you want to load in to Excel.
If you don’t have something handy, here is a list of 18 million random numbers, split into 6 columns, 3 million rows.
Step 1 – Connect to your data thru Power Query
Go to Data ribbon and click on “Get Data”. Point to the source where your data is (CSV file / SQL Query / SSAS Cube etc.)
Step 2 – Load data to Data Model
In Power Query Editor, do any transformations if needed. Once you are ready to load, click on “Close & Load To..” button.
Tell Power Query that you want to make a connection, but load data to model.
Now, your data model is buzzing with more than million cells.
Step 3 – Analyze the data with Pivot Tables
Go and insert a pivot table (Insert > Pivot Table)
Excel automatically picks Workbook Data Model. You can now see all the fields in your data and analyze by calculating totals / averages etc.
You can also build measures (thru Power Pivot, another powerful feature of Excel) too.
How to view & manage the data model
Once you have a data model setup, you can use,
- Data > Queries & Connections: to view and adjust connection settings
- Relationships: to set up and manage relationships between multiple tables in your data model
- Manage Data Model: to manage the data model using Power Pivot
Alternative answer – Can I not use Excel…
Of course, Excel is not built for analyzing such large volumes of data. So, if possible, you should try to analyze such data with tools like Power BI [What is Power BI?] This gives you more flexibility, processing power and options.
Watch the answer & demo of Excel Data Model
I made a video explaining the interview question, answer and a quick demo of Excel data model with 2 million rows. Check it out below or on my YouTube Channel.
Resources to learn about Excel Data Model
- How to set up data model and connect multiple tables
- A quick but detailed introduction to Power Query
- Create measures in Excel Pivot Tables – Quick example
- Create data model in Excel – Support.Office.com
- How to use Data Model in Excel – GorillaBI.com
How do you analyze large volumes of data in Excel?
What about you? Do you use the data model option to analyze large volumes of data? What other methods do you rely on? Please post your tips & ideas in the comments section.
31 Responses to “Beautiful Budget vs. Actual chart to make your boss love you”
Would be considerably easier just to have a table with the variance shown.
On Step 3, how do you "Add budget and actual values to the chart again"?
There are a few ways to do it.
Easy:
1) Copy just the numbers from both columns (Select, CTRL+C)
2) Select the chart and hit CTRL+V to paste. This adds them to chart.
Traditional:
1) Right click on chart and go to "select data..."
2) From the dialog, click on "Add" button and add one series at a time.
One more way to accomplish it is just select the columns into chart. Press Ctrl+C and then press Ctrl+V
Regards
Neeraj Kumar Agarwal
Unfortunately, this doesn't seem to work for me in Excel 2010. The "Var 1" and "Var 2" columns cannot combine two fonts to display the symbol and the figure side-by-side.
Secondly, there is no option to Click on “Value from cells” option when formatting the label options. The only options provided are Series Name, Category Name or Value.
@TheQ47... the emoji font also has normal English letters, so if you use that font, then you should be ok. I am assuming your computer doesn't have that font or hasn't been upgraded for emoji support.
Reg. Excel 2010, you can manually link each label to a cell value. Just select one label at a time (click on labels, wait a second, click on an individual label) and press = and link it to the label var 1 or var 2.
I am using excel 2010, please explain how to apply Step 12
Regards
Neeraj Kumar Agarwal
Hi Neeraj,
"Value from cells" option is only available in Excel 2013 or above. In older versions, you have to manually adjust the label value by linking each label seperately.
Read this please: https://chandoo.org/wp/change-data-labels-in-charts/
Sir, you are just awesome.
Your creativity has no limit.
Regards
Neeraj Kumar Agarwal
Hi Chandoo,
I just found your website, and really love it. It helps me a lot to be an Excel expert 😉
Currently I am facing with a problem at step 11:
Var1 Var2
D30%
A5%
B0%
B4%
B7%
C10%
C13%
D27%
I42%
Though at mapping table, I used windings, here formula uses calibra. How I can change it? I am able to change only the whole cell. In this case numbers will be Windings too.
Thanks for your help!
Hi Mariann... Welcome to Chandoo.org and thanks for your comment.
If you wanted to use symbols from wingdings and combine them with % numbers, then you need to setup two labels. One with symbol, in wingdings font and another with value in normal font. Just add the same series again to the chart, make it invisible, add labels. You may need to adjust the alignment / position of label so everything is visible.
[…] firs article explains how you can enhance your charts with symbols. You can simply insert any supported symbol into your data and charts. To some extend you can […]
You're a good person, thank you to share your knowledge with us, I will try to do in my work
Great visualization of variance. My question is that is this possible in powerbi?
How would you go about it?
HELLO, WHY CANT I FIND VALUES FOR LABELS IN EXCEL 2013
Dear chanddo sir,
What to do if we have dynamic range for Chart. How this will work. can you able to make the same thing works on dynamic range.
Sir Chandoo,
Good Day!
First, I'd like to say that I am very grateful for your work and for sharing all these things with us.
I tried to do this chart but it seems that the symbols don't work with text (abs(var%),"0%") unless we keep the Windings font style.
The problem is, it converts the text into symbol as well and you wont see the 0% anymore. I'm using Windows 7.
WOW - Segoe UI Emoji
This is the greatest discovery for me this month 🙂 Thanks for sharing.
Here's my two-cents:
https://wmfexcel.com/2019/02/17/a-compelling-chart-in-three-minutes/
Sir This is awesome chart, and very easy to made because of your way to explain is very simple , everyone can do. Thank you
one problem i am facing, I hv made this chart , but when i am inserting data table to chart it is showing two times , how can i resolve this
in this chart when i am adding new month data for example first i made this chart jan to mar but when i add data for the apr month graphs updated automatically but labels are missing for that new month
Hi Renuka,
Please make sure the formulas for labels are also calculated for extra months. Just drag down the series and set label range to appropriate address.
So I am playing with the Actual chart here - but amounts are bigger than your - you have 600 as Budget - my budget is 104,000 - is there a way to shorten that I am unaware of
thank you - I LOVE YOUR SITE
Thanks for the tips and tricks on Excel. In the Planned versus Actual chart examples, you use multiple values (ex. multiple Categories in above). How can this be done when we have only 1 set of values? For example if I have only this:
Planned Actual
SOW Budget 417480 367551
How can I create a single bar chart like the one above?
Thank you Chandoo.
This one is just perfect for my Quarterly Review presentation on Operational Budget against Actual Performance for the Hospital I'm currently working with.
Just Subscribed today (10 minutes ago)
Is there a way to make the table of data into a pivot table to be able to add a slicer for the graph due to many different categories and months?
Hi, I tried to modify you template with something appropriate for me, and I found a problem. this template was modified by me started with excel 2010, then 2016 and finally 2019. Same thing - somehow appear an error - or didn't show the emoticons for positive percentage or doubled the emoticons for some rows. I suspect to be from excel. if is need it I can sand you my xlsx for study. Please help if you can.
Hi Chandoo,
Could you please check the Var Formula in Step1. You have mentioned budget-actual and when i did this i got different values but when reversed like actual-budget i got the actual value what you have demonstrated in step1.
Please share your view.
This is a great chart (budget vs. actual). However, in trying recreate it, I cannot color in the UP Down bars individually, and they all become formatted with the same color. I'm using Office 365. Look forward to the feedback.
Thanks.
Dan
pls explain in detail step 7
While in the Excel sheet you have used following formula for Var
Var = Actual - Budget
But
in the note, you have written
Var = Budget - Actual
Good Presentation and Data information.thank you so much chandoo.