As part of our Excel Interview Questions series, today let’s look at another interesting challenge. How-to handle more than million rows in Excel?
You may know that Excel has a physical limit of 1 million rows (well, its 1,048,576 rows). But that doesn’t mean you can’t analyze more than a million rows in Excel.
The trick is to use Data Model.
Excel data model can hold any amount of data
Introduced in Excel 2013, Excel Data Model allows you to store and analyze data without having to look at it all the time. Think of Data Model as a black box where you can store data and Excel can quickly provide answers to you.
Because Data Model is held in your computer memory rather than spreadsheet cells, it doesn’t have one million row limitation. You can store any volume of data in the model. The speed and performance of this just depends on your computer processor and memory.
How-to load large data sets in to Model?
Let’s say you have a large data-set that you want to load in to Excel.
If you don’t have something handy, here is a list of 18 million random numbers, split into 6 columns, 3 million rows.
Step 1 – Connect to your data thru Power Query
Go to Data ribbon and click on “Get Data”. Point to the source where your data is (CSV file / SQL Query / SSAS Cube etc.)
Step 2 – Load data to Data Model
In Power Query Editor, do any transformations if needed. Once you are ready to load, click on “Close & Load To..” button.
Tell Power Query that you want to make a connection, but load data to model.
Now, your data model is buzzing with more than million cells.
Step 3 – Analyze the data with Pivot Tables
Go and insert a pivot table (Insert > Pivot Table)
Excel automatically picks Workbook Data Model. You can now see all the fields in your data and analyze by calculating totals / averages etc.
You can also build measures (thru Power Pivot, another powerful feature of Excel) too.
How to view & manage the data model
Once you have a data model setup, you can use,
- Data > Queries & Connections: to view and adjust connection settings
- Relationships: to set up and manage relationships between multiple tables in your data model
- Manage Data Model: to manage the data model using Power Pivot
Alternative answer – Can I not use Excel…
Of course, Excel is not built for analyzing such large volumes of data. So, if possible, you should try to analyze such data with tools like Power BI [What is Power BI?] This gives you more flexibility, processing power and options.
Watch the answer & demo of Excel Data Model
I made a video explaining the interview question, answer and a quick demo of Excel data model with 2 million rows. Check it out below or on my YouTube Channel.
Resources to learn about Excel Data Model
- How to set up data model and connect multiple tables
- A quick but detailed introduction to Power Query
- Create measures in Excel Pivot Tables – Quick example
- Create data model in Excel – Support.Office.com
- How to use Data Model in Excel – GorillaBI.com
How do you analyze large volumes of data in Excel?
What about you? Do you use the data model option to analyze large volumes of data? What other methods do you rely on? Please post your tips & ideas in the comments section.
27 Responses to “9 Box grid for talent mapping – HR for Excel – Template & Explanation”
Great stuff! I can understand how to add a slicer to the pivot table, but how do you implement the departmental selector on the 'Filter' formula scheme?
Just saw this on your Youtube channel, and it’s areat idea...!
An easy way to overcome the "ugliness" of pivot tables and get it to look nice (in the format of the Output sheet), would be to simply build a sheet with the nice map at the top, a pivot underneath it and a slicer next to formatted map and then reference each of the 9 cells in the formatted map to the “related” cell in the Pivot.
Keep up the good work!
/Claus
Thanks Claus. That is a great idea 🙂
Hi Chandoo,
This is great! Curious how to make additional columns operate the same as the Department column (ex. have a "manager column") that would allow you to sort a 9 box by manager, area, or team in addition to department?
Feel free to email me if needed! mfry01@minnetronixmedical.com
Happy New Year
Madison Fry
I am curious about the smae thing. I would like to populate the 9 box with other views as well by adding additional columns. IE., I would like to add location, region, etc. Thank you.
This is great, thank you!
How can i see the whole data set of all the teams in the output table. Need a formula that will pick up all the employees
Hello,
Love the template. Thank you. Question - the drop down to pick a department on the Output tab does not seem to work on the downloadable template. Am I doing something incorrectly?
Thank you!
Hi Heather... Thank you. I am using Excel 365 to make the calculations. If you are using an older version of Excel, then the drop-down filter won't work.
Hi
I was able to follow your 9 box grid and modified based my needs. However, you tutorial did not show how to you create the filter for the "Pick a department. Can you kindly share how to create that filter that updated the grid. Thank you.
I am working on this project but I am struggling with the data validation for the department. I copy the worksheets data entry and output as the managers want to see different tabs for each managers.
I updated the source reference for each tab but It does not update the grid based on the new source. The list was updated but it does not populate the grid based on the performance and potential listed.
In addition the hyperlink Update Data and View Talent Map no longer works. Can you please help me.
I keep getting this error message in the pivot table:
This formula is invalid or incomplete: 'The expression is not valid or appears to be incomplete. Please review and correct the expression.
The following syntax error occurred during parsing: Invalid token, Line 1, Offset 14, ‘.
Hi, I used your 9-box excel template with excel 365. First off, thank you so very much. It is incredibly helpful!! My only question is that the boxes aren't big enough for all of the employees (specifically the middle which we call 'Core Employee'). Is there a way to make the boxes larger? Even though it is in excel, I am not able to increase row height (like I normally do in a speadsheet). Any ideas? Thanks again, Jody
Hi Chandoo,
Thanks for the great content. Re. 9 box grid, pls advise how do I increase the size of the box to accommodate more names?
Hi Chandoo,
I figured it out. Excel 365 has the format row height on the ribbon. Thank you
Merci Chandoo pour le modèle proposé,
j'ai une question et un souhait est il possible de développer davantage ce modèle en insérant la photo de chaque employé.
Hi Chandoo!
Great tutorial and tool, thank you! Your tutorial didn't include how to create additional filters on the "Output" tab. Could you please share how you did it?
Can this be done exactly in google sheets?
Hi Chandoo,
Thanks for the video it was really helpful. Is there any way to multi select the dropdown to display multiple or all departments rather than just one at once?
Hello Prish
I have Microsoft 365 and I am struggling to make the boxes larger/unable to increase row height; any idea how you made this work? Specifically in the Output tab where the map is?
Many thanks
Hello Jody, I have Microsoft 365 and I am struggling to make the boxes larger/unable to increase row height; any idea how you made this work? Specifically in the Output tab where the map is? Many thanks
Is there a way to change the 9 box wording descriptions, i.e. Work Horses, to our own internal langauge?
You can edit the file. The descriptions are textboxes.
Hi Chandoo, this is awesome and has worked perfectly. Due to a big organisation the 9 box grid on the output file is too small. I tried adjusting using the row/width ribbon under the format ribbon however it doesn't seem to work. Is there an easier way to adjust this?
Thanks!
When I drag the formula, it doesn't work, and the order I use with the data changes. In the beginning, the order is it is " candidates," " potential," and " performance," but when it goes to another column, it is " Potential," Performance," and "Candidates."Can you help me? Thank you very much, sending love from vietnam
Hi- I am working on the 9 grid project and I am trying to expand the box since I have over 100 names on a few of the columns. How do I do that?
Hi, Thank you this is great stuff and really useful.
As well as department as demonstrated on your clip, how can I display all candidates on the grid at once?
Many thanks in advance