How to consolidate data that is different shapes [BYOD]

Posted on February 16th, 2015 in Excel Howtos , Pivot Tables & Charts , VBA Macros - 12 comments

Last week, I asked my email newsletter readers to submit “one data analysis problem you are struggling with”. We called it BYOD – Bring your own data. More than 100 people have emailed various interesting (and often very difficult) problems. This week (between 16th of February to 20th of February), let’s take a look at some of these problems and solve them.

Consolidating data in different shapes

We can use either VBA or Excel’s consolidation features to combine data that has same shape (ie same number & type of columns). Here is one way to do it.

But what if we need to consolidate data that is in different shapes?

Something like this:

consolidating-data-in-different-shapes-excel-problem

In such cases, we can use 3 powerful tools.

  1. Multiple Consolidation Ranges – Pivot Tables
  2. VBA
  3. Power Query

So let’s examine how to use these approaches to consolidate data in different shapes.

Multiple consolidation ranges – Pivot

The first approach involves using a hidden feature in Excel, called as multiple consolidation ranges.

  1. Let’s say your data is spread across multiple worksheet tabs
  2. Go to any tab and press ALT+D P (press D first, release the key, press P)
  3. This opens Insert Pivot dialog from Excel 2003 days

    Insert a multiple consolidation ranges - pivot table using Excel

  4. In the next screen, select “I will create page fields”
  5. In the step 3, select & add the ranges one at a time and click finish.
    How to set up multiple consolidation ranges in Excel - Pivot tables
  6. And your consolidation is done!

[Related: Introduction to Excel Pivot Tables]

Combining data in multiple shapes – Using VBA & Power Query

For these 2 methods, please watch below video.

(Click here to see this video on Chandoo.org youtube channel)

Download Example Workbook

Please click here to download the example workbook. It contains only the VBA solution. For pivot table solution, use the above steps. For Power Query solution, follow the video and create it on your Excel.

How do you consolidate such data?

My preference is to use VBA as consolidation tends to be a repetitive task (every month we need to consolidate) and it works in any version of Excel. That said, I also like the flexibility and diversity Power Query offers. You can do so much more than just consolidating with PQ.

What about you? Have you faced any such consolidation challenges in your work? How did you solve them? Please share your thoughts and solutions in the comments section.

Written by Chandoo
Tags: , , , , , , , ,
Home: Chandoo.org Main Page
? Doubt: Ask an Excel Question

12 Responses to “How to consolidate data that is different shapes [BYOD]”

  1. Dani says:

    You can add a column with the company name, the range names company1, company2 by starting with a blank query and the function = Excel.CurrentWorkbook(), it will list all the range names in the workbook then you can filter the list for names that contains "company", display the data and unpivot all the data at once. The result will be identical to your query with and additional column for the company name. Furthermore from now on all new range names that includes the word "company" will be included.

    Here is the M code:
    let
    Source = Excel.CurrentWorkbook(),
    #"Filtered Rows" = Table.SelectRows(Source, each Text.Contains([Name], "company")),
    #"Expand Content" = Table.ExpandTableColumn(#"Filtered Rows", "Content", {"Column1", "Column2", "Column3", "Column4"}, {"Content.Column1", "Content.Column2", "Content.Column3", "Content.Column4"}),
    #"First Row as Header" = Table.PromoteHeaders(#"Expand Content"),
    #"Filtered Rows1" = Table.SelectRows(#"First Row as Header", each ([Product] "Product")),
    #"Unpivoted Columns" = Table.UnpivotOtherColumns(#"Filtered Rows1", {"Product", "company1"}, "Attribute", "Value"),
    #"Reordered Columns" = Table.ReorderColumns(#"Unpivoted Columns",{"company1", "Product", "Attribute", "Value"})
    in
    #"Reordered Columns"

  2. Dani says:

    Just realize this approach will work only if all tables have the same column headers, it is not the case here.

    However we can achieve the same result by instead creating a power query function and using it for each range name that starts with "company". Just test it with your file and it works.

    M code for the function:
    let
    FormatTable=(t as table)=>
    let
    Source = t,
    #"First Row as Header" = Table.PromoteHeaders(Source),
    #"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"First Row as Header", {"Product"}, "Attribute", "Value"),
    #"Renamed Columns" = Table.RenameColumns(#"Unpivoted Other Columns",{{"Attribute", "Year"}})
    in
    #"Renamed Columns"
    in FormatTable

    M code for the query:
    let
    Source = Excel.CurrentWorkbook(),
    #"Filtered Rows" = Table.SelectRows(Source, each Text.StartsWith([Name], "company")),
    #"Added Custom" = Table.AddColumn(#"Filtered Rows", "Custom", each FormatTable([Content])),
    #"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Content"}),
    #"Expand Custom" = Table.ExpandTableColumn(#"Removed Columns", "Custom", {"Product", "Year", "Value"}, {"Product", "Year", "Value"})
    in
    #"Expand Custom"

  3. Rizwan says:

    Dear Dani,

    Please elaborate it further.

    Regards, Rizwan

  4. Gretchen says:

    I would use Data Consolidation and Consolidate by Category.

  5. pmondollot says:

    Didn't know the multiple consolidation ranges. Thanks!!

  6. Miguel says:

    Awesome stuff, Chandoo!

    Here's another take on Power Query with a really direct approach.

    ---
    let
    Source = Excel.CurrentWorkbook(),
    #"Filtered Rows" = Table.SelectRows(Source, each Text.Contains([Name], "company")),
    #"Added Custom" = Table.AddColumn(#"Filtered Rows", "Custom", each Table.PromoteHeaders([Content])),
    #"Added Custom1" = Table.AddColumn(#"Added Custom", "Custom.1", each Table.UnpivotOtherColumns([Custom], {"Product"}, "Attribute", "Value")),
    #"Removed Other Columns" = Table.SelectColumns(#"Added Custom1",{"Name", "Custom.1"}),
    #"Expand Custom.1" = Table.ExpandTableColumn(#"Removed Other Columns", "Custom.1", {"Product", "Attribute", "Value"}, {"Product", "Attribute", "Value"}),
    #"Changed Type" = Table.TransformColumnTypes(#"Expand Custom.1",{{"Attribute", Int64.Type}}),
    #"Grouped Rows" = Table.Group(#"Changed Type", {"Name", "Product", "Attribute"}, {{"Total", each List.Sum([Value]), type number}}),
    #"Sorted Rows" = Table.Sort(#"Grouped Rows",{{"Attribute", Order.Ascending}}),
    #"Pivoted Column" = let #"Changed Type" = Table.TransformColumnTypes(#"Sorted Rows", {{"Attribute", type text}}, "en-US"),
    #"Pivoted Column" = Table.Pivot(#"Changed Type", List.Distinct(#"Changed Type"[Attribute]), "Attribute", "Total", List.Sum)
    in #"Pivoted Column"
    in
    #"Pivoted Column"

    ----
    Note: this was all done within the PQ UI. I'm not much of a fan of the functions unless there's no other way of doing it.

    Once you have the final table just enable the total row feature of the table and that is it 🙂

  7. Shariq says:

    How can you consolidate data in multiple sheets and multiple format? is there a simple solution for this? can anyone help?

  8. Jordun says:

    You can use Data-Consolidate too.

  9. Shurahbeel Malik says:

    Very useful and informative

Leave a Reply