fbpx

Combine multiple Excel files using Power Query [Full example + download]

Power Query - 4 comments

Say you want to combine multiple Excel files, but there is a twist. Each file has few tabs (worksheets) and you want to combine like for like, ie , all Sheet1s to one dataset, all Sheet2s to another dataset…

To make matters interesting each sheet has a different format.

What now?

Of course Power Query to the rescue.

power query man

This is an advanced example of Power Query. If you are a beginner, start with these pages.

Combine multiple Excel files – the problem

Imagine you work in Finance. Your job involves paying employees for their business travel expenses. Every time someone goes on a business trip, they submit a trip expense report. This is an Excel template with two tabs.

  • Travel details tab: for gather personal and travel details
  • Expense details tab: for itemized expense details

As you have a lot of employees, you don’t want to manually scan the files and combine the data. Here is a sample of how these files look.

personal details & travel details - sample data
Tab 1 – Travel details
expense details - itemized - sample data
Tab 2 – Expense Details

You want to combine all the expense files in to one big, consolidated & refreshable travel expense workbook.

combine multiple files using power query

Using Power Query to combine files

"from folder" - Power query

Some of you may already know Power Query’s “Get data from Folder” feature. This helps us easily get & combine multiple excel files in a folder. Unfortunately, this alone will not be helpful for us as our file has two different tabs and we need to combine them separately 😉

Here is the process we need to follow.

Start by placing all the expense reports in to one folder. This can be a folder on your computer or on a network / shared drive.

Now go to “Get Data > From File > Folder”

folder path screen - power query

Point to the folder path and Power Query will show all the files in that folder.

Once satisfied with the list of files (don’t worry if you need to exclude some files, you can do that while editing the query by applying filters), click on “Combine & Edit”.

combine and edit option - files list

Now you will get another screen asking you choose which tabs / tables you want to bring. As we have two sets of consolidations, let’s start with the first one – travel details tab. Select that and proceed.

combine files screen with sample data from one file - power query combine files

At this point, Power Query will create a folder called “Transform sample” and places a few things in it. PQ will also create a query for all the merged data. This is how your Power Query window could look.

editing transform sample - powerquery combine files

Editing the Transform sample query

As you can see, the default combined query data can be useless for our situation. So let’s proceed by editing “Transform sample file from reports” query.

What is Transform sample really?

In this sample query, you can make any changes and PQ will apply them to all the files in the folder before combining them to one gain data set.

Steps to turn travel details to a table

Our travel details sample needs to become one row table so that we can effectively merge multiple files. To do so, follow these steps:

  1. Remove blank / heading rows on the top.
  2. Remove any nulls or unnecessary rows from column 1
  3. Transpose the table
  4. Promote first row to headers

This is how the output would look after the process.

transforming travel details to a tabular format - steps required

Combine all files

Now that we have edited transform sample, time to go back to the “reports” query to see the output. If you are happy with it, rename the query and load it in to Excel (or Power BI).

Combined travel details

combined data

Combining expense details

The process is same for expense details consolidation. Start by creating a fresh “from folder” query. As expense details are in a table, there is no need to do any additional changes to the transform sample. Simply combine everything from “expenses” tables and you are done.


Combined expense details

combined data - expenses

Download sample files to practice this

Power Query can be tricky to explain with blog posts alone. That is why I made few sample files and consolidated workbook. Click here to download everything.

Try to merge the files in “reports” folder using your own logic / transformation steps. Share your story / tips in the comments.

I get an error when merging data from files

There are many reasons why Power Query may show an error when connecting to a folder. Here is a check list to help you.

  • Make sure the folder path is valid and accessible. If you created the query on one computer and try to refresh it from another, chances are it won’t work. Use shared network drives or change path in Power Query steps before refreshing.
  • Files are loaded, but merged query errors. This can happen if you edited the transform sample. Usually Power Query adds “Changed type” steps automatically after you do something. These changed type steps refer to column names in the query and change data types. If you edit the transform sample and alter the column structure of table, then the query will fail. The solution? Simple, delete all the automatically added changed type steps.
  • Some files should not be loaded, but they load and mess up the results. Before making any transformations, set up filters based on file type or names. This way you can prevent loading unnecessary files.

Do you merge / combine files with Power Query?

I do this all the time. My recent win was to merge 24 PDF credit card statements (2 types of cards over last 12 months) to one big table of data so that I can see trends and find out where I spend most.

What is your experience with combine multiple Excel files / folder query feature? What are some of your favorite tricks with this? Please post them in the comments section.

This article is inspired from a comment by Sourav.

Chandoo

Hello Awesome...

My name is Chandoo. Thanks for dropping by. My mission is to make you awesome in Excel & your work. I live in Wellington, New Zealand. When I am not F9ing my formulas, I cycle, cook or play lego with my kids. Know more about me.

I hope you enjoyed this article. Visit Excel for Beginner or Advanced Excel pages to learn more or join my online video class to master Excel.

Thank you and see you around.

Related articles:

Written by Chandoo
Tags: , , , , ,
Home: Chandoo.org Main Page
? Doubt: Ask an Excel Question

4 Responses to “Combine multiple Excel files using Power Query [Full example + download]”

  1. Kelly says:

    Chandoo- can you share your solution for combining 24 PDF documents as mentioned above.

    Do you merge / combine files with Power Query?

    I do this all the time. My recent win was to merge 24 PDF credit card statements (2 types of cards over last 12 months) to one big table of data so that I can see trends and find out where I spend most.

    I have a similar problem that I am solving for. Thanks

    • Chandoo says:

      Hmm, I will need to find some generic PDF dataset as I don't want to share my credit card statements online 😀
      But here is the process, if you want to try.

      1) You need to use Power BI Power Query (as Excel PQ doesn't yet support PDF import)
      2) Place all PDFs in a folder and connect with "From Folder" query
      3) PQ will detect structured tables in your PDFs. Select the correct one.
      4) Edit transform sample so you can change the results or multi-select tables and append them to one big table.

      Once the final query has what you need, just select entire table in PQ, press CTRL+C and paste it in Excel for further analysis or leave in PBI for visualizing.

      Give it a try

  2. Nam says:

    Is there a MAcro Language that can launch the Power Query?

  3. Pradeep Nair says:

    Hi Chandoo, My problem with Power Query is that my individual sheets are password protected and hence will not append. How can you solve this?

Leave a Reply


« »