This is a guest post by Vijay, our in-house VBA Expert.
Last week, we learned how to use SQL and query data inside Excel. This week, lets talk about how we can use VBA to consolidate multiple data sheets from different workbooks into one single worksheet.
Consolidate Data Demo
First, lets take a look at the consolidate data VBA code.
Consolidating Data from different Excel files – the setup
There is one master file (or sheet) which needs to be consolidated by pulling data from multiple source files containing raw data (having the same data structure).
Lets try to make a generic consolidation macro so that we can use this almost anywhere.
We start of by creating a simple table on our sheet, we will call this List.
- On this table essentially we are defining everything that our VBA code needs to know to copy and paste data.
- We start by telling the name of the Excel workbook and then the complete path (location) of the file.
- In the next 2 cells we define what are the starting cell and the ending cell that contains our data.
- Next we are put the name of the worksheet where the data will be pasted. In our example the sheet remains the same however as per your requirements you may put a different sheet name.
- The last option is to specify where to paste the copied data and we only need to tell the start cell address, the code will automatically select the next empty cell in that column and then paste the data from that point onwards.
Let’s understand the code.
Dim strWhereToCopy As String, strStartCellColName As String
Dim strListSheet As StringstrListSheet = “List”
On Error GoTo ErrH
Sheets(strListSheet).Select
Range(“B2”).Select
‘this is the main loop, we will open the files one by one and copy their data into the masterdata sheet
Set currentWB = ActiveWorkbook
Do While ActiveCell.Value <> “”
strFileName = ActiveCell.Offset(0, 1) & ActiveCell.Value
strCopyRange = ActiveCell.Offset(0, 2) & “:” & ActiveCell.Offset(0, 3)
strWhereToCopy = ActiveCell.Offset(0, 4).Value
strStartCellColName = Mid(ActiveCell.Offset(0, 5), 2, 1)
Application.Workbooks.Open strFileName, UpdateLinks:=False, ReadOnly:=True
Set dataWB = ActiveWorkbook
Range(strCopyRange).Select
Selection.Copy
currentWB.Activate
Sheets(strWhereToCopy).Select
lastRow = LastRowInOneColumn(strStartCellColName)
Cells(lastRow + 1, 1).Select
Selection.PasteSpecial xlPasteValues, xlPasteSpecialOperationNone
Application.CutCopyMode = False
dataWB.Close False
Sheets(strListSheet).Select
ActiveCell.Offset(1, 0).Select
Loop
Exit Sub
ErrH:
MsgBox “It seems some file was missing. The data copy operation is not complete.”
Exit Sub
End Sub
We have used the Workbook object to accomplish this task and also the Error handler to trap any errors that may come in case any file is missing.
The current code will display a message box when it is not able to open any file and will stop.
We start by assigning the workbook where we want to consolidate the date to the variable currentWB by using the statement:
Set currentWB = ActiveWorkbook
After this a looping construct has been used to go through all the inputs provided one by one and open the workbooks, it has been assumed these workbooks to contain on the data that we need to copy hence I did not specify the source sheet name, however this can be easily added to this code to add more functionality.
Inside our loop are the 4 variables which are assigned the
1) File name,
2) Copy Range,
3) Where To Copy and
4) Which Column contains the starting cell to paste data.
We open the data workbook by using the Application.Workbooks.Open method.
Once we have our first data workbook open, we assign this to the dataWB variable so that we can easily switch between the two workbooks and close them when the operation has been completed.
Next we select the data that has been assigned to the copy range and copy to the clipboard.
We then switch back to our main workbook and select the sheet where we want to paste the data, I have assigned this to the variable called “strWhereToCopy”. This allows us to paste data onto separate sheets within the same workbook.
I have also made use of UDF (user defined function) to find the last cell in the column that we specify.
Once we have found the last row we then select the next empty cell below that and paste our data then.
Additional things that may be used to enhance this code
1. Since we are using the same instance of Excel we may allow the user to preserve the format of the data being pasted.
2. Allow the user with the option to clear data before new is pasted.
Download Consolidate Data from different files Demo file
Click here to download the workbook.
Please Note: You would need to create the data files on your system, this download only contains the code template to consolidate.
More on VBA & Macros
If you are new to VBA, Excel macros, go thru these links to learn more.
- More Examples on Consolidation
- What is VBA & Macros? Introduction
- Excel VBA Example Macros
- VBA tutorial videos
Join our VBA Classes
If you want to learn how to develop applications like these and more, please consider joining our VBA Classes. It is a step-by-step program designed to teach you all concepts of VBA so that you can automate & simplify your work.
15 Responses to “Highlight Employees by Performance Rating – Conditional Formatting Challenge”
While this might solve the question Shelly asked, there is another option that might be more useful - a pivot table could make a list of people who fall into the various categories, so, if you needed to simply see who got in the top bracket to give them a bonus, you would have that list
Simply sorting by the rankings would work too, but you would knock them out of alphabetical order.
Normal
0
false
false
false
EN-US
X-NONE
X-NONE
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin-top:0in;
mso-para-margin-right:0in;
mso-para-margin-bottom:10.0pt;
mso-para-margin-left:0in;
line-height:115%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;}
The solution I chose makes use of the percentile formula.
The percentile formula returns the value representing the K-th percentile of a range of values. The range of values is the first criteria, and K is the second criteria in the formula.
I applied Conditional Formatting according to the formulas in the order below:
5% =$C6>=PERCENTILE($C$6:$C$33,0.95) Dark Blue
15% =$C6>=PERCENTILE($C$6:$C$33,0.85) Light Blue
65% =$C6>=PERCENTILE($C$6:$C$33,0.1) Green
10% =$C6>=PERCENTILE($C$6:$C$33,0.05) Light Red
5% =$C6<PERCENTILE($C$6:$C$33,0.05) Dark Red
The issue I noted with this approach is that Zambi was not highlighted in my solution as it is in the solution provided. Unless I am mistaken, and I very well may be, the 10th percentile for this data set is at 2.21, so Zambi would fall above the 10th percentile with a PR of 2.3.
The first step to this was figuring out the 'buckets'; what scores should fall into each range. In attempting to match the formatting of the spreadsheet, I determined the buckets below.
5% = 95% to 100%
10% = 90% up to but not including 95%
65% = 10% up to but not including 90%
10% = 5% up to but not including 10%
5% = under 5%
After that, it is a relatively simple matter to plug the necessary values into the conditional formatting formulas as shown above.
One final consideration is that while the buckets above match the color banding on the spreadsheet, I believe that the original request suggests a different color banding with 6 buckets shown below.
Top 5% = 95 to 100% Dark blue
Top 10% = 85 up to but not including 95% Light blue
Top 65% = 35 up to but not including 85% Green
Bottom 10% = 10% down to but not including 5% Light Red
Bottom 5% = 5% or under Dark Red
This leaves one final bucket of 10 to 35% (exclusive of both values) that is not highlighted and so would remain white.
Thank you Chandoo and Shelly for an interesting and useful exercise. This is certainly a valuable technique to have in my reporting bag of tricks.
Use of PERCENTILE is a smarter way of doing it. Below is my solution.
First 5 % = Apply conditional formatting (Dark Blue) as highlight ">=" =PERCENTILE(C:C,0.95)
Next 15% = Apply conditional formatting (Lighter Blue) as highlight between =PERCENTILE(C:C,0.95)-0.01 and =PERCENTILE(C:C,0.8)
Next 65% = Apply conditional formatting as highlight (Olive Green) between =PERCENTILE(C:C,0.8)-0.01 and =PERCENTILE(C:C,0.15)
Next 10% = Apply conditional formatting as highlight (Lighter Red) between =PERCENTILE(C:C,0.15)-0.01 and =PERCENTILE(C:C,0.05)
Bottom 5% = Apply conditional formatting (Red) as less than =PERCENTILE(C:C,0.05)
I agree, this is a challenge faced by HR managers every year and use of percentile formulae is the most popular solution which permits further processing like making bell curve, applying increments based on segmentation etc.
Hi Chandoo,
I came at the same solution as yours (not looking at yours first) but I have hard coded the conditions in the conditional formatting. For example:
=AND($C6>=$D$10,$C6<$D$9)
I have done the same thing 5 times for each condition. This makes the formatting independent of the order of specification. I think it will work better across versions of excel.
To copy the same thing in all sheets, Shelly can copy these formatted cells with format painter and apply it to the relevant cells in next sheet and so on! I know 700 sheets will be difficult but I dont know of any other way to apply conditional formating rules to the whole sheet.
First i have used percentile formula in the next column of "percentile Threshold" where E5, E6.. is input to colour code.
The idea behind doing this is to replicate the formula for any range and any threshold
=PERCENTILE($C$3:$C$30,1-E5)
=PERCENTILE($C$3:$C$30,1-E6)
=PERCENTILE($C$3:$C$30,1-E7)
=PERCENTILE($C$3:$C$30,1-E8)
=PERCENTILE($C$3:$C$30,1-E9)
Now i have given logic to different employee by applying "if Formula"
=+IF(J3>=$G$5,1,IF(J3>=$G$6,2,IF(J3>=$G$7,3,IF(J3>=$G$8,4,5))))
where 'J" referes to PR and "G" refers to percentile derived from above mentioned formula.
once again it is replicable (just change reference points)
Now comes the major part of Conditional Formatting, i have used "use a formula to determine which cells to be formatted"
Formula =$j=5, format "required colour" Applies to "$I$3:$J$30"
plus put tick on stop if true
This solves the query, important point that this is repeatable and can be done for n number of departments
Thanks !
I had done some reading on it and in Excel 2010 a new function has been introduced, percentile.exc. Attaching a video which also talks why the old percentile function shouldn't be used as it acts erroneous at times. Might be worth a watch Chandoo,
http://www.itechtalk.com/thread10579.html
@Deepa
Quit correct.
Where ever you use statistical spreadsheet functions and are using excel 2010 you should use the new versions of the functions as MS did a lot of work to speed up and fix errors in the old functions.
Warning: If you use the new Excel 2010 statistical functions in Named Formulas most of them will crash excel so do keep that in mind.
Hello Chandoo,
When i first read the challenge file, i thought, the color that need to be applied for a given rule, also need to be picked dynamically as given in rule set. But in the solution file, i found that color is hard Coded. So in case, someone has same data, but wants different colors, he/she needs to goto manage rules and change colors.
Let me know if my understanding is correct, and if yes, can we also make the color to be applied dynamic?
Thanks
Kishore
HI I ALSO USED THE PERCENTILE FUNCTION. HOWEVER, I WENT A STEP FURTHER AND USING THE SMALL() FUNCTION I SORTED THE DATA BY PERCENTILE SO THE COLOSCHEME WOULD BE GROUPED BASED ON THE VALUE. THIS WAY IT IS BETTER AND EASIER TO VIEW.
[...] recently posted a challenge to help a reader with a [...]
Hi, i have got doubt regarding to the percentages that has been put in chandoo's spreadsheet, i cant understadn how he put directly. can some one please explain how chandoo put the percetages straight way that i stated below..
5%
15%
60%
10%
5%
I have stumbled on this post as the solution has been already given so I have taken the liberty to record a video where I show the implementation of it as well as adding a filtering feature which I hope can prove to be useful.
Thank you
http://www.xlninja.com/2012/06/28/how-to-use-excel-to-highlight-employee-performance-rating/
[...] scriu nici macar un cuvant din urmatorul articol. Astazi mi-am citit mailul si hopa challenge de la Chandoo. Cum puteam sa refuz asa ceva si m-am apucat de citit, iar dupa 5 min i-am spus sotului ca pe asta [...]
Question for Chandoo:
I came to your site late but am totally loving these challenges 🙂
I guess it all boils down to how the bins are set up.
I agree with the PERCENTILE.INC function.
pls help me understand where I am wrong.
I have determined following the bins:
bottom 5% <=2.00 (F6:F33 <=PERCENTILE(range,.05))
lower 15% (5+10) <= 2.40 (F6:F33 <=PERCENTILE(range,.15))
lower 80% (5+10+65) <=3.46 (F6:F33 <=PERCENTILE(range,.80))
lower 95% (5+10+65+15) <=4.00 (F6:F33 =PERCENTILE(range,.95))
top 5% <=4.20 (F6:F33 <=PERCENTILE(range,1.00))
I find that only Tom is highest scorer and unique top 5% achiever.
I notice that Chandoo has included Christy and Daniel in top 5% achievers. How can there be 3 people in top 5% out of a population of 28 (5% of 28 = 1.4, i.e. only one person can achieve that status)?
I tried different ways but cannot get to that distribution.
Rest of the work is simply organizing the conditional formatting rules with Stop If True box checked.
Thanks for your insights