This series of articles will give you an overview of how to manage spreadsheet risk. These articles are written by Myles Arnott from Excel Audit
- Part 1: An Introduction to managing spreadsheet risk
- Part 2: How companies can manage their spreadsheet risk
- Part 3: Excel’s auditing functions
- Part 4: Using external software packages to manage your spreadsheet risk
In the first article in this series we highlighted the risks that poorly managed spreadsheet solutions can introduce to a business. In this article we will demonstrate how companies can manage this risk.
A formal governance framework
The first, and arguably most important step is to ensure that the senior management team buy into the need for a robust spreadsheet risk management framework, and that they define and effectively communicate their spreadsheet risk management policy.
Spreadsheets identified and catalogued
It is impossible to know the level of spreadsheet risk in an organization without first identifying and then risk assessing all of the spreadsheets. It is therefore necessary to create a catalog of all of the spreadsheets and then to gather the key information about each spreadsheet to enable a risk assessment to be carried out.
The two key factors for determining the spreadsheet risk are the probability of there being an error and the impact that that error could have.
Risk = Probability of an error X impact if an error were to occur
The probability of error is related to the complexity of the spreadsheet. Complexity attributes differ across companies but include:
- Spreadsheet size (Mbs)
- Spreadsheet design (hard coded numbers in formulae, poor model structuring etc)
- The number of users
- The use of complex formulae (particularly array formulae, nested formulae etc)
- The number of cells populated
- The number of internal and external links
- The use of VBA
The impact of the error is related to how critical the spreadsheet is within the business. Each company will have a slightly different definition of the impact levels of spreadsheets, but generally:
- A spreadsheet is low impact if it is not used as part of a critical business process and an error would not have a material impact on the business.
- A spreadsheet is medium impact if it contains confidential information and an error could have a material impact on the business.
- A spreadsheet is high impact if it contains highly confidential information and an error would have a significant impact on the business. Spreadsheets used within processes that fall under external regulation (such as Sarbanes-Oxley and Solvency II) are deemed to be of high impact.
Finally, the spreadsheets should be placed in order of risk. Those identified as business critical and high risk should be prioritized for detailed review and placed under control.
This is clearly an on-going process. As new spreadsheets are developed they will need to pass through the risk assessment process as defined by the company’s spreadsheet risk management policy. A periodic review should also be carried out to ensure that all spreadsheets have been correctly categorized.
A best practice standard
The company should define its own best practice spreadsheet development standard that is applied to spreadsheets deemed to be medium or high impact. The standard should clearly outline the standards and conventions to which a spreadsheet should be built. New developments can then be reviewed to ensure that they adhere to the standard.
We advocate the use of the Excel Best Practice Standard from the Spreadsheet Standards Review Board (‘SSRB’).
We also recommend that tailored schedules are added to the standard to reflect your specific design standards. For example this could be a specific color scheme, use of logo or the use of specific text within the header or footer (e.g. document security levels).
Testing
A fundamental, but often overlooked step in the Excel model development cycle is testing. All spreadsheets (but especially business critical spreadsheets) need to be first peer reviewed and then rigorously tested.
It helps to consider the steps that an IT department would take to ensure that something they deliver is correct. It will pass through stages of unit and system testing prior to quality assurance and finally user acceptance testing. So why should a spreadsheet being used for a critical process be any different?
The fact is that no matter how hard we try, humans make errors. The purpose of testing is to identify them and get them resolved before the model goes into the live environment.
Remember that in the first article we highlighted the fact that 94% of spreadsheets and 5% of all formulae within spreadsheets contain errors.
Here is Scott Adams’ view on spreadsheet testing in Dilbert
Training
All staff should be trained so that they have sufficient Excel knowledge for their role and to use the spreadsheets that they are responsible for. As part of the induction process all staff should also be taught the company’s best practice standard.
Whilst this sounds obvious, research has shown that few companies prioritize investment in spreadsheet training.
Documentation
A key risk with spreadsheets is that they are often built and used by one individual within a team (often referred to as a “key man dependency”). If this person is ill or leaves unexpectedly the other members are totally reliant on the documentation left behind. From experience this rarely exists.
Each spreadsheet that is used within a process should as a bare minimum have documentation stating:
- the purpose of the spreadsheet;
- how the spreadsheet fits within the process;
- the source of all inputs for the spreadsheet;
- all key assumptions and drivers;
- key calculations;
- distribution list for outputs.
Spreadsheets that are part of as critical business process should have detailed documentation. This should include a technical specification and user notes.
Security
All business critical and confidential spreadsheets should be subject to access control. Security controls can be implemented across three levels:
- Directory level: Only specific individuals have access to key directories
- File level: Confidential and critical spreadsheets should be password protected to restrict access
- Cell level: Non-input cells should be password protected
Change control, backups and archives
To minimize the risk of losing the current version of a spreadsheet and ensuring that the correct version is being used at all times, all business critical spreadsheets should be backed up, archived and subject to change control procedures.
So, in summary..,
the characteristics of a well-managed environment are:
- a formal governance framework, sponsored by the senior management team, is in place for all spreadsheet development;
- a catalog of spreadsheets is maintained and prioritized by risk profile;
- a best practice standard is applied to the development of all new spreadsheets;
- all new spreadsheets pass through a formal risk assessment, are peer reviewed and formally tested;
- staff are provided with sufficient training to carry out their roles;
- all spreadsheets and their associated processes are well documented;
- access to critical spreadsheets is subject to security controls;
- spreadsheets are subject to change control and are regularly backed up and archived.
What next?
In the next article we will look at the built in Excel functions that can help you to manage spreadsheet risk.
What about you?
How do you (or your company) manage spreadsheet risk? What best practices & guidelines you follow? Please share using comments.
Thank you Myles
Many thanks to Myles for writing this series. Your experience in this area is invaluable. If you enjoy this series, drop a note of thanks to Myles thru comments. You can also reach him at Excel Audit or his linkedin profile.
42 Responses to “Prevent Duplicate Data Entry using Cell Validations”
Hi Chandoo, I need you help in the following problem.
I'm trying to get a direccion from a found result. With this dirreccion I will want the before cell value. For example, If result of a find is 38 localized in cell $C$2, I need to get previus value (cell $B$2 ), maybe Andrés.
Do you know some way to do that?
Thank you for you help.
Hi Chandoo
Thanks for this. One thing though: In my pre-2007 version of Excel, the COUNTIF function doesn't recognise a semicolon (;), but requires a comma.
Is the semicolon an Excel 2007 thing?
Jair... I am not sure I understand what you want. what do you mean by Dirreccion?
@Lincoln: I am sorry, often I forget that I am using European version of excel where the delimiter is ; instead of ,. I have corrected the formula now.
Thanks for this nice tip, i used to do a find all after filling every new items which was cumbersome.
Do you know a way to extend this validation search to other tabs/sheets ?
Thanks for you attention. I'm trying to get of value continue from a found value. Let me show a example:
Name Years
John 35
Maria 28
Teresa 32
If I search the max years, the result is 35, but I need that result to be John. Do you know how I can do it?
@Subbu.. you can easily extend the validation to other sheets by pasting the data validations. See the latest article here: http://chandoo.org/wp/2009/10/28/copy-data-validations/
@Jair.. you can use the large() or small() formulas to do this. for eg. =index(A1:A3,large(B1:B3,1)) will get you the name of the person with highest "years". More help here: http://chandoo.org/excel-formulas/large.html
Hi, I don't know if I'm using bad the formula or its performance is diferent for my Office version. Large() formula return the value in the cell, in my example 35. The index() formula use a range, row and column. I'm using the large() as number of row, and it is bad because into the range don't have row 35. This is my perception. What do you think?
Hi, I going to prove, with this solution by JLD http://jldexcelsp.blogspot.com/2008/07/extraer-direccion-de-celda-en-matriz.html
Hello,
I am trying to attempt data validation in Excel Mobile, but the DV tool isnt available. I want to prevent duplicates is all, any advice on acheiving this in Excel Mobile? Thanks..
@Jair... my french aint that good. it starts at "merci" and ends at "beau coup".
Anyhow, you need to merge the large with vlookup to do this. I am not sure if you have solved the problem. Otherwise let me know with details and I can write the formula in comments.
@Chad... I have never used excel mobile, so I have no idea. May be they have not implemented data validations in excel mobile.
Any excel mobile users out there?
Hi Chandoo, the proposed solution by JlD is interesting. He created a macro to get values when the matrix is not one dimensional, how on my problem. This fuction for me.
I would like to share you my work, how can I upload?
@Jair.. sorry for such a delayed reply.. you can upload the files to skydrive and link them here. Or you can email them to me at chandoo.d @ gmail.com and I will upload them somewhere. But it could take forever if you email files to me as I am a bit lazy.
[...] Day 31: Advanced Data Validation Tricks in Excel – Part 2 [...]
Hi,
Can you help me in Microstrategy?
Br,
Moin
really wonderful article. I feel it is implementing Primary Key concept into spreadsheets.
Hi article on data validation. Excel is a very versatile platform to work with and we use it for all kinds of data tabulation. In fact this must have been the most rudimentary data management tools I must have worked with and knowing such tips only adds functuionality to our user experience. Great article. looking forawrd to read more.
Hi Chandoo,
Thanks for such a nice idea.
I tried copy paste the data into the validated area, but the pop-up msg (warning msg) doesn't came. Is it something that we need to update the data manually each time,.
Do we have any option where we can bulk upload the number and it throws a warning message that the data already exits and do we want to continue with this ?
Please do reply me.
Thank you.
Regards,
Vasanth.
It was really useful chandoo...thanks a lot...
Tried this in excel 2010 and it did not work?
Could the newer excel have changed that much?
@Leo
It works fine in Excel 2010
The formula used above =COUNTIF($B$4:$B$11,B4)<=1
only applies to the range B4:B11
Did you adjust the range to your data?
This page helped me accurately to find solution of my question. thanx
we want to prevent duplicate entries in three columns combined, using data validation, i.e. say, column A has first name and Column B has middle name, Column C has last name. the first name can be duplicate, middle name can be duplicate, last name can be duplicate, but not all three at the same time.
I want to prevent duplicate entries in three columns combine, using data validation, i.e. say, column A has first name and Column B has middle name, Column C has last name. the first name can be duplicate, middle name can be duplicate, last name can be duplicate, but not all three at the same time.
Hi, I've tried above validation method to prevent duplicate value from entering into the cells. It's work, when user key in the data into the selected range. However, it's not working when user copy-&-paste the info into the same range.
Please advice. Thanks.
Hi Chandoo,
Does it work in Excel 2007?
Please Reply
thanks chandoo........
it does not work when data is copy pasted...any solution for this??
hi i m shaloo and i want to know in excel if i write duplicate no.then it says or show about we are write duplicate no.
Hi Chandoo
I've tried using this with a Named Range, which is actually a column in a Table as DV wont accept a table reference, and it wont work.
Also tried using Offset to specify the Named Range, but that wont work either.
Is it possible to use Named Ranges with DV?
Thanks
Kris
I have tried the above formula on a table column. The Error box does not pop up, there is only the small ! next to the cell with the duplicate. The column I am working with is formulas that produce a date. Is the reason it doesn't work that the cells contain formulas rather than data?
The formula works but only if I enter data in cell above it. So for example, if I have "123" in B11 it does not allow me to enter "123" in B10, B9, B8, etc. But I can still enter "123" in B12. Please help! 🙂
Great tip.. thanks a lot
I have 21 years of experience working as data entry assistant. I constantly read several blogs to keep myself up-to-date with the advances in data entry profession. I really enjoyed this blog post. From my several years of experience, I agree with you 100% when you say, “ We all know that data validation is a very useful feature in Excel. You can use data validation to create a drop-down list in a cell and limit the values user can enter. ”
Keep blogging. I will come here again.
--data entry assistant
Hi,
This is really very helpful.
Thank you
how can i assign two validation on a single cell
one is for list validation (means the data should be from that range)
second i want to prevent them from repetition
how can i do this ?
P7 to P506 have GR# which are for list
i want to prevent C column to not to repeat and should be from the P column
@Junaid
Can you please post the question in the Chandoo.org Forums
http://forum.chandoo.org/
You have to register to be able to post questions
Please attach a file so that a specific answer can be delivered.
i made an account but there is no option available to post questions ??
where can i ??
@Jubaid
Goto http://forum.chandoo.org/
Goto Ask an Excel Question
Post New thread
Type your question
Attach a file
Please attach a file so that a specific answer can be delivered.
friend can any one tell me the formula
exname location qty
gaurav 1 1
rofan 2 5
sandeep 3 6
gaurav 4 3
rofan 5 4
sandeep 6 8
gaurav 7 9
If this is a data.
if i want a formula by which if i type gaurav then all the location and qty should be shown in a new page.
i had 5,00,000 sku so if i punch one name i can get the entire details
@Gaurav
Can you please post the question at the Chandoo.org Forums
http://forum.chandoo.org/
Please attach a sample file for a quicker more targeted response
IF(ISERROR(INDEX($B$3:$C$9,SMALL(IF($B$3:$B$9=$B$12,ROW($B$3:$B$9)-ROW($C$2)),ROW(A1:C1)),2)),"",INDEX($B$3:$C$9,SMALL(IF($B$3:$B$9=$B$12,ROW($B$3:$B$9)-ROW($C$2)),ROW(A1:C1)),2))
please explain
YOUR COUNTIF FORMULA IS REALLY HELPFUL AND WORKS. I TRIED SEVERAL SITES BUT THEIR FORMULA DOES NOT WORK. ONLY YOU HAVE GIVEN A RIGHT FORMULA!
THANK YOU!!!!!