Using Solver to Assign Items to Buckets

Share

Facebook
Twitter
LinkedIn

In April 2011, Mdsuhair asked a question at the Chandoo.org Forums

What formula can I use to divide a group of numbers into 2 groups so that the Sum of each of the 2 groups is as equal as possible?

This is a candidate for Solver.

This post will walk us through the solution to the problem using Solver. It uses Solver and screen shots from Excel 2007.

All versions of Excel have solver available. Users should note that the screen shots may not match your version although the functionality will, some of the functions appear in different locations in different versions of Solver.

What and Where Is Solver

What is Solver

Solver is an Excel add-in that can solve problems by enabling a Target cell to achieve some goal.

This goal may be to minimse, maximise, or achieve some target value.

It solves the problem by adjusting a number of input cells according to a set of criteria or constraints which are defined by the user.

Where is Solver

Solver is an Excel add-in supplied with Excel, but not enabled by default.

To enable solver

File, Excel Options, Add-ins, Manage Excel Add-ins, Select the Solver Add-in checkbox

Solver will now appear as a New Tab on the Data, Analysis Tab

Solver Example

This post is based around a worked example Solver Example File, the file is compatible with all versions of Excel.

Defining the Problem

Mdsuhair had a series of 8 Items each which had a value:

 

He wants to know which items should be combined so that the sum of the values of the items is nearly equal.

To do this we need to assign each item into a Bucket. Namely Bucket A and Bucket B.

We will put values of 1 into a Bucket for each Item to show that it is assigned to that Bucket and a value of 0 to show that the bucket is empty (In/Out) Value.

We can start by adding a Total Column, which counts items in Buckets A + Bucket B for each Item, It then totals the totals.

We also know that we need to work out the Value of each Item in each Bucket

We do this by multiplying the Items Value by the Buckets (In/Out) value

Finally we need to add up the values in each Bucket and work out the difference between them

Mdsuhair’s original problem was to minimise the difference between the sum of the values in the 2 buckets.

Now we can place values of 1 in the buckets manually and Excel will show us the value of each Bucket and the Difference between them in Cell G11.

I have applied some conditional formatting to show when a cell has a value > 0.

The problem is that there are 28 or 256 combinations of answers, and to test them all manually at 1 every 5 seconds would take 21.3 minutes, assuming we can keep up that pace and remember which was the best combination.

This is where solver comes to the fore.

Applying Solver

To apply solver we need to define a series of requirements, rules and constraints.

These requirements, rules and constraints guide solver and set limits which allow solver to quickly narrow in on the answer.

What are our rules

Our main requirement is to minimise the difference between the value of the 2 buckets.

The difference between the 2 buckets in our example is cell G11, the sum of Bucket 2 values minus the Sum of Bucket 1 values.

We want to have G11 as low as possible but greater than or equal to 0.

 

We also know that an item can only be in Bucket A or Bucket B, it can’t be in both and can’t be Broken apart.

That is 2 Constraints for each Item

Firstly The Total column must be equal to 1

Secondly the Buckets Values must be Integers

We also know that the Total Number of Items is 8, this is another constraint.

We will discuss how these constraints are used in the next section

The Solver Window

This section will explain the solver window and its use in defining the problem within solver.

A Blank Solver Window

A Filled Solver Window

Set Target Cell:

This is the Target cell which is the cell which you are trying to solve the problem for.

Our Target cell is G11, The difference between the 2 Buckets values

Equal To:

The Equal To: section defines what we want to do with our Target Cell.

We want to achieve the same value in each Bucket and so the difference between the Buckets will be 0.

It might sound strange but we don’t want to minimise that difference. A minimal value will be achieved when all the Items are placed in Bucket A, as our equation for G11 will then have 0 – Total which is –Total, which is more minimal than 0.

Another way to constrain this is to Change G11 to =Abs(G10-H10)

This allows us to use Min as an Equal To: Value

But for now we can just leave G11 as =G10-H10 and we will set the Equal To: section as 0.

By Changing Cells:

Changing Cells: refers to the cells which will be modified by Solver to try and solve the problem.

We want to let Solver change the number of items in each bucket, this is the range: $C$2:$D$9

Hint: You can try the Guess button next to the Range Reference and Solver will take a Guess at what cells the problem is dependent on.

Always check this if you use it, especially in complex models.

Subject to the Constraints:

Constraints are the rules which define the limits of the possible solutions to the problem

We will add several constraints for our rules:

1. The Total column must be equal to 1 for each Item

2. The Bucket Value must be an Integer

3. The total contents of the 2 buckets must be 8 items

4. You could add a further constraint that each Bucket should hold the same number of items

Hint: As a general Rule, Under Constrain rather than over constrain! You can always add more constraints later.

To do this we will use the Add Constraint Button

1. We need to add a constraint for each cell in the Range E2:E9 that it is only allowed to be = 1

This constraint must be applied for each cell in the range E2:E9

2. We need to add a constraint for each cell in the Range C2:D9 that it is only allowed to be an integer

This constraint must be applied for each cell in the range C2:D9

3. We need to add a constraint for the Total of the 2 Buckets, E10=8

You can Change or Delete Constraints if you make a mistake by selecting the appropriate constraint and using the Change or Delete Buttons

Save and Load Solves Parameters

Selecting the Options Button there is the Option to Save Model and Load Models.

Hint: The Save/Load Models has been shifted onto the main Solver dialog in Excel 2010.

Using the Save Model and Load Model options you can Save and the Load the Solver Parameters for your model. The Save Model saves the parameters in a Range of cells as shown below.

This allows an easy way to actually setup and/or change the solver parameters.

Hint: Setup one constraint using solver then Save the model. Edit the model on the worksheet and re-load the model as required.

Note: That the parameters although when saved show as True/False or Numbers are all Excel Equations, see above.

This means you can edit them to change the Constraints and Parameters as required and re-load them into solver.

I have included 3 sets of Parameters for our model.

These are:

  • Base Case – Forces bucket values to be equal, Allows uneven bucket counts
  • Equal Sized Buckets – Forces each bucket to contain the same number of items
  • Force an error – Which forces an error in the solver model

Load each model and try them at your leisure.

Running the Solver Model

Warning: Solver is a computationally complex add-in, so once your model is setup, Save your Workbook.

Prior to running the model there are a few parameters we should look at to ensure the model solves correctly.

On the main Solver window select the Options button. (Some of these parameters are on the Main Solver window in Solver 2010)

Generally you can accept the defaults but in this case we will change the following

Assume Linear Model – Select

Assume Non-Negative – Select

Note: Solver in Excel 2010 will return a better answer without these 2 parameters enabled by default

The other 2 parameters which you may need to change from time to time is

Precision: Precision is a number from 0 to 1 and higher means more precise

Tolerance: Tolerance shows how far away from a Number, an Integer constraint is allowed to be

The use of the Estimates, Derivatives and Search parameters are beyond the scope of this post. I direct you to the Excel Help on these subjects, by selecting the Help button.

Run the Model

To Run the Model, select the Solve Button from the main Solver menu.

The main status bar in Excel will flash up a number of statistics about the internal workings of the Solver add-in. Generally these flash by and are too fast to read. If a model is too complex it may stall and you won’t see any movement for a while. Solver generally recovers from these problems itself.

 

Once the Solver model finishes it will display a dialog of the results and allow you to do several things

First thing to note is that “Solver Found a Solution.

If it has found a solution, the worksheet cells will be changed to show the solution

You now have 4 options:

  • Run a Report
  • Save a Scenario
  • Return to the model
  • Check Your Results

Run a Report

Run a report by clicking the report you want.

A new sheet will be added to your workbook depending on the report but will be called:

  • Answer Report 1,
  • Sensitivity Report 1
  • Limits Reports 1

etc.

Note, that not all reports are relevant at all times, depending on the Constraints you have applied.

Save a Scenario

Selecting the Save a Scenario button takes you to the Save Scenario dialog.

 

Type in a Name and the Scenario of your model is saved as a Scenario.

Scenarios as available for use in the Scenario Manager, which is accessed from the Data, What-If-Analysis Tab

Return to the model

You can return to your model and either:

  • Keep Solver Solution
  • Restore Original Values

Check the Results

Solver is probably the most Black Box’ish of systems within Excel. As such any results it puts out must be manually checked for suitableness before further use.

These checks for realness, should as a start confirm that the results meet all the criteria supplied.

Are the results roughly what were expected?

Are any Minimums or Maximums violated?

 

What If Solver Doesn’t Find a Solution?

From time to time Solver will return with an error that a “Solver could not find a feasible solution.

When this happens it is indicating one of several possibilities:

  • Your model is over or under constrained
  • Your model constraints are impossible to meet
  • Your model constraints have an error

Start by checking the current constraints for errors and ambiguities

Eg: in our case we have 8 items so requiring the Count of the two buckets to be 20 is impossible to meet

 

My Solver Answer has Strange Numbers?

In solver up to and including Excel 2007, solver would commonly return numbers like 3.5E-18.

This is 0.0000000000000000035, which is effectively 0

If your model returns these, feel free to go through the model and change them to 0, in our model we should also check as the corresponding 1, may in fact be 0.9999999999999999965.

It should be noted that this problem in Solver in Excel 2010 does not occur as often but will still occur.

 

What and How have you used solver in the past?

What and How have you used solver in the past?

Let us know in the comments below:

Facebook
Twitter
LinkedIn

Share this tip with your colleagues

Excel and Power BI tips - Chandoo.org Newsletter

Get FREE Excel + Power BI Tips

Simple, fun and useful emails, once per week.

Learn & be awesome.

Welcome to Chandoo.org

Thank you so much for visiting. My aim is to make you awesome in Excel & Power BI. I do this by sharing videos, tips, examples and downloads on this website. There are more than 1,000 pages with all things Excel, Power BI, Dashboards & VBA here. Go ahead and spend few minutes to be AWESOME.

Read my storyFREE Excel tips book

Overall I learned a lot and I thought you did a great job of explaining how to do things. This will definitely elevate my reporting in the future.
Rebekah S
Reporting Analyst
Excel formula list - 100+ examples and howto guide for you

From simple to complex, there is a formula for every occasion. Check out the list now.

Calendars, invoices, trackers and much more. All free, fun and fantastic.

Advanced Pivot Table tricks

Power Query, Data model, DAX, Filters, Slicers, Conditional formats and beautiful charts. It's all here.

Still on fence about Power BI? In this getting started guide, learn what is Power BI, how to get it and how to create your first report from scratch.

30 Responses to “Rescue oddly shaped data – Battle between Formulas, VBA and Power Query”

  1. MF says:

    Nice use of Power Query! Power Query is simply awesome! But somehow a lot of people are punishing themselves by not using it (not learning it).

    An imperfect 4th approach for consideration... no codes at all...
    Select myrange.
    Go to Special --> Blank
    Delete Cell --> Shift cell left
    90% done... now we just need to move the data of 2nd column to the bottom of 1st column
    Of course... Power Query is the best.
    Cheers,

  2. There is another way but it involves multiple steps:
    Copy the values in column E, move the cursor to F5, Paste Special with Skip Blanks, OK
    Copy the values in column D, move the cursor to F8, Paste Special with Skip Blanks, OK
    And so on.
    This works perfectly, albeit a little clumsily apart from the values in B17 and C16, which can be moved with simple copy and paste

  3. Robson says:

    Power Query Forever! I do not know how I survived for so long without knowing and using this tool, I can not recommend it to my colleagues, but by the way they prefer to suffer to learn.

    My congratulations here from Brazil.

  4. Haz says:

    I rolled my eyes when I saw that data

    Using decimal places is a nice trick to order data, thanks for that

    And tweaking the first formula a bit, you can use OFFSET instead of INDIRECT

    =OFFSET($A$1, MIN(IF(myrange, ROW(myrange)), ROWS(A$1:A1))-1, RIGHT(TEXT(MIN(IF(myrange, ROW(myrange) + COLUMN(myrange)*0.00001), ROWS(A$1:A1)), ".00000"), 5)-1)

    • Michael Connor says:

      Tried the above formula with the downloaded oddly shaped data file and I could not get it to work. I get #value without ctrl+shift+enter, and #ref with ctrl+shift+enter.

      • Haz says:

        Sorry, it was SMALL, not MIN.
        Add with CTRL+SHIFT+ENTER.

        • Michael Connor says:

          Thank you for your formula. Like the indirect formula I tested this one in older versions of EXCEL and it worked without ALTERATION in EXCEL 95. Very impressive.

  5. Bertie Hechter says:

    Too complicated

    Use =Sum to summarize all the sells to the left and Bobs Your Uncle

  6. Michael Connor says:

    I tested this formula in versions of Excel all the way back to Excel 95

    =IF(ISERROR(INDIRECT("R"&SUBSTITUTE(TEXT(SMALL(IF(MyRange"",ROW(MyRange)+COLUMN(MyRange)*0.00001),ROWS(A$1:A9)),"00000.00000"),".","C"),FALSE)),"",(INDIRECT("R"&SUBSTITUTE(TEXT(SMALL(IF(MyRange"",ROW(MyRange)+COLUMN(MyRange)*0.00001),ROWS(A$1:A9)),"00000.00000"),".","C"),FALSE)))

    So there are multiple ways of cleaning up messy data by formulas.

    • Chandoo says:

      Wow.. Excel 95. Who knew people still use that. But as you have shown, Excel has all these beautiful and powerful functions for 23 years. It has data sciency stuff before DS was even a thing.

      • Michael Connor says:

        I had a problem with pasting the formula in the original post.
        Formula should be: =IF(ISERROR(INDIRECT("R"&SUBSTITUTE(TEXT(SMALL(IF(myrange"",ROW(myrange)+COLUMN(myrange)*0.00001),ROWS(A$1:A1)),"00000.00000"),".","C"),FALSE)),"",(INDIRECT("R"&SUBSTITUTE(TEXT(SMALL(IF(myrange"",ROW(myrange)+COLUMN(myrange)*0.00001),ROWS(A$1:A1)),"00000.00000"),".","C"),FALSE)))

        EXCEL even in a 16 bit version, is a very robust and capable program.

  7. Michael Connor says:

    I don't like the VBA code. If you have a blank row in MyRange, the last entry in the range is doubled up in the paste.here range.

    • Chandoo says:

      Not really. The macro is writing one cell at a time from paste.here. You have to clean the range before, which I was too lazy to write. But a line like Range(range("paste.here"), range("paste.here").end(xldown)).clearcontents should do the trick.

      • Michael Connor says:

        Adding Range(range("paste.here"), range("paste.here").end(xldown)).clearcontents fixed the problem.

  8. A Rakesh Patro says:

    for step split column by delimiter i am not getting option of split into rows or columns. Can you help me in this

  9. Marc says:

    Thanks Chandoo for promoting Power Query.

    To simplify further, you can "Unpivot Columns" instead of right click on the newly created column and split it by comma in to rows in step 3 of Power Query.

  10. johan says:

    i used

    =LOOKUP(10000,B5:F5)

    and got the answers. I just plagiarized this formula somewhere and use it, maybe you can explain why it works.
    Regards

    • Chandoo says:

      @Johan... I am not sure if the formula works correctly. When I tested it with the sample data in this post, it showed #N/As in two cells. Essentially, it will only give first value in each row. So if a row has multiple values, then subsequent values are missed. LOOKUP() function goes thru a list and finds the first value that is less than or equal to the input - in this case 10000 in B5:F5.

  11. Ben says:

    I have the need to convert pdf's to excel on occasion and they often come out a mess like this. I have used:

    Cell G2 =COUNT(myrange)
    Cell G3 =IFERROR(IF(G2-1<1,"",G2-1),"") copied down to G100
    Cell H2 =IFERROR(LARGE(myrange,G2),"") copied down to H100

  12. Patrick says:

    Waouw...

    =IFERROR(INDIRECT("R" & SUBSTITUTE(TEXT(SMALL(IF(myrange "", ROW(myrange) + COLUMN(myrange)*0.00001),
    ROWS(A$1:A1)), "00000.00000"), ".", "C"), FALSE), "")

    but CTRL Shift Enter with {} before and after 🙂 😀

  13. Peter B says:

    Another possibility.
    This assumes that you have a row index 'k' to use in the SMALL function and a column index 'h' to identify the columns of 'myRange'.
    If you define 'coord' to refer to
    =k+h/10 [assuming h<10]
    then it will be possible to recover values later based upon location within 'myRange'. The formula 'nb' that identifies non-blanks by coordinates is given by
    = SMALL( IF(myRange"", coord), k )
    Finally, to unpick the pieces
    = INDEX( myRange, INT(nb), 10*MOD(nb, 1) )

  14. Peter B says:

    Whilst I am here and making trouble the PQ solution is also a tad over-complicated. All that is needed is to unpivot the entire table and remove the Attribute column.

    The advanced editor would show
    let
    Source = Excel.CurrentWorkbook(){[Name="myRange"]}[Content],
    #"Unpivoted Columns" = Table.UnpivotOtherColumns(Source, {}, "Attribute", "Value"),
    #"Removed Columns" = Table.RemoveColumns(#"Unpivoted Columns",{"Attribute"})
    in
    #"Removed Columns"

  15. vivian.liu says:

    1.fill the blank cells with 0
    2.the requested column value=sum of those mess number column
    but this can be used in only one column has value

  16. Juan Carlos Barreto says:

    Chandoo

    And if we use the formula SEARCH (100000000, B5: F5)

    JC

  17. Daniel Dion says:

    Another approach with Power Query, it will still work if the number of columns changed:
    let
    Source = Excel.CurrentWorkbook(){[Name="myrange"]}[Content],
    #"Added Custom" = Table.AddColumn(Source, "List", each Record.ToList(_)),
    #"Removed Other Columns" = Table.SelectColumns(#"Added Custom",{"List"}),
    #"Expanded LIst" = Table.ExpandListColumn(#"Removed Other Columns", "List"),
    #"Filtered Rows" = Table.SelectRows(#"Expanded LIst", each ([List] null))
    in
    #"Filtered Rows"

  18. Bob says:

    Nowadays, you can just use TOCOL on Excel 2024, MS 365, and Web Excel. It has a parameter to ignore blanks/errors/both.

Leave a Reply