Multi Variate Optimization (or something like it)

This year I've been talking about skills in testing, and today I want to talk about a big one: Multi-Variate optimization. 

Sounds impressive, don't it?

Don't worry. While Multi Variate Optimization is important in test, it is also a simple concept, one you probably use at work every day ... and one that you can get better at.

We all all familair with optimizing a single variable: Trying to get the best price for a product without trading off anything else.

Multi-variable optimization involved trading off other things. 

So You find a cheaper product ... if you'll put up with two weeks of shipping.

Or you find a higher interest rate for your cash ... if you are willing to tie it up in a certificate of deposit at your bank.

Perhaps you want to invest in stocks with no guarenteed return but a higher historical return than CDs? Then again, if they have a bad couple of years, you could lose money.

This is optimizing across multiple variables, and it is harder.

What is the value of a free lunch?

For a simple multi-variable exercise, try finding a hotel near New York University in Manhattan for a few days - you can look on hotels.com. Here are a few things to consider:

- Price. Cheap is good.

- Distance. Too far from the University, you'll need to take transportation ($$ and time).

- Quality. Clean and roomy is good.

- Wireless. Free internet is good.

- Snacks. Free coffee, free breakfast, and other amenities are good.

- Guest Rating. Higher is better.

- Exercise. A pool or exercise equipment is a plus.

- Time. If you spend more than an hour looking, well, you could have been doing some billable work instead and afford a better room now, couldn't you?

As you can probably guess, guest ratings tend to run in opposition to price. (Pools and exercise rooms also take up square feets and add to price). What you want is a diamond in the rough, a perfect combination - but the law of supply and demand tells us this is unlikely to happen. Because price and ratings tend to complate, if you try to optimize for all of these, you'll likely find something middle-of-the-road that isn't great at anything.

The key to solving this sort of problem, to get the "best" answer, is to figure out what really matters to the client -- who for purposes of this exercise might be you. The "right" answer for a single business traveller in their early 20's might be different than the "right" answer for a family of 5.

What does this have to do with software?

Remember the iron square - quality, cost, features, and time? 

It turns out to be an optimization problem, just one with multiple dimensions.

Anybody can pick a middle-of-the-road solution -- and they often do! The trick is to find the right solution of the four variables on this project, right now.

For example: Generally I would suggest pegging quality at extremely high, because poor quality is a false economy. But that isn't always the case; sometimes you are working on a quick SQL script to make a one-time change to the database, a script that will never be maintained. If you know you are going to throw the script away when you are done, and know that the run is going to be one-time, than you might skip certain aspects of process, documentation, or quality to get the work done.

Now don't look at me so shocked.

The problem comes when, a month later, the manager of Operations asks to run "Matt's SQL update" just "one more time" to "fix the data for the quarterly reports." Pretty soon you find yourself running the query every month, adding parameters for the month, then year, then ... this little "one-time" hack isn't.

All this points back to Heusser's Law of Hackery: Anytime you have a recurring business process that is known by the name of the author, not the business process it enables, you probably have a problem.

In that case, it would be better to have fixed the offending code that was creating bad information in the database. Again, the one-time fix might have been appropriate, and low code-quality might have been fine -- if we actually threw away the code after the first use.

The optimizations are all over software projects - they are what to work on next in project management, how deep to to testing vs wide in QA, how much time to spend on new features vs. regression tests. Tradeoffs in tooling, which slow us down now but (we think might) speed us up later are optimization problems. We tend to understand this in performance, where we talk about designing code that trades off bandwidth vs CPU vs memory size vs disk vs memory speed vs disk vs ... a list.

How To Get Better

The two obivous pieces to me here are understanding the context and then optimizing for it. Learning to optimize is probably the easy part - you can play chess, Carcassone, Settlers of Catan, Dominion, or even the game of my youth, Car Wars. Car Wars might be the best example, where you have to trade off price, handling, space (bigger cars have more and cost more), engine size, acceleration (bigger engines take up more space and cost more money), armor (which is heavier and more costly on larger vehicles) and weapons (which take up space, have weight, and cost. Some are more accurate than others. Some are heavier ...)

You get the picture. 

The context part  - that one is a lot harder to get to. If someone knows a game or simulation that can help you learn the context, I am all years. For me to learn a bit about this skill, it took a lot years of reading, learning, paying attention, failing ... and learning from my mistakes.

Perhaps we can talk a bit about that next time.