Prove This

On my second professional model, now that I thought I was an expert 😉 , my manager came to me and said “Prove this…”. He had the very common situation where an associate wanted to make a major investment, but could not convince upper management. This is a perfect application for simulation – a model can provide objective information on which to base such a decision.

This was a much better situation than my first experience. This time I had a motivated, involved stakeholder. I had a clear objective. I had important meaningful work. Life was good.

For a while.

Until the model started to “dis-prove this”. Experimentation led me to believe that other alternatives might be better. I told myself that I must be wrong. I double checked but I could not find any errors. Then my boss and the stakeholder told me I was wrong. I triple checked but I could still not find any errors. Life was no longer so good.

What went wrong?

We started with the result. When I set out to prove a conclusion, I put my integrity at risk. The best I could hope for was that the model actually “proved” what they wanted. A typical client reaction to that situation is an empty “I already knew that!” feeling and the perception on his part that I provided very little value. Worse is when the model contradicts their conclusion. It does not support a “known fact“. In that case, stakeholders might think I am incompetent or that simulation offers no value.

But the worst case of all would have been if the model disproved what the stakeholder wanted, but I kept “fixing it” until it supported their conclusion. My client may be satisfied, but do you think he will ever bring me real work? Unlikely. I would have just proven to him that a model can be made to produce whatever result you want, and that my integrity is low enough to do that.

When similar situations arose later in my career, my responses were:

    –I will be happy to evaluate that situation for you, but I cannot promise what the results will be.
    –If what you really want is just a supporting statement, I cannot provide it without objective criteria on which to base it.

While the above does not always create good will, it does allow me to keep my integrity. As much as I hate to admit it, intentionally misusing a model to create invalid results is often easy. Integrity is often the most important thing that you and I can provide as simulationists. A simulationist without integrity should look for another line of work.

Dave Sturrock
VP Products – Simio LLC

Model This

When I first started modeling, my boss came to me and said “Model this…” and then proceeded to describe an area of the plant that he thought “might benefit from having a model”. Unfortunately there were no specific objectives beyond that.

To me, a new simulationist, that sounded like an ideal project. Nothing to prove… Nothing specific to evaluate… No one waiting on specific results (because none were asked for)… It even sounded like a good opportunity to learn how to model. But it was not.

“Model this” generally results in a useless project. A waste of time. Without clients or clear objectives, I could not know what to model. Without clear objectives, I had little motivation to learn how to model tricky situations; I instead tended to bypass them to work on aspects more fun or interesting. In fact, for the same reason I often did not even recognize modeling challenges, so I never learned to deal with them.

Moreover, when it was all done, what did I have to show for my time? Perhaps a cool-looking animation. It probably did not have many aspects of reality to it. Reality is driven by close interaction with the stakeholders – oops, I did not have any of them. Why should anyone waste his or her time sharing domain knowledge with me, when I was basically just modeling for fun?

And worse, after I “finished” the model, I was overconfident of my modeling skills – after all, I modeled everything I set out to model, right? Of course I was never forced to really verify and validate against the real system, so I never really had any idea how good the model really was.

Avoid “model this”. Always push for clear project objectives. More on that next time.

Dave Sturrock
VP Products – Simio LLC