On my second professional model, now that I thought I was an expert 
, my manager came to me and said “Prove this…”. He had the very common situation where an associate wanted to make a major investment, but could not convince upper management. This is a perfect application for simulation – a model can provide objective information on which to base such a decision.
This was a much better situation than my first experience. This time I had a motivated, involved stakeholder. I had a clear objective. I had important meaningful work. Life was good.
For a while.
Until the model started to “dis-prove this”. Experimentation led me to believe that other alternatives might be better. I told myself that I must be wrong. I double checked but I could not find any errors. Then my boss and the stakeholder told me I was wrong. I triple checked but I could still not find any errors. Life was no longer so good.
What went wrong?
We started with the result. When I set out to prove a conclusion, I put my integrity at risk. The best I could hope for was that the model actually “proved” what they wanted. A typical client reaction to that situation is an empty “I already knew that!” feeling and the perception on his part that I provided very little value. Worse is when the model contradicts their conclusion. It does not support a “known fact“. In that case, stakeholders might think I am incompetent or that simulation offers no value.
But the worst case of all would have been if the model disproved what the stakeholder wanted, but I kept “fixing it” until it supported their conclusion. My client may be satisfied, but do you think he will ever bring me real work? Unlikely. I would have just proven to him that a model can be made to produce whatever result you want, and that my integrity is low enough to do that.
When similar situations arose later in my career, my responses were:
-
–I will be happy to evaluate that situation for you, but I cannot promise what the results will be.
–If what you really want is just a supporting statement, I cannot provide it without objective criteria on which to base it.
While the above does not always create good will, it does allow me to keep my integrity. As much as I hate to admit it, intentionally misusing a model to create invalid results is often easy. Integrity is often the most important thing that you and I can provide as simulationists. A simulationist without integrity should look for another line of work.
Dave Sturrock
VP Products – Simio LLC
Tags: Integrity, misuse, Objectives, projects, simio, simulation, simulationist, software
I have experience with a “Prove This” simulation as well. A large company had already agreed to make a sizeable capital investment and wanted a model just to show that the investment was sound. I was a young modeler at the time and didn’t have the experience to see this as a problem. As it turns out, the data showed a vast improvement in productivity was possible. Ever since, I have been cautious of a “Prove This” simulation. Oddly enough, the company went bankrupt less than a year later, not due to this project, but due to gross mismanagement on the rest of the business.
It is totally true, one of the major mistakes of simulation it to model a
system to prove an existing result. There will be no value. Especially if
the model goes in the opposite direction of the results ”conclusion”.
Simulation is not a tool to prove ”support” something already exists.
Simulation is not a method for backward approval of something. In
conclusion, never Simulate to justify/prove what’s already decided
Talking about integrity, people may give up their integrity to satisfy their customers at least what they expected. The meaning of evaluating a model is proving its feasibility not fixing the results to satisfy customers’ needs.
As a simulationist, if we just keep fixing the results but not telling the truth. What will happen? The model will lose its real function when taking into real world obviously. We should know our duty clearly and keep our integrity. Telling the truth at least let your customers know, even they insisting their thoughts