# Predicting Process Variability

Systems rarely perform exactly as predicted. A person doing a task may take six minutes one time and eight minutes the next. Sometimes variability is due to outside forces, like materials that behave differently based on ambient humidity. Some variability is fairly predictable such as tool that cuts slower as it gets dull with use. Others seem much more random, such as a machine that fails every now and then. Collectively we will refer to these as process variability.

How good are you are predicting the impact of process variability? Most people feel that they are fairly good at it.

For example, if someone asked you what is the probability of rolling a three in one role of a common six-sided die, you could probably correctly answer one in six (17%). Likewise, you could probably answer the likelihood of flipping a coin twice and having it come up heads both times, one in four (25%).

But what about even slightly more complex systems? Say you have a single teller at a bank who always serves customers in exactly 55 seconds and customers come in exactly 60 seconds apart. Can you predict the average customer waiting time? I am always surprised at how many professionals get even this simple prediction wrong. (If you want to check your answer, look to the comment attached to this article.)

But let’s say that those times above are variable as they might be in a more typical system. Assume that they are average processing times (using exponential distributions for simplicity). Does that make a difference? Does that change your answer? Do you think the average customer would wait at all? Would he wait less than a minute? Less than 2 minutes? Less than 5 minutes? Less than 10 minutes? I have posed this problem many times to many groups and in an average group of 40 professionals, it is rare for even one person to answer these questions correctly.

This is not a tough problem. In fact this problem is trivial compared to even the smallest, simplest manufacturing system. And yet those same people will look at a work group or line containing five machines and feel confident that they can predict how a random downtime will impact overall system performance. Now extend that out to a typical system with all its variability in processing times, equipment failures, repair times, material arrivals, and all the other common variability. Can anyone predict its performance? Can anyone predict the impact of a change?

With the help of simulation, you can.

This simple problem can be easily solved with either queuing theory or a simple model in your favorite simulation program. More complex problems will require simulation. After using your intuition to guess the answer, I’d suggest that you determine the correct answer for yourself. If you want to check your answer look at the comment attached to this article.

And the next time you or someone you know is tempted to predict system performance, I hope you will remember how well you did at predicting performance of a trivial system. Then use simulation for an accurate answer.

Dave Sturrock
VP Products – Simio LLC

# Simulation in Healthcare

Over the years, I have had several occasions to use medical facilities for myself and my family. Some visits were routine, such as for a diagnostic tests or images. Others were for much more critical visits to an emergency department. As my visits spanned many facilities and many time periods, I observed a dramatic difference in the service provided. In the case of bad service I just had to wonder “Didn’t anyone ever study this operation? Did anyone ever simulate it?”

Simulation can bring significant benefits to healthcare, just as it does in other types of systems. Some of those benefits come from the simulation’s ability to:
• Account for variability in human behavior
• Account for variability in demand
• Capture complexities and interdependencies
• Capture system performance over a period of time
• Support continuous process improvement and evaluation of new scenarios
• Provide an objective basis for evaluating policies and strategies

Here are a few possible applications to illustrate how simulation is often used in the healthcare industry:

New Facility Design – Evaluate design to assure that present and future objectives will be met. Reduce capital costs by “running” the facility under various scenarios and identifying excess capacity . Reduce operating costs by supporting lean and six sigma analyses. Increase throughput through process flow optimization and identification of bottlenecks and capacity constraints.

Emergency Department (ED) – Decrease LOS (Length of Stay) and LWBS (Leave Without Being Seen) yielding higher patient satisfaction. Improve staff efficiency and improve room and resource utilization resulting in lower costs.

Outpatient Lab and Surgery – Determine optimal staff and resource allocation. Balance scheduled demand with the often-critical unscheduled demand. Decrease lab and diagnostic turn-around time. Identify non-value-added and redundant processes.

Ambulance Service – Evaluate operational scenarios for both road and air-based vehicles. Evaluate new technology to determine their effect on the entire system. Pre-plan dynamic utilization-based response guidelines to optimize performance during major ED demand periods.

Vaccine Distribution – Evaluate regional material stocking strategies, distribution strategies, and staffing.

Often the benefits from these studies are reported in the millions of dollars so they are well worth the undertaking.

One source of additional information is the Society for Simulation in Healthcare which is having their annual conference in January. Another source is the Society for Health Systems which offers the latest in process analytics, tools, techniques and methodologies for performance improvement.

Dave Sturrock
VP Products – Simio LLC

# Data Collection Basics Part 2

Last week in Data Collection Basics (Part 1) I discussed data collection, introducing the topics of identifying required data and then locating or creating that data. Once you have some data, you typically need to do some analysis on it before you can effectively use that data.

Select Distribution. Typically input data to a simulation model is specified as a distribution. If you have estimated data you must select the most appropriate distribution (for example a minimum time, typical time, and maximum time may be represented as a Triangular distribution). If you have actual data, then you will need to run a statistical analysis on it. Many software products (some generic and some simulation-specific) are available to help you with selecting (fitting) a distribution and its shape parameters, and even with cleaning the data to eliminate bad observations.

Analyze Sensitivity. Once you have some data you can build it into your model and start making trial runs. Particularly if you have relied on an estimate, you might want to run your model with values above and below the estimated values to determine system sensitivity to that parameter. If you find that the system is sensitive to an estimated value (e.g. the results change significantly with a change to the input parameter), then you can determine if it is worth a greater investment to obtain a more reliable value. This is one potential solution to the problems of bias and inaccuracy discussed in the initial article. But more than that, it is also a good way to iteratively determine how much time to spend on your input data.

Adjust Detail. Sometimes the quality of the available data can help you determine the appropriate level of detail for a model. If the data you intend to use is not very good, then there is little point to building a highly detailed model. This is not to imply that such a model is of no value, after all every model is just a representation or estimate of reality – no model will be perfect. But it is important to represent to your stakeholders the relative accuracy of the model and its underlying data.

This was a quick overview of some steps to data collection. Whole textbook chapters have been written about each of these, so be sure to look for greater detail when you are ready.

Dave Sturrock
VP Products – Simio LLC

# Data Collection Basics

Even though the people responsible for building models are often the “data collection people”, I know very few associates who think this is a particularly enjoyable part of their job. But data collection is a necessary part of most simulation projects. An early task in each simulation project should be to identify what data will be needed and how that data will be obtained.

Identify Data. There are many different types of data that you will potentially need. Like other aspects of simulation, the identifying required data is best done iteratively. Start by looking at the major areas of your model: arrival sections, processing sections, storage areas, departure areas, internal movement and similar aspects. For each area, then consider the key parameters necessary to describe it. For example, in an arrival area: What is arriving? Are there many different types of entities? Do they each have descriptive attributes that are important? Do you expect the arrivals to follow some type of a time-based pattern? Considering questions such as these will also help you define the model and modeling approach and iteratively, more detail on the exact data required.

Locate Data. With the current level of automation and electronic tracking, the availability of data has become more prevalent. If it’s an existing system, there may already be data that is routinely collected. If it is a new system, the vendor may have access to data collected on similar systems. In either case, the existence of data does not necessarily make your job easy. For example, perhaps you are interested in a processing time on an operation, and that processing time is automatically captured. But what may not be obvious is exactly what that number represents. Does it (sometimes) include time when the process was failed (perhaps short failures are imbedded but long failures are not)? Does it (sometimes) include time when an operator went on break and forgot to properly log out? Detecting and cleaning such situations can be a tedious and frustrating part of using existing data.

Create Data. If the data you need does not exist or cannot be appropriately cleaned, you must often create it. On an existing system, the most accurate method is to electronically capture the data or have manual studies done to determine it. Either of these can be very expensive. An alternate approach is to get estimates from people who know – people running or managing the operation. Although fast and inexpensive, this may introduce bias and inaccuracy. Likewise on a system that does not yet exist, you may need to rely on specifications provided by a vendor, again possibly introducing bias and inaccuracy. More on dealing with this situation later.

This was a quick overview of some initial steps to consider in data collection. Next week I will discuss some additional steps on what to do next with that data. Until then, Happy Modeling!

Dave Sturrock
VP Products – Simio LLC

# Simulation and Disaster Management

While the last couple months have been pretty dry where I live here in the Northeastern part of the U.S., in the Southeastern part several severe hurricanes have already hit and it looks like more are coming. While every severe storm can have serious consequences, often the major difference between a severe storm and an outright disaster is the level of preparation.

Of course weather is just one of many potential causes of disasters. We have all seen floods, fires, earthquakes, and other disasters around the world that have been made much worse through inadequate planning and poor execution. Simulation can play a major role in preparing communities to avoid or at least reduce the impact of such disasters.

More accurate weather prediction is due in part to simulation. Combining advanced detection technology with sophisticated simulations has allowed us to become much better at predicting storm paths and severity. This allows for improved warnings and appropriate responses.

Simulation use in evacuation planning has a very high potential, but is not used as much as it could be. Communities should be able to examine various scenarios and evaluate the best ways to move people to safety, well before a dangerous situation actually occurs.

First-responder rescue efforts can also be pre-planned and evaluated. Where should various types of equipment be stored? How can it be moved? Who will staff it? What procedures should be used for various types of disasters?

As for relief scenarios, they too could be planned ahead of time with the assistance of simulation. What equipment and supplies should be stockpiled and where? How can it be quickly relocated? Who will staff it? The logistics of a large scale disaster-relief effort, including health care provisions, security at all levels, and even communications, (all of which often involve multi-organization coordination) is a great opportunity to showcase the true benefits of using simulation.

Large corporations and other organizations can also do their own simulation-based planning. Contingency plans for various scenarios can minimize the impact of a local or regional event and help ensure that a single event does not cripple the entire organization.

Louisiana State University has a relatively new center for disaster management and has organized a conference November 16-18 dealing with some of these issues.

Be Prepared” is a motto that anyone planning for a disaster should live by; Simulation helps make that a bit easier.

Dave Sturrock
VP Products – Simio LLC

# Keep Simulation Projects Simple Too

We all have stories about company decisions that make us shake our head. If you have ever worked for a large organization, it may have seemed that some of their decisions were, shall we say, sub-optimal.

For example, one particular organization was using a “home grown” time reporting system that was simple, efficient, and worked well. However upper management felt the need to buy a more sophisticated “name brand” system. Unfortunately it was poorly designed and overly complicated. Rollout required extensive training and retraining to learn the simplest tasks. It was so difficult to use that many employees simply stopped using it in favor of informal arrangements with their managers (who also found it difficult to use). As a result, the company spent a lot of money and wasted a lot of employee time, and in the end they had a system that produced inferior results.

If this was an isolated case, it could be easily forgiven. But I expect most people working for large organizations could cite similar situations. Large organizations often tend to replace simplicity with complexity.

Last week in Keep Simulation Simple I talked about KIS; the Keep It Simple concept of doing just enough to do it well and no more!

I discussed how KIS could be applied to model building, but you can also extend the KIS principle to many other aspects of simulation, especially the tools you routinely use.

For time tracking, you can buy expensive highly integrated software systems like the organization above, or for desk- bound employees you can buy software that will sense or periodically ask and record what you are working on. But the cheaper, simpler and more effective solution is simply using a spreadsheet or paper form and having the employee take two minutes at the end of each day and record time against their tasks for that day. Sure there can sometimes be reason for other methods, but for the majority of us the spread sheet solution is superior.

For project management, choose the simplest tool that will meet your needs. Some projects are complex enough that they need project management software like Microsoft Project or something even better. But in many cases, such software results in a waste of time when a simple spreadsheet could meet your needs. In my experience project management software is often overkill for the types of projects we usually encounter.

In simulation software there is some inclination to buy the most comprehensive software that you can afford. But it is often better just to buy the simplest software that is likely to meet your short to intermediate-term needs. An important caution here – make sure that your software has an adequate upgrade path so that as your needs evolve you can migrate into more feature-rich software without losing your initial investment in software, training, and models.

Stay vigilant for time wasters – they often come disguised as “cool technology” and “time savers”.
Keep It Simple.

Dave Sturrock
VP Products – Simio LLC

# Keep Simulation Simple

I mentioned a while back that I am a Boy Scout. OK, maybe my boyhood days are long gone, but I still consider myself to be a Scout. I learned many lessons as a Scout; lessons that continue to serve me well today. One of those is KIS or Keep It Simple.

I remember learning primitive camping skills. Many novice campers would bring too much gear, requiring hauling and storing it, and just in general complicating camp life. The simple (KIS) approach is to bring only what you absolutely need. Many novice campers would also select poor camp sites and then spend time dealing with dampness, bugs, discomfort, safety issues and more. The simple approach is to avoid those issues by selecting a good camp site. Then in both cases, you spend all that saved time enjoying the camp and doing what you came to do. KIS pays off.

KIS applies equally well to many aspects of simulation. When things go wrong, it can often be traced back to too much complexity.

• How many people are subjected to overly complex management procedures?
• Are the procedures used for planning and tracking your work making the most effective use of everyone’s time?
• Is every aspect of your work done effectively?
• The basic concept of KIS is to do just enough to do it well and no more! Does this mean you should not do your best? No. But it does mean that you should segment your work into small phases and KIS on each phase.

In model-building for example, let’s say a stakeholder expresses desire for a detailed model for the 10 areas of his system. One common approach is to go off and create exactly what the stakeholder asked for. Unfortunately, this will probably be wrong. A better approach is to pick one representative area, and do a very high-level model of that one area. Then review that model and results with the stakeholder. In most cases, you will both learn a lot and you may jointly decide on a different approach. Then perhaps do a detailed model of that same area or perhaps extend that high-level model to a few more areas. Again you will probably learn something that will change your approach or objectives. For each phase, you want to do the simplest thing (KIS) that will meet the objectives for that phase. In this way, you will minimize any wasted effort and come much more quickly to exactly what the stakeholder needs.

Let’s consider model-building at a much more detailed level. A common mistake by novices is to build a large section of a model (perhaps even an entire system) all at once. And then you hit “Go” and it does not work. Why doesn’t it work? There are perhaps a thousand possible reasons to investigate. Even worse, there are most likely dozens of small or large problems, each potentially obscuring the others. Verifying and validating such a model is a daunting task. A much better approach is to start by selecting a very small (KIS) portion of the model to build and verify that it works. Then repeat. When a problem is discovered in any new section, it is generally easy to find it because you know it is a result of that latest section just added. Again, “Keep It Simple”.

Remember, Keep It Simple. Work effectively and exceed your stakeholder expectations one simple step at a time.

Dave Sturrock
VP Products – Simio LLC

# We Are Not Making Swiss Watches

Please bear with me while I mention three apparently unrelated topics.

I have a very good friend Harry who often offers me advice. Fortunately for me, the fact that I don’t want the advice has no impact on whether I will receive it. 😉 I have to admit that on those rare occasions when I have been accused of expressing some wisdom, it can often be traced back to conversations years ago with Harry. I always admired his ability to “cut to the chase” and identify what is really important.

For those born into the age of digital watches, it was not too long ago that most watches were completely mechanical with lots of moving parts. Expensive watches were made with high precision and featured exceptional accuracy. Less expensive watches were made to lower tolerances and did not keep time quite as well – every now and then they might need to be slightly adjusted to reflect accurate time. If you really needed to always have the correct time, you would want to have a watch made by the Swiss, since they were world-renowned for their quality. Unfortunately, Swiss watches also commanded a very high price and most people could not afford that luxury. However, most inexpensive watches were still good enough for typical day-to-day use.

Building models is fun and addicting! When building models it is very easy to get so involved that you forget the big picture. This is equally likely to happen when attempting to get some model detail “just right” or fine tuning an animation to make it more life-like. For example, I once modeled a material handling system where I was helping to evaluate and fine-tune several competing designs. After I had completed the model, I found myself spending hours fine-tuning the animation of AGVs unloading onto the processing devices.

So how are Harry’s sage advice, Swiss watches, and modeling related? Quite frequently when Harry would find me working on a model aspect like the one above, a typical conversation went something like this:

Dave: “Arrgh! I can’t get this exactly right.”
Harry: “Are the results correct and validated?”
Harry: “Can someone look at the animation and understand what is happening?”
Dave: “Yes.”
Harry: “Well Dave, you are not making Swiss watches here…”

This was Harry’s way of bringing me back to focus on what is really important. While everyone wants to have the most realistic model and animation, for the vast majority of projects that level of realism is no more necessary than a Swiss watch. That’s not to imply that shoddy work is acceptable. Shoddy work in never acceptable – every project should be completed at least at the level of detail and quality necessary to meet the project objectives. But even though an extra level of realism is nice, it’s a luxury that most cannot afford. And that is especially true if spending time on the luxuries causes you to neglect the basics.

In fact, you should apply this concept even more broadly. When a project is not going as well as expected, first look closely to see if you or anyone involved is spending valuable time on things that are not absolutely necessary. If so, refocus on your priorities.

I urge you to join me following Harry’s advice: Concentrate on the basics first and leave the “luxuries” for later.

Dave Sturrock
VP Products – Simio LLC

# Executing a Simulation Project for the Simple Minded

This week I thought I would step back and offer a somewhat light-hearted summary of some of the things we have covered. Here are five simple steps for executing a simulation project.

1. Figure out what to model. You can do this by being brilliantly insightful (which you might be), or by just talking to some stakeholders. Ask them questions. How would you use this model? What are your problems? What type of solution are you looking for? Will you invest time in this? Will you make decisions based on this model? Are you going to roll your eyes and laugh out loud once I leave the room? Are you going to tell your spouse about this project over dinner this evening to demonstrate that you indeed do have a sense of humor?

2. Build something. It doesn’t have to be the world-changing model you devised in step 1, but a close enough approximation. It should do at least one useful thing from the list of game-changing things that’s on the feature-list from #1. Oh, and it should sort of work (even if requiring the assistance of some chanting, prayer and promises to recycle more).

3. (Option A) Deliver it! Get your project in the hands of users. Even if it’s incomplete and not fully validated. It is possible that everyone that sees it runs screaming in the other direction. Mothers protect their children in its presence. But, get it out there and work like heck to deal with the aftermath of the great steaming heap you’ve unleashed upon the world.

3. (Option B) Make Perfect, Wait, Deliver it! This avoids the problems with Option A because people will no longer run screaming. But, nobody cares about your project now because everyone is flying around with jet-packs on their back and 16-core processors are embedded in people’s brain as an outpatient procedure. The problem moved on and your solution (however “perfect”) is now irrelevant.

4. Present. Present. Present. The law of large numbers says that the larger the number of stakeholders exposed to the project (see Step 3a), the more people you’ll encounter with average coordination who will trip and fall when trying to run away from your solution. Some of these people will buy-in while still in a semi-dazed state. Voila! You have happy stakeholders.

5. Refine. Armed with a few active stakeholders, see what you can learn from them. What are they trying to accomplish? How do they use the model? What do they say between the screams of frustration? Figure out how to lower the pain quickly and treat them gently. During these brief spites of happiness that your stakeholders have, other stakeholders who come into contact with them think “Hey, Joe seems to be happy — even though he’s got this far-away look in his eyes, maybe this model is useful. Let me try it out…” Bing! You have another “happy” stakeholder.

And the story goes on.

For the really, really simple minded here’s the summary:
Decide what to model, complete an imperfect prototype, get stakeholder buy-in, keep improving, get more stakeholder buy-in. Lather, rinse, repeat. SUCCESS!

Thanks go to Dharmesh Shah of the OnStartups blog who provided the basis for this article. For anyone who has ever been, or wanted to be, part of a startup, you might find this blog interesting and entertaining.

Dave Sturrock
VP Products – Simio LLC

# Well – Managed – Agility

In previous articles we discussed a definition of agility and the potential oxymoron of managed agility. I coined the phrase “well managed agility” as part of the solution. I was asked why I used the word “well” in that phrase. The short answer is that anyone can say they are agile. And anyone can say that they manage agile. But doing both and doing them well is the key.

Good management in general is a topic I’ll leave for later. Today, I will concentrate on managing the process of change.

The most important part of managing agility is to have a single person (or a small team that works cohesively) responsible for the agility. Ideally this person, who I’ll call the agility manager, should be knowledgeable and sensitive enough to understand:
1) What the stakeholders want,
2) What the stakeholders need, and
3) The issues involved in delivering the above two items.

The project demands a lot from the agility manager. It demands the ability to get inside the heads of stakeholders and really understand not only what they want, but what they really need. Sometimes even the stakeholders don’t know what they need until they see it (or miss the lack of it). The agility manager must also understand the issues and tradeoffs on the development side of actually meeting those needs (including deadlines).

Some common agility challenges include:
• The system being modeled changes (perhaps the design has evolved, or “assumed” aspects have just come to light)
• The stakeholder objectives change (date, modeling details, purpose of model)
• Modeling problems are discovered (modeling or data collection may be more difficult than expected)
Often changes like these make current plans invalid. Of course the stakeholders want it all and many conscientious modelers will want to say yes to satisfy the stakeholders.

To do the job well, the agility manager must be someone who understands all the above aspects and who can take the broader view to determine what action best serves the stakeholders? Choices like adding feature A versus doing a more thorough job implementing existing feature B, when you don’t have the time or resources to do both. Or when a project is running behind schedule, is it better to schedule extra effort, remove some features, or postpone the delivery date? Only a very knowledgeable manager can correctly make these decisions by carefully weighing the benefits/rewards of each action.

Even better, the most effective manager will try to avoid having to make big decisions like the ones above, by instead making correct calls on numerous similar small decisions that arise on a routine basis. In a future installment I will discuss managing the “routine” and some tools and processes to facilitate that.

Dave Sturrock
VP Products – Simio LLC