by Antonio Rodriguez (N.I.H.)
As presented at the 2018 Winter Simulation Conference
Even the best scientific minds cannot repeatedly produce desired results when working in sub-optimal systems. Complex enterprises are difficult to understand and manage. Cause-and-effect relationships are often separated in time and space, making real improvements challenging. To understand how complex systems work it is essential that we employ tools that accurately map and quantify the dynamics that drive results. Computer modeling and simulation (CMAS) is a valuable design, planning, management, and overall analytical decision-support tool to achieve effective and efficient results. CMAS could become a ubiquitous tool in the lengthy and complex environment of the clinical research (CR) enterprise. Without the comprehensive understanding gained when applying CMAS, organizations may continue to be overwhelmed by problems such as unnecessary bottlenecks, high costs, low productivity, and the inability of retaining critical staff. The approach explained here may complement or even replace traditional methods when organizations pursue greater enterprise capability.
Introduction
The continued development of computer modeling and simulation (CMAS) products have facilitated new capabilities for understanding interactions among multiple factors across producing spectrums. The use of computer models provide access to new understanding unavailable in the physical world—an understanding that leads to improvements required to achieve and sustain efficient systems where contributors can proudly realize their objectives.
The clinical research (CR) field is complex and could benefit from the application of CMAS to drive new understanding and improvement in its diverse enterprises over time. CMAS tools today are more robust and user friendly than ever, and include accurate animations to better communicate with all stakeholders. Simulations are more realistic when they include quality, scheduling, resource utilization, unexpected events, and constraints in the model. These topics will be addressed in the discussion section and will be integrated in the proposed approach to use CMAS in the CR enterprise
Discussion
In a tight fiscal or highly competitive environment, efforts to accomplish critical enterprise results could be guided using methods and tools that take into account variability in 1) the demand for outputs, 2) the capacity and constraints of its processes, 3) production schedules, and 4) unexpected events. Efficient and effective end-to-end processes result mainly when a pre-defined sequence of actions is well-balanced in terms of capacity and demand. To achieve a sustainable balance of end-to-end processes, the study and improvement effort may also need to include critical support functions.
Inadequate capacity throughout the sequence of activities and decisions will cause unnecessary delays and quality problems in some areas, only to be compounded with costly overcapacity in other areas. The cumulative effect of these prevalent unbalanced systems produces unnecessarily high-cost products and services, and frustrated stakeholders.
Like the private sector, federal government is frequently challenged by limits in budget and human resources. My experience in the pharmaceutical and electronics industries is that more effort was placed on improving operations on the manufacturing side than I have witnessed in operations, research and development, and support service areas in federal government agencies. This was done, for example, 1) through changing facility layouts; 2) changing inspection methodologies; 3) applying statistical methods to demonstrate that changes made to raw materials, processes, or equipment accomplished desired results; and 4) applying new technologies to fundamentally change established processes.
However, even in manufacturing facilities the problem of unbalanced processes or activities caused the cost of converting inputs to outputs (e.g., production planning, purchasing raw materials, production scheduling, assembly, filling, testing, sterilizing, packing, distribution) to be higher than necessary. Staff has to be hired, trained, and provided with an adequate work environment; equipment has to be purchased, calibrated, maintained, and replaced; production has to be planned; new products have to be pilot tested; facilities have to be secured, sterilized, operated, maintained, renovated, and in some cases expanded; products have to be packed, distributed, or scraped and disposed of; electricity, fuel, and water have to be purchased and in some cases even generated; and regulatory requirements have to be met and documented. The aforementioned activities are performed regularly and describe what may need to be done to produce outputs.
Knowing what is needed to produce a product is necessary, but not sufficient, to achieve effective and efficient end-to-end processes. Multiple bottlenecks will occur unless a balance between capacity and demand in most activities is carefully managed. This does not mean a one-to-one ratio between capacity and demand, it means sufficient capacity to meet the demand without creating bottlenecks larger than what the customers or participants are willing to tolerate—this is a moving target.
Consistent efficiency and effectiveness are a function of multiple factors such as balanced capacity and demand in the sequence of activities and decisions (i.e.
Production schedule production schedule
CMAS is an excellent tool to model different scheduling scenarios quickly to inform decision-makers on the likelihood of success (Pegden 2017). Different schedules for the same products will require resources at different times. Experimenting with the availability of limited resources in a computer model and seeing the impact on various scheduling scenarios helps decision-makers select more predictable scenarios. The complexity rapidly increases as the quantity and mix of products increase. Complex schedules are subject to change. Being able to anticipate the impact these changes will have on the availability of resources and the delivery of materials is essential to minimize scheduling disruptions and effectively execute production plans. Schedule changes may impact the balance needed for efficiency by altering available capacity or by lowering expected internal demand throughout the process. Schedules could be monitored periodically to determine if changes will impact resources needed to reduce bottlenecks, or even better, achieve bottleneckfree processes.
Quality
Many have attempted to define quality. It is difficult to do. Juran mentioned “fitness for use” (Juran 1989, p. 15) as a definition that received wide acceptance. He also stated that fitness for use “does not provide the depth needed by managers to choose courses of action” (Juran 1989, p. 15). W. Edwards Deming stated that “quality can be defined only in terms of the agent” (Deming 1986, p. 168). Different agents may view or define quality in different ways. Therefore, any serious attempt to provide quality requires operationally defined terms agreed on by all significant agents or stakeholders (Ackoff 1994).
Shewhart examined the subject of quality from different perspectives (Shewhart 1931). He addressed, for example, the popular conception of quality—quality of product, quality of process, and quality as a relationship. The subject is complex, but Deming summarized the various constructs addressed by Shewhart as “the difficulty of defining quality is to translate future needs of the user into measurable characteristics, so that a product can be designed and turned out to give satisfaction at a price that the user will pay” (Deming 1986, p. 169).
Process quality impacts production capacity. Rework (i.e., making changes to something already done) and scrap (i.e., having to throw away work done and start over again) diminish available production capacity and add unnecessary cost to the final product. If a finished product in the hands of a consumer adds value or satisfaction for the price paid, it should be considered efficient and effective. However, the performance and cost of competing products or providers of the same product may change the final verdict of the customer or sponsor. Rework and scrap levels, and their impact on capacity and demand throughout processes, can be easily assessed using CMAS.
Without consistent quality upstream, enterprise resources will be consumed without producing the expected demand in an organization’s domain of the process. Also, rework and scrap in one step of the process will waste some available capacity and deliver an inadequate amount of output downstream. This inefficiency unnecessarily increases the cost of work already performed. Therefore, every step in the process must perform well on a consistent basis if efficiency and effectiveness are to be sustained
Clinical research
The process of “making important discoveries that improve health and save lives” (NIH Who We Are 2017) is lengthy and complex. The National Institutes of Health (NIH) director, Francis Collins, described it as follows:
The translation of basic biological discoveries into clinical applications that improve human health is an intricate process that involves a series of complex steps: the discovery of basic information about the pathogenesis of a disease; an assessment of whether that information has the potential to lead to a clinical advance; development of candidate diagnostics, devices, or therapeutics; optimization of the candidates in preclinical settings; regulatory assessment of the data to determine the potential for human use; testing in human clinical trials; application for approval for widespread clinical use; and, ultimately, the assessment of approved diagnostics, devices, and therapeutics during widespread use in real-world settings (Collins 2011, p. 1).
Since this overall translational effort is a complicated progression, it may be helpful to start by focusing on a critical and sometimes limiting portion of this whole effort: clinical research. Information gained by following the approach depicted in this document could be helpful in guiding efforts upstream and downstream. “Through clinical research, clinician-investigators translate laboratory discoveries into better treatments, therapies and interventions to improve the nation’s health” (NIH Clinical Center 2018). The CR enterprise is expected to accomplish patient safety and research goals with fewer bottlenecks and more agility.
As applied to CR, what is the meaning of quality? What is a quality research protocol (final product)? What is a quality research process (all the steps and decisions that contribute to the final product)? Does the meaning of quality change throughout the process? To what extent can quality be achieved without operational definitions? (Lewis 1929). Without operationally defining the outputs of the different steps in a process, multiple agents will not have shared knowledge and actionable definitions to achieve optimal enterprise results (Deming 1986).
The NIH is expected to make “important discoveries that improve health and save lives” (NIH Who We Are 2018) Part of the NIH mission is met through CR. “Clinician–investigators translate laboratory discoveries into better treatments, therapies and interventions to improve the nation’s health” (NIH Clinical Center 2018).
What are the results expected from the CR enterprise? How many completed protocols should be expected and when? How many protocols will be in production or active every year? What are the resources needed for each one? Should a complete support system be provided to every Principal Investigator (PI), to a group of PIs, or should they be centrally provided for all? If a partial support portfolio is going to be provided, what should it include and in what quantities? How long should a research effort be funded and how should it be supported to provide for a fair likelihood of success? What should be the yield of approved protocols that reach the treatment phase? What impact does an approved protocol have on the retention of investigators and support staff, if it is canceled due to unanticipated constraints?
As in manufacturing, the CR support system is also complex. It includes staff from a myriad of professions that must be recruited and retained such as credentialed physicians, dentists, PhD researchers, research nurses, nurse practitioners, study coordinators, bioethics specialists, protocol navigators, fellows, QA/improvement specialists, contract research organization representatives, pharmacists, dietitians, medical technologists, imaging technologists, therapists, regulatory affairs staff, patient representatives, medical records and medical supply staff, patient recruitment and patient services personnel, radiation safety officers, human resources staff, and other administrative and acquisition professionals. Determining the level of staff support needed, when and where, can be better determined by using CMAS than by using other available practices such as adjusting the level of resources annually based on the previous year’s results. CMAS is also a more robust practice than making resource allocation decisions based on benchmarking studies, or industry standards developed for other institutions, or organizational history.
Professionals depend upon, for example, laboratory and hospital equipment, libraries, information technology, websites, medicines, transportation services, language interpreters, pharmacies, amenities, visa services for international patients and caregivers, background materials for reviewers, climate control systems, aseptic processing areas, protective apparel, microbiological agents, and radioactive materials. Therefore, planners and managers in the CR enterprise should address many of the aforementioned questions to adequately source them on an on-going basis.
What and how many of each type of resource is needed throughout the steps of the process to minimize unwanted bottlenecks? Identifying, prioritizing, and investing in some of these resources that seem scarce, without examining the impact on the whole process, may only shift bottlenecks. It may bring temporary relief to a part of the process, while making the whole less efficient, or even worse, less effective.
In addition, the CR system depends on scientists with outstanding credentials from outside the NIH who evaluate the quality of research and advise scientific directors, institute and center directors, the NIH deputy director for Intramural Research, and the NIH director. The contribution that they make happens before and during treatment of patients participating in protocols. Scientists also have to be identified and recruited; their participation requires a great deal of coordination. Their recruitment and coordination effort is another part of the overall process that needs to be orchestrated to avoid bottlenecks in the review and approval of protocols
Optimizing every function independently seldom, if ever, achieves overall net optimization (Deming 2000). It usually results in the creation of many bottlenecks and frustration throughout the overall process. Understanding and managing the whole CR process as a system of interdependent components is essential to maximize efficiency, while achieving the effectiveness desired. CMAS provides an excellent and efficient alternative to view and experiment with the whole process (Deming 2000).
