Top Five Mistakes in Clinical Protocol Design

By Lisa R. Sanders, Ph.D., R.A.C., Sr. Clinical Scientist II

Almost a Holy Grail for the pharma/biotech world, the perfect clinical study protocol requires no amendments, collects only the data needed for the planned analysis, is fully feasible, and is enrolled on time. The reality is that the average clinical trial protocol requires 2-3 amendments (with >40% occurring before a subject is enrolled)[1], collects a large amount of data that is not associated with any key endpoint or regulatory requirement[2], and includes so many entry criteria that it can be almost impossible to enroll in a timely fashion. And while it is widely recognized in our industry that protocol design and complexity is a key cost driver, our industry struggles to make the changes necessary to address this issue. Over the past 10 years as a clinical and regulatory scientist I have seen plenty of clinical study protocols from a lot of different companies: small virtual companies, medium-sized biotech companies with several products in the pipeline, and well-established global pharma with numerous marketed products. But regardless of company size or experience, there are protocol issues repeated again and again. Timelines, business pressures, egos, compartmentalization...all can contribute to a flawed clinical protocol. And since everything else flows from the protocol, issues are propagated down the line.  Here are the five mistakes I have seen most often and suggestions on how to mitigate them.

1. Too many objectives

A recent protocol I reviewed had nine different study objectives. Another had 20 (but to be fair, five were exploratory). As we try to pack more and more into a study as a cost saving measure, the number of objectives is increasing. The downstream affect is more endpoints and assessments, increased cost, more complicated logistics, and a muddled sample size determination and data analysis. Focus first on the key questions:

• Why do I need this study?
• What is the primary scientific question I need to answer?
• What are the regulatory requirements that must be met by this study?
• What must I learn to support the next step in development?

This gives the basis for the handful of objectives your study will support. To keep your protocol focused, at Cato Research we recommend no more than 1-2 (preferably 1) major/primary objectives and no more than 3 minor/secondary objectives.

2. Not first creating clear, concise, well written, and agreed on objectives

  All too often we see protocol objectives that are too broad, unclear, or missing key information. For example: “To determine the safety and tolerability of Drug X.” So broad an objective does not give us much to work with to determine endpoints, assessments, and design the data analysis. A well written objective will include all of the following information:

• Product being tested
• Dosing: level(s), frequency, duration, and mode of administration
• Study population
• Clinical endpoint
• Parameter measured
• Time of assessment
• Control

Adding to our previous example: The primary objective of Study CLIN-002 is to evaluate the safety and tolerability of Drug X capsules at four dose levels (5 mg, 10 mg, 15 mg, 20 mg), versus placebo, when administered orally once daily for 28 days to subjects with hyperlipidemia by assessing adverse events, physical examinations and vital signs, 12‑lead electrocardiogram, clinical blood chemistries, hematology, and urinalysis during the treatment period. A specific objective such as this provides clear direction for your endpoints, assessments, and statistical analysis. You can be sure the data collected will evaluate the specific question you are trying to answer. And your objectives require input from others BEFORE they are finalized (see Step 3).

3. Not seeking interdisciplinary input (a.k.a., Not enough cooks in the kitchen) A clinical protocol merges scientific data, medical expertise, regulatory requirements, and operational logistics. But all too often medical writers go it alone in the protocol development process, and only look for input on the final product, when substantive changes are time consuming and often shunned. And input from the clinical operations perspective can be overlooked entirely. Interdisciplinary input is key from the start of what is a three-stage process:

• Objectives and endpoints: Input from medical experts and statisticians at the objective stage is key to ensure they can be supported by measurable, collectable, and analyzable endpoints. Regulatory input is needed to ensure that data required by regulatory authorities is not overlooked. An interdisciplinary review of the study objectives and their supporting endpoints at this stage is a must.
• Synopsis: Creation of the study synopsis through the addition of logistical and data collection, and data analysis details requires input from medical, data management, statistics, and clinical operations. This ensures your protocol design meets applicable guidelines, is logistically sound, and produces analyzable data before all the body text is added. Again, interdisciplinary review and agreement is critical at this stage, while your protocol is still in outline form.
• Full protocol: Armed with a fully vetted synopsis, creating the full protocol basically becomes a “fill in the blank” exercise. The key design decisions are behind you, and you can proceed knowing it is unlikely there will be many surprises during the final review process.

At Cato Research, we plan an interdisciplinary round table meeting at each stage of the protocol development process. This provides an efficient approach to garner interdisciplinary input that allows discussion and consensus among the different disciplines.

4. Just doing what you (or someone else) did last time  It is not unusual to do a protocol review and find some entry criteria, endpoints, or assessments that just do not make sense in the context of the study. For example, having a criteria that prohibits pregnancy or breast feeding in a congenital hemophilia or dementia study. And when we ask where an item came from, it is not uncommon to hear, “That was included in other protocols I looked at,” or “That was in the last protocol we ran in this indication.” It can make a lot of sense to go back and look at what you did last time--why reinvent the wheel, right? A review of published studies and those active in clinicaltrials.gov can provide insight into current design trends for a specific indication. Your last protocol for Disease X was thoroughly reviewed and successfully run, so it may be a good starting point. But over time and across authors, the specific reasons why a certain endpoint or entry criterion was included can be lost. The study population is different, the indication has changed, the “gold standard” endpoint has changed, the statistical approaches have evolved. Exercise caution when following the example of earlier protocols. Critically evaluate the basic design elements to ensure they still apply. And do not skimp on Step 3 just because you had an example to start with.

5. Redundancy, Repetition, Reiteration  Call it what you will, but repeating the same thing in multiple sections of a protocol can create serious headaches. Consider the following four examples of protocol text from four different sections of a single protocol (synopsis, methodology, time and events table, and protocol body, respectively):

• Vital signs including pulse, blood pressure, and respiratory rate will be routinely monitored during treatment, minimally at each assessment.
• Vital signs will be collected 10 minutes after administration of dose
• Vital signs (10-20 minutes after administration of dose), includes blood pressure, heart rate, respiratory rate, temperature
• Vital signs are taken at each dose

Sites may follow different timing for vital sign collection, based on where in the protocol they are looking. Vitals could be taken at the time of the dose, between 10 and 20 minutes after dosing, at 20 minutes after dosing, and some sites will not include temperature. While this could be considered a minor problem in this example, it provides an excellent illustration of what can happen when the same detail is repeated in multiple protocol sections. Throw protocol amendments into the mix and you can end up with a mess. So how do we fix this? Simple--eliminate repetition of details wherever possible.

• Keep the synopsis and methodology sections to a high level descriptions and put the details in the protocol body text. Refer to specific sections if needed.
• Avoid footnotes in a time and events table, and instead reference the appropriate protocol section or a study procedures manual.
• Consider developing standardized template text to be used across protocols for common, recurrent items such as the assessment of physical health, medical history, concomitant medications, adverse events, and blood and urine collections.
• Whenever possible, you can keep the nitty gritty (e.g., collection time windows, study product logistics) out of the protocol and in a separate study procedures manual. This also eliminates the need to amend the protocol for minor procedural changes.
• Review, review, and review (see Step 3). Getting fresh eyes on the final protocol is a great way to catch inaccuracies and repetition.

There is often a great deal of pressure to author a protocol quickly to meet a filing deadline, a corporate milestone, or to keep development moving forward quickly. But protocol amendments are costly and time consuming to produce and implement. Having from the outset a concise, well-written, substantively designed, logistically feasible protocol that minimizes amendments will more than make up for the additional up front time. [1] http://csdd.tufts.edu/news/complete_story/pr_ir_sep-oct_2011 (accessed 25 Sep 14) [2] http://csdd.tufts.edu/news/complete_story/pr_ir_sep_oct_2014 (accessed 25 Sep 14)