Chapter Webinar with ACP Mid-Florida

A Real Quantitative Approach to Evaluate the Effectiveness of Tabletop Exercises 

About the Presentation

Operations-based exercises readily lend themselves to quantitative measurement tools through observation (e.g., the Crisis Management Team met withinx minutes of the incident). However, discussion-based exercises, such as a tabletop exercise (TTX), do not.Existing TTX evaluation methods that use quantitative tools gather information from participant responses to statements using a 1-5 numerical Likert scale. However, many of the statements focus on exercise delivery (e.g., “Participants discussed...”), or are only conclusory (e.g., “The exercise increased my understanding about the capabilities of participating departments.”). While some believe these methods are sufficient, organizations need alternative methodologies to gather more reliable data. If an organization uses the wrong TTX evaluation methodology, it will not accurately determine effectiveness or measure improvement over time.

This presentation discussion will examine the “pros and cons” of different TTX evaluation methodologies, including the Homeland Security Exercise Evaluation Program (HSEEP). The panel will then discuss how new, cost-effective TTX evaluation tools can more reliably and accurately determine whether the TTX has improved the organization’s preparedness, which may pave the way toward developing an industry standard.

About our Speakers

John Duda, CEO of SummitET Inc.

John R. Duda is the CEO of SummitET Inc., a veteran-owned small business that provides proven full-spectrum preparedness solutions to systematically address threats and hazards through a wide range of services. Mr. Duda has supported government and private sector clients as program manager for multiple domestic and international planning and exercise programs, and he has designed and facilitated numerous exercises. Since 2012, Mr. Duda has conducted research and has coauthored an article that discusses existing and new methods that can be used to evaluate the effectiveness of tabletop exercises.

Scott J. Glick

Scott J. Glick is the General Counsel of SummitET Inc. Mr. Glick previously served as Director of Preparedness and Response and Senior Counsel in the National Security Division at the U.S. Department of Justice (DOJ), where he led DOJ’s national preparedness policy and planning efforts. Since retiring from DOJ, Mr. Glick has worked with numerous government agencies on ways to infuse policies and plans into exercises. During his career, Mr. Glick has facilitated, designed, and participated in national-level tabletop and full field exercises, and has coauthored numerous articles, including an article on methods to evaluate the effectiveness of tabletop exercises.

 Meeting information: