The mandate
After-action reviews are not optional. FEMA's Homeland Security Exercise and Evaluation Program (HSEEP) requires an After Action Report/Improvement Plan (AAR/IP) after every exercise.[1] On the military side, CJCSI 3150.25H requires units to submit lessons learned to the Joint Lessons Learned Information System (JLLIS).[2] State emergency management agencies have their own mandates. The pattern is universal: when something important happens, you are required to capture what you learned.
The problem is not the mandate. The problem is the tools available to fulfill it.
The gap
The Department of Defense built JLLIS to be the central repository for military lessons learned. In 2015, the Government Accountability Office found that JLLIS had "limited functionality" and that many units avoided using it.[3]The Army's SOLLIMS (Stability Operations Lessons Learned and Information Management System) was eventually sunsetted. On the civilian side, most agencies rely on Word document templates from the FEMA PrepToolkit — literally filling in a Word doc, emailing it, and filing it on a shared drive.
This is the state of the art for most of the U.S. government: Word templates and shared folders.
What Word templates cannot do
A Word document captures text. That is all it does. Here is what it cannot do:
- Detect patterns across events. When the same communication breakdown happens in three consecutive exercises, a Word document on a shared drive does not alert anyone. Each AAR exists in isolation.
- Link lessons to corrective actions.An AAR might recommend "improve interoperable communications." A Word file has no mechanism to assign that action, set a deadline, or track whether it was completed before the next exercise.
- Track whether lessons were actually applied. The most common finding in government AAR programs: the same lessons appear in report after report, year after year, because there is no system connecting the lesson to the fix.
- Alert when history repeats.A hurricane response exercise in 2024 surfaces the same shelter coordination issue documented in 2019. Without structured data, no one knows — until someone happens to remember, or doesn't.
What structured AAR tools can do
The alternative is not a better Word template. It is structured data — lessons captured in a format that a system can search, link, and analyze.
- Capture lessons as structured data. Each lesson gets tagged with event type, capability area, severity, and responsible unit. Not as an afterthought — as the natural way lessons are entered.
- Assign and track corrective actions with deadlines. Every identified gap links to a specific corrective action, an owner, and a due date. Status is visible to leadership without chasing emails.
- Detect recurring patterns automatically. When a new lesson matches a pattern from previous events, the system surfaces the connection. Three exercises with the same communication failure is no longer invisible — it is flagged.
- Generate compliance reports for leadership. HSEEP requires documentation. Structured tools generate AAR/IP documents automatically from the data, in the required format, without manual reformatting.
The cost of not changing
There is a phrase that comes up repeatedly in emergency management:
"The same mistakes happen every 3-5 years because the people who learned the lesson moved on."
This is not a failure of people. It is a failure of systems. People rotate. Administrations change. Institutional memory, when it lives only in Word documents on shared drives, has the lifespan of the person who wrote it.
The cost is measured in repeated mistakes, repeated spending, and — in emergency management — repeated failures that affect real communities. A shelter coordination breakdown that was identified in an exercise but never fixed becomes a shelter coordination breakdown during an actual hurricane.
Side by side
| Capability | Word Template | Structured AAR Tool |
|---|---|---|
| Capture lessons | Yes (free text) | Yes (structured + searchable) |
| Search across past AARs | Manual (open each file) | Instant (full-text + tag search) |
| Detect recurring patterns | No | Automatic |
| Track corrective actions | No | Assigned, tracked, reported |
| Alert on repeated failures | No | Automatic |
| Generate HSEEP-compliant reports | Manual formatting | Auto-generated |
| Survive staff turnover | Rarely | By design |
Notes
- [1] FEMA Homeland Security Exercise and Evaluation Program (HSEEP), January 2020 revision. Requires AAR/IP for all exercises conducted under HSEEP doctrine. Available at preptoolkit.fema.gov.
- [2]Chairman of the Joint Chiefs of Staff Instruction 3150.25H, "Joint Lessons Learned Program," directs use of JLLIS for lessons learned collection, validation, and dissemination.
- [3]GAO-15-243, "DOD Should Enhance Lessons Learned System for Joint Operations," February 2015. Found that JLLIS had limited search functionality, inconsistent data entry, and low adoption rates across services.