Governance Nest: After Action Report (AAR) Process

Governance Nest: After Action Report (AAR) Process

From our guiding principles:

  • We’re on an never ending journey of growth & improvement: We value growing as individuals, as a cooperative, and as a broader community. We respectfully challenge each other and our assumptions, constantly searching for new areas to grow, whatever “grow” might mean. We recognize our current plan can always be improved and that the world is dynamic and constantly changing - thus we collectively need to adapt and change too.

The business need:

As a learning organization - it’s helpful to systematically drive process improvements via thoughtful reviews of significant events across the Co-op. To do this, it’s helpful to have a framework to consistently execute reviews and capture learnings. This post proposes to reapply (loosely) the After Action Report (AAR) process and is part of the Governance Improvement Process (GIP)

The process:

The objective of this process is to quickly yet systematically collect feedback after an event with the intent to improve future operations. Based on the “After Action Report” process developed by the US Military and loosely linked to the TC 25-20 “A leader’s guide to After-Action Reviews” this survey is not intended to find blame, fault, or identify scapegoats. The sole purpose of this process is to document what worked and what did not work as a way of informing future efforts and to build a culture of self reflection and continuous improvement. At Index Coop, we are on a never ending journey of self improvement, and the AAR is a tool we can use to do that.

After-action reviews are conducted during or immediately after each event and start with the focus of the intended objectives - what was the objective of the action. The audience should include the contributors, leaders, and ancillary stakeholders, remembering to involve as many participants in the discussion as feasible. It is helpful to use open-ended questions and if possible, link responses to documented core principles of the DAO. The output is to document the strengths and weaknesses of the execution, identify any unintended consequences (positive or negative) and to finally develop a short list of recommendations which may include process changes, objective adjustments, or additional training.

Getting it done in 3 steps:

  1. Planning phase

A solid outline of the effort up front results in better results and better perception of a fair and equitable process. If you rush the planning phase you end up “learning by doing” and later interviews can be disproportionately weighted as the process is refined. You can protect against this by setting up a thoughtful interview process.

Specific steps in planning:

  • Develop the timeline & outline the plan
  • Determine the list of target interviewees
  • Pilot 3 to 4 interviews to test process
  • Select and train qualified AAR contributors
  1. Conducting phase

This is where the team goes out and solicits feedback on the event. It is important to find interviewers who can record responses impartially - they don’t have to be impartial, but they need to recognize the survey is not about them, it’s about getting accurate feedback. As conducting, seek maximum participation and maintain focus on the objectives - avoid casting blame and keep the interview focused on the scope, don’t let it turn into a general bitch fest. As you capture results, record the key themes and try to prioritize the input based on the subjects responses.

  1. Wrapping up phase

As surveys return, draft an impartial analysis on the data to identify the following points:

  1. General sentiment of the action (positive / negative)
  2. Summary of what worked and what didn’t work
  3. Identify any urgent issues that need to be resolved
  4. Develop a list of recommendations for future improvements

For clarity, the final report must contain the following components:

  • Summary of sentiment: showing the depth and range of opinions
    ** Outline of work worked based on feedback
    ** Outline what did not work and needs to be improved
  • Collection of improvement ideas
    ** As best possible - summarize & quantify recommendations
  • AAR Process suggestions
    ** What about this process worked and what did not work
    ** How to adjust the format for future investigations

Inaugural execution: After Action Report for the Index Coop Council

To note: this After Action Review will use the March 08 Retro meeting with Chase Chapman as a starting point - that effort will go deep on collecting and quantifying community sentiment. This second step will go deep on collecting recommended actions and improvements while making sure we have captured as much feedback as possible. Once both efforts are completed, it is our intention to build a retro / AAR process that anyone can use in the future to collect community feedback and to articulate future improvement ideas.

Plan Outline:

Refine plan March 3-4

Surveyor kick off call March 08

Survey period March 08-15

Reconcile results March 15-18

Publish report March 21

Target interview list:

ICC members - past and current:

  • Pepperoni joe, Matthew Graham, Metfanmike, Lemonade Alpha, JD, Mel.eth, Cavalier

Engaged community members:

  • Post commentators
  • Anyone who voted on V1 or would like to participate

Survey execution:

  • Start with an overview of the the IC principles
  • Review the objectives of this exercise (to capture lessons learned and to improve)
  • Summarize the scope of the conversation (the effectiveness of of the IIC from x to y)
  • Let them know all responses and interviews are anonymous
  • Ask if they are ok to have the session recorded so you can go back and clarify

Part 1: Baseline questions

  1. What was the objective of the Index Coop Council?
  2. On a scale of 1 to 10 how effective was the process? (1 not effective, 10 very)
  3. On a scale of 1 to 10 how transparent was the process? (1 not transparent , 10 very transparent)
  4. How would you classify your role: (ICC member, nest/pod leader, community member

Part 2: What worked well?

  • Hint 1 look for specific examples to get the person thinking about the question. If you hear “we all got along” - dive into that with “can you think of a specific example of a time when “you got along” - tell me about that? Specific examples will led to a richer conversation than empty phrases which don’t add depth to the analysis
  • Hint 2: The answer will turn into a rambling list - capture each response to “what worked” and dig into each question below for each “what worked”
    • Example: “we got along famously and the process was clear, we moved way faster then we ever could have, and those who disagree are just a bunch of ninnies” becomes:
      • A1: we got along Ask Q2, Q3,
      • A2: Process was clear Ask Q2, Q3,
      • A3: we moved fast Ask Q2, Q3,
      • A4 (to what didn’t work) Not everyone agreed (were ninnies)
  • Hint 3: people always want to jump to “what didn’t work” - don’t let them do this. Focus on what worked and get them to indicate a few things that they perceive went well. This sets a solid foundation for the conversation and makes sure you do capture items that need to be celebrated. There will be rare cases where the person has nothing good to share… this should not be the norm.

Q1: What worked well?

Q2: How impactful was that to the (success/failure) of the effort?

Q3: how would you suggest we improve on that?

(iterate back to the list of what worked)

Part 3: What didn’t work so well (what do we need to fix for the future?)

  • Hint: keep the person focused on “what worked” until their list is exhausted
  • Once they run out of “what worked” then focus on what did not work”

Q1: What was something that did not go so well?

Q2: How impactful was that to the (success/failure) of the effort?

Q3: how would you suggest we fix that in the future?

Part 4: Wrapping up

Q1: Of all we talked about, what one thing should we do differently?

Q2: Is there anyone else whom you think we should speak with?

Q3: What would you recommend we do next?

Close with a review of the timeline and let them know when they should see a summary report

5 Likes

First few interviews completed today and I am humbled by the quality of the conversations. The power of the Index Coop is the people, and we need to explore every chance to have these brilliant people co-create the future. Its not quick, its not efficient, and it takes more time than we think it should - but we land on better decisions that will strengthen us into something unforkable.

Sneak peak into Day 1 recommendations:
What top things would you recommend for ICC V2?

3 Likes

Day 2: another round of amazing proposals for V2 inclusion. Our target is to reach 30 of the 100 contributors in the Coop. If you have not received an invitation and want to talk, ping me on discord (#shawn16400) we will send you an invite.

#4 The next IIP should be for a standing council (not every 6 months)*
4 Council members put up a sizable bond - which can be slashed for misbehavior**
7.At least one woman and at least one diversity candidate on the council (can’t be the same person)**
correction:

1.Adopt the Amazon framework of Type 1 and Type 2 decisions (don’t spend time on the small stuff)
5. Nests develop strategies, Nest leaders synchronize strategies, ICC solves disputes**
5. Council Needs to be able to hold Nest leaders accountable to adhering to defined strategy

2 Likes

Day 3: Are we just enamored with DAO Porn? We need to accept some centralization to compete. Today delivered clearly articulated contrarian views - all great stuff!
We will continue interviewing through early next week -
if you have not received an invite and want to, ping @shawn16400 on discord.

1 Like

I had a conversation today about quadratically prioritizing the actions & principles we are collecting into a segmented, coherent, bottom up proposal. I wonder what it would look like. Feedback snippets from the day:




2 Likes

After hitting our target, we are winding down the interviews with a few more missed connections. Last chance for input, if you have not met with @asira or @shawn16400 and want to provide feedback, please reach out. Given the passing of v2, the pressure comes off a little, but the delivery of results becomes a bit weird, but we will figure it out.
Samples from today and yesterday:

4 Likes