APWG Growth & Performance Measures: Q4 Updates

Overview

This post is intended to

  • clarify APWG current objective and key performance indicators
  • provide an up to date reference for monthly reporting during Q4 2021 @pujimak_in
  • ensure transparency and opportunity for community feedback
    CC: @lanks @Vanita @Tudou
KPI BASELINE TARGET OCTOBER OBJECTIVE SOURCE FREQUENCY
Strategic
APAC Median Buy Size - ? WIP Strategic Dune 30 / 90 Days
APAC Net Buys - ? WIP Strategic Dune 30 / 90 Days
APAC % Trade Volume - ? WIP Strategic Dune 30 / 90 Days
APAC % N$F - ? WIP Strategic F. Nest 30 / 90 Days
Tactical
IB/BD Relationship 23 26 31 IB/BD Notion Month (Mth)
Bi Weekly BD IB Community Call 10 10 - IB/BD Count Month (Mth)
Community Autonomy BD 2.5 3.0 - IB/BD Survey End Q4
Community Autonomy IB 1.7 2.2 - IB/BD Survey End Q4
Community Autonomy GM 2.8 3.3 - GM Survey End Q4
Website Users Top 10 22% 22% 9.84% GM GA4 Mth / Qtr
Website user Top 50 25% 25% 18.69% GM GA4 Mth / Qtr
Website users Top 100 24% 24% 16.72% GM GA4 Mth / Qtr
Gold/Silver/Bronze Owls 45% 45% 51% TOC Discord Mth / Qtr
Women/Non-Binary Contributors 15% 15% 13.5% TOC Discord Mth / Qtr
Lastest Update: 4 Nov 2021

Discussion

Strategic Objective: Deliver quantifiable value ≥ 5 x APWG Requested Budget = ~$770k quantified in terms of net inflow, trade volume, raised awareness and minimised costs

  • A [Strategic KPI] APAC % Net $ Inflow
  • B [Strategic KPI] APAC % Trade Volume
  • C [Strategic KPI] APAC Median Buy Size
  • D [Strategic KPI] APAC Net Buys
    Target: baseline unknown
    Status: Experimental WIP requiring Analytics & Treasury Support

Although the strategic objectives remain unchanged our approach to quantifying value continues to evolve. We recognise that measuring quantifiable value is a challenge for which we continue to seek answers. As such, there has been a number of changes to the tactical objectives and key performance indicators identified in the Q4 APWG proposal as discussed below.

Increase Net $ Inflow through strategic Business Development & Institutional Business partnerships
[KPI] Growth % APAC Net Inflow
Target: baseline unknown
Status: Experimental
Changed: from a tactical to a strategic objective as defined by requiring on-chain data

We believe it is safe to say that a % of global trade volumes, net flows and assets under management, are attributable to people in APAC countries. However, we have not yet established a way to attribute Net $ Inflow to APAC trading or activity, so at this point, we can neither baseline nor quantify the value of APAC work in N$F terms.

The primary challenge is the disconnect between on-chain ($) and off-chain (marketing) data sources, and so we are in the process of experimenting with data, in search of trends and testing hypotheses’. For this purpose, a second google analytics account was established to install the former Universal Analytics (UA). UA began collecting data on 17 October to provide functionality that GA4 no longer allows.

  • Account views: allows us to create and maintain an APAC filtered (by country) view of website analytics
  • Users by the time of day: provides 24/7 website activity reporting to the hour level
  • Run API scripts for the filtered APAC account view

We are looking to surface answers to the following questions:-

Q: Are there any trends in the data to indicate typical times and days of high activity in APAC?
Q: Can off-chain data (website user activity) be mapped to on-chain transactions?

We will monitor and look to export (API scripts) day and time day from UA to see if there is any correlation between website users and signed or rejected transactions. If we can correlate APAC ~25% website user activity to x% trade volume (TV), the hypothesis is that following an event OR during spikes in APAC web user-activity, we could expect to see an increase from baseline % TV. There may be no correlation. And at this point, we have very limited data to work with (est. Oct 21), but if the scripts can automate hourly data feeds our talented data analysts will soon identify or rule out this type of approach @anthonyb.eth @prairiefi

Q: Can we attribute purchase intent signals on the website - events such as unlock wallet or click buy buttons - to signed and or rejected transactions?

At the moment the only product being “tracked” is DPI. We would like to see this expanded to include all products. Work to explore the purchase funnels and signals that correlate to transactions is currently on hold as we await the delivery of a marketing facing website. Working with @DevOnDeFi once the website is live we can progress the tagging of events on the website in order to see how APAC website users interact with our products online. APAC specific funnel information could enable future KPI’s to better reflect the GMWG AAARRR marketing funnel - awareness, acquisition, activation, retention, revenue and referrals @MrMadila

Tactical Objective 1 Drive [Strategic Success Measures] through Business Development & Institutional Business partnerships
1.1
[KPI] # Strategic Relationship established/progressed by IB BD team
Baseline: 23
Target: 26

Tactical Objective 2:: Support new contributors to develop IB BD context, depth of market and product knowledge and empower high-impact contributors through increased autonomy
2.1
[KPI] # Attendance IB BD Bi-Weekly Community Call
Baseline: TBC
Target: 10
2.2
[KPI] Self Reported Autonomy of community members in relation to Business Development, Institutional Business and Growth Marketing
Baseline: TBC
Target: 20% growth of self-reported 3 or 4 level contributors

Due to the difficulty of quantifying N$F we have created tactical objectives 1 & 2 in reference to strategic KPI’s. The actual tactical KPI’s are focused on establishing relationships, processes and removing barriers to participation.

Establishing a Bi-Weekly IB BD Community call was identified as a key process to overcome time zone barriers to participation with existing IB BD Working Group Calls. It is hoped that these calls will help the APWG community to develop context together as a cohort and further decentralise knowledge in order to build autonomy.

We will baseline self-reported autonomy at the first meeting via a simple Typeform survey.

On a scale of 0 - 4: at this stage of your journey, how confident do you feel contributing to the operations of each of the Index Coop’s existing working groups? (BD, IB, GM)

0 = Not Applicable
1 = learning & seeking support
2 = capable & seeking guidance
3 = experienced & seeking collaborators
4 = professional & seeking full autonomy

At the beginning of Q1 2022, we will again survey the community to identify if these meetings have helped community members develop autonomy and confidence in the areas of institutional business, business development and growth marketing. The first meeting is scheduled for Tuesday 2 November at which point baselines will be added to the summary table.

Tactical Objective 3: Increase $ Aggregate Trade Volume by raising awareness of Index Coop Products through regional content development and events with CTA that drive visitors to the IC website.
3.1
[KPI] # Website Users Top 10
Baseline Target: 22%
3.2
[KPI] # Website Users Top 50
Baseline Target: 25%
3.3
[KPI] # Website Users Top 100
Baseline Target: 24%

These new website users KPI’s were added as we gained access to Google Analytics data since the development of the proposal. Global GMWG has significantly more budget and resources for content development and as we are not competing with them APWG goal will look to monitor and maintain this baseline presence. It is expected that these KPI’s could change significantly once the new website is live with translated copy that will better serve APAC markets or audiences from countries such as China, Japan and South Korea.

Tactical Objective 4: Identify, develop and retain contributors owls through the use of APAC onboarding community & talent scouting and mentoring initiatives.
4.1
[KPI] # Gold, Silver, Bronze level owls
Baseline: 48%
Target: 45%
4.2
[KPI] Representation of women/non-binary contributors (3% above industry standard 1)
Baseline: 14%
Target: 15%

Removed

The following KPIs listed in the Q4 APWG proposal have been removed

  • [KPI] # New IB BD opportunities identified: removed as duplicate refer strategic relationships established/progressed
  • [KPI] # Event participants: removed because we will instead measure engagement following events based on website user activity
    [KPI] # Impressions while we intend to develop regional content it is unlikely we can effectively measure impression on channels that we do not own, if the content is appropriate for sharing on Twitter we may be able to measure impressions but we do not intend Twitter impressions to be an APWG KPI.
  • Community: outreach focused on crypto native audiences such as young professional and university students. We note here that our focus on onboarding APAC university students signalled support for the development of an internship program, however, this project is undertaken by and sits best with TOC Talent @bradwmorris @fishbiscuit

Q4 Ongoing Edits

4 November: Update Oct actual performance & changes to autonomy KPIs
5 Nov: Reporting Community KPI’s

4 Likes

<3 Awesome to see this challenge being tackled further in depth and with a SME such as yourself!

Growth could certainly piggyback on some of this work for broad application.

3 Likes

APWG keeps killing it - Great to see this information laid out. We are only as strong as our ability to work together across our entire organization. This document really highlights how the work of each different WG feeds into the others and supports the entire organization.

2 Likes

EDITS: 4 Nov 2021 (GMT+13)

The following table was removed from the forum post above with updated performance metrics and October: Actual results

KPI TARGET OBJECTIVE DATA SOURCE TIME
APAC Median Buy Size Baseline Req. Strategic Dune Analytics 30 / 90 Days
APAC Net Buys Baseline Req. Strategic Dune Analytics 30 / 90 Days
APAC % Trade Volume Baseline Req. Strategic Dune Analytics 30 / 90 Days
APAC % N$F Baseline Req. Strategic Finance Nest 30 / 90 Days
IB/BD Relationship 23 IB/BD Notion Month (Mth)
Bi Weekly BD IB Community Call 10 IB/BD Head Count Month (Mth)
Self-Identified Autonomy = 3 or 4 20% IB/BD Survey Quarter (Qtr)
Website Users Top 10 22% GM Google Analytics Mth / Qtr
Website user Top 50 25% GM Google Analytics Mth / Qtr
Website users Top 100 24% GM Google Analytics Mth / Qtr
Gold/Silver/Bronze Owls 45% TOC Discord Dash Mth / Qtr
Women/Non-Binary Contributors 15% TOC Discord Dash Mth / Qtr
Last Update: 31 October 2021

The primary change to the forum post was to self-identified autonomy levels.

  • Information on each of the levels was added to the forum post.
  • KPI used to measure self reported autonomy have changed from a % to the group average score for each of BD IB & GM.

The new KPI’s better reflect APWG opportunities to build context, learn and develop shared understanding. At this capture only 10 people have reported their level of autonomy and it is key to note that our key BD people @Lanks @pujimak_in are not include in this reporting.

2 Likes

This looks very comprehensive. @lee0007!

I am new to forum, is there anywhere to catch-up on a bit more background on these metrics? (N$F etc)

Would love to read more on this. Thanks!

1 Like

I’d recommend talking to the experts by joining a weekly community call for the Analytics Working Group or bi weekly Growth Marketing working group. This conversation is a good read Request for Feedback: Index Coop KPIs v2.0

1 Like