Reaffirming Our Focus (North Star & Core KPIs)

[anon10525910]

Yesterday, I posted 3 Immediate Strategic Needs for the Coop.

The goal of today’s post is to solidify 2 of those needs: North Star Metrics & Core KPIs.

At the end of this post, I also share some thoughts on empowering high-impact contributors, though no concrete proposals. That comes next.

Where are we today

Over the past 3-4 weeks, I’ve been reviewing community materials & having conversations with many of you to learn what you believe we need to focus on.

Through this process, two core points are clear:

1. We are aligned on what fundamentally matters
2. There are a lot of additional metrics that tell important stories

With that in mind, here is what I recommend

1. We reaffirm our commitment to 5 core metrics
2. We enable individual community members & working groups to use more nuanced metrics

Today, I am focusing on #1. I break core metrics into 1) North Star Metrics and 2) Core KPIs. The core question for you to consider is: "Are we still roughly aligned?”

I am confident we are.

North Star Metrics

North Star metrics are supposed to be the measurement that’s most predictive of an organization’s long-term success. It is an exercise in simplifying the overall strategy into terms all can remember, understand, and apply [Learn more: Mixpanel & Grow & Amplitude].

Well-crafted north star metrics have 3 attributes:

1. Measures user value
2. Represents a product strategy
3. Is a leading indicator of revenue

Crafting a single north star metric for the Coop is not easy. Our products already provide value to users in different ways (FLI vs. DPI/CGI).

So, for this first iteration, I recommend we use 2 North Stars to guide us as we learn more:

• $ Total Value Locked
• $ Aggregate Trade Volume

These are the strong metrics to use as North Stars:

• They measure user value
• They represent a product strategy
• They are leading indicators of revenue

These can be then broken down further (i.e. by product & incentivized/unincentivized) to provide more actionable insight.

Core Key Performance Indicators (KPIs)

Core KPIs are used to answer, “How are we doing against our north star?”

We’ve largely circled around a similar set of KPIs since launch. Based on all prior posts & conversations, I recommend we focus on 3 to help us answer, “how are we doing?” against our North Stars.

1. # Total Unit Supply
2. # Total Unique Holders
3. $ Protocol Revenue

Again, these metrics can be broken down further, i.e. by product and incentivized vs. unincentivized.

Next Steps - Core Metrics

Request for feedback

• What might be missing from these core metrics?
• Generally, what is unclear? What questions come up?

Intended next steps

• Ensure we have tracking/tooling for core metrics across current Coop products
• Update the Weekly Planning Deck to focus on these core metrics

Addendum: Empowering High-Impact Contributors

Finally, let’s touch on Strategic Need #3: Empowering high-impact contributors.

As described in Roles, Contributions, and Rewards and October Rewards Distribution & Methodology, not all work gets compensated. These posts do a great job describing how Treasury Rewards ought to work.

It has been clear, though, that Treasury Rewards fail to enable a certain kind of high-autonomy, high-impact work. The Growth Working Group was the first Index Coop-created solution to this problem.

The core questions I’m thinking about when it comes to empowering high-impact contributors include:

1. How do we remain fluid while we learn how to best run the Coop?
2. How do we empower more leaders to step up in response to community needs?
3. Who makes what decisions?
4. How are they held accountable?
5. How do we enable community members to answer: a) What should I work on? b) How do I know my work will be used? c) How do I know my work will make an impact? d) Will I be compensated for my work? How?

One model that continues to pop up as an answer is “working groups.” This model is both validated outside of the Coop and battle-tested within the Coop by the Growth Working Group. Broadly speaking, this structure has answered many of those questions.

Iterating on this structure so that others can take on high-impact/high-autonomy work is one of my focus areas right now.

My intended next step is to share for feedback a rough outline on how we can scale working groups.

Stay tuned

[anon10525910]

TLDR;

North Stars

• $ Total Value Locked
• $ Aggregate Trade Volume

Core KPIs

• # Total Unit Supply
• # Total Unique Holders
• $ Protocol Revenue

[LemonadeAlpha]

I think this is directionally right.

We use DPI unincentivized supply growth as the north star for the GWG, as that is the one number we can look at every week and say number go up = good, number go down = bad.

While its a solid indicator, we don’t use TVL because price appreciation can influence this upwards in the midst of stagnant or even declining growth. We can’t influence the price of the index via underlying assets outside of the methodology (indirect).

Aggregate trade volume is also a solid KPI, but I think it obfuscates the signal we’d be trying to get from a product to some extent. DPI volume would be great but volume absent supply growth is probably suboptimal to target.

I might be misunderstanding but the FLI stands to capture the lion’s share of its revenue from mint/redeem volume and regular trading volume does not book revenue. I’d be interested to hear why TVL and Volume are good indicators for FLI as I haven’t begun to think about the product metrics as deeply yet.

I’d argue that most products should use one of those two (mint volume, supply growth) since they are definitionally leading indicators of revenue while volume is not necessarily indicative of revenue for any of our products and TVL is not necessarily the most indicative of revenue for products like FLI.

Just my two cents, its splitting hairs at some point and can get behind this framework if we want to leave it simple and accessible!

As always, awesome work and thanks for doing this.

[dylan]

As a North Star metric, I think TVL and Volume are simpler to understand than mint volume or supply growth. Example: TVL is used to measure all projects on DeFi Pulse. Unit supply and mint volume may be better suited for Core KPIs.
My 2 cents: the current North Star + KPIs are good enough.

Something to note (that may be obvious): when tracking supply growth we are still subject to price appreciation. Since a substantial quantity of DPI is LP-ing in the ETH/DPI pool, when ETH rallies relative to DPI the pool gets arb’d and more DPI enters* the system.

[DarkForestCapital]

Hey Greg,

This looks great from a product and growth perspective, it’s clear that decisions will now have to be made with these metrics in mind and we can question how each choice helps to reach our goals.

One thing it would be good to clarify is how things that are not quantifiable by these metrics will be treated? For example there is a load of great work on the Index Coop side of things that doesn’t have a direct impact on these, but does ensure we are making the Coop successful and sustainable. Things like:

• Analytics
• Treasury management and reporting
• Investment committee, research and analysis
• Governance and meta-governance
• Design and branding
• Language ops

Clearly we’re not treating the above as having less value to the Coop but they are more difficult to quantify without an immediate measurable. It would be good to understand how you are thinking about these areas of the community and their contribution relative to the KPIs set out above (or not!).

[0x_Dev]

Great post prior to this @anon10525910 and this one too. Thank you for putting together.

While I can see some potential issues with TVL and Volume, I think their pros outweigh the cons. As @dylan says, TVL has become quite enshrined via DFP. I would like to see these and maybe a couple of other metrics displayed live on the homepage above the fold for browsers of our website too: great social proof and a focusing point to refer to.

I think the internal KPIs are solid.

Good points from @DarkForestCapital re less directly linked work re KPIs. Hopefully we can account for the value of good content, good design, and language ops too) - I can think of a few qualitative and quantitative tests we can run there. But, I’d +1 re analytics, community, general infra, treasury management, governance, compliance/legal, and of course engineering etc. I wonder if these areas can also break out into named working groups for leaders who petition the treasury committee (or a wider committee of working group heads) for INDEX budget, which is then allocated as the working group leads see fit each month. This way each working group has a community at the ground level and a leadership on top of it, which can represent its value and explain why the group deserves a % monthly allocation of INDEX.

[verto0912]

Thanks, @anon10525910, for me, these metrics largely make sense.

I would maybe push back against the aggregate trading volume as being a North Star. It sounds rather strange for an index fund manager to have trading volume as something we aspire to.

Another reason aggregate trading volume might be a problem is measuring it across 4-5 and more products. How do we set a target for aggregate trading volume across multiple products? I think this is rather challenging to estimate.

I would propose # total unique holders to be elevated to a north star metric and aggregate volume used as one of the core KPIs.

Another area I don’t see in your post, which I think is important, is liquidity. Having sufficient liquidity certainly measures user value and the utility of our products. If there’s no liquidity, other things don’t quite matter. Would love to get your thoughts on this point.

Look forward to your post on empowering high-impact contributors. It sounds like you are leaning towards working groups as a solution. It would be great to understand the experience of the Growth Working Group, the learnings and why you think that model is effective and can scale.

[Matthew_Graham]

I agree with @LemonadeAlpha - we should not use TVL due to it being heavily influenced by price.

We could look at market share/dominance. So measure how much of the market we are capturing, this gives us a relative performance to peers.

Average trading day volume is good as a measure of liquidity. I don’t see this a driving KPI, but if liquidity is to low, it is a problem as we can not facilitate larger trades via a DPI-ETH pool… To low may indicate we need to introduce/increase incentives.

Mint volume is a great measure and I would suggest Net Supply Growth is better again. So we track the net change in volume of DPI essentially. The more DPI in circulation the better.

We should also have data around how the DPI token is used, is it in a liquidity pool like DPI-ETH on Uniswap trying to be productive (IL versus Fee APY). This is a bit more along the lines of know your client.

DPI per user address distribution is a good measure that tells us a bit about our clients. Ie: lots of small balances, then gas is big topic, but if the majority of holders are whales - they are less worried about gas prices.

I personally think we have KPI for the product itself and then we have a range of metrics for knowing our clients and how DPI is being used. The later helps us make informed decisions and anticipate how are actions will be received by the market.

Revenue/Income is key metric - as there is a fee structure, we need to measure the key drivers that influence revenue. Here Revenue is absolute fees, Income is Index Coop portion of revenue after sharing fees with the methodologist.

I would target either Income or Net Income - Net Income measures Index Coop Income minus associated expenses. Is the product adding value to INDEX holders are we spending more on the product then what we generate from it…

@dylan is there a way to trade the arb transactions and filter these out from the data ?
I think this arb data skews our results with false data. The more volatile price is, the more arb bots act which we see through congestion on the network via gas prices.

[0x_Dev]

Thinking about this further and noting @verto0912 and @Matthew_Graham 's points:

• Volume does like an odd north start metric. E.g. Vanguard wouldn’t put this front and center. Suggest demoting this from North Star roster.
• Unique addresses holding seems like a good one. We can use this as a proxy (kinda) for clients, which is ultimately what matters most. Clients buy the products, hold the value locked, etc. Suggest elevating to North Star roster.
• Unit supply: also suggest elevating this to North Star roster - in combo with TVL, unit supply is a helpful, contextual sidekick.

FWIW, as I think through website stuff more, I’m liking these highest order social proof points more and more:

• No of products
• TVL
• No of unique wallets holding our products

[MrMadila]

Hi guys, just wanting to throw in my thoughts surrounding the use of TVL (aka AUM, aka AUV ) and echo the sentiments of @Matthew_Graham, @LemonadeAlpha and @0x_Dev

Whilst being a popular and “easy to understand at a glance” figure widely used within DeFi, it can be a rather blunt metric and so should be fully understood and contextualised when using.

For illustration: DPI circulating supply could double with the up and coming exchange launches and large external treasury purchases. But if in the same period we witness a 50% market pull back TVL would essentially appear flat despite the hard work and huge success in increasing product demand. Conversely the user base could be suffering (people selling in order to buy… cryptopunks, stonks ?!) yet the market could be pumping and giving a false impression that Index products are in demand.

It is also worth noting here that currently the streaming fees across the products that are considered revenue are not yet crystallised profit/losses (this is being looked at by the treasury team I believe). Again what looks like $15,000 of generated revenue this week @ $370 per DPI could be worth more or less depending on the future price when a decision is made to sell or convert.

As you can see things start to get technical and further decisions into accounting methods need to be considered in a wider context. Which is why my feeling is that TVL would be better used as a KPI than a North Star. And a form of unit supply (aka circulating supply) etc be used instead. At the very least TVL should be immediately accompanied with such.

The last few months have seen exponential growth in the price of many of the tokens within the products (at time of writing!) so it is understandable to be drawn to those nice big ol’ numbers! I hate to be the party pooper but in the periods of market consolidations, volatility and draw downs TVL could prove to be much less palatable number than it has proven to be recently.

TVL is by no means not a useful metric but I believe we should factor in these limitations in our decision marking.

Lastly a big shout out to @anon10525910 and the rest of the contributors for putting in such a huge amount of time, thought and care into this. The quality of ideas and debate here is something we should all feel proud to be a part of.

[Matthew_Graham]

How do we move this forward ??

I think we need a working space to flush this topic out. The pace of the governance forum is too slow and engagement on discord and team meetings is greater. This process needs to be as inclusive as possible.

May I suggest, each working group drafts its own KPIs. We should have overall community goals and each working group should have its own targets that feed into the overall goal. To achieve this I think we need a series of meetings. Let’s start with flushing out Index Coop overall goals. The sooner we do this the better.

Curiously, do we need SETs approval/permission on install these KPIs ?

Would there be KPIs and a road map to transition towards independence such that all community roles are elected by the community and paid solely by Index Coop ? Long Term Index Coop needs to be self governing.

[verto0912]

Great point and something I’d like to echo. What is the consensus/approval mechanism here?

@Matthew_Graham

May I suggest, each working group drafts its own KPIs.

@Matthew_Graham I’m not sure we have any working groups formalised, beyond GWG, which is also coming to an end of its term.

From all the recent discussions, it seems that we might be looking at things in a wrong order. Perhaps figuring out how we actually govern ourselves in a way that is robust, decentralised and empowers the community should be the first priority. KPIs are important but if there’s no governance & org structure to execute on them, then they have less impact.

[MrMadila]

Just echoing @Matthew_Graham and @verto0912. Makes sense to formalise decision making structures before making decisions.

[mrvls_brkfst]

I like @Matthew_Graham 's point about Net Income and would make it a KPI. Overall, the mix selected by @anon10525910 looks good but what about bumping those North Stars down to KPIs and having # Total Unique Holders as the only North Star, our Pole Star? …community and everything else follows. That way, we strive to meet the KPI but always in the context of growing distribution of DPI, CGI, FLI, INDEX, and all that follow.

[anon10525910]

[Quick note to acknowledge that there has been a lot of great feedback here!!* I will be reviewing it all & synthesizing a.s.a.p. so that we can keep the convo moving forward together ]

[anon10525910]

Allllllrighty folks, there is a lot here and I’m doing my best to synthesize this in a way that makes it actionable. I know I am not capturing everyone’s nuance, please bear with me.

Here is the TLDR; that I think I’m understanding

1. $ TVL is a crude metric, let’s deprecate it
2. $ Trading Volume is a bit odd, let’s deprecate it
3. # Holders seems great, let’s elevate it
4. # Unit Supply Growth & Mint Volume are more fine-tuned metrics that measure what really matters. Got plus ones.
5. $ Income getting plus ones.

Other metrics discussed: Liquidity, market dominance, average trading day volume, net supply growth, DPI token usage, DPI per user address distribution

[MrMadila]

+1 for Market dominance (Market capture?) and Net supply growth as KPI’s

[jdcook]

+1 on elevating # Holders . We build index products to give specific market segments (users) a product that they love. We must always remain true to the “users”. While it is harder in DeFi and Web 3.0 to identify your users and understand them (compared to TradFi and Web 2.0), that should not stop us from keeping the users of our products as our North Star, imo.

[anon10525910]

Back for take 2 This time I’m hitting specific questions & feedback.

To set the table:

• This is an evolving conversation we’re having together
• Our principles are front & center for me throughout it.
• I am sharing thoughts while acknowledging that this doesn’t capture my full thinking or hit all nuance.

I’m excited to pick up these convos in the coming days!

@DarkForestCapital

One thing it would be good to clarify is how things that are not quantifiable by these metrics will be treated? For example there is a load of great work on the Index Coop side of things that doesn’t have a direct impact on these, but does ensure we are making the Coop successful and sustainable.

I am definitely thinking about alllll that Personally, I view everything you laid out (Analytics → Language ops) as directly impacting our overall success, growth and sustainability. They are critical pieces of the puzzle.

Happy to go deeper, but that is the TLDR; Everything you laid out is directly impacting today and will directly impact tomorrow and to infinity (cc @0x_Dev, i know you had a similar question!)

@verto0912

Another area I don’t see in your post, which I think is important, is liquidity. Having sufficient liquidity certainly measures user value and the utility of our products. If there’s no liquidity, other things don’t quite matter. Would love to get your thoughts on this point.

Interesting, I hadn’t thought about liquidity that seems more like an input than an output to me. For example, we could juice up liquidity with Liquidity Mining, and that wouldn’t necessarily indicate Index Coop success. Would love to hear how you think about it differently!

@verto0912

It would be great to understand the experience of the Growth Working Group, the learnings and why you think that model is effective and can scale.

100% - excitied to pick up this convo where we left off on today’s org call!

@Matthew_Graham

Let’s start with flushing out Index Coop overall goals. The sooner we do this the better.

amen to this

@Matthew_Graham

This process needs to be as inclusive as possible.

amen to this too! Posting in the forum like this was step #1 to get convo going out in the open so that all can participate. I’m excited to continue this convo publicly & together in the forums, on calls, and in discord.

@MrMadila

Makes sense to formalise decision making structures before making decisions.

I hear you (& @verto0912 & @Matthew_Graham, and i know many others as well) and understand how important this is.

This specific post, and ensuing conversation, is intentionally narrow.

It is driving towards rough alignment on the core metrics that we believe are helpful for coordinating our efforts today – rather than a set in stone decision or single solution for the larger org. challenges we all feel and started to talk about today

Based on all the responses here, we’ve made solid progress!!

[dylan]

Yeah, I think given the pool size + price changes in DPI/ETH + assumption the pool is always basically balanced we should be able to correct for the arb transactions from total supply