This looks great from a product and growth perspective, it’s clear that decisions will now have to be made with these metrics in mind and we can question how each choice helps to reach our goals.
One thing it would be good to clarify is how things that are not quantifiable by these metrics will be treated? For example there is a load of great work on the Index Coop side of things that doesn’t have a direct impact on these, but does ensure we are making the Coop successful and sustainable. Things like:
Treasury management and reporting
Investment committee, research and analysis
Governance and meta-governance
Design and branding
Clearly we’re not treating the above as having less value to the Coop but they are more difficult to quantify without an immediate measurable. It would be good to understand how you are thinking about these areas of the community and their contribution relative to the KPIs set out above (or not!).
Great post prior to this @gregdocter and this one too. Thank you for putting together.
While I can see some potential issues with TVL and Volume, I think their pros outweigh the cons. As @dylan says, TVL has become quite enshrined via DFP. I would like to see these and maybe a couple of other metrics displayed live on the homepage above the fold for browsers of our website too: great social proof and a focusing point to refer to.
I think the internal KPIs are solid.
Good points from @DarkForestCapital re less directly linked work re KPIs. Hopefully we can account for the value of good content, good design, and language ops too) - I can think of a few qualitative and quantitative tests we can run there. But, I’d +1 re analytics, community, general infra, treasury management, governance, compliance/legal, and of course engineering etc. I wonder if these areas can also break out into named working groups for leaders who petition the treasury committee (or a wider committee of working group heads) for INDEX budget, which is then allocated as the working group leads see fit each month. This way each working group has a community at the ground level and a leadership on top of it, which can represent its value and explain why the group deserves a % monthly allocation of INDEX.
Thanks, @gregdocter, for me, these metrics largely make sense.
I would maybe push back against the aggregate trading volume as being a North Star. It sounds rather strange for an index fund manager to have trading volume as something we aspire to.
Another reason aggregate trading volume might be a problem is measuring it across 4-5 and more products. How do we set a target for aggregate trading volume across multiple products? I think this is rather challenging to estimate.
I would propose # total unique holders to be elevated to a north star metric and aggregate volume used as one of the core KPIs.
Another area I don’t see in your post, which I think is important, is liquidity. Having sufficient liquidity certainly measures user value and the utility of our products. If there’s no liquidity, other things don’t quite matter. Would love to get your thoughts on this point.
Look forward to your post on empowering high-impact contributors. It sounds like you are leaning towards working groups as a solution. It would be great to understand the experience of the Growth Working Group, the learnings and why you think that model is effective and can scale.
Average trading day volume is good as a measure of liquidity. I don’t see this a driving KPI, but if liquidity is to low, it is a problem as we can not facilitate larger trades via a DPI-ETH pool… To low may indicate we need to introduce/increase incentives.
Mint volume is a great measure and I would suggest Net Supply Growth is better again. So we track the net change in volume of DPI essentially. The more DPI in circulation the better.
We should also have data around how the DPI token is used, is it in a liquidity pool like DPI-ETH on Uniswap trying to be productive (IL versus Fee APY). This is a bit more along the lines of know your client.
DPI per user address distribution is a good measure that tells us a bit about our clients. Ie: lots of small balances, then gas is big topic, but if the majority of holders are whales - they are less worried about gas prices.
I personally think we have KPI for the product itself and then we have a range of metrics for knowing our clients and how DPI is being used. The later helps us make informed decisions and anticipate how are actions will be received by the market.
Revenue/Income is key metric - as there is a fee structure, we need to measure the key drivers that influence revenue. Here Revenue is absolute fees, Income is Index Coop portion of revenue after sharing fees with the methodologist.
I would target either Income or Net Income - Net Income measures Index Coop Income minus associated expenses. Is the product adding value to INDEX holders are we spending more on the product then what we generate from it…
@dylan is there a way to trade the arb transactions and filter these out from the data ?
I think this arb data skews our results with false data. The more volatile price is, the more arb bots act which we see through congestion on the network via gas prices.
Volume does like an odd north start metric. E.g. Vanguard wouldn’t put this front and center. Suggest demoting this from North Star roster.
Unique addresses holding seems like a good one. We can use this as a proxy (kinda) for clients, which is ultimately what matters most. Clients buy the products, hold the value locked, etc. Suggest elevating to North Star roster.
Unit supply: also suggest elevating this to North Star roster - in combo with TVL, unit supply is a helpful, contextual sidekick.
FWIW, as I think through website stuff more, I’m liking these highest order social proof points more and more:
Whilst being a popular and “easy to understand at a glance” figure widely used within DeFi, it can be a rather blunt metric and so should be fully understood and contextualised when using.
For illustration: DPI circulating supply could double with the up and coming exchange launches and large external treasury purchases. But if in the same period we witness a 50% market pull back TVL would essentially appear flat despite the hard work and huge success in increasing product demand. Conversely the user base could be suffering (people selling in order to buy… cryptopunks, stonks ?!) yet the market could be pumping and giving a false impression that Index products are in demand.
It is also worth noting here that currently the streaming fees across the products that are considered revenue are not yet crystallised profit/losses (this is being looked at by the treasury team I believe). Again what looks like $15,000 of generated revenue this week @ $370 per DPI could be worth more or less depending on the future price when a decision is made to sell or convert.
As you can see things start to get technical and further decisions into accounting methods need to be considered in a wider context. Which is why my feeling is that TVL would be better used as a KPI than a North Star. And a form of unit supply (aka circulating supply) etc be used instead. At the very least TVL should be immediately accompanied with such.
The last few months have seen exponential growth in the price of many of the tokens within the products (at time of writing!) so it is understandable to be drawn to those nice big ol’ numbers! I hate to be the party pooper but in the periods of market consolidations, volatility and draw downs TVL could prove to be much less palatable number than it has proven to be recently.
TVL is by no means not a useful metric but I believe we should factor in these limitations in our decision marking.
Lastly a big shout out to @gregdocter and the rest of the contributors for putting in such a huge amount of time, thought and care into this. The quality of ideas and debate here is something we should all feel proud to be a part of.
I think we need a working space to flush this topic out. The pace of the governance forum is too slow and engagement on discord and team meetings is greater. This process needs to be as inclusive as possible.
May I suggest, each working group drafts its own KPIs. We should have overall community goals and each working group should have its own targets that feed into the overall goal. To achieve this I think we need a series of meetings. Let’s start with flushing out Index Coop overall goals. The sooner we do this the better.
Curiously, do we need SETs approval/permission on install these KPIs ?
Would there be KPIs and a road map to transition towards independence such that all community roles are elected by the community and paid solely by Index Coop ? Long Term Index Coop needs to be self governing.
Great point and something I’d like to echo. What is the consensus/approval mechanism here?
@Matthew_Graham I’m not sure we have any working groups formalised, beyond GWG, which is also coming to an end of its term.
From all the recent discussions, it seems that we might be looking at things in a wrong order. Perhaps figuring out how we actually govern ourselves in a way that is robust, decentralised and empowers the community should be the first priority. KPIs are important but if there’s no governance & org structure to execute on them, then they have less impact.
I like @Matthew_Graham 's point about Net Income and would make it a KPI. Overall, the mix selected by @gregdocter looks good but what about bumping those North Stars down to KPIs and having # Total Unique Holders as the only North Star, our Pole Star? …community and everything else follows. That way, we strive to meet the KPI but always in the context of growing distribution of DPI, CGI, FLI, INDEX, and all that follow.
+1 on elevating # Holders . We build index products to give specific market segments (users) a product that they love. We must always remain true to the “users”. While it is harder in DeFi and Web 3.0 to identify your users and understand them (compared to TradFi and Web 2.0), that should not stop us from keeping the users of our products as our North Star, imo.
I am sharing thoughts while acknowledging that this doesn’t capture my full thinking or hit all nuance.
I’m excited to pick up these convos in the coming days!
I am definitely thinking about alllll that Personally, I view everything you laid out (Analytics → Language ops) as directly impacting our overall success, growth and sustainability. They are critical pieces of the puzzle.
Happy to go deeper, but that is the TLDR; Everything you laid out is directly impacting today and will directly impact tomorrow and to infinity (cc @DevOnDeFi, i know you had a similar question!)
Interesting, I hadn’t thought about liquidity that seems more like an input than an output to me. For example, we could juice up liquidity with Liquidity Mining, and that wouldn’t necessarily indicate Index Coop success. Would love to hear how you think about it differently!
100% - excitied to pick up this convo where we left off on today’s org call!
amen to this
amen to this too! Posting in the forum like this was step #1 to get convo going out in the open so that all can participate. I’m excited to continue this convo publicly & together in the forums, on calls, and in discord.
This specific post, and ensuing conversation, is intentionally narrow.
It is driving towards rough alignment on the core metrics that we believe are helpful for coordinating our efforts today – rather than a set in stone decision or single solution for the larger org. challenges we all feel and started to talk about today
Based on all the responses here, we’ve made solid progress!!
Just catching up on this convo. Agree with most of what has been written.
+1 for # Holders as North Star. Essentially we are selling index products, hence customer (aka user) numbers should be a priority
+1 for # Unit Supply Growth & Mint Volume Again, high number of units sold/minted implies high demand aka happy customers
Tbh I’m struggling a bit to grasp what exactly differentiates NorthStar from KPIs. I assume it’s sort of long term strategic goals vs. tactical “how-to-get-there” metrics.
Anyway, I’d see trading volume and liquidity as KPIs, as they determine how much we can sell (or how big of a trade). Plus I liked the store analogy used a few times before, in that it should be our goal to place our products in as many “stores” (aka exchanges) as possible. Essentially this comes back to product availability.
Things like revenue/income are just an outcome of our efforts, they’ll come naturally if we successfully achieve our north star and KPIs.
Interesting that nobody mentioned $INDEX price appreciation as KPI. This shows how product focused we are!
Regarding TVL, I’m also not a big fan of using this metric, as it can be misleading when looking at it isolated.
Fantastic read and amazing value content. I was thinking whether there is any target for said working groups in place to achieve on a regular basis. We can use SWOT analysis for different products and Index Coop as a whole on quarterly basis to see where we are and “what” needed to be done.