Engineering and Product Constraints

TL;DR

  • Currently, the Index Coop is constrained by its engineering resources - we cannot simultaneously launch and operate new products, as well as improve existing ones.
  • Black Wednesday made clear that the existing highest priority for engineering is to improve the safety, profitability, and robustness of our existing product base [namely, the FLI suite].
  • This means that new product launches need to be pushed back and timelines readjusted.
  • The product and engineering teams are working on setting new timelines with methodologists, developing systems to proactively inform stakeholders, and creating risk models to improve FLI operations.

Situation

Index Coop now has had a few massively successful products. We have accomplished much over these past 6 months and should be proud of where we are today. This success has led to tremendous AUM growth, brand recognition, and community strength.

With this increasing success, we have more individuals, both external methodologists, and internal community members, that are interested in launching more products with us. It’s clear that launching new products is very important and definitely something we need to keep doing.

At the same time, it is critical to maintain and improve our existing products. Black Wednesday exposed some severe technical and non-technical weaknesses that have cost some of our customers millions of dollars. On top of that, we have seen the profitability of our products significantly decline, with periods of high volatility often causing $50K+ in operating expenses per day. If we don’t maintain and improve our core product base, we risk building the future of our organization on shaky ground and exposing ourselves to brand-destroying, catastrophic events.

Currently, the bottleneck is our engineering resources - we do not have enough engineering resources to simultaneously launch and operate new products, as well as improve existing ones. Launching new products right now risks the development team getting stuck putting out fires all day on a large suite of products, without having time to invest in longer-term, high-impact, improvements.

It has become a priority among the product and development teams to shift focus from fielding new products, launching new products, and operating them to focusing on making deep improvements to our existing products that increase their profitability, safety, and robustness.

Complication

There are products that we have committed to launching with high-profile methodologists that we are incredibly enthusiastic about launching. Additionally, there are several other products in the pipeline that the community is very excited about that we will likely not be able to launch in the near term.

Some to highlight are the Stable Yield Index and the Synthetix Debt Mirror Index. In order to increase the shared knowledge base within the community, we will go over some of the existing blockers with each of these products.

Stable Yield Index:

  • Requires integrations to several protocols to access the yield those contain.
  • Such integrations take significant development effort that we don’t have yet.
  • Rebalancing assets that are currently deposited in other products [ex. USDC in a Yearn Vault] is operationally complex as it requires pulling the asset out of the protocol, rebalancing, and then depositing back into the protocol.
  • Currently that wrapping/unwrapping process is very centralized and requires significantly more operational effort as we have to do a multisig transaction for each wrap/unwrap.
  • Smart contract multi-sig interactions are complex to execute and require incredible precision.
  • With 9 assets, that would be 18 multi-sig transactions for each rebalance which, by itself, would be hours of time for our engineering team.

Synthetix Debt Mirror:

  • The main issue with the Synthetix Debt Mirror is that it requires a weekly rebalancing pace.
  • Similar to SYI, the way we make money from the SMI is by farming it on other protocols and extracting those revenues.
  • Thus it faces the similar operational process described above.
  • Furthermore, that process now occurs at a weekly pace instead of a monthly pace which also massively increases the amount of operational overhead.

Our relationship with methodologists, both internal and external, is critical to our success and thus such situations need to be managed with extreme empathy and care.

Question

How can we:

  • Give our engineering team the space to work on mission-critical, high effort, and longer-term product improvements.
  • Maintain our relationships with the top methodologists that have passed DG2 or are in the Product Onboarding Process.
  • Set the proper expectations with methodologists proposing new products that we are excited about and want to do eventually.

Answer

Giving our engineering team space:

  • We need to protect our smart contract engineering resources and make sure they have the time and energy to focus on long-term, high-leverage product improvements.
  • This means:
    • Halting the launch of new products till we get our existing products to be more profitable, safe, and robust.
    • Communicating with existing methodologists clearly our existing problems, our reprioritization, and a new timeline for launch.
    • Building the capacity within the Coop to handle more product operations like rebalancing [Dylan with the engineering working group] and maintenance [FLI team] so our smart contracts team can focus on building.

Maintaining our relationship with existing methodologists

  • Try to scope down requirements as much as possible:
    • For example, if with the SMI, we can agree with a monthly rebalance to start with, and only farming half the assets on Compound [an existing Set Protocol integration], we have dramatically reduced the launch and operational costs to something the Coop has developed the capacity to take care of independently.
    • That would be something we could support today.
    • In the future, we can invest more into developing the product to make it fully featured.
  • Empathize with our methodologists, communicate our situation, and provide a clear guideline for when they could expect a product launch.

Expectation setting for new products:

  • Though we can not immediately launch new products today, that does not mean we need to switch off our product onboarding process.
  • We should still form relationships with methodologists, develop the expertise within the Index Coop to be able to vet and spec out new products and guide them through our governance process.
  • The key is setting expectations early on in the process so there are no misalignments that lead to frustration later down the line.

Next Steps

Most internal methodologists going through the product onboarding process are aware of the existing resource constraints and new timelines. Product leaders at the Index Coop will have calls with external methodologists over the following weeks to explain our situation and preserve our relationship.

Dylan and the Engineering Working Group are gaining more proficiency in managing operational processes like rebalances to significantly reduce the workload on the smart contract team.

The FLI Team within the Product Working Group is collaborating with the Smart Contract team to manage the parameters for the FLI and building a risk model that makes managing the parameters more rigorous. As community stakeholders, you can expect transparency and more information on such models as we develop them. Along with maintaining the parameters of the FLI suite, we will also be working on educational content to make sure our investors are fully informed about the product.

There is plenty of work to do outside of launching new products. Outside of maintaining and operating existing products, the Product Working Group has realized we have significant weaknesses in stakeholder management and will be focusing on how to improve there. We need to empathize deeply with our investors, methodologists, community, and team members to make sure we are all fully informed and in sync. Hopefully, this forum post serves as the first step in that direction.

17 Likes

Love the details and transparency in this post! I also think this is exactly the right approach to take. We’ve hit product-market fit for a few key products that have driven amazing growth for the Coop. Focusing on scalability (incl stakeholder management and reducing tech debt) is the natural next step for many startups in this phase, and will set us up for even more growth in the future.

One note – I’m surprised not to see an effort towards expanding engineering resources, given that resource constraints are driving some of this issue. Curious if you can shed more light there!

6 Likes

I think @puniaviision has focused his post on the chanlleges to product and our external partners, there is absolutely and effort to look at growing engineering resource:

  • Coop members (@tgreco @emault @afromac and @allan.g ) are rapidly coming up to speed on many of the (none solidity) aspects of the FLI products.
  • @dylan is onboarding more coop contributors to support different areas of the coops Engineering activities.
  • we have approached third party groups to manage the full process of coding a verification of some desired features within the Set protocol.
  • Set labs are advertising to recruit https://twitter.com/jobsincrypto/status/1381426848366747648?s=20
  • I think we are discussing more engineering bounties / grants to target specific activities.
8 Likes

Oh awesome! This is great, thanks for the detailed response @overanalyser :slight_smile:

@overanalyser @puniaviision , I’ve been looking to get involved with some dev work in addition to the FLI experts. I think a quick way we could offload some work from the dev team is to transition the FLI parameter updates to me and/or @tgreco if he’s up for it.

That would streamline and speed up FLI maintenance and clear up at least one or two tasks for the current dev team.

That would require accessing the Index Coop multi-sig to do the parameter update transactions, which we don’t have the infra to delegate right now.

Outside of the FLI, you may find it useful to participate in the Engineering Working Group calls!

1 Like

yea! I think this post by Punia is the opening salvo of starting to get the entire Coop hive-mind on the necessary effort to expand our building (eng. & pm) abilities.

and much more can/needs to be done.

Some rough/quick questions bouncing around my brain:

  • How can we leverage Gitcoin?
  • How can we start sharing more, publicly about what is being built?
  • What high-leverage engineering bounties can we put together? How do we spread the word about those bounties to various engineering communities?
  • What resources can we look to for inspiration/guidance? Working in Public (book), Stripe’s Documentation, Yearn’s gitbook, Yearn’s Project Board , Chainlink docs, leaders at Web2 open source companies, etc…
  • How do we help Web2 engineers learn Solidity? This tweet points to some resources
5 Likes

@puniaviision Thank for bringing these issues to the community in a very detailed and thoughtful post. I’ve been thinking it over the past few days along with @overanalyser’s INDEX.labs post.

I am going to ask some pointed questions and make some tough comments so I want to first acknowledge that I completely understand, emphathize, and agree with you that Index Coop is currently constrained by its Engineering resources.

I want to be a part of solving these product constraints and have a lot of questions and thoughts for clarification on your post that I would love to further discuss in the Product Working Group meeting tomorrow.

1. How can we transfer product operational responsibilities from Engineering to other teams and contributors?

I understand why this is the case for product like the Stable Yield Index (SYI) or Instrinsic Productivity for DPI as these require engineering/development effort.

It is very unclear to me why the Synthetix Debt Mirror (SMI) or sectoral indices like mine and Kiba’s proposal for a Data Economy Index (DATA) are constrained by Engineering resources.

  • Why do we need Engineering resources to launch a sectoral index like DATA or Category Winners (CWI)?**

  • Is rebalancing of the TokenSets the only Engineering resource constraint for launching and maintaining DATA and CWI? What are the other Engineering constraints for sectoral indices like DPI?**

  • @dylan Could methodologists and the Product Working Group takeover many of these responsibilities from Set Engineering or the Engineering Working Group?

2. Concerns about the Index Methodologist Program and the Product Launch Moratorium:

As an Index Coop contributor, INDEX investor, and aspiring methodologist, this is both incredibly disappointing and frustrating. With only a single external methodologist and an undefined moratorium on new product launches, it is abundantly clear to me that the Index Methodologist Program is not working as intended.

As an aspiring external methodologist, I am extremely concerned that I will not be able to participate in the Methodologist Program by building, launching, and maintaining new sectoral indices like the Data Economy Index (DATA) with the Index Cooperative.

To be clear - I am not criticizing you, Set Protocol, DeFi Pulse, or anybody else at the Index Coop. We have a fantastic community and I want to be a part of the solution so we can start launching new products again as quickly as possible.

I look forward to continuing the conversation to find a solution in the PWG meeting!

5 Likes

I can’t stress this enough, 18 months is shortsighted and completely misses the problem it is supposed to solve. The number one thing that drew me to this community was the long-term thinking and only using 18 months to find the best methodologists in the world is mind-boggling.

This problem got even worse after the Set team decided to pull out all the possible engineering resources from the coop (as set by the Engineering WG Q2 OKR’s) before that 18 month period even ended.

2 Likes

Following up on my post. I spoke with @dylan and @puniaviision about the post and my comment on Discord.

Here are @dylan’s comments from our Discord thread:

and…

@dylan @richard @puniaviision I would happily help kick off the expert groups and to start the process of responsibly transitioning ownership of product/index rebalances to the community.

1 Like

the Set team decided to pull out all the possible engineering resources from the coop

That is not at all an accurate description of the situation. Improvements to Index Coop products & operations continue to make up the vast majority of Set Labs’ internal engineering roadmap

3 Likes

I made sure to include the word possible to acknowledge that. Removing every resource at this point is still unfeasible.

Thank you for clarifying.

I used the Ethereum Docs and CryptoZombies to get my head around the concepts and figure out what questions to ask. This helped me to write my first contract.

We’re also working on documentation for our repos that will make onboarding a bit easier.

EWG has had discussions on related content creation

I think Richard has a bounty out for a couple important ones on the bounty spreadsheet

3 Likes

One of the ways we might address this problem is establishing a partnership with Raid Guild to work on new products. If we were able to get work into their pipeline we would have the confidence they are in progress and being handled as proper projects with Index Coop serving as the client.

Raid Guild is a DAO of crypto OGs and have a network with many of the roles we need support with.

Although they are also experiencing more demand than supply right now they are focused on expanding internal resources and working on ways to prioritize product work. Like staking to get projects moved up in queue.

I am currently an Apprentice with Raid Guild and getting up to speed with the CDWG at Index Coop. My passion is product development process and would be willing to push this along if it is an option we would like to explore.

3 Likes

hi @puniaviision

Thank you taking the time to write the post and for shedding some light on the engineering work load. Reading over the post, something I am failing to grasp here is timeline and specific tasks being worked on.

I understand there is work being done documenting a lot of the rebalancing process to enable new contributors via PWG to help out. This is great, but it was presented in the comments.

I struggle to comprehend based on current Engineering capacity how many hours we spend on occurring tasks. For example, what is engineering resource utilisation for occurring tasks like maintenance - DPI rebalancing is x days per month and not expanding the functionality of on a product that was brought to market knowing it required ongoing development.

For example, what specific engineering tasks relating to FLI are being worked on ?
Is TokenSets integrating functionality like flash loans, something not available on Compound but is on other protocols not integrated into TokenSets.

What I am trying to communicate, is I understand the pitched narrative, but knowing Set Labs was working on products in private (cough FLI) without sharing them with Index Coop - I want deeper insight into what specific things are being worked on.

Naturally, this is a huge blind spot for Index Coop and the DAO community is reliant on external Set Labs communications for updates.

7 Likes

Would it be okay to dig into this question a little bit?

At a thematic level, the smart contract engineering team [AFAIK] right now is concerned with infrastructure level upgrades to the FLI suite, documentation to be able to onboard the community, and any quick wins around automation.

Struggling to understand the underlying question/concern. Having our SC team regularly report their hours for the things they are working on would be significant additional overhead. Wondering if there are other things we could do here to build up trust.

Who is head of the Engineering Working Group for the Coop? Index Coop Org Chart

Pretty confident @dylan leads the EWG, but I’ll let him confirm.

If you haven’t voted on IIP-49 yet, take 1 minute to cast your vote here :point_right: Snapshot

IIP-49: DPI MetaGoverance Vote Delegation to Multisig will free up hours each week of Set engineering resources to focus on improvements that impact Coop produts.

Deadline: ~4 hours from now (11:00 am PT / 6:00 pm UTC)
INDEX needed: ~6,000 INDEX to reach quorum
Current Status: 100% FOR

3 Likes

@gregdocter Is there a easy reference to know what number of INDEX to reach quorum is? Did this reach quorum?

I voted YES on this a few days ago and in hindsight should have done more campaigning to get it to pass.

3 Likes