Developer Activity as Signal

  • Commentary
  • September 20, 2020

report published by Delphi Digital, an independent crypto research firm, finds that, for low market cap projects, high developer activity correlates with token price growth; “Low market cap projects which went onto achieve greater than 10x price growth had on average 2.5 times more daily active developers and 3.5 times more daily commits than projects with negative returns.” The report suggests that measures of developer activity, such as daily commits or Delphi’s new ‘DMR’ metric (a ratio of normalized developer activity to market cap), could serve as an initial filter useful for identifying signal among early stage projects.

Code quality is one of the challenges in rigidly defining measures of developer activity, and illustrates a limitation with the metric. Measures of commits and lines-of-code’s frequency and quantity are not necessarily indicative of such changes’ quality: one commit might contain thousands of lines-of-code or a single character, and an efficient developer might achieve in fewer lines-of-code that which takes a less-efficient developer more. A more representative picture of developer activity would supplement information about commits with an assessment of their quality— one would want to know when comparable frequencies of developer activity were of different value added. Yet comparing the respective quality of two projects’ sets of commits is time consuming, and presupposes expertise in the programming languages and architecture (what were the constraints and were there more efficient alternatives?), as well as a solid understanding of a project’s goals (is this a bug v. feature?).

Software forks also pose tradeoffs for metrics. On the one hand, forked projects can add new features, bug fixes, etc by selectively adopting commits from the parent chain—such activity may add value to the project even if the project’s own ‘developer community’ is not ultimately responsible for the work. On the other hand, a project’s deficiency in active development could be masked by ‘double counting’ commits made to the parent repository as progress made by the project. Users of developer metrics should recognize that forked project’s activity may be artificially inflated or deflated depending on how this issue is treated.

Overall, Delphi’s research provides reason to revisit developer activity as a metric, and goes further than other groups in offering a quantified basis for its utility. While developer activity may be useful as an initial and minimal filter, like most other tools, it is best deployed judiciously, both in the appropriate contexts and with an understanding of its limitations. Readers also should note that publicly surfacing the metric could change its predictive utility going forward, given the ease with which projects could inflate measures of development activity in absence of simple methods for evaluating its quality.