Back to top

Arcs of involvement in Drupal (or open source)

I've been working with Drupal since 4.6 and was somewhat involved in the process of making 4.7. That was about 8 years ago. Here are some truths I've learned in that time.

There is at least one arc to people's involvement: people go up and people go down

Think about your involvement in Drupal. Did it start at some level, stay at that level, and continue at that same level indefinitely? Probably no. You are somewhere on an arc of involvement. You may be on a mini-arc within a bigger arc. When you first heard of Drupal you didn't immediately jump in and file bug reports and help do usability studies and write documentation. You tried it out. Then you did a little contribution and you liked it (or hated it) but your involvement probably kept increasing for a while. Right now maybe you're headed up or down - maybe you're involvement is decreasing right now because of a big project at work but you know it will increase 2 months from now because your next project will make it easy to contribute to a given module (or core?). Let's look at some real world arcs of involvement.

Let's look just at Drupal Core involvement. I've used the data from marvil07 which includes analysis on commits to the main branch of core (originally CVS and then migrated to git) since 2000. Here are the top committers for the years 2000 to 2003, inclusive:

Person Commits
Dries 2341
Kjartan 334
Steven 157
jeroen 102
natrak (pseudonym of Kjartan) 82
ax 9
Jeremy 9
Marco 6
Kristjan 4
moshe weitzman 4
Al 3
Gerhard 2
Gabor Hojtsy 2

Very interesting! I only recognize 8 of those 13 people, but they are the foundations of Drupal for the first quarter of its life. Let's look at the next 3 years: 2004 to 2007.

Person Commits
Dries 3537
Steven 1048
Gabor Hojtsy 894
drumm 655
killes 557
chx 331
webchick 124
pwolanin 93
Neil (pseudonym of drumm?) 82
moshe weitzman 69
webernet 69
Gerhard 67
Kjartan 65
JonBob 59

So, looking at the 2004 to 2007 time frame we see a few interesting new faces. Gabor has shot up in involvement. There are some brand new faces (drumm, chx, webchick, pwolanin, JonBob) who seem to be doing a lot of work. Moshe, Gerhard (killes), and Kjartan are still doing a lot of work. And lots of people who were essential contributors from 2000 to 2003 are no longer in the top list.

Now, let's take a look at the top contributors from 2008 and see what has happened.

Name 2008 2009 2010 2011 2012 2013
Dries 897 1446 1482 670 506 309
webchick 251 1014 1042 726 1093 909
catch 101 187 122 304 761 529
sun 23 205 241 83 350 94
Gabor Hojtsy 404 204 175 62 51 3
jhodgdon 0 106 197 97 448 236
Damien Tournoud 139 129 164 47 32 12
chx 103 108 132 70 103 79
yched 11 159 113 29 50 77
David_Rothstein 18 72 170 56 173 153
Dave Reid 70 111 77 31 45 9
andypost 0 44 100 21 61 98
pwolanin 140 67 70 23 16 55
Berdir 0 65 71 19 134 158
c960657 21 81 58 13 21 2
Crell 33 51 37 46 302 43
dww 31 68 46 18 11 3
aspilicious 0 0 84 43 200 23
effulgentsia 0 34 80 13 165 84
mr.baileys 0 30 70 25 11 12
plach 0 23 56 30 32 29
xjm 0 0 0 106 226 101
casey 0 7 91 7 4 2
Greg Dunlap 0 0 0 104 130 0
JohnAlbin 8 50 36 13 12 9
moshe weitzman 25 50 36 4 16 3
scor 16 19 58 11 16 18
boombatower 66 55 21 12 7 1
pillarsdotnet 0 0 3 84 26 5
Bojhan 1 45 32 6 15 23
yoroy 2 31 40 12 7 1
Jeff Burnz 0 3 64 13 1 0
quicksketch 17 48 20 11 11 28
bfroehle 0 0 11 66 20 3
Rob Loach 25 34 19 21 106 25
mgifford 0 22 40 11 22 34
rfay 0 16 40 14 4 0
fago 4 12 25 32 30 22
droplet 0 0 7 61 97 30
bleen18 0 13 41 13 15 3
mfb 24 36 19 10 8 2
bjaspan 10 47 14 0 1 0
ksenzee 2 15 38 7 14 0
drewish 28 41 8 11 9 0
Jacine 0 2 41 14 7 3
agentrickard 9 13 28 16 7 5
drumm 121 52 3 0 11 2
beejeebus 17 17 22 15 37 19
carlos8f 0 7 38 8 2 0
arianek 0 49 2 0 1 0
tstoeckler 0 3 30 16 30 31
jbrown 0 2 23 22 6 6
brianV 0 35 8 3 4 6
marcingy 2 6 22 18 31 17
JamesAn 0 42 3 0 3 0
Jody Lynn 2 21 10 13 6 2
dmitrig01 21 14 21 9 0 0
Heine 14 11 21 11 13 3
TR 0 0 4 39 8 1
Josh Waihi 2 26 14 2 2 2
cwgordon7 24 24 17 1 0 0

Do you see some interesting patterns in there? I think I do. Over the course of years we can see that people spark up and then fall off. Look at drumm, who was key in the 2004 to 2007 time period. His involvement in core has steadily fallen since 2008. JamesAn was pretty busy in 2009 and has done a lot less in core since then. I'm not picking on these people to criticize, just pointing out a pattern. It happens. It's OK. That last table was based on the ongoing behavior of the top contributors from 2008. Let's look at the list of top contributors in 2013:

Person Commits
alexpott 1327
webchick 909
catch 529
dawehner 391
tim.plunkett 347
Dries 309
jhodgdon 236
damiankloip 214
Berdir 158
David_Rothstein 153
swentel 106
Wim Leers 105
xjm 101

Compare the 2013 list to the 2008-biased list. Alexpott, dawehner (2 commits in 2008), tim.plunkett, damiankloip, swentel, Wim Leers are either not present or barely there in the list from 2008. That's right: 6 of the top 13 contributors in 2013 were not in the top of the list from 2008.

As Emma Jane Westby said in her Reddit "ask me anything":

Lead, Follow, or Step aside. It's a question that I have to force myself to answer on a pretty regular basis. "Am I leading effectively? Am I happy to follow someone else's lead? Or am I trolling, and is it time to step aside?" OSS communities are so tight-knit that it makes it really difficult for someone prominent to "quit" their community.

My conclusion from this is that people come and go. And in some cases, they come back and go again and...you get the idea.

It's nice when we can rely on people, but it's naive to think they will always be there always doing the same amount of amazing work in the same way. Does it mean that Drupal is dying because some people have left? Hardly. It is part of a normal cycle.

Why do people get hyper involved?

I think it's fair to say that most people in these lists are "hyper involved." Dave Reid has said that a big part of his involvement (core involvement peaked in 2009) was because he was learning Drupal which then turned into a job. I know that some of my biggest involvement was possible when I was purposefully between jobs and living on the thrill of commits and fixing bugs for other people. So, what are some of the motivators?

  • The thrill of seeing their code committed (in this data it's to core, even)
  • The thrill of closing out issues - klausi has said he just loves closing issues
  • Learning PHP/Drupal/web development (the core queue is one of the best free universities I've ever seen)
  • Feeling wanted - people submitting issues and getting bugs fixed means you are valued in the world
  • They get a new job that requires them to work on Drupal and...hey...might as well contribute it

But, once that hyper involvement is gone, once Dave got a job, once my purposeful stint between jobs was over, it becomes harder to find time for hyper involvement. Should we be mad at Dave for reducing his involvement in order to be a good employee? Should we be mad at Palantir for hiring him and giving him at least partially funded time to work on contributing code?

Why people decrease involvement?

  1. They get burned out on the guantlet that is our issue queue or on using git instead of hg or on...whatever
  2. They do more work in less visible areas of the project (drumm with infrastructure and drupal.org, rfay with testing)
  3. They have a babby
  4. Their spouse's work situation changes
  5. Their hobby, which they like more than Drupal, becomes their job
  6. They get a new job that doesn't involve Drupal (you could argue that Drupal jobs are so prevalant that this is more of a choice than a force)
  7. They lose the joy from getting code committed
  8. Their original involvement level was unsustainable (in between jobs, before a job, part of learning) unsustainable trends must end

Which, if any of those, should we do something about?

What about the Security Team?

As a Security Team member of many years and Security Team Lead for 2 years, I've looked very very closely at a lot of metrics about the team. In addition to metrics we've gathered more subjective data: we have brainstorming sessions every 6 months and we've also done formal surveys of the team. Based on all that feedback we made improvements:

  • to educating module maintainers
  • improved APIs to make them harder to create vulnerabilities
  • to the infrastructure we use to communicate with researchers and maintainers
  • to the advisory creation process

And I can say with a strong degree of certainty that these changes have greatly reduced the number of vulnerabilities that exist and improved the speed we publish security patches. But...they can't help that a team member no longer gets a thrill from finding a bug. They can't help that people have babies or other shifting personal priorities that make them care less about fixing bugs. They can't help a lot of things that cause people to be less involved in fixing security issues. So? So we keep working smarter but we also just keep working: we actively recruit new people. There was a time when everyone on the security team was a brand new member. There was a time when they all made their first blunder or got confused about how CSRF works and how to protect against it. New people will make mistakes too and that's OK. Mistakes are part of learning.

Another example of someone who was "new": Look at Dawehner's core involvement from 2008 until now. It's like a rocketship. 2 patches in 2008 and above 390 in this past (incomplete) year. Wow. We should be looking out for more people with the potential to move on a path like that. We should find out what activated him and see if we can create those conditions for other people.

Make new friends, but keep the old

I'm not by any means saying that we should accept people leaving the project. I am sad when people decide in some way to leave the project or reduce involvement or...whatever. But I respect their decision.

A few thoughts:

  • We should not encourage people to do unsustainable things. If someone is stepping down, respect that need.
  • If and when people step down, they should do so considerately as our Code of Conduct reminds us.
  • It's better to know that someone has left (and get someone to fill that void) than it is to have a ghost maintainer.

What does it mean to leave in a considerate way. I think this is worth a lot of thinking. It's easy to justify your own behavior - after all, most people think they are doing the "right" thing and convincing others to do that same right thing is pretty natural human behavior. While I'm being somewhat relativistic here, I want to be clear: it's easy to leave some role, see others replacing you who make mistakes, and come back and criticise them in an unconstructive manner. That's not really helpful.

So, I respect the issues where Nate Haug and Dave Reid asked to be removed from MAINTAINERS.txt. In theory I should hope that everyone has the energy and consistency that webchick and Dries bring to the project (look at their commits per year from 2000 through 2013 and you get a sense of what I mean). But, that's not realistic. So I think Nate and Dave for the work they've done and hope that they may be in a moment of reduced core involvement that will turn back up. But until then, we can find other people to do that work.

It's important to set realistic expectations for people both in terms of what can be done per time period and how long that energy can be maintained.

The Drop is always moving - because everything else is too

I've read a lot of articles and books about Drupal over time. When I first learned it...that's about all I did was learn it and build horrible sites for myself exploring site architectures. A lot of people say that Drupal 8 is too confusing. If you feel that way I'll ask: are you still learning about the technologies involved as voraciously as you were when you first learned Drupal? Are you still learning about Drupal with as much passion as you were at first? If not...consider if Drupal is more complex or if you are just complacent. PHP is not going to sit stagnantly. Web technology is not going to sit stagnantly. They are moving forward and we must move with them.

Category: 
People Involved: