This is a transcript of What's Up With That Episode 6, a 2023 video discussion between Sharon (yangsharon@chromium.org) and Elly (ellyjones@chromium.org).
The transcript was automatically generated by speech-to-text software. It may contain minor errors.
What does it mean for Chrome to be open source? What exactly is Chromium? Can anyone contribute to the browser? Answering all that is today's special guest, Elly. She worked all over Chrome and ChromeOS, and is passionate about accessibility, the open web, and free and open-source software.
Notes:
Links:
00:00 SHARON: Hello, and welcome to “What's Up With That?” the series that demystifies all things Chrome. I‘m your host, Sharon. And today, we’re talking about open source. What does it mean to be open source? I‘ve heard of Chrome, but what’s Chromium? What are all the ways you can get involved? Answering those questions and more is today‘s special guest, Elly. Elly currently works on the Chrome content team, which is focused on making the web more fun and interesting to use. Previously, she’s worked all over Chrome and Chrome OS. She's passionate about accessibility, the open web, and free and open-source software. Welcome, Elly.
00:34 ELLY: Thank you, Sharon.
00:34 SHARON: All right. First question I think is pretty obvious. What is open source? What does that mean?
00:40 ELLY: Yeah, so open source is a pretty old idea. And it basically just means, in the purist sense, that the source code for a program is open to be read by others.
00:51 SHARON: OK. And Chrome, the source code is available to be read by anyone. What else is it? Open source - I‘ve heard of open-source community. It seems like there’s a lot to it. So why don't you just tell us more about open source, generally?
01:03 ELLY: Yeah, for sure. There‘s quite a bit of nuance here. And there’s been differing historical interpretations of some of these terms, so I‘ll - there’s two big camps that are important to talk about. One is open source, which means what I said - the source is available to be viewed. There's also the idea of free software, which is software that actually has license terms that allow for people to modify it, to make their own derivative versions of it, and that kind of thing. And so historically, there was a pretty big difference between those things. These days, the two concepts are often talked about pretty interchangeably because a lot of open-source projects are free software, and all free software projects basically are open source. But the distinction used to be very important and is still pretty important, I guess. Chromium is both open source and free software. So we ship under a license that allows for - not only for everyone to read and look at our code, but also for other folks to make their own versions of Chromium. So, yeah, Chromium, both open source and free software.
01:56 SHARON: OK, very cool. And you mentioned Chromium in there. But I think for most people, when they think of the browser, they call it Chrome. So what is the difference between Chrome and Chromium? Are they the same thing? I think people, myself included, sometimes use those interchangeably, especially when you work on it. So what is the difference there?
02:16 ELLY: Yeah, fantastic question. So Chromium is an open-source and free software web browser that is made by the Chromium Foundation, which is like an actual .org that exists somewhere on the internet. Chrome is a Google-branded web browser that is basically made by taking Chromium, which is an open-source and free software web browser, adding some kind of Google magic to it, like integrations with some Google services, some kind of media codecs that maybe aren't themselves free software, that kind of thing, bundling that up into a more polished product which we call Google Chrome, and then shipping that as a web browser. So Chromium is an open-source project. Google Chrome is a Google product that is built on top of Chromium.
03:03 SHARON: OK. So Google Chrome is a Chromium-based browser, which is a term I think that people who work in any browser stuff - it‘s a term that they’ve all [INAUDIBLE] before.
03:08 ELLY: Yeah, exactly. And in fact, you alluded to the fact that we sometimes use those terms interchangeably. And especially at Google, we sometimes get a little confused about what we‘re talking about sometimes because we’re - the Google Chrome team are the biggest contributors to Chromium, the open-source project. And so we tend to sometimes talk about the two things as though they‘re the same. But there’s a really important difference for folks who are working on other Chromium-derived browsers. So if you‘re working on a Chromium derivative that a Linux distribution ships, for example, your browser is based on Chromium, and it’s really not Chrome. It‘s Chromium, right? It is the open-source browser that Chrome is based on. But it’s not the same thing at all.
03:52 SHARON: Yeah, if you want to learn a bit more about basing things on Chromium, the content episode is a good one to check out. We talk a bit about that and embedding Chrome in Chromium and what that means. So -
04:03 ELLY: Yeah, absolutely.
04:03 SHARON: check it out if you [INAUDIBLE]...
04:03 ELLY: And there‘s also, in the Chromium source tree, there’s actually a thing called Content Shell, which is a minimal demonstration browser. It‘s like the rendering engine from Chromium wrapped in the least amount of browser possible to make it work. And we use it for testing, but it’s also a really good starting point if you're trying to learn how to build a Chromium derivative browser.
04:22 SHARON: OK, very neat. So I think a next very natural question to come out of this is, why is Chrome or Chromium - Chromium rather - going to try to be good about using those correctly here - but why is Chromium open source?
04:40 ELLY: Yeah, so this is the decision that we made right when we were starting the project actually. And it‘s based on this really fundamental idea that the web benefits when users have better browsers. So if we, like the Chromium project, come up with some super clever way of doing something, or we come up with some really ingenious optimization to our JavaScript Engine or something like that, it’s better for the web, better for everyone, and ultimately even better for Google as a business if those improvements are actually adopted by other people and taken by other people and used by them. So it is better for us if other people make use of anything clever that we do. And separately from that, there‘s this idea that’s really prevalent in open-source communities of, if people can read the code, they‘re more likely to find bugs in it. And that’s something that Chromium constantly benefits from, is folks who are outside the project, just kind of looking through our code base, reading and understanding it, spotting maybe security flaws that are in there. That kind of research is so much easier to do when the source code is just there, and you‘re not trying to reverse-engineer something you can’t see the source to. So we get a lot of benefit from being open-source like that. And those are the reasons we had originally, and those still all hold totally true today, I think.
05:51 SHARON: That makes sense. Yeah, it seems, at first, a bit odd for a big company like Google to make something like this open source. But there are other massive open-source things at Google - Android, I think, being the other canonical example, which we don‘t know too much about, but we won’t be getting too into that. But there are other big open-source projects around.
06:08 ELLY: Yeah, absolutely. And there‘s also, like - there’s Go. That's an open-source programming environment, like a language and a compiler and a bunch of tools around it that is open source built by Google. There are plenty of other open-source and free software projects built by large corporations, often for really the same reasons. We benefit because the entire web benefits from better technology.
06:32 SHARON: Yeah, I think some of the Build stuff we do is open source. Check out the previous episode for that. And that's, yeah, exactly - not strictly only used by -
06:37 ELLY: Yeah, and by the way, partly because we're open source - like, for example, the Chromium base library, which is part of our C++ software environment - our base library is regularly used in other projects, even things that are totally unrelated to browsers, because it provides a high-quality implementation of a lot of basic things that you need to do. And so that code is being used in so many places we would never have anticipated and has done honestly more good in the world than it would do if it was just part of a really excellent browser.
07:13 SHARON: Something that someone on my first team told me was, if you've changed anything in base, that probably is going to get run any time the internet gets run, somewhere in that stack, which, if you think about it, is so crazy.
07:26 ELLY: Oh, Yeah. Absolutely. Early in my career, I added a particular error message to part of the Chrome network stack. And that network stack, too, is one of those components that gets reused in a lot of places. And so occasionally, I‘ll be running some completely other program. Like, I’ll be running a video game or something, and I‘ll see that error message that I added being emitted from this program. And I’m like, oh, my code is living on in a place I would never have really thought of.
07:51 SHARON: Oh, that's very cool.
07:51 ELLY: Yeah.
07:51 SHARON: Yeah.
07:51 ELLY: It‘s one of those unique open-source experiences in my book, of seeing your own work being used like that by other folks you wouldn’t have anticipated.
07:57 SHARON: Yeah, that‘s very cool. So something I think I’ve heard you say before that I thought sounded very cool was the open-source dream. So can you tell us a bit more about what that is. What is that vision? It sounds very nice.
08:09 ELLY: Yeah, so I talked about this a little bit. And earlier, I cautioned against conflating open-source and free software. But it really is more of the free software dream than the open-source dream, in some sense. That dream is this idea that if we have software that is made available for free, under licenses that let people modify it and make derivative works and keep using it, that over time, everyone will get access to really high-quality and freely-available software. And we will have a situation where the software that people need is built by their communities, built by the people who are in those communities, instead of being something that they have to buy from a company that makes it. It‘ll be something they can instead produce for themselves. And over time, I think that this has really played out in that way. If you look at the state of operating systems today, for example, there are these really high-quality, freely-available open-source free software operating systems that are readily available and anyone can use, and they really do meet the needs for a lot of folks. And then, in fact, it kind of circles back to where Linux is a high-quality, free software open-source operating system that Google can then turn around and make really good use of to build something like Chromium OS, which is another free software open-source project that uses Linux as one of its major components. And then we get to produce a product that the Chromium OS engineering team would have had to spend a lot of time if we weren’t able to make use of that existing Linux kernel work. So you get into this cycle of giving back and sharing and benefiting from the effects of other people sharing. That's the free software dream to me.
09:57 SHARON: It does - yeah, that sounds great. And for sure - I try to use open-source options when I can. When I edit these videos, I use something open-source. It feels appropriate for what we‘re doing here. So, yeah, that sounds like it would be - it’s a good system that everyone contributes to and everyone benefits from. And that's really nice.
10:10 ELLY: Yeah, absolutely.
10:16 SHARON: So going away from that towards the more less open-source part, so what kind of things in Chrome, the browser, are not open source? You mentioned a couple of things earlier. Can you tell us a bit more about some of those things?
10:27 ELLY: Yeah, I‘m going to caveat this by saying that I don’t personally work on the stuff I‘m about to talk about. And so my knowledge is more superficial. There’s a couple things I‘m pretty confident about. So one is, for example, there’s a few video formats that Chrome can play that Chromium cannot play because Google has agreements with the companies that make those codecs that allow us to basically license and embed their thing and ship it as part of Chrome. But those agreements, we can‘t really extend them to everyone who might make a Chromium browser. And so it ends up in a situation where there is a closed-source component that’s included in Chrome to make that possible. I‘m struggling to think of another example right off the top of my head. I believe that there’s also a couple things in Chrome that are integrating with Google APIs, where they‘re features that are Chrome-specific because they’re Google-specific. And one of the things that is generally true between the two products is that Chrome will have more Google integrations and more Google magic and more Google smarts than Chromium will. And so I think some of those are actually closed-source components that come from Google that get embedded into Chrome. But because they‘re a closed-source, we wouldn’t want to put them into Chromium.
11:37 SHARON: Right. It seems like, yeah, I can sign into Chrome. I don‘t expect that I’d be able to sign in with my gmail.com into, say, Chromium. I‘m not sure it’s actually part of it, but that's a guess.
11:49 ELLY: Yeah, so that does work, except that you need to - any Chromium distributor needs to go and talk to - basically, talk to the sign-in team to get an API key that allows their browser to sign in. There is a process for doing that. It doesn‘t actually require any closed-source code components. But there is still a thing where you have to talk to the accounts team and basically be like, hey, we’re a legitimate web browser, and we want to allow users to sign in. Because we don‘t want a situation where bots or malware are doing fake user sign-ins from - pretending to be Chromium. That’s bad.
12:25 SHARON: Right. That makes sense. Yeah, and I think because of where Chrome and Chromium are positioned, I think there will be some interesting comparisons and differences between Chrome, Chromium, and other internal google3 projects. So that‘s kind of the term for things that are closed-source Google - the typical Maps, Search, all that stuff - and also comparing Chromium to other open-source projects. So we’ve talked a bit about the similarities and differences between Chrome and Google internal. Are there any other things you can think of that are either similar or different between Chrome the project and the people who work on it and how people do things internally at Google?
13:11 ELLY: Yeah. So internally at Google, there‘s this very powerful, very custom-built whole technology stack around the projects. There is a continuous integration system. There’s an editor. There‘s a source control system. There’s all of this stuff. Within Google, all of that is custom. And it‘s all fitted to Google’s needs. And a lot of it is just built from scratch, frankly. Whereas for Chromium, we‘re using essentially off-the-shelf open-source stuff to meet a lot of those needs. So, for example, for version control, we’re just using Git, which is I think the most popular version control system in the world right now. It‘s definitely open source. And our build system, for example, which is like GN and Ninja put together, those are both free software open-source projects. Admittedly, both of them were, I think, started as part of Chromium because we had those needs. But they, themselves, are free software components that anyone else can also use to build a Chromium. And the reason why that’s done that way - like, why doesn‘t - it’s actually a really good question. Why doesn‘t Chrome, which is a Google project, use all of this amazing infrastructure for engineering that Google has? And the answer is, we want the Chromium project to be possible to work on for people who don’t work at Google. And so we can‘t say, oh, hey, whenever you’re going to make a change, you have to commit it into Google‘s internal source control system. That wouldn’t work at all. So we‘re almost - because we want to be an open-source project, and because we want to have contributors from outside of Google, we end up almost pushed into using this pretty open free software stack, which I - to be honest, from my perspective, has a lot of other benefits. When we have new folks joining the team, we can actually offer them tools they’re already pretty familiar with. They don‘t have the feeling that new Googlers sometimes get, where they’re totally disoriented. Like, everything they know about programming doesn‘t apply anymore. We actually be like, hey, here’s Git. You know how to use this. Here‘s Gerrit, which is another piece of open-source software that we use. They may not have used Gerrit before, but a lot of projects do. And so they might have run into it previously. So it has pluses and minuses, definitely. So that’s a big difference. There‘s also a bit of what I would say is a cultural difference more than anything else because most Google projects that are not open source - so I’m not talking about things like Android or Go or something like that - but projects that are really just not open source, like Search, their ecosystem of discussion and culture and stuff is very much inside Google. Whereas for Chromium, we constantly are getting ideas and suggestions and code changes and stuff from outside of Google. And so we also tend to have perspectives from outside of Google in our discussions more often as we work on Chromium. So part of that is at the level of, if we‘re going to make a change, we would have maybe input coming in on that change from Mozilla even. They’re a group we collaborate with a ton on web standards. And so we would have their perspective in the discussion. Whereas if we were working entirely within Google, we might not have those external perspectives. So culture-wise, I feel like Chromium has more perspectives in the room sometimes when we're thinking about stuff.
16:26 SHARON: That makes sense because browsers exist across other companies too, and there‘s a lot of compatibility and standards and stuff. So just in that nature of things, you have to have a lot more of this collaboration. If you make a change, it’ll affect all of the embedders maybe, and then you have to think about this. And, yeah, there's a lot more discussion - [INTERPOSING VOICES]
16:42 ELLY: Yeah, absolutely.
16:42 SHARON: If you‘re Search, you’re like, OK, we‘re going to, I don’t know, do our thing.
16:47 ELLY: Yeah, you have more - I don‘t know if “autonomy” is the right word. But, yeah. I want to caveat this by saying I’m not on Search. And so maybe it‘s totally different. But that’s how it looks to me as a person who works on Chrome.
16:59 SHARON: Yeah. Yeah. And I think in terms of actual development and making code changes and stuff, I think probably the biggest difference is that because anyone can download the source repository and make changes and all that, the actual programming and changes you do, you do those on a computer. Maybe that's a machine you SSH into or a cloud top or whatever. But you have to actually download all of the code. Whereas with all of the google3 stuff, everything happens in a cloud somewhere. So everything is all connected, and you just do things through the browser pretty much.
17:29 ELLY: That‘s very true. Actually, there’s another important facet that just occurred to me, which is, because Chromium is open source - and in particular, some open-source projects will use this model where they send out a release every so often. So they‘ll be like, we’re shipping a new major release of our program, and here‘s the source that corresponds to that. So there are companies that do that. But we actually do what’s called developing in the open. So our main Git repository that stores our source is public. Which means that as soon as you put in a commit, or even if you just put it up for code review, that‘s public. Everyone on the internet can see what we’re doing live, which is really pretty interesting in terms of its effects on you. So for example, if you‘re in - you’re working inside google3, and you‘re like, I have this really cool, wild idea, I’m going to go and make an experimental branch and just make a prototype of it and see what happens, you can just go do that. It‘s not a problem. But if you’re working in Chromium, and you go and make your wild prototype experimental branch, you have a pretty good chance that someone‘s going to notice that. And then maybe you get a news story that’s like, hey, Chromium might be adding this amazing feature. And you‘re like, oh, no, that was my wild, experimental idea. I didn’t intend for this to happen. But now people have really picked up on it, and people outside of the company that you‘ve never met are starting to get excited about something that you never really intended to build and just wanted to try. So it’s a different way of working. You‘re sort of always in the public eye a little bit. And you want to be a little bit more considerate about how something might look to people way outside of your team and outside of your context. Whereas teams that are inside google3 I don’t think have to think about that as much.
19:07 SHARON: Yeah, I mean, for me, I‘ve only really worked in Chromium full time and all that. And I’ve just gotten used to the fact that all of my code changes are fully public and anyone can look at them. Whereas I think people who work in anything that‘s not like that - people in the company you work, I can see it. But not just anyone out there. So I don’t know. I‘ve gotten used to it, but I think it’s not a typical thing to [INAUDIBLE].
19:30 ELLY: Oh, yeah. Absolutely. And in fact, this is something that folks who are transferring into Chrome from other parts of Google sometimes have a little difficulty with, is if you‘re used to writing a commit message where maybe the only description in the commit message is go/doc about my project, for Chromium that doesn’t fly because only Googlers can actually follow those links. And so the commit message to a non-Googler doesn‘t say anything. And so you actually have to start thinking, how am I going to explain this whole thing I’m doing to a non - to a person who doesn‘t have any of this Google-specific context about what it is. You go through this little mental - you cross this little mental bridge where you actually are forced to reframe your own work away from, what are Google’s business goals, and towards, how does this fit Chromium, the open-source project, that other people also use? It's interesting and occasionally a little frustrating, but interesting and usually really beneficial.
20:26 SHARON: Yeah, for sure. And I think from people I've talked to, it just seems like another, briefly, difference between internal Google stuff and Chromium is that internal Google just has a ton of tools you can use.
20:37 ELLY: Yes, absolutely.
20:37 SHARON: Which both means a lot of things that are maybe a bit challenging in Chromium are probably easier, but also maybe finding the right tool is hard. But -
20:42 ELLY: Oh, yeah. That is very much the case. I have only limited experience working inside google3. But I definitely have experienced the profusion of tools and also the fact that the tools are just honestly amazing. And it makes total sense. Google has many, many engineers whose whole job is to build great tools. And Chromium is just not that big of a project. We just don‘t have that many folks that are working on it. The folks who do build infrastructure work for Chromium do amazing work, but there’s not hundreds of them. And so it's not on the same level.
21:12 SHARON: Yeah. And what you said earlier makes me have - gives me - has - makes me wonder - and this ties us into the next thing - of other open-source projects, they just do a release, and they don't maybe do development in the open. And having not actually worked on other open-source projects really, I kind of assumed that this development in the open was the norm. So how common do you think or you know that that practice is?
21:45 ELLY: Gosh, I would really be guessing, to be honest with you. But I would say the development in the open is by far the norm these days. And when you see projects that follow the big release model instead, the way that looks is they‘ll be like, hey, version 15 is out, and here’s the source for version 15. You can look at it. But the development, as it happens, happens internally. I would tend to associate that with being maybe big company projects that have a lot of confidentiality concerns. So for example, if you‘re building the software that goes with some cool, new hardware for your company, you don’t want to start checking that software into Git publicly because then people are going to read it and be like, ooh, this has support for a billion-megapixel camera. That must be coming in the new thing. And so I think that the big release model might be, these days, more prevalent when people are doing hardware integrations, where there‘s other components that are shipping at a fixed time and you don’t want your source to be open until that point. But honestly, the developing in the open model is, I think, much more common these days. Historically, back in the '70s and ‘80s, when you would buy an operating system and it would come with source, that was just a thing that you got as part of the package, then it was much more of the source is released with the OS model. Whereas these days, because distributed development is so easy with modern version control systems, it’s just so common to just develop in the open like we do.
23:11 SHARON: Oh, cool. I didn't know that. So compared to other open-source projects, what are some similarities and differences that Chromium has to others that you may be familiar with?
23:25 ELLY: Ooh. All the ones I‘m familiar with are quite a bit smaller than Chromium. And so it’s going to be hard to talk about it because, frankly -
23:32 SHARON: That's probably the common difference, though, right? Probably very few are as big as Chromium.
23:32 ELLY: Oh, yeah. So in particular, one of the hardest problems in open source - in running an open-source project is managing how humans relate to other humans. The code problems are often relatively easy. The problems of how do we make decisions about the direction of a project that maybe has a hundred contributors who speak 10 different languages across a dozen time zones, that‘s a hard problem. And so I often talk about the idea between open source, open development, and then open governance. And so open source is just, like, you can see the source. Open development is you can see the development process. So the Git repo is open. The bug tracker is open. The mailing lists, where we do a lot of our discussion, are open. So we do open development. But then you have this next step of open governance, where the big decisions about where the project is going are made in the open. And for Chromium, some of those are made in the open, especially when it’s really about the web platform or that kind of thing. But some of them are not. For example, if we‘re deciding that we’re going to do some cool new UI design, that design and the initial development of it might not necessarily be - or sorry, the development would be done in the open, but the designing of it might not. That might be a discussion between a few UX designers who all work at Google in a Google internal place. And so Chromium has a bit of open governance but not all the way. A lot of smaller projects have super open governance. So they‘ll literally be like, hey, should we rewrite this entire thing in Rust? And they’ll make that decision by arguing about it on a mailing list, where everyone can see. And that‘s totally, totally fine. Because Chromium is so big, we can’t make those kinds of decisions by having every Chromium engineer have their opinion and just post. It would be complete chaos. And because we‘re big and prominent, a lot of the work that we do is very much in the public eye. And so even discussions that are maybe relatively speculative - like that example I gave before, where you have an idea and you’re like, wouldn‘t it be neat if we did this? It’s easy for that to turn into people inferring what Google‘s intentions are with Title Case, like, Big Important Thing, and turning that into a lot when you would not have intended it to be that way. And so we do end up keeping our governance relatively on the closed side compared to other open-source projects I’ve worked on. Other than that, in terms of engineering practices and what we do to get the code written, we uphold a super high standard of quality. And in particular - which is not to say that most open-source projects don‘t, because they totally do. But Chromium, in my opinion, is really, really thoughtful about not just, hey, how should code review work, but really evolving stuff like, how should we bring new developers into this project? What should that feel like? Those are discussions that we have. And I often feel like those are discussions that other open-source projects don’t talk about as much. What else is different for us? I‘m not sure. I think that those are some of the big ones. The differences in scale are such that it’s almost hard to talk about. The difference between an open-source project that maybe has 5 contributors and one that has 500 is very, very large.
27:07 SHARON: With the open governance thing you mentioned, something that that made me think of is maybe Blink Intents, where you submit a thing to a list and then that gets discussed. So that's part of the Chromium project, I think, right? That falls under that category.
27:20 ELLY: Yep. Yep.
27:20 SHARON: And so that's where, if you want to make a change to Blink, the rendering engine, you do this process of posting it to a list, and then people weigh in.
27:25 ELLY: Yeah, absolutely. So Blink really does do open governance in a way that I, honestly, very much admire. Blink and the W3C and a lot of these groups that are setting standards for the internet do do open governance. Because, frankly, it‘s the only way for them to work. It would not be good or healthy for the web if it was just like, we’re going to do whatever - whatever we, Google, have decided to do and good luck everyone else. That would be very bad. So yeah, Blink definitely does do open governance. But when it gets to things that are more part of the browsers' behavior and features, we tend to have the governance a little more closed.
28:08 SHARON: Right. And I think an example of Blink being more open governance is the fact that BlinkOn is open to anyone to participate to. And that‘s the channel that we’re posting this on right now. It just happened to make sense that I figured most of the audience who is watching Blink [INAUDIBLE] already are interested in these, too. So that's why - [INTERPOSING VOICES]
28:27 ELLY: Yeah, absolutely.
28:27 SHARON: And for people who may not have - may have found these videos that don‘t know about BlinkOn. That’s what that is.
28:34 ELLY: Yeah. And just in that vein of open governance for Blink, especially, there‘s also this idea of being a standard and then having things be compatible with it. So the web platform is a collection of standards. And other browsers have to implement those standards, too. And so for example, if we make up a standard that is very difficult or impossible for, like, Firefox to implement, that’s not good. That‘s fragmenting the web platform. That’s a bad thing. Whereas the Chromium UI, like how the omnibox works in Chromium, for example, isn‘t a standard. It doesn’t matter whether Firefox or Edge or Opera or whoever have the same omnibox behavior as us, right? And so there‘s much less of a need to all agree. And instead, it’s almost a little bit better to have some variety there so that users can get a little bit more of a choice and that collectively more things get tried in that vein. So there‘s places where agreement and standardization are really important. And then there’s places where it‘s actually OK for each individual browser to go off on its own a bit and be like, hey, we thought of this cool, new way to do bookmarks. And so we have built this. And it doesn’t matter whether the other browsers agree about it because bookmarks are not a thing that interoperates between browsers.
29:44 SHARON: Yeah, that makes sense. So now let‘s talk about some of the actual details of what it’s like to work on Chromium and make changes, write code, and new ideas. So I think you mentioned a few things, like bug tracking. That‘s all public, in the open, apart from, of course, security-sensitive things and other [INAUDIBLE] are hidden. What else is there? Code review - that was Gerrit. You mentioned that. So You can see all the comments that everyone leaves on everyone’s changes.
30:16 ELLY: Oh, Yeah. And for better or for worse, by the way. It‘s good to bear in mind that if you’re like - you‘re going to type like a slightly jerk message to someone on a code review, that’s going to be preserved for all time, and everyone's going to be able to see it.
30:29 SHARON: Yeah. Yeah. Be nice to people. [CHUCKLES] Version control - that‘s Git. Probably people will know about that. Something that might be worth mentioning is that a lot of people who contribute to Chromium, and if you look at things like Gerrit and Chromium Code Search - that’s also public, of course - looks a lot like Google internal code search, but obviously it's open source. So a lot of people have @chromium.org emails.
31:00 ELLY: Yes.
31:00 SHARON: So why are there separate emails? Because you can use at a google.com or a GMail or any email. So why have this @chromium.org email thing?
31:05 ELLY: Yeah, so there‘s a few different reasons for that. So chromium.org emails are available to members of the project, which is a little bit nebulously defined, but it’s definitely not just Googlers. And so there‘s a couple reasons why people like having those. So for some folks, it’s sort of a signal that you are acting as a member of the open-source project rather than acting with your Google hat on, if you like. And so for example, I help run the community moderation team for Chromium. And so when I‘m doing work for that team, I’m very careful to use my chromium.org account because I want it to be clear that I‘m enforcing the Chromium community guidelines, which are something that was agreed upon by a whole bunch of Chromium members, not just Googlers. And so I’m not enforcing Google‘s code of conduct. I’m enforcing Chromium‘s code of conduct in my role as a Chromium project person. So sometimes you deliberately put on your Chromium hat so that you can make it clear that you are acting on behalf of their project. Some folks - and I’m also one of these folks, by the way - just happen to really be big fans and supporters of free software and of open source. And so if I have the choice between wearing my corporate identity and wearing my open-source project member identity, I might just wear my open-source project member identity and decide to actually contribute that way. And so a lot of the folks who‘ve been on Chromium - or have been on Chrome, I should say, for a while, that’s part of their reasoning. They joined because they were excited to work on something that was open. And so they have this open-source identity, this Chromium identity, that they use for that. There‘s a third factor, and this touches on one of the sometimes less pleasant parts of working in open source, which is our commit log and our bug tracker and all of that stuff are public. And what that means is that everyone on the internet can go see them. And that is often great, but it’s occasionally not great. So for example, if you go and make an unpopular UI change, people on the internet know that that was you. And that might not be something that you‘re necessarily super ready to deal with. So for example, way, way, way, way early in my career, I made a change to Chromium OS because I was working - I was on the Chrome OS team as a brand Noogler. So this is I’ve been at Google maybe five or six months. I made a change to Chrome OS. Somebody happened to notice it and take issue with it. I don‘t even remember what the change was or the issue. But they happened to notice it and take issue with it. They showed up in our IRC channel, because we used IRC at the time, which was also public because the whole project was very open like that, and really just started yelling at me personally about it. And I’m like, this is not a cool experience. This is something that if this was a Google coworker of mine, I would be talking to HR about this. But it‘s actually just a random person on the internet. And so there are some folks who use their Chromium username as a little bit of a layer of insulation almost, where it’s like, I want to work on this project, but I don‘t - maybe my Google username has my full name in it. I don’t necessarily want every change I make to be done like that. And so if you don‘t do that, you can end up in a situation where you make a change, and then it’s really attributed to you as though it was your personal idea and you did this bad thing. And that‘s not a risk that everyone wants to take as part of doing their work. And so sometimes people have a chromium.org account really because they want an identity that’s separate from their Google account - that has a different name on it, that has different stuff like that. And so one of the things that I‘m always cautious to remind folks of on my team is, if you’re working with someone who has a chromium.org account, always use that chromium.org account when you‘re speaking in public, always, always, always, because you don’t want to break that veil if someone is relying on it.
35:09 SHARON: Right. Yeah, that makes sense. And I think, in general, whenever you are signing up for interacting in these public spaces, generally, I think it's encouraged to use your chromium.org account. So for example, Slack, which is the modern - current IRC often -
35:27 ELLY: It hurts my soul to hear you say that.
35:32 SHARON: Well - [LAUGHS]
35:32 ELLY: I‘m a die-hard IRC user. I’ve been using IRC for 30 years. And I was one of the few people who was I think very sad when we decided to move off IRC. But you're right, that it is the modern IRC option.
35:44 SHARON: I think a lot of people are very die hard about IRC. So, you know, but modern or not, that‘s what’s currently being used.
35:49 ELLY: Absolutely.
35:55 SHARON: So Slack is where anyone can join and discuss Chromium stuff. And generally, that kind of thing, you're encouraged to use your chromium.org account.
36:01 ELLY: Yeah, absolutely. And to be fair to Slack also, the Slack has probably 30 times as many people in it as the IRC channel ever did. So I think that it‘s pretty clear that Slack is more popular than IRC was. But, yeah, no, we use our Chromium identities a lot, really, really on purpose. And to be honest, I would like it if we use them even more. Sometimes you will see folks who actually have both identities signed up. So they’ll have their google.com and their Chromium, and that‘s always confusing for everyone. So if it was up to me, I would say everyone has a Chromium identity, and they’d just all use it when they're contributing.
36:39 SHARON: Yeah, that‘s definitely one of these unique two Chromium [INAUDIBLE] pain points of someone [INAUDIBLE] use their maybe - often, they’re the same for most people. But sometimes they‘re different. Sometimes they’re very subtly different, and it's -
36:53 ELLY: Absolutely.
36:53 SHARON: you end up sending your [INAUDIBLE]...
36:53 ELLY: I also - I have met a couple folks who the Google username they really wanted wasn't available, but it was available for chromium.org. And so they picked a shorter, cooler username for chromium.org, which is totally - totally fine to do. But then, every time you have to remember, oh, I know them by this longer Google username, but actually they use this shorter username for Chromium.
37:13 SHARON: Yeah, you have to remember their real life name. You have to remember their work email. And then now you have to remember another work email.
37:19 ELLY: Well, we have software that can help with that a bit.
37:25 SHARON: Yeah, for sure. So as part of that, and that‘s, in a way - a thing that to me feels very related is there’s a thing called being a committer in Chromium. So what does it mean to be a committer? And what does it entail?
37:37 ELLY: Yeah, so committers are basically people who are trusted to commit to CLs, for want of a better way of putting it. So the way the project is structured, anyone can upload a CL. And anyone anywhere on the internet can upload a CL. It has to be reviewed by the OWNERS of the directories that it touches or whatever. But there are some files that are actually, like, OWNERS equals star. So for example, the build file in Chrome browser, because everybody needs to edit it all the time, it just has OWNERS equal star. And there‘s a comment that’s like, hey, if you‘re making a huge change, ask one of these people. But otherwise, you’re just freely allowed to edit it. And so if the committer system didn‘t exist, anyone on the internet would be allowed to edit a bunch of parts of the project without any review, which is pretty bad. And so there’s this extra little speed bump where it‘s like, you have to send in a few CLs to show that you’re really a legit person who‘s contributing to the project. And once you’ve done that, you get this committer status, which actually allows you to push the button that makes Gerrit commit your change into the tree. And that‘s what it does mechanically. We culturally tend to have it mean something a little different than that, but it’s - culturally, it‘s like a sign of trust of the other project members in you. So getting that committer status really means, we collectively trust you to not totally screw things up. That’s what it is. And so you have to be a committer to actually be in an OWNERS file, for example. You can‘t be listed as an owner until you’re a committer. Because if you‘re not a committer yet, we’re not really - if we‘re not trusting you to commit code, we’re not really going to trust you to review other people‘s code. And, yeah, when you’re new joining the project, it‘s actually a pretty big milestone to become a committer. You become a committer after you’ve been working for anywhere from three to six months, I would say. And it‘s definitely this moment of being like, yeah, I’ve really arrived. I‘m no longer new on the project. I’m now a full committer.
39:51 SHARON: Can you briefly tell us what the steps, mechanically, to becoming a committer are?
39:51 ELLY: Yeah, so you need to have landed enough CLs to convince people you know what you‘re doing. And there is no hard and fast limit, but it’s like - it should be convincing. And so I often hear maybe 15 to 20 of nontrivial CLs is a pretty good number. Having done that, you need someone to propose you or nominate you for committership. So there‘s actually - there’s a mailing list for having these discussions. And so whoever‘s going to nominate you, who has to already be a committer, they’ll send mail to that list, basically being like, I would like to nominate this person for committer. There‘s a comment period during which people can reply. And then if there’s nobody who is raising a big objection to you being a committer, after - I don‘t know what the actual time period is - but after some amount of time, the motion carries with no objections, and then your Chromium account becomes a committer. I think Google accounts can also be committers as well, but I’ve only ever done this process for Chromium accounts. And so those threads - what‘s going on in those threads is mostly people endorsing the request. So let’s say that I have someone who‘s new on my team who I want to propose as a committer. I’ll start the thread nominating them as a committer, and then I‘ll go and talk to maybe two or three of the people who have reviewed a lot of their changes, and I’ll be like, hey, would you endorse this person for a committer? If so, please post in this thread. And so in the thread, there will actually be a couple of replies that are like, plus 1, or, yes, this seems like a good fit. Very rarely, there might be a reply, which is like, hey, I saw some - I saw some stuff on this CL that shows that maybe this person isn‘t quite ready. We had a whole bunch of back and forth comments, and eventually it really didn’t seem like they understood what I was asking for. And I feel like they‘re not really ready yet. Sometimes that will happen. But usually the threads - by the time someone’s nominating you, you‘re already in good shape. So that’s the mechanical process. And then there is - it might actually just be Eric, individually, who goes through and flips the bits on people being committers based on the threads. I‘m not sure. But there’s some process by which those threads turn into people being committers.
42:14 SHARON: OK, cool. Is there an analog of this either internally at Google or in other open-source projects? Because internally at Google, there‘s the concept of readability, which means you are vouched for that you know how to code in this one language, which has some similarities. That’s maybe a similar thing. Are there any similar notions in other projects you've seen?
42:38 ELLY: Yeah, so many projects have this notion of being a member. And that often combines our notions of committer and sometimes code owner. And so they might - or for some open-source projects, you‘ll actually hear “maintainer” as the thing. And so they’ll be like, only people who are project members can upload changes in the first place. And only people who are maintainers can merge those changes. So that little speedbump on entry is pretty common. Because it‘s a fact of life that if you are on the public internet and you have no barriers to entry, you’re going to have spam in your community no matter what you do. And so that kind of split is super, super common. For some projects that don‘t do open development, the entire thing might happen inside a company or inside an organization anyway. And then there is no notion of committer status because you’re just hired onto that team and then you can commit. But for projects that do open development and free software projects, there is often a sense of, these are the people who are roughly trusted to land code. And for a lot of projects, especially bigger ones, there‘s actually a two-tiered model, where maybe you have people who are domain experts on a specific thing, like, they maintain some subsystem. And they’re trusted to make whatever changes they need or approve other people‘s changes in that area. But then at the wider scale, there’s what‘s often called a steering committee or a core group or something. And those groups have authority over the whole project and the direction of everything that’s going on. And so you‘ll often see that kind of model in larger projects. At smaller scales, it’s often literally a list of one to five people who all have commit access to the same Git repo, and there's no - no structure on top of that. But for bigger projects, governance becomes a real concern. And so people start thinking about that.
44:35 SHARON: All right. Now, let‘s switch topics to talking about the more day-to-day logistics of working on Chromium. So if you’re not a Googler, don't work at Google, to what extent can you effectively contribute to Chromium, the project?
44:48 ELLY: Yeah, so that depends where you‘re coming from, both whether you’re part of another large organization, like maybe you work at Microsoft, you work at Opera, Vivaldi, one of those companies, or if you‘re really an IC lone contributor. If you’re in a large organization, probably your org will have its own structure around how you should contribute anyway. And so you might just want to talk about that. So I‘ll really focus on the individual contributor angle. And so for engineers specifically, like if you’re a programmer who wants to contribute to the code base, that‘s awesome. The best approach I think is really to find an area that you’re passionate about because it‘s so much more fun and enjoyable to contribute when you’re doing something you care about. So find an area you care about. Get in touch with the team that works on that area, either through their mailing lists or find their component in Monorail or find them in the OWNERS files or whatever. Get in touch with those folks. Ask them what are good places for you to contribute as a new person. That‘s often a really great way to get started. And you’ll have a person you can go to for advice to be like, hey, how do I go about doing this thing? My experience has been that Chromium contributors are pretty much all super helpful. And so they‘re very willing to just give you guidance or do whatever. And you’ll then know who to send your code reviews to.
46:01 SHARON: Cool. Yeah. And if you're not an engineer, what are some ways you can also contribute?
46:06 ELLY: Yeah, so there‘s a whole bunch of these. And by the way, these all apply to basically every open-source project, so not just Chromium specifically. So open-source projects, if you are a good writer, if you enjoy doing technical writing or you enjoy doing UX writing or you want to do that kind of thing, almost every open-source project out there is looking for people to contribute documentation. And Chromium is no exception at all to that. So high-quality documentation, we love that stuff. Or even if you’re just honing that craft and you want to practice, Chromium is not a bad spot to do that. If you‘re a UX designer or a visual designer, a lot of open-source projects will actually appreciate your contributions of you bringing in, like, hey, I thought of a way that this user experience could feel or how the screen could look or something like that. They’ll often appreciate that kind of input or design work. If you are someone who speaks multiple languages, translations are another great way to contribute to open-source projects. A lot of open-source projects don‘t have access to the same kind of - Chromium has access to a translation team within Google who do a lot of our translations. A lot of open-source projects don’t have that. And so contributing translations of documentation, of user-facing interface, stuff like that, can be super valuable. And the last thing I‘ll say, which can be done by really anyone - you don’t even need special skills for this one - is try early releases of stuff. So try development branches. If you‘re a Chrome user, try running Beta or Dev or Canary. And then when something doesn’t feel right or when it‘s - when it doesn’t work for you or it crashes or whatever, file bugs. And try to get practiced at filing good bugs, with details and info and steps to reproduce the bug and stuff like that. That‘s such a huge help as a developer of any open-source projects - to get that early-user feedback and be able to correct problems before they make it to the stable channel. And on Chromium, I’ve run into a few folks who just - their main contribution to the project is really just that they file great bugs all the time. There‘s a few folks who all they really do is they run Canary on Mac, and they notice when something doesn’t feel quite right. And so they file stuff that‘s like, maybe the engineering team wouldn’t necessarily have noticed it. But when someone calls it out, we‘re like, oh, that actually does feel kind of janky, and now we can go fix that. And getting that feedback early is so, so valuable. So there’s a lot of different ways. Those are some, but there's plenty more, too.
48:21 SHARON: OK. Cool. Yeah, and a few things on that. If you want to really try out random things, you can go to chrome://flags, play around there, see what happens. In terms of going back a bit for being an engineer, there‘s other web-adjacent stuff that you can do that we won’t get into too much now. But that can be things like adding web platform test, web standard stuff. And for people who are into security, we have a VRP, Vulnerability Rewards Program. But if you know about that, probably you‘re into the whole security space. This is not how you’re going to - maybe this is how you heard about it, and you want to get into it. But, anyway -
48:59 ELLY: Yeah. I will say, if you‘re a security researcher and you aren’t familiar with the Chromium VRP, you should go take a look because it's - Chromium is a really interesting project to audit for security. And the VRP can make it very worth your while to do so if you find good bugs.
49:12 SHARON: Mm-hmm. Yeah. And going back a bit earlier to being an engineer, like an IC, who is not at Google or any of these other big companies, there are other barriers to entry to being a contributor, right?
49:28 ELLY: Oh, yeah.
49:28 SHARON: So I definitely encountered this after my internship. I worked on Chrome. I was like, hey, I know what‘s going on now at the end of it. A couple things we didn’t finish. I‘ll go home, and I will keep working on this - good intentions. And I got home, got my laptop, which was a pretty good laptop, but still a laptop. I downloaded Chrome. That took a very long time. I built it for the first time, which always takes a bit longer. But that took so long. And even the incremental builds just took so long that I was like, OK, this is not happening. I’m in school right now. I‘ve got other things to worry about. So how feasible is it for a typical person, let’s say, to actually make changes in Chromium?
50:05 ELLY: Yeah, that is unfortunately probably the biggest barrier to entry for individuals who want to make technical contributions. Obviously, it doesn‘t affect you if you’re contributing documentation translations, whatever. But if you‘re trying to modify the code, yeah, the initial build is going to be very slow, and then the incremental builds are going to be very slow. And a lot of the ancillary tasks are slow too, like running the test suite or running stuff in a debugger. The project is just very big. And that’s something that I think a lot of folks on the Chromium team wish we could reduce. But Chromium is big because the web is big and because what people want it to do is big. And so it‘s not just big for no reason. But it does make it harder to get started as a contributor. I’ve had this experience, too. I have a modern laptop sitting on the desk over there. And it takes seven to eight hours to do a clean Chromium build on that. Whereas on my work workstation, which has access to Goma, Google‘s compile farm, it takes a few minutes. And the large organizations that contribute also all have compile farms for the same reason. It’s just so slow to work when you‘re only doing local building and don’t have access to a ton of compilation power.
51:12 SHARON: Mm-hmm. Yeah. I wonder if we could, I don't know, do a thing for people who are individuals who contribute more. Probably that would be really hard to do. Probably people have thought about it. But, yeah.
51:24 ELLY: It would be nice if we could. I don't know what the challenges would be offhand, but it would be very cool if we could somehow make that available.
51:30 SHARON: All right. That all sounds very cool. I know I learned a lot. Hopefully some of you learned a lot, too. I think if you are working within Google, it‘s really easy to not really interact with any of this more open-source stuff, depending on which part you work on. Maybe you work on a part that’s very Google Chrome specific. I know before I was working on Fuchsia, so that was before Launch. So that was not really something we were open to the public about anyway. And a lot of even the typical Chrome tools I was unfamiliar with. So I think depending on which part you work on, this stuff - it's all there, but you might not have had a chance to interact with. So thank you, Elly, for telling us about it and giving us some context about free and open-source software in general.
52:08 ELLY: Yeah, of course.
52:08 SHARON: Is there anything you would like to give a shout out? Normally, we shout out a specific Slack channel. I think in this case, the Slack in general is the shout out. Anything else?
52:20 ELLY: The Slack, in general, definitely deserves it. Honestly, I‘m going to go a little bit larger scale here. I’m going to shout out all of the folks who have contributed to Chromium, both at Google and elsewhere. It is the work of many hands. And it would not be what it is without the contributions from the folks at Google, the folks at Microsoft, folks at Yandex, folks at Naver. All of these different browsers and projects and all of the different individuals that have contributed, like everyone in the AUTHORS file - so shout out to all of those folks. And also, I really want to shout out the open-source projects not even part of Chromium that we use and rely on every day. So for example, we use LLVM, which is a separate open-source project for our compilation toolchain. And I think I would not be exaggerating to say that Chromium couldn‘t exist in its current form without the efforts of a bunch of other open-source projects that we’re making use of. And so I‘m really hopeful and optimistic that Chromium can live up to that. We’re standing on the shoulders of a lot of other open-source projects to build the thing that we‘ve built. And I’m hopeful that, in turn, other projects are going to stand on our shoulders to build yet cooler stuff and yet - yet better programs and build a yet better open-source community. So shout out to all of the authors of all the open-source software that Chromium uses, which is a lot of people. But they deserve it.
53:37 SHARON: Yeah, for sure. It‘s very cool how it’s very - all very related. And even within Chrome, I think people stick around longer than typical other projects. And it‘s cool to see people around, like a decent number of them, from before Chrome launched. And that’s probably [INAUDIBLE] to a generally more positive engineering culture. So that's very good.
53:58 ELLY: I think so. But I'm biased, of course.
53:58 SHARON: Yeah, maybe. [LAUGHS] Cool. You mentioned mailing lists a bunch. Any favorites that you have?
54:08 ELLY: Oh, yeah. chromium-dev is the mailing list of my heart, I would say. It‘s the main open-source development mailing list for us. It’s a great place for all of your newbie questions. If you‘re just like, how the heck do I even check out the source, that’s a good place to ask. The topic-specific mailing lists, especially net-dev and security-dev, are really good if you have questions in those specific areas. But honestly, all of the mailing lists on chromium.org are good. I haven‘t yet encountered one where I’m like, that mailing list is bad. So check them all out.
54:33 SHARON: Cool. All right. Check out every single mailing list. Sounds good.
54:38 ELLY: Yeah, every mailing list, every Slack channel.
54:38 SHARON: All right. Great.
54:38 ELLY: You're all good.
54:38 SHARON: Every Slack channel, I think - yeah, I'll add myself to the rest of them. All right. Well, thank you very much, Elly.
54:45 ELLY: Of course.
54:45 SHARON: Thank you for chatting with us. And see you all next time.
54:51 ELLY: All right. Thank you, Sharon. Easter egg - in the second part of this video, Elly is drinking soda.
