Caves, Clusters, and Weak Ties: The Six Degrees World of Inventors – understanding small-world networks

Your company’s scientists and investors can be antennas that bring great ideas into your company. The key, says HBS professor Lee Fleming, is understanding small-world networks.

by Sara Grant, HBS Working Knowledge

http://hbswk.hbs.edu/item.jhtml?id=4516&t=innovation&nl=y

Six degrees of separation seems to work well for B-list actors—but does it have anything to say about innovation and business?

HBS associate professor Lee Fleming believes it does, and his work looks specifically at how ideas and innovation flow across company boundaries through small (and getting smaller) communities and collaborations of inventors. Fleming and his colleagues found, for example, that at the end of the last decade, half of the patented inventors in Silicon Valley could trace an indirect collaborative path to one another. Can you say Kevin Bacon?

"Our work and more recent work on knowledge diffusion demonstrates that knowledge flows along these collaborative relationships, even years after they were formed," says Fleming. At the same time, the world of inventors "is getting smaller," he says, "inventors are more connected to their colleagues in outside firms, and that knowledge is diffusing in both directions."

One implication for executives: Don’t lock your scientists, researchers, and inventors in ivory towers. Your organization will benefit by the knowledge that flows to them from outside your company. Instead of asking ‘How can I make my R&D lab more productive?’ Fleming says a better question to ponder is, ‘How can I commercialize from this web of connected engineers?’

Sara Grant: Can you describe your latest research for our readers regarding small worlds and innovation.

Lee Fleming: The concept of small worlds started in the 1960s, with research done here in Boston. Stanley Milgram, the renowned social psychologist, randomly chose people from phone books out of Kansas and Nebraska and asked them to forward a letter to a friend of his in Boston through personal acquaintances. He found that—conditional on the letter actually getting to his friend—there were six referrals. This idea has passed into popular culture and urban folklore as the six degrees of separation.

That’s where it lay until a decade ago when a few researchers out of the Santa Fe Institute formalized the model by developing mathematical definitions of a small world. In layman’s terms, the best way to think about a small-world network is that there are local clusters, much like caves. Within these caves you have a small clan of people tightly connected together, with strong relationships between each other. But a couple of members in each cave—if you have a collection of caves within reach of each other—know other members of other caves, and it’s this combination of a tight, local clustering with an occasional weak, distant tie to other clusters, that is the essence of a small world. Some argue that clustering is good for innovation, but since clusters get stale, you also need the non-redundant information that flows in from outside the local cluster. Hence, the advantage of small worlds is that they combine tight local clustering with distant ties.

My colleagues and I study networks and innovation, and were a little skeptical about these ideas. We gathered thirty years of U.S. patent data and wrote up some code that identifies the inventors and links them to each other for three million patents and two million inventors. Our first cut was to look at the relationships across different regions, for example, Boston and Silicon Valley. We focused our research on these two regions because they’re both technologically dynamic and because of previous research on the influence of networks on regional innovation.

More on this article
Boston Inventor Networks mid-1990s http://hbswk.hbs.edu/item_sidebar.jhtml?id=4516

As we expected, a small-world structure is no panacea for innovation. We’re teasing out the econometric details right now in our research, but it appears that the distant ties are much more important than the clustering. Clustering has a negative influence, unless each cluster contains people with a variety of backgrounds. However, in addition to testing the small worlds idea, we found something more interesting: inventors in regions are becoming more connected. By the end of the ’90s, half of the patented inventors in Silicon Valley could trace an indirect collaborative path to one another. For example, if I co-authored a patent with you and you co-authored another patent with someone else, then your friend and I would be indirectly connected through you. This process cascades to the point where smaller clusters link up into larger clusters and so on, and eventually you have a black hole that sucks in thousands of inventors into a single massive network. Thus begins this process of "agglomeration."

sidebar http://hbswk.hbs.edu/item.jhtml?id=4516&t=innovation&nl=y

Agglomeration also happened here in Boston (the illustration shows the beginning of Boston’s agglomeration in the mid-1990s), but it lagged, mainly because DEC (Digital Equipment Corporation) was falling apart. And so with the largest component failing, inventors and local universities like MIT could not coalesce around DEC, as they did out in Silicon Valley with IBM and Stanford.

Perhaps most importantly, our work and more recent work on knowledge diffusion demonstrates that knowledge flows along these collaborative relationships, even years after they were formed. For managers the critical message here is, the world of inventors is getting "smaller" in the sense that inventors are more connected to their colleagues in outside firms, and that knowledge is diffusing in both directions. So with regard to agglomeration and connection, the world of inventors is indeed shrinking and becoming a "smaller world."

Q: How do managers react to this?

A: Interestingly, their preliminary reaction has been bi-modal. The inventors and the inventors who have become managers tend to think the connectedness is interesting and fun, and they agree that information flows across these links—that that’s the lifeblood of research and creativity. At the other extreme, we found two CEO managers in Silicon Valley who were vehemently opposed to these ideas and complained about how they hired their scientists and trained them for a year and then they put them in silos so they wouldn’t lose them to outside job opportunities. This reaction strikes us as a little short-sighted—and back-handed evidence that these flows really exist. I think you have to acknowledge the fact that information has always flowed across firm boundaries. It’s not an old argument, and the visual and econometric evidence now backs it up.

But it moves the question from how to develop your isolated internal R&D function to how to manage a collection of transient professionals who are constantly communicating and moving. Instead of hiring someone fresh out of school and putting her to work in your lab and expecting her to spend her career there, you now hire more mid-career professionals who bring a deeper, richer human and social capital to your firm. It changes the question of how do I make my R&D lab more productive to how can I commercialize from this web of connected engineers?

You’re enmeshed within this wash of information, and your inventors create and they help others create and others help them create. Then the challenge becomes, which of these streams of invention do you decide to commercialize and how do you construct proprietary advantage around something which, to start with, is very much out there and, to varying degrees, not really proprietary.

That’s one extreme. Obviously a lot of firms still try and erect the walls, but I think in general the network data show, and other research has shown, that things are getting more open and spilling more quickly across boundaries. It seems that with all the network’s and technology we have now, it’s easier to communicate and people are free to do it more, and they do. And our visual patent networks are just one example of the many relationships we build and maintain. There are scientific, personal, community, church, educational, and many other networks.

Q: Sounds like technology makes it harder for managers to actually put the clamps on this spillage.

A: It does. And other movements as well, like the open source movement, where the entire idea of proprietary invention is anathema. It’s not unlike the norms of science. It’s an ironic and fascinating time to study technology, science, and invention. In some ways the processes of invention in firms are moving towards an open science model. For example, Novartis and the Broad Institute just announced that their collaborative research will be placed in the public domain. At the same time, however, you also have universities now moving to a more closed model, whereas in the past universities used to do the open science. So many different cross-currents occur, and you end up with a fertile, exciting, and messy web.

Q: Do managers have a fear of losing knowledge management in the organization? Maybe managers want to exert more control over that area.

A: Oh, absolutely. It’s a double-edged sword. How you prosper within a geographical region, such as Silicon Valley and, I’m sure, Boston, is a challenge for managers. I’ve become less sure recently about the arguments that Boston is less networked. If you travel to Kendall Square, there are so many buildings going up, the interaction, the collaborative atmosphere. You just can’t walk through [Kendall Square] without feeling the excitement. I can’t imagine that Boston has not become a more open and networked region recently.

Q: You point to the importance of location. For example, Silicon Valley was perhaps more open to information flows than, say, Boston or Philadelphia. Why is location key to the small-world network? Is it just helpful for the flow?
It’s this combination of a tight, local clustering with an occasional weak, distant tie to other clusters, that is the essence of a small world.

A: Well, it has to do with job movement, at least in the data we have. When inventors move across organizations, they take ties they have with previous organizations and carry them to the new one. Cultural and historical reasons also exist for why one region would be more or less open to information flows. There are also legal issues. For instance, Massachusetts has non-compete agreements where you can’t share information or work on something that you used to work on in the old firm; whereas in California, they don’t exist. You can debate on whether or not they’re enforceable in Massachusetts—quite often they’re not. But in some states, you just don’t have them at all. So that factors into a manager’s practice and thinking. Policymakers—like Massachusetts’ governor Mitt Romney—are also beginning to question the logic of a strong non-compete regime.

I’ve worked as an engineer in Silicon Valley for seven years and I don’t think I ever heard of anyone mentioning non-competes. It was just assumed that you had your intellectual capital and if you weren’t happy where you were, or if your employer wasn’t compensating you enough, or if they didn’t sponsor you creatively, then you’d go somewhere else for a better deal. That too can lead to more information and knowledge flow and creativity. I strongly suspect it’s better for the region’s overall innovative productivity. But it also means risk—and opportunity—for firms in the region.

For example, if you don’t have some complementary assets, such as better manufacturing, marketing, or product development, you are going to be at a severe disadvantage when competing in the "fast lane." I would argue, though, that it’s not good for a firm to simply close its walls and say, we’re not going to share, we’re not going to get our ideas taken away, because then its funnel of new ideas will dry up.

Q: So how can managers foster these networks to their advantage?

A: Well, there are two approaches that I would categorize as trust or litigate. Since I am an engineer and not a lawyer, I fall into the trust camp, but there are obviously minimal legal defenses that you absolutely have to build. If things become overly litigious, however, it really mucks up the information flow and a technical professional’s productivity. Ideally, you need to make your technical staff aware of the tradeoffs and the technical/competitive landscape without building barriers to creativity.

One thing is to appoint a technical gatekeeper that will keep close watch on the technical activities of potential competitors. The gatekeeper then needs to make her colleagues aware of who else is playing (or planning to play) in the same market. In contrast, many firms working on similar technologies do not compete and have no plans to compete. Information flow with inventors from those firms can be very helpful on both sides—not unlike an informal R&D alliance. As an added nuance, however, you need to be aware that the firms that are not direct competitors might be indirectly tied to your direct competitors, so it gets tricky. Simple awareness of the shrinking world of inventors is the first step. Set up unobtrusive legal protection, make your technical professionals aware of the issues, and then trust them to manage their knowledge sharing.