A thought on innovation: industry vs. academia

When I was a graduate student in Computer Science at Stanford, I spent a lot of time struggling over whether I wanted to pursue a career in academia or the software industry.  I was torn because academia seemed like a place where you could pursue great ideas and research without concern for market acceptance or financial viability.  That, of course, was also precisely academia's problem.  The things you did in academia often never saw the light of day in the "real world" and were relegated to a library shelf (or, these days, a server) somewhere.  On the flip side, the software industry never seemed to tackle the "big" technical problems of society because the market wasn't there, or at least not readily apparent.

I was reminded of this conflict a few months ago when I attended Stanford's 4th Annual Media X Conference, which is a conference that brings together researchers and professors from across the University with representatives from industry.  The idea is to promote discussion and collaboration, with the hope that corporations adopt and/or collaborate on research projects underway at the university.  Conceivably, industry could even "charter" research projects that the faculty would carry out on their behalf.

This is an interesting idea, and I'm certainly rooting for its success, especially because a friend of mine, Ellen Levy, runs the Media X program.  But, as I sat through the rapid-fire presentations of the professors, and talked with them during the breakout sessions, I couldn't help but think this idea is destined to fail.  Unless the university (and all universities) change one fatal flaw in their approach to solving big problems: collaboration.   Where people inside corporations--at least well-run ones--tend to have one common goal (the short and long-term success of the corporation), people inside of research universities all have individual goals: the success of their individual research so that they can get funding to do more research, get their books published, and build up their personal academic "brand."  Please don't get me wrong--I'm not suggesting in any way that the motivation of academic researchers is disingenuine or selfish.  Almost universally, academics have chosen their craft because they are trying to contribute to the great body of social knowledge in a pure form. However, they mostly seem to be living on individual islands, and are at pains to point out why what they are doing is different from what all of their colleagues are doing.  This simply is not a good way to make great progress.  If there are two smart people working on a problem, I believe they will always solve it better if they work together.  Three is even better than two.  Now there may be some limit (five or ten?) but these people need to be peers, able to challenge each others' ideas

My mother and my twin brother (and several close friends) are research professors at major universities, so you'd think I'd be headed for an earful from them all after this post.  However, I think they all agree with my aha moment, as they have all expressed simimlar frustrations with academia.  Now, there are plenty of problems with the way the private sector solves big problems--namely, that they define "big" by the market size instead of the severity to society (think global warming, poverty, illiteracy, etc.).  That's why we need academia to be effective at solving these big problems.  Programs like MediaX do a service to the private sector by teaching us what's happening in the research labs  of America's finest universities.   But they could learn something from the private sector as well, namely that it's hard to turn a theory into a world-changing technology or product without an environment that better rewards team thinking and behavior.   More focus on the problem, less on the problem solver.

Comments

[eric butler (moonglum01)]

"If there are two smart people working on a problem, I believe they will always solve it better if they work together."

....and (thinking out loud) what if one of the smart people is at Microsoft and one is at Apple and both are trying to come up with the next great application?

Personally, I would argue that academia, in reality (despite all semblances to the contrary) does collaborate to a much greater extent than private industry. As you know, every good piece of research must be founded on a thorough review of current theory and practice. By their very nature academic researchers, well... research. To say that there is less direct collaboration may be true, but, less collaboration does not mean a lower awareness to what is going on in their field of study.

I think history has demonstrated that, in the long run, chaotic, market driven knowledge supplemented with academic research has produced a much more relevant body of knowledge than a command driven (Soviet) model of innovation.

----

Another thought on, "If there are two smart people working on a problem, I believe they will always solve it better if they work together."

For me, this is 100% true - I work best as part of a team. That said, I work with people who are brilliant and do great work when left alone, but are awful as part of a team. Put 10 academic researchers together in a room - you don't get collaboration, you get the most petty of petty bickering.

Academics - not right or wrong, just, well... different.

[Miami Office Space]

yes totally agree. the problems of academia is a place where you could pursue great ideas and research without concern for market acceptance or financial viability
nice blog, I'm a huge fan