Unlocking Trapped Engineers

Millions of software engineers are caged. Their talent remains unseen behind the iron bars of the misconception of a shortage of software engineers. An estimated 1 million technology jobs will go unfilled by 2020, according to a report put forth by Microsoft in 2012. Publications like The Wall Street Journal and even the White House repeatedly cite this projection, ringing a nationwide alarm of a severe shortage in technical talent.

This alarm comes with good intentions, aiming to nurture more engineers through education and immigration reform. Political agendas aside, it’s worth scrutinizing the basis of this projection.

Using statistics from the Bureau of Labor Statistics (BLS), it’s projected that 1.4 million positions will be open in computing with only 400,000 computer science grads. Hence, there will be a shortage of 1 million programmers.

Many citations incorrectly say that BLS projects the shortage when, in actuality, BLS economist Jennifer Chi clarified in an email that all of these attempts are made by third parties. “[It is] an incorrect comparison of the total employment and total labor force projections, which are two separate and fundamentally different measures.”

This projection does not account for the vast number of non-traditional programmers who are self-taught, never graduated, dropped out or simply have a different degree. How can we base a talent shortage on CS degree production when 59.8 percent of those with software engineer, programmer or computer scientist titles in one study did not carry a CS degree; 36 percent of IT workers do not hold a college degree at all, according to the Economic Policy Institute; 40 percent of programmers on Stack Overflow, one of the largest developer websites, are self-taught; or 14 percent of the members of some teams at Google alone don’t have a college degrees?

A study looked at patterns between 10,000 resumes and job performance over 14 years, and found that there is simply no correlation between having a degree and being a good software engineer. How can we claim there’s an enormous mismatch of supply and demand when a significant portion of engineers isn’t accounted for in the infamous labor projection?

Ironically, the outcry of an engineering talent shortage is indirectly reinforcing the problem. Despite the clear evidence that proves otherwise, we’re perpetuating the false notion that there’s only one entry into the world of programming.

There are many great engineers out there. But the unprecedented speed at which software technology moves means by the time you need more engineers, your systems are already strained. There’s no time to validate hundreds of applicants blindly. One wrong engineer could change the whole equation.

So you use blunt instruments, like prestigious alma maters and CS degrees, as a proxy for talent, potentially overlooking an endless supply of engineers who don’t fit the profile — and thus the perception of a talent shortage. Software leaders’ deep-seated fear of hiring anyone that will slow them down is systematically locking out millions of skilled engineers who lack a high pedigree while forging a so-called talent shortage.

Unlocking raw skill

Some will say that market economics tells us that programmers wouldn’t have notoriously high salaries if there weren’t a skills shortage. But closer examination of salary trends for software engineers tells us their overall average salaries haven’t increased more than half a percent per year, according to the Economic Policy Institute.

But there’s one exception that’s often amplified in the media. Extraordinary engineers, who sit at the top of the bell curve of talent, are rumored to earn millions. But this concept is nothing new or unique to engineering. Just like any other profession or creative skill, there’s a limit to how many people are the best-of-the-best.

The economic modeling specialists looked at median earnings for programmers and computer software engineers from 2000-2010 using the BLS’s Current Population Survey database, and found little growth in the last decade:

It’s a curious case. On the one hand, just about all executives say they struggle to find the technical talent. Meanwhile, computer science programs have been exploding with student applicants at an unprecedented rate over the past few years.

In a previous piece on how brick-and-mortar institutions are unable to keep up with the pace of technology, we found that programs actually have to turn away students:

Plus, oddly enough, unemployment has been relatively high for grads in the field (engineering: 7 percent; computer science: 7.8 percent; information systems: 11.7 percent). Wouldn’t you expect these numbers to be much lower in a field suffering from a so-called talent shortage?

And it’s not just new grads. If there’s a shortage in engineering talent, then why has the unemployment rate been rising for software engineers overall? It grew from 2 percent in 2014 to 3.2 percent in 2015. This is interesting, considering the average unemployment rate for college-educated people usually hovers around 2 percent.

Unraveling the misconception

Good engineers are plentiful. But there’s a palpable misconception about what credentials a “real” programmer should have next to their name. Programmers who took alternative paths to engineering are overshadowed behind engineers with CS degrees.

Jacob Kaplan-Moss, a contributor of the web framework Django, recently met a student at Kansas University who did a great presentation on predicting seasonal floods using flow analysis with tools like Python, Linux and Django. When Kaplan-Moss was looking to hire Python programmers, he naturally approached her. Despite having just written thousands of lines of Python while inventing an original cluster distributed geographic pipeline, she said: “Oh, no I can’t do that. I’m not really a ‘programmer.’”

If even an accomplished, impressive researcher feels a barrier to enter the field, there’s clearly something wrong with the perception of engineers. Rather than focusing on what programmers do, there’s a persistent myth about what programming must be like. How do we unlock skilled programmers, coming from nontraditional, but equally legitimate, paths, into our field?

The onus is on companies

Given that it can collectively take well over 30 days to hire a single engineer, the obsession with computer science degrees makes sense. Why go through hiring cycles to vet engineers blindly?

So this is what it all boils down to: This pressure to hire engineers fast is baked into the DNA of companies. Driven by an ingrained fear of slowing progress, engineering managers’ use of blunt instruments to save time in finding talent is perpetuating the facade of a talent shortage.

But considering that a major chunk of programmers don’t have CS degrees, and degrees are no predictor of success, companies should stop relying on resumes to vet engineering candidates. Instead, they should ask candidates to prove their programming skills. Benchmarking talent by pure programming skills would unlock hundreds of thousands of engineers.

While everyone else keeps amplifying their budget to play tug-of-war with the same few engineers, smart engineering managers have a major advantage by focusing efforts on seeking out, testing and hiring large subsets of non-traditional engineers who are less sought-after.

One million programming jobs may go unfilled by 2020, but only if companies aren’t willing to pause and rethink the barriers that are locking out programmers who entered the field without a CS degree.