Data Engineer

Engineering at CrunchBase

Our Mission

It’s nearly impossible to find an authoritative source of truth about companies that is both always up-to-date and freely available. Our mission is to continue building the largest high-quality, live, openly editable and accessible dataset of company and people information in the world -- and to build a delightful product with slick, easy-to-use APIs around that dataset.

We love building enjoyable user experiences, complex data schemata, APIs, and scalable systems that support tens of millions of interactions per day. Our team is a group of highly skilled engineers working on formidable problems throughout the stack.

How We Operate

We have a flexible model for team organization at CrunchBase, centered around the principles of agile software development, allowing us to fluidly adapt to new requirements and situations as they arise. We tend towards two main types of teams:

• Feature-oriented teams:operate independently and own their projects from cradle to grave, building everything they need to execute. While they don't necessarily have to do all the work themselves, it's their responsibility to make sure the business needs are met all the way into production.

• Function-oriented teams: provide support to feature teams and have deeper expertise on particular parts of the stack. It's their responsibility to build platforms that are so good the other teams want to use them, rather than being forced into it.

We also ship code as early as we possibly can; getting betas into customer hands is a top priority. Our goal is to tune our process to customer needs as efficiently and quickly as possible. CrunchBase is built with a variety of tools - a core part of our engineering philosophy is to use the right tool for the job:

• Ruby, Scala, Java

• Node.js, Angular.js, Typescript

• Redis, PostgreSQL, ElasticSearch

• Git, Nginx, Docker, NewRelic, Jenkins

• AWS - EC2, ELB, VPC, S3, R53, RDS, SQS, etc.

Data Engineering at CrunchBase

Our data team is responsible for building and maintaining the infrastructure for our data needs. As CrunchBase grows, so too does the amount of data we process, the number of sources it comes from, and the number of ways that people want to slice and dice it. We are currently building out a robust pipeline for our core data and we’re continually expanding use cases for it, so it must be both scalable and flexible.

At its core, CrunchBase is a data company, and data engineering is at the heart of our platform and will propel us into the future. The responsibilities of data engineers at CrunchBase include:

• Architect and build new dimensional data models and schema designs to improve accessibility, efficiency, consistency and quality of both internal and production data.

• Build, monitor, and maintain analytics and production data ETL pipelines.

• Provide the foundation for a data-driven culture by empowering other engineers and the Product team to ask questions of the dataset in an easy, reliable way.

• Enable data scientists to implement NLP and ML algorithms at scale, in fault-tolerant, highly available systems.

Qualifications

• Solid understanding of computer science and software engineering fundamentals.

• Motivated to participate in ongoing learning and growth through pair programming, code reviews, application of new technologies and best practices.

• Excellent verbal and written communication skills.

• Familiarity with tools we presently use is a plus, but not required -- if you know something better, we may use that instead!

What CrunchBase offers

• Game changing opportunities to revolutionize the way people interact with companies.

• Competitive salary and significant equity in a well-funded, high-growth company.

• Daily catered lunch and free drinks with healthy snacks.

• Unlimited Paid Time Off (PTO).

• Full medical, dental, and vision coverage.

• Free One Medical Group membership.

• 401(k) plan.

• Free access to TechCrunch conferences & events.

• Comped Uber rides home on nights and weekends.

• Prime location in the SoMa district of SF, near CalTrain, BART and Muni stops.

• A collaborative team of creative, bright coworkers.