Data Engineer

Department: Operations

Location: Atlanta

MailChimp is the world’s largest marketing automation platform. More than 15 million people and businesses use MailChimp to design and send 1 billion emails a day. We empower small businesses with a suite of powerful and easy-to-use email, marketing automation, and analytics tools that integrate with hundreds of popular applications and services.

MailChimp’s Operations team is responsible for infrastructure and tooling, as well as provisioning, configuring, and monitoring the services in our datacenters. Team members work closely with our Development, Marketing, and Data teams to provide the infrastructure needed to move the company and our products forward.

We are seeking an experienced engineer who would be part of the Data Engineering team with a focus on maintaining and improving our pools of Elasticsearch clusters, in addition to further developing and optimizing our Kafka data pipeline. We take a pragmatic and practical approach to our stacks; we use proven components and build our own logic and complexity on top of well understood building blocks. We are growing fast and need people who can help advance our infrastructure to deal with the needs of today and tomorrow.

The ideal candidate will have a strong history of Linux experience, Operational Elasticsearch/Lucene/Solr experience, independent troubleshooting skills, and a love for automation and monitoring. Additional areas of experience in MySQL, Cassandra, Hadoop, Kafka and other stateful technologies are a big plus.

Responsibilities

  • Design, build, optimize, and support systems for storing, aggregating, and analyzing large amounts of data in Elasticsearch
  • Improve and maintain our data pipeline infrastructure on Kafka and help improve its performance and reliability
  • Work closely with internal teams to ensure that their current and future logging and reporting needs are met
  • Work with data science and delivery teams to support their customer-facing and research needs
  • Participate in on-call rotations with the Data Engineering team

Requirements

  • Operational experience with Elasticsearch, Lucene, and/or Solr, to include: query tuning, data recovery, monitoring, performance optimization, capacity planning, and schema design
  • Deep understanding of Lucene internals and data structures; experience designing and tuning IR/search queries and algorithms
  • JVM tuning and monitoring experience
  • Kernel tuning
  • CentOS support and admin experience
  • Scripting and coding (python, bash, php, go, ruby, java)
  • Building systems with configuration management tools
  • Familiarity with other infrastructure pieces of our stacks (nginx, mysql, redis, kafka)

Bonus points for

  • Data warehousing experience
  • Experience supporting both batch and real time analytics
  • Designing, working with, and supporting distributed data systems
  • Building and optimizing data pipelines built on Logstash, Flume, FluentD, Rsyslog, or similar log processing tools
  • Batch processing workflows and tools (Hive, Airflow, Luigi, etc)
  • Experience working with Kafka, RedisMQ, SQS, or other message queuing systems
  • Experience with Puppet profiles, roles, and other patterns needed for large scale configuration management implementations

MailChimp is a founder-owned, highly profitable, and private company located in the heart of Atlanta. We offer our 500+ employees an exceptional workplace, extremely competitive compensation, fully paid benefits (for employees and their families), and generous profit sharing. We hire humble, collaborative, and ambitious people, and give them endless opportunities to grow and succeed.

We love our hometown and support sustainable urban renewal. Our office is in the historic Ponce City Market, right on the Atlanta Beltline. If you'd like to be considered for this position, please apply below. We look forward to meeting you!

MailChimp is an equal opportunity employer, and we value diversity at our company. We don't discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.

Big and getting bigger

MailChimp has 15 million users who email 16 billion subscribers, with 12,000 new accounts created every day.

A home for square pegs

We don’t work like most tech companies, and we don’t look like them either. Our engineers have come to MailChimp from many different paths.

Benefits

We encourage our employees to live their best lives through wellness programs, education opportunities.