Big Data Engineer - Boston, MA

data-plus-math-loho

We are looking for a savvy Big Data Engineer to join our world class team of analytics experts.

The right candidate will be excited by the prospect of optimizing or even re-designing our company’s data architecture to support our next generation of products and data initiatives.

The Senior Big Data Engineer will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. 

You will support our software developers, database architects, data analysts and data scientists and will ensure optimal data delivery architecture is consistent throughout ongoing projects. You must be self-directed and comfortable supporting the data needs of multiple teams, systems and products. 

 

Responsibilities:

  • Create and maintain optimal data pipeline architecture

  • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

  • Build the infrastructure required for optimal ingestion, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.

  • Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.

  • Be a member of our agile team and must contribute to daily standups, planning, and sprint retrospectives

  • Be a pillar of technological knowledge and lead by example in code, feature development, and documentation.

Requirements:

  • 5+ years of experience in a Data Engineering role

  • Bachelor’s degree or higher, preferably, with a concentration in a computational field such as Computer Science, Mathematics, Statistics, Physics, Engineering

  • Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.

  • Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.

  • Experience with big data tools: Hadoop, Spark, Kafka, etc.

  • Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.

  • Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.

  • Experience with AWS cloud services: EC2, EMR, RDS, Redshift

  • Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.

  • Strong analytic skills related to working with unstructured datasets.

  • Hands on experience with Git (Version Control) and UML Diagramming

 

Traits

  • A desire to solve business problems with technology.

  • Great communication skills, and the ability to influence stakeholders.

  • Strong interpersonal skills and exceptional character

  • Interest, willingness and demonstrated ability to quickly pick up new technology quickly

  • A self-starter who brings energy, passion, and creativity to work every day

 

Sorry, Data Plus Math is unable to offer work/visa sponsorship at this time.

Apply Below!