Data Engineer

Overview

The position provides the opportunity to engage in cutting-edge research in a highly-collaborative environment that feels more like a startup than a big company. You will contribute to developing products that have an impact on millions of learners around the globe.


Responsibilities:

  • Implementing scalable, automated ETL processes
  • Maintaining, administering and improving current ETL processes
  • Integration of the data coming from a variety of sources
  • Solving engineering problems
  • Designing solutions for new business requirements
  • Creating and conducting internal training
  • Building continuous knowledge and know-how improvement culture

We work in agile teams in which members can wear many hats, so you’ll have the opportunity to work on different project aspects.


Skills

  • Programming skills, especially Python
  • Awareness of best programming practices
  • Experience with versioning system (git)
  • Experience with UNIX systems
  • Knowledge about ETL processes
  • Knowledge about AWS Services
  • Knowledge about cloud computing / architecture
  • Analytical thinking
  • Willingness to learn 

Tools

  • Python
  • Spark
  • Amazon Web Services (S3, Lambda,GLUE, Redshift, Athena, Step Functions)
  • Databricks
  • GIT/Bitbucket
  • Snowflake

Example projects


  • Fully automated system for data transforming

  • Applications utilized for testing of data quality

  • Processes preparing data structure for business reporting for many various e-learning platforms:

    • How often students solve the tests

    • What results they achieve

    • How often students pass the exams

    • From where over the world students are most engagement