Data Engineer in New York, NY at Open Systems Technologies

Date Posted: 11/3/2019

Job Snapshot

Job Description

An up and coming startup is seeking a strong Data Engineer. You will help design, build and maintain data feeds. 

Responsibilities

  • Design, develop and deploy ETL pipelines and web scraping projects
  • Work with large quantities of structured and unstructured data
  • Implement monitoring mechanisms for both streaming and batch jobs to ensure 99.999% uptime and to track data quality
  • Design and build infrastructure to support data

Qualifications

  • BS or MS degree in Computer Science, Math, Statistics or other technical field
  • 2-3+ years of applied software engineering experience (especially in startups, finance or adtech)
  • Must have expertise in the below technologies
    • Python
    • ETL
    • Scala
    • Pandas, Pyspark
  • Experience deploying code in a cloud environment - AWS preferred
  • Experience with various storage technologies
Job category:
  • Information Technology
Job keywords:
  • Python
  • ETL
  • Data
  • Scala