Middle Big Data/Python Engineer

  • Lviv, Ukraine
  • full employment

General

  • - 2+ years of commercial experience in Python programming
  • - 1+ years of commercial experience in AWS services
  • - Track record of working in Scrum / Agile software teams
  • - At least Intermediate written and spoken English

Responsibilities

  • - Data Wrangling
  • - ETL jobs development and support in AWS environment
  • - Data validation and Data quality report development, support, monitoring.
  • - Create plans, test files, and scripts for data unit/integration testing
  • - Create supporting documentation, such as metadata and diagrams of entity relationships, business processes, and process flow

 

Technical Skills

  • - Python OOP skills
  • - Understanding of python memory management
  • - Good knowledge in usage of Pandas, Numpy, Matplotlib, Boto3
  • - Experience accessing and retrieving data from large data sources (e.g. REST API, SFTP, MS SQL, MYSQL, ORACLE, POSTGRES)
  • - Agile development process

Soft Skills

  • - Self-starter with a willingness to become a data expert and to learn new skills
  • - Results oriented with the ability to multi-task and adjust priorities when necessary
  • - Ability to work independently or in a team environment with internal /external customers
  • - Ability to articulate and train technical concepts to a non-technical audience
  • - Strategic thinking and analytical capability
  • - Strong verbal/written communication skills.

 

WIll be a plus

  • - Spark/Pyspark experience
  • - AWS Redshift, Athena, Hive
  • - AWS EMR cluster management, configuration
  • - Hadoop ecosystem (YARN/Mesos, RACK Node, NAMED Node)
  • - DataWarehousing (design, implement - DIM, FACTS)
  • - Experience with Microservice architecture
  • - Experience with Docker
  • - Solid knowledge in SQL engines such as Presto, Redshift Spectrum, Hive
  • - Solid understanding of Data Transformation techniques in clusterized systems
  • - Experience in creating and tuning SQL Queries in clusterized/non-clusterized systems
  • - Experienced Linux administrator
  • - Experience with serverless services such as AWS lambda
  • - Experience with CI tools (such as Jenkins)
Apply Yourself Refer Someone Else