Sr Big Data Engineer
8+ yrs IT experience and good expertise in SDLC/Agile
6+ yrs in programing language (Python, Scala, Spark)
Mandatory:
Hands on experience in writing advanced SQL queries, familiarity with variety of database
Tools: - Big-Data, Spark, Python, Scala
Experience in building and optimizing 'Big-Data' pipelines, architecture, and data sets
Experience in Hadoop file formats like ORC, Avro, Parquet, CSV
Cloud platform: AWS, Snowflake
Hands on experience in AWS Bigdata env
Desired:
Kafka streaming and Shell scripting
Scheduling tools like Control-M, Oozie
NoSQL databases like MongoDB
Implement Python flex APIs to share data insights to digital systems