Responsibilities

- Design, build, and optimize scalable data pipelines using Azure Data Factory and other Azure services to ensure seamless data flow and transformation.

- Develop and manage data storage solutions with Azure Data Lake Storage and Blob Storage, ensuring high availability and data security.

- Collaborate with cross-functional teams to define data architecture and integrate data from diverse sources, including RESTful APIs and third-party platforms.

- Optimize ETL/ELT processes to support data analysis and reporting needs, ensuring high performance and cost-efficiency.

- Implement and maintain data quality processes, including monitoring via Azure Monitor to identify and address pipeline errors or bottlenecks.

- Design and develop Azure solutions for advanced data warehousing and reporting.

- Mentor junior data engineers in Azure best practices, coding standards, and process optimization to foster team growth and productivity.

- Stay updated on emerging trends in Azure technologies and incorporate them into the company’s data engineering practices.

Qualifications

- Experience: 5–7 years in data engineering roles, with at least 3 years of hands-on experience in the Azure ecosystem.

- Skills:

- Expertise in Azure Data Factory, Azure Data Lake Storage, and Blob Storage.

- Proficiency in SQL and programming languages like Python, PySpark, or Scala.

- Strong understanding of data integration techniques, including working with REST APIs.

- Knowledge of monitoring and optimization tools like Azure Monitor and Azure Logic Apps.

- Education: Bachelor’s degree in computer science or a related field. Master’s degree preferred.

- Certifications: Azure-specific certifications like Microsoft Certified: Azure Data Engineer Associate or Azure Solutions Architect Expert are highly valued.

- Soft Skills: Excellent problem-solving abilities, strong communication skills, and a collaborative mindset.