Job Summary:

We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. The ideal candidate will have extensive experience working with Azure and Databricks, and will play a crucial role in our Copilot/AI rollout. This role involves designing, implementing, and optimizing data pipelines and architectures to support our AI initiatives.

Key Responsibilities:

• Design and Develop Data Pipelines: Create and maintain scalable data pipelines using Azure Data Factory, Databricks, and other Azure services.
• Data Integration: Integrate data from various sources, ensuring data quality and consistency.
• Optimize Data Workflows: Implement best practices for data processing and storage, optimizing performance and cost.
• Collaborate with AI Teams: Work closely with AI and machine learning teams to understand data requirements and deliver high-quality data solutions.
• Monitor and Troubleshoot: Monitor data pipelines for performance and reliability, troubleshooting issues as they arise.
• Documentation: Maintain comprehensive documentation of data workflows, processes, and systems.
• Mentorship: Provide guidance and mentorship to junior data engineers and other team members.

Qualifications:

• Education: Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
• Experience: 5+ years of experience in data engineering, with a focus on Azure and Databricks.
• Technical Skills:
• Proficiency in Azure Data Factory, Databricks, and other Azure services.
• Strong programming skills in Python, SQL, and Spark.
• Experience with data modeling, ETL processes, and data warehousing.
• Familiarity with CI/CD pipelines and version control systems (e.g., Git).

Soft Skills:

• Excellent problem-solving and analytical skills.
• Strong communication and collaboration abilities.
• Ability to work in a fast-paced, dynamic environment.

PLEASE NOTE I AM ONLY LOOKING TO SPEAK TO US CITIZENS AND GREEN CAR HOLDERS - NO THIRD PARTIES!