Job Description: Title - Sr Python Data Engineer Location - Charlotte, NC / Iselin, NJ (Hybrid) Duration- Contract to Hire Responsibilities & Requirements: We are seeking an experienced Python Data Engineer to join our team for a critical platform migration project. The ideal candidate will be responsible for developing, testing, and optimizing Python applications to ensure smooth migration and integration with the platform. This role requires strong expertise in Python, microservices, and integration with various data storage solutions.
Key Responsibilities: - Develop and maintain Python applications and microservices for platform migration.
- Design and develop robust Data processes using Python and PySpark to extract, transform, and load data from various sources.
- Implement best practices for data management, including data governance and security.
- Optimize data pipelines for performance, scalability, and reliability.
- Rewire existing Python microservices to connect with the cluster and S3 storage.
- Collaborate with cross-functional teams to assess and implement compatibility with the environment.
- Update and optimize Python code to ensure compatibility with new data formats and cloud computer environments.
- Implement and support CI/CD pipelines, including automated testing and deployment processes.
- Troubleshoot and resolve issues related to Python applications and their integration with the platform.
- Monitor and troubleshoot ETL processes, ensuring timely data delivery and addressing any data quality issues.
- Contribute to document, including code comments and technical guides, to support future development and maintenance.
Qualifications: - Proven experience as a Python Developer, with a strong portfolio of relevant projects.
- knowledge of Python programming, including frameworks such as Django or Flask.
- Experience with microservices architecture and API development.
- Familiarity with data storage solutions like S3 and databases such as Oracle and Postgres.
- Understanding of CI/CD pipelines and automated testing tools.
- Strong problem-solving skills and the ability to work collaboratively in a team environment.
- Excellent communication skills and attention to detail.
Preferred Skills: - Lead complex technology initiatives
- Design, code, test, debug, and document for projects and programs
- Software Development experience
- Experience in Python along with SQL proficiency
- Experience in building and deploying microservices and cross channel restful Application Programming Interface (API).
- Experience with GCP services, BigQuery, DataProc, Apache Beam.
- Experience with secure DevOps and deployment automation to cloud environments
- Experience with Cloud technologies
- Experience in ReactJS, Python Frameworks, UML, NoSQL, MongoDB, other relational database management system (RDBMS), Kafka pub/sub model, ETL.
- Experience in Data analysis, Data Import and Data Feeds.