Senior Big Data Developer - $130-150k - Hybrid - Pheonix AZ, New York, Fort Lauderdale.

Role Overview: We are seeking a skilled Senior Big Data Developer to join our team. The ideal candidate will bring extensive experience in big data processing and analysis, with at least 3 years of hands-on expertise in Apache Spark. You will be responsible for designing, implementing, and optimizing data pipelines and applications that handle large-scale data in real-time and batch processing environments.

Key Responsibilities:

• Data Pipeline Development: Design, develop, and maintain large-scale, distributed data pipelines using Apache Spark.
• Performance Optimization: Implement best practices for optimizing Spark jobs for performance and scalability.
• Integration: Work with diverse data sources, including HDFS, NoSQL databases, relational databases, and cloud storage.
• Collaboration: Partner with data scientists, analysts, and stakeholders to understand requirements and deliver data solutions.
• Real-time Processing: Develop real-time data streaming applications using Spark Streaming or similar technologies.
• Code Quality: Write clean, maintainable, and reusable code, following best practices for version control, testing, and documentation.
• Troubleshooting: Identify and resolve issues in Spark jobs and data pipelines.

Required Skills and Qualifications:

Experience:

• 8 to 12 years in software development, with a focus on big data solutions.
• 3+ years of hands-on experience in Apache Spark (batch and streaming).

Technical Skills:

• Proficient in programming languages such as Scala, Python, or Java.
• Strong understanding of distributed computing principles.
• Experience with big data ecosystems (Hadoop, HDFS, Hive, Kafka).
• Familiarity with cloud platforms like AWS, Azure, or GCP.
• Proficient in SQL and experience with relational databases.

Tools and Frameworks:

• Version control systems (Git, GitHub, Bitbucket).
• CI/CD pipelines and DevOps practices.

Soft Skills:

• Excellent problem-solving and analytical skills.
• Strong communication and teamwork abilities.

Others:

• Knowledge of machine learning frameworks and integration with Spark (e.g., MLlib).
• Experience with containerization and orchestration tools (Docker, Kubernetes).
• Certifications in Big Data or Cloud technologies.
• Experience in developing solutions for Payment Networks and Payment or Financial transaction processing.