Platform Data Engineer - (DataBricks, PySpark, AWS)
Comcast
Kabelnetzbetreiber
45071 West Chester, OH - USA
Senior Fachkraft / Projektleiter
Experteer Overview
In this role you will design and optimize scalable data pipelines for enterprise datasets, enabling analytics and business decisions. You will work within the Data Product Engineering Team to evolve the enterprise Data Lake and support high-volume, near real-time data processing. Expect collaboration with cross-functional groups to deliver impactful data products using modern cloud and streaming technologies. This is a hands-on, performance-focused position with opportunities to mentor others and influence the data platform's long-term evolution.
Verantwortungsbereiche
- Design, develop, and optimize scalable data pipelines for workforce, billing, interaction, and other datasets
- Build cloud-native data solutions using AWS, Databricks, and PySpark
- Develop batch and streaming data processing frameworks integrating source systems
- Leverage Kafka and Databricks streaming for near real-time processing
- Drive performance tuning, automation, and operational improvements across the platform
- Provide production support, troubleshooting, and root-cause analysis for critical workflows
- Work with large-scale distributed systems and tens of TB data
- Use MWAA to orchestrate data workflows
- Collaborate with Product, Data Governance, Analytics, and Engineering across onshore/offshore models
- Support data warehousing initiatives and promote data quality, scalability, reliability
- Mentor junior engineers and contribute to team growth
- Participate in architectural discussions for long-term platform evolution
Zentrale Anforderungen
- Bachelor's degree or equivalent practical experience in a related field
- 5+ years of Data Engineering or related experience
- Strong hands-on with AWS, PySpark, Databricks
- Experience building and maintaining large-scale ETL/ELT pipelines
- Knowledge of distributed systems and large-volume data processing
- Experience with data warehousing concepts and modern data architectures
- Experience with Apache Airflow/MWAA
- Kubernetes fundamentals (pod lifecycle, job orchestration)
- Python development for data engineering
- Experience supporting production data platforms and driving operational excellence
- Strong communication and cross-team collaboration skills
Stellenbeschreibung
In this role you will design and optimize scalable data pipelines for enterprise datasets, enabling analytics and business decisions. You wi…
Leading careersFor members only
Take your next career step
1M+ top positions worldwide with salary benchmarks
Be discreetly found and contacted by headhunters
Exclusively for senior-level professionals and executives
Already a member?