Sobre este proyecto
it-programming / web-development
Abierto
We are seeking an experienced Senior Data Engineer to design, build, and maintain a robust cloud-based data platform. This role involves working with a modern data stack including Airbyte, Apache Airflow, dbt, Amazon Redshift, Apache Superset, AWS, Python, and SQL. The ideal candidate will be responsible for the full lifecycle of data infrastructure, ensuring scalability, reliability, and performance.
Responsibilities:
* Build scalable ELT (Extract, Load, Transform) pipelines and data integrations to support various business needs.
* Develop and customize Airbyte connectors to integrate diverse data sources.
* Implement and manage Python API integrations for seamless data flow.
* Orchestrate complex data workflows using Apache Airflow, ensuring efficient and timely data processing.
* Create and maintain dbt (data build tool) models to transform raw data into analytics-ready data warehouses.
* Design and build interactive dashboards and data visualizations using Apache Superset.
* Ensure high data quality, implement comprehensive monitoring solutions, and design an architecture that is ready for ai/ml applications.
Requirements:
* Strong, proven experience with key data engineering tools and technologies including Airbyte, Apache Airflow, dbt, Amazon Redshift, Python, and SQL.
* Demonstrated experience with REST APIs for data integration and extensive knowledge of AWS cloud services.
* Solid understanding of enterprise data integration patterns and best practices.
* Proficiency in data modeling techniques, data pipeline monitoring, and ELT methodologies.
* Excellent English communication skills, both written and verbal.
* Availability to work with significant overlap during US Eastern business hours.
Preferred Qualifications:
* Advanced experience with AWS services relevant to data engineering.
* Familiarity with data sources such as QuickBooks and Salesforce for integration purposes.
* Experience with Apache Superset for advanced analytics and dashboarding.
* Prior experience in building data pipelines specifically for ai/ml initiatives.
* Knowledge and practical experience in data governance principles and implementation.
Categoría Programación y Tecnología
Subcategoría Programación Web
¿Cuál es el alcance del proyecto? Cambio mediano
Plazo de Entrega: No definido
Habilidades necesarias