About this project
it-programming / web-development
Open
We are seeking an experienced Senior Data Engineer to design, build, and maintain a robust cloud-based data platform. This role involves working with a modern data stack including Airbyte, Apache Airflow, dbt, Amazon Redshift, Apache Superset, AWS, Python, and SQL. The ideal candidate will be responsible for the full lifecycle of data infrastructure, ensuring scalability, reliability, and performance.
Responsibilities:
* Build scalable ELT (Extract, Load, Transform) pipelines and data integrations to support various business needs.
* Develop and customize Airbyte connectors to integrate diverse data sources.
* Implement and manage Python API integrations for seamless data flow.
* Orchestrate complex data workflows using Apache Airflow, ensuring efficient and timely data processing.
* Create and maintain dbt (data build tool) models to transform raw data into analytics-ready data warehouses.
* Design and build interactive dashboards and data visualizations using Apache Superset.
* Ensure high data quality, implement comprehensive monitoring solutions, and design an architecture that is ready for ai/ml applications.
Requirements:
* Strong, proven experience with key data engineering tools and technologies including Airbyte, Apache Airflow, dbt, Amazon Redshift, Python, and SQL.
* Demonstrated experience with REST APIs for data integration and extensive knowledge of AWS cloud services.
* Solid understanding of enterprise data integration patterns and best practices.
* Proficiency in data modeling techniques, data pipeline monitoring, and ELT methodologies.
* Excellent English communication skills, both written and verbal.
* Availability to work with significant overlap during US Eastern business hours.
Preferred Qualifications:
* Advanced experience with AWS services relevant to data engineering.
* Familiarity with data sources such as QuickBooks and Salesforce for integration purposes.
* Experience with Apache Superset for advanced analytics and dashboarding.
* Prior experience in building data pipelines specifically for ai/ml initiatives.
* Knowledge and practical experience in data governance principles and implementation.
Category IT & Programming
Subcategory Web development
What is the scope of the project? Medium-sized change
Delivery term: Not specified
Skills needed