Data Engineer with BigQuery, Real Time Warehouse Platform

Remote.Bulgaria
Remote.Georgia
Remote.Kazakhstan
Remote.Poland
Алмати
Астана
Белград
Варна
Варшава
Вроцлав
Днепър
Ереван
Киев
Клуж-Напока
Краков
Ларнака
Лвов
Лодз
Люблин
Одеса
Рига
София
Тбилиси
Харков

Ако сте получили информация за тази свободна позиция от нашите рекрутери, прочетете нашата Политика за поверителност на личните данни.

Project overview

We are building a modern data warehousing platform that supports both real time and batch analytical workflows. The project focuses on designing scalable data pipelines, implementing a medallion architecture, and enabling seamless collaboration across data engineering, analytics, and data science teams.

Team

You will collaborate with a cross functional team of data engineers, data scientists, and analysts. The team follows an open communication style, uses Agile practices, and works closely to define data requirements and deliver scalable solutions.

Position overview

We are looking for a Data Engineer who will design, develop, and optimize data pipelines and warehouse layers on Google BigQuery. You will work with real time streaming, batch processing, and MLflow based workflows while ensuring data quality, reliability, and performance across all components of the platform.

Technology stack

Google BigQuery, Google Cloud Pub/Sub, Apache Beam, Dataflow, Cloud Composer, Apache Airflow, Python, SQL, MLflow, Terraform, Great Expectations, dbt tests

Responsibilities

Plan, develop, and maintain ETL and ELT pipelines across Bronze, Silver, and Gold layers
Implement the medallion architecture with a focus on data lineage and quality
Build and optimize real time data streaming pipelines using Pub/Sub and Apache Beam on Dataflow
Create and orchestrate batch workflows using Cloud Composer and Apache Airflow
Write performant and cost efficient BigQuery SQL using partitioning, clustering, and query optimization
Collaborate with analysts and data scientists to understand data needs and deliver reliable solutions
Prepare and maintain technical documentation, data dictionaries, and monitoring dashboards

Requirements

4 years of Data Engineering experience
3 or more years of experience designing and building cloud based ETL and ELT pipelines
Hands on experience with data modeling including dimensional modeling and schema design
Experience working with real time streaming architectures and event driven data processing
Proficiency in SQL and at least one programming language such as Python, Java, or Scala
Experience with Google Cloud Pub/Sub for message driven ingestion
Practical experience building Apache Beam pipelines on Google Cloud Dataflow
Experience with workflow orchestration in Cloud Composer or Apache Airflow
Hands on experience with optimizing BigQuery SQL, partitioning, clustering, resource usage, and cost management

Nice to have

Experience using BigQuery ML for model creation and deployment
Foundational understanding of machine learning workflows and feature engineering
Experience with Terraform for infrastructure provisioning
Familiarity with data quality frameworks such as Great Expectations or dbt tests
Google Cloud Professional Data Engineer certification

Търсите сходни възможности?

Try AI chatbots with our ready-made prompt to discover similar roles that match your skills and interests.

Най-търсени позиции

1 of 1

Senior Data Designer

БългарияКипърЛатвияПолшаРумънияСърбияУкрайна

Accountable for creating high-quality Enterprise Data Warehouse data designs by understanding and documenting data sources, ingestion methods, mappings, modeling, and defining required data harmonization, calculations, derivations, and transformation logic

Python Data Engineer (Azure), Investment Management

САЩ

We are seeking a Python Data Engineer (Azure) to join our team. In this role, you will be responsible for creating Microsoft Fabric pipelines and organizing ETL/ELT processes supporting a leading investment management company

Data Quality Engineer, Data Reliability Platform

GeorgiaАрменияБългарияКипърЛатвияПолшаРумънияСърбия

The project includes implementing monitoring rules, building automated validation workflows, and supporting data teams with insights that help maintain data quality at scale

Senior Data Engineer

САЩ

We are looking for an experienced Senior Data Engineer to design, build, and maintain scalable data solutions in the Azure cloud.