Beschreibung

Data Engineer specialized in distributed data processing using Apache Spark and Scala.

Experience in designing, developing, and optimizing ETL pipelines on Big Data architectures, working with large-scale datasets in critical production environments.

Specialized in:

Spark job optimization and performance tuning
Batch ETL pipeline development
Workflow orchestration with Airflow
Big Data environment migrations
Distributed processing and scalability

I have worked on projects focused on financial data processing, system integrations, and data platform modernization, participating in migrations to cloud architectures and Databricks environments.

Main stack:

Scala, Spark, Airflow, Hive, SQL, Databricks, PostgreSQL, Cloudera, CI/CD, and APIs.

Branchenexpertise

Sprachen

Spanisch
Muttersprachlich oder zweisprachig
Englisch
Konversationssicher

Arbeitsortpräferenzen

Nur remote

Führt Projekte hauptsächlich remote aus

BOSONIT S.L.
Data Engineer
BANKEN & VERSICHERUNGEN
Januar 2022 - Heute (4 Jahre und 5 Monate)
Madrid, Spanien
Desarrollo, mantenimiento y evolución de pipelines ETL para el procesamiento de mensajes de pago (SWIFT, ISO 20022, SEPA, ACH) Procesamiento batch de datos desde capa landing (S3) hasta capa common, aplicando validaciones técnicas y funcionales Normalización de múltiples fuentes de datos en un modelo común para su posterior explotación Optimización de jobs Spark reduciendo tiempos de ejecución de varias horas a minutos mediante mejoras en particionado, configuración y lógica de procesamiento la de procesos
ETL/ELT Apache Spark Data Engineer Scala Databricks
BINAIA
Big Data Engineering Mentor
BILDUNG & E-LEARNING
Juli 2023 - Heute (2 Jahre und 11 Monate)
Madrid, Spanien
• Mentoring new Big Data trainees, providing guidance on both theoretical and practical aspects of Big Data technologies.

• Conduct bi-weekly follow-ups to ensure learning progress. The mentorship program begins with foundational knowledge in Hadoop, HDFS, Hive, Apache Spark, Scala/Python, followed by practical ETL simulations and hands-on experience with Apache Airflow for building and orchestrating data pipelines.
Coaching and mentoring Scala Apache Spark ETL/ELT Databricks

Sei die erste Person, die David empfiehlt

Teile Deine Erfahrung aus der Zusammenarbeit mit diesem Freelancer.

Agatha Frydrych

Backend Java Software Engineer

4.7

(3)

Baptiste Duhen

Fullstack developer

4.6

(4)

Amed Hamou

Senior Lead Developer

(2)

Audrey Champion

Web developer

4.3

(3)

Anmelden, um Profile zu sehen

Grado Superior
MEDAC
2022
Grado Superior
Certified Associate Developer for Apache Spark 3.0
Databricks
Certified Associate Developer for Apache Spark 3.0