Edward Plata

Data engineer — lakehouses, streaming, and platforms other engineers can safely build on.

New York, NY

Data engineer with 7 years building analytics platforms across health-tech, consulting and financial data. I own claims ingestion and a shared-savings payout pipeline at Vytalize Health, and before that built streaming pipelines and a self-service platform for 200+ analysts at Accenture. I work in Databricks, dbt, PySpark, Delta Lake and Kafka, and I care most about the parts that make a platform trustworthy: governance standards, observability, and pipelines that recover on their own.

Experience

Feb 2025 – Present

Data Engineer

Vytalize Health · New York, NY

Own: claims ingestion · shared-savings payouts · lakehouse governance

  • Own the BCDA/FHIR claims ingestion path (PySpark, Delta Lake, dbt) under a zero data-loss (RPO 0) contract, with idempotent, replay-safe loads that retry without duplicating records.
  • Own the shared-savings payout pipeline end to end: ingestion, reconciliation, and the provider payment figures it produces.
  • Architected the medallion (Bronze/Silver/Gold) lakehouse on Databricks and made Unity Catalog its governance layer for lineage and access control, so analytics and ML teams use regulated data self-serve.
  • Set the dbt governance standard (tests, docs, lineage) and enforced it with CI checks, so engineers outside the platform team change downstream models safely.
  • Built pipeline observability on OpenTelemetry and Databricks job metrics, with alerting that cut time-to-detect on pipeline failures from hours to minutes.
  • Run ingestion across Azure and AWS with automated failover between the two clouds.

Sep 2021 – Jan 2025

Data Engineering Consultant

Accenture

Platform for 200+ analysts · dbt standards across 50+ engineers

  • Built a self-service analytics platform on Databricks for 200+ analysts, with standard notebook environments and data contracts.
  • Drove dbt modelling, testing, CI and lineage standards to adoption across 50+ engineers, cutting review cycles 20%; they became the default way analytics-engineering teams shipped data.
  • Shipped knowledge search on AWS Kendra with retrieval-augmented generation (RAG), cutting analyst query-resolution time 67% across the consulting practice.
  • Built Kafka and Spark Structured Streaming pipelines that fed live operational dashboards.
  • Cut query latency on high-volume warehouse workloads by redesigning partitioning and file layout (Z-ordering) and rewriting SQL.
  • Designed self-serve experimentation infrastructure (statistics libraries, metric stores), so product teams ran A/B tests on their own.

Jul 2019 – Aug 2021

Data Engineer

Verisk / Argus

ETL automation · data-quality reporting

  • Replaced manual nightly ETL with Pentaho and GitLab CI/CD workflows, removing ~40 hours/month of ops toil.
  • Shipped a data-quality dashboard (Tableau + SQL) that showed validation failures to engineers as they happened, cutting QA cycle time 50%.

Skills

Languages
Python · SQL · R · PySpark · C++ (learning)
Lakehouse & warehouses
Databricks · Delta Lake · Unity Catalog · Snowflake · BigQuery · DuckDB · Postgres
Movement & orchestration
Kafka · Spark Structured Streaming · dbt · Airflow · Dagster · dlt
Cloud & infra
AWS · Azure · GCP · Docker · Kubernetes · Vercel
Reliability
OpenTelemetry · SLOs · CI/CD · monitoring & alerting
Analytics & ML
A/B testing · causal inference · statistical modelling · scikit-learn

Education

New York, NY

BS, Statistics

Baruch College, CUNY

New York, NY

Data Science Immersive

Galvanize