Edward Plata
Data engineer — lakehouses, streaming, and platforms other engineers can safely build on.
New York, NY
Data engineer with 7 years building analytics platforms across health-tech, consulting and financial data. I own claims ingestion and a shared-savings payout pipeline at Vytalize Health, and before that built streaming pipelines and a self-service platform for 200+ analysts at Accenture. I work in Databricks, dbt, PySpark, Delta Lake and Kafka, and I care most about the parts that make a platform trustworthy: governance standards, observability, and pipelines that recover on their own.
Experience
Feb 2025 – Present
Data Engineer
Vytalize Health · New York, NY
Own: claims ingestion · shared-savings payouts · lakehouse governance
- Own the BCDA/FHIR claims ingestion path (PySpark, Delta Lake, dbt) under a zero data-loss (RPO 0) contract, with idempotent, replay-safe loads that retry without duplicating records.
- Own the shared-savings payout pipeline end to end: ingestion, reconciliation, and the provider payment figures it produces.
- Architected the medallion (Bronze/Silver/Gold) lakehouse on Databricks and made Unity Catalog its governance layer for lineage and access control, so analytics and ML teams use regulated data self-serve.
- Set the dbt governance standard (tests, docs, lineage) and enforced it with CI checks, so engineers outside the platform team change downstream models safely.
- Built pipeline observability on OpenTelemetry and Databricks job metrics, with alerting that cut time-to-detect on pipeline failures from hours to minutes.
- Run ingestion across Azure and AWS with automated failover between the two clouds.
Sep 2021 – Jan 2025
Data Engineering Consultant
Accenture
Platform for 200+ analysts · dbt standards across 50+ engineers
- Built a self-service analytics platform on Databricks for 200+ analysts, with standard notebook environments and data contracts.
- Drove dbt modelling, testing, CI and lineage standards to adoption across 50+ engineers, cutting review cycles 20%; they became the default way analytics-engineering teams shipped data.
- Shipped knowledge search on AWS Kendra with retrieval-augmented generation (RAG), cutting analyst query-resolution time 67% across the consulting practice.
- Built Kafka and Spark Structured Streaming pipelines that fed live operational dashboards.
- Cut query latency on high-volume warehouse workloads by redesigning partitioning and file layout (Z-ordering) and rewriting SQL.
- Designed self-serve experimentation infrastructure (statistics libraries, metric stores), so product teams ran A/B tests on their own.
Jul 2019 – Aug 2021
Data Engineer
Verisk / Argus
ETL automation · data-quality reporting
- Replaced manual nightly ETL with Pentaho and GitLab CI/CD workflows, removing ~40 hours/month of ops toil.
- Shipped a data-quality dashboard (Tableau + SQL) that showed validation failures to engineers as they happened, cutting QA cycle time 50%.
Skills
- Languages
- Python · SQL · R · PySpark · C++ (learning)
- Lakehouse & warehouses
- Databricks · Delta Lake · Unity Catalog · Snowflake · BigQuery · DuckDB · Postgres
- Movement & orchestration
- Kafka · Spark Structured Streaming · dbt · Airflow · Dagster · dlt
- Cloud & infra
- AWS · Azure · GCP · Docker · Kubernetes · Vercel
- Reliability
- OpenTelemetry · SLOs · CI/CD · monitoring & alerting
- Analytics & ML
- A/B testing · causal inference · statistical modelling · scikit-learn
Education
New York, NY
BS, Statistics
Baruch College, CUNY
New York, NY
Data Science Immersive
Galvanize