Abnormally-Precise, Cloud-Native Email Security

Data Platform Engineer

Platform EngineerPlatform EngineerFull TimeRemoteTeam 501-1,000H1B SponsorCompany Site LinkedIn

Location

United States

Posted

56 days ago

Salary

$123.3K - $145K / year

3 yrs expEnglishAirflowAWSPy SparkPythonSparkSQL

Job Description

• Develop reusable ingestion frameworks (Python/Airflow/AWS Glue) for APIs and unstructured sources beyond Fivetran, handling various data formats (JSON, Parquet, etc.). • Own the end-to-end Medallion (bronze/silver/gold) architecture for core domains, ensuring robust lineage and metadata across diverse data sources. • Implement data observability (native tests, alerts, lineage hooks); lead incident management and root-cause analysis (RCA) for data. • Help standardize reusable “paved-road” patterns (e.g. CI templates, ingestion operators) to improve developer productivity. • Prepare datasets for AI/LLM use cases (feature stores, embeddings/RAG prep).

Job Requirements

3–5+ years of data engineering with strong Python and SQL; hands-on Spark/PySpark (ideally via AWS Glue).
Deep experience in AWS (S3, IAM, Lambda, CloudWatch) running secure, observable data workloads.
Proficiency operating Snowflake (warehouse sizing, RBAC, resource monitors, clustering/partitioning).
Proven governance/security patterns: masking policies, row-level security, and auditability.
Orchestration experience (Airflow/MWAA) and event/file/API ingestion beyond managed connectors.
CI/CD for data with GitHub Actions; test/promotion workflows; secrets and PII handling.
Solid grasp of Medallion architecture, dimensional modeling (star schema), and data quality frameworks.
Ownership of incident management and RCA with measurable reduction in MTTR.