Artificial Intelligence & Machine Learning for National Security

Senior Data Engineer

Data EngineerData EngineerFull TimeRemoteTeam 11-50Since 2020Company Site LinkedIn

Location

United States

Posted

14 days ago

Salary

Not specified

Bachelor DegreeEnglishAirflowApacheDistributed SystemsIo TKafkaPostgre SQLPython

Job Description

• Implement real-time data pipelines with MQTT and Redpanda for stream processing. • Implement offline data pipelines using Dagster for batch processing. • Parse and process binary message formats from various data sources. • Build data warehouses using Postgres, Apache Iceberg, Parquet, and S3. • Design data models that allow for high-performance queries. • Validate and normalize data sources. • Improve local development and CI/CD using modern tooling and GitHub Actions.

Job Requirements

Expertise in time-series data processing and analysis (windowing, resampling, interpolation, etc.)
Experience with binary message parsing.
Proficiency in Python for data engineering workflows.
Experience with row-based & columnar-based data formats.
Experience with OLTP & OLAP databases.
Knowledge of distributed systems, streaming architectures, and batch processing patterns.
Hands-on experience with a batch orchestrator such as Dagster/Airflow.
Hands-on experience with a streaming platform such as Redpanda/Kafka.
Hands-on experience with binary message formats such as Protobuf.
Nice To Have: Experience with IoT devices and sensors.
Nice To Have: Digital signal processing experience.
Nice To Have: Geospatial analysis and GIS experience.
Nice To Have: Familiar with working in monorepos.

Benefits

Unlimited PTO — Take the time you need to recharge and maintain work-life balance.
Dedicated Sick Time — Your health and well-being come first.
Comprehensive Health & Benefits – Medical, dental, and vision coverage to keep you and your family protected.
11 Paid Holidays — Enjoy time off throughout the year to celebrate and spend with loved ones.
Professional Development — Educational opportunities and resources to help you grow your skills and advance your career.
Collaborative Environment — Work directly with leadership in our flat organizational structure, where your ideas and contributions matter.
Mission-Driven Work — Contribute to projects that directly support national security and make a real-world impact.
Growth Opportunities — Join us during an exciting expansion phase where you can help shape our future.
401(k) with company match.
Onsite / Remote / Flexible work arrangements or hybrid options (position dependent).
Relocation assistance (position dependent).
Referral bonuses.
Performance bonuses.
Life insurance and disability coverage.
Technology home office setup stipend.
Professional certification reimbursement (position dependent).