Data Engineer
Location
United States
Posted
57 days ago
Salary
Not specified
PythonRustPostgre SQLApache IcebergApache ParquetAmazon S3MQTTRedpandaApache KafkaDagsterApache AirflowProtocol BuffersTime Series Data ProcessingBinary Message ParsingOLTPOLAPDistributed SystemsStreaming ArchitecturesBatch Processing
Job Description
This description is a summary of our understanding of the job description. Click on 'Apply' button to find out more.
Role Description
We’re seeking a skilled Data Engineer to build the next-generation data management and artificial intelligence platform for maritime domain awareness.
What you’ll do:
- Implement real-time data pipelines with MQTT and Redpanda for stream processing.
- Implement offline data pipelines using Dagster for batch processing.
- Parse and process binary message formats from various data sources.
- Build data warehouses using Postgres, Apache Iceberg, Parquet, and S3.
- Design data models that allow for high-performance queries.
- Validate and normalize data sources.
- Improve local development and CI/CD using modern tooling and GitHub Actions.
Qualifications
- Expertise in time-series data processing and analysis (windowing, resampling, interpolation, etc.)
- Proficiency in Python and Rust for data engineering workflows.
- Experience with binary message parsing.
- Experience with row-based & columnar-based data formats.
- Experience with OLTP & OLAP databases.
- Knowledge of distributed systems, streaming architectures, and batch processing patterns.
- Hands-on experience with a batch orchestrator such as Dagster/Airflow.
- Hands-on experience with a streaming platform such as Redpanda/Kafka.
- Hands-on experience with binary message formats such as Protobuf.
Requirements
- Experience with IoT devices and sensors (nice to have).
- Digital signal processing experience (nice to have).
- Geospatial analysis and GIS experience (nice to have).
- Familiar with working in monorepos (nice to have).
Benefits
- Unlimited PTO — Take the time you need to recharge and maintain work-life balance.
- Dedicated Sick Time — Your health and well-being come first.
- Comprehensive Health & Benefits – Medical, dental, and vision coverage to keep you and your family protected.
- 11 Paid Holidays — Enjoy time off throughout the year to celebrate and spend time with loved ones.
- Professional Development — Educational opportunities and resources to help you grow your skills and advance your career.
- Collaborative Environment — Work directly with leadership in our flat organizational structure, where your ideas and contributions matter.
- Mission-Driven Work — Contribute to projects that directly support national security and make a real-world impact.
- Growth Opportunities — Join us during an exciting expansion phase where you can help shape our future.
- 401(k) with company match.
- Onsite / Remote / Flexible work arrangements or hybrid options (position dependent).
- Relocation assistance (position dependent).
- Referral bonuses.
- Performance bonuses.
- Life insurance and disability coverage.
- Technology home office setup stipend.
- Professional certification reimbursement (position dependent).
Job Requirements
- Expertise in time-series data processing and analysis (windowing, resampling, interpolation, etc.)
- Proficiency in Python and Rust for data engineering workflows.
- Experience with binary message parsing.
- Experience with row-based & columnar-based data formats.
- Experience with OLTP & OLAP databases.
- Knowledge of distributed systems, streaming architectures, and batch processing patterns.
- Hands-on experience with a batch orchestrator such as Dagster/Airflow.
- Hands-on experience with a streaming platform such as Redpanda/Kafka.
- Hands-on experience with binary message formats such as Protobuf.
- Experience with IoT devices and sensors (nice to have).
- Digital signal processing experience (nice to have).
- Geospatial analysis and GIS experience (nice to have).
- Familiar with working in monorepos (nice to have).
Benefits
- Unlimited PTO — Take the time you need to recharge and maintain work-life balance.
- Dedicated Sick Time — Your health and well-being come first.
- Comprehensive Health & Benefits – Medical, dental, and vision coverage to keep you and your family protected.
- 11 Paid Holidays — Enjoy time off throughout the year to celebrate and spend time with loved ones.
- Professional Development — Educational opportunities and resources to help you grow your skills and advance your career.
- Collaborative Environment — Work directly with leadership in our flat organizational structure, where your ideas and contributions matter.
- Mission-Driven Work — Contribute to projects that directly support national security and make a real-world impact.
- Growth Opportunities — Join us during an exciting expansion phase where you can help shape our future.
- 401(k) with company match.
- Onsite / Remote / Flexible work arrangements or hybrid options (position dependent).
- Relocation assistance (position dependent).
- Referral bonuses.
- Performance bonuses.
- Life insurance and disability coverage.
- Technology home office setup stipend.
- Professional certification reimbursement (position dependent).
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Data Engineer57 days ago
Full TimeRemoteTeam 201-500H1B No Sponsor
Senior Data Engineer delivering data science services and streaming data pipelines
AirflowAWSCloudKafkaPythonSQLTerraform
Data Engineer57 days ago
Full TimeRemoteTeam 51-200H1B No Sponsor
Senior Data Engineer responsible for designing data infrastructure for Tekmetric
AirflowApacheETLJavaPythonScalaSparkSQLTableau
United States
Data Engineer57 days ago
Full TimeRemoteTeam 51-200H1B No Sponsor
Data Engineer transforming raw external data into powerful datasets at Machinify.
AirflowAWSCloudKafkaPythonSparkSQL
California
Manager, Ads Data Engineering
NetflixWhere you come to do the best work of your life. Follow @WeAreNetflix on Twitter, IG, Facebook, & Youtube for more
Data Engineer57 days ago
Full TimeRemoteTeam 10,001+Since 1997H1B Sponsor
Data Engineering Manager leading ads data initiatives at Netflix
HadoopSparkSQL