Upbound

Upbound delivers a single point of control to manage all your applications and infrastructure across teams and clouds.

Data Engineer – AI

Data EngineerData EngineerFull TimeRemoteTeam 11-50Since 2017H1B No SponsorCompany SiteLinkedIn

Location

California

Posted

143 days ago

Salary

Not specified

10 yrs expEnglishAirflowCloudElastic SearchKubernetesSpark

Job Description

• Define and drive the technical vision for data platforms that support AI-powered features in Crossplane and Upbound Spaces • Lead the design of data pipelines that transform infrastructure and data into training datasets for ML models • Architect vector search and RAG systems that leverage Crossplane Control Planes & Upbound Marketplace as a knowledge store • Build data infrastructure that processes resources, extensions, and compositions for semantic search • Establish frameworks for collecting, processing, and analyzing infrastructure configuration data • Design data pipelines that handle Crossplane-specific data • Create infrastructure for indexing and searching Upbound Marketplace content, documentation, and community patterns • Develop metrics and monitoring for AI features integrated with Upbound's control plane architecture • Design data systems that power AI agents for infrastructure provisioning & operations, helping users generate and optimize Crossplane compositions • Create feature engineering platforms that extract signals from control plane operations, resource status, and reconciliation patterns • Implement data infrastructure for training models that predict infrastructure failures, optimize resource allocation, and suggest configuration improvements • Drive the development of knowledge graph representations of infrastructure dependencies and relationships

Job Requirements

  • 10+ years of software/data engineering experience with at least 4 years in technical leadership roles
  • Proven track record building data platforms that support production systems at scale
  • Deep expertise in both traditional data engineering (Spark, Airflow, data lakes) and ML-specific infrastructure (feature stores, model serving)
  • Experience with vector databases (Pinecone, Weaviate, Qdrant, Milvus, pgvector, Opensearch, ElasticSearch)
  • Demonstrated experience with LLM applications, including RAG architectures and semantic search implementations
  • Understanding of Kubernetes, cloud-native architectures, and infrastructure-as-code principles
  • Strong understanding of data requirements for AI/ML systems: training pipelines, feature stores, and inference infrastructure
  • Hands-on experience building knowledge bases and semantic search systems for technical documentation and code
  • Experience with embedding models for code and technical documentation
  • Knowledge of time-series data processing for infrastructure metrics and events
  • Understanding of graph databases and their application to infrastructure dependency modeling

Benefits

  • Health insurance
  • 401(k) matching
  • Flexible work hours
  • Paid time off
  • Remote work options

Related Categories

Related Job Pages

More Data Engineer Jobs

Principal Data Architect – Databricks

Arbi Arredobagno

The perfect design for every bathroom

Data Engineer143 days ago
Full TimeRemoteTeam 51-200Since 1987H1B No Sponsor

Principal Technical Architect at Argano providing strategic direction for technical architecture.

United States
$136.2K - $223.7K / year

Data Engineer Manager

Superlanet

Advisory, Staffing, and Multi-State Employer of Record Solutions for Clinicians, by Clinicians.

Data Engineer144 days ago
Full TimeRemoteTeam 51-200Since 2017H1B No Sponsor

Data Engineer Manager leading healthcare data platform development remotely

ApacheAzureCloudETLKafkaPySparkScalaSparkSQLUnity
Texas
$135K - $190K / year

Data Engineer

Zapier

Get your software working together, automatically.

Data Engineer144 days ago
Full TimeRemoteTeam 501-1,000Since 2011H1B No Sponsor

Data Engineer building scalable data systems at Zapier.

AWSAzureCloudGoogle Cloud PlatformPythonSparkSQLTypeScript
United States
$141.1K - $211.7K / year

Senior Data Engineer

InStride

Partnering with businesses to create life-changing workforce education programs through a leading academic network.

Data Engineer147 days ago
Full TimeRemoteTeam 51-200H1B Sponsor

Sr. Data Engineer building scalable data infrastructure at InStride

CloudMongoDBPythonTypeScript
Arizona + 22 moreAll locations: Arizona, California, Colorado, Connecticut, Florida, Illinois, Kansas, Louisiana, Nevada, New Hampshire, New Jersey, New York, Ohio, Oregon, Maryland, Massachusetts, Michigan, Missouri, Pennsylvania, Texas, Virginia, Washington, Wisconsin
$150K - $165K / year