HHiring Reality
← Smartsheet

Principal Software Engineer - Observability & Telemetry Data

Smartsheet
Location
Bellevue, WA, USA
Posted
19 days ago
Department
Engineering - Developers
What they actually want (must-haves)
  • 10+ years of experience building and operating large-scale distributed systems, data platforms, or observability infrastructure
  • Deep Observability Expertise with production ownership of metrics, logs, and distributed tracing
  • Telemetry Data Engineering experience with large-scale data architecture
  • Practical experience with the Databricks platform and MLflow
  • Practical experience with OpenTelemetry collectors and sampling strategy
  • Experience with high-volume log and telemetry pipelines
Nice to have
  • Experience instrumenting LLM or agentic systems
  • Data reliability engineering practice
  • Telemetry cost engineering or FinOps at scale
  • Experience in regulated environments
  • Knowledge of Prometheus, Grafana, and Alertmanager
What the job really is

As a Principal Software Engineer in Observability & Telemetry Data at Smartsheet, you will architect and oversee the telemetry data platform, ensuring effective collection, modeling, and storage of telemetry data. You will collaborate across engineering teams to establish standards, lead the migration to OpenTelemetry, and integrate AI telemetry with existing data platforms. Your role will also involve mentoring engineers, participating in production support, and driving improvements in system resiliency.

Benefits
  • Employer subsidized medical/vision and dental coverage for full-time employees
  • 401k Match to help you save for your future
  • Monthly stipend to support your work and productivity
  • Flexible Time Away Program, plus Sick Time Off
  • 12 paid holidays per year
  • Up to 24 weeks of Parental Leave
Things to weigh
  • No salary listed in the posting, but a range is provided
  • Role involves significant cross-team collaboration and influence without direct ownership of all roadmaps
  • Participation in on-call rotation may be expected
  • The position may require navigating complex design trade-offs between fidelity and cost
Job score3.8/5
Benefits5/5
Freshness4/5
Career value5/5
Role clarity5/5
Pay transparency0/5

Applying to Smartsheet?

See how your résumé matches this role — and tailor it from what actually gets interviews.