Changing a Tire on a Moving Car: Migrating to Datadog’s New Event Store without Downtime
Senior Software Engineer | Datadog
Husky is Datadog’s third-generation event analytics database that powers more than a dozen products and stores petabytes of customer data. It is written in Go from the standard library up, and allows the Event Platform team to deliver on internal customers' long-standing feature requests that were impractical with the legacy system. This talk will walk through two such feature requests (late data ingestion and querying over unfaceted fields), and explain how these seemingly simple requests required starting from scratch with the new system.
In this talk, Ryan will also cover the process of migrating thousands of customers across multiple Datadog products from the legacy system to Husky without downtime. This painstaking process involved double-writing data for months and exhaustive query shadowing to ensure compatibility with the legacy system.