Cisco and Splunk Cisco time sequence mannequina univariate zero-shot time sequence foundational mannequin designed for observability and safety metrics. It’s launched as Hugging Face’s Open Weight Checkpoint below the Apache 2.0 license and is focused at predicting workloads with out task-specific fine-tuning. This mannequin extends TimesFM 2.0 with an specific multiresolution structure that fuses coarse and fine-grained historical past inside one context window.
Why we’d like multiresolution context for observability?
Manufacturing indicators usually are not easy single-scale alerts. Weekly patterns, long-term development and saturation are solely seen at coarse decision. See saturation occasions, visitors spikes, and incident dynamics at 1- or 5-minute decision. Typical time sequence basis fashions function at a single decision with a context window of 512 to 4096 factors, however TimesFM 2.5 extends this to 16384 factors. Even for one minute of knowledge, this covers as much as two weeks, and sometimes much less.
That is an observability challenge as knowledge platforms usually solely maintain previous knowledge in aggregated type. High-quality-grained samples expire and solely exist as a 1-hour rollup. The Cisco Time Sequence mannequin is constructed for this storage sample. We deal with coarse-grained historical past as a first-class enter that improves predictions at finer resolutions. This structure operates instantly on a multi-resolution context, moderately than appearing as if all inputs reside on a single grid.
Multiresolution enter and prediction targets
Formally, the mannequin consumes pairs of contexts (xc,×f). The coarse context (x_c) and fantastic context (x_f) every have a most size of 512. (x interval)c) is fastened at an interval of 60 instances xf). A typical observability configuration makes use of a 1-hour aggregation of 512 hours and a 1-minute worth of 512 minutes. Each sequence finish on the identical predicted minimize level. This mannequin makes use of the imply and a set of quantiles from 0.1 to 0.9 to foretell a variety of 128 factors at excessive decision.
Structure, TimesFM core with decision embedding
Internally, Cisco Time Sequence fashions reuse the TimesFM patch-based decoder stack. The enter is normalized, patched into non-overlapping chunks, and handed by way of a residual embedding block. The transformer core consists of fifty decoder-only layers. The ultimate residual block maps the tokens to the horizon. The analysis group removes positional embedding and as an alternative depends on patch order, multiresolution construction, and new decision embedding to encode construction.
Two additions make the structure multi-resolution succesful. A particular token, sometimes called ST in experiences, is inserted between the coarse and fantastic token streams. It exists in sequence area and marks boundaries between resolutions. A decision embedding, sometimes called RE, is added to mannequin area. One embedding vector is used for all coarse tokens and one for all fantastic tokens. Ablation research within the paper present that each elements enhance high quality, particularly in lengthy context situations.
The decoding process can be multi-resolution. The mannequin outputs imply and quantile predictions for high-resolution time intervals. Throughout lengthy horizon decoding, newly predicted fantastic factors are added to the fantastic context. Aggregating these predictions updates the high-level context. This creates an autoregressive loop through which each resolutions evolve concurrently throughout prediction.
Coaching knowledge and recipes
The Cisco Time Sequence mannequin is skilled by persevering with pre-training primarily based on TimesFM weights. The ultimate mannequin has 500 million parameters. For coaching, we use AdamW for bias, norm, and embeddings, and Muon for hidden layers utilizing a cosine studying fee schedule. This loss is a mix of the imply squared error of the common prediction and the quantile loss over quantiles from 0.1 to 0.9. The group trains over 20 epochs and selects the optimum checkpoint by validation loss.
The dataset is giant and observability biased. The Splunk group experiences roughly 400 million metric time sequence from its personal Splunk Observability Cloud deployment. These are collected over 13 months at 1-minute decision, with some aggregated to 5-minute decision. The researchers say the ultimate corpus accommodates greater than 300 billion distinctive knowledge factors, together with roughly 35 % one-minute observability, 16.5 % five-minute observability, 29.5 % GIFT Eval pre-training knowledge, 4.5 % Chronos dataset, and 14.5 % artificial KernelSynth sequence.
Benchmark outcomes for observability and GIFT analysis
The analysis group evaluates the mannequin primarily based on two key benchmarks. The primary is an observability dataset derived from Splunk metrics at 1- and 5-minute resolutions. The second is a filtered model of GIFT Eval, which removes the dataset that leaks TimesFM 2.0 coaching knowledge.
For 1-minute decision observability knowledge with 512 fantastic steps, the Cisco Time Sequence mannequin with 512 multiresolution contexts reduces the common absolute error to 0.4788 from 0.6265 in TimesFM 2.5 and 0.6315 in TimesFM 2.0, with related enhancements in common absolute scale error and steady rating chance rating. An identical achieve is seen at 5 minute decision. At each resolutions, the mannequin outperforms the Chronos 2, Chronos Bolt, Toto, and AutoARIMA baselines below the normalized metrics used on this paper.
Within the filtered GIFT Eval benchmark, the Cisco Time Sequence mannequin matches the bottom TimesFM 2.0 mannequin and exhibits aggressive efficiency with TimesFM-2.5, Chronos-2, and Toto. The primary declare isn’t common superiority, however moderately sustaining normal predictive high quality whereas including sturdy benefits in lengthy context home windows and observability workloads.
Essential factors
- The Cisco Time Sequence Mannequin is a univariate zero-shot time sequence basis mannequin that extends the TimesFM 2.0 decoder-specific spine with a multiresolution structure for observability and safety metrics.
- The mannequin makes use of a multiresolution context with coarse and fantastic sequence, every as much as 512 steps lengthy. The coarse decision is 60 instances the fantastic decision, predicting 128 fantastic decision steps in common and quantile output.
- The Cisco Time Sequence mannequin is skilled on over 300B knowledge factors, greater than half of which come from observability, blended Splunk machine knowledge, GIFT Eval, Chronos datasets, and artificial KernelSynth sequence, with roughly 0.5B parameters.
- On observability benchmarks at 1- and 5-minute resolutions, the mannequin achieves decrease error than TimesFM 2.0, Chronos, and different baselines whereas sustaining aggressive efficiency on the general-purpose GIFT Eval benchmark.
Please verify paper, blog and HF model card. Please be at liberty to test it out GitHub page for tutorials, code, and notebooks. Please be at liberty to comply with us too Twitter Remember to affix us 100,000+ ML subreddits and subscribe our newsletter. grasp on! Are you on telegram? You can now also participate by telegram.
Asif Razzaq is the CEO of Marktechpost Media Inc. As a visionary entrepreneur and engineer, Asif is dedicated to harnessing the potential of synthetic intelligence for social good. His newest endeavor is the launch of Marktechpost, a man-made intelligence media platform. It stands out for its thorough protection of machine studying and deep studying information, which is technically sound and simply understood by a large viewers. The platform boasts over 2 million views monthly, demonstrating its recognition amongst viewers.
