Splunk: Data Retention

How long does Splunk data remain searchable?

A Splunk event (log entry) remains searchable until it is at least 90 days old.  The age of a log event is calculated from its timestamp, and not when it was ingested into Splunk.  

What happens to old data in Splunk after 90 days?

The data is rolled off to archive storage where it is no longer searchable.  That data is stored until the events are 1 year old, and then they are permanently deleted.

My data has been frozen, how do I get it back?

Thawing frozen data takes a considerable amount of time, disk storage, and processing power.  A request to do so should only be considered in the case of a University emergency, and the decision to thaw the data would need be approved by Senior ITS management.

Wait! You said that data is unsearchable after 90 days, but when I search my index I see data that is older than 90 days... What gives?

Splunk guarantees that data remains searchable for at least 90 days.   Under the hood, Splunk groups many log events into "buckets" and stores them on disk. Sometimes newer log data can get mixed in with older log data into the same bucket. This can happen, for instance, if somebody sends very old data or they have bad data with timestamps way in the future or past. This then would get mixed with the regular incoming data.  What results is that buckets can have a mix of events that span many days or weeks.  Splunk will retain the entire bucket until all the data contained within is at least 90 days old.  Therefore you may see older data, but anything past 90 days is incomplete. Searching past 90 days results is a bad idea due to the "swiss cheese effect."

Print Article

Details

Article ID: 243
Created
Tue 6/25/24 6:06 PM
Modified
Wed 6/26/24 11:28 AM