Purging policy

Overview

Purging policy controls how long tracked transaction instance data is retained in the BAM datastore before it is permanently deleted. This article describes the principles for designing a purging policy governance model — how to set appropriate retention periods per transaction, how to align retention with compliance requirements, and how to manage storage costs as your BAM environment scales.

For the mechanics of configuring a purging policy on an individual transaction, see Purging policy.

Business value

Without deliberate purging governance, retention periods default to 180 days across all transactions regardless of whether that period is appropriate. High-volume transactions accumulate large volumes of instance data quickly, increasing storage costs and slowing down query performance. Mission-critical transactions with compliance requirements may need longer retention than the default. Applying the right retention period per transaction type keeps storage costs controlled and query performance consistent over time.

How it works

Each business transaction in BAM has its own purging policy setting. The default retention period is 180 days. You can reduce this period on a per-transaction basis. Purging runs as a background job once daily and permanently deletes all transaction instances and stage instances older than the configured threshold for each transaction.

Purging is irreversible — deleted instance data cannot be recovered. Purging applies to tracked instance data only; business process and transaction configurations are not affected.

Key practices

Segment transactions by retention requirement before setting purging periods

Before configuring any purging policy, classify your transactions by their retention requirement:

Configure the purging period for each transaction to match its retention category, not the default. Applying the same retention period to all transactions is the most common governance mistake in BAM environments.

Use the shortest retention period that satisfies your requirements

Longer retention periods consume more storage and produce larger query result sets. Do not retain data longer than you need it. For most operational integration workflows, 30–90 days is sufficient for incident investigation purposes. Reserve longer retention for transactions that have explicit compliance, audit, or contractual requirements.

When in doubt, start with the operational investigation period (30–90 days) and review after three months of production data. If no incidents have required data older than 30 days, shorten the retention period further.

Align retention periods with your organisation's data retention policy

If your organisation has a documented data retention policy, map each BAM transaction type to the appropriate policy category before configuring purging periods. BAM's per-transaction granularity makes it possible to satisfy different policy requirements within the same BAM environment.

For transactions that process personal data, confirm with your data protection team whether the BAM datastore falls within the scope of your data protection obligations. Tracked properties that include names, email addresses, or other personal identifiers may be subject to data minimisation requirements that affect both what you track and how long you retain it.

Proactively reduce retention on high-volume transactions

High-volume transactions — those that generate thousands of instances per day — are the most important to manage with short retention periods. A transaction producing 10,000 instances per day with a 180-day default will accumulate 1.8 million instances in the datastore before any purging occurs, degrading query performance and increasing storage costs significantly.

For high-volume transactions, the recommended approach is:

1. Estimate the expected daily instance volume before go-live.
2. Decide the minimum retention period needed for operational purposes (typically 7–14 days for high-volume, low-criticality flows).
3. Set the purging period at go-live — do not rely on updating it later after the data has already accumulated.

Coordinate purging policy changes with your operations team

Changing a purging policy to a shorter period does not immediately delete existing data that exceeds the new threshold — it takes effect from the next daily background job run. Once the new period is active, data older than the new threshold will be deleted on subsequent daily runs.

Before shortening a retention period on a production transaction, confirm with your operations team that:

• No active investigations require access to instance data that would fall outside the new threshold.
• Any compliance holds or legal holds on data have been reviewed.
• Support teams have exported or archived any historical instances they may need for ongoing cases.

Monitor storage growth as part of regular BAM operations

Review the storage utilisation of your BAM datastore periodically — at minimum quarterly, and more frequently in the first three months after a new high-volume transaction goes live. Early detection of unexpectedly high data accumulation gives you time to adjust purging periods before storage costs become significant or query performance degrades.

The BAM Getting Started > Storage information article describes the storage architecture and how to view storage metrics for your BAM environment.

Apply consistent retention periods within business process groups

If multiple teams operate separate business processes in a shared BAM environment, align on a set of standard retention tiers (for example, 14 days / 90 days / 365 days) and require teams to justify deviations. Ad hoc per-transaction retention periods across dozens of processes are difficult to audit and govern over time.

Document the agreed retention tiers and the criteria for each tier as part of your BAM governance documentation, and review them annually.

Example scenario

A platform operations team governs a BAM environment shared by three integration teams. They define three standard retention tiers:

Each team must categorise their transactions against these tiers when onboarding to BAM. A finance integration team running payment transactions applies the long-term tier. A logistics team running shipment milestone updates — 5,000 instances per day — applies the short-term tier. All other teams default to the standard tier.

The platform operations team reviews storage utilisation quarterly and flags any transaction exceeding 500,000 accumulated instances for retention tier review.

Related articles

• Purging policy
• Storage information
• Storage management
• Transaction design
• Business processes