Integration Testing: How to Catch Failures Before They Reach Customers

An integration can pass its happy-path test and still fail on the first busy day. Real systems send duplicates, omit optional fields, change schemas, expire credentials, enforce rate limits, and return success before a business transaction is complete.

A strong integration testing strategy checks contracts, component behaviour, end-to-end outcomes, and recovery. It does not rely on one fragile test environment that must contain every system.

Start With the Business Invariant

Define what must remain true:

• Every paid order creates at most one fulfilment request.
• Posted invoices balance with their source transactions.
• A customer can never see another customer's record.
• Inventory cannot become available from a failed receipt.

Tests should prove these invariants across failure and retry, not only compare HTTP status codes.

Use a Testing Pyramid for Integrations

Component tests

Test mapping, validation, state transitions, idempotency, and error classification with controlled inputs. These tests should be fast and numerous.

Contract tests

Verify that consumers and providers agree on the requests, responses, messages, and fields actually used. Consumer-driven contract testing can detect breaking provider changes without deploying the entire estate.

Integration tests

Exercise real interfaces for authentication, pagination, rate limiting, webhook verification, file transfer, and platform-specific behaviour.

End-to-end tests

Use a small set of critical journeys to confirm the final business state across systems. Keep them focused because they are slower and more fragile.

Production verification

Use synthetic checks, canaries, reconciliation, and monitored test transactions where appropriate. Some provider behaviour cannot be reproduced perfectly outside production.

Build a Failure Catalogue

Include:

• Timeout before and after the provider commits.
• Duplicate request or event.
• Out-of-order and delayed messages.
• Missing, null, unknown, and oversized fields.
• Invalid encoding, dates, and currencies.
• Authentication expiry and permission denial.
• Rate limiting and quota exhaustion.
• Partial batch success.
• Dependency outage.
• Schema and version change.
• Replay after a long delay.

For each, define retry, dead-letter, alert, reconciliation, and human recovery.

Test Idempotency Explicitly

Send the same command concurrently and after a timeout. Confirm it produces one business result and returns a stable reference.

Test the scope and expiry of idempotency keys. A key reused for a genuinely new order should not suppress it, while a retry should not create a duplicate.

Control Test Data

Create identifiable, isolated records with deterministic setup and cleanup. Do not share one mutable “test customer” across parallel tests.

Mask or synthesise personal data. Define how test events are excluded from finance, customer communication, analytics, and fulfilment.

Manage External Dependencies

Use mocks for fast component tests, provider sandboxes for interface behaviour, and a limited number of real-system tests. Document where the sandbox differs from production.

Record representative provider responses so failures can be reproduced without depending on availability, but refresh fixtures as contracts evolve.

Test Deployment Compatibility

Use expand-and-contract changes:

1. Add new fields or behaviour without removing the old.
2. Deploy providers.
3. Move consumers.
4. Verify old usage has stopped.
5. Remove the old contract.

Contract verification should be a deployment gate for both sides.

Verify Recovery, Not Only Detection

Trigger the dead-letter queue, replay a message, rotate a credential, restore connectivity, and reconcile a missing record. Confirm the operator can identify scope, prevent duplication, and close the exception.

A runbook that has never been exercised is an assumption.

Define Release Evidence

Before release, require passed contracts, critical journeys, failure tests, permission tests, performance at expected peak, reconciliation, monitoring, and rollback. Record known gaps and their owner.

DualByte's system integration service can help design testable contracts and prove integrations under realistic failure conditions.

Sources

• Pact: What Is Contract Testing?
• Pact: Consumer-Driven Contract Process
• Microsoft Azure Architecture Center: Event-Driven Architecture