Runbook: Balance Discrepancy

Overview

A balance discrepancy occurs when a wallet's stored balance doesn't match the sum of its transactions.

Detection

Automated

• Nightly reconciliation job compares balances
• Alert triggers if mismatch > 1 cent

Manual

• Member reports incorrect balance
• Staff notices balance doesn't match receipt

Severity

Discrepancy	Severity	Response Time
Single wallet, < R10	Low	24 hours
Single wallet, > R10	Medium	4 hours
Multiple wallets	High	1 hour
Systematic pattern	Critical	Immediate

Investigation Steps

1. Identify Affected Wallets

-- Find wallets with balance mismatch
SELECT
  wa.id,
  wa.member_id,
  wa.balance as stored_balance,
  COALESCE(SUM(wt.amount), 0) as calculated_balance,
  wa.balance - COALESCE(SUM(wt.amount), 0) as discrepancy
FROM wallet_account wa
LEFT JOIN wallet_transaction wt ON wt.wallet_id = wa.id
GROUP BY wa.id
HAVING wa.balance != COALESCE(SUM(wt.amount), 0);

2. Review Transaction History

-- Get recent transactions for affected wallet
SELECT
  id,
  type,
  amount,
  balance_after,
  reference,
  description,
  posted_at
FROM wallet_transaction
WHERE wallet_id = '<wallet_id>'
ORDER BY posted_at DESC
LIMIT 50;

3. Check for Common Causes

Missing Transaction

• Payment completed but transaction not posted
• Check payment provider webhook logs
• Verify outbox processing

-- Check for orphaned payments
SELECT * FROM payment_intent
WHERE status = 'COMPLETED'
AND source_id LIKE 'wallet-%'
AND NOT EXISTS (
  SELECT 1 FROM wallet_transaction
  WHERE reference = payment_intent.id
);

Duplicate Transaction

• Same payment posted twice
• Check for duplicate references

-- Find duplicate references
SELECT reference, COUNT(*)
FROM wallet_transaction
WHERE wallet_id = '<wallet_id>'
AND reference IS NOT NULL
GROUP BY reference
HAVING COUNT(*) > 1;

Race Condition

• Concurrent balance updates
• Check timestamps of recent transactions

Application Bug

• Incorrect amount calculation
• Review recent code changes

4. Determine Root Cause

Document:

• When discrepancy first appeared
• What operation preceded it
• Whether pattern exists across wallets

Resolution

For Single Wallet Discrepancy

If Missing Credit

1. Verify payment was received
2. Post adjustment transaction:

INSERT INTO wallet_transaction (
  id, wallet_id, type, amount, balance_after,
  reference, description, posted_at
) VALUES (
  uuid_generate_v4(),
  '<wallet_id>',
  'ADJUSTMENT',
  <missing_amount>,
  <corrected_balance>,
  'RECON-<date>-<ticket>',
  'Balance correction - missing credit from <source>',
  NOW()
);

UPDATE wallet_account
SET balance = <corrected_balance>, updated_at = NOW()
WHERE id = '<wallet_id>';

If Erroneous Debit

1. Verify original transaction was incorrect
2. Post reversal:

INSERT INTO wallet_transaction (
  id, wallet_id, type, amount, balance_after,
  reference, description, posted_at
) VALUES (
  uuid_generate_v4(),
  '<wallet_id>',
  'ADJUSTMENT',
  <reversal_amount>,  -- positive to credit back
  <corrected_balance>,
  'REV-<original_txn_id>',
  'Reversal of erroneous charge',
  NOW()
);

UPDATE wallet_account
SET balance = <corrected_balance>, updated_at = NOW()
WHERE id = '<wallet_id>';

For Multiple Wallet Discrepancies

1. Pause affected operations if ongoing
2. Identify common factor (time range, merchant, transaction type)
3. Bulk reconciliation:

-- Update all wallets to match transaction sum
UPDATE wallet_account wa
SET balance = (
  SELECT COALESCE(SUM(amount), 0)
  FROM wallet_transaction
  WHERE wallet_id = wa.id
),
updated_at = NOW()
WHERE id IN (<affected_wallet_ids>);

4. Post adjustment transactions for audit trail

For Systematic Issues

1. Escalate to engineering
2. Halt related transactions if needed
3. Root cause analysis
4. Deploy fix
5. Full reconciliation

Post-Resolution

Verification

-- Confirm no remaining discrepancies
SELECT COUNT(*) as remaining_discrepancies
FROM wallet_account wa
WHERE wa.balance != (
  SELECT COALESCE(SUM(amount), 0)
  FROM wallet_transaction
  WHERE wallet_id = wa.id
);

Communication

• Notify affected members if balance changed
• Update support ticket
• Document in incident log

Prevention

• Review for code fix if applicable
• Add additional monitoring
• Update reconciliation frequency if needed

Contacts

Role	Contact
Wallet Engineering	#wallet-eng Slack
On-call Engineer	PagerDuty
Finance/Reconciliation	finance@company.com