A complete technical case: inventory reservation under concurrency

Illustrative scenario created for this report. The goal is to show how the competencies combine; the examples do not constitute a production-ready implementation.

An application needs to let a customer reserve units of an item. There are multiple organizational customers (tenants), simultaneous requests, retries after timeouts, and an event that notifies another service of the reservation's creation.

11.1. The work starts with the contract

Asking to “create a reservation endpoint” leaves important decisions open. Before delegating, the engineer identifies the invariants:

• The available quantity can never become negative.
• During the contracted key-retention window, repeating the same intent must produce a single logical reservation.
• Reusing the same key with different content must produce an explicitly defined result.
• A customer can only act on their own data.
• A partial failure must not produce a reservation without traceability of the corresponding event.
• The system must allow the outcome to be discovered when the connection drops after the database commit.

The engineer also settles with the domain whether the reservation expires, how it is canceled, and how long the idempotency key remains valid. The contract must say what happens when the key expires and when the reservation expires before it: recovering the original operation and starting a new intent with a new key are distinct behaviors. These decisions change the modeling.

A condensed contract might be:

goal: "Reserve units of an item for the authenticated customer"
scope:
  includes: [reservation API, persistence, event, tests, telemetry]
  excludes: [payment, changes to the fulfillment flow]
invariants:
  - "availability >= 0"
  - "one logical reservation per customer and key during the retention window"
  - "same key with a different payload produces a conflict"
  - "no access to another customer's data"
evidence:
  - "test with concurrent requests competing for the last item"
  - "retry test after commit and before the response"
  - "isolation test between customers"
  - "event publication recovery test"
pending_decisions:
  - "expiration policy for the reservation and the key"

The fields are an illustrative template. The contract is only ready for full implementation once the pending items that affect semantics have been resolved.

11.2. The engineer defines the atomicity boundary

A naive solution reads the availability and then writes the reduction. Two requests can observe the same stock and act on a premise that is no longer true.

For an item represented by a single row, a conditional operation can be part of the solution:

-- Illustrative fragment: quantity must be validated as a positive integer.
-- The customer's identity comes from authentication, not from the request body.
UPDATE inventory
SET available = available - :quantity
WHERE tenant_id = :authenticated_tenant
  AND item_id = :item_id
  AND available >= :quantity
RETURNING available;

In PostgreSQL, the semantics of concurrent updates under Read Committed include re-evaluating the condition against the updated version of the row after waiting for a concurrent transaction. It is this documented property, together with the schema constraints, that should ground the reasoning; the visual shape of the query does not prove its adequacy. F28

The fragment does not on its own solve idempotency, reservation creation, full authorization, expiration, or event publication. One possible design guarantees a single inventory row per (tenant_id, item_id) and uses a unique constraint on (tenant_id, idempotency_key) in the operations log. Key coordination, the conditional decrement, the reservation, the retrievable result, and the outbox insert all take part in the same transaction.

The retry path queries the committed result and compares the relevant parameters. Concurrency handling needs to distinguish a conflict, an operation not yet committed, and a completed result, without allowing a second effect. The contract also defines whether refusals, such as insufficient stock, stay bound to the key or allow a new attempt. This is a design proposal for the example, whose implementation requires its own validation.

The retry contract also needs to distinguish the same intent from a different operation with identical parameters. The explicit-identifier approach presented by AWS helps formulate this aspect. F29

11.3. The event requires handling partial failure

Writing the reservation and publishing directly to a broker are two distinct operations. If one happens and the other fails, there can be divergence.

One possibility is to write an event to an outbox in the same transaction as the reservation and use a publisher to deliver it later. This pattern addresses the boundary between persistence and publication. Delivery may repeat messages; consumers need to handle duplicates according to the contract. The AWS documentation describes the pattern and its conditions. F30

The engineer decides whether this design matches the requirements and defines how to observe delays, retry publication, and recognize processed events. AI can implement each component and produce alternatives. The coherence of the contract between them remains an engineering decision.

For the example, each event has a stable identifier. The consumer can record that identifier and apply its effect in the same local transaction, or perform an operation whose contract is idempotent. If the effect happens in another service, the guarantee also depends on that service. The outbox alone does not provide exactly-once effect at the destination.

11.4. The work is split along already-defined interfaces

One possible organization would be:

These fronts can be people, agents, or combinations. The split should only be parallel where dependencies allow. The initial investigation may reveal that a domain decision needs to be resolved before any implementation.

11.5. The evidence goes beyond the happy path

The engineer asks for observable results for specific scenarios:

1. Two requests compete for the last unit: only one gets the reservation.
2. Two requests use the same key: there is a single effect on inventory.
3. The same key reappears with a different quantity: the specified conflict occurs.
4. The process terminates after the commit and before the response: the retry recovers the defined result.
5. A customer tries to reference another customer's item: there is no improper exposure or mutation.
6. The broker accepts the event, but the publisher terminates before recording the publication: resumption may repeat the message; the consumer's deduplication or idempotency prevents a new effect.
7. The reservation expires during concurrent operations: the agreed rule preserves consistency.

Isolated mocks can help with local tests, but they do not demonstrate the database's locking and isolation behavior. The relevant tests should exercise that behavior, coordinating the relevant sequence of operations with barriers or synchronization points. Simply firing requests at the same time may fail to reproduce the intended race. When feasible, introducing a version deliberately vulnerable to the race makes it possible to confirm that the test detects the intended defect.

11.6. The result needs to be deployable and observable

The delivery includes the change, the compatible migration, the relevant tests, and the operational signals. The evidence should identify the validated version. If integration with another change alters the behavior, the corresponding validation needs to be repeated on the integrated result.

For a gradual rollout, one can monitor reservation conflicts, latency, errors, outbox delays, and invariant divergences. Abort conditions should be defined before exposure. Data changes require their own care: rolling back the binary does not automatically undo a mutation that has already been persisted.

The chapter on canarying from Google's SRE Workbook offers a reference for combining release automation, small changes, and evaluation of a version before widening its exposure. F33

11.7. Where the engineer's competence lay

The biggest human contribution lay in defining the intent, recognizing concurrency and partial failures, choosing the contracts, separating fronts, demanding independent evidence, and preparing for operations. The implementation was a field of delegable execution within those conditions.

When finished, the engineer records the useful learnings: an example concurrency test, the idempotency policy, a migration procedure, and the diagnostic signals. The next similar task starts from a better position.