When a user completes a transaction while offline, the transaction is added to the "pending" object store in their IndexedDB database with a status of "ready".
Periodically, the client attempts to transmit the pending transactions to the server. It does this by:
1. Quering for the first entry in the pending store with the state "ready" and changes it to "pending."
2. Attempts to transmit the transaction to the server.
3. Updating the status of the entry in the pending store with the result of the server request. Upon success, the entry is removed from the pending store. Upon network failure, the status is reset to "ready".
If for any reason Step #3 fails, then the pending transaction will be forever stuck in the "pending" state.
This has happened in the past because there was a bug in the code that executed Step #3. However, in rare cases it could also happen if the user closes the browser while Step 2 is in progress.
To recover from failure in Step #3, we can add a "checkoutTime" property to the pending entries, which tracks when the entry was marked as "pending". During step #1, we can identity entries that have been checked out far longer than an HTTP request can expected to last (5 minutes, for example), and restore them to "ready" status.