Synchronization

Every change to a transaction or a subscriber, whoever made it, goes into a sequence. A ping tells you that the sequence has grown, and seq returns the changed objects, whole, from where you left off.

Requests

The model

There is no webhook that carries the data. A ping carries only the shop's ID and how long its sequence is now. You answer it at once, and fetch what changed with seq:

Seq is a log. Each change, such as a payment, a capture, or a renewal, appends the object it changed, and {seq} is a position in it. Ask from 0, and seq returns every transaction and subscriber the shop has; ask from where your last call ended, and it returns what changed since. So synchronizing is a loop:

seq := LoadSeq()                  -- 0 the first time
loop
    data := GET("https://api.gjold.com/v2/{shopid}:seq:" ‖ seq)
    for each change in data.changes do
        Apply(change)             -- if its rev is higher than yours
    end for
    if data.seq = seq then break  -- caught up
    seq := data.seq
    StoreSeq(seq)
end loop

Run it when a ping comes, and when your server starts. A ping's "seq" is where the sequence ends, so a ping whose "seq" you have reached needs no call.

Why it works

• Nothing is lost. Seq keeps every change, so a client that was down for a day catches up from where it stopped, and a client that lost its database starts again from 0.
• Nothing is applied twice, or out of order. Each object comes whole, with a "rev" that counts its versions, so the newer copy always wins, however often or late a copy arrives.
• Order is kept where it matters. Objects arrive in the order they were created, and a subscriber before its charges.
• It scales. One call returns many changes, and an object that changed many times may come once, in its latest state.

Where it goes wrong

The requests are simple, but synchronizing at scale is not: two pings at once start two synchronizations, which race; a lock that a crash leaves held stops them all; and a "seq" stored before its changes are applied loses them. The implementation note on synchronizing goes through each, and how to avoid it, and Receiving pings how to check and answer a ping.