Even if you can’t make it to the great outdoors, AMPS now makes it easy to visit a range of data in the transaction log.

For years, AMPS has had the ability to use a bookmark subscription to replay messages from the transaction log. A bookmark subscription begins at a bookmark – a point in the transaction log – and the subscription would continue until the point that the application unsubscribed.

Bookmark subscriptions allow applications to resume subscriptions after being disconnected without losing messages, and also provide a way to recreate the sequence of messages to an instance (for example, for auditing or backtesting purposes).

Come to Think of It, Please Fence Me In

But what if an application only wants part of the transaction log (say, messages published between 48 hours ago and 24 hours ago)? The conventional way to solve this with AMPS is for the application to request a bookmark subscription with a timestamp from 48 hours ago to start the replay and then check each message as it arrives to decide if replay has passed the part of the transaction log that was of interest. If the message doesn’t already have a timestamp as part of the message data, the subscription would also need to use the timestamp option so that AMPS would include the time at which the message was processed on the local instance. This approach requires the AMPS server to send more messages than the application would process, since there is no way for the server to know where the application would stop processing messages. Since replay is typically very fast, this could easily be millions of messages more than the application needs.

Starting in the AMPS 5.3.2 preview, AMPS now provides a Bookmark Range feature. This feature lets you decide the exact range of messages for your bookmark subscription to receive. AMPS client applications can now receive an exact set of messages from a subscription during a specific timeframe, including open of business to close using timestamp bookmarks. No longer will clients have to check for that end marker message to see if they should unsubscribe. Now you can specify what specific bookmark you would like AMPS to stop sending messages for the subscription.

Where to Begin, Where to End

AMPS now allows the subscription to set not only the beginning point but also the ending point. Bookmark range also allows you to tell AMPS whether the subscription should receive the initial bookmark, as opposed to regular bookmark replays, where the bookmark provided is not included in the message set returned to the subscription. With a bookmark range, you specify the inclusiveness you would like for your beginning and ending range, and you can choose to include the beginning (and ending) bookmark.

To set a beginning and ending point, a subscription provides a subscription range specifier with the bookmark. The format of the subscription range specifier is as follows:

<begin_interval_specifier> <begin_bookmarks> : <end_bookmarks> <end_interval_specifier>

where the begin_interval_specifier is one of:

and the end_interval_specifier is one of:

Subscriptions may request a completed acknowledgement with their range subscription. AMPS will return the completed acknowledgement when the stopping point is reached.

All currently supported bookmark formats are allowed to be used in bookmark range, this includes timestamps.

To receive all messages matching your subscription for June 4, 2020 you could specify a bookmark like the following: [20200604T000000:20200605T000000). This bookmark tells AMPS to deliver all the messages that match the subscription beginning at midnight June 4 (inclusive) and ending at midnight June 5 (exclusive). (AMPS timestamps are in the UTC timezone, so if your business is in a different time zone, you can adjust the timestamp as needed.)

Plan for the Future

Currently bookmarks not in the transaction log are treated as unknown bookmarks. Subscriptions with unknown bookmarks are set to begin after the last message recorded in the transaction log at the point of subscription, this includes timestamps in the future. On a regular bookmark subscription with a future timestamp as the bookmark the subscription would begin immediately after the last message in the transaction log. The new bookmark range subscription feature supports future timestamps not being returned as the last message, but rather will hold the subscription open until the given timestamp. Future timestamps can be used in both the begin_bookmarks and the end_bookmarks. An application can now enter a subscription to collect all records for the full business day ahead and then complete, even when the application is started before business hours. The subscription would just set two future timestamps along with the inclusiveness that the application needs. AMPS will begin delivering messages at the start time specified and stop delivering at the end time specified. Again, a completed acknowledgement will be sent (if requested) at the point where the subscription ends.

Bookmark range also supports bookmark lists in both the beginning and ending point. AMPS will find the earliest bookmark in the starting list and the latest bookmark in the stopping list. The replay will then act as if those two bookmarks were given as the range start and stop bookmarks.

More Than Just Your Normal Open Ended Subscription

As you can see above, bookmark range can be used in a variety of ways. You can receive messages from a specific range in the past to retrace your steps or set up a future subscription to gather messages for the work ahead. In either case, AMPS automatically begins and ends the subscription exactly at the point you choose, without your application having to do any extra work, and without sending any unnecessary messages.

The new bookmark range feature can be combined with other awesome AMPS features including, but not limited to: filters, select list, and rate.

What are you most excited about with this new feature? How will you use this to make your applications more efficient? Let us know in the comments!