[mongodb-dev] StorageEngine::getAllCommittedTimestamp()

['Mark McAuliffe' via mongodb-dev]

Greetings,

It appears that MongoDB 3.6.5 introduced a new storage API
method: StorageEngine::getAllCommittedTimestamp(). The documentation for
this method says: "Returns the all committed timestamp. All transactions
with timestamps earlier than the all committed timestamp are committed."
This seems to be saying that the "all committed" timestamp is an exclusive
upper. That is, if getAllCommittedTimestamp were to return (100, 0) then
all transactions with strictly lower timestamps must be committed, but
(100, 0) itself presumably isn't. However, my experiments with WiredTiger
suggest that it actually reports an "all committed" timestamp that is
inclusive. For example, in a situation where the oldest active transaction
has timestamp (1003, 0), WiredTiger's getAllCommittedTimestamp returned
(1002, 4294967295) -- it went out of its way to return a timestamp that is
less than (1003, 0). Can someone please confirm that the "all committed"
timestamp is actually inclusive, not exclusive?

Thank you,
- Mark

['Eric Milkie' via mongodb-dev]

Hi Mark,
The function API does not currently dictate whether a transaction with
exactly the timestamp returned is committed or not, as it originally did
not matter to the code logic calling this function. It turns out that
WiredTiger did originally return the timestamp of the earliest uncommitted
transaction; this turned out to be problematic for some logic involving
visibility and replication, so later in development we changed it to return
"one" less than the timestamp of the earliest uncommitted transaction, thus
making the function inclusive as you noticed. If there are no uncommitted
transactions, it returns the latest timestamp currently known.
I should change the function comment to reflect this; thanks for pointing
it out.
-Eric

On Tue, Jun 19, 2018 at 6:27 PM, 'Mark McAuliffe' via mongodb-dev <

Post by 'Mark McAuliffe' via mongodb-dev
Greetings,
It appears that MongoDB 3.6.5 introduced a new storage API
method: StorageEngine::getAllCommittedTimestamp(). The documentation for
this method says: "Returns the all committed timestamp. All transactions
with timestamps earlier than the all committed timestamp are committed."
This seems to be saying that the "all committed" timestamp is an exclusive
upper. That is, if getAllCommittedTimestamp were to return (100, 0) then
all transactions with strictly lower timestamps must be committed, but
(100, 0) itself presumably isn't. However, my experiments with WiredTiger
suggest that it actually reports an "all committed" timestamp that is
inclusive. For example, in a situation where the oldest active transaction
has timestamp (1003, 0), WiredTiger's getAllCommittedTimestamp returned
(1002, 4294967295) -- it went out of its way to return a timestamp that is
less than (1003, 0). Can someone please confirm that the "all committed"
timestamp is actually inclusive, not exclusive?
Thank you,
- Mark
--
You received this message because you are subscribed to the Google Groups
"mongodb-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/mongodb-dev.
To view this discussion on the web visit https://groups.google.com/d/ms
gid/mongodb-dev/e4b38efd-4a13-414f-9c0b-1f588c5c656d%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-dev/e4b38efd-4a13-414f-9c0b-1f588c5c656d%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

['Mark McAuliffe' via mongodb-dev]

Thank you for your reply, Eric. It is very helpful to have this
confirmation. My original (exclusive upper) implementation ran into trouble
(perhaps the same sort of trouble that you did?). Switching to inclusive
upper appears to have solved those problems, so an updated comment would
definitely be helpful.
Thank you again,
- Mark

Post by 'Eric Milkie' via mongodb-dev
Hi Mark,
The function API does not currently dictate whether a transaction with
exactly the timestamp returned is committed or not, as it originally did
not matter to the code logic calling this function. It turns out that
WiredTiger did originally return the timestamp of the earliest uncommitted
transaction; this turned out to be problematic for some logic involving
visibility and replication, so later in development we changed it to return
"one" less than the timestamp of the earliest uncommitted transaction, thus
making the function inclusive as you noticed. If there are no uncommitted
transactions, it returns the latest timestamp currently known.
I should change the function comment to reflect this; thanks for pointing
it out.
-Eric
On Tue, Jun 19, 2018 at 6:27 PM, 'Mark McAuliffe' via mongodb-dev <

Post by 'Mark McAuliffe' via mongodb-dev
Greetings,
It appears that MongoDB 3.6.5 introduced a new storage API
method: StorageEngine::getAllCommittedTimestamp(). The documentation for
this method says: "Returns the all committed timestamp. All transactions
with timestamps earlier than the all committed timestamp are committed."
This seems to be saying that the "all committed" timestamp is an exclusive
upper. That is, if getAllCommittedTimestamp were to return (100, 0) then
all transactions with strictly lower timestamps must be committed, but
(100, 0) itself presumably isn't. However, my experiments with WiredTiger
suggest that it actually reports an "all committed" timestamp that is
inclusive. For example, in a situation where the oldest active transaction
has timestamp (1003, 0), WiredTiger's getAllCommittedTimestamp returned
(1002, 4294967295) -- it went out of its way to return a timestamp that is
less than (1003, 0). Can someone please confirm that the "all committed"
timestamp is actually inclusive, not exclusive?
Thank you,
- Mark
--
You received this message because you are subscribed to the Google Groups
"mongodb-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
<javascript:>.
Visit this group at https://groups.google.com/group/mongodb-dev.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-dev/e4b38efd-4a13-414f-9c0b-1f588c5c656d%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-dev/e4b38efd-4a13-414f-9c0b-1f588c5c656d%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.