You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
tap-rest-api keeps a copy of the last extracted record in the bookmark (aka state) together with the last recorded index or timestamp. The extra information is used to ignore the same record in the next run with the bookmark whose start index/time is inclusive.
However, It is not a good practice to include a raw record in the bookmark items mainly for security reasons.
So we should
Sort the last record (except the extraction time key), by keys
create a md5 digest
include the digest instead of raw record in the bookmark.
To ensure backward compatibility, the dup check is made for both raw record and the digest.
The text was updated successfully, but these errors were encountered:
* Use digest for dup check instead of raw record #24
* make timestamp in bookmark integer
* handle integerized timestamp from bookmark
* handle None bookmark case for timestamp
tap-rest-api keeps a copy of the last extracted record in the bookmark (aka state) together with the last recorded index or timestamp. The extra information is used to ignore the same record in the next run with the bookmark whose start index/time is inclusive.
However, It is not a good practice to include a raw record in the bookmark items mainly for security reasons.
So we should
To ensure backward compatibility, the dup check is made for both raw record and the digest.
The text was updated successfully, but these errors were encountered: