Problem with checkpoints in the export API

This note is for any API users. If you aren’t an API user (or don’t know
what that means) you can ignore this email.

One of the unanticipated consequences of the migration that happened
yesterday was that we accidentally broke the "previous_export"
functionality of the export API[1]. What this means is that any existing
export_id’s no longer correctly point to the correct state of the data when
that export was taken. More information below.

What are the symptoms of this?

··· * * If you are hitting an export API using the "previous_export" parameter and a previously generated export ID, you will get back empty or incorrect data, instead of the newest data you would expect.

Did we lose any data?

No, all your data is perfectly safe and sound. The only issue is with the
checkpoints.

What can you do?
*
*
Unfortunately, we can’t reconstruct the checkpoint IDs anymore. What this
means is that you should do an initial export with no "previous_export"
value passed in (which may take a long time to complete), and then use that
as a new starting checkpoint. From there everything should behave as it did
previously.

How did this happen?

It was an oversight on our part, as the checkpointing system relied on the
assumption that the backend database would never change. We’re looking into
removing this requirement so that this problem won’t happen in the future.

If you have any questions or need any help resolving this, don’t hesitate
to contact us at commcarehq-support@dimagi.com

thanks,
HQ team


[1] https://wiki.commcarehq.org/display/commcarepublic/Export+API