Applying change_log

Developers
1

Hello,

I am applying the change log historical-auditcare-data-migration

the steps going well, migrating the data on descending order(date from recent to old data).

commcare local instance setup and project app were built in 2018, but the migration is still processing back to the 2014 year like this.

2022-09-01 09:46:23,992 INFO [corehq.apps.auditcare.couch_to_sql] Batch finished: 2016-06-08 20:00:00 - 2016-06-08 19:00:00

2022-09-01 09:46:23,994 INFO [corehq.apps.auditcare.couch_to_sql] Batch finished: 2016-06-08 19:00:00 - 2016-06-08 18:00:00

2022-09-01 09:46:24,194 INFO [corehq.apps.auditcare.couch_to_sql] Batch finished: 2016-06-08 21:00:00 - 2016-06-08 20:00:00

2022-09-01 09:46:24,500 INFO [corehq.apps.auditcare.couch_to_sql] Batch finished: 2016-06-08 18:00:00 - 2016-06-08 17:00:00
.............

2022-09-01 10:04:11,796 INFO [corehq.apps.auditcare.couch_to_sql] Batch finished: 2014-11-20 06:00:00 - 2014-11-20 05:00:00
2022-09-01 10:04:11,800 INFO [corehq.apps.auditcare.couch_to_sql] Starting batch: 2014-11-20 01:00:00 - 2014-11-20 00:00:00
2022-09-01 10:04:11,801 INFO [corehq.apps.auditcare.couch_to_sql] Starting batch: 2014-11-20 00:00:00 - 2014-11-19 23:00:00
2022-09-01 10:04:11,802 INFO [corehq.apps.auditcare.couch_to_sql] Starting batch: 2014-11-19 23:00:00 - 2014-11-19 22:00:00
2022-09-01 10:04:11,802 INFO [corehq.apps.auditcare.couch_to_sql] Starting batch: 2014-11-19 22:00:00 - 2014-11-19 21:00:00
2022-09-01 10:04:11,803 INFO [corehq.apps.auditcare.couch_to_sql] Starting batch: 2014-11-19 21:00:00 - 2014-11-19 20:00:00
2022-09-01 10:04:11,829 INFO [corehq.apps.auditcare.couch_to_sql] Batch finished: 2014-11-20 01:00:00 - 2014-11-20 00:00:00
2022-09-01 10:04:11,832 INFO [corehq.apps.auditcare.couch_to_sql] Batch finished: 2014-11-19 21:00:00 - 2014-11-19 20:00:00
2022-09-01 10:04:11,833 INFO [corehq.apps.auditcare.couch_to_sql] Batch finished: 2014-11-19 23:00:00 - 2014-11-19 22:00:00
2022-09-01 10:04:11,834 INFO [corehq.apps.auditcare.couch_to_sql] Batch finished: 2014-11-19 22:00:00 - 2014-11-19 21:00:00
2022-09-01 10:04:11,836 INFO [corehq.apps.auditcare.couch_to_sql] Batch finished: 2014-11-20 00:00:00 - 2014-11-19 23:00:00
2022-09-01 10:04:11,841 INFO [corehq.apps.auditcare.couch_to_sql] Starting batch: 2014-11-19 20:00:00 - 2014-11-19 19:00:00

Is that normal?

2

Hello,

Migration is successful. I don’t know why it processed till 2012-12-31

2022-09-01 10:24:40,570 INFO [corehq.apps.auditcare.couch_to_sql] Starting batch: 2012-12-31 01:00:00 - 2012-12-31 00:00:00
2022-09-01 10:24:40,583 INFO [corehq.apps.auditcare.couch_to_sql] Batch finished: 2012-12-31 01:00:00 - 2012-12-31 00:00:00
2022-09-01 10:24:40,585 INFO [corehq.apps.auditcare.couch_to_sql] Batch finished: 2012-12-31 02:00:00 - 2012-12-31 01:00:00
2022-09-01 10:24:40,588 INFO [corehq.apps.auditcare.utils.migration] Migration Successfull
No batches to process

3

Hi Demis, yes, that is normal - it looks like the process chunks by hour from the date of the first SQL record back until 2012-12-31. I think this is because the chunks are processed in parallel, and we don't know in advance how to divide into even portions. The chunks with no data in them should just complete quickly.

1 Like