Incomplete data-export while using the CommCare Data Export Tool

Hi,

I am not able to get the complete data exported through the commcare
advance export tool into the sqlite. We exported the same data to xlsx and
it is downloading the full-data. However, when we do it onto SQLite it is
not exporting all the records. We have also tried with the database
SQL-Server and faced the same problem.

If it helps: We found that the number of records getting exported from
commcare is correct when we do it to Excel (eg: 1903 records). When we do
it SQLite or SQL-Server, it is not exporting all the records (eg: 1865
records on both SQLite and SQL-Server). The same pattern is observed with
other forms also (the number of records on Excel is correct, but both the
other ones show the same number which is less).

Please note that we are using the same syntax for both the export methods
(xlsx and sql). The only change is in the “output-format” parameter and
rest all are the same.

Kindly request your urgent support/help on this.

Thanks,
Gurudayal

Hi Simon,

Thanks for your reply. Please note that we are not getting any errors,
except that a warning is coming which we reported earlier (the one that you
mentioned that we can ignore). Other than that we are not getting any
errors/warnings. I am pasting the warning again (just in case if there is
a relation).

“C:\Python27\lib\site-packages\sqlalchemy\engine\default.py:573:
SAWarning: Unicode type received non-unicodebind param
value.param.append(processorskey)”

Thanks in advance for your guidance in helping us to resolve this problem.

Warm Regards,
Gurudayal

··· On Tuesday, 16 December 2014 20:53:03 UTC+5:30, Gurudayal Sahu wrote: > > Hi, > > I am not able to get the complete data exported through the commcare > advance export tool into the sqlite. We exported the same data to xlsx and > it is downloading the full-data. However, when we do it onto SQLite it is > not exporting all the records. We have also tried with the database > SQL-Server and faced the same problem. > > If it helps: We found that the number of records getting exported from > commcare is correct when we do it to Excel (eg: 1903 records). When we do > it SQLite or SQL-Server, it is not exporting all the records (eg: 1865 > records on both SQLite and SQL-Server). The same pattern is observed with > other forms also (the number of records on Excel is correct, but both the > other ones show the same number which is less). > > Please note that we are using the same syntax for both the export methods > (xlsx and sql). The only change is in the "output-format" parameter and > rest all are the same. > > Kindly request your urgent support/help on this. > > Thanks, > Gurudayal > > >

Hi Gurudayal

This is a bit puzzling. Are there any errors that get output during the
export?

··· On 16 December 2014 at 17:23, Gurudayal Sahu wrote: > > Hi, > > I am not able to get the complete data exported through the commcare > advance export tool into the sqlite. We exported the same data to xlsx and > it is downloading the full-data. However, when we do it onto SQLite it is > not exporting all the records. We have also tried with the database > SQL-Server and faced the same problem. > > If it helps: We found that the number of records getting exported from > commcare is correct when we do it to Excel (eg: 1903 records). When we do > it SQLite or SQL-Server, it is not exporting all the records (eg: 1865 > records on both SQLite and SQL-Server). The same pattern is observed with > other forms also (the number of records on Excel is correct, but both the > other ones show the same number which is less). > > Please note that we are using the same syntax for both the export methods > (xlsx and sql). The only change is in the "output-format" parameter and > rest all are the same. > > Kindly request your urgent support/help on this. > > Thanks, > Gurudayal > > > -- > > --- > You received this message because you are subscribed to the Google Groups > "CommCare Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to commcare-developers+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. >


Simon Kelly
Senior Engineer | Dimagi South Africa

Hi Gurudayal

Please could you send an email to commcarehq-support@dimagi.com with the
details and also include the export config file you are using and the
command line command (excluding any usernames or passwords).

This will create a support ticket that we can use to track this issue.

Thanks
Simon

··· On 17 December 2014 at 11:03, Gurudayal Sahu wrote: > > Hi Simon, > > Thanks for your reply. Please note that we are not getting any errors, > except that a warning is coming which we reported earlier (the one that you > mentioned that we can ignore). Other than that we are not getting any > errors/warnings. I am pasting the warning again (just in case if there is > a relation). > > "C:\Python27\lib\site-packages\sqlalchemy\engine\default.py:573: > SAWarning: Unicode type received non-unicodebind param > value.param.append(processors[key](compiled_params[key]))" > > Thanks in advance for your guidance in helping us to resolve this problem. > > Warm Regards, > Gurudayal > > > > On Tuesday, 16 December 2014 20:53:03 UTC+5:30, Gurudayal Sahu wrote: >> >> Hi, >> >> I am not able to get the complete data exported through the commcare >> advance export tool into the sqlite. We exported the same data to xlsx and >> it is downloading the full-data. However, when we do it onto SQLite it is >> not exporting all the records. We have also tried with the database >> SQL-Server and faced the same problem. >> >> If it helps: We found that the number of records getting exported from >> commcare is correct when we do it to Excel (eg: 1903 records). When we do >> it SQLite or SQL-Server, it is not exporting all the records (eg: 1865 >> records on both SQLite and SQL-Server). The same pattern is observed with >> other forms also (the number of records on Excel is correct, but both the >> other ones show the same number which is less). >> >> Please note that we are using the same syntax for both the export methods >> (xlsx and sql). The only change is in the "output-format" parameter and >> rest all are the same. >> >> Kindly request your urgent support/help on this. >> >> Thanks, >> Gurudayal >> >> >> -- > > --- > You received this message because you are subscribed to the Google Groups > "CommCare Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to commcare-developers+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. >


Simon Kelly
Senior Engineer | Dimagi South Africa