Commcare-Export tool Error while exporting Devnagiri data into mysql DB

Hello there,

While using the Commcare-Export tool for exporting Data, the Data is
Exported correctly in the Excel File and also in SQLite DB, however when we
try to Export the Data in the MySql DB, the export breaks and gives us the
following Error : " UnicodeEncodeError: ‘latin-1’ codec can’t encode
characters in position 0-3: ordinal not in range(256) "
(Refer attached Screenshot for the same)

The data is imported correctly into the DB until the Hindi Text is
Encountered, once Hindi text is encountered, it breaks the process and
gives the error.

We understand that the error may be due to the Devnagiri Text being
inserted into the DB, so we tried to solve this issue by changing all the
data columns to " utf8_unicode_ci ", but still the Problem persists.

Any feedback would be appreciated.

Thanks in Advance.
:

You could try append this to the MySQL connection URL: “?charset=utf8”

Please let me know if that works and I’ll update the documentation to
include it.

··· On 1 December 2016 at 15:11, zeeshan Azmi wrote:

Hello there,

While using the Commcare-Export tool for exporting Data, the Data is
Exported correctly in the Excel File and also in SQLite DB, however when we
try to Export the Data in the MySql DB, the export breaks and gives us the
following Error : " UnicodeEncodeError: ‘latin-1’ codec can’t encode
characters in position 0-3: ordinal not in range(256) "
(Refer attached Screenshot for the same)

The data is imported correctly into the DB until the Hindi Text is
Encountered, once Hindi text is encountered, it breaks the process and
gives the error.

We understand that the error may be due to the Devnagiri Text being
inserted into the DB, so we tried to solve this issue by changing all the
data columns to " utf8_unicode_ci ", but still the Problem persists.

Any feedback would be appreciated.

Thanks in Advance.
:


You received this message because you are subscribed to the Google Groups
"commcare-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Simon Kelly
Senior Engineer | Dimagi South Africa

Hello Simon,

Thanks for the speedy reply,

As you mentioned, we did try adding the “?charset=utf8” to the connection
string for commcare-export tool.

However the problem still persists.

We are getting the error at the same point at which the commcare export
tool used to interrupt the execution previously, without using the
“charset” string.

We are using the following command:
web@web-System-Product-Name:~/working/hindi$ commcare-export --output-format
sql --output mysql://username:password@DBserver/DBname?charset=utf8
–project project_name --query config_file

Please refer the attached image for detailed error.

Any feedback would be appreciated.

Thanks in Advance.

You could try append this to the MySQL connection URL: “?charset=utf8”

I think the simplest approach is going to be to use a different MySQL
library.

Please install the pymysql library as follows:

pip install pymysql

And then use the following DB URL format:

mysql+pymysql://:@/?charset=utf8

Thanks
Simon

··· On 2 December 2016 at 08:52, zeeshan Azmi wrote:

Hello Simon,

Thanks for the speedy reply,

As you mentioned, we did try adding the “?charset=utf8” to the connection
string for commcare-export tool.

However the problem still persists.

We are getting the error at the same point at which the commcare export
tool used to interrupt the execution previously, without using the
"charset" string.

We are using the following command:
web@web-System-Product-Name:~/working/hindi$ commcare-export --output-format
sql --output mysql://username:password@DBserver/DBname?charset=utf8
–project project_name --query config_file

Please refer the attached image for detailed error.

Any feedback would be appreciated.

Thanks in Advance.

You could try append this to the MySQL connection URL: “?charset=utf8”


You received this message because you are subscribed to the Google Groups
"commcare-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Simon Kelly
Senior Engineer | Dimagi South Africa

Hello Simon,

I have installed the pysql library,.Its working smoothly now.

Thanks,
Zishan.

Great.

··· On 8 December 2016 at 06:35, zeeshan Azmi wrote:

Hello Simon,

I have installed the pysql library,.Its working smoothly now.

Thanks,
Zishan.


You received this message because you are subscribed to the Google Groups
"commcare-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Simon Kelly
Senior Engineer | Dimagi South Africa