Hi Simon,
Thanks for your suggestion.
I have upgraded the commcare-export tool by following command:
pip install -U commcare-export
but still, it is giving me same error (as sent in previous mail's attached
file), as follows:
Note: I am sending last lines of error message:
sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError)
(1050, "Table 'registration' already exists") [SQL: u'\nCREATE TABLE
Registration (\n\tid VARCHAR(255) NOT NULL, \n\tusername VARCHAR(32),
\n\tform.sevikaname VARCHAR(32), \n\tform.awwnumber VARCHAR(32),
\n\tstarted_time VARCHAR(32), \n\treceived_on VARCHAR(32), \n\tcaseid
VARCHAR(32), \n\tformid VARCHAR(41), \n\tform.beatid VARCHAR(32),
\n\tcase_name VARCHAR(32), \n\tform.coid VARCHAR(32), \n\tcompleted_time
VARCHAR(32), \n\tform.uniqueid VARCHAR(32), \n\tPRIMARY KEY (id)\n)\n\n']
Please suggest the way forward to resolve this, as it is now become
mission critical issue.
For your current reference, please find below snapshot:
[image: Inline image 1]
···
--
Thanks,
*Ajay Singh*
*Corporate** Renaissance **Group*
Cell (India): +91-9711628763
Email: ajay.singh@crgoup.co.in
[image: CR Group] http://www.crgroup.com/[image: Rapid-Fire Business
Intelligence & Analytics] http://www.tableau.com/
On Mon, Dec 12, 2016 at 4:08 PM, Simon Kelly skelly@dimagi.com wrote:
Hi Ajay
The issue you're experiencing is a bug in the version of the export tool
you are using. Upgrading to the latest version will fix it:
pip install -U commcare-export
Thanks
Simon
On 9 December 2016 at 20:46, Ajay Singh ajay.singh@crgroup.co.in wrote:
Hi Simon,
Please find below answers to your queries:
-
Operating System: Windows 10 Pro
-
Python Version and pip freeze
[image: Inline image 1]
-
Exact output is attached as file 'CommCare-Export_Error.txt'.
Please analyse and suggest.
--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763 <+91%2097116%2028763>
Email: ajay.singh@crgoup.co.in
[image: CR Group] http://www.crgroup.com/[image: Rapid-Fire Business
Intelligence & Analytics] http://www.tableau.com/
On Fri, Dec 9, 2016 at 7:43 PM, Simon Kelly skelly@dimagi.com wrote:
What operating system are you using? Can you please show the output of
the following commands:
python --version
pip freeze
Please also send the exact output from the export tool.
Thanks.
On 9 December 2016 at 15:51, Ajay Singh ajay.singh@crgroup.co.in wrote:
Hi Simon,
Please refer below the steps of my query:
-
If I am exporting the data for first time, it is getting exported to
MySQL perfectly.
-
If I am exporting the same data/form, 2nd time, without any
change/insert in my form, the commcare-export tool executing without any
error.
-
Now, if I am adding more records in my form, and exporting same form
with some added records,
then commcare-export tool showing error message like, "Table Already
Exists". Yes table was created in step #1.
And, I am running following command for export:
commcare-export --username AAAAAA --password ###### --output-format sql
--output mysql://root:root@localhost:3306/
myDbName --project my-test-proj --query D:
QueryFile.xlsx
Issue is coming only when, MySQL table already exists, and I am adding
some more records in my form, then exporting the form's data. See step #3
above.
--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763 <+91%2097116%2028763>
Email: ajay.singh@crgoup.co.in
[image: CR Group] http://www.crgroup.com/[image: Rapid-Fire Business
Intelligence & Analytics] http://www.tableau.com/
On Fri, Dec 9, 2016 at 6:52 PM, Simon Kelly skelly@dimagi.com wrote:
Please show the exact command you are running (without usernames etc)
On 9 December 2016 at 15:00, Ajay Singh ajay.singh@crgroup.co.in wrote:
Hi Simon,
Please help us on the requested query.
--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763 <+91%2097116%2028763>
Email: ajay.singh@crgoup.co.in
[image: CR Group] http://www.crgroup.com/[image: Rapid-Fire
Business Intelligence & Analytics] http://www.tableau.com/
On Fri, Dec 9, 2016 at 1:48 AM, Ajay Singh ajay.singh@crgroup.co.in wrote:
Thanks Simon.
Its all becomes possible because of prompt support from your side.
Now the data is getting exported to MySQL table from CommCare HQ.
Its Ok, but now, I am facing next challenge, and explaining step by
step:
-
If I am exporting the data for first time, it is getting exported
to MySQL perfectly.
-
If I am exporting the same data/form, 2nd time, without any
change/insert in my form, the commcare-export tool executing without any
error.
-
Now, if I am adding more records in my form, and exporting same
form with some added records,
then commcare-export tool showing error message like, "Table
Already Exists". Yes table was created in step #1.
From commcare-export tool behavior, I have observed that, it is
creating the table every time, you export the data.
My expectation is,
if table does not exists, then it should create the table,
else if table exists, then it should insert new records in the
table, without showing error message like, "Table Already Exists".
Note: I am not using --since parameter, as I am looking for
automating this data export process, and can not do dynamic date change in
my command line.
Please suggest the resolution.
--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763 <+91%2097116%2028763>
Email: ajay.singh@crgoup.co.in
[image: CR Group] http://www.crgroup.com/[image: Rapid-Fire
Business Intelligence & Analytics] http://www.tableau.com/
On Thu, Dec 8, 2016 at 6:33 PM, Simon Kelly skelly@dimagi.com wrote:
Great that you got it working!
On 8 December 2016 at 14:54, Ajay Singh ajay.singh@crgroup.co.in wrote:
Hi Simon,
Many-many thanks for your prompt support and reply.
I referred the documentation again, and modified my --query file
for 'id' field.
Now I got succeeded in Commcare-Export to MySQL data
integration.
Once again, many thanks to Simon, Ben and Ethan, and whole
CommCare Support team.
--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763 <+91%2097116%2028763>
Email: ajay.singh@crgoup.co.in
[image: CR Group] http://www.crgroup.com/[image: Rapid-Fire
Business Intelligence & Analytics] http://www.tableau.com/
On Thu, Dec 8, 2016 at 1:08 PM, Simon Kelly skelly@dimagi.com wrote:
As is mentioned in the documentation: Note that for use with a
SQL database, a field named id is required.
On 8 December 2016 at 00:23, Ajay Singh <ajay.singh@crgroup.co.in wrote:
Hi Ethan,
Thank you.
The URL,
python - ImportError: No module named MySQLdb - Stack Overflow
odule-named-mysqldb
has given me idea about installing MySQL-Python from pip win,
but unfortunately it did not worked for me.
However, I surfed and got a site: https://www.youtube.com/
watch?v=soeIgysM-XU
which has given me the idea to install: MySQL-python-1.2.5.win32-py2.7.exe
to resolve MySQL issue, and I got succeeded to resolve MySQL error at my
end.
Thank you. I moved one step ahead.
Now when I am using commcare-export to export data to MySQL, it
is automatically adding an 'id' field as primary key in MySQL table, but
there no such field or value coming from CommCare HQ or in my --query file,
so the insert statement is getting failed. Although the data is downloading
and showing on screen, but no inserts in MySQL.
Please refer below error message and suggest:
[image: Inline image 1]
--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763 <+91%2097116%2028763>
Email: ajay.singh@crgoup.co.in
[image: CR Group] http://www.crgroup.com/[image: Rapid-Fire
Business Intelligence & Analytics] http://www.tableau.com/
On Thu, Dec 8, 2016 at 12:45 AM, Ajay Singh < ajay.singh@crgroup.co.in> wrote:
Thanks Ethan,
I am going thru with the given URL and will update you on my
progress.
--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763 <+91%2097116%2028763>
Email: ajay.singh@crgoup.co.in
[image: CR Group] http://www.crgroup.com/[image: Rapid-Fire
Business Intelligence & Analytics] http://www.tableau.com/
On Thu, Dec 8, 2016 at 12:32 AM, Ethan Soergel < esoergel@dimagi.com> wrote:
Did you see my reply on December 2nd? I'll copy that again
here in case you missed it:
It looks like that module may have some binary components that
need to be installed separately. Here is a stack overflow thread with a
number of potential solutions to this issue:
python - ImportError: No module named MySQLdb - Stack Overflow
odule-named-mysqldb
When I searched for your error message on google, I found a
number of reports of similar issues. "pip install mysql" might not be
enough, you may need to install some dependencies through your operating
system's package manager (apt-get, for Ubuntu), as suggested in the above
link. SQLAlchemy is a library used for interacting with SQL databases from
python programs.
On Wed, Dec 7, 2016 at 1:36 PM, Ajay Singh < ajay.singh@crgroup.co.in> wrote:
Hi Support team,
This is to put in revised followup, regarding CommCare-Export
tool's data export to MySQL database;
we have tried in many ways, as whatever available resources
in your site as well as in github, but every time, we are facing issue
related to 'sqlalchemy'.
We have installed required libraries for MySQL, as suggested
by Simon in last mail (
"pip install pymysql"), but still issue not resolved.
Kindly suggest and help us to resolve, are we missing some
steps related to configuration of 'sqlalchemy'?, also please elaborate as
what is this 'sqlalchemy'?
Will appreciate your quick help in this context, as it is
long pending request at your end.
--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763 <+91%2097116%2028763>
Email: ajay.singh@crgoup.co.in
[image: CR Group] http://www.crgroup.com/[image:
Rapid-Fire Business Intelligence & Analytics]
http://www.tableau.com/
On Fri, Dec 2, 2016 at 9:29 PM, Ajay Singh < ajay.singh@crgroup.co.in> wrote:
Thanks Simon,
I have installed MySQL libraries by using
"pip install pymysql" command. It got installed successfully.
On running CommCare export command, I am getting below error
message.
I am using following format for MySQL db URL:
mysql://username:password@host:port/database_name
[image: Inline image 1]
--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763 <+91%2097116%2028763>
Email: ajay.singh@crgoup.co.in
[image: CR Group] http://www.crgroup.com/[image:
Rapid-Fire Business Intelligence & Analytics]
http://www.tableau.com/
On Fri, Dec 2, 2016 at 8:46 PM, Simon Kelly < skelly@dimagi.com> wrote:
You can install the package with
"pip install pymysql"
On 2 December 2016 at 16:40, Ajay Singh < ajay.singh@crgroup.co.in> wrote:
Hi Support Team,
We are trying to export data from CommCare Export tool to
MySQL Database. We are using following DB URL format:
mysql://username:password@host:port/database_name
But the tool is giving following error message. Please
review and suggest the solution.
[image: Inline image 1]
--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763 <+91%2097116%2028763>
Email: ajay.singh@crgoup.co.in
[image: CR Group] http://www.crgroup.com/[image:
Rapid-Fire Business Intelligence & Analytics]
http://www.tableau.com/
--
Simon Kelly
Senior Engineer | Dimagi South Africa
--
You received this message because you are subscribed to the
Google Groups "CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to commcare-developers+unsubscrib
e@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
--
Simon Kelly
Senior Engineer | Dimagi South Africa
--
Simon Kelly
Senior Engineer | Dimagi South Africa
--
Simon Kelly
Senior Engineer | Dimagi South Africa
--
Simon Kelly
Senior Engineer | Dimagi South Africa
--
Simon Kelly
Senior Engineer | Dimagi South Africa
