Hello Simon,
As shared in my earlier mail, regarding exporting CommCare data to MySQL
using commcare-export tool,
I have re-installed and re-configured complete python and commcare setup in
my laptop, but the issue is as it is,
i.e. first export to MySQL, then adding some records in CommCare web, then
exporting same form's data to existing table in MySQL, it is showing same
error
"Table 'registration' already exists", that I was facing previously. In
my expectations, it should automatically sync incremental records.
Simon, now its too long way to hit same ground again and again, so I will
request for your help,
Lets take my machine's control over Google hangout and investigate the
problem their in.
We propose and request this session for Monday, 19-Dec-2016.
Please share your contact number so that we can reach you. In next mail, I
am sending you meeting invite for Monday, 19-Dec-2016, for remote session.
Please find my cell number in below signature panel.
···
-- Thanks, *Ajay Singh* *Corporate** Renaissance **Group* Cell (India): +91-9711628763 Email: ajay.singh@crgoup.co.in[image: CR Group] http://www.crgroup.com/[image: Rapid-Fire Business
Intelligence & Analytics] http://www.tableau.com/
On Wed, Dec 14, 2016 at 2:17 AM, Ajay Singh ajay.singh@crgroup.co.in wrote:
Hi Simon,
Thanks for your suggestion and continuous support. That's your kindness.
I have followed the way you suggested, i.e. uninstalled and re-installed
commcare-export, but sorry to say, the same issue still persists.I am getting same error message, i.e.
"Table 'registration' already exists"Next, I am thinking to uninstall the complete python setup, including
commcare-export from my laptop, and will do complete setup again.Lets see, what will happen.
Please suggest.
--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763
Email: ajay.singh@crgoup.co.in[image: CR Group] http://www.crgroup.com/[image: Rapid-Fire Business
Intelligence & Analytics] http://www.tableau.com/On Tue, Dec 13, 2016 at 6:14 PM, Simon Kelly skelly@dimagi.com wrote:
Hi Ayjay
I don't think this is a library issue. Nobody else using the tools has
had this issue. I'm sure it's something to do with the specific
configuration on your computer. It's possible that there are stale files
that need to be removed.Please try uninstalling and re-installing commcare-export:
pip uninstall commcare-export
pip install commcare-exportOn 12 December 2016 at 20:53, Ajay Singh ajay.singh@crgroup.co.in wrote:
Hi Simon,
Thanks for your suggestion.
By using the given way (i.e. mysql+pymysql://root:root@localhost:3306/myDbName),
I am getting following error:sqlalchemy.exc.InternalError: (pymysql.err.InternalError) (1050, u
"Table 'registration' already exists") [SQL: u'\nCREATE TABLE
registration
(\n\tid VARCHAR(255) NOT NULL, \n\tusername VARCHAR(32)
COLLATE utf8_bin, \n\tform.sevikaname
VARCHAR(32) COLLATE utf8_bin,
\n\tform.awwnumber
VARCHAR(32) COLLATE utf8_bin, \n\tstarted_time
VARCHAR(32) COLLATE utf8_bin, \n\treceived_on VARCHAR(32) COLLATE utf8_bin,
\n\tcaseid VARCHAR(32) COLLATE utf8_bin, \n\tformid VARCHAR(41) COLLATE
utf8_bin, \n\tform.beatid
VARCHAR(32) COLLATE utf8_bin, \n\tcase_name
VARCHAR(32) COLLATE utf8_bin, \n\tform.coid
VARCHAR(32) COLLATE utf8_bin,
\n\tcompleted_time VARCHAR(32) COLLATE utf8_bin, \n\tform.uniqueid
VARCHAR(32) COLLATE utf8_bin, \n\tPRIMARY KEY (id)\n)\n\n']I am using MySQL-Python Library 1.2.5, which was installed by the setup
'MySQL-python-1.2.5.win32-py2.7.exe'Can you please suggest me the relevant MySQL-Python library, in which
this issue was fixed.I will also keep trying some other ways available in the internet (if
any), and will update you accordingly.--
AjayOn Monday, December 12, 2016 at 6:29:04 PM UTC+5:30, Simon Kelly wrote:
The only other thing I can think of is that it's not using the correct
MySQL library. Please try change the connection url to the following:mysql+pymysql://root:root@localhost:3306/myDbName
On 12 December 2016 at 14:44, Ajay Singh ajay....@crgroup.co.in wrote:
Hi Simon,
Thanks for your suggestion.
I have upgraded the commcare-export tool by following command:
pip install -U commcare-exportbut still, it is giving me same error (as sent in previous mail's
attached file), as follows:Note: I am sending last lines of error message:
sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError)
(1050, "Table 'registration' already exists") [SQL: u'\nCREATE TABLE
Registration
(\n\tid VARCHAR(255) NOT NULL, \n\tusername VARCHAR(32),
\n\tform.sevikaname
VARCHAR(32), \n\tform.awwnumber
VARCHAR(32),
\n\tstarted_time VARCHAR(32), \n\treceived_on VARCHAR(32), \n\tcaseid
VARCHAR(32), \n\tformid VARCHAR(41), \n\tform.beatid
VARCHAR(32),
\n\tcase_name VARCHAR(32), \n\tform.coid
VARCHAR(32), \n\tcompleted_time
VARCHAR(32), \n\tform.uniqueid
VARCHAR(32), \n\tPRIMARY KEY (id)\n)\n\n']Please suggest the way forward to resolve this, as it is now become
mission critical issue.For your current reference, please find below snapshot:
[image: Inline image 1]
--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763
Email: ajay....@crgoup.co.in[image: CR Group] http://www.crgroup.com/[image: Rapid-Fire
Business Intelligence & Analytics] http://www.tableau.com/On Mon, Dec 12, 2016 at 4:08 PM, Simon Kelly ske...@dimagi.com wrote:
Hi Ajay
The issue you're experiencing is a bug in the version of the export
tool you are using. Upgrading to the latest version will fix it:
pip install -U commcare-exportThanks
SimonOn 9 December 2016 at 20:46, Ajay Singh ajay....@crgroup.co.in wrote:
Hi Simon,
Please find below answers to your queries:
Operating System: Windows 10 Pro
Python Version and pip freeze
[image: Inline image 1]Exact output is attached as file 'CommCare-Export_Error.txt'.
Please analyse and suggest.
--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763
Email: ajay....@crgoup.co.in[image: CR Group] http://www.crgroup.com/[image: Rapid-Fire
Business Intelligence & Analytics] http://www.tableau.com/On Fri, Dec 9, 2016 at 7:43 PM, Simon Kelly ske...@dimagi.com wrote:
What operating system are you using? Can you please show the output
of the following commands:python --version pip freeze
Please also send the exact output from the export tool.
Thanks.
On 9 December 2016 at 15:51, Ajay Singh ajay....@crgroup.co.in wrote:
Hi Simon,
Please refer below the steps of my query:
If I am exporting the data for first time, it is getting
exported to MySQL perfectly.If I am exporting the same data/form, 2nd time, without any
change/insert in my form, the commcare-export tool executing without any
error.Now, if I am adding more records in my form, and exporting same
form with some added records,
then commcare-export tool showing error message like, "Table
Already Exists". Yes table was created in step #1.And, I am running following command for export:
commcare-export --username AAAAAA --password ######
--output-format sql --output mysql://root:root@localhost:3306/
myDbName --project my-test-proj --query D:
QueryFile.xlsxIssue is coming only when, MySQL table already exists, and I am
adding some more records in my form, then exporting the form's data. See
step #3 above.--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763
Email: ajay....@crgoup.co.in[image: CR Group] http://www.crgroup.com/[image: Rapid-Fire
Business Intelligence & Analytics] http://www.tableau.com/On Fri, Dec 9, 2016 at 6:52 PM, Simon Kelly ske...@dimagi.com wrote:
Please show the exact command you are running (without usernames
etc)On 9 December 2016 at 15:00, Ajay Singh ajay....@crgroup.co.in wrote:
Hi Simon,
Please help us on the requested query.
--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763
Email: ajay....@crgoup.co.in[image: CR Group] http://www.crgroup.com/[image: Rapid-Fire
Business Intelligence & Analytics] http://www.tableau.com/On Fri, Dec 9, 2016 at 1:48 AM, Ajay Singh < ajay....@crgroup.co.in> wrote:
Thanks Simon.
Its all becomes possible because of prompt support from your
side.Now the data is getting exported to MySQL table from CommCare
HQ.Its Ok, but now, I am facing next challenge, and explaining
step by step:
If I am exporting the data for first time, it is getting
exported to MySQL perfectly.If I am exporting the same data/form, 2nd time, without any
change/insert in my form, the commcare-export tool executing without any
error.Now, if I am adding more records in my form, and exporting
same form with some added records,
then commcare-export tool showing error message like, "Table
Already Exists". Yes table was created in step #1.From commcare-export tool behavior, I have observed that, it is
creating the table every time, you export the data.My expectation is,
if table does not exists, then it should create the table,
else if table exists, then it should insert new records in the
table, without showing error message like, "Table Already
Exists".Note: I am not using --since parameter, as I am looking for
automating this data export process, and can not do dynamic date change in
my command line.Please suggest the resolution.
--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763
Email: ajay....@crgoup.co.in[image: CR Group] http://www.crgroup.com/[image: Rapid-Fire
Business Intelligence & Analytics] http://www.tableau.com/On Thu, Dec 8, 2016 at 6:33 PM, Simon Kelly ske...@dimagi.com wrote:
Great that you got it working!
On 8 December 2016 at 14:54, Ajay Singh < ajay....@crgroup.co.in> wrote:
Hi Simon,
Many-many thanks for your prompt support and reply.
I referred the documentation again, and modified my --query
file for 'id' field.Now I got succeeded in Commcare-Export to MySQL data
integration.Once again, many thanks to Simon, Ben and Ethan, and whole
CommCare Support team.--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763
Email: ajay....@crgoup.co.in[image: CR Group] http://www.crgroup.com/[image:
Rapid-Fire Business Intelligence & Analytics]
http://www.tableau.com/On Thu, Dec 8, 2016 at 1:08 PM, Simon Kelly < ske...@dimagi.com> wrote:
As is mentioned in the documentation: Note that for use
with a SQL database, a field named id is required.On 8 December 2016 at 00:23, Ajay Singh < ajay....@crgroup.co.in> wrote:
Hi Ethan,
Thank you.
The URL,
http://stackoverflow.com/quest
ions/22252397/importerror-no-module-named-mysqldb
has given me idea about installing MySQL-Python from pip
win, but unfortunately it did not worked for me.However, I surfed and got a site: https://www.youtube.com/
watch?v=soeIgysM-XUwhich has given me the idea to
install: MySQL-python-1.2.5.win32-py2.7.exe to resolve
MySQL issue, and I got succeeded to resolve MySQL error at my end.Thank you. I moved one step ahead.
Now when I am using commcare-export to export data to
MySQL, it is automatically adding an 'id' field as primary key in MySQL
table, but there no such field or value coming from CommCare HQ or in my
--query file, so the insert statement is getting failed. Although the data
is downloading and showing on screen, but no inserts in MySQL.Please refer below error message and suggest:
[image: Inline image 1]
--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763
Email: ajay....@crgoup.co.in[image: CR Group] http://www.crgroup.com/[image:
Rapid-Fire Business Intelligence & Analytics]
http://www.tableau.com/On Thu, Dec 8, 2016 at 12:45 AM, Ajay Singh < ajay....@crgroup.co.in> wrote:
Thanks Ethan,
I am going thru with the given URL and will update you on
my progress.--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763
Email: ajay....@crgoup.co.in[image: CR Group] http://www.crgroup.com/[image:
Rapid-Fire Business Intelligence & Analytics]
http://www.tableau.com/On Thu, Dec 8, 2016 at 12:32 AM, Ethan Soergel < esoe...@dimagi.com> wrote:
Did you see my reply on December 2nd? I'll copy that
again here in case you missed it:It looks like that module may have some binary components
that need to be installed separately. Here is a stack overflow thread with
a number of potential solutions to this issue:
http://stackoverflow.com/quest
ions/22252397/importerror-no-module-named-mysqldbWhen I searched for your error message on google, I found
a number of reports of similar issues. "pip install mysql" might not be
enough, you may need to install some dependencies through your operating
system's package manager (apt-get, for Ubuntu), as suggested in the above
link. SQLAlchemy is a library used for interacting with SQL databases from
python programs.On Wed, Dec 7, 2016 at 1:36 PM, Ajay Singh < ajay....@crgroup.co.in> wrote:
Hi Support team,
This is to put in revised followup, regarding
CommCare-Export tool's data export to MySQL database;
we have tried in many ways, as whatever available
resources in your site as well as in github, but every time, we are facing
issue related to 'sqlalchemy'.We have installed required libraries for MySQL, as
suggested by Simon in last mail (
"pip install pymysql"), but still issue not resolved.Kindly suggest and help us to resolve, are we missing
some steps related to configuration of 'sqlalchemy'?, also please elaborate
as what is this 'sqlalchemy'?Will appreciate your quick help in this context, as it
is long pending request at your end.--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763
Email: ajay....@crgoup.co.in[image: CR Group] http://www.crgroup.com/[image:
Rapid-Fire Business Intelligence & Analytics]
http://www.tableau.com/On Fri, Dec 2, 2016 at 9:29 PM, Ajay Singh < ajay....@crgroup.co.in> wrote:
Thanks Simon,
I have installed MySQL libraries by using
"pip install pymysql" command. It got installed
successfully.On running CommCare export command, I am getting below
error message.I am using following format for MySQL db URL:
mysql://username:password@host:port/database_name[image: Inline image 1]
--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763
Email: ajay....@crgoup.co.in[image: CR Group] http://www.crgroup.com/[image:
Rapid-Fire Business Intelligence & Analytics]
http://www.tableau.com/On Fri, Dec 2, 2016 at 8:46 PM, Simon Kelly < ske...@dimagi.com> wrote:
You can install the package with
"pip install pymysql"On 2 December 2016 at 16:40, Ajay Singh < ajay....@crgroup.co.in> wrote:
Hi Support Team,
We are trying to export data from CommCare Export
tool to MySQL Database. We are using following DB URL format:
mysql://username:password@host:port/database_nameBut the tool is giving following error message.
Please review and suggest the solution.[image: Inline image 1]
--
Thanks,
Ajay Singh
Corporate* Renaissance *Group
Cell (India): +91-9711628763
Email: ajay....@crgoup.co.in[image: CR Group] http://www.crgroup.com/[image:
Rapid-Fire Business Intelligence & Analytics]
http://www.tableau.com/--
Simon Kelly
Senior Engineer | Dimagi South Africa--
You received this message because you are subscribed to
the Google Groups "CommCare Developers" group.
To unsubscribe from this group and stop receiving emails
from it, send an email to commcare-developers+unsubscrib
e@googlegroups.com.
For more options, visit https://groups.google.com/d/op
tout.--
--
Simon Kelly
Senior Engineer | Dimagi South Africa--
Simon Kelly
Senior Engineer | Dimagi South Africa--
Simon Kelly
Senior Engineer | Dimagi South Africa--
Simon Kelly
Senior Engineer | Dimagi South Africa--
Simon Kelly
Senior Engineer | Dimagi South Africa--
You received this message because you are subscribed to the Google
Groups "CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Simon Kelly
Senior Engineer | Dimagi South Africa--
You received this message because you are subscribed to the Google
Groups "CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Simon Kelly
Senior Engineer | Dimagi South Africa--
You received this message because you are subscribed to a topic in the
Google Groups "CommCare Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/to
pic/commcare-developers/1Euu2eUUPB0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
commcare-developers+unsubscribe@googlegroups.com.For more options, visit https://groups.google.com/d/optout.