Understanding the Data Export tool connecting automatically to a web-hosted SQL database

Users
1

Hi all --

I have been trying to learn more about the CommCare data export tool, and
how it could connect to automatically do a regular upload to a SQL database
that's hosted on a web server. I understand that if I did that, then
Tableau or some other app could access it.

So, I was looking at the wiki site on the data export tool --
https://confluence.dimagi.com/display/commcarepublic/CommCare+Data+Export+Tool#
… and I was a little unsure at the start.

A few quite basic, newbie questions:

1-- My mental image is that the target SQL database would be sitting on a
webserver somewhere, always accessible. But when I read the above
instructions, it looks like it is really focusing on doing the steps of
installing python and so forth on a local standalone PC.

Of course I do see that one of the three options in those instructions is a
linux machine, but I’m still feeling like they’re talking about a desktop
machine. Can you explain if these are the same instructions I would do
from the command line of a hosted server and all that, if I am hoping to
make the link to a SQL database that’s sitting on my webhosted space?

[Note, just to give you too much information, that I have in fact made a
couple of blank databases on my inexpensive webhost. The hosting provider
is Bluehost, let's say the web domain name is www.example.com, I made one
MySQL database called example_commcaret1 and one PostgreSQL database called
example_commcaret2 … and for both of them I have a userID/password created
with strong permissions.]

... So can anyone clear up this “desktop or not” question? Would I simply
follow the instructions in the above wiki page, but do it from the command
line of my hosted webspace?

2-- Am I right that when I set up this automatic export with the Data
Export tool, I can make it export for only a specific subset of my commcare
data? It doesn’t have to be the entire database? For example, I could make
it only be records that were owned by a certain mobile users group, etc?

3-- Any other caveats, or any tips? For example, is my Bluehost webspace
likely going to be able to handle this kind of automatic SQL export
(provided that I do the configurations to access the command line, set up
SSH/shell access, etc?)

Thanks much!

Eric

2

Hey Eric,

Quick thoughts inline:

I have been trying to learn more about the CommCare data export tool, and

how it could connect to automatically do a regular upload to a SQL database
that's hosted on a web server. I understand that if I did that, then
Tableau or some other app could access it.

So, I was looking at the wiki site on the data export tool --
CommCare Data Export Tool - CommCare Public - CommCare Public
… and I was a little unsure at the start.

A few quite basic, newbie questions:

1-- My mental image is that the target SQL database would be sitting on a
webserver somewhere, always accessible. But when I read the above
instructions, it looks like it is really focusing on doing the steps of
installing python and so forth on a local standalone PC.

Of course I do see that one of the three options in those instructions is
a linux machine, but I’m still feeling like they’re talking about a desktop
machine. Can you explain if these are the same instructions I would do
from the command line of a hosted server and all that, if I am hoping to
make the link to a SQL database that’s sitting on my webhosted space?

[Note, just to give you too much information, that I have in fact made a
couple of blank databases on my inexpensive webhost. The hosting provider
is Bluehost, let's say the web domain name is www.example.com, I made one
MySQL database called example_commcaret1 and one PostgreSQL database called
example_commcaret2 … and for both of them I have a userID/password created
with strong permissions.]

... So can anyone clear up this “desktop or not” question? Would I simply
follow the instructions in the above wiki page, but do it from the command
line of my hosted webspace?

Yeah, the documentation was written assuming a desktop computer, but a
similar process should be used for a server. Moreover, the script can run
on a totally separate machine as the database if you wish (though it will
be much more performant if it's running on the same machine as the
database).

2-- Am I right that when I set up this automatic export with the Data
Export tool, I can make it export for only a specific subset of my commcare
data? It doesn’t have to be the entire database? For example, I could make
it only be records that were owned by a certain mobile users group, etc?

Yes, you should be able to define additional filters (though unfortunately
docs are not great).

3-- Any other caveats, or any tips? For example, is my Bluehost webspace
likely going to be able to handle this kind of automatic SQL export
(provided that I do the configurations to access the command line, set up
SSH/shell access, etc?)

None that I can think of. Don't think it's particularly computationally
expensive, so as long as you have a reasonable host and plenty of space for
the data you should be all set!

cheers,
Cory

··· > > > > Thanks much! > > Eric > > > -- > You received this message because you are subscribed to the Google Groups > "commcare-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to commcare-users+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. >