Which API is best for exporting forms data to external web app

Users
1

Hi all -- I have an new API question, different from the one you helped me
with a month or two ago.

I have hired a developer that is going to make a PHP web application that
will regularly export from CommCare our entire body of submitted forms,
into its own local database. Then, from there, it will do statistical
calculations and create a web dashboard of charts, graphs, and
visualizations so we can better understand our project activities.

I want to be sure which API or method is best to tell the developers to
learn and use for this.

-- The new web app will always keep in its database a copy of the whole
collection of forms from our CommCare project space. (We are not working
with cases for this.)

-- Then, once a day, their app will make a new API call to our CommCare
project to determine which new forms have been added since the last time.
It will then export all of those new forms into their correct places in
their own database. (Naturally, in some circumstances, their app may
choose to reset itself and do a new, fresh download of all of the forms
from scratch... or we can tell it to do a "right now" refresh of the
database instead of waiting for the daily.)

the APIs to use for this might be either the Data Export Tool or the List
Forms API. But perhaps I'm missing big parts of the picture.

-- Can you tell me what API and method will be best to implement this daily
exporting of all forms content to a remote web app?

-- Any general pointers on why Data Export Tool, List Forms API, or another
API would be the best for this?

-- Can you provide any other general pointers about the method they should
use... to make it easier for these guys to start planning and reading
through the API documentation?

Huge thanks. I think that asking this clearly at the beginning can help
reduce confusion later on...

Eric

··· From my novice look through the API documentation, my first sense is that
2

Hey Eric,

I'd definitely recommend the data export tool which was designed for this
use case. Behind the scenes it uses the list forms API, but hopefully it
makes things easier for your team to work with.

Cory

··· On Thu, Dec 22, 2016 at 5:36 AM, Eric Stephan wrote:

Hi all -- I have an new API question, different from the one you helped me
with a month or two ago.

I have hired a developer that is going to make a PHP web application that
will regularly export from CommCare our entire body of submitted forms,
into its own local database. Then, from there, it will do statistical
calculations and create a web dashboard of charts, graphs, and
visualizations so we can better understand our project activities.

I want to be sure which API or method is best to tell the developers to
learn and use for this.

-- The new web app will always keep in its database a copy of the whole
collection of forms from our CommCare project space. (We are not working
with cases for this.)

-- Then, once a day, their app will make a new API call to our CommCare
project to determine which new forms have been added since the last time.
It will then export all of those new forms into their correct places in
their own database. (Naturally, in some circumstances, their app may
choose to reset itself and do a new, fresh download of all of the forms
from scratch... or we can tell it to do a "right now" refresh of the
database instead of waiting for the daily.)

From my novice look through the API documentation, my first sense is that
the APIs to use for this might be either the Data Export Tool or the List
Forms API. But perhaps I'm missing big parts of the picture.

-- Can you tell me what API and method will be best to implement this
daily exporting of all forms content to a remote web app?

-- Any general pointers on why Data Export Tool, List Forms API, or
another API would be the best for this?

-- Can you provide any other general pointers about the method they should
use... to make it easier for these guys to start planning and reading
through the API documentation?

Huge thanks. I think that asking this clearly at the beginning can help
reduce confusion later on...

Eric

--
You received this message because you are subscribed to the Google Groups
"commcare-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

3

Great info, thanks! we'll see how many hundred requests for additional info
this results in. :slight_smile:

Thanks
Eric

4

It seems to me that the list forms API is one the one we need.

We don't really need to go through the Excel format. What I would expect is
making a call to a REST web service and receive the data in Json format,
which is described here:
https://confluence.dimagi.com/display/commcarepublic/List+Forms

We would take care of parsing the received Json and update our local
database.

Please advise if this is the wrong approach.

Thank you!

··· On Thursday, December 22, 2016 at 1:45:08 PM UTC+7, Cory Zue wrote: > > Hey Eric, > > I'd definitely recommend the data export tool which was designed for this > use case. Behind the scenes it uses the list forms API, but hopefully it > makes things easier for your team to work with. > > Cory > > On Thu, Dec 22, 2016 at 5:36 AM, Eric Stephan <estep...@gmail.com > wrote: > >> >> >> Hi all -- I have an new API question, different from the one you helped >> me with a month or two ago. >> >> I have hired a developer that is going to make a PHP web application that >> will regularly export from CommCare our entire body of submitted forms, >> into its own local database. Then, from there, it will do statistical >> calculations and create a web dashboard of charts, graphs, and >> visualizations so we can better understand our project activities. >> >> I want to be sure which API or method is best to tell the developers to >> learn and use for this. >> >> -- The new web app will always keep in its database a copy of the whole >> collection of forms from our CommCare project space. (We are not working >> with cases for this.) >> >> -- Then, once a day, their app will make a new API call to our CommCare >> project to determine which new forms have been added since the last time. >> It will then export all of those new forms into their correct places in >> their own database. (Naturally, in some circumstances, their app may >> choose to reset itself and do a new, fresh download of all of the forms >> from scratch... or we can tell it to do a "right now" refresh of the >> database instead of waiting for the daily.) >> >> From my novice look through the API documentation, my first sense is that >> the APIs to use for this might be either the Data Export Tool or the List >> Forms API. But perhaps I'm missing big parts of the picture. >> >> -- Can you tell me what API and method will be best to implement this >> daily exporting of all forms content to a remote web app? >> >> -- Any general pointers on why Data Export Tool, List Forms API, or >> another API would be the best for this? >> >> -- Can you provide any other general pointers about the method they >> should use... to make it easier for these guys to start planning and >> reading through the API documentation? >> >> Huge thanks. I think that asking this clearly at the beginning can help >> reduce confusion later on... >> >> Eric >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "commcare-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to commcare-user...@googlegroups.com . >> For more options, visit https://groups.google.com/d/optout. >> > > >
5

Hi Marco,

You certainly could do that. The data export tool is just intended to make
the process you're describing a bit easier and handle things like
checkpoints and common field mappings for you. It does essentially the same
thing as you have outlined.

thanks,
Cory

··· On Tue, Dec 27, 2016 at 1:57 PM, Marco Pravato wrote:

It seems to me that the list forms API is one the one we need.

We don't really need to go through the Excel format. What I would expect
is making a call to a REST web service and receive the data in Json format,
which is described here:
List Forms - CommCare Public - CommCare Public

We would take care of parsing the received Json and update our local
database.

Please advise if this is the wrong approach.

Thank you!

On Thursday, December 22, 2016 at 1:45:08 PM UTC+7, Cory Zue wrote:

Hey Eric,

I'd definitely recommend the data export tool which was designed for this
use case. Behind the scenes it uses the list forms API, but hopefully it
makes things easier for your team to work with.

Cory

On Thu, Dec 22, 2016 at 5:36 AM, Eric Stephan estep...@gmail.com wrote:

Hi all -- I have an new API question, different from the one you helped
me with a month or two ago.

I have hired a developer that is going to make a PHP web application
that will regularly export from CommCare our entire body of submitted
forms, into its own local database. Then, from there, it will do
statistical calculations and create a web dashboard of charts, graphs, and
visualizations so we can better understand our project activities.

I want to be sure which API or method is best to tell the developers to
learn and use for this.

-- The new web app will always keep in its database a copy of the whole
collection of forms from our CommCare project space. (We are not working
with cases for this.)

-- Then, once a day, their app will make a new API call to our CommCare
project to determine which new forms have been added since the last time.
It will then export all of those new forms into their correct places in
their own database. (Naturally, in some circumstances, their app may
choose to reset itself and do a new, fresh download of all of the forms
from scratch... or we can tell it to do a "right now" refresh of the
database instead of waiting for the daily.)

From my novice look through the API documentation, my first sense is
that the APIs to use for this might be either the Data Export Tool or the
List Forms API. But perhaps I'm missing big parts of the picture.

-- Can you tell me what API and method will be best to implement this
daily exporting of all forms content to a remote web app?

-- Any general pointers on why Data Export Tool, List Forms API, or
another API would be the best for this?

-- Can you provide any other general pointers about the method they
should use... to make it easier for these guys to start planning and
reading through the API documentation?

Huge thanks. I think that asking this clearly at the beginning can help
reduce confusion later on...

Eric

--
You received this message because you are subscribed to the Google
Groups "commcare-users" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to commcare-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"commcare-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

6

Hi Cory,

many thanks, I think we will try by using the APIs at first, and report if
there's any any problem.

Happy Holidays!

··· On Tuesday, December 27, 2016 at 7:46:29 PM UTC+7, Cory Zue wrote: > > Hi Marco, > > You certainly could do that. The data export tool is just intended to make > the process you're describing a bit easier and handle things like > checkpoints and common field mappings for you. It does essentially the same > thing as you have outlined. > > thanks, > Cory > > On Tue, Dec 27, 2016 at 1:57 PM, Marco Pravato <lord...@gmail.com > wrote: > >> It seems to me that the list forms API is one the one we need. >> >> We don't really need to go through the Excel format. What I would expect >> is making a call to a REST web service and receive the data in Json format, >> which is described here: >> https://confluence.dimagi.com/display/commcarepublic/List+Forms >> >> We would take care of parsing the received Json and update our local >> database. >> >> Please advise if this is the wrong approach. >> >> Thank you! >> >> >> >> On Thursday, December 22, 2016 at 1:45:08 PM UTC+7, Cory Zue wrote: >>> >>> Hey Eric, >>> >>> I'd definitely recommend the data export tool which was designed for >>> this use case. Behind the scenes it uses the list forms API, but hopefully >>> it makes things easier for your team to work with. >>> >>> Cory >>> >>> On Thu, Dec 22, 2016 at 5:36 AM, Eric Stephan wrote: >>> >>>> >>>> >>>> Hi all -- I have an new API question, different from the one you helped >>>> me with a month or two ago. >>>> >>>> I have hired a developer that is going to make a PHP web application >>>> that will regularly export from CommCare our entire body of submitted >>>> forms, into its own local database. Then, from there, it will do >>>> statistical calculations and create a web dashboard of charts, graphs, and >>>> visualizations so we can better understand our project activities. >>>> >>>> I want to be sure which API or method is best to tell the developers to >>>> learn and use for this. >>>> >>>> -- The new web app will always keep in its database a copy of the whole >>>> collection of forms from our CommCare project space. (We are not working >>>> with cases for this.) >>>> >>>> -- Then, once a day, their app will make a new API call to our CommCare >>>> project to determine which new forms have been added since the last time. >>>> It will then export all of those new forms into their correct places in >>>> their own database. (Naturally, in some circumstances, their app may >>>> choose to reset itself and do a new, fresh download of all of the forms >>>> from scratch... or we can tell it to do a "right now" refresh of the >>>> database instead of waiting for the daily.) >>>> >>>> From my novice look through the API documentation, my first sense is >>>> that the APIs to use for this might be either the Data Export Tool or the >>>> List Forms API. But perhaps I'm missing big parts of the picture. >>>> >>>> -- Can you tell me what API and method will be best to implement this >>>> daily exporting of all forms content to a remote web app? >>>> >>>> -- Any general pointers on why Data Export Tool, List Forms API, or >>>> another API would be the best for this? >>>> >>>> -- Can you provide any other general pointers about the method they >>>> should use... to make it easier for these guys to start planning and >>>> reading through the API documentation? >>>> >>>> Huge thanks. I think that asking this clearly at the beginning can help >>>> reduce confusion later on... >>>> >>>> Eric >>>> >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "commcare-users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to commcare-user...@googlegroups.com. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "commcare-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to commcare-user...@googlegroups.com . >> For more options, visit https://groups.google.com/d/optout. >> > >