I work on a project that involves massive amounts of data from various sources, but our main source is CommCare. I want to transfer data from another platform to CommCare. Can you help me with this? I'm very concerned about the data's archieve and security.
Hi @Bayan
There are various ways to transfer data from another platform to CommCare. The method you choose will depend on the nature of the data, how often you want to transfer the data, and the capabilities of the other platform.
The nature of the data
Is the data about identifiable things, like people or items or places? Then you would probably want to store that data in CommCare as cases.
Or is the data about events, or individual data points that one could think of as survey data? Then you would probably want to store them as form question values.
Frequency
If you want to transfer the data just once, or infrequently like, maybe annually or perhaps even monthly, and if the data can fit in an Excel spreadsheet, then I would recommend using a Case Import (Data > Import Cases from Excel). This is your simplest (cheapest in terms of effort?) option.
If you want to transfer the data more frequently, things get complicated quickly.
CommCare HQ offers a range of APIs. You can find API documentation for programmers here.
CommCare also offers integration via Zapier. Zapier is great at moving data between CommCare and a range of different sources and destinations. You can find instructions here.
Capabilities of the other platform
Your final set of options will be determined by how you can get data out of the other platform; whether it allows you to export as Excel or CSV; whether it allows you to forward data to a remote API, or automatically exported somewhere that Zapier has access to, like Google Sheets; whether it offers APIs that can be polled; etc.
I hope I have given you some options to think about, and places to find more information.
Thanks,
Norman
Hi @Norman_Hooper,
We want to download 5-year project data (2018-2023). We had been using a Commcare App to follow family planning clients. In this case, what is advisable? Download data as case export or as a form export? I have read that if I export case data, I will get only last updated information about a case and not all the information about the client/case.
Thanks
That is correct, a case export will allow you to export the current state of your cases.
A form export will allow you to download selected data from every form that updated those cases, so you can get a complete history of changes.
If you have too much data for an Excel spreadsheet and you intend to store it in a database, and if you have a Pro Plan subscription, you could choose to use the CommCare Data Export Tool. It is a command line tool that can export form or case data from CommCare HQ directly into a database like Azure SQL / SQL Server / PostgreSQL / etc. If this fits your use case, you can find more information here: https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2143955952/CommCare+Data+Export+Tool+DET
Thanks,
Norman
Hi @Norman_Hooper,
Thank you for your explanation below. We are not using a Pro Plan Subscription.
I am sorry but there is there is a topic that confuses me a bit: loading and saving cases when it comes to case management. I keep reading Commcare material available on help site. I have found terms like: saving info without loading and loading info without saving. I were wondering if you could explain to me the difference with just a practical example
Thanks
Could you give a link to the CommCare material you are referring to?
For now, I will assume you mean Advanced Case Management - Saving Data to Case Properties and Advanced Case Management - Referencing Data from the Case.
Let's start with loading data. Let us imagine you have a form to change a patient's address. You may want to load their current known address, and allow the user to modify it. To do that, you can drag the case property for the address into the "Default Value" field. That loads case data into a form question.
Next, saving data. On the form settings page, in the "Case Management" tab, you can save form question values to case properties.
If you save data without first loading it, then you are "saving info without loading". And likewise, if you set a default value for a form question, but you don't save that form question value back to the case property, then you are "loading info without saving".
I hope I have understood your question properly.
Thanks,
Norman
Hi @Norman_Hooper,
Thanks so much.