Here's a question. I was describing our forthcoming HIV outreach commcare
app to the country manager that oversees the project that our app will run
within. He had a nice basic question about backups of the data. The
scenarios for wanting such a backup: (A) imagine that someday one of our
own staff members exhibits total lack of competence and goes in and carries
out the steps to delete one or multple users.. thus deleting the data. How
could it be recovered? (B) imagine that CommCareHQ does have some kind of
hack, or improbable data loss on the official servers.
First off, I have confidence in the cloud-based system that Dimagi runs
CommCare on. But I wanted to have a clear answer to him. And I applaud
people that show healthy paranoia!
(1) what is the best method by which we would do a full export of all of
our project's data, so we could store that data backup on our own local
drives, so that if a failure, hack, or deletion did happen on the
cloud-based server, we could upload that backed-up data from our local
drives into the cloud, and all the open and closed cases, history, data and
mobile function would be present in the application as desired, and we
could proceed?
(2) Any other comments on what the best practice is with regards to
ensuring our data security in case of improbable, but possible, events?
Thanks for the vote of confidence. We do have quite a lot of infrastructure
around backups in our production environment including multiple levels of
redundancy in data storage as well as offsite snapshot backups for all data
so that our customers don't have to worry about this. But agree paranoia
certainly doesn't hurt.
Rolling your own backup solution would be quite complicated. You can pretty
easily set something up to pull all the data down using the exports or the data
export tool https://help.commcarehq.org/display/commcarepublic/CommCare+Data+Export+Tool.
And you could use the case importer to get the cases back to the state they
were in before the catastrophic even occurred. However, backing up the
entire history in a seamless way that allows you to reimport that would be
quite hard and involve a lot of manual coding.
Cory
···
On Tue, Feb 17, 2015 at 11:32 PM, Eric Stephan wrote:
Hello folks --
Here's a question. I was describing our forthcoming HIV outreach commcare
app to the country manager that oversees the project that our app will run
within. He had a nice basic question about backups of the data. The
scenarios for wanting such a backup: (A) imagine that someday one of our
own staff members exhibits total lack of competence and goes in and carries
out the steps to delete one or multple users.. thus deleting the data. How
could it be recovered? (B) imagine that CommCareHQ does have some kind of
hack, or improbable data loss on the official servers.
First off, I have confidence in the cloud-based system that Dimagi runs
CommCare on. But I wanted to have a clear answer to him. And I applaud
people that show healthy paranoia!
(1) what is the best method by which we would do a full export of all of
our project's data, so we could store that data backup on our own local
drives, so that if a failure, hack, or deletion did happen on the
cloud-based server, we could upload that backed-up data from our local
drives into the cloud, and all the open and closed cases, history, data and
mobile function would be present in the application as desired, and we
could proceed?
(2) Any other comments on what the best practice is with regards to
ensuring our data security in case of improbable, but possible, events?
Hi Cory -- Great, thanks for the info. It makes sense.
To entertain one of the paranoid scenarios... let's assume that we do
someday assign administrative rights to some staffer, and that staffer
makes an incompetent (or vindictive) step and deletes one or more mobile
users, thus erasing the data associated with them.
In that case, is there a process by which we could go to Dimagi and ask
that part of the app's data... or all of the app's data... be recovered
from a backup?
There's no precedent for this scenario, but it would definitely be
physically possible to recover that data.
That said, our backup systems are largely meant to only be used in the
event of a significant hardware or software failure and there isn't a
"quick and easy" way for us to recover data that is intentionally deleted,
so it's not something I would necessarily build into your workflow.
We do make it intentionally hard to delete user data for this exact purpose
(the incompetent scenario at least, not the vindictive), and we recommend
always archiving data instead of deleting it so that it can be recovered if
needed.
Cory
···
On Fri, Feb 20, 2015 at 3:30 AM, Eric Stephan wrote:
Hi Cory -- Great, thanks for the info. It makes sense.
To entertain one of the paranoid scenarios... let's assume that we do
someday assign administrative rights to some staffer, and that staffer
makes an incompetent (or vindictive) step and deletes one or more mobile
users, thus erasing the data associated with them.
In that case, is there a process by which we could go to Dimagi and ask
that part of the app's data... or all of the app's data... be recovered
from a backup?
