Hard delete domain on self-hosted instance

Developers
1

What is the best way to hard delete a domain and all data in a self-hosted environment?

Just to provide some context - I have backup files to restore for a project space that comes from the Dimagi hosted instance but the domain had existed on my local instance prior to receiving those files. I would like to get rid of the domain and restore it using the backups I've been given. Running cchq monolith django-manage delete_domain [project space name] does a soft delete which I assume is insufficient if I am to restore from this backup (perhaps I'm wrong there?).

Thanks!

2

Hi Ed

That command does a hard delete on (almost) all the domain's data except for the domain object itself. You could delete the domain object directly via the shell which would allow importing the backup. There is a small risk that there could be some residual data from the old domain but it would mostly be things like audit logs and not applications, forms or cases etc.

If you want to be completely safe you could drop all the databases and then re-configure them.

3

Thanks Simon! I don't believe any data was ever captured in the local instance - we had merely tested transferring the forms and config across. That said, I'm happy to start from scratch. What I'm a bit short on information wise is the process for blitzing stuff - I'm not aware of where everything sits and don't want to make assumptions.
Is there documentation somewhere for:
PostgreSQL:

        Name         |   Owner    | Encoding |   Collate   |    Ctype    |   Access privileges   
---------------------+------------+----------+-------------+-------------+-----------------------
 commcarehq          | commcarehq | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 commcarehq_p1       | commcarehq | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 commcarehq_p2       | commcarehq | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 commcarehq_proxy    | commcarehq | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 commcarehq_synclogs | commcarehq | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 commcarehq_ucr      | commcarehq | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 formplayer          | commcarehq | UTF8     | en_US.UTF-8 | en_US.UTF-8 |

couchdb:

  "_dbs",
  "_nodes",
  "_replicator",
  "_users",
  "_global_changes.1624022307",
  "_replicator.1624022306",
  "_users.1624022306",
  "commcarehq.1624025546",
  "commcarehq__apps.1624025549",
  "commcarehq__auditcare.1624025545",
  "commcarehq__domains.1624025549",
  "commcarehq__fixtures.1624025550",
  "commcarehq__fluff-bihar.1624025558",
  "commcarehq__m4change.1624025558",
  "commcarehq__meta.1624025550",
  "commcarehq__receiverwrapper.1624025558",
  "commcarehq__users.1624025550"

...and what I should be deleting / not deleting. E.g. with postgresql can I just use dropdb, what environment should I operate from for the various data sources (blob / files, couch, postgresql). Are there manage.py commands for any of these interactions I should be using (deleting data / restoring data) or should I be interacting with the core tools directly (pg_restore, dropdb, createdb, etc.)?

4

There's some documentation on this here and here. As you might imagine, this isn't something we do often, so I'd definitely make sure you understand what the commands are doing, as the operations are irreversible.

Depending on the complexity of your setup, you might even find it more straightforward to backup your environment configuration to your machine, then flash the servers themselves and redeploy everything from scratch.

1 Like
5

Thanks Ethan, that might indeed be easier.
I have a follow up question...
There's an option to export case management data - what are the chances that can be used to export from one instance and import into another?
Also - we have had a request to copy data from one project space to another - is that feasable (assuming both are compatible / identical apps)?
Thanks again!

6

There's an option to export case management data - what are the chances that can be used to export from one instance and import into another?

You mean importing the spreadsheet from "export case data"? Yeah, that should work, though you'll have to delete or ignore the case ID column so it can generate new ones. This might get tricky if you have complex case relationships, so I'd definitely try it out first with your data structure.

Also - we have had a request to copy data from one project space to another - is that feasable (assuming both are compatible / identical apps)?

You should reach out to support for that. I believe they can give you a raw data dump suitable for populating a new environment, but I don't think there's any straightforward way to populate another project on the same environment (since the IDs all have to be unique). The case export/import approach would likely be more tractable.

1 Like
7

Hey Simon, I have a follow-up question on this. What would the process be to delete the domain object? We're in a situation where we deleted a domain but would like to re-use the domain name. The domain was created and forms were created under the app but no user data was ever input.
Let me know your thoughts when you have a mo?
Thanks!

9

Hi Ed

You could run this script from a shell which will delete all domains with that name including all their data.
There is no way to recovery the data after doing this!

from corehq.apps.domain.dbaccessors import iter_all_domains_and_deleted_domains_with_name

domain_name = "my_domain"
for domain in iter_all_domains_and_deleted_domains_with_name(domain_name):
    domain.delete()
1 Like