Archiving forms to be removed from data export

Hello, my current workflow is to export data on a daily basis to generate reports in R. Thus, when I archive a form, the form continues to be available in the daily export data. Is it possible to archive a form so that it is removed from the exported data? I could not find any previous threads that discuss this specific scenario.

Thank you,
Will

Hi Will,

Unfortunately Daily Saved Exports are the one place in the system where archived forms are still exported in the system. The other export channels will all have archived forms removed, would one of them work for your purpose?

If you are looking for minimal steps, the data export tool is capable of being set up to provide this as a one step operation, but you’ll need to be comfortable running python on the command line.

-Clayton

Clayton,

Thanks for the prompt response! I’m working on setting up the data export tool from my Mac system. The issue I’ve run into is the “commcare-export” command not being found on command line despite installing Homebrew and Python. Have you heard of this issue before?

Thanks,
Will

Are you running the pip2 install step once python is available through homebrew? I’m not as familiar with the install and setup process on mac.

-Clayton

Yes, I’ve run the pip2 install commcare-export command and it seems to have installed everything correctly. But when I run “commcare-export” command, I get this following error:
-bash: commcare-export: command not found

Do I need to be in a specific directory to access the command? Any help on finding the root of the issue would be appreciated.

Will

That’s really odd, you shouldn’t need to be in a specific directory, as once the library is added to pip it should be available.

Do you have a python virtual environment set up? That might explain quite a bit about how things are configured. It’s possible, for instance that your environment is set up with python3 rather than python2, so it’s possible your pip install put commcare-export into a command set that isn’t currently on your path.

I’m not sure which version of python my virtual environment is referencing so maybe that could be the issue. Do you have some resources to set up my virtual environment properly?

Python virtual environments are a really handy tool that let you encapsulate all of the resources and requirements that you install for different packages (say, commcare-export) into their own clean python setup, to avoid any confusion about what’s installed and how it overlaps with other configurations. The most important thing to confirm before getting started with them is whether you are using the python2 or python3 runtime. commcare-export should work with python2.7 or python 3, but depending on your OS setup you may need to be quite explicit about which one you are using.

-Clayton

Providing an update that it was an issue of downloading the correct python version-once I downloaded 2.7 and confirmed the path in my bash profile, the command worked. One more quick question: I want to export ALL variables from all my forms when using the data export tool. Is it possible to specify all variables/columns without having to list them out in the excel query file?

Hi Will,

Unfortunately there isn’t, because in many configurations the DET needs manual input from the query file for things like what to name the potentially ambiguous columns and similar. We are working on tooling that increases the ease of generating DET templates in the future, but they aren’t available just yet.

-Clayton