Remove rows or subtract values in fluff table when form is archived

Hi,

I have a report which use fluff with calculator like this:
https://github.com/dimagi/commcare-hq/blob/master/custom/m4change/user_calcs/anc_hmis_report_calcs.py#L24

Problem is that when I make form which initially pass business rules and
is already indexed as "archived", previous computation is not removed /
subtracted in fluff table in Postgres, even when I run
ptop_fast_reindex_fluff. case.get_forms() method doesn't return this
form though.

Do you happen to know if there is an easy way to achieve this with
calculator built in the way I showed in the link, or we need to rebuild
whole report and put XFormInstance as document_class in Fluff model?

Best regards
Pawel

Hi Pewel

That's a good question and you are correct that at present that is the case
however this seems like something we should handle so I've modified the
functionality to remove all existing data before adding new date for a
document. delete rows for docs before updateing by snopoke · Pull Request #77 · dimagi/fluff · GitHub

We'll need to re-index any existing data.

··· On 7 April 2014 12:46, Paweł Reise wrote:

Hi,

I have a report which use fluff with calculator like this:
https://github.com/dimagi/commcare-hq/blob/master/
custom/m4change/user_calcs/anc_hmis_report_calcs.py#L24

Problem is that when I make form which initially pass business rules and
is already indexed as "archived", previous computation is not removed /
subtracted in fluff table in Postgres, even when I run
ptop_fast_reindex_fluff. case.get_forms() method doesn't return this form
though.

Do you happen to know if there is an easy way to achieve this with
calculator built in the way I showed in the link, or we need to rebuild
whole report and put XFormInstance as document_class in Fluff model?

Best regards
Pawel

--

--- You received this message because you are subscribed to the Google
Groups "CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Simon Kelly
Senior Engineer | Dimagi South Africa

Hi Simon,

thanks for this. Two concerns:

  • Do we have to to make re-index every time we archive a form? If so,
    would it be possible to provide some solution to make it automatic? If
    it can't be done in fluff code, can we add a task to our custom domain,
    where we would re-index data if any form was archived?
  • There is one additional scenario to handle: when the case has only one
    form, and user marks this form as "archived", the case changes its type
    to "CommCareCase-Deleted" and is not included in re-indexing, so
    existing data for this case is not removed.

Best regards
Pawel

··· On 04/08/2014 11:24 AM, Simon Kelly wrote: > Hi Pewel > > That's a good question and you are correct that at present that is the > case however this seems like something we should handle so I've > modified the functionality to remove all existing data before adding > new date for a document. https://github.com/dimagi/fluff/pull/77 > > We'll need to re-index any existing data. > > > On 7 April 2014 12:46, Paweł Reise <preise@soldevelo.com > wrote: > > Hi, > > I have a report which use fluff with calculator like this: > https://github.com/dimagi/commcare-hq/blob/master/custom/m4change/user_calcs/anc_hmis_report_calcs.py#L24 > > Problem is that when I make form which initially pass business > rules and is already indexed as "archived", previous computation > is not removed / subtracted in fluff table in Postgres, even when > I run ptop_fast_reindex_fluff. case.get_forms() method doesn't > return this form though. > > Do you happen to know if there is an easy way to achieve this with > calculator built in the way I showed in the link, or we need to > rebuild whole report and put XFormInstance as document_class in > Fluff model? > > > Best regards > Pawel > > -- > > --- You received this message because you are subscribed to the > Google Groups "CommCare Developers" group. > To unsubscribe from this group and stop receiving emails from it, > send an email to commcare-developers+unsubscribe@googlegroups.com > . > For more options, visit https://groups.google.com/d/optout. > > > > > -- > Simon Kelly > Senior Engineer | Dimagi South Africa > -- > > --- > You received this message because you are subscribed to the Google > Groups "CommCare Developers" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to commcare-developers+unsubscribe@googlegroups.com > . > For more options, visit https://groups.google.com/d/optout.

I think Simon meant to sent this to the broader list:

Pawel, the re-index is only necessary this once. If we didn't re-index then
only data for cases that are modified would get re-calculated and hence we
would miss any old data that has already been processed and for which there
are no future changes.

Your point about the case deletion is very valid, I've created a ticket for
it here: http://manage.dimagi.com/default.asp?107866

··· On Thu, Apr 10, 2014 at 12:56 PM, Paweł Reise wrote:

Hi Simon,

thanks for this. Two concerns:

  • Do we have to to make re-index every time we archive a form? If so,
    would it be possible to provide some solution to make it automatic? If it
    can't be done in fluff code, can we add a task to our custom domain, where
    we would re-index data if any form was archived?
  • There is one additional scenario to handle: when the case has only one
    form, and user marks this form as "archived", the case changes its type to
    "CommCareCase-Deleted" and is not included in re-indexing, so existing data
    for this case is not removed.

Best regards
Pawel

On 04/08/2014 11:24 AM, Simon Kelly wrote:

Hi Pewel

That's a good question and you are correct that at present that is the
case however this seems like something we should handle so I've modified
the functionality to remove all existing data before adding new date for a
document. delete rows for docs before updateing by snopoke · Pull Request #77 · dimagi/fluff · GitHub

We'll need to re-index any existing data.

On 7 April 2014 12:46, Paweł Reise preise@soldevelo.com wrote:

Hi,

I have a report which use fluff with calculator like this:

https://github.com/dimagi/commcare-hq/blob/master/custom/m4change/user_calcs/anc_hmis_report_calcs.py#L24

Problem is that when I make form which initially pass business rules and
is already indexed as "archived", previous computation is not removed /
subtracted in fluff table in Postgres, even when I run
ptop_fast_reindex_fluff. case.get_forms() method doesn't return this form
though.

Do you happen to know if there is an easy way to achieve this with
calculator built in the way I showed in the link, or we need to rebuild
whole report and put XFormInstance as document_class in Fluff model?

Best regards
Pawel

--

--- You received this message because you are subscribed to the Google
Groups "CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Simon Kelly
Senior Engineer | Dimagi South Africa


You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--


You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.