Case filtering performance

Hi guys,
I had been using fixtures for a particular data set and filtering upon
them. Now I'm using cases. I have noticed considerable difference in the
time it takes to filter the items on one of their attributes. Is this
expected ? Is case filtering slower than fixture filtering ?
Thanks

Yashseth,

Three questions:

  1. Is a filter or is this inside of a form?
  2. Were the fixtures and cases of comparable size?
  3. Are you running on j2me phones, or on Android?

-Clayton

··· On Wed, Jul 3, 2013 at 1:51 AM, yashseth wrote:

Hi guys,
I had been using fixtures for a particular data set and filtering upon
them. Now I'm using cases. I have noticed considerable difference in the
time it takes to filter the items on one of their attributes. Is this
expected ? Is case filtering slower than fixture filtering ?
Thanks

--
You received this message because you are subscribed to the Google Groups
"commcare-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

It is a filter.
The fixtures and cases are of comparable sizes.
The filtering is extremely slow on j2me phone but runs fine with higher end
android phones(tested with dual cores and quad cores)

··· On Wednesday, July 3, 2013 11:21:16 AM UTC+5:30, yashseth wrote: > > Hi guys, > I had been using fixtures for a particular data set and filtering upon > them. Now I'm using cases. I have noticed considerable difference in the > time it takes to filter the items on one of their attributes. Is this > expected ? Is case filtering slower than fixture filtering ? > Thanks >

Herehttps://www.commcarehq.org/a/capilot/apps/view/5404c6c2fbdfff1656490b57133ba368/modules-0/forms-0/?edit=trueis the form.
Some background on the data.
We have a hierarchical data structure as Village>>Mediator>>Group>>Persons.
The Village, Mediator, Group data comes from fixtures whereas the person
data comes from cases. Each person is a case. One of the attributes of a
case is the groupID. The filtering occurs when you select a groupID, and
based on that all cases with that groupID have to be selected and shown one
by one.

··· On Wednesday, July 3, 2013 11:21:16 AM UTC+5:30, yashseth wrote: > > Hi guys, > I had been using fixtures for a particular data set and filtering upon > them. Now I'm using cases. I have noticed considerable difference in the > time it takes to filter the items on one of their attributes. Is this > expected ? Is case filtering slower than fixture filtering ? > Thanks >

Can you point me to the filter logic/form so I can look? We do some runtime
optimizations for doing that kind of filtering, and you might be right on
the edge of which cases it can catch.

-Clayton

··· On Tue, Jul 9, 2013 at 1:35 AM, yashseth wrote:

It is a filter.
The fixtures and cases are of comparable sizes.
The filtering is extremely slow on j2me phone but runs fine with higher
end android phones(tested with dual cores and quad cores)

On Wednesday, July 3, 2013 11:21:16 AM UTC+5:30, yashseth wrote:

Hi guys,
I had been using fixtures for a particular data set and filtering upon
them. Now I'm using cases. I have noticed considerable difference in the
time it takes to filter the items on one of their attributes. Is this
expected ? Is case filtering slower than fixture filtering ?
Thanks

--
You received this message because you are subscribed to the Google Groups
"commcare-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

I'll see whether there's anything we can do to optimize this use case in
the future.

In the mean time I do have a few suggestions. CommCare is able to optimize
some accesses to the casedb depending on whether the request is indexable.
Indexable queries are things that are simple one-to-one evaluations like "a
= b", other more complex queries involving xpath manipulations must be
evaluated from scratch each time.

Specific suggestions:

  1. If you have multiple case types on the phone and are doing a query for
    more than one case, including a predicate: [@case_type = 'mytype'] will
    exclude the other types of cases from evaluation, and is optimized to
    always be very fast.

  2. It looks like your form treats "data/selected_group" as both a singular
    response (case[group_id = /data/selected_group]) and a multiple response (
    selected(/data/selected_group,group_id)). If it's a singular response,
    using the singular form may improve your query time, since CommCare can
    index that evaluation.

  3. If selected_group is a multiple response, you could try to split up your
    predicate into two chunks, and to use the second clause in it first, since
    it looks index-able.

So convert

instance('casedb')/casedb/case[selected(/data/selected_group,group_id) and
id=current()/../attendee_id]

to

instance('casedb')/casedb/case[id=current()/../attendee_id][
selected(/data/selected_group,group_id)]

and see if that helps. It may take a while on the first time you access
the db (compared to further accesses) since the query indexing is a runtime
optimization.

-Clayton

··· On Tue, Jul 9, 2013 at 12:22 PM, yashseth wrote:

Herehttps://www.commcarehq.org/a/capilot/apps/view/5404c6c2fbdfff1656490b57133ba368/modules-0/forms-0/?edit=trueis the form.
Some background on the data.
We have a hierarchical data structure as
Village>>Mediator>>Group>>Persons. The Village, Mediator, Group data comes
from fixtures whereas the person data comes from cases. Each person is a
case. One of the attributes of a case is the groupID. The filtering occurs
when you select a groupID, and based on that all cases with that groupID
have to be selected and shown one by one.

On Wednesday, July 3, 2013 11:21:16 AM UTC+5:30, yashseth wrote:

Hi guys,
I had been using fixtures for a particular data set and filtering upon
them. Now I'm using cases. I have noticed considerable difference in the
time it takes to filter the items on one of their attributes. Is this
expected ? Is case filtering slower than fixture filtering ?
Thanks

--
You received this message because you are subscribed to the Google Groups
"commcare-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.