DimagiPilot - Sorting Bihar reports by 'chw_name' field

Hi,
Currently I’m working on the bihar reports, most of functionality is
already finished - but we still have some problems with sorting records
using ‘chw_name’ field. This field is concatenation of two user data fields:

chw_name = “username” + “full_name”.

ElasticSearch “full_cases” doesn’t provide such field in his mappings - In
order to get such field from mapping, I assume I should modify that mapping
and add new field which will contain data mentioned above.
I read that I could use there, a ‘nested_type’ query to get related user
data based on ‘user_id’ field of case, but I wonder If there is any other
ways to solve this sorting issue. I would appreciate any
insights/suggestions.

Regards,
Marcin

Hi Marcin,

We are planning on phasing out full_cases, i’d look at the report_cases
field.

What’s the source of the information you want to display the chw_name
information? Is it the case’s xform history?

With regard to getting that mapping into the case document - this is a
todo that cory and I discussed to get as much of that into the case
document in couch so as to send it as unmodified as possible to
elasticsearch. The idea would be to have the metadata ofthe transaction
(xform) information be understandable without lookup incouch by some
computation/dereferencing happening upon submission.

Changes to the report_xforms index will have the case blocks indexed which
is another way to get access to sortable metadata about case transactions.
These are still being debugged and tested though - i should have a version
ready in the next few days. If you would ike to take a look at that let me
know and we can talk over the changes.

Dan

··· On Fri, Oct 18, 2013 at 11:49 AM, sienioslav wrote:

Hi,
Currently I’m working on the bihar reports, most of functionality is
already finished - but we still have some problems with sorting records
using ‘chw_name’ field. This field is concatenation of two user data fields:

chw_name = “username” + “full_name”.

ElasticSearch “full_cases” doesn’t provide such field in his mappings -
In order to get such field from mapping, I assume I should modify that
mapping and add new field which will contain data mentioned above.
I read that I could use there, a ‘nested_type’ query to get related user
data based on ‘user_id’ field of case, but I wonder If there is any other
ways to solve this sorting issue. I would appreciate any
insights/suggestions.

Regards,
Marcin


You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi Dan

Unfortunately the source information of “chw_name” field isn’t in case
xform history - it is a new field which is a concatenation of 'username’
and ‘fullname’ fields from related user object.

Regarding ‘report_xforms’, I think I could take a look into that
changes(once It will be ready for testing). Is there any branch available
with these changes, just let me know where can I found it.

Regards,
Marcin

W dniu niedziela, 20 października 2013 19:34:57 UTC+2 użytkownik Daniel
Myung napisał:

··· > > Hi Marcin, > > We are planning on phasing out full_cases, i'd look at the report_cases > field. > > What's the source of the information you want to display the chw_name > information? Is it the case's xform history? > > With regard to getting that mapping into the *case* document - this is a > todo that cory and I discussed to get as much of that into the case > document in couch so as to send it as unmodified as possible to > elasticsearch. The idea would be to have the metadata ofthe transaction > (xform) information be understandable without lookup incouch by some > computation/dereferencing happening upon submission. > > Changes to the report_xforms index will have the case blocks indexed which > is another way to get access to sortable metadata about case transactions. > These are still being debugged and tested though - i should have a version > ready in the next few days. If you would ike to take a look at that let me > know and we can talk over the changes. > > Dan > > > > On Fri, Oct 18, 2013 at 11:49 AM, sienioslav <sieni...@gmail.com wrote: > >> Hi, >> Currently I'm working on the bihar reports, most of functionality is >> already finished - but we still have some problems with sorting records >> using 'chw_name' field. This field is concatenation of two user data fields: >> >> chw_name = "username" + "full_name". >> >> ElasticSearch "full_cases" doesn't provide such field in his mappings - >> In order to get such field from mapping, I assume I should modify that >> mapping and add new field which will contain data mentioned above. >> I read that I could use there, a 'nested_type' query to get related user >> data based on 'user_id' field of case, but I wonder If there is any other >> ways to solve this sorting issue. I would appreciate any >> insights/suggestions. >> >> Regards, >> Marcin >> >> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "CommCare Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to commcare-developers+unsubscribe@googlegroups.com . >> For more options, visit https://groups.google.com/groups/opt_out. >> > >

Hi Marcin,

This has been merged into the master branch now.

I would recommend you don’t block the feature on being able to sort by
username / chw name yet, and make that a nice to have. I think any solution
is going to be pretty involved. That said, Dan is considering extending the
indices to include additional server-metadata as he mentioned, but I don’t
think you should block on that change.

Cory

··· On Mon, Oct 21, 2013 at 3:13 PM, sienioslav wrote:

Hi Dan

Unfortunately the source information of “chw_name” field isn’t in case
xform history - it is a new field which is a concatenation of 'username’
and ‘fullname’ fields from related user object.

Regarding ‘report_xforms’, I think I could take a look into that
changes(once It will be ready for testing). Is there any branch available
with these changes, just let me know where can I found it.

Regards,
Marcin

W dniu niedziela, 20 października 2013 19:34:57 UTC+2 użytkownik Daniel
Myung napisał:

Hi Marcin,

We are planning on phasing out full_cases, i’d look at the report_cases
field.

What’s the source of the information you want to display the chw_name
information? Is it the case’s xform history?

With regard to getting that mapping into the case document - this is a
todo that cory and I discussed to get as much of that into the case
document in couch so as to send it as unmodified as possible to
elasticsearch. The idea would be to have the metadata ofthe transaction
(xform) information be understandable without lookup incouch by some
computation/dereferencing happening upon submission.

Changes to the report_xforms index will have the case blocks indexed
which is another way to get access to sortable metadata about case
transactions. These are still being debugged and tested though - i should
have a version ready in the next few days. If you would ike to take a look
at that let me know and we can talk over the changes.

Dan

On Fri, Oct 18, 2013 at 11:49 AM, sienioslav sieni...@gmail.com wrote:

Hi,
Currently I’m working on the bihar reports, most of functionality is
already finished - but we still have some problems with sorting records
using ‘chw_name’ field. This field is concatenation of two user data fields:

chw_name = “username” + “full_name”.

ElasticSearch “full_cases” doesn’t provide such field in his mappings -
In order to get such field from mapping, I assume I should modify that
mapping and add new field which will contain data mentioned above.
I read that I could use there, a ‘nested_type’ query to get related user
data based on ‘user_id’ field of case, but I wonder If there is any other
ways to solve this sorting issue. I would appreciate any
insights/suggestions.

Regards,
Marcin


You received this message because you are subscribed to the Google
Groups “CommCare Developers” group.
To unsubscribe from this group and stop receiving emails from it, send
an email to commcare-developers+**unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.


You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

If you want to sort by a concatenated username + full_name - that really is
just sorting by username. There’s no need to index and sort over a new
field that concatenates the two.

Support sort by username, and present it as a concatenation of the
user+username. In your rendering of the row, I’d make a cache of users so
you don’t do repeated user queries to get their full names.

So i’d query report_xforms and do a nested query. It should work in the
case where form.case is the case block in the form:

Dan

··· On Tue, Oct 22, 2013 at 2:11 AM, Cory Zue wrote:

Hi Marcin,

This has been merged into the master branch now.

I would recommend you don’t block the feature on being able to sort by
username / chw name yet, and make that a nice to have. I think any solution
is going to be pretty involved. That said, Dan is considering extending the
indices to include additional server-metadata as he mentioned, but I don’t
think you should block on that change.

Cory

On Mon, Oct 21, 2013 at 3:13 PM, sienioslav sienioslaw@gmail.com wrote:

Hi Dan

Unfortunately the source information of “chw_name” field isn’t in case
xform history - it is a new field which is a concatenation of 'username’
and ‘fullname’ fields from related user object.

Regarding ‘report_xforms’, I think I could take a look into that
changes(once It will be ready for testing). Is there any branch available
with these changes, just let me know where can I found it.

Regards,
Marcin

W dniu niedziela, 20 października 2013 19:34:57 UTC+2 użytkownik Daniel
Myung napisał:

Hi Marcin,

We are planning on phasing out full_cases, i’d look at the report_cases
field.

What’s the source of the information you want to display the chw_name
information? Is it the case’s xform history?

With regard to getting that mapping into the case document - this is a
todo that cory and I discussed to get as much of that into the case
document in couch so as to send it as unmodified as possible to
elasticsearch. The idea would be to have the metadata ofthe transaction
(xform) information be understandable without lookup incouch by some
computation/dereferencing happening upon submission.

Changes to the report_xforms index will have the case blocks indexed
which is another way to get access to sortable metadata about case
transactions. These are still being debugged and tested though - i should
have a version ready in the next few days. If you would ike to take a look
at that let me know and we can talk over the changes.

Dan

On Fri, Oct 18, 2013 at 11:49 AM, sienioslav sieni...@gmail.com wrote:

Hi,
Currently I’m working on the bihar reports, most of functionality is
already finished - but we still have some problems with sorting records
using ‘chw_name’ field. This field is concatenation of two user data fields:

chw_name = “username” + “full_name”.

ElasticSearch “full_cases” doesn’t provide such field in his mappings

  • In order to get such field from mapping, I assume I should modify that
    mapping and add new field which will contain data mentioned above.
    I read that I could use there, a ‘nested_type’ query to get related
    user data based on ‘user_id’ field of case, but I wonder If there is any
    other ways to solve this sorting issue. I would appreciate any
    insights/suggestions.

Regards,
Marcin


You received this message because you are subscribed to the Google
Groups “CommCare Developers” group.
To unsubscribe from this group and stop receiving emails from it, send
an email to commcare-developers+**unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.


You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi,

Thank for your insights. I could be wrong, but I think it isn’t a good idea
in our case, to do query on “report_xforms” along with sorting by
’form.meta.username’ field, and retrieving case from “form.case” block,
since after this operation could be very time consuming along with our
retrieving methods. Moreover, if we would switch to "report_xforms"
mapping, we couldn’t use filters(like filtering by groups, or open/close
cases), because this mapping doesn’t contain those fields.

I think we should stick to “report_cases” mapping, and try find some way to
inject username into this mapping while indexing records, we tried to
modify mapping by using nested objects(just like it is done in
"reportxform_mapping.py"), but It failed because there is no ‘username’ in
couchdb case document.

Do you think, if there is any other correct way to solve with this problem?

Marcin

W dniu wtorek, 22 października 2013 22:19:36 UTC+2 użytkownik Daniel Myung
napisał:

··· > > If you want to sort by a concatenated username + full_name - that really > is just sorting by username. There's no need to index and sort over a new > field that concatenates the two. > > Support sort by username, and present it as a concatenation of the > user+username. In your rendering of the row, I'd make a cache of users so > you don't do repeated user queries to get their full names. > > So i'd query report_xforms and do a nested query. It should work in the > case where form.case is the case block in the form: > > https://github.com/dimagi/commcare-hq/blob/master/corehq/apps/api/es.py#L399 > > Dan > > > On Tue, Oct 22, 2013 at 2:11 AM, Cory Zue <cz...@dimagi.com >wrote: > >> Hi Marcin, >> >> This has been merged into the master branch now. >> >> I would recommend you don't block the feature on being able to sort by >> username / chw name yet, and make that a nice to have. I think any solution >> is going to be pretty involved. That said, Dan is considering extending the >> indices to include additional server-metadata as he mentioned, but I don't >> think you should block on that change. >> >> Cory >> >> >> >> On Mon, Oct 21, 2013 at 3:13 PM, sienioslav <sieni...@gmail.com wrote: >> >>> Hi Dan >>> >>> Unfortunately the source information of "chw_name" field isn't in case >>> xform history - it is a new field which is a concatenation of 'username' >>> and 'fullname' fields from related user object. >>> >>> Regarding 'report_xforms', I think I could take a look into that >>> changes(once It will be ready for testing). Is there any branch available >>> with these changes, just let me know where can I found it. >>> >>> Regards, >>> Marcin >>> >>> W dniu niedziela, 20 października 2013 19:34:57 UTC+2 użytkownik Daniel >>> Myung napisał: >>> >>>> Hi Marcin, >>>> >>>> We are planning on phasing out full_cases, i'd look at the report_cases >>>> field. >>>> >>>> What's the source of the information you want to display the chw_name >>>> information? Is it the case's xform history? >>>> >>>> With regard to getting that mapping into the *case* document - this is >>>> a todo that cory and I discussed to get as much of that into the case >>>> document in couch so as to send it as unmodified as possible to >>>> elasticsearch. The idea would be to have the metadata ofthe transaction >>>> (xform) information be understandable without lookup incouch by some >>>> computation/dereferencing happening upon submission. >>>> >>>> Changes to the report_xforms index will have the case blocks indexed >>>> which is another way to get access to sortable metadata about case >>>> transactions. These are still being debugged and tested though - i should >>>> have a version ready in the next few days. If you would ike to take a look >>>> at that let me know and we can talk over the changes. >>>> >>>> Dan >>>> >>>> >>>> >>>> On Fri, Oct 18, 2013 at 11:49 AM, sienioslav wrote: >>>> >>>>> Hi, >>>>> Currently I'm working on the bihar reports, most of functionality is >>>>> already finished - but we still have some problems with sorting records >>>>> using 'chw_name' field. This field is concatenation of two user data fields: >>>>> >>>>> chw_name = "username" + "full_name". >>>>> >>>>> ElasticSearch "full_cases" doesn't provide such field in his mappings >>>>> - In order to get such field from mapping, I assume I should modify that >>>>> mapping and add new field which will contain data mentioned above. >>>>> I read that I could use there, a 'nested_type' query to get related >>>>> user data based on 'user_id' field of case, but I wonder If there is any >>>>> other ways to solve this sorting issue. I would appreciate any >>>>> insights/suggestions. >>>>> >>>>> Regards, >>>>> Marcin >>>>> >>>>> -- >>>>> >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "CommCare Developers" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to commcare-developers+**unsubscribe@googlegroups.com. >>>>> For more options, visit https://groups.google.com/**groups/opt_out >>>>> . >>>>> >>>> >>>> -- >>> >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "CommCare Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to commcare-developers+unsubscribe@googlegroups.com >>> . >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >> >> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "CommCare Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to commcare-developers+unsubscribe@googlegroups.com . >> For more options, visit https://groups.google.com/groups/opt_out. >> > >

ahh, i see the trickiness.

So yeah, cory’s discussion point on updating our metadata stored within the
case itself.

So will need to check with the other guys once they get back from summit,
but I’m seeing 2 ways moving forward.

1: Update the entire case document and how we store them in couch to
include this metadata - either as some new first class properties, or as a
block of _computed metadata that’s useful for processing.

2: Prior to sending the case document to reportcases, we add and index the
new metadata as we need.

Long term the preference I believe is for us to do it via option 1. I’ll
check in with the other devs on seeing if we want to look at that option
moving forward.

Dan

··· On Fri, Oct 25, 2013 at 10:20 AM, sienioslav wrote:

Hi,

Thank for your insights. I could be wrong, but I think it isn’t a good
idea in our case, to do query on “report_xforms” along with sorting by
’form.meta.username’ field, and retrieving case from “form.case” block,
since after this operation could be very time consuming along with our
retrieving methods. Moreover, if we would switch to "report_xforms"
mapping, we couldn’t use filters(like filtering by groups, or open/close
cases), because this mapping doesn’t contain those fields.

I think we should stick to “report_cases” mapping, and try find some way
to inject username into this mapping while indexing records, we tried to
modify mapping by using nested objects(just like it is done in
"reportxform_mapping.py"), but It failed because there is no ‘username’ in
couchdb case document.

Do you think, if there is any other correct way to solve with this problem?

Marcin

W dniu wtorek, 22 października 2013 22:19:36 UTC+2 użytkownik Daniel Myung
napisał:

If you want to sort by a concatenated username + full_name - that really
is just sorting by username. There’s no need to index and sort over a new
field that concatenates the two.

Support sort by username, and present it as a concatenation of the
user+username. In your rendering of the row, I’d make a cache of users so
you don’t do repeated user queries to get their full names.

So i’d query report_xforms and do a nested query. It should work in the
case where form.case is the case block in the form:
https://github.com/dimagi/commcare-hq/blob/master/
corehq/apps/api/es.py#L399https://github.com/dimagi/commcare-hq/blob/master/corehq/apps/api/es.py#L399

Dan

On Tue, Oct 22, 2013 at 2:11 AM, Cory Zue cz...@dimagi.com wrote:

Hi Marcin,

This has been merged into the master branch now.

I would recommend you don’t block the feature on being able to sort by
username / chw name yet, and make that a nice to have. I think any solution
is going to be pretty involved. That said, Dan is considering extending the
indices to include additional server-metadata as he mentioned, but I don’t
think you should block on that change.

Cory

On Mon, Oct 21, 2013 at 3:13 PM, sienioslav sieni...@gmail.com wrote:

Hi Dan

Unfortunately the source information of “chw_name” field isn’t in case
xform history - it is a new field which is a concatenation of 'username’
and ‘fullname’ fields from related user object.

Regarding ‘report_xforms’, I think I could take a look into that
changes(once It will be ready for testing). Is there any branch available
with these changes, just let me know where can I found it.

Regards,
Marcin

W dniu niedziela, 20 października 2013 19:34:57 UTC+2 użytkownik Daniel
Myung napisał:

Hi Marcin,

We are planning on phasing out full_cases, i’d look at the
report_cases field.

What’s the source of the information you want to display the chw_name
information? Is it the case’s xform history?

With regard to getting that mapping into the case document - this is
a todo that cory and I discussed to get as much of that into the case
document in couch so as to send it as unmodified as possible to
elasticsearch. The idea would be to have the metadata ofthe transaction
(xform) information be understandable without lookup incouch by some
computation/dereferencing happening upon submission.

Changes to the report_xforms index will have the case blocks indexed
which is another way to get access to sortable metadata about case
transactions. These are still being debugged and tested though - i should
have a version ready in the next few days. If you would ike to take a look
at that let me know and we can talk over the changes.

Dan

On Fri, Oct 18, 2013 at 11:49 AM, sienioslav sieni...@gmail.comwrote:

Hi,
Currently I’m working on the bihar reports, most of functionality is
already finished - but we still have some problems with sorting records
using ‘chw_name’ field. This field is concatenation of two user data fields:

chw_name = “username” + “full_name”.

ElasticSearch “full_cases” doesn’t provide such field in his mappings

  • In order to get such field from mapping, I assume I should modify that
    mapping and add new field which will contain data mentioned above.
    I read that I could use there, a ‘nested_type’ query to get related
    user data based on ‘user_id’ field of case, but I wonder If there is any
    other ways to solve this sorting issue. I would appreciate any
    insights/suggestions.

Regards,
Marcin


You received this message because you are subscribed to the Google
Groups “CommCare Developers” group.
To unsubscribe from this group and stop receiving emails from it,
send an email to commcare-developers+unsubscribe@googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_outhttps://groups.google.com/groups/opt_out
.


You received this message because you are subscribed to the Google
Groups “CommCare Developers” group.
To unsubscribe from this group and stop receiving emails from it, send
an email to commcare-developers+**unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.


You received this message because you are subscribed to the Google
Groups “CommCare Developers” group.
To unsubscribe from this group and stop receiving emails from it, send
an email to commcare-developers+**unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.


You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi Dan,

Thank for your reply, I’ll be waiting about any new information in this
case.

Regards,
Marcin

W dniu piątek, 25 października 2013 17:13:39 UTC+2 użytkownik Daniel Myung
napisał:

··· > > ahh, i see the trickiness. > > So yeah, cory's discussion point on updating our metadata stored within > the case itself. > > So will need to check with the other guys once they get back from summit, > but I'm seeing 2 ways moving forward. > > 1: Update the entire case document and how we store them in couch to > include this metadata - either as some new first class properties, or as a > block of _computed metadata that's useful for processing. > > 2: Prior to sending the case document to reportcases, we add and index the > new metadata as we need. > > Long term the preference I believe is for us to do it via option 1. I'll > check in with the other devs on seeing if we want to look at that option > moving forward. > > Dan > > > On Fri, Oct 25, 2013 at 10:20 AM, sienioslav <sieni...@gmail.com wrote: > >> Hi, >> >> Thank for your insights. I could be wrong, but I think it isn't a good >> idea in our case, to do query on "report_xforms" along with sorting by >> 'form.meta.username' field, and retrieving case from "form.case" block, >> since after this operation could be very time consuming along with our >> retrieving methods. Moreover, if we would switch to "report_xforms" >> mapping, we couldn't use filters(like filtering by groups, or open/close >> cases), because this mapping doesn't contain those fields. >> >> I think we should stick to "report_cases" mapping, and try find some way >> to inject username into this mapping while indexing records, we tried to >> modify mapping by using nested objects(just like it is done in >> "reportxform_mapping.py"), but It failed because there is no 'username' in >> couchdb case document. >> >> Do you think, if there is any other correct way to solve with this >> problem? >> >> Marcin >> >> W dniu wtorek, 22 października 2013 22:19:36 UTC+2 użytkownik Daniel >> Myung napisał: >>> >>> If you want to sort by a concatenated username + full_name - that really >>> is just sorting by username. There's no need to index and sort over a new >>> field that concatenates the two. >>> >>> Support sort by username, and present it as a concatenation of the >>> user+username. In your rendering of the row, I'd make a cache of users so >>> you don't do repeated user queries to get their full names. >>> >>> So i'd query report_xforms and do a nested query. It should work in the >>> case where form.case is the case block in the form: >>> https://github.com/dimagi/**commcare-hq/blob/master/** >>> corehq/apps/api/es.py#L399 >>> >>> Dan >>> >>> >>> On Tue, Oct 22, 2013 at 2:11 AM, Cory Zue wrote: >>> >>>> Hi Marcin, >>>> >>>> This has been merged into the master branch now. >>>> >>>> I would recommend you don't block the feature on being able to sort by >>>> username / chw name yet, and make that a nice to have. I think any solution >>>> is going to be pretty involved. That said, Dan is considering extending the >>>> indices to include additional server-metadata as he mentioned, but I don't >>>> think you should block on that change. >>>> >>>> Cory >>>> >>>> >>>> >>>> On Mon, Oct 21, 2013 at 3:13 PM, sienioslav wrote: >>>> >>>>> Hi Dan >>>>> >>>>> Unfortunately the source information of "chw_name" field isn't in case >>>>> xform history - it is a new field which is a concatenation of 'username' >>>>> and 'fullname' fields from related user object. >>>>> >>>>> Regarding 'report_xforms', I think I could take a look into that >>>>> changes(once It will be ready for testing). Is there any branch available >>>>> with these changes, just let me know where can I found it. >>>>> >>>>> Regards, >>>>> Marcin >>>>> >>>>> W dniu niedziela, 20 października 2013 19:34:57 UTC+2 użytkownik >>>>> Daniel Myung napisał: >>>>> >>>>>> Hi Marcin, >>>>>> >>>>>> We are planning on phasing out full_cases, i'd look at the >>>>>> report_cases field. >>>>>> >>>>>> What's the source of the information you want to display the chw_name >>>>>> information? Is it the case's xform history? >>>>>> >>>>>> With regard to getting that mapping into the *case* document - this >>>>>> is a todo that cory and I discussed to get as much of that into the case >>>>>> document in couch so as to send it as unmodified as possible to >>>>>> elasticsearch. The idea would be to have the metadata ofthe transaction >>>>>> (xform) information be understandable without lookup incouch by some >>>>>> computation/dereferencing happening upon submission. >>>>>> >>>>>> Changes to the report_xforms index will have the case blocks indexed >>>>>> which is another way to get access to sortable metadata about case >>>>>> transactions. These are still being debugged and tested though - i should >>>>>> have a version ready in the next few days. If you would ike to take a look >>>>>> at that let me know and we can talk over the changes. >>>>>> >>>>>> Dan >>>>>> >>>>>> >>>>>> >>>>>> On Fri, Oct 18, 2013 at 11:49 AM, sienioslav wrote: >>>>>> >>>>>>> Hi, >>>>>>> Currently I'm working on the bihar reports, most of functionality is >>>>>>> already finished - but we still have some problems with sorting records >>>>>>> using 'chw_name' field. This field is concatenation of two user data fields: >>>>>>> >>>>>>> chw_name = "username" + "full_name". >>>>>>> >>>>>>> ElasticSearch "full_cases" doesn't provide such field in his >>>>>>> mappings - In order to get such field from mapping, I assume I should >>>>>>> modify that mapping and add new field which will contain data mentioned >>>>>>> above. >>>>>>> I read that I could use there, a 'nested_type' query to get related >>>>>>> user data based on 'user_id' field of case, but I wonder If there is any >>>>>>> other ways to solve this sorting issue. I would appreciate any >>>>>>> insights/suggestions. >>>>>>> >>>>>>> Regards, >>>>>>> Marcin >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> --- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "CommCare Developers" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to commcare-developers+**unsubscrib** >>>>>>> e@googlegroups.com. >>>>>>> For more options, visit https://groups.google.com/**grou**ps/opt_out >>>>>>> . >>>>>>> >>>>>> >>>>>> -- >>>>> >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "CommCare Developers" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to commcare-developers+**unsubscribe@googlegroups.com. >>>>> For more options, visit https://groups.google.com/**groups/opt_out >>>>> . >>>>> >>>> >>>> -- >>>> >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "CommCare Developers" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to commcare-developers+**unsubscribe@googlegroups.com. >>>> For more options, visit https://groups.google.com/**groups/opt_out >>>> . >>>> >>> >>> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "CommCare Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to commcare-developers+unsubscribe@googlegroups.com . >> For more options, visit https://groups.google.com/groups/opt_out. >> > >

I recommend moving forward without username sort for now and making this a
"nice to have" feature for later on.

··· On Mon, Oct 28, 2013 at 1:50 PM, sienioslav wrote:

Hi Dan,

Thank for your reply, I’ll be waiting about any new information in this
case.

Regards,
Marcin

W dniu piątek, 25 października 2013 17:13:39 UTC+2 użytkownik Daniel Myung
napisał:

ahh, i see the trickiness.

So yeah, cory’s discussion point on updating our metadata stored within
the case itself.

So will need to check with the other guys once they get back from summit,
but I’m seeing 2 ways moving forward.

1: Update the entire case document and how we store them in couch to
include this metadata - either as some new first class properties, or as a
block of _computed metadata that’s useful for processing.

2: Prior to sending the case document to reportcases, we add and index
the new metadata as we need.

Long term the preference I believe is for us to do it via option 1. I’ll
check in with the other devs on seeing if we want to look at that option
moving forward.

Dan

On Fri, Oct 25, 2013 at 10:20 AM, sienioslav sieni...@gmail.com wrote:

Hi,

Thank for your insights. I could be wrong, but I think it isn’t a good
idea in our case, to do query on “report_xforms” along with sorting by
’form.meta.username’ field, and retrieving case from “form.case” block,
since after this operation could be very time consuming along with our
retrieving methods. Moreover, if we would switch to "report_xforms"
mapping, we couldn’t use filters(like filtering by groups, or open/close
cases), because this mapping doesn’t contain those fields.

I think we should stick to “report_cases” mapping, and try find some way
to inject username into this mapping while indexing records, we tried to
modify mapping by using nested objects(just like it is done in
"reportxform_mapping.py"), but It failed because there is no ‘username’ in
couchdb case document.

Do you think, if there is any other correct way to solve with this
problem?

Marcin

W dniu wtorek, 22 października 2013 22:19:36 UTC+2 użytkownik Daniel
Myung napisał:

If you want to sort by a concatenated username + full_name - that
really is just sorting by username. There’s no need to index and sort over
a new field that concatenates the two.

Support sort by username, and present it as a concatenation of the
user+username. In your rendering of the row, I’d make a cache of users so
you don’t do repeated user queries to get their full names.

So i’d query report_xforms and do a nested query. It should work in the
case where form.case is the case block in the form:
https://github.com/dimagi/commcare-hq/blob/master/corehq/
apps/api/es.py#L399

Dan

On Tue, Oct 22, 2013 at 2:11 AM, Cory Zue cz...@dimagi.com wrote:

Hi Marcin,

This has been merged into the master branch now.

I would recommend you don’t block the feature on being able to sort by
username / chw name yet, and make that a nice to have. I think any solution
is going to be pretty involved. That said, Dan is considering extending the
indices to include additional server-metadata as he mentioned, but I don’t
think you should block on that change.

Cory

On Mon, Oct 21, 2013 at 3:13 PM, sienioslav sieni...@gmail.comwrote:

Hi Dan

Unfortunately the source information of “chw_name” field isn’t in
case xform history - it is a new field which is a concatenation of
’username’ and ‘fullname’ fields from related user object.

Regarding ‘report_xforms’, I think I could take a look into that
changes(once It will be ready for testing). Is there any branch available
with these changes, just let me know where can I found it.

Regards,
Marcin

W dniu niedziela, 20 października 2013 19:34:57 UTC+2 użytkownik
Daniel Myung napisał:

Hi Marcin,

We are planning on phasing out full_cases, i’d look at the
report_cases field.

What’s the source of the information you want to display the
chw_name information? Is it the case’s xform history?

With regard to getting that mapping into the case document - this
is a todo that cory and I discussed to get as much of that into the case
document in couch so as to send it as unmodified as possible to
elasticsearch. The idea would be to have the metadata ofthe transaction
(xform) information be understandable without lookup incouch by some
computation/dereferencing happening upon submission.

Changes to the report_xforms index will have the case blocks indexed
which is another way to get access to sortable metadata about case
transactions. These are still being debugged and tested though - i should
have a version ready in the next few days. If you would ike to take a look
at that let me know and we can talk over the changes.

Dan

On Fri, Oct 18, 2013 at 11:49 AM, sienioslav sieni...@gmail.comwrote:

Hi,
Currently I’m working on the bihar reports, most of functionality
is already finished - but we still have some problems with sorting records
using ‘chw_name’ field. This field is concatenation of two user data fields:

chw_name = “username” + “full_name”.

ElasticSearch “full_cases” doesn’t provide such field in his
mappings - In order to get such field from mapping, I assume I should
modify that mapping and add new field which will contain data mentioned
above.
I read that I could use there, a ‘nested_type’ query to get related
user data based on ‘user_id’ field of case, but I wonder If there is any
other ways to solve this sorting issue. I would appreciate any
insights/suggestions.

Regards,
Marcin


You received this message because you are subscribed to the Google
Groups “CommCare Developers” group.
To unsubscribe from this group and stop receiving emails from it,
send an email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google
Groups “CommCare Developers” group.
To unsubscribe from this group and stop receiving emails from it,
send an email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google
Groups “CommCare Developers” group.
To unsubscribe from this group and stop receiving emails from it, send
an email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google
Groups “CommCare Developers” group.
To unsubscribe from this group and stop receiving emails from it, send
an email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.