A little help orienting myself in the API documentation

Users
1

Hello folks --

I am in a situation where we are considering doing a bit of integration
between our commcare project and a third-party web application (PHP/mySQL)
that might, in certain circumstances, want to connect with the commcareHQ
API, authenticate itself as a given user, and basically issue some commands
to submit one form of a type that we have already of course defined and are
making use of in our app. Furthermore, that form, that we'd like this
other app to complete and submit via the API, has its case management set
up so that it requires an existing case, as well as in some conditions will
create a new case of a different case type.

I am not the programmer that will do this coding, but I do want to be
oriented as to the basic sections of the API documentation apply to the
various steps, so that I can get this programmer (who doesn't know
commcare) at least ... oriented to start off with. :slight_smile:

I am looking at the table of about 20 APIs that I see listed
on CommCare HQ APIs - CommCare Public - CommCare Public.

I'm a little bit uncertain about which API will be able to concretely carry
out the following steps. I was hoping that someone could look at my crude
list of expected steps below, and tell me which API has the commands to
allow these to be invoked for that step. Sorry if this seems needy or even
wrong-headed -- it will just make it easier for me to be confident that I'm
basically oriented, and also to start my team off in the right mindset.

This PHP application, after having collected a bunch of information in its
own forms and variables, will then hopefully carry out these steps. For
each one of these, can you tell me which API our programmer should look at
to start learning how to make it happen? Again, sorry if this seems
painfully obvious to you:

1-- the third party app makes a call to authenticate itself as a given
user, using its password ---> ________ api

[Now we ultimately want to complete a form whose name is "C1 Referral
Followup" which however requires the selection of an existing case for
which to complete the form. Therefore:]

2-- Their app asks the commcare system for the list of cases that the
logged-in user has available to them that are allowed for that C1 Referral
Followup form. (in other words, the list of available cases that the user
is normally presented with when they go to the C1 form on the normal mobile
phone app.) ... so that their web app can then present that list to the
user and allow them to select one of them. ----> __________ api

3-- Their app takes a number of fields/variables that they have already
collected from the user, and transmits that data to the CommCareHQ server,
specifying which of the C1 form's fields are being populated with what
pieces of data, and resulting in the form being fully submitted and saved
to the CommCareHQ server. ---> ______ api

4-- The CommCareHQ system, which has already been configured in its case
management that under certain circumstances a new case should be created
and passing certain values to the new case, does that as normal --- Does
that process happen automatically in the CommCareHQ app since the Case
Management for that form has been specified right there on the CommCareHQ
server? (I hope!) Or does the remote web app need to invoke something for
that additional case to be created?

5-- And just to be clear, the other existing case management configurations
on CommCareHQ will also handle the data submitted themselves, right? no
additional things that the remote app would have to do for that to happen?

6-- Do you have any other notes, or things that I should flag, about
possible complexities in the story that I just told above?

Thanks much for your inputs here, this will set me off on the right foot.

Eric

2

Hey Eric,

Apologies for brevity, but quick responses inline.

Hello folks --

I am in a situation where we are considering doing a bit of integration
between our commcare project and a third-party web application (PHP/mySQL)
that might, in certain circumstances, want to connect with the commcareHQ
API, authenticate itself as a given user, and basically issue some commands
to submit one form of a type that we have already of course defined and are
making use of in our app. Furthermore, that form, that we'd like this
other app to complete and submit via the API, has its case management set
up so that it requires an existing case, as well as in some conditions will
create a new case of a different case type.

I am not the programmer that will do this coding, but I do want to be
oriented as to the basic sections of the API documentation apply to the
various steps, so that I can get this programmer (who doesn't know
commcare) at least ... oriented to start off with. :slight_smile:

I am looking at the table of about 20 APIs that I see listed on
CommCare HQ APIs - CommCare Public - CommCare Public.

I'm a little bit uncertain about which API will be able to concretely
carry out the following steps. I was hoping that someone could look at my
crude list of expected steps below, and tell me which API has the commands
to allow these to be invoked for that step. Sorry if this seems needy or
even wrong-headed -- it will just make it easier for me to be confident
that I'm basically oriented, and also to start my team off in the right
mindset.

This PHP application, after having collected a bunch of information in its
own forms and variables, will then hopefully carry out these steps. For
each one of these, can you tell me which API our programmer should look at
to start learning how to make it happen? Again, sorry if this seems
painfully obvious to you:

1-- the third party app makes a call to authenticate itself as a given
user, using its password ---> ________ api

Authentication will happen on every request so this does not exist.

[Now we ultimately want to complete a form whose name is "C1 Referral
Followup" which however requires the selection of an existing case for
which to complete the form. Therefore:]

2-- Their app asks the commcare system for the list of cases that the
logged-in user has available to them that are allowed for that C1 Referral
Followup form. (in other words, the list of available cases that the user
is normally presented with when they go to the C1 form on the normal mobile
phone app.) ... so that their web app can then present that list to the
user and allow them to select one of them. ----> __________ api

List cases
https://confluence.dimagi.com/pages/viewpage.action?pageId=12224287
(filter by user id or owner id)

3-- Their app takes a number of fields/variables that they have already
collected from the user, and transmits that data to the CommCareHQ server,
specifying which of the C1 form's fields are being populated with what
pieces of data, and resulting in the form being fully submitted and saved
to the CommCareHQ server. ---> ______ api

Submission
https://confluence.dimagi.com/display/commcarepublic/Submission+API

4-- The CommCareHQ system, which has already been configured in its case
management that under certain circumstances a new case should be created
and passing certain values to the new case, does that as normal --- Does
that process happen automatically in the CommCareHQ app since the Case
Management for that form has been specified right there on the CommCareHQ
server? (I hope!) Or does the remote web app need to invoke something for
that additional case to be created?

You will need to submit valid casexml
https://github.com/dimagi/commcare/wiki/casexml20 to create/modify cases.

5-- And just to be clear, the other existing case management
configurations on CommCareHQ will also handle the data submitted
themselves, right? no additional things that the remote app would have to
do for that to happen?

Once the cases are created/updated they will look "normal" to CommCare, so
if you set the type and properties the same the app will treat them the
same.

6-- Do you have any other notes, or things that I should flag, about
possible complexities in the story that I just told above?

The submission + casexml API will be the tricky part.

··· On Wed, Sep 2, 2015 at 4:52 AM, Eric Stephan wrote:

Thanks much for your inputs here, this will set me off on the right foot.

Eric

--
You received this message because you are subscribed to the Google Groups
"commcare-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

3

Hello Cory and all -- Thanks very much for your very useful info about the
commcare API process earlier in this thread. I now have a "second level"
of questions to make sure I am oriented correctly about the API, so that
when I work with the coding side of our team, I'm sure we are headed in the
right direction.

(In fact, in another thread you have given help to my colleague working on
this same project, who is learning about the API process so he can guide
his coders. That thread is
Redirecting to Google Groups .)

** As a reminder, we have a fully working commcare project that involves a
few separate apps and two different levels of referrals. To over-simplify,
our outreach workers fill out form A1 when they interview people on the
street. When we determine that person needs a referral to HIV testing, it
creates a referral by creating a new child case of type "referral." Then,
over in the HIV testing clinic, the clinicians pick up their commcare phone
and go to form *C1 *which allows them to open up that "referral" case and
fill out a form describing the results of the HIV test visit. And if the
results of that test were HIV positive, then the app is designed to create
a further referral to ART care, by creating a new child case, of type
"art-client," which will later on be picked up over in the ART clinic in
form D1.

** That is all working fine. But now our goal is to make it so that over in
that middle clinic -- the HIV testing clinic described above that would
normally use our smartphone app to load up cases on form C1 and sometimes
create child cases -- we can use the CommCare API to actually build that
full "C1 form" process into a PHP-based web app that is currently operating
in those clinics. Therefore, the web app would replace that smartphone in
that one clinic's role in the chain that I describe above.

(1) My first question is just confirming my understanding. Earlier, I had
been hoping that the info that the web app would have to submit through the
API would be somewhat minimal, and that much of the complex plumbing stuff
would be taken care of on the configurations already existing on the
CommCare servers. But now I am understanding that when the person using
their PHP web app goes to do the C1 form, and they select which 'referral'
case they want to open and fill out the form for, the web app will actually
have to do a lot of loading up of values from that existing case... and
then, after the web app contributes the few new pieces of info collected by
the web app (like the results of the HIV test etc.), if a child
case/referral is indeed required to be made, then their web app will also
have to generate all the right code to fully create the new ongoing case
("art-client") and then submit that.
--> Other than confirming that I understand that correctly, my only
question here is probably obvious to all -- can you remind me which API is
used to load the values from the existing case into variables in the web
app, so that they can later be submitted correctly in the XML?

(2) My second question is simply trying to orient myself in the XML that
the C1 form created. Since the web app will have to output this XML
faithfully, I wanted to make sure I understood the general overview, so
that I and the coding team will be headed in the right direction with the
same understanding.
... So I filled out one of our C1 forms completely, giving responses in
which the patient tested HIV positive and therefore a child case is indeed
created. I am now attaching the XML that resulted from that C1 form
submission -- I have highlighted with a little color. Here are some
embarrassingly high-level questions to make sure I'm oriented OK. And if I
appear to be misunderstanding things, please point that out too:

** The first section, in green -- this seems simply submitting exactly the
fields that were completed when the C1 form was filled out. This seems
straightforward.
** The second section, whose tags start n0, in blue, seems to be only those
the fields that I had configured to save their values to the PARENT case
rather than the case itself. That I also basically understand.
** The third section, whose tags start n1, in yellow, seems to be all the
info that is needed for the CHILD case that is being created because the
person tested HIV positive. That also seems clear.
** But the fourth section, whose tags start n2, in grey, is confusing to
me. It seems that it is simply submitting a few fields that were already
submitted up in the first section (the "simply the values from this form"
one.) For example, it is submitting the "hct-services-provided" field
again, exactly as it did up above. Can you tell what this grey "n2" section
is for?
** The last section, n3, just seems like various metadata about versions
and deviceIDs.

(3) A few odd questions:

-- When we go to analyze our app's data, we have very attentive to make
sure we can do analysis that is able to recall the exact links between a
given A1 form submission, its resulting exact C1 form submission, and its
resulting D1 form submission. Looking at the data, I have found that the
workable key value to connect all of those is the A1 form's info.caseid,
the C1 form's parents.parent.case.@case_id, and the D1 form's
parents.parent.parent.case.@case_id. Is there any chance that when we
implement this API to allow a web app to take the place of the mobile app's
C1 form, that that chain of key values could be spoiled? That method of
linking the three forms will still work, right?

-- it's a little bit daunting to see all of those identifier values placed
into the headers of those sections in the XML. for example, things like
case_id, user_id, instanceID, owner_id, deviceID, etc. But I'm guessing
that all those IDs are nicely obtained through the API calls that we will
use to locate and load up the case. It can't hurt for me to ask, are any of
those values not actually fully required? It is good for me to ask in case
it saves us a bit of effort :slight_smile:

Giant thanks for your help, and tolerance of my unbelievably verbose
approach :slight_smile:

Eric

xml-for-our-C1-form.docx (18.8 KB)

4

Hi Eric

I'll take a quick stab at answering your questions. Here is another
resource that may be helpful:

The general process that happens when you fill out a form is as follows
(you may already know this):

  • Form session created with necessary data
  • As questions get completed in the form the values in the main XML
    'instance' (see xform-spec) get populated via the 'bind' expressions
  • Once the form is complete the instance XML is ready to submit to the
    server with all values populated.

Now let me try and answer your questions below.

Hello Cory and all -- Thanks very much for your very useful info about
the commcare API process earlier in this thread. I now have a "second
level" of questions to make sure I am oriented correctly about the API, so
that when I work with the coding side of our team, I'm sure we are headed
in the right direction.

(In fact, in another thread you have given help to my colleague working on
this same project, who is learning about the API process so he can guide
his coders. That thread is
Redirecting to Google Groups .)

** As a reminder, we have a fully working commcare project that involves a
few separate apps and two different levels of referrals. To over-simplify,
our outreach workers fill out form A1 when they interview people on the
street. When we determine that person needs a referral to HIV testing, it
creates a referral by creating a new child case of type "referral." Then,
over in the HIV testing clinic, the clinicians pick up their commcare phone
and go to form *C1 *which allows them to open up that "referral" case and
fill out a form describing the results of the HIV test visit. And if the
results of that test were HIV positive, then the app is designed to create
a further referral to ART care, by creating a new child case, of type
"art-client," which will later on be picked up over in the ART clinic in
form D1.

** That is all working fine. But now our goal is to make it so that over
in that middle clinic -- the HIV testing clinic described above that would
normally use our smartphone app to load up cases on form C1 and sometimes
create child cases -- we can use the CommCare API to actually build that
full "C1 form" process into a PHP-based web app that is currently operating
in those clinics. Therefore, the web app would replace that smartphone in
that one clinic's role in the chain that I describe above.

(1) My first question is just confirming my understanding. Earlier, I had
been hoping that the info that the web app would have to submit through the
API would be somewhat minimal, and that much of the complex plumbing stuff
would be taken care of on the configurations already existing on the
CommCare servers. But now I am understanding that when the person using
their PHP web app goes to do the C1 form, and they select which 'referral'
case they want to open and fill out the form for, the web app will actually
have to do a lot of loading up of values from that existing case... and
then, after the web app contributes the few new pieces of info collected by
the web app (like the results of the HIV test etc.), if a child
case/referral is indeed required to be made, then their web app will also
have to generate all the right code to fully create the new ongoing case
("art-client") and then submit that.
--> Other than confirming that I understand that correctly, my only
question here is probably obvious to all -- can you remind me which API is
used to load the values from the existing case into variables in the web
app, so that they can later be submitted correctly in the XML?

With reference to my explanation above, the form engine uses the 'external
instance' API on the phone to load case properties and other values form
the session:

Without using a form engine you could manually insert values from the case
into the XML by parsing the 'bind' expressions to see which values are
required.

(2) My second question is simply trying to orient myself in the XML that
the C1 form created. Since the web app will have to output this XML
faithfully, I wanted to make sure I understood the general overview, so
that I and the coding team will be headed in the right direction with the
same understanding.
... So I filled out one of our C1 forms completely, giving responses in
which the patient tested HIV positive and therefore a child case is indeed
created. I am now attaching the XML that resulted from that C1 form
submission -- I have highlighted with a little color. Here are some
embarrassingly high-level questions to make sure I'm oriented OK. And if I
appear to be misunderstanding things, please point that out too:

** The first section, in green -- this seems simply submitting exactly the
fields that were completed when the C1 form was filled out. This seems
straightforward.
** The second section, whose tags start n0, in blue, seems to be only
those the fields that I had configured to save their values to the PARENT
case rather than the case itself. That I also basically understand.
** The third section, whose tags start n1, in yellow, seems to be all the
info that is needed for the CHILD case that is being created because the
person tested HIV positive. That also seems clear.
** But the fourth section, whose tags start n2, in grey, is confusing to
me. It seems that it is simply submitting a few fields that were already
submitted up in the first section (the "simply the values from this form"
one.) For example, it is submitting the "hct-services-provided" field
again, exactly as it did up above. Can you tell what this grey "n2" section
is for?
** The last section, n3, just seems like various metadata about versions
and deviceIDs.

The way we process the XML on the server is to extract all the 'case'
elements and treat them as a transaction on the case to which the apply
(via the @case_id attribute). The reason there are duplicate values is to
avoid having to do any evaluation logic on the server to determine what the
case values should be. All the logic happens in the form.

The n2 section corresponds to the 'main' case that was selected for this
form.

(3) A few odd questions:

-- When we go to analyze our app's data, we have very attentive to make
sure we can do analysis that is able to recall the exact links between a
given A1 form submission, its resulting exact C1 form submission, and its
resulting D1 form submission. Looking at the data, I have found that the
workable key value to connect all of those is the A1 form's info.caseid,
the C1 form's parents.parent.case.@case_id, and the D1 form's
parents.parent.parent.case.@case_id. Is there any chance that when we
implement this API to allow a web app to take the place of the mobile app's
C1 form, that that chain of key values could be spoiled? That method of
linking the three forms will still work, right?

Provided that the correct ID values are supplied in the form XML the server
will process it as normal and the linking will work.

-- it's a little bit daunting to see all of those identifier values placed
into the headers of those sections in the XML. for example, things like
case_id, user_id, instanceID, owner_id, deviceID, etc. But I'm guessing
that all those IDs are nicely obtained through the API calls that we will
use to locate and load up the case. It can't hurt for me to ask, are any of
those values not actually fully required? It is good for me to ask in case
it saves us a bit of effort :slight_smile:

  • case_id
    • the ID's of the various cases
    • If a new case is being created you should generate a new UUID for
      the case (only do this for case blocks that have a 'create'
      element in them)
    • List Cases (version 3+) - CommCare Public - CommCare Public
    • user_id
    • The ID of the user filling out the form (this should be a mobile
      worker)
··· On 21 October 2015 at 07:16, Eric Stephan wrote: - https://confluence.dimagi.com/display/commcarepublic/List+Mobile+Workers - instanceID - the ID of this form - each form must have a unique ID - you should generate an ID and insert it prior to submission - we use UUIDv4 - owner_id - This is the ID of a mobile worker or group who should be the owner of this case - If it is the ID of mobile worker then it should be the same a 'user_id' - It should only be a group ID if you are doing case sharing - If it is a group ID then it must be the ID of a group which the user is part of - A users group IDs are included in the data from the user list API: https://confluence.dimagi.com/display/commcarepublic/List+Mobile+Workers - deviceID - the ID of the device submitting the data - this can be empty

I hope that answers your questions.
Best regards

--
Simon Kelly
Senior Engineer | Dimagi South Africa