Looking for DHIS2 + CommCare information -- Who uses it?

Hello all–

I am realizing that I should learn more about how well DHIS2 can work with
CommCare. It might prove to be a valuable tool for our analysis needs that
are stemming from our HIV outreach/referrals app… these analysis needs
get quite complex, involving tracing of individuals across various levels
of referrals; calculation of complex indicators; and accounting and tracing
client recruitment coupons… and also with our needs to possibly
"segregate" sections of our data so that some organizations can log in and
only see data related to their own organization.

My questions are:

– Who among us has connected CommCare with DHIS2, and who can tell me that
the connection process was complex and disappointing, or smooth and
rewarding?

– What are the strongest online user groups for DHIS2? I’ve been searching
for online DHIS2 forums, and haven’t found one central one… the DHIS2
folks don’t see to be as strong on the community side as I’m used to over
here :slight_smile:

– I’m not super impressed by the tutorials/learning materials I’ve found
so far on the DHIS2 site – so if you know a learning resource other than
those, please tell me!

Thanks folks!

Eric

Hi Eric -

We’ve been increasing our work on DHIS2 integration. Sheel Shah manages
our overall work on MOTECH which acts as the integration layer. We have it
live in a few countries and are adding it more. Since its been mostly
Dimagi or Grameen staff who have done the mapping/configuration to date, my
guess is your experience if you tried it would be closer to complex than
smooth - though I hope rewarding in any case!.

That said, we are getting it easier and easier with each release of
MOTECH. The document is going to be coming out over the next few
weeks/months as we get all the kinks worked out. We can offline if you
want to setup a sandbox and play with it, would be great to get your
feedback. Ping me if that’s something you want to look in to.

I don’t know good answers to the questions on the DHIS2 Forum or
Tutorials. I know DHIS2 does a lot of in person capacity building so that
may be an avenue if it works with your schedule.

Regards,
Jonathan

··· On Tue, Feb 2, 2016 at 2:28 AM, Eric Stephan wrote:

Hello all–

I am realizing that I should learn more about how well DHIS2 can work with
CommCare. It might prove to be a valuable tool for our analysis needs that
are stemming from our HIV outreach/referrals app… these analysis needs
get quite complex, involving tracing of individuals across various levels
of referrals; calculation of complex indicators; and accounting and tracing
client recruitment coupons… and also with our needs to possibly
"segregate" sections of our data so that some organizations can log in and
only see data related to their own organization.

My questions are:

– Who among us has connected CommCare with DHIS2, and who can tell me
that the connection process was complex and disappointing, or smooth and
rewarding?

– What are the strongest online user groups for DHIS2? I’ve been
searching for online DHIS2 forums, and haven’t found one central one… the
DHIS2 folks don’t see to be as strong on the community side as I’m used to
over here :slight_smile:

– I’m not super impressed by the tutorials/learning materials I’ve found
so far on the DHIS2 site – so if you know a learning resource other than
those, please tell me!

Thanks folks!

Eric


You received this message because you are subscribed to the Google Groups
"commcare-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hi Eric,

At VaxTrac we’re successfully forwarding CommCare data to MoTech and DHIS2.
Our engineer Samantha can tell you a lot more and commiserate with your
experience.

You’re welcome to reach out to her at sam.rabenjamina@vaxtrac.com or myself
at duncan.spencer@vaxtrac.com

··· On Tuesday, February 2, 2016 at 10:33:32 AM UTC-5, Jonathan Jackson wrote: > > Hi Eric - > > We've been increasing our work on DHIS2 integration. Sheel Shah manages > our overall work on MOTECH which acts as the integration layer. We have it > live in a few countries and are adding it more. Since its been mostly > Dimagi or Grameen staff who have done the mapping/configuration to date, my > guess is your experience if you tried it would be closer to complex than > smooth - though I hope rewarding in any case!. > > That said, we are getting it easier and easier with each release of > MOTECH. The document is going to be coming out over the next few > weeks/months as we get all the kinks worked out. We can offline if you > want to setup a sandbox and play with it, would be great to get your > feedback. Ping me if that's something you want to look in to. > > I don't know good answers to the questions on the DHIS2 Forum or > Tutorials. I know DHIS2 does a lot of in person capacity building so that > may be an avenue if it works with your schedule. > > Regards, > Jonathan > > On Tue, Feb 2, 2016 at 2:28 AM, Eric Stephan <estep...@gmail.com > wrote: > >> >> Hello all-- >> >> I am realizing that I should learn more about how well DHIS2 can work >> with CommCare. It might prove to be a valuable tool for our analysis needs >> that are stemming from our HIV outreach/referrals app... these analysis >> needs get quite complex, involving tracing of individuals across various >> levels of referrals; calculation of complex indicators; and accounting and >> tracing client recruitment coupons... and also with our needs to possibly >> "segregate" sections of our data so that some organizations can log in and >> only see data related to their own organization. >> >> My questions are: >> >> -- Who among us has connected CommCare with DHIS2, and who can tell me >> that the connection process was complex and disappointing, or smooth and >> rewarding? >> >> -- What are the strongest online user groups for DHIS2? I've been >> searching for online DHIS2 forums, and haven't found one central one... the >> DHIS2 folks don't see to be as strong on the community side as I'm used to >> over here :) >> >> -- I'm not super impressed by the tutorials/learning materials I've found >> so far on the DHIS2 site -- so if you know a learning resource other than >> those, please tell me! >> >> Thanks folks! >> >> Eric >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "commcare-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to commcare-user...@googlegroups.com . >> For more options, visit https://groups.google.com/d/optout. >> > >
1 Like

Thank you to everyone that has replied to me in this thread and offline –
Sorry for the delay.

It is great to hear that an “official route” is emerging. Gives me the
sense that there is a correct path to take in this challenge.

(I also sense that a strongly combined CommCare plus DHIS2 would be an
incredible thing to work with. If I can simply find the time to learn it.)

I will follow up with Sheel and also in the Vaxtrac guys that you were so
great to refer me to.

Thanks again!
Eric

Hey Eric, I think I got a hold of you offline, but wanted to follow up
here just in case.

At OpenFn.org we’ve got an open-source CommCare to DHIS2 integration. It
relies on CommCare’s JSON data forwarding, and lets a user define a "job"
that pushes CommCare cases or forms to DHIS2 using the Events API and
DataValues API in real-time. Solid error handling and even source data
manipulation if you’ve got data entry or validation mismatch problems.
(It’s like Zapier, but allows you to manipulate data after it’s submitted,
performs updates using external IDs, and will store new submissions from
CommCare even if you exceed your monthly limit.) Let me know if you want to
check it out!

Best,
Taylor

··· On Friday, 5 February 2016 03:52:59 UTC+2, Eric Stephan wrote: > > Thank you to everyone that has replied to me in this thread and offline -- > Sorry for the delay. > > It is great to hear that an "official route" is emerging. Gives me the > sense that there is a correct path to take in this challenge. > > (I also sense that a strongly combined CommCare plus DHIS2 would be an > incredible thing to work with. If I can simply find the time to learn it.) > > I will follow up with Sheel and also in the Vaxtrac guys that you were so > great to refer me to. > > Thanks again! > Eric >

Ah yes, also another great option. Another plug for Taylor and team: I
believe they also have CommCare->SalesForce integration as well.

thanks,
Jonathan

··· On Mon, Feb 8, 2016 at 3:59 AM, Taylor Downs wrote:

Hey Eric, I think I got a hold of you offline, but wanted to follow up
here just in case.

At OpenFn.org we’ve got an open-source CommCare to DHIS2 integration. It
relies on CommCare’s JSON data forwarding, and lets a user define a "job"
that pushes CommCare cases or forms to DHIS2 using the Events API and
DataValues API in real-time. Solid error handling and even source data
manipulation if you’ve got data entry or validation mismatch problems.
(It’s like Zapier, but allows you to manipulate data after it’s submitted,
performs updates using external IDs, and will store new submissions from
CommCare even if you exceed your monthly limit.) Let me know if you want to
check it out!

Best,
Taylor

On Friday, 5 February 2016 03:52:59 UTC+2, Eric Stephan wrote:

Thank you to everyone that has replied to me in this thread and offline
– Sorry for the delay.

It is great to hear that an “official route” is emerging. Gives me the
sense that there is a correct path to take in this challenge.

(I also sense that a strongly combined CommCare plus DHIS2 would be an
incredible thing to work with. If I can simply find the time to learn it.)

I will follow up with Sheel and also in the Vaxtrac guys that you were so
great to refer me to.

Thanks again!
Eric


You received this message because you are subscribed to the Google Groups
"commcare-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hi Taylor
I am interested in knowing more about the Commcare to DHIS2 integration at
OpenFn.org

Yaw

··· On 8 February 2016 at 08:59, Taylor Downs wrote:

Hey Eric, I think I got a hold of you offline, but wanted to follow up
here just in case.

At OpenFn.org we’ve got an open-source CommCare to DHIS2 integration. It
relies on CommCare’s JSON data forwarding, and lets a user define a "job"
that pushes CommCare cases or forms to DHIS2 using the Events API and
DataValues API in real-time. Solid error handling and even source data
manipulation if you’ve got data entry or validation mismatch problems.
(It’s like Zapier, but allows you to manipulate data after it’s submitted,
performs updates using external IDs, and will store new submissions from
CommCare even if you exceed your monthly limit.) Let me know if you want to
check it out!

Best,
Taylor

On Friday, 5 February 2016 03:52:59 UTC+2, Eric Stephan wrote:

Thank you to everyone that has replied to me in this thread and offline
– Sorry for the delay.

It is great to hear that an “official route” is emerging. Gives me the
sense that there is a correct path to take in this challenge.

(I also sense that a strongly combined CommCare plus DHIS2 would be an
incredible thing to work with. If I can simply find the time to learn it.)

I will follow up with Sheel and also in the Vaxtrac guys that you were so
great to refer me to.

Thanks again!
Eric


You received this message because you are subscribed to the Google Groups
"commcare-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

This thread has really allowed me to start the process of orienting myself
on this kind of integration.

The various discussions I have had have pointed at a few avenues… I may
not be accurate in my comments below…

– integrate by means of the MOTECH app that creates a connecting
application to direct commcare data automatically to DHIS2. A strength
seems to be that it can even create more than 1-on-1 connections among
apps… could even be bringing four apps into sync on an ongoing basis. (a
bit too much for me to comprehend now)

– integrate via OpenFN, which appears possibly a bit less complex in its
setup and function, but might be more 1-on-1

– Also I learned that an alternative to DHIS2 itself could be in fact
using Salesforce as a data analysis repository. hadn’t thought of that. I
was used to thinking of Salesforce as something used by big for-profit
companies… not by HIV programs like mine…

will keep learning more. Wish I could put the world on pause and simply sit
down and train myself, over unlimited time, on a few of these apps.

Hey Yaw,

Are you free for a quick call so I can walk you through our tools?

CommCare provides a data forwarding service that pushes forms and/or cases
as JSON to a secure URL. We then manipulate this source data and send
messages to DHIS2, using either the Events API or the DataValueSets API.
(Could add more of DHIS2’s apis, but that’s all we’ve plugged in for now.)

This whole process can be done from a command line using fn-lang
https://github.com/OpenFn/fn-lang, language-common
https://github.com/OpenFn/language-common, and language-DHIS2
https://github.com/OpenFn/language-dhis2*, or via our hosted service,
which you can check out here https://github.com/OpenFn/core-docs. (See
the screenshots for an idea of the flow, and look at the sample
"expressions" here
https://github.com/OpenFn/language-dhis2#current-datavaluesets-api-expressiontoo-bulky
to see what the data transformation looks like.)

Hope this is a good start. Feel free to reach out to me on email (
taylor@openfn.org) or Skype (taylor.downs) if you’re up for a quick walk
through!

Taylor

*Note that since CommCare delivers valid JSON, we don’t need a special
"source adaptor" to set up a CommCare to DHIS2 integration. Open Data Kit,
Ona.io, and plenty of other applications can deliver valid JSON to a URL
when something happens (a form submission, for example) so all of these
tools can be integrated to DHIS2 in a matter of minutes.

··· On Wed, 24 Feb 2016 at 07:33 YAW ANTWI-ADJEI wrote:

Hi Taylor
I am interested in knowing more about the Commcare to DHIS2 integration at
OpenFn.org

Yaw

On 8 February 2016 at 08:59, Taylor Downs downs.taylor@gmail.com wrote:

Hey Eric, I think I got a hold of you offline, but wanted to follow up
here just in case.

At OpenFn.org we’ve got an open-source CommCare to DHIS2 integration. It
relies on CommCare’s JSON data forwarding, and lets a user define a "job"
that pushes CommCare cases or forms to DHIS2 using the Events API and
DataValues API in real-time. Solid error handling and even source data
manipulation if you’ve got data entry or validation mismatch problems.
(It’s like Zapier, but allows you to manipulate data after it’s submitted,
performs updates using external IDs, and will store new submissions from
CommCare even if you exceed your monthly limit.) Let me know if you want to
check it out!

Best,
Taylor

On Friday, 5 February 2016 03:52:59 UTC+2, Eric Stephan wrote:

Thank you to everyone that has replied to me in this thread and offline
– Sorry for the delay.

It is great to hear that an “official route” is emerging. Gives me the
sense that there is a correct path to take in this challenge.

(I also sense that a strongly combined CommCare plus DHIS2 would be an
incredible thing to work with. If I can simply find the time to learn it.)

I will follow up with Sheel and also in the Vaxtrac guys that you were
so great to refer me to.

Thanks again!
Eric

You received this message because you are subscribed to the Google Groups

“commcare-users” group.

To unsubscribe from this group and stop receiving emails from it, send an

email to commcare-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Hi,

We are using openfn to integrate with DHIS2, for cases it can be tricky to avoid duplicate (once sync the phone send the registration and follow up form millisecond apart, this create 2 push) but we managed it.

If one is interested we are working on openHIM meditor that work with the configuration from openFN (it use openfn core),in order to sustainable solution when the project ends. https://github.com/SwissTPH/openfn-openhim-mediator

br

2 Likes

This looks super cool. Have started watching the repo and I’m happy to help out if you need anything that side. (Might also dovetail nicely with OpenFn/microservice—a project we’re hoping to release in the next month or so which creates a dockerized, OpenFn webserver, sans UI, using our open-source tools to run a single job, with trigger, credential, etc., as a microservice!)

1 Like

Exciting to see the work your team is putting in @pmpdelcroix! Connecting CommCare to DHIS2 is a use case we see very often. What type of integration are you setting up for - aggregated data, anonymous events, tracked entities?

Hi,

We take case data from CommCare and we sent it to a DHIS2 tracker via the openFn SaS, we will try Prod data integration with the openHIM mediator shortly, I’ll keep you posted.

br

Dear all,

We activated the forwarding to openHIM + our openFN mediator and more than 1000 cases were sent in a day, we still have some small updates to do but it looks good.

I do have an issue with CommCare, in our openFn mediator we have a trigger that check if the case need to be sent to DHIS2 or not (basically we are sending only closed cases) when it is not closed we configure our mediator the answer 409 (conflict) but commcare treat it like a 401/404 and try to resend the case over and over. Is there a way to have comcare managing the 409 like openHIM as a “Completed” but not “successful” push ? (we could also use 406 - Not acceptable)

br

Hi Patrick,

Glad to hear that things are working overall.

Treating a 409 is a bit tricky overall because although Form Forwarding will only send records once, Case Forwarding will send records repeatedly as they are updated.

Out of curiosity is there a reason why a 200 wouldn’t be an appropriate response for the mediator in this case? It sounds like the record has been processed by the downstream server in your case, even if it hasn’t elected to create a matching record, so a 201 might not be valid, but a 200 response doesn’t seem inaccurate.

-Clayton

Thank you for the fast answer,

having something which is not 2XX nor 5XX enable us to see the ratio of the completed vs successful on openhim, it can help to detect issues on the trigger.

image

If 409 is an issue another 4xx can be used: 406(Not Acceptable), 412(Precondition Failed) 428(Precondition Required), or even 304 (Not Modified, ) but the later is a bit far fetched

Br

Got it. I think it’s somewhat common to use 409 to report an existing duplicate, so I think that would be a reasonable thing for the Case/Form forwarders to do. I can add that as a suggestion for the Open Source roadmap.

-Clayton

1 Like

Hello Clayton,

Do you think changing this line to https://github.com/dimagi/commcare-hq/blob/a85b0e7c3f87e96846613a6ea9481e168fbcbd9f/corehq/motech/requests.py#L265

if not (200 <= response.status_code < 300 or response.status_code = 409):

Will do the trick ? (it will be handled as succes but it is easy to change)

thanks in advance

best regards