Local proxies in Myanmar?

Users
1

Hi everyone,

Context

··· =======

We are implementing an MNCH project in rural Myanmar : community health
workers and midwives are provided a smartphone with a diagnosis helper /
patient registry / medical files application. CommCare is our framework,
mainly thanks to its case sharing feature.

This mHealth project is a part of a bigger initiative that aims to build
capacity and support these CHW. Our main activity consists of monthly
refresher trainings.

We operate in Dala, a peri-urban zone very close to Rangoon. We'll scale
the pilot project up to the Kayin (Karen) State at the end of the year,
a much more rural and politically sensitive region.

Connectivity issues

We are facing a connectivity problem. In the villages, connections are
unreliable at best and often unusable most of the time. My guess is the
network backbone doesn't support the increased traffic incurred by the
mobile phone boom of the last 2/3 years. You can actually connect to the
mobile network, but the backbone seems so congested it becomes unusable.
We are talking pings to 8.8.8.8 > 30000ms here (no typo).

However, during the monthly refresher training meetings, we can sync the
phones using the ADSL connexion of our Dala office. It is slow, painful
and unreliable, but at least we can update and sync the devices. To give
you an idea, last time it took me an entire day to update 18 phones…

Obviously the situation will be much worse in Kayin State. We do hope
that the state of network connexions will improve in Myanmar with the
arrival of two new operators (Ooredoo from Qatar and Telenor from
Norway), either by building new networks or by forcing the current
operator MPT to invest and upgrade its current systems. However for the
time being connecting to a CommCare server over the internet will prove
to be a little complicated.

Local servers

I was thinking about deploying our local installs of CommCare servers.
Each base would have a small, portable server coupled with a WIFI access
point (a laptop / small computer like a BeagleBone or Raspberry Pi and a
WIFI dongle configured as Infrastructure Access Point should do it).

During the monthly trainings, this server would be carried to the
training location. Users will directly connect through the LAN WIFI and
update / sync their devices. Back at the base, M&E staff could get their
reports directly from the server on the LAN. This portable server could
also be carried during the field visits that our monitors do on a
regular basis, hence syncing the devices more often.

With these portable CommCare servers, we wouldn't rely on dodgy Internet
connexions, everything would be done over LANs.

In fact, local proxies

Apart from the obvious extra cost of buying / installing / maintaining /
backuping these servers, the main issue will be to keep all these
CommCare instances in sync in term of forms / users management /
application building. The last thing I'd want would be to manually
manage a cluster of unrelated CommCare installations : users connecting
to a local, LAN server should have access to the same data as if they
were connecting to a unique central instance. The distributed nature of
this architecture should be hidden from the user (granted, minus sync
delays between servers).

With this configuration, local servers would initiate connections and
sync to a central server overnight when connexion are more reliable.
They would push their forms gathered form the field and pull application
/ users / configuration / forms updates.

Implementation

So I was wondering :

  1. Am I over-engineering this stuff ? Is a simpler, more straightforward
    workflow possible ?

  2. Did someone already implement such an infrastructure ? How did it go ?

  3. Does CommCare support such a feature already : local proxies ?

  4. As a first step, even if the users management was totally
    decentralized and not consolidated between servers, at least I'd like to
    have reports and applications updated to local proxies on a nightly
    basis. I see a cron job running a python script getting reports /
    applications through the API from a central server then pushing these to
    a local instance. Do you think it could work ?

My apologies for this long email…

Many thanks,

--
Charles Flèche
mHealth advisor
Télécoms Sans Frontières http://www.tsfi.org
Première Urgence - Aide Médicale Internationale http://www.pu-ami.org
+95 9 431 978 25
Skype: charles.fleche

2

Charles,

Before jumping into too many details, I have one question for you.

Have you tried configuring a test app against our server in india at "
india.commcarehq.org"? It's significantly closer geographically and may
have better connectivity.

We have some tools to help mitigate long-term offline connections, but we
don't currently have a micro-node setup for handling the process you're
describing currently. We can chat about some potential options once it's
clear whether the india server is a better fit.

-Clayton

··· On Mon, Aug 11, 2014 at 3:34 AM, Charles Flèche wrote:

Hi everyone,

Context

We are implementing an MNCH project in rural Myanmar : community health
workers and midwives are provided a smartphone with a diagnosis helper /
patient registry / medical files application. CommCare is our framework,
mainly thanks to its case sharing feature.

This mHealth project is a part of a bigger initiative that aims to build
capacity and support these CHW. Our main activity consists of monthly
refresher trainings.

We operate in Dala, a peri-urban zone very close to Rangoon. We'll scale
the pilot project up to the Kayin (Karen) State at the end of the year, a
much more rural and politically sensitive region.

Connectivity issues

We are facing a connectivity problem. In the villages, connections are
unreliable at best and often unusable most of the time. My guess is the
network backbone doesn't support the increased traffic incurred by the
mobile phone boom of the last 2/3 years. You can actually connect to the
mobile network, but the backbone seems so congested it becomes unusable. We
are talking pings to 8.8.8.8 > 30000ms here (no typo).

However, during the monthly refresher training meetings, we can sync the
phones using the ADSL connexion of our Dala office. It is slow, painful and
unreliable, but at least we can update and sync the devices. To give you an
idea, last time it took me an entire day to update 18 phones...

Obviously the situation will be much worse in Kayin State. We do hope that
the state of network connexions will improve in Myanmar with the arrival of
two new operators (Ooredoo from Qatar and Telenor from Norway), either by
building new networks or by forcing the current operator MPT to invest and
upgrade its current systems. However for the time being connecting to a
CommCare server over the internet will prove to be a little complicated.

Local servers

I was thinking about deploying our local installs of CommCare servers.
Each base would have a small, portable server coupled with a WIFI access
point (a laptop / small computer like a BeagleBone or Raspberry Pi and a
WIFI dongle configured as Infrastructure Access Point should do it).

During the monthly trainings, this server would be carried to the training
location. Users will directly connect through the LAN WIFI and update /
sync their devices. Back at the base, M&E staff could get their reports
directly from the server on the LAN. This portable server could also be
carried during the field visits that our monitors do on a regular basis,
hence syncing the devices more often.

With these portable CommCare servers, we wouldn't rely on dodgy Internet
connexions, everything would be done over LANs.

In fact, local proxies

Apart from the obvious extra cost of buying / installing / maintaining /
backuping these servers, the main issue will be to keep all these CommCare
instances in sync in term of forms / users management / application
building. The last thing I'd want would be to manually manage a cluster of
unrelated CommCare installations : users connecting to a local, LAN server
should have access to the same data as if they were connecting to a unique
central instance. The distributed nature of this architecture should be
hidden from the user (granted, minus sync delays between servers).

With this configuration, local servers would initiate connections and sync
to a central server overnight when connexion are more reliable. They would
push their forms gathered form the field and pull application / users /
configuration / forms updates.

Implementation

So I was wondering :

  1. Am I over-engineering this stuff ? Is a simpler, more straightforward
    workflow possible ?

  2. Did someone already implement such an infrastructure ? How did it go ?

  3. Does CommCare support such a feature already : local proxies ?

  4. As a first step, even if the users management was totally decentralized
    and not consolidated between servers, at least I'd like to have reports and
    applications updated to local proxies on a nightly basis. I see a cron job
    running a python script getting reports / applications through the API from
    a central server then pushing these to a local instance. Do you think it
    could work ?

My apologies for this long email...

Many thanks,

--
Charles Flèche
mHealth advisor
Télécoms Sans Frontières http://www.tsfi.org
Première Urgence - Aide Médicale Internationale http://www.pu-ami.org
+95 9 431 978 25
Skype: charles.fleche

--
You received this message because you are subscribed to the Google Groups
"commcare-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

3

Hi Charles,

What you're describing is an architecture that the project that was
predecessor to CommCare HQ was actually designed to solve. As a result a
lot of what you're looking for could be supported relatively cleanly by the
underlying technology. Right now the primary data store of CommCare is
CouchDB which is designed to handle bidirectional replication across a
number of separate nodes via its replication engine
http://guide.couchdb.org/draft/replication.html. If you do end up going
with the network of servers approach I'd recommend you look into using that
as your underlying sync mechanism. There will be some problems with it,
since not all CommCare data is stored in CouchDB anymore, but a lot of
things (including specifically reports) should just work. We had good
success with this architecture for a 40-clinic deployment in Zambia with
very bad network (where each clinic worked fully offline and then synced
with couchdb replication in the background).

All that said, this will be a bit flaky and the devil is in the details.
Your API based approach will make a lot of that cleaner and allow your
central server to be cloud HQ (we don't support database-level access so
replication wouldn't be an option) but at the expense of having to
explicitly do work for each category of data that you want to push over.

When you think you have a plan I'm happy to provide input via the
developers list.

Cory

··· On Mon, Aug 11, 2014 at 11:51 AM, Clayton Sims wrote:

Charles,

Before jumping into too many details, I have one question for you.

Have you tried configuring a test app against our server in india at "
india.commcarehq.org"? It's significantly closer geographically and may
have better connectivity.

We have some tools to help mitigate long-term offline connections, but we
don't currently have a micro-node setup for handling the process you're
describing currently. We can chat about some potential options once it's
clear whether the india server is a better fit.

-Clayton

On Mon, Aug 11, 2014 at 3:34 AM, Charles Flèche mhealth-myanmar@tsfi.org wrote:

Hi everyone,

Context

We are implementing an MNCH project in rural Myanmar : community health
workers and midwives are provided a smartphone with a diagnosis helper /
patient registry / medical files application. CommCare is our framework,
mainly thanks to its case sharing feature.

This mHealth project is a part of a bigger initiative that aims to build
capacity and support these CHW. Our main activity consists of monthly
refresher trainings.

We operate in Dala, a peri-urban zone very close to Rangoon. We'll scale
the pilot project up to the Kayin (Karen) State at the end of the year, a
much more rural and politically sensitive region.

Connectivity issues

We are facing a connectivity problem. In the villages, connections are
unreliable at best and often unusable most of the time. My guess is the
network backbone doesn't support the increased traffic incurred by the
mobile phone boom of the last 2/3 years. You can actually connect to the
mobile network, but the backbone seems so congested it becomes unusable. We
are talking pings to 8.8.8.8 > 30000ms here (no typo).

However, during the monthly refresher training meetings, we can sync the
phones using the ADSL connexion of our Dala office. It is slow, painful and
unreliable, but at least we can update and sync the devices. To give you an
idea, last time it took me an entire day to update 18 phones...

Obviously the situation will be much worse in Kayin State. We do hope
that the state of network connexions will improve in Myanmar with the
arrival of two new operators (Ooredoo from Qatar and Telenor from Norway),
either by building new networks or by forcing the current operator MPT to
invest and upgrade its current systems. However for the time being
connecting to a CommCare server over the internet will prove to be a little
complicated.

Local servers

I was thinking about deploying our local installs of CommCare servers.
Each base would have a small, portable server coupled with a WIFI access
point (a laptop / small computer like a BeagleBone or Raspberry Pi and a
WIFI dongle configured as Infrastructure Access Point should do it).

During the monthly trainings, this server would be carried to the
training location. Users will directly connect through the LAN WIFI and
update / sync their devices. Back at the base, M&E staff could get their
reports directly from the server on the LAN. This portable server could
also be carried during the field visits that our monitors do on a regular
basis, hence syncing the devices more often.

With these portable CommCare servers, we wouldn't rely on dodgy Internet
connexions, everything would be done over LANs.

In fact, local proxies

Apart from the obvious extra cost of buying / installing / maintaining /
backuping these servers, the main issue will be to keep all these CommCare
instances in sync in term of forms / users management / application
building. The last thing I'd want would be to manually manage a cluster of
unrelated CommCare installations : users connecting to a local, LAN server
should have access to the same data as if they were connecting to a unique
central instance. The distributed nature of this architecture should be
hidden from the user (granted, minus sync delays between servers).

With this configuration, local servers would initiate connections and
sync to a central server overnight when connexion are more reliable. They
would push their forms gathered form the field and pull application / users
/ configuration / forms updates.

Implementation

So I was wondering :

  1. Am I over-engineering this stuff ? Is a simpler, more straightforward
    workflow possible ?

  2. Did someone already implement such an infrastructure ? How did it go ?

  3. Does CommCare support such a feature already : local proxies ?

  4. As a first step, even if the users management was totally
    decentralized and not consolidated between servers, at least I'd like to
    have reports and applications updated to local proxies on a nightly basis.
    I see a cron job running a python script getting reports / applications
    through the API from a central server then pushing these to a local
    instance. Do you think it could work ?

My apologies for this long email...

Many thanks,

--
Charles Flèche
mHealth advisor
Télécoms Sans Frontières http://www.tsfi.org
Première Urgence - Aide Médicale Internationale http://www.pu-ami.org
+95 9 431 978 25
Skype: charles.fleche

--
You received this message because you are subscribed to the Google Groups
"commcare-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"commcare-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

Hi Clayton,

··· On 11/08/2014 22:21, Clayton Sims wrote: > Have you tried configuring a test app against our server in india at > "india.commcarehq.org "?

Many thanks for the suggestion. It seems like india.commcare.org domain
cannot be resolved at the moment :

$ nslookup india.commcare.org commcare.org
Server: commcare.org
Address: 198.1.112.104#53

** server can't find india.commcare.org: NXDOMAIN

I'll try using the Indian server when it is available again. However,
hitting a closer machine may not solve my connectivity issues as in some
area I just cannot connect at all or the network backbone seems so
congested that if you can actually connect to a mobile network BTS, you
can't do anything with it. Even SMS are extremely unreliable in these areas…

Thanks for the india.commcare.org tip anyway.

Charles

--
Charles Flèche
mHealth advisor
Télécoms Sans Frontières http://www.tsfi.org
Première Urgence - Aide Médicale Internationale http://www.pu-ami.org
+95 9 431 978 25
Skype: charles.fleche

5

Hi Charles,

Your nslookup uses india.commcare.org and it should be india.commcarehq.org.

Sincerely,
Craig

··· On Tuesday, August 12, 2014 7:52:31 AM UTC+5:45, Charles Flèche wrote: > > Hi Clayton, > > > On 11/08/2014 22:21, Clayton Sims wrote: > > Have you tried configuring a test app against our server in india at > > "india.commcarehq.org "? > > > Many thanks for the suggestion. It seems like india.commcare.org domain > cannot be resolved at the moment : > > $ nslookup india.commcare.org commcare.org > Server: commcare.org > Address: 198.1.112.104#53 > > ** server can't find india.commcare.org: NXDOMAIN > > > I'll try using the Indian server when it is available again. However, > hitting a closer machine may not solve my connectivity issues as in some > area I just cannot connect at all or the network backbone seems so > congested that if you can actually connect to a mobile network BTS, you > can't do anything with it. Even SMS are extremely unreliable in these > areas… > > Thanks for the india.commcare.org tip anyway. > > Charles > > > > > -- > Charles Flèche > mHealth advisor > Télécoms Sans Frontières http://www.tsfi.org > Première Urgence - Aide Médicale Internationale http://www.pu-ami.org > +95 9 431 978 25 > Skype: charles.fleche >
6

Hi Cory,

Many thanks for your swift answer.

There will be some problems with it, since not all CommCare
data is stored in CouchDB anymore

Is there a documentation (I'm thinking code comments) that states what
is stored in CouchDB and what is in PostgreSQL ? For my geeky curiosity,
what is the rational behind using two types of DB ?

but a lot of things (including specifically reports) should
just work. We had good success with this architecture for a 40-clinic
deployment in Zambia with very bad network (where each clinic worked
fully offline and then synced with couchdb replication in the background).

Seems like a challenging deployment, congratulation for that ! How did
you do the sync if the clinics were fully offline : shuffling USB keys
around ?

All that said, this will be a bit flaky and the devil is in the details.
Your API based approach will make a lot of that cleaner and allow your
central server to be cloud HQ (we don't support database-level access so
replication wouldn't be an option) but at the expense of having to
explicitly do work for each category of data that you want to push over.

Yeah, I'm not totally confident with brute-force DB storage replication,
especially in a mixed PostgreSQL / CouchDB environment, but as I have no
experience with such procedures so it is probably just me being too
defensive.

When you think you have a plan I'm happy to provide input via the
developers list.

My contract here in Myanmar ends pretty soon and it is still unclear if
I'll stay longer in order to implement this distributed architecture or
if it will be my successor's duty. However I intend to leave with at
least a few ideas on how to solve our current issues.

Context:

  • All servers are LAN hosted on LANs, behind our firewalls
  • Master server (M) is hosted in our main office, with a descent (well,
    Myanmar-descent…) internet connection and is always online.
  • Proxy servers (P) are hosted in remote bases, with an unreliable and
    slow internet connection (3G), are mostly offline but can be connected
    overnight for data sync

Note : it is unclear if our ISP allows static IPs for our main office
connection. If not, the main office server will be another P, while M
would be hosted in the cloud HQ. For simplicity, let's consider M to be
hosted behind our firewall for the time being.

In a ideal world, every P is the exact copy of M. However, a minimal
system allows an application to be built on M, then pulled from P on a
regular basis. P pushes its forms to M overnight.

User management is decentralized : M does not know anything about the
users. In fact, M has just a few test mobile users configured. It
doesn't know anything about cases too. It just stores forms from every
bases. We assume that formid and caseid are random enough not to
collide, coming from different P.

  1. Does the API allow to pull / push applications ?

  2. Does the API allow to push forms to another server ?

  3. What would be the best strategy to adopt for pushing new forms from P
    to M ? Keeping a list on P of already pushed forms (in the DB itself
    maybe) ? Querying for existing formids on M before pushing the ones not
    already on M ?

  4. Would reporting break if forms state unknown caseids (as M would
    known nothing about the mobiles users / cases on P) ?

Many thanks everyone,

··· On 12/08/2014 02:29, Cory Zue wrote:

--
Charles Flèche
mHealth advisor
Télécoms Sans Frontières http://www.tsfi.org
Première Urgence - Aide Médicale Internationale http://www.pu-ami.org
+95 9 431 978 25
Skype: charles.fleche

7

Your nslookup uses india.commcare.org and it should be india.commcarehq.org.

Whoopsie… Thanks for that Craig.

··· On 12/08/2014 11:21, Craig A. wrote:

--
Charles Flèche
mHealth advisor
Télécoms Sans Frontières http://www.tsfi.org
Première Urgence - Aide Médicale Internationale http://www.pu-ami.org
+95 9 431 978 25
Skype: charles.fleche

8

Hi Charles,

You will run into trouble trying to get a fixed ipaddress here in Myanmar.
Its very expensive and most of the companies that say they offer fixed
ipaddresses dont really, theyre are just fixed ipaddresses on the isp's
subnet.

Brian

··· On Tuesday, August 12, 2014 9:53:28 AM UTC+6:30, Charles Flèche wrote: > > Hi Cory, > > Many thanks for your swift answer. > > > > On 12/08/2014 02:29, Cory Zue wrote: > > There will be some problems with it, since not all CommCare > > data is stored in CouchDB anymore > > > Is there a documentation (I'm thinking code comments) that states what > is stored in CouchDB and what is in PostgreSQL ? For my geeky curiosity, > what is the rational behind using two types of DB ? > > > > > but a lot of things (including specifically reports) should > > just work. We had good success with this architecture for a 40-clinic > > deployment in Zambia with very bad network (where each clinic worked > > fully offline and then synced with couchdb replication in the > background). > > > Seems like a challenging deployment, congratulation for that ! How did > you do the sync if the clinics were fully offline : shuffling USB keys > around ? > > > > > All that said, this will be a bit flaky and the devil is in the details. > > Your API based approach will make a lot of that cleaner and allow your > > central server to be cloud HQ (we don't support database-level access so > > replication wouldn't be an option) but at the expense of having to > > explicitly do work for each category of data that you want to push over. > > > Yeah, I'm not totally confident with brute-force DB storage replication, > especially in a mixed PostgreSQL / CouchDB environment, but as I have no > experience with such procedures so it is probably just me being too > defensive. > > > > > When you think you have a plan I'm happy to provide input via the > > developers list. > > > My contract here in Myanmar ends pretty soon and it is still unclear if > I'll stay longer in order to implement this distributed architecture or > if it will be my successor's duty. However I intend to leave with at > least a few ideas on how to solve our current issues. > > Context: > - All servers are LAN hosted on LANs, behind our firewalls > - Master server (M) is hosted in our main office, with a descent (well, > Myanmar-descent…) internet connection and is always online. > - Proxy servers (P) are hosted in remote bases, with an unreliable and > slow internet connection (3G), are mostly offline but can be connected > overnight for data sync > > Note : it is unclear if our ISP allows static IPs for our main office > connection. If not, the main office server will be another P, while M > would be hosted in the cloud HQ. For simplicity, let's consider M to be > hosted behind our firewall for the time being. > > In a ideal world, every P is the exact copy of M. However, a minimal > system allows an application to be built on M, then pulled from P on a > regular basis. P pushes its forms to M overnight. > > User management is decentralized : M does not know anything about the > users. In fact, M has just a few test mobile users configured. It > doesn't know anything about cases too. It just stores forms from every > bases. We assume that formid and caseid are random enough not to > collide, coming from different P. > > 1. Does the API allow to pull / push applications ? > > 2. Does the API allow to push forms to another server ? > > 3. What would be the best strategy to adopt for pushing new forms from P > to M ? Keeping a list on P of already pushed forms (in the DB itself > maybe) ? Querying for existing formids on M before pushing the ones not > already on M ? > > 4. Would reporting break if forms state unknown caseids (as M would > known nothing about the mobiles users / cases on P) ? > > > Many thanks everyone, > > -- > Charles Flèche > mHealth advisor > Télécoms Sans Frontières http://www.tsfi.org > Première Urgence - Aide Médicale Internationale http://www.pu-ami.org > +95 9 431 978 25 > Skype: charles.fleche >
9

Hum, good to know, thanks for that Brian. What about IPv6 ? Would
CommCare work on an IPv6 stack ? Would ISPs provide an IPv6 address block ?

So a Master cloud server may be the best option here.

··· On 12/08/2014 10:31, bmcdonald@proximitydesigns.org wrote: > You will run into trouble trying to get a fixed ipaddress here in > Myanmar.

--
Charles Flèche
mHealth advisor
Télécoms Sans Frontières http://www.tsfi.org
Première Urgence - Aide Médicale Internationale http://www.pu-ami.org
+95 9 431 978 25
Skype: charles.fleche

10

Hey Charles,

Very quick answers (dropping cc-users via bcc).

There will be some problems with it, since not all CommCare

data is stored in CouchDB anymore

Is there a documentation (I'm thinking code comments) that states what is
stored in CouchDB and what is in PostgreSQL ?

Not formal documentation. You can look for subclasses of a django Model as
opposed to a couchdbkit Document.

For my geeky curiosity, what is the rational behind using two types of DB ?

We chose couchdb as our primary data store and then ran into some problems
with it (mostly around performance and reporting) so started moving some
things to SQL.

but a lot of things (including specifically reports) should

just work. We had good success with this architecture for a 40-clinic
deployment in Zambia with very bad network (where each clinic worked
fully offline and then synced with couchdb replication in the background).

Seems like a challenging deployment, congratulation for that ! How did you
do the sync if the clinics were fully offline : shuffling USB keys around ?

Sorry, they functioned fully offline via a LAN, but had netsticks that
could eventually sync via the cell network.

All that said, this will be a bit flaky and the devil is in the details.

Your API based approach will make a lot of that cleaner and allow your
central server to be cloud HQ (we don't support database-level access so
replication wouldn't be an option) but at the expense of having to
explicitly do work for each category of data that you want to push over.

Yeah, I'm not totally confident with brute-force DB storage replication,
especially in a mixed PostgreSQL / CouchDB environment, but as I have no
experience with such procedures so it is probably just me being too
defensive.

When you think you have a plan I'm happy to provide input via the

developers list.

My contract here in Myanmar ends pretty soon and it is still unclear if
I'll stay longer in order to implement this distributed architecture or if
it will be my successor's duty. However I intend to leave with at least a
few ideas on how to solve our current issues.

Context:

  • All servers are LAN hosted on LANs, behind our firewalls
  • Master server (M) is hosted in our main office, with a descent (well,
    Myanmar-descent…) internet connection and is always online.
  • Proxy servers (P) are hosted in remote bases, with an unreliable and
    slow internet connection (3G), are mostly offline but can be connected
    overnight for data sync

Note : it is unclear if our ISP allows static IPs for our main office
connection. If not, the main office server will be another P, while M would
be hosted in the cloud HQ. For simplicity, let's consider M to be hosted
behind our firewall for the time being.

In a ideal world, every P is the exact copy of M. However, a minimal
system allows an application to be built on M, then pulled from P on a
regular basis. P pushes its forms to M overnight.

User management is decentralized : M does not know anything about the
users. In fact, M has just a few test mobile users configured. It doesn't
know anything about cases too. It just stores forms from every bases. We
assume that formid and caseid are random enough not to collide, coming from
different P.

(this should be a fine assumption)

  1. Does the API allow to pull / push applications ?

(Assume you are referring to the HQ APIs in this section. The couch
replication would support all of these things out of the box.)

We currently only support pull, but adding support for push is something
that could be done relatively easily.

  1. Does the API allow to push forms to another server ?

Yes. You would have to use the submission API.

  1. What would be the best strategy to adopt for pushing new forms from P
    to M ? Keeping a list on P of already pushed forms (in the DB itself maybe)
    ? Querying for existing formids on M before pushing the ones not already on
    M ?

Simplest would likely be keeping a list of whether the form had been
pushed, which only gets set once you get a 2XX response from the submission
API. You could also try using the built in form forwarding (which basically
does that), although the backoff it has might be too aggressive for your
setup.

  1. Would reporting break if forms state unknown caseids (as M would known
    nothing about the mobiles users / cases on P) ?

If you are able to forward the forms in order you can rely on case
processing to recreate the cases on M, so this theoretically could work
(you would also need to sync users to have the reports work).

API docs are here in case you hadn't seen them:
https://confluence.dimagi.com/display/commcarepublic/CommCare+HQ+APIs

Cory

··· On 12/08/2014 02:29, Cory Zue wrote:

Many thanks everyone,

--
Charles Flèche
mHealth advisor
Télécoms Sans Frontières http://www.tsfi.org
Première Urgence - Aide Médicale Internationale http://www.pu-ami.org
+95 9 431 978 25
Skype: charles.fleche

--
You received this message because you are subscribed to the Google Groups
"commcare-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

11

Many thanks for your clear answers Cory. I'll add them to the
documentation being written for phase 2 of the project.

··· -- Charles Flèche mHealth advisor Télécoms Sans Frontières http://www.tsfi.org Première Urgence - Aide Médicale Internationale http://www.pu-ami.org +95 9 431 978 25 Skype: charles.fleche