OTA and PostURL via SSL

Hi all,

I am trying to configure our CommCare app to send and receive data to
CommCare HQ over SSL. I have manually exported the SSL cert from
https://data.commcarehq.org and installed it on the phone and I have changed
the PostURL/PostTestURL and ota-restore-url settings in my profile.xml from
http to https. However, I still receive the typical “Certificate failed
verification” error when the app attempts to communicate with CommCareHQ.
What is required to get the communication to run over SSL?

Thanks,
Stephen

Hi Stephen,

Which phone model are you using? Can you hit the https URL in the
phone’s browser without it complaining about the root certificate
verification issues? Our certificate is from thawte, and was chosen
so as have its root certification be as widely shipped on stock phones
as possible, so as to work out of the box. However, it still could be
a possibility that our “lowest common denominator” decision may have
missed a few models…

Dan

··· On Jan 4, 7:20 am, Stephen Lorenz wrote: > Hi all, > > I am trying to configure our CommCare app to send and receive data to > CommCare HQ over SSL. I have manually exported the SSL cert fromhttps://data.commcarehq.organd installed it on the phone and I have changed > the PostURL/PostTestURL and ota-restore-url settings in my profile.xml from > http to https. However, I still receive the typical "Certificate failed > verification" error when the app attempts to communicate with CommCareHQ. > What is required to get the communication to run over SSL? > > Thanks, > Stephen

Hi Dan,

Of course, that makes sense. This project doesn’t actually start collecting
data until February so we don’t yet have a phone with a data plan. So all
testing which involves communicating with CommCareHQ is being done through
the emulator–which must be using a keystore which is missing a thawte root
cert. Within the next two weeks we will be able to test the communication
piece on real phones (Nokia 2730) and this will be one of the first things I
check. In the meantime I will work on trying to import the thawte root cert
in my emulator.

Best,
Stephen

··· On Tue, Jan 4, 2011 at 10:52 PM, Dan Myung wrote:

Hi Stephen,

Which phone model are you using? Can you hit the https URL in the
phone’s browser without it complaining about the root certificate
verification issues? Our certificate is from thawte, and was chosen
so as have its root certification be as widely shipped on stock phones
as possible, so as to work out of the box. However, it still could be
a possibility that our “lowest common denominator” decision may have
missed a few models…

Dan

On Jan 4, 7:20 am, Stephen Lorenz stephen.lor...@gmail.com wrote:

Hi all,

I am trying to configure our CommCare app to send and receive data to
CommCare HQ over SSL. I have manually exported the SSL cert
fromhttps://data.commcarehq.organd installed it on the phone and I have
changed
the PostURL/PostTestURL and ota-restore-url settings in my profile.xml
from
http to https. However, I still receive the typical “Certificate failed
verification” error when the app attempts to communicate with CommCareHQ.
What is required to get the communication to run over SSL?

Thanks,
Stephen

Stephen,

The Nokia Emulators are actually pretty bad about including appropriate Root
CA’s. However, my local test with https is also behaving strangely on a
physical 2700, which is odd. I’ll look into this and let you
know whether there’s anything that needs to be resolved for https to work
properly.

-Clayton

··· On Wed, Jan 5, 2011 at 7:04 AM, Stephen Lorenz wrote:

Hi Dan,

Of course, that makes sense. This project doesn’t actually start
collecting data until February so we don’t yet have a phone with a data
plan. So all testing which involves communicating with CommCareHQ is being
done through the emulator–which must be using a keystore which is missing a
thawte root cert. Within the next two weeks we will be able to test the
communication piece on real phones (Nokia 2730) and this will be one of the
first things I check. In the meantime I will work on trying to import the
thawte root cert in my emulator.

Best,
Stephen

On Tue, Jan 4, 2011 at 10:52 PM, Dan Myung dmyung@dimagi.com wrote:

Hi Stephen,

Which phone model are you using? Can you hit the https URL in the
phone’s browser without it complaining about the root certificate
verification issues? Our certificate is from thawte, and was chosen
so as have its root certification be as widely shipped on stock phones
as possible, so as to work out of the box. However, it still could be
a possibility that our “lowest common denominator” decision may have
missed a few models…

Dan

On Jan 4, 7:20 am, Stephen Lorenz stephen.lor...@gmail.com wrote:

Hi all,

I am trying to configure our CommCare app to send and receive data to
CommCare HQ over SSL. I have manually exported the SSL cert
fromhttps://data.commcarehq.organd installed it on the phone and I have
changed
the PostURL/PostTestURL and ota-restore-url settings in my profile.xml
from
http to https. However, I still receive the typical “Certificate failed
verification” error when the app attempts to communicate with
CommCareHQ.
What is required to get the communication to run over SSL?

Thanks,
Stephen

Hello all,

I have done a bit more research into our SSL/HTTPS woes and I believe I have
uncovered two issues, one involving GETting from the
https://data.commcarehq.org/ota_restore URL and the other involving POSTing
to https://data.commcarehq.org/receiver/submit/.

The ota_restore issue appears to involve basic http authentication; it works
when accessed via http (port 80) but not via https (port 443). You can try
this in your browser. In both cases (http and https) you are prompted to
login. With http://data.commcarehq.org/ota_restore your username/password
are accepted and you receive the ota_restore XML document. With
https://data.commcarehq.org/ota_restore your username/password are never
accepted; you keep on being prompted to login.

The other issue involves POSTing to the
https://data.commcarehq.org/receiver/submit/ URL. There is no
authentication required to POST to a domain, so this is not the issue. I
examined the phone logs and then wrote a simple curl script to emulate an
data.commcarehq.org/receiver/submit/ POST. When POSTing to the
non-https URL (http://data.commcarehq.org/receiver/submit/) I
receive the following successful response from the CommCareHQ server:

<?xml version='1.0' encoding='UTF-8'?> 1 1 Thanks! 2000 Submission 2011-01-24 09:52:12.732186 84861 200 1 8708cdb6-27c9-11e0-ab9a-00505697291a

When I change the URL to POST to the https URL (
https://data.commcarehq.org/receiver/submit/) I receive the
following failed response:

<?xml version='1.0' encoding='UTF-8'?> 1 2000 Submission 2011-01-24 06:10:45.646407 26 200 97409250-27aa-11e0-89d5-00505697291a

The only difference between these two POSTs was changing http to https. If
I look at my submissions in CommCare HQ, I see the http POST but not the
https POST.

Assuming what I am trying to do is possible, am I missing something or is
there an underlying problem?

Best,
Stephen

··· On Wed, Jan 5, 2011 at 2:36 PM, Clayton Sims wrote:

Stephen,

The Nokia Emulators are actually pretty bad about including appropriate
Root CA’s. However, my local test with https is also behaving strangely on a
physical 2700, which is odd. I’ll look into this and let you
know whether there’s anything that needs to be resolved for https to work
properly.

-Clayton

On Wed, Jan 5, 2011 at 7:04 AM, Stephen Lorenz stephen.lorenz@gmail.comwrote:

Hi Dan,

Of course, that makes sense. This project doesn’t actually start
collecting data until February so we don’t yet have a phone with a data
plan. So all testing which involves communicating with CommCareHQ is being
done through the emulator–which must be using a keystore which is missing a
thawte root cert. Within the next two weeks we will be able to test the
communication piece on real phones (Nokia 2730) and this will be one of the
first things I check. In the meantime I will work on trying to import the
thawte root cert in my emulator.

Best,
Stephen

On Tue, Jan 4, 2011 at 10:52 PM, Dan Myung dmyung@dimagi.com wrote:

Hi Stephen,

Which phone model are you using? Can you hit the https URL in the
phone’s browser without it complaining about the root certificate
verification issues? Our certificate is from thawte, and was chosen
so as have its root certification be as widely shipped on stock phones
as possible, so as to work out of the box. However, it still could be
a possibility that our “lowest common denominator” decision may have
missed a few models…

Dan

On Jan 4, 7:20 am, Stephen Lorenz stephen.lor...@gmail.com wrote:

Hi all,

I am trying to configure our CommCare app to send and receive data to
CommCare HQ over SSL. I have manually exported the SSL cert
fromhttps://data.commcarehq.organd installed it on the phone and I have
changed
the PostURL/PostTestURL and ota-restore-url settings in my profile.xml
from
http to https. However, I still receive the typical “Certificate
failed
verification” error when the app attempts to communicate with
CommCareHQ.
What is required to get the communication to run over SSL?

Thanks,
Stephen

Hi Stephen,

Thanks for the detailed analysis of the HTTP vs. HTTPS issues for OTA
restore. We’ve observed similar behavior on other instances of HQ code
we’ve run for HTTPS. The short answer is that we can reproduce it, and it’s
a server side problem on our end. The combination of apache proxying
interacting with the django instance is the root cause of the problem.
We’re going to look into the issue and try to tweak the configuration to
try and get the behavior consistent for both 443 and 80.

Dan

··· On Mon, Jan 24, 2011 at 10:11 AM, Stephen Lorenz wrote:

Hello all,

I have done a bit more research into our SSL/HTTPS woes and I believe I
have uncovered two issues, one involving GETting from the
https://data.commcarehq.org/ota_restore URL and the other involving
POSTing to https://data.commcarehq.org/receiver/submit/.

The ota_restore issue appears to involve basic http authentication; it
works when accessed via http (port 80) but not via https (port 443). You
can try this in your browser. In both cases (http and https) you are
prompted to login. With http://data.commcarehq.org/ota_restore your
username/password are accepted and you receive the ota_restore XML document.
With https://data.commcarehq.org/ota_restore your username/password are
never accepted; you keep on being prompted to login.

The other issue involves POSTing to the
https://data.commcarehq.org/receiver/submit/ URL. There is
no authentication required to POST to a domain, so this is not the issue. I
examined the phone logs and then wrote a simple curl script to emulate an
data.commcarehq.org/receiver/submit/ POST. When POSTing to
the non-https URL (http://data.commcarehq.org/receiver/submit/)
I receive the following successful response from the CommCareHQ server:

<?xml version='1.0' encoding='UTF-8'?> 1 1 Thanks! 2000 Submission 2011-01-24 09:52:12.732186 84861 200 1 8708cdb6-27c9-11e0-ab9a-00505697291a

When I change the URL to POST to the https URL (
https://data.commcarehq.org/receiver/submit/) I receive the
following failed response:

<?xml version='1.0' encoding='UTF-8'?> 1 2000 Submission 2011-01-24 06:10:45.646407 26 200 97409250-27aa-11e0-89d5-00505697291a

The only difference between these two POSTs was changing http to https. If
I look at my submissions in CommCare HQ, I see the http POST but not the
https POST.

Assuming what I am trying to do is possible, am I missing something or is
there an underlying problem?

Best,
Stephen

On Wed, Jan 5, 2011 at 2:36 PM, Clayton Sims csims@dimagi.com wrote:

Stephen,

The Nokia Emulators are actually pretty bad about including appropriate
Root CA’s. However, my local test with https is also behaving strangely on a
physical 2700, which is odd. I’ll look into this and let you
know whether there’s anything that needs to be resolved for https to work
properly.

-Clayton

On Wed, Jan 5, 2011 at 7:04 AM, Stephen Lorenz stephen.lorenz@gmail.comwrote:

Hi Dan,

Of course, that makes sense. This project doesn’t actually start
collecting data until February so we don’t yet have a phone with a data
plan. So all testing which involves communicating with CommCareHQ is being
done through the emulator–which must be using a keystore which is missing a
thawte root cert. Within the next two weeks we will be able to test the
communication piece on real phones (Nokia 2730) and this will be one of the
first things I check. In the meantime I will work on trying to import the
thawte root cert in my emulator.

Best,
Stephen

On Tue, Jan 4, 2011 at 10:52 PM, Dan Myung dmyung@dimagi.com wrote:

Hi Stephen,

Which phone model are you using? Can you hit the https URL in the
phone’s browser without it complaining about the root certificate
verification issues? Our certificate is from thawte, and was chosen
so as have its root certification be as widely shipped on stock phones
as possible, so as to work out of the box. However, it still could be
a possibility that our “lowest common denominator” decision may have
missed a few models…

Dan

On Jan 4, 7:20 am, Stephen Lorenz stephen.lor...@gmail.com wrote:

Hi all,

I am trying to configure our CommCare app to send and receive data to
CommCare HQ over SSL. I have manually exported the SSL cert
fromhttps://data.commcarehq.organd installed it on the phone and I have
changed
the PostURL/PostTestURL and ota-restore-url settings in my profile.xml
from
http to https. However, I still receive the typical “Certificate
failed
verification” error when the app attempts to communicate with
CommCareHQ.
What is required to get the communication to run over SSL?

Thanks,
Stephen

Stephen,

Thanks for the in-depth analysis into the situation! We’ll be sure to let
you know ASAP when this is resolved.

-Clayton

··· On Mon, Jan 24, 2011 at 11:22 AM, Daniel Myung wrote:

Hi Stephen,

Thanks for the detailed analysis of the HTTP vs. HTTPS issues for OTA
restore. We’ve observed similar behavior on other instances of HQ code
we’ve run for HTTPS. The short answer is that we can reproduce it, and it’s
a server side problem on our end. The combination of apache proxying
interacting with the django instance is the root cause of the problem.
We’re going to look into the issue and try to tweak the configuration to
try and get the behavior consistent for both 443 and 80.

Dan

On Mon, Jan 24, 2011 at 10:11 AM, Stephen Lorenz <stephen.lorenz@gmail.com wrote:

Hello all,

I have done a bit more research into our SSL/HTTPS woes and I believe I
have uncovered two issues, one involving GETting from the
https://data.commcarehq.org/ota_restore URL and the other involving
POSTing to https://data.commcarehq.org/receiver/submit/.

The ota_restore issue appears to involve basic http authentication; it
works when accessed via http (port 80) but not via https (port 443). You
can try this in your browser. In both cases (http and https) you are
prompted to login. With http://data.commcarehq.org/ota_restore your
username/password are accepted and you receive the ota_restore XML document.
With https://data.commcarehq.org/ota_restore your username/password are
never accepted; you keep on being prompted to login.

The other issue involves POSTing to the
https://data.commcarehq.org/receiver/submit/ URL. There is
no authentication required to POST to a domain, so this is not the issue. I
examined the phone logs and then wrote a simple curl script to emulate an
data.commcarehq.org/receiver/submit/ POST. When POSTing to
the non-https URL (http://data.commcarehq.org/receiver/submit/)
I receive the following successful response from the CommCareHQ server:

<?xml version='1.0' encoding='UTF-8'?> 1 1 Thanks! 2000 Submission 2011-01-24 09:52:12.732186 84861 200 1 8708cdb6-27c9-11e0-ab9a-00505697291a

When I change the URL to POST to the https URL (
https://data.commcarehq.org/receiver/submit/) I receive the
following failed response:

<?xml version='1.0' encoding='UTF-8'?> 1 2000 Submission 2011-01-24 06:10:45.646407 26 200 97409250-27aa-11e0-89d5-00505697291a

The only difference between these two POSTs was changing http to https.
If I look at my submissions in CommCare HQ, I see the http POST but not the
https POST.

Assuming what I am trying to do is possible, am I missing something or is
there an underlying problem?

Best,
Stephen

On Wed, Jan 5, 2011 at 2:36 PM, Clayton Sims csims@dimagi.com wrote:

Stephen,

The Nokia Emulators are actually pretty bad about including appropriate
Root CA’s. However, my local test with https is also behaving strangely on a
physical 2700, which is odd. I’ll look into this and let you
know whether there’s anything that needs to be resolved for https to work
properly.

-Clayton

On Wed, Jan 5, 2011 at 7:04 AM, Stephen Lorenz <stephen.lorenz@gmail.com wrote:

Hi Dan,

Of course, that makes sense. This project doesn’t actually start
collecting data until February so we don’t yet have a phone with a data
plan. So all testing which involves communicating with CommCareHQ is being
done through the emulator–which must be using a keystore which is missing a
thawte root cert. Within the next two weeks we will be able to test the
communication piece on real phones (Nokia 2730) and this will be one of the
first things I check. In the meantime I will work on trying to import the
thawte root cert in my emulator.

Best,
Stephen

On Tue, Jan 4, 2011 at 10:52 PM, Dan Myung dmyung@dimagi.com wrote:

Hi Stephen,

Which phone model are you using? Can you hit the https URL in the
phone’s browser without it complaining about the root certificate
verification issues? Our certificate is from thawte, and was chosen
so as have its root certification be as widely shipped on stock phones
as possible, so as to work out of the box. However, it still could be
a possibility that our “lowest common denominator” decision may have
missed a few models…

Dan

On Jan 4, 7:20 am, Stephen Lorenz stephen.lor...@gmail.com wrote:

Hi all,

I am trying to configure our CommCare app to send and receive data to
CommCare HQ over SSL. I have manually exported the SSL cert
fromhttps://data.commcarehq.organd installed it on the phone and I have
changed
the PostURL/PostTestURL and ota-restore-url settings in my
profile.xml from
http to https. However, I still receive the typical “Certificate
failed
verification” error when the app attempts to communicate with
CommCareHQ.
What is required to get the communication to run over SSL?

Thanks,
Stephen

Hi Dan and Clayton,

Thanks for the updates!

Best,
Stephen

··· On Mon, Jan 24, 2011 at 12:34 PM, Clayton Sims wrote:

Stephen,

Thanks for the in-depth analysis into the situation! We’ll be sure to let
you know ASAP when this is resolved.

-Clayton

On Mon, Jan 24, 2011 at 11:22 AM, Daniel Myung dmyung@dimagi.com wrote:

Hi Stephen,

Thanks for the detailed analysis of the HTTP vs. HTTPS issues for OTA
restore. We’ve observed similar behavior on other instances of HQ code
we’ve run for HTTPS. The short answer is that we can reproduce it, and it’s
a server side problem on our end. The combination of apache proxying
interacting with the django instance is the root cause of the problem.
We’re going to look into the issue and try to tweak the configuration to
try and get the behavior consistent for both 443 and 80.

Dan

On Mon, Jan 24, 2011 at 10:11 AM, Stephen Lorenz < stephen.lorenz@gmail.com> wrote:

Hello all,

I have done a bit more research into our SSL/HTTPS woes and I believe I
have uncovered two issues, one involving GETting from the
https://data.commcarehq.org/ota_restore URL and the other involving
POSTing to https://data.commcarehq.org/receiver/submit/.

The ota_restore issue appears to involve basic http authentication; it
works when accessed via http (port 80) but not via https (port 443). You
can try this in your browser. In both cases (http and https) you are
prompted to login. With http://data.commcarehq.org/ota_restore your
username/password are accepted and you receive the ota_restore XML document.
With https://data.commcarehq.org/ota_restore your username/password are
never accepted; you keep on being prompted to login.

The other issue involves POSTing to the
https://data.commcarehq.org/receiver/submit/ URL. There is
no authentication required to POST to a domain, so this is not the issue. I
examined the phone logs and then wrote a simple curl script to emulate an
data.commcarehq.org/receiver/submit/ POST. When POSTing to
the non-https URL (http://data.commcarehq.org/receiver/submit/)
I receive the following successful response from the CommCareHQ server:

<?xml version='1.0' encoding='UTF-8'?> 1 1 Thanks! 2000 Submission 2011-01-24 09:52:12.732186 84861 200 1 8708cdb6-27c9-11e0-ab9a-00505697291a

When I change the URL to POST to the https URL (
https://data.commcarehq.org/receiver/submit/) I receive the
following failed response:

<?xml version='1.0' encoding='UTF-8'?> 1 2000 Submission 2011-01-24 06:10:45.646407 26 200 97409250-27aa-11e0-89d5-00505697291a

The only difference between these two POSTs was changing http to https.
If I look at my submissions in CommCare HQ, I see the http POST but not the
https POST.

Assuming what I am trying to do is possible, am I missing something or is
there an underlying problem?

Best,
Stephen

On Wed, Jan 5, 2011 at 2:36 PM, Clayton Sims csims@dimagi.com wrote:

Stephen,

The Nokia Emulators are actually pretty bad about including appropriate
Root CA’s. However, my local test with https is also behaving strangely on a
physical 2700, which is odd. I’ll look into this and let you
know whether there’s anything that needs to be resolved for https to work
properly.

-Clayton

On Wed, Jan 5, 2011 at 7:04 AM, Stephen Lorenz < stephen.lorenz@gmail.com> wrote:

Hi Dan,

Of course, that makes sense. This project doesn’t actually start
collecting data until February so we don’t yet have a phone with a data
plan. So all testing which involves communicating with CommCareHQ is being
done through the emulator–which must be using a keystore which is missing a
thawte root cert. Within the next two weeks we will be able to test the
communication piece on real phones (Nokia 2730) and this will be one of the
first things I check. In the meantime I will work on trying to import the
thawte root cert in my emulator.

Best,
Stephen

On Tue, Jan 4, 2011 at 10:52 PM, Dan Myung dmyung@dimagi.com wrote:

Hi Stephen,

Which phone model are you using? Can you hit the https URL in the
phone’s browser without it complaining about the root certificate
verification issues? Our certificate is from thawte, and was chosen
so as have its root certification be as widely shipped on stock phones
as possible, so as to work out of the box. However, it still could be
a possibility that our “lowest common denominator” decision may have
missed a few models…

Dan

On Jan 4, 7:20 am, Stephen Lorenz stephen.lor...@gmail.com wrote:

Hi all,

I am trying to configure our CommCare app to send and receive data
to
CommCare HQ over SSL. I have manually exported the SSL cert
fromhttps://data.commcarehq.organd installed it on the phone and I have
changed
the PostURL/PostTestURL and ota-restore-url settings in my
profile.xml from
http to https. However, I still receive the typical “Certificate
failed
verification” error when the app attempts to communicate with
CommCareHQ.
What is required to get the communication to run over SSL?

Thanks,
Stephen

Hi Stephen,

Just an update on our end. I’ve tested HTTPS for the OTA restore on our
newer codebase of the CommCareHQ code. Good news is that the https issue is
no longer an issue.

The bad news is that it seems to relegate the issue to the older instance of
CommCareHQ running using a different server setup. It’s been an ongoing
issue centered around the combination of the apache-mod-wsgi setup we have
running on the data.commcarehq.org instance, and/or quirks with the older
version of the django_digest authentication scheme. Current code now uses a
more streamlined server setup, hence the issue being resolved.

Would you be ok to wait until we release the updated version of commcarehq
with these fixes? The timeline for release is early February. Sorry for
the inconvenience. We’ll let you know when there are public URLs to test
against.

Dan

··· On Mon, Jan 24, 2011 at 12:52 PM, Stephen Lorenz wrote:

Hi Dan and Clayton,

Thanks for the updates!

Best,
Stephen

On Mon, Jan 24, 2011 at 12:34 PM, Clayton Sims csims@dimagi.com wrote:

Stephen,

Thanks for the in-depth analysis into the situation! We’ll be sure to let
you know ASAP when this is resolved.

-Clayton

On Mon, Jan 24, 2011 at 11:22 AM, Daniel Myung dmyung@dimagi.com wrote:

Hi Stephen,

Thanks for the detailed analysis of the HTTP vs. HTTPS issues for OTA
restore. We’ve observed similar behavior on other instances of HQ code
we’ve run for HTTPS. The short answer is that we can reproduce it, and it’s
a server side problem on our end. The combination of apache proxying
interacting with the django instance is the root cause of the problem.
We’re going to look into the issue and try to tweak the configuration to
try and get the behavior consistent for both 443 and 80.

Dan

On Mon, Jan 24, 2011 at 10:11 AM, Stephen Lorenz < stephen.lorenz@gmail.com> wrote:

Hello all,

I have done a bit more research into our SSL/HTTPS woes and I believe I
have uncovered two issues, one involving GETting from the
https://data.commcarehq.org/ota_restore URL and the other involving
POSTing to https://data.commcarehq.org/receiver/submit/.

The ota_restore issue appears to involve basic http authentication; it
works when accessed via http (port 80) but not via https (port 443). You
can try this in your browser. In both cases (http and https) you are
prompted to login. With http://data.commcarehq.org/ota_restore your
username/password are accepted and you receive the ota_restore XML document.
With https://data.commcarehq.org/ota_restore your username/password
are never accepted; you keep on being prompted to login.

The other issue involves POSTing to the
https://data.commcarehq.org/receiver/submit/ URL. There
is no authentication required to POST to a domain, so this is not the issue.
I examined the phone logs and then wrote a simple curl script to emulate
an data.commcarehq.org/receiver/submit/ POST. When
POSTing to the non-https URL (
http://data.commcarehq.org/receiver/submit/) I receive the
following successful response from the CommCareHQ server:

<?xml version='1.0' encoding='UTF-8'?> 1 1 Thanks! 2000 Submission 2011-01-24 09:52:12.732186 84861 200 1 8708cdb6-27c9-11e0-ab9a-00505697291a

When I change the URL to POST to the https URL (
https://data.commcarehq.org/receiver/submit/) I receive
the following failed response:

<?xml version='1.0' encoding='UTF-8'?> 1 2000 Submission 2011-01-24 06:10:45.646407 26 200 97409250-27aa-11e0-89d5-00505697291a

The only difference between these two POSTs was changing http to https.
If I look at my submissions in CommCare HQ, I see the http POST but not the
https POST.

Assuming what I am trying to do is possible, am I missing something or
is there an underlying problem?

Best,
Stephen

On Wed, Jan 5, 2011 at 2:36 PM, Clayton Sims csims@dimagi.com wrote:

Stephen,

The Nokia Emulators are actually pretty bad about including appropriate
Root CA’s. However, my local test with https is also behaving strangely on a
physical 2700, which is odd. I’ll look into this and let you
know whether there’s anything that needs to be resolved for https to work
properly.

-Clayton

On Wed, Jan 5, 2011 at 7:04 AM, Stephen Lorenz < stephen.lorenz@gmail.com> wrote:

Hi Dan,

Of course, that makes sense. This project doesn’t actually start
collecting data until February so we don’t yet have a phone with a data
plan. So all testing which involves communicating with CommCareHQ is being
done through the emulator–which must be using a keystore which is missing a
thawte root cert. Within the next two weeks we will be able to test the
communication piece on real phones (Nokia 2730) and this will be one of the
first things I check. In the meantime I will work on trying to import the
thawte root cert in my emulator.

Best,
Stephen

On Tue, Jan 4, 2011 at 10:52 PM, Dan Myung dmyung@dimagi.com wrote:

Hi Stephen,

Which phone model are you using? Can you hit the https URL in the
phone’s browser without it complaining about the root certificate
verification issues? Our certificate is from thawte, and was chosen
so as have its root certification be as widely shipped on stock
phones
as possible, so as to work out of the box. However, it still could
be
a possibility that our “lowest common denominator” decision may have
missed a few models…

Dan

On Jan 4, 7:20 am, Stephen Lorenz stephen.lor...@gmail.com wrote:

Hi all,

I am trying to configure our CommCare app to send and receive data
to
CommCare HQ over SSL. I have manually exported the SSL cert
fromhttps://data.commcarehq.organd installed it on the phone and I have
changed
the PostURL/PostTestURL and ota-restore-url settings in my
profile.xml from
http to https. However, I still receive the typical “Certificate
failed
verification” error when the app attempts to communicate with
CommCareHQ.
What is required to get the communication to run over SSL?

Thanks,
Stephen

Hi Dan,

I just confirmed with the person running the program that waiting until
early/mid-February is fine. He will not be submitting real patient data
until that point. Thank you for looking into this.

Best,
Stephen

··· On Tue, Jan 25, 2011 at 11:50 AM, Daniel Myung wrote:

Hi Stephen,

Just an update on our end. I’ve tested HTTPS for the OTA restore on our
newer codebase of the CommCareHQ code. Good news is that the https issue is
no longer an issue.

The bad news is that it seems to relegate the issue to the older instance
of CommCareHQ running using a different server setup. It’s been an ongoing
issue centered around the combination of the apache-mod-wsgi setup we have
running on the data.commcarehq.org instance, and/or quirks with the older
version of the django_digest authentication scheme. Current code now uses a
more streamlined server setup, hence the issue being resolved.

Would you be ok to wait until we release the updated version of commcarehq
with these fixes? The timeline for release is early February. Sorry for
the inconvenience. We’ll let you know when there are public URLs to test
against.

Dan

On Mon, Jan 24, 2011 at 12:52 PM, Stephen Lorenz <stephen.lorenz@gmail.com wrote:

Hi Dan and Clayton,

Thanks for the updates!

Best,
Stephen

On Mon, Jan 24, 2011 at 12:34 PM, Clayton Sims csims@dimagi.com wrote:

Stephen,

Thanks for the in-depth analysis into the situation! We’ll be sure to let
you know ASAP when this is resolved.

-Clayton

On Mon, Jan 24, 2011 at 11:22 AM, Daniel Myung dmyung@dimagi.comwrote:

Hi Stephen,

Thanks for the detailed analysis of the HTTP vs. HTTPS issues for OTA
restore. We’ve observed similar behavior on other instances of HQ code
we’ve run for HTTPS. The short answer is that we can reproduce it, and it’s
a server side problem on our end. The combination of apache proxying
interacting with the django instance is the root cause of the problem.
We’re going to look into the issue and try to tweak the configuration to
try and get the behavior consistent for both 443 and 80.

Dan

On Mon, Jan 24, 2011 at 10:11 AM, Stephen Lorenz < stephen.lorenz@gmail.com> wrote:

Hello all,

I have done a bit more research into our SSL/HTTPS woes and I believe I
have uncovered two issues, one involving GETting from the
https://data.commcarehq.org/ota_restore URL and the other involving
POSTing to https://data.commcarehq.org/receiver/submit/.

The ota_restore issue appears to involve basic http authentication; it
works when accessed via http (port 80) but not via https (port 443). You
can try this in your browser. In both cases (http and https) you are
prompted to login. With http://data.commcarehq.org/ota_restore your
username/password are accepted and you receive the ota_restore XML document.
With https://data.commcarehq.org/ota_restore your username/password
are never accepted; you keep on being prompted to login.

The other issue involves POSTing to the
https://data.commcarehq.org/receiver/submit/ URL. There
is no authentication required to POST to a domain, so this is not the issue.
I examined the phone logs and then wrote a simple curl script to emulate
an data.commcarehq.org/receiver/submit/ POST. When
POSTing to the non-https URL (
http://data.commcarehq.org/receiver/submit/) I receive
the following successful response from the CommCareHQ server:

<?xml version='1.0' encoding='UTF-8'?> 1 1 Thanks! 2000 Submission 2011-01-24 09:52:12.732186 84861 200 1 8708cdb6-27c9-11e0-ab9a-00505697291a

When I change the URL to POST to the https URL (
https://data.commcarehq.org/receiver/submit/) I receive
the following failed response:

<?xml version='1.0' encoding='UTF-8'?> 1 2000 Submission 2011-01-24 06:10:45.646407 26 200 97409250-27aa-11e0-89d5-00505697291a

The only difference between these two POSTs was changing http to https.
If I look at my submissions in CommCare HQ, I see the http POST but not the
https POST.

Assuming what I am trying to do is possible, am I missing something or
is there an underlying problem?

Best,
Stephen

On Wed, Jan 5, 2011 at 2:36 PM, Clayton Sims csims@dimagi.com wrote:

Stephen,

The Nokia Emulators are actually pretty bad about including
appropriate Root CA’s. However, my local test with https is also behaving
strangely on a physical 2700, which is odd. I’ll look into this and let you
know whether there’s anything that needs to be resolved for https to work
properly.

-Clayton

On Wed, Jan 5, 2011 at 7:04 AM, Stephen Lorenz < stephen.lorenz@gmail.com> wrote:

Hi Dan,

Of course, that makes sense. This project doesn’t actually start
collecting data until February so we don’t yet have a phone with a data
plan. So all testing which involves communicating with CommCareHQ is being
done through the emulator–which must be using a keystore which is missing a
thawte root cert. Within the next two weeks we will be able to test the
communication piece on real phones (Nokia 2730) and this will be one of the
first things I check. In the meantime I will work on trying to import the
thawte root cert in my emulator.

Best,
Stephen

On Tue, Jan 4, 2011 at 10:52 PM, Dan Myung dmyung@dimagi.comwrote:

Hi Stephen,

Which phone model are you using? Can you hit the https URL in the
phone’s browser without it complaining about the root certificate
verification issues? Our certificate is from thawte, and was chosen
so as have its root certification be as widely shipped on stock
phones
as possible, so as to work out of the box. However, it still could
be
a possibility that our “lowest common denominator” decision may have
missed a few models…

Dan

On Jan 4, 7:20 am, Stephen Lorenz stephen.lor...@gmail.com wrote:

Hi all,

I am trying to configure our CommCare app to send and receive data
to
CommCare HQ over SSL. I have manually exported the SSL cert
fromhttps://data.commcarehq.organd installed it on the phone and I have
changed
the PostURL/PostTestURL and ota-restore-url settings in my
profile.xml from
http to https. However, I still receive the typical “Certificate
failed
verification” error when the app attempts to communicate with
CommCareHQ.
What is required to get the communication to run over SSL?

Thanks,
Stephen