Date and bool

Hi Brian

I played with this a bit and following instructions here [1] got it to work
by installing ‘ndg-httpsclient’. I’ve also added this to the requirements
for the package so it should install in future.

Re the type support, here is the PR
https://github.com/dimagi/commcare-export/pull/41 (pretty irritating how
different DB’s support types differently!). Once that gets merged I’ll
create a new release.

[1] http://stackoverflow.com/a/29099439

··· On 29 June 2015 at 16:33, Brian DeRenzi wrote:

Hi Simon,

I played with this a bit over the weekend. I didn’t try to address the
date/bool issues, but rather the issue with the SSL connection. The reason
I want to fix that issue is that I have the following setup:

Cron calls a shell script
Shell script calls python and redirects stderr to a file
Shell script checks the stderr file to see if there were any issues, then
emails me if there were

Since the insecure platform warning [1] shows up on standard error, I get
an email every minute. I’ve disabled the email feature for the moment. I
upgraded to 2.7.10, but that didn’t help. I tried hacking in my local copy
of the code to implement the pyopenssl fix [2], to no success. The requests
module is the one calling urllib3 (I think there are some unnecessary
imports at the top there with the urlopen and what not), but even
implementing the suggested fix [3] didn’t help. By then I’d spent too much
time on it and had to move on to other pieces.

Nothing urgent, but any thoughts? Also happy to punt on the warning if you
have different approach to monitoring things (besides my hack
pipe-to-file-and-email method)

Brian

[1]
https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning
[2] https://urllib3.readthedocs.org/en/latest/security.html#pyopenssl
[3]
http://stackoverflow.com/questions/27981545/surpress-insecurerequestwarning-unverified-https-request-is-being-made-in-pytho

On Fri, Jun 26, 2015 at 5:51 PM, Brian DeRenzi bderenzi@cs.uct.ac.za wrote:

Hi Simon,

Thanks so much for sitting with me and working through this. Just a
friendly reminder to please add dates and bools.

Brian


Simon Kelly
Senior Engineer | Dimagi South Africa

Hey Brian

I’ve created a 0.12.0 release. I’ve also added a ‘–strict-types’ parameter
which will prevent the tool from changing column types once they have been
created. This may result in runtime errors if data is an unexpected type
but that’s probably preferable. It will still create the table for you if
it doesn’t already exist and also add new columns if they don’t exist (but
only when it gets data for those columns).

Let me know if you have any issues. Hopefully we can get things running
smoothly soon!
Simon

··· On 30 June 2015 at 04:25, Brian DeRenzi wrote:

Super, thanks Simon! Hopefully we’ll have “real-time” graphs up and
running this week.

(and crazy how much code was required to support across the DBs…)

On Mon, Jun 29, 2015 at 5:30 PM, Simon Kelly skelly@dimagi.com wrote:

Hi Brian

I played with this a bit and following instructions here [1] got it to
work by installing ‘ndg-httpsclient’. I’ve also added this to the
requirements for the package so it should install in future.

Re the type support, here is the PR
https://github.com/dimagi/commcare-export/pull/41 (pretty irritating
how different DB’s support types differently!). Once that gets merged I’ll
create a new release.

[1] http://stackoverflow.com/a/29099439

On 29 June 2015 at 16:33, Brian DeRenzi bderenzi@cs.uct.ac.za wrote:

Hi Simon,

I played with this a bit over the weekend. I didn’t try to address the
date/bool issues, but rather the issue with the SSL connection. The reason
I want to fix that issue is that I have the following setup:

Cron calls a shell script
Shell script calls python and redirects stderr to a file
Shell script checks the stderr file to see if there were any issues,
then emails me if there were

Since the insecure platform warning [1] shows up on standard error, I
get an email every minute. I’ve disabled the email feature for the moment.
I upgraded to 2.7.10, but that didn’t help. I tried hacking in my local
copy of the code to implement the pyopenssl fix [2], to no success. The
requests module is the one calling urllib3 (I think there are some
unnecessary imports at the top there with the urlopen and what not), but
even implementing the suggested fix [3] didn’t help. By then I’d spent too
much time on it and had to move on to other pieces.

Nothing urgent, but any thoughts? Also happy to punt on the warning if
you have different approach to monitoring things (besides my hack
pipe-to-file-and-email method)

Brian

[1]
https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning
[2] https://urllib3.readthedocs.org/en/latest/security.html#pyopenssl
[3]
http://stackoverflow.com/questions/27981545/surpress-insecurerequestwarning-unverified-https-request-is-being-made-in-pytho

On Fri, Jun 26, 2015 at 5:51 PM, Brian DeRenzi bderenzi@cs.uct.ac.za wrote:

Hi Simon,

Thanks so much for sitting with me and working through this. Just a
friendly reminder to please add dates and bools.

Brian


Simon Kelly
Senior Engineer | Dimagi South Africa


Simon Kelly
Senior Engineer | Dimagi South Africa