App builder changes retrospective

Hi all

Just a little retrospective on the recent app builder changes that caused a
bit of havoc over the last couple of days.

First let me enumerate the sequence of events:

  • On Monday evening we deployed to prod which included a lazy migration
    of app documents and also included a bug in the app profile.xml template.
  • Once the bug was discovered the deploy was rolled back to the
    previously deployed version but a few apps had already been migrated and so
    were not compatible with the older code.
  • The solution was to fix the bug in the profile.xml template and
    re-deploy.
  • A second bug appeared which affected Cloucare and was fixed by Cory
    yesterday.

Why did this happen?

  1. I made the change to the profile.xml template whilst testing other
    app builder changes but never went back to test this specific change. This
    was an oversight on my part.
  2. The bug in Cloudcare happened because both Danny and myself failed to
    realize the dependency between the two and so didn’t consider looking there
    during the re-factoring.

How to make sure it doesn’t happen in the future:

  • Run the tests properly dammit!
  • Take a moment to consider what else might be affected by the changes.
··· -- Simon Kelly Senior Engineer | Dimagi South Africa

thanks Simon.

i suspect it also affects this:
https://help.commcarehq.org/display/commcarepublic/Application+Structure+API
though
i don’t know if anyone uses that API.

we could use kibana to check how frequently it is accessed (if at all).

and yeah, good to be super careful when changing the underlying structure
of anything that might end up being referenced externally or via APIs.

thanks,
Cory

··· On Wed, Nov 13, 2013 at 2:34 PM, Simon Kelly wrote:

Hi all

Just a little retrospective on the recent app builder changes that caused
a bit of havoc over the last couple of days.

First let me enumerate the sequence of events:

  • On Monday evening we deployed to prod which included a lazy
    migration of app documents and also included a bug in the app profile.xml
    template.
  • Once the bug was discovered the deploy was rolled back to the
    previously deployed version but a few apps had already been migrated and so
    were not compatible with the older code.
  • The solution was to fix the bug in the profile.xml template and
    re-deploy.
  • A second bug appeared which affected Cloucare and was fixed by Cory
    yesterday.

Why did this happen?

  1. I made the change to the profile.xml template whilst testing other
    app builder changes but never went back to test this specific change. This
    was an oversight on my part.
  2. The bug in Cloudcare happened because both Danny and myself failed
    to realize the dependency between the two and so didn’t consider looking
    there during the re-factoring.

How to make sure it doesn’t happen in the future:

  • Run the tests properly dammit!
  • Take a moment to consider what else might be affected by the changes.


Simon Kelly
Senior Engineer | Dimagi South Africa


You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Thanks Cory, it doesn’t seem anyone is using that API. Also the API doesn’t
include any of the parts of the app that were changed.

··· On 13 November 2013 14:29, Cory Zue wrote:

thanks Simon.

i suspect it also affects this:
https://help.commcarehq.org/display/commcarepublic/Application+Structure+API though
i don’t know if anyone uses that API.

we could use kibana to check how frequently it is accessed (if at all).

and yeah, good to be super careful when changing the underlying structure
of anything that might end up being referenced externally or via APIs.

thanks,
Cory

On Wed, Nov 13, 2013 at 2:34 PM, Simon Kelly skelly@dimagi.com wrote:

Hi all

Just a little retrospective on the recent app builder changes that caused
a bit of havoc over the last couple of days.

First let me enumerate the sequence of events:

  • On Monday evening we deployed to prod which included a lazy
    migration of app documents and also included a bug in the app profile.xml
    template.
  • Once the bug was discovered the deploy was rolled back to the
    previously deployed version but a few apps had already been migrated and so
    were not compatible with the older code.
  • The solution was to fix the bug in the profile.xml template and
    re-deploy.
  • A second bug appeared which affected Cloucare and was fixed by Cory
    yesterday.

Why did this happen?

  1. I made the change to the profile.xml template whilst testing other
    app builder changes but never went back to test this specific change. This
    was an oversight on my part.
  2. The bug in Cloudcare happened because both Danny and myself failed
    to realize the dependency between the two and so didn’t consider looking
    there during the re-factoring.

How to make sure it doesn’t happen in the future:

  • Run the tests properly dammit!
  • Take a moment to consider what else might be affected by the
    changes.


Simon Kelly
Senior Engineer | Dimagi South Africa


You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Simon Kelly
Senior Engineer | Dimagi South Africa

ok great. thanks for checking.

··· On Wed, Nov 13, 2013 at 6:08 PM, Simon Kelly wrote:

Thanks Cory, it doesn’t seem anyone is using that API. Also the API
doesn’t include any of the parts of the app that were changed.

On 13 November 2013 14:29, Cory Zue czue@dimagi.com wrote:

thanks Simon.

i suspect it also affects this:
https://help.commcarehq.org/display/commcarepublic/Application+Structure+API though
i don’t know if anyone uses that API.

we could use kibana to check how frequently it is accessed (if at all).

and yeah, good to be super careful when changing the underlying structure
of anything that might end up being referenced externally or via APIs.

thanks,
Cory

On Wed, Nov 13, 2013 at 2:34 PM, Simon Kelly skelly@dimagi.com wrote:

Hi all

Just a little retrospective on the recent app builder changes that
caused a bit of havoc over the last couple of days.

First let me enumerate the sequence of events:

  • On Monday evening we deployed to prod which included a lazy
    migration of app documents and also included a bug in the app profile.xml
    template.
  • Once the bug was discovered the deploy was rolled back to the
    previously deployed version but a few apps had already been migrated and so
    were not compatible with the older code.
  • The solution was to fix the bug in the profile.xml template and
    re-deploy.
  • A second bug appeared which affected Cloucare and was fixed by
    Cory yesterday.

Why did this happen?

  1. I made the change to the profile.xml template whilst testing
    other app builder changes but never went back to test this specific change.
    This was an oversight on my part.
  2. The bug in Cloudcare happened because both Danny and myself
    failed to realize the dependency between the two and so didn’t consider
    looking there during the re-factoring.

How to make sure it doesn’t happen in the future:

  • Run the tests properly dammit!
  • Take a moment to consider what else might be affected by the
    changes.


Simon Kelly
Senior Engineer | Dimagi South Africa


You received this message because you are subscribed to the Google
Groups “CommCare Developers” group.
To unsubscribe from this group and stop receiving emails from it, send
an email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Simon Kelly
Senior Engineer | Dimagi South Africa


You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.