Just a little retrospective on the recent app builder changes that caused a
bit of havoc over the last couple of days.
First let me enumerate the sequence of events:
On Monday evening we deployed to prod which included a lazy migration
of app documents and also included a bug in the app profile.xml template.
Once the bug was discovered the deploy was rolled back to the
previously deployed version but a few apps had already been migrated and so
were not compatible with the older code.
The solution was to fix the bug in the profile.xml template and
re-deploy.
A second bug appeared which affected Cloucare and was fixed by Cory
yesterday.
Why did this happen?
I made the change to the profile.xml template whilst testing other
app builder changes but never went back to test this specific change. This
was an oversight on my part.
The bug in Cloudcare happened because both Danny and myself failed to
realize the dependency between the two and so didn't consider looking there
during the re-factoring.
How to make sure it doesn't happen in the future:
Run the tests properly dammit!
Take a moment to consider what else might be affected by the changes.
···
--
Simon Kelly
Senior Engineer | Dimagi South Africa
we could use kibana to check how frequently it is accessed (if at all).
and yeah, good to be super careful when changing the underlying structure
of anything that might end up being referenced externally or via APIs.
thanks,
Cory
···
On Wed, Nov 13, 2013 at 2:34 PM, Simon Kelly wrote:
Hi all
Just a little retrospective on the recent app builder changes that caused
a bit of havoc over the last couple of days.
First let me enumerate the sequence of events:
On Monday evening we deployed to prod which included a lazy
migration of app documents and also included a bug in the app profile.xml
template.
Once the bug was discovered the deploy was rolled back to the
previously deployed version but a few apps had already been migrated and so
were not compatible with the older code.
The solution was to fix the bug in the profile.xml template and
re-deploy.
A second bug appeared which affected Cloucare and was fixed by Cory
yesterday.
Why did this happen?
I made the change to the profile.xml template whilst testing other
app builder changes but never went back to test this specific change. This
was an oversight on my part.
The bug in Cloudcare happened because both Danny and myself failed
to realize the dependency between the two and so didn't consider looking
there during the re-factoring.
How to make sure it doesn't happen in the future:
Run the tests properly dammit!
Take a moment to consider what else might be affected by the changes.
--
Simon Kelly
Senior Engineer | Dimagi South Africa
we could use kibana to check how frequently it is accessed (if at all).
and yeah, good to be super careful when changing the underlying structure
of anything that might end up being referenced externally or via APIs.
thanks,
Cory
On Wed, Nov 13, 2013 at 2:34 PM, Simon Kelly skelly@dimagi.com wrote:
Hi all
Just a little retrospective on the recent app builder changes that caused
a bit of havoc over the last couple of days.
First let me enumerate the sequence of events:
On Monday evening we deployed to prod which included a lazy
migration of app documents and also included a bug in the app profile.xml
template.
Once the bug was discovered the deploy was rolled back to the
previously deployed version but a few apps had already been migrated and so
were not compatible with the older code.
The solution was to fix the bug in the profile.xml template and
re-deploy.
A second bug appeared which affected Cloucare and was fixed by Cory
yesterday.
Why did this happen?
I made the change to the profile.xml template whilst testing other
app builder changes but never went back to test this specific change. This
was an oversight on my part.
The bug in Cloudcare happened because both Danny and myself failed
to realize the dependency between the two and so didn't consider looking
there during the re-factoring.
How to make sure it doesn't happen in the future:
Run the tests properly dammit!
Take a moment to consider what else might be affected by the
changes.
--
Simon Kelly
Senior Engineer | Dimagi South Africa
we could use kibana to check how frequently it is accessed (if at all).
and yeah, good to be super careful when changing the underlying structure
of anything that might end up being referenced externally or via APIs.
thanks,
Cory
On Wed, Nov 13, 2013 at 2:34 PM, Simon Kelly skelly@dimagi.com wrote:
Hi all
Just a little retrospective on the recent app builder changes that
caused a bit of havoc over the last couple of days.
First let me enumerate the sequence of events:
On Monday evening we deployed to prod which included a lazy
migration of app documents and also included a bug in the app profile.xml
template.
Once the bug was discovered the deploy was rolled back to the
previously deployed version but a few apps had already been migrated and so
were not compatible with the older code.
The solution was to fix the bug in the profile.xml template and
re-deploy.
A second bug appeared which affected Cloucare and was fixed by
Cory yesterday.
Why did this happen?
I made the change to the profile.xml template whilst testing
other app builder changes but never went back to test this specific change.
This was an oversight on my part.
The bug in Cloudcare happened because both Danny and myself
failed to realize the dependency between the two and so didn't consider
looking there during the re-factoring.
How to make sure it doesn't happen in the future:
Run the tests properly dammit!
Take a moment to consider what else might be affected by the
changes.
--
Simon Kelly
Senior Engineer | Dimagi South Africa