Page returns 500 error while opening application link

demis08 · May 10, 2021, 3:42pm

Hi there,

i got the following error while trying to open a file as shown below.

the associated log is as shown below

2021-05-10 15:32:31,039 ERROR Internal Server Error: /a/fmoh-echis/apps/view/49a390c8938c90f81c00f0a65d00bc27/
Traceback (most recent call last):
File "/home/cchq/www/echis/releases/2021-05-10_13.14/python_env-3.6/lib/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/home/cchq/www/echis/releases/2021-05-10_13.14/python_env-3.6/lib/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/home/cchq/www/echis/releases/2021-05-10_13.14/python_env-3.6/lib/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/cchq/www/echis/releases/2021-05-10_13.14/python_env-3.6/lib/python3.6/site-packages/sentry_sdk/integrations/django/views.py", line 67, in sentry_wrapped_callback
return callback(request, *args, **kwargs)
File "/home/cchq/www/echis/releases/2021-05-10_13.14/python_env-3.6/lib/python3.6/site-packages/django/views/decorators/http.py", line 40, in inner
return func(request, *args, **kwargs)
File "/home/cchq/www/echis/releases/2021-05-10_13.14/corehq/apps/domain/decorators.py", line 119, in _inner
return call_view()
File "/home/cchq/www/echis/releases/2021-05-10_13.14/corehq/apps/domain/decorators.py", line 88, in call_view
def call_view(): return view_func(req, domain_name, *args, **kwargs)
File "/home/cchq/www/echis/releases/2021-05-10_13.14/corehq/apps/app_manager/views/apps.py", line 666, in view_app
return view_generic(request, domain, app_id, release_manager=True)
File "/home/cchq/www/echis/releases/2021-05-10_13.14/corehq/ex-submodules/dimagi/utils/couch/resource_conflict.py", line 20, in new_fn
return fn(*args, **kwargs)
File "/home/cchq/www/echis/releases/2021-05-10_13.14/corehq/apps/app_manager/views/view_generic.py", line 161, in view_generic
context.update(get_releases_context(request, domain, app_id))
File "/home/cchq/www/echis/releases/2021-05-10_13.14/corehq/apps/app_manager/views/releases.py", line 219, in get_releases_context
multimedia_state = app.check_media_state()
File "/home/cchq/www/echis/releases/2021-05-10_13.14/corehq/apps/hqmedia/models.py", line 982, in check_media_state
for media in self.all_media():
File "/home/cchq/www/echis/releases/2021-05-10_13.14/python_env-3.6/lib/python3.6/site-packages/memoized.py", line 20, in _memoized
cache[key] = value = fn(*args, **kwargs)
File "/home/cchq/www/echis/releases/2021-05-10_13.14/corehq/apps/hqmedia/models.py", line 777, in all_media
form.validate_form()
File "/home/cchq/www/echis/releases/2021-05-10_13.14/corehq/apps/app_manager/models.py", line 1097, in validate_form
form = self.wrapped_xform()
File "/home/cchq/www/echis/releases/2021-05-10_13.14/corehq/apps/app_manager/models.py", line 1089, in wrapped_xform
return XForm(self.source)
File "/home/cchq/www/echis/releases/2021-05-10_13.14/corehq/apps/app_manager/models.py", line 722, in get
source = source.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa7 in position 436903: invalid start byte

Ethan_Soergel · May 11, 2021, 6:59pm

Hi Demis,

From that stack trace, it looks like there is an invalid character in the form. The form source is stored as an xml attachment, and this is a failure when attempting to interpret that document as utf-8. Unfortunately I don't think we have any handy utilities to resolve this issue, but you can try viewing the attachment on its own to see if you can find the offending character and maybe get clarity on what the issue was.

Farid_Rener · May 11, 2021, 7:46pm

Hi Demis,

The form source ('attachment') is actually stored in the blobdb, and it could be that the source has been corrupted somehow (due to power failure or something else).

You can first do what Ethan suggests, which is to figure out which form is broken. The page at: /a/{DOMAIN}/apps/download/{APP_ID}/ might show you more information.

Otherwise, I would iterate through each form in a django shell (./manage.py shell, or cchq {env} django-manage shell) as follows:

from corehq.apps.app_manager.models import Application

app_id = "49a390c8938c90f81c00f0a65d00bc27"
app = Application.get(app_id)
forms = [form for form in app.get_forms()]

for form in forms:
     try:
         form.wrapped_xform()
     except Exception:
         print(f"Bad form is: {form.unique_id}")
         break

You might then be able to pull the raw source (without decoding it to utf-8) with:

attachment = app.lazy_fetch_attachment(f"{form.unique_id}.xml")
print(attachment)

By outputting the raw source of the form, you might see that the file has been truncated, or contains some weird characters, that you may be able to fix and write back to the blob db.

However, in your case, if this is one of your linked apps, it might be easier to pull the source back down from HQ. If it isn't a linked app, you could also try reverting the application to a previous build.

Thanks

demis08 · May 13, 2021, 8:25pm

Thanks Farid,

the script is really supportive. I got a single form_id while running the above command.

Bad form is: f7883048f57b5ae7c8519ad9eb2886d6800e103a

using the blobdb client tool, i tried to display the content of file,...no luck.

mc cat minionew/echis/f7883048f57b5ae7c8519ad9eb2886d6800e103a

mc cat miniold/echis/f7883048f57b5ae7c8519ad9eb2886d6800e103a

Looking into the /a/{DOMAIN}/apps/download/{APP_ID}/ displays the following and trying to download the files brings an error