Unable to install new application due to multimedia issue

We've been unable to install new apps with multimedia and have observed that when we try to upload an image through the web browser we can't see it in preview either as seen below:

What could be the cause of this?

More Information about our issue: So it seems the link between postgres and blob is broken or there are might be a permission issue because we can see that the files are uploading fine but we can't access the images on the frontend nor app preview.
All services are up and the only error we get it the lack of the picture preview and/or the error when we try to install the app without multimedia. The app installs without multimedia though. Any ideas on what's going on here?

Hello Niko,

What happens when you copy that image URL and try to open in a new tab, what error are you getting?

To further get an idea, could you check below?

  1. Does the application multimedia page load fine with all media ?
  2. If the media doesn't load, is it missing all the media or the only newly uploaded media?

The other thing that I would recommend trying is to perform a code deploy (cchq deploy) and see if the error disappears.

Hi Sravan,

  1. the application media page does not load fine with all media, only a small fraction (10/132) of the images load.

  2. uploading new multimedia shows the image from the original topic post (no preview) and failure to install

  3. cchq deploy does not resolve the issue, we are running on the latest code

Hello

What happens when you copy that image URL and try to open in a new tab, what error are you getting?

If this is a remote application, you could try 'Force pull all multimedia' that pulls all of the multimedia from remote and sets up any missing multimedia.

Hi,

copy that image URL and try to open in a new tab:

If this is a remote application, you could try 'Force pull all multimedia':

It's not a remote application. Both the test and production project spaces (linked) are on the local instance (commcare.health.gov.mw) . The multimedia issues are being seen on the test project space and then being pushed down to the live space hence the failure to install apps from both project spaces.

Okay, it appears that there is a 500 error when you access that URL. You can investigate into the respective error traceback from the Django log files to find what the error is or you can share here for me to look into.

For reference, the error will be logged under /home/cchq/www/<envname>/logs/<django-log-file>. You can tail that file while you try to refresh the URL to print the traceback.

When we refresh the URL:

2022-09-26 14:14:54,867 ERROR [django.request] Internal Server Error: /hq/multimedia/file/CommCareImage/79df1fdeba8589f0859302cd04b922d1/
Traceback (most recent call last):
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/blobs/mixin.py", line 191, in fetch_attachment
meta = db.metadb.get(parent_id=self._id, key=key)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/blobs/metadata.py", line 188, in get
raise BlobMeta.DoesNotExist(repr(kw))
corehq.blobs.models.BlobMeta.DoesNotExist: {'parent_id': '79df1fdeba8589f0859302cd04b922d1', 'key': 'cbc80afa3e334823a1f228761d5b2f63'}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/python_env-3.9/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/python_env-3.9/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/python_env-3.9/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
return self.dispatch(request, *args, **kwargs)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/middleware.py", line 234, in inner
response = fn(*args, **kwargs)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/apps/hqmedia/views.py", line 959, in dispatch
return super(ViewMultimediaFile, self).dispatch(request, *args, **kwargs)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/python_env-3.9/lib/python3.9/site-packages/django/views/generic/base.py", line 98, in dispatch
return handler(request, *args, **kwargs)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/apps/hqmedia/views.py", line 993, in get
data, content_type = self.multimedia.get_display_file()
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/apps/hqmedia/models.py", line 200, in get_display_file
data = self.fetch_attachment(self.attachment_id, stream=True).read()
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/blobs/mixin.py", line 194, in fetch_attachment
raise ResourceNotFound(
couchdbkit.exceptions.ResourceNotFound: CommCareImage 79df1fdeba8589f0859302cd04b922d1 attachment: 'da03264f2bee45696170a158c5a1ef56-437x437.PNG'

When we try to upload multimedia:

2022-09-26 14:21:35,962 ERROR [django.request] Internal Server Error: /hq/multimedia/file/CommCareImage/79df1fdeba8589f0859302cd04b922d1/
Traceback (most recent call last):
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/blobs/mixin.py", line 191, in fetch_attachment
meta = db.metadb.get(parent_id=self._id, key=key)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/blobs/metadata.py", line 188, in get
raise BlobMeta.DoesNotExist(repr(kw))
corehq.blobs.models.BlobMeta.DoesNotExist: {'parent_id': '79df1fdeba8589f0859302cd04b922d1', 'key': 'cbc80afa3e334823a1f228761d5b2f63'}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/python_env-3.9/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/python_env-3.9/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/python_env-3.9/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
return self.dispatch(request, *args, **kwargs)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/middleware.py", line 234, in inner
response = fn(*args, **kwargs)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/apps/hqmedia/views.py", line 959, in dispatch
return super(ViewMultimediaFile, self).dispatch(request, *args, **kwargs)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/python_env-3.9/lib/python3.9/site-packages/django/views/generic/base.py", line 98, in dispatch
return handler(request, *args, **kwargs)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/apps/hqmedia/views.py", line 993, in get
data, content_type = self.multimedia.get_display_file()
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/apps/hqmedia/models.py", line 200, in get_display_file
data = self.fetch_attachment(self.attachment_id, stream=True).read()
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/blobs/mixin.py", line 194, in fetch_attachment
raise ResourceNotFound(
couchdbkit.exceptions.ResourceNotFound: CommCareImage 79df1fdeba8589f0859302cd04b922d1 attachment: 'da03264f2bee45696170a158c5a1ef56-437x437.PNG'
2022-09-26 14:21:38,724 ERROR [django.request] Internal Server Error: /hq/multimedia/file/CommCareImage/79df1fdeba8589f0859302cd04b922d1/
Traceback (most recent call last):
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/blobs/mixin.py", line 191, in fetch_attachment
meta = db.metadb.get(parent_id=self._id, key=key)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/blobs/metadata.py", line 188, in get
raise BlobMeta.DoesNotExist(repr(kw))
corehq.blobs.models.BlobMeta.DoesNotExist: {'parent_id': '79df1fdeba8589f0859302cd04b922d1', 'key': 'cbc80afa3e334823a1f228761d5b2f63'}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/python_env-3.9/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/python_env-3.9/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/python_env-3.9/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
return self.dispatch(request, *args, **kwargs)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/middleware.py", line 234, in inner
response = fn(*args, **kwargs)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/apps/hqmedia/views.py", line 959, in dispatch
return super(ViewMultimediaFile, self).dispatch(request, *args, **kwargs)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/python_env-3.9/lib/python3.9/site-packages/django/views/generic/base.py", line 98, in dispatch
return handler(request, *args, **kwargs)
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/apps/hqmedia/views.py", line 993, in get
data, content_type = self.multimedia.get_display_file()
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/apps/hqmedia/models.py", line 200, in get_display_file
data = self.fetch_attachment(self.attachment_id, stream=True).read()
File "/home/cchq/www/monolith/releases/2022-09-21_15.34/corehq/blobs/mixin.py", line 194, in fetch_attachment
raise ResourceNotFound(
couchdbkit.exceptions.ResourceNotFound: CommCareImage 79df1fdeba8589f0859302cd04b922d1 attachment: 'da03264f2bee45696170a158c5a1ef56-437x437.PNG'

Hello Niko,

This indicates that the multimedia referred to by the application are missing in your environment. This could happen if you imported the application from another environment without properly importing the multimedia. Did you recently migrate from Dimagi's server to your environment?

We managed to resolve this issue. The solution was to delete and replace all multimedia on the local instance with new images. Users are now able to do a fresh installation.