commcareHQ deployment error

Developers
1

Hi

I recently got the following error while trying to deploy commcareHQ.

[172.19.3.41] sudo: git submodule update --init --recursive -q
[172.19.3.41] sudo: git clean -ffd
[172.19.3.41] sudo: find . -name '*.pyc' -delete
[172.19.4.42] out: warning: af4d7f22fd470077ca28e85d4ec9678923cc5bb2:.gitmodules, multiple configurations found for 'submodule.subm
odules/couchforms-src.path'. Skipping second one!
[172.19.4.42] out: warning: af4d7f22fd470077ca28e85d4ec9678923cc5bb2:.gitmodules, multiple configurations found for 'submodule.subm
odules/couchforms-src.url'. Skipping second one!
[172.19.4.42] out: warning: af4d7f22fd470077ca28e85d4ec9678923cc5bb2:.gitmodules, multiple configurations found for 'submodule.subm
odules/couchexport-src.path'. Skipping second one!
[172.19.4.42] out: warning: af4d7f22fd470077ca28e85d4ec9678923cc5bb2:.gitmodules, multiple configurations found for 'submodule.subm
odules/couchexport-src.url'. Skipping second one!

.............................................
...............................................
..............................................

Fatal error: sudo() received nonzero return code 1 while executing!

Requested: git fetch origin --tags -q
Executed: sudo -S -p 'sudo password:' -H -u "cchq" /bin/bash -l -c "cd /home/cchq/www/echis/releases/2020-04-13_16.19 >/dev/null
&& git fetch origin --tags -q"

Aborting.
!!! Parallel execution exception under host u'172.19.4.42':
Process 172.19.4.42:

Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/home/ansible/.virtualenvs/ansible/local/lib/python2.7/site-packages/fabric/tasks.py", line 240, in inner
submit(task.run(*args, **kwargs))
File "/home/ansible/.virtualenvs/ansible/local/lib/python2.7/site-packages/fabric/tasks.py", line 172, in run
return self.wrapped(*args, **kwargs)
File "/home/ansible/.virtualenvs/ansible/local/lib/python2.7/site-packages/fabric/decorators.py", line 52, in inner_decorator
return func(*args, **kwargs)
File "/home/ansible/.virtualenvs/ansible/local/lib/python2.7/site-packages/fabric/decorators.py", line 180, in inner
return func(*args, **kwargs)
File "/home/ansible/commcare-cloud/src/commcare_cloud/fab/operations/release.py", line 49, in update
sudo('git fetch origin --tags -q')
File "/home/ansible/.virtualenvs/ansible/local/lib/python2.7/site-packages/fabric/network.py", line 654, in host_prompting_wrappe
r
return func(*args, **kwargs)
File "/home/ansible/.virtualenvs/ansible/local/lib/python2.7/site-packages/fabric/operations.py", line 1121, in sudo
stderr=stderr, timeout=timeout, shell_escape=shell_escape,
File "/home/ansible/.virtualenvs/ansible/local/lib/python2.7/site-packages/fabric/operations.py", line 952, in _run_command
error(message=msg, stdout=out, stderr=err)
File "/home/ansible/.virtualenvs/ansible/local/lib/python2.7/site-packages/fabric/utils.py", line 359, in error
return func(message)
File "/home/ansible/.virtualenvs/ansible/local/lib/python2.7/site-packages/fabric/utils.py", line 55, in abort
raise env.abort_exception(msg)
Exception: sudo() received nonzero return code 1 while executing!

Requested: git fetch origin --tags -q
Executed: sudo -S -p 'sudo password:' -H -u "cchq" /bin/bash -l -c "cd /home/cchq/www/echis/releases/2020-04-13_16.19 >/dev/null
&& git fetch origin --tags -q"

Fatal error: One or more hosts failed while executing task 'update'

Underlying exception:
sudo() received nonzero return code 1 while executing!

Requested: git fetch origin --tags -q
Executed: sudo -S -p 'sudo password:' -H  -u "cchq"  /bin/bash -l -c "cd /home/cchq/www/echis/releases/2020-04-13_16.19 >/dev/n

ull && git fetch origin --tags -q"

Aborting.

2

Hi Demis,

I have downloaded the repo manually and I haven't seen the errors running the same git commands manually but redeployment fails and we are looking into it

Thanks,
Sanjay

3

The error here appears like this in the stack trace:

[12.345.67.890] out: fatal: git upload-pack: not our ref 59cbfc3f2b9469367c4946b32075ea6e49dc4ff7
[12.345.67.890] out: fatal: remote error: upload-pack: not our ref 59cbfc3f2b9469367c4946b32075ea6e49dc4ff7
[12.345.67.890] out: Errors during submodule fetch:
[12.345.67.890] out:    corehq/apps/hqmedia/static/hqmedia/MediaUploader

This is triggered by upgrading git. We've merged a workaround which should allow deploys to complete if you update to the latest commcare-cloud.

4

Hi Demis,

Can you please confirm re-deploy working are not on Echis env?

Thanks,
Sanjay

5

Hi Sanjay,

Finally deployment works!!

I am using two upstream server: my own github(for pushing local changes ) and also dimagi's commcare-cloud repo for latest code update, it mess up the code.

for the time being, I just created another control machine and pull fresh copy of commcare-cloud repo, and then it worked

thanks @sanjay.chinthala and @Ethan_Soergel for ur support