Commcare deploy error

robynton · August 20, 2022, 5:11pm

Good day! I have a fresh installation of Ubuntnu 18.04. Used quick install script to install in as monolith server. At the end of script, when executing deploy step, I've got next warning about ownership in git repository and next error about parallel execution mode, running commcare-cloud monolith deploy produce the same warning and error.

So warning:

(cchq) lamp@monolith:~/commcare-cloud$ commcare-cloud monolith deploy
[192.169.233.133] Executing task '_task'
[192.169.233.133] run: git rev-parse HEAD
[192.169.233.133] Passphrase for private key:
[192.169.233.133] out: fatal: detected dubious ownership in repository at '/home/cchq/www/monolith/releases/2022-08-18_16.41'
[192.169.233.133] out: To add an exception for this directory, call:
[192.169.233.133] out:
[192.169.233.133] out: git config --global --add safe.directory /home/cchq/www/monolith/releases/2022-08-18_16.41
[192.169.233.133] out:
Warning: run() received nonzero return code 128 while executing 'git rev-parse HEAD'!
Diff generation skipped. Supply a Github token to see deploy diffs.
New version details:
Branch deployed : commcare: master
Here's the complete diff on github: https://github.com/dimagi/commcare-hq/compare/fatal: detected dubious ownership in repository at '/home/cchq/www/monolith/releases/2022-08-18_16.41'
To add an exception for this directory, call:
git config --global --add safe.directory /home/cchq/www/monolith/releases/2022-08-18_16.41...91cf5d37ba5cc66a15994210f46e8a244b2d9501

And error:

Are you sure you want to preindex and deploy to monolith? [y/N]y
Vault Password for 'monolith':
Sending email: lamp has initiated a CommCare HQ deploy to monolith
Ubuntu 18.04.6 LTS
Enter passphrase for key '/home/lamp/.ssh/id_rsa':
Connection to 192.169.233.133 closed.
commcare-cloud monolith fab deploy_commcare --set code_branch=master --branch master
patched-fab -f /home/lamp/commcare-cloud/src/commcare_cloud/fabfile.py monolith deploy_commcare --set code_branch=master --disable-known-hosts --system-known-hosts /home/lamp/environments/monolith/known_hosts
Using commcare-hq branch master
[192.169.233.133] Executing task 'deploy_commcare'
[192.169.233.133] Executing task '_setup_release'
[192.169.233.133] Executing task 'create'
[192.169.233.133] sudo: mkdir -p /home/cchq/www/monolith/releases/2022-08-19_18.38

Fatal error: Needed to prompt for a connection or sudo password (host: 192.169.233.133), but input would be ambiguous in parallel mode

Aborting.
!!! Parallel execution exception under host '192.169.233.133':

Process 192.169.233.133:
Traceback (most recent call last):
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/network.py", line 449, in connect
sock=sock
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/paramiko/client.py", line 446, in connect
passphrase,
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/paramiko/client.py", line 764, in _auth
raise saved_exception
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/paramiko/client.py", line 735, in _auth
filename, pkey_class, passphrase
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/paramiko/client.py", line 586, in _key_from_filepath
key = klass.from_private_key_file(key_path, password)
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/paramiko/pkey.py", line 235, in from_private_key_file
key = cls(filename=filename, password=password)
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/paramiko/rsakey.py", line 55, in init
self._from_private_key_file(filename, password)
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/paramiko/rsakey.py", line 175, in _from_private_key_file
data = self._read_private_key_file("RSA", filename, password)
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/paramiko/pkey.py", line 308, in _read_private_key_file
data = self._read_private_key(tag, f, password)
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/paramiko/pkey.py", line 334, in _read_private_key
data = self._read_private_key_pem(lines, end, password)
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/paramiko/pkey.py", line 386, in _read_private_key_pem
raise PasswordRequiredException("Private key file is encrypted")
paramiko.ssh_exception.PasswordRequiredException: Private key file is encrypted

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/tasks.py", line 240, in inner
submit(task.run(*args, **kwargs))
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/tasks.py", line 172, in run
return self.wrapped(*args, **kwargs)
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/decorators.py", line 52, in inner_decorator
return func(*args, **kwargs)
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/decorators.py", line 180, in inner
return func(*args, **kwargs)
File "/home/lamp/commcare-cloud/src/commcare_cloud/fab/operations/release.py", line 203, in create
sudo('mkdir -p {}'.format(env.code_root))
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/network.py", line 654, in host_prompting_wrapper
return func(*args, **kwargs)
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/operations.py", line 1121, in sudo
stderr=stderr, timeout=timeout, shell_escape=shell_escape,
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/operations.py", line 929, in _run_command
channel=default_channel(), command=wrapped_command, pty=pty,
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/state.py", line 397, in default_channel
chan = _open_session()
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/state.py", line 389, in _open_session
return connections[env.host_string].get_transport().open_session()
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/network.py", line 156, in getitem
self.connect(key)
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/network.py", line 148, in connect
user, host, port, cache=self, seek_gateway=seek_gateway)
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/network.py", line 528, in connect
password = prompt_for_password(text)
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/network.py", line 605, in prompt_for_password
handle_prompt_abort("a connection or sudo password")
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/utils.py", line 184, in handle_prompt_abort
abort(reason % "input would be ambiguous in parallel mode")
File "/home/lamp/.virtualenvs/cchq/lib/python3.6/site-packages/fabric/utils.py", line 55, in abort
raise env.abort_exception(msg)
Exception: Needed to prompt for a connection or sudo password (host: 192.169.233.133), but input would be ambiguous in parallel mode

Fatal error: One or more hosts failed while executing task 'create'

Underlying exception:
Needed to prompt for a connection or sudo password (host: 192.169.233.133), but input would be ambiguous in parallel mode

Aborting.

Connecting to server with ssh -A and rerun of deploy didn't help.
Would be thankfull to any suggestions.

CharlSmit · August 22, 2022, 10:57am

Hi, @robynton .

I see the following line in the error report:

paramiko.ssh_exception.PasswordRequiredException: Private key file is encrypted

As far as I know this means the private key file is encrypted and the script will want to ask for a password, hence the error

Exception: Needed to prompt for a connection or sudo password (host: 192.169.233.133), but input would be ambiguous in parallel mode

You can try reconnecting to the server using ssh -A after adding the private key to the ssh agent.

Steps to add to ssh agent:

eval ssh-agent (this makes sure the ssh agent is running)
ssh-add (this will add these keys to the agent by default)
ssh-add -l ([optional] verify that the correct private key has been added)

Please let me know if this works for you.

robynton · August 22, 2022, 8:21pm

Thank you sir fo detailed explanation! It worked!

But when I run commcare-cloud monolith django-manage check_services I've got error about missing any users:

ssh lamp@192.169.233.133 -t -o UserKnownHostsFile=/home/lamp/environments/monolith/known_hosts 'sudo -iu cchq bash -c '"'"'cd /home/cchq/www/monolith/current; python_env/bin/python manage.py check_services'"'"''
Ubuntu 18.04.6 LTS
SUCCESS (Took 0.62s) kafka : Kafka seems to be in order
SUCCESS (Took 0.00s) redis : Redis is up and using 1.01M memory
FAILURE (Took 0.06s) postgres : default:commcarehq:OK p1:commcarehq_p1:OK p2:commcarehq_p2:OK proxy:commcarehq_proxy:OK synclogs:commcarehq_synclogs:OK ucr:commcarehq_ucr:OK No users found in postgres
SUCCESS (Took 0.01s) couch : Successfully queried an arbitrary couch view
SUCCESS (Took 0.01s) celery : OK
SUCCESS (Took 0.11s) elasticsearch : Successfully sent a doc to ES and read it back
SUCCESS (Took 0.27s) blobdb : Successfully saved a file to the blobdb
FAILURE (Took 0.01s) formplayer : Could not connect to formplayer: https://capi.gstat.gov.ua/formplayer/serverup
SUCCESS (Took 0.00s) rabbitmq : RabbitMQ OK
Connection to 192.169.233.133 closed.

So, the silly question is what is default user/password, or how I can create one.
Thanks in advance!

Babacar_Gueye · August 23, 2022, 7:53am

Hi @robynton
Yiou have to create a superuser for the postgres
cchq monolith django-manage make_superuser blablabla@domaine.com
Just make sure to replace blablabla

robynton · August 23, 2022, 9:36pm

Thank you very much @Babacar_Gueye ! Just missed that line from documentation, my fault.