Hey guys,
Hope all is well. Let me preface this with a thank you—I know you've got a
lot going on and don't rely on ansible monolith deployments for your core
work, so I realize that any help you provide here is going above and
beyond. Thank you for that!
My objective is to get ansible-playbook -i inventories/monolith -u root -e
'@vars/dev/dev_private.yml' -e '@vars/dev/dev_public.yml' deploy_stack.yml
running on a freshly provisioned Ubuntu 14.04.5 LTS (GNU/Linux
3.13.0-125-generic x86_64) droplet with 2 gigs of memory.
While I think that's a solid goal for the whole CommCare open-source
community, I'd like to disclose that we've also got a client at Open
Function that wants to connect CommCare to another system using OpenFn, but
CommCare needs to be hosted on their servers due to regulatory issues.
Note that we made a couple of changes vagrant and edited some ansible
scripts. You can see this work here:
Getting it to work by taylordowns2000 · Pull Request #1 · rorymckinley/commcare-sandbox · GitHub. One
significant change is that we are running the vagrant stuff as root.
To the issues:
Issue #1:
TASK [couchdb : Set CouchDB username and password]
···
***************************** ok: [165.227.172.214] => (item={u'username': u'commcarehq', u'name': u'commcarehq', u'is_https': False, u'host': u'165.227.172.214', u'password': u'commcarehq', u'port': 5984}) failed: [165.227.172.214] (item={u'username': u'commcarehq', u'name': u'commcarehq__users', u'is_https': False, u'host': u'165.227.172.214', u'password': u'commcarehq', u'port': 5984}) => {"cache_control": "must-revalidate", "content": "{\"error\":\"unauthorized\",\"reason\":\"You are not a server admin.\"}\n", "content_length": "64", "content_type": "text/plain; charset=utf-8", "date": "Thu, 05 Oct 2017 11:10:34 GMT", "failed": true, "item": {"host": "165.227.172.214", "is_https": false, "name": "commcarehq__users", "password": "commcarehq", "port": 5984, "username": "commcarehq"}, "msg": "Status code was not [200]: HTTP Error 401: Unauthorized", "redirected": false, "server": "CouchDB/1.6.1 (Erlang OTP/R16B03)", "status": 401, "url": "http://165.227.172.214:5984/_config/admins/commcarehq"} to retry, use: --limit @/vagrant/ansible/deploy_stack.retryPLAY RECAP
165.227.172.214 : ok=135 changed=90 unreachable=0
failed=1
Possible solution 1: This task runs twice, but each user in "items" has
the same username and password. The failure can be stepped over, as we
don't need to (and can't) set up two different couchdb users with
commcarehq:commcarehq on the same box.
*Issue #2&3: *For both couchdb2 and redis, monit fails. After I reboot the
system and start monit manually they pass and redis is running, but
couchdb2 still shows "Execution failed". After another system reboot, and
manually starting monit, both now show as running and being monitored.
monit status: Process 'couchdb2'
status Execution failed
monitoring status Monitored
data collected Thu, 05 Oct 2017 11:59:49
TASK [couchdb2 : monit]
fatal: [165.227.172.214]: FAILED! => {"changed": false, "failed": true,
"msg": "couchdb2 process not presently configured with monit", "name":
"couchdb2", "state": "monitored"}
RUNNING HANDLER [monit : reload monit]
to retry, use: --limit @/vagrant/ansible/deploy_stack.retry
PLAY RECAP
165.227.172.214 : ok=36 changed=20 unreachable=0
failed=1
TASK [redis : monit]
fatal: [165.227.172.214]: FAILED! => {"changed": false, "failed": true,
"msg": "redis process not presently configured with monit", "name":
"redis", "state": "monitored"}
RUNNING HANDLER [monit : reload monit]
RUNNING HANDLER [redis : restart redis]
RUNNING HANDLER [redis : restart rsyslog]
to retry, use: --limit @/vagrant/ansible/deploy_stack.retry
PLAY RECAP
165.227.172.214 : ok=17 changed=10 unreachable=0 failed=1
Issue 4:
TASK [touchforms : Touchforms user]
An exception occurred during task execution. To see the full traceback, use
-vvv. The error was: ImportError: No module named django
fatal: [165.227.172.214 -> 165.227.172.214]: FAILED! => {"changed": false,
"failed": true, "module_stderr": "Traceback (most recent call last):\n
File "/tmp/ansible_iUft9p/ansible_module_django_user.py", line 144, in
\n main()\n File
"/tmp/ansible_iUft9p/ansible_module_django_user.py", line 125, in main\n
user.create_user()\n File
"/tmp/ansible_iUft9p/ansible_module_django_user.py", line 84, in
create_user\n superuser=repr(self.superuser),\n File
"/usr/local/lib/python2.7/dist-packages/sh.py", line 1427, in call\n
return RunningCommand(cmd, call_args, stdin, stdout, stderr)\n File
"/usr/local/lib/python2.7/dist-packages/sh.py", line 774, in init\n
self.wait()\n File "/usr/local/lib/python2.7/dist-packages/sh.py",
line 792, in wait\n self.handle_command_exit_code(exit_code)\n File
"/usr/local/lib/python2.7/dist-packages/sh.py", line 815, in
handle_command_exit_code\n raise exc\nsh.ErrorReturnCode_1: \n\n RAN:
/home/cchq/www/dev/current/python_env/bin/python manage.py shell
--plain\n\n STDOUT:\n\n\n STDERR:\nTraceback (most recent call last):\n
File "manage.py", line 9, in \n import django\nImportError: No
module named django\n\n", "module_stdout": "Traceback (most recent call
last):\n File "manage.py", line 9, in \n import
django\nImportError: No module named django\n\n", "msg": "MODULE FAILURE"}
to retry, use: --limit @/vagrant/ansible/deploy_stack.retry
Possible solution: Here, we need to SSH in and then:
su - cchq
cd www/dev/current
source python_env/bin/activate
pip install -r requirements/requirements.txt
At this point the whole ansible playbook succeeds, but when we visit our
IP, we get the maintenance page and see this in the nginx logs:
2017/10/05 13:56:16 [error] 1064#1064: *18 connect() failed (111:
Connection refused) while connecting to upstream, client: 186.106.251.211,
server: 165.227.172.214, request: "GET /favicon.ico HTTP/1.1", upstream:
"http://165.227.172.214:9010/favicon.ico", host: "165.227.172.214",
referrer: "https://165.227.172.214/solutions/"
After activating the python_env we run runserver as cchq:
./manage.py runserver 0.0.0.0:9010
File "/home/cchq/www/dev/current/python_env/local/lib/python2.7/site-packages/django/db/backends/postgresql/base.py", line 176, in get_new_connection
connection = Database.connect(**conn_params)
File "/home/cchq/www/dev/current/python_env/local/lib/python2.7/site-packages/psycopg2/init.py", line 130, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
django.db.utils.OperationalError: ERROR: pgbouncer cannot connect to server
At this point, we're wondering:
- Why isn't the server running itself?
- And how do we get it to run?
Best,
Taylor
-
are very few.