The basic workflow is to add -K or --ask-become-pass to the command
line which will prompt you for the 'sudo' password.
If you're also using ansible vault you will want --ask-vault-pass as
well. Supplying two passwords is inconvenient so you can actually store the
'become' password in the vault file as "ansible_become_pass".
Simon Kelly
Director of Server Engineer | Dimagi
···
On 30 October 2017 at 13:11, wrote:
Simon, this is fantastic—thank you. Very quick one:
- I ran ansible-playbook -i inventories/monolith -e
'@vars/dev/dev_private.yml' -e '@vars/dev/dev_public.yml' deploy_stack.yml
-u root --tags=users
- Then ansible-playbook -i inventories/monolith -e
'@vars/dev/dev_private.yml' -e '@vars/dev/dev_public.yml' deploy_stack.yml
-u ansible
on a brand new box, but get:
TASK [apt] ************************************************************
fatal: [159.203.132.215]: FAILED! => {"changed": false, "failed": true,
"module_stderr": "Shared connection to 159.203.132.215 closed.\r\n",
"module_stdout": "sudo: a password is required\r\n", "msg": "MODULE
FAILURE", "rc": 1}
Was I misinterpreting the suggestion? Or is there a change that must be
made either to the ansible user setup or with visudo to get this working?
Conceivably, we could set up password-less sudo privs for ansible like
this: ansible ALL=(ALL) NOPASSWD:ALL
Best,
Taylor
On Monday, October 30, 2017 at 5:18:30 AM UTC-3, Simon Kelly wrote:
Hey answers inline:
- Right now, we're only able to get ansible to run if we set a
FORMPLAYER_INTERNAL_AUTH_KEY to an empty string, but since we later have
issues with Formplayer I'm worried that this isn't the right move. What
should we do here?
That is a shared secret between HQ and Formplayer which allows
formplayer to authenticate API calls to HQ. You should set it to a secret
key with a reasonable amount of entropy.
- With which user do you run ansible on a Digital Ocean box? (I
noticed an "ansible" user gets configured, but presumably you've got to
first run as root. Are you meant to run once as root and then after it
fails subsequently run as ansible?)
It's usually necessary to run it as a privileged user once to setup the
user accounts. We would normally run just the 'users' tag as root and from
then on we can run it as the 'ansible' user.
- Where do you clone the commcare-hq-deploy repo and with which
user do you run fab deploy?
For running deploy you can have the repo anywhere you like as long as
you have access to the machines you're deploying to from there. For deploy
you don't require any external dependencies (other than those defined in
requirements.txt.
And then one higher-level question to make sure I'm understanding things
correctly: It seems as though deployment on a new box requires the cloning
and configuration of three separate repos: (1) commcarehq-ansible
https://github.com/dimagi/commcarehq-ansible, (2) commcare-hq-deploy
https://github.com/dimagi/commcare-hq-deploy, and (3) formplayer
https://github.com/dimagi/formplayer. If we're trying to get this
down to a single repo (or at least a single README) can you describe the
relationship between these three repos theoretically and in
user/directory terms? It would be amazing to know where you clone each of
them and how you run them in relation to each other. We've been doing
all the ansible stuff as root and the django stuff as cchq, but that
may not be right.
None of these repo's need to be anywhere specific. The formplayer repo in
particular should not be needed for anything. Currently when we deploy
formplayer it pulls the latest version from our Jenkins build server.
The other two are related as follows (at least for our setup):
- ansible repo: stores all the ansible playbooks and vault files
(with all the secret keys etc).
- deploy repo: has the deploy scripts and the ansible inventory files
(required for both deploy and ansible)
We always setup one of the VMs in our clusters as a 'control' machine
from where we can run the ansible playbooks (and also normal deploys if we
want). Once you have an account on this machine you can follow the
instructions in the readme: https://github.com/dim
agi/commcarehq-ansible#setting-up-a-dev-account-on-ansible-
control-machine
This should setup the 'commcare-hq-deploy' repo and also the python
virtualenv for ansible. It will also create some bash aliases that make it
easier to run ansible playbooks.
I hope that answer's your questions. Let me know if you have follow ups.
Cheers
Simon
--
You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Simon, I have been following along and have tried the ansible load as
well...
the core ansible load went very well......few hiccups along the way but
made it through it.
I am at the deployment stage but I have ran into this error.....
File
"/home/harperjo/commcarehq-ansible/commcare-hq-deploy/fab/fabfile.py", line
272, in read_inventory_file
return get_inventory(filename).get_group_dict()
AttributeError: 'Inventory' object has no attribute 'get_group_dict'
harperjo@GCS-1:~/commcarehq-ansible/commcare-hq-deploy$
this is on my fav dev deploy execution. I am wondering if i missed
something in the fabfile that points to my ansible environment file?
your thoughts?
···
On Monday, October 30, 2017 at 5:54:41 AM UTC-7, Simon Kelly wrote:
>
> The basic workflow is to add `-K` or `--ask-become-pass` to the command
> line which will prompt you for the 'sudo' password.
>
> If you're also using ansible vault you will want `--ask-vault-pass` as
> well. Supplying two passwords is inconvenient so you can actually store the
> 'become' password in the vault file as "ansible_become_pass".
>
>
> Simon Kelly
> Director of Server Engineer | Dimagi
>
> On 30 October 2017 at 13:11, <tay...@openfn.org > wrote:
>
>> Simon, this is fantastic—thank you. Very quick one:
>>
>> 1. I ran ansible-playbook -i inventories/monolith -e
>> '@vars/dev/dev_private.yml' -e '@vars/dev/dev_public.yml' deploy_stack.yml
>> -u root --tags=users
>> 2. Then ansible-playbook -i inventories/monolith -e
>> '@vars/dev/dev_private.yml' -e '@vars/dev/dev_public.yml' deploy_stack.yml
>> -u ansible
>>
>> on a brand new box, but get:
>>
>> TASK [apt]
>> ***********************************************************************************************************************************************************************************
>> fatal: [159.203.132.215]: FAILED! => {"changed": false, "failed": true,
>> "module_stderr": "Shared connection to 159.203.132.215 closed.\r\n",
>> "module_stdout": "sudo: a password is required\r\n", "msg": "MODULE
>> FAILURE", "rc": 1}
>>
>> Was I misinterpreting the suggestion? Or is there a change that must be
>> made either to the ansible user setup or with visudo to get this working?
>> Conceivably, we could set up password-less sudo privs for ansible like
>> this: ansible ALL=(ALL) NOPASSWD:ALL
>>
>> Best,
>> Taylor
>>
>> On Monday, October 30, 2017 at 5:18:30 AM UTC-3, Simon Kelly wrote:
>>>
>>> Hey answers inline:
>>>
>>>>
>>>> 1. Right now, we're only able to get ansible to run if we set a
>>>> FORMPLAYER_INTERNAL_AUTH_KEY to an empty string, but since we later have
>>>> issues with Formplayer I'm worried that this isn't the right move. What
>>>> should we do here?
>>>>
>>>> That is a shared secret between HQ and Formplayer which allows
>>> formplayer to authenticate API calls to HQ. You should set it to a secret
>>> key with a reasonable amount of entropy.
>>>
>>>>
>>>> 1. With which user do you run ansible on a Digital Ocean box? (I
>>>> noticed an "ansible" user gets configured, but presumably you've got to
>>>> first run as root. Are you meant to run once as root and then after it
>>>> fails subsequently run as ansible?)
>>>>
>>>> It's usually necessary to run it as a privileged user once to setup the
>>> user accounts. We would normally run just the 'users' tag as root and from
>>> then on we can run it as the 'ansible' user.
>>>
>>>>
>>>> 1. Where do you clone the commcare-hq-deploy repo and with which
>>>> user do you run fab deploy?
>>>>
>>>> For running deploy you can have the repo anywhere you like as long as
>>> you have access to the machines you're deploying to from there. For deploy
>>> you don't require any external dependencies (other than those defined in
>>> requirements.txt.
>>>
>>>
>>>> And then one higher-level question to make sure I'm understanding
>>>> things correctly: It seems as though deployment on a new box requires the
>>>> cloning and configuration of three separate repos: (1)
>>>> commcarehq-ansible , (2)
>>>> commcare-hq-deploy , and
>>>> (3) formplayer . If we're trying
>>>> to get this down to a single repo (or at least a single README) can you
>>>> describe the relationship between these three repos theoretically *and*
>>>> in user/directory terms? It would be amazing to know where you clone each
>>>> of them and how you run them in relation to each other. We've been
>>>> doing all the ansible stuff as root and the django stuff as cchq, but
>>>> that may not be right.
>>>>
>>>
>>> None of these repo's need to be anywhere specific. The formplayer repo
>>> in particular should not be needed for anything. Currently when we deploy
>>> formplayer it pulls the latest version from our Jenkins build server.
>>>
>>> The other two are related as follows (at least for our setup):
>>>
>>> - ansible repo: stores all the ansible playbooks and vault files
>>> (with all the secret keys etc).
>>> - deploy repo: has the deploy scripts and the ansible inventory
>>> files (required for both deploy and ansible)
>>>
>>> We always setup one of the VMs in our clusters as a 'control' machine
>>> from where we can run the ansible playbooks (and also normal deploys if we
>>> want). Once you have an account on this machine you can follow the
>>> instructions in the readme:
>>> https://github.com/dimagi/commcarehq-ansible#setting-up-a-dev-account-on-ansible-control-machine
>>>
>>> This should setup the 'commcare-hq-deploy' repo and also the python
>>> virtualenv for ansible. It will also create some bash aliases that make it
>>> easier to run ansible playbooks.
>>>
>>> I hope that answer's your questions. Let me know if you have follow ups.
>>>
>>> Cheers
>>> Simon
>>>
>>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "CommCare Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to commcare-developers+unsubscribe@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
Hi John
I'm glad it's gone well. The error you're seeing is a bug after we upgraded
too a new version of ansible. If you update the deploy repo to the latest
version and make sure your requirements are up to date it should work.
Cheers
Simon
Simon, I have been following along and have tried the ansible load as
well...
the core ansible load went very well......few hiccups along the way but
made it through it.
I am at the deployment stage but I have ran into this error.....
File "/home/harperjo/commcarehq-ansible/commcare-hq-deploy/fab/fabfile.py",
line 272, in read_inventory_file
return get_inventory(filename).get_group_dict()
AttributeError: 'Inventory' object has no attribute 'get_group_dict'
harperjo@GCS-1:~/commcarehq-ansible/commcare-hq-deploy$
this is on my fav dev deploy execution. I am wondering if i missed
something in the fabfile that points to my ansible environment file?
your thoughts?
···
On 02 Dec 2017 02:42, "John Harper" wrote:
On Monday, October 30, 2017 at 5:54:41 AM UTC-7, Simon Kelly wrote:
The basic workflow is to add -K or --ask-become-pass to the command
line which will prompt you for the 'sudo' password.
If you're also using ansible vault you will want --ask-vault-pass as
well. Supplying two passwords is inconvenient so you can actually store the
'become' password in the vault file as "ansible_become_pass".
Simon Kelly
Director of Server Engineer | Dimagi
On 30 October 2017 at 13:11, tay...@openfn.org wrote:
Simon, this is fantastic—thank you. Very quick one:
- I ran ansible-playbook -i inventories/monolith -e
'@vars/dev/dev_private.yml' -e '@vars/dev/dev_public.yml' deploy_stack.yml
-u root --tags=users
- Then ansible-playbook -i inventories/monolith -e
'@vars/dev/dev_private.yml' -e '@vars/dev/dev_public.yml' deploy_stack.yml
-u ansible
on a brand new box, but get:
TASK [apt] ************************************************************
fatal: [159.203.132.215]: FAILED! => {"changed": false, "failed": true,
"module_stderr": "Shared connection to 159.203.132.215 closed.\r\n",
"module_stdout": "sudo: a password is required\r\n", "msg": "MODULE
FAILURE", "rc": 1}
Was I misinterpreting the suggestion? Or is there a change that must be
made either to the ansible user setup or with visudo to get this working?
Conceivably, we could set up password-less sudo privs for ansible like
this: ansible ALL=(ALL) NOPASSWD:ALL
Best,
Taylor
On Monday, October 30, 2017 at 5:18:30 AM UTC-3, Simon Kelly wrote:
Hey answers inline:
- Right now, we're only able to get ansible to run if we set a
FORMPLAYER_INTERNAL_AUTH_KEY to an empty string, but since we later have
issues with Formplayer I'm worried that this isn't the right move. What
should we do here?
That is a shared secret between HQ and Formplayer which allows
formplayer to authenticate API calls to HQ. You should set it to a secret
key with a reasonable amount of entropy.
- With which user do you run ansible on a Digital Ocean box? (I
noticed an "ansible" user gets configured, but presumably you've got to
first run as root. Are you meant to run once as root and then after it
fails subsequently run as ansible?)
It's usually necessary to run it as a privileged user once to setup the
user accounts. We would normally run just the 'users' tag as root and from
then on we can run it as the 'ansible' user.
- Where do you clone the commcare-hq-deploy repo and with which
user do you run fab deploy?
For running deploy you can have the repo anywhere you like as long as
you have access to the machines you're deploying to from there. For deploy
you don't require any external dependencies (other than those defined in
requirements.txt.
And then one higher-level question to make sure I'm understanding
things correctly: It seems as though deployment on a new box requires the
cloning and configuration of three separate repos: (1)
commcarehq-ansible https://github.com/dimagi/commcarehq-ansible, (2)
commcare-hq-deploy https://github.com/dimagi/commcare-hq-deploy, and
(3) formplayer https://github.com/dimagi/formplayer. If we're trying
to get this down to a single repo (or at least a single README) can you
describe the relationship between these three repos theoretically and
in user/directory terms? It would be amazing to know where you clone each
of them and how you run them in relation to each other. We've been
doing all the ansible stuff as root and the django stuff as cchq, but
that may not be right.
None of these repo's need to be anywhere specific. The formplayer repo
in particular should not be needed for anything. Currently when we deploy
formplayer it pulls the latest version from our Jenkins build server.
The other two are related as follows (at least for our setup):
- ansible repo: stores all the ansible playbooks and vault files
(with all the secret keys etc).
- deploy repo: has the deploy scripts and the ansible inventory
files (required for both deploy and ansible)
We always setup one of the VMs in our clusters as a 'control' machine
from where we can run the ansible playbooks (and also normal deploys if we
want). Once you have an account on this machine you can follow the
instructions in the readme: https://github.com/dim
agi/commcarehq-ansible#setting-up-a-dev-account-on-ansible-
control-machine
This should setup the 'commcare-hq-deploy' repo and also the python
virtualenv for ansible. It will also create some bash aliases that make it
easier to run ansible playbooks.
I hope that answer's your questions. Let me know if you have follow ups.
Cheers
Simon
--
You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
okay thank
I will check the ansible version I have on the control and my host machines
to ensure it is at the correct level.
should i just blow away my commcare-hq-deploy folder and pull the new one
down using wget?
ill have to configure my fabfile.py, environment.yml and others again
correct?
···
On Friday, December 1, 2017 at 10:46:34 PM UTC-7, Simon Kelly wrote:
>
> Hi John
>
> I'm glad it's gone well. The error you're seeing is a bug after we
> upgraded too a new version of ansible. If you update the deploy repo to the
> latest version and make sure your requirements are up to date it should
> work.
>
> Cheers
> Simon
>
> On 02 Dec 2017 02:42, "John Harper" <john....@grableservices.com > wrote:
>
> Simon, I have been following along and have tried the ansible load as
> well...
>
> the core ansible load went very well......few hiccups along the way but
> made it through it.
>
> I am at the deployment stage but I have ran into this error.....
>
> File
> "/home/harperjo/commcarehq-ansible/commcare-hq-deploy/fab/fabfile.py", line
> 272, in read_inventory_file
> return get_inventory(filename).get_group_dict()
> AttributeError: 'Inventory' object has no attribute 'get_group_dict'
> harperjo@GCS-1:~/commcarehq-ansible/commcare-hq-deploy$
>
>
> this is on my fav dev deploy execution. I am wondering if i missed
> something in the fabfile that points to my ansible environment file?
>
> your thoughts?
>
>
> On Monday, October 30, 2017 at 5:54:41 AM UTC-7, Simon Kelly wrote:
>
>> The basic workflow is to add `-K` or `--ask-become-pass` to the command
>> line which will prompt you for the 'sudo' password.
>>
>> If you're also using ansible vault you will want `--ask-vault-pass` as
>> well. Supplying two passwords is inconvenient so you can actually store the
>> 'become' password in the vault file as "ansible_become_pass".
>>
>>
>> Simon Kelly
>> Director of Server Engineer | Dimagi
>>
>> On 30 October 2017 at 13:11, wrote:
>>
>>> Simon, this is fantastic—thank you. Very quick one:
>>>
>>> 1. I ran ansible-playbook -i inventories/monolith -e
>>> '@vars/dev/dev_private.yml' -e '@vars/dev/dev_public.yml' deploy_stack.yml
>>> -u root --tags=users
>>> 2. Then ansible-playbook -i inventories/monolith -e
>>> '@vars/dev/dev_private.yml' -e '@vars/dev/dev_public.yml' deploy_stack.yml
>>> -u ansible
>>>
>>> on a brand new box, but get:
>>>
>>> TASK [apt]
>>> ***********************************************************************************************************************************************************************************
>>> fatal: [159.203.132.215]: FAILED! => {"changed": false, "failed": true,
>>> "module_stderr": "Shared connection to 159.203.132.215 closed.\r\n",
>>> "module_stdout": "sudo: a password is required\r\n", "msg": "MODULE
>>> FAILURE", "rc": 1}
>>>
>>> Was I misinterpreting the suggestion? Or is there a change that must be
>>> made either to the ansible user setup or with visudo to get this working?
>>> Conceivably, we could set up password-less sudo privs for ansible like
>>> this: ansible ALL=(ALL) NOPASSWD:ALL
>>>
>>> Best,
>>> Taylor
>>>
>>> On Monday, October 30, 2017 at 5:18:30 AM UTC-3, Simon Kelly wrote:
>>>>
>>>> Hey answers inline:
>>>>
>>>>>
>>>>> 1. Right now, we're only able to get ansible to run if we set a
>>>>> FORMPLAYER_INTERNAL_AUTH_KEY to an empty string, but since we later have
>>>>> issues with Formplayer I'm worried that this isn't the right move. What
>>>>> should we do here?
>>>>>
>>>>> That is a shared secret between HQ and Formplayer which allows
>>>> formplayer to authenticate API calls to HQ. You should set it to a secret
>>>> key with a reasonable amount of entropy.
>>>>
>>>>>
>>>>> 1. With which user do you run ansible on a Digital Ocean box? (I
>>>>> noticed an "ansible" user gets configured, but presumably you've got to
>>>>> first run as root. Are you meant to run once as root and then after it
>>>>> fails subsequently run as ansible?)
>>>>>
>>>>> It's usually necessary to run it as a privileged user once to setup
>>>> the user accounts. We would normally run just the 'users' tag as root and
>>>> from then on we can run it as the 'ansible' user.
>>>>
>>>>>
>>>>> 1. Where do you clone the commcare-hq-deploy repo and with which
>>>>> user do you run fab deploy?
>>>>>
>>>>> For running deploy you can have the repo anywhere you like as long as
>>>> you have access to the machines you're deploying to from there. For deploy
>>>> you don't require any external dependencies (other than those defined in
>>>> requirements.txt.
>>>>
>>>>
>>>>> And then one higher-level question to make sure I'm understanding
>>>>> things correctly: It seems as though deployment on a new box requires the
>>>>> cloning and configuration of three separate repos: (1)
>>>>> commcarehq-ansible ,
>>>>> (2) commcare-hq-deploy ,
>>>>> and (3) formplayer . If we're
>>>>> trying to get this down to a single repo (or at least a single README) can
>>>>> you describe the relationship between these three repos theoretically
>>>>> *and* in user/directory terms? It would be amazing to know where you
>>>>> clone each of them and how you run them in relation to each other. We've
>>>>> been doing all the ansible stuff as root and the django stuff as cchq,
>>>>> but that may not be right.
>>>>>
>>>>
>>>> None of these repo's need to be anywhere specific. The formplayer repo
>>>> in particular should not be needed for anything. Currently when we deploy
>>>> formplayer it pulls the latest version from our Jenkins build server.
>>>>
>>>> The other two are related as follows (at least for our setup):
>>>>
>>>> - ansible repo: stores all the ansible playbooks and vault files
>>>> (with all the secret keys etc).
>>>> - deploy repo: has the deploy scripts and the ansible inventory
>>>> files (required for both deploy and ansible)
>>>>
>>>> We always setup one of the VMs in our clusters as a 'control' machine
>>>> from where we can run the ansible playbooks (and also normal deploys if we
>>>> want). Once you have an account on this machine you can follow the
>>>> instructions in the readme:
>>>> https://github.com/dimagi/commcarehq-ansible#setting-up-a-dev-account-on-ansible-control-machine
>>>>
>>>> This should setup the 'commcare-hq-deploy' repo and also the python
>>>> virtualenv for ansible. It will also create some bash aliases that make it
>>>> easier to run ansible playbooks.
>>>>
>>>> I hope that answer's your questions. Let me know if you have follow ups.
>>>>
>>>> Cheers
>>>> Simon
>>>>
>>>> --
>>>
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "CommCare Developers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to commcare-developers+unsubscribe@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "CommCare Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to commcare-developers+unsubscribe@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>
>
>
You shouldn't need to blow it away. You should just be able to do a git
pull to got the latest version.
···
On 02 Dec 2017 16:31, "John Harper" wrote:
okay thank
I will check the ansible version I have on the control and my host
machines to ensure it is at the correct level.
should i just blow away my commcare-hq-deploy folder and pull the new one
down using wget?
ill have to configure my fabfile.py, environment.yml and others again
correct?
On Friday, December 1, 2017 at 10:46:34 PM UTC-7, Simon Kelly wrote:
Hi John
I'm glad it's gone well. The error you're seeing is a bug after we
upgraded too a new version of ansible. If you update the deploy repo to the
latest version and make sure your requirements are up to date it should
work.
Cheers
Simon
On 02 Dec 2017 02:42, "John Harper" john....@grableservices.com wrote:
Simon, I have been following along and have tried the ansible load as
well...
the core ansible load went very well......few hiccups along the way but
made it through it.
I am at the deployment stage but I have ran into this error.....
File "/home/harperjo/commcarehq-ansible/commcare-hq-deploy/fab/fabfile.py",
line 272, in read_inventory_file
return get_inventory(filename).get_group_dict()
AttributeError: 'Inventory' object has no attribute 'get_group_dict'
harperjo@GCS-1:~/commcarehq-ansible/commcare-hq-deploy$
this is on my fav dev deploy execution. I am wondering if i missed
something in the fabfile that points to my ansible environment file?
your thoughts?
On Monday, October 30, 2017 at 5:54:41 AM UTC-7, Simon Kelly wrote:
The basic workflow is to add -K or --ask-become-pass to the command
line which will prompt you for the 'sudo' password.
If you're also using ansible vault you will want --ask-vault-pass as
well. Supplying two passwords is inconvenient so you can actually store the
'become' password in the vault file as "ansible_become_pass".
Simon Kelly
Director of Server Engineer | Dimagi
On 30 October 2017 at 13:11, tay...@openfn.org wrote:
Simon, this is fantastic—thank you. Very quick one:
- I ran ansible-playbook -i inventories/monolith -e
'@vars/dev/dev_private.yml' -e '@vars/dev/dev_public.yml' deploy_stack.yml
-u root --tags=users
- Then ansible-playbook -i inventories/monolith -e
'@vars/dev/dev_private.yml' -e '@vars/dev/dev_public.yml' deploy_stack.yml
-u ansible
on a brand new box, but get:
TASK [apt] ************************************************************
fatal: [159.203.132.215]: FAILED! => {"changed": false, "failed": true,
"module_stderr": "Shared connection to 159.203.132.215 closed.\r\n",
"module_stdout": "sudo: a password is required\r\n", "msg": "MODULE
FAILURE", "rc": 1}
Was I misinterpreting the suggestion? Or is there a change that must be
made either to the ansible user setup or with visudo to get this working?
Conceivably, we could set up password-less sudo privs for ansible like
this: ansible ALL=(ALL) NOPASSWD:ALL
Best,
Taylor
On Monday, October 30, 2017 at 5:18:30 AM UTC-3, Simon Kelly wrote:
Hey answers inline:
- Right now, we're only able to get ansible to run if we set a
FORMPLAYER_INTERNAL_AUTH_KEY to an empty string, but since we later have
issues with Formplayer I'm worried that this isn't the right move. What
should we do here?
That is a shared secret between HQ and Formplayer which allows
formplayer to authenticate API calls to HQ. You should set it to a secret
key with a reasonable amount of entropy.
- With which user do you run ansible on a Digital Ocean box? (I
noticed an "ansible" user gets configured, but presumably you've got to
first run as root. Are you meant to run once as root and then after it
fails subsequently run as ansible?)
It's usually necessary to run it as a privileged user once to setup
the user accounts. We would normally run just the 'users' tag as root and
from then on we can run it as the 'ansible' user.
- Where do you clone the commcare-hq-deploy repo and with which
user do you run fab deploy?
For running deploy you can have the repo anywhere you like as long as
you have access to the machines you're deploying to from there. For deploy
you don't require any external dependencies (other than those defined in
requirements.txt.
And then one higher-level question to make sure I'm understanding
things correctly: It seems as though deployment on a new box requires the
cloning and configuration of three separate repos: (1)
commcarehq-ansible https://github.com/dimagi/commcarehq-ansible,
(2) commcare-hq-deploy https://github.com/dimagi/commcare-hq-deploy,
and (3) formplayer https://github.com/dimagi/formplayer. If we're
trying to get this down to a single repo (or at least a single README) can
you describe the relationship between these three repos theoretically
and in user/directory terms? It would be amazing to know where you
clone each of them and how you run them in relation to each other. We've
been doing all the ansible stuff as root and the django stuff as cchq,
but that may not be right.
None of these repo's need to be anywhere specific. The formplayer repo
in particular should not be needed for anything. Currently when we deploy
formplayer it pulls the latest version from our Jenkins build server.
The other two are related as follows (at least for our setup):
- ansible repo: stores all the ansible playbooks and vault files
(with all the secret keys etc).
- deploy repo: has the deploy scripts and the ansible inventory
files (required for both deploy and ansible)
We always setup one of the VMs in our clusters as a 'control' machine
from where we can run the ansible playbooks (and also normal deploys if we
want). Once you have an account on this machine you can follow the
instructions in the readme: https://github.com/dim
agi/commcarehq-ansible#setting-up-a-dev-account-on-ansible-
control-machine
This should setup the 'commcare-hq-deploy' repo and also the python
virtualenv for ansible. It will also create some bash aliases that make it
easier to run ansible playbooks.
I hope that answer's your questions. Let me know if you have follow
ups.
Cheers
Simon
--
You received this message because you are subscribed to the Google
Groups "CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Though if you've made changes to the fabfile you'll want to commit those in
a branch and then merge the h updated master branch into your environment
branch.
···
On 03 Dec 2017 08:21, "Simon Kelly" wrote:
You shouldn't need to blow it away. You should just be able to do a git
pull to got the latest version.
On 02 Dec 2017 16:31, "John Harper" john.harper@grableservices.com wrote:
okay thank
I will check the ansible version I have on the control and my host
machines to ensure it is at the correct level.
should i just blow away my commcare-hq-deploy folder and pull the new one
down using wget?
ill have to configure my fabfile.py, environment.yml and others again
correct?
On Friday, December 1, 2017 at 10:46:34 PM UTC-7, Simon Kelly wrote:
Hi John
I'm glad it's gone well. The error you're seeing is a bug after we
upgraded too a new version of ansible. If you update the deploy repo to the
latest version and make sure your requirements are up to date it should
work.
Cheers
Simon
On 02 Dec 2017 02:42, "John Harper" john....@grableservices.com wrote:
Simon, I have been following along and have tried the ansible load as
well...
the core ansible load went very well......few hiccups along the way but
made it through it.
I am at the deployment stage but I have ran into this error.....
File "/home/harperjo/commcarehq-ansible/commcare-hq-deploy/fab/fabfile.py",
line 272, in read_inventory_file
return get_inventory(filename).get_group_dict()
AttributeError: 'Inventory' object has no attribute 'get_group_dict'
harperjo@GCS-1:~/commcarehq-ansible/commcare-hq-deploy$
this is on my fav dev deploy execution. I am wondering if i missed
something in the fabfile that points to my ansible environment file?
your thoughts?
On Monday, October 30, 2017 at 5:54:41 AM UTC-7, Simon Kelly wrote:
The basic workflow is to add -K or --ask-become-pass to the command
line which will prompt you for the 'sudo' password.
If you're also using ansible vault you will want --ask-vault-pass as
well. Supplying two passwords is inconvenient so you can actually store the
'become' password in the vault file as "ansible_become_pass".
Simon Kelly
Director of Server Engineer | Dimagi
On 30 October 2017 at 13:11, tay...@openfn.org wrote:
Simon, this is fantastic—thank you. Very quick one:
- I ran ansible-playbook -i inventories/monolith -e
'@vars/dev/dev_private.yml' -e '@vars/dev/dev_public.yml' deploy_stack.yml
-u root --tags=users
- Then ansible-playbook -i inventories/monolith -e
'@vars/dev/dev_private.yml' -e '@vars/dev/dev_public.yml' deploy_stack.yml
-u ansible
on a brand new box, but get:
TASK [apt] ******************************
fatal: [159.203.132.215]: FAILED! => {"changed": false, "failed":
true, "module_stderr": "Shared connection to 159.203.132.215 closed.\r\n",
"module_stdout": "sudo: a password is required\r\n", "msg": "MODULE
FAILURE", "rc": 1}
Was I misinterpreting the suggestion? Or is there a change that must
be made either to the ansible user setup or with visudo to get this
working? Conceivably, we could set up password-less sudo privs for ansible
like this: ansible ALL=(ALL) NOPASSWD:ALL
Best,
Taylor
On Monday, October 30, 2017 at 5:18:30 AM UTC-3, Simon Kelly wrote:
Hey answers inline:
- Right now, we're only able to get ansible to run if we set a
FORMPLAYER_INTERNAL_AUTH_KEY to an empty string, but since we later have
issues with Formplayer I'm worried that this isn't the right move. What
should we do here?
That is a shared secret between HQ and Formplayer which allows
formplayer to authenticate API calls to HQ. You should set it to a secret
key with a reasonable amount of entropy.
- With which user do you run ansible on a Digital Ocean box? (I
noticed an "ansible" user gets configured, but presumably you've got to
first run as root. Are you meant to run once as root and then after it
fails subsequently run as ansible?)
It's usually necessary to run it as a privileged user once to setup
the user accounts. We would normally run just the 'users' tag as root and
from then on we can run it as the 'ansible' user.
- Where do you clone the commcare-hq-deploy repo and with which
user do you run fab deploy?
For running deploy you can have the repo anywhere you like as long
as you have access to the machines you're deploying to from there. For
deploy you don't require any external dependencies (other than those
defined in requirements.txt.
And then one higher-level question to make sure I'm understanding
things correctly: It seems as though deployment on a new box requires the
cloning and configuration of three separate repos: (1)
commcarehq-ansible https://github.com/dimagi/commcarehq-ansible,
(2) commcare-hq-deploy
https://github.com/dimagi/commcare-hq-deploy, and (3) formplayer
https://github.com/dimagi/formplayer. If we're trying to get this
down to a single repo (or at least a single README) can you describe the
relationship between these three repos theoretically and in
user/directory terms? It would be amazing to know where you clone each of
them and how you run them in relation to each other. We've been
doing all the ansible stuff as root and the django stuff as cchq,
but that may not be right.
None of these repo's need to be anywhere specific. The formplayer
repo in particular should not be needed for anything. Currently when we
deploy formplayer it pulls the latest version from our Jenkins build server.
The other two are related as follows (at least for our setup):
- ansible repo: stores all the ansible playbooks and vault files
(with all the secret keys etc).
- deploy repo: has the deploy scripts and the ansible inventory
files (required for both deploy and ansible)
We always setup one of the VMs in our clusters as a 'control' machine
from where we can run the ansible playbooks (and also normal deploys if we
want). Once you have an account on this machine you can follow the
instructions in the readme: https://github.com/dim
agi/commcarehq-ansible#setting-up-a-dev-account-on-ansible-c
ontrol-machine
This should setup the 'commcare-hq-deploy' repo and also the python
virtualenv for ansible. It will also create some bash aliases that make it
easier to run ansible playbooks.
I hope that answer's your questions. Let me know if you have follow
ups.
Cheers
Simon
--
You received this message because you are subscribed to the Google
Groups "CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
well I am getting closer on this one. Deploy on a dev platform seemed to go well until the following task.
File "/home/harperjo/commcarehq-ansible/commcare-hq-deploy/fab/fabfile.py", line 544, in conditionally_stop_pillows_and_celery_during_migrate
execute_with_timing(supervisor.stop_pillows)
File "/home/harperjo/commcarehq-ansible/commcare-hq-deploy/fab/utils.py", line 37, in execute_with_timing
execute(fn, *args, **kwargs)
File "/home/harperjo/.local/lib/python2.7/site-packages/fabric/tasks.py", line 418, in execute
error(err, exception=d['results'])
File "/home/harperjo/.local/lib/python2.7/site-packages/fabric/utils.py", line 358, in error
return func(message)
File "/home/harperjo/.local/lib/python2.7/site-packages/fabric/utils.py", line 54, in abort
raise env.abort_exception(msg)
Exception: One or more hosts failed while executing task 'stop_pillows'
Underlying exception:
sudo() received nonzero return code 123 while executing!
Requested: scripts/supervisor-group-ctl stop pillowtop
Executed: sudo -S -p 'sudo password:' -H -u "cchq" /bin/bash -l -c "cd /home/cchq/www/dev/releases/2017-12-14_17.01 >/dev/null && scripts/supervisor-group-ctl stop pillowtop"
i tried to run the script on the host server itself at the following folder
/home/cchq/www/dev/releases/current/dev/scripts
./supervisor-group-ctl stop pillowtop
this was the output.....
root@monolith:/home/cchq/www/dev/current/scripts# ./supervisor-group-ctl stop pillowtop
Error: stop requires a process name
stop Stop a process
stop :* Stop all processes in a group
stop Stop multiple processes or groups
stop all Stop all processes
root@monolith:/home/cchq/www/dev/current/scripts#
not seeing a reason for the error......any ideas on this one. I have ran fab dev clean_releases and tried it again.......still fails here.