Datadog - alternatives?

Developers
1

Are there alternatives to Datadog that I can integrate with the system? I have to say Datadog's billing system is atrocious. I don't know if it's always been like this, but for example, none of the items listed on their pricing page appear on my monthly bill, instead we are billed for the same host under different headings and a bunch of on-demand metrics.
If anyone has any insight into how to manage datadog billing, I'm all ears, otherwise we're likley just going to can it and go with something else.

I see many others complaining about the same thing:

2

Hi Ed

We currently only support Datadog and Prometheus (which you would have to host yourself).

It should be fairly straight forward to implement a new provider in CommCare that would only give you 'custom' metrics. You'd then need to setup your own VM / service monitoring.

Let me know if you'd like more details on adding a new provider.

1 Like
3

Excellent, I've used Prometheus on other projects, I think I may just switch to this. What is required config wise to enable it from Commcare?
Thanks for the prompt response Simon.

4

Hi Ed

It's been a while since we've had an environment use Prometheus so there are likely to be some teething issues.

If you're using commcare-cloud you can create a new YAML file in your environment folder as follows:

[env]/prometheus.yml

prometheus_monitoring_enabled: True

Configure your inventory:
inventory.ini

[prometheus]
10.x.x.x  # host to run prometheus

[graphana]
10.x.x.x  # host to run graphana

[alertmanager]
10.x.x.x  # host to run alertmanager

[prometheus_proxy]
10.x.x.x  # host to host the prometheus / graphana nginx proxy

Run the following commcare-cloud commands:

cchq <env> ap deploy_prometheus.yml
cchq <env> ap deploy_commcare.yml
cchq <env> update-supervisor-confs

I will also draw up a doc explaining more about how we've got it set up and reply back here once I've got that.

5

Thanks Simon, I’ll give that a go on our QA and report back.

9

Hi Simon,
While working on installing Prometheus server on a client machine, I came across this forum post.
It appears that the role cloudalchemy.node-exporter has been renamed on Ansible Galaxy, and it now uses an underscore instead of a hyphen.

The necessary change should be applied to the file /src/commcare_cloud/ansible/deploy_prometheus.yml in the task named "Prometheus Node Exporter" as shown below:

`- name: Prometheus Node Exporter
  hosts: all
  become: true
  tasks:
    - import_role:
        name: cloudalchemy.node_exporter
  tags: node_exporter
`

It would be great if someone could update the main repository with this change. Aside from this issue, the rest of the installation process runs smoothly.

1 Like
10

@demis08 I see that the cloudalchemy.node_exporter role has been deprecated in favour of the node exporter role in prometheus-community/ansible.

I think the new role is a drop in replacement but it would be great if someone using prometheus could test it:

  1. Install the collection:

    ansible-galaxy collection install prometheus.prometheus

  2. Update deploy_prometheus.yml:

    -         name: cloudalchemy.node_exporter
+         name: prometheus.prometheus.node_exporter
2 Likes