Update-config failing

Developers
41

Good point, thanks! Here's the output of the sqlmigrate users 0016_hqapikey

BEGIN;
--
-- Create model HQApiKey
--
CREATE TABLE "users_hqapikey" ("id" serial NOT NULL PRIMARY KEY, "key" varchar(128) NOT NULL, "name" varchar(255) NOT NULL, "created" timestamp with time zone NOT NULL, "ip_allowlist" inet[] NOT NULL, "user_id" integer NOT NULL);
--
-- MIGRATION NOW PERFORMS OPERATION THAT CANNOT BE WRITTEN AS SQL:
-- Raw Python operation
--
ALTER TABLE "users_hqapikey" ADD CONSTRAINT "users_hqapikey_user_id_84116bfd_fk_auth_user_id" FOREIGN KEY ("user_id") REFERENCES "auth_user" ("id") DEFERRABLE INITIALLY DEFERRED;
CREATE INDEX "users_hqapikey_key_919a7182" ON "users_hqapikey" ("key");
CREATE INDEX "users_hqapikey_key_919a7182_like" ON "users_hqapikey" ("key" varchar_pattern_ops);
CREATE INDEX "users_hqapikey_user_id_84116bfd" ON "users_hqapikey" ("user_id");
COMMIT;

...the SQL appeared to run without any output at all:

1898×828 44.6 KB

The table was created, constraints are there etc. so all appeared to run OK
I'll run the rest of the migrations now

EDIT done, the rest of the migrations and the users_hqapikey insert appeared to work fine, it has updated the table.
If I'm following correctly, I should now be going back to this post
The output of that SQL indicates that there are records in the tastypie_apikey table but hqwebapp_apikeysettings does not exist.
Output of showmigrations:

I see it's listed there (hqwebapp.0004_apikeysettings)
If I get the SQL using python manage.py sqlmigrate hqwebapp 0004_apikeysettings:

BEGIN;
--
-- Create model ApiKeySettings
--
CREATE TABLE "hqwebapp_apikeysettings" ("id" serial NOT NULL PRIMARY KEY, "ip_whitelist" inet[] NULL, "api_key_id" integer NOT NULL UNIQUE);
ALTER TABLE "hqwebapp_apikeysettings" ADD CONSTRAINT "hqwebapp_apikeysetti_api_key_id_7bd96ab8_fk_tastypie_" FOREIGN KEY ("api_key_id") REFERENCES "tastypie_apikey" ("id") DEFERRABLE INITIALLY DEFERRED;
COMMIT;

...and run that manually at the dbshell, it does a ROLLBACK at the end...
OK weird... I've just checked and the table now exists as does that constraint...
So at this point back to this post I have 35 records in tastypie_apikey and 0 in hqwebapp_apikeysettings
For reference, I've included the output of the migrate_multi users 0016_hqapikey and migrate_multi here:

42

Update - hopefully we're nearly through the woods. A deploy was successful and the server is back up though formplayer is down. I have this in the formplayer log:

2021-05-10 13:46:59.964 [main] ERROR o.s.boot.SpringApplication - Application run failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Invocation of init method failed; nested exception is org.flywaydb.core.internal.exception.FlywaySqlException:
Unable to obtain connection from database (jdbc:postgresql://197.x.x.x:5432/formplayer?prepareThreshold=0) for user 'commcare': FATAL: no pg_hba.conf entry for host "197.x.x.x", user "commcare", database "formplayer", SSL off
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SQL State  : 28000
Error Code : 0
Message    : FATAL: no pg_hba.conf entry for host "197.x.x.x", user "commcare", database "formplayer", SSL off

I don't want to manually edit pg_hba.conf - I assume Ansible is handling it. Are we perhaps missing a new setting in one of the yml files? Thanks for the help @Simon_Kelly

43

Glad we're making progress! The DB connection issue is configured by commcare-cloud. You can run this command:

cchq monolith ap deploy_postgres.yml

It should give you a diff of changes to make and then ask to confirm them before actually making the changes.

1 Like
44

OK, changes implemented - the only items I wasn't sure of were related to pgbouncer - perhaps you know if they're an issue or not:

TASK [postgresql : Set vm.dirty_background_bytes] *******************************************************************************************************************************************
skipping: [197.x.x.x]
[WARNING]: Could not match supplied host pattern, ignoring: pgbouncer

TASK [pgbouncer stop and disable excess processes] ******************************************************************************************************************************************
fatal: [197.x.x.x]: FAILED! => {"msg": "to count backwards make stride negative"}
...ignoring

TASK [pgbouncer remove conf for excess processes] *******************************************************************************************************************************************
fatal: [197.x.x.x]: FAILED! => {"msg": "to count backwards make stride negative"}
...ignoring

TASK [pgbouncer remove unix socket directory for excess processes] **************************************************************************************************************************
fatal: [197.x.x.x]: FAILED! => {"msg": "to count backwards make stride negative"}
...ignoring

Regardless, that got formplayer up, ta muchly!

45

Thanks for the logs, I haven't seen that before so we'll take a look and try figure out why they failed. Those are all cleanup tasks which I don't expect would do anything on your setup anyway so probably OK that they failed. One question, was that during the 'check' run or during the actual playbook run where it's applying the changes?

1 Like
46

It was during the check run.
I'll be doing a monolith deploy on a new server from scratch probably later today and will post anything interesting I come across.

1 Like
47

I'll be doing a monolith deploy on a new server from scratch probably later today and will post anything interesting I come across.

Please do. We just recently went over the monolith installation instructions and fixed a few issues that had recently arisen, so please let us know if you run into any friction there.

1 Like