Error by Uploading Cases/Forms : DatabaseError: error with no message from the libpq

Dear CommCare Developers community,
I’ve the latest version of commcareHQ installed in a digital ocean node
Ubuntu 14.04 server (4gb RAM):
All Service are up and running Postgresql 9.6 (the same problem with 9.4).
couchdb. Elasticsearch, kafka (0.10.1.0 and the same problem with 0.10.1.1),
zookeeper and clery are up and running.
After uploading 2-3 forms to the server I become the following errors:
1- exceptions.AttributeError --> See the log here

Doc Type: XFormInstance
Domain: endev
DB Type: form-sql
DB Name: form-sql
Is Deletion: False
Date Created: 15 Feb 2017 16:51
Date of Last Attempt: 15 Feb 2017 16:51
Date of Next Attempt: 15 Feb 2017 16:56
Pillow: AppFormSubmissionTrackerPillow
Total Attempts: 1
Current Attempt: 1
Error Type: exceptions.AttributeError
Stack Trace: can't set attribute

File "./corehq/ex-submodules/pillowtop/pillow/interface.py", line 109, in
process_with_error_handling
self.process_change(change)
File "./corehq/ex-submodules/pillowtop/pillow/interface.py", line 168, in
process_change
self._processor.process_change(self, change)
File "./corehq/ex-submodules/pillowtop/processors/form.py", line 19, in
process_change
doc = change.get_document()
File "./corehq/ex-submodules/pillowtop/feed/interface.py", line 60, in
get_document
self.document = self.document_store.get_document(self.id)
File "/home/demo/commcare-hq/corehq/form_processor/document_stores.py",
line 22, in get_document
return
self.form_accessors.get_form(doc_id).to_json(include_attachments=True)
File
"/home/demo/commcare-hq/corehq/form_processor/interfaces/dbaccessors.py",
line 111, in get_form
return self.db_accessor.get_form(form_id)
File
"/home/demo/commcare-hq/corehq/form_processor/backends/sql/dbaccessors.py",
line 233, in get_form
return FormAccessorSQL.get_forms([form_id])[0]
File
"/home/demo/commcare-hq/corehq/form_processor/backends/sql/dbaccessors.py",
line 1135, in getitem
self._fetch_all()
File
"/home/demo/commcare-hq/corehq/form_processor/backends/sql/dbaccessors.py",
line 1128, in _fetch_all
self._result_cache = list(self.queryset)
File
"/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/models/query.py",
line 1253, in iter
setattr(instance, column, values[pos])

2- Database Error: error with no message from libpq... I get this error
randomly and restarting Kafka the other services didn't help. Rebooting the
server is helping but only for a 2 or 3 submissions...
Any ideas how to solve this error?

Traceback (most recent call last):

File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in
_bootstrap

self.run()

File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run

self._target(*self._args, **self._kwargs)

File "./corehq/ex-submodules/pillowtop/pillow/interface.py", line 82, in
run

self.process_changes(since=self.get_last_checkpoint_sequence(), 

forever=True)

File "./corehq/ex-submodules/pillowtop/pillow/interface.py", line 94, in
process_changes

self.process_with_error_handling(change)

File "./corehq/ex-submodules/pillowtop/pillow/interface.py", line 111, in
process_with_error_handling

handle_pillow_error(self, change, ex)

File "./corehq/ex-submodules/pillowtop/pillow/interface.py", line 181, in
handle_pillow_error

error.save()

File
"/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/models/base.py",
line 708, in save

force_update=force_update, update_fields=update_fields)

File
"/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/models/base.py",
line 736, in save_base

updated = self._save_table(raw, cls, force_insert, force_update, using, 

update_fields)

File
"/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/models/base.py",
line 820, in _save_table

result = self._do_insert(cls._base_manager, using, fields, update_pk, 

raw)

File
"/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/models/base.py",
line 859, in _do_insert

using=using, raw=raw)

File
"/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/models/manager.py",
line 122, in manager_method

return getattr(self.get_queryset(), name)(*args, **kwargs)

File
"/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/models/query.py",
line 1039, in _insert

return query.get_compiler(using=using).execute_sql(return_id)

File
"/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py",
line 1060, in execute_sql

cursor.execute(sql, params)

File
"/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/backends/utils.py",
line 79, in execute

return super(CursorDebugWrapper, self).execute(sql, params)

File
"/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/backends/utils.py",
line 64, in execute

return self.cursor.execute(sql, params)

File
"/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/utils.py",
line 95, in exit

six.reraise(dj_exc_type, dj_exc_value, traceback)

File
"/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/backends/utils.py",
line 64, in execute

return self.cursor.execute(sql, params)

DatabaseError: error with no message from the libpq

Any help or suggestions would be appreciated!

Thanks

-Speedy

Hi there,
after reinstalling commcarehq using the dockerized services I still have
the same error.
It seems that the error is produced from postgres: please find attached the
postgres log.

postgres_1 | WARNING: there is already a transaction in progress

postgres_1 | ERROR: duplicate key value violates unique constraint
"pillowtop_djangopillowcheckpoint_pkey"

postgres_1 | DETAIL: Key
(checkpoint_id)=(fluff.OpmUserFluffPillow.testhq) already exists.

postgres_1 | STATEMENT: INSERT INTO
"pillowtop_djangopillowcheckpoint" ("checkpoint_id", "sequence",
"timestamp", "old_sequence", "sequence_format") VALUES
('fluff.OpmUserFluffPillow.testhq', '0',
'2017-04-09T10:34:03.889459'::timestamp, NULL, 'text')

restarting postgres and the other services didn't help. Any idea how to fix
that?
Best
-Speedy

··· Am Mittwoch, 15. Februar 2017 18:39:32 UTC+1 schrieb Speedy: > > Dear CommCare Developers community, > I’ve the latest version of commcareHQ installed in a digital ocean node > Ubuntu 14.04 server (4gb RAM): > All Service are up and running Postgresql 9.6 (the same problem with > 9.4). couchdb. Elasticsearch, kafka (0.10.1.0 and the same problem with > 0.10.1.1), zookeeper and clery are up and running. > After uploading 2-3 forms to the server I become the following errors: > 1- exceptions.AttributeError --> See the log here > > Doc Type: XFormInstance > Domain: endev > DB Type: form-sql > DB Name: form-sql > Is Deletion: False > Date Created: 15 Feb 2017 16:51 > Date of Last Attempt: 15 Feb 2017 16:51 > Date of Next Attempt: 15 Feb 2017 16:56 > Pillow: AppFormSubmissionTrackerPillow > Total Attempts: 1 > Current Attempt: 1 > Error Type: exceptions.AttributeError > Stack Trace: can't set attribute > > File "./corehq/ex-submodules/pillowtop/pillow/interface.py", line 109, in > process_with_error_handling > self.process_change(change) > File "./corehq/ex-submodules/pillowtop/pillow/interface.py", line 168, in > process_change > self._processor.process_change(self, change) > File "./corehq/ex-submodules/pillowtop/processors/form.py", line 19, in > process_change > doc = change.get_document() > File "./corehq/ex-submodules/pillowtop/feed/interface.py", line 60, in > get_document > self.document = self.document_store.get_document(self.id) > File "/home/demo/commcare-hq/corehq/form_processor/document_stores.py", > line 22, in get_document > return > self.form_accessors.get_form(doc_id).to_json(include_attachments=True) > File > "/home/demo/commcare-hq/corehq/form_processor/interfaces/dbaccessors.py", > line 111, in get_form > return self.db_accessor.get_form(form_id) > File > "/home/demo/commcare-hq/corehq/form_processor/backends/sql/dbaccessors.py", > line 233, in get_form > return FormAccessorSQL.get_forms([form_id])[0] > File > "/home/demo/commcare-hq/corehq/form_processor/backends/sql/dbaccessors.py", > line 1135, in __getitem__ > self._fetch_all() > File > "/home/demo/commcare-hq/corehq/form_processor/backends/sql/dbaccessors.py", > line 1128, in _fetch_all > self._result_cache = list(self.queryset) > File > "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/models/query.py", > line 1253, in __iter__ > setattr(instance, column, values[pos]) > > 2- Database Error: error with no message from libpq... I get this error > randomly and restarting Kafka the other services didn't help. Rebooting the > server is helping but only for a 2 or 3 submissions... > Any ideas how to solve this error? > > > > > > > Traceback (most recent call last): > > File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in > _bootstrap > > self.run() > > File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run > > self._target(*self._args, **self._kwargs) > > File "./corehq/ex-submodules/pillowtop/pillow/interface.py", line 82, in > run > > self.process_changes(since=self.get_last_checkpoint_sequence(), > forever=True) > > File "./corehq/ex-submodules/pillowtop/pillow/interface.py", line 94, in > process_changes > > self.process_with_error_handling(change) > > File "./corehq/ex-submodules/pillowtop/pillow/interface.py", line 111, > in process_with_error_handling > > handle_pillow_error(self, change, ex) > > File "./corehq/ex-submodules/pillowtop/pillow/interface.py", line 181, > in handle_pillow_error > > error.save() > > File > "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/models/base.py", > line 708, in save > > force_update=force_update, update_fields=update_fields) > > File > "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/models/base.py", > line 736, in save_base > > updated = self._save_table(raw, cls, force_insert, force_update, > using, update_fields) > > File > "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/models/base.py", > line 820, in _save_table > > result = self._do_insert(cls._base_manager, using, fields, update_pk, > raw) > > File > "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/models/base.py", > line 859, in _do_insert > > using=using, raw=raw) > > File > "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/models/manager.py", > line 122, in manager_method > > return getattr(self.get_queryset(), name)(*args, **kwargs) > > File > "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/models/query.py", > line 1039, in _insert > > return query.get_compiler(using=using).execute_sql(return_id) > > File > "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", > line 1060, in execute_sql > > cursor.execute(sql, params) > > File > "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/backends/utils.py", > line 79, in execute > > return super(CursorDebugWrapper, self).execute(sql, params) > > File > "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/backends/utils.py", > line 64, in execute > > return self.cursor.execute(sql, params) > > File > "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/utils.py", > line 95, in __exit__ > > six.reraise(dj_exc_type, dj_exc_value, traceback) > > File > "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/backends/utils.py", > line 64, in execute > > return self.cursor.execute(sql, params) > > DatabaseError: error with no message from the libpq > > Any help or suggestions would be appreciated! > > Thanks > > -Speedy >

Hi Speedy,

It looks like your database got into a funny state. You can try deleting
the duplicate record that is causing the error to occur (this shouldn't
have any negative side effects on a dev environment).

Ben

··· On Sun, Apr 9, 2017 at 6:55 AM, Speedy wrote:

Hi there,
after reinstalling commcarehq using the dockerized services I still have
the same error.
It seems that the error is produced from postgres: please find attached
the postgres log.

postgres_1 | WARNING: there is already a transaction in progress

postgres_1 | ERROR: duplicate key value violates unique constraint
"pillowtop_djangopillowcheckpoint_pkey"

postgres_1 | DETAIL: Key (checkpoint_id)=(fluff.OpmUserFluffPillow.testhq)
already exists.

postgres_1 | STATEMENT: INSERT INTO "pillowtop_djangopillowcheckpoint"
("checkpoint_id", "sequence", "timestamp", "old_sequence",
"sequence_format") VALUES ('fluff.OpmUserFluffPillow.testhq', '0',
'2017-04-09T10:34:03.889459'::timestamp, NULL, 'text')

restarting postgres and the other services didn't help. Any idea how to
fix that?
Best
-Speedy
Am Mittwoch, 15. Februar 2017 18:39:32 UTC+1 schrieb Speedy:

Dear CommCare Developers community,
I’ve the latest version of commcareHQ installed in a digital ocean node
Ubuntu 14.04 server (4gb RAM):
All Service are up and running Postgresql 9.6 (the same problem with
9.4). couchdb. Elasticsearch, kafka (0.10.1.0 and the same problem with
0.10.1.1), zookeeper and clery are up and running.
After uploading 2-3 forms to the server I become the following errors:
1- exceptions.AttributeError --> See the log here

Doc Type: XFormInstance
Domain: endev
DB Type: form-sql
DB Name: form-sql
Is Deletion: False
Date Created: 15 Feb 2017 16:51
Date of Last Attempt: 15 Feb 2017 16:51
Date of Next Attempt: 15 Feb 2017 16:56
Pillow: AppFormSubmissionTrackerPillow
Total Attempts: 1
Current Attempt: 1
Error Type: exceptions.AttributeError
Stack Trace: can't set attribute

File "./corehq/ex-submodules/pillowtop/pillow/interface.py", line 109,
in process_with_error_handling
self.process_change(change)
File "./corehq/ex-submodules/pillowtop/pillow/interface.py", line 168,
in process_change
self._processor.process_change(self, change)
File "./corehq/ex-submodules/pillowtop/processors/form.py", line 19, in
process_change
doc = change.get_document()
File "./corehq/ex-submodules/pillowtop/feed/interface.py", line 60, in
get_document
self.document = self.document_store.get_document(self.id)
File "/home/demo/commcare-hq/corehq/form_processor/document_stores.py",
line 22, in get_document
return self.form_accessors.get_form(doc_id).to_json(include_attachm
ents=True)
File "/home/demo/commcare-hq/corehq/form_processor/interfaces/dbaccessors.py",
line 111, in get_form
return self.db_accessor.get_form(form_id)
File "/home/demo/commcare-hq/corehq/form_processor/backends/sql/dbaccessors.py",
line 233, in get_form
return FormAccessorSQL.get_forms([form_id])[0]
File "/home/demo/commcare-hq/corehq/form_processor/backends/sql/dbaccessors.py",
line 1135, in getitem
self._fetch_all()
File "/home/demo/commcare-hq/corehq/form_processor/backends/sql/dbaccessors.py",
line 1128, in _fetch_all
self._result_cache = list(self.queryset)
File "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packa
ges/django/db/models/query.py", line 1253, in iter
setattr(instance, column, values[pos])

2- Database Error: error with no message from libpq... I get this error
randomly and restarting Kafka the other services didn't help. Rebooting the
server is helping but only for a 2 or 3 submissions...
Any ideas how to solve this error?

Traceback (most recent call last):

File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in
_bootstrap

self.run()

File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run

self._target(*self._args, **self._kwargs)

File "./corehq/ex-submodules/pillowtop/pillow/interface.py", line 82,
in run

self.process_changes(since=self.get_last_checkpoint_sequence(),

forever=True)

File "./corehq/ex-submodules/pillowtop/pillow/interface.py", line 94,
in process_changes

self.process_with_error_handling(change)

File "./corehq/ex-submodules/pillowtop/pillow/interface.py", line 111,
in process_with_error_handling

handle_pillow_error(self, change, ex)

File "./corehq/ex-submodules/pillowtop/pillow/interface.py", line 181,
in handle_pillow_error

error.save()

File "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/models/base.py",
line 708, in save

force_update=force_update, update_fields=update_fields)

File "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/models/base.py",
line 736, in save_base

updated = self._save_table(raw, cls, force_insert, force_update,

using, update_fields)

File "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/models/base.py",
line 820, in _save_table

result = self._do_insert(cls._base_manager, using, fields,

update_pk, raw)

File "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/models/base.py",
line 859, in _do_insert

using=using, raw=raw)

File "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packa
ges/django/db/models/manager.py", line 122, in manager_method

return getattr(self.get_queryset(), name)(*args, **kwargs)

File "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packa
ges/django/db/models/query.py", line 1039, in _insert

return query.get_compiler(using=using).execute_sql(return_id)

File "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packa
ges/django/db/models/sql/compiler.py", line 1060, in execute_sql

cursor.execute(sql, params)

File "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packa
ges/django/db/backends/utils.py", line 79, in execute

return super(CursorDebugWrapper, self).execute(sql, params)

File "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packa
ges/django/db/backends/utils.py", line 64, in execute

return self.cursor.execute(sql, params)

File "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packages/django/db/utils.py",
line 95, in exit

six.reraise(dj_exc_type, dj_exc_value, traceback)

File "/home/demo/.virtualenvs/cchq/local/lib/python2.7/site-packa
ges/django/db/backends/utils.py", line 64, in execute

return self.cursor.execute(sql, params)

DatabaseError: error with no message from the libpq

Any help or suggestions would be appreciated!

Thanks

-Speedy

--


You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.