Import cases

Hey,

I want import cases from production to my local machine and I have problem
with this function. I get error in my local celeryd attached below.

I have the newest changes form master branch and I clear a .pyc files. Any
suggestions?

Task corehq.apps.importer.tasks.bulk_import_async with id
25aab2ff-b40a-45d2-a332-de93dbf30cf5 raised exception:
‘DatabaseError(“DatabaseWrapper objects created in a thread can only be
used in that same thread. The object with alias ‘default’ was created in
thread id 140736101340096 and this is thread id 4331731376.”,)’

Task was called with args: (<corehq.apps.importer.util.ImporterConfig
object at 0x10a064ad0>, u’opm’, ‘255471afdce14ba6bbaf69c16cb04fe0’) kwargs:
{}.

The contents of the full traceback was:

Traceback (most recent call last):
File “/usr/local/lib/python2.7/site-packages/celery/app/trace.py”, line
283, in trace_task
uuid, retval, SUCCESS, request=task_request,
File “/usr/local/lib/python2.7/site-packages/celery/backends/base.py”,
line 256, in store_result
request=request, **kwargs)
File
"/usr/local/lib/python2.7/site-packages/djcelery/backends/database.py",
line 29, in _store_result
traceback=traceback, children=self.current_task_children(request),
File “/usr/local/lib/python2.7/site-packages/djcelery/managers.py”, line
42, in _inner
return fun(*args, **kwargs)
File “/usr/local/lib/python2.7/site-packages/djcelery/managers.py”, line
181, in store_result
’meta’: {‘children’: children}})
File “/usr/local/lib/python2.7/site-packages/djcelery/managers.py”, line
87, in update_or_create
return get_queryset(self).update_or_create(**kwargs)
File “/usr/local/lib/python2.7/site-packages/djcelery/managers.py”, line
70, in update_or_create
obj, created = self.get_or_create(**kwargs)
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”,
line 405, in get_or_create
return self.get(**lookup), False
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”,
line 328, in get
num = len(clone)
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”,
line 144, in len
self._fetch_all()
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”,
line 965, in _fetch_all
self._result_cache = list(self.iterator())
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”,
line 238, in iterator
results = compiler.execute_sql()
File
"/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py",
line 838, in execute_sql
cursor = self.connection.cursor()
File
"/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py",
line 160, in cursor
self.validate_thread_sharing()
File
"/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py",
line 421, in validate_thread_sharing
% (self.alias, self._thread_ident, thread.get_ident()))
DatabaseError: DatabaseWrapper objects created in a thread can only be used
in that same thread. The object with alias ‘default’ was created in thread
id 140736101340096 and this is thread id 4331731376.

··· -- Just to let you know,

Hi Lukasz,

What version of celery and djcelery are you running locally (pip freeze)?
If it’s different from the versions in requirements/requirements.txt you’ll
want to update.

Also, doublecheck the setting you have in localsettings
for CELERY_RESULT_BACKEND. Locally I have it set
to ‘djcelery.backends.database:DatabaseBackend’.

Also note that the preferred way of starting a celery worker is now
"./manage.py celery worker" as “./manage.py celeryd” has been deprecated.

Lastly, if you still can’t get asynchronous tasks to work, you can probably
just set CELERY_ALWAYS_EAGER to True in your localsettings and they should
run synchronously without the need to start a worker process.

-Giovanni

··· On Thu, Oct 6, 2016 at 7:35 AM, Łukasz Wyszomirski < lwyszomirski@soldevelo.com> wrote:

Hey,

I want import cases from production to my local machine and I have problem
with this function. I get error in my local celeryd attached below.

I have the newest changes form master branch and I clear a .pyc files. Any
suggestions?

Task corehq.apps.importer.tasks.bulk_import_async with id
25aab2ff-b40a-45d2-a332-de93dbf30cf5 raised exception:
‘DatabaseError(“DatabaseWrapper objects created in a thread can only be
used in that same thread. The object with alias ‘default’ was created in
thread id 140736101340096 and this is thread id 4331731376.”,)’

Task was called with args: (<corehq.apps.importer.util.ImporterConfig
object at 0x10a064ad0>, u’opm’, ‘255471afdce14ba6bbaf69c16cb04fe0’)
kwargs: {}.

The contents of the full traceback was:

Traceback (most recent call last):
File “/usr/local/lib/python2.7/site-packages/celery/app/trace.py”, line
283, in trace_task
uuid, retval, SUCCESS, request=task_request,
File “/usr/local/lib/python2.7/site-packages/celery/backends/base.py”,
line 256, in store_result
request=request, **kwargs)
File “/usr/local/lib/python2.7/site-packages/djcelery/backends/database.py”,
line 29, in _store_result
traceback=traceback, children=self.current_task_children(request),
File “/usr/local/lib/python2.7/site-packages/djcelery/managers.py”,
line 42, in _inner
return fun(*args, **kwargs)
File “/usr/local/lib/python2.7/site-packages/djcelery/managers.py”,
line 181, in store_result
’meta’: {‘children’: children}})
File “/usr/local/lib/python2.7/site-packages/djcelery/managers.py”,
line 87, in update_or_create
return get_queryset(self).update_or_create(**kwargs)
File “/usr/local/lib/python2.7/site-packages/djcelery/managers.py”,
line 70, in update_or_create
obj, created = self.get_or_create(**kwargs)
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”,
line 405, in get_or_create
return self.get(**lookup), False
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”,
line 328, in get
num = len(clone)
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”,
line 144, in len
self._fetch_all()
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”,
line 965, in _fetch_all
self._result_cache = list(self.iterator())
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”,
line 238, in iterator
results = compiler.execute_sql()
File “/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py”,
line 838, in execute_sql
cursor = self.connection.cursor()
File “/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py”,
line 160, in cursor
self.validate_thread_sharing()
File “/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py”,
line 421, in validate_thread_sharing
% (self.alias, self._thread_ident, thread.get_ident()))
DatabaseError: DatabaseWrapper objects created in a thread can only be
used in that same thread. The object with alias ‘default’ was created in
thread id 140736101340096 and this is thread id 4331731376.


Just to let you know,


You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hey,

My versions:
celery==3.1.18
django-celery==3.1.16

I set CELERY_RESULT_BACKEND as you mention and this not help, still the same issue.

I have the same situation when I set CELERY_ALWAYS_EAGER to True, but I have attached error on browser.

Thanks,
Łukasz

··· On 6 October 2016 at 13:58:30, Giovanni Capalbo (gcapalbo@dimagi.com) wrote:

Hi Lukasz,

What version of celery and djcelery are you running locally (pip freeze)? If it’s different from the versions in requirements/requirements.txt you’ll want to update.

Also, doublecheck the setting you have in localsettings for CELERY_RESULT_BACKEND. Locally I have it set to ‘djcelery.backends.database:DatabaseBackend’.

Also note that the preferred way of starting a celery worker is now “./manage.py celery worker” as “./manage.py celeryd” has been deprecated.

Lastly, if you still can’t get asynchronous tasks to work, you can probably just set CELERY_ALWAYS_EAGER to True in your localsettings and they should run synchronously without the need to start a worker process.

-Giovanni

On Thu, Oct 6, 2016 at 7:35 AM, Łukasz Wyszomirski lwyszomirski@soldevelo.com wrote:
Hey,

I want import cases from production to my local machine and I have problem with this function. I get error in my local celeryd attached below.

I have the newest changes form master branch and I clear a .pyc files. Any suggestions?

Task corehq.apps.importer.tasks.bulk_import_async with id 25aab2ff-b40a-45d2-a332-de93dbf30cf5 raised exception:
‘DatabaseError(“DatabaseWrapper objects created in a thread can only be used in that same thread. The object with alias ‘default’ was created in thread id 140736101340096 and this is thread id 4331731376.”,)’

Task was called with args: (<corehq.apps.importer.util.ImporterConfig object at 0x10a064ad0>, u’opm’, ‘255471afdce14ba6bbaf69c16cb04fe0’) kwargs: {}.

The contents of the full traceback was:

Traceback (most recent call last):
File “/usr/local/lib/python2.7/site-packages/celery/app/trace.py”, line 283, in trace_task
uuid, retval, SUCCESS, request=task_request,
File “/usr/local/lib/python2.7/site-packages/celery/backends/base.py”, line 256, in store_result
request=request, **kwargs)
File “/usr/local/lib/python2.7/site-packages/djcelery/backends/database.py”, line 29, in _store_result
traceback=traceback, children=self.current_task_children(request),
File “/usr/local/lib/python2.7/site-packages/djcelery/managers.py”, line 42, in _inner
return fun(*args, **kwargs)
File “/usr/local/lib/python2.7/site-packages/djcelery/managers.py”, line 181, in store_result
’meta’: {‘children’: children}})
File “/usr/local/lib/python2.7/site-packages/djcelery/managers.py”, line 87, in update_or_create
return get_queryset(self).update_or_create(**kwargs)
File “/usr/local/lib/python2.7/site-packages/djcelery/managers.py”, line 70, in update_or_create
obj, created = self.get_or_create(**kwargs)
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”, line 405, in get_or_create
return self.get(**lookup), False
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”, line 328, in get
num = len(clone)
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”, line 144, in len
self._fetch_all()
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”, line 965, in _fetch_all
self._result_cache = list(self.iterator())
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”, line 238, in iterator
results = compiler.execute_sql()
File “/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py”, line 838, in execute_sql
cursor = self.connection.cursor()
File “/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py”, line 160, in cursor
self.validate_thread_sharing()
File “/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py”, line 421, in validate_thread_sharing
% (self.alias, self._thread_ident, thread.get_ident()))
DatabaseError: DatabaseWrapper objects created in a thread can only be used in that same thread. The object with alias ‘default’ was created in thread id 140736101340096 and this is thread id 4331731376.


Just to let you know,


You received this message because you are subscribed to the Google Groups “CommCare Developers” group.
To unsubscribe from this group and stop receiving emails from it, send an email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups “CommCare Developers” group.
To unsubscribe from this group and stop receiving emails from it, send an email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Strange, I have the same versions locally and it’s working. Have you tried
running “pip install -r requirements/requirements.txt”? Specifically, we
just upgraded to Django 1.8 and I wonder if something is off with the
package versions you have locally.

Note that you’ll need to restart the celery worker after doing so.

··· On Thu, Oct 6, 2016 at 8:20 AM, Łukasz Wyszomirski < lwyszomirski@soldevelo.com> wrote:

Hey,

My versions:
celery==3.1.18
django-celery==3.1.16

I set CELERY_RESULT_BACKEND as you mention and this not help, still the
same issue.

I have the same situation when I set CELERY_ALWAYS_EAGER to True, but I
have attached error on browser.

Thanks,
Łukasz

On 6 October 2016 at 13:58:30, Giovanni Capalbo (gcapalbo@dimagi.com) wrote:

Hi Lukasz,

What version of celery and djcelery are you running locally (pip freeze)?
If it’s different from the versions in requirements/requirements.txt you’ll
want to update.

Also, doublecheck the setting you have in localsettings
for CELERY_RESULT_BACKEND. Locally I have it set to ‘djcelery.backends.
database:DatabaseBackend’.

Also note that the preferred way of starting a celery worker is now
"./manage.py celery worker" as “./manage.py celeryd” has been deprecated.

Lastly, if you still can’t get asynchronous tasks to work, you can
probably just set CELERY_ALWAYS_EAGER to True in your localsettings and
they should run synchronously without the need to start a worker process.

-Giovanni

On Thu, Oct 6, 2016 at 7:35 AM, Łukasz Wyszomirski < lwyszomirski@soldevelo.com> wrote:

Hey,

I want import cases from production to my local machine and I have
problem with this function. I get error in my local celeryd attached below.

I have the newest changes form master branch and I clear a .pyc files.
Any suggestions?

Task corehq.apps.importer.tasks.bulk_import_async with id
25aab2ff-b40a-45d2-a332-de93dbf30cf5 raised exception:
‘DatabaseError(“DatabaseWrapper objects created in a thread can only be
used in that same thread. The object with alias ‘default’ was created in
thread id 140736101340096 and this is thread id 4331731376.”,)’

Task was called with args: (<corehq.apps.importer.util.ImporterConfig
object at 0x10a064ad0>, u’opm’, ‘255471afdce14ba6bbaf69c16cb04fe0’)
kwargs: {}.

The contents of the full traceback was:

Traceback (most recent call last):
File “/usr/local/lib/python2.7/site-packages/celery/app/trace.py”,
line 283, in trace_task
uuid, retval, SUCCESS, request=task_request,
File “/usr/local/lib/python2.7/site-packages/celery/backends/base.py”,
line 256, in store_result
request=request, **kwargs)
File “/usr/local/lib/python2.7/site-packages/djcelery/backends/database.py”,
line 29, in _store_result
traceback=traceback, children=self.current_task_children(request),
File “/usr/local/lib/python2.7/site-packages/djcelery/managers.py”,
line 42, in _inner
return fun(*args, **kwargs)
File “/usr/local/lib/python2.7/site-packages/djcelery/managers.py”,
line 181, in store_result
’meta’: {‘children’: children}})
File “/usr/local/lib/python2.7/site-packages/djcelery/managers.py”,
line 87, in update_or_create
return get_queryset(self).update_or_create(**kwargs)
File “/usr/local/lib/python2.7/site-packages/djcelery/managers.py”,
line 70, in update_or_create
obj, created = self.get_or_create(**kwargs)
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”,
line 405, in get_or_create
return self.get(**lookup), False
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”,
line 328, in get
num = len(clone)
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”,
line 144, in len
self._fetch_all()
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”,
line 965, in _fetch_all
self._result_cache = list(self.iterator())
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”,
line 238, in iterator
results = compiler.execute_sql()
File “/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py”,
line 838, in execute_sql
cursor = self.connection.cursor()
File “/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py”,
line 160, in cursor
self.validate_thread_sharing()
File “/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py”,
line 421, in validate_thread_sharing
% (self.alias, self._thread_ident, thread.get_ident()))
DatabaseError: DatabaseWrapper objects created in a thread can only be
used in that same thread. The object with alias ‘default’ was created in
thread id 140736101340096 and this is thread id 4331731376.


Just to let you know,


You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups
"CommCare Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Yes, I run this command when I downloaded the newest code from master branch.

··· On 6 October 2016 at 15:23:11, Giovanni Capalbo (gcapalbo@dimagi.com) wrote:

Strange, I have the same versions locally and it’s working. Have you tried running “pip install -r requirements/requirements.txt”? Specifically, we just upgraded to Django 1.8 and I wonder if something is off with the package versions you have locally.

Note that you’ll need to restart the celery worker after doing so.

On Thu, Oct 6, 2016 at 8:20 AM, Łukasz Wyszomirski lwyszomirski@soldevelo.com wrote:
Hey,

My versions:
celery==3.1.18
django-celery==3.1.16

I set CELERY_RESULT_BACKEND as you mention and this not help, still the same issue.

I have the same situation when I set CELERY_ALWAYS_EAGER to True, but I have attached error on browser.

Thanks,
Łukasz

On 6 October 2016 at 13:58:30, Giovanni Capalbo (gcapalbo@dimagi.com) wrote:

Hi Lukasz,

What version of celery and djcelery are you running locally (pip freeze)? If it’s different from the versions in requirements/requirements.txt you’ll want to update.

Also, doublecheck the setting you have in localsettings for CELERY_RESULT_BACKEND. Locally I have it set to ‘djcelery.backends.database:DatabaseBackend’.

Also note that the preferred way of starting a celery worker is now “./manage.py celery worker” as “./manage.py celeryd” has been deprecated.

Lastly, if you still can’t get asynchronous tasks to work, you can probably just set CELERY_ALWAYS_EAGER to True in your localsettings and they should run synchronously without the need to start a worker process.

-Giovanni

On Thu, Oct 6, 2016 at 7:35 AM, Łukasz Wyszomirski lwyszomirski@soldevelo.com wrote:
Hey,

I want import cases from production to my local machine and I have problem with this function. I get error in my local celeryd attached below.

I have the newest changes form master branch and I clear a .pyc files. Any suggestions?

Task corehq.apps.importer.tasks.bulk_import_async with id 25aab2ff-b40a-45d2-a332-de93dbf30cf5 raised exception:
‘DatabaseError(“DatabaseWrapper objects created in a thread can only be used in that same thread. The object with alias ‘default’ was created in thread id 140736101340096 and this is thread id 4331731376.”,)’

Task was called with args: (<corehq.apps.importer.util.ImporterConfig object at 0x10a064ad0>, u’opm’, ‘255471afdce14ba6bbaf69c16cb04fe0’) kwargs: {}.

The contents of the full traceback was:

Traceback (most recent call last):
File “/usr/local/lib/python2.7/site-packages/celery/app/trace.py”, line 283, in trace_task
uuid, retval, SUCCESS, request=task_request,
File “/usr/local/lib/python2.7/site-packages/celery/backends/base.py”, line 256, in store_result
request=request, **kwargs)
File “/usr/local/lib/python2.7/site-packages/djcelery/backends/database.py”, line 29, in _store_result
traceback=traceback, children=self.current_task_children(request),
File “/usr/local/lib/python2.7/site-packages/djcelery/managers.py”, line 42, in _inner
return fun(*args, **kwargs)
File “/usr/local/lib/python2.7/site-packages/djcelery/managers.py”, line 181, in store_result
’meta’: {‘children’: children}})
File “/usr/local/lib/python2.7/site-packages/djcelery/managers.py”, line 87, in update_or_create
return get_queryset(self).update_or_create(**kwargs)
File “/usr/local/lib/python2.7/site-packages/djcelery/managers.py”, line 70, in update_or_create
obj, created = self.get_or_create(**kwargs)
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”, line 405, in get_or_create
return self.get(**lookup), False
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”, line 328, in get
num = len(clone)
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”, line 144, in len
self._fetch_all()
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”, line 965, in _fetch_all
self._result_cache = list(self.iterator())
File “/usr/local/lib/python2.7/site-packages/django/db/models/query.py”, line 238, in iterator
results = compiler.execute_sql()
File “/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py”, line 838, in execute_sql
cursor = self.connection.cursor()
File “/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py”, line 160, in cursor
self.validate_thread_sharing()
File “/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py”, line 421, in validate_thread_sharing
% (self.alias, self._thread_ident, thread.get_ident()))
DatabaseError: DatabaseWrapper objects created in a thread can only be used in that same thread. The object with alias ‘default’ was created in thread id 140736101340096 and this is thread id 4331731376.


Just to let you know,


You received this message because you are subscribed to the Google Groups “CommCare Developers” group.
To unsubscribe from this group and stop receiving emails from it, send an email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups “CommCare Developers” group.
To unsubscribe from this group and stop receiving emails from it, send an email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups “CommCare Developers” group.
To unsubscribe from this group and stop receiving emails from it, send an email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups “CommCare Developers” group.
To unsubscribe from this group and stop receiving emails from it, send an email to commcare-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.