Hi
Am getting the following error after i tried to delete and regenerate elasticsearch indices. following the link below:
Problem:
-
I delete and regenerate elasticsearch indices using the following cmd.
curl -XDELETE 172.19.3.41:9200/index name -
regenreate elasticsearch:
sudo -iu cchq
cd www/echis/current
source python_env-3.6/bin/activate
./manage.py ptop_preindex
All indices recreated -
But the size of Indices doesn't much the original size before deletion. So that I applied to each index.
./manage.py ptop_reindexer_v2 case --cleanup
Output: Are you sure you want to delete the current index (if it exists)? y/n
y
2019-11-12 11:45:21,679 INFO es_utils Initializing elasticsearch mapping for [case]
2019-11-12 11:45:23,601 INFO reindexer setting checkpoint to {TopicPartition(topic='case', partition=0): 3806, TopicPartition(topic='case', partition=3): 3103, TopicPartition(topic='case', partition=9): 1660, TopicPartition(topic='case', partition=4): 1504, TopicPartition(topic='case', partition=5): 3401, TopicPartition(topic='case', partition=6): 1727, TopicPartition(topic='case', partition=1): 1404, TopicPartition(topic='case', partition=7): 942, TopicPartition(topic='case', partition=2): 438, TopicPartition(topic='case', partition=8): 2330, TopicPartition(topic='case-sql', partition=2): 831116, TopicPartition(topic='case-sql', partition=6): 407248, TopicPartition(topic='case-sql', partition=8): 423195, TopicPartition(topic='case-sql', partition=3): 455878, TopicPartition(topic='case-sql', partition=1): 579424, TopicPartition(topic='case-sql', partition=0): 1598718, TopicPartition(topic='case-sql', partition=7): 429793, TopicPartition(topic='case-sql', partition=5): 717302, TopicPartition(topic='case-sql', partition=4): 581104, TopicPartition(topic='case-sql', partition=9): 547762}
2019-11-12 11:45:23,601 INFO manager (CaseToElasticsearchPillow-hqcases_2016-03-04) setting checkpoint: {"case,0": 3806, "case,3": 3103, "case,9": 1660, "case,4": 1504, "case,5": 3401, "case,6": 1727, "case,1": 1404, "case,7": 942, "case,2": 438, "case,8": 2330, "case-sql,2": 831116, "case-sql,6": 407248, "case-sql,8": 423195, "case-sql,3": 455878, "case-sql,1": 579424, "case-sql,0": 1598718, "case-sql,7": 429793, "case-sql,5": 717302, "case-sql,4": 581104, "case-sql,9": 547762}
Processing 0 documents: ...
Processed 0/0 of 0 documents in 0:00:00.233131 (? remaining)
Processed 0/0 of 0 documents (0 previously processed).
Note: Not able to repopulate the data.
- when i look at the django.log:
tail -f /home/cchq/www/echis/log/django.log
I got the following error:
elasticsearch.exceptions.RequestError: TransportError(400, 'SearchPhaseExecutionException[Failed to ex
ecute phase [query], all shards failed; shardFailures {[OSPeX2RUROWtiiGpguQsNg][hqusers_2017-09-07][0]
: RemoteTransportException[[es1][inet[/172.19.3.40:9300]][indices:data/read/search[phase/query]]]; nes
ted: SearchParseException[[hqusers_2017-09-07][0]: query[ConstantScore(+cache(domain.exact:fmoh-echis)
cache(domain_memberships.domain.exact:fmoh-echis) +cache(base_doc:couchuser))],from[-1],size[-1]: Par
se Failure [Failed to parse source [{"query": {"filtered": {"filter": {"and": [{"or": [{"term": {"doma
in.exact": "fmoh-echis"}}, {"term": {"domain_memberships.domain.exact": "fmoh-echis"}}]}, {"term": {"b
ase_doc": "couchuser"}}]}, "query": {"match_all": {}}}}, "sort": [{"username.exact": {"order": "asc"}}
], "from": 0, "size": 0, "_source": ["_id", "username", "first_name", "last_name", "doc_type", "is_act
ive", "location_id", "__group_ids"]}]]]; nested: SearchParseException[[hqusers_2017-09-07][0]: query[C
onstantScore(+cache(domain.exact:fmoh-echis) cache(domain_memberships.domain.exact:fmoh-echis) +cache(
base_doc:couchuser))],from[-1],size[-1]: Parse Failure [No mapping found for [username.exact] in order
to sort on]]; }{[OSPeX2RUROWtiiGpguQsNg][hqusers_2017-09-07][1]: RemoteTransportException[[es1][inet[
/172.19.3.40:9300]][indices:data/read/search[phase/query]]]; nested: SearchParseException[[hqusers_201
7-09-07][1]: query[ConstantScore(+cache(domain.exact:fmoh-echis) cache(domain_memberships.domain.exact
:fmoh-echis) +cache(base_doc:couchuser))],from[-1],size[-1]: Parse Failure [Failed to parse source [{"
query": {"filtered": {"filter": {"and": [{"or": [{"term": {"domain.exact": "fmoh-echis"}}, {"term": {"
domain_memberships.domain.exact": "fmoh-echis"}}]}, {"term": {"base_doc": "couchuser"}}]}, "query": {"
match_all": {}}}}, "sort": [{"username.exact": {"order": "asc"}}], "from": 0, "size": 0, "_source": ["
_id", "username", "first_name", "last_name", "doc_type", "is_active", "location_id", "__group_ids"]}]]
]; nested: SearchParseException[[hqusers_2017-09-07][1]: query[ConstantScore(+cache(domain.exact:fmoh-
echis) cache(domain_memberships.domain.exact:fmoh-echis) +cache(base_doc:couchuser))],from[-1],size[-1
]: Parse Failure [No mapping found for [username.exact] in order to sort on]]; }]')