I have a cluster containing 2 nodes and we are only using 5 primary shards/Index. I've recently noticed some performance issues when rebuilding reports, so that i would like to add five additional primary shards for newly created indices. Basically for hqcases index
Is there any API to resize sharding seamlessly?
Hi Demisew
With only 2 nodes in the cluster I highly doubt that increasing the number of shards is going to give any benefit. That would only help if you have more than 5 ES nodes.
Could you describe more about the performance issues you had rebuilding reports?
Building static data source
reports->household report-> edit datasource->House hold datasource-> rebuild data source.
to check progress click on preview data. It displays the following page.
on the above picture u can see the progress of rows updated by refreshing the page. around 50 rows updated per second
UCR rebuilds don't generally use Elasticsearch. Are you expecting more than 50 rows per second?
I thought every report related tasks are connected with elasticsearch. what is the issue then?
Hi Demis,
Since this has been a point of confusion in the past one thing I wanted to make sure is quite clear: UCR rebuilds are only needed when there are core structural changes to UCR's. They shouldn't be needed in day-to-day normal operation, as the HQ data pipeline keeps the data sources up to date live as forms come into the server.
Since 'rebuilding' the entire data source in the production projects space can take some time, our expected workflows for developing reports are to pilot changes against a smaller subset of data which could be rebuilt quickly before deploying rebuilds of reports into the real production data space.
-Clayton
