Unique ID beneficiary no sometimes getting duplicates

Hi, We are using a unique ID beneficiary no in our application. It seems to work fine most of the time. Ive double checked the code and seems ok and no error messages. But sometimes, we are getting two beneficiaries with same no e.g. SCT005 comes up twice. I cant work out where to look for the error. Does anyone know if either of the following affect it?

  1. Not syncing. Some of the field workers dont sync very often. Does this matter?
  2. Updating the app. Ive made a few changes to the app - largely on the hindi translation so nothing significant but they are different versions. Could they have an effect?

Any other ideas on where to look for the error? Very grateful for any help on this one.

thanks Nicola

Hi Nicola,

It depends a lot on what sort of ID scheme you're using. Have you seen this help page that goes into unique IDs in depth?

https://confluence.dimagi.com/display/commcarepublic/Generating+a+Unique+ID+for+beneficiaries

With that system, IDs should be guaranteed unique as long as:

  • each mobile worker's ID prefix is unique
  • each mobile worker uses only one device

To investigate your current system, I'd start by looking up the beneficiary cases with duplicate IDs, then find the form which created each beneficiary. Were they created by the same user? Is the device ID the same? How close together are the creation times? You can also check the Application Status report to see the last time each mobile worker synced, and what app version they are using.

Hi Ethan, thanks - I have followed that link and checked it all.

Yes they were created by the same user on the same device. They were about 2 hours apart.

He had not synced - can you explain how often they need to sync? Can this be done without wifi?

Will an app version make a difference ? He isnt using the latest version but the changes to the latest version were very minor - Can you explain how that would affect the ID nos?

Thanks - Ive read a lot - espeically all the pages around 'Generating a unique ID' - IM pretty sure the error is something to do with syncing or app versions but I cant fund a page to explain how these affect the unique IDs.

thanks Nicola

Sorry - one more question. You say each mobile worker must have a unique ID prefix. I have it set up so that all the members of the case sharing group have the same ID. Is that wrong?

However, in this case all the forms have been done by the same mobile worker. THere are two mobile workers in the group but one hasnt entered any data yet.

thanks very much, really appreciate your help!
Nicola

I have it set up so that all the members of the case sharing group have the same ID.

This could be the issue. The reason the ID prefix is unique for each user is so that no two users can create conflicting IDs. The counter should also be on the user's case so it doesn't matter how often they sync, as there's no interdependence. Imagine the following scenario:

With a shared prefix of ABC and a counter on the case sharing group:

  • User A and B both sync and see that the last used ID number was 4
  • User A registers a new patient, using ID ABC-5
  • User A registers a new patient, using ID ABC-6
  • User B registers a new patient, using ID ABC-5
  • User A syncs and sees that the last used ID number was 5
  • User A registers a new patient, using ID ABC-6

With a different prefix and counter for each user

  • User A and B both sync and see that the last used ID number was 4
  • User A registers a new patient, using ID ABA-5
  • User A registers a new patient, using ID ABA-6
  • User B registers a new patient, using ID ABB-5
  • User A syncs and sees that the last used ID number was 6 (User B's update doesn't affect User A)
  • User A registers a new patient, using ID ABA-7

Having different counters means that they can operate independently without syncing. Having different prefixes means that even when multiple users happen to have the same counter, there's no conflict.

Will an app version make a difference?

It'll only make a difference if the new app version contained changes to the ID generation logic. If nothing's changed recently, it shouldn't matter.

Ethan's scenario would normally be my assumption of what happened, but it sounds like it may not be the case here due to what Nicola mentioned earlier in the thread

Yes they were created by the same user on the same device. They were about 2 hours apart.

I'd still very strongly recommend assigning unique prefix codes per user, but that may be more of a long term issue.

My default guess would actually have been that syncing caused the problem due to some unexplained behavior where the sync payload didn't include the updated Usercase value for the current ID counter, but if the user isn't syncing currently that wouldn't quite explain things.

Nicola - it would be helpful to go find the Form Submission in HQ which created the first case, and the subsequent second case with the same ID. If you look at the source XML for these two docs there are a few things that would be good to confirm to understand the current state.

1 - There should be a usercase update in the first submission where the counter is incremented to the next value. Confirming that this exists and set the counter forwards would be helpful
2 - It would be good to confirm that there's no indication that there were issues with the first submission. One very unlikely but plausible scenario is that there was an issue with the form structure and it was processed correctly on the server, but not on the device. If there's any indication on the form submission that something unusual occured, that would be helpful to know.
3 - You can confirm in the Meta block of the submission that the userid and deviceid are the exact same (you may have already done this, but just confirming). If they aren't, that may indicate a lead.
4 - In the usercase itself, you can click on the ID Counter property to see a history of its values over time. It would be good to confirm that the ID wasn't re-set back to its original count after being set forward for any reason

One final guess as well: Are your users using incomplete forms as part of their workflow? If so, you might need to set the way you use the incrementing ID slightly differently to ensure it's updated each time the form is loaded, as opposed to just on the first form load.

You could see if the incomplete forms issue was to blame by checking the form start and end times in the submissions. If one form was started before the other was ended, that might indicate that they started both forms against the same counter.

-Clayton

HI this is really helpful. Thank you. Ive been working through your points. Some of it is quite complicated for me so doing my best! I checked the ID counter history and it isnt working. It seems to work sometimes but its definitely the problem. However, Ive checked it through several times again the pages in the help section. I just cant work out whats causing it to move forwards sometimes and then sometimes go back. Ive attached the screenshot so you can see. Do you think this is because the fieldworker isnt syncing? Should i tell him to sync after every form?

thanks very much,
nicola

In case it helps, this is my coding... any advice is very gratefully recieved as Im at a real loss here!

thanks Nicola

Last bit of the counter code

thanks

Hi Nicola,

I think the answer we are looking for is definitely somewhere within these four forms

Specifically, I think you should take a look at the two forms where the property is set to 4, especially because the cadence here doesn't really make sense with the second '4' coming more than a week later.

My guess is that these two submissions came from different devices, and that if you look at the deviceid metadata they will have different ID's. Alternatively, I'd check to see if the second from has a form_start date that is on 9-30 instead of 10-08, but I think that's less likely, because it looks like there are two "chains" of IDs incrementing between the two devices.

What to do about the situation depends a bit on what's happening on the ground. If there are two users re-using the same login on different devices, I'd follow ethan's suggestion of providing a second user with a different prefix. That will give you a separate chain of ID's per device.

If, alternatively, this really is one user who is using two different devices they will need to make sure they are syncing between. There are a few alternatives which would involve a per-user-per-id prefix in addition to the current ones, but not worth getting into details on that before determining the need.

-Clayton

Hi Clayton, yes i think this is the reason. Thank you! I had checked the device ID before but only for the two that are the same ie. SCT006 both have same device ID but SCT005 - which is in the middle of them (timewise) has a different device ID. I will find out from the fieldworker. thanks for all your help!
Nicola