How to do migrations on user data?

yves · 22 May 2023, 07:30

We use the user data in FusionAuth to store some additional attributes for our users.

The schema naturally evolves and so from time to time we need to migrate the schema, add attributes, remove attributes, convert items.

What are best practices on how to do this in FusionAuth?

Is there anything in FusionAuth supporting this? Or is the advice to do this outside of FusionAuth?

dan · 23 May 2023, 16:07

@yves This is similar to any ElasticSearch schema changes. (That is, adding attributes is easy, changing data types and removing attributes is tougher.)

There's no particular guidance available.

For testing, you can set the index name using fusionauth-app.user-search-index.name. More here: https://fusionauth.io/docs/v1/tech/reference/configuration

Hope that helps. If you have more specific questions, feel free to ask.

yves · 23 May 2023, 16:29

@dan Thanks for this!

I guess I'm struggling to understand how FusionAuth internally saves the user data. Is this unstructured (e.g. JSON) inside the relational database?

I guess I can't give this more structure, e.g. defining data types, and so on?

I guess there's also no "direct" access to the user data via an API? At least I didn't find anything.

dan · 25 May 2023, 19:31

@yves

I guess I'm struggling to understand how FusionAuth internally saves the user data. Is this unstructured (e.g. JSON) inside the relational database?

You can examine the database schema FusionAuth uses here: https://fusionauth.io/docs/v1/tech/installation-guide/fusionauth-app#advanced-installation

I guess I can't give this more structure, e.g. defining data types, and so on?

The structure for the .data fields is implicit. This means that you can define the schema by creating a value. So if I set user.data.isPremium to true, then ES will understand that user.data.isPremium is of type boolean.

A couple of notes:

if you change the datatype across users, ES will get confused. Here's docs on how to deal with that: https://fusionauth.io/docs/v1/tech/admin-guide/troubleshooting#mapperparsingexception
there's an open issue for allowing schema enforcement. Please upvote it or add other feedback
using the PATCH method for arrays in the user.data field is a bit fragile. Here's docs on how to choose this method correctly: https://fusionauth.io/docs/v1/tech/apis/#the-patch-http-method

I guess there's also no "direct" access to the user data via an API? At least I didn't find anything.

You can update, patch or remove values from the user.data field using the normal user APIs. Or did I misunderstand your question?