Syncing Bubble Database to Xano

There has been a lot of learning over the past weeks as we have gone deeper down into the rabbit hole of making this a reality. I have watched a lot of videos, I have consumed a lot of forum posts and content and I have experimented with the Xano Connector plugin in Bubble quite extensively. A plan of action has formed on how to migrate our Bubble database to Xano and here I will detail how that plan looks like.

Spoiler alert, basically we need two identical databases running in parralel so that data is synced across them and then once this magic machine is running, we can start to build out the app functionality to use Xano instead of Bubble.

Step 1 - Duplicate existing user database

The user field is super important as it is how you will go about filtering your data later. Unlike in Bubble, Xano does not have a native "Created by" field so this is something you need to add within your Xano database structure. Every single table that you are referring to user data, you will need to make a "Created by" field.

I also find that we often make use of the "Last modified" field so as a general good practice for the Xano database, on every table that you set up in the instance, add a "Last modified", "Created by" and "Bubble ID" fields. Obviously the user table does not need "Created by". The Bubble ID will come in handy as we start to build the relationships across the data so it is super useful to have it in from the start.

With your user table setup, you can use the Bubble API or a csv export to create your users into the Xano database. This makes it pretty simple and the main point is having the user's email and their Bubble ID. The rest of the fields are not so important at the moment as we can pick them up later by looping through the records.

Step 2 - Syncing new users

The next step is to think about the overall plan. We want both the Bubble database and the Xano database to contain the same records. When a user is created in Bubble, we want the user to be automatically created in Xano. If the user deletes their account in Bubble, it should also delete their data in Xano.

This is pretty simple to do, on the signup flow for the user, simply make an api connector endpoint in Bubble to sign the user up into Xano. You can then do the same on your delete user flow. This will ensure that the system is in sync and your user count in Xano and Bubble should be the same. Happy days.

Step 3 - Follow same process for other data types

Here we are looking to do the same sync process for the other data types. First of all, we want to import all of the existing Bubble records into Xano. This can be done via the API or it can be done via a CSV export. Don't be too concerned about grabbing all of the fields accurately straight away.

Data types often change after they have been created. At this stage, we're not so bothered by making the tables completely accurate with the same field nor are we so bothered with the relationship fields being setup 100% correctly. What we are bothered with is having the records be the same in number across both Xano and Bubble.

We want to have a sync in the number of records for every single table (or data type if we're in Bubble). What is important is that we have the Bubble ID for these records as this will let us call that record later via the Bubble API and populate all the data that we need including the relationships.

Find out in your Bubble application when a record is created and also when a record is deleted and create endpoints via the API connector to add / remove records in your Xano database. That will ensure that all data is synced into Bubble and you now effectively have two databases with the same number of records.

Step 4 - Duplicate the front-end

I build most of my apps with a single app page where all the magic happens for the user. This makes it relatively simple to know what I need to change when I am doing this switch to Xano. I can create a new page in Bubble and just duplicate that entire page. This will now give me both duplicate databases and duplicate front-ends. This is where the fun begins!

We now have a legacy app page where the existing app runs without issue. This legacy page is producing records for both Xano and Bubble and we don't want to touch this at all. We want to start on the new app page in de-coupling it from Bubble data and starting to port in our Xano database via the Xano Connector plugin.

That's the next challenge

There is a lot to do still and we still do not have all of our records in Xano having all the fields of information which they should have, that is not so much of an issue though at this stage. What is important is that we can use the Xano Connector to start replacing repeating groups and calls for data on our new app page to call directly from Xano and remove as much of the Bubble database and workflows as possible.

A final note to leave this post, you may be thinking, what about Option Sets in Bubble? From our playing around, it is fine to keep these in Bubble, what you will need to do though, is create the individual fields in your Xano database for each part of an Option Set that is saved. I'll mention that more in the next post.