Mass Data for multiple apps

My company builds apps for small business owners who perform services similar to Instacart & doordash but on an Individual & personalized scale/experience. Each small business owner gets their own app complete with their logo as the icon and their brand colors throughout the app. With this being said, grocery stores alone can carry 10,000s of items each. There is approximately 150 stores that each business owner will need to be able to access if they chose to offer that store as a service for their clients. What’s the best way to connect to these items and stores so I’m not spending days imputing 10,000s of items per store per app.

Is there a way to host all this info on one app/platform and have all the other individual apps pull info from it and also customize each item to their needs?

Hi @Zwood,

Do I understand correctly that you’d like to have an app which has 1 500 000 individual SKUs available for its users (150 * 10000)?

Best,
Victor.

Hi @Victor that’s a massive DB :hot_face:

I have about 2K users with 15 columns of data and the App is so slow-witted trying to load the DB (I’m not using so many list, just a few filtered with no autorefresh.), I’m thinking try Xano, i hope this can improve the speed.

Regards

Sort of… More like 150 SKUs as each store has their own and at least one of each item with it’s UPC and images that can be assigned to multiple stores, have multiple prices per store as each store through out the country has a different price for the exact same item as well as multiple prices per business owner as each owner will have the option to markup the cost just like Instacart does if they so desire. So the database could have anywhere from 10,000 items to 1.5M items or even more.

Hi @Zwood,

In my opinion:
(a) for such kind of data load, an external backend database should be used
(b) maybe it would be possible to design the backend and the app in a way, that it will be able to split the data and handle it efficiently, but I’d say this will require having quite advanced technical skills
(c) with millions of records, there are a lot of other factors which come into play (query tuning, indexing, scalability, etc etc etc), which require a lot of knowledge about how backend works and how to provide this data to frontend (app).

In my personal opinion, this is a full-code kind of task.

Best,
Victor.

2 Likes

@Victor @Fercho so the API that I found contains 800000 branded grocery items, 34000 generic grocery items, & 30000 ingredient items and their images, UPC codes, and much more. This api is founded on RapidAPI.com, I’ll get a total of 864000 grocery items. far less than 1.5M I was talking about gathering. I was looking at AirTable to be the backend data host and Aldalo would just pull the information from AirTable. Would this work with the amount of items in this API? If so, I tried watching the tutorial videos and visiting the help section to connect to RapidAPI but have no luck in getting it to work properly. I was also thinking the possibility of allowing the user to duplicate an item and assign it to a specific store.

Hi @Zwood,

1,500,000 and 864,000 is not a big difference.

You are going to use Airtable as a backend data host - but even in Enterprise plan they have a limit of 100,000 records per Base (and on the Pro plan it is 50,000).
Airtable API has a limit of 5 calls per second / per base, so for a higher workload, there is a risk of hitting this limit.

I’ve already stated my opinion - this is a full-code kind of task, unless you decide to impose some limits.

Best,
Victor.