When million records and dashboard loads slow...

Status
Not open for further replies.
twisted1919

twisted1919

Administrator
Staff member
I am aware that at some point, given the number of records in database and the server where mailwizz is hosted, the dashboard from /backend and /customer starts to load very slow.
I know the exact reason for this, but it's not really something one could easily fix without breaking other parts of the app.

My current idea is to let the dashboard as is, then if the slow loading issue appears, replace the dashboard with something else, a lighter dashboard, but what should this dashboard contain and how it should look like, any idea ?
 
twisted1919 said:
I am aware that at some point, given the number of records in database and the server where mailwizz is hosted, the dashboard from /backend and /customer starts to load very slow.
I know the exact reason for this, but it's not really something one could easily fix without breaking other parts of the app.

My current idea is to let the dashboard as is, then if the slow loading issue appears, replace the dashboard with something else, a lighter dashboard, but what should this dashboard contain and how it should look like, any idea ?
Click to expand...
Hey buddy, i guess i can help you out with that as these days i have been trying and using other Email Marketing tools and checking their apps how they work and what exactly they are doing to show things on dashboard.
But it will be a difficult task, and for the current dashboard i would suggest let be like this and once we have some other solutions or lighter dashboard inputs then you can decide how you wanna go about it.

Rest please see me PM :p and we can also discuss this part over there, i have few ideas which i can share with my login details so that you can also check, I'll share that with you asap.

Thanks
 
Last edited:
@twisted1919 Thanks for acknowledging & addressing this problem..
Most of the Mailwizz Users may not be facing this problem initially as total subscriber count is less initially, but its a big problem once u have millions of records in your mailwizz DB.
This is the biggest problem that some1 will face with millions of records as i am facing.
I would like to suggest few things which can be considered while solving this problem.

1. We can create 2 Type of Dashboard. i.e a) Default Version b) Lite Version
2. Admin can have choose his own dashbard version and can set customer dashboard Version(default/Lite)
3. Some Few More Tables can be added into Database which will have latest Dashboard statics stored into it, this Statistic will be updated in a schedule interval be a deamon/cronjob. Lite dashboard will just display information from these tables rather than doing live Query onto Live DB
4. Lite Dashbard will Show last updated Time Stamp on each stats and will have a refresh button..user can also get latest information/statics by a button.
5. Admin can set time interval after which states will be refreshed/renewed.

I hope it may help.

Regards
 
@Jamie Whittingham - If you need the list exported and its going to save you any hassle, as a short term quick fix, though not designed to be the way you interact with mailwizz, anything over 1,000,000 to be exported quickly and efficiently you can export it directly out of the database for example: MySQL by going to your table that would be something like: `mw_list_subscriber` then directly export it from there.

On occasion i have done this as a quick fix on very large lists whee i just need the email addresses without wanting to put any strain on resources, You can easily reference the list id and export everything assigned to that list id directly from the database.

In my example:
Code: 
 SELECT * FROM `mw_list_subscriber` WHERE `list_id` = 5

Hope this helps as a short term quick-fix.

Thanks
 
Hi @BirdyUK , thanks for the tip. I was going to do that but then I noticed that it would not handle bounces / blacklist / unsubscribers .....

I was under the impression that there was a lot of JOIN's and multi query relationships before we get handed the export file .....

Any ideas on which SQL statements are used to remove the bounces etc etc

cheers
 
@Jamie Whittingham - When you need all the reports i probably wouldnt advise exporting different sections of your database because it will start to get a bit messy for you and if there is another way you can get round it within mailwizz i would recommend doing it that way or making some trial and errors.

-- Quick Note: Before you start exporting sections of your database i would advise doing an overall backup to ensure nothing is lost or broken while doing the trial and error approach.

To export other lists that are relevent to the campaign your wanting to keep the data relation to Delivery and Bounce logs you can do that like this, Find your campaign list id so for example: ( wE2dx3fvbrf3w ) in the campaigns table find the true numeric id of the campaign within the ID column, Lets say in this case my campaign id is '9'

You can then perform a query like this:

Code: 
 SELECT * FROM `mw_campaign_delivery_log` WHERE `campaign_id` = 9

You can also do this on bounce logs and so on, All the relvent tables that you need will use a relationship so its not to hard to track down what it is need to export but just so you know, make sure to backup or export the entire database before you do anything as its often easier to break things than fix things :)

As for my answer i know its not the best, Im suffering with an immense Migrane today so forgive me, its been tough typing out what i have done! Hope this helps, when i get chance ill edit this and give a better example.

Thanks
 
@twisted1919
do we have any solution planned in next release for this problem or we have to wait for long or to design our own solution..
just want to know that we are facing a lot of difficulty in our testing platform... where we have 11Million recorded and its difficult to load dashboard it takes more than 15 min to load dashbaord..
 
@frm.mwz @Jamie Whittingham @BirdyUK
i have developed my own code and solved this problem
We optimized some query..and dis some modification on the code now it loads in just mili seconds even if you have 100 millions of records..
I can offer this modification to any one who is facing the same problem..on a price

Limitation: Live DASHBOARD States are updated on every x minutes(default was around 5 min) x can be set by you. i have set it on every 60min on my system admin and 30min for customer.

PM me
installation support will be provided
 
pradeep sharma said:
@frm.mwz @Jamie Whittingham @BirdyUK
i have developed my own code and solved this problem
We optimized some query..and dis some modification on the code now it loads in just mili seconds even if you have 100 millions of records..
I can offer this modification to any one who is facing the same problem..on a price

Limitation: Live DASHBOARD States are updated on every x minutes(default was around 5 min) x can be set by you. i have set it on every 60min on my system admin and 30min for customer.

PM me
installation support will be provided
Click to expand...

Is your solution more like an extension?
 
is it the list sizes or is it the number of campaigns that had been sent?

I.e. What is the work around with out recoding? Can i delete old campaigns? Will this alleviate it?
 
I dont know if this was related BUT I had this issue last week. But my system resources were just fine....
I went and deleted a ton of old servers and old campaigns... then the next day it cleared up... not right away but the next day. Maybe a cron ran that cleared it all up? But mine went back to normal after i did that 12 hours or so later. (for what its worth).
 
Status
Not open for further replies.
Back
Top