Ideas for CLI Monitor Dashboard

twisted1919

twisted1919

Administrator
Staff member
Hello guys,

As you know, in mailwizz there are a number of command line commands, most popular one is "send-campaigns" one, which can be called from command line to trigger email sending.
Every time there is an issue with delivery, we apply https://kb.mailwizz.com/articles/debug-send-campaigns-command/ which basically will show what mailwizz is doing, but in command line.

Now, this is fine for developers and people that can read a console output and make sense from it, but can we do better, can we make this info available to anyone in a nice dashboard, in realtime without affecting the application performance when sending emails?
We believe we can and in last weeks we fiddled with this idea and the technology we will use in order to make this happen and we'd like to build such dashboard that shows in realtime what is going one with the sending process.

When send-campaigns is running, we have access to all the campaigns sending at that point, to customers these campaigns belong to, to delivery servers used for these campaigns but also to the subscribers which will receive the campaign, so knowing this, what type of stats would you like to see in this dashboard?

We can do things like the number of emails sent per minute, hour, etc.
We can also do a breakdown on the domains the emails went out for, even the numbers of emails a domain got in the last minute/hour etc, then a breakdown for delivery servers and a graph that shows sending in realtime based on speed.
What else?

Note 1: because of the complexity involved and the fact it will need to be maintained in the long run, this will be a separate paid app. People contributing ideas here will get discounts.

Note 2: We want this app to perform extraordinary fast, which means we will store all the info in memory, which means there will be a limit to the quantity of data we will show and that data will be, most likely, removed between restarts.
This also means we can't have historic records(like keep what happened 6 months ago), we don't care. We care about what currently is going on, realtime (of course, almost realtime, but you get the point)

Note 3: We will send your data from mailwizz to this app over the wire, which means we'd like to send as less data as possible to make it travel as fast as possible.

Thank you in advance for any insight.
 
just some ideas:
- monitor custom callback trigger
- monitor bounce and notification when XX bounce
- monitor the sending speed and send a notification when getting slower
- live monitoring the number of subscribers
 
If this is a separate UI, it'd be nice if to have single sign-on between the two or some way to easily access the dashboard while working in the main mailwizz UI.
 
Here's my idea:
- Percentage of Success, Giveups for every campaign in real-time
- millisecond speed of processing each subscriber including the email preparation, etc. and the most important is the delivery server communication speed. it will give a lot of insights.
Code: 
// > subscriber_start
//      process_subscriber
// > before_prepare_email
//      prepare_email
// > after_prepare_email
// > before_send_to_subscriber
//      send_to_subscriber
// > delivery_server_before_send_email
//      ds_before_send_email
// > mailer_before_send_email
//      mailer_send_email
// > mailer_after_send_email
//      after_mailer_send
// > delivery_server_after_send_email
//      ds_after_send_email
// > subscriber_end
//      subscriber_end
 
JFOC said:
just some ideas:
- monitor custom callback trigger
- monitor bounce and notification when XX bounce
- monitor the sending speed and send a notification when getting slower
- live monitoring the number of subscribers
Click to expand...
For now we'd like to focus on the data produced by the send-campaigns command, so only last two things from your list would be doable.

007 said:
If this is a separate UI, it'd be nice if to have single sign-on between the two or some way to easily access the dashboard while working in the main mailwizz UI.
Click to expand...
We don't really want to add complexity with single sign on, but we will make it easy to login.

Dev1@OptiBiz1 said:
Here's my idea:
- Percentage of Success, Giveups for every campaign in real-time
- millisecond speed of processing each subscriber including the email preparation, etc. and the most important is the delivery server communication speed. it will give a lot of insights.
Click to expand...
Doable.

Dev1@OptiBiz1 said:
I am thinking this is something like influxdb?
Click to expand...
We'd like to avoid adding an extra piece of software. See below answer.

majid1f said:
is this program develop with PHP or other languages like Go lang?
Click to expand...
It's going to be written in Go.

Dev1@OptiBiz1 said:
@twisted1919 is this a separate hosted solution?
Click to expand...
Yes, it's going to be a different app. Being written in go, you'll get just a binary file which you execute and will start a web application which will present you all the data.
 
twisted1919 said:
Hello guys,

As you know, in mailwizz there are a number of command line commands, most popular one is "send-campaigns" one, which can be called from command line to trigger email sending.
Every time there is an issue with delivery, we apply https://kb.mailwizz.com/articles/debug-send-campaigns-command/ which basically will show what mailwizz is doing, but in command line.

Now, this is fine for developers and people that can read a console output and make sense from it, but can we do better, can we make this info available to anyone in a nice dashboard, in realtime without affecting the application performance when sending emails?
We believe we can and in last weeks we fiddled with this idea and the technology we will use in order to make this happen and we'd like to build such dashboard that shows in realtime what is going one with the sending process.

When send-campaigns is running, we have access to all the campaigns sending at that point, to customers these campaigns belong to, to delivery servers used for these campaigns but also to the subscribers which will receive the campaign, so knowing this, what type of stats would you like to see in this dashboard?

We can do things like the number of emails sent per minute, hour, etc.
We can also do a breakdown on the domains the emails went out for, even the numbers of emails a domain got in the last minute/hour etc, then a breakdown for delivery servers and a graph that shows sending in realtime based on speed.
What else?

Note 1: because of the complexity involved and the fact it will need to be maintained in the long run, this will be a separate paid app. People contributing ideas here will get discounts.

Note 2: We want this app to perform extraordinary fast, which means we will store all the info in memory, which means there will be a limit to the quantity of data we will show and that data will be, most likely, removed between restarts.
This also means we can't have historic records(like keep what happened 6 months ago), we don't care. We care about what currently is going on, realtime (of course, almost realtime, but you get the point)

Note 3: We will send your data from mailwizz to this app over the wire, which means we'd like to send as less data as possible to make it travel as fast as possible.

Thank you in advance for any insight.
Click to expand...
# clear indication, when DS is switched, ie how much of each camp was processed til then, and against quota
# allow setting for amount of memory which will in turn determine how much data can be kept (couple minutes, hours, etc)
# start / stop of manual sending (with reminder or option to disable send crons)
# ability to add a few notes to each monitored camp
# option to save in file (eg csv) the results displayed
# adjust send time for pausing so stats make sense (in general and in this app)
# adjust send time for last empty run to close out camp to not count (in general and in this app)
# show ping time history (eg once a minute) to each DS
Those are a few initial ideas only, once we see it, we probably have more ;)
Thank you so much for continuing to make MailWizz greater from release to release!
 
twisted1919 said:
Hello guys,

As you know, in mailwizz there are a number of command line commands, most popular one is "send-campaigns" one, which can be called from command line to trigger email sending.
Every time there is an issue with delivery, we apply https://kb.mailwizz.com/articles/debug-send-campaigns-command/ which basically will show what mailwizz is doing, but in command line.

Now, this is fine for developers and people that can read a console output and make sense from it, but can we do better, can we make this info available to anyone in a nice dashboard, in realtime without affecting the application performance when sending emails?
We believe we can and in last weeks we fiddled with this idea and the technology we will use in order to make this happen and we'd like to build such dashboard that shows in realtime what is going one with the sending process.

When send-campaigns is running, we have access to all the campaigns sending at that point, to customers these campaigns belong to, to delivery servers used for these campaigns but also to the subscribers which will receive the campaign, so knowing this, what type of stats would you like to see in this dashboard?

We can do things like the number of emails sent per minute, hour, etc.
We can also do a breakdown on the domains the emails went out for, even the numbers of emails a domain got in the last minute/hour etc, then a breakdown for delivery servers and a graph that shows sending in realtime based on speed.
What else?

Note 1: because of the complexity involved and the fact it will need to be maintained in the long run, this will be a separate paid app. People contributing ideas here will get discounts.

Note 2: We want this app to perform extraordinary fast, which means we will store all the info in memory, which means there will be a limit to the quantity of data we will show and that data will be, most likely, removed between restarts.
This also means we can't have historic records(like keep what happened 6 months ago), we don't care. We care about what currently is going on, realtime (of course, almost realtime, but you get the point)

Note 3: We will send your data from mailwizz to this app over the wire, which means we'd like to send as less data as possible to make it travel as fast as possible.

Thank you in advance for any insight.
Click to expand...


This is quite required feature.
 
I would benefit from a dashboard where I could choose to list the statistics for either the campaigns or the delivery servers. If delivery servers are listed, I would like to know how many e-mails have been sent for the particular DS in the last minute, last hour & last 24 hours. It would be super if the same dashboard could also list average delivery error percentages and average delivery times for the previously mentioned time frames as well.

Then narrowing down by selecting an individual DS, the monitor would list the campaigns for that DS along with their respective stats, which are being brainstormed here.

The way I am using the application, the combined amount of DSs with my customers is in the hundreds, and getting an overview of what's going on, currently, in the system, is quite hard. Currently I can only look at the campaigns screen to get an overview, and in there I can see the sent amounts & percentages increase, but that's pretty much it. Having a dashboard where I can monitor single DS/campaign health easily would be a lifesaver in many situations.

Some possible neat statistics would be (again, for campaign & DS respectively) the top recipient domains in the queue (possibly even drill-down statistics on these, e.g. domain-based listing?), customer based statistics (who is sending the most # emails, most number of campaigns, most delivery errors, etc.) and the ability to view a small portion of the log (maybe for the last cron job, or latest 100 rows etc.) per campaign or DS.
 
You must log in or register to reply here.
Back
Top