Ideas for CLI Monitor Dashboard

Discussion in 'Announcements' started by twisted1919, Apr 8, 2019.

  1. twisted1919

    twisted1919 Administrator Staff Member

    Joined:
    Dec 27, 2014
    Messages:
    10,254
    Likes Received:
    2,382
    Hello guys,

    As you know, in mailwizz there are a number of command line commands, most popular one is "send-campaigns" one, which can be called from command line to trigger email sending.
    Every time there is an issue with delivery, we apply https://kb.mailwizz.com/articles/debug-send-campaigns-command/ which basically will show what mailwizz is doing, but in command line.

    Now, this is fine for developers and people that can read a console output and make sense from it, but can we do better, can we make this info available to anyone in a nice dashboard, in realtime without affecting the application performance when sending emails?
    We believe we can and in last weeks we fiddled with this idea and the technology we will use in order to make this happen and we'd like to build such dashboard that shows in realtime what is going one with the sending process.

    When send-campaigns is running, we have access to all the campaigns sending at that point, to customers these campaigns belong to, to delivery servers used for these campaigns but also to the subscribers which will receive the campaign, so knowing this, what type of stats would you like to see in this dashboard?

    We can do things like the number of emails sent per minute, hour, etc.
    We can also do a breakdown on the domains the emails went out for, even the numbers of emails a domain got in the last minute/hour etc, then a breakdown for delivery servers and a graph that shows sending in realtime based on speed.
    What else?

    Note 1: because of the complexity involved and the fact it will need to be maintained in the long run, this will be a separate paid app. People contributing ideas here will get discounts.

    Note 2: We want this app to perform extraordinary fast, which means we will store all the info in memory, which means there will be a limit to the quantity of data we will show and that data will be, most likely, removed between restarts.
    This also means we can't have historic records(like keep what happened 6 months ago), we don't care. We care about what currently is going on, realtime (of course, almost realtime, but you get the point)

    Note 3: We will send your data from mailwizz to this app over the wire, which means we'd like to send as less data as possible to make it travel as fast as possible.

    Thank you in advance for any insight.
     
    Vpul Shah, Osjtya and Ankit like this.
  2. JFOC

    JFOC Member

    Joined:
    May 31, 2017
    Messages:
    62
    Likes Received:
    4
    S.E:
    Expired
    L.T:
    Regular
    L.C:
    1
    just some ideas:
    - monitor custom callback trigger
    - monitor bounce and notification when XX bounce
    - monitor the sending speed and send a notification when getting slower
    - live monitoring the number of subscribers
     
  3. 007

    007 Member

    Joined:
    Sep 22, 2017
    Messages:
    83
    Likes Received:
    12
    S.E:
    2019-11-21 02:15:23
    L.T:
    Regular
    L.C:
    2
    If this is a separate UI, it'd be nice if to have single sign-on between the two or some way to easily access the dashboard while working in the main mailwizz UI.
     
  4. Dev1@OptiBiz1

    Dev1@OptiBiz1 Member

    Joined:
    Feb 17, 2016
    Messages:
    41
    Likes Received:
    10
    Here's my idea:
    - Percentage of Success, Giveups for every campaign in real-time
    - millisecond speed of processing each subscriber including the email preparation, etc. and the most important is the delivery server communication speed. it will give a lot of insights.
    Code:
    // > subscriber_start
    //      process_subscriber
    // > before_prepare_email
    //      prepare_email
    // > after_prepare_email
    // > before_send_to_subscriber
    //      send_to_subscriber
    // > delivery_server_before_send_email
    //      ds_before_send_email
    // > mailer_before_send_email
    //      mailer_send_email
    // > mailer_after_send_email
    //      after_mailer_send
    // > delivery_server_after_send_email
    //      ds_after_send_email
    // > subscriber_end
    //      subscriber_end
     
  5. Dev1@OptiBiz1

    Dev1@OptiBiz1 Member

    Joined:
    Feb 17, 2016
    Messages:
    41
    Likes Received:
    10
    I am thinking this is something like influxdb?
     
  6. majid1f

    majid1f Member

    Joined:
    Feb 23, 2015
    Messages:
    118
    Likes Received:
    20
    Last edited: Apr 9, 2019
  7. Dev1@OptiBiz1

    Dev1@OptiBiz1 Member

    Joined:
    Feb 17, 2016
    Messages:
    41
    Likes Received:
    10
  8. twisted1919

    twisted1919 Administrator Staff Member

    Joined:
    Dec 27, 2014
    Messages:
    10,254
    Likes Received:
    2,382
    For now we'd like to focus on the data produced by the send-campaigns command, so only last two things from your list would be doable.

    We don't really want to add complexity with single sign on, but we will make it easy to login.

    Doable.

    We'd like to avoid adding an extra piece of software. See below answer.

    It's going to be written in Go.

    Yes, it's going to be a different app. Being written in go, you'll get just a binary file which you execute and will start a web application which will present you all the data.
     
  9. frm.mwz

    frm.mwz Well-Known Member

    Joined:
    Mar 8, 2016
    Messages:
    3,707
    Likes Received:
    677
    S.E:
    2019-11-27 02:17:39
    L.T:
    Regular
    L.C:
    7
    # clear indication, when DS is switched, ie how much of each camp was processed til then, and against quota
    # allow setting for amount of memory which will in turn determine how much data can be kept (couple minutes, hours, etc)
    # start / stop of manual sending (with reminder or option to disable send crons)
    # ability to add a few notes to each monitored camp
    # option to save in file (eg csv) the results displayed
    # adjust send time for pausing so stats make sense (in general and in this app)
    # adjust send time for last empty run to close out camp to not count (in general and in this app)
    # show ping time history (eg once a minute) to each DS
    Those are a few initial ideas only, once we see it, we probably have more ;)
    Thank you so much for continuing to make MailWizz greater from release to release!
     
  10. Ankit

    Ankit Active Member

    Joined:
    Aug 12, 2015
    Messages:
    116
    Likes Received:
    35
    S.E:
    Expired
    L.T:
    Regular
    L.C:
    8

    This is quite required feature.
     
  11. Vpul Shah

    Vpul Shah Active Member Support Staff

    Joined:
    Feb 23, 2015
    Messages:
    752
    Likes Received:
    120
    S.E:
    2019-12-22 13:03:57
    L.T:
    Regular
    L.C:
    1
    This is really needed feature +1.
     
  12. aleksiCoon

    aleksiCoon New Member

    Joined:
    Jun 1, 2017
    Messages:
    1
    Likes Received:
    0
    S.E:
    Expired
    L.T:
    Regular
    L.C:
    2
    I would benefit from a dashboard where I could choose to list the statistics for either the campaigns or the delivery servers. If delivery servers are listed, I would like to know how many e-mails have been sent for the particular DS in the last minute, last hour & last 24 hours. It would be super if the same dashboard could also list average delivery error percentages and average delivery times for the previously mentioned time frames as well.

    Then narrowing down by selecting an individual DS, the monitor would list the campaigns for that DS along with their respective stats, which are being brainstormed here.

    The way I am using the application, the combined amount of DSs with my customers is in the hundreds, and getting an overview of what's going on, currently, in the system, is quite hard. Currently I can only look at the campaigns screen to get an overview, and in there I can see the sent amounts & percentages increase, but that's pretty much it. Having a dashboard where I can monitor single DS/campaign health easily would be a lifesaver in many situations.

    Some possible neat statistics would be (again, for campaign & DS respectively) the top recipient domains in the queue (possibly even drill-down statistics on these, e.g. domain-based listing?), customer based statistics (who is sending the most # emails, most number of campaigns, most delivery errors, etc.) and the ability to view a small portion of the log (maybe for the last cron job, or latest 100 rows etc.) per campaign or DS.
     

Share This Page