Is MailWizz double sending emails?

Lakjin

Active Member
I have a problem ever since updating to the latest MailWizz. Emails send fine, but campaigns appear to be stuck at 100% processing for hours. The campaigns eventually finish, but I'm a bit worried about why it is taking literally hours at 100% processing. It takes the regular few hours to get to 100% processing, but once i it is there, a campaign can be at 100% processing for 12 hours or more. Is MailWizz sending out double emails? What is going on? Why is it taking so long to finish campaigns?

Attached is an example screenshot of a campaign at 100% processing that is still sending out emails (I verified via MySQL requests that it is still processing emails).
azNGYNH.png
 

twisted1919

Administrator
Staff member
@Lakjin - I doubt the app sends double emails. When the app gives you the impression it hangs it means it actually reprocessing emails that failed delivery, and it does this 3 times till it gives up so that can take a while to complete.
The Campaign Delivery Logs from backend > misc should give all the info.
 

Lakjin

Active Member
@Lakjin - I doubt the app sends double emails. When the app gives you the impression it hangs it means it actually reprocessing emails that failed delivery, and it does this 3 times till it gives up so that can take a while to complete.
The Campaign Delivery Logs from backend > misc should give all the info.
This is probably what is happening. Thank you. Will take a look.
 

frm.mwz

Well-Known Member
I have a problem ever since updating to the latest MailWizz. Emails send fine, but campaigns appear to be stuck at 100% processing for hours. The campaigns eventually finish, but I'm a bit worried about why it is taking literally hours at 100% processing. It takes the regular few hours to get to 100% processing, but once i it is there, a campaign can be at 100% processing for 12 hours or more. Is MailWizz sending out double emails? What is going on? Why is it taking so long to finish campaigns?

Attached is an example screenshot of a campaign at 100% processing that is still sending out emails (I verified via MySQL requests that it is still processing emails).
azNGYNH.png
besides the above
considering the miniscule difference of 0.1% (182 of 160927 emails) this could be also unsubscribes diminishing the 'out of {total}'
 

Lakjin

Active Member
@Lakjin - I doubt the app sends double emails. When the app gives you the impression it hangs it means it actually reprocessing emails that failed delivery, and it does this 3 times till it gives up so that can take a while to complete.
The Campaign Delivery Logs from backend > misc should give all the info.
This is the query being run, can you tell me what MailWizz is doing?

SELECT `t`.`subscriber_id` AS `t0_c0`, `t`.`subscriber_uid` AS `t0_c1`, `t`.`email` AS `t0_c3`, `t`.`list_id` AS `t0_c2`, `t`.`ip_address` AS `t0_c4`, `t`.`source` AS `t0_c5`, `t`.`date_added` AS `t0_c7` FROM `mw_list_subscriber` `t` LEFT JOIN `mw_list_field_value` `fieldValues40` ON (`fieldValues40`.`subscriber_id`=`t`.`subscriber_id`) LEFT JOIN `mw_list_field_value` `fieldValues18` ON (`fieldValues18`.`subscriber_id`=`t`.`subscriber_id`) LEFT JOIN `mw_list_field_value` `fieldValues19` ON (`fieldValues19`.`subscriber_id`=`t`.`subscriber_id`) LEFT OUTER JOIN `mw_campaign_delivery_log` `deliveryLogs` ON (`deliveryLogs`.`subscriber_id`=`t`.`subscriber_id`) AND (deliveryLogs.campaign_id = '3882') WHERE (((t.list_id='3') AND (t.status='confirmed') AND (`fieldValues40`.`field_id` = 40 AND (`fieldValues40`.`value` = 'yes') ) AND (`fieldValues18`.`field_id` = 18 AND (`fieldValues18`.`value` = 'yes') ) AND (`fieldValues19`.`field_id` = 19 AND (`fieldValues19`.`value` = 'no') )) AND ((t.list_id='3') AND (t.status='confirmed'))) AND (deliveryLogs.subscriber_id IS NULL) GROUP BY t.subscriber_id ORDER BY t.subscriber_id DESC LIMIT 2000 OFFSET 14000

And this

SELECT COUNT(DISTINCT t.subscriber_id) as counter FROM `mw_list_subscriber` `t` LEFT JOIN `mw_list_field_value` `fieldValues40` ON (`fieldValues40`.`subscriber_id`=`t`.`subscriber_id`) LEFT JOIN `mw_list_field_value` `fieldValues18` ON (`fieldValues18`.`subscriber_id`=`t`.`subscriber_id`) LEFT JOIN `mw_list_field_value` `fieldValues19` ON (`fieldValues19`.`subscriber_id`=`t`.`subscriber_id`) LEFT JOIN `mw_list_field_value` `fieldValues46` ON (`fieldValues46`.`subscriber_id`=`t`.`subscriber_id`) LEFT OUTER JOIN `mw_campaign_delivery_log` `deliveryLogs` ON (`deliveryLogs`.`subscriber_id`=`t`.`subscriber_id`) AND (deliveryLogs.campaign_id = '3882') WHERE (((t.list_id='3') AND (t.status='confirmed') AND (`fieldValues40`.`field_id` = 40 AND (`fieldValues40`.`value` = 'yes') ) AND (`fieldValues18`.`field_id` = 18 AND (`fieldValues18`.`value` = 'yes') ) AND (`fieldValues19`.`field_id` = 19 AND (`fieldValues19`.`value` = 'no') ) AND (`fieldValues46`.`field_id` = 46 AND (`fieldValues46`.`value` != 'no') )) AND ((t.list_id='3') AND (t.status='confirmed'))) AND (deliveryLogs.subscriber_id IS NULL)
 
Last edited:

Lakjin

Active Member
This is the query being run, can you tell me what MailWizz is doing?

SELECT `t`.`subscriber_id` AS `t0_c0`, `t`.`subscriber_uid` AS `t0_c1`, `t`.`email` AS `t0_c3`, `t`.`list_id` AS `t0_c2`, `t`.`ip_address` AS `t0_c4`, `t`.`source` AS `t0_c5`, `t`.`date_added` AS `t0_c7` FROM `mw_list_subscriber` `t` LEFT JOIN `mw_list_field_value` `fieldValues40` ON (`fieldValues40`.`subscriber_id`=`t`.`subscriber_id`) LEFT JOIN `mw_list_field_value` `fieldValues18` ON (`fieldValues18`.`subscriber_id`=`t`.`subscriber_id`) LEFT JOIN `mw_list_field_value` `fieldValues19` ON (`fieldValues19`.`subscriber_id`=`t`.`subscriber_id`) LEFT OUTER JOIN `mw_campaign_delivery_log` `deliveryLogs` ON (`deliveryLogs`.`subscriber_id`=`t`.`subscriber_id`) AND (deliveryLogs.campaign_id = '3882') WHERE (((t.list_id='3') AND (t.status='confirmed') AND (`fieldValues40`.`field_id` = 40 AND (`fieldValues40`.`value` = 'yes') ) AND (`fieldValues18`.`field_id` = 18 AND (`fieldValues18`.`value` = 'yes') ) AND (`fieldValues19`.`field_id` = 19 AND (`fieldValues19`.`value` = 'no') )) AND ((t.list_id='3') AND (t.status='confirmed'))) AND (deliveryLogs.subscriber_id IS NULL) GROUP BY t.subscriber_id ORDER BY t.subscriber_id DESC LIMIT 2000 OFFSET 14000

And this

SELECT COUNT(DISTINCT t.subscriber_id) as counter FROM `mw_list_subscriber` `t` LEFT JOIN `mw_list_field_value` `fieldValues40` ON (`fieldValues40`.`subscriber_id`=`t`.`subscriber_id`) LEFT JOIN `mw_list_field_value` `fieldValues18` ON (`fieldValues18`.`subscriber_id`=`t`.`subscriber_id`) LEFT JOIN `mw_list_field_value` `fieldValues19` ON (`fieldValues19`.`subscriber_id`=`t`.`subscriber_id`) LEFT JOIN `mw_list_field_value` `fieldValues46` ON (`fieldValues46`.`subscriber_id`=`t`.`subscriber_id`) LEFT OUTER JOIN `mw_campaign_delivery_log` `deliveryLogs` ON (`deliveryLogs`.`subscriber_id`=`t`.`subscriber_id`) AND (deliveryLogs.campaign_id = '3882') WHERE (((t.list_id='3') AND (t.status='confirmed') AND (`fieldValues40`.`field_id` = 40 AND (`fieldValues40`.`value` = 'yes') ) AND (`fieldValues18`.`field_id` = 18 AND (`fieldValues18`.`value` = 'yes') ) AND (`fieldValues19`.`field_id` = 19 AND (`fieldValues19`.`value` = 'no') ) AND (`fieldValues46`.`field_id` = 46 AND (`fieldValues46`.`value` != 'no') )) AND ((t.list_id='3') AND (t.status='confirmed'))) AND (deliveryLogs.subscriber_id IS NULL)

@twisted1919 I think I may have figured out what is going on. The two queries above are from findSubscribers and countSubscribers functions. I think what is happening, new subscribers are added to our list as MailWizz is processing campaigns sent to the list. When MailWizz gets to the end of a campaign, it keeps picking up new subscribers which is why it stays at 100% processing for so long. Any fix for this behavior? For example, only send campaigns to subscribers on the list at the time the campaign has started? An easy way to implement this is to modify the queries to only look for subscribers that where on the list on or before the time the campaign started.
 

Lakjin

Active Member
@twisted1919 I think I may have figured out what is going on. The two queries above are from findSubscribers and countSubscribers functions. I think what is happening, new subscribers are added to our list as MailWizz is processing campaigns sent to the list. When MailWizz gets to the end of a campaign, it keeps picking up new subscribers which is why it stays at 100% processing for so long. Any fix for this behavior? For example, only send campaigns to subscribers on the list at the time the campaign has started? An easy way to implement this is to modify the queries to only look for subscribers that where on the list on or before the time the campaign started.
@twisted1919 Specifically, this would work: make sure `date_added` in `mw_list_subscriber` is less than or equal to `started_at` in `mw_campaign` when processing subscribers. I assume both are recorded in same timezone, right? How would I add this to the send campaign command? I really need this.
 

twisted1919

Administrator
Staff member
@Lakjin - open apps/common/models/Campaign.php and look for:
PHP:
protected function addRegularCriteria(CDbCriteria $criteria)
{
    if (!empty($this->option->regular_open_unopen_campaign_id) && !empty($this->option->regular_open_unopen_action)) {
        if ($this->option->regular_open_unopen_action == CampaignOption::REGULAR_OPEN_UNOPEN_ACTION_OPEN) {
            $criteria->with['trackOpens'] = array(
                'select'    => false,
                'together'  => true,
                'joinType'  => 'INNER JOIN',
                'on'        => 'trackOpens.campaign_id = :opensUnopensCid',
                'condition' => 'trackOpens.id = (SELECT id FROM {{campaign_track_open}} WHERE campaign_id = :opensUnopensCid AND subscriber_id = t.subscriber_id ORDER BY id ASC LIMIT 1)',
                'params'    => array(':opensUnopensCid' => $this->option->regular_open_unopen_campaign_id),
            );
            return true;
        } elseif ($this->option->regular_open_unopen_action == CampaignOption::REGULAR_OPEN_UNOPEN_ACTION_UNOPEN) {
            $criteria->with['trackOpens'] = array(
                'select'    => false,
                'together'  => true,
                'joinType'  => 'LEFT OUTER JOIN',
                'on'        => 'trackOpens.campaign_id = :opensUnopensCid',
                'condition' => 'trackOpens.subscriber_id IS NULL',
                'params'    => array(':opensUnopensCid' => $this->option->regular_open_unopen_campaign_id),
            );
            return true;
        }
        return false;
    }

    return true;
}

and make it:
PHP:
protected function addRegularCriteria(CDbCriteria $criteria)
{
    // ADD THESE 2 LINES
    $criteria->addCondition('t.date_added <= :cdate');
    $criteria->params[':cdate'] = $this->date_added;
    
    if (!empty($this->option->regular_open_unopen_campaign_id) && !empty($this->option->regular_open_unopen_action)) {
        if ($this->option->regular_open_unopen_action == CampaignOption::REGULAR_OPEN_UNOPEN_ACTION_OPEN) {
            $criteria->with['trackOpens'] = array(
                'select'    => false,
                'together'  => true,
                'joinType'  => 'INNER JOIN',
                'on'        => 'trackOpens.campaign_id = :opensUnopensCid',
                'condition' => 'trackOpens.id = (SELECT id FROM {{campaign_track_open}} WHERE campaign_id = :opensUnopensCid AND subscriber_id = t.subscriber_id ORDER BY id ASC LIMIT 1)',
                'params'    => array(':opensUnopensCid' => $this->option->regular_open_unopen_campaign_id),
            );
            return true;
        } elseif ($this->option->regular_open_unopen_action == CampaignOption::REGULAR_OPEN_UNOPEN_ACTION_UNOPEN) {
            $criteria->with['trackOpens'] = array(
                'select'    => false,
                'together'  => true,
                'joinType'  => 'LEFT OUTER JOIN',
                'on'        => 'trackOpens.campaign_id = :opensUnopensCid',
                'condition' => 'trackOpens.subscriber_id IS NULL',
                'params'    => array(':opensUnopensCid' => $this->option->regular_open_unopen_campaign_id),
            );
            return true;
        }
        return false;
    }

    return true;
}
 

frm.mwz

Well-Known Member
@Lakjin - open apps/common/models/Campaign.php and look for:
PHP:
protected function addRegularCriteria(CDbCriteria $criteria)
{
    if (!empty($this->option->regular_open_unopen_campaign_id) && !empty($this->option->regular_open_unopen_action)) {
        if ($this->option->regular_open_unopen_action == CampaignOption::REGULAR_OPEN_UNOPEN_ACTION_OPEN) {
            $criteria->with['trackOpens'] = array(
                'select'    => false,
                'together'  => true,
                'joinType'  => 'INNER JOIN',
                'on'        => 'trackOpens.campaign_id = :opensUnopensCid',
               'condition' => 'trackOpens.id = (SELECT id FROM {{campaign_track_open}} WHERE campaign_id = :opensUnopensCid AND subscriber_id = t.subscriber_id ORDER BY id ASC LIMIT 1)',
                'params'    => array(':opensUnopensCid' => $this->option->regular_open_unopen_campaign_id),
            );
            return true;
        } elseif ($this->option->regular_open_unopen_action == CampaignOption::REGULAR_OPEN_UNOPEN_ACTION_UNOPEN) {
            $criteria->with['trackOpens'] = array(
                'select'    => false,
                'together'  => true,
                'joinType'  => 'LEFT OUTER JOIN',
                'on'        => 'trackOpens.campaign_id = :opensUnopensCid',
                'condition' => 'trackOpens.subscriber_id IS NULL',
                'params'    => array(':opensUnopensCid' => $this->option->regular_open_unopen_campaign_id),
            );
            return true;
        }
        return false;
    }

    return true;
}

and make it:
PHP:
protected function addRegularCriteria(CDbCriteria $criteria)
{
    // ADD THESE 2 LINES
    $criteria->addCondition('t.date_added <= :cdate');
    $criteria->params[':cdate'] = $this->date_added;
   
    if (!empty($this->option->regular_open_unopen_campaign_id) && !empty($this->option->regular_open_unopen_action)) {
        if ($this->option->regular_open_unopen_action == CampaignOption::REGULAR_OPEN_UNOPEN_ACTION_OPEN) {
            $criteria->with['trackOpens'] = array(
                'select'    => false,
                'together'  => true,
                'joinType'  => 'INNER JOIN',
                'on'        => 'trackOpens.campaign_id = :opensUnopensCid',
               'condition' => 'trackOpens.id = (SELECT id FROM {{campaign_track_open}} WHERE campaign_id = :opensUnopensCid AND subscriber_id = t.subscriber_id ORDER BY id ASC LIMIT 1)',
                'params'    => array(':opensUnopensCid' => $this->option->regular_open_unopen_campaign_id),
            );
            return true;
        } elseif ($this->option->regular_open_unopen_action == CampaignOption::REGULAR_OPEN_UNOPEN_ACTION_UNOPEN) {
            $criteria->with['trackOpens'] = array(
                'select'    => false,
                'together'  => true,
                'joinType'  => 'LEFT OUTER JOIN',
                'on'        => 'trackOpens.campaign_id = :opensUnopensCid',
                'condition' => 'trackOpens.subscriber_id IS NULL',
                'params'    => array(':opensUnopensCid' => $this->option->regular_open_unopen_campaign_id),
            );
            return true;
        }
        return false;
    }

    return true;
}
perhaps to consider as a (for some) meaningful option (as they could chose 'send to all subs that were subd at the beginning of the campaign' or 'send to all subs that were subd til end of the campaign' (then what about unsubs during the campaign, but perhaps that is not necessary)
what do you think @twisted1919
 

Lakjin

Active Member
@Lakjin - open apps/common/models/Campaign.php and look for:
PHP:
protected function addRegularCriteria(CDbCriteria $criteria)
{
    if (!empty($this->option->regular_open_unopen_campaign_id) && !empty($this->option->regular_open_unopen_action)) {
        if ($this->option->regular_open_unopen_action == CampaignOption::REGULAR_OPEN_UNOPEN_ACTION_OPEN) {
            $criteria->with['trackOpens'] = array(
                'select'    => false,
                'together'  => true,
                'joinType'  => 'INNER JOIN',
                'on'        => 'trackOpens.campaign_id = :opensUnopensCid',
                'condition' => 'trackOpens.id = (SELECT id FROM {{campaign_track_open}} WHERE campaign_id = :opensUnopensCid AND subscriber_id = t.subscriber_id ORDER BY id ASC LIMIT 1)',
                'params'    => array(':opensUnopensCid' => $this->option->regular_open_unopen_campaign_id),
            );
            return true;
        } elseif ($this->option->regular_open_unopen_action == CampaignOption::REGULAR_OPEN_UNOPEN_ACTION_UNOPEN) {
            $criteria->with['trackOpens'] = array(
                'select'    => false,
                'together'  => true,
                'joinType'  => 'LEFT OUTER JOIN',
                'on'        => 'trackOpens.campaign_id = :opensUnopensCid',
                'condition' => 'trackOpens.subscriber_id IS NULL',
                'params'    => array(':opensUnopensCid' => $this->option->regular_open_unopen_campaign_id),
            );
            return true;
        }
        return false;
    }

    return true;
}

and make it:
PHP:
protected function addRegularCriteria(CDbCriteria $criteria)
{
    // ADD THESE 2 LINES
    $criteria->addCondition('t.date_added <= :cdate');
    $criteria->params[':cdate'] = $this->date_added;
   
    if (!empty($this->option->regular_open_unopen_campaign_id) && !empty($this->option->regular_open_unopen_action)) {
        if ($this->option->regular_open_unopen_action == CampaignOption::REGULAR_OPEN_UNOPEN_ACTION_OPEN) {
            $criteria->with['trackOpens'] = array(
                'select'    => false,
                'together'  => true,
                'joinType'  => 'INNER JOIN',
                'on'        => 'trackOpens.campaign_id = :opensUnopensCid',
                'condition' => 'trackOpens.id = (SELECT id FROM {{campaign_track_open}} WHERE campaign_id = :opensUnopensCid AND subscriber_id = t.subscriber_id ORDER BY id ASC LIMIT 1)',
                'params'    => array(':opensUnopensCid' => $this->option->regular_open_unopen_campaign_id),
            );
            return true;
        } elseif ($this->option->regular_open_unopen_action == CampaignOption::REGULAR_OPEN_UNOPEN_ACTION_UNOPEN) {
            $criteria->with['trackOpens'] = array(
                'select'    => false,
                'together'  => true,
                'joinType'  => 'LEFT OUTER JOIN',
                'on'        => 'trackOpens.campaign_id = :opensUnopensCid',
                'condition' => 'trackOpens.subscriber_id IS NULL',
                'params'    => array(':opensUnopensCid' => $this->option->regular_open_unopen_campaign_id),
            );
            return true;
        }
        return false;
    }

    return true;
}

Thank you good sir. I tried figuring this out on my own but I wasnt even close.

Question. '$this->date_added' is the date / time the campaign was added? Can we do it so we use the date / time that the campaign was started? Because a campaign may be dated a few days before a campaign is actually started, and the campaign would then miss all those newer subs.
 

twisted1919

Administrator
Staff member
@Lakjin - makes sense, this should do it:
Code:
$criteria->addCondition('t.date_added <= :cdate');
$criteria->params[':cdate'] = date('Y-m-d', strtotime($this->send_at));
 

twisted1919

Administrator
Staff member
Sorry, it;s:
Code:
$criteria->addCondition('t.date_added <= :cdate');
$criteria->params[':cdate'] = $this->send_at;
 
Top