Email validation before sending

Pierre

Pierre

New Member
I'm experimenting with the different email validation extensions, and I have a few questions about how they work.
- Does checking emails before sending a message slow down sending speed?
- What happens if an email is detected as invalid?
- Is it sent?
- Does its status change in the database?
- What happens with "unkown" emails?

Finally, do some of the service providers offered work better than others?

Regards
 
Details after a few tests:

- Checks before sending: the emails all leave, good or bad.
- Import with verification: all emails are imported and confirmed, even bad ones...

I have the impression that nothing passes except the tokens used at the supplier...

Don't work ?

(I've have account, API and tocken of course)
 
Pierre said:
Details after a few tests:

- Checks before sending: the emails all leave, good or bad.
- Import with verification: all emails are imported and confirmed, even bad ones...

I have the impression that nothing passes except the tokens used at the supplier...

Don't work ?

(I've have account, API and tocken of course)
Click to expand...
Can you tell us which email verification provider are you using? Also can you confirm that in customer area you enable this provider and allow check zones to check for verification on import and also on sending campaigns?
 
I tried with the first two listed in the backend: Bulk email checker Email List Verify

I confirm that I have put the API, authorizations, and I have selected all the conditions for use: campaign, import, etc.
 

Attachments

  • screen.png
    screen.png
    37.1 KB · Views: 17
Pierre said:
I tried with the first two listed in the backend: Bulk email checker Email List Verify

I confirm that I have put the API, authorizations, and I have selected all the conditions for use: campaign, import, etc.
Click to expand...
Response codes have been updated from Email List Verify docs and will be available in the next release, "unkown" now will be treated as errors.
Pierre said:
Does its status change in the database?
Click to expand...
For all emails which will have a bad response, status will be changed in "Blacklisted".
 
holgroup said:
I have just done the same with EmailListVerify, I cant tell what has been checked (if anything)
Click to expand...
If you check your EmailListVerify you will found all emails that has been checked and the statuses for this emails, mailwizz will read only the messages posted by EmailListVerify and based on that response will block or no the email which has been verified.
 
I had already imported my list once into MailWizz, I had some emails that were missing TLDs and they errored on the import which is fine. Made an empty copy of the list, enabled ELV for verification and added credits, tried to import the exact same list with the "missing tld records" removed, and now Mailwizz crashes, very unhelpful error but at least now I can see it consuminmg credits:

1714461786792.png

Bombed and doesnt load anymore:
1714461897500.png
And zero subscribers in Mailwizz
 
I'm so sick of this, thought maybe its because the original list was still there, deleted that and tried again; STILL BOMBED!!!!

1714462401889.png
 
Last edited:
And yet if I disable the ELV verification, once again the list imports okay without these errors. Showing that Mailwizz is incapable of recovering from errors sent by the external third party application. This should be address in code and the application should recover not just sulk and stop working:

1714463085985.png
 
Tried again lowering batch size from 500 to 100, with ELV turned on, did one batch of 100 and then sat there dumbstuck before bombing again. Works fine with ELV disabled:

1714489694985.png

Browser console has this error:
1714489787581.png
 
Seems to be on the 3rd batch of 300 the destination script doesnt reply withion the max_execution_time = 300 seconds and the console network tab of chrome says:
504 Gateway Time-out

The server didn't respond in time.
 
Gave up with email list verify, an tried bulk email checker instead. Got to 900 records, and the BOOOOOOOOOM crashed again. This is such a frustrating process. I am processing "The importer runs in batches of 50 subscribers with a pause of 5 seconds between the batches" and it still crashes. 504 gateway timeout AGAIN:

Error, aborting the import process!
Error, aborting the import process!
 
First, here's a small explanation related to what is going on:
You are importing 50 emails in each HTTP call, which isn't a problem if you don't use external services.
But when you do use external services to verify emails, each email is verified with the email verification provider.
Now, email verification isn't exactly a fast process, if it takes one second to verify one email, we're down to 50 seconds to verify the entire batch of 50 subscribers. Your web server and php by default have a limit of 30 seconds per request, after this time, they'll kill the request and move on. Which is exactly what happens in your case, email validation service responding too slow, go over the timeout limit of your web server and php, and you end up with this error.

How to fix it?
You have a few options:
1. Increase the timeout on server, from 30 seconds to something much higher.
2. Clean the list externally and import it in MailWizz already cleaned.
3. Try to use the CLI import instead, since the CLI is not subject to timeouts, it might work better. See https://www.mailwizz.com/kb/how-do-i-enable-command-line-import-for-subscribers/

I understand it is frustrating, but this is just how some things work in some cases.
 
Okay thats all well, and thanks for explaining in detail, what you say makes sense. However excepting:
1) My server PHP max_execution_time is set to 300, which is 5 minutes
2) I am verifying batches of 50 emails with a 5 second pause between
3) I have spoken to emaillistverify.com and gave them all of the they say that their systems can handle much more than we are pushing through
4) I can get to 400 emails in batches of 100 very easily, and I can do batches of 50 again very easily, but on occasions the whole thing hangs and the import process aborts
5) This also does not explain, especially as it would appear that this timeout is expected / usual behaviour, why the verification script process does not make allowance for this and accommodate instead of just importing the import process.
6) It also does not make sense why this has now happened with two different providers.

If these 503 / 504 errors are expected, the import routines should handle the timeout gracefully and then restart that batch after having waited for a short period of time especially as this is an ajax request anyway and you wont have to handle a page reload. In the verification settings, the user should be able to specify (a) if the external service timesout, how long to wait to try that batch again and (b) how many times to retry before eventually giving up and aborting. This at least would prevent a perpetual loop.

Thanks as always for your help, support and guidance.
 
holgroup said:
) My server PHP max_execution_time is set to 300, which is 5 minutes
Click to expand...
What about your web server? Did you check the logs? You'll see there the answer.

holgroup said:
2) I am verifying batches of 50 emails with a 5 second pause between
Click to expand...
It does not matter, some of the batches will finish faster than others because the verification will take less time

holgroup said:
3) I have spoken to emaillistverify.com and gave them all of the they say that their systems can handle much more than we are pushing through
Click to expand...
Sure, but this is not about volume, but about response time.

holgroup said:
I can get to 400 emails in batches of 100 very easily, and I can do batches of 50 again very easily, but on occasions the whole thing hangs and the import process aborts
Click to expand...
Yes, see my answer for #2

holgroup said:
5) This also does not explain, especially as it would appear that this timeout is expected / usual behaviour, why the verification script process does not make allowance for this and accommodate instead of just importing the import process.
Click to expand...
This is why we give you the ability to set the number of people you import in a batch. It is also why we offer cli import.
There's not much you can do in this case since the web server will cut you no matter what.

holgroup said:
6) It also does not make sense why this has now happened with two different providers.
Click to expand...
As I said, it is a metter of how fast they are able to proces sthe emails you throw at them VS the server timeouts. There's no black magic here, it's just the requests won't get into trouble if they stay under a certain amount of time.

holgroup said:
If these 503 / 504 errors are expected, the import routines should handle the timeout gracefully and then restart that batch after having waited for a short period of time especially as this is an ajax request anyway and you wont have to handle a page reload. In the verification settings, the user should be able to specify (a) if the external service timesout, how long to wait to try that batch again and (b) how many times to retry before eventually giving up and aborting. This at least would prevent a perpetual loop.
Click to expand...
I am sure we can improve in this area. I am going to look into it and see some ways to do retries. Thanks for pointing out.

Maybe try the import from command line and see if you have better luck?
If not, maybe for the time being, clean them externally, and in the next release, when #6 is going to be taken care of, this issue will go away.

Best.
 
I am part way through CLI import process which is going well, however I suspect that even this isnt working properly and checking / verifying everything that is being imported.

I have 19834 subscribers imported so far, and they should have been verified with BulkEmailChecker, but on that, while I can see the validation count increasing. I have only done 3.7k passed / 1.4k failed / 3.3k unknown oin the dashboard which is a far different number from the number of subscribers imported.

Any thoughts?
 
One thing I can think of is if you have duplicates in your list, then they won't be checked a second time, nor at mailwizz side, nor at bulkmailchecker side.

Other than this, I can say for sure that the validation is triggered for each email address, exception above.
 
Thanks for prompt response. Okay I already have these subscribers imported into another list which I did without validation, now this is being imported into a fresh (was empty) list. I already removed duplicates in excel before I started the import process . From mailwizz:
1714584442805.png
From BulkEmailChecker:
1714584466875.png
 
I just took a look in the code to make sure I don't speak from memories.
The code responsible for this looks like:
PHP: 
$url = sprintf('%s/?key=%s&email=%s', $apiUrl, $apiKey, $email);
        try {
            $response = (string)(new GuzzleHttp\Client())->get($url)->getBody();
        } catch (Exception $e) {
            $response = '';
        }

        if (empty($response)) {
            return $emails->itemAt($email);
        }
So the HTTP request is made regardless, BUT if it fails for some reason, we assume the email is valid and move on, so the explanation here would be that there are failed requests, but the number is just too high, and I think you should open a ticket with them and have them offer some insights into this, and we can continue from there.
I'll have to ping my colleague @ghimes to take a look at this since i'll be away for a few days. If anything, open a support ticket and discuss with him.
 
You must log in or register to reply here.
Back
Top