Recommendations for Enhancing Mailwizz’s Tracking Domain System with Detailed Technical Insights

[pradeep sharma]

Hi @twisted1919,

I have thoroughly reviewed the existing tracking domain system within Mailwizz and have identified several areas that require enhancements to meet the evolving technological standards and usage scenarios that have changed significantly over the last 2-3 years.

Current Challenges:

1. HTTPS Adoption: The widespread adoption of HTTPS by browsers and its enforcement by search engines like Google (which penalizes non-SSL email links) underscore the need for SSL support in all tracking domains. Nearly all modern browsers default to HTTPS, making it critical for tracking links to also be secure to maintain email deliverability and reputation.
2. SSL Implementation Difficulties: Many domains are hosted on shared IPs, making it challenging to implement wildcard SSL certificates through standard web panel systems like cPanel. While cPanel and similar platforms support wildcard subdomains, they do not facilitate wildcard SSL on shared IPs. This creates a barrier for users with moderate technical skills to achieve a secure setup.
3. CNAME Verification Limitations: The current CNAME-based system does not effectively verify if the application is behind a proxy or load balancer, which is a common scenario given the prevalence of cloud services and CDNs. This can lead to misconfigurations and security vulnerabilities.
4. Issues with Non-SSL Tracking Domains: Achieving a wildcard SSL on an IP typically restricts you to non-SSL tracking domains. In practice, this is insufficient as most applications and browsers enforce HTTPS, rendering non-SSL tracking ineffective and potentially harmful to user trust and email effectiveness.
5. Integration with Cloudflare: Using Cloudflare for DNS management and proxy tracking offers excellent protection against spam and blacklisting. However, because Cloudflare masks real DNS records and shows proxy IPs instead, it complicates the traditional CNAME-based verification process.
6. Tracking Server Configuration: Currently, Mailwizz supports a CDN across the application but lacks a similar, robust system for tracking. I strongly recommend the implementation of a dedicated tracking server configuration (e.g., tracking-server.application-domain.com). This domain should internally point to the same directory and handle all tracking operations, segregating it from other application traffic to optimize performance.

Technical Recommendations for Server Settings:

• Server Timeout Settings for Tracking Servers: The fastcgi and proxy timeout settings on the tracking server should be very short (preferably under 2 seconds). This setting is crucial because when sending volumes increase, the server needs to handle a significant number of email open requests efficiently—typically, an email open involves multiple HTTP requests for CDN assets and tracking. Keeping timeout settings low ensures that the server can manage these requests without overload.
• Application Server Settings: Contrastingly, the application server should have longer timeout settings (around 300 seconds) to accommodate time-intensive operations such as live list imports and large file transfers.

We can have the application to be on the same server but we need to track the main application to be served using a different hostname so that we can configure low timeouts on Tracking and high timeouts on the main application.

From a practical standpoint, these contrasting scenarios (fast-tracking vs. slow application processes) need to be addressed to optimize both the user experience and system performance.


Proposed Solutions:

1. Backend-Approved Tracking Domain Verification: Introduce an option within customer groups for backend-approved tracking domain verification. While customers can request domain tracking, actual verification and configuration would be managed by the backend admin. This ensures that domains are properly configured and secure before activation.
2. Public Key-Based Signature for Verification: Transition from CNAME to public key-based signatures for domain verification. Mailwizz could embed a unique signature meta tag across its pages, similar to Google's method for web domain verification. This would allow our scripts to verify the presence of this signature on the tracking domain’s server.
3. Automating SSL Configuration and Domain Alias Setup: Implement outgoing webhooks for tracking domain requests, which would automate the configuration of domain aliases and SSL certifications on our servers. This is crucial because manual setups can be prone to errors and are inefficient for scaling.
4. Separate Tracking Host: Establish a separate tracking host to facilitate server restarts and configurations without disrupting the main application’s operations, thereby enhancing uptime and reliability for everyday operations.
5. Signature-Based Cron Job Verification: Modify the server’s cron job processes to verify tracking domains based on the signature method rather than CNAME, enhancing compatibility with Cloudflare-proxied domains.

Implementing these suggestions will not only resolve current security and functionality issues but also significantly improve Mailwizz’s infrastructure for handling modern email campaign needs effectively. Cloudflare, in particular, has been instrumental in protecting against IP blacklisting, an issue I've personally navigated in the past, emphasizing the importance of these advancements.


I hope these detailed insights and suggestions can be considered for future updates to enhance the efficiency, security, and scalability of the Mailwizz platform.

Best regards,

Pradeep Sharma

Skype: sexy_virus2

 

[twisted1919]

I'm checking on these, give me some time.

P.S: ChatGPT much ?

 

[pradeep sharma]

I'm checking on these, give me some time.

P.S: ChatGPT much ?

I take help in removing Grammatical mistakes and rewriting the content written by my own for better understanding.

Some of My posts are rewritten to remove any grammatical mistake correct English grammar and better understandability to fellow Forum Members.


Here is what i wrote by myself and asked CHATGPT to streamline and rewrite this, Please go through this and it will also help to understand my Point about how Important is is to keep application root level Tracking under a different domain/subdomain

######

Hi @twisted1919

We have gone through the Tracking domain system and as a technical person i am aware of how it works, the tracking domain system in Mailwizz was built in very initial days and since then very few improvements is done. But since last 2-3 years Scinaria is changed in many ways as

Current Challenges.
1. almost all browsers are by default enabling https (SSL) and even google is penalizing emails if links are not served over SSLs

2. we see SSL everywhere, and its good for all of us

3. Many domains are hosted on single IPs, So Getting a WildCard Domain Is almost not at all supported in any Web panel system like cpanel, etc. Though they support wild card Subdomain but not wild card on IP So for a for even a moderate expertise user its very difficult to get this achieved.

4. The Current method to verify the CNAME-based system does not validate if the Application in behind the Public IP

5. If you can achieve a wild card on IP you will only be able to achieve the NON-SSL Based tracking domain. but in a practical scenario, it will not suffice but most of the applications (even browsers ) are forcing https by default so, even though we are able to achieve it but it does work properly.

6. It always good to keep DNS in Cloudflare and proxied tracking via Cloudflare. It Protects your IPs being spam and Blackign listed by SPAMHAUSE, due to the use in tracking domains/links/CDNs served by Emails Sent.

7. So if we proxy Traffic via Cloudflare on the Tracking domain, the current CNAME-based verification will not work, as Cloudflare doesn't return CNAME but returns Proxy IPs when we check for host query on DNS .
8. The Most Important Part, We have a facility for CDN in application-wide but we don't have anything for Tracking, I strongly be recommended to have a TRACKING SERVER like setting mailwizz So we can put a custom tracking server domain like (tracking-server.application-domain.com). tracking-server.application-domain.com must point to the same directory internally or by what whatever,it should point to the application-domain. and all customer/campaign tracking domains must ve verified against this(tracking-server.application-domain.com)

Try to understand my point that why its very important to keep your root level tracking under a different domain/subdomain because for tracking server, Server Timeout parameters like fastcgi_connect_timeout, fastcgi_send_timeout 300s;fastcgi_read_timeout,proxy_connect_timeout,proxy_read_timeout are kept very very small i.e (under 2 second or max 5 seconds) if you increases it it will not given any benefits but will add overload on high throughput applications, when we intended to increase sending volumes we must keep in mind that out servers must be able to handles atleasts 25% volume of emails send as open open email requests, an average email open request with takes average 7-8 different http request on server for both CDN assets and tracking So if our server must be able to handle request of such a big volume, and this can be achieved if we keep timeput as low as possibleI Aloways keep my Tracking timeout 2 second, in practically if my server is not abe to serve the request on 2 seconds then it has no meaning, because preple expect redirect in less then 2 second and don't wait,So Low timeout must be on the tracking server/host but it for the application its a big problem, in the application side we must keep atrleast 300 Seconds because our live list import and file transfers can take more time then that. So From a Practical point of view These are 2 just opposite scenarios We must address this!!


Possible solutions :

1. Add an Option to allow verification of Tracking domain and configuration approved from the backend, an option on the customer group to enable tracking domain verification approved by the backend if this option is enabled customer can add a tracking domain request but it can't verify and enable it. customer can just request it but configuration and proper setup to be done by the backend admin only. if we disable this option then the customer can able to verify this automatically if proper DNS CNAME Setup is done by the customer and at same time application must be able to issue r get ssl on tracking hostname and apply on tracking-server.

2. All must switch from CNAME to better Signatur of Public Key-based verification. for example, mailwizz must add a signature string on a webpage like meta-app-signature=some-randome-string and this must it on all pages of mailwizz. we can add this on the header. this is the same way Google do verification of web domains in many of its services,

3. In tracking domain Verification our script will fetch the content on the tracking domain URL ie https://customer-tracking-domain.customer-domain.com and will look for the signature which must be present in the application tracking server if the Tracking server is enabled !!. if it finds it then verification must to granted and the customer or admin from the backend will be able to verify the tracking domain and use it as per the campaign,

4. All tracking domain add requests must have an outgoing webhook post request with the tracking domain and some other important details so that the process of Creating a Domain Alias, Granting SSL /creating a separate host entry in Apache or NGINX server can be automated. without this, an option of the customer side to verify ssl domain has no meaning because they can't configure SSL and create domain alias entries in our NGINX or Apache web server.
So if We are creating a webhook then this can be achieved, but this thing requires RESTART of the webserver to take effect, that is why we wanted to have a separate tracking host either so that the user can split the tracking on separate server, We also need to restart NGINX or apache of Tracking server not the Webserver of our main application where many customers are using mailwizz for various operations daily,

5. Verification of corn job which runs on the server to verify the tracking domain must be verified using a Signature not based on CNAME.This way we can achieve all the leverage of Cloudflare, Custome Timeouts for Application, and Tracking and application, Trakcing domain verification by admin or automation. Trust me Cloudflare is a real Savior Massiah in terms of Protecting IPs from Blacking listing even if you are not using them for Sending emails, because links or CDN used in email resolve to any IPs will lead to blacklisting of IPs, and all domain hosted on these IPs. will also be potentially in domain blacking listing, i have encountered this problem in the past and since then I don't want my IP to be revealed.


Regards

Pradeep

###############