When AOL and Comcast bounce messages back to the sender, they generate a bounce known as an Abuse Reporting Format or ARF for short.
As a 10,000 ft overview, here's how it works.
The original email campaign sender composes an email, they record the unique Message-Id of each outgoing email,
and also, inside of the body of the email they add an unsubscribe link that looks something like this:
<a href=http://www.mycompany.com/newsletter/unsubscribe.aspx?emailID=123>To Unsubscribe, click here</a>
Why do they do this?
Well, by keeping track of the Message-Id they know when that specific message was
sent out, what campaign it belonged to, and even what email address it should be mapped to.
It really depends upon how much information you want to keep track of.
If for some reason the Message-Id is not returned in the bounce, another way of keeping track
of which email address bounced, is the Unsubscribe link. In the Unsubscribe link, is an id (or key)
that maps back to the campaign databse. If we know the key of the email address that bounced, the sender can remove that email address.
We will look at this in depth below.
When this message is sent to an email address that no longer exists at AOL or Comcast,
a bounce is generated. Inside of that bounce are 2 attachments,
1) An ARF report, and
2) The original message that was sent (the one we composed above).
Using ListNanny to Process the Bounces
ListNanny has the capability to parse and read these ARF reports. A typical ARF report may look something like:
------=_NextPart_000_05A7_01C9075B.90749E70
Content-Type: message/feedback-report;
name="ATT01192.dat"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="ATT01192.dat"
Feedback-Type: abuse
User-Agent: AOL SComp
Version: 0.1
Received-Date: Mon, 25 Aug 2008 20:24:18 -0400
Source-IP: 63.134.242.204
Reported-Domain: smtp.myserver.com
Redacted-Address: redacted
Redacted-Address: redacted@
The original sender can sometimes use this data to scrub their lists.
However, to prevent email addresses from leaking (due too fake bounces and unscrupulous spammers),
AOL and Comcast redact the original recipient's email address.
To help cope with this, the one thing included with this bounce, is the original message that was sent.
However, again, AOL and Comcast remove the recipient's email address (they don't make this easy).
The way to handle this situation, is to use that Messge-Id and Unsubscribe link we recorded from above.
Processing the Bounces
What we need to do, is use ListNanny to parse the bounce and:
1) Find the ARF Report
2) Find the Message-Id value of the original message, and
3) Find the Unsubscribe link in the original message.
Once we have that information, we can remove the offending email address from our database.
Below is a code example that demonstrates this functionality.
C#
static void AolARFBounce()
{
//Load an AOL bounce
NDR n = NDR.ParseFile("c:\\temp\\AOLArfReport.eml");
//get at the ARF Report
ARFReport arf = n.FindARFReport();
if( arf == null )
{
//check to see if it's a .dat file
foreach( NDRPart part in n.BaseMessage.MimeParts )
{
if(( part.Filename != null ) && (part.Filename.EndsWith(".dat") ) )
{
string body = part.DecodedText();
arf = new ARFReport(body);
break;
}
}
}
if( arf != null )
{
//A custom method we can use for processing ARF reports
ProcessArfReport( arf );
}
//lets find the Message-Id value of the message that was originally sent
//Lets also find all of the hrefs in the original email
if( n.OriginalMessage != null )
{
if( n.OriginalMessage.MessageId != null )
{
//Extract the messageId value, and process it
string messageId = n.OriginalMessage.MessageId.Value;
ProcessMessageId( messageId );
}
NDRPart part = n.OriginalMessage.HtmlMimePart;
if( part != null )
{
string[] hrefs = part.HRefs();
//loop through all of the Hrefs in our email to find the
//unsubscribe.aspx href. This href will contain our
//email address key
foreach( string href in hrefs )
{
Console.WriteLine( href );
if( href.ToLower().IndexOf("unsubscribe.aspx") > -1 )
{
//a method to process the url
ProcessUrl( href );
break;
}
}
}
}
}
static void ProcessMessageId( string messageId )
{
//in this method we would update our database with
//any statistics we want to record (or remove the emal address)
//of the offending recipoient
}
static void ProcessUrl(string href )
{
//this url will look something like
//http://www.mycompany.com/campaign/Unsubscribe.aspx?emailId=123
//We will want to extract the "123" part, and update the database to remoe
//the email address that maps to the key "123"
}
static void ProcessArfReport( ARFReport arf )
{
//in this method, simply write out the results to the console screen.
//Normally we would implement any business rule related to ARF processing here.
//check to see if the ARF report was found
if( arf != null )
{
//write out the various properties
Console.WriteLine( arf.ARFFeedbackType );
if( arf.ARFFeedback != null )
{
Console.WriteLine( arf.ARFFeedback );
}
if( arf.AuthenticationResults != null )
{
Console.WriteLine( arf.AuthenticationResults );
}
if( arf.OriginalMailFrom != null )
{
Console.WriteLine( arf.OriginalMailFrom );
}
if( arf.OriginalRcptTo != null )
{
Console.WriteLine( arf.OriginalRcptTo );
}
if( arf.ReceivedDate != null )
{
Console.WriteLine( arf.ReceivedDate );
}
if( arf.RemovalRecipient != null )
{
Console.WriteLine( arf.RemovalRecipient );
}
if( arf.ReportedDomain != null )
{
Console.WriteLine( arf.ReportedDomain );
}
if( arf.ReportedUri != null )
{
Console.WriteLine( arf.ReportedUri );
}
if( arf.SourceIP != null )
{
Console.WriteLine( arf.SourceIP );
}
if( arf.UserAgent != null )
{
Console.WriteLine( arf.UserAgent );
}
if( arf.Version != null )
{
Console.WriteLine( arf.Version );
}
Console.WriteLine( "====================");
Console.WriteLine( "write out all of the fields of the ARF Report" );
Console.WriteLine( "====================");
if( arf.ARFFields != null )
{
for( int i=0;i<arf.ARFFields.Count;i++)
{
NDRHeader field = arf.ARFFields[i];
string name = field.Name;
string completeValue = field.ValueComplete;
Console.WriteLine( "{0}:{1}", name, completeValue );
}
}
}
}
VB.NET
Shared Sub AolARFBounce()
'Load an AOL bounce
Dim n As NDR = NDR.ParseFile("c:\temp\AOLArfReport.eml")
'get at the ARF Report
Dim arf As ARFReport = n.FindARFReport()
If arf Is Nothing Then
'check to see if it's a .dat file
Dim part As NDRPart
For Each part In n.BaseMessage.MimeParts
If Not (part.Filename Is Nothing) And part.Filename.EndsWith(".dat") Then
Dim body As String = part.DecodedText()
arf = New ARFReport(body)
Exit ForEach
End If
Next part
End If
If Not (arf Is Nothing) Then
'A custom method we can use for processing ARF reports
ProcessArfReport(arf)
End If
'lets find the Message-Id value of the message that was originally sent
'Lets also find all of the hrefs in the original email
If Not (n.OriginalMessage Is Nothing) Then
If Not (n.OriginalMessage.MessageId Is Nothing) Then
'Extract the messageId value, and process it
Dim messageId As String = n.OriginalMessage.MessageId.Value
ProcessMessageId(messageId)
End If
Dim part As NDRPart = n.OriginalMessage.HtmlMimePart
If Not (part Is Nothing) Then
Dim hrefs As String() = part.HRefs()
'loop through all of the Hrefs in our email to find the
'unsubscribe.aspx href. This href will contain our
'email address key
Dim href As String
For Each href In hrefs
Console.WriteLine(href)
If href.ToLower().IndexOf("unsubscribe.aspx") > - 1 Then
'a method to process the url
ProcessUrl(href)
Exit ForEach
End If
Next href
End If
End If
End Sub 'AolARFBounce
Shared Sub ProcessMessageId(messageId As String)
'in this method we would update our database with
'any statistics we want to record (or remove the emal address)
'of the offending recipoient
End Sub
Shared Sub ProcessUrl(href As String)
'this url will look something like
'http://www.mycompany.com/campaign/Unsubscribe.aspx?emailId=123
'We will want to extract the "123" part, and update the database to remoe
'the email address that maps to the key "123"
End Sub
Shared Sub ProcessArfReport(arf As ARFReport)
'in this method, simply write out the results to the console screen.
'Normally we would implement any business rule related to ARF processing here.
'check to see if the ARF report was found
If Not (arf Is Nothing) Then
'write out the various properties
Console.WriteLine(arf.ARFFeedbackType)
If Not (arf.ARFFeedback Is Nothing) Then
Console.WriteLine(arf.ARFFeedback)
End If
If Not (arf.AuthenticationResults Is Nothing) Then
Console.WriteLine(arf.AuthenticationResults)
End If
If Not (arf.OriginalMailFrom Is Nothing) Then
Console.WriteLine(arf.OriginalMailFrom)
End If
If Not (arf.OriginalRcptTo Is Nothing) Then
Console.WriteLine(arf.OriginalRcptTo)
End If
If Not (arf.ReceivedDate Is Nothing) Then
Console.WriteLine(arf.ReceivedDate)
End If
If Not (arf.RemovalRecipient Is Nothing) Then
Console.WriteLine(arf.RemovalRecipient)
End If
If Not (arf.ReportedDomain Is Nothing) Then
Console.WriteLine(arf.ReportedDomain)
End If
If Not (arf.ReportedUri Is Nothing) Then
Console.WriteLine(arf.ReportedUri)
End If
If Not (arf.SourceIP Is Nothing) Then
Console.WriteLine(arf.SourceIP)
End If
If Not (arf.UserAgent Is Nothing) Then
Console.WriteLine(arf.UserAgent)
End If
If Not (arf.Version Is Nothing) Then
Console.WriteLine(arf.Version)
End If
Console.WriteLine("====================")
Console.WriteLine("write out all of the fields of the ARF Report")
Console.WriteLine("====================")
If Not (arf.ARFFields Is Nothing) Then
Dim i As Integer
For i = 0 To arf.ARFFields.Count - 1
Dim field As NDRHeader = arf.ARFFields(i)
Dim name As String = field.Name
Dim completeValue As String = field.ValueComplete
Console.WriteLine("{0}:{1}", name, completeValue)
Next i
End If
End If
End Sub
I realize this has been an extremely brief entry, and many pages could be written on processing bounces.
This article it targeted towards those email senders who already understand the basic architecture of sending
mailing campaigns. If anyone has any questions, no matter how small, about this article,
feel free to contact me, using the Contact Us page.
Thanks,
Dave Wanta