Mega Code Archive

Retrieve E Mail from a POP3 Server with ASP, Visual Basic, VB Script, Delphi, Cold Fusion and more

Title: Retrieve E-Mail from a POP3 Server with ASP, Visual Basic, VB Script, Delphi, Cold Fusion and more. Question: Quiksoft Corporation's issue of E-Mail Secrets will show you how to build a reliable POP3 application the easy way. Developers who are new to POP3 as well as seasoned POP3 developers will find this article indispensable. Read this article and have a basic, reliable POP3 application up and running within an hour. Sample code and applications included. Answer: Retrieve E-Mail from a POP3 Server Your app can easily retrieve Internet e-mail from a POP3 server once you learn the tricks... Building an application to access POP3 mail can be very difficult without the right help. I have been building POP3 apps for 6 years now and have worked closely with professional developers the world over -- helping them to do the same. In this issue of E-Mail Secrets, I am going to show you how to build a reliable POP3 application the easy way. Developers who are new to POP3 as well as seasoned POP3 developers will find this article indispensable. We will cover topics from basic to advanced and offer tips, tricks and secrets that developers of all levels can benefit from. What you will learn from this brief article will save you weeks or more of learning, research and development. I am confident that you can read this article and have a basic, reliable POP3 application up and running within an hour. In this edition you will learn: How to connect to a POP3 server What you need to know about protecting POP3 passwords that are processed with your application. How to get the total count of messages in the mailbox How to download full messages or headers only How to download messages or headers in small groups to enable paging capabilities in your app How to delete a message from the server How to parse downloaded messages and access all message parts, including attachments, HTML, etc... How to uniquely identify messages on the server How to synchronize your app with a POP3 server and enable it to work offline How to detect if a message has already been downloaded by your app 5 things you can't do with a POP3 server Bonus: How to store and organize downloaded messages locally This edition also contains sample code and applications including: VB script code demonstrating most POP3 procedures A simple VB6 POP3 client A simple ASP POP3 client Introduction There are many reasons why you may want to build an application to retrieve e-mail from a POP3 server. Automated e-mail processing, inbox notifications and message scanning are some uses for POP3 applications -- other than building a full blown desktop or web based POP3 client. What does it take to write a functional and dependable POP3 application? You need a basic understanding of what POP3 can and can not do, and you need a good component to provide that functionality. You will also need a component that can parse and decode downloaded messages. The code samples throughout the newsletter are written in VB Script. I purposely kept them very simple to avoid confusing you with unnecessary functionality, bells, whistles and other bulk. We will be using the POP3 and Message components of the EasyMail Objects for retrieving and parsing the messages. In the bonus section we will use the EasyMail MailStore object to store and manage downloaded messages on the local hard drive. The EasyMail Objects are a set of COM objects that will enable your application to send, retrieve, merge, compose, view, edit, store and print Internet e-mail messages. We will only be using a portion of the EasyMail Objects in this issue. To run the samples you will need to download and install the EasyMail Objects. The combination of VB Script and COM components is easy to understand and easily adaptable to other programming languages or environments such as Visual Basic, ASP, C++, Delphi, etc... Also available for download are two sample POP3 client applications. One in Visual Basic and the other in ASP. You can download the samples and the EasyMail Objects here. Lets get started... Logging In and Getting Message Count The following example shows how to login to a POP3 server and get the number of available messages. It uses the EasyMail POP3 object to connect to the POP3 server and get the message count: set objPOP3 = CreateObject("EasyMail.POP3") objPOP3.MailServer="mail.domain.com" objPOP3.Account="account" objPOP3.Password="password" objPOP3.Connect cnt = objPOP3.GetDownloadableCount() MsgBox cnt & " messages available." objPOP3.Disconnect Are your POP3 passwords at risk? Continue on to find out why your passwords may be vulnerable and how to protect them... Protecting the POP3 Password with APOP Traditional POP3 servers expect to receive the account password in plain text format. The problem with plain text format is that it can easily be intercepted by hackers, giving them complete access to your POP3 account. An extension to POP3, titled APOP, has been developed as a countermeasure. With APOP authentication, the password is never sent across the network. This is great, but you may be asking, "How can the server authenticate a user if the password is never sent across the network?" The answer is that the client application creates a digest that is computed from the password and the current time. The client application sends the digest to the server in-lieu-of the password. The server performs the same calculation to create its own digest and then compares it to the digest it received from the client. If the two digests match, the user is authenticated. Since the digest is computed from the current time, it changes with every login. If it is intercepted by a hacker, it is useless because it will no longer be valid, since the time has changed. The internals of this authentication are quite complex and I will not go into them here. The EasyMail POP3 object makes APOP authentication very easy and handles all the complexities for you. set objPOP3 = CreateObject("EasyMail.POP3") objPOP3.MailServer="mail.domain.com" objPOP3.Account="account" objPOP3.Password="password" objPOP3.AuthMode=256+16 objPOP3.Connect cnt = objPOP3.GetDownloadableCount() MsgBox cnt & " messages available." objPOP3.Disconnect By setting the AuthMode property to 256, we enable APOP authentication. Since APOP is an extension to POP3, it is not supported by all POP3 servers. By adding 16 to the AuthMode property we instruct the EasyMail POP3 object to automatically downgrade to plain text authentication if the server does not support APOP. This enables our application to use APOP if available, and automatically use conventional POP3 authentication if APOP is not supported by the server. Downloading Messages In this example, we will download and parse every message in the mailbox. When the operation is complete, we will use a message box to display the subject of the first message. We will not display the subject of every message, to keep things simple. A word of caution: Since this code downloads every message in the mailbox, it may take a long time to execute if the mailbox contains many messages. It is often more efficient to download messages in small groups as the next sample will demonstrate, however some applications will benefit from downloading all messages at once because they will not be dealing with large mailboxes, and the code is very easy and straight forward: set objPOP3 = CreateObject("EasyMail.POP3") objPOP3.MailServer="mail.domain.com" objPOP3.Account="account" objPOP3.Password="password" objPOP3.Connect objPOP3.DownloadMessages(0) MsgBox "Subject: " & objPOP3.Messages(1).Subject objPOP3.Disconnect The zero parameter to DownloadMessages() instructs the EasyMail POP3 object not to mark the message for deletion after downloading. The DownloadMessages() method automatically parses the downloaded messages and exposes the data via the Messages collection. More on message deletion and parsing later... A More Efficient Download Efficient POP3 applications download message headers first, and do not download the complete message unless the user specifically requests it. The message headers contain dates and addressing information plus the subject, and this is enough to build a table or list box to display the messages to the user. When the user selects a specific message to view, you can download the entire message and display the contents. This approach saves time and bandwidth. Also, as mentioned in the previous example, it is often more efficient to download messages in small batches rather than downloading every message in one pass. This type of efficiency becomes more important with larger mailboxes and/or slower connections to the server. set objPOP3 = CreateObject("EasyMail.POP3") objPOP3.MailServer="mail.domain.com" objPOP3.Account="account" objPOP3.Password="password" objPOP3.Connect 'download first 5 headers and 'display their subjects nMsgCount=objPOP3.GetDownloadableCount() nLastMessage = 5 if nLastMessage nMsgCount then nLastMessage = nMsgCount end if for x = 1 to nLastMessage objPOP3.DownloadSingleMessageHeaders(x) MsgBox objPOP3.Messages(x).Subject next 'download full message from first header 'and display attachment count if objPOP3.Messages.Count then nFullMsg=objPOP3.DownloadSingleMessage(1) nCnt=objPOP3.Messages(nFullMsg).Attachments.Count MsgBox "Attachment count: " & nCnt objPOP3.Messages.Delete(nFullMsg) end if objPOP3.Disconnect This sample downloads the first 5 messages on the server, or every message if there are 5 or less. The execution speed of this sample should be very fast, unless you have a slow connection to the server. Finally, we simulate the user selecting the first message in the list and download it in its entirety. We can then display the attachment count. This code can easily be expanded and adapted to provide paging capabilities in your application. Enabling your application to display multiple pages of 10-20 messages each will increase its speed and decrease its resource consumption. Up next: How to parse downloaded messages and access their parts including attachments, HTML and more... Parsing and Accessing Downloaded Message Parts In the beginning of this article I mentioned that we would be using the EasyMail Message object to do the parsing and decoding. You may have noticed that we have successfully parsed messages, but have never used the EasyMail Message object. The reason for this is that the EasyMail POP3 object is fully integrated with the EasyMail Message object. Beneath the hood the EasyMail POP3 object loads an instance of the EasyMail Message object for each downloaded message. The EasyMail Message object is responsible for parsing the message and exposing its parts such as its subject, body text, attachments, etc... Here is a brief sample that demonstrates some of its basic functionality: set objPOP3 = CreateObject("EasyMail.POP3") objPOP3.MailServer="mail.domain.com" objPOP3.Account="account" objPOP3.Password="password" objPOP3.Connect ret = objPOP3.DownloadSingleMessage(1) if ret=0 then set Message = objPOP3.Messages(1) s = "Date: " & Message.Date & vbcrlf s = s & "From: " & Message.FromAddr & vbcrlf s = s & "Attachments: " & Message.Attachments.Count s = s & vbcrlf s = s & "Subject: " & Message.Subject s = s & vbcrlf & vbcrlf s = s & Message.BodyText MsgBox s end if The call to DownloadSingleMessage(1) instructs the EasyMail POP3 object to download the first message in the mailbox. After the message is downloaded, the DownloadSingleMessage() method internally creates an instance of the EasyMail Message object and adds it to the POP3 object's Messages collection. Finally DownloadSingleMessage() passes the downloaded data to the new Message object which parses and decodes it. This all happens with one call to DownloadSingleMessage(). Most of the EasyMail POP3 object's download methods will behave in this way, but there are some methods that download the message to a disk file without parsing. See DownloadSingleMessageToFile() in the EasyMail Objects documentation for more information. The EasyMail Message object is extremely fast and powerful. With it you can get access to every message part including attachments, HTML, etc... Please see the EasyMail Objects documentation to learn more about the EasyMail Message object and its capabilities. Identifying Messages on a POP3 Server Up until now we have been identifying each message on the server by its message number. The message number is the message's ordinal position in the mailbox. i.e. the first (and oldest) message in the mailbox is 1, the second message is 2, etc... The POP3 protocol is based around this numbering system, but these numbers can change from session to session. A session in POP3 speak is the period between a connection and disconnection from a mailbox. POP3 sessions are exclusive. That is, the POP3 server will only allow one user at a time to be logged on to a specific mailbox. Message numbers are guaranteed to remain constant within any session. New messages arriving in the mailbox during a session, will not be available until the next session, and deleted messages are not physically removed until the session is ended. So POP3 guarantees that message numbers will not change during the course of one single session. This is relatively simple to understand and does not present any problems until you want operations to span multiple sessions. Lets say that you want to create an application that downloads messages, stores them on the users hard drive and then disconnects. Later when the user wants to view the message, they run your app which loads the messages from the hard drive. When the user wants to perform an operation on a previously downloaded message such as deleting it from the server, your application can connect to the POP3 server easy enough, but the message's ordinal position may have changed since the last session. So how do you find the message to delete? To facilitate this type of operation and to assist you in synchronizing application data with a POP3 server, the POP3 server assigns every message a unique-id. The POP3 RFC (1939) states: "The unique-id of a message is an arbitrary server-determined string, consisting of one to 70 characters in the range 0x21to 0x7E, which uniquely identifies a message within a [mailbox] and which persists across sessions." However, the RFC later goes on to state: "Clients should be able to handle a situation where two identical copies of a message in a [mailbox] have the same unique-id." So how do you access a message by its unique-id and what do you do if two identical messages in the same mailbox have the same unique-id? Personally I have not known a mail server to duplicate unique-ids in one mailbox, so I am not going to address that, but you should be aware of it none-the-less. If you are really concerned, you can add additional checks and processing to handle this situation if it should arise. The EasyMail POP3 object contains two methods to enable you to work with unique-ids: GetMessageNumFromID() and GetMessageID(). GetMessageNumFromID() returns a message number given its unique-id. GetMessageID() does exactly the opposite and returns a unique-id, given a message number. If your application will need to synchronize its locally stored messages between sessions, it can call GetMessageID() and store the unique-id with the local message data. Upon subsequent sessions, you can pass that unique-id to GetMessageNumFromID() to retrieve the current message number for that unique-id and thus perform operations on it. A message's unique-id can also be used to determine if it has already been downloaded by your app during a previous session. In our example, we want our application to download and store only the new messages that have arrived since our last session. How do you figure out which messages are new? POP3 does not support this capability on the server end, but if your app stores the unique-id of each message it has downloaded, you can loop through the mailbox and find any message with a unique-id not stored locally. These are the new messages. So you may be asking yourself, "How do you store messages on the local hard drive?" and "How do you delete messages from the server?" The answer to these questions will come later in this article. Some applications such as ASP are always disconnected. A typical ASP POP3 application may logon to the server to download the first 10 headers, display the headers in an HTML table and then disconnect from the POP3 server. When the user selects a message from the table, the ASP application will have to reconnect to the mailbox and perform an action on the selected message and disconnect again. This is an example of where using unique-ids works well even though no messages are being stored locally (at least on disk). Please continue on to learn how to delete messages from the server... Deleting Messages from the Server How and when to delete messages from the POP3 server depends largely on the objectives of your applications. Some POP3 applications delete the messages as soon as they are downloaded. These applications typically store a copy of the messages locally and thus there is no need to keep them on the server. However, removing messages from the server and storing them locally will make them unavailable from any computer other than the one that retained the local copy. For this reason, many newer POP3 applications do not delete the message from the server, until you delete it from the local store. Some applications, such as web based e-mail, do not delete the message or store a local copy. The message is downloaded into memory each time it is accessed. This enables the same messages to be easily accessed from different computers such as one at work and one at home. The EasyMail POP3 object enables you to delete a message from the POP3 server in many different ways. Here are some examples: Delete a message from the server by its ordinal position: 'delete the message from the server 'where x is the messages ordinal position 'of the message on the server objPOP3.DeleteSingleMessage(x) If the message you want to delete is in the POP3.Messages collection, you can delete it from the server like this: 'delete the first message in the Messages 'collection from the server objPOP3.Messages(1).DeleteFromServer You can also mark messages for deletion when they are downloaded with DownloadMessages() by passing in a 2 as the only parameter to the method: objPOP3.DownloadMessages(2) When a message is deleted from the server, the server flags the message for deletion but does not physically delete the message until the session is terminated. The RollBack() method of the EasyMail POP3 object can be called prior to ending the session to un-flag the messages so they will not be deleted when the session ends. Five things you can not do with POP3 POP3 is an effective, yet simple protocol. Because it is simple, it will not do everything that you might want to do with your mail system. Many of the limitations may be bridged by clever programming, but it is important to understand that they are not a part of the POP3 protocol. I talk to lots of developers who are frustrated because they can not figure the command to do some operation on a POP3 server. Many times the simple answer is that the command does not exist. POP3 has a very limited command set. Create Folders on the Server The POP3 protocol supports only one folder, the Inbox. Clients such as Outlook Express enable users to manage a local store of downloaded messages on their hard drive, but it is important to understand that this is not a function of POP3. If you want your application to store and manage downloaded messages, check out the EasyMail MailStore object. Have two users logged into one mailbox at the same time The POP3 server will only allow one user to be logged in to a mailbox at one time. It is important to properly disconnect from the POP3 server, or the mailbox may remain locked until a timeout occurs on the server. Determine if a message has been read The POP3 protocol does not provide a mechanism for the server to indicate if messages have been read. Clients such as Outlook Express store data on the local hard drive and thus can mark those messages as read or unread in the local store, but not on the server. Determine if there are new messages on the server The POP3 protocol does not provide a mechanism to enable servers to indicate the number of new messages on the server. Your application can store data locally and thus programatically determine the number of new messages on the server. If you want your application to store and manage downloaded messages, check out the EasyMail MailStore object. Sort the messages The POP3 server orders messages from oldest to newest. The POP3 protocol does not support a mechanism to sort messages on the server. Many applications simulate this by downloading the messages and then sort them locally prior to displaying them to the user. Next, bonus information on how to store, organize and manage downloaded messages on the local hard drive... Bonus: Storing and Managing Messages Locally Since POP3 does not provide a way to organize and manage messages on the server, many POP3 applications, such as Outlook Express, download the messages from the server and store them on the local hard drive. It is typical for these applications to provide an easy to use interface that allows users to organize messages in folders. If you would like to add this type of functionality to your application, we have a solution that was designed just for the purpose. The EasyMail MailStore object enables your application to easily store, organize and retrieve messages in a specialized, lightweight database system on the local hard drive. Creating the store Set objMailStore = CreateObject("EasyMail.MailStore") objMailStore.CreateStore "c:\", 0 objMailStore.OpenStore "c:\", 0 objMailStore.CreateFolder "Inbox", 0 objMailStore.CloseStore A store can be created in any directory. Here we create the store in the root directory and create a folder called "Inbox". We can create a virtually unlimited number of folders, and nest them too. i.e. Inbox\Save. Downloading a message and adding it to the store set objPOP3 = CreateObject("EasyMail.POP3") objPOP3.MailServer="mail.domain.com" objPOP3.Account="account" objPOP3.Password="password" objPOP3.Connect ret = objPOP3.DownloadSingleMessage(1) if ret=0 then set Message = objPOP3.Messages(1) Set objMailStore = CreateObject("EasyMail.MailStore") objMailStore.OpenStore "c:\", 0 objMailStore.SelectFolder "Inbox", 0, 0 strFile=objPOP3.Messages(1).TempMsgSourceFile objMailStore.AddMessageFile strFile, "", 0 objMailStore.CloseStore end if objPOP3.Disconnect We begin by using the EasyMail POP3 object to download a message. In this case, to keep things simple, we simply download the first message in the mailbox. To add the downloaded message to the store we pass in the raw unparsed RFC-822 message file. This is available from the TempMsgSourceFile property of the EasyMail Message object. We add the message to the "Inbox" folder in our store. When a message is added to the store, its header is parsed and a copy of the header data is stored is a special database format that will enable our application to retrieve it in the future very quickly without reparsing. Retrieving a message from the store Set objMailStore = CreateObject("EasyMail.MailStore") ret = objMailStore.OpenStore("c:\", 0) ret = objMailStore.SelectFolder("Inbox", 0, 0) objMailStore.MoveFirst 0 set env = objMailStore.Envelope data = "Date: " & env.DateAdded & vbcrlf data = data & "From: " & env.FromAddr & vbcrlf data = data & "Attachments: " & env.AttachmentCount data = data & vbcrlf data = data & "Subject: " & env.Subject MsgBox data objMailStore.CloseStore One of the really cool features of the EasyMail MailStore object is the way that it allows you to list the stored messages. The Envelope property of the EasyMail MailStore object, provides quick access to the basic message information that we would normally want to display in a list box for the user. i.e. subject, date, priority, etc... I am just skimming the surface of the EasyMail MailStore object here. If your e-mail application needs to store messages locally, you owe it to yourself to explore its capabilities further. More information can be found here on our website. Conclusion... As I said in the beginning, POP3 is pretty easy once you know how the protocol operates. A reliable set of e-mail components, such as the EasyMail Objects is also a must. In addition to what I have demonstrated in this article, the EasyMail Objects will enable your application to send and print e-mail messages too.