Data Mining - Analyzing Metrics for RSS Feeds, Blogs + Websites

I've been a bit remiss in posting to this blog because I've been deep in planning and co-writing an e-book on RSS Metrics, as well as plotting out a new RSS Metrics blog. I have a lot of blogposts written but I haven't decided yet (1) whether this blog will parallel the new Metrics blog over at MarketingStudies.net; (2) whether the two blogs will discuss separate issues; or (3) whether I should drop this blog. My choice will most likely be #2, as I'd like to tie discussions here with some over at GeoPlotting. Look for some new posts here starting no later than next Tuesday Sep 6/05, EDT. I am also experimenting with an alternative to podcasting and may have some metrics lessons on audio for free listening here.

(c) Copyright 2005-present, Raj Kumar Dash, http://netmetrics.blogspot.com

This blog straddles the line between technical and non-technical discussions. This posting leans more toward the technical in terms of how to use the information here. However, if you are an RSS Publisher, knowing what the PAT (Peak Access Time) metric represents is worthwhile and may help keep your web site and RSS Feed running smoothly.

PAT or Peak Access Time refers to the time of day that your Feed URL and the associated details pages are being accessed on a given day. This metric is best calculated over a longer duration, such as a month or even a year or longer.

The general principle is to split a 24 hour day into several periods, typically one hour intervals. (Some people prefer 2- or 4-hour intervals.) Then, for each day across a date range, we calculate the number of times any file related to your RSS Feed has been accessed.

We want ALL accesses counted, including duplicate accesses by the same IP addresses. Let's assume that we'll calculate file access in one hour intervals, starting at 12 midnight and ending at 11:59 pm. For each interval, how many RSS Feed-related files were accessed each day? As an example, let's pick the first interval, 12:00 midnight to 12:59 am. What is the total number of Feed-file accesses in that interval on all days of our date range? Calculate this total for each of the remaining 23 one-hour time slots over the date range. The interval with the highest total is the PAT, or Peak Access Time.

In reality, PAT is calculated by totalling the size of each of the Feed-files accessed during each time slot, not by their access count. However, for simplicity, by assuming each Feed-file is the same size, we essentially get the same result with less computing effort. In fact, this an acceptable measure for a single RSS Feed because we just want a relative measure to see what time of day, over a long duration, the Feed is most commonly accessed.

What is the value of the PAT? We'll discuss just two uses for PAT:

1. Estimating Server Resources - This is of value to your technical crew. It just tells them that if, say, 1:00 pm - 1:59 pm every weekday (Mon-Fri) is the PAT, then they should ensure that your web server is not performing any unnecessary tasks during that interval, in order to free up computing resources for the RSS Feed. In this situation, it's best to calculate access counts over 4- or 6-hour intervals.
   Since there are six 4-hour intervals in a 24 hour day, you may want to calculate the PAT and the next highest total as well, and free up your web server during those two slots.
   


2. Advertising Rates - This is of value to your marketing department. If your RSS Feed distinctly shows the difference in Feed access over different time slots, marketing can assign different advertising rates to different time slots. However, in this scenario, to maximize the peak access count, it is best to set the time slot to a larger interval, such as 4- to 6-hour slots. This rate plan is analogous to costing
   ads in a magazine by their page placement: highest cost on inside and back covers, lowest cost in the classifieds area, etc. (You do not want to have too many different prices, so minimize them by reducing the number of time slots in a 24 hour period simply by using a larger interval. I suggest 4 or 6 hours simply because they divide into 24 evenly.)
   

It is worthwhile to calculate the PAT every day, then total it up per time slot over intervals such as a week, each month, quarterly, and over the "lifetime" of the feed. Doing so will help you to see any seasonal trends, which you can take advantage of. (Again, your tech crew may assign extra resources during peak season, and your marketing dept can adjust advertising rates for off-peak seasons.) As you can see, PAT may be calculated for a single day, initially, but can be aggregated into a suite of values for different purposes.

(c) Copyright 2005, Raj Kumar Dash, http://netmetrics.blogspot.com

This is the second is a series of blog posts discussing a variety of RSS Metrics. Please note that the names I use for various RSS Metrics in this blog are my own and may not be standard. In truth, there are no standards, hence why I've made up my own terms - albeit based on terms used for other types of Internet-related metrics. These terms are subject to change in later postings (but will be identified if they change).

Before we can calculate I-CTR for a particular day, we need to define some terms:

RSS Feed URL
This is the URL of the RSS/XML or Atom/XML file, or generator, that represents the RSS Feed. (By generator, I mean any web script that dynamically generates RSS or Atom content.)

RSS Headline Items
When you view an RSS Feed in a typical standalone RSS Reader, you usually see a Feed list in a left window pane, a list of headline items in a top-right window pane, and the description of the selected headline in a bottom-right window pane. Each time you click on a different headline item, you will see the associated description displayed in the bottom pane.

RSS Item Description
Each headline has an associated description. This is usually either a direct excerpt of the full "story" or a summary of the "story".

RSS Item Details Page
This is the web page that contains the "full text" of an RSS headline item. So if you are running a newsfeed, for each item you will have a headline, a description (summary), and the full-story. The full story usually appears by itself on a web page. Depending on the services you are offering, the full story may be on your own web site or elsewhere (we assume here that it is on yours). Note that the some RSS Feeds display the full-text of each item in the feed instead of descriptions. We are assuming here, that we are not dealing with such Feeds. All RSS Readers display a link from an item's description to the full-text story's web page (providing the feed contains such links).

DR - Daily Readership
The ADR, Average Daily Readership, was explained in a previous posting. The DR is the Readership for a given day. It is the number of unique IP addresses that requested ANY details page. For example, if 1000 unique IP addresses requested anywhere from, say, 1-10, details pages each today, then the DR for today is 1000.

DS - Daily Subscribers
This value is an upper bound on DR. DS is the number of unique IP addresses that requested the RSS Feed URL on a given day (via an "update feed" command in their RSS Reader). Note that the reason DS does not usually equal DR is that not everyone who updates a feed reads any of the items. If they do read an item, they may or may not click on the "read details" link for an item. So while there may be 1000 unique feed requests on an given day, maybe only 300 subscribers actually read any details pages.

Item Click-Through Rate - I-CTR
This term refers to a metric that measures the relative popularity of a particular RSS Feed headline item. The assumptions are that all subscribers use a standalone RSS Reader to view a feed. (For simplicity, we are ignoring web-based RSS aggregation services.) Furthermore, we are assuming that all RSS Reader applications only show a headline's description if explicitly requested when the subscriber selects a headline. (Compare this to some RSS Readers that simultaneously show all item headlines and their descriptions each time the feed is updated.)

The Feed CTR (F-CTR) for a given day is just the DR divided by the DS, both for the same given day. So if 1000 unique IP addresses requested the Feed URL that day but only 300 unique IP addresses requested one or more details pages, then the F-CTR = DR/DS = 300/1000 = 0.3 = 30% click-through rate for the entire feed.

If you want to calculate the I-CTR (CTR for a single item), calculate the DR(i) = DR for this item. For example, let's say you want the DR(i) for item 1 in the feed. Let's say that 150 unique IP addresses requested the details page for headline item 1. So DR(1) = 150. Then the I-CTR for item 1 is I-CTR(1) = 150/DS = 150/1000 = 0.15 = 15% click-through rate.

There is a way to achieve a more accurate I-CTR, however it depends on being able to tell when the description of a particular headline item is viewed. There is a method called "pixl tracking" that places a visible or invisible image into each Feed item's description. The image must reside on a web site that you have access to. Futhermore, you need a unique way to identify each Feed item. You can then count the number of times unique IP addresses viewed the description for a particular item by counting accesses to the image. I'll refer to this as DS(i), or the number of subscribers for the day for item i. DS(i) is less than or equal to DS. We'll discuss pixl tracking in greater depth in a later post.

(c) Copyright 2005, Raj Kumar Dash, http://netmetrics.blogspot.com

I've been taking a bit of a time-out from my RSS blogs to work on an e-book with Rok Hrastnik of RSS Diary. Rok is a busy young man with several writing projects on the go, one of them being "The Truth About RSS Metrics", a short e-book that he has asked me to be an associate author on. Rok is handling most of the marketing-related sections and I'm handling most of the technical aspects. However, the book is not aimed at tech-heads per se. Its intent is to expose RSS Metrics packages and general techniques to Internet publishers, advertisers, and direct marketers. [A much more technical book, written by myself, will also be out within the year and will focus on the mathematical and statistical techniques for collecting and analyzing RSS Metrics. This latter book will include actual code samples.]

(c) Copyright 2005, Raj Kumar Dash, http://netmetrics.blogspot.com