Web Sense

Don’t Use a Web Site that Does Not Cookie its Visitors

In his article Internet Says: "Me Want Cookie," WSJ 5/5/2008, L Gordon Crovitz points out that “Web sites do a poor job of explaining how and why this information [cookies] is used…” and that “Web sites and marketers have failed to explain why cookies are harmless…. So people don’t understand how the Web works, so fear they are being spied on and manipulated.” The consequence of which is more and more web users are deleting their cookies, and “Do-Gooders” are lobbying for and politicians are considering legislation to restrict cookies.

Crovitz’s primary justification for cookies is that they make advertising more relevant – visitors get ads for relevant products and services and “in exchange for seeing targeted advertising, we get access to Web sites, usually free.” Relevant advertising and free access are nice benefits from cookies but aren’t their most important benefit. The most important benefit of cookies and the reason I try to avoid Web sites that don’t cookie their visitors is that cookies enable Web sites to distinguish between visitors (not identify who they are just that they are different visitors). This is the foundation of the Web measurement necessary to improve users’ Web experience. Without cookies Web sites don’t have the information they need to improve their site. Sites that don’t cookie their visitors don’t care enough about their visitors to want to give them a better user experience.

Today most major sites use cookies to understand how their visitors use their sites. While companies such as DoubleClick specialize in the use of cookies for marketing purposes, the majority of cookie usage is for site improvement (most Web sites use cookies for both marketing and site improvement). Cookies provide the information necessary to answer questions such as:

Can a site’s visitors find the products they’re looking for? Is the check-out process confusing? How well does online support work? What features are useful and how often are they used? What kind of enhancements would best serve our customers? What types of capacity are needed? How can we improve the customer experience? Did the changes made to the site really improve the user experience?

Comparing the online experience with brick and mortar facilities, the absence of cookies would be like prohibiting companies from looking at and identifying their customers when they came to their facility. Today, banks observe customer behavior and use that information to do things such as have special windows for Merchants who come in with large volumes of cash, to have a single line for retail customers that is serviced by multiple windows, and to position customer service personnel. Retail establishments observe customer behavior to determine where and how to place and displace products, how to design traffic flows, as well as the placement of checkout stations. Grocery stores use membership cards to identify what products customers buy together to better place products making it easier for customers to find what they want. If done properly, it does improve profits, but it also improves the customer experience. No one would even think about prohibiting commercial establishments from observing their customers and using that information to improve their business. In my opinion, Web sites that don’t cookie their visitors don’t care enough about them to want to improve their site and should be avoided.

They only reason to prohibit online merchants from doing so is, as Mr. Crovitz says fear of the unknown – what cookies can and cannot do. Mr. Crovitz’s solution is to have better disclosure statements and uses as an example Walt Mossberg and Kara Swisher’s disclosure statement. Here's what is says:

“Some of the advertisers and Web analytics firms used on this site may place 'tracking cookies' on your computer. We are telling you about them right upfront, and we want you to know how to get rid of these tracking cookies if you like."

Read more »

Tracking cookies are small text files that can tell such companies what you are doing online, even though they usually don't record your name or other personably identifiable information. These cookies are used by these companies to try and match ads to a user's interests. They are used all over the Web, but in most cases, their presence is only disclosed deep inside privacy policies.

We want you to know how to get rid of these tracking cookies if you like. Here are links to pages where you can opt out of the cookies set by our ad-placement contractor and our analytics contractor:

• http://www.doubleclick.com/us/about_doubleclick/privacy/
• http://www.omniture.com/privacy/2o7

"We'd prefer a totally opt-in system, but, as far as we know, the ad industry doesn't have a practical one as of now.”

I enjoy reading Mossberg’s column and respect his judgment. However, in this case I think Mossberg and Swisher’s approach is wrong. It’s great that they tell people upfront about cookies. However, their whole emphasis is on what to do if you don’t like cookies and their explanation of the benefits of cookies is tepid at best. They subtlety pander to the fear of the unknown while pretending that they are objective. They should explain how valuable cookies are and that they enable Web sites to get better. Perhaps they don’t truly appreciate the importance of cookies in Web site improvement. Perhaps they are only aware of cookies as a marketing tool. They should advise their readers to keep their cookies and to avoid sites that don’t cookie their visitors.

I strongly agree with Mr. Corvitz’s position that there is quite a bit of misunderstanding and ignorance about cookies and their value. I propose that those of us who understand cookies and their benefits do something about it to combat the “real risk that someday soon we’ll find the untested hands of regulators in the cookie jar.” Web sites should have an information page explaining the benefits of cookies in access through a “cookie link” that only talks about cookies and their benefits in clear, easily understandable language (no legalize). Measurement companies like Omniture, digital Agencies like Digitas, and associations like the Web Analytics Association should take out ads in major publications explaining the value of cookies (if they don’t they may be out of business). Web professionals of all kinds should write their elected representatives explaining the value of cookies and asking them to refrain from misguided restrictions of their use. Let’s not be intimidated by fear-mongers and the ignorant.

Let’s standup for what we know to be true.

Data Integrators at X Change 2008

Data Integrators, are you ready for X Change 2008?

I'm looking forward to X Change 2008 for two reasons: Food & Conversations.

Conversations.

Last year we were just starting to integrate online and offline data. To be perfectly honest, all I could talk about were the "challenges" involved in data integration. Much has happened since then. We've actually done it. For me, the most interesting integration is that of Online Support and Call Center data.

To get a good picture of what's happening, we use a scatter plot with Online Support on one axis and Call Center usage on the other. We then divide the graph into four quadrants and look at different user segments such as online tenure and customer value. Wow do the patterns differ depending on the segment. Differ is interesting, but they differ in ways that are actionable for both the Call Center and Online Support.

That is valuable because we can use those differences to improve service and reduce costs. For example in many cases, in the first month or so after enrollment, clients are light users of both the Call Center and Online Support. Very quickly, however, there is a group that uses the Call Center over and over with very little use of Online Support. After about a year, the majority of this group is in the low callers and high online user group. By comparing their call types and online behavior, we are developing training and support programs that promise to reduce Call Center volume -- eliminate or at least substantially reduce the large number of calls made by customers with two to twelve months of experience. Because we can actually monitor the customer behavior based on the training and support they have received, we will be able to tell quite accurately what works and what doesn't.

We also look at repeat Call Center usage. Pilot programs have been started in which special training is given to a sample of Call Center Callers and their subsequent behavior monitored both online and offline. Without a doubt some things work (dramatically reduce Call Center volume) and some don't. Some work with certain segments but not others. What's great is that we have facts instead of guesses to support our conclusions based on our actions.

So what has this got to do with X Change 2008?

Well last year there was a group of hardy souls who were just starting data integration projects. In the X Change 2008 sessions, I'd like to compare notes and share insights with these pioneers. Then in the evening (we were all staying at the same hotel and didn't have to drive anywhere - at least if you reserve early enough) over drinks complain, vent, and get the relief that is only possible by sharing your pain with someone who has had the same experience.

So all you data integrators who have suffered the pain of data integration (How many user ids don't match? What do you do when you get records that show a single caller called 40,000 times in a single month?) come to X Change 2008. We'll share insights, swap stories, and get revitalized.

The Food.

I've met the chef at the Ritz and tasted his food. Each Ritz Carlton has a specialty and the San Francisco Ritz's specialty is food. It really is special. I've also seen all the rooms (dinning, conference, and bedrooms). Very, very nice.

Web Analytics -- An Employment Guide -- Where do you excel - Politics or Analytics?

To meet the growing demand for our web consulting services, we are recruiting newly graduated college students as well experienced analysts. As the use of the web becomes a requirement rather than an option, web analytics is becoming an increasingly popular high-paid career. (In the early days of electricity, there were electric companies - companies that used electricity - and non-electric companies. Today every company uses electricity and the distinction between an electric and non-electric company is meaningless. Similarly the distinction between a web company and a non-web company is rapidly disappearing.) In this environment, we've met some very good candidates. Many of whom have a choice between us (while we're the largest independent web analytics consultancy, we're still less than 20 people) and a large agency or general purpose consulting firm. Even though our starting salaries are equal to or greater than the large firm's offer, the candidates often choose the larger organization.

Is this a good decision or not? For those who know me, you know that one of my favorite saying is: "Know yourself and know others: in one hundred battles you will win one hundred times." Well what does that have to do with choosing an employer? Very simply, before you make the decision you need to know what you like, what your strengths are, and what the opportunities and success requirements of your future employer are. If you really like analysis and are good at it, you will do best at Semphonic. On the other hand if analysis is something you don't mind doing, and you have very good political skills you will do best at a large firm (agency or consulting).

With only one basic kind of service, web analytics, Semphonic must provide outstanding, cost effective consulting with every engagement. A large firm provides a wide range of services. Services that can range from web design, IT operations, media buying, to consulting not even associated with the web. Since their relationship is based on a wide range of services, each one does not have to be excellent, just good enough. Consequently, after a certain point, increased analytic capability is irrelevant. When that point is reached, politics becomes the critical success factor in one's career. Being a powerful, skillful member of the compensation committee, being able to smooze with other senior partners as well as client executives is more important than insightful analysis. The super stars of a large firm are those with exception political skills, and they make the really big bucks, more than a successful analyst at Semphonic. However, as with super stardom in any profession there aren't very many super stars and the competition is intense. The high salaries of these super stars requires that non-star employees (everyone else) work long and hard as they have to generate the billable hours that generate the revenue needed to pay the super stars super salaries.

Semphonic on the other hand has very little politics, a flat organization, and low overhead so that a much greater proportion of its revenue can go directly to the analysts doing the work. Its critical success factor is excellence of service. "Good enough" is not good enough. Consequently, increased analytic capability is very relevant. Semphonic gives its analysts more responsibility, more opportunity for different types of analysis, less stress, and more money than any large firm.

So what's the bottom line? If you're good at politics and like the trappings of a big firm and want a chance for super stardom (remember most people don't make star status) go for a big firm. If you like web analytics go for Semphonic. Choosing an employer is an important decision. You spend a large portion of you waking hours working. Make sure it's something that fits who you are.

If Semphonic sounds like a place for you, e-mail career@semphonic.com with your resume and a letter telling us why you like web analytics.

Why Ask Semphonic?; Web Analytics -- The T'ai Chi Solution

Why "Ask Semphonic"? Because the questions are often more valuable than the answers.

In T'ai Chi (a "soft" marshal art that uses the soft to overcome the hard and the mind to overcome physical strength), there is a saying that I think applies very well to web analytics. "Know yourself and know others: in one hundred battles you will win one hundred times." The question is how best to know yourself and others from a web analytics perspective.

Just when you think you've gotten your web analytics under control, things change. Changes can be from any of three major perspectives: business, web site technology, or analytic tools. A marketing channel is no longer as effective as it once was. Your customer base is changing. You have a new product. Competition changes. Then there is always web site technology. You may redo your site in Flash or some other rich media with capabilities you never had before. Clients may be accessing your site through their cell phones, or RSS feeds may all of a sudden take off. Web 2.0 is here and Web 3.0 is on the way. Finally, there is your analytic tool. Pricing, capabilities, or even availability may change. What do you do?

So how do you know your web-analytic-self as well as web-analytic-others so that you can "in one hundred battles win one hundred times" with all this change going on? Start by asking questions. Start with some overview questions such as does your web analytics support your business objectives and how does it do it? Then get into the details of how. What site KPI's do you measure? How? Do you use segmentation? How? Do you track online campaigns? How? Do you track links? How? Do you measure any online processes? How? Do you do longitudinal analysis? How? What kind of web reporting do you do? How do you do it? Keep on going.

Put together a comprehensive list of questions that covers your web site from all three perspectives: business, web site technology, and web analytic tool. As your experience and expertise increase, keep adding to your list of questions and periodically re-ask them. You'll find that the questions are more important than the answers. Why, because the answers are continually changing while the questions don't change very much. If you have a good set of questions, any time there is a change in your environment all you have to do go to your question list and ask. On the other hand, if you just had a list of answers, how would you handle change?

If questions are the answer to how "in one hundred battles you will win one hundred times," what's the best way to take advantage of Ask Semphonic? Prepare your questions. To make sure you're not missing any, categorize them by the three perspectives, business, web site technology, web analytic tools. Select a few you think are the most important and submit them. Then during Ask Semphonic listen to the questions as much as you do to the answers.

When listening to the questions, remember that there are four different types of questions:

1. Questions you know the answers to.
2. Questions you understand but don't know the answers.
3. Questions you know but don't understand.
4. Questions you don't even know.

All four are important, but the last two, questions you don't understand and questions you don't even know about can be the most important. You may not need them today, but write them down. They could very well be the questions that give you the winning edge sometime in the future.

What I like about Ask Semphonic is that it gives you the best of both questions and answers. In short run the answers may be most valuable, but in the long run it's the questions. Semphonic consultants are a great resource for answers. Participants, and at this writing there are over 50, are a fantastic resource for questions.

The first Ask Semphonic takes place on December 11 at 11:00AM PST, 2:00PM EST (check out http://www.semphonic.com/analytics/asksem.asp) and will be about the Omniture acquisition of Visual Sciences. This will not only affect Visual Sciences and Omniture customers, but also has the potential of affecting others in so far as it changes the market dynamics of web analytic tools. The answers given today, may or may not be relevant a year from now. However, I'm sure most of the questions will be relevant. If you're forearmed with the right questions, you will be able to "Know yourself and know others: in one hundred battles you will win one hundred times."

Will Omniture Learn From Google?

What's Google doing that Omniture could benefit from? As Jessica Vascellaro writes in the Wall Street Journal, November 5, 2007, "[Google] hopes to entice others to develop features to exploit the mobile web.... If Google succeeds at rallying developers ... it could open the way for consumers to start doing more easily on their web phones what they can already do on the Web." Why is Google doing this? It's not because they are a philanthropic organization. It's because they want to own the market and they realize that they cannot provide all the features that consumers want as fast as they want them. They can, however, own the platform.

Omniture is in a similar position. With Web 2.0, technology is changing too fast for any one company to provide all the functionality that customers of web analytics software will need. The scope and complexities of rich media are such that Omniture will always be in a catch-up mode if it tries to do everything itself. Take Flash for example, Omniture is finally getting it's act together, but it's taken awhile.

During this catch-up period, what did customers do? Some suffered in silence. However, others wrote their own code. Since Omniture did not provide a robust API, these customers wrote code and placed it below the line line in the Omniture js file that says "Do not modify below this line". It usually worked until a new version of the js file came out. Then the customer had the choice of staying with their modified js file or figuring out how to modify the new js file so that they could continue using the capabilities they had developed. Let me tell you it wasn't always a very prettry picture. Another popular capability to add "below the line" is enhanced link tracking. The list goes on.

There are also features that customers would like, but even going "below the line" won't help. In particular, enhancements to the Excel interface. The existing Omniture Excel interface has a very nice GUI and is fine for new users or users with limited Excel expertise. However, there is a large group of web analysts who are not only experienced Omniture users but also very proficient at Excel. These users would love a robust Excel API. With a robust Excel API, Omniture could take credit for the power of Excel in reporting and analyzing the web data it collects and make lots of analyst very happy.

With Omniture's purchase of Visual Sciences, does it have any incentive to become a platform for web analytics and provide robust APIs so that others can develop Omniture features? In the short run probably not. However, who knows how short the short run will be. As Web 2.0 continues to take over, a smart competitor may take the software platform route and entice developers to develop enhanced features for their platform and eventually take over the market. Look at Apple. It had a better user interface, but didn't open it up and didn't capture the market. If they had, Windows may have been an also ran operating system.

In the meantime, what's a web analyst to do. You don't want to go below the line, but yet you need features that Omniture does not easily provide. Well, we can help. Our Omniture implementation guide (http://www.semphonic.com/analytics/impguides.asp) provides all sorts of tips on how to get the most out of Omniture without going below the line. We also provide an implementation review that provides solutions that don't require going below the line. In the longer run, let's hope that Omniture will learn from Google and start thinking of itself as a web analytics platform and not as a provider of all things to all people -- an impossible task.

Any features you'd like to see in Omniture that a 3rd party developer could add if there was a robust API. Do you have any horror stories related to modifications to the js file?

Omniture and Web 2.0

Now that Omniture is on the verge of swallowing Visual Sciences, the question is how well will Omniture move from Web 1.0 to Web 2.0. Will the Visual Sciences acquisition help or will it distract? Another important question is what will Web 2.0 measurement look like? Will it be supported by a powerful new application or will it be supported by a platform that accepts components from a wide range of third-party developers? My guess - and it's just guess - is that Web 2.0 will eventually go the way of a te platform with components from others.

Why, because there are too many possibilities in Web 2.0. Web 2.0 measurement will have address a wide range of challenges including mobile device support, qualitative in addition to quantitative analysis, integration of multiple data sources, rich media, as well as address issues of visitor identification. Making Web 2.0 measurement even more complicated, most of these challenges will not have just one solution. One size solution fits all solutions won't cut it in Web 2.0 and beyond. I don't think anyone vendor will be able to be all things to all web sites. If they try, they won't succeed. Rather the vendor that develops a te platform that accepts a wide range of third-party add-ons will be enormously profitable.

In their Wall Street Journal article, Strategies for Being a Platform Leader, September 27, 2007, Annabelle Gawer and Michael A. Cusumano write about what it takes to be a platform leader. Two of the qualities of a te platform include solving "an essential technological problem for many players in an industry" and "be[ing] easy to connect to or build upon." In addition, a successful platform leader must "create economic incentives that encourage other firms to develop complementary applications for the platform, and at the same time protect its own ability to profit from its innovations. This balancing act is perhaps the greatest challenge to platform leadership."

The question is can Omniture do this. So far, these qualities have not been their strengths. However, now that they are acquiring a competitor will they have the vision and the ability to move from a killer application to a te platform? Will the Visual Sciences acquisition go to their heads and prevent them taking the steps needed for success in Web 2.0 What do you think?

Omniture Acquires Visual Sciences: Don't Panic, But Don't Hide Your Head In The Sand

The announcement of Omniture's acquisition of Visual Sciences has created quite a stir. My advice is "Don't panic, but don't hide your head in the sand." Those of us immersed in web analytics are analysts, so let's analyze the situation.

For Visual Sciences clients, issues include moving to a new analytics platform with similar yet different features and capabilities, a different contract, different performance and a different support structure. While Omniture clients may assume that this won't affect them, they could be wrong. Omniture may be distracted by the effort to digest the Visual Sciences acquisition and performance and support may suffer. To aid the migration of Visual Sciences clients it may add features such as the HBX Report Builder which would be a very nice improvement.

With a larger customer base, Omniture may change its pricing structure. As Phil Kemelor points out in his Blog, Omniture's digestion of Visual Sciences is going to take some time and take some unexpected turns. (Omniture's digestion of Instadia's 200 clients is projected to take 18 months - how long will it take to digest Visual Sciences 2,000 clients).

Take advantage of the time it's going to take and take charge of your own fate - this advice is for both Visual Sciences and Omniture clients. What should you do?

1. Make sure your analytic house is in order (both Visual Sciences and Omniture clients);
2. Determine when and under what circumstances you want to migrate (Visual Sciences clients);
3. Keep abreast of changes in web analytics platforms, both technical and contractual (both Visual Sciences and Omniture clients).

#1 Make Sure Your Analytic House Is In Order

1. Using a structured perspective on your web metrics (business objectives to web KPIs to supporting statistics to application implementation) make sure that you are getting the information you need and you understand how your measurement application (Visual Sciences or Omniture) is providing that information;
2. Make sure you have meaningful standards and conventions for your web measurement. If you don't have any or if they are inadequate initiate an effort to remedy that;
3. Make sure that data collection is separated from data reporting and analysis. For example, if you are using an Excel interface, make sure that calculations and formatting for reporting and analysis are done in different worksheets than data collection.

#2 Determine When and Under What Circumstances You Want to Migrate

You got plenty of time, so take advantage of it and migrate when it makes sense for you. For example, if you are going from an HTML based web site to a Flash based web site, you are going to have to change your tagging for Flash whether you migrate or not so why not do the migration in conjunction with the move to Flash. If you don't have good web metrics standards and conventions, establish them and do their implementation as part of the migration. If you're not moving to a new web technology, don't need to improve your measurement standards and conventions, and don't need any new features, wait. "If it works, don't fix it." If you do migrate, consider out sourcing the migration, as it is not an expertise that you will need on an ongoing basis.

# 3 Keep Abreast of Changes in Web Analytics Platforms, Both Technical and Contractual.

Wait, watch, and listen. Monitor Omniture's digestion process of Visual Sciences for technical and contractual issues. Get your information not just from Omniture, but also from clients, analysts, and consultants. Also, keep abreast if what other vendors such as WebTrends are doing.

When migration makes sense for you -- technical, contractual, or support reasons -- make sure you know what you have, what you want and how your going to get there. Thus armed, migration may turn out to be a blessing in disguise. For Omniture clients, this may be a period of "interesting times" with lots of changes. Make sure you're set-up to take advantage of new features. Speak-up if performance or support suffers. Keep Omniture "honest" by monitoring industry developments.

You use your analytic skills to optimize your web site. Use them to optimize the Omniture acquisition of Visual Sciences for you.

Web 2.0 and the Opportunity for Web Measurement Standards

We depend upon standards and conventions all of the time. Often we don’t even realize that we do at least until they are not followed or understood. In the U.S., a raised hand, palm facing out means stop. In Iraq, it means “greetings,” “hi,” or “hello.” When a U.S. soldier wants a car with Iraqis to stop and raises his hand, the Iraqis think he is waving hello and keeps on driving, often to their detriment.

In accounting there are definite conventions regarding the classification of money and resources. Revenue and expense, asset and liability accounts are clearly differentiated. These conventions are essential for meaningful financial measurement and reporting. Executives that don’t follow these conventions can, as Enron executives found out, end up in jail. While not having or not following conventions and standards in web measurement won’t get you killed or land you in jail, it will make web measurement and analysis more difficult and prevent you from maximizing its benefits.

Except for some very general measures such as page views and sessions (even these are not always the same) there are generally no accepted web measurement standards and conventions across web sites. Even in a single web site there are more often than not no clear, well followed standards. Looking at URL names is frequently not unlike an archeological excavation. If directory “x” was used, it was when Joe was web master. If directory “y” was used, it was when Jane was web master. What’s wrong with that? It makes some of the most meaningful analyses difficult if not impossible.

For sites with large number of pages, most big sites, analysis on an individual page basis, except for pages like the home page, is not meaningful. There are just too many pages. What is meaningful, however, is analysis by content group. An online brokerage site wants to know whether information on fees is more effective than information on trading tools in getting prospect to sign-up. If all the “fee” pages are not in a single directory and all the “tool” pages in their own directory, it is difficult if not impossible to compare “fee” and “tool” performance, especially if measurement analysts are not informed when a new page is added that does not adhere to a standard.

And then there are links. If you don’t know how links are used, you haven’t a chance of knowing how well or how poorly your site’s navigation is working. More importantly, you don’t know how to improve it. I’ve seen sites where all of the links on a page were named “View Details,” impossible to differentiate, impossible to improve.

Because most major web metrics tools are based on actual names (URL, links, etc.) it is difficult to retrofit a measurement on a site without a naming convention. On the other hand, because web 1.0 sites do have names for their pages and links, there is at least something that can be measured even if it is not optimal.

With web 2.0 rich-internet-application (RIA) techniques such as Ajax, or Flash, if nothing is done to implement measurement, there will be no measurement. A site implementing a RIA solution is starting with a clean measurement slate that will be empty if nothing is done. So with web 2.0 you must take some action if you want measurement. There is no default, not even a bad one. If you have to do something, you might as well do it right.

What’s right? First of all follow the KISS (keep it simple s…..) principle. Establish simple, clear standards / conventions for naming and categorizing. If they’re not simple and clear, no one will follow them. Make sure they are flexible enough to measure what’s needed today, but also flexible enough to accommodate future requirements (future requirements that are not yet known). Make sure that everyone involved with your web site from designers to programmers to QA is aware of, buys into, and commits to the standards. Make sure that there is a process in place that supports their use.

While every site will have different requirements there are a few basic things that need to be tracked.

• Base pages. In web 1.0 almost everything a viewer saw was a page. In web 2.0. Pages are more like platforms on which events take place. So while there are potentially not as many pages, they are important. An event that takes place on one base page needs to be differentiated from an event that takes place on another base page;
• Events. Events nare actions that take place on a base page without going to another base page. Event names will have two parts. First, the name of base page on which it occurred. Second, a name that is descriptive and consistent across the entire site. For example, deleting a column in a report would have “delete” in the name as well as the column being deleted;
• Links. Links are actions that take viewers from one page to another (either on the site or off the site). The link name should consist of the from-page name (the page on which it originates), a to-page name (the page to which it takes a viewer), as well as link name that is descriptive and consistent across the entire site. The combination of these three elements should be unique across the site;
• Content Area. All pages should belong to a content area so that they can be meaningfully combined and compared and usage statistics correctly aggregated. Content areas can be either hierarchical or flat depending on site requirements. (For a more detailed discussion of content types checkout Functionalism at http://www.semphonic.com/functionalism).

I’m sure there are lots of things I’ve forgotten, overlooked, and omitted. I welcome all comments and suggestions. There is still a lot of thinking and work to be done to effectively measure and analyze web 2.0 sites.

Customer Satisfaction

Our financial services clients are continually struggling with how to measure success on their sites that provide online services as opposed to their prospecting sites. Online service sites very often don’t have definitive success criteria like a prospecting or e-commerce site does.

On an e-commerce site selling a product is a success. On a prospecting site getting a prospect to sign-up or enroll is a success. However, on online services sites where customers do tasks such as manage their investments, bank accounts, or credit cards it’s different. Is it better if customers to come to the site more often or less often? It depends. Is it better if customers process more transactions? It depends.

What really matters is how satisfied they are with the online service. One way to judge customer satisfaction is by customer loyalty. Do your customers continue to do their online processing over time? The time frame necessary to tell depends on the service. Depending on the purpose of a financial services account, a loyal customer may log-on daily, weekly, monthly, or just twice a year. In some cases, account balance is an important success criteria but it requires merging online with off-line information which in many cases is difficult to do.

Another method of measuring customer satisfaction is the direct method – just ask them. We have a large contingent of financial services clients who use online satisfaction surveys to see how well they’re doing with their online services and it makes good sense. It’s inexpensive, flexible and has rapid turnaround. However, it can be misleading for several reasons.

First of all there is the sample composition issue. Most online satisfaction surveys are random and reflect the composition of online visitors. That sounds good, but it is not necessarily so. Consider an online service site where 20% of the visitors generate 80% of the revenue. That means that 80% of the visitors generate only 20% of the revenue. In many cases these two segments have very different online requirements. If they do, it makes most sense to make site enhancements for the 20% of customers who generate 80% of the revenue. If you do that and the enhancements are successful it’s very possible that your online satisfaction survey rating will go down. In fact that’s what happened with one of our clients.

They made significant improvements for their most profitable customers and their satisfaction ratings dropped. Why, because it actually made it more difficult for their more numerous but least profitable customers. Fortunately, we had enough information on survey respondents so that we were able to segment the survey responses by customer profitability.

Segmented by profitability, we were able to show that satisfaction rating increased for the most profitable customers even though it decreased overall. We are now developing a measurement system that will enable us to weight each individual response by profitability so that the web site owner can determine if the increase in satisfaction of the most profitable customers is enough to offset the decrease in satisfaction of the least profitable segment.

Segmentation, segmentation, segmentation. The more we work with satisfaction survey results the more, we realize the importance of meaningful segmentation. In many cases, length of usage is an important segment in understanding customer satisfaction with online services. Long time site users often react differently than new users to site changes. This raises the question of what to do.

This is a tough question to answer. Long time customers may have larger account balances and may appear to be more profitable. However, new customers may be “checking-out” services and have considerable long-term potential and a higher life time value. If satisfaction survey results differ by length of usage, we recommend developing a “first time user experience” which is different than the “long-term user experience” and addresses first time user concerns. Other segments we have found useful in evaluating satisfaction survey results include frequency of usage and usage of other similar services.

Another important factor in satisfaction surveys is outages. It sounds obvious, the more a site is down, the lower the satisfaction survey results. However, it’s not always so clear cut. We have a client who looked at the user satisfaction survey results on the two or three days following a site outage and could find no significant correlation with site outages and user satisfaction. Did users really not care whether the site was up or down? To find the answer, we looked at outages and satisfaction over a 6 month period and made a few adjustments.

Since this site had a very low weekend usage, we discounted weekend outages. Next we allocated outages that occurred at the end of a month between the month in which they occurred and the following month. Finally, we took into account the cumulative number of outages. With these few adjustments, we found that there was an extremely high correlation between site outages and user satisfaction. In this particular case, the best use of web resources was not site redesign but improving site availability.

What’s the next step in improving the value of online? Well, I’m segmentation prejudiced. So I recommend to all of our clients that use or want to use online satisfaction surveys that we work with them to integrate their satisfaction surveys within their measurement systems so that they can segment survey results by meaningful, actionable segments. One thing to be careful about, as you start looking at different segments are sample sizes. Are the segmented sample big enough for meaningful conclusions? If not you may want to consider a larger total sample or serving the sample to random samples of specified segments.

All said and done, customer satisfaction can be measured, but as with all measurement some upfront planning, attention to details and an understanding of your customers makes the difference between meaningful and misleading results.