My corporate duties these days at Testware Associates tend towards the managerial-responding to requests for proposals, comfortably relaxing in my corner office while others do the real work, waxing poetically in StickyMinds.com articles. However, I sometimes forego management nirvana to participate in several of the many and varied testing engagements under way. This article will focus on cookies and cookie testing techniques that Testware has employed in Web testing.

Stateless and Stateful Systems
I must start our adventure with a few textbook definitions. According to whatis.com, a stateless system has "no record of previous interactions and each interaction request has to be handled based entirely on information that comes with it." On the flip side, a stateful system does keep a record of previous interactions.

To elaborate on stateless systems, we will consider the Hypertext Transfer Protocol (HTTP), which is the protocol used to exchange files on the World Wide Web. HTTP was used by your Web browser and the StickyMinds.com Web server to display the very page you are reading now.

The Stateless HTTP
Basically, the conversation between your browser and the StickyMinds server using HTTP went like this:

Your browser: "Hey StickyMinds Web server! Can I please have the page http://www.stickyminds.com/sitewide.asp…"

StickyMinds server: "Yes. The document you requested does exist."

StickyMinds server: "Here is the text of the document:."

 

Once your browser received the last byte of this page using HTTP, the server essentially "forgot" about what you had just done. If you now go elsewhere on the StickyMinds site, the StickyMinds server responds to your new request as above, without memory of your earlier request. This isn't a bad thing for the StickyMinds site; no harm, no foul. It does not need to know anything about your earlier request in order to respond to your new request. But are there cases in which state does matter for a Web-based system?

To State or Not to State on the Web (with apologies to Shakespeare)
As we'll see, state certainly does matter on the Web! Take everyone's favorite destroyer of shareholder's equity and purveyor of books and music, amazon.com. If there weren't ways to overcome the stateless nature of HTTP and maintain state on the Web, the following would not be possible:

FIGURE 1 Welcome screen

• The nice "Hello, " message that greets returning shoppers on the Amazon home page. Without state, how could the site have any knowledge of your name?
• The virtual shopping cart. In the absence of state, how could Amazon keep track of the individual items I add to my cart and the quantity of each item?

FIGURE 2 Shopping cart

So, how does amazon.com work around the stateless HTTP protocol to accomplish the "magic" above? Part of the answer is cookies.

Maintaining State with Cookies
Whatis.com notes that a cookie is "information a Web site puts on your hard disk so that it can remember something about you at a later time."

How and where cookies are stored depends on the browser and operating system you use. Internet Explorer (IE) stores each cookie in a separate file, usually underneath the Windows operating system folder. Netscape stores all cookies in a single file named cookies.txt, usually in a folder underneath the browser's installation folder.

Per-Session Cookies and Cookie Expiration
When a Web server sets a cookie on your system, it can optionally give that cookie an expiration date. As time marches on, any cookies with expiration dates in the "past" are deleted.

If the Web server does not give a cookie an expiration date, that cookie is a per-session cookie. Per-session cookies are deleted when you close your Web browser; they only exist for the single Web surfing session beginning when you start the browser and ending when you close the browser.

Cookie Usage by Amazon.com
Let's make the cookie concepts we've discussed more concrete by examining how amazon.com uses cookies. In doing so, we will also encounter a common problem in "cookie testing"-figuring out what the hieroglyphic-like information in the cookie means! We'll navigate through the site to discover where cookies are employed.

To start, I deleted all Netscape cookies from my PC and set the cookie option to prompt me whenever the Web site sets a cookie. Next, I navigated to www.amazon.com.

We get a prompt indicating that the site wants to set a "session-id" cookie. I then open Netscape's cookies.txt file and copy/paste the cookie details into a "cookie log" with my observations for later analysis. A word of warning: some sites are highly active with cookies, setting or modifying them on every page you visit. Creating the cookie log on these types of sites will be time consuming and drive you to a certain level of insanity. Getting as much info as possible in advance about cookie activity from the developers is your best bet in this situation.

So we record the following data for the first cookie.

.amazon.com TRUE / FALSE 994320128 session-id 102-7224116-8052958

The prompt that Netscape presented me with indicated the cookie will expire on Thursday July 5, 2001, one week from my visit. (We'll explore the details in the next two sections.)

The second cookie set by amazon contained the following data and also expires
on 7/5/2001.

.amazon.com TRUE / FALSE 994320181 session-id-time 994320000

Amazon's third cookie contained the following and expires on 1/1/2036. Since my laptop will be reduced to either paperweight or landfill status by then, this is pretty much a "permanent cookie" relative to the useful life of my laptop.

.amazon.com TRUE / FALSE 2082787330 ubid-main 077-4356846-2652328

The fourth cookie is a per-session cookie, since the Netscape prompt did not include an expiration date. Since per-session cookies aren't written to the hard drive, examining the cookie content can be done only through the actual Netscape prompt.

FIGURE 3 Per-session cookie

The fifth cookie expires on 1/1/2036 and contained the following data.

.amazon.com TRUE / FALSE 2082787787 x-main hQFiIxHUFj8mCscT@Yb5Z7xsVsOFQjBf

After accepting this fifth cookie, the amazon.com home page (finally!) displayed. The URL of the home page was http://www.amazon.com/exec/obidos/subst/home/home.html/102-7224116-8052958.

Have we seen that number sequence at the end of the URL before? Yes, it's the session ID stored in the first cookie.

A sixth cookie containing the following data and expiring on 6/29/2001 was then set.

http://www.amazon.com/ FALSE / FALSE 993797034 seenpop 1

Upon accepting this cookie, a secondary browser window popped up with a free shipping promotion notice. A logical guess at this cookie's purpose, then, would be that it tracks whether or not you've seen the promotion popup ad.

After all of these cookies were set, my Netscape cookies.txt file looked like this:

FIGURE 4 Cookie file

Why are there are only five cookies in the file? The per-session cookie is kept in memory only; it is not written to the cookies.txt file.

So What's Inside a Cookie?
Before we attempt to analyze all of the cookies set by amazon.com, let's take a quick look at cookie structure and the meaning of cookie data.

The first cookie set by amazon was:

.amazon.com TRUE / FALSE 994320128 session-id 102-7224116-8052958

Using the information at www.cookiecentral.com, I'll break the cookie down into its individual fields from left to right and describe what each field is used for.

• .amazon.com is the domain for which this cookie is valid. Only cookies set by machines in the amazon.com domain can read this cookie. (Note: Bugs in Web browser cookie implementation have allowed unauthorized sites to access cookies in the past.)
• FALSE (1^st occurrence) is a flag indicating whether or not the cookie can be accessed by all machines in the domain.
• / is the path the cookie is valid for.
• FALSE (2^nd occurrence) is a secure flag indicating whether or not a secure (encrypted) connection is needed to access the cookie.
• 994320128 is the UNIX expiration time of the cookie. UNIX time is the number of seconds since January 1, 1970 00:00:00 GMT.
• session-id is the name of the variable stored by this cookie.
• 102-7224116-8052958 is the value of this cookie.

Amazon.com Cookie Analysis
Our cookie experiment for amazon.com showed us that simply loading the Amazon home page creates six cookies: one per-session (non-persistent) cookie and five persistent cookies. Since the site design documents and developers are unavailable to us, let's put the cookie data into a table and try to decipher what the cookies are used for and the meaning of the cookie data. We will consider only the first five cookies, since we determined the purpose of the sixth cookie (www.amazon.com FALSE / FALSE
993797034 seenpop 1) is to "remember" if the user sees the promotional popup.

cookie #	domain	accessible by all machines	path	secure connection needed	expiration	name	value
1	.amazon.com	TRUE	/	FALSE	994320128	session-id	102-7224116- 8052958
2	.amazon.com	TRUE	/	FALSE	994320181	session-id-time	994320000
3	.amazon.com	TRUE	/	FALSE	2082787330	ubid-main	077-4356846- 2652328
4	.amazon.com	TRUE	/	FALSE	(per-session cookie)	obidos_path	(see figure 3)
5	.amazon.com	TRUE	/	FALSE	2082787787	x-main	hQFiIxHU Fj8mCscT @Yb5Z7xs VsOFQjBf

The first cookie is a session ID assigned to my shopping session by the amazon server. The primary giveaway here is the variable name "session-id." (I can hear you saying, "Thanks for the revelation, Rich.") Another clue is that the data in its value field, 102-7224116-8052958, can be found at the end of the home page URL which appeared after the 5^th cookie was set, www.amazon.com/…/home.html/102-7224116-8052958. Cookie 1 expires on 7/5/2001, based on the warning dialog I saw in Netscape before the cookie was set. So the UNIX expiration time 994320128 in this cookie must  orrespond to 7/5/2001.

The second cookie's purpose isn't obvious. Based on its name and value, session-id-time and 994320000, respectively, I would guess it is the maximum possible "end" time in UNIX time of my amazon.com session. I know from the Netscape warning above that this cookie expires on 7/5/2001, so I can infer that the expiration value of 994320181 in this cookie also corresponds to 7/5/2001 (approximately 3 minutes from the UNIX time listed in the previous cookie).

The purpose of cookies 3 and 5 is yet even harder to decipher. The names of cookies 3 and 5, ubid-main and x-main, don't lend us any immediate understanding. Both of these cookies expire in 2036, so whatever amazon is tracking here, it must be of long-term use.

The fourth cookie, the only per-session / nonpersistent cookie, contains a long value with the substring "continue-shopping-url." I would guess this cookie value tells the amazon Web server where to send me if I click the "Continue Shopping" button on the shopping cart page. As before, I'm having a hard time determining with certainty what this cookie is used for without further investigation.

I'd have to talk to the amazon developers, possibly bribe them with pizza/beer, or get access to some design documents or specifications to get any further here.

Cookie Tasting, er, Testing
Now that we're in the know about what cookies are, how they're used to provide state in Web systems, and what cookie contents look like, let's address how to test sites that use cookies.

1. Disabling Cookies
This is probably the easiest area of cookie testing. What happens to the Web site if all cookies are disabled? Start by closing all instances of your browser and deleting all cookies from your PC set by the site under test. The cookie file is kept open by the browser while it's running, so you must close the browser to delete the cookies. Closing the browser also removes any per-session cookies in memory.

Disable all cookies and attempt to use the site's major features and functions. Most of the time, you will find that these sites won't work when cookies are disabled. This isn't a bug, but rather a fact of life: disabling cookies on a site that requires cookies (of course!) disables the site's functionality.

With cookies disabled, your testing job is somewhat reduced. Can the user perform any operations on the site? Is it obvious to the Web site user that he must have cookies enabled to use the site? Is the Web server recognizing that its attempts to set cookies are failing? If so, does it send a page to the user stating, in plain language, that cookies must be enabled for the site to work? Or can the user frustratingly attempt the same operation many times in a row without a clue as to why the site isn't working?

Amazon.com passes this test and then some. I was able to use all major aspects of the site-searching, shopping cart, checkout functions-even though cookies were completely disabled. It appears that state maintenance was being taken care of server-side, based on the session ID at the end of the home page URL. For example, I chose the Yamaha CD-ROM kit on the amazon home page and added it to my shopping cart. The shopping cart page URL was  www.amazon.com/…/one-click-thank-you-confirm/107-0357560-1728507. Changing the rightmost digit from 7 to 8, and posting this edited URL, lost my shopping cart and brought up the following error page, lending further support to the probability of server-side state maintenance with a session ID in the URL.

 

FIGURE 5 Bug alert

This server-side state maintenance allows someone to shop at amazon.com even if they have totally disabled cookies-an intelligent design. If cookies are enabled, though, we saw previously that amazon sets a session ID cookie to "remember" your session ID. Why? If you leave the site with something in your shopping cart and then return, the session ID cookie is used to resume your previous shopping session and shopping cart state.

2. Selectively Rejecting Cookies
What happens to the site if some cookies are accepted and others are rejected? Start by deleting all cookies from your PC set by the site under test and set your browser's cookie option to prompt you whenever a Web site attempts to set a cookie. Exercise the site's major functions. You will be prompted for each and every cookie the site attempts to set. Accept some and reject others. (Analyze site cookie usage in advance and draw up a test plan detailing what cookies to reject/accept for each function.) How does the site hold up under this selective cookie rejection? As above, does the Web server detect that certain cookies are being rejected and respond with an appropriate message? Or does the site malfunction, crash, corrupt data, or misbehave in other ways?

Let's strategize a selective cookie rejection test for the amazon.com home page. Each test case will require either accepting or rejecting each of the six cookies, so there are 2^6 = 64 possible test cases. A few of the test cases are enumerated in the following table, proceeding as if counting in binary with "reject" being a 0 value and "accept" being a 1 value.

 

test case #	cookie 1 (persistent)	cookie 2 (persistent)	cookie 3 (persistent)	cookie 4 (per session)	cookie 5 (persistent)	cookie 6 (persistent)
1	reject	reject	reject	reject	reject	reject
2	reject	reject	reject	reject	reject	accept
3	reject	reject	reject	reject	accept	reject
4	reject	reject	reject	reject	accept	accept
5	reject	reject	reject	accept	reject	reject
…
64	accept	accept	accept	accept	accept	accept

If I were to run the fourth test case, for example, I would reject the first four cookies every time amazon.com tries to set them, but allow amazon to set the fifth and sixth cookies.

Based on amazon's performance in the disabling cookies test, I would guess that the site would pass most or nearly all of the selective cookie rejection test cases. The first test case is equivalent to the disabling cookies test performed previously, but I'll leave it in the table for completeness. I executed test cases 2 and 5, closing the browser and deleting the cookies before starting each test case. Both passed: I was able to use the site's major functions, as above, without problem. Looks like the site designers ensured that "problems" with cookies would have little or no effect on a customer's ability to shop at  amazon.com.

Note that the test cases above only deal with the cookies being rejected or accepted when amazon.com first tries to create them. We also should test rejecting and accepting cookie modifications. Allow a cookie to initially be set. If/when the Web server attempts to subsequently modify that cookie, what happens if you disallow the change, retaining the "old" value?

3. Corrupting Cookies
Now's our chance to really abuse the site under test! Exercise the site's major features. Along the way, as cookies are created and modified, try things like

• Altering the data in the persistent cookies. Since the per-session cookies are stored only in memory, they aren't readily accessible for editing.

  First example: in the first cookie written by amazon.com, change the name session-id to something different, perhaps ses-id or sexqion-id. Remember, you will have to close the browser to edit the cookies. After editing the cookie, restart the browser and reload/continue using the site. Did the corrupted cookie cause the site to malfunction? Is any data lost or corrupted in the database? If I visit the amazon site, close the browser, restart the browser and go back to amazon.com, my "previous" session is maintained based on the session ID in the cookie. However, if I corrupt the session ID variable name, amazon detects the corruption and recovers by discarding all six of the cookies and recreating them with new values.

  Second example: change the session-id value in the data field by adding 1 to the rightmost digit; 102-7224116-8052958 becomes 102-7224116-8052959. Are you now looking at someone else's shopping session? Anything lost or corrupted in the database?

• Selectively deleting cookies. Allow the cookie to be written (or modified), perform several more actions on the site, then delete that cookie. Continue using the site. What happens? Is it easy to recover? Any data loss or corrupted?

4. Cookie Encryption
The last cookie test I'll mention is a simple one. While investigating cookie usage on the site you're testing, pay particular attention to the meaning of the cookie data. Sensitive information like usernames and passwords should NOT be stored in plain text for all the world to read; this data should be encrypted before it is sent to your computer. I've tested many sites where this seemingly obvious rule has been violated. A case can certainly be made that certain types of sensitive data-credit card numbers, for example-should never be stored in cookies, even encrypted.

Based on the amazon.com cookie analysis we performed above, I'd say amazon easily passes the cookie encryption test. No sensitive user or credit card information is stored in plain text.

Wrap Up
State information can be maintained in Web systems by the use of cookies. Other methods for maintaining state include hidden form fields and embedding state data in HTML links; I recommend that Web testers explore these methods as well. Our job as testers is to find out, by talking to developers, reading system documentation, or experimenting with the Web site, which of these technologies are being used and to design tests accordingly.

Further Information

• www.whatis.com (excellent Web-based technical encyclopedia)
• www.cookiecentral.com (tons of info about cookies)
• Persistent Client State HTTP Cookies Preliminary Specification
• "Behind Closed Doors: What every tester should know about Web privacy"; by Russ Smith, January/February 2001 issue of STQE magazine (discussion of HTTP GET and POST methods and cookies)
• "E-Business Testing: Test Techniques and Tools -
  Risk-Based E-Business Testing Part 2"; by Paul Gerrard, posted 2/7/2001 on
  StickyMinds.com (discussion of HTTP GET and POST methods, cookies, hidden form fields, and security testing).