2

Abstract

Folder navigation is the main way that personal computer users retrieve their own

files. People dedicate considerable time to creating systematic structures to facilitate

such retrieval. Despite the prevalence of both manual organization and navigation,

there is very little systematic data about how people actually carry out navigation, or

about the relation between organization structure and retrieval parameters. The aims

of our research were therefore to study users’ folder structure, personal file navigation,

and the relations between them. We asked 296 participants to retrieve a total of 1,131

of their active files and analyzed each of the 5,035 navigation steps in these retrievals.

Folder structures were found to be shallow (files were retrieved from mean depth of

2.86 folders), with small folders (a mean of 11.82 files per folder) containing many

subfolders (M = 10.64). Navigation was largely successful and efficient with

participants successfully accessing 94% of their files and taking 14.76 seconds to do

this on average. Retrieval time and success depended on folder size and depth. We

therefore found users’ decision to avoid both deep structure and large folders to be

adaptive. Finally, we used a predictive model to formulate the effect of folder depth

and folder size on retrieval time, and suggested an optimization point in this trade-off.

3

Personal file navigation (navigation for short) is a two-phase process. First, users

manually traverse their organizational hierarchy until they reach the folder in which the target

file is stored. Second, they locate the file within that folder (Bergman, Beyth-Marom,

Nachmias, Gradovitch, & Whittaker, 2008).

Most information retrieval research has focused on public data sources such as

databases, libraries and the web, developing various theories and methods for organizing and

retrieving such public information. Yet all of us expend considerable effort organizing and

accessing our personal information, using predominantly manual methods to prepare for

subsequent retrieval. Surprisingly little is known about this process, in terms of how

successful people are at organizing and retrieving their personal data.

This paper therefore attempts to empirically investigate various questions relating to

navigational retrieval, personal folder organization and the relationship between them. We

present large-scale quantitative data about: a) participants’ folder structure and organizational

strategies; b) navigation success and efficiency; and c) the effects of folder structure on

retrieval success and efficiency. In contrast to previous research that focused on file structure

alone, our study also quantitatively investigated file navigation retrieval in a natural setting,

and examined the effect of structure on folder navigation.

There has been some prior research on how people organize their personal

information. Early studies looked at the organization of personal paper archives (Malone,

1983; Whittaker & Hirschberg, 2001) finding two prevalent strategies: filing and piling.

Because of the characteristics of filing cabinets and folders, early studies found only few

instances of complex subfoldering of paper archives (Cole, 1981). More recent work has

documented organizational strategies across different types of digital data, detailing how

people organize emails (Whittaker & Sidner, 1996), web data (Abrams, Baecker, & Chignell,

1998; Tauscher & Greenberg, 1997), photos (Kirk, Sellen, Rother, & Wood, 2006),

documents (Gonçalves & Jorge, 2003; Hardof-Jaffe, Hershkovitz, Abu-Kishk, Bergman, &

Nachmias, 2009a, 2009b; Henderson & Srinivasan, 2009; Jones, Phuwanartnurak, Gill, &

4

Bruce, 2005) or common strategies across all these data types (Bergman, Beyth-Marom, &

Nachmias, 2006, 2008; Boardman & Sasse, 2004). While such studies have looked at how we

manually organize personal information, less attention has been paid to how people exploit

these structures to access that information.

Some recent studies document the problems people experience with organizing

personal information. People find it hard to organize emails, making folders that are either too

big or too small (Fisher, Brush, Gleave, & Smith, 2006; Whittaker & Sidner, 1996). For

example, Whittaker and Sidner (1996) found that almost 40% of email folders contain 2 or

fewer items and Henderson and Srinivasan (2009) showed that 8% of file folders created are

empty, showing that people create structures that they fail to actively exploit for organization.

In contrast, with digital photos, people create large folder structures that contain

heterogeneous pictures from many different events, making it hard to find older digital photos

(Whittaker, Bergman, & Clough, 2009). Other studies show that web bookmark folders are

often not useful in supporting retrieval of web documents (Abrams et al., 1998; Aula, Jhaveri,

& Kaki, 2005; Tauscher & Greenberg, 1997). And when users are asked to explain their

organization in PIM (personal information management) ‘desktop tours’, they usually express

dissatisfaction and modify their organization as they give the tour (Boardman & Sasse, 2004;

Whittaker & Sidner, 1996).

One response to these organizational problems has been to propose a move to desktop

search. Much novel desktop search technology has been developed over the last few years,

e.g. Google Desktop, Microsoft Windows Search, and Macintosh Spotlight. According to its

advocates, desktop search promises to minimize users’ organizational problems, because it

reduces the need to manually organize personal information, which is automatically indexed

by the search engine. Search has other potential advantages: it allows flexible and efficient

ways to query one’s personal information (Cutrell, Dumais, & Teevan, 2006; Russell &

Lawrence, 2007). Despite its promise, however, various studies still show a strong preference

for navigation over search when both are available for accessing personal information

5

(Barreau & Nardi, 1995; Boardman & Sasse, 2004; Kirk et al., 2006; Teevan, Alvarado,

Ackerman, & Karger, 2004). Moreover, the use of improved search engines has been shown

to have little effect on this preference (Bergman et al., 2008). Bergman et al. (2008) showed

that regardless of search engine quality, there was a strong preference for navigation. Search

was predominantly used as a last resort only when users could not remember the location of a

file. There was also little evidence that using improved desktop search leads people to change

their filing habits to become less reliant on hierarchical file organization.

It therefore seems that (at least for the foreseeable future) manual file organization

and navigation will be critical PIM behaviors. This paper therefore attempts to explore and

quantify various research questions relating to three topics: folder structure, navigation

performance and the effect of structure on retrieval.

Folder structure

There are important trade-offs to be made in organizing files and folders. Folder

hierarchies may lie between two extremes: (a) broad and shallow or (b) deep and narrow.

Broad shallow hierarchies allow faster access to folders, but increase the time needed to scan

within each folder. In contrast, deep narrow hierarchies allow faster scanning of each folder,

but users have to access more folders overall. Previous work is inconclusive about which of

these strategies people most commonly use.

In an early study, Barreau (1995) studied 7 participants using DOS, OS/2, Windows,

and Macintosh operating systems. Only three of her participants used folders at all, the other

four grouped their files simply by placing them on separate floppy disks. More recent studies

have generated contradictory findings about the structure of personal file systems. Gonçalves

& Jorge (2003) studied the folder structure of 11 computer scientists using Windows (8),

Linux (2) and Solaris OS (1). Their results show extremely deep, narrow hierarchies. The

average directory depth was found to be 8.45, with an average branching factor (which is an

estimate of the mean number of subfolders per folder) of 1.84. In contrast, a larger scale study

by Henderson and Srinivasan (2009) looked at the folder structure of 73 university employees

6

using Windows OS. The structures they found were much shallower, being only 3.4 folders

deep on average (similar results were obtained by Boardman & Sasse, 2004). Folders tended

to be broader with an average of 4.1 subfolders per folder, for non-leaf folders. Both studies

found relatively small numbers of files per folder: 13 for Gonçalves & Jorge (2003) and 11.1

for Henderson & Srinivasan (2009).

However, one significant limitation of the above studies is that they examine the

user’s entire folder archive, which may contain thousands of inactive files in archival

structures that have not been touched for years. For example, Gonçalves & Jorge (2003)

document that over half the files in the users’ system had not been modified for over a year.

Instead, our study focused on active parts of the structure from which the user had recently

retrieved files. Other work has documented a strong tendency to access recent personal

information (Bergman et al., 2008, Dumais et al., 2003, Tang et al., 2008), and we wanted to

focus on these more typical access situations.

Our study investigated the following research questions regarding folder structure:

1.1 Depth: At what depth in the folder hierarchy are active files stored? – Are they stored in

deep structures as found in Gonçalves & Jorge (2003) or shallow ones as in Henderson &

Srinivasan (2009)?

1.2 Size: How big are file folders?

1.3 Internal Structure: How many subfolders and files are in each folder?

1.4 Relations between Structure and Depth: Does folder depth affect folder size, number of

subfolders and percentage of subfolders?

1.5 Subfoldering Distribution: What percentage of each folder is taken up with subfolders?

How is this subfolder percentage distributed across all folder items? What explains this

distribution?

7

Navigation Success

Prior research has consistently shown that navigation is the main way in which users

retrieve their files (Bergman, Beyth-Marom, Nachmias et al., 2008; Boardman & Sasse, 2004;

Kirk et al., 2006; Teevan et al., 2004). However, no prior research has quantified how people

actually navigate to their folders in their natural setting. Our study examined retrieval success

rate and the time users took to navigate to their files. We can interpret these results in terms of

users’ memory of their file locations.

Our research questions for navigation success were:

2.1 Success Rate: How often are participants successful in retrieving their files?

2.2 Factors Affecting Success: We collected information about retrieval strategies. We

examined the number of retrieval steps, i.e. the number of times a user opened a new folder,

as well as step duration – the time taken to scan each folder. We asked the following

questions: What is the distribution of retrieval outcome? How does retrieval outcome relate to

retrieval time, number of steps per retrieval and step duration? And what do these results

imply for users’ memory for file location?

The Effects of Structure on Retrieval

While prior work has documented different organizational strategies, it hasn’t

examined the effect of these strategies on retrieval. It seems, however, that there are trade-

offs in how users choose to organize their information. Broad shallow hierarchies reduce the

number of folders to be scanned, but increase the time to scan the contents of each folder. In

contrast, narrow, deep hierarchies reduce scan time per folder, but mean that more folders

have to be accessed overall.

Although the effect of structure on retrieval has not been examined for personal files,

it has been studied extensively for menu navigation (Jacko & Salvendy, 1996; Kiger, 1984;

Miller, 1981; Snowberry, Parkinson, & Sisson, 1984) and for Web page navigation (Furnas,

1997; Kim, Li, Moy, & Ni, 2001; Larson & Czerwinski, 1998; Shneiderman, 1997; Zaphiris

8

& Mtei, 1997; Zhang, Zhu, & Greenwood, 2004). Overall, breadth is better than depth in

terms of both error rate and retrieval time, i.e. choosing broad shallow hierarchies leads to

more effective retrieval. For example, Miller (1981) tested 4 artificial menu structures with 64

bottom level nodes: 26 (6 levels of depth with 2 items of breadth), 43 (three levels of depth

each with 4 items of breadth), 82 (two levels of depth with 8 items of breath) and 641 (64 top

level items). Of the four structures, the 82 supported fastest retrieval and lowest error rate.

These results suggest that some hierarchical organization reduces the visual overcrowding

found in the 641 structure; however, deep structures should also be avoided. Indeed later

studies (which did not test the ‘no hierarchy’ option) found that retrieval time is positively

correlated with depth for both menus and Web pages (Furnas, 1997; Jacko & Salvendy, 1996;

Kiger, 1984; Kim et al., 2001; Zaphiris & Mtei, 1997). For web design, a widely quoted

heuristic for navigation design is the “three clicks rule,” which states that the user should be

able to get from the homepage to any other page on the site within three mouse clicks,

arguing for shallow organizational structure (Zhang, Zhu, & Greenwood, 2004).

Our research questions for the effect of structure on retrieval were:

3.1 Folder Depth and Retrieval Time: Does folder depth affect retrieval time?

3.2 Folder Size and Retrieval Time: Does folder size affect step duration and retrieval time?

3.3 Folder Size, Folder Depth and Success: Do structural elements (folder size and depth)

affect retrieval success?

3.4 Predictive Modeling: How do folder depth and size predict retrieval time?

Method

Previous work examined organizational strategies in relatively small numbers of

participants. In contrast, in our study, to increase external validity, we collected data from

large numbers of users sampled in a naturalistic setting. The requirement for lightweight, non-

intrusive data collection led us to a procedure in which we recruited users and videotaped

their screens as they accessed files from their own computers. We did not install software on

people’s machines to record organization and retrieval behaviors. Installation is error prone,

9

and pilot interviews showed that users were concerned about its intrusiveness and potential

implications for their privacy.

Other studies have tried to profile people’s entire document collections (Gonçalves &

Jorge, 2003; Henderson & Srinivasan, 2009; Tang et al., 2007). However, this runs the risk of

cataloguing large numbers of documents that may not have been accessed for very long

periods. Instead, we wanted to look at typical access behaviors. Other research shows that

users tend to most frequently access recent information items regardless of whether these are

files, web pages or emails (Bergman, Beyth-Marom, Nachmias et al., 2008; Dumais et al.,

2003; Tang et al., 2008). We therefore videoed participants navigating to files in their Recent

Documents list, i.e. personal files that they had recently spontaneously retrieved and opened

from their own computers, as part of their everyday computer use. There were a number of

other important benefits to this approach. Focusing on recent files meant users were trying to

access files that we were confident were present on users’ disks and that were definitely

retrievable by the user. It also allowed us to identify active files without having to manipulate

or access participants’ file systems, avoiding encroaching on their privacy.

Participants

Participants were 296 everyday computer users: 163 males, 133 females. The large

majority of participants were students and employees at Sheffield University. The participants

were directly approached by the researchers in the university and students’ hall of residence

(non random selection). We knocked on their doors in the evenings and asked them to spare a

few minutes for the study. Participants’ ages ranged from 16 to 64 years (M = 26.44, SD =

9.63). The majority of participants were Windows OS users (246: 181 XP, 62 Vista, 3

Windows 2000), 43 used a Mac, and 7 used a Linux operating system.

Procedure

Participants used their own computers for the retrieval task. The tester printed out the

participants’ Recent Documents list, asking them to navigate to each file (the target) in that

10

list in order. Participants were asked to click on the target file once but not open it. We did

this to preserve users’ privacy as these files might contain sensitive information. Participants

were asked to close all folders before each navigation task took place, so that all retrievals

started from the desktop. Participants were asked to skip a file in the list when they had

already navigated to that target folder during a previous access task. We did this to prevent

access to these items being primed because that folder had already been accessed. We asked

our participants to access only files saved on their computer and to avoid retrieving files on

external drives (such as a memory stick) and email attachments that hadn’t been saved as files

on their hard drive. The procedure took approximately 10 minutes.

Retrievals

Our study includes 1,131 valid retrievals. Of the initial overall set of 1,158 recorded

retrievals, we excluded 2% that were deemed invalid for the following reasons: 15 retrievals

were interrupted by external events such as phone calls or instant messenger alerts. In a

further 6 retrievals, participants did not follow the above procedure (e.g. they moved the

mouse-pointer over the Recent Documents list to look up the file’s path instead of using the

printout); 3 participants used a library computer so the Recent Documents list did not contain

any of their personal files; for 2, the video recording was not clear enough to be analyzed, and

1 participant had deleted all files on the list prior to the experiment.

The target files of these retrievals were in various formats: 469 text files (e.g., doc

files), 160 pictures (e.g., jpg files), 126 pdf files, 64 Excel files, 49 MP3 files, 40 PowerPoint

files, 28 video clips (e.g., avi files), 16 SPSS files, 14 html files, 48 files in unidentified

format and 117 files in other, less common formats.

Retrieval Time Measurements

Recordings of user interactions were made using a high definition digital video

camera (1080). This was sufficient resolution to allow the user interaction to be timed

accurately, with text on screen being readable by our analysts almost all the time.

11

We measured retrieval time by analyzing the videos frame-by-frame. In a pilot, it was

found that in the camera’s default setting, frames were not of equal duration, making timing

calculations very complex. This problem was resolved by adjusting the camera so that frames

were recorded at a fixed rate of 25 frames per second, making each frame 40 milliseconds

(0.04 second) long.

Retrieval Time: Retrieval time was measured from the first mouse movement made

by a participant in the navigation, until the moment when they either clicked on the target file

(in successful retrievals) or announced that they could not find it (in failure retrievals).

Step Duration: We use the term ‘step’ for each folder opened in the navigation

process. In our study, we measured 5,035 steps. Step duration was measured from the time a

folder was opened until the time the user either (a) clicked on the next folder, (b) reverted to a

parent folder (if the relevant item was not found), (c) clicked on the target file, or (d) said, “I

give up.” We excluded the time taken from clicking on a folder to that folder’s opening, as

pilots showed that this time was inconsistent across different computers depending on their

configuration and performance. Because of this correction, the total time for aggregated steps

is slightly shorter than the overall retrieval time.

Research Limitations

As users, we are very oriented to the semantics of our files. We organize files and

give files and folders names based on their intrinsic meaning. Semantics undoubtedly affects

navigation success and retrieval time. However, our research focuses on structural rather than

semantic elements and their effect on retrieval. Each person’s semantic organization is highly

individual (Boardman & Sasse, 2004), making it hard to compare the effects of semantics

across individuals. Evaluating these effects was beyond the scope of this research and should

be addressed in future work.

12

Results

Folder Structure

The video recordings provide information about the users’ folder organization

strategies. We were able to collect information about the organization of the folders that

participants accessed as they navigated to the target file. In this section, we describe

properties of the hierarchical structure, such as folder depth, size and breadth (number of

subfolders).

1.1 At what depth in the folder hierarchy are active files stored?

Folder depth is the number of steps in the folder path that the participants traversed to

get directly to the folder containing the target file. The folder depth of the desktop is 0, and

the folder depth of the root folder (e.g., My Documents) is 1. Figure 1 presents a frequency

distribution of the depth of the target file for 1,054 successful retrievals. We obviously could

not determine the depth when users failed to find the target. We excluded 8 additional

retrievals because the recordings were not clear enough. We treated shortcuts in different

ways depending on whether we were analyzing folder structure or retrieval. In the current

section we are interested in folder structure, when participants used a shortcut for access, we

identified the depth of the file rather than of the shortcut, as we were interested in where the

location and context of where the file was logically stored. Section 3.1 describes how we

treat shortcuts in retrieval context. Full numerical results are in Table 3 second column in the

appendix.

The mean folder depth of the target was 2.86 (SD = 1.85). The median folder depth

was 3. Furthermore, the majority of retrieved files (82%) were stored at depths of 4 or less

(see Figure 1). This is in clear contrast to previous studies which report overall depths of 8.45

for entire archives (Gonçalves & Jorge, 2003). There was also considerable use of the

desktop: in 115 retrievals (11% of all retrievals) participants used a desktop folder shortcut

and in 75 (7%) retrievals they used files placed on the desktop.

13

Figure 1. Frequency distribution of depth of target file.

A possible explanation for the shallow hierarchical position of active files is that

people rely on default locations (such as My Documents and My Pictures). However, only

136 retrievals (12%) were made from such default storage locations (e.g. files retrieved

directly from My Documents folder, as opposed to subfolders inside it). The default location

folders used in these retrievals contained an average of 19.42 files on the average (SD

=37.28). This clearly indicates that these folders are not large enough to serve as the users’

only file repository. Lack of reliance on defaults implies that the majority of participants

made efforts to construct their own organizational hierarchies rather than relying on

placement by the application.

These findings inform us about hierarchical depth of the folder containing the target

file; in the next sections (1.2 – 1.5), we report on folders at each individual step in the retrieval

14

process. Whenever our results include depth we report only Direct Navigation retrievals:

retrievals in which the user went directly to the target file, without making mistakes by

accessing irrelevant folders. In this case, the hierarchical depth of each folder along the path

was consistent with the step number (each step increases the depth, except the last one from

which the file is retrieved). We omitted results regarding folder depth 0, as in the first step,

participants did not navigate using a folder but used either a menu (e.g. Start ->My

Documents) or the desktop instead.

1.2 How big were file folders?

On average, folders that participants used in their navigation contained 22.46

information items (i.e., files and subfolders), (SD = 32.30). The median folder size was much

smaller: 15 information items. This difference between the average and median was due to a

long tail of very big folders, some of which contained a large number of machine-generated

files (e.g., picture folders populated by camera software or music folders managed by music

software).

1.3 How many subfolders and files were in each folder?

On average, participants’ folders contained 10.64 subfolders (SD = 23.54) and 11.82

files (SD = 27.47). When calculating the average percentage of subfolders in relation to all

information items (files and subfolders), we find that about half of the information items in

the folders were subfolders (M = 54%, SD = 36%). This is again striking: instead of

organizing information into a small number of folders containing huge numbers of files, the

large number of subfolders suggests that users spend time and effort to create structure in

their file system, in anticipation of future retrievals.

1.4 Does folder depth affect folder size, number of subfolders and percentage of

subfolders?

The average folder size at different depth levels is represented by the top diamond

line in Figure 2 (for numerical values including standard deviations, see Table 4 in the

15

appendix). As is evident from the graph, there is a negative correlation between folder depth

and folder size, with folders becoming smaller at greater depths (Pearson r(2,248)= -0.13, p <

0.01). A possible explanation for this is that deeper folders are added later than shallower

ones, so participants have less time to populate them with files and subfolders. Alternatively,

participants keep active files on higher levels to promote accessibility.

Figure 2. Folder properties at different depths.

Figure 2 also shows the mean number of files (center triangle line) and subfolders

(bottom square line). Both graphs seem to decay with depth at approximately the same rate.

Although there is a small negative correlation between folder depth and percentage of

subfolders (r(2,642) = -0.06, p < 0.01), each folder depth has an average of about 50% files

and 50% subfolders (except for the more infrequent 7-11-level deep folders) as confirmed in

Table 4.

This constant average percentage of subfolders disconfirms the common intuition that

higher folder levels serve as structural aids; they are populated mostly by folders whereas

deeper folder levels contain mostly files.

16

1.5 What is the distribution of the percentage of subfolders in all folder items? And

what explains this distribution?

A histogram of subfolder distribution of all information items in folders is presented

in Figure 3.

Figure 3. Percentage of subfolders in folders.

Figure 3 clearly shows a bi-modal distribution of subfolder percentages. Moreover,

32% of the folders contain either only files (331 folders – 12 % of all folders measured) or

only subfolders (521 – 20% of the folders). What explains this bi-modal distribution? Why do

some of the folders contain exclusively or mainly files, while other folders contain

exclusively or only subfolders? The answer to this question is not in the folder structure: we

found (in the previous section) that folder depth has little effect on subfolder percentage. The

explanation relates to the difference between Target Folders (folders containing the target

files) and those which are navigated through on the way to the target. Figure 4 divides Figure

17

3 into two histograms: Target Folders and Navigation Folders (folders that precede the target

in the navigation path).

Figure 4. The subfolders histogram divided into target and navigation folders.

Figure 4 shows that Target Folders contained mostly files and are ‘responsible’ for the

“all files” peak in the bi-modal distribution of Figure 3, while Navigation Folders contained

mostly subfolders and are ‘responsible’ for the “all subfolders” peak in the bi-modal

distribution. An independent sample t test shows that the subfolder percentage of Target

Folders (M = 13%, SD = 22%) was significantly smaller than the subfolder percentage of the

Navigation Folders (M = 65%, SD = 31%), t(2,641)=37.52, p<0.01. The effect (a difference

of 52% between averages) is l