RFC1431

Network Working Group P. Barker Request for Comments: 1431 University College London

                                                       February 1993

                          DUA Metrics

Status of this Memo

This memo provides information for the Internet community. It does not specify an Internet standard. Distribution of this memo is unlimited.

Abstract

This RFC is being distributed to members of the Internet community in order to solicit their reactions to the proposals contained in it. While the issues discussed may not be directly relevant to the research problems of the Internet, they may be interesting to a number of researchers and implementers.

This document defines a set of criteria by which a DUA implementation, or more precisely a Directory user interface, may be judged. Particular issues covered include terminal requirements; style of interface; target user; default object classes and attribute types; use of DAP; error handling. The focus of the note is on "white pages" DUAs: this is a reflection of the current information base. Nevertheless much of the document will be applicable to DUAs developed for other types of Directory usage.

Please send comments to the author or to the discussion group <osi- [email protected]>.

Overview

The purpose of this document is to define some metrics by which DUA products can be measured. It should be first be noted that the use of the term "DUA" is rather misleading. There is an assumption here that the DUA is implemented correctly and is able to "talk" valid X.500 protocol: this is a sine qua non. Instead, this document seeks to draw out the characteristics of Directory user interfaces. However, the term DUA is persisted with as it is used by most people when referring to Directory user interfaces. The format of these DUA metrics is essentially a questionnaire which extracts a detailed description of a user interface. DUAs come in very different forms. Many make use of windowing environments, offering a "high-tech" view of the Directory, while others are designed to work in a terminal environment. Some interfaces offer extensive control over the Directory, and thus may be well-suited to Directory managers, while others are aimed more at the novice user. Some interfaces are configurable to allow searches for any attribute in any part of the DIT, while others lack this generality but are focussed on handling the most typical queries well. In many aspects, it is almost impossible to say that one DUA is better than other from looking at the responses to question in this document. A flexible management tool will be better for management than a DUA aimed at servicing simple look-ups, and vice-versa. Furthermore, in other areas, there are several radically different approaches to a problem, but it is not as yet clear whether one approach is better than another. One example of this is the extent to which a DUA provides an abstraction of the underlying DIT hierarchy, either emphasising the world as a tree or trying to conceal this from the user.

However, other aspects, such as whether the DUA can actually find the entries required, and if so, how quickly, can be directly measured in some way. Throughout this document, some of the questions posed are annotated with a square-bracketed points score and an explanation as to how the points should be allocated. For example, a question might be appended with "[2 if yes]", indicating score 2 points for an affirmative answer to that question. These points scores should be

collated in Table 1 at the end of the document, and this table constitutes a measure of the DUA. The metrics are on a section by section basis, which should help the reader who is seeking, for example, a DUA with good management capabilities which runs on a wide variety of platforms, to focus on the critical aspects of a DUA for the particular requirement.

General Information

This section contains general information about the implementation under discussion.

5. Describe the hardware and software platforms on which the DUA will

   run.  Some DUAs are split into a user interface part, and a DUA
   server part, communicating by means of a protocol.  If the DUA is
   of the type where the DUA protocol machinery and user agent are
   implemented in a single process, complete only the user interface
   section, and indicate "n/a" for the DUA server and
   communications protocol questions.

  (a)  User interface part [1 per platform, up to a maximum of 4]

       i.  Hardware (If appropriate, can summarise as, for example,

      ii.  O/S (state version if critical)

           A.  UNIX (be sure to indicate which flavour - e.g., SYSV,

       i.  Hardware (If appropriate, can summarise as, for example,

      ii.  O/S (state version if critical)

           A.  UNIX (be sure to indicate which flavour - e.g., SYSV,

     iii.  How does the user interface communicate with the DUA
           server?

           A.  Directory Assistance Service, as described in RFC1202

           C.  LDAP protocol, as described in Internet Draft OSI-DS
               26

  (c)  Name any other software required to run the DUA which is not
       supplied with the operating system or with the DUA software
       itself.  Examples might include X.500 DAP libraries, or

6. Is the software free? If the DUA needs other packages, are these

Conformance to OSI Standards

Directory protocols

7. Please list all conformance testing work applied to the DUA

   implementation (here the term DUA is used correctly in the sense
   of the DUA protocol machinery) [2 if any conformance work has been

Protocol stacks

For the next two questions, [2 per stack supported for up to 4 stacks]

8. Which of the following transport and network layer protocols does

   the DUA support:

9. Does the DUA support other transport and "network" layer

   protocols?

10. Does the DUA also run over any lightweight stack? If so, describe

Schema

11. Does the DUA support the full schema in X.520 and X.521 (y/n)?

   (Omissions should be described in response to a later question) [2

DIT structure

12. Does the DUA only follow object class hierarchies which conform to

Conformance to Research Community Standards

The COSINE and Internet Directory Pilots have agreed a set of extensions to the standard, which make for a more cohesive pilot. This section is about conformance to these extensions.

13. Does the DUA fully support RFC1274, "The COSINE and Internet

   X.500 Schema" (y/n)?  (Omissions should be described in response

14. Can the DUA handle referrals whose network addresses conform to

   RFC1277, "Encoding Network Addresses to support operation over

15. Does the DUA handle the Distinguished Name string syntax described

   in OSI-DS 23, "A String Representation of Distinguished Names"

16. Does the DUA use the user-friendly naming query resolution

   described in OSI-DS 24, "Using the OSI Directory to achieve User

17. Does the DUA make use of the Quality of Service schema extensions

   described in OSI-DS 15, "Handling QOS (Quality of service) in the

The General Style of the DUA

18. Is this a "white pages" interface, designed to give access to

   information about people within organisations?  If not, state the

19. If this is a white pages DUA, who is it principally designed to

   serve?  Indicate more than one of the following categories if
   appropriate (but please do not fill in so may categories as to
   hide due emphasis):  [mark allocated should be the highest for any
   single classification]

  (a)  The ordinary user, who has no understanding of X.500, the
       hierarchical DIT, the state of advancement of the pilot, etc.

  (b)  A secretary who wants to do telephone or room number look-ups

  (c)  A computer-literate user, who habitually uses a wide-range of

20. Which best describes the use the DUA makes of the user's terminal?

21. Does the DUA tend to emphasise or de-emphasise the DIT hierarchy?

22. Describe the interface in your own terms (up to about 50 words)

Schema

Object Classes and Attribute Types

Some DUAs are tightly focussed on answering particular queries: for example, white pages look-ups for information about people. Others offer more general capabilities. Please answer this question accordingly.

23. If the DUA has a tight focus, state:

  (c)  Other attribute types which may be configured.  This might be
       answered as, for example, "all barring photo and audio", or

24. If the DUA has more general capabilities, state:

  (a)  State any object classes in X.521 which cannot be searched for

  (b)  State any object classes in RFC1274 which cannot be searched

  (d)  State any attributes in RFC1274 which cannot be displayed....

DIT structure

25. DUAs often have a default object class hierarchy (e.g., it might

   assume countries at the root of the DIT, organisations immediately
   under countries, and people somewhere under organisations.
   Describe the type of hierarchy which the DUA most closely accords
   to:

  (c)  Default hierarchy offered, but many hierarchies are supported

27. State any hierarchies, which are valid according to X.521's

   suggested DIT structure, but which cannot be queried (exclude
   hierarchies which cannot be queried because the DUA does not query

Entering queries

The term "querying" is used here as a generic term for finding an entry, whether it be as a simple look-up, or the prelude to a modification operation.

28. Which best describes the query entry style?

  (c)  Querying is by "navigating" around the DIT, the user

Strategy for locating entries

A number of strategies are employed by DUAs to find the entry the user is looking for. These have implications for user-friendliness and performance. For example, an interface which makes extensive use of search operations may be excellent at finding entries, but at the cost of being intolerably slow.

29. Which of the following strategies most closely accords with the

   behaviour of the DUA?

  (b)  The DUA offers users a list of entries, and invites the user

  (c)  The DUA only tries read operations (i.e., the DN must be

  (d)  The DUA tries read operations first, then searches for

  (e)  The DUA tries read operations first, then offers a list of

  (f)  User explicitly controls the X.500 operation which is invoked

30. Does the DUA allow a user to "list" (either by the list

   operation or by a single level search operation) all the child

   If so, does it do so:

32. Will the DUA optionally follow links to other entries by using

   attributes such as seeAlso and roleOccupant with a DN syntax?  ..

Displaying results

33. Are the strings used to describe attribute types freely and

34. Name any attribute types where the attribute values may be

   presented in local formats?  (For example, it may be possible to
   configure the display of telephone numbers so that local numbers
   are shown as extensions, rather than with the full international

35. Does the DUA allow for the display of more than one result at a

   time (showing attribute values other than the name of the entry)?
   If so, how many entries may be displayed in response to a single

36. Does the DUA support the notion of a quick synopsis, where a small

   core of attributes is retrieved initially, and a larger set is

37. What does the DUA do with attribute types it doesn't support in

38. What does the DUA do with attributes which have are not in its

   sub-schema, and which have a non-standard syntax?

10. Association Handling

This section is concerned with how a DUA handles its association with the Directory.

39. How/where is the access point to the Directory configured? If

   more than one method, indicate which ways are possible.  [1 per
   method, up to maximum of 3]

40. Does the DUA allow for automatic connection to a back-up DSA if

41. Can the DUA keep connections open to more than one DSA at a time?

42. Does the DUA keep an idle connection open to the DSA(s). If not,

44. Does the DUA make use of asynchronous operations?

       If so, is this true for:

45. Does the DUA use size and time limits by default? [2 if no

11. Suitability for management

This section is intended to establish the range of operations supported by the DUA and, in particular, whether it is suitable for management tasks.

46. Is it possible to invoke all the operations in the Directory

   Abstract Service?  If not, say which operations it does use [2 if

47. Is the user given full control over the service controls? If not,

   say which may be controlled, or none at all [2 if full, 1 if some

48. Is it possible to manage system attributes with the DUA? If so,

   indicate which DSA implementations for which this DUA provides

49. Access control notwithstanding, does the DUA allow the following?

  (a)  Attribute management [2 for all below, 1 for some]

  (b)  Entry management [2 for all below, 1 for some]

12. Query Resolution

This section discusses the process of query resolution. While two DUAs may both be able to resolve a query using the same information, one may do so much more quickly than the other. Some DUAs may be more "economic" in their use of DAP operations to achieve the same results. Some DUAs may find the correct results even when the users' input corresponds rather weakly to Directory names. Three aspects of query resolution are measured:

o  Does the DUA actually find the required entry?

o  If the required entry is found, how many entries were returned as
   well?

o  How "expensive" was the query in terms of underlying X.500
   operations, whether the query was resolved successfully or not?

The following set of queries might all conceivably be resolved such that the author's Directory entry be found. The queries are split into 2 groups: the first group SHOULD pose no difficulties for a reasonable DUA; the second group are more problematic. In each case, award [2] marks if the query found the author's entry successfully. The expensiveness of each query should be measured using the following formula, which introduces the notion of SearchStones! The SearchStone rating is calculated by adding together the total operations used in attempting to resolve a query, weighted thus:

o  Bind [5]

o  Read operation [1]

o  List operation [2]

o  Search single level for countries, organisations or
   localities [3]

o  Search single level for organisational units, people or roles [3]

o  Search subtree [5]

  Note: The single level searches have been separated into two
  categories in acknowledgement that certain types of search are
  much more likely to span multiple DSAs than others.  The
  weightings are the same for the moment because of the
  pervasiveness of the Quipu implementation, which replicates all
  sibling entries in a single DSA, whatever the level in the DIT.

The notion of SearchStones merits some further explanation and the statement of some caveats.

The idea is to give some broad brush view of the work being undertaken by a DUA to retrieve an entry. There will be some correspondence between a low SearchStone rating and a DUA responding quickly, and vice-versa, although this correlation is not consistent, for reasons given below. It would be desirable to be able to have some timing information for the resolution of queries, but such results would only be meaningful if the tests were for target entries widely distributed throughout the DIT. Maybe this is something for the future? In the meantime it is worth noting some of the factors which militate against simple minded interpretation of the SearchStones.

o  The DIT is not uniform, with the depth varying considerably

o  While the DIT is currently mastered mostly by DSAs of a single
   implementation, this will be decreasingly the case, and other DSAs
   may have very different performance profiles.

o  Different directory domains are already adopting different
   strategies on information replication with profound performance
   implications.

o  No weighting is given to different search filters, or to boolean
   combinations of filters.

While acknowledging the difficulty of the exercise, there are counter arguments:

o  Some DUAs are better than others at finding the required results

o  Some DUAs will get the required results more quickly than most

o  DUA designers have to build DUAs in the knowledge that the DIT is
   heterogeneous with respect to DSA implementation and DIT structure

One possible way forward would be to refine the test queries such that they better represented the diversity of the DIT. However, as a first step, the tests are restricted to queries which could reasonably be constructed as searches for the author's entry. The author's entry is held in part of the DIT which is representative of much of the current DIT. It is suggested that in order to normalise the tests as much as possible, that testing be performed by connecting to the target DSA directly. The DSA's name is "cn=Vicuna, c=GB", and the addresses of the DSA may be found in the presentation address attribute for that entry. Note that the SearchStone rating should be shown even for queries which cannot be resolved.

First, the straightforward queries:

50. NAME=Paul Barker, OU=Computer Science, O=University College

   London, C=GB

51. NAME=Paul Barker, OU=Computer Science, O=UCL, C=GB

52. NAME=Barker, OU=Computer, O=UCL, C=GB

53. NAME=Barker, O=UCL, C=GB

54. NAME=p barker, O=university college, C=GB

55. NAME=paul b, OU=cs, O=university college, C=GB

More difficult queries:

56. NAME=p b, O=university college, C=uk

57. NAME=Paul Barker, OU=Computer Networking, O=london college, C=GB

58. NAME=Paul Baker (sic), OU=cs, O=ucl, C=Britain

59. NAME=p baker (sic), O=UCL, C=England

60. NAME=Paul Barker, OU=Directories, O=london, C=United Kingdom

Anonymous

Search

RFC1431

Namespaces

More

Page actions

Contents

Overview

General Information

Conformance to OSI Standards

Directory protocols

Protocol stacks

Schema

DIT structure

Conformance to Research Community Standards

The General Style of the DUA

Schema

Object Classes and Attribute Types

DIT structure

Entering queries

Strategy for locating entries

Displaying results

Navigation

Navigation

Wiki tools

Wiki tools

Anonymous

Search

RFC1431

Contents

Overview

General Information

Conformance to OSI Standards

Directory protocols

Protocol stacks

Schema

DIT structure

Conformance to Research Community Standards

The General Style of the DUA

Schema

Object Classes and Attribute Types

DIT structure

Entering queries

Strategy for locating entries

Displaying results

Navigation

Wiki tools

Page tools