 | Level: Introductory Eric Hann (ehann@us.ibm.com), Team Lead, IBM
Benjamin Boltz (boltz@us.ibm.com), Developer, IBM
23 May 2002 This article is an introduction to OnDemand and gives scenarios for
how to make OnDemand solutions user friendly and high performing. IBM® Content
Manager OnDemand allows companies to manage large quantities of printed
output, such as reports, invoices, or statements.
Introduction
Your company has made the decision to purchase the IBM Content Manager OnDemand (OnDemand) solution. They've put you in charge. You've gone to OnDemand University, attended the workshops, and you've spent a great deal of time on the OnDemand Web Support Web site.
When your supervisor asks you if you are prepared to design an OnDemand solution that performs well and is user friendly, how will you answer?
This article is intended to help you answer that question positively and confidently. Although we won't discuss every possibility available for correctly designing an OnDemand solution, we will provide you with an idea of what you should think about while creating your solution.
Most people new to OnDemand understand that OnDemand stores documents and allows you to retrieve these documents on a PC or with a Web browser. They may also understand the basic functions of the individual components or have the ability to create an application that will store a document type. However, most people new to OnDemand, and even some who have been using OnDemand for a while, have no idea if the way they use their OnDemand solution is user friendly or even if it is performing well. Their end users have grown accustomed to the way things are and nobody has suggested improvements to the solutuion.
What is OnDemand?
In order to help us learn how to design our OnDemand solution, we first need to understand what OnDemand is.
OnDemand allows an end user to retrieve a specific document from storage in the shortest amount of time with the least amount of user interaction. For the solution to perform well and be user friendly, you must give some thought to how the data will be used.
In extremely simple terms, OnDemand is an electronic version of your local library back when your local library required you to use a manual card catalog system. Here is how your local library and OnDemand compare:
|
Local Library
|
OnDemand
| | Page in a book | Page of a document | | Chapter in book | Document inside the stored object | | Book | Stored object | | Card in drawer | Row in database table | | Card catalog drawer | Database table | | Card catalog cabinet | Folder |
At the library, when you want to research a specific chapter in a book, you go to the card catalog cabinet, open a specific card catalog drawer, find a specific card that tells you where the book is located, go get the book and then turn to the page you want to see.
With OnDemand's PC or Web viewer, you select a folder that is attached to certain database tables. You then enter your search criteria. When you press Enter, you will get a document hit list, which is a copy of specific rows in a database table. When you select one of these rows, you retrieve the document that you are interested in.
Maximizing performance and user friendliness
Going back to our local library, we know what book we want to see but when we get to the card catalog cabinet, there are four card catalog drawers that have exactly the same label on the front. In order to find out where our book is located, we have to search all four of the drawers. This will obviously be much slower than searching a single card drawer.
The same is true for OnDemand. If your search criteria must go across multiple application group tables, your query will take longer to complete. This means that your system works harder to perform the query and your end user must wait longer for results.
Our mission as OnDemand solution designers is to figure out the best possible way to design our application so that we aren't searching across multiple application group tables. Multiple application group tables may come from the same application group or they may come from multiple application groups.
Before we start designing our application, we need to understand the four pieces of an OnDemand application that are going to mean the most to our design:
-
Application - The OnDemand application is for indexing. It gathers from the data the information needed to insert database rows and the resources needed for the data being loaded. Resources may be anything from logos, special fonts, page definitions, form definitions, etc. The application also contains the information for viewing and printing the data.
-
Application Group - The application group is a table builder. It tells us what colums need to be added to a table when the table is built. During loading, it makes sure that our application provided us with the correct field (column) information to successfully insert a row.
-
Folder - The OnDemand folder provides the end user with a "view" of the database. It will provide a user with a "fill-in-the-blank" approach to SQL and will determine what database tables will be queried.
-
Storage - Cache storage or Optical/Tape Storage such as TivoliTM Storage Management (TSM) for UNIX® or Windows®. This is where our data resides. When we retrieve a document, we will get it from either cache or from our long-term storage.
Although there are many more variables involved in the overall OnDemand solution, these are the four that a successful OnDemand solution design are most concerned with.
If OnDemand were simply used for a single report type, designing for success would be easy. For most customers, a normal OnDemand solution is used daily by thousands of people who are retrieving hundreds of data types. Therefore, designing for performance and friendliness takes a three-step approach.
Step 1: Determine how data will be retrieved
OnDemand finds documents based on information we collect in our database. Therefore, it is important to understand how people will search. When a user enters a query, the solution should return meaningful information to the user as quickly as possible. This means we will only search across a very limited number of database tables and we will only return a limited number of query matches. At our very best, there will only be a single query match.
When a user opens a folder, the database query has already started. Selecting a folder limits us to the number of application group tables that we will be searching against. We'll discuss this part of the query in Step 3.
A folder provides an interface that allows the end user to query database tables. The fields of a folder correspond to application group fields, which correspond to database index and filter fields. There should be a segment field available that narrows what tables we search. A folder may also have a server-based text search field. Using a text search field is the slowest way to search for data as we first match the other query fields and then search the matching documents for words or phrases that match our text search. You should avoid providing text search fields for OnDemand users. Your end user will be happier with the efficiency of query fields.
Once a query is requested, here is a high-level view of what happens:
- If part of the query is a segment field, use this to select only the tables that meet this criteria.
- If index fields are part of the query, search the indexes of the tables selected by the segment search to choose matching table rows.
- If filter fields are chosen, look at the selected rows to narrow our rows to only those rows with matching filter fields.
- Return the matching query rows, in the form of a hit list, to the end user.
Although it would be tempting to make all fields index fields, you need to understand the tradeoff. When you create an index, you have taken information already stored in your database table and will use database space to add that information again in a separate table, as well as using space for the overhead involved in creating an index. If all of your table fields are indexes, your database would be unnecessarily huge without any additional benefit.
Having too many fields, index or filter, is also not a good idea. Each database column that you tell the application group to create for a table is additional space needed in your database. You need to only create fields (database columns) for things you need to search against. At the very minimum, you should provide at least one index field. This field should be the most unique value in the data, such as Customer Number, Social Security Number, Phone Number, etc. It should also be a field that an end user will normally search on.
This is why it is imperative to understand how your customers plan to search for documents. If your customers will search on a single field, you obviously will only need that field. If 80% of your users will query using three fields and another 20% need other fields as well as the three fields used by others, it may be best to have only three indexes and leave other fields as simply filter fields.
An application group table is limited by the number of rows that it is able to have. You set this within the application group properties. By default, an application group has 10 million rows. Once you've stored the 10 million rows, OnDemand closes the first table and opens a second table. By doing this, you've just very slightly degraded performance if a query must cross both tables. The performance change from crossing one table to two tables is extremely small and will not be noticed. However, searching across 100 tables is a different matter. At the same time, if you set your row limit too high, performance will also degrade because it takes too long to look through the table to match your query.
This is where segment fields come in. You should always specify a segment value to improve performance, usually report date or statement date. If one does not exist in the report, you can always use a load date by specifying it in the application. This value should be chronological to provide the best segmentation. A segment field will allow us to limit the number of tables we choose to search. If our segment is "load date" and we fill a table four times per year, we are able to limit our search to a single table simply by adding the month and year to our search. At most, a month might carry over into a second table. However, we have successfully narrowed our search simply by narrowing the tables we search across.
Step 2: Make data retrieval efficient
Now that we understand how users will retrieve data, the next step is to determine how often they will retrieve the data. Data that is retrieved often should remain in cache storage until it is no longer needed by 90% of your users. Data that is retrieved infrequently can be stored on long-term storage. Cache storage is the fastest means to deliver data to your end users. Once the demand for the data is no longer high, the data may be removed from cache and the remaining users are still able to retrieve the data from long-term storage. Having a platter/tape placed into a drive and then having the drive spin up and deliver the data to your end user takes considerably longer than retrieving from cache. It gets worse if too many people are retrieving from long-term storage and a drive is not currently available to honor your retrieval request.
Your company might have certain limitations regarding the amount of cache storage available for you. However, the general rule of thumb is to keep data in cache for as long as possible. In general, you do not want your end users to wait for data to be delivered from long-term storage.
Step 3: Designing folders for performance and usability
A folder may have a single application group or it may have several application groups assigned to it. The best situation is a single application group in a folder. However, this is not always the useful solution for the data we must work with.
There are few basic OnDemand principles that will help you:
- Data objects should use the same application when:
- Data objects should use the same application group but a different application when:
- Before you design an application group, you should consider if there is any chance that you may have more than one application that will load to it. You have the ability to expand the Application ID field to include more applications. However, if you do not add an Application ID field, you cannot go back and add this field later.
- Application groups may reside in the same folder as long as there are common search fields within each of the application groups. However, if the end user does not have equal access to all application groups, you are concerned with query retrieval time or you expect that each application group will have a large quantity of tables that a segment search will not narrow for you, you should consider having individual folders for each application group. You can limit the number of Application Groups searched in a folder by using user and group permissions.
- A folder should be your first query as well as your first query restriction. An end user should only see folders containing application groups with information he or she will use. A folder should only contain application groups, or portions of application groups that are very similar and would be considered illogical to separate. And the user should only see folders that pertain to their job. For example, inventory documents should not be available in a payroll folder. You can limit the number of folders a user sees in the folder list by using user and group permissions.
You need to strive to accomplish the following:
- Application = One to Many Data Objects
- Application Group = One to Many Applications
- Folder = One to One or Few Application Groups
So, now that we have the basics, let's go through an uncomplicated scenario and design an OnDemand solution.
Your company has the following six reports they would like to store in OnDemand:
-
Balance Sheet - AFP Data
-
Sales Detail Report - Line Data
-
Inventory Detail Report - AFP Data
-
Transaction Detail Report - Line Data
-
Income Statement - PDF
-
Payroll Ledger - AFP Data
Example of a Bad Solution:
Using just this information, a very simplistic but poor design would look as follows:
- We create an application for each report and an application group for each application.
- We make all fields an index field.
- We make a single folder accessed by all users that contains all six application groups. Any time we search this single folder using a common index field, we will be searching across all six application groups. If we don't have a segment field, we will search across all of the tables in all six of the application groups. To make matters worse, because we made every field an index field, our database is much larger than it needs to be.
Example of a Good Solution:
With a little more research about how the data is used you could determine the following:
-
Balance Sheet - AFP Data Major Query on account number, date
Minor query on account description
(Used by executives and accountants)
-
Sales Detail Report - Line Data Major Query on account number, date
Minor Query on account description
(Used by executives, accountants, sales)
-
Inventory Detail Report - AFP Data Major Query on product number, date
Minor Query on product description, transaction type
(Used by executives, accountants, inventory control, sales)
-
Transaction Detail Report - Line Data Major query on account number, date
Minor query on account description, transaction type
(Used by accountants)
-
Income Statement - PDF Major query on account number, date
Minor query on account description
(Used by executives, accountants)
-
Payroll Ledger - AFP Data Major query on employee number, last name, date, Social Security Number
Minor query on First name, Department
(Used by accountants, humans resources)
And a better OnDemand design using the information learned about these reports could be:
The Payroll Ledger is the only report used by the Human Resources Department. Therefore, we are going to create the following:
- Folder "Payroll Ledger"
- Application group "payledge"
- Application "payledge"
Next, we have the Inventory Detail Report. This is the only report viewed by the inventory control department.
- Folder "Inventory Detail Report"
- Application Group "invreport"
- Application "invreport"
The next report we are going to look at, Transaction Detail Report, is similar in need to the others. However, it has a requirement of "transaction type" and this field isn't needed by the other accounting reports.
- Folder "Transaction Detail Report"
- Application Group "transdtl"
- Application "transdtl"
Finally, we have three reports, Balance Sheet, Sales Detail Report, Income Statement, with all different data types. They all have the exact same query needs. The only thing we have to watch for is that our Sales Detail Report is the only one that is used by the Sales Department. However, these reports are good candidates for a single application group.
- Folder:
- Application Group: "execreport"
- Application "balsheet" "salesrpt" "incomestmnt"
Our solution will require six applications, four application groups and five folders with access controlled by five groups. Each query will search across a minimum of a single table. Most of our searches will be index scans (index fields), as opposed to table scans (filter fields). Because we have the date field listed as our segment date, if we load our data in date order and require that the user enters a date, we have an excellent opportunity to restrict our queries to the fewest tables possible.
Our example isn't the only possibility for an excellent OnDemand solution design. There are several things we are able to do, such as query restrictions, user group restrictions, application group permissions, etc. The OnDemand Support Web site is an excellent resource to assist you with other design possibilities.
As the designer of your OnDemand Solution, you will likely be presented with a wide variety of reports to archive. They won't all be line data or AFP data and they'll all have different query needs. The best designs that you will achieve will require understanding who uses the reports and how the reports will be queried. Detailed planning, before you begin to build your solution, will help you achieve a design that remains efficient for many years to come.
AFP, IBM, DB2, and Tivoli are trademarks or registered trademarks of IBM Corporation in the United States, other countries, or both.
Resources Learn
Get products and technologies
About the authors | | | Joining Content Manager Ondemand Support in 1998, Eric Hann is the Content Manager, OnDemand Support Team Lead, based in Boulder, Colorado. Eric played a key role in the initial design and implementation of IBM's Web-Based support initiative for Content Manager products. Eric can be reached at ehann@us.ibm.com. |
| | | Benjamin Boltz is a Content Manager, OnDemand developer, based in Boulder, Colorado. Ben started at IBM in 1996 as an installer for a service offering that eventually became the product Content Manager OnDemand. |
Rate this page
| |
