Migrating LDAP servers with WebSphere Portal V4.2 and IBM Directory Integrator V5.1

Document options Document options requiring JavaScript are not displayed Sample code Learn and share! Exchange know-how with your peers -- try our new Pass It Along beta app Rate this page Help us improve this content

Level: Intermediate

Nick Lawrence, Advisory IT Specialist , IBM Software Services for WebSphere EMEA
Christian Chbteauvieux, Advisory IT Specialist , IBM Software Services for WebSphere EMEA

04 Jun 2003

This paper describes how a large Microsoft Active Directory-based portal could be migrated to an IBM WebSphere Portal V4.2-based portal using IBM Directory Server V5.1 as the LDAP server. As part of a proof-of-concept, services consultants used IBM Directory Integrator V5.1 to migrate existing users, while modfiying the LDAP heirarchy, to meet the customer's requirements.

© Copyright International Business Machines Corporation 2003. All rights reserved.

Introduction

This document is derived from a customer situation with a large industrial company in Germany. This company has a large portal, based on Microsoft® Active Directory®, covering about 200,000 users and 10,000 groups. Although the company wanted to maintain this system, it also wanted to develop a true enterprise LDAP for use with IBM ® WebSphere® Portal Version 4.2 (hereafter called WebSphere Portal) and other future products. The company also wanted to see that migrating to IBM Directory Server using IBM Directory Integrator would be practical.

This paper is not intended to be a comprehensive guide to every circumstance that you might encounter in performing similar work; rather, it describes what we did in a single customer situation. However, we believe it can provide a framework that you will find useful in applying to your own situation.

This document consists of two parts. The first part describes how we used Directory Integrator to pull user and group information from the Active Directory servers and to load it into the Directory Sever servers. The second part describes how we modified an existing WebSphere Portal installation to use a different LDAP server and schema.

We should note that this case was favourable to our efforts in some ways. The customer required that all user ID's be unique, even though the users were spread across a deep group structure. This was helpful because it prevented the Two Bobs problem in WebSphere applications, where the user's login as [uid=]bob matches two different distinguished names (DNs) , each featuring the same base DN, with no way to distinguish between them. Also, although the customer had already done a lot of work on their WebSphere Portal system, it was not yet in production, which enabled us to work quickly and complete the majority of the work in a day.

We used the following levels of software:

• IBM Directory Integrator 5.1 (formerly known as MetaMerge)
• IBM Directory Server 5.1 (formerly known as Secureway Directory)
• IBM WebSphere Portal Version 4.2

Back to top

Migrating user and group information

This part of the document describes how to migrate user registry information from one LDAP server to another; specifically, it describes migrating from Active Directory to Directory Server. It first describes how to set up Directory Server, and then at describes migrating users, groups and passwords.

Setting up IBM Directory Server

First, we installed Directory Server using the default installation settings. We configured it to support a dc=customer, dc=com suffix to look similar to the suffix already defined in the customer's Active Directory implementation. We flattened its structure into two branches:

• cn=users, dc=customer, dc=com, to contain user entries
• cn=groups, dc=customer, dc=com, to contain group entries

We used the cn attribute as a naming attribute for these two branches to mimic the existing Active Directory model and to match WebSphere Portal's default model. We would recommend using another attribute (for example, ou) if this were not the case.

We configured Directory Server to automatically encrypt user passwords in the SHA1 algorithm. The customer can changes this setting at a later date, if desired.

We did no specific tuning to improve Directory Serverperformance because the out-of-box performance was good enough for this proof of concept. In a production environment, however, we would normally tune the Directory Server.

Migrating users

For the proof of concept, we decided to move user entries from the German Active Directory domain only (dc=de, dc=customer, dc=com in Active Directory), limiting the number of records to approximately 50,000. For the migration, we used a Directory Integrator assemblyLine , which only moved a set of standard inetOrgPerson attributes (cn, givenName, mail, sn and uid). Using this assemblyLine simplifed the task and allowed us to avoid customizing the Directory Sever schema to fit the custom Active Directory attributes, which were not needed by the portal. At a later date, the Directory Sever schema can be customised to integrate more attributes into the assemblyLine.

We configured the assemblyLine to only move records with LDAP uid fields matching the following regular expression:

[a-zA-Z]{3}\d[a-zA-Z]{2,4}

For example: cia8fe, bok8fel...

Making this choice allowed us to avoid moving non-customer users (for example, special Windows users).

About assemblyLine

The assemblyLine is available in the Directory Integrator 5.1 XML configuration file, rs.xml, and is called populateIDS. You can start it manually, but we set up a simple Timer eventHandler (called CronJob in the Directory Integrator configuration file) in Directory Integrator so that it is automatically started every Sunday at 10:00 a.m., as well.

To see the exact parameters used in the configuration of Directory Integrator, download the attached configuration file, rs.xml, which is a copy of what we used with this customer.

The populateIDS assemblyLine

There is nothing particularly hard in this assemblyLine. There are only two connectors, an iterator and an update connector, reading users from Active Directory and updating Directory Server.

The readAD connector filters entries from Active Directory using the following LDAP search filter:

(&(objectclass=user)(!(objectclass=computer))(sn=*))

Migrating groups from Active Directory

We divided the task of migrating groups into two parts. The customer's Active Directory structure contained a mix of static groups, each of which contains a reference to its users (through the LDAP member attribute of the group object), and non-static groups, in which user entries define the group to which they belong (through the Active Directory's member attribute of the user object). We imported these two types of groups using two Directory Integrator assemblyLines in the rs.xml configuration file, populateIDSgroups and populateIDSgroupsFromADusers, respectively. The next two sections describe these two assumplyLines in more detail.

Active Directory can define groups with no members in it, but this is invalid in the standard LDAP groupOfUniqueNames objectclass used in Directory Server. Therefore, we set a dummy member, cn=root, to allow propagation of such groups. This allowed the propagation of several thousand groups in Directory Server.

We migrated all (about 10,000) Active Directory customer groups, except for nested groups (or groups of groups). Nested groups were not required by the proof of concept, but could be migrated very easily in the future.

You can start assemblyLines manually, but we set up a simple Timer eventHandler, called CronJob in the Directory Integrator configuration file, so that they are automatically started every Sunday at 10:00 AM.

The populateIDSgroups assemblyLine

The populateIDSgroups assemblyLine includes two connectors, an iterator and an update connector, for reading groups from Active Directory and updating Directory Server. In Directory Server, groups are created as groupOfUniqueNames entries, storing their members into the uniqueMember attribute. In Active Directory, this list of member attribute (called member) contains references to Active Directory user entries, whose values are the users' distinguished names. Therefore, a translation is required for Directory Sever to reference its own entries, and not Active Directory entries. In other words, the values of the member attributes in Active Directory must be translated to fit the user's DN in Directory Server. We used the uniqueMember attribute mapping of the update connector output map to accomplish this translation.

This assemblyLine is very simple, and would need to be adapted to import nested groups. Currently every DN is translated by string manipulation. That is, if a group had a member

cn=memberID, cn=groups, dc=de, dc=customer, dc=com

it would be translated by the attribute map as

uid=memberID, cn=users, dc=customer, dc=com

in Directory Server, even if the memberID is a group and not a user.

The populateIDSgroupsFromADusers assemblyLine

Initially, the populateIDSgroups assemblyLine did not move the expected number of groups in Directory Server. After investigation, we discovered that user entries in Active Directory can also define which group they belong to, using the memberOf attribute of the user object, and that groups will not have their member field referencing the user's entry; that is, there is no cross-reference). Therefore, the later assemblyLine did not move all group membership information and we had to write the populateIDSgroupsFromADusers assemblyLine.

This assemblyLine is slightly more complicated, but still fairly simple. Its first connector reads user entries from Active Directory and applies the same filtering as explained in the populateIDS assemblyLine section.

The Active Directory user's corresponding entry in Directory Sever is searched by the lookupDN connector, and once it is found, the entry's DN in Directory Sever is retrieved.

The recurse script updates every Directory Server-matching group found in the user's active Directory memberOf attribute. First, the script strips the group identifier from the Active Directory group (cn is the group naming attribute in Active Directory), and then invokes a passive connector, updateGroups, to update the group in Directory Sever that has the same group identifier. The group update ensures that the user's DN, which is retrieved by the lookupDN connector, is added to the set of existing group members, instead of replacing the uniqueMember group attribute value. The script uses the updateGroups connector uniqueMember attribute map to specify this update:

if (current.getAttribute("uniqueMember") != null) ret.value = 
   current.getAttribute("uniqueMember").addValue(work.getString("userDn"));

Migrating passwords from Active Directory

At this stage, the alternate portal system, based on the IBM Directory Server, was almost ready for use. However, a very important piece of information was missing: the user passwords, from Active Directory, which are necessary to allow users to log in using their current Windows/Active Directory password.

Because Active Directory uses its own encryption techniques to hash passwords, it is impossible to get these passwords out in a format that would be understandable by any other system. Active Directory does not use SHA1 which could have allowed us to simply copy the password hashes to Directory Server.

We evaluated several options to propagate passwords from Active Directory to Directory Sever. First, we suggested using the Directory Integrator password synchronizer, which intercepts Windows NT user ID and password change requests and propagates the changes to other password repositories (using an Directory Integrator assemblyLine) before the NT system changes the password. We rejected this option because there was no staging environment available at the customer that would have allowed us to test the solution with theActive Directory system. Also, we did not want to make changes to the production Active Directory server.

The second option was to customize WebSphere Portal and WebSphere Application Server to authenticate users against Active Directory, but authorize users against Directory Server (for example, by computing the groups the user belong to) . This option would have required us to write a custom login module for Application Server and to extend WebSphere Member Services (WMS) for WebSphere Portal. We abandoned this idea because we thought it would be difficult to implement in a short period of time, and because customizing WMS is not considered good practice, because the interface could change in Version 5 of WebSphere Portal.

Ultimately, we came up with a much simpler and easier solution. Because the customer does not wish to use password interceptors on its Windows domains, and it still wants Active Directory to be the primary repository for passwords, we changed the portal login page so that the user ID and password typed into WebSphere Portal's login form are sent to Directory Integrator rather than to the li page of the portal. We simply changed the URL in WebSphere Portal's login.jsp file.

When Directory Integrator gets the credentials from the user's browser, it first validates the user password in Active Directory. Directory Integrator must first look for the user specified with the provided user ID, and then try to rebind to Active Directory using the matched DN and provided password.

Next, if the credentials are validated, Directory Integrator looks for the corresponding user in Directory Sever and updates its password. Then Directory Integrator returns an HTTP redirect to return the user to the rest of the portal login script (that is, the li page of the portal) to log the user into the portal. If the credentials are not validated, the user is returned to WebSphere Portal's login screen with an error.

Therefore, passwords are propagated in Directory Sever each time the user logs into WebSphere Portal. Admittedly this is not very efficient, and in a production environment this should only be done if the supplied password does not match the one in Directory Server. In fact, at first we implemented such functionality in Directory Integrator. The first check by the Directory Integrator assemblyLine would be to check whether the supplied password is valid in Directory Server, and stop its execution if it is. However the customer insisted on the principle that users who had recently changed their password on Active Directory should not be able to login to the portal using their former password. Therefore, we disabled this functionality.

Of course, you need to take special steps to put this system into production. First, you should writing a JSP page to send the form data directly to Directory Integrator, instead of sending an HTTP redirect to the user's browser. You should also consider configuring SSL at all edges of the system to ensure password secrecy. Furthermore, you could use message queues if performance becomes a problem.

In summary, the password synchronization solution that we implemented consists of an Directory Integrator HTTP EventHandler waiting for the user credentials received from the WebSphere Portal login page, and an assemblyLine to perform the authentication job against Active Directory and password propagation on Directory Server. The following sub-sections describe how we implemented this solution.

The PasswordSyncEH eventHandler

The PasswordSyncEH HTTP event handler waits for incoming HTTP connections on port 88. When such a connection arrives, if the HTML user ID and password parameters have been received, then the handler starts the passwordSync assemblyLine. Iif the credentials were not received, it returns a form to the user's browser, and therefore behaves like a small Web server.

The event handler's SyncPwd action map starts the SyncPassword assemblyLine if the uid and password is received. It passes the event object, communicates the credentials to the assemblyLine, and waits for the assemblyLine to complete execution. After the assemblyLine has completed, Directory Integrator sends an HTTP redirect to the user's browser, which logs him into the portal (through the /wps/portal/.cmd/li script).

The SyncPassword assemblyLine

The SyncPassword assemblyLine includes four connectors:

The authenticateIDS connector runs first to check whether the user-provided password is already valid within Directory Sever. If it is valid, this connector aborts the assemblyLine's execution because no propagation is needed at this stage. This connector is disabled by configuration. The customer did not accept this connector because it would not prevent users from changing their password on Active Directory and still logging into the portal using their former password.

The dumpEntry script component logs information into the assemblyLine log. However, it is also disabled; we used it for debugging.

The findDN LDAP connector, preconfigured to connect to Active Directory, tries to lookup the Active Directory entry matching the user ID and password provided by the eventHandler. It connects as the wpsbind user, performs an LDAP search, and when a matching entry is found, tries to rebind to the directory using the matched DN and the user password. The code for these actions is in the connector's Lookup successful hook:

thisConnector.connector.setParam("ldapUsername", work.getString("$dn")); 
thisConnector.connector.setParam("ldapPassword", work.getString("http.qs.password")); 
 
task.logmsg("User DN found in AD, now trying to initialize the auth connector"); 
 
try 
{ 
 thisConnector.connector.initialize(null); 
} 
catch (e) 
{ 
 response = "401.htm"; 
 task.logmsg("user " + work.getString("http.qs.uid") + " 
 has supplied a wrong password"); 
 system.skipEntry(); 
}

If the lookup fails, the connector skips the entry. Because this assemblyLine is a single-pass assemblyLine (no iterators are defined in it), skipping the entry means executing the assemblyLine epilog and terminating the execution.

The last connector, updateIDS, updates the user's password in Directory Server. It modifies the entry whose user ID matches the one provided by the user.

Although this exercise uses Active Directory and Directory Sever as directory servers, you could use any LDAP-compliant directory server. With a little adaptation effort, you could also use this assemblyLine with non-LDAP servers as well. For example, you could change an Directory Integrator connector type from LDAP to JDBC or SOAP, and authenticate against a relational database, a web services server, or other server.

Back to top

Modifying the WebSphere Portal environment

In this section, you see to modify an existing WebSphere Portal installation to use a different LDAP server and schema. There are three parts to this task.

First, you modify the LDAP strings in DB2 to match the new LDAP schema.
Important: This is not a supported operation. You should have a good knowledge of DB2 and SQL commands in order to perform this operation correctly in your environment, because the structure of IDs will differ from that shown here. You should also take particular care if your version of WebSphere Portal is different from 4.2.0. We do not expect the relevant parts of the database structure to be different within the 4.x family, but we have not tested with all versions. If you are in any doubt, particularly with production systems, you should contact your local IBM WebSphere Services group.

Then, you change the wms.xml file that controls WebSphere Portal's relationship with the LDAP.

Finally, you change the WebSphere Application Server security settings.

Modifying DB2 setup

To modify the DB2 setup for LDAP:

Stop the Application Server and any processes that might use WebSphere Portal and WMS databases.

Open a DB2 command prompt and make a connection to the database. If you are not already logged in as the DB2 administrator, you will need to connect in that role. The command will be something like:

> connect to wpsdb user db2admin using [password]

Next, scan the tables in the database to see which of them include a reference to a distinguished name. Look for a full LDAP reference such as:

cn=wpsadmin,ou=portal,ou=ses1,ou=qi,ou=fe,dc=de,dc=customer,dc=com.

In our customer's case, WebSphere Portal and WMS facilities shared a database, but you may need to scan both if they are separate. The easiest way to do this is to sample each database in the DB2 Control Center, although it is also possible to do on the command line. Table 1 shows the tables that we found needed to be updated.

When you find a table that contains one or more distinguished names, you need to update any reference to wpsadmin, wpsbind or the group wpsadmins. Whether or not you choose to update any further references is a matter of choice. If you do not update the references, those users will effectively cease to exist within the portal, and any preferences, settings, and personal pages will be lost, with their redundant data still in the database. The majority of the user references inside the Portal are made using internal ID's that are mapped against distinguished names, and, once that mapping is lost, so are the settings for that user.

Unless the Portal is in pre-production, it will probably not be acceptable to leave users without their settings. On the other hand, it can be quite an effort to update all users manually, so it will almost certainly be worth considering a programmatic update process if many users need to be migrated.

In order to update the distinguished names in the tables, you need to replace them with the structure of distinguished names that your new LDAP uses. In our case, we were given a free hand to choose the structure, and decided on a standard schema to suit WebSphere Portal. The commands we used were similar to this:

> update member set unique_id = 'uid=wpsadmin,cn=users,dc=customer,dc=com' 
   where user_id ='cn=wpsadmin,ou=portal,ou=ses1,ou=qi,ou=fe,dc=de,dc=customer,
   dc=com' 
> update member set unique_id = 'uid=wpsbind,cn=users,dc=customer,dc=com' 
   where user_id = 'cn=wpsbind,ou=portal,ou=ses1,ou=qi,ou=fe,dc=de,dc=customer,
   dc=com' 
> update member set unique_id = 'cn=wpsadmins,cn=groups,dc=customer,dc=com' 
   where user_id = 'cn=wpsadmins,ou=portal,ou=ses1,ou=qi,ou=fe,dc=de,dc=customer,
   dc=com'

You see that wpsadmins, as a group, has the prefix cn= instead of uid= and belongs to cn=groups rather than cn=users in our target Directory Sever schema. Of course, these may differ according to your circumstances.

Table 1shows the table names and columns in those tables that we found needed to be changed. All of them are in the wpsdb database, and all are owned by db2admin. This was sufficient in our case, but your situation may vary. We abandoned Active Directory's former organizational structure, so we did not need to update tables such as ORGENTITYNAME.

Table 1: DB2 tables that needed modification

You should find this process fairly straightforward throughout the tables that need to be updated. The only real problem that we found was in the MEMBER table. The requirement to flatten the organizational hierarchy (for performance and maintainability) and the requirement for unique entries in the TRUNC_USER_ID column, meant we could not effectively transfer the references to organizational DNs in the hierarchy to the new schema. We left the references as they were without apparent adverse effects, but we would not recommend further testing.

When you are confident that you have successfully changed all the desired DNs from the old schema format to the new, you can commit your changes and disconnect from the database.

Modifying XML files

The next step is to modify the WMS settings in WebSphere Portal so that it recognizes the new LDAP. You can find the relevant file, called wms.xml, in [was_root]\lib\app\xml.

See the WebSphere Portal InfoCenter's topic called Managing membership page for an overview of the wms.xml. Table 2 shows all the settings in the [Directory] section that enable our customer's system to work correctly.

Table 2. The customer's Directory settings

Because we flattened the LDAP structure during the migration from Active Directory to Directory Sever the orgUnit* attributes no longer have meaning within the new structure. We kept them for completeness, but we set their values to be the same as those for the org* attributes. Also, the grpDummyMember setting is only required for Secureway-derived products (in order to maintain X500 compatibility). You need to add it if your are moving from another LDAP to Directory Server.

If you are using, or planning to use, the credential vault service, you need to update the vaultservice.properties file. You can find this in [was_root]\lib\app\config\services. The last line of this file is similar to this:

systemcred.dn=cn=wpsadmin,ou=portal,ou=ses1,ou=qi,ou=fe,dc=de,dc=customer,dc=com

You should change this to your new Portal administrator DN. In our case, that is:

systemcred.dn=uid=wpsadmin,cn=users,dc=customer,dc=com

The last XML file that you may need to change is attributemap.xml, which can be found under [wps_root]\wms\xml. This file controls how WebSphere Portal looks for certain attributes in the LDAP, because different schemas can have different names for a similar attribute. For this customer situation, we were able to keep the Directory Sever schema very simple and needed to change a few of the attributes. Table 3 shows the mapping between the objectAttribute and pluginAttribute names that we used.

Table 3. Attribute mapping

We made the changes as shown in the table, and left existing object/plugin pairs in place, even though we do not expect to use them.

We do not use preferredLanguage, even though most users of the inetOrgPerson schema do. If you are moving fromActive Directory to another LDAP and want to enable preferredLanguage, you must uncomment it from the attributeMap.xml file. A bug in Active Directory causes problems when used with preferredLanguage. Similarly if you are moving to Active Directory from another LDAP, you must make sure that you out preferredLanguage.

Modifying WebSphere Application Server setup

The last step is to modify the security configuration for WebSphere Application Server.

1. Within the Administrator's Console, open the Security Center, and de-select Enable Global Security on the first panel. Before you click OK,

2. Select the Authentication panel to change the LDAP settings. The settings we used are fairly typical for enabling authentication against a standard Directory Sever image configured for Portal. You may need to make changes if your schema structure is different.
   
   Table 4. Authentication settings for LDAP.
   

   Parameter	Value
   Security Server ID	uid=wpsbind,cn=users,dc=customer,dc=com
   Security Server Password	[wpsbind's password]
   Host	fe0portal02.deilab1.customer.com
   Port	389
   Base DN	dc=customer,dc=com
   Bind DN/PWD	[leave blank]

3. Select the Secureway as the LDAP type, and then click on the adjacent Advanced button.

4. When the sub-panel opens up, change the object class of a user from ePerson to inetOrgPerson, and then click OK.
   Tip: You can also make further performance tuning modifications here, such as removing references to groupOfNames and member from the group filters, and arranging search orders so that the object class comes before the prefix. However, the single change of the user object class brings the settings into line with a vanilla WebSphere Portal installation.

5. Click OK on the sub-panel, and again on the Security Center. A message advises that the settings will not take effect until the node is restarted. Therefore, restart the node, and then open the Admin Console and Security Center again.

You also need to reset the Admin Role, just as if you were installing a new instance of WebSphere Portal.

1. On the final tab and delete the user who is currently listed in the Admin Role.

2. Type the DN of your admin user into the search box. In our case, we typed:
   
   uid=wpsadmin,cn=users,dc=customer,dc=com

You could search for wpsadmin with a * in the search box, unless you have a very large number of users that could be returned. In our customer's case, there were 200K users. You may also encounter problems if you have a space in the DN.
   

3. Give that user the Admin Role.
4. On the first tab of the Security Console, re-enable security, and then restart the node.
5. In the Admininstator's Console, make sure that WebSphere Portal starts up with Application Server.
   Tip: If you make an error in the security settings that prevents you from getting into the Admininstator's Console, you can disable Application Server security from within DB2 by connecting to the Application Server database and issuing the following commands:

   > connect to wasdb user db2admin using [password] > update EJSADMIN.SECURITYCFG_TABLE set SECURITYENABLED=0 > commit

   
   
6. Assuming, however, that all is well, you can test your changes. Open a Web browser, navigate to your WebSphere Portal site, and log on as wpsadmin. If everything is as you expected, you have succeeded!
7. You can, of course, also test the validity of the changes with any other user who has already been migrated to the new LDAP, or if you have not yet migrated the passwords to your new Directory Server, you can create a new user and perform further tests.

Back to top

Conclusion

As you can see, the process of configuring WebSphere Portal and IBM Directory Integrator to migrate data from one LDAP to another is reasonably simple. Even though performing the work quickly for a proof of concept allowed us to leave a few rough edges, we could very easily rectify them when we had more time.

Back to top

Download

Name	Size	Download method
rs.xml	150 KB	FTP|HTTP

	Information about download methods

About the authors

		Nick Lawrence is an Advisory IT Specialist in the IBM Software Services for WebSphere EMEA group, based in Hursley Labs. He has specialised in WebSphere Portal for the last two years, with a particular focus on Portal security.

		Christian Chbteauvieux is an Advisory IT Specialist in the IBM Software Services for Tivoli EMEA group. He recently joined IBM from Metamerge, the company which was bought when IBM acquired Directory Integrator, where he also had an EMEA responsibility.

Rate this page

Please take a moment to complete this form to help us better serve you. Did the information help you to achieve your goal? Yes No Don't know   Please provide us with comments to help improve this page:   How useful is the information? 1 2 3 4 5 Not useful Extremely useful

Share this.... Digg this story del.icio.us Slashdot it!

Back to top