From 1d5c02ac40d7af22005eb681bf9d6615f386b7e2 Mon Sep 17 00:00:00 2001 From: Jen Malloy Date: Wed, 29 Mar 2017 12:54:50 -0400 Subject: [PATCH] fixed RHSSO-942 --- .../topics/user-storage/cache.adoc | 28 +++++----- .../topics/user-storage/import.adoc | 53 +++++++++---------- .../topics/user-storage/javaee.adoc | 34 ++++++------ .../topics/user-storage/migration.adoc | 22 ++++---- .../user-storage/registration-query.adoc | 31 +++++------ .../topics/user-storage/rest.adoc | 6 +-- 6 files changed, 82 insertions(+), 92 deletions(-) diff --git a/server_development/topics/user-storage/cache.adoc b/server_development/topics/user-storage/cache.adoc index c4d535ef91..8a983a7276 100644 --- a/server_development/topics/user-storage/cache.adoc +++ b/server_development/topics/user-storage/cache.adoc @@ -1,14 +1,14 @@ === User Caches -When a user is loaded by id, username, or email queries it will be cached. When a user is cached, it iterates through -the entire `UserModel` interface and pulls this information to a local in-memory only cache. In a cluster, this cache -is still local, but it becomes an invalidation cache. When a user is modified, it is evicted. This eviction event -is propagated to the entire cluster so that other nodes' user cache is also invalidated. +When a user is loaded by ID, username, or email queries it is cached. When a user is cached, it iterates through +the entire `UserModel` interface and pulls this information to a local in-memory-only cache. In a cluster, this cache +is still local, but it becomes an invalidation cache. When a user is modified, it is evicted. This eviction event +is propagated to the entire cluster so that the other nodes' user cache is also invalidated. ==== Managing the user cache -You can get access to the user cache by calling `KeycloakSession.userCache()`. +You can access the user cache by calling `KeycloakSession.userCache()`. [source,java] ---- @@ -41,11 +41,11 @@ public interface UserCache extends UserProvider { } ---- -There are methods for evicting a specific users, users contained in a specific realm, or the entire cache. +There are methods for evicting specific users, users contained in a specific realm, or the entire cache. ==== OnUserCache Callback Interface -You may want to cache additional information that is specific to your provider implementation. The User Storage SPI +You might want to cache additional information that is specific to your provider implementation. The User Storage SPI has a callback whenever a user is cached: `org.keycloak.models.cache.OnUserCache`. [source,java] @@ -55,8 +55,8 @@ public interface OnUserCache { } ---- -Your provider class should implement this interface if it wants this callback. The `UserModel` delegate parameter -is the `UserModel` instance returned by your provider. The `CachedUserModel` is an expanded `UserModel` interface. +Your provider class should implement this interface if it wants this callback. The `UserModel` delegate parameter +is the `UserModel` instance returned by your provider. The `CachedUserModel` is an expanded `UserModel` interface. This is the instance that is cached locally in local storage. [source,java] @@ -94,14 +94,10 @@ public interface CachedUserModel extends UserModel { } ---- -This `CachedUserModel` interface allows you to evict the user from cache and get the provider `UserModel` instance. -The most interesting method is `getCachedWith()`. This returns a map that allows you to cache additional information -pertaining to the user. For example, credentials are not part of the `UserModel` interface. If you wanted to cache -credentials in memory, you would implement `OnUserCache` and cache your user's credentials using the `getCachedWith()` -method. +This `CachedUserModel` interface allows you to evict the user from the cache and get the provider `UserModel` instance. +The `getCachedWith()` method returns a map that allows you to cache additional information pertaining to the user. For example, credentials are not part of the `UserModel` interface. If you wanted to cache credentials in memory, you would implement `OnUserCache` and cache your user's credentials using the `getCachedWith()` method. ==== Cache Policies -Each configured user storage provider can specify unique cache policies. Go to the admin console management page -for your provider to see how to do this. +On the administration console management page for your user storage provider, you can specify a unique cache policy for each configured user storage provider. diff --git a/server_development/topics/user-storage/import.adoc b/server_development/topics/user-storage/import.adoc index 6e4e4b2310..f51fc4a712 100644 --- a/server_development/topics/user-storage/import.adoc +++ b/server_development/topics/user-storage/import.adoc @@ -1,29 +1,29 @@ === Import Implementation Strategy -When implementing a user storage provider, there's another strategy you can take. Instead of using user federated storage, -you can create a user locally in the {{book.project.name}} built in user database and copy attributes from your external -store into this local copy. There are a bunch of advantages to this approach. +When implementing a user storage provider, there's another strategy you can take. Instead of using user federated storage, +you can create a user locally in the {{book.project.name}} built-in user database and copy attributes from your external +store into this local copy. There are many advantages to this approach. -* {{book.project.name}} basically becomes a persistence user cache for your external store. Once the user is imported +* {{book.project.name}} basically becomes a persistence user cache for your external store. Once the user is imported you'll no longer hit the external store thus taking load off of it. * If you are moving to {{book.project.name}} as your official user store and deprecating the old external store, you -can slowly migrate applications to use {{book.project.name}}. When all applications have been migrated, unlink the +can slowly migrate applications to use {{book.project.name}}. When all applications have been migrated, unlink the imported user, and retire the old legacy external store. There are some obvious disadvantages though to using an import strategy: -* Looking up a user for the first time will require multiple updates to {{book.project.name}} database. This can -be a big performance loss under load and put a lot of strain on the {{book.project.name}} database. The user federated +* Looking up a user for the first time will require multiple updates to {{book.project.name}} database. This can +be a big performance loss under load and put a lot of strain on the {{book.project.name}} database. The user federated storage approach will only store extra data as needed and may never be used depending on the capabilities of your external store. -* With the import approach, you have to keep local keycloak storage and external storage in sync. The User Storage SPI +* With the import approach, you have to keep local keycloak storage and external storage in sync. The User Storage SPI has capability interfaces that you can implement to support synchronization, but this can quickly become painful and messy. -To implement the import strategy you simply check to see first if the user has been imported locally. If so return the -local user, if not create the user locally and import data from the external store. You can also proxy the local user +To implement the import strategy you simply check to see first if the user has been imported locally. If so return the +local user, if not create the user locally and import data from the external store. You can also proxy the local user so that most changes are automatically synchronized. -This will be a bit contrived, but we can extend our `PropertyFileUserStorageProvider` to take this approach. We +This will be a bit contrived, but we can extend our `PropertyFileUserStorageProvider` to take this approach. We begin first by modifying the `createAdapter()` method. .PropertyFileUserStorageProvider @@ -50,24 +50,24 @@ begin first by modifying the `createAdapter()` method. ---- In this method we call the `KeycloakSession.userLocalStorage()` method to obtain a reference to local {{book.project.name}} -user storage. We see if the user is stored locally, if not, we add it locally. Also note that we call -`UserModel.setFederationLink()` and pass in the id of the `ComponentModel` of our provider. This sets a link between +user storage. We see if the user is stored locally, if not, we add it locally. Also note that we call +`UserModel.setFederationLink()` and pass in the ID of the `ComponentModel` of our provider. This sets a link between the provider and the imported user. NOTE: When a user storage provider is removed, any user imported by it will also be removed. This is one of the purposes of calling `UserModel.setFederationLink()`. Another thing to note is that if a local user is linked, your storage provider will still be delegated to for methods -that it implements from the `CredentialInputValidator` and `CredentialInputUpdater` interfaces. Returning `false` +that it implements from the `CredentialInputValidator` and `CredentialInputUpdater` interfaces. Returning `false` from a validation or update will just result in {{book.project.name}} seeing if it can validate or update using local storage. Also notice that we are proxying the local user using the `org.keycloak.models.utils.UserModelDelegate' class. -This class is an implementation of `UserModel`. Every method just delegates to the `UserModel` it was instantiated with. +This class is an implementation of `UserModel`. Every method just delegates to the `UserModel` it was instantiated with. We override the `setUsername()` method of this delegate class to synchronize automatically with the property file. For your providers, you can use this to _intercept_ other methods on the local `UserModel` to perform synchronization -with your extern store. For example, get methods could make sure that the local store is in sync. Set methods -keep external store in sync with local one. +with your external store. For example, get methods could make sure that the local store is in sync. Set methods +keep the external store in sync with the local one. NOTE: If your provider is implementing the `UserRegistrationProvider` interface, your `removeUser()` method does not need to remove the user from local storage. The runtime will automatically perform this operation. Also @@ -97,15 +97,15 @@ public interface ImportedUserValidation { ---- Whenever a linked local user is loaded, if the user storage provider class implements this interface, then the -`validate()` method is called. Here you can proxy the local user passed in as a parameter and return it. That -new `UserModel` will be used. You can also optionally do a check to see if the user exists still in the external store. -if `validate()` returns `null`, then the local user will be removed from the database. +`validate()` method is called. Here you can proxy the local user passed in as a parameter and return it. That +new `UserModel` will be used. You can also optionally do a check to see if the user still exists in the external store. +If `validate()` returns `null`, then the local user will be removed from the database. ==== ImportSynchronization Interface -With the import strategy you can see that it would be possible for the local user copy could get out of sync with -external storage. For example, maybe a user has been removed from the external store. The User Storage SPI has -an additional interface you can implement to deal with this. `org.keycloak.storage.user.ImportSynchronization`. +With the import strategy you can see that it is possible for the local user copy to get out of sync with +external storage. For example, maybe a user has been removed from the external store. The User Storage SPI has +an additional interface you can implement to deal with this, `org.keycloak.storage.user.ImportSynchronization`: [source,java] ---- @@ -117,8 +117,5 @@ public interface ImportSynchronization { } ---- -This interface is implemented by the provider factory. Once this interface is implemented by the provider factory, -the admin console management page for the provider will show additional options. There is a button that will allow -you to manually force a synchronization. This invokes the `ImportSynchronization.sync()` method. Also, some additional -configuration options will show up that allow you to automatically schedule a synchronization. Automatic syncs invoke -the `syncSince()` method. +This interface is implemented by the provider factory. Once this interface is implemented by the provider factory, the administration console management page for the provider shows additional options. You can manually force a synchronization by clicking a button. This invokes the `ImportSynchronization.sync()` method. Also, additional configuration options are displayed that allow you to automatically schedule a synchronization. Automatic synchronizations invoke the `syncSince()` method. + diff --git a/server_development/topics/user-storage/javaee.adoc b/server_development/topics/user-storage/javaee.adoc index eaeb9da53b..ee52128bf2 100644 --- a/server_development/topics/user-storage/javaee.adoc +++ b/server_development/topics/user-storage/javaee.adoc @@ -1,16 +1,16 @@ === Leveraging Java EE -The user storage providers can be packaged within any Java EE component so long as you set up the `META-INF/services` -file correctly to point to your providers. For example, if your provider needs to use third party libraries, you -can package up your provider within an ear and store these third pary libraries in the ear's `lib/` directory. -Also note that provider jars can make use of the `jboss-deployment-structure.xml` file that EJBs, WARS, and EARs -can use in a JBoss/Wildfly environment. See the JBoss/Wildfly documentation for more details on this file. It -allows you to pull in external dependencies among other fine grain actions. +The user storage providers can be packaged within any Java EE component if you set up the `META-INF/services` +file correctly to point to your providers. For example, if your provider needs to use third-party libraries, you +can package up your provider within an EAR and store these third-party libraries in the `lib/` directory of the EAR. +Also note that provider JARs can make use of the `jboss-deployment-structure.xml` file that EJBs, WARS, and EARs +can use in a JBoss/Wildfly environment. For more details on this file, see the JBoss/Wildfly documentation. It +allows you to pull in external dependencies among other fine-grained actions. -Implementations of `UserStorageProviderFactory` are required to be plain java objects. But, we also currently support -implementing `UserStorageProvider` classes as Stateful EJBs. This is especially useful if you want to use JPA -to connect to a relational store. This is how you would do it: +Implementations of `UserStorageProviderFactory` are required to be plain java objects. But we also currently support +implementing `UserStorageProvider` classes as Stateful EJBs. This is especially useful if you want to use JPA +to connect to a relational store. This is how you would do it: [source,java] ---- @@ -47,13 +47,13 @@ public class EjbExampleUserStorageProvider implements UserStorageProvider, } ---- -You have to define the `@Local` annotation and specify your provider class there. If you don't do this, EJB will +You have to define the `@Local` annotation and specify your provider class there. If you do not do this, EJB will not proxy the user correctly and your provider won't work. -You must put the `@Remove` annotation on the `close()` method of your provider. If you don't, the stateful bean -will never be cleaned up and you may eventually see error messages. +You must put the `@Remove` annotation on the `close()` method of your provider. If you do not, the stateful bean +will never be cleaned up and you might eventually see error messages. -Implementations of `UserStorageProviderFactory` are required to be plain java objects. Your factory class would +Implementations of `UserStorageProviderFactory` are required to be plain java objects. Your factory class would perform a JNDI lookup of the Stateful EJB in its create() method. [source,java] @@ -76,12 +76,12 @@ public class EjbExampleUserStorageProviderFactory } ---- -This example also assumes that you've defined a JPA deployment in the same jar as the provider. This means a `persistence.xml` +This example also assumes that you have defined a JPA deployment in the same JAR as the provider. This means a `persistence.xml` file as well as any JPA `@Entity` classes. -WARNING: When doing JPA any additional datasource you use must be an XA datasource. The {{book.project.name}} datasource - is non-xa. If you interact with two or more non-xa datasource in the same transaction, the server will barf with - an error message. You can only have one non-xa resource in a single transaction. +WARNING: When doing JPA any additional datasource you use must be an XA datasource. The {{book.project.name}} datasource + is non-xa. If you interact with two or more non-xa datasource in the same transaction, the server returns + an error message. Only one non-xa resource is permitted in a single transaction. See the JBoss/Wildfly manual for more details on deploying an XA datasource. diff --git a/server_development/topics/user-storage/migration.adoc b/server_development/topics/user-storage/migration.adoc index 7a53a3c529..7bede85ffe 100644 --- a/server_development/topics/user-storage/migration.adoc +++ b/server_development/topics/user-storage/migration.adoc @@ -33,13 +33,9 @@ has capability interfaces that you can implement to support synchronization, but ==== UserFederationProvider vs. UserStorageProvider -The first thing to notice is that `UserFederationProvider` was a complete interface. You implemented every method -in this interface. However, `UserStorageProvider` has instead broken up this interface into multiple capability interfaces that -you implement as needed. +The first thing to notice is that `UserFederationProvider` was a complete interface. You implemented every method in this interface. However, `UserStorageProvider` has instead broken up this interface into multiple capability interfaces that you implement as needed. -`UserFederationProvider.getUserByUsername()` and `getUserByEmail()` have exact equivalents in the new SPI. The difference -between the two is how you import. If you are going to continue with an import strategy, you no longer call -`KeycloakSession.userStorage().addUser()' to create the user locally. Instead you call `KeycloakSession.userLocalStorage().addUser()`. +`UserFederationProvider.getUserByUsername()` and `getUserByEmail()` have exact equivalents in the new SPI. The difference between the two is how you import. If you are going to continue with an import strategy, you no longer call `KeycloakSession.userStorage().addUser()` to create the user locally. Instead you call `KeycloakSession.userLocalStorage().addUser()`. The `userStorage()` method no longer exists. The `UserFederationProvider.validateAndProxy()` method has been moved to an optional capability interface, `ImportedUserValidation`. @@ -51,7 +47,7 @@ then the `ImportedUserValidation.validate()` method is not called at all. The `UserFederationProvider.isValid()` method no longer exists in the later model. The `UserFederationProvider` methods `synchronizeRegistrations()`, `registerUser()`, and `removeUser()` have been -moved to the `UserRegistrationProvider` capability interface. This new interface is optional to implement so if your +moved to the `UserRegistrationProvider` capability interface. This new interface is optional to implement so if your provider does not support creating and removing users, you don't have to implement it. If your earlier provider had switch to toggle support for registering new users, this is supported in the new SPI, returning `null` from `UserRegistrationProvider.addUser()` if the provider doesn't support adding users. @@ -61,12 +57,12 @@ and `CredentialInputUpdater` interfaces, which are also optional to implement de updating credentials. Credential management used to exist in `UserModel` methods. These also have been moved to the `CredentialInputValidator` and `CredentialInputUpdater` interfaces. One thing to note that if you do not implement the `CredentialInputUpdater` interface, then -any credentials provided by your provider can be overridden locally in {{book.project.name}} storage. So if you want +any credentials provided by your provider can be overridden locally in {{book.project.name}} storage. So if you want your credentials to be read-only, implement the `CredentialInputUpdater.updateCredential()` method and return a `ReadOnlyException`. The `UserFederationProvider` query methods such as `searchByAttributes()` and `getGroupMembers()` are now encapsulated -in an optional interface `UserQueryProvider`. If you do not implement this interface, then users will not be viewable +in an optional interface `UserQueryProvider`. If you do not implement this interface, then users will not be viewable in the admin console. You'll still be able to login though. ==== UserFederationProviderFactory vs. UserStorageProviderFactory @@ -77,9 +73,9 @@ If you have implemented synchronization logic, then have your new `UserStoragePr ==== Upgrading to a New Model -The User Storage SPI instances are stored in a completely different set of relational tables or Mongo schema. {{book.project.name}} +The User Storage SPI instances are stored in a completely different set of relational tables or Mongo schema. {{book.project.name}} automatically runs a migration script. If any earlier User Federation providers are deployed for a realm, they are converted -to the later storage model as is, including the `id` of the data. This migration will only happen if a User Storage provider exists +to the later storage model as is, including the `id` of the data. This migration will only happen if a User Storage provider exists with the same provider ID (i.e., "ldap", "kerberos") as the earlier User Federation provider. So, knowing this there are different approaches you can take. @@ -88,8 +84,8 @@ So, knowing this there are different approaches you can take. of imported users. Then, when you upgrade {{book.project.name}}, just deploy and configure your new provider for your realm. . The second option is to write your new provider making sure it has the same provider ID: `UserStorageProviderFactory.getId()`. Make sure this provider is in the `deploy/` directory of the new {{book.project.name}} installation. Boot the server, and have - the built-in migration script convert from the earlier data model to the later data model. In this case all your earlier linked imported + the built-in migration script convert from the earlier data model to the later data model. In this case all your earlier linked imported users will work and be the same. If you have decided to get rid of the import strategy and rewrite your User Storage provider, we suggest that you remove the earlier provider -before upgrading {{book.project.name}}. This will remove linked local imported copies of any user you imported. \ No newline at end of file +before upgrading {{book.project.name}}. This will remove linked local imported copies of any user you imported. diff --git a/server_development/topics/user-storage/registration-query.adoc b/server_development/topics/user-storage/registration-query.adoc index b4eb9ebfd7..b5fccce094 100644 --- a/server_development/topics/user-storage/registration-query.adoc +++ b/server_development/topics/user-storage/registration-query.adoc @@ -1,8 +1,8 @@ === Add/Remove User and Query Capability interfaces -One thing we have not done with our example is allow it to add and remove users or change passwords. Users defined in our example are -also not queryable or viewable in the admin console. To add these enhancements, our example provider must implement +One thing we have not done with our example is allow it to add and remove users or change passwords. Users defined in our example are +also not queryable or viewable in the administration console. To add these enhancements, our example provider must implement the `UserQueryProvider` and `UserRegistrationProvider` interfaces. ==== Implementing UserRegistrationProvider @@ -26,7 +26,7 @@ file to disk. } ---- -Then, the implementation of the `addUser()` and `removeUser()` methods becomes pretty simple. +Then, the implementation of the `addUser()` and `removeUser()` methods becomes simple. .PropertyFileUserStorageProvider [source,java] @@ -53,10 +53,10 @@ Then, the implementation of the `addUser()` and `removeUser()` methods becomes p ---- Notice that when adding a user we set the password value of the property map to be `UNSET_PASSWORD`. We do this as -we can't have null values for a property in the property value. We also have to modify the `CredentialInputValidator` +we can't have null values for a property in the property value. We also have to modify the `CredentialInputValidator` methods to reflect this. -`addUser()` will be called if the provider implements the `UserRegistrationProvider` interface. If your provider has +`addUser()` will be called if the provider implements the `UserRegistrationProvider` interface. If your provider has a configuration switch to turn of adding a user, returning `null` from this method will skip the provider and call the next one. @@ -74,7 +74,7 @@ the next one. } ---- -Since we can now save our property file, probably also makes sense to allow password updates. +Since we can now save our property file, it also makes sense to allow password updates. .PropertyFileUserStorageProvider [source,java] @@ -120,12 +120,12 @@ We can now also implement disabling a password too. } ---- -With these methods implemented, you'll now be able to change and disable the password for the user in the admin console. +With these methods implemented, you'll now be able to change and disable the password for the user in the administration console. ==== Implementing UserQueryProvider -Without implementing `UserQueryProvider` the admin console would not be able to view and manage users that were loaded -by our example provider. Let's look at implementing this interface. +Without implementing `UserQueryProvider` the administration console would not be able to view and manage users that were loaded +by our example provider. Let's look at implementing this interface. .PropertyFileUserStorageProvider [source,java] @@ -155,8 +155,8 @@ by our example provider. Let's look at implementing this interface. } ---- -The `getUser()` method simple iterates the key set of the property file delegating to `getuserByUsername` to load a user. -Notice that we are indexing this call based on the `firstResult` and `maxResults` parameter. If your external store +The `getUser()` method simple iterates over the key set of the property file delegating to `getuserByUsername` to load a user. +Notice that we are indexing this call based on the `firstResult` and `maxResults` parameter. If your external store doesn't support pagination, you'll have to do similar logic. .PropertyFileUserStorageProvider @@ -183,8 +183,8 @@ doesn't support pagination, you'll have to do similar logic. } ---- -The first declaration of `searchForUser()` takes a string paraeter. This is supposed to be a string that you use to -search username and email attributes to find the user. This string can be a substring which is why we use the `String.contains()` +The first declaration of `searchForUser()` takes a string parameter. This is supposed to be a string that you use to +search username and email attributes to find the user. This string can be a substring, which is why we use the `String.contains()` method when doing our search. .PropertyFileUserStorageProvider @@ -205,7 +205,7 @@ method when doing our search. ---- The `searchForUser()` method that takes a `Map` parameter can search for a user based on first, last name, username, and email. -We only store usernames, so we only search based on usernames. We delegate to `searchForUser()` for this. +We only store usernames, so we only search based on usernames. We delegate to `searchForUser()` for this. .PropertyFileUserStorageProvider @@ -227,4 +227,5 @@ We only store usernames, so we only search based on usernames. We delegate to ` } ---- -We don't store and groups or attributes, so the other methods just return an empty list. +We do not store groups or attributes, so the other methods return an empty list. + diff --git a/server_development/topics/user-storage/rest.adoc b/server_development/topics/user-storage/rest.adoc index 49668ecdae..7ca6302a90 100644 --- a/server_development/topics/user-storage/rest.adoc +++ b/server_development/topics/user-storage/rest.adoc @@ -1,7 +1,7 @@ === REST Management API -You can create, remove, and update your user storage provider deployments through the admin REST api. The User Storage SPI +You can create, remove, and update your user storage provider deployments through the administrator REST API. The User Storage SPI is built on top of a generic component interface so you will be using that generic API to manage your providers. The REST Component API lives under your realm admin resource. @@ -10,8 +10,7 @@ The REST Component API lives under your realm admin resource. /admin/realms/{realm-name}/components ---- -We will only show this REST API interaction with the Java client. Hopefully you can extract how do do this from -curl from this api. +We will only show this REST API interaction with the Java client. Hopefully you can extract how to do this from curl from this API. [source,java] ---- @@ -100,3 +99,4 @@ realmResource.components().component(component.getId()).update(component); realmREsource.components().component(component.getId()).remove(); ---- +