Filters

JPA criteria API is used for search entities from underlying database. JPA predicate has to be constructed for all supported filter properties. Predicate can be constructed two ways:

• by service - predicates are constructed in AbstractReadDtoService#toPredicates method. This predicates are statically implemented and not overidable.
• By registered filter builders - A mechanism for dynamic registry and composing of filters has been developed in the application for searching the data in IdM (identities, roles, tree structure components, etc.). A new filter can be registered for existing REST services in any module. Filter builder can be registered by custom module and can override filter builder from other module (e.g. filter in custom module can override core filter behavior).

Filter can be used by authoritazation policy evaluator for securing data access.

Filter builder

The filter (FilterBuilder) is registered with the given key (FilterKey):

• entityClass - a domain type for which it is intended
• name - the name of the item during which the filter is actively evaluated if it is stated in the filtering criteria (=> get parameter)

Evaluating of filters is done by FilterManager, which searches over the domain type during construction:

• collects all registered active filters by a key:
  • complying with the given domain type
  • complying with the name of the item existing in the parameters by which it is being filtered
• and calls method getPredicate over all the found filters the results of which (if there are some) are merged into unified searching criteria via operator and.

A filter must implement these methods:

• getName - returns the item name - see above
• getPredicate - construction of the searching criteria themselves
• find - returns data (objects by the domain type - see above) solely by the predicate constructed above

The following ready-made abstract classes can be used for constructing a new filter:

• BaseFilterBuilder - provides the base implementation for working with the configuration (see below)
• AbstractFilterBuilder - provides the base implementation of find method based on the repository handed in the constructor

Example filter

This example filter is meant for searching for identities by the username.

Example filter test

For every filter is highly recommended create also tests for filtering. There is example:

Filter configuration

Filters are configurable thanks to interface Configurable via the standard application configuration.

Implemented filters

UuidFilter

A generic filter for finding any entity (child of AbstractEntity) by its uuid (parameter in the filter - id).

UsernameIdentityFilter

Finding any entity by its username (parameter in the filter - username).

DefaultManagersFilter

Finding managers of the entity by the IR, tree structures and set guarantees. The manager is the identity which has its IR on an organizational structure component superordinate to a structure component of the subordinate's IR. The manager (guarantee) can also be configured directly to the IR.

Filter parameters

• managersFor - a key parameter; uuid of the subordinate for which the searched are managers
• managersByTreeType - an additional parameter; if it is entered, the managers are being searched for only by the IR with the given structure type
• managersByContract - an additional parameter; if it is entered, the managers are being searched for only by the given IR
• includeGuarantees - True => it is being searched for managers by tree structures and managers (guarantee) configured directly to the IR. False => only managers by tree structures.
• validContractManagers - an additional parameter; valid now or in future contracts. True => managers of contracts ended in the past will not be returned. False => managers of contracts ended in the past will be returned. All managers otherwise.

Configuration:

DefaultManagerContractBySubordinateContractFilter

@since 10.4.0

Find managers' contracts by subordinate contract. Finding managers' contracts by tree structures and set guarantees. The manager contract is on an organizational structure component superordinate to a structure component of the subordinate's contract. The manager (guarantee) can also be configured directly to the subordinate contract.

Filter parameters

• managersByContract - a key parameter; uuid of the subordinate contract for which the searched are managers' contracts
• includeGuarantees - True => it is being searched for managers' contracts by tree structures and by direct guarantees. False => only managers' contracts by tree structures.
• validContractManagers - an additional parameter; valid now or in future contracts. True => manager contracts of subordinate contract ended in the past will not be returned. False => manager contracts of subordinate contract ended in the past will be returned. All managers' contracts otherwise.

Configuration:

DefaultSubordinatesFilter

Finding subordinates of the entity by the IR, tree structures and set guarantees. The subordinate is the identity which has its IR on an organizational structure component subordinate to a structure component of the manager's IR. The guarantee can also be configured directly to the IR.

Filter parameters

• subordinatesFor - a key parameter; uuid of the manager for which the searched are subordinates
• subordinatesByTreeType - an additional parameter; if it is entered, the subordinates are being searched for only by the IR with the given structure type
• includeGuarantees - True => it is being searched for subordinates by tree structures and subordinates by managers (guarantee) configured directly to the IR. False => only subordinates by tree structures.

Configuration:

DefaultContractByManagerFilter

@since 9.7.0

Finding subordinate contracts of the manager by the IR, tree structures and set guarantees. The subordinate contract is the contract which has its IR on an organizational structure component subordinate to a structure component of the manager's IR. The guarantee can also be configured directly to the IR.

Filter parameters

• subordinatesFor - a key parameter; uuid of the manager for which the searched are subordinate contracts
• subordinatesByTreeType - an additional parameter; if it is entered, the subordinate contracts are being searched for only by the IR with the given structure type
• includeGuarantees - True => it is being searched for subordinate contracts by tree structures and subordinate contracts by managers (guarantee) configured directly to the IR. False => only subordinate contracts by tree structures.

Configuration:

GuaranteeManagersFilter

Finding managers of the entity by the set guarantees. The manager (guarantee) has to be configured directly to the IR.

Filter parameters

• managersFor - a key parameter; uuid of the subordinate for which the searched are managers
• managersByTreeType - if it is entered, the filter is disabled (=> returns no data ~ disjunction)
• includeGuarantees - has to be true => it is being searched for managers configured directly to the IR. False => returns no data ~ disjunction.
• validContractManagers - an additional parameter; valid now or in future contracts. True => managers of contracts ended in the past will not be returned. False => managers of contracts ended in the past will be returned. All managers otherwise.

Configuration:

GuaranteeContractBySubordinateContractFilter

@since 10.4.0

Finding managers' contracts by the given subordinate contract. The manager (guarantee) has to be configured directly to the IR. All managers' contracts are returned.

Filter parameters

• managersByContract - a key parameter; uuid of the subordinate contract for which the searched are managers' contracts
• validContractManagers - an additional parameter; valid now or in future contracts. True => managers of contracts ended in the past will not be returned. False => managers of contracts ended in the past will be returned. All managers otherwise.

Configuration:

GuaranteeSubordinatesFilter

Finding subordinates by directly configured managers to the IR.

Filter parameters

• subordinatesFor - a key parameter; uuid of the manager for which the searched are subordinates
• subordinatesByTreeType - if it is entered, the filter is disabled (=> returns no data ~ disjunction)
• includeGuarantees - has to be true => it is being searched for subordinates by directly configured managers to the IR. False => returns no data ~ disjunction.

Configuration:

ContractByGuaranteeFilter

@since 9.7.0

Finding subordinate contracts by directly configured managers to the IR.

Filter parameters

• subordinatesFor - a key parameter; uuid of the manager for which the searched are subordinate contracts
• subordinatesByTreeType - if it is entered, the filter is disabled (=> returns no data ~ disjunction)
• includeGuarantees - has to be true => it is being searched for subordinate contracts by directly configured managers to the IR. False => returns no data ~ disjunction.

Configuration:

EavCodeManagersFilter

The second option for finding managers in the same way as DefaultManagersFilter with a difference that the superordinate component is not being searched for by tree structure but by the value of an extended (eav) tree component parameter - a code of a different component is stated in the tree component, where there are managers (having IR on the given component) defined.

EavCodeManagerContractBySubordinateContractFilter

@since 10.4.0

The second option for finding managers' contracts in the same way as DefaultManagerContractBySubordinateContractFilter with a difference that the manager contract is not being searched for by tree structure but by the value of an extended (eav) tree component parameter - a code of a extended attribute is stated in the tree component, where there are managers (having contract on the given component) defined.

<del>EavCodeSubordinatesFilter</del>

The second option for finding subordinates in the same way as DefaultManagersFilter with a difference that the superordinate component is not being searched for by tree structure but by the value of an extended (eav) tree component parameter - a code of a different component is stated in the tree component, where there are managers (having IR on the given component) defined. Otherwise the functions and parameters are identical.

EavCodeContractByManagerFilter

The second option for finding subordinate contracts in the same way as DefaultContractByManagerFilter with a difference that the superordinate component is not being searched for by tree structure but by the value of an extended (eav) tree component parameter - a code of a different component is stated in the tree component, where there are managers (having IR on the given component) defined. Otherwise the functions and parameters are identical.

Search by group of identifiers

@since 15.17.1 (16.1.0 in the 16.x.x line)

A generalized filter for finding several records at once by a list of their identifiers. The user pastes a list of values (typically copied from a spreadsheet or from an export) into a single filter field and the agenda returns all records matching any of the values.

The filter is registered under the identifiers key. Searched attributes differ by entity type, therefore one filter builder is needed for every searched entity type - unlike ExternalIdentifiableFilter or DisableableFilter, which have a single generic builder for all entities.

Two artifacts form the abstraction:

• IdentifiersFilter - filter dto interface. Declares the identifiers parameter and provides getIdentifiers / setIdentifiers as default methods, so a filter dto only has to add the interface.
• AbstractIdentifiersFilterBuilder - base class of the filter builder. Implements getName and getPredicate (values are combined by or, each attribute as an in statement). A concrete builder only defines the searched attributes.

Where it is used

Adding the filter to another agenda

Three steps are needed.

1. Add IdentifiersFilter to the filter dto.
2. Register a filter builder extending AbstractIdentifiersFilterBuilder and define the searched attributes.
3. Add the filter field to the agenda table on the frontend (see below).

Contract of the searched attributes

getIdentifierPaths must return at least one path - an empty list would build builder.or() without arguments, which is a predicate that is never true, so the agenda would silently return nothing. The contract is enforced by Assert.notEmpty.

• Use root attributes, or attributes of a *ToOne relation owned by the root (the foreign key is on the root table). A join into a collection multiplies rows of the main query and the criteria are not distinct, so duplicates would end up in the paged result. An inverse *ToOne relation without a unique constraint has the same issue.
• Joining a *ToOne relation inside the method is allowed, but use an explicit left join. The path root.get(relation).get(attribute) renders an implicit inner join, which silently drops records with an empty relation - even when they match on another path of the same or predicate. This is why AccAccountIdentifiersFilter joins systemEntity explicitly with JoinType.LEFT: an account without a system entity has to be found by its own `uid`.
• Prefer paths on the same table and on indexed columns. A condition over the root table and a joined table combined by or cannot use an index on either side, so the whole main table is scanned (twice when paged). It can be an acceptable trade off, but it has to be a conscious decision.

Where the trade off was accepted:

• AccAccountIdentifiersFilter - searches two tables (`acc_account` and `sys_system_entity`), so the query does not use an index. The existing quick search (text) in the same agenda behaves identically, so it is not a regression.
• IdmTreeNodeIdentifiersFilter - both attributes are on the root table, but neither can use an index. `code` is not the leading column of `ux_tree_node_code (tree_type_id, code)` and `name` is not indexed at all. Tree structures are small compared to identities or accounts.

Limits and behavior

• Exact match including letter case. The predicate is built as path.in(identifiers) without lower(), so `Novak` and `novak` are not the same value. This differs from the quick search (text) in the same agendas, which is case insensitive. Values copied from an export with a different letter case will return nothing.
• One value can match more records. Attributes that are not unique are searched as well - for example a role base code, a role name or an identity last name. Searching for `payroll` can return `payroll|dev`, `payroll|test` and `payroll|prod`.
• At most 200 values can be inserted at once on the frontend. When the limit is exceeded, the whole input is refused and a warning is displayed. The inserted text stays in the input field so it can be shortened.
• The backend cap is 500 values for any filter, evaluated centrally in AbstractReadDtoService and configurable by idm.sec.core.filter.check.size.maximum. Exceeding it ends with FilterSizeExceededException (HTTP 400). The frontend limit is deliberately lower, so the user gets a readable message instead of a server error.
• Values are not deduplicated. Inserting the same value twice sends two existence check requests and the value is matched twice.

Frontend

The agenda table uses the Advanced.Filter.CreatableSelectBox component. Inserted values are split by a separator (comma by default) and sent as a repeated identifiers query parameter.

Component properties:

• manager - manager of the filtered agenda. Required when useCheck is used.
• useCheck - highlights which of the inserted values exist in the agenda. Values that do not exist are marked red.
• maxValues - maximum count of values inserted at once, `200` by default (exported as MAX_VALUES).
• separator - separator of the inserted values, `,` by default.
• help - help content built by Advanced.Filter.getIdentifiersHelp. The attributes parameter is an already localized list of the searched attributes, because the method cannot know the attributes of an entity from another module. Set multipleResults when one value can match more records.

Existence check and the COUNT permission

The existence check calls the /search/count endpoint of the agenda, which is guarded by the *_COUNT authority (ROLE_COUNT, ACCOUNT_COUNT, TREENODE_COUNT and so on). The permission is not implied by READ and has to be granted explicitly.

The check is therefore evaluated by EntityManager#canCount before the request is sent. A user without the permission gets no highlighting at all and no request is sent, while filtering itself keeps working.

Filter agenda

Agenda of registered filter builders is available from menu Setting-> Modules -> Filters.

Supported features:

• All reqistered dynamic filters are shown.
• All filters implemented internally in service (description Internal service implementation (toPredicates). is hard coded for this filters). 
• Currently not active filters can be activated (button is available).
• Filters are grouped by entity, which can be filtered by.