Apache Camel

Unit4 Integration Kit uses Apache Camel as routing and mediation engine. Apache Camel is a powerful open source integration framework based on known Enterprise Integration Patterns with powerful Bean Integration. At the core of the Camel framework is a routing engine, or more precisely a routing engine builder. It allows you to define your own routing rules, decide from which sources to accept messages, and determine how to process and send those messages to other destinations. The routing engine will selectively move a message around, based on the route’s configuration. In Camel’s case, routes are configured with a combination of enterprise integration patterns and a domain-specific language. Camel uses an integration language that allows you to define complex routing rules, akin to business processes. We should also mention what Camel isn’t. Camel isn’t an enterprise service bus (ESB), although some call Camel a lightweight ESB because of its support for routing, transformation, monitoring, orchestration, and so forth. Camel doesn’t have a container or a reliable message bus. For that reason, we prefer to call Camel an integration framework rather than an ESB.

CamelContext

The Camel context is the runtime container where Camel runs it. It initializes type converters, routes, endpoints, EIPs etc. A CamelContext object represents the Camel runtime system, which keeps all the pieces together.

Channels

When Camel executes a route, the controller in which it executes the route is called Channel. A Channel is responsible for chaining the processors execution, passing the Exchange from one to another, alongside monitoring the route execution. It also allow us to implement interceptors to run any logic on some route’s events, such as when a Exchange is going to a specific Endpoint.

Components

Components are the extension point in Camel to add connectivity to other systems. The core of Camel is very small to keep dependencies low, promote embeddability, etc. and as a result contains only 13 essential components. There are over 80 components outside the core. To expose these systems to the rest of Camel, Components provide an Endpoint interface. By using URIs, you can send or receive messages on Endpoints in a uniform way. To date, there are over 80 components in the Camel ecosystem that range in function from data transports, to DSLs, data formats, and so on. From a programming point of view, components are fairly simple: they’re associated with a name that’s used in a URI, and they act as a factory of endpoints. For example, a FileComponent is referred to by file in a URI, and it creates FileEndpoints. The endpoint is perhaps an even more fundamental concept in Camel.

Dead letter channel error handler

The DeadLetterChannel error handler is similar to the default error handler except for the following differences: - The dead letter channel is the only error handler that supports moving failed messages to a dedicated error queue, which is known as the dead letter queue. - Unlike the default error handler, the dead letter channel will, by default, handle exceptions and move the failed messages to the dead letter queue. - The dead letter channel supports using the original input message when a message is moved to the dead letter queue. This pattern is often used with messaging. Instead of allowing a failed message to block new messages from being picked up, the message is moved to a dead letter queue to get it out of the way.

Default error handler

Camel is preconfigured to use the DefaultErrorHandler, which covers most use cases. The default error handler is preconfigured and doesn’t need to be explicitly declared in the route. In every Camel route, there is a Channel that sits between each node in the route graph. The default error handler is configured with these settings: - No redelivery - Exceptions are propagated back to the caller

Deployment

When an Integration Flow is deployed for a tenant, a workflow has to be executed to create the proper tenant context for that Integration Flow. All configuration properties that are marked as tenant-specific in the Integration Flow have to be filled with a value. The configuration properties to be filled in during deployment are determined @deployment time by fetching these properties from Integration Flow Management. Validation of configuration values will be applied. If mapping tables are part of the Integration Flow, the mapping table must be filled conform design. Validation of values in rows will be applied.

Endpoint

A Component acts like a factory to instantiate Endpoints for our use. We don’t directly use a Component, we reference instead by defining a Endpoint URI, that makes Camel infer about the Component that it needs to be using in order to create the Endpoint. An endpoint is an abstraction that models the end of a message channel through which a system can send or receive messages.

Enterprise Integration Patterns

Application Integration enables independently designed applications to work together. Key capabilities of application integration solutions include the reliably moving of messages among endpoints, message validation, mapping, transformation and orchestration – often referred to as Enterprise Integration Patterns.

Exchange

An exchange in Camel is the message’s container during routing.

Flow History

During routing, messages are contained in an exchange and these exchanges flow from one processor to another. So messages follow different steps of an Integration Flow. For the purpose of proof of delivery and error tracking, runs of Integration Flows (transactions) and all their corresponding steps are stored in so-called Flow History of U4IK.

Generic Message Processor (GMP)

A Generic Message Processor (GMP) is a scalable rule based routing & mediation and EIP processing engine.

Integration Flow

An Integration Flow is a step-by-step movement of a message, which originates from an endpoint in the role of a consumer. The message then flows through a processing component, which could be an EIP or a component. The message is finally sent to a target endpoint that’s in the role of a producer. During routing, messages flow from one processor to another; as such, you can think of an Integration Flow as a graph having specialized processors as the nodes, and lines that connect the output of one processor to the input of another.

Message

Messages are the entities used by systems to communicate with each other when using messaging channels. Messages flow in one direction from a sender to a receiver. Messages are the data itself that is transferred inside a Camel route. Messages have a body (a payload), headers, and optional attachments

Processor

Processors are used to manipulate and mediate messages in between Endpoints. All of the EIPs are defined as Processors or sets of Processors. As of writing, Camel supports over 40 patterns from the EIP book and many other useful Processors. The processor is a core Camel concept that represents a node capable of using, creating, or modifying an incoming exchange. During routing, exchanges flow from one processor to another; as such, you can think of a route as a graph having specialized processors as the nodes, and lines that connect the output of one processor to the input of another. So how do exchanges get in or out of this processor graph? To find out, we’ll need to look at both components and endpoints.

Recoverable and irrecoverable errors

When it comes to errors, we can divide them into recoverable and irrecoverable errors. An irrecoverable error is an error that remains an error now matter how many times you try to perform the same action again. A recoverable error, on the other hand, is a temporary error that might not cause a problem on the next attempt. A good example of such an error is a problem with the network connection resulting in a java.io.IOException. On a subsequent attempt, the network issue could be resolved and your application could continue to operate.

Route

Routes on Camel are the heart of the processing. It consists of a flow, that start on a endpoint, pass through a stream of processors/convertors and finishes on another endpoint. it is possible to chain routes by calling another route as the final endpoint of a previous route. A route can also use other features, such as EIPs, asynchronous and parallel processing. Camel’s routing engine is what actually moves messages under the hood. This engine isn’t exposed to the developer, but you should be aware that it’s there and that it does all the heavy lifting, ensuring that messages are routed properly. A route is the step-by-step movement of a Message from an input queue, through arbitrary types of decision making (such as filters and routers) to a destination queue (if any). Routes are obviously a core abstraction for Camel. The simplest way to define a route is as a chain of processors. Each route in Camel has a unique identifier that’s used for logging, debugging, monitoring, and starting and stopping routes. Routes also have exactly one input source for messages, so they’re effectively tied to an input endpoint. To wire processors and endpoints together to form routes, Camel defines a DSL. To define a route, a DSL is used. The DSLs provide a nice abstraction for Camel users to build applications with. Under the hood, though, a route is actually composed of a graph of processors. A route first starts with a consumer (think “from” in the DSL) that populates the initial exchange. At each processor step, the out message from the previous step is the in message of the next. In many cases, processors don’t set an out message, so in this case the in message is reused. At the end of a route, the MEP of the exchange determines whether a reply needs to be sent back to the caller of the route. If the MEP is InOnly, no reply will be sent back. If it’s InOut, Camel will take the out message from the last step and return it.

Secret

Client certificates can be used in Integration Flows to setup secure connections with third-party webservice endpoints. A single client certificate (that is issued by a third party for a certain tenant) can be used in different Integration Flows for that tenant. So a client certificate is Integration Flow agnostic. Therefore, client certificates have to be registered upfront before deploying the Integration Flow for a tenant. That's where Tenant Secret Management comes in. Tenant Secret Management provides a secured Azure key vault per tenant and facilitates uploading client certificates as secret versions with a proper identifier.

Time-to-live (TTL)

A predefined Time-To-Live (TTL) will be applied on Flow History data throughout the system, so the transactional data will be stored with a predefined TTL. The retention time of any data stored in the solution is configurable to make sure that Flow History data @rest is stored no longer than necessary to be in line with GDPR Article 30(1.f).

Trigger

A Trigger is a component which is responsible, based on an external input, for initiating Integration Flows.