Introduction to Apache Camel

Take any integration project and you have multiple applications talking over multiple transports on multiple platforms. As you can imagine, in large enterprise applications this can get complex very fast. Much of the complexity systems from two issues:

1. dealing with the specifics of applications and transports, and
2. coming up with good solutions to integration problems.

Making your applications speak transports and APIs is relatively easy on its own. But what happens when you want to route a message to another application? You then have to take care of mapping the message to the application plus handle any new concepts related to the application. Add a dozen other applications into the mix and you've got quite a headache on your hands. Ignoring the mechanics of how to connect with multiple transports and APIs, we can focus on the high level design of how applications interact. Fortunately, most solutions to enterprise integration problems have been formalized already. Gregor Hohpe and Bobby Woolfe's book, Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions, boils down years of experience from enterprise architects into a set of sixty five Enterprise Integration Patterns (EIPs). This is great but we still have to hand code all parts of these patterns; these are not packaged solutions, only recommendations.

Apache Camel was created with the intention of addressing these two issues. That's why Unit4 Integration Kit uses Apache Camel as routing and mediation engine. Camel is heavily based on EIPs. Although EIPs describe integration problems and solutions and also provide a common vocabulary, the vocabulary isn’t formalized. Camel tries to close this gap by providing a language to describe the integration solutions. There’s almost a one-to-one relationship between the patterns described in Enterprise Integration Patterns and the Camel DSL.

Apache Camel is a powerful open source integration framework based on known Enterprise Integration Patterns with powerful Bean Integration. Apache Camel uses URIs so that it can easily work directly with any kind of Transport or messaging model such as HTTP, ActiveMQ, JMS, JBI, SCA, MINA or CXF Bus API together with working with pluggable Data Format options. Apache Camel lets you work with the same API regardless which kind of Transport used, so learn the API once and you will be able to interact with all the Components that is provided out-of-the-box.

Camel-related books are also available, in particular the Camel in Action book, presently serving as the Camel bible.

Camel lets you create the Enterprise Integration Patterns to implement routing and mediation rules via Spring based Xml Configuration. Here is a link to the XML schema documents for Xml Configuration with Camel releases. You can refer to http://camel.apache.org/schema/ to see the individual XSDs available.

At the core of the Camel framework is a routing engine, or more precisely a routing engine builder. It allows you to define your own routing rules, decide from which sources to accept messages, and determine how to process and send those messages to other destinations. The routing engine will selectively move a message around, based on the route’s configuration. In Camel’s case, routes are configured with a combination of enterprise integration patterns and a domain-specific language. Camel uses an integration language that allows you to define complex routing rules, akin to business processes. We should also mention what Camel isn’t. Camel isn’t an enterprise service bus (ESB), although some call Camel a lightweight ESB because of its support for routing, transformation, monitoring, orchestration, and so forth. Camel doesn’t have a container or a reliable message bus. For that reason, we prefer to call Camel an integration framework rather than an ESB.

In this section some of the concepts and terminology that are fundamental to Camel are explained. This section is not meant as a complete Camel tutorial, but as a first step in that direction.

Messaging model

In Camel, there are two abstractions for modeling messages, both of which we’ll cover in this section. - org.apache.camel.Message: The fundamental entity containing the data being carried and routed in Camel - org.apache.camel.Exchange: The Camel abstraction for an exchange of messages. This exchange of messages has an “in” message and as a reply, an “out” message

We’ll start by looking at Message to understand how data is modeled and carried in Camel. Then we’ll look at how a “conversation” is modeled in Camel by the Exchange.

Message

Messages are the data itself that is transferred inside a Camel route. A Message can have a body, which is the data itself and headers, which are, like properties on a Exchange, key-value entries that can be used along the processing. One important aspect to keep in mind, however, is that along a Camel route our Messages are changed – when we convert the body with a Type converter, for instance – and when this happens, we lose all our headers. So, Message headers must be seen as ephemeral data, that will not be used through the whole route. For that type of data, it is better to use Exchange properties. The Message body can de made of several types of data, such as binaries, JSON, etc. Messages are the entities used by systems to communicate with each other when using messaging channels. Messages flow in one direction from a sender to a receiver:

Messages are the entities used by systems to communicate with each other when using messaging channels. Messages flow in one direction from a sender to a receiver Messages are uniquely identified with an identifier of type java.lang.String. The identifier’s uniqueness is enforced and guaranteed by the message creator, it’s protocol dependent, and it doesn’t have a guaranteed format. For protocols that don’t define a unique message identification scheme, Camel uses its own UID generator. Messages have a body (a payload), headers, and optional attachments:

• Headers and attachments: Headers are values associated with the message, such as sender identifiers, hints about content encoding, authentication information, and so on. Headers are name-value pairs; the name is a unique, case-insensitive string, and the value is of type java. lang.Object. This means that Camel imposes no constraints on the type of the headers. Headers are stored as a map within the message. A message can also have optional attachments, which are typically used for the web service and email components.
• Body: The body is of type java.lang.Object. That means that a message can store any kind of content. It also means that it’s up to the application designer to make sure that the receiver can understand the content of the message. When the sender and receiver use different body formats, Camel provides a number of mechanisms to transform the data into an acceptable format, and in many cases the conversion happens automatically with type converters, behind the scenes.
• Fault flag: Messages also have a fault flag. Some protocols and specifications, such as WSDL and JBI, distinguish between output and fault messages. They’re both valid responses to invoking an operation, but the latter indicates an unsuccessful outcome. In general, faults aren’t handled by the integration infrastructure. They’re part of the contract between the client and the server and are handled at the application level.

During routing, messages are contained in an exchange.

Exchange

An exchange in Camel is the message’s container during routing. An exchange also provides support for the various types of interactions between systems, also known as message exchange patterns (MEPs). MEPs are used to differentiate between one-way and request-response messaging styles. The Camel exchange holds a pattern property that can be either - InOnly: A one-way message (also known as an Event message). For example, JMS messaging is often one-way messaging. - InOut: A request-response message. For example, HTTP-based transports are often request reply, where a client requests to retrieve a web page, waiting for the reply from the server. Exchanges – or MEPs (Message Exchange Patterns) – are like frames where we transport our data across the integrations on Camel. A Exchange can have 2 messages inside, one representing the input and another one representing the output of a integration. The output message on Camel is optional, since we could have a integration that doesn’t have a response. Also, an Exchange can have properties, represented as key-value entries, that can be used as data that will be used across the whole route.

The Exchange interface provides an abstraction for an exchange of messages, that is, a request message and its corresponding reply or exception message. In Camel terminology, the request, reply and exception messages are called in, out and fault messages. An exchange contains following elements: - Exchange ID: A unique ID that identifies the exchange. Camel will generate a default unique ID, if you don’t explicitly set one. - MEP: A pattern that denotes whether you’re using the InOnly or InOut messaging style. When the pattern is InOnly, the exchange contains an in message. For InOut, an out message also exists that contains the reply message for the caller. - Exception: If an error occurs at any time during routing, an Exception will be set in the exception field. - Properties: Similar to message headers, but they last for the duration of the entire exchange. Properties are used to contain global-level information, whereas message headers are specific to a particular message. Camel itself will add various properties to the exchange during routing. You, as a developer, can store and retrieve properties at any point during the lifetime of an exchange. - In message: This is the input message, which is mandatory. The in message contains the request message. - Out message: This is an optional message that only exists if the MEP is InOut. The out message contains the reply message.

Camel's architecture

The figure below shows the architecture of Apache Camel:

At a high level, Camel is composed of processors, components, and routes. All of these are contained within the CamelContext. The routing engine uses routes as specifications for where messages are routed. Routes are defined using one of Camel’s domain-specific languages (DSLs). Processors are used to transform and manipulate messages during routing and also to implement all the EIP patterns, which have corresponding keywords in the DSL languages. Components are the extension points in Camel for adding connectivity to other systems. To expose these systems to the rest of Camel, components provide an endpoint interface.

Camel concepts

CamelContext

The Camel context is the runtime container where Camel runs it. It initializes type converters, routes, endpoints, EIPs etc. A CamelContext object represents the Camel runtime system, which keeps all the pieces together.

Routes

Routes on Camel are the heart of the processing. It consists of a flow, that start on a endpoint, pass through a stream of processors/convertors and finishes on another endpoint. it is possible to chain routes by calling another route as the final endpoint of a previous route. A route can also use other features, such as EIPs, asynchronous and parallel processing. Camel’s routing engine is what actually moves messages under the hood. This engine isn’t exposed to the developer, but you should be aware that it’s there and that it does all the heavy lifting, ensuring that messages are routed properly.

A route is the step-by-step movement of a Message from an input queue, through arbitrary types of decision making (such as filters and routers) to a destination queue (if any). Routes are obviously a core abstraction for Camel. The simplest way to define a route is as a chain of processors. Each route in Camel has a unique identifier that’s used for logging, debugging, monitoring, and starting and stopping routes. Routes also have exactly one input source for messages, so they’re effectively tied to an input endpoint.

To wire processors and endpoints together to form routes, Camel defines a DSL. To define a route, a DSL is used. The DSLs provide a nice abstraction for Camel users to build applications with. Under the hood, though, a route is actually composed of a graph of processors.

A route first starts with a consumer (think “from” in the DSL) that populates the initial exchange. At each processor step, the out message from the previous step is the in message of the next. In many cases, processors don’t set an out message, so in this case the in message is reused. At the end of a route, the MEP of the exchange determines whether a reply needs to be sent back to the caller of the route. If the MEP is InOnly, no reply will be sent back. If it’s InOut, Camel will take the out message from the last step and return it.

Channels

When Camel executes a route, the controller in which it executes the route is called Channel. A Channel is responsible for chaining the processors execution, passing the Exchange from one to another, alongside monitoring the route execution. It also allow us to implement interceptors to run any logic on some route’s events, such as when a Exchange is going to a specific Endpoint.

Processors

Processors are used to manipulate and mediate messages in between Endpoints. All of the EIPs are defined as Processors or sets of Processors. As of writing, Camel supports over 40 patterns from the EIP book and many other useful Processors. The processor is a core Camel concept that represents a node capable of using, creating, or modifying an incoming exchange. During routing, exchanges flow from one processor to another; as such, you can think of a route as a graph having specialized processors as the nodes, and lines that connect the output of one processor to the input of another. So how do exchanges get in or out of this processor graph? To find out, we’ll need to look at both components and endpoints.

Components

Components are the extension point in Camel to add connectivity to other systems. The core of Camel is very small to keep dependencies low, promote embeddability, etc. and as a result contains only 13 essential components. There are over 80 components outside the core. To expose these systems to the rest of Camel, Components provide an Endpoint interface. By using URIs, you can send or receive messages on Endpoints in a uniform way. To date, there are over 80 components in the Camel ecosystem that range in function from data transports, to DSLs, data formats, and so on. From a programming point of view, components are fairly simple: they’re associated with a name that’s used in a URI, and they act as a factory of endpoints. For example, a FileComponent is referred to by file in a URI, and it creates FileEndpoints. The endpoint is perhaps an even more fundamental concept in Camel.

Endpoints

A Component acts like a factory to instantiate Endpoints for our use. We don’t directly use a Component, we reference instead by defining a Endpoint URI, that makes Camel infer about the Component that it needs to be using in order to create the Endpoint. An endpoint is an abstraction that models the end of a message channel through which a system can send or receive messages.

One of the things that make Camel easy to use is the endpoint URI. By specifying a URI, you can identify the component you want to use and how that component is configured. You can then decide to either send messages to the component configured by this URI, or to consume messages from it.

In Camel, you configure endpoints using URIs, such as file:data/inbox?delay=5000, and you also refer to endpoints this way. At runtime, Camel will look up an endpoint based on the URI notation. Endpoint URIs are divided into three parts: a scheme, a context path, and options:

The scheme denotes which Camel component handles that type of endpoint. In this case, the scheme of file selects the FileComponent. The FileComponent then works as a factory creating the FileEndpoint based on the remaining parts of the URI. The context path data/ inbox tells the FileComponent that the starting folder is data/inbox. The option, delay=5000 indicates that files should be polled at a 5 second interval.