Integration through Messaging Systems – Part 1

Lets discuss about Integration through messaging systems in terms of Transmission protocols.

Transmission of data from one system to another is the most elemental task for any kind of  Systems Integration. If you cannot transmit data from one point to another, you cannot integrate. It is therefore essential to understand the building blocks of data.

Integration through messaging systems

Part1: Transmission Protocols

Unlike OSI model, TCP/IP model was implemented first and later built as a model. In fact, the first implementation of TCP/IP model did not go by segregation of functionality by layers. In its current form, TCP/IP conceptual model contains four layers, Application, Transport, Internet, Network Access Layer. The illustration below shows the OSI model alongside the four-layer TCP/IP model.

The highest layer, Application layer, is the one that most users see. It could be a protocol visible on a browser, or an abstraction of the transport that an integration system is addressing without worrying about its implementation details. In a more popular TCP/IP four-layer model, the Application Layer is inclusive of the Presentation and Session layers. The Presentation layer concerns itself with the translation of data from Application Layer to the network format and vice-versa, while the session layer handles connections – starting the connection, maintaining it and terminating it at the end of the session.

Further down, there is the Transport Layer. This layer deals with the coordination of data transfer between end systems. It is the layer where data transmission reliability, integrity, congestion control and multiplexing are managed. It controls the pace of producers and consumers of data.

Next up is Network layer, also called Internet Layer, that packs data into IP Datagrams. The datagrams contain enough information for any router to forward these packets to the next hop on its journey to the destination.

The next two layers, Data Link layer and Physical layer are about how the data is sent from node to node. Suffice to say, this layer abstracts the physical representation of the system’s connection with the network including cable type, radio frequency links (Similar to 802.11 wireless systems), etc.

Now, going back to why we want to understand these layers in the first place – When you talk about a TCP or a UDP protocol, did you know that you are talking about a protocol that is implemented at the Transport Layer? Also, that when you connect through a TCP connection, it is taking care of the functionality of the layers below it – Network, Data Link and Physical Layers. In other words, if you are implementing a TCP connection in your code, your code would remain the same regardless of how your server is connected to the network – either through a wireless router, or an ethernet cable, or some other source of network connection. TCP and UDP are the two most popular protocols of the Transport Layer.

At the Application Layer, there are numerous protocols available for implementation. There are protocols that are competing, and those that are complementing each other. Some of these are ubiquitous, almost irreplaceable. In the context of Systems Integration, some commonly used ones are TELNET, FTP, SMTP, SNMP, AMQP, HTTP, MQTT.

Not going into specifics of each of these protocols, it can be summed-up that protocols implemented at the Application Layer (with the abstraction of Transport Layer built-in) can exercise certain flexibility to provide add-on features for the developer community. Architecture patterns like Hub-and-Spoke, Bus architecture were implemented at the Application Layer to render robustness to the system at the Application Layer.

Most application-layer protocols operate with a broker between the sender and the receiver. The introduction of the broker (which could be a server in some architectures), simplifies a lot of work being done in the Application Layer. The broker could take messages on behalf of the consumer while it is down, and deliver it when the consumer is back online, making it a durable subscription; the broker could handle routing to multiple clients, thereby providing the abstraction for publish/subscribe model; the broker could balance the load within a cluster of receivers; the broker could also provide transaction management among multiple receivers; the broker could provide connection pooling; the list is really endless. The concept of adding a broker, as simple as it might seem, provides a powerful way of abstracting complexity away from the developer. By adding these features within the messaging system, developer can rest easy and focus more on their process flow rather than coding for message handling overhead. Developers can focus more on their processes than working toward implementation of wrappers around their transport layer connections.

In our next post, we’ll take up a few protocols and see how their implementation helps specific use-cases. We’ll delve deeper into some of them, and the problems they solve. So, stay tuned!

At Prowess Software, we are constantly tasked with finding the right integration approach for our clients. We help assess their current use of messaging protocols, and we help optimize them in their current middleware implementation.

Click for: Application Layer Protocols