TIBCO StreamBase Container Connection Vs Module Reference

Background

It was observed, as part of the deployment of a StreamBase application, that with a greater number of module references, the application takes longer to load and more heap space. This results in inefficient use of heap space and longer start-up time. On further analysis, it was found that each module reference in a StreamBase application is treated as a new class and gets compiled internally, hence more module references result in great compile time. This process cascades and eventually, the startup time also increases.

Procedure

Steps followed for optimal use of heap space and better start-up time

• An analysis was done to find the TIBCO StreamBase processes (sbapp), that are being invoked multiple times. All framework processes that are singleton processes that only take input and execute asynchronously may fit the scenario.
• This issue can be alleviated by using container connections for the sbapp, which does not send back any response
• Define the common process’ endpoint as its container. 
• Replace all module references to the module reference with an output stream connected to this container’s top-level application input stream
• In the Output Stream properties, the Avanced tab, “Container connection” should be set to the name of the container that is newly defined. This is the pattern for taking any leaf module that does not return a tuple to the calling application and making it a singleton, called from every point of use

Set the singleton sb app in engine config file as below:

log = { moduleName = “com.prowess.sb.fw.log” moduleParameterSettings = { parameters = { } } containerParameterSettings = { } }

A TIBCO StreamBase application process (sbapp) that has a module reference to a common function is shown in the screenshot below

The referred module from the above screenshot points to a sbapp that is a singleton process, as seen below.

The singleton process can be configured as a container and can be called as shown in the screenshot below.

Conclusion

After the above changes were made to a large process containing many module-references, and later replaced with container connections, there was a sizable improvement in the startup time and heap space usage. It helped that the newly created container was readily available for the processes during start-up rather than instantiating and compiling for each module reference invocation.

It is important to plan and define the containers in the StreamBase configuration file wherever appropriate so that the application runs with the best usage of the resources. Having said that, it is also important to understand the use-case for module references. A module-reference may be needed if your use-case is dependent on the response of the invoked process. It is also effective to create, compile and keep the class ready for execution, if it is not invoked too many times. If all invocations are modified for container connections, it may be an overkill to manage too many containers and unnecessary.