Archive for category TIBCO

BusinessEvents 4: using Scorecards in a clustered environment

Scorecards have a special purpose in TIBCO BE. There are often used for static (as in Java-style static) values which should be globally accessible through the whole engine. As Java statics, TIBCO describes its main purpose in instance dependent Variables which are only valid in the context of one Processing Unit in an Inference engine.
Here a little citation of the TIBCO documentation on this topic (p.144 – Understanding and Working With Scorecards):

A scorecard is a special type of concept. A scorecard serves as a set of static
variables that is available throughout the project.

It is more accurate to say there is one instance of a scorecard per inference agent.
Each inference agent in an application has its own instance of the score card.
Scorecards are not shared between agents.

So you can use Scorecards without any Issues as long as you consider the per Instance rule. Now I have a project where Timeouts are calculated an stored in Scorecards. This solution works perfectly in single instance environment because of the special behavior described above. Now there comes a new Requirement into the project. The Agents should be clustered over 2 Instances to ensure high availability. This really ruins the simplicity of the scorecard approach.

Digging a little deeper into the documentation I came to the conclusion that all the scorecards had to be stored in the cache-server. Knowing this, it came to me mind that the only thing I must accomplish would be, that all inference agents use the same instance of the scorecard.

To achieve the single Instance per Agent approach, TIBCO uses Instance keys for every Processing Unit. So if you could change this instance key, all agents had to us the same instance of the scorecards and the timeout-solution would still work. Searching the documentation the answer was also found really quickly on page 144 (Understanding and Working With Scorecards).
On that page TIBCO described how to set this instance key via CDD.

Any agent that uses scorecards, and also uses Cache Manager, must be assigned a
unique key so that the correct scorecard can be retrieved from the cache. The key
is set in the Processing Unit tab of the CDD.

I also tried this approach on a little test project and it worked right away.

What this means is, you can change the behavior of Scorecards depending on your needs. If you need a per Instance Variable you just have to set different Instance keys for every Agent. On the other hand if you want a globally shared Variable you can achieve this by setting the same value to all instances.

, ,

No Comments

handling a few thousand simultaneous connections in TIBCO BusinessWorks

Scaling with TIBCO BusinessWorks can sometimes be a bit tricky. Recently I began testing some scenarios how to scale a Webservice a bit larger. The first source of information was of course the official documentation and to look at the proposed best practice values for such an engine.
To start small, I tried a HTTP Receiver with a 32bit JVM runtime. I set the heap to the maximum amount possible (something about 1.7gig) and tried how many connections I could handle with that. After a few hundred (300-400) the engine alsways ran into an Out-Of-Memory Exception. From that point the engine was often not recoverable and had to be killed.
After that I tried my luck with an 64bit JVM. Theoretically, with more RAM more connections should be possible, so lets go for it.
I increased the heap size to about 4gig. With that value, the engine actually consumed something about 6gig of memory (I only had 8gig on my test machine). Running the same test as before the connection count increased linearly. That was something I didn’t expect. First of I expeceted the Memory consumption should lower on the amount of connections (because of the more reusable objects) and an increase in CPU load because of the more and more complicated handling of the larger heap on the JVM end.
Despite that I came close to handle about one thousand connections. This seems pretty good but was not enough for what I had in mind. The only possibility I saw at that point was to increase the Memory further and further to get more connections running. A second concern which came to my mind was the thread handling. In a default Tomcat installation ever connection gets its own thread. This does consume a lot of memory but also increases the thread count of the server engine dramatically.
With that in mind I remembered something I read about Tomcat 6 a while ago. For the purpose of handling a lot of simultaneous and enduring Javascript requests Tomcat introduced a new kind of connector engine which uses the Java NIO Framework. To explain this a little, Sun introduced this framework in Java 1.4 to handle IO over a single thread mechanism and use mostly OS provided functions for memory allocation and interaction. So in theory this could be the ideal framework for handling a lot of network IO. Tomcat introduced this feature with the c10k problem specifically in mind. So I began searching around how I could get a similar behvior out of BusinessWorks.
What I didn’t know at that point was, that TIBCO already introduced this feature into BusinessWorks with version 5.7. You can switch the HTTP connector engine with some parameter in the HTTP Connection resource.

HTTP Connection - engine selector

HTTP Connection - engine selector

So I changed the connector engine and restartet the test. This time I started small. I set the heap to 512MB and limited the maxProcessor to 500. What I then saw was unexpected. The engine filled up right to the ten thousand requests I send. There occured no Out-Of-Memory at all. What was also interesting was, that the engine held 10k connections despite the maxprocessors was set to 500.
So to conclude the result, the new connector is quite impressive when you need to handle a lot of simultanous connection and have not a lot of memory to spare. On the other hand, when you use it, you loose some of the TIBCO integrated features to limit your load. Further to that, the TIBCO documentation states that due to the single thread arcitecture you increase latency. So as always there is a tradeoff.

One final sidenote. I had some issues with the BusinessWorks 5.7.1 engine so I upgraded it to 5.7.2. Than it ran without a glitch.

, , , ,

3 Comments

TIBCO Designer Panel too small

Recently I ran into some rather trivial problem which isn’t really addressed by the TIBCO Designer. I had a process which wouldn’t fit into the Design Panel. There was just not enough space on the canvas to fit in the actual flow.
After asking around I came to the conclusion that every designer (from different colleges I work with) had a different resolution for the Design canvas. Nobody knew any kind of property where you can set this resolution, so I began searching around.
The designer uses basically 2 Folder for its configs. One is the installation folder with the designer.tra (already explored this one). The other one is the .TIBCO folder in your user home directory. In that folder there exists a file with the name “Designer5.prefs”.
The content of this file consists mainly of position data of the various dialogs. Further to that it includes all the values which you can set through the designer preferences window. Back to the actual topic, I found the following two values:

graph.height.pref=712
graph.width.pref=1515

These values represent the default size which the Designer allocated for its canvas. As you can see the default is pretty small on this one. I also found installations where both values where ten times larger then this. So far I found no performance penalty to this.
The only pattern I found is that newer installation had smaller values as default. Where this value comes from and how it is determined stays unclear to me.

One other thing I found during my research. If the panel is to small for the current process you can drag one activity right next to the border and then start to move it via keyboard (‘Shift + Cursor’). By doing so you drag the icon out of the canvas, but the position will be updated internally. After that you just need to refresh the process view and voila you have expanded the canvas size. But this is just a quick and dirty work-around.

, ,

No Comments