The graph shows how the number of "hanged" API calls, counted in the control, depends on call execution time. Points on the graphs represent maximum numbers of user requests "hanged" in between risk control cycles, i.e., maximum of riskCounterValue in the series of tests for given call execution time. Remember that some user requests are denied when the Decision Rule is enforced and the number of "hanged" resource threads does not increase.

The horizontal red line in Figure 2 marks the amount of allowed maximum concurrent users in the Portal. The purpose of the control is to keep maximum riskCounterValue strictly below the red line. The closer the points in the graphs are to the red line, the more probability that the riskThreshold may be reached or exceeded.

As we can see, the behavior of the control is not obvious. For some values of call execution time (from 3250 ms to 1500 ms), the control yields user requests and number of "hanged" resource threads gets close to and exceeds the allowed maximum of concurrent users in the Portal. This is the interval during which the control is ineffective in the given conditions. At the same time, there are two intervals - from 100 ms to 1000 ms and from 3500 ms to 4000 ms - where the control is effective: the Decision Rule with the particular riskThreshold reliably protects Portal from "hanged" API calls and leaves enough concurrent request threads to serve other user requests.

The graph also shows that smaller riskThreshold provides better protection. However, if riskThreshold for a resource is set too low, the resource may become unavailable in most of user sessions just due to the slight fluctuations in the network latency. This is another subject for balance and tuning.

Conclusion
The proposed solution of run-time control of "hanged" resource calls allows a Portal to isolate the resources that are in trouble and to continue its work with remaining resources making a minimal impact on the performances. The solution effectiveness depends on several tuning parameters: the ratio between request frequency and frequency of the risk control cycles, the value of the risk threshold, and the expected call execution time.

Tuning is not a trivial task in this case - it requires intensive testing. Moreover, the numbers given in the article are specific to my test Portal and you should expect other values in the tests on your Portal though you find the same dependencies. On the other hand, if certain performance degradation is acceptable, it is recommended to perform risk control cycles in the scope of every API call that significantly simplifies tuning of the solution parameters.

References

• WebLogic RMI Features and Guidelines: Guidelines on Using the RMI Timeout http://e-docs.bea.com/wls/docs81/rmi/rmi_api.html
• Nyberg, G., Patrick, R., Bauerschmidt, P., McDaniel, J., and Mukherjee, R. "Mastering BEA WebLogic Server: Best Practices for Building and Deploying J2EE Applications." Wiley E-Book published March 2004. ISBN: 0-471-48090-8.