Refer to the RFA.NET reference guide, the EventQueue.Dispatch method returns an approximate number of events left in the queue after success. Therefore, this value is not accurate.

From the code, you are using the Event Queue with Statistics so you can use the EventQueue.EventQueueCurrentSize property. This property returns the current number of events stored in the queue if statistics are enabled and always zero (0) if not. It is a proper way to verify the number of events remaining in the statistic queue.

The application is a slow consumer which causes the event queue’s maximum depth is reached. To avoid this issue, the application must make sure that the client.processEvent() method returns as fast as possible. Running long processing tasks inside the client.processEvent() method must be avoided. Otherwise, you can use multiple event queues and each event queue will be dispatched by its own application thread in order to distribute workload among event queues and increase the application’s performance.

Hi @James.Kibirige

Can you please confirm how you are obtaining the Queue Depth - using EventQueueCurrentSize or the return value from the Dispatch() method call?

Also, can you confirm if you are enabling Statistics when you Create the Event Queue by providing a second parameter set to True?

public static EventQueue Create(RFA_String name,
     bool statistics
 )

Also, please confirm the version of RFA.NET you are using.

Also, if you refer to the RFA.Net Developers guide,you will note that it states "Using the existing registerNotificationClient() method is equivalent to enabling high threshold notification with a high threshold value of 1. It might not make sense to use both to call the dispatch method if the application is using both as notifications."

In other words, you will be notified each time the queue depth is 1 or higher. This could explain why you are always seeing a queue size of 1 (assuming you are using EventQueueCurrentSize to determine the queue depth).

If you are going to use the notification client then you should make use of the high/low threshold notifications are appropriate. Otherwise, you can use the return value from the Dispatch() call to get an approximate queue size instead.

@James.Kibirige

Not sure how many RIC you subscribe and how about the update rate of the RICs you are testing?

As Umer said, first of all, please make sure that you already turn on EventQueue statistic when you create the queue by setting the flag to true.

eventQueue = EventQueue.Create(new RFA_String("myEventQueue"),true);

I guess that you are monitoring the EventQueueCurrentSize so the number of events in the queue depending on the frequency of the update with events from RFA internal. Also, it depends on how fast your application can dispatch and process the event. If your application has a long-running task or app logic inside the process event it can block the event processing and the number of events in the queues may keep increasing. Anyway, if you want to test, you can use one of the RFA consumer examples such as StarterConsumer to subscribing to high update RIC such as currency RIC like JPY=, GBP=, and AUD=. I add those RICs to the itemList config in StarterConsumer.cfg and see the EventQueueCurrentSize more than 1 from the example.

Hi @moragodkrit, @Umer Nalla,

Thank you for the quick response to my original question.

The application is already set up to report statistics:

• _context.EventQueue = EventQueue.Create(new RFA_String("rfaQueue"), true);

Where can I download the RFA.Net Developers guide?

In regards to the dispatch method do you thus suggest logging within the main control loop for the application where the call to Dispatch() is used. I don't want to prematurely Dispatch the EventQueue while the application runs.

Yes we are trying to ascertain why the EventQueue fills up causing the application to restart. I am not sure of the rate that update events arrive from RFA Internal, and would like to know how to measure this effectively. The main control loop manages the rate of EventQueue dispatch and we can use this to ascertain the rate that messages are processed on the queue.

Thanks for the advice I will apply this in the application as part of the testing I am currently doing.

Thanks

James Kibirige

Hi guys,

I am still having a problem with the Dispatch method the application is using the following version of the method:

• _context.EventQueue.Dispatch(5000); --with a parameter timeOut of 5000

With this current set up when the application runs it logs the EventQueue size as 0 or -1 intermittently which isn't what I was expecting from the application.

Thanks

James Kibirige

Hi @moragodkrit, @Umer-Nalla ,

I have been monitoring the logs after I made adjustments to the application to log the Current Queue size. The application has generated the following alert:

• Queue threshold of '4000' breached

Unfortunately the logging I have to get the Current Queue Size is as follows:

• DEBUG : NewsAnalytics.Service.RdmRunner: Current Queue Size: 0

So even when I capture the return value of the Dispatch() method the Queue size is always 0:

• var dispatchReturn = _context.EventQueue.Dispatch(5000);

This is a problem because it means we cannot accurately measure or log the Current event queue depth, we need to understand why the Event queue is filling up. Is their anything that I am missing?

Thanks

James Kibirige

Hi @James.Kibirige

The reason your queue is filling up is because your application is unable to process all the events in a timely manner. It is taking too long to process each event it receives and therefore, you are building up a backlog in the event queue - a scenario described as "Slow consumer".

See a previous post on suggestions on how to deal with slow consumer scenarios.

You can also refer to this article - which although is based around RF Java - still provides some useful context.

As to why the dispatch method is not returning the event queue size correctly, you can raise a support ticket and investigate this issue in more detail offline.

Hi @James.Kibirige

I just tried a quick test here with RFA.Net v8.1.0.L1 example StarterConsumer subscribing to several highly traded RICs.

I added the following line of code in the Run() method

int dispatchReturn = eventQueue.Dispatch(10);
AppUtil.Log(AppUtil.LEVEL.INFO,string.Format("Dispatch : {0}\r\n ", dispatchReturn));

and I see positive values being output:

2020/01/13 12:26:36.134 INFO    Dispatch : 0
2020/01/13 12:26:36.160 INFO    Dispatch : 58
2020/01/13 12:26:36.181 INFO    Dispatch : 57
and so on.

Obviously the return value fluctuates between positive and -1, depending on how many events are in the queue.

Hi @James.Kibirige

With regards the support ticket link, this may be controlled by your employer in terms of who can raise a ticket - I have reached out to your Refinitiv Account manager to try and identify who is the authorised user in your organisation. You may also be able to determine this by speaking to your internal Market Data team.

With regards your other points, I am somewhat confused. It is not up to the API to restart the application / process / service - that would be an application level implementation. All I can assume is that the restart is not being done as expected - perhaps the application code is not performing a proper cleanup and shutdown of the RFA API. Please refer to the Cleanup() method of the StarterConsumer example that comes with the RFA SDK or the Tutorials example for an idea of what RFA objects need to be deactivated, destroyed, released etc. for a clean RFA shutdown.

If, however, you are referring to the Event Queue reset as described in the developer guide, you will note this only occurs if you have set a EventQueueMaxDepth greater than 0. As explained in the documentation, this is different to the EventQueueHighThreshold and EventQueueLowThreshold values. Obviously, if you set do set EventQueueMaxDepth, then all the events in the queue will be discarded - as described in the developer guide. This may not be a desirable outcome - If an application needs to retain all events, the application should not enable the event queue maximum depth functionality.

For further details please refer to section Event Queue with Statistics of the RFA developer guide (and the EventQueue section of RFA Reference Manual). It provides further details on the whole mechanism including actions required on the part of the developer to reset the event queue when ready - otherwise it will not restart queuing events again.

Hi @James.Kibirige

Based on my past experience, the most likely explanation are that the queue is gradually building over the week. This would suggest that your application is slightly too slow to process all the events it receives in a timely manner.

Just as an example, say it was receiving on average 1000 events an hour, perhaps it can only process around 970 - so the event queue is gradually increasing in size at a rate of 700 events per day on average.This would mean that over 7 days a backlog of approx 5000 events has build up in the queue.

As the message 'Max queue depth of '5000' breached. Application will be restarted.' is an application generated message, it is the applications responsibility to restart or reset RFA etc.

Did you read the article and post I linked previously, they talk about things you can try to improve the rate at which your application processes the events it receives.

I did reach out to your account manager but have not heard back in terms of your organisation's approved RDC contact. I will try an alternative source.

Regards,
Umer