question

Upvotes
Accepted
aharonya avatar image
9 ●2 ●2 ●6

RFA.NET - Seperate threads - event dispatching and event processing

Hi,

As part of using RFA.Net in order to get market price information, we implemented:

QueueDispatcher which runs in the background in thread and do 

eventQueue.Dispatch();

In the other hand we created EventClient implemented Client interface, and use it in the interest registration (registrations are in stream mode).

My question: As it seems in debug, the dispatching (i.e. QueueDispatcher) and the event processing made by the same thread. Due to large number of Rics I'm a little bit worry about that and I'd like to know if - is there a way to seperate the Event processing thread from the dispatching thread?

treprfarfa-apistreaming-pricesmulti-threading
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

1 Answer

Upvotes
Accepted
moragodkrit.chumsri_1 avatar image
7.6k ●15 ●6 ●9

@aharonya

Basically, if an Event becomes available following a call to Dispatch(), RFA invokes the Client callback method specified when opening the Event Stream. Thus the event processing made by the same thread.

If your concern is the scenario that it has a large number of Rics or high update rate, you might need to clone the event and then put it in your own queue and process the Event in other thread. Basically, event objects are only valid within the function context of the Event Handler function call (i.e., ProcessEvent()). Once the application returns from ProcessEvent() the Event object is no longer valid. If the application wishes to asynchronously process the Event it will need to make its own copy for later processing by using the Clone() method.

An application may create a copy of an Event by using the Clone() method while in the Client callback method. Unlike the Event provided in the Client callback method, the application is responsible for cleaning up a cloned Event via the Event’s Destroy() method. Note that a Clone() call makes a heap allocation and may affect performance.

Also using Horizontal Scaling feature may help in this case. When Horizontal Scaling enables RFA applications to use multiple instances of the Consumer on multi-core processors to dynamically scale the number of Session instances that applications use. Since each instance of a horizontally-scaled adapter processes messages on its own connection (independently of other adapter instances), applications can utilize this feature to dramatically increase Response Message and Request Message throughput. You can find more details about Horizontal Scaling from RFA Developer Guide section Horizontal Scaling.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

aharonya avatar image aharonya

@moragodkrit

Thanks for your reponse. One more question regarding it: If we'll stay with the current implementation (means one thread for dispatching and processing). In case of the heavy event processing "blocks" the eventQueue.Dispatch(); what will happen with the awaits events? There will be a bottle-nack? Or there is some sophisticated mechanism which care of sending just the freshests events? For our business needs, we don't need every update. It will be fair enough to get some of them per minute/second.

Thanks!

moragodkrit.chumsri_1 avatar image moragodkrit.chumsri_1 aharonya

Yes, there will be a bottle-neck. If you have a time-consuming task in the processEvent callback, the first issue is memory growth issue because there are a lot of events left in the event queue. The second issue is that the data the application layer receives is delayed due to the bottleneck.

Instead of sending streaming request, you may need to send a snapshot request or a snapshot request+view(to selected only fields you interest) every minute or sec.

Click below to post an Idea Post Idea