question

Upvotes
Accepted
15 2 0 3

How to use OmmConsumer.reigster for multithreading?

Hi

I'm trying to expedite multithreading on EMA based on the example of series400.example410__MarketPrice__HorizontalScaling.

I'm still wondering how I specify the thread for getting callback from EMA.

I tried to run `OmmConsumer.register` on separated thread from original one, but all request streams is receiving on the same thread.

Any help will be very appreciated.

Regards

elektronrefinitiv-realtimeelektron-sdkrrtema-apielektron-message-api
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
Accepted
20.3k 73 10 20

Hi @cho.seungjin

Are you executing your multiple OmmConsumer instances each on a seperate thread as illustrated in the 410 example?

And are you then calling registerclient using the different OmmConsumer instances e.g.

consumer1.openItem("VOD.L", "ELEKTRON_DD"); consumer2.openItem("BT.L", "ELEKTRON_DD"); 

If I run the 410 example with some extra code to dump the Thread name in the OnRefresh and OnUpdate callbacks I see the following:

Thread:pool-2-thread-1 Item Name: VOD.L Service Name: ELEKTRON_DD Item State: Open / Ok / None / 'All is well' 
Fid: 1 Name = PROD_PERM DataType: UInt Value: 5625 
..... 
Thread:pool-5-thread-1 Item Name: BT.L Service Name: ELEKTRON_DD Item State: Open / Ok / None / 'All is well' 
Fid: 1 Name = PROD_PERM DataType: UInt Value: 5625 
..... 
Thread:pool-2-thread-1 Item Name: VOD.L Service Name: ELEKTRON_DD 
Fid: 1025 Name = QUOTIM DataType: Time Value: 10:12:44:0 
Fid: 1071 Name = COLID_2 DataType: Enum Value: 15 

Thread:pool-5-thread-1 Item Name: BT.L Service Name: ELEKTRON_DD 
Fid: 345 Name = BID_TONE DataType: Rmtes Value: blank Fid: 
345 Name = BID_TONE DataType: Rmtes Value: blank 

As you can see the BT.L stream and VOD.L stream are on different threads.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Hi @Umer Nalla

It probably didn't work because our code used Consumer instance as a Singletone. After changed it to create every time, it works. Thank you for your tips.

Upvotes
20.3k 73 10 20

Hi @cho.seungjin

Just be aware that each OmmConsumer instance represents a 'session' - along with the overhead associated with a session - such as connect and login to server, dictionary download etc. Therefore, only create additional instances if you really need them. Also, you should do not create and discard OmmConsumer instances during the runtime of your application. Ideally create them at the startup / initialisation phase, continue to use those instances during the runtime and only unintialise during the shutdown phase..

I came across a scenario where a developer was creating and unintialising OmmConsumer instances frequently during the runtime of his application and this is not a recommended approach.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
15 2 0 3

Hi, @Umer Nalla

Ok, I'll keep the number of OmmConsumer not bigger than total core size which is about 8.

Also our application is holding OmmConsumer instance throughout the application lifecycle, but It looks ok to hold just handle id rather than whole OmmConsumer instance, am i right?

And your comment is really helpful but it seems something that I should have discover from official document or anything similar.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
20.3k 73 10 20

Hi @cho.seungjin

Ideally you should only use the number of OmmConsumer instances that you need - based on testing with your watchlist. For example a recent client developer I helped used 2 instances for his watchlist of approx 40,000 instruments - but this may differ for you because your RICs may be more volatile or you may do more processing in your callback handlers. Using more instances than you need could have a detrimental impact on the overall performance of your application.

I am not sure what you mean about holding the handle id? What I meant about keeping OmmConsumer instance for the lifecycle of your program is not to make the mistake of some developers who keep creating and uninitializing the OmmConsumer instances each time they want to request some data...

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
15 2 0 3

This makes me confusing about my understanding so far.

If 2 instances can deal with more that 40,000 instruments, I don't see any benefit from multithreading at least in my case. Because, our consumer application is just passing the message to backend system, so there are no such heavy operation on callback handlers. And even the consumer didn't do anything but log the message, the EMS sever keep throwing the `disconnection` message for server side overflow.

And handle id is returned by `consumer.registerClient`, so I thought that, without saving the reference of OmmConsumer, someone also can unregister when they want.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
20.3k 73 10 20

Hi @cho.seungjin

As I mentioned in my post above, the number of OmmConsumer instances should be arrived at by testing. If you find that using more than one OmmConsumer stops the disconnects then this is appropriate for your application - what works for one developer's application may not apply for another application. There are many factors that can affect this scenario.

In your other post i did say 'it could be that due to the high volatility nature of the instrument, your application may be struggling to cope with the update rate for your 12,000 items - causing the disconnect from the server'.This is the most common cause for server side overflow reported by developers.

If testing with more than one OmmConsumer still results in disconnect, I recommend you raise a Premium Support ticket so that they can investigate in more detail offline. There may some other issue with your code etc causing the disconnects.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
15 2 0 3

Hi @Umer Nalla

Thank you for prompt response

So, Isn’t there any easy way to get best fit for TSE Market? Just keep failing until find marginal value that works possibly at that moment. I thought there might be some kind of best practice or suggested architecture.

the link you gave didn’t work for me but let me just contact local sales person

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
20.3k 73 10 20

Hi @cho.seungjin

There are too many factors which determine how an application consuming realtime data will perform for example:

  • watchlist size,
  • average update rate,
  • peak update rates,
  • size of updates, #what exactly you do with the updates when you receive them,
  • how long your code spends processing the update before returning control back to the API from the callback,
  • network capacity /network card
  • PC performance
  • and so on...

Developers generally test and tweak their application over a period of time to arrive at a workable fit - before deploying in production - with some margin to allow for exceptional scenarios.

You can also test your application by generating dummy data using the IProvider examples - you can modify the code to better reflect your typical data. Or you can also use the sink_driven_src example to generate data at variable rates, message sizes etc. Sink_driven_src also allows you to replay data captured using rmdstestclient as described in this article. These tools are used by developers to simulate market conditions and test their application.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Click below to post an Idea Post Idea