Few questions for you:

Are you using one of the API examples to subscribe.

Is it tick-by-tick real time data

Do you have any chain RIC's in the subscription?

Can you share the instrument list?

I believe my code started from one of the examples. It is nothing more complicated than

consumer.registerClient(EmaFactory.createReqMsg().serviceName("ELEKTRON_DD").payload(batch), appClient);

where batch is an OmmArray of RICs. RICs are a collection of equity tickers on ".NB" exchange. It is a tick-by-tick real time data subscription but I do not even even get any update msgs. I think I get 100 refresh msgs and them memory goes through the roof and my app sort of hangs. I do not think I have any chain RICs (I do not even know what they are). These are all single name equity tickers.

Thanks

I'll offer a different explanation.

I'm not myself using the EMA api but I think it works by having a background thread that reads as fast as it can from the socket and puts that into an in-memory queue. Your application logic then essentially dispatches from that queue. If you are not processing messages fast enough then you'll see memory usage go through the roof.

I would say: Do the math. How fast are you processing messages? And what is the input rate of messages?. Some financial instruments update like 100 times per second so if you subscribe to 400 of those bastards your processing logic should be able to process insanely fast (like no more than 0.025 msecs per message). If not you'll fall behind and memory usage will grow and grow fast. Ok, my example is of course kinda a worst case scenario, but you may be underestimating how slow your application logic really is compared to the update rate of the instruments you're subscribed to. Don't ask for more than you can chew.

All in all when you say "I think I get 100 refresh msgs and them memory goes through the roof", the more correct phrasing would be that you have processed 100 msgs. In the background your application will have received a lot more.

Your local TREP may offer feeds that are conflated so that you won't get overwhelmed with messages. Ask your admin.

If you really want to process an instrument set which updates at very high rates then ETA is a better choice than EMA because the former offers more choice on message dispatching, threading and what not.

(all of the above is merely a guess at what is going on :-))

Hello @Axegine

Could you please give us a Java heap dump file when the problem occurred and a list of 400 RICs?

You can use the following tools that available in the <JDK_HOME>/bin folder to capture the Java heap dump

1. jvisualvm: This is a graphical interface to capture and analyze the Java memory heap dump. Please see more detail about this jvisualvm tool in http://docs.oracle.com/javase/8/docs/technotes/guides/visualvm/index.html
2. jmap: This is a command line tool to capture the heap dump of inputted Java process id. Please see more detail about this jmap tool in https://docs.oracle.com/javase/8/docs/technotes/tools/unix/jmap.html

jmap command format:

$>jmap
–dump:format=b,file=<capture file> <application process id>

Example jmap command:

• $> jmap
  -dump:format=b,file=heap.bin 8956

Hello @Axegine

To let us investigate this issue in detail, could you please give me the heap dump file when the problem occurs?