question

Upvotes
Accepted
skajiwara avatar image
3 ●0 ●2 ●3

EMA Java: "channel out of buffers errorId=-4" error with over 3,000 RICs

Hi,

I'm using EMA Java from Scala 2.11.

I `registerClient()` 3,558 RICs at once by the following code:

val client = new MyClient()  // sub-class of OmmConsumerClient

val rics = List("/1418.T", ...)  // Ordinary stocks in Tokyo Stock Exchange, whose length is 3558

rics.map { ric =>
  consumer.registerClient(reqMsg.serviceName("ELEKTRON_DD").name(ric), client)}

Although `MyClient#on*Msg()` have almost empty implementation, my program uses a lot of memory (memory usage of the process is around 6GB on 16GB-memory OSX).

It leads to the following runtime error:

19:17:15.024 [pool-2-thread-1] ERROR c.t.e.a.OmmConsumerImpl - loggerMsg
    ClientName: EmaConsumer_1
    Severity: Error
    Text:    Call to rsslReactorDispatchLoop() failed. Internal sysError='0' Error text='channel out of buffers errorId=-4 errorText=channel out of buffers'. 
loggerMsgEnd

My questions are:

  • Is it usual to consume such a large memory with 3,558 RICs registration?
    • If it's unusual, I will use some heap profiler (no idea currently) to detect potential memory leak.
    • If it's usual, could you tell me about how much memory would be required to get 3,558 RICs' feed? I appreciate if I can hear some reasonable tuning points.

Thank you.

elektronrefinitiv-realtimeelektron-sdkrrtema-apielektron-message-apimemory
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Nipat Kunvutipongsak avatar image Nipat Kunvutipongsak

@skajiwara It seems like the problem didn't occur when the application tried to send a batch request with a lot of items, but the problem happened after that 'request' state (when the application received data), am I correct?

I have replicated this problem by putting 4000 items into a request message with EMA Java version emaj3.0.3.L1.all (from Elektron-SDK1.0.7java). However, it encountered the message below only.

"Failed to rsslElementEntry.encode() while encoding rssl elementlist. Reason='INVALID_DATA'"

Could you please confirm whether my understanding is correct or not?

Nipat Kunvutipongsak avatar image Nipat Kunvutipongsak Nipat Kunvutipongsak

Also, can I know the version of EMA/Elektron-SDK the application is using? Perhaps, I can't reproduce the problem because of different API version. Thanks.

1 Answer

Upvotes
Accepted
Jirapongse avatar image
38.1k ●71 ●35 ●53
@skajiwara

It could be the size of GuaranteedOutputBuffers used by the underlying API (ETA).

You can try to increase its value via the configuration. For example:

<ChannelGroup>
	<ChannelList>
		<Channel>
			<Name value="Channel_1"/>	
			...
			<GuaranteedOutputBuffers value="5000"/>
			<Host value="hostname"/>
			<Port value="14002"/>
				...
		</Channel>

config.png (26.9 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Click below to post an Idea Post Idea