question

Upvotes
Accepted
Steven McCoy avatar image
1.2k ●23 ●28 ●42

IndexOutOfBoundsException publishing a large RFA SymbolList

The following exception appears in RFA/Java when publishing a symbol list with ~3,000 instruments as the constituents of NYSE:

2016-06-22 13:04:19,034 ERROR [main] Shinboru(Shinboru.java:466): catching
java.lang.IndexOutOfBoundsException
        at com.reuters.io.ArrayWriter.checkRemaining(Unknown Source)
        at com.reuters.io.DataArrayWriterWireFormatV1.writeByte(Unknown Source)
        at com.reuters.io.DataArrayWriter.writeByte(Unknown Source)
        at com.reuters.rfa.internal.rwf.RwfEncoder.encodeFieldListInit(Unknown Source)
        at Shinboru.OnRequest(Shinboru.java:510)
        at Provider.republish(Provider.java:749)
        at Provider.OnDictionaryResponse(Provider.java:735)
        at Provider.OnRespMsg(Provider.java:588)
        at Provider.OnOMMItemEvent(Provider.java:565)
        at Provider.processEvent(Provider.java:534)
        at com.reuters.rfa.internal.common.ResponseMessage.dispatchToClient(Unknown Source)
        at com.reuters.rfa.internal.common.ResponseMessage.dispatch(Unknown Source)
        at com.reuters.rfa.internal.common.EventQueueImpl.localDispatch(Unknown Source)
        at com.reuters.rfa.internal.common.EventQueueImpl.dispatch(Unknown Source)
        at Shinboru.mainloop(Shinboru.java:460)
        at Shinboru.run(Shinboru.java:432)
        at Shinboru.main(Shinboru.java:570)

Re-implementing the provider in UPA/Java did not encounter similar issues.

RFA/Java provider source code, UPA/Java provider source code (please note the line numbers differ from the above trace).

edit: With additional trace the symbol list fails at item #2957.

edit #2: Tried with increased buffer 65,535 bytes to no avail.

The actual source code used above, basically adds DSS/REST request: shinborujava.txt

treprfarfa-api
shinborujava.txt (20.2 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

1 Answer

Upvote
Accepted
Lars-at-Addicticks avatar image
1.5k ●3 ●5 ●7

What happens if your increase the size of your encoder ?

Currently you have it at 5000 bytes. (OMM_PAYLOAD_SIZE)

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Steven McCoy avatar image Steven McCoy

RFA/Java is unlike C++ technically supposed to take the payload size as a hint and automatically increase the buffer as needed. I have however tried with a buffer size of 65535 bytes with no change in result.

Steven McCoy avatar image Steven McCoy Steven McCoy

I guess the RFA encoder is simply a lot more inefficient than the UPA one, increasing to 100,000 bytes worked.

Lars-at-Addicticks avatar image Lars-at-Addicticks Steven McCoy

Yeah, well, I'm not sure what the downside is of allocating say 100K to the encoder always. I guess what happens in the background is that some byte array of the size of 100K is allocated. Most applications will tend to say: so what? Personally this is what I would do to avoid getting obscure errors - even if my real requirement is a lot smaller.

Click below to post an Idea Post Idea