question

Upvote
Accepted
31 0 0 3

TREP trimming data

Hi,

I'm using last version of RFA.NET library (8.1.1.1) to communicate with TREP. Everything works until I start sending bigger messages. It looks like if the message is bigger than 1.5KB it is trimmed and I will get just part of the data. I checked and message buffer is really smaller.

We use our own data model for this without adding it to directory.


I test like this:

I check encoded buffer size before provider sends and compare with encoded buffer size on consumer side.

  • Provider
    • Non-interactive provider
    • Connection RSSL
    • RFA.NET (8.1.1.1)
  • Consumer
    • Consumer
    • Connection RSSL
    • RFA.NET (8.1.1.1)


Do you know why this happens and what to do to fix that?

I would expect this can be because some fragmentation, but I will get just one message not more. Can it be caused by incorrect settings of TREP?


Update:

Thanks for trace configuration.

My feed is set up to stay live even if provider goes down and items are always live if I don't send suspect explicitly.

I tested once more with trace and I found strange behavior.

In my test I was pushing small messages to TREP and I was increasing its size. After couple of iterations, I did this:

  1. I started Consumer (ok) - see RSSLConsumerTrace_25328.xml
  2. It received old data (ok) - see RSSLConsumerTrace_25328.xml
  3. I started Provider and it sent new data (ok) - see RSSLProviderTrace_25328.xml
  4. Consumer got new data (ok) - see RSSLConsumerTrace_25328.xml
  5. I stopped Consumer and Provider (ok) - see RSSLConsumerTrace_25328.xml and RSSLProviderTrace_25328.xml
  6. I started Consumer again and expected data from point 3 but I got whole bunch of strange messages. (NOT OK) - see RSSLConsumerTrace_32076.xml

All the files are in this zip: Test.zip

treprfarfa-api.net
test.zip (387.4 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
Accepted
7.6k 15 6 9

@jan.skalicky

  • Are you using the same custom dictionary file on both the Application and the TREP server side? I'm not sure that there is a problem with the type of data in the dictionary on the TREP server or not.
  • What kind of data type did you use to publish the fid on the NI Provider side? Is this opaque data as Umer said?


From the tracing log, it looks like you are using field ID 1 and from our data dictionary it is PROD_PERM and data type is UINT64. But data you are publishing is not the UInt64 so I'm not sure how the ADH caching this kind of data after you stop the NI Provider. I guess that the data you receive from the trace file end with 25328 come from the Provider which still active so the TREP just fan out the message from the provider to the consumer app. It could be the reason that there is no problem when you test it when the Provider still active.

But after the provider stops, and then you start the consumer again, the TREP sends multiple refresh messages with a blank field list to the consumer instead.

<!-- Incoming Message from '<host>:14002' on 'localhost' interface -->

<!-- Time: 13:31:44:405 -->

<!-- rwfMajorVer="14" rwfMinorVer="1" -->

<refreshMsg domainType="128" streamId="3" containerType="RSSL_DT_FIELD_LIST" flags="0x10A8 (RSSL_RFMF_HAS_MSG_KEY|RSSL_RFMF_SOLICITED|RSSL_RFMF_HAS_QOS|RSSL_RFMF_HAS_PART_NUM)" groupId="42" partNum="12" qosDynamic="0" qosRate="1" qosTimeliness="1" dataState="RSSL_DATA_OK" streamState="RSSL_STREAM_OPEN" code="RSSL_SC_NONE" text="" dataSize="3">

<key flags="0x7 (RSSL_MKF_HAS_SERVICE_ID|RSSL_MKF_HAS_NAME|RSSL_MKF_HAS_NAME_TYPE)" serviceId="772" name="Px.Groups" nameType="1"/>

<dataBody>

<fieldList flags="0x8 (RSSL_FLF_HAS_STANDARD_DATA)">

</fieldList>

</dataBody>

</refreshMsg>


<!-- Incoming Message from '<host>:14002' on 'localhost' interface -->

<!-- Time: 13:31:44:405 -->

<!-- rwfMajorVer="14" rwfMinorVer="1" -->

<refreshMsg domainType="128" streamId="3" containerType="RSSL_DT_FIELD_LIST" flags="0x10A8 (RSSL_RFMF_HAS_MSG_KEY|RSSL_RFMF_SOLICITED|RSSL_RFMF_HAS_QOS|RSSL_RFMF_HAS_PART_NUM)" groupId="42" partNum="13" qosDynamic="0" qosRate="1" qosTimeliness="1" dataState="RSSL_DATA_OK" streamState="RSSL_STREAM_OPEN" code="RSSL_SC_NONE" text="" dataSize="3">

<key flags="0x7 (RSSL_MKF_HAS_SERVICE_ID|RSSL_MKF_HAS_NAME|RSSL_MKF_HAS_NAME_TYPE)" serviceId="772" name="Px.Groups" nameType="1"/>

<dataBody>

<fieldList flags="0x8 (RSSL_FLF_HAS_STANDARD_DATA)">

</fieldList>

</dataBody>

</refreshMsg>


The blank field list from the above sample traces is the outgoing message from the TREP server. Therefore, the problem seems to relate to the data caching mechanism on the TREP side. I would suggest you contact the Refinitiv TREP support team to help you verify the setup and you can provide the RSSL trace log to the team to investigate the issue further.


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
20.3k 73 10 20

Hi @jan.skalicky

Can you confirm if you are Posting data to TREP or implementing a Provider application to send the data? If a provider, then is it interactive or non-interactive?

Also, the data you are sending what form is it - a simple FieldList or a more complex Reuters Domain model - if so which domain? Or are you sending an opaque Buffer?

Please provide more details on exactly what you are doing, the connection type, the payload etc so that we can assist you better.

How are you confirming the data is being trimmed? How do you consume the data?

Also, if you can confirm which version of TREP so that we can check for any know TREP issues.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

I added info in my question. I just need to ask internally which version of TREP we have.

Upvotes
20.3k 73 10 20

Hi @jan.skalicky

Thanks for updating the original post with the additional information.

Are you publishing the data using a Custom Domain Model specified in the RespMsg and then consuming the data by specifying the Custom Domain in your ReqMsg too?

Unless one of my colleagues has a quick answer, I suspect this would require a more detailed investigation (and I will be out of the office for most of next week). In view of this, I would recommend you create a Support Ticket - so that you can work with Developer support to verify your Provider code and ensure the encoding of the message is correct etc.

You can also enable the RSSL Trace for your Provider and Consumer to confirm what exact payload is being transmitted by the Provider to TREP and likewise what payload is being received by the Consumer from TREP. See this post on how to enable the Trace

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

I added new findings with trace logs to question. Can you please check? I asked our internal market data support for access to support accounts, but I did not get it so far, therefore I would like to continue here for now, if it is ok.

Upvotes
7.6k 15 6 9

@jan.skalicky

You may also turn on the RSSL Trace log on both Provider and Consumer side so you can compare the raw data that sent and receive by API.

You can find the instruction to turn on the trace file from this post.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

I added new findings with trace logs to question. Can you please check? I asked our internal market data support for access to support accounts, but I did not get it so far, therefore I would like to continue here for now, if it is ok.

Click below to post an Idea Post Idea