Hi @ansn

Thanks for confirming that you are indeed populating the databuffer correctly - just needed to be eliminate that possibility - as this is a common mistake we have seen being made by other developers.

Also thanks for sharing the the 2nd trace file where you published 'umer' to the same fid for the two different instruments. I can see that on the outgoing post the identical data is being sent:

PostMsg anderstest42 <fieldEntry fieldId="32650" data="1B25 3075 6D65 72"/>

PostMsg anderstest4 <fieldEntry fieldId="32650" data="1B25 3075 6D65 72"/>

This confirms that the API and your application is working and encoding the outgoing payload correctly.

However, when you subscribe to the items we get back different values.

Refresh anderstest42 <fieldEntry fieldId="32650" data="1B25 3075 6D65 72"/>

Refresh anderstest4 <fieldEntry fieldId="32650" data="7465 7374 3437"/>

This indicates that the problem lies elsewhere i.e. the TREP infrastructure components.

As per my initial post, I recommend you speak to your Market Data team so they can check if there is anything reported in their TREP logs.
If they confirm that they do not see any issues reported, then I recommend they raise a support ticket with our TREP support team.
If your Market Data team passes on the Trace files you have generated, they will provide evidence that your application is behaving correctly and publishing the right data and that the problem lies somewhere in TREP.

One final thing you could try - just for elimination purpose is to remove the '1B 25 30' bytes from the RMTES String payload -

e.g. something like

dataBuffer.SetFromString(value, DataBuffer.DataBufferEnum.StringRMTES) 

- and run the test again with ASCII only string value as per your above tests.

As you say, if it works once it should work a 2nd time, but I just want to eliminate the possibility that TREP is perhaps occasionally not coping well this switching function.

I tried recreating your scenario here and published two RICs in sequence with identical payload including FID 32650 and I could not get it to fail (with or without the above 3 bytes).

Hi @ansn

The above code does not resemble RFA.NET code - I can only assume you are using some internal wrapper? Without sight of the actual underlying RFA.net code it would not be possible to confirm if there are any issues with said code.

There a couple of things to try:

• Check with your Market Data team if there any issues reported in the TREP (and ATS if you are posting to one) log files.
• Enable the low level trace for RFA.net so we can see the message flow between our app and the server

If you have access to the RFA config files, you can enable the low level trace using the following parameters:

\Connections\<Connection_RSSL>\traceMsgToFile = True 
\Connections\<Connection_RSSL>\traceMsgFileName = "<your trace file location and name>"

where <Connection_RSSL> should be replaced with the actual connection being used by whichever Session you acquire from the config.

Trace attached rfatrace_5984.txt

The point I want to get across is that the code is working in general, except for particular instruments. That means it's unlikely to be an issue with the code.

The code that calls the RFA looks like this

this.Msg.Clear();
this.Attrib.Clear();

Attrib.ServiceName = RFAserviceName; // i.e. new RFA_String("SERVICE")
Attrib.NameType = RDM.INSTRUMENT_NAME_TYPES.INSTRUMENT_NAME_RIC;
Attrib.Name = new RFA_String(instrument);
Msg.AttribInfo = Attrib;

Msg.MsgModelType = RDM.MESSAGE_MODEL_TYPES.MMT_MARKET_PRICE;
Msg.IndicationMask = PostMsg.IndicationMaskFlag.MessageInit | PostMsg.IndicationMaskFlag.MessageComplete;
Msg.IndicationMask |= PostMsg.IndicationMaskFlag.WantAck;
Msg.SeqNum = PostID; // don't need the fine granularity provided by seqnum. Just set it to postId.
Msg.PostID = PostID;
if ((Msg.HintMask & PostMsg.HintMaskFlag.Seq) == 0) throw new System.Exception("Flag not set");
pendingAcks.Add(PostID, new AckInfo { InstrumentName = instrument });
++PostID;

{
FieldList container = new FieldList();
container.Clear();
container.SetInfo(this.RDMdict.DictId, (short)Dictionary.DICTIONARY_TYPES.DICTIONARY_RECORD_TEMPLATES);
flItr.Start(container);
foreach (var field in instrumentForm.Fields)
{
msgEntry.Clear();
msgEntry.FieldID = field.Key;
msgEntry.Data = field.Value;
flItr.Bind(msgEntry);
}
flItr.Complete();

Msg.Payload = container;
}


OMMHandleItemCmd cmd = new OMMHandleItemCmd();
cmd.Handle = this.loginHandler.LoginStreamHandle; // Off-stream posting, using our loginHandlers stream (and its event handler for ack messages)
cmd.Msg = Msg;
consumer.Submit(cmd);

Hi @ansn

As mentioned, without your underlying code, it is difficult to investigate what is going wrong.

However, based on the trace I can see that data being sent to server for two instruments is different in length.

For anderstest42 your application is sending the following value:

<fieldEntry fieldId="32650" data="1B25 3030 2E39 3132 3632 3734 3138 3031 3931 37"/>

For anderstest4 your application is sending the following value:

<fieldEntry fieldId="32650" data="1B25 3030 2E35 3438 3830 3038 3235 3836 3235 3837"/>

When you then subscribe to those items, you are receiving back the following payload:

for anderstest42:

<fieldEntry fieldId="32650" data="1B25 3030 2E39 3132 3632 3734 3138 3031 3931 37"/>

and for anderstest4:

<fieldEntry fieldId="32650" data="7465 7374 3437"/>

incidentally - the above hex data converts to 'test47' - does this mean anything to you?

What are you contributing the data to - what does service 'SERVICE' represent? Is it some contributions engine, which perhaps has some rules defined for what length/values it can accept for the fid 32650 and is therefore being rejected? You may well get an ACK from the ADS because it considers the post a valid post msg format, but it is then being discarded by the upstream server.

Also, I should point out that your current wrapper implementation is incredibly inefficient - just in that single trace file you are performing connect, login, source directory download, logout, disconnect four times - which is a considerable overhead given that you are only posting and subscribing two items.

As a test - please post the identical value to the field for both anderstest4 and anderstest42 and see what happens.

The "test47" string is what the field value is from before my test, set by another client using the EMA api.

I've already told you everything I know about SERVICE. It's an internal persistent cache that's only updated with Post message, configured in the ADH. No interactive providers are attached to it.

The wrapper code is deliberately inefficient during upstart. It is however a lot more efficient than the raw RFA api in terms of developer time spent coding stuff, and it does not sacrifice performance once a subscription is established.

New trace attached where I send the string "umer". rfatrace_332.txt

Hi @ansn

Thanks for sharing the code. I cannot see how instrumentForm.Fields and its constituent Value are populated, so cannot be sure if the FieldEntry.Data is being populated with the correct DataBuffer content.

Please see attached file for an example of how FieldEntry.Data is populated for different field types e.g. for a string type field (such as 32650):

field.FieldID = 296 // Ask MMID 1 - Alphanumeric/RMTEST_STRING field 
RFA_String AskStr = new RFA_String("Ask MMID data");                            
dataBuffer.SetFromString(AskStr, DataBuffer.DataBufferEnum.StringRMTES); 
field.Data = dataBuffer;                             
fieldListWIt.Bind(field);

The attached code is extracted from the Encoder.cs file which can be found in the Examples\Common\ folder of the RFA.NET SDK.

Logically, and as you can verify from the trace, instrumentForm.Fields and its constituent Value must be populated properly since it works for one instrument but not the other, having run exactly the same code with only the instrument name differing.

Anyway here is the code if you want to prove me wrong

public void FillOut(string instrument, string field, object value)

{

RDMFidDef def = GetFidDef(field);

Open(instrument).Fields[def.FieldId] = Encoder.Encode(def, value);

}

public Data Encode(RDMFidDef fidDef, object value)
{
byte type_indicator = RFATypes.OMMType2DataBufferEnum(fidDef);
return Encode(type_indicator, value);
}

switch (fidDef.OMMType)
{
case OMMTypeEnum.RWF_TYPE_XML:
return DataBuffer.DataBufferEnum.XML;
case OMMTypeEnum.RWF_TYPE_OPAQUE:
return DataBuffer.DataBufferEnum.Opaque;
case OMMTypeEnum.RWF_TYPE_RMTES_STRING:
return DataBuffer.DataBufferEnum.StringRMTES;
case OMMTypeEnum.RWF_TYPE_UTF8_STRING:
return DataBuffer.DataBufferEnum.StringUTF8;
case OMMTypeEnum.RWF_TYPE_ASCII_STRING:
return DataBuffer.DataBufferEnum.StringAscii;
case OMMTypeEnum.RWF_TYPE_DATE_TIME:
return DataBuffer.DataBufferEnum.DateTime;
case OMMTypeEnum.RWF_TYPE_TIME:
return DataBuffer.DataBufferEnum.Time;
case OMMTypeEnum.RWF_TYPE_DATE:
return DataBuffer.DataBufferEnum.Date;
case OMMTypeEnum.RWF_TYPE_REAL:
return DataBuffer.DataBufferEnum.Real;
case OMMTypeEnum.RWF_TYPE_DOUBLE:
return DataBuffer.DataBufferEnum.Double;
case OMMTypeEnum.RWF_TYPE_FLOAT:
return DataBuffer.DataBufferEnum.Float;

etc etc etc

private Data Encode(byte type_indicator, object value, ErrMsgr default_prepend) {
DataBuffer result = new DataBuffer();
if (value == null)
{
result.SetBlankData(type_indicator);
return result;
}
switch (type_indicator) {

case DataBufferEnum.StringRMTES: // A nice string type with a private encoding scheme and no public encoder for it.
byte[] utf8_part = Encoding.UTF8.GetBytes(dynamic_cast<string>(value, default_prepend)); // The RMTESConverter is one-way only.
byte[] rmtes = new byte[3 + utf8_part.LongLength]; // Tell the RMTES string that we contain utf8.
new byte[] { 0x1B, 0x25, 0x30 }.CopyTo(rmtes, 0); // see https://developers.refinitiv.com/article/encoding-and-decoding-non-ascii-text-using-ema-and-rfa-cnet
utf8_part.CopyTo(rmtes, 3);
result.SetBuffer(new ThomsonReuters.RFA.Common.Buffer(rmtes), type_indicator);
break;
case DataBufferEnum.StringUTF8:
case DataBufferEnum.XML:
result.SetBuffer(new ThomsonReuters.RFA.Common.Buffer(Encoding.UTF8.GetBytes(dynamic_cast<string>(value, default_prepend))), type_indicator);
break;
case DataBufferEnum.StringAscii:
result.SetFromString(new RFA_String(dynamic_cast<string>(value, default_prepend)), type_indicator);
break;

case DataBufferEnum.Double:
result.SetDouble(dynamic_cast<double>(value, default_prepend),type_indicator);
break
case DataBufferEnum.Real: // Radix 10 floating point number implementation (!?)
result.SetFromString(new RFA_String(dynamic_cast<double>(value, default_prepend).ToString()), type_indicator);
break;

etc etc etc

static T dynamic_cast<T>(object value, ErrMsgr prepend_err = null)
{
try
{
return (T)(dynamic)value;
}
catch ....

Anders

Hi Umer

Thanks for confirming that our implementation is correct.

Thanks also for the suggestion. I tried the SetFromString() method with the same result.

You off course also need the source code in case it turns out to be a bug in the RFA api.

The issue is a tricky to reproduce. We use some instruments and suddenly they freeze and cannot be updated nor deleted by the RFA api. Oddly, the EMA api work fine. It can change the stuck instruments, but it doesn't get them unstuck for RFA.net.

The freezing typically happens after a few minutes of use, disconnecting, connecting, and re-requesting etc. After it gets stuck we can't get them unstuck.

I'll revert back to our market data team.

Hi @ansn

Thanks for the update.

It is very odd what you are seeing - as the underlying Reuters Wire Format is used by both RFA and EMA - if you enable the low level trace in EMA you should see the same PostMsg being sent to the server as per your RFA code.
It would be interesting to see if there any subtle differences at your end between the APIs in the trace messages...

Good idea. Unfortunately I don't have immediate access to the other implementation, but will forward your request.

Other than being different, the other code was run from another location (actually another country, but hitting the same TREP instance). We tried using the same credentials (username, appid, network position) from RFA with the same result - stuck instrument.

Anders