question

Upvotes
Accepted
KAKIT.LAI avatar image
7 ●2 ●2 ●6

Tick history raw data extract is not sort in timestamp

Hi,

I have a python code to use REST API to extract tick history raw from DSS. The extraction works but the file output is not sort by timestamp but instead sort by FIDs. How can I fix that?


Below is the example using my program to extract 0005.HK on Nov 1 from 0930 - 1000 local exchange time, for FIDs 22, 25 and 6. Here's a snapshot of the extraction using my program. Its sorted by BID (22), following by ASK (25) and then by TRDPRC_1 (6)


capture2.png


I compare the same extraction using TRTH with the following template which is same as my program

Below is the extract from TRTH. You can see the data is sorted by timestamp.


capture5.png


trth-rest-api
capture2.png (100.2 KiB)
capture5.png (228.8 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

6 Answers

Upvotes
Accepted
Jirapongse avatar image
38.1k ●71 ●35 ●53

@KAKIT.LAI

The request contains only one FID.

1637642086562.png

Therefore, the result contains only values of the TRDPRC_1 field.

1637642161325.png


1637642086562.png (30.0 KiB)
1637642161325.png (30.4 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
Jirapongse avatar image
38.1k ●71 ●35 ●53

@KAKIT.LAI

Please share the request message that you are using.

I used the following request.

{
  "ExtractionRequest": {
    "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.TickHistoryRawExtractionRequest",
    "IdentifierList": {
      "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",  
      "InstrumentIdentifiers": [{
        "Identifier": "0005.HK",
        "IdentifierType": "Ric"
      }],"ValidationOptions": {"AllowHistoricalInstruments": true}
    },
    "Condition": {
      "MessageTimeStampIn": "LocalExchangeTime",
      "ReportDateRangeType": "Range",
      "QueryStartDate": "2021-11-01T01:30:00.000Z",
      "QueryEndDate": "2021-11-01T02:00:00.000Z",
      "ExtractBy": "Ric",
      "SortBy": "SingleByRic",
      "DomainCode": "MarketPrice",
      "Fids": "22,25,6",
      "DisplaySourceRIC": true,
      "FidListOperator": "OR"
    }
  }
}

The output is:

1636949150330.png


1636949150330.png (102.6 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
KAKIT.LAI avatar image
7 ●2 ●2 ●6

This is the code


    "ExtractionRequest": {
        "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.TickHistoryRawExtractionRequest",
        "IdentifierList": {
        "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",  
        "InstrumentIdentifiers": list_of_ric2,
        "UseUserPreferencesForValidationOptions":"false"
        },    
        "Condition": {
        "DateRangeTimeZone": DateRangeTimeZone,
        "TimeRangeMode": TimeRangeMode,
        "MessageTimeStampIn": MessageTimeStampIn,
        "ReportDateRangeType": "Range",
        "QueryStartDate": start_datetime,
        "QueryEndDate": end_datetime,
        "ExtractBy": ExtractBy,
        "SortBy": SortBy,
        "DomainCode": DomainCode,
        "DisplaySourceRIC":"false",
        "Fids": ",".join(fids)
        }
    }
    }


And this is what I input into my program


capture.png



capture.png (25.3 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Jirapongse avatar image Jirapongse

@KAKIT.LAI

The request message doesn't have the FidListOperator property. I checked and found that, on my account, the default value is "AND" so the output is:

1636962252327.png

Please check the request message with other tools, such as Postman or curl.

1636962252327.png (30.0 KiB)
Upvotes
KAKIT.LAI avatar image
7 ●2 ●2 ●6

Thanks. I tried but still giving me the same output...


"Condition": {
        "DateRangeTimeZone": DateRangeTimeZone,
        "TimeRangeMode": TimeRangeMode,
        "MessageTimeStampIn": MessageTimeStampIn,
        "ReportDateRangeType": "Range",
        "QueryStartDate": start_datetime,
        "QueryEndDate": end_datetime,
        "ExtractBy": ExtractBy,
        "SortBy": SortBy,
        "DomainCode": DomainCode,
        "DisplaySourceRIC":"false",
        "Fids": ",".join(fids),
        "FidListOperator": "AND"
        }
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Jirapongse avatar image Jirapongse

@KAKIT.LAI

With the "AND" operator, the output should be like this:

1637028897894.png

Please check the request message with other tools, such as Postman or curl.

1637028897894.png (107.5 KiB)
Upvotes
KAKIT.LAI avatar image
7 ●2 ●2 ●6

Sorry for the delay in getting back. I tried again and output still sorted by the FIDs instead of time of the update. The following is the actual parameter passed to in ExtractionRequest.


{'ExtractionRequest': {'@odata.type': '#DataScope.Select.Api.Extractions.ExtractionRequests.TickHistoryRawExtractionRequest', 'IdentifierList': {'@odata.type': '#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList', 'InstrumentIdentifiers': [{'Identifier': '3333.HK', 'IdentifierType': 'Ric'}], 'UseUserPreferencesForValidationOptions': 'false'}, 'Condition': {'DateRangeTimeZone': 'Local Exchange Time Zone', 'TimeRangeMode': 'Inclusive', 'MessageTimeStampIn': 'LocalExchangeTime', 'ReportDateRangeType': 'Range', 'QueryStartDate': '2021-11-08T10:00:00.000Z', 'QueryEndDate': '2021-11-08T10:30:00.000Z', 'ExtractBy': 'Entity', 'SortBy': 'SingleByRic', 'DomainCode': 'MarketPrice', 'DisplaySourceRIC': 'false', 'Fids': '6', 'FidListOperator': 'AND'}}}

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
KAKIT.LAI avatar image
7 ●2 ●2 ●6

I have fixed the code and issue is fixed. You can close the ticket. Thanks for the help.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Click below to post an Idea Post Idea