question

Upvotes
Accepted
Naveen Tad avatar image
1 ●0 ●0 ●0

How to Fix, TRTH On Demand Raw Data Extraction Errors

Hello ,


I am working on setting up a python downloader for “On Demand Raw Data Extraction” using the documentation provided here (https://developers.refinitiv.com/en/api-catalog/refinitiv-tick-history/refinitiv-tick-history-rth-rest-api/tutorials#rest-api-tutorial-8-on-demand-raw-data-extraction)


I’m running into an issue where I get a 400(Bad Request) response. See below for details


On Demand Extraction Request Url : https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractRaw


Python Function


baseUrl = "https://selectapi.datascope.refinitiv.com/RestApi/v1/"

requestHeaders = { "Prefer":"respond-async", "Content-Type":"application/json" }

# Get Location URL

defgetLocationUrl( token):

ExtractionUrl = baseUrl + "Extractions/ExtractRaw"

requestHeaders["Authorization"] = "token " + token

params = { "ExtractionRequest": { "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.TickHistoryRawExtractionRequest", "IdentifierList": { "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList", "InstrumentIdentifiers": [ { "Identifier" : "AB.PS" , "IdentifierType" : "Ric" } ] }, "Condition": { "ReportDateRangeType": "Delta", "DaysAgo": 1 } } }


print(ExtractionUrl)

r = requests.post( ExtractionUrl, headers=requestHeaders , params=params)

if (r.status_code==200):

print ("YAY")

else:

print(r.status_code)

print(r.reason )

print(r.text)


Response Code


r.status.code = 400

r.reason = ‘Bad Request’

r.text = {"error":{"code":"6fe04886-854a-45d5-bc57-4d6ab225a700","message":"Query option 'ExtractionRequest' was specified more than once, but it must be specified at most once."}}


Per the Response message , ‘ExtractionRequest’ was specified more than once, but as you can see in the below, It was only specified once


params = { "ExtractionRequest": { "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.TickHistoryRawExtractionRequest", "IdentifierList": { "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList", "InstrumentIdentifiers": [ { "Identifier" : "AB.PS" , "IdentifierType" : "Ric" } ] }, "Condition": { "ReportDateRangeType": "Delta", "DaysAgo": 1 } } }


Can someone please look into this and help us out with resolving this issue.


Thanks,

Naveen.

tick-history-rest-apiextraction
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

zoya faberov avatar image zoya faberov

Hello @Naveen Tad,

From looking at the code, it seems that you are not passing requestBody into Post request, but only URL, headers and parameters.

I am not fully sure if this is the only issue with the code, but would suggest that by following the steps in the answer and starting with both the working request to replace and the working python code skeleton, you should arrive at a working code per your requirement the quickest way.

4 Answers

Upvotes
Accepted
zoya faberov avatar image
23k ●22 ●9 ●14

Hello @Naveen Tad,

In my understanding, as the tutorial you looking at is par of Postman starter collection, discusses step-by-step the approach for Postman requesting, and Raw request is available and downloadable with our Postman starter collection,

If you would like to run the same/similar Raw extraction request, but in your python code,

The quickest way to get it working in Python would be:

  1. Install free tool Postman if you have not had a chance, via www.getpostman.com
  2. Download fully working RTH REST Tutorials Postman collection
  3. Import collection, run
    1. Authentication request
    2. Raw data extraction request
    3. You have a fully working "naked" request at hand
  4. Download working on-demand example code for Python in RTH Python Code Samples
  5. Swap the request in on-demand example with your required request, from Postman collection.
  6. Any other on-demand DSS HTTP REST request that you will have ready and working in Postman, you will be able to quickly have working from python code the same way, by creating a copy of the working example and just swapping the payload of the request into the example in place of the request.

Does this help, will you be able to proceed?


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
Naveen Tad avatar image
1 ●0 ●0 ●0

Thanks. Will test with sample examples and get back on this thread if any issues.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
Naveen Tad avatar image
1 ●0 ●0 ●0

I downloaded the sample program RTH_OnDemand_IntradayBars.py and just ran it with my ID and I get ‘Quote Exceeded Error’. Please take a look at the below logs /


$ python ./RTH_OnDemand_IntradayBars.py

Authentication token (valid 24 hours):

**********************************

HTTP status of the response: 202

Extraction is not complete, we shall poll the location URL:

https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractRawResult(ExtractionId='0x0793c595679ca9d4')

As we received a 202, we wait 30 seconds, then poll again (until we receive a 200)

HTTP status of the response: 202

As we received a 202, we wait 30 seconds, then poll again (until we receive a 200)

HTTP status of the response: 200


jobId: 0x0793c595679ca9d4


Extraction notes:

Extraction Services Version 14.5.42294 (737b0965c07f), Built Apr 8 2021 13:43:46

User ID: 9027207

Extraction ID: 2000000265791722

Schedule: 0x0793c595679ca9d4 (ID = 0x0000000000000000)

Input List (2 items): (ID = 0x0793c595679ca9d4) Created: 06/01/2021 16:13:47 Last Modified: 06/01/2021 16:13:47

Report Template (16 fields): _OnD_0x0793c595679ca9d4 (ID = 0x0793c59567bca9d4) Created: 06/01/2021 16:12:09 Last Modified: 06/01/2021 16:12:09

Schedule dispatched via message queue (0x0793c595679ca9d4), Data source identifier (1795B0303CCE47368DE78572C98769BE)

Schedule Time: 06/01/2021 16:12:10

Processing started at 06/01/2021 16:12:10

Processing completed successfully at 06/01/2021 16:13:48

Extraction finished at 06/01/2021 16:13:48 UTC, with servers: tm02n02, TRTH (89.76 secs)

Instrument <RIC,CARR.PA> expanded to 1 RIC: CARR.PA.

Instrument <RIC,IBM.N> expanded to 1 RIC: IBM.N.

Total instruments after instrument expansion = 2

Quota Message: INFO: Tick History Cash Quota Count Before Extraction: 0; Instruments Approved for Extraction: 0; Tick History Cash Quota Count After Extraction: 0, 100% of Limit; Tick History Cash Quota Limit: 0

Quota Message: ERROR: The RIC 'CARR.PA' in the request would exceed your quota limits. Adjust your input list to continue.

Quota Message: ERROR: The RIC 'IBM.N' in the request would exceed your quota limits. Adjust your input list to continue.

Quota Message: WARNING: Tick History Cash Quota has been reached or exceeded


Content response headers (AWS server): type: text/plain


Saving compressed data to file: .Python_Test.step4.csv.gz ... please be patient

Finished saving data to file:.Python_Test.step4.csv.gz


Decompressed data:


Content response headers (AWS server): type: text/plain


Saving compressed data to file:.Python_Test.step4.csv.gz ... please be patient

Finished saving compressed data to file:.Python_Test.step5.csv.gz

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
zoya faberov avatar image
23k ●22 ●9 ●14

Hello @Naveen Tad,

As you login into DSS GUI, please click on the menu Your Name->Usage->Quota.

I can see two possibilities: if your user id is not permissioned for Intraday bars request, you should not see any quota, or if you are permissioned, you have exceeded your quota and should see it as well.

---

If you are permissioned for TickHistoryRaw request, once you have have your TickHistoryRaw request tuned and working in Postman, you could replace the request in example code with yours= working request, and run that instead.

Let us know how this works for you?

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Click below to post an Idea Post Idea