question

Upvotes
Accepted
timothy.liu.1010 avatar image
1 ●0 ●0 ●0

AWS server return different data type when using REST API with python2 instead of python3

Hi,


I am trying to use REST API in python to extract tick history. I was able to follow the example provided in the download section and extract tick data successfully (in .csv.gzip format) in my development environment, which is in python3. However, when I tested exactly the same code in my production environment (python2) I got an error saying the return data is not in gzip format.

I checked the `Content-Type` field in HTTP response, I found AWS return a xml data, instead of gzip, in my production environment. Message as below:

`Content response headers (AWS server): type: application/xml`

The only difference, as far as I can tell for now, is that I am running with python2. Would this be the problem?

Thanks so much for helping

tick-history-rest-api
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

veerapath.rungruengrayubkul avatar image veerapath.rungruengrayubkul

Hi @timothy.liu.1010,

Could you share the snippet code and application log when the issue occurred, so that I will try to replicate the issue?

timothy.liu.1010 avatar image timothy.liu.1010 veerapath.rungruengrayubkul

I don't know the best way to share snippet here, but my code basically follows the example file in official doc: TRTH_OnDemand_IntradayBars.py, except we are extracting Time and Sales, instead of Intraday Summary.


Which kind of log are you looking for? I can try to paste it here.


Thanks

1 Answer

Upvotes
Accepted
veerapath.rungruengrayubkul avatar image
11.3k ●25 ●8 ●13

Hi @timothy.liu.1010,

I have tried the TRTH_OnDemand_IntradayBars.py example which is modified to use TickHistoryTimeAndSales extraction with Python 2.7.9 version, but cannot find the issue. Could you provide the request message you are using? Is there any modification on the example except the request?

Are you able replicate the issue on demand? Could you 1) enable debug logging, 2) add the X-Client-Session-Id header, and 3) log the response header of the RawExtractionResults endpoint?

Please provide the application's console log.

Below is the sample of modified code on Step 5.

#Step 5: get the extraction results, using the received jobId.
...
#   Advisory: avoid incomplete output - decompress then download
# 1) enable debug logging
logging.basicConfig(level=logging.DEBUG)

requestUrl = "https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/RawExtractionResults" + "('" + jobId + "')" + "/$value"

#AWS requires an additional header: X-Direct-Download
if useAws:
    requestHeaders={
        "Prefer":"respond-async",
        "Content-Type":"text/plain",
        "Accept-Encoding":"gzip",
        "X-Direct-Download":"true",
        "Authorization": "token " + token,
        # 2) add the X-Client-Session-Id header
        "X-Client-Session-Id": "9FBAA1E1-DFF3-435C-A0CA-1C9A6C46C266"
    }
else:
    requestHeaders={
        "Prefer":"respond-async",
        "Content-Type":"text/plain",
        "Accept-Encoding":"gzip",
        "Authorization": "token " + token
    }

r5 = requests.get(requestUrl,headers=requestHeaders,stream=True)
#Ensure we do not automatically decompress the data on the fly:
r5.raw.decode_content = False
if useAws:
    # 3) log the response header
    for red in r5.history:
        print (red.headers)
    
    print ('Content response headers (AWS server): type: ' + r5.headers["Content-Type"] + '\n')
    #AWS does not set header Content-Encoding="gzip".
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Click below to post an Idea Post Idea