question

Upvotes
Accepted
f.palma avatar image
1 ●0 ●0 ●1

error in from_json_to_df in PyDSWS

Hi, I saw there is the same question on the forum, but without any actual solution. Sometimes, when I try to make a request: 

import PyDSWS

ds = PyDSWS.Datastream(username=RefinitivCredentials.username, password=RefinitivCredentials.password)
data = ds.get_data(tickers=tickers, start=start_date, end=end_date, freq=frequency, fields=fields)

I get this:

File "/usr/local/lib/python3.7/site-packages/PyDSWS/datastream.py", line 102, in get_data

df = self.from_json_to_df(response)

File "/usr/local/lib/python3.7/site-packages/PyDSWS/datastream.py", line 41, in from_json_to_df

if response_json['Dates']:

KeyError: 'Dates'"


Moreover, it seems that it does not always happen and I'm not actually able to reproduce it through https://product.datastream.com/DswsClient/Docs/TestRestV1.aspx?



datastream-apidsws-apierror
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

1 Answer

Upvotes
Accepted
Jirapongse avatar image
38.1k ●71 ●35 ●53

@f.palma

Typically, the response will have Dates field like this:

   {
        "AdditionalResponses": [
           ...
        ],
        "DataTypeNames": null,
        "DataTypeValues": [
          ...
        ],
        "Dates": [
            "/Date(1600128000000+0000)/",
            "/Date(1600214400000+0000)/",
            "/Date(1600300800000+0000)/",
            "/Date(1600387200000+0000)/",
            "/Date(1600646400000+0000)/",
            "/Date(1600732800000+0000)/",
            "/Date(1600819200000+0000)/",
           ...
        ],

The error indicates that the response doesn't have the "Dates" field. We may need to verify the returned response. One way to do that is by using a proxy. You can refer to Use mitmdump to Capture Refinitiv Real-Time - Optimized Content to setup a proxy.

Then run the mitmdump.exe with the following parameter.

C:\Program Files (x86)\mitmproxy\bin>mitmdump.exe --flow-detail=3

Proxy server listening at http://*:8080

Next, set proxy to the python code.

import os
os.environ['HTTP_PROXY']="http://127.0.0.1:8080"
os.environ['HTTPS_PROXY']="http://127.0.0.1:8080"
os.environ['REQUESTS_CA_BUNDLE']="c:\\temp\\mitmproxy-ca.pem"

The certificate files will be created in the config directory (~/.mitmproxy or %USERPROFILE%/.mitmproxy by default). Please change the location of "c:\\temp\\mitmproxy-ca.pem".

After that, you will see data sent through the proxy.

Please share the response when the problem occured.


1602844865542.png (33.4 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Click below to post an Idea Post Idea