question

Upvotes
Accepted
Joey avatar image
1 ●0 ●0 ●0

Issue retrieving data from datastream python API - possibly a requests 2.22 issue

Hi I'm using the python API for datastream with no issues locally but struggling to trigger these scripts on my server on a schedule.

The problem I'm facing is that I fly blindly on the server with very little error information - simply the result is just the get_data function retrieves nothing.

This server needs to have the API url whitelisted (as all web urls are otherwise blocked). I have whitelisted the below - Can anyone confirm this is correct?

https://product.datastream.com/DSWSClient/V1/DSService.svc/rest/

https://datastreamprod.int.refinitiv.com/dswsclient/V1/DSService.svc/rest/


Secondly my python installation is different on this server where the requests library is version 2.22 and my local requests library is 2.24. Aligning is very difficult (corporate IT). Are there any known issues utilizing this version of requests through the python API?

datastream-apidsws-api
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

2 Answers

Upvotes
Accepted
Gurpreet avatar image
13k ●32 ●12 ●18

Hello @joseph.duncan2,

I have been able to get more debugging information by enabling logging and http-debug. You can try and see if this helps troubleshoot the issue:

# add this code into your python dsws application
import logging
import http.client

http.client.HTTPConnection.debuglevel = 1
logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True

This generates additional debugging information which you can pipe into a file. Here is what I capture:

DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): product.datastream.com:443
DEBUG:urllib3.connectionpool:https://product.datastream.com:443 "POST /DSWSClient/V1/DSService.svc/rest/GetToken HTTP/1.1" 200 355

DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): product.datastream.com:443
DEBUG:urllib3.connectionpool:https://product.datastream.com:443 "POST /DSWSClient/V1/DSService.svc/rest/GetData HTTP/1.1" 200 360

send: b'POST /DSWSClient/V1/DSService.svc/rest/GetToken HTTP/1.1\r\nHost: product.datastream.com\r\nUser-Agent: python-requests/2.25.1\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\nContent-Length: 111\r\nContent-Type: application/json\r\n\r\n'
send: b'{"Password": "****", "Properties": [{"Key": "__AppId", "Value": "PythonLib 1.0.8"}], "UserName": "****"}'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: application/json; charset=utf-8
header: X-Powered-By: ASP.NET
header: Date: Thu, 11 Feb 2021 14:37:22 GMT
header: Content-Length: 355

send: b'POST /DSWSClient/V1/DSService.svc/rest/GetData HTTP/1.1\r\nHost: product.datastream.com\r\nUser-Agent: python-requests/2.25.1\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\nContent-Length: 584\r\nContent-Type: application/json\r\n\r\n'
send: b'{"DataRequest": {"DataTypes": [{"Properties": null, "Value": "VO"}, {"Properties": null, "Value": "P"}], "Instrument": {"Properties": null, "Value": "VOD"}, "Date": {"End": "", "Frequency": "", "Kind": 0, "Start": "2017-01-01"}, "Tag": null}, "Properties": {"Key": "Source", "Value": null}, "TokenValue": "fMSwrVbhK****222870"}'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: application/json; charset=utf-8
header: X-Powered-By: ASP.NET
header: Date: Thu, 11 Feb 2021 14:37:22 GMT
header: Content-Length: 360


  Instrument Datatype     Value       Dates
0        VOD       VO  36773.80  2017-01-01
1        VOD        P    199.85  2017-01-01


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
Jirapongse avatar image
38.1k ●71 ●35 ●53

@joseph.duncan2

Yes, the URL is https://product.datastream.com/DSWSClient/V1/DSService.svc/rest/.

I am using requests 2.22.0 and it works fine.



icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Click below to post an Idea Post Idea