For a deeper look into our Eikon Data API, look into:

Overview |  Quickstart |  Documentation |  Downloads |  Tutorials |  Articles

question

Upvotes
Accepted
gibran.afzal1 avatar image
1 ●0 ●0 ●0

Eikon python api not able to fetch data

I recently moved to a new work machine and am unable to get the eikon python api working. This was working on my previous machine.

- I have eikon desktop running
- I have an app id which I am able to register with the python api.
- When calling get_data() I get errors such as the below:

020-09-29 09:19:04,160 P[28316] [MainThread 19712] Error code 502

I think the eikon can't get past our corporate firewall.

Is it possible to specify the proxy for the eikon process to use? (Note I have the env variables HTTP_PROXY/HTTPS_PROXY set and confirm they WORK for other http requests in python)


import eikon as ek

ek.set_app_key('....'')

ek.get_data('CL', 'Close')

eikoneikon-data-apiworkspaceworkspace-data-apirefinitiv-dataplatform-eikonpythonapi
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

11 Answers

Upvotes
Accepted
pierre.faurel avatar image
4.3k ●2 ●4 ●5

I have another test that you can run easily. That could confirm if your issue is caused by the corporate proxy.

Open a console windows then type :

python -c "import httpx; print(httpx.get('http://localhost:9060/api/status'))"

Chek if there is error in output.

Now, type:

> set NO_PROXY=localhost:9000,localhost:9060
> python -c "import httpx; print(httpx.get('http://localhost:9060/api/status'))"

Check the output (it should be <Response [200 OK]> )

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
umer.nalla avatar image
20.3k ●73 ●10 ●20

Hi @gibran.afzal1

Can you try installing a specific version of httpx library and see if it helps

pip install httpx==0.14.2

We have noticed a problem in one of the Eikon Library dependencies httpx which is used in the requests library. For other users with similar issues, they have been able to resolve the issue for the time being by installing the specific version.

We will of course fix this in the near future.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
pierre.faurel avatar image
4.3k ●2 ●4 ●5

Hi,

We need more infotmation than 502 error code.

Could you follow this steps :

  1. To check if Eikon proxy is avalable, open http://localhost:9000/api/
  2. check your eikon version (pip list => eikon ???)
    Last 1.1.6.post3 version fixed the issue with httpx 0.15.x versions, so you can keep any version
  3. get result from request as below:
    result, error = ek.get_data('CL', 'Close')
    error should contains more detazil than 502 error code


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
gibran.afzal1 avatar image
1 ●0 ●0 ●0

Hi

1. This gives the following error:

{"code":500,"message":"Cannot find module \".\"","statusMessage":"Internal Server Error"}

I assume it is not running. any idea what step I am missing

2. eikon = 1.1.6.post3


Thanks

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

pierre.faurel avatar image pierre.faurel

Sorry, the right url is http://localhost:9060/api/status

Result should be : 

{"statusCode":"ST_PROXY_READY","version":"2.6.3"}

To get more detail, could you activate log before ek.get_data() call:

ek.set_log_level(10)
ek.get_data(...)

then share traces.

Upvotes
gibran.afzal1 avatar image
1 ●0 ●0 ●0

Hi Pierre,

The correct url you provided gives:

{"statusCode":"ST_PROXY_READY","version":"2.6.0"}

This is what i get with the increased log level


2020-09-30 08:52:45,853 P[28316] [MainThread 19712] Request:{'Entity': {'E': 'DataGrid_StandardAsync', 'W': {'requests': [{'instruments': ['CL'], 'fields': [{'name': 'Close'}]}]}}} 2020-09-30 08:52:45,853 P[28316] [MainThread 19712] Request to http://localhost:9060/api/v1/data     headers = {'Content-Type': 'application/json', 'x-tr-applicationid': '5dee7c58886b4ddb939898c91e5641acc339a40a'}     params = None 2020-09-30 08:52:45,916 P[28316] [MainThread 19712] HTTP Response code: 502 2020-09-30 08:52:45,917 P[28316] [MainThread 19712] HTTP Response: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html> <!-- FileName: index.html Language: [en] --> <!--Head--> <head> <meta content="text/html; charset=UTF-8" http-equiv="Content-Type"> <meta http-equiv="X-UA-Compatible" content="IE=7" /> <title>RWE Web Gateway - Notification</title> <script src="/mwg-internal/de5fs23hu73ds/files/javascript/sw.js" type="text/javascript" ></script> <!-- <link rel="stylesheet" href="/mwg-internal/de5fs23hu73ds/files/default/stylesheet.css" /> --> <style type="text/css">
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

pierre.faurel avatar image pierre.faurel

Sorry for the delay to reply !

Could you detail Eikon Desktop version ?

To investigate deeper, could you execute following lines :

import eikon as ek
ek.set_log_level(1)
ek.set_app_key(("xxxxxxxxx")
result, error = ek.get_data('CL', 'Close')

then share output.

Take care to anonymize personal data in following messages :

...
2020-10-02 19:47:41,262 P[52340] [MainThread 52336] Request to http://localhost:9060/api/handshake
    headers = {'Content-Type': 'application/json', 'x-tr-applicationid': 'xxxxxxxxxxxxx'}
    params = None
2020-10-02 19:47:41,891 P[52340] [MainThread 52336] HTTP request response 200: {"access_token":"xxxxxxxxxxxxx","expires_in":1209600,"token_type":"bearer"}
2020-10-02 19:47:41,892 P[52340] [MainThread 52336] Response : 200 - {"access_token":"xxxxxxxxxxxxx","expires_in":1209600,"token_type":"bearer"}
2020-10-02 19:47:41,892 P[52340] [MainThread 52336] Application ID: xxxxxxxxxxxxx
...
2020-10-02 19:47:41,893 P[52340] [MainThread 52336] Request to http://localhost:9060/api/v1/data
    headers = {'Content-Type': 'application/json', 'x-tr-applicationid': 'xxxxxxxxxxxxx', 'Authorization': 'xxxxxxxxxxxxx'}
    params = None
Upvotes
gibran.afzal1 avatar image
1 ●0 ●0 ●0

Hi


my desktop is 4.0.52 (4.0.52055)


For the snippet of code you gives the error shown in my previous reply.

(Note I haven't included the rest of the output as its essentially the css/html for our internal proxy error page) 

2020-10-05 11:56:03,689 P[47692] [MainThread 45204] entity: DataGrid_StandardAsync 2020-10-05 11:56:03,692 P[47692] [MainThread 45204] payload: {'requests': [{'instruments': ['CL'], 'fields': [{'name': 'Close'}]}]} 2020-10-05 11:56:03,693 P[47692] [MainThread 45204] Request:{'Entity': {'E': 'DataGrid_StandardAsync', 'W': {'requests': [{'instruments': ['CL'], 'fields': [{'name': 'Close'}]}]}}} 2020-10-05 11:56:03,694 P[47692] [MainThread 45204] Request to http://localhost:9060/api/v1/data
    headers = {'Content-Type': 'application/json', 'x-tr-applicationid': 'xxx'}     params = None 2020-10-05 11:56:03,728 P[47692] [MainThread 45204] HTTP request response 502: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html>

1601895712096.png (36.7 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
pierre.faurel avatar image
4.3k ●2 ●4 ●5

Hi,

Interesting information is in HTTP response.

502 error code is related to a gateway error, and the response you get isn't expected at all. To debug at lower level, could you set environment variable HTTPX_LOG_LEVEL=trace , then run again the snippet.

There is 3 HTTP responses to check:

1. Request: GET http://localhost:9060/api/status

2020-10-05 15:57:22,706 P[18028] [MainThread 30724] HTTP request response 200: {"statusCode":"ST_PROXY_READY","version":"2.6.3"}

2. Request to http://localhost:9060/api/handshake

2020-10-05 15:57:23,209 P[18028] [MainThread 30724] HTTP request response 200: {"access_token":"xxxxx","expires_in":1209600,"token_type":"bearer"}

3. Request to http://localhost:9060/api/v1/data

2020-10-05 15:57:23,215 P[18028] [MainThread 30724] Request to http://localhost:9060/api/v1/data 
 ... 
DEBUG [2020-10-05 15:57:23] httpx._client - HTTP Request: POST http://localhost:9060/api/v1/data "HTTP/1.1 307 Temporary Redirect" 
... 
DEBUG [2020-10-05 15:57:27] httpx._client - HTTP Request: POST http://localhost:9060/api/udf "HTTP/1.1 200 OK" 
... 
2020-10-05 15:57:27,480 P[18028] [MainThread 30724] HTTP request response 200: {"responses":[{"columnHeadersCount":1,"data":[["CL",null]],"error":[{"code":251658244,"col":1,"message":"Error: Field 'CLOSE' was not found in response for the instrument 'CL'","row":0}],"headerOrientation":"horizontal","headers":[[{"displayName":"Instrument"},{"displayName":"CLOSE","field":"CLOSE"}]]...}]}

I assume that there isn't error in steps 1 and 2.

Could check traces in step 3 ?

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
gibran.afzal1 avatar image
1 ●0 ●0 ●0

Hi

I set the env variable, and restarted the browser.


1

http://localhost:9060/api/status

statusCode"ST_PROXY_READY"version"2.6.0"


2

http://localhost:9060/api/handshake

code404message"Not Found"statusMessage"Not Found"


3

http://localhost:9060/api/v1/data


code500message"Cannot find module \".\""statusMessage"Internal Server Error"
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
pierre.faurel avatar image
4.3k ●2 ●4 ●5

Sorry, I didn't ask to open urls but just set env variable then restart the code snippet 

import eikon as ek 
ek.set_log_level(1) 
ek.set_app_key(("xxxxxxxxx") 
result, error = ek.get_data('CL', 'Close')

and check traces.

I just shared what I have as request response in my console window.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
gibran.afzal1 avatar image
1 ●0 ●0 ●0

Hi

When doing e.set_app_key() I get:

TRACE [2020-10-05 16:28:34] httpcore._async.http_proxy - forward_request proxy_origin=(b'http', b'proxy-rwest-uk.energy.local', 8080) proxy_headers=[(b'proxy-authorization', b'Basic =')] method=b'GET' url=(b'http', b'localhost', 9060, b'/api/status') 2020-10-05 16:28:34,025 P[47892] [MainThread 33532] forward_request proxy_origin=(b'http', b'proxy-rwest-uk.energy.local', 8080) proxy_headers=[(b'proxy-authorization', b'Basic =')] method=b'GET' url=(b'http', b'localhost', 9060, b'/api/status') TRACE [2020-10-05 16:28:34] httpcore._async.connection_pool - reusing idle http11 connection=<AsyncHTTPConnection http_version=HTTP/1.1 state=4> 2020-10-05 16:28:34,026 P[47892] [MainThread 33532] reusing idle http11 connection=<AsyncHTTPConnection http_version=HTTP/1.1 state=4> TRACE [2020-10-05 16:28:34] httpcore._async.connection - connection.arequest method=b'GET' url=(b'http', b'proxy-rwest-uk.energy.local', 8080, b'http://localhost:9060/api/status') headers=[(b'proxy-authorization', b'Basic ='), (b'host', b'localhost:9060'), (b'x-tr-applicationid', b'5dee7c58886b4ddb939898c91e5641acc339a40a')] 2020-10-05 16:28:34,028 P[47892] [MainThread 33532] connection.arequest method=b'GET' url=(b'http', b'proxy-rwest-uk.energy.local', 8080, b'http://localhost:9060/api/status') headers=[(b'proxy-authorization', b'Basic ='), (b'host', b'localhost:9060'), (b'x-tr-applicationid', b'5dee7c58886b4ddb939898c91e5641acc339a40a')] TRACE [2020-10-05 16:28:34] httpcore._async.http11 - send_request method=b'GET' url=(b'http', b'proxy-rwest-uk.energy.local', 8080, b'http://localhost:9060/api/status') headers=[(b'proxy-authorization', b'Basic ='), (b'host', b'localhost:9060'), (b'x-tr-applicationid', b'5dee7c58886b4ddb939898c91e5641acc339a40a')]


For the rest of the code i get (I've removed some of the personally identifiable authentication stuff):

2020-10-05 16:29:08,350 P[47892] [MainThread 33532] entity: DataGrid_StandardAsync 2020-10-05 16:29:08,350 P[47892] [MainThread 33532] payload: {'requests': [{'instruments': ['CL'], 'fields': [{'name': 'Close'}]}]} 2020-10-05 16:29:08,351 P[47892] [MainThread 33532] Request:{'Entity': {'E': 'DataGrid_StandardAsync', 'W': {'requests': [{'instruments': ['CL'], 'fields': [{'name': 'Close'}]}]}}} 2020-10-05 16:29:08,352 P[47892] [MainThread 33532] Request to http://localhost:9060/api/v1/data
    headers = {'Content-Type': 'application/json', 'x-tr-applicationid': ''}     params = None TRACE [2020-10-05 16:29:08] httpcore._async.connection_pool - removing connection from pool=<AsyncHTTPConnection http_version=HTTP/1.1 state=4> 2020-10-05 16:29:08,353 P[47892] [MainThread 33532] removing connection from pool=<AsyncHTTPConnection http_version=HTTP/1.1 state=4> TRACE [2020-10-05 16:29:08] httpcore._async.http_proxy - forward_request proxy_origin=(b'http', b'proxy-rwest-uk.energy.local', 8080) proxy_headers=[(b'proxy-authorization', b'Basic =')] method=b'POST' url=(b'http', b'localhost', 9060, b'/api/v1/data') 2020-10-05 16:29:08,354 P[47892] [MainThread 33532] forward_request proxy_origin=(b'http', b'proxy-rwest-uk.energy.local', 8080) proxy_headers=[(b'proxy-authorization', b'Basic =')] method=b'POST' url=(b'http', b'localhost', 9060, b'/api/v1/data') TRACE [2020-10-05 16:29:08] httpcore._async.connection_pool - adding connection to pool=<AsyncHTTPConnection http_version=UNKNOWN state=0> 2020-10-05 16:29:08,355 P[47892] [MainThread 33532] adding connection to pool=<AsyncHTTPConnection http_version=UNKNOWN state=0> TRACE [2020-10-05 16:29:08] httpcore._async.connection - open_socket origin=(b'http', b'proxy-rwest-uk.energy.local', 8080) timeout={'connect': None, 'read': None, 'write': None, 'pool': None} 2020-10-05 16:29:08,356 P[47892] [MainThread 33532] open_socket origin=(b'http', b'proxy-rwest-uk.energy.local', 8080) timeout={'connect': None, 'read': None, 'write': None, 'pool': None} TRACE [2020-10-05 16:29:08] httpcore._async.connection - create_connection socket=<httpcore._backends.asyncio.SocketStream object at 0x000002A989FF5C10> http_version='HTTP/1.1' 2020-10-05 16:29:08,366 P[47892] [MainThread 33532] create_connection socket=<httpcore._backends.asyncio.SocketStream object at 0x000002A989FF5C10> http_version='HTTP/1.1' TRACE [2020-10-05 16:29:08] httpcore._async.connection - connection.arequest method=b'POST' url=(b'http', b'proxy-rwest-uk.energy.local', 8080, b'http://localhost:9060/api/v1/data') headers=[(b'proxy-authorization', b'Basic ='), (b'host', b'localhost:9060'), (b'content-type', b'application/json'), (b'x-tr-applicationid', b'5dee7c58886b4ddb939898c91e5641acc339a40a'), (b'content-length', b'120')] 2020-10-05 16:29:08,367 P[47892] [MainThread 33532] connection.arequest method=b'POST' url=(b'http', b'proxy-rwest-uk.energy.local', 8080, b'http://localhost:9060/api/v1/data') headers=[(b'proxy-authorization', b'Basic ='), (b'host', b'localhost:9060'), (b'content-type', b'application/json'), (b'x-tr-applicationid', b'5dee7c58886b4ddb939898c91e5641acc339a40a'), (b'content-length', b'120')] TRACE [2020-10-05 16:29:08] httpcore._async.http11 - send_request method=b'POST' url=(b'http', b'proxy-rwest-uk.energy.local', 8080, b'http://localhost:9060/api/v1/data') headers=[(b'proxy-authorization', b'Basic ='), (b'host', b'localhost:9060'), (b'content-type', b'application/json'), (b'x-tr-applicationid', b'5dee7c58886b4ddb939898c91e5641acc339a40a'), (b'content-length', b'120')] 2020-10-05 16:29:08,368 P[47892] [MainThread 33532] send_request method=b'POST' url=(b'http', b'proxy-rwest-uk.energy.local', 8080, b'http://localhost:9060/api/v1/data') headers=[(b'proxy-authorization', b'Basic ='), (b'host', b'localhost:9060'), (b'content-type', b'application/json'), (b'x-tr-applicationid', b'5dee7c58886b4ddb939898c91e5641acc339a40a'), (b'content-length', b'120')] TRACE [2020-10-05 16:29:08] httpcore._async.http11 - send_data=Data(<120 bytes>) 2020-10-05 16:29:08,369 P[47892] [MainThread 33532] send_data=Data(<120 bytes>) DEBUG [2020-10-05 16:29:08] httpx._client - HTTP Request: POST http://localhost:9060/api/v1/data "HTTP/1.1 502 cannotconnect" 2020-10-05 16:29:08,387 P[47892] [MainThread 33532] HTTP Request: POST http://localhost:9060/api/v1/data "HTTP/1.1 502 cannotconnect" TRACE [2020-10-05 16:29:08] httpcore._async.http11 - receive_event=Data(<5707 bytes>) 2020-10-05 16:29:08,388 P[47892] [MainThread 33532] receive_event=Data(<5707 bytes>) TRACE [2020-10-05 16:29:08] httpcore._async.http11 - receive_event=Data(<10220 bytes>) 2020-10-05 16:29:08,389 P[47892] [MainThread 33532] receive_event=Data(<10220 bytes>) TRACE [2020-10-05 16:29:08] httpcore._async.http11 - receive_event=Data(<2638 bytes>) 2020-10-05 16:29:08,393 P[47892] [MainThread 33532] receive_event=Data(<2638 bytes>) TRACE [2020-10-05 16:29:08] httpcore._async.http11 - receive_event=EndOfMessage(headers=[]) 2020-10-05 16:29:08,394 P[47892] [MainThread 33532] receive_event=EndOfMessage(headers=[]) TRACE [2020-10-05 16:29:08] httpcore._async.http11 - response_closed our_state=DONE their_state=DONE 2020-10-05 16:29:08,395 P[47892] [MainThread 33532] response_closed our_state=DONE their_state=DONE 2020-10-05 16:29:08,397 P[47892] [MainThread 33532] HTTP request response 502: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

pierre.faurel avatar image pierre.faurel

In your traces, the request is not complete : Authorization header is missing.

2020-10-05 16:29:08,352 P[47892] [MainThread 33532] Request to http://localhost:9060/api/v1/data
    headers = {'Content-Type': 'application/json', 'x-tr-applicationid': ''}     params = None 

Whereas request headers should be:

2020-10-06 23:57:09,942 P[42552] [MainThread 35360] Request to http://127.0.0.1:9060/api/v1/data
    headers = {'Content-Type': 'application/json', 'x-tr-applicationid': 'xxxxxxxxxxxxxxxxxxx', 'Authorization': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'}

=> Did you remove it ?

If not, that means that token wasn't retrieve on handshake request.
=> This handshake is not in your trace. Could you share it ?

This is log between following following logged messages.

2020-10-06 23:57:07,472 P[42552] [MainThread 35360] Try to handshake on url http://localhost:9060/api/handshake...
...
2020-10-06 23:57:08,208 P[42552] [MainThread 35360] HTTP request response 200: {"access_token":"xxxxxxxxxxxxxxxxxxxxxx","expires_in":1209600,"token_type":"bearer"}

According to your output, it seems that your machine is configured with a proxy.=> => Could check this ?

=> if it's possible, could you disabled it for test ?

Upvotes
gibran.afzal1 avatar image
1 ●0 ●0 ●0

Hi

Yes we have a corporate proxy - unfortunately there is no way to remove it as without it I can't reach the internet.

I searched for the string 'Try to handshake' and could not see it.

Is there a way to pass in a proxy to the eikon library?

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

pierre.faurel avatar image pierre.faurel

It seems that the proxy intercepts request sent to localhost.

It'll be interesting to check "ping localhost" response in a console window.

Is it possible for you to confifure the proxy manually as below ?

You can keep empty Address and Port, and just add localhost; 127.0.01 in the exception list.
Checkbox "Don't use the proxy server for local address" could be optional but can you can test with and without ?

1602070970818.png (307.6 KiB)
Click below to post an Idea Post Idea