For a deeper look into our Eikon Data API, look into:

Overview |  Quickstart |  Documentation |  Downloads |  Tutorials |  Articles

question

Upvotes
Accepted
32 5 4 2

A CSV download via the API scheduled every work day fails at least once per week with no apparent cause. How do I fix it?

#WARNING: the Eikon terminal must be running in order to connect to the API

#import packages
import eikon as ek # the Eikon Python wrapper package
import pandas as pd
import numpy as np
import datetime
from datetime import timedelta, date, datetime
from pandas.tseries.offsets import BDay
 
#connects to Bill's Eikon terminal
ek.set_app_key('XXXX')
 
#retreive the RICs from Eikon
df_rics,e = ek.get_data("lists('Inv Trust List')","TR.RIC")
 
#convert that into a list and set as an object
ric_list = df_rics['Instrument'].tolist()
 
#Slice, loop and concatenate the retrieval request - must be done in order to avoid a backend request timeout which
 #happens when we use too many RICs. n can be toggled below. 
n = 25
 df = pd.DataFrame()
for ric_chunk in [ric_list[i:i + n] 
                  for i in range(0, len(ric_list), n)]:
    tmp_df, e = ek.get_data(ric_chunk,
                    ['TR.RNSFilerName',
                     'TR.RNSAnnouncedDate',
                     'TR.RNSTransactionType',
                     'TR.RNSARNumShrsTransacted',
                     'TR.RNSARPctOSTransacted',
                     'TR.RNSARTransactionPrice',
                     'TR.RNSARMktValTransaction',
                     'TR.RNSARTotShrsPostTrans',
                     'TR.RNSARPctOSPostTrans'])
    df = tmp_df.append(df)
 
#set boundary dates of the download
end_date = date.today()
 start_date = end_date - BDay(2)
 
#set those dates in the necessary string format
end_date_str = datetime.strftime(end_date, "%Y-%m-%d")
 start_date_str = datetime.strftime(start_date, "%Y-%m-%d")
 
#denote dates in datetime format
df['RNS Announced Date'] = pd.to_datetime(df['RNS Announced Date'])
 
#set the date constraints as an object
mask = (df['RNS Announced Date'] >= start_date_str) & (df['RNS Announced Date'] < end_date_str)
 
#set the dataframe as only those with values inside the boundary dates
df = df.loc[mask]
 
#rename some of columns headings, some for compatibility purposes - some programs don't like the '£' symbol
df.rename(columns={'RNS Announced Date': 'Transaction Date','RNS AR Price (at Transaction) - £': 'RNS AR Price (at Transaction) GBP',
                    'RNS AR Market Value of Transaction - £': 'RNS AR Market Value of Transaction - GBP'},
           inplace=True)
 
#create file name and export as CSV
todays_date = date.today()
 todays_date_str = datetime.strftime(todays_date, "%Y%m%d")
 df.to_csv('Daily API Download_' + todays_date_str + '.csv')
eikoneikon-data-apiworkspaceworkspace-data-apirefinitiv-dataplatform-eikonpythonapi
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

This is the error


 
                  

Error code 503 | Server Error: API Proxy is not available

Traceback (most recent call last):

File "daily_api_download.py", line 15, in <module>

df_rics,e = ek.get_data("lists('Inv Trust List')","TR.RIC")

File "C:\Users\william\AppData\Local\Continuum\anaconda3\lib\site-packages\eikon\data_grid.py", line 186, in get_data

result = eikon.json_requests.send_json_request(_endpoint, payload, debug=debug)

File "C:\Users\william\AppData\Local\Continuum\anaconda3\lib\site-packages\eikon\json_requests.py", line 121, in send_json_request

_raise_for_status(response)

File "C:\Users\william\AppData\Local\Continuum\anaconda3\lib\site-packages\eikon\json_requests.py", line 244, in _raise_for_status

raise EikonError(response.status_code, error_msg)

eikon.eikonError.EikonError: Error code 503 | Server Error: API Proxy is not available

UPDATE

Hi @zoya.farberov, as explained on another post, I have been logging my errors and there are three scenarios that occur:

1) download works (happens c.25% of the time),

2) download fails because Eikon Terminal is not open at the time of running (happens c.25% of the time) -this is a Windows Task Scheduling error and not one I need help fixing,

3) download fails and I get the error code attached (happens 50% of the time)

Please would you help me resolve scenario 3. The error code that is generated is attached. ThanksError log.zip

error-log.zip (660 B)

Hi @zoya.farberov , please would you get back to me on this? Many thanks

Hi @pierre.faurel,

Are you able to take a look at the follow-up information on the issue and the logs from @bill39?

Thanks


Show more comments
Upvote
Accepted
4.3k 2 4 5

Hi @bill39

The two fixes I proposed are workaround to detect errors and manage it when it's possible (that's what I named "secure your code").

1) Check that session is started and available with this code:

cnx_state = ek.get_desktop_session().get_open_state()
if cnx_state != ek.Session.State.Open:
     exit(-1)

If the cnx_state is equal to ek.Session.State.Closed, that means Eikon API Proxy is not avalaible (need to restart Eikon ?).

2) Catch 2504 error code to request again

    while retry:
        try:
            tmp_df, e = ek.get_data(ric_chunk,                             ['TR.RNSFilerName',
                ... 
               'TR.RNSARPctOSPostTrans'])
            retry = False
        except EikonError as e:
            if e.code not in [2504]:
                retry = False
            else:
                time.sleep(0.5) # delay retry in 500 msec

Note that you can add other error code than 2504 in the list to catch (ex: 400).

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Hi @pierre.faurel, please would you help me integrate this into my code? Firstly for part 1), I am getting

AttributeError: module 'eikon' has no attribute 'get_desktop_session'

and secondly I have already incorporated an error catching code to stop error code 400 (which I don't understand either) following guidance from one of your colleagues. I don't know how I should be incorporating the two together, unfortunately now this has gotten beyond my skill level - I'm a stock analyst by trade, not a programmer! Please help. Many thanks. Ideally we would set up a remote session on my computer where we can go through it and amend the code as necessary.

Upvotes
23k 22 9 14

Hello @bill39,

I understand that the cause of the issue is always error 503? And you seem to be seeing the error consistently, as opposite to once or twice? If this is correct, this is not quite the expected behavior.

Please see this relevant previous discussion, the Eikon troubleshooting guide is very often a key tool to determine the specific cause for the issue, hopefully, helpful to you.

Some other relevant factors are: are you on the recent version of Eikon? Are you on the latest version of the Eikon library, as some important fixes have been released in version of 1.1.6.post2. The issues fixed also appear to be tracked to python 3.8+, if you are on python 3.8+, the use of 1.1.6.post2 is recommended.

Hope that this info is of help.



icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Yes 503 is always the error. I have python version 3.8.+. For some reason the troubleshooting guide link you have provided directs me to a page that 'cannot be found'. Please would you email me the troubleshooting instructions for error code 503? Thanks

Hello @bill39,

Please try again- the article link should work now.

The dev portal went through a major update today and some links were migrated.

Thanks for letting us know

The API request ran today when I executed it manually but it didn't run when run by my Task Scheduler early this morning. What could be the cause of the requests sometimes working and sometimes not? And how do I fix it? Thank you

Hello @bill39,

Does it run via scheduler in general for you? If you schedule it in 1 min, does it run?

Eikon/Refinitiv Workspace desktop needs to be up, running and logged in, Eikon Proxy within it up and running as part of it, for Eikon Data API python to interact with it. If that is not the case, the task will not run successfully. By default, Eikon desktop is logged out at night, do you find yourself logging back in, in the morning?

If I schedule it in 1 minute it does sometimes work but not always. I.e. some days it will run every time I try and other days it does not. I have also set a task to open the Eikon terminal every morning before the API download so it is always on when the API runs, yes. I also run a macro in an excel file every morning that uses Eikon data and sometimes this fails also. When this macro fails the buyback also fails but there are times when the macro runs and the API download fails.

Upvotes
23k 22 9 14

Hello @bill39,

As you mention that, for the most part, the scheduled task succeeds, and sometimes it does not, I think it would be helpful to collect any error or info at that time that it fails. Try introducing

import eikon as ek
...
import logging.config 
ek.set_log_level(logging.DEBUG)
ek.set_app_key(XXXXXX

please collect and share any info that is logged at the time of the failed call?

Let's see if we can find out more to help us track down the issue.

---

Would like to also note, that the resulting CSV content is subject to Eikon, Eikon Excel and any data obtained via Eikon Data API single user license, to be used by logged in Eikon user only. While sharing of the content would require Enterprise level product and license.


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Noted, thanks. Thanks, I have added the logging lines to my code. Where will the error log be stored? When running the script manually today the error I receive is:


Traceback (most recent call last):

File "daily_api_download.py", line 39, in <module>

'TR.RNSARPctOSPostTrans'])

File "C:\Users\william\AppData\Local\Continuum\anaconda3\lib\site-packages\eikon\data_grid.py", line 186, in get_data

result = eikon.json_requests.send_json_request(_endpoint, payload, debug=debug)

File "C:\Users\william\AppData\Local\Continuum\anaconda3\lib\site-packages\eikon\json_requests.py", line 118, in send_json_request

_check_server_error(result)

File "C:\Users\william\AppData\Local\Continuum\anaconda3\lib\site-packages\eikon\json_requests.py", line 194, in _check_server_error

raise EikonError(int(server_response['ErrorCode']), error_message)

eikon.eikonError.EikonError: Error code 2504 | UDF Core request failed. Gateway Time-out

_______________________________________


Thanks

Hi,

error code 2504 is something you can catch to retry to send your request.

You can update your code as below:

import time

...

for ric_chunk in [ric_list[i:i + n]
    retry = True
    while retry:
        try:
            tmp_df, e = ek.get_data(ric_chunk,
                            ['TR.RNSFilerName',
                             'TR.RNSAnnouncedDate',
                             'TR.RNSTransactionType',
                             'TR.RNSARNumShrsTransacted',
                             'TR.RNSARPctOSTransacted',
                             'TR.RNSARTransactionPrice',
                             'TR.RNSARMktValTransaction',
                             'TR.RNSARTotShrsPostTrans',
                             'TR.RNSARPctOSPostTrans'])
            retry = False
        except EikonError as e:
            if e.code not in [2504]:
                retry = False
            else:
                time.sleep(0.5) # delay retry in 500 msec
    df = tmp_df.append(df)

For the Error code 503 | Server Error: API Proxy is not available, if you continue to reproduce (a fix with eikon 1.1.6.post3 was published on pypi), as you trigger your script to run daily, that's possible that Eikon Proxy turned down.

You can secure your script with :

ek.set_app_key('XXXX')

cnx_state = ek.get_desktop_session().get_open_state()
if cnx_state != ek.Session.State.Open:
    exit(-1)


How would you suggest I integrate that into my current code? It already splits my retreival into chunks like this:


n = 10
 df = pd.DataFrame()
for ric_chunk in [ric_list[i:i + n] 
                  for i in range(0, len(ric_list), n)]:
    tmp_df, e = ek.get_data(ric_chunk,
                    ['TR.RNSFilerName',
                     'TR.RNSAnnouncedDate',
                     'TR.RNSTransactionType',
                     'TR.RNSARNumShrsTransacted',
                     'TR.RNSARPctOSTransacted',
                     'TR.RNSARTransactionPrice',
                     'TR.RNSARMktValTransaction',
                     'TR.RNSARTotShrsPostTrans',
                     'TR.RNSARPctOSPostTrans'])
    df = tmp_df.append(df)

Additionally, what do you mean by "securing my script"? What does that code do?

Thank you

Show more comments
Click below to post an Idea Post Idea