question

Upvotes
Accepted
BenL avatar image
1 ●0 ●0 ●1

Extractions/RawExtractionResults AWS download - getting 200 OK with data rather than 302 redirect when using - X-Direct-Download

Hi,

I'm using the Extractions/RawExtractionResults method with X-Direct-Download set to true to try to get a 302 redirect to the AWS S3 link. I have tried this using Postman and .Net Core. I have followed the guides at:

private HttpClient Client = new HttpClient(new HttpClientHandler
        {
            AllowAutoRedirect = false,
        });

var uri = new Uri($"https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/RawExtractionResults('{job.JobId}')/$value");

var request = new HttpRequestMessage(HttpMethod.Get, uri);
request.Headers.Add("Authorization", $"Token {authToken}");
request.Headers.Add("Prefer", "respond-async");
request.Headers.Add("X-Direct-Download", "true");

var result = await this.Client.SendAsync(request);

I expect result to have a 302 status code with the redirect to the S3 bucket in the header but it always has a 200 with the actual data. I have also tried using postman with automatic redirects turned off.

UPDATE

I want to clarify what I am trying to achieve and the steps I am following:

1) Get Authentication token using /Authentication/RequestToken endpoint and username/password - no issues.

2) Use the token to get a list of completed jobs from the /Jobs/JobGetCompleted endpoint - no issues.

3) Use a job ID obtained in step 2) to get the extraction data from using the Extractions/RawExtractionResults endpoint. This is where I am having issues. The endpoint gives a 200 response and returns the data which is what I would expect. The problem is when I try to use the "X-Direct-Download" header. I still get a 200 response and the data rather than a 302 redirect.

This happens when auto redirects are turned off in Postman and when redirects are disabled in C# as in the code sample above.


tick-history-rest-apipostmanaws
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

4 Answers

Upvotes
Accepted
Jirapongse avatar image
38.1k ●71 ●35 ●53

@BenL

Refer to the RefinitivTick History User Guide, AWS download supports for Tick History Time and Sales, Tick History Market Depth, Tick History Intraday Summaries, and Tick History Raw reports.

Please check if that job is in the AWS-supported templates and if it is a scheduled or on-demand extraction.




icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
zoya faberov avatar image
23k ●22 ●9 ●14

Hello @BenL ,

Believe you should be able to do work it this way.

I have just tested with Postman, the quickest for me, following the relevant section in article Boost Tick History downloads with AWS

1. Changing the setting:

redirectoff.gif

For my test, I have used RTH Tutorial 4.

2. 4b - Requested extraction - status was 202

3. 4c - Checked status, till status became 200

4. 4d - Requested get tick data, but adding AWS download now:

xdirect.gif

I think the last step is where the difference may be, and hope this helps


redirectoff.gif (22.9 KiB)
xdirect.gif (55.5 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

BenL avatar image BenL
Sorry, I feel like I have followed all the steps and am still getting a 200 rather than a 302. Is there an example Job ID you can share?
Upvotes
zoya faberov avatar image
23k ●22 ●9 ●14

Hello @BenL

Sharing the jobId will not help, I believe, as you will not be able to access my job, only yours.

Can we try the following:

Is your Postman set to disable re-direct as described in the article and my previous answer?

Do you have any job, that you have created, has completed, and are able to get result of currently, from Postman successfully, or please create one?

Next, I am able to run request:

{
                {protocol}}{
                {host}}{
                {api}}Extractions/RawExtractionResults('{
                {jobId}}')/$value

When I run it without x-direct the result is 200 + data

without.gif

When I run it with x-direct the result is

with.gif

Can you please try the same, do you have a recent Postman, mine is not the very latest but is quite recent Postman v8.6.2, perhaps this could be the difference as well.

Let us know if you are able to reproduce?


without.gif (45.1 KiB)
with.gif (22.9 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

BenL avatar image BenL

I am getting the completed jobs from: https://hosted.datascopeapi.reuters.com/RestApi/v1/Jobs/JobGetCompleted

is this not correct?

Upvotes
zoya faberov avatar image
23k ●22 ●9 ●14

Hello @BenL ,

Let me step back, to make sure that I fully understand your requirement:

Do you require to obtain the list of the complete jobs? Then please use JobGetCompleted call to do this.

Are you interested in downloading a result of your job over AWS, and using x-direct-download header, but still failing to do so? Then please test via Postman as suggested to verify you are able to do so and let us know what you observe.

To better understand the semantics of DSS REST requesting, please refer to RTH REST tutorials , especially tutorials REST API Tutorials Introduction and REST API Tutorial 3: On Demand data extraction workflow.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

BenL avatar image BenL
Hi @zoya.farberov - I have updated my question explaining what I am trying to do. I get the list of completed jobs from JobGetCompleted and then what to use the job IDs returned to get an S3 download link.
Click below to post an Idea Post Idea