For a deeper look into our DataScope Select SOAP API, look into:

Overview |  Quickstart |  Documentation |  Downloads |  Tutorials

question

Upvotes
Accepted
Md.Hisham.TR avatar image
5 ●0 ●0 ●2

Downloading compressed file on TRTH

I run into out of memory exception when I add new lines to response body by "using (var gzip = new GZipInputStream(streamResponse.Stream))" after "var streamResponse = ExtractionsContext.GetReadStream(extractionResult);".

I tried to get L1 data for one day for two chain ric "0#HSI:", "0#HCEI:" and got the above issue

is it possible that I just download the compressed file only? How can I do that? I will decompress the file in later stages.

Or is there a better option regarding my above issue?


It does work if I submit a request with a smaller number of identifiers. But this is not the ideal solution.

Though I will not run into “out of memory” exception, the needed time is still way too long.

dss-rest-apidatascope-selectdsstick-history-rest-api
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

1 Answer

Upvotes
Accepted
Jirapongse avatar image
38.1k ●71 ●35 ●53

@Mohamed.Hisham

Please refer to the .Net SDK Tutorial 5: On Demand extraction, file IO tutorial.

It demonstrates how to set AutomaticDecompression to false and save the data in compressed format.

//Direct download from AWS ?
if (awsDownload) { extractionsContext.DefaultRequestHeaders.Add("x-direct-download", "true"); };

DssStreamResponse streamResponse = extractionsContext.GetReadStream(extractionResult);
using (FileStream fileStream = File.Create(dataOutputFile))
    streamResponse.Stream.CopyTo(fileStream);

//Reset header after direct download from AWS ?
if (awsDownload) { extractionsContext.DefaultRequestHeaders.Remove("x-direct-download"); };

Console.WriteLine("Saved the compressed data file to disk:\n" + gzipDataOutputFile);
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Click below to post an Idea Post Idea