question

Upvotes
Accepted
89 2 6 9

cut file, file 0 kb

Please tell me the answer to who worked with such problems

Using an API example to access Tick History

For example: RIC ESH1 (raw market-by-price)

Something does not download the full contract from the start date to the current date.

Cuts the file size.

How to download the contract completely so that the dates inside the file do not cut?

So for example, why 1UROF0, 1UROF1, 1UROF8, 1UROF9, 1UROG0, 1UROG1 file size 0 kb

Why?

code:

using System;
using System.Linq;
using System.Diagnostics;
using System.IO;
using ThomsonReuters.Dss.Api.StandardExtractions;
using ThomsonReuters.Dss.Api;
using ThomsonReuters.Dss.Api.Content;
using ThomsonReuters.Dss.Api.Extractions;
using ThomsonReuters.Dss.Api.Extractions.ExtractionRequests;
using ThomsonReuters.Dss.Api.Extractions.ReportExtractions;
using ThomsonReuters.Dss.Api.Extractions.ReportTemplates;
using ThomsonReuters.Dss.Api.Extractions.Schedules;
using ThomsonReuters.Dss.Api.Extractions.SubjectLists;
using ThomsonReuters.Dss.Api.Search;
using ThomsonReuters.Dss.Core.RestApi;

namespace RefinitivHistoryTesting
{
    class Program
    {
        static void Main(string[] args)
        {
            //-----------------------------------------------------------------
            //Connect and authenticate to the DSS server:
            //-----------------------------------------------------------------
            Uri dssUri = new Uri("https://hosted.datascopeapi.reuters.com/RestApi/v1/");
            string dssUserName = "****";
            string dssUserPassword = "*****";

            var out_dir = @".\Storage\";
            var SearchContext = new SearchContext(dssUri, dssUserName, dssUserPassword);
            var ExtractionsContext = new ExtractionsContext(dssUri, dssUserName, dssUserPassword);
            var availableMbPFields = ExtractionsContext.GetValidContentFieldTypes(ReportTemplateTypes.TickHistoryRaw);
            var availableTnSFields = ExtractionsContext.GetValidContentFieldTypes(ReportTemplateTypes.TickHistoryTimeAndSales);


            ExtractionsContext.Options.AutomaticDecompression = true;

            var dateRange = DateTimeRange.Create(
                new DateTimeOffset(2008, 01, 01, 0, 0, 0, TimeSpan.Zero),
                new DateTimeOffset(2020, 12, 22, 0, 0, 0, TimeSpan.Zero));

            var request = new HistoricalChainResolutionRequest
            {
                ChainRics = new[] { "0#1URO:" },//{ "0#ESH1m" },//{ "0#1URO:" },
                Range = dateRange
            };

            var chain_results = SearchContext.HistoricalChainResolution(request);
            foreach (var chain in chain_results)
            {
                Console.WriteLine("Chain Identifier: {0} {1}, Constituents: {2}", chain.IdentifierType, chain.Identifier, chain.Constituents.Count());
                foreach (var constituent in chain.Constituents)
                {
                    var searchResults = SearchContext.HistoricalSearch(
                        HistoricalSearchRequest.Create(
                            constituent.Identifier, constituent.IdentifierType, dateRange, HistoricalResultsBy.Ric, null));

                    //foreach (var result in from pos in searchResults where pos.DomainCode == "8" select pos)
                    foreach (var result in searchResults) // выкачивает 8 и 6, если имя контракта или все файлы если chain

                    //var result = searchResults.FirstOrDefault(pos => pos.Identifier.Contains("1UROF0") &&pos.DomainCode=="8");
                    //if (result != null)
                    {
                        var startDate = result.FirstDate;//new DateTimeOffset(2021, 01, 01, 0, 0, 0, TimeSpan.Zero);//result.FirstDate;
                        var lastDate = result.LastDate;//new DateTimeOffset(2021, 03, 05, 0, 0, 0, TimeSpan.Zero);//result.LastDate;
                        Console.WriteLine("{0},\t {1},\t {2},\t {3},\t {4}", result.Identifier, result.IdentifierType, startDate, lastDate, result.DomainCode);
                        
                        var extractionResult = ExtractionsContext.ExtractRaw(

                                //!!!--- Это должно соответствовать модели данных Real Time API MarketByPrice---!!!
                                new TickHistoryRawExtractionRequest
                                {
                                    Condition = new TickHistoryRawCondition
                                    {
                                        ReportDateRangeType = ReportDateRangeType.Range,
                                        QueryStartDate = startDate, //new DateTimeOffset(2016, 07, 25, 20, 0, 0, TimeSpan.Zero),
                                        QueryEndDate = lastDate, //new DateTimeOffset(2016, 08, 02, 19, 59, 59, TimeSpan.Zero),
                                        ExtractBy = TickHistoryExtractByMode.Ric,
                                        MessageTimeStampIn = TickHistoryTimeOptions.GmtUtc,
                                        SortBy = TickHistorySort.SingleByRic,
                                        DomainCode = TickHistoryRawDomain.MarketByPrice,
                                        DisplaySourceRIC = true
                                    },
                                    IdentifierList = new InstrumentIdentifierList
                                    {
                                        InstrumentIdentifiers = new[]
                                        {
                                            InstrumentIdentifier.Create(result.IdentifierType, result.Identifier)
                                        }
                                    },

                                });

                        using (var response = ExtractionsContext.RawExtractionResultOperations.GetReadStream(extractionResult))
                        using (var reader = new StreamReader(response.Stream))
                        {
                            var fileName = out_dir + "_" + result.Identifier + "_" + startDate.DateTime.ToString("dd.MM.yyyy") + "_" + lastDate.DateTime.ToString("dd.MM.yyyy") + "_MarketByPrice.txt";
                            using (var fileStream = File.Create(fileName))
                                response.Stream.CopyTo(fileStream);
                        }
                        
                    }
                }
            }

            Console.WriteLine("Press Enter to exit");
            Console.ReadLine();
        }
    }

}
tick-history-rest-api
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvote
Accepted
23k 22 9 14

Hello @s61670,

I just looked into one RIC, "1UROF0". To some it up, I think this will require an input from an RTH content expert, as not all instruments in all markets have Raw history stored.

1. HistoricalChainResolution Chain "0#1URO:" expands, RIC "1UROF0" is present

2. HistoricalSearch Ric 1UROF0,

{
  "Request": {
    "Identifier": "1UROF0",
    "IdentifierType": "Ric",
    "Range": {
      "Start": "2008-01-01T00:00:00.000Z",
      "End": "2021-01-01T00:00:00.000Z"
    }
  }
}

results:

"value": [
        {
            "Identifier": "1UROF0",
            "IdentifierType": "Ric",
            "Source": "",
            "Key": "VjF8MHgzMDAwMDAwMDAwMDAwMDAwfDB4MzAwMDAwMDAwMDAwMDAwMHx8fHx8fHwxVVJPRjB8",
            "Description": "Historical Instrument",
            "InstrumentType": "Unknown",
            "Status": "Valid",
            "DomainCode": "6",
            "FirstDate": "2019-08-17T00:00:00.000Z",
            "LastDate": "2020-01-25T00:00:00.000Z",
            "History": []
        },
        {
            "Identifier": "1UROF0",
            "IdentifierType": "Ric",
            "Source": "",
            "Key": "VjF8MHgzMDAwMDAwMDAwMDAwMDAwfDB4MzAwMDAwMDAwMDAwMDAwMHx8fHx8fHwxVVJPRjB8",
            "Description": "Historical Instrument",
            "InstrumentType": "Unknown",
            "Status": "Valid",
            "DomainCode": "8",
            "FirstDate": "2019-08-05T00:00:00.000Z",
            "LastDate": "2020-01-25T00:00:00.000Z",
            "History": []
        }

3. Therefore, we can request raw history "1UROF0":

{
  "ExtractionRequest": {
    "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.TickHistoryRawExtractionRequest",
    "IdentifierList": {
      "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",  
      "InstrumentIdentifiers": [{
        "Identifier": "1UROF0",
        "IdentifierType": "Ric"
      }] ,
      "ValidationOptions": null,
      "UseUserPreferencesForValidationOptions": false
    },
    "Condition": {
      "MessageTimeStampIn": "GmtUtc",
      "ReportDateRangeType": "Range",
      "QueryStartDate": "2019-08-05T00:00:00.000Z",
      "QueryEndDate": "2020-01-25T00:00:00.000Z",
      "ExtractBy": "Ric",
      "SortBy": "SingleByRic",
      "DomainCode": "MarketPrice",
      "DisplaySourceRIC": true
    }
  }
}

That results in:

{
    "@odata.context": "https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#RawExtractionResults/$entity",
    "JobId": "0x077752f443174f26",
    "Notes": [
        "All identifiers were invalid.  No extraction performed."
    ],
    "IdentifierValidationErrors": [
        {
            "Identifier": {
                "@odata.type": "#ThomsonReuters.Dss.Api.Content.InstrumentIdentifier",
                "Identifier": "1UROF0",
                "IdentifierType": "Ric",
                "Source": ""
            },
            "Message": "Not found"
        }
    ]
}

It is likely that only some data for this instrument (and other instruments from your list) is made available as Raw history. There is a possibility that you have run into a content gap that is not intentional. However, to tell for sure, would need an RTH content expert, and this forum is mainly comprised of developers, the members of the forum are developers working with Refinitiv APIs, the moderators are Refinitiv API experts.

For a content question such as this, the best approach for a Refinitiv customer is to submit it directly, to Refinitiv Helpdesk Online -> Tick History, or by dialing Refinitiv Helpdesk local number.

In this case, we are going to refer to content experts for investigation, and get back to you and the community here.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
89 2 6 9

The screen shows that the file is being cut.

Full RIC history is not downloaded

There is no current date, the number of lines is more than 13 million.

Other RICs are downloaded empty without data. What could be the problem?

0 kb.PNG

2.PNG


0-kb.png (231.9 KiB)
2.png (285.4 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Click below to post an Idea Post Idea