DownloaderDataProvider

QuantConnect.Lean.Engine.DataFeeds.DownloaderDataProvider

DownloaderDataProvider()

DownloaderDataProvider(data_downloader: IDataDownloader)

Bases: BaseDownloaderDataProvider

Data provider which downloads data using an IDataDownloader or IBrokerage implementation

Signature descriptions:

Creates a new instance
Creates a new instance using a target data downloader used for testing

new_data_request

new_data_request: _EventContainer[
    Callable[
        [Object, DataProviderNewDataRequestEventArgs], Any
    ],
    Any,
]

Event raised each time data fetch is finished (successfully or not)

fetch

fetch(key: str) -> Stream

Determines if it should downloads new data and retrieves data from disc

Parameters:

Name	Type	Description	Default
`key`	`str`	A string representing where the data is stored	required

Returns:

Type	Description
`Stream`	A Stream of the data requested.

filter_and_group_download_data_by_symbol

filter_and_group_download_data_by_symbol(
    download_data: List[BaseData],
    symbol: Union[Symbol, str, BaseContract],
    data_type: Type,
    exchange_time_zone: Any,
    data_time_zone: Any,
    downloader_start_time_utc: Union[datetime, date],
    downloader_end_time_utc: Union[datetime, date],
) -> Iterable[IGrouping[Symbol, BaseData]]

Filters and groups the provided download data by symbol, based on specified criteria.

Parameters:

Name	Type	Description	Default
`download_data`	`List[BaseData]`	The collection of download data to process.	required
`symbol`	`Union[Symbol, str, BaseContract]`	The symbol to filter the data for.	required
`data_type`	`Type`	The type of data to filter for.	required
`exchange_time_zone`	`Any`	The time zone of the exchange.	required
`data_time_zone`	`Any`	The desired time zone for the data.	required
`downloader_start_time_utc`	`Union[datetime, date]`	The start time of data downloading in UTC.	required
`downloader_end_time_utc`	`Union[datetime, date]`	The end time of data downloading in UTC.	required

Returns:

Type	Description
`Iterable[IGrouping[Symbol, BaseData]]`	An enumerable collection of groupings of download data, grouped by symbol.

get_downloaded_data

get_downloaded_data(
    downloader_data_parameters: List[
        DataDownloaderGetParameters
    ],
    symbol: Union[Symbol, str, BaseContract],
    exchange_time_zone: Any,
    data_time_zone: Any,
    data_type: Type,
) -> Iterable[IGrouping[Symbol, BaseData]]

Retrieves downloaded data grouped by symbol based on IDownloadProvider.

Parameters:

Name	Type	Description	Default
`downloader_data_parameters`	`List[DataDownloaderGetParameters]`	Parameters specifying the data to be retrieved.	required
`symbol`	`Union[Symbol, str, BaseContract]`	Represents a unique security identifier, generate by ticker name.	required
`exchange_time_zone`	`Any`	The time zone of the exchange where the symbol is traded.	required
`data_time_zone`	`Any`	The time zone in which the data is represented.	required
`data_type`	`Type`	The type of data to be retrieved. (e.g. Data.Market.TradeBar)	required

Returns:

Type	Description
`Iterable[IGrouping[Symbol, BaseData]]`	An IEnumerable containing groups of data grouped by symbol. Each group contains data related to a specific symbol.

get_stream

get_stream(key: str) -> Stream

Get's the stream for a given file path

This codeEntityType is protected.

need_to_download

need_to_download(file_path: str) -> bool

Main filter to determine if this file needs to be downloaded

This codeEntityType is protected.

Parameters:

Name	Type	Description	Default
`file_path`	`str`	File we are looking at	required

Returns:

Type	Description
`bool`	True if should download.

dispose

dispose() -> None

The stream created by this type is passed up the stack to the IStreamReader The stream is closed when the StreamReader that wraps this stream is disposed

on_new_data_request

on_new_data_request(
    e: DataProviderNewDataRequestEventArgs,
) -> None

Event invocator for the new_data_request event

This codeEntityType is protected.

download_once

download_once(
    key: str, download: Callable[[str], Any]
) -> Stream

Helper method which guarantees each requested key is downloaded only once concurrently if required based on need_to_download

This codeEntityType is protected.

Parameters:

Name	Type	Description	Default
`key`	`str`	A string representing where the data is stored	required
`download`	`Callable[[str], Any]`	The download operation we want to perform once concurrently per key	required

Returns:

Type	Description
`Stream`	A Stream of the data requested.