LeanData

QuantConnect.Util.LeanData

Bases: Object

Provides methods for generating lean data file content

SECURITY_TYPE_AS_DATA_PATH

SECURITY_TYPE_AS_DATA_PATH: HashSet[str]

The different SecurityType used for data paths

generate_line

generate_line(
    data: IBaseData,
    resolution: Resolution,
    exchange_time_zone: Any,
    data_time_zone: Any,
) -> str

generate_line(
    data: IBaseData,
    security_type: SecurityType,
    resolution: Resolution,
) -> str

Signature descriptions:

• Converts the specified base data instance into a lean data file csv line. This method takes into account the fake that base data instances typically are time stamped in the exchange time zone, but need to be written to disk in the data time zone.

• Converts the specified base data instance into a lean data file csv line

generate_relative_zip_file_path

generate_relative_zip_file_path(
    symbol: Union[Symbol, str, BaseContract],
    date: Union[datetime, date],
    resolution: Resolution,
    tick_type: TickType,
) -> str

generate_relative_zip_file_path(
    symbol: str,
    security_type: SecurityType,
    market: str,
    date: Union[datetime, date],
    resolution: Resolution,
) -> str

Generates the relative zip file path rooted in the /Data directory

generate_zip_file_name

generate_zip_file_name(
    symbol: Union[Symbol, str, BaseContract],
    date: Union[datetime, date],
    resolution: Resolution,
    tick_type: TickType,
) -> str

generate_zip_file_name(
    symbol: str,
    security_type: SecurityType,
    date: Union[datetime, date],
    resolution: Resolution,
    tick_type: Optional[TickType] = None,
) -> str

Signature descriptions:

• Generates the zip file name for the specified date of data.

• Creates the zip file name for a QC zip data file

generate_zip_file_path

generate_zip_file_path(
    data_directory: str,
    symbol: Union[Symbol, str, BaseContract],
    date: Union[datetime, date],
    resolution: Resolution,
    tick_type: TickType,
) -> str

generate_zip_file_path(
    data_directory: str,
    symbol: str,
    security_type: SecurityType,
    market: str,
    date: Union[datetime, date],
    resolution: Resolution,
) -> str

Generates the full zip file path rooted in the data_directory

get_daily_calendar

get_daily_calendar(
    exchange_time_zone_date: Union[datetime, date],
    exchange: SecurityExchange,
    extended_market_hours: bool,
) -> CalendarInfo

get_daily_calendar(
    exchange_time_zone_date: Union[datetime, date],
    exchange_hours: SecurityExchangeHours,
    extended_market_hours: bool,
) -> CalendarInfo

Helper method to return the start time and period of a bar the given point time should be part of

Parameters:

Name	Type	Description	Default
exchange_time_zone_date	Union[datetime, date]	The point in time we want to get the bar information about	required
exchange	Optional[SecurityExchange]	The associated security exchange	None
extended_market_hours	bool	True if extended market hours should be taken into consideration	required
exchange_hours	Optional[SecurityExchangeHours]	The associated exchange hours	None

Returns:

Type	Description
CalendarInfo	The calendar information that holds a start time and a period.

try_parse_path

try_parse_path(
    file_path: str,
    symbol: Optional[Union[Symbol, str, BaseContract]],
    date: Optional[Union[datetime, date]],
    resolution: Optional[Resolution],
    tick_type: Optional[TickType],
    data_type: Optional[Type],
) -> Tuple[
    bool,
    Union[Symbol, str, BaseContract],
    Union[datetime, date],
    Resolution,
    TickType,
    Type,
]

try_parse_path(
    file_name: str,
    symbol: Optional[Union[Symbol, str, BaseContract]],
    date: Optional[Union[datetime, date]],
    resolution: Optional[Resolution],
) -> Tuple[
    bool,
    Union[Symbol, str, BaseContract],
    Union[datetime, date],
    Resolution,
]

try_parse_path(
    file_name: str,
    symbol: Optional[Union[Symbol, str, BaseContract]],
    date: Optional[Union[datetime, date]],
    resolution: Optional[Resolution],
    is_universes: Optional[bool],
) -> Tuple[
    bool,
    Union[Symbol, str, BaseContract],
    Union[datetime, date],
    Resolution,
    bool,
]

Parses file name into a Security and DateTime

Parameters:

Name	Type	Description	Default
file_path	Optional[str]	File path to be parsed	None
symbol	Optional[Union[Symbol, str, BaseContract]]	The symbol as parsed from the fileName	required
date	Optional[Union[datetime, date]]	Date of data in the file path. Only returned if the resolution is lower than Hourly	required
resolution	Optional[Resolution]	The resolution of the symbol as parsed from the file_path	required
tick_type	Optional[Optional[TickType]]	The tick type	None
data_type	Optional[Optional[Type]]	The data type	None
file_name	Optional[str]	File name to be parsed	None
is_universes	Optional[Optional[bool]]	Outputs whether the file path represents a universe data file.	None

use_daily_strict_end_times

use_daily_strict_end_times(
    settings: IAlgorithmSettings,
    request: BaseDataRequest,
    symbol: Union[Symbol, str, BaseContract],
    increment: timedelta,
    exchange_hours: SecurityExchangeHours = None,
) -> bool

use_daily_strict_end_times(
    settings: IAlgorithmSettings,
    data_type: Type,
    symbol: Union[Symbol, str, BaseContract],
    increment: timedelta,
    exchange_hours: SecurityExchangeHours,
) -> bool

use_daily_strict_end_times(
    daily_strict_end_time_enabled: bool,
    data_type: Type,
    symbol: Union[Symbol, str, BaseContract],
    increment: timedelta,
    exchange_hours: SecurityExchangeHours,
) -> bool

use_daily_strict_end_times(data_type: Type) -> bool

Signature descriptions:

• Helper method to determine if we should use strict end time

• True if this data type should use strict daily end times

aggregate_quote_bars

aggregate_quote_bars(
    bars: List[QuoteBar],
    symbol: Union[Symbol, str, BaseContract],
    resolution: timedelta,
) -> Iterable[QuoteBar]

Aggregates a list of second/minute bars at the requested resolution

Parameters:

Name	Type	Description	Default
bars	List[QuoteBar]	List of QuoteBars	required
symbol	Union[Symbol, str, BaseContract]	Symbol of all QuoteBars	required
resolution	timedelta	Desired resolution for new QuoteBars	required

Returns:

Type	Description
Iterable[QuoteBar]	List of aggregated QuoteBars.

aggregate_ticks

aggregate_ticks(
    ticks: List[Tick],
    symbol: Union[Symbol, str, BaseContract],
    resolution: timedelta,
) -> Iterable[QuoteBar]

Aggregates a list of ticks at the requested resolution

Parameters:

Name	Type	Description	Default
ticks	List[Tick]	List of quote ticks	required
symbol	Union[Symbol, str, BaseContract]	Symbol of all ticks	required
resolution	timedelta	Desired resolution for new QuoteBars	required

Returns:

Type	Description
Iterable[QuoteBar]	List of aggregated QuoteBars.

aggregate_ticks_to_trade_bars

aggregate_ticks_to_trade_bars(
    ticks: List[Tick],
    symbol: Union[Symbol, str, BaseContract],
    resolution: timedelta,
) -> Iterable[TradeBar]

Aggregates a list of ticks at the requested resolution

Parameters:

Name	Type	Description	Default
ticks	List[Tick]	List of trade ticks	required
symbol	Union[Symbol, str, BaseContract]	Symbol of all ticks	required
resolution	timedelta	Desired resolution for new TradeBars	required

Returns:

Type	Description
Iterable[TradeBar]	List of aggregated TradeBars.

aggregate_trade_bars

aggregate_trade_bars(
    bars: List[TradeBar],
    symbol: Union[Symbol, str, BaseContract],
    resolution: timedelta,
) -> Iterable[TradeBar]

Aggregates a list of second/minute bars at the requested resolution

Parameters:

Name	Type	Description	Default
bars	List[TradeBar]	List of TradeBars	required
symbol	Union[Symbol, str, BaseContract]	Symbol of all tradeBars	required
resolution	timedelta	Desired resolution for new TradeBars	required

Returns:

Type	Description
Iterable[TradeBar]	List of aggregated TradeBars.

generate_relative_factor_file_path

generate_relative_factor_file_path(
    symbol: Union[Symbol, str, BaseContract],
) -> str

Generates relative factor file paths for equities

generate_relative_universes_directory

generate_relative_universes_directory(
    symbol: Union[Symbol, str, BaseContract],
) -> str

Generates the relative directory to the universe files for the specified symbol

generate_relative_zip_file_directory

generate_relative_zip_file_directory(
    symbol: Union[Symbol, str, BaseContract],
    resolution: Resolution,
) -> str

Generates the relative zip directory for the specified symbol/resolution

generate_universes_directory

generate_universes_directory(
    data_directory: str,
    symbol: Union[Symbol, str, BaseContract],
) -> str

Generates the directory to the universe files for the specified symbol

generate_zip_entry_name

generate_zip_entry_name(
    symbol: Union[Symbol, str, BaseContract],
    date: Union[datetime, date],
    resolution: Resolution,
    tick_type: TickType,
) -> str

Generate's the zip entry name to hold the specified data.

get_common_tick_type

get_common_tick_type(
    security_type: SecurityType,
) -> TickType

Gets the tick type most commonly associated with the specified security type

Parameters:

Name	Type	Description	Default
security_type	SecurityType	The security type	required

Returns:

Type	Description
TickType	The most common tick type for the specified security type.

get_common_tick_type_for_common_data_types

get_common_tick_type_for_common_data_types(
    type: Type, security_type: SecurityType
) -> TickType

Get the TickType for common Lean data types. If not a Lean common data type, return a TickType of Trade.

Parameters:

Name	Type	Description	Default
type	Type	A Type used to determine the TickType	required
security_type	SecurityType	The SecurityType used to determine the TickType	required

Returns:

Type	Description
TickType	A TickType corresponding to the type.

get_consolidator_start_time

get_consolidator_start_time(
    period: timedelta,
    start_time: timedelta,
    time: Union[datetime, date],
) -> datetime

Helper method to calculate the start time of a consolidator bar given a period, and anchor start time and the current data time

get_data_type

get_data_type(
    resolution: Resolution, tick_type: TickType
) -> Type

Gets the data type required for the specified combination of resolution and tick type

Parameters:

Name	Type	Description	Default
resolution	Resolution	The resolution, if Tick, the Type returned is always Tick	required
tick_type	TickType	The TickType that primarily dictates the type returned	required

Returns:

Type	Description
Type	The Type used to create a subscription.

get_next_daily_end_time

get_next_daily_end_time(
    symbol: Union[Symbol, str, BaseContract],
    exchange_time_zone_date: Union[datetime, date],
    exchange_hours: SecurityExchangeHours,
) -> datetime

Helper method to get the next daily end time, taking into account strict end times if appropriate

is_common_lean_data_type

is_common_lean_data_type(base_data_type: Type) -> bool

Determines if the Type is a 'common' type used throughout lean This method is helpful in creating SubscriptionDataConfig

Parameters:

Name	Type	Description	Default
base_data_type	Type	The Type to check	required

Returns:

Type	Description
bool	A bool indicating whether the type is of type TradeBarQuoteBar or OpenInterest.

is_valid_configuration

is_valid_configuration(
    security_type: SecurityType,
    resolution: Resolution,
    tick_type: TickType,
) -> bool

Helper method to determine if a configuration set is valid

option_use_scale_factor

option_use_scale_factor(
    symbol: Union[Symbol, str, BaseContract],
) -> bool

Helper method that defines the types of options that should use scale factor

parse_data_security_type

parse_data_security_type(
    security_type: str,
) -> SecurityType

Matches a data path security type with the SecurityType

Parameters:

Name	Type	Description	Default
security_type	str	The data path security type	required

Returns:

Type	Description
SecurityType	The matching security type for the given data path.

parse_key

parse_key(
    key: str,
    file_name: Optional[str],
    entry_name: Optional[str],
) -> Tuple[None, str, str]

Helper to separate filename and entry from a given key for DataProviders

Parameters:

Name	Type	Description	Default
key	str	The key to parse	required
file_name	Optional[str]	File name extracted	required
entry_name	Optional[str]	Entry name extracted	required

parse_time

parse_time(
    line: str,
    date: Union[datetime, date],
    resolution: Resolution,
) -> datetime

Helper method that will parse a given data line in search of an associated date time

read_symbol_from_zip_entry

read_symbol_from_zip_entry(
    symbol: Union[Symbol, str, BaseContract],
    resolution: Resolution,
    zip_entry_name: str,
) -> Symbol

Creates a symbol from the specified zip entry name

Parameters:

Name	Type	Description	Default
symbol	Union[Symbol, str, BaseContract]	The root symbol of the output symbol	required
resolution	Resolution	The resolution of the data source producing the zip entry name	required
zip_entry_name	str	The zip entry name to be parsed	required

Returns:

Type	Description
Symbol	A new symbol representing the zip entry name.

set_strict_end_times

set_strict_end_times(
    base_data: IBaseData, exchange: SecurityExchangeHours
) -> bool

Helper method that if appropiate, will set the Time and EndTime of the given data point to it's daily strict times

Parameters:

Name	Type	Description	Default
base_data	IBaseData	The target data point	required
exchange	SecurityExchangeHours	The associated exchange hours	required

supports_extended_market_hours

supports_extended_market_hours(data_type: Type) -> bool

Helper method to determine if the specified data type supports extended market hours

Parameters:

Name	Type	Description	Default
data_type	Type	The data type	required

Returns:

Type	Description
bool	Whether the specified data type supports extended market hours.

try_parse_security_type

try_parse_security_type(
    file_name: str,
    security_type: Optional[SecurityType],
    market: Optional[str],
) -> Tuple[bool, SecurityType, str]

Parses file name into a Security and DateTime

Parameters:

Name	Type	Description	Default
file_name	str	File name to be parsed	required
security_type	Optional[SecurityType]	The security_type as parsed from the file_name	required
market	Optional[str]	The market as parsed from the file_name	required

use_strict_end_time

use_strict_end_time(
    daily_strict_end_time_enabled: bool,
    symbol: Union[Symbol, str, BaseContract],
    increment: timedelta,
    exchange_hours: SecurityExchangeHours,
) -> bool

Helper method to determine if we should use strict end time

Parameters:

Name	Type	Description	Default
symbol	Union[Symbol, str, BaseContract]	The associated symbol	required
increment	timedelta	The datas time increment	required