Built-in Historical Data Types in qteasy

Data types are a core qteasy concept: information used in a trading strategy, such as historical market data or macroeconomic series. Information is not the same as raw data. In older versions, qteasy equated data types with DataSource table columns, which was inadequate because:

  1. Some raw fields alone carry little meaning and must be combined with other data

  2. Some fields encode multiple kinds of information extractable from different angles

  3. The meaning of information is dynamic—the same data can be interpreted differently

Equating information with data is careless: logically, data is only a carrier of information.

In the new qteasy definition, a data type is information extracted from complex stored data. Some can be read directly from tables, for example:

  • Stock open price, read directly from the price table

Some data must be assembled from multiple tables, for example:

  • Adjusted stock price requires price data, adjustment factors, and computation

Sometimes one column yields different information, for example:

  • Previous close and daily change are different information from the same price table

qteasy redefines data types as the information we need, with simple APIs to extract it quickly, support custom types, and chart, compare, and analyze results.

Each data type is implemented as a class for easy customization. qteasy also ships extensive built-in types covering many tables so downloaded data is fully usable.

Logically, defining a data type requires:

  • Data source: which tables and columns provide this information?

  • Acquisition: how is the information extracted?

Many built-in types must be defined simply and parametrically; the class model stays small enough for parameter-driven definitions.

Usage and output — data types expose two unified APIs:

API1: data_type.get(start, end, frequency) — symbol-agnostic interval data at one frequency, Series-like output.

API2: data_type.get(shares, start, end, frequency) — per-share interval data at one frequency.

A third API: data_type.get(shares, start, end, frequency, **kwargs)

Built-in and custom types share the same return format.

Acquisition categories depend on how data is read and produced—direct reads vs composition tables, multi-table queries such as adjusted prices, etc. Distinct patterns need distinct handling.

Ignoring pure multi-field math types (except adjusted price), acquisition types include:

acquisition_type controls symbolised vs un-symbolised output and Period vs TimeStamp layouts:

Supported output layouts are fixed at data-type definition time

‘basics’:

basics — time-independent reference data; kwargs: one table and column

‘direct’:

direct — read one table/column over an interval and frequency

‘adjustment’:

adjustment — read A and B from two tables and adjust A with B (e.g. adjusted price)

‘relations’:

relations — relational output between A and B (eq/ne/gt/or/nor, etc.)

‘operation’:

operation — arithmetic between A and B (+/-/*//)

‘event_status’:

event_status — fill status over event impact windows (suspension, rename, etc.)

‘event_multi_stat’:

event_multi_stat — multiple statuses over event windows (e.g. management roster)

‘event_signal’:

event_signal — signals at event dates (limit up/down, listings, dividends, etc.)

‘composition’:

composition — filter composition tables and pivot rows/columns; plus special procedural types

‘compound’:

compound — combine multiple fields at one timestamp (e.g. financial statements)

Output Type

‘symbolised’:

Symbolised Period data: DataFrame-like, rows = time periods, columns = share codes

‘un-symbolised’:

Un-symbolised Period data: Series-like, rows = time periods, single column

Class design is parametric: parameters fix acquisition method and target tables

One public method dispatches by acquisition type, reads target tables, cleans, and returns data.

Type metadata can encode field semantics, e.g. open usable in the latest cycle but close not

A hook method may be overridden for custom acquisition logic

Many built-in types are created via a mapping table from type IDs to key parameters.

1, DATA_TYPE_MAP:

DATA_TYPE_MAP maps each data type ID to its storage table (unique ID and location per type).

Data table mapping columns: htype_name (key): data type name

freq (key): available frequency — min/d/w/m/q

asset_type (key): E, IDX, FT, FD, OPT, THS, SW, etc.

description: detailed description for search

acquisition_type: acquisition method

table_name: storage table name

column: column name in the table

“””