FAQ
How to set up installation environments for different Python versions
In the qteasy Tutorial, we introduced how to install qteasy using a virtual environment created with venv. However, sometimes users may need to install qteasy on different Python versions, such as Python 3.9, Python 3.10, Python 3.11, Python 3.12, etc. Here, we introduce a method to install qteasy on different Python versions.
Creating environments with different Python versions using Anaconda:
Anaconda is a Python distribution for scientific computing that includes conda, Python, and over 180 scientific packages and their dependencies. Therefore, we can use Anaconda to create environments with different Python versions and then install qteasy within those environments.
Anaconda can be downloaded from its official website: Anaconda. Download the version corresponding to your operating system and then install it.
After installing Anaconda, we can use the conda command to create environments with different Python versions and install Python packages within them. For example:
$ conda create -n py39 python=3.9 # 创建python3.9环境
$ conda activate py39 # 激活python3.9环境
$ pip install qteasy # 在python3.9环境下安装qteasy
In a conda environment, packages installed using pip can also be correctly identified and managed.
Here are some commonly used conda commands:
$ conda env list # 查看所有的环境
$ conda activate py39 # 激活python3.9环境
$ conda deactivate # 退出当前环境
$ conda remove -n py39 --all # 删除python3.9环境
$ conda list # 查看当前环境下安装的包
$ conda list -n py39 # 查看python3.9环境下安装的包
Use a domestic pip mirror source
In China, due to network conditions, installing Python packages using pip can sometimes be slow or even fail. In such cases, we can use domestic pip mirror sources, such as the pip mirror source from Tsinghua University.
When installing Python packages using pip, you can use the -i parameter to specify the pip mirror source, for example:
$ pip install -i https://pypi.tuna.tsinghua.edu.cn/simple qteasy
How to upgrade QtEasy to the latest version?
When using Qteasy, you may need to upgrade to the latest version. Before upgrading, it is recommended to check the Release History to understand the user-visible changes in each version. The method for upgrading Qteasy to the latest version is as follows:
$ pip install qteasy --upgrade
Alternatively, use the following command:
$ pip isntall qteasy -U
To use the Tsinghua University mirror source within China, you can use the following command:
$ pip install -i https://pypi.tuna.tsinghua.edu.cn/simple qteasy --upgrade
How to install TA-Lib
The complete TA-Lib package cannot be installed via pip because pip install ta-lib only installs a Python wrapper for the TA-Lib package. Users must first install the C language version of TA-Lib before they can use it in Python.
Some users can install the C language TA-Lib package using the following method:
conda install -c conda-forge libta-lib
Methods for installing the C language TA-Lib package on different operating systems:
Windows
Download ta-lib-0.4.0-msvc.zip and extract it to
C:\ta-lib.Download and install
Visual Studio Community(2015 or later), and select the[Visual C++]feature.Open the Windows Start Menu and launch
[VS2015 x64 Native Tools Command Prompt]Move to
C:\ta-lib\c\make\cdr\win32\msvcnmake
Mac OS
First, install Homebrew, and then install the C language TA-LIB package through Homebrew:
$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
$ brew install ta-lib
If using Apple Silicon chips, you can:
$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
$ arch -arm64 brew install ta-lib
Linux
Download ta-lib-0.4.0-src.tar.gz, then:
$ tar -xzf ta-lib-0.4.0-src.tar.gz
$ cd ta-lib/
$ ./configure --prefix=/usr
$ make
$ sudo make install
After the C language TA-LIB package is installed, the Python TA-Lib package can be installed via pip:
$ pip install ta-lib
To install TA-Lib on an arm64-based Mac, you can use the following command:
$ arch -arm64 python -m pip install --no-cache-dir ta-lib
If you are using an Arm64 architecture computer but encounter an import error ImportError: after installing ta-lib, it may be due to an environment issue. Please refer to this article (https://blog.csdn.net/Shepherdppz/article/details/138253619) to resolve your problem.
Install Qteasy in a higher version of Python environment
The solution is to upgrade to the latest version of qteasy. Newer versions of qteasy have been tested in Python 3.7 to 3.12 environments and function correctly in these environments.
$ pip install qteasy --upgrade
Earlier versions of qteasy were not adequately tested in newer versions of python, resulting in some compatibility issues. These issues primarily exist in qteasy distributions of version 1.1.4 and earlier.
Meanwhile, earlier versions of qteasy used some APIs that are about to be deprecated in higher versions of pandas and numpy, which may have caused compatibility issues with higher versions of pandas and numpy. These issues have been fixed in versions of qteasy from v1.1.7 onwards.
After adding configuration information to qteasy.cfg, a database connection failure message appears.
Some users have reported that even after configuring the database connection information in the qteasy.cfg file, they still encounter a database connection failure message, receiving an error similar to the following:
C:\Users\yuewe\Documents\GitHub\qteasy-1.0.26\qteasy\database.py:2504: RuntimeWarning: (1045, "Access denied for user 'yuewe'@'localhost' (using password: NO)"), Can not set data source type to "db", will fall back to default type
warnings.warn(f'{str(e)}, Can not set data source type to "db",'
If this is your first time encountering this problem, you should probably first check your qteasy.cfg file. Mainly check the following two aspects:
Is the configured key incorrect? The stable version of
qteasyuses database connection information configuration keys such aslocal_db_host/local_db_port, etc., instead ofdatabase_host. There were some errors in the official documentation before, but these errors have been corrected in the latest documentation.When providing configuration information in the file, please do not add other characters such as
",<, and>, etc. Otherwise, these characters will be interpreted as part of thetokenor database name, causing the database connection to fail.
When parsing configuration files, qteasy automatically converts them to the correct format based on the configuration type. For example, the database port 3306 should be an int variable, which can be used directly.
local_db_port = 3306
That’s it. qteasy will convert the string 3306 to an int type 3306.
The following configuration file example is correct:
tushare_token = 2dff3f034aa966479c81e4b4b0736fb081b740abb2xxxxxxxxxxxxxxxxxxxxx
local_data_source = database
local_db_host = localhost
local_db_port = 3306
local_db_user = user_name
local_db_password = pass_word
local_db_name = ts_db
When reading data from the database, a prompt suggests installing sqlalchemy.
Sometimes, when using a database as a data source and reading data from the database, you may see the following message:
UserWarning: pandas only supports SQLAlchemy connectable (engine/connection) or database string URI or sqlite3 DBAPI2 connection. Other DBAPI2 objects are not tested. Please consider using SQLAlchemy.
The above message suggests that users use sqlalchemy.
In summary, this is a UserWarning. No error actually occurred; the data read was successful. The reason for this warning message is related to the version of pandas you have installed. You do not need to install sqlalchemy; qteasy has removed the dependency on sqlalchemy. To confirm why this warning is appearing, please check your pandas version.
import pandas as pd
pd.__version__
If you are using pandas version 1.1 or higher, you may encounter this UserWarning, but it generally shouldn’t affect usage. This is because qteasy uses pymysql as its database connection API, but pandas has gradually removed support for pymysql since version 1.1. Although tests show that qteasy can still read data correctly in pandas version 1.5, you will receive a warning message.
The above issues are on my modification list. In the upcoming minor upgrade, the dependency on the pandas SQL API will be removed, and pymysql will be used directly to read the database. This will prevent this warning message from appearing. If you do not want to see this warning message, you can also downgrade the pandas version to 1.1.0. qteasy has consistently used a lower version of pandas during its early development, and most stability tests are based on pandas version 1.1. If you use pandas version 1.1, this warning message will not appear, and all other functions will work normally.
Download data in batches from tushare
Sometimes we need to download data in batches, pausing for a certain period between each batch. For example, some data tables on tushare have a frequency limit, limiting the maximum number of downloads per minute; exceeding this limit will cause download failures. In this case, we can achieve such batch downloads and pauses using the following two configuration parameters:
QT_CONFIG.hist_dnld_delay: The default value is 0. If set to a positive integer, it represents the number of downloads in each batch when downloading in batches.QT_CONFIG.hist_dnld_delay_evy: The default value is 0. If set to a positive integer, it represents the number of seconds to wait between two download batches. The following configuration will make it wait one minute after downloading 600 sets of data:
>>> qt.configure(hist_dnld_delay=60, hist_dnld_delay_evy=600)
>>> qt.configure(hist_dnld_retry_cnt=3) # 同时减少重试的次数以缩短报错前等待的时间，这个配置并不是必要的
After setting it up as described above, even when using parallel downloading, qteasy will not submit all tasks to the process pool at the same time. Instead, it will submit them in batches and wait for a period of time before submitting the next batch.
The above features were added in version v1.1.11 of qteasy. If your version of qteasy is lower, please upgrade to the latest version.
How to get the minimum workflow from data to backtesting/optimization working?
It is recommended to complete the process in one minute as described in the “Quick Start Guide”: Configure tushare Token → Download data → Perform a backtest using the built-in strategy. For the complete workflow (Configure data source → Download data → Define strategy and backtest → Parameter optimization → Simulated/live run) and corresponding tutorials, please see [Quick Start Guide - End-to-End Roadmap / Tutorial](getting_started.md#End-to-End Roadmap–Tutorial). Following the order of the tutorials in this roadmap will allow you to complete the entire process from scratch.
Why is backtesting/optimization slow?
Possible reasons include:
Data volume and window length: The longer the backtesting interval, the larger the asset pool, and the larger the strategy’s
window_length, the more data needs to be prepared and calculated for a single backtest.Number of parameter groups: In optimization mode (mode=2), the more parameter combinations there are, the more times a single backtest needs to be performed; this has been mitigated through multi-process parallelism, but it is still related to the number of machine cores and the scale of the task.
Initial Numba Compilation: Core backtesting functions (such as
backtest_stepandbacktest_batch_steps) use Numba JIT. Compilation is triggered on the first run or when the parameter type changes, resulting in a one-time delay. Subsequent backtests of the same type will reuse the cache, which will significantly improve the speed.Data Preparation: If the data in the DataSource is not prepared in advance, data retrieval and window assembly will be triggered before backtesting, which will also increase the first run time.
For further information on backtesting engines and performance design, please refer to Backtesting Engines and Performance and Backtesting Engines and Performance (Design Perspective) in “Architecture and Design”.
How to understand “future-proof functions”?
Qteasy avoids the strategy from “seeing” future data during backtesting by its mechanism: At each time step, the engine only injects a window of historical data that is visible at that time step into the strategy. The strategy retrieves only the data within that window using get_data(dtype_id) in realize(). Metrics calculated on this window (such as EMA) are consistent with the historical data that is “only visible at that time” in live trading, thus avoiding unintentional use of future data. For detailed design rationale (why not perform full-time axial quantization, how data windows are injected step-by-step, etc.), see Design Intent and Unique Advantages in “Architecture and Design”.
What is use_latest_data_cycle and how is it set?
use_latest_data_cycle (often abbreviated as ulc) controls whether the data window used by the strategy includes the current root candlestick (the latest data at the current moment): When set to True, the window can include the current bar (such as the closing price), corresponding to “whether the latest price at the time is allowed during trading”; when set to False, the window strictly contains data from “past closing prices”. This choice will have a significant impact on returns in long-term backtesting, and qteasy exposes it as a configurable item in the strategy declaration and runtime configuration. For its meaning and impact, see Design Intent and Unique Advantages - use_latest_data_cycle; for specific settings, see the data_types / window_length declarations in the strategy class and How the Strategy Declares and Uses Data, as well as the configuration instructions in the current version documentation and API.