This section describes how to set up high-speed data load.
Setup flow
Set the parameters required for high-speed data load in postgresql.conf. After that, start or restart the instance.
The table below lists the postgresql.conf parameters that must be changed, and the values that must be added to their current values. After editing postgresql.conf, start or restart the instance.
Parameter | Setting | Required |
|---|---|---|
max_prepared_transactions | Add the number of transactions that can be prepared by parallel workers during data load to the parameter's current value. The resulting value must be equal to or greater than the maximum number of parallel workers used with this feature. | Mandatory |
max_worker_processes | Number of parallel workers to perform data load. | Mandatory |
max_parallel_workers | Add the maximum number of parallel workers to be used in a parallel query by this feature to the parameter's current value. The resulting value must be equal to or greater than the number of parallel workers used with this feature. | Mandatory |
Example
The example below shows how to configure 2 instances of high-speed data load being executed simultaneously using a degree of parallelism of 4.
max_prepared_transactions = 13 #Example if the initial value was 5: 5 + 2 x 4 = 13 max_worker_processes = 16 #Example if the initial value was 8: 8 + 2 x 4 = 16 max_parallel_workers = 12 #Example if the initial value was 4: 4 + 2 x 4 = 12
Note
As shown in the example above, set the value of max_prepared_transactions, max_worker_processes and max_parallel_workers multiplied by the number of instances of this feature executed simultaneously.
The table below lists the postgresql.conf parameters that must also be checked.
Parameter | Setting | Required |
|---|---|---|
dynamic_shared_memory_type | Implementation of dynamic shared memory to be used by the instance. The default value is recommended. | Mandatory |
See
Refer to "Resource Consumption" in the PostgreSQL Documentation for information on the parameters.
Execute CREATE EXTENSION to install the high-speed data load extension. The extension needs to be installed on each database.
Example
The example below installs the extension on the "postgres" database.
postgres=# CREATE EXTENSION pgx_loader; CREATE EXTENSION
Note
Only superusers can install the high-speed data load extension.
The high-speed data load extension can only be installed on the public schema.
