Skip to contents

Split Order data into smaller chunks

Source: R/DWF_Efforts_Split.R
Efforts_Split.Rd

This function extracts Order data without extracting the zipped archive. The function reads CSV files inside the zipped file, selects specified columns, and splits the data into smaller chunks of specified row size, saving each chunk as a separate file.

Usage

Efforts_Split(Path_Zip, Path_Output, ChunkSize = 1e+05)

Arguments

Path_Zip

Character. The file path to the zip archive containing the CSV file. The file must be a ZIP archive containing a single CSV file.

Path_Output

Character. The directory where the split files will be saved. The directory must exist.

ChunkSize

Integer. The number of rows per chunk file. Default: 100,000. Note: Larger chunk sizes may require significant memory and processing power.

Value

A character vector of file path(s) to the created chunk files.

Note

This function is not intended to be used directly by the user or in the IAS-pDT, but only used inside the Efforts_Process function.

Author

Ahmed El-Gabbas