Quantcast
Channel: Windows PowerShell forum
Viewing all articles
Browse latest Browse all 21975

Merging multiple Large (500 MB+) CSV files into one to comparing it to another bigger CSV files

$
0
0

Hi Experts,

I have a requirement to combine multiple large CSV files (500 MB+) files into one, it is working super fine with csv files of smaller size (till 200 MB) and doing what I want to do i.e. merge all and create a single csv file with single header and data from all CSVs. My system configuration is Windows XP with 3GB RAM. 

The error that I get is the out of memory exception on files with 200+ MB, Is there any way I can change the memory allocation to powershell as 3 GB is fine for working with 500 MB+ files ?

Also, is there anyother way to do this process.

Currently I am using 

Import-Csv processes.csv

and Export-Csv to export csv in a merged CSV,

Thanks in Advance.

Prajesh




Viewing all articles
Browse latest Browse all 21975

Latest Images

Trending Articles



Latest Images

<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>