Out of memory problem

两个目录下有66K多个文件,需要运行Kappa进行计算比较。作了一个批处理,

for %%f in (C:nan_wkspmetric_analysis_radar_nldasdatanwbi_deg8_selradar_clas*.asc) DO (
kappa %%f C:nan_wkspmetric_analysis_radar_nldasdatanwbi_deg8_selnldas_combo_clasl%%~nf.asc -k:C:nan_wkspmetric_analysis_radar_nldaskappakappa.txt)

在执行到1万个的时候,提示 Not enough storage is available to process this command,以及 out of memory,退出!不知道是什么原因。起先以为是Kappa代码有问题,但理论上讲,代码是用托管模式写的,正常退出时dotnet会负责回收的。Kappa代码里连接了ArcGIS的相关代码,这部分是非托管的,有可能是异常引起非托管资源被占用,从而积累导致问题。

后来在网上找了一下,在微软KB里有类似的描述,http://support.microsoft.com/kb/126962,建议修改注册表,增加shared memory。

但终因为不知道改成多少合适,而且改Kappa代码也很容易,决定不采用这种方法(workaround)。用c#写小段代码,实现批处理,寄希望于通过托管系统自身的回收机制,能解决out of memory的问题。代码十分简单。

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Diagnostics;

namespace Kappa_bat
{
    class Program
    {
        static void Main(string[] args)
        {
            string nldas_dir = @"C:nan_wkspmetric_analysis_radar_nldasdatanwbi_deg8_selnldas_combo_clas";
            string radar_dir = @"C:nan_wkspmetric_analysis_radar_nldasdatanwbi_deg8_selradar_clas";
            string out_dir = @"C:nan_wkspmetric_analysis_radar_nldaskappa";

            string kappa_file = Path.Combine(out_dir, "kappa.txt");
            string log_file = Path.Combine(out_dir, "logfile.txt");

            string[] radarfiles = Directory.GetFiles(radar_dir, "*.asc");
            foreach (string f in radarfiles)
            {
                Process proc = new Process();
                proc.StartInfo.FileName = "kappa.exe";
                string fn = Path.GetFileName(f);
                string nldas_fn = Path.Combine(nldas_dir, "l" + fn);
                proc.StartInfo.Arguments = nldas_fn + " "+ f+ " -l:"+log_file+" -k:"+kappa_file ;
                proc.StartInfo.UseShellExecute = false;
                proc.StartInfo.RedirectStandardOutput = true;
                proc.Start();
                proc.WaitForExit();
                Console.WriteLine(fn);
            }
            Console.WriteLine("Done.");
        }
    }
}

Leave a Reply

Your email address will not be published. Required fields are marked *