Yearly Archives: 2007

Extending Python by reusing existing C++ codes


Extending Python by reusing existing C++ codes

Zhuotong Nan ([email protected])

Why I do this?

First let me show you some codes wrote in Python:

(code snippet)

This class is to call an external executable, which will generate necessary data files in a proper file format for further work. When I come to port this class to Windows, I found it is unable to work since os.fork and os.wait do not exist in Windows, and much more worse, os.execvp cannot run as expected. My initial solution is to look for their equivelant under Windows but hard to solve it in a fast way. The alternative choice is to implement Read_xmrg2 as an module of Python. Fortunately, we have c source codes of Read_xmrg2 (which can be found at http://www.weather.gov/oh/hrl/dmip/2/xmrgformat.html). I start to look for some information regarding this implementation.

Extending using SIP

This solution is conventionally called extending Python. But the common implementation presented by Python help is hard to do. More efforts should be placed in doing the conversion between C/C++ data types and Python’s. Since I have background on Qt, a helpful tool called SIP (http://www.riverbankcomputing.co.uk/sip/) draws my attention. There are a number of tools to do the Python bindings. Some studies shows SIP is easier to do such kind of bindings. Although SIP comes with a nice reference file, it is a little bit hard for a newbie to start.

Step 1 Tuning existing c codes

Since I prefer C++ style, I changed existing Read_xmrg2 codes to a C++ class. Following is its simple header file (dmip2.h).

  (code snippet)

Note reverse_byte_order and its short equivalent are used by read_xmrg2 functions. I made them private. Also I declared read_xmrg2 as static. Header file should be generated since SIP will link this header file to generate conversion codes. One work we remains is to convert main function in original c codes to a common function.

(code snippet)

Keep in mind, we should add DMIP2 as a domain prefixing to every method (read_xmrg, constructor, reverse_byte_order, etc). Next, I commented some codes since as a function there is no necessity to check argument number. Also comment declarations of functions reverse_byte_order and reverse_byte_order_short since we declare them in the class definition. Replace original argv[1] to variable fn. Note I also add a fflush(stdout) to force the print information to show immediately, otherwise the caller (Python function) also prints messeges which will mess up the console output.

Then, make the class compiled as a shared library (with extension as DLL). In Eclipse C++ dev, we only need to switch Artifact type to Shared Library as shown below (figure 2), and then compile codes to generate dmip2_py.dll.


figure 1. DLL building settings in Eclipse c++

Step 2 Generate Python module using SIP

2.1 Install SIP

Get SIP package (sip-4.7.1zip) from http://www.riverbankcomputing.co.uk/sip/. SIP supports all versions of Python since v2.3. Version 2.4 was installed in my computer because some other Python modules depend only this version. Unpack SIP zip file to a directory (sip-4.7.1) under Python v2.4 directory (generally c:Python24). Open command window, enter the directory SIP located, and then type "c:python25python configure.py -p win32-g++" which will configure SIP with MinGW compiler I installed in my environment. Default settings of configuration are suitable for most cases. After its configuration, type "mingw32-make" to compile and "mingw32-make install" to install SIP module with Python. More information is available in the reference file (sipref.html) under doc directory of SIP installtion directory.

2.2 Generate a SIP specification file

(code snippet)

Note here if the header file of the wrapped class is not at the same directory as the sip files (dmip2.sip), we should specify its relative location with #include directive. I save this specifation file in sip diectory (figure 2). Since we are only interested in read_xmrg2 function, this sip specification file exposes only this declaration. Change %Module on your demand. In Python codes, we can call read_xmrg2 by DMIP2PY.DMIP2.read_xmrg2() if we import this module.


figure 2. sip folder contains sip files, while src folder c++ header files as wells as source files.

 

2.3 Built a configure file for faciliating building module

(code snippet)

The line os.system(…) will enable generated files put the same folder as sip files. You should change dmip2.sip according your naming. Other point should be mentioned is makefile.extra_libs where specifies the name of DLL files produced in Step 1. If the DLL is in different location, make corresponding change to makefile.extra_lib_dirs. I prefer copying DLL file(s) (for ex. dmip2_py.dll) to sip directory.

Save to configure.py in the sip directory, run it by typing "python configure.py" in command window. Be sure you have set python location (for ex. c:python24) and MinGW binary location (for ex. c:mingwbin) to your system path variable. Otherwise please specify the absolute path here.

This compilation will produce some new files. See figure 3 for my case.


figure 3. The compilation will produce ultimate python module which wraps DLL functionality.

Step 3 Using generated Python module in Python codes

We need to make some changes to runProgram to employ generated Python module.

(code snippet)

Here we first import DMIP2PY module, and enhance the function by enabling it runs on varying operating system. Please note if we use import directive as shown in the above case, we have to locate the function by MODULENAME.CLASSNAME.FUNCTIONNAME(). But if we import functions by "from DMIP2PY import *", we can refer to the function by CLASSNAME.FUNCTIONNAME().

Do not forget to copy the compiled Python module (dmip2py.pyd) and associated DLL file (dmip2_py.dll) to the same directory as the calling python file.

Conclusion

We walked through how to extend python by using SIP step by step. It’s not much hard to implement your real application. Efforts should be made to the SIP specification file. More details can be found in the reference manual located in the doc directory of sip source package. But for simple scenarios, simply do enhancement based on the above example. SIP also have special design for Qt (www.trolltech.com) application. In this context, it is the unique choice for Qt users.

Sources codes can be availabe on request. Send your comments and suggestions to [email protected].

All rights reversed.

IF YOU ARE INTESTED IN COMPLETE VERSION, PLEASE SEND ME EMAIL OR REQUEST BY FOLLOWING COMMENTS.

Download PDF version (174KB)

Protector Suite 5.6.0 build 3152

The latest version of Protector Suite for SONY VAIO SZ series can work both
XP and Vista.
1. uninstall any previous version
2. reboot
3. download this versio from
http://dlsvr03.asus.com/pub/ASUS/nb/R1F/FingerPrint_Vista32bit_070126.zip
and install it.
4. reboot
I tested it on my XP sp2, works fine. This also supports both IE 6 (I don’t
know if it works with IE 7) and Firefox 2.0.x.

Ref: http://forum.notebookreview.com/showthread.php?t=101461

危险警告

今天,我们办公室几个已经有手机的同事收到一条危险警告短信,说U. Pitt的某建筑
楼内可能有炸弹,要求大家撤离或远离。在中午时刻,再次收到短信,危险已经清除。
据同事讲这在这里是经常的事情。

管理制度

尽管从我刚到Pitt,书功就一直跟我讲她的管理是如何混乱,但这些天手续下来,感觉
还是十分顺利,尽管中间会将一些事情弄错。今天上午去申请了学校的ID,用它有很多
好处,比如坐公车不用车,可以借书等。包括拍照,前后10分钟左右。然后去申请
Email Account,也很快就办好了。
跟朋友谈到办事效率的时候,我就在想,中国为什么大家人浮于事,跟部门的分工明确
怕有不少的关系。
国内办事情,经常去这个部门,然后人家告诉你缺这个缺那个,这个东西得要哪位领导
的签字,那个东西还要补充材料。等东西都准备好了,过去的时候发现办事的人不在,
那就继续等吧,事情就是这样被往后拖。办事态度肯定是一回事,制度可能是更重要的
原因。
这里办每个事都可以找到详细的说明,告诉你要准备什么材料,然后拿着这些材料到负
责的部门,很快就能完,在上班时间肯定是有人在的,不会让人白跑。
国内不同部门间交叉太多,简单的事都要经过多个部门的手。我所在工程学院,学生上
千,教授几十,但行政人员只有几个,chair还是教授兼的。看国内的单位吧,哪个单
位不是行政的一大把,而且每个人都把自己看作一个官,有人上门找就要显摆一下。
不过中国人多,如果不这么大把行政人员,都下岗了,也是社会的一大压力。

美国夏令时结束

今天是十一月份的第一个星期天,美国夏令时(DST)早上2点结束。时间回到正常的国
际时差(匹兹堡现在与国内时差13小时)。07年美国政府从原有夏令时又延长了2个
月,以最大化利用白天的阳光资源,减少能源浪费。但这样时间拔来拔去的,也挺复
杂,今天就一不小心就多睡了一个小时,计算机时间是自动调过来了,朋友不说我还不
知道有这回事。

办事效率

据说U. Pitts的管理在全美大学里都是比较乱的。今天跟书功跑了一趟OIS (Office of International Services),仅完成了部分registration,还要办ssn,mobile phone,Pitts Id,以后也许还要弄个driver license开开,想着就头痛。

到美国了

昨天从Beijing到Chicago再到Pittsburg,近二十小时。WSG过来机场接。整个过程很顺利。
今天转了一下U. Pitts和Downtown。

有朋自远方来

L同志打电话过来的时候,张口就叫我大名,很熟的样子,手机的新的,却不报上他自己的名字,害的我琢磨了好一阵子,才想可能是L这个家伙。他说已经在兰州。下班后叫上CXC和WLZ,跑到地质宾馆,见面后免不了一阵感慨,光阴易逝。CXC甚至是大学毕业后都没见着,怕有10余年了。C感叹说如果在大街上碰着,估计都不敢贸然相认。

借这个机会,我联系了在兰的全部同学,在胖妈妈聚会。除了我借口开车,不敢多喝,其余TX们我想都喝好了,L估计心情不错,7、8两估计是有的。到静宁路的歌来美KTV的时候,他倒在沙发就睡,怎么叫也不醒,你说睡好了也可以,2小时后他居然又吐了,不划算。不过大家的心情都是很好的。这次人来的也比较全,说是大家都在兰,这个城市也不大,但有TX我都是1年多没见着了,只偶尔通过电话。

前面去北京的时候还见着HCY,和HZJ。HCY刚从日本回来,我也是2年多没见着了,他媳妇刚给他生了个漂亮女baby。HZJ前面从刚果金回来后一直没见,现在常驻长沙,在北京也难得一见。这次还麻烦HZJ驱车满北京城跑,完了还送到机场,真是太麻烦他了。

ArcIMS ArcXML 元数据搜索中文关键词问题

或者在c#代码中直接用中文字符构建 SEARCH_METADATA将搜索失败,提示非法的ArcXML
解决方法是使用中文的unicode表示,即旞类似的方式
提供了一个转换函数:

/// <summary>
/// 将汉字变成 黑&27827;的ascii编码
/// </summary>
/// <param name="text"></param>
/// <returns></returns>
public static string UnicodeEncode(string text)
{
char[] chars = System.Web.HttpUtility.HtmlEncode(text).ToCharArray();
System.Text.StringBuilder result = new System.Text.StringBuilder(text.Length + (int)(text.Length * 0.1));

foreach (char c in chars)
{
int value = Convert.ToInt32(c);
if (value > 127)
result.AppendFormat("&#{0};", value);
else
result.Append(c);
}

return result.ToString();
}

ESRI本身提供的metadata explorer搜索中文关键词时有问题,是因为它的代码将中文解释成了 &amp;26789;类似,而不是梥,可能是将&再次解释了,搜索时将不返回结果。不知道9.1及以后版本的arcims metadata explorer里是否已经更正。