Main contents
Archive for the 'Programming' Category
February 5th, 2009
The Win32 extensions for Python have been a lifesaver for me many times. I consider them one of the best ways of scripting the windows platform. Unfortunately there’s not as much documentation as one could hope for.
A useful trick I discovered recently is the following. Say you’re trying to use a COM interface such as Microsoft Message Queuing. You can create a message queue object as follows:
import win32com, win32com.client
serverInfo = win32com.client.Dispatch("MSMQ.MSMQQueueInfo")
However, the next step requires constants from the messaging API such as MQ_SEND_ACCESS. These can be made available to win32com.client.constants in two ways. One involves running makepy, but a more robust alternative is as follows:
import win32com, win32com.client
win32com.client.gencache.EnsureDispatch("MSMQ.MSMQQueueInfo")
win32com.client.gencache.EnsureDispatch("MSMQ.MSMQMessage")
from win32com.client import constants
serverInfo = Dispatch("MSMQ.MSMQQueueInfo")
serverInfo.FormatName = <format name>
return serverInfo.Open(constants.MQ_SEND_ACCESS, constants.MQ_DENY_NONE)
The order is important. Once EnsureDispatch has been called, win32com.client.constants can be imported and will have the correct values available.
Posted in Programming, python | No Comments »
July 6th, 2008
Spent some time exploring Lua. This language has caught my interest, not least because Lightroom is largely implemented in it. I’ve played around with it briefly before on Linux, and I have to admit that the experience of getting it up and running is much less painful on Ubuntu than on Windows. $ apt-get install lua is much faster than downloading tarballs, creating visual studio solutions from scratch, mucking around with paths and dependencies and resolving any number of LNK2005 errors. But I got there in the end. Specifically, I
- Downloaded the Lua source from Lua.org.
- Created a Visual Studio DLL project containing all the source files apart from lua.c and luac.c
- Built this DLL to create lua.dll
- Created another project with just lua.c, and linked to the above DLL
- Built this project to create lua.exe. Now I’ve got a lua environment.
- Downloaded the luabind source from Rasterbar Software. This seems to be based on the powerful Boost.Python library, which I’ve had a lot of success with in the past.
- Created another Visual Studio project for luabind, add the luabind source files and link with lua.dll. Luabind does come with a bjam make file, but I usually find bjam a complete headache to use.
- Created yet another project with the sample code from the luabind tutorial, built and linked against the luabind library and lua.dll
After all this, (and several hours of tweaking linker settings) I’ve now got the tutorials working to the extent that I can now do two things. I can expose C++ functions into a Lua program, which is very handy, and something I’ve found very useful in Python. But of course, the big draw to Lua is that it can be embedded in an existing C or C++ program! So now I’ve also got to the point where I can contain a Lua interpreter within a C++ application, pass a string of Lua code to that interpreter to be executed, and even let provide the interpeter with the ability to call out to C++ functions elsewhere in the program.
So despite the the hoops I had to jump through, I think this could be a very powerful environment. Now I just need to learn the Lua language itself, but as they say, if you can’t learn a new language in a weekend you shouldn’t call yourself a programmer.
Posted in Programming, c++, lua | No Comments »
February 13th, 2008
As I type this on my windows XP desktop machine, I’m also logged in (via my other desktop machine running Ubuntu Linux) to an HP-UX system running on itanium, a Solaris 7 machine on a Sparc processor, two Solaris 10 machines on AMD64 , an AIX machine running on PowerPC 3 and another Linux box on intel i686.
So, that’s five operating systems, at least seven OS versions, and five processor architectures, with a nice mix of big and little endian-ness.
And the really nice thing is that I’ve just got my networking code to compile and run on all of them. (Although I’m looking at you, HP-UX, for not handling non-blocking sockets the same as the others!)
Posted in Programming | No Comments »
October 27th, 2007
Had a chance to play around with some interesting software tools recently. Some that have caught my attention have been:
- Ubuntu - very nice Linux distribution, rapidly becoming the one to beat. We use it at the company I’m contracting for, and it’s been my first exposure to a Debian based system. apt-get is very nice, I’ve had very little software installation problems. But as ever I feel that Linux is lagging in the display driver department.
- lighttpd. I have a new favourite webserver. Thanks to apt-get, I went from having no webserver to having a fully functional installation of lighttpd serving up cgi pages in, oh, about three minutes. Hooray for Linuxy goodness!
- Haskell. Rapidly becoming one of my favourite languages, and I’m convinced that any decade now I’ll be able to do something useful with it, as opposed to merely alternating between being in awe of it’s elegance and banging my head against trying to understand category theory just so I can do some simple I/O. But that said, I took one step closer to this goal with the discovery of.
- Haxr. Getting it installed involved jumping through a few Cabal hoops, as it seemed to need some older packages, but once I had it running, I very quickly had a Haskell CGI program served up via Lighttpd (see above!) and talking xml-rpc over the network! Being convinced, as I am, of the importance of low coupling/high cohesion, I have a feeling that xml-rpc or json-rpc could be very nice ways of tying together complex and disparate systems. I’ve seen enough companies spend many many-years trying to put together distributed serialisation mechanisms, and I suspect that often the simplest approach is best. Being able to create a service using a language choice that is totally invisible to the client application is very handy.
Now I just need to find a nice persistance layer for Haskell, and I’ll be a happy man.
Posted in Haskell, Programming | No Comments »
September 1st, 2007
It’s been nice working at a company where everyone is a computational polyglot – you can debate the merits of LISP or the shortcomings of Smalltalk over coffee. Of course there are downsides – simply to get the build system to work requires getting Perl, C, Bash and Python to work together harmoniously.
However, working in a semantically rich language, like Python, or talking to people who know Lisp, which is probably the most versatile language in existence, really does through into sharp relief how semantically impoverished C is. One problem is that people frequently don’t make use of language abstractions, even where they exist. But it’s very nice to be able to introduce them, and benefit from the resulting simplification of the code base.
I discovered the new python ‘with’ statement, which in the absence of deterministic destructors is a pretty nice way of handling resource management. Writing lock-safe code is now as simple as this:
with Guard(lock):
threadUnsafeOperation()
where Guard is as simple as
class Guard:
def __init__(self,lock):
self.lock=lock
def __enter__(self):
self.lock.acquire()
def __exit__(self,exceptionType,exceptionValue,traceback):
self.lock.release()
I’ve really been missing the constructor/destructor pairs from C++ in other languages, and this seems to fit the bill pretty nicely, especially where the pattern you’re looking for is ‘ScopeGuard’
Posted in Programming, python | No Comments »
June 7th, 2007
Having put in all the work for a nice infrastructure for my current project, I think that it’s beginning to pay off. Now I can make changes to my TurboGears app in JEdit, have the changes picked up by CherryPy’s auto-reload feature and test and debug in a very tight cycle. Then when I’m happy with them I can push them upstream with a quick subversion check in, and a few seconds later my changes are running on the production machine. All in all, I feel as if programming could actually become pleasant again. And I must say that TurboGears is a nice way to develop GUI’s – describing the layout in XHTML is far nicer than all the grief I used to have to go back in the day when I was building desktop apps with MFC.
Posted in Programming, TurboGears, python | No Comments »
June 1st, 2007
I think that I’ve spent maybe 10 minutes today writing actual production code, and many, many hours dealing with dropping SSH connections, spotty wireless access, wrestling with TurboGears’ logging system, trying to figure out why code that used to compile on Solaris 9 doesn’t anymore since the system upgrade to 10, and just generally trying to put into place a workflow that will actually make me productive for my current projects.
It’s clear to me that being able to write software is only one, fairly small component of successfully creating systems. Being able to manage the complexity of disparate, changing systems without going crazy is critical. And being able to choose wisely between multiple technology options is also critical – the ability to distinguish between the package that is going to be a lifesaver and the one that will send you to an early grave as it continually causes more problems than it solves.
Posted in Programming, work | No Comments »
May 30th, 2007
Languages I code in:
Continuously for the last 8 years
Regularly for several years:
- Java
- Haskell
- Javascript
- Visual Basic
- FoxPro (yuck!)
Operating Systems I use regularly
- Windows (Since NT3.1/ Windows 95)
- Linux (including Mandrake, Ubuntu, Red Hat)
- FreeBSD (isn’t ports nice?)
- Solaris
- Mac OSX (can’t run a studio without it!)
- Other interesting combinations like Cygwin on Windows, Parallels on Mac, VirtualPC etc.
- I typically work in heterogeneous environments with multiple different systems having to work together.
Database systems I’ve used
- MS SQL Server
- MySQL
- Postgresql
- Oracle
- Access
- FoxPro
- sqlite
Source Control Systems I know
- Bitkeeper
- Perforce
- Subversion
- Microsoft Visual Source Safe (hardly counts, I know)
- darcs
Types of development I do
- Web based AJAX applications, using TurboGears or ModPython
- Win32 applications, using MFC, WTL, or the Win32 api.
- Low-level C++ libraries: eg. TCP/IP communications and medical imaging.
- Cross platform GUIs using toolkits like GTK.
- Anything to do with databases, especially fun legacy environments with multiple disparate data sources that all need glued together!
- Medical Imaging – I’ve developed a powerful C++ and Python implementation of the DICOM standard that’s being used at a number of institutions, and I maintain the associated support website : http://dicomlib.swri.ca
- Creating systems and workflow to efficiently deliver new services.
- Analysing, simplifying and improving legacy systems.
People I work for
- Major Banks
- Oil companies
- Non-profits
- Research hospitals
- Consulting firms
- Friends
- Software development companies
- Event managers
- Photographers
- Anyone else…
What is my philosophy of systems development?
“Il semble que la perfection soit atteinte non quand il n’y a plus rien à ajouter, mais quand il n’y a plus rien à retrancher.”
(Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.)
Antoine de Saint-Exupéry
Posted in Programming, c++, python, work | No Comments »
May 30th, 2007
Created a new site for a client on WebFaction, using TurboGears. A few small bumps here and there, but mostly a smooth and fairly painless process. Went from not having a site to having a fully functional TurboGears site with postgres backend, subversion and trac implementation, in about a day. (Including homeschooling all morning and rigging a photoshoot in the evening!)
The site doesn’t do a whole lot yet, but I have a hunch that setting up the infrastructure will have been a significant part of the job. I think that WebFaction could be a very useful resource – they can get you up and running with a site complete with Rails/TurboGears/Django/Framework of your choice quite efficiently.
Posted in Programming, python, work | No Comments »
May 18th, 2007
So, getting somewhere with TurboGears, although I’ve been banging my head a little against SQLObject. While it’s indubitably a nice tool if you’re looking for an object persistence mechanism, it can get a bit frustrating if you actually want to treat your relational database like a relational database. There seem to be a lot of tools available that promise to shield you from dealing with SQL, but generally speaking I don’t want to be shielded. The strength of the relational model is not that you can rapidly access a big list of data elements, but that you can slice, join and view this data in a huge variety of ways.
Specifically I’ve been trying to implement a many-to-many relation where the intermediate table has attributes, and then do a join across this table filtered by these attributes. Trivial in SQL, and frustratingly difficult in SQLObject. In fact I’ve shelved it for now while I concentrate on other issues.
Posted in Programming, databases | No Comments »