For me, that time is now, that library is ConfigParser, and the replacement is included below the fold. It’s called conf, and it means that the only interaction you as a programmer are required to have with your config file is to assign and/or read values to/from it. Assigning and reading look exactly like normal attribute assignment/reading.

Subversion/TRAC string: https://svn.coriolinus.net/OOconf
distutils packaged version: conf-0.2.0

As always, I’m too cheap to pay a thousand dollars for a site certificate, so please ignore any certificate mismatch errors you encounter when viewing the https side of the site. If you don’t trust me enough to click through, you’ll still find the current version under the fold.

import os
import codecs
import json
from ConfigParser import SafeConfigParser

def internal(func):
	def tf(self, *args, **kwargs):
		self.__dict__['__conf__'].__dict__['__internal__'] += 1
		try:
			return func(self, *args, **kwargs)
		finally:
			self.__dict__['__conf__'].__dict__['__internal__'] -= 1
	return tf

class Section(object):
	def __init__(self, conf, name, fullname=None):
		if fullname is None:
			fullname = name

		self.__dict__['__conf__'] = conf
		self.__name__ = name
		self.__fullname__ = fullname

		self.__dict__['subsections'] = set()
		self.__dict__['attributes'] = set()

		#this lines MUST be the last in __init__
		self.__restrictedvars__ = set((i for i in dir(self) if '__' not in i))

	@internal
	def __setattr__(self, name, value):
		if self.restricted(name):
			if (not self.__conf__.__restrict__) or self.__conf__.__internal__ > 0:
				self.__dict__[name] = value
			else:
				raise AttributeError("Namespace conflict: %s restricted for Conf use." % name)
		else:
			self.__conf__.__new_data__ = True
			self.attributes.add(name)
			self.__conf__.__cp__.set(self.__fullname__, name, json.dumps(value))

	@internal
	def __getattr__(self, name):
		if self.restricted(name):
			return self.__dict__[name]
		else:
			return json.loads(self.__conf__.__cp__.get(self.__fullname__, name))

	@internal
	def __delattr__(self, name):
		if self.restricted(name):
			if (not self.__conf__.__restrict__) or self.__conf__.__internal__ > 0:
				del self.__dict__[name]
			else:
				raise AttributeError("Namespace conflict: %s restricted for Conf use." % name)
		else:
			self.__conf__.__new_data__ = True
			self.attributes.remove(name)
			self.__conf__.__cp__.remove_option(self.__name__, name)

	@internal
	def restricted(self, name):
		"""
		This function returns true for all attributes which should be stored locally, not in the
		configuration file proper.
		"""
		if name.startswith('__') or name.endswith('__'):
			return True
		if name in self.__restrictedvars__:
			return True

		return False

	@internal
	def add_section(self, name):
		fullname = ''.join((self.__fullname__, '.', name))

		if hasattr(self, name):
			raise ValueError("Namespace conflict: %s already in use" % fullname)

		if not self.__conf__.__cp__.has_section(fullname):
			self.__conf__.__cp__.add_section(fullname)
		self.subsections.add(name)
		self.__dict__[name] = Section(self.__conf__, name, fullname)
		self.__restrictedvars__.add(name)
		self.__new_data__ = True

	@internal
	def remove_section(self, name):
		fullname = ''.join((self.__fullname__, '.', name))

		if not hasattr(self, name):
			raise ValueError("Can't remove section %s, as it doesn't exist" % fullname)

		sub = getattr(self, name)
		for subsub in list(sub.subsections):
			sub.remove_section(subsub)

		self.__conf__.__cp__.remove_section(fullname)
		self.subsections.remove(name)
		self.__restrictedvars__.remove(name)
		del self.__dict__[name]
		self.__new_data__ = True

class Conf(Section):
	"""
	Automatic storage and retrieval of arbitrary values into a config file. 

	Uses type information and automatic reconversions to store a variety of primitive types in a
	perfectly human-readable format. Primitive types are those encodable by the json module.

	Usage:
	>>> from conf import Conf
	>>> conf = Conf() #or Conf(filename, defaultsectionname)
	>>> conf.foo = 'hello world'
	>>> conf.bar = 723
	>>> conf.baz = False
	>>> conf.flush()

	[exit, start a new session here]

	>>> conf = Conf()
	>>> conf.foo
	'hello world'
	>>> conf.bar
	723
	>>> conf.baz
	False

	If you want implicit file creation, you need to use a with statement:
	>>> with Conf() as conf:
	...      conf.foo = 2783.1
	...
	>>> del conf
	>>> with Conf() as otherConf:
	...      otherConf.foo
	...
	2783.1
	"""
	def __init__(self, filename ='.conf', sectionName=None):
		"""
		Initialize a new Conf object.
		"""
		self.__dict__['__internal__'] = 0
		self.__dict__['__restrict__'] = False
		#the above is magic; it must come first

		#sanity check
		if sectionName is not None and '.' in sectionName:
			raise ValueError("Namespace: '.' cannot be part of a section name")

		#initialize the superclass
		Section.__init__(self, self, sectionName if sectionName is not None else 'config')

		self.__filename__ = filename
		self.__cp__ = SafeConfigParser()

		#load and initialize the configparser
		if os.path.exists(filename):
			with codecs.open(filename, 'rb', 'utf8') as cf:
				self.__cp__.readfp(cf, filename)

		if sectionName is None:
			topLevelSections = [s for s in self.__cp__.sections() if s.count('.') == 0]
			if len(topLevelSections) == 1:
				sectionName = topLevelSections[0]
				self.__name__ = sectionName
				self.__fullname__ = sectionName
			else:
				sectionName = 'config'

		#create the default section
		if not self.__cp__.has_section(sectionName):
			self.__cp__.add_section(sectionName)

		#load default section attributes
		self.attributes = set(self.__cp__.options(sectionName))

		#load the various sections
		#first, sort them by the number of dots they contain
		secs = [(s.count('.'), s) for s in self.__cp__.sections() if s != sectionName]
		secs.sort()
		for name in [sec for count, sec in secs]:
			if '.' not in name:
				self.subsections.add(name)
				self.__dict__[name] = Section(self, name)
				self.__dict__[name].attributes = set(self.__cp__.options(name))
				self.__restrictedvars__.add(name)
			else:
				fullname = name
				rest, name = name.rsplit('.', 1)
				sec = self
				for part in rest.split('.'):
					sec = getattr(sec, part)
				sec.subsections.add(name)
				sec.__dict__[name] = Section(self, name, fullname)
				sec.__dict__[name].attributes = set(self.__cp__.options(fullname))
				sec.__restrictedvars__.add(name)

		self.__new_data__ = False

		#this must come last:
		self.__internal__ = 0
		self.__restrict__ = True
		#This is used in combination with the @internal decorator. Each method so decorated
		# increments this variable on entry, and decrements it on exit. They can then check: is
		# __internal__ > 0? If yes, they were called from within this Conf object, and can adjust
		# their behavior accordingly.
		#
		#Note that we initialize it here to 0. When this __init__ exits, it decrements to -1. This
		# is intentional. Since each internal function starts by incrementing it, this means that
		# only if the variable is > 0 was its caller also internal.

	@internal
	def __enter__(self):
		return self

	@internal
	def __exit__(self, exc_type, exc_value, traceback):
		if self.__new_data__:
			self.flush()

	@internal
	def flush(self):
		with codecs.open(self.__filename__, 'wb', 'utf8') as cf:
			self.__cp__.write(cf)
		self.__new_data__ = False

	@internal
	def add_section(self, name):
		"""
		Create a new section in the conf file. This will become a dotted extension of the conf object.

		For example:
		>>> c = Conf()
		>>> c.foo = 'hello world'
		>>> c.add_section('bar')
		>>> c.bar.baz = 'world says hello'

		The above turns into a config file which looks like this:
		[config]
		foo = hello world

		[bar]
		baz = world says hello
		"""
		if hasattr(self, name):
			raise ValueError("Namespace conflict: %s already in use" % name)
		if '.' in name:
			raise ValueError("Namespace: '.' cannot be part of a section name")

		if not self.__conf__.__cp__.has_section(name):
			self.__conf__.__cp__.add_section(name)
		self.subsections.add(name)
		self.__dict__[name] = Section(self, name)
		self.__restrictedvars__.add(name)
		self.__new_data__ = True

	@internal
	def remove_section(self, name):
		"""
		Remove a section and all included data.
		"""
		if not hasattr(self, name):
			raise ValueError("Can't remove section %s, as it doesn't exist" % name)

		sub = getattr(self, name)
		for subsub in sub.subsections:
			sub.remove_section(subsub)

		self.__cp__.remove_section(name)
		self.subsections.remove(name)
		self.__restrictedvars__.remove(name)
		del self.__dict__[name]
		self.__new_data__ = True

conf.py by coriolinus is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
Permissions beyond the scope of this license may be available at http://www.coriolinus.net/contact/.

I also learned some things:

1. I am very, very out of practice. More than four hours into the exercise, I was still debugging. The bugs were things like MSSQL Optional Feature Not Implemented, not actual logic errors, but still. That’s too long given the complexity of the task.
2. I have a lot of fun coding. In a basically unprecedented move, I was delaying leaving work until the guy whose office I was borrowing made me leave so he could lock up. Especially given that I’d had an 11 hour day at that point, this is a significant development.
3. For all that I rag on MS products, the MSSQL query designer really does take a lot of work out of the process of writing complex queries.

Actually, 90 minutes into the exercise, a civilian contractor came by and worked magic and solved the problem for which I was writing code in the first place. I kept working, using the excuse that my version will be more featureful than the Army’s, and that by having the source to it the Army will benefit. The real reason is much simpler: I’m having way too much fun to just give this project up. I am perpetually at the 50% mark and working rapidly towards completion; I’m not going to let this just escape me.

More images here, at the flickr set.

Say you took all the programming languages I have ever encountered and put them into a table. If you were to assign a column to ease of use, a column to language power, and a third column to the product of the previous two columns, then reverse sort the table by the last column, Python would be the first language in the table.

I have to laugh at parts of the release document. For example, the claim that programming languages started simple and easy to learn, and that the high-level concepts of modern languages discourage people from learning to code. “Hello World” is a one-liner only in a high-level language; it’ll take a beginner a few days of work and tons of documentation to accomplish in assembly. I also loved the section titled “Rules for naming Variables”: “[TODO]”

In general, this looks like a rehash of QBasic, of which all the warts are still present. The language lesson introduces GOTO long before talking about subroutines. Subs (not functions) access and modify global variables, and are limited to the current file’s scope. The major differences seem to be that most of the library functions have been rewritten in an object-oriented style and additional libraries for GUI stuff and Turtle programming have been added. Even though the library uses objects, the user has no way of writing their own.

I can see some junior coder at Microsoft doing this as a project for promotion points. It very much feels like a student project. However, I can’t see any programmer using it to actually teach programming. Small Basic might be useful if your ambition is to one day work up to Visual Basic as an enterprise programming language, but people interested in learning modern programming techniques would be better advised to look elsewhere for their first language.

Quite recently, I upgraded to iTunes 7.7. My little utility stopped working. When I finally got around to looking into the error, I found this:

Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import win32com.client
>>> itunes = win32com.client.Dispatch("iTunes.Application")
>>> help(itunes._print_details_)
Help on method _print_details_ in module win32com.client.dynamic:

_print_details_(self) method of win32com.client.CDispatch instance
Debug routine - dumps what it knows about an object.

>>> itunes._print_details_()
AxDispatch container iTunes.Application
Methods:
Props:
Get Props:
Put Props:
>>>

You’ll note that as far as COM is concerned, iTunes no longer publishes any methods or properties in its only externally instantiable interface. In other words, it’s become a black box with no inputs or outputs. You can tell that it’s there, but you can’t interact with it in any way.

I may try again in some other language; it’s possible that iTunes is somehow no longer compatible with the python win32com implementation, but still works with others. Still, it’s annoying.

[edit 20080914]

Things work nicely again in iTunes 8.0.0.35; hopefully they stay working this time.

Also, I discovered that my code example above is not very helpful. It seems that the methods in the win32com client object are lazily added the first time they are called, so _print_details_() will only ever show methods that have already been called successfully. I suppose I should be happy that they implemented COM support at all, instead of being annoyed that introspection doesn’t work as well as python has accustomed me to.

XML and Java have the same sort of flavor to them: they’re reasonably good and very widely used; they’re the sort of product that design committees everywhere aspire to create. Their flaws only really become visible after something better comes along. In Java’s case, Python demonstrated that a whole lot of the structure and required text that gives Java code its rigidity can be stripped away, leaving a language that’s a joy to develop in. However, there hasn’t been an analogous improvement on XML.

Until yesterday.

Protocol Buffers have a non-descriptive name; I had no idea what to expect when I clicked the link to the announcement that Google put out. As it turns out, they’re a generic data serialization format (much like XML), except without all the human-readability business that so bloats actual XML. From the announcement:

Protocol Buffers allow you to define simple data structures in a special definition language, then compile them to produce classes to represent those structures in the language of your choice. These classes come complete with heavily-optimized code to parse and serialize your message in an extremely compact format. Best of all, the classes are easy to use: each field has simple “get” and “set” methods, and once you’re ready, serializing the whole thing to – or parsing it from – a byte array or an I/O stream just takes a single method call.

In case you missed that, all you have to write is the schema. All the encoding and decoding crap that you have to wade through in XML has already been abstracted away; they generate classes to do that for you. This is, in fact, cooler than sliced bread.

Of course, there do exist times when XML might better serve your needs:

However, protocol buffers are not always a better solution than XML – for instance, protocol buffers would not be a good way to model a text-based document with markup (e.g. HTML), since you cannot easily interleave structure with text. In addition, XML is human-readable and human-editable; protocol buffers, at least in their native format, are not. XML is also – to some extent – self-describing. A protocol buffer is only meaningful if you have the message definition (the .proto file).

In my experience, the human-readability and self-documentation inherent in XML have always been bonus features not essential to the core mission, which was getting data from Point A to Point B. However, I’ve had to spend countless hours wrangling with DOM and SAX, dealing with the problem of getting the data into and out of that intermediate form.

There is one wart that I noticed: you still have to create and read the Messages entirely distinctly from your own native class structure. The natural thing to do, if you want to use this to serialize and deserialize a class, would be just to put all the members into the Message definition and put the methods into a subclass of the generated class. However, that is expressly forbidden. All is not lost, though: all you really need, at simplest, is a pair of methods like this:

class AClass(object):
     ...
     def toPBuff(self):
          out = AClassPBuff()
          for member in dir(self):
               if not (callable(member) or '__' in member or member in self.__excludeFromSerialize):
                    setattr(out, member, getattr(self, member))
          return out

     @classmethod
     def fromPBuff(cls, pBuff):
          out = AClass()
          for member in dir(out):
               if not (callable(member) or '__' in member or member in self.__excludeFromSerialize):
                    setattr(out, member, getattr(pBuff, member))
          return out

In short, even if only in terms of making efficient use of developer time, this is already an awesome project. Once you count in that it is also faster and slimmer than the alternatives, this becomes astonishingly cool. Expect it to be making appearances in my code from now on.

“But,” you object, “what happened to that C++ project you were working on?”

Basically, I got bored**. I have a fairly high tolerance for doing background research before jumping into a project, but the sheer volume of required reading to even have an idea of how to start is fairly daunting. I just figured it was better to code something than walk away entirely.

I was going to stay a little bit on focus, write a spinning globe in C++ using OpenGL. Then I started looking into online documentation and found absolutely nothing that gave me a starting point. I couldn’t even find a website that told me which header files to include to get at the API, let alone documentation of the API itself.

Clearly, that’s a project for another day. Taking the second best option, I decided to see what was doable with PyGame. I’d heard the name before without having actually used it for anything; it was only natural to see what it could do. As it turns out, it focuses more on 2d graphics than 3d. A rotating image of a globe would amuse nobody, so I changed my design goal, and ended up with this.

Playing with graphics, once you get past the initial hurdles, is kind of fun. There may be more of this later.

** “But what about the computer games and novels you can be trusted to fritter away your time with before starting any productive work at all?” XCOM crashed one time too many, its open-source replacement is years from playability, and I thought it’d be a neat feather in my hat if I could get some measurable percentage done on my own replacement today. As things turned out, that feather eluded my grasp.

Then I got to the part about the virtual keyword.

In every other object-oriented language I’ve ever learned, complete support for polymorphism was just assumed. If you subclass some class and override some of its methods, then naturally whenever you call those methods on an instance of your subclass, you get your overrides. I can’t even describe this without sounding tautological, because all the terminology in OOP just assumes that things work this way. Polymorphism is such a fundamental concept that I never really considered that a language would call itself Object Oriented without supporting it inherently.

Then I got to C++. Consider this code:

class A {
public:
     void method();
};

class B : public A {
public:
     void method();
};

void demo_fail();

void A::method() {
     std::cout << "    A::method() called\n";
}

void B::method() {
     std::cout << "    B::method() called\n";
}

void demo_fail() {
     A *a = new A();
     A *b = new B();

     std::cout << "Calling a->method()...\n";
     a->method();
     std::cout << "Calling b->method()...\n";
     b->method();
     std::cout << "\n...What?\n";
}

You’d expect the above code to happily call A’s method, then B’s. Instead, the output looks like this:

It turns out that C++ does support polymorphism; it’s just that someone decided not to enable it by default. The virtual keyword is the key here; by adding it to the prototype of the method of the superclass, things start working again:

class Alpha {
public:
     virtual void method();
};

class Bravo : public Alpha {
public:
     void method();
};

void demo_succeed();

void Alpha::method() {
     std::cout << "    Alpha::method() called\n";
}

void Bravo::method() {
     std::cout << "    Bravo::method() called\n";
}

void demo_succeed() {
     Alpha *alpha = new Alpha();
     Alpha *bravo = new Bravo();

     std::cout << "Calling alpha->method()...\n";
     alpha->method();
     std::cout << "Calling bravo->method()...\n";
     bravo->method();
     std::cout << "\nThe |virtual| keyword is essential.\n";
}

The problem here isn’t so much that you need an arbitrary keyword to enable one of the fundamentals of OOP; it’s where that keyword has to go. Useful programming in the modern era depends on libraries of code that the individual coder doesn’t need, or want, to maintain. One of the nice things about OOP is that you can use a library as long as it’s useful, and even a bit longer: just subclass and extend until you have all the functionality you need. In C++, that only works if the library author has correctly anticipated exactly which functions you will need to override. In practice, this means that the author probably just stuck the virtual keyword onto every public and protected function, neatly negating the tiny compiler optimization that was the whole point of not making the ‘virtual’ behavior the default.

I have to admit that I am starting to dread getting to the section on exceptions. According to a friend of mine, C++ exceptions are hacktastic.