A Surprising Encounter

Here's a video of me at the NHS Hackday in London a couple of months ago. I was hanging with Rob Tweed, undoubtedly the most interesting and forward thinking man in Mumps. And the man behind   EWD, which puts Mumps into the enterprise and behind an array of extremely trendy modern web-based front ends.

Our team's project, (entirely based on Rob's work, with data wrangling from Chris), was to pull NHS data into a Semantic Web format where it could be queried with SPARQL and shown in a variety of widgets from the MIT SMART platform for healthcare records.

Rob has big ambitions. In a world where healthcare IT and patient records are a scandalous litany of expensive, failed and cancelled projects wasting millions of dollars, Mumps based VistA is fast, solid (with 40 years of development behind it), and free (as in speech) software. Thanks to Rob and SMART it can start to present its data through standardized HTML5 apps that are as slick and up-to-date as anyone else's. And Rob has plans to make the whole thing available as single downloadable virtual machine. (I think, in VirtualBox format.)

To repeat, that's the equivalent of a £12 billion system (except ... er ... working) as a free download. That's the virtue of free software, folks.

(Of course, yes there is still a huge installation, training, integration cost involved in putting a system like this in place, but if you start with something that a) exists and is b) free then you might actually get some kind of result for your 12 billion.)

Update : Why is this surprising? Well, partly because I'm kind of out of the Mumps world at the moment, and went along to the NHS hackday without any thought Rob would be there or that I'd end up in the Mumps team. But I've been a fan of his work so it was great to meet him.  Also, I'm a SemWeb sceptic, so I have no idea what I'm doing in a project which is about using RDF and SPARQL as its standard file-format, the way SMART does. :-) 

Wow. This guy has ported Mumps to Android.

Nice.

I'm still sparring with Rob Tweed over MUMPS's dynamic scoping.

The Mumps scope conversation is hotting up (see Rob's answer and comments). And he leads me to a new MUMPS blog.

I just answered a question about MUMPS on programmers.stackexchange.com

Having been out of that world for a couple of years, my thinking is that the NoSQL movement is probably the best and worst thing to happen to MUMPS. Ultimately it's likely to both vindicate and kill it. Because somebody, at some point, is going to reinvent MUMPS's database and query structure almost identically, but with no connection to the MUMPS tradition. Then people will rave about this new storage system. But no-one will ever choose a MUMPS derivative again.

For example, a month or two ago, I was talking with a colleague about using redis to cache a look-up of something in our Django application. We had a large number of records addressed by a triple of three keys, and needed to quickly get subsets matching one or two of these keys (but different combinations at different times).

This is the kind of thing that MUMPS eats for breakfast. But we were finding it hard to squash into redis's key,val pair structure. Even with dictionaries. (Same would be true of memcached etc.) For the first time in my life, I found myself actually regretting that I couldn't write this module in Cache ObjectScript.

Dell Streak tablets part of their health-care strategy.

Andy Oram at O'Reilly on VistA.

I wouldn't dare guess whether VistA will continue as a niche product or will suddenly emerge to eat up the U.S. market for electronic medical systems. But I think it's definitely something to watch.

Philip Greenspun's suggestions for reform of the US Health system.

Here come the healthcare apps.

Tim O'Reilly on open patient record initiatives

OK. It's been six months or more since I last updated this blog.

Astute readers will probably have realized that I've fallen out of the Cache world. My day-job is now working with Python and Django which, of course, makes me very happy.

But I have a bit of sadness too. I learned a LOT from working with Cache and ObjectScript. Some of it was painful. Some of it was weird. But much was enlightening. I often found myself realizing how different a programmer's world could be from that which I considered mainstream. I'm grateful that I had the chance to experience that. It made me a better (and humbler) programmer. Humbler because it doesn't matter how bad something looks, chances are it may have seemed, or even been, the right solution to some problem, somewhere, somewhen. Seeing that, teaches perspective.

Furthermore, I stand by the things I've written before. I still believe that both the theme of old-skool legacy medical systems coming to terms with the modern web 2.0 environment, and Intersystems' platform's potential for reinventing itself, are interesting.

I'd like to know more and follow both. OTOH, my new day-job and new set of concerns are taking up way too much time. If I could, I'd have attended the Slipstream Workshop this coming week, but as it's a work-day, I won't be able to. Good luck for everyone who's attending, anyway.

Finally, this is not goodbye. If any MUMPS related thoughts or news comes my way, I'll still pass them on. And if anyone else out there in the Cache or MUMPS communities would like to take over and continue this blog, I'd be happy to add you as a co-author. Just drop me a note in the comments here.

L8r peeps ...

PS : I'm extremely impressed with GitHub at the moment. So much so that I've decided that it's worth moving Twistah across to it from the rather unexciting Google Code. The new repository is http://github.com/interstar/Twistah/tree/master.

The anti-MUMPS troll at the Daily WTF? (admittedly, we've all been there) has another anti-MUMPS story.

If you've been following the discussion over Mumps : the IT world's best kept secret? you'll have noticed that one thing that stands out about Caché / MUMPS is its unusually structured data-store of persistent, multi-dimensional ragged-arrays. And Rob Tweed makes a case that this is a better fit than a relational database for internet scale applications. He cites Google's BigTable and Amazon's S3 as examples of a trend away from relational dbs in favour of key value pairs.

Today Steve Yegge has an interesting meditation on a similar theme, about "domain modelling schools" and the need for programmers to have experience of multiple tools and organizing principles. (And while he doesn't note the MUMPS model, it would be another interesting one to add to the list.)

Worth reading, also, this Cringely piece on how relational databases don't work so well in a parallel cloud where disk-access is often a bottleneck so more of the live database is moved into memory. (Killer quote : "Google has THE ENTIRE INTERNET IN MEMORY AT ONCE.") Disk is just used for backup / persistence."

More discussion of Rob's future of Mumps is going on on the mailing list and in comments to his original post.

Rob Tweed asks if MUMPS is dead

Worth reading his thoughts. Some intriguing ideas, such as his own work making the MUMPS db accessible from Python and other scripting languages, and the parallels between the MUMPS db and Google Application Engine and Amazon's AWS.

Workaround for the Scoping Gotcha

My colleague pointed out to me today that you can get around the scoping issue I mentioned by writing your method like this :

 
  Method f() {
    set x = 6
    set m = "write !,"_x ; difference
    xecute m
  }
 

What's the difference? Instead of including the name of the variable x inside the string, you concatenate the value of the local variable x with the string at run-time, before the string is finally evaluated. Because the value of x is resolved before the xecute, things work as you'd expect.

That's true. But I still regard this as a serious bug. An equivalent python program :

 
class A :
    def f(self) :
        x = 6
        m = "print x"
        exec m

x = 9
a = A()
a.f()
 

prints what you'd expect : the number 6.

What's really going on to cause this issue in Caché? My guess is that the OO layer in Caché is compiled down to plain Caché .int routines which don't have a significantly different semantics or scoping rules to Caché ObjectScript. In order to get the effect of the private world inside the object, at compile time the variable x is renamed to some object-specific equivalent. Of course, this renaming affects the real references to x, but not the string assigned to m.

When the VM then tries to xecute the string, it encounters the name x but finds no binding in the local execution frame and so, according to COS's dynamic scoping rules, has to look down the stack for a frame which does have a binding, the place from which we call y.f(), where x is bound to 9.

Of course, this hypothesis might be completely wrong. Expert correction is welcome.

Don't be confused. This is NOT Caché ObjectScript. :-)

Xecute Scoping

A nasty gotcha in scoping when you try to use Caché's "interpret a string containing code" command xecute inside an object.

You can execute (ie. eval) a string containing a piece of Caché ObjectScript with xecute like this :

set m = "write 2+2"
xecute m

xecute only works on statements, not expressions. But you can use the alternative @ for that. Eg.

set m = "2+2"
write @m

However, if you try to use this inside a class definition, something odd happens.

When a class is compiled, the string which contains code is obviously not affected by the pre-processor. But it seems that when the method is run and the string xecuted, the variable names in the string don't get bound to the scope within the method. Instead they get their values from the global frame or somewhere else entirely.

Look at the following example.

A class :

Class MY.ScopeGotcha Extends %Persistent [ ClassType = persistent, ProcedureBlock ]
{

  Method f() {
    set x = 6
    set m = "write !,x"
    xecute m
  }

}

Now try running it from the terminal like this :

 set x = 9
 set y = ##class(MY.ScopeGotcha).%New()
 do y.f()

Guess what it prints ... :-)

You know what I really wish?

I wish Caché had decent regular expressions. The same ones that Perl made ubiquitous and are supported in C++, Java, Python, Ruby, PHP, Javascript and every other language you can think of.

It's hard to write a wiki without good string processing and regexes.

Intersystems's documentation for Caché is notoriously bad.

Here's a classic example I found today for the "super" keyword.

Yes, I realize "super" has something to do with super-classes and inheritance and so the "extends" keyword is relevant ... but ... er ... wouldn't you expect the "super" example to actually *use* the "super" keyword? Somewhere?

Or if there is no "super" keyword, because "extends" is how you declare inheritance relationships, what on earth is this page doing here?

Update : after further consideration, the only way I can decode this is that "super" is the name of the list of super-classes. That's why the super "keyword" can "have a value".

This is typical of Intersystems' often garbled approach to documentation. Don't they know what the word "keyword" actually means???