| Andrew Cooke | Contents | Latest | RSS | Twitter | Previous | Next

C[omp]ute

Welcome to my blog, which was once a mailing list of the same name and is still generated by mail. Please reply via the "comment" links.

Always interested in offers/projects/new ideas. Eclectic experience in fields like: numerical computing; Python web; Java enterprise; functional languages; GPGPU; SQL databases; etc. Based in Santiago, Chile; telecommute worldwide. CV; email.

Personal Projects

Lepl parser for Python.

Colorless Green.

Photography around Santiago.

SVG experiment.

Professional Portfolio

Calibration of seismometers.

Data access via web services.

Cache rewrite.

Extending OpenSSH.

Last 100 entries

Santiago 30m Bike Route; Mapa de Ciclovias en Santiago; How Unreliable is UDP?; SE Santiago 20m Bike Route; Cameron's Rap; Configuring libxml with Eclipse; Reducing Combinatorial Complexity With Occam - AI; Sentidos Comunes (Chilean Online Magazine); Hilary Mantel: The Assassination of Margaret Thatcher - August 6th 1983; NSA Interceptng Gmail During Delivery; General IIR Filters; What's happening with Scala?; Interesting (But Largely Illegible) Typeface; Retiring Essentialism; Poorest in UK, Poorest in N Europe; I Want To Be A Redneck!; Reverse Racism; The Lost Art Of Nomography; IBM Data Center (Photo); Interesting Account Of Gamma Hack; The Most Interesting Audiophile In The World; How did the first world war actually end?; Ky - Restaurant Santiago; The Black Dork Lives!; The UN Requires Unaninmous Decisions; LPIR - Steganography in Practice; How I Am 6; Clear Explanation of Verizon / Level 3 / Netflix; Teenage Girls; Formalising NSA Attacks; Switching Brakes (Tektro Hydraulic); Naim NAP 100 (Power Amp); AKG 550 First Impressions; Facebook manipulates emotions (no really); Map Reduce "No Longer Used" At Google; Removing RAID metadata; New Bike (Good Bike Shop, Santiago Chile); Removing APE Tags in Linux; Compiling Python 3.0 With GCC 4.8; Maven is Amazing; Generating Docs from a GitHub Wiki; Modular Shelves; Bash Best Practices; Good Emergency Gasfiter (Santiago, Chile); Readings in Recent Architecture; Roger Casement; Integrated Information Theory (Or Not); Possibly undefined macro AC_ENABLE_SHARED; Update on Charges; Sunburst Visualisation; Spectral Embeddings (Distances -> Coordinates); Introduction to Causality; Filtering To Help Colour-Blindness; ASUS 1015E-DS02 Too; Ready Player One; Writing Clear, Fast Julia Code; List of LatAm Novels; Running (for women); Building a Jenkins Plugin and a Jar (for Command Line use); Headphone Test Recordings; Causal Consistency; The Quest for Randomness; Chat Wars; Real-life Financial Co Without ACID Database...; Flexible Muscle-Based Locomotion for Bipedal Creatures; SQL Performance Explained; The Little Manual of API Design; Multiple Word Sizes; CRC - Next Steps; FizzBuzz; Update on CRCs; Decent Links / Discussion Community; Automated Reasoning About LLVM Optimizations and Undefined Behavior; A Painless Guide To CRC Error Detection Algorithms; Tests in Julia; Dave Eggers: what's so funny about peace, love and Starship?; Cello - High Level C Programming; autoreconf needs tar; Will Self Goes To Heathrow; Top 5 BioInformatics Papers; Vasovagal Response; Good Food in Vina; Chilean Drug Criminals Use Subsitution Cipher; Adrenaline; Stiglitz on the Impact of Technology; Why Not; How I Am 5; Lenovo X240 OpenSuse 13.1; NSA and GCHQ - Psychological Trolls; Finite Fields in Julia (Defining Your Own Number Type); Julian Assange; Starting Qemu on OpenSuse; Noisy GAs/TMs; Venezuela; Reinstalling GRUB with EFI; Instructions For Disabling KDE Indexing; Evolving Speakers; Changing Salt Size in Simple Crypt 3.0.0; Logarithmic Map (Moved); More Info; Words Found in Voynich Manuscript

© 2006-2013 Andrew Cooke (site) / post authors (content).

OOPSLA Summary

From: "andrew cooke" <andrew@...>

Date: Sat, 28 Oct 2006 18:27:35 -0300 (CLST)

[Sent this to various people at work]

This is a summary of the stuff I thought was interesting at OOPSLA.
The level of detail is a bit inconsistent, so if you get to a bit
which makes no sense just skip on to another paragraph.


OOPSLA is probably the best conference I've been to.  Having said
that, there's still a lot of room for improvement.  There's a
surprising amount of time spent on either long single-person
presentations, or discussions between a bunch of people (many of whom
have given presentations earlier).  It gives the distinct impression
that there's an OOPSLA clique who run the place and talk to the rest
of us.

And having said that, some of those people are pretty good.  One of
the highlights was Guy Steele's presentation of Fortress, a new
language (still in development, part of a DARPA funded competition)
for scientific computing (in fact, I think it could be very general).


Scientific Computing

Fortress is cool on a pile of new fronts: extensibility, parallelism,
notation.

Notation: at one point they seriously considered programming in Latex
(which gained a round of applause).  The idea is that you write the
program pretty much like you write maths.  So there's a big emphasis
on operators - set union is the "U" character.  Obviously this
requires unicode and/or special editors, so there's also an ASCII
equivalent (the word "UNION" in this case).

Extensibility: the language is designed to have almost everything
implemented in libraries.  So there's a kernel that the "end user"
doesn't see that much, and libraries (which is what the end user
relies on) layered on top of that.  This raises a whole pile of issues
related efficiency - the compiler needs specialised knowledge about
library code to make it efficient - which are solved by having traits
(rather like Java interfaces) that indicate various algebraic
properties.  An example of how this is useful below.

Parallelism: Given the above, it's no surprise that the exact approach
to parallelism depends on the libraries you use.  This seems like it
would be a complete mess, but they have a neat scheme that separates
parallel processing into distribution and collection.  Distribution
decides what goes where, and collection collects the results.  You can
perhaps see how knowing if collection is associative or commutative,
for example, affects what algorithms you can use.  And that
information comes from the traits.

(Parallelism cont).  There's a bunch of maths behind this (Steele
continually name-checked Haskell).  The collection process above is a
catamorphism, which I didn't recognise, but it's the banana stuff from
various papers of Bird et al.  Which in turn is somehow related to
categories (initial algebras generate structures and co-algebras can
be though of harvesting information; the two are related by reversing
arrows in the category diagram - this wasn't in the talk, but I think
it's what's happening).

Anyway, you won't need to know this to use Fortress.  If it all works
you'll be able to program scientific code much like a cross between
maths and Fortran (did I say it supported checking of physical
units?), and everything will magically work in a whole variety of
different hardware configurations.


Aspects, DSLs

Another thing that surprised me was the absence of Microsoft (I'm
writing this during an address from one of the organisers, and they've
just shown a list of sponsors that includes MS!), and any research on
MS languages.  This was very much a Java conference.  As a
consequence, perhaps, there wasn't that much in the way of innovation
with languages.  Aspects were the one idea kicking around within
languages; other than that people were relying on separate tools.

The general feeling about aspects is that there's something useful
there, but that the current approach isn't a final solution.  One
interesting research paper showed a variation that suggested a very
restricted functionality, something like a "lens" that can be used to
view objects within a certain scope.  This lens is implemented as a
kind of local multiple inheritance that is imposed on the object in
question.  So, for example, you can adapt an object to match a
different interface before calling a library.  That's what I use
aspects for, but it's way, way less powerful than what we have now.

Tools, rather than language extensions, were also the emphasis in a
session on Domain Specific Modelling (DSM) that I attended.  This is
closely related to Domain Specific Languages (DSL), which I am a big
fan of.  Perhaps because of that I was pretty disappointed with the
talks here.  Some good ideas, but usually without much foundation in
theory.  That gives some nice engineering, but little meat to take
away and use in a different situation.  And what I thought was the big
new thing - strongly typed meta-programming, which lets you extend
existing languages "at will" without needing to introduce a separate
tool - was never mentioned.

One talk was an exception, and introduced me to some new maths
(co-algebras) which, it turned out, are related to some old maths I
only vaguely understand (category theory).  This is the same kind of
thing bubbling under Fortress's parallelism - it's there, but very few
people are talking about it.


Agile Development

In parallel with the main conference are a set of tutorials.  These
cost extra, but I had signed up for two (a thank-you to Sonya for
contributing her allocation to me).

My first tutorial was on "Crystal Clear" - an approach to agile
development.  The presentation, by Alastair Cockburn, was pretty good
- very down to earth and pragmatic.  He also answered our questions,
even if only to say, sometimes, that we needed to find our own
solutions to locale-specific problems - the biggest thing I finally
understood is that Agile (or, at least, Cockburn's take on it) is
flexible.  There's no one right way and each project requires not just
its own methodology, but also frequent reflection and revision.

(At the same time, there are a core set of targets ("properties") and
general approaches.  I will give some of these later.)

Acknowledging this flexibility, Cockburn recommends a regular
"reflection workshop" where people discuss the previous iteration (or
whatever - once a month was suggested at one point), identify things
to keep, things to try, and problems that need to be elevated to
management.

This relates to the quality talk at ADASS and comments Evan has been
making recently, I think.

But there's also another, related angle.  Agile isn't just about
technical decisions.  Or rather, technical discussions aren't always
conclusive.  There's a very important reliance on inter-personal
skills and negotiation so that the team as a whole can resolve
conflict.  This goes further than someone making a final technical
decision - it's about "vibes": trust, mutual respect, understanding...

Cockburn repeatedly emphasised delivery.  Frequency of delivery, not
"iterations" (planning cycles) is a core property of Agile; focusing
on iterations at the expense of delivery is a common problem.

A related point was the idea of a "friendly user".  Someone we can
deliver to internally.  Obviously this works better for client
software; in the context of our system it seems to be mainly an
argument for less formal delivery processes.

At some point we started discussing agile development and remote
teams.  While disclaiming expertise in this area, Cockburn made a
number of recommendations.  The most important was the need for
"discipline" - a stricter adherence to the Agile principles.

I think this makes sense, but "discipline" is a tricky thing to
understand.  The following quote was helpful: "High discipline teams
work in the same way all the time, by personal choice."

So discipline is an internal thing each developer cultivates; it's not
enforced directly from outside.  The question then, is how to instil
this.  See communication issues.

Are we disciplined?  One measure of this is the common architecture.
Can we all write down the architecture?  Do we agree on it?  I think
we are getting close to this.

Personally, I also wonder if we should all be using a common
technology.

Anyway - are we agile?  From a list of 7 properties of successful
projects, 3 were highlighted as critical to Crystal (Cockburn's thing):
  -- Frequent delivery
  -- Osmotic Communication
  -- Reflective Improvement

Osmotic communication means that even if you don't know what was said,
you know who was talking and what the subject was.

It's interesting to consider how we score against these points.
Cockburn's own summary of the summary was: "Monitor the project by the
quality of communications and morale in the community."


Dynamic and Static Types

Not directly relevant to work, but very interesting, was Philip
Wadler's talk.  He started with logic, introduced types, illustrated
the Howard-Curry isomorphism, and went on to show how Haskell's type
classes are like Java's generics.

He then backtracked (or I think he would have - he kind of ran out of
time) and showed how Haskell could learn from Java(!).  The result was
an approach to static typing that is implemented by erasure - you end
up with a static typed layer on top of a dynamically typed
sub-language (i think it's object-functional; type classes are
dispatched on the first argument, which becomes an "object", as far as
i can tell).  And one cool thing that lets you do is make a statically
typed full-stack system: you write everything in a single, statically
typed language, which compiles to SQL, something for the server (I
missed what - perhaps Haskell?) and Javascript for the AJAX client.

Which is pretty damn cool.  It might fix, for example, the problem we
have in the portal (Ruby on Rails, although the problem is more
general than that one platform), where the Javascript code is fragile
to changes in the HTML pages (assuming pages and Javascript are
instead generated by a single coherent system).


Service Oriented Architecture

My second tutorial was on SOA.  It was given by someone who had
developed middleware at Siemens so that everything from embedded
devices to mainframes were running within a single system.  So he had
a take on SOA that was distinct from the usual web-services view.

Much like Agile, SOA becomes more diffuse as you try to pin it down.
Throughout the session I was asking myself: "what is SOA; how does it
compare to what we have; how do we understand the differences?".

Cockburn described Agile in terms of properties projects should have
and recommended methods for achieving them.  In the same way, SOA
seems to have the following targets:
  -- Strong decoupling
  -- Orchestration and composition at a high level

One way to think of decoupling is to make binding as late as possible.
That might mean using well-defined (and language independent)
interfaces, registries and federation.  This can push binding back not
just to runtime, but to individual interactions and, with asynchronous
messaging, to something even less rigid (the presenter made a strong
case that asynchronous messaging also helps performance).

Another way in which services are coupled is via shared state.  To the
point where "pure SOA" ("pure" both as "extreme" and also as "pure
functional language"), and related middleware, gives no way to manage
state.

Orchestration and composition at a high (system) level (as well as at
lower levels) are also simplified by asynchronous messaging and a lack
of state (there seems to be a strong connection between SOA,
workflows, and functional programming - I can't understand why this
isn't being picked up more by the FP community.  Perhaps because for
them it is just stating the obvious?).

Web services, with an emphasis on cross-platform availability rather
than efficiency, managing state, or reducing footprint, fit nicely
within this vision.  Other applications of SOA are going to need to
make, and justify compromises.

Looking at our own system, we clearly need to place more emphasis on
state.  We will need transactions, and a middleware that will help
handle them (as I believe Mule will).  Inter-operability, at least at
lower levels, is not an issue.  In general, then, our system seems to
make justifiable changes to the web service paradigm.

However, one area that does worry me is the strong reliance on
synchronous messaging.  Even if this is constructed on to pof
asynchronous transports, it still makes clients with state fragile.  I
wonder how much impact this will have on making our system
reconfigurable (moving orchestration into more explicit workflows).

Much of the above is from my own reflections (and chatting to Sonya,
who I think would avoid the word "compromise" and instead argue SOA is
a broader church), in response to the tutorial.  Returning to the
details of the tutorial, things were generally encouraging.

In particular, it was easy to map between our own system and the final
architecture used by the presenter.  At a high level, we have similar
layering.  At a lower level, our messaging client/server approach was
also present.


Negatives

The above are the main highlights.  I was a little disappointed /
surprised by a couple of things too.  There was a distinct lack of
theory for a conference with an "academic" reputation (although in a
welcome speech it was described as only within the "top 50" of cited
conferences).  And several times I heard people making very naive,
way-oversimplified statements about capitalism and free markets
(particularly in relation to large systems).


Positives

Maybe the most positive thing to come from this conference is the
degree to which computing theory is driving computing practice (even
if few people were talking about it).  Two examples struck me.

First, Wadler's talk showed just how much creeps in slowly - I never
really considered how close the relationship was between Haskell and
Java (Haskell was name-checked repeatedly through the conference).

Second, category theory was mentioned in two very unrelated areas,
driving both the support for parallelism in Fortress and the
management of constraints across different "views" of chip hardware
design.  I can see this kind of thing extending to, say, the
management of state in SOA.  This is hugely relevant to what we do.

And Portland Airport has free wifi.  Woot!

Comment on this post