| Andrew Cooke | Contents | Latest | RSS | Twitter | Previous | Next

C[omp]ute

Welcome to my blog, which was once a mailing list of the same name and is still generated by mail. Please reply via the "comment" links.

Always interested in offers/projects/new ideas. Eclectic experience in fields like: numerical computing; Python web; Java enterprise; functional languages; GPGPU; SQL databases; etc. Based in Santiago, Chile; telecommute worldwide. CV; email.

Personal Projects

Lepl parser for Python.

Colorless Green.

Photography around Santiago.

SVG experiment.

Professional Portfolio

Calibration of seismometers.

Data access via web services.

Cache rewrite.

Extending OpenSSH.

Last 100 entries

Real-life Financial Co Without ACID Database...; Flexible Muscle-Based Locomotion for Bipedal Creatures; SQL Performance Explained; The Little Manual of API Design; Multiple Word Sizes; CRC - Next Steps; FizzBuzz; Update on CRCs; Decent Links / Discussion Community; Automated Reasoning About LLVM Optimizations and Undefined Behavior; A Painless Guide To CRC Error Detection Algorithms; Tests in Julia; Dave Eggers: what's so funny about peace, love and Starship?; Cello - High Level C Programming; autoreconf needs tar; Will Self Goes To Heathrow; Top 5 BioInformatics Papers; Vasovagal Response; Good Food in Vina; Chilean Drug Criminals Use Subsitution Cipher; Adrenaline; Stiglitz on the Impact of Technology; Why Not; How I Am 5; Lenovo X240 OpenSuse 13.1; NSA and GCHQ - Psychological Trolls; Finite Fields in Julia (Defining Your Own Number Type); Julian Assange; Starting Qemu on OpenSuse; Noisy GAs/TMs; Venezuela; Reinstalling GRUB with EFI; Instructions For Disabling KDE Indexing; Evolving Speakers; Changing Salt Size in Simple Crypt 3.0.0; Logarithmic Map (Moved); More Info; Words Found in Voynich Manuscript; An Inventory Of 3D Space-Filling Curves; Foxes Using Magnetic Fields To Hunt; 5 Rounds RC5 No Rotation; JP Morgan and Madoff; Ori - Secure, Distributed File System; Physical Unclonable Functions (PUFs); Prejudice on Reddit; Recursion OK; Optimizing Julia Code; Cash Handouts in Brazil; Couple Nice Music Videos; It Also Works!; Adaptive Plaintext; It Works!; RC5 Without Rotation (2); 8 Years...; Attack Against Encrypted Linux Disks; Pushing Back On NSA At IETF; Summary of Experimental Ethics; Very Good Talk On Security, Snowden; Locusts are Grasshoppers!; Vagrant (OpenSuse and IDEs); Interesting Take On Mandela's Context; Haskell Cabal O(n^2) / O(n) Fix; How I Am 4; Chilean Charity Supporting Women; Doing SSH right; Festival of Urban Intervention; Neat Idea - Wormholes Provide Entanglement; And a Link....; Simple Encryption for Python 2.7; OpenSuse 13.1 Is Better!; Little Gain...; More Details on Technofull Damage; Palmrest Cracked Too....; Tecnofull (Lenovo Support) Is Fucking Useless; The Neuroscientist Who Discovered He Was a Psychopath; Interpolating Polynomials; Bottlehead Crack as Pre-amp; Ooops K702!; Bottlehead Crack, AKG K701; Breaking RC5 Without Rotation; Great post thank you; Big Balls of Mud; Phabricator - Tools for working together; Amazing Julia RC5 Code Parameterized By Word Size; Chi-Square Can Be Two-Sided; Why Do Brits Accept Surveillance?; Statistics Done Wrong; Mesas Trape from Bravo; European Report on Crypto Primitives and Protocols; Interesting Omissions; Oryx And Crake (Margaret Atwood); Music and Theory; My Arduino Programs; Elliptic Curve Crypto; Re: Licensing Interpreted Code; Licensing Interpreted Code; ASUS 1015E-DS03 OpenSuse 12.3 SSD; translating lettuce feature files into stub steps files; Re: translating lettuce feature files into stub steps files; A Tale of Two Psychiatrists; The Real Reason the Poor Go Without Bank Accounts

© 2006-2013 Andrew Cooke (site) / post authors (content).

Mule + Java Interfaces

From: "andrew cooke" <andrew@...>

Date: Fri, 13 Jan 2006 15:53:20 -0300 (CLST)

These are some notes I wrote at work to clarify how messaging with Mule
could (should?) work in our system.  One of the side issues here is that
it's unclear what J2EE buys us.

[...]

Consider two services implemented as:

class edu.noao.nsa.msg.SomeMessage {
  // serializable data transfer object with getters/setters
}

interface edu.noao.nsa.serviceA.ServiceA {
  public SomeResponse doSomething(SomeMessage message);
}

class edu.noao.nsa.serviceA.impl.ServiceAImpl implements ServiceA {
  // implements the functionality as a simple bean
}

@Local interface edu.noao.nsa.serviceA.ejb.ServiceALocal
extends ServiceA {}

@Remote interface edu.noao.nsa.serviceA.ejb.ServiceARemote
extends ServiceA {}

@Stateless class edu.noao.nsa.serviceA.ejb.ServiceA
extends ServiceAImpl {
  // do injection from @Resource here
}

class edu.noao.nsa.serviceB.impl.ServiceB {
  private ServiceA serviceA; // plus getters/setters
  public Object someProcess(...) {
    ...
    response = serviceA.doSomething(message);
    ...
  }
}

note the package structure:

edu
 +- noao
     +- nsa
         +- msg
         |   +- serializable message beans
         +- serviceA
         |   +- ServiceA as interface
         |   +- impl
         |   |   +- ServiceA implemented as bean
         |   +- ejb
         |   |   +- wrappers for EJB3
         |   +- test etc
         +- serviceB
             +- ServiceB as interface
             +- impl
             |   +- ServiceB implemented as bean
             +- ejb
                 +- wrappers for EJB3

Now for testing we can write Java unit tests on the impl classes.  We can
even connect together services (using Spring) to test more than one
service at a time.

Note - testing service together may seem more like integration testing,
but in the VOI we follow this same pattern down inside the VOI.  So the
VOI itself is composed of sub-services (components) that plug together as
required.  These then test as described above, but can still be deployed
separately.

The EJB wrappers do two things:
- provide Local/Remote interfaces
- do the necessary injection via @Resource


So as monolithic Java code, that just works.  Either in J2EE or in some
Spring based container.  What extra work needs to be done to deploy the
services on separate machines?

First we need to implement a proxy that intercepts the message from
serviceB.  We already have a set of MessageSender implementations which
send a message to Mule.  So the code is:

class edu.noao.nsa.serviceA.mule.ServiceAMsgProxy implements ServiceA {
  private MessageSender messageSender;
  public void setMessageSender...;
  public MessageSender getMessageSender...;
  public SomeResponse doSomething(SomeMessage message) {
    return (SomeResponse)getSender().sendSynchronous(message);
  }
}


That's it.


Seriously.  No more Java code.  That will send the message object to Mule,
which can be configured to send the message anywhere, via any messaging
system (JMS / convert it to XML / send it as an email / SOAP /
in-memory/VM etc).

This proxy is injected into ServiceB.  ServiceB sees the same interface as
before and talks to it as before.

At the ServiceA end, no extra code is needed at all.  Instead you either:
- deploy ServiceA within Mule, in which case routing is trivial
  (practically automatic)
- deploy ServiceA in EJB, in which case you configure Mule to call the EJB
  (we extend Mule to call EJBs - this was a strange ommission in Mule, but
extension - adding a new "provider" - is quite simple).

So the advantages are:
- Write and test pure Java code
- Type checking of inter-service contracts via interfaces
- No messaging code appears in the "business logic"
- All routing decisions are isolated in Mule
- Only a single, simple proxy at the sender is required
- No interface at all is required at the receiver
- The final deployment structure is not fixed in the design
- Progressively more complex messaging technologies can be configured -
Mule provides synchronous transport over asynchronous messaging

One disadvantage is organising the build so that the cross-dependencies
work OK.  This reflects the cross-dependencies in the contracts that each
service relies on - it's not an artifact of bad engineering, but a
reflection that services do depend on each other via their interfaces, and
that this should be verified as early as possible.

I suspect the best build solution is to use two passes - once to generate
the interfaces and a second time to compile code that depends on all the
interfaces available.

Andrew

[Fwd: (Strictly limited) circular dependencies with M2]

From: "andrew cooke" <andrew@...>

Date: Sat, 14 Jan 2006 11:25:32 -0300 (CLST)

---------------------------- Original Message ----------------------------
Subject: (Strictly limited) circular dependencies with M2
From:    "andrew cooke" <andrew@...>
Date:    Sat, January 14, 2006 11:14
To:      users@...
--------------------------------------------------------------------------

Hi,

I have been asked to configure Maven 2 for our new project at work.  The
project architecture is a set of services, each of which will be a
separate sub-project in Maven 2.

However, each service depends on the *external interfaces* of the services
it connects to.

I know that generally circular dependencies are a "bad thing", so I want
to explain carefully why the dependency we have is not:

 Each service has a contract that it implements.  A calling (dependent)
 service depends on this contract.  If the contract is broken, the system
 will fail.

 Since the contract is critical, it makes sense to test it as early as
 possible.  Preferably at compile time.  We can do this (to a limited
 extent) by expressing the contract as an interface.  So any code that
 uses service X compiles againts the interface that X provides.

 We can then insert proxies and messaging so that the services can exist
 on different machines, etc.  More details at
 http://www.acooke.org/cute/MuleJavaIn0.html

Of course, we do not want "deep" dependencies of one service on another,
so the external interfaces are restricted to:

 - Java interfaces that the service implements
 - Data transfer objects (serializable, simple beans)

How should this be structured?  Since the data transfer objects are shared
between services and have some additional constraints from the messaging
technology (serializable; it would also be useful to have a carefully
designed inheritance hierarchy for simplifying routing decisions based on
class) I think they should go into a separate "messaging" project.  The
interfaces that each service provides, however, belong in the appropriate
service.

So we have the package structure:

 project
  +- messaging
  |   +- data transfer objects
  +- service 1
  |   +- external interface
  |   +- implementation
  +- service 2
  .   +- external interface
  .   +- implementation
  +...

To build this I need to:

 - compile messaging (data transfer objects)
 - compile external interfaces for each service
 - package external interfaces and message objects
   (necessary for the messaging system)
 - compile implementations for each service
 - package each service (both interface and implementation)

(I may also need to package services as ejb3 beans)

If I do this, all dependencies are resolved correctly.

My question, then, is how do I do this with Maven 2?  Sorry I had to write
so much to ask it, and further apopolgies if this is really obvious.  I've
looked around the documentation, but can't see how this level of control
is provided.

Thanks,
Andrew

[Fwd: RE: (Strictly limited) circular dependencies with M2]

From: "andrew cooke" <andrew@...>

Date: Sat, 14 Jan 2006 13:40:37 -0300 (CLST)

---------------------------- Original Message ----------------------------
Subject: RE: (Strictly limited) circular dependencies with M2
From:    "Allison, Bob" <robert.allison@...>
Date:    Sat, January 14, 2006 13:25
--------------------------------------------------------------------------

The normal way of handling this kind of dependency tree is to put the
common stuff (each service's interface and DTOs) is a separate project.
Then each service's project depends on this common stuff project.

Make sense?

[Fwd: RE: (Strictly limited) circular dependencies with M2]

From: "andrew cooke" <andrew@...>

Date: Sat, 14 Jan 2006 13:42:02 -0300 (CLST)

---------------------------- Original Message ----------------------------
Subject: RE: (Strictly limited) circular dependencies with M2
From:    "andrew cooke" <andrew@...>
Date:    Sat, January 14, 2006 13:32
--------------------------------------------------------------------------


It makes sense, but it seems a bit ugly.  I guess there's nothing to stop
the two projects sharing the same Java package namespace.  Hmmm.

OK, I guess I'll go with that.  It seems like Maven is being a bit
intrusive, though.  On the other hand, I guess it will drive home to
everyone the distinction between the two.

Thanks,
Andrew

[Fwd: New, clean, elegeant, succint, correct, consistent, tidy, compact, ordered, directed, cycle-free and generally wonderful maven structure]

From: "andrew cooke (noao)" <acooke@...>

Date: Thu, 23 Feb 2006 19:54:43 -0300 (CLST)

I don't think I'm breaking any rules by forwarding this from work...

---------------------------- Original Message ----------------------------
Subject: New, clean, elegeant, succint, correct, consistent, tidy,
compact,      ordered, directed, cycle-free and generally wonderful maven
structure
From:    "andrew cooke (noao)" <acooke@...>
--------------------------------------------------------------------------

This is how I've left CVS.  PLease shout if it doesn't make sense.

You can get the new tree by doing "cvs checkout nsa-maven".  DON'T try
doing an update into your old tree ;o)


A note on package contents
--------------------------

messaging-iface contains (should contain?  Evan?) message beans (which are
built from simple serializable objects) and interfaces to the services
that messaging implements (messaging itself may be empty at the moment,
but remains as a placeholder in case it is needed in the future).

Other service iface classes define the interfaces that the service
provides, in terms of the message objects.

So a very simple interface looks like:

package edu.noao.nsa.<name>
import edu.noao.nsa.messaging.MyBean

public Interface MyService {
  // argument is a message bean from messaging-iface
  public void doSomething(MyBean msg);
}


Dependencies
------------

This is the dependency tree - code can only refer to code for which there
is a path going *up*.

So, for example, nothing can refer to code in the messaging package.
However, everything except toolkit and mulex can refer to the
messaging-iface package, which contains the message POJOs and the
interfaces that the messaging infrastructure provides.

Another example - the generic <name> package (ie any typical future
package) only has access to <name>-iface (which it implements) and
messaging-iface.  So it can do its own work, create message objects and
send messages (but nothing else).

There is one complication.  The old messaging code was not written with
this structure in mind.  For now, every service depends on messaging-old.
However, this will be phased out once we have scavenged that code for
anything that might be useful and people have eliminated dependencies.

Oh, and "toolkit" is a placeholder for common (service-independent)
utility code, currently empty.

toolkit
|
 \- messaging-iface------\    messaging-old
    |                     |   :
    |\-security-iface----\|   :
    |  |                  |   :
    |   \-security........|...:
    |                     |   :
    |\-dataservice-iface-\|   :
    |  |                  |   :
    |   \-dataservice.....|...:
    |                     |   :
    |\-logging-iface-----\|   :
    |  |                  |   :
    |   \-logging.........|...:
    |                     |   :
    |\-voi-iface---------\|   :
    |  |                  |   :
    |   \-voi.............|...:
    |                     |   :
     \-<name>-iface------\|   :
       |                  |   :
        \-<name>..........|...:   mulex
                          |   :   |
                          \-messaging

[Fwd: Suggested naming standard/structure for a service]

From: "andrew cooke" <andrew@...>

Date: Thu, 23 Feb 2006 19:57:17 -0300 (CLST)

In fact, I think the J2EE remote interface should also be in -iface.

---------------------------- Original Message ----------------------------
Subject: Suggested naming standard/structure for a service
From:    "andrew cooke (noao)" <acooke@...>
-------------------------------------------------------------------------

[not sure anyone but alvaro cares about this, but fyi]

trying to get the security/npa code into a format consistent with
everything we've discussed so far, this is what i think the final "new
proposal alert" code will look like:

in nsa-maven/maven/newproposalalert-iface:
edu
+-noao
  +-nsa
    +-sec
      +-npa
        +-iface
          +-NewProposalAlertIface (interface provided to messaging,
              taking and returning message beans as arguments)

in nsa-maven/maven/newproposalalert:
edu
+-noao
  +-nsa
    +-sec
      +-npa
        +-NewProposalAlert (local interface, expressed in local classes
        |   relevant to internal logic)
        +-iface
        | +-ejb (optional)
        | | +-NewProposalAlertBean (ejb imp. of NewProposalAlertIface,
        | |     has NewProposalAlert implementation hard-wired, perhaps
        | |     by extending NewProposalAlertImpl)
        | +-spring
        |   +-NewProposalAlertBean (spring imp. of NewProposalAlertIface,
        |       receives a NewProposalAlert implementation - an instance
        |       of NewProposalAlterImpl - via spring injection; unpacks
        |       message and calls NewProposalAlert; packages result)
        +-impl
          +-NewProposalAlterImpl (implementation of NewProposalAlert)

in nsa-maven/maven/messaging-iface:
the message beans that appear in NewProposalAlterIface

Comment on this post