andrew cooke: blog

Info on Anon P2P etc

Useful thread and a link to a related thesis.

http://news.ycombinator.com/item?id=3566305

Andrew

Permalink | Comment on this post

Previous Entries

For comments, see relevant pages (permalinks).

Hardware Transactional Memory coming to Intel

http://software.intel.com/en-us/blogs/2012/02/07/transactional-synchronization-in-haswell/

Andrew

Permalink

Rome - 3 Dreams of Black

I'm poking aroud webgl at the moment; this by far the best demo I've come
across (the only demo that's shown real artistic sensibility) -
http://www.ro.me/

Andrew

Permalink

New Technologies

These are a bunch of random notes I wrote for work when someone asked what was
"new".  I realise some of this is quite old - that's partly me not knowing
stuff, and partly because I wasn't sure what was required.  Anyway, feedback
is welcomed.

The original was a wiki page; I've had to reformat it here.


Cloud Computing

 Various ideas / acronyms related to "the cloud" are worth understanding:

  IAAS - Infrastructure As A Service - This is where we are heading with our
  VPS.  The idea is that infrastricture is something that you buy as a service
  (obviously). VPS is still tied to the idea of individual machines; IAAS is
  pushing that more towards have a network of machines that grows on demand.
  AWS http://aws.amazon.com/ are dominant, but there are also a bunch of other
  players like RackSpace http://www.rackspace.com/, etc.

  PAAS - Platform As A Service - GAE http://code.google.com/appengine/ (Google
  App Engine) is the prime example.  They have layered software over IAAS to
  make everything simpler to use.  You develop programs according to the
  platform's API and end up with a system that automatically scales with
  demand.

  SAAS - Software As A Service - Google docs https://docs.google.com
  etc. Software that comes from the cloud and stores data there, using only a
  "thin client".

 Writing a web service for GAE in Python is a very productive way to get a
 site running quickly that will handle loads well (you can use much of Django
 - the only new thing to learn is that way that data storage works - see "big
 data" elsewhere).

Big Data

 Related to the above, and driven by companies like Facebook, Twitter and
 Google, are various ideas related to "big data".  The most visible of these
 are a whole slew of "NoSQL" databases.  These are just starting to die off -
 many didn't do much that SQL can't do anyway, like Object and XML databases
 before - but various niches remain:

  Memcaced http://memcached.org/ and Redis http://redis.io/ are proving useful
  for caching data in-memory;

  HBase http://hbase.apache.org/ and Google's Big Table
  http://research.google.com/archive/bigtable.html work on problems so large
  that SQL really does fail (closely related are Hadoop
  http://hadoop.apache.org/ and Map reduce
  http://code.google.com/edu/parallel/mapreduce-tutorial.html, which allow
  data stored in those databases to be processed and collected) (Update:
  Amazon just released their database, Dynamo
  http://www.allthingsdistributed.com/2012/01/amazon-dynamodb.html);

  Neo4j http://neo4j.org/ is focussed on storing graphs.

 Associated with NoSQL http://en.wikipedia.org/wiki/NoSQL are various spin-off
 technologies - libraries for serialisation of data like Avro
 http://avro.apache.org/ and Thrift http://thrift.apache.org/ - and the rising
 importance of data visualisation.

 ZeroMQ http://www.zeromq.org/ feels like it fits here somehow, if only
 because it follows the trend of reinventing stuff that already worked - it's
 the most fashionable messaging library at the moment (simple and easy to use,
 but without the reliability guarantees of "enterprise" messaging - think of
 it more like TCP on steroids).

Web

 Web programming is changing rapidly. Web applications are becoming more and
 more dynamic. This is supported on the client by increasing use of Javascript
 (the latest library is Backbone.js http://documentcloud.github.com/backbone/
 which helps assemble web pages on the client from fragments of data).  This
 requires the server, increasingly, to "push" data (a typical HTTP page used
 to be "pulled" by the client; now the server sends data even when the client
 doesn't "know" that an update is needed).  Push used to imply Comet
 http://en.wikipedia.org/wiki/Comet_(programming) (a technology hack using
 stalled HTTP requests), but HTML 5 http://www.html5rocks.com/en/ (which is
 the future of HTML - XHTML is a largely unused dead-end) is introducing web
 sockets http://www.html5rocks.com/en/tutorials/websockets/basics/ that allow
 bi-directional transfer (like TCP sockets).

 Kind-of related to server push is the idea that servers should be
 asynchronous and event driven. Instead of having a thread for each
 connection, there's just one thread in the server, which reacts to incoming
 client requests on demand.  For this to work the thread cannot "block" (eg
 while waiting for data to be read from the database) so everything is handled
 via callbacks (eg so a database read is handled by a library function (which
 is a separate thread or process); once the data have been read from the
 database a callback is used to transfer the data to the main thread to
 continue processing).  That is very like the way events are handled in
 Javascript on the client, which is where Node.js http://nodejs.org/ and
 server-side Javascript come in (if you know how traditional GUIs work, with a
 single render thread, this is very similar).  Related: JSON
 http://people.planetpostgresql.org/andrew/index.php?/archives/244-Under-the-wire.html
 as a database type http://en.wikipedia.org/wiki/Document-oriented_database;
 Javscript http://code.google.com/p/plv8js/wiki/PLV8 in the database.

 A common theme here is communication; another aspect of that is SPDY
 http://dev.chromium.org/spdy, which is intended to replace HTTP, offering
 more efficient mutliplexing of sessions.  See this article
 http://www.igvita.com/2012/01/18/building-a-modern-web-stack-for-the-realtime-web/
 for an overview of how this affects the technology stack.

 Back on the client, new technologies include WebGL
 http://www.chromeexperiments.com/webgl (this lets you do graphics on the GPU,
 so you can have 3D effects etc), Canvas
 http://www.html5rocks.com/en/tutorials/canvas/integrating/ and SVG
 http://www.html5rocks.com/en/tutorials/raphael/intro/ (for displaying data),
 liquid layouts
 http://designfestival.com/resizing-fixed-fluid-or-responsive-layouts/ (very
 fluid, handling everything from phones to tables to desktop brownsers in a
 single page) layouts, web storage http://en.wikipedia.org/wiki/Web_Storage
 (on the client), much better support for fonts
 http://designmodo.com/custom-web-fonts-fontface, and CSS3 (or processors like
 Sass http://sass-lang.com/) (Update: possibly relevant book
 http://shop.oreilly.com/product/0636920018421.do).

 A big thread running through much of current web work is the idea of social
 cues and interaction - sharing, feedback, progressing, belonging, etc.

Small Computers

 Many of the web developments are being driven by mobile phones. And phone
 hardware technology is now starting to spread - we're starting to see the
 "internet of things" (where famously, even your fridge and toaster have
 internet connections).  One example of this is the Nest Labs thermostat
 http://www.nest.com/; it is "just" a house heating thermostat, but "done like
 Apple".  Another example is Raspberry PI http://www.raspberrypi.org/ - a
 computer capable of running Linux for just $25. Small computers are going to
 be everywhere, very soon.

 Connected to this, there's the idea of a general "dumbing down" of consumer
 computing.  You can see this in the iPad, for example (one take is that it's
 intended to help people consume, not create).  The general idea is of
 "computing appliances" (with restricted choice of software from "app stores")
 replacing general computing.  Another driver for this is the crackdown on
 "pirating".

Programming Languages

 Java is looking increasingly outdated, but there's still no obvious
 replacement. The next big language is going to have to combine two sets of
 ideas: functional programming and better types.  Functional programming, when
 it works, tends to lead to simpler, more compact code, that is easier to make
 parallel across multiple cores.  Better types means that people want
 programming in Java to be as quick and easy as programming in Python, while
 keeping Java's speed and ability to detect errors at compile time.

 For a long time it looked like Scala http://www.scala-lang.org/ would be the
 future, but increasingly the consensus seems to be that it's just too damn
 complicated.  Clojure http://clojure.org/ is a more recent contender, but it
 probably doesn't have enough ideas from the "better types" to win out. More
 obscure (and less ambitious) possibilities include Kotlin
 http://blog.jetbrains.com/kotlin/, Xtend http://www.eclipse.org/Xtext/xtend/
 and Rust http://www.rust-lang.org/.

 For system level programming Go http://golang.org/ is actually pretty good. I
 don't think it's going to be huge, but it really is a better C (and way, way
 nicer than C++).  It's curiously old-fashioned compared to some of the
 languages above, but perhaps that's an advantage.

 Meantime, in the background, Javascript is becoming more and more important,
 even though people don't really like to use it (the Good Parts
 http://shop.oreilly.com/product/9780596517748.do book is excellent BTW).  It
 is becoming more important on the client (see elsewhere) and is also starting
 to play a role on the server when writing asynchronous servers (again, see
 elsewhere).  This has triggered various attempts to make it more friendly:
 Coffeescript http://coffeescript.org/ is a pre-processor that makes it look
 more like Python; Clojurescript https://github.com/clojure/clojurescript
 allows programs written in Clojure to be "compiled" to Javascript; Closure
 http://code.google.com/closure/ (completely unrelated to Clojure) is a
 compiler from Javascript to Javascript (!).

 Increasingly, languages are supporting distributed and parallel computing.
 Erlang http://www.erlang.org/ was an early success in this space. Clojure, as
 mentioned above, is also pushing new ideas (actors and software transactional
 memory).  It's also possible that we need a completely new paradigm -
 non-deterministic computing
 http://my-inner-voice.blogspot.com/2012/01/many-core-processors-everything-you.html.

3D Printing

 Is as mindblowingly awesome as jet packs, IMHO.


Andrew

Permalink

Shoddy Macs

How can any self-respecting engineer buy a computer where they *glue* the
cable to the disk?  These aren't machines made for people that take things
apart - they're made for *consumers* that never expect to get their hands
dirty.

Mac owning engineers: you're selling your souls for a shiny trinket.  These
aren't machines made, and made well, to be cared for.  They are intended as
disposable trophies.  Polished turds.

You should be ashamed of yourselves.  Really.

(Pauli's Macbook Pro is dead and I have it in pieces).

Andrew

Permalink

Profiling Go Programs

This is a really nice article that shows the practical details of profiling
(CPU and memory) Go programs.  In addition, the way that the profiler output
is displayed (via graphvix, with node sizes scaled to reflect the measured
values) is very neat.

http://blog.golang.org/2011/06/profiling-go-programs.html

Andrew

Permalink

Access to Bloomberg Data Feed

http://www.computerworlduk.com/news/it-business/3334375/bloomberg-market-data-api-made-public/
http://open.bloomberg.com/

Not clear to me what limitations exists (volume limits etc).

Andrew

Permalink

Automated JVM Leak Detection

The system I am currently working on has a leak on redeploying to Tomcat
(probably similar to what is described here
http://blogs.oracle.com/fkieviet/entry/classloader_leaks_the_dreaded_java)

This looks like it would automate the detection and solution:

 http://www.plumbr.eu/
 http://www.plumbr.eu/blog/screencast-how-to-use-plumbr

Andrew

Permalink

Fashion (Men's)

http://www.wwd.com/runway/mens-fall-collections-2012/review/thom-browne/slideshow#/slideshow/article/5547182/5547183

Andrew

Permalink

Left Drifts Centre; Right Becomes Vicious

When you have people like Blair and Obama in power it's easy to think that the
right can't be worse.  So it's worth noting that it can.

http://www.guardian.co.uk/commentisfree/2012/jan/22/housing-benefit-cap-62p-a-day

Andrew

Permalink

Jolly Good Idea Chaps, Wot?

http://www.guardian.co.uk/uk/2012/jan/15/queen-royal-yacht-diamond-jubilee-gove

Words fail me.  Original comment here was more or less a string of
profanities.  If this happens I will consider changing nationality.

Andrew

Permalink

I Can Vote!

I just registered to vote here in Santiago, Chile.  That's pretty awesome - I
don't think a Chilean in my position could do the same in the UK.  If anyone
else is interested in doing the same, it's pretty easy - you need to follow
the instructions here
http://www.extranjeria.gov.cl/filesapp/solicitud_certificado_avecindamiento.pdf
which basically means:

  - Go to the international police (who have moved recently - they are now in
    Morande) and request a certificate saying that you have been resident in
    Chile for the last 5 years (excluding holidays - I think you can leave for
    up to 6 months, but don't quote me on that).  This costs some small amount
    (all the rest below is free).

  - Send that certificate, and copies of your residency card and carnet, and
    the form at the link, to the ministry of the interior.

  - Wait 2-3 months (yes, it says 10 days on the form....) and you'll receive
    another certificate.

  - Take that certificate and your carnet to the local vote registration
    office (for Providencia it's in Manuel Montt).

It may also be worth explaining why I registered.  I live in a nice district
of Santiago - it's fairly central and contains many of the older embassies.
The mayor belongs to the ultra-right UDI party (this follows logically from
the social class of the area - that's how Santiago is); during the
dictatorship he was a member of the secret police.

Last November Labbe, the mayor, helped organise, in the Providencia sports
club, a "homage" to the life of Miguel Krassnoff - a brigadier in the secret
police, who was tried and jailed for abuses of human rights.

Krassnoff tortured the parents of a friend of ours.

Fuck that.

Andrew

Permalink

Info on Anon P2P etc

Previous Entries

Hardware Transactional Memory coming to Intel

Rome - 3 Dreams of Black

New Technologies

Shoddy Macs

Profiling Go Programs

Access to Bloomberg Data Feed

Automated JVM Leak Detection

Fashion (Men's)

Left Drifts Centre; Right Becomes Vicious

Jolly Good Idea Chaps, Wot?

I Can Vote!

C[omp]ute

Recent Threads

Recent Replies