| Andrew Cooke | Contents | Latest | RSS | Twitter | Previous | Next

C[omp]ute

Welcome to my blog, which was once a mailing list of the same name and is still generated by mail. Please reply via the "comment" links.

Always interested in offers/projects/new ideas. Eclectic experience in fields like: numerical computing; Python web; Java enterprise; functional languages; GPGPU; SQL databases; etc. Based in Santiago, Chile; telecommute worldwide. CV; email.

Personal Projects

Lepl parser for Python.

Colorless Green.

Photography around Santiago.

SVG experiment.

Professional Portfolio

Calibration of seismometers.

Data access via web services.

Cache rewrite.

Extending OpenSSH.

Last 100 entries

NSA Interceptng Gmail During Delivery; General IIR Filters; What's happening with Scala?; Interesting (But Largely Illegible) Typeface; Retiring Essentialism; Poorest in UK, Poorest in N Europe; I Want To Be A Redneck!; Reverse Racism; The Lost Art Of Nomography; IBM Data Center (Photo); Interesting Account Of Gamma Hack; The Most Interesting Audiophile In The World; How did the first world war actually end?; Ky - Restaurant Santiago; The Black Dork Lives!; The UN Requires Unaninmous Decisions; LPIR - Steganography in Practice; How I Am 6; Clear Explanation of Verizon / Level 3 / Netflix; Teenage Girls; Formalising NSA Attacks; Switching Brakes (Tektro Hydraulic); Naim NAP 100 (Power Amp); AKG 550 First Impressions; Facebook manipulates emotions (no really); Map Reduce "No Longer Used" At Google; Removing RAID metadata; New Bike (Good Bike Shop, Santiago Chile); Removing APE Tags in Linux; Compiling Python 3.0 With GCC 4.8; Maven is Amazing; Generating Docs from a GitHub Wiki; Modular Shelves; Bash Best Practices; Good Emergency Gasfiter (Santiago, Chile); Readings in Recent Architecture; Roger Casement; Integrated Information Theory (Or Not); Possibly undefined macro AC_ENABLE_SHARED; Update on Charges; Sunburst Visualisation; Spectral Embeddings (Distances -> Coordinates); Introduction to Causality; Filtering To Help Colour-Blindness; ASUS 1015E-DS02 Too; Ready Player One; Writing Clear, Fast Julia Code; List of LatAm Novels; Running (for women); Building a Jenkins Plugin and a Jar (for Command Line use); Headphone Test Recordings; Causal Consistency; The Quest for Randomness; Chat Wars; Real-life Financial Co Without ACID Database...; Flexible Muscle-Based Locomotion for Bipedal Creatures; SQL Performance Explained; The Little Manual of API Design; Multiple Word Sizes; CRC - Next Steps; FizzBuzz; Update on CRCs; Decent Links / Discussion Community; Automated Reasoning About LLVM Optimizations and Undefined Behavior; A Painless Guide To CRC Error Detection Algorithms; Tests in Julia; Dave Eggers: what's so funny about peace, love and Starship?; Cello - High Level C Programming; autoreconf needs tar; Will Self Goes To Heathrow; Top 5 BioInformatics Papers; Vasovagal Response; Good Food in Vina; Chilean Drug Criminals Use Subsitution Cipher; Adrenaline; Stiglitz on the Impact of Technology; Why Not; How I Am 5; Lenovo X240 OpenSuse 13.1; NSA and GCHQ - Psychological Trolls; Finite Fields in Julia (Defining Your Own Number Type); Julian Assange; Starting Qemu on OpenSuse; Noisy GAs/TMs; Venezuela; Reinstalling GRUB with EFI; Instructions For Disabling KDE Indexing; Evolving Speakers; Changing Salt Size in Simple Crypt 3.0.0; Logarithmic Map (Moved); More Info; Words Found in Voynich Manuscript; An Inventory Of 3D Space-Filling Curves; Foxes Using Magnetic Fields To Hunt; 5 Rounds RC5 No Rotation; JP Morgan and Madoff; Ori - Secure, Distributed File System; Physical Unclonable Functions (PUFs); Prejudice on Reddit; Recursion OK; Optimizing Julia Code

© 2006-2013 Andrew Cooke (site) / post authors (content).

System Re-factoring

From: "andrew cooke" <andrew@...>

Date: Sat, 18 Jul 2009 14:09:37 -0400 (CLT)

I'm planning to re-arrange my system, so that I can turn off my main
computer on an evening, but still have my own local network, stream music,
etc.

This is made a little more complex by living in Chile.  The only low-power
computer I can find is this - http://www.sym.cl/ficha/8001/ - which
appears to support Linux (good) but, in that form factor, cannot take PCI
cards, has space for only one disk (no RAID), and one network port.

I have a spare disk and memory, and found somewhere that sells a USB to
network adapter that is compatible with Linux -
http://www.tecno.cl/prod/ficha.asp?Prod=3876 - so the total price is a bit
over 100 pounds / 200 dollars (I realise that in the US I could get
significantly more for my money, but that's life...).

I think the VIA chip will be able to manage my firewall and stream music,
but I am less sure it's capable of running an IMAP server (particularly if
I want to search email).  So I need to change my email solution.  I think
that means getting with the times and moving to gmail (currently I run my
own imap server that feeds web-mail (squiirelmail)).

I have been subscribed for some email lists for many (over 10) years, and
always had the idea that I was creating my own archive, but web search
with Google is so much better than anything I can come up (and Beagle
still doesn't handle maildirs correctly).  So I am going to unsubscribe
from everything and simply have personal email to gmail.  As a backup I
will use pop to grab mail once a day (say) to my "work" computer, where it
can be stored on RAID.  And I am going to stop filtering with procmail by
subject/list and instead go with date (directories for year and month). 
I've just checked and it seems I can reprocess an existing file in maildir
simply by passing it (ie the filename) procmail at the command line; so I
can rewrite all my old email too, and archive it up (just in case).

On the hardware front, the plan is to buy the USB network hub first, see
if it really does work OK, and only then get the CPU.  Then configure that
and connect everything up perhaps when Paulina is away in Brazil.  It's
been difficult to find much info on the VIA chip and Linux (I get the
impression it was superseded by Atom, and that these things have only
become popular as netbooks, but I have to work with what I can find).  And
if I can turn my main computer off for half the day, this should pay for
itself in less than a year in saved electricity!

Thought I'd write all this down to see if it makes sense.  It seems to.  I
will post more updates here as I work with the hardware.

Andrew

PS This also means no more updates to LEPL for a while.  In fact after/in
parallel with this I want to generate some "art" using LEPL, rather than
extend it, so for the next few months don't expect to write any new
functionality.  Release 3.0 came out yesterday, and so far has had just
over 100 downloads with no complaints, so I hope it is OK...

Mail-based Blog

From: "andrew cooke" <andrew@...>

Date: Sat, 18 Jul 2009 14:26:02 -0400 (CLT)

I knew writing here would turn up some issues...

How will this site work?  I quite like the way it works currently -
basically that I send emails to myself, that can be read in public, and
which works as I kind of shared notebook for ideas and solutions I stumble
across.  I don't think I have any regular readers, but I do get a lot of
targeted google hits on particular issues.

So I don't want to change the interface, but that means handling
"arbitrary" email address to my account.  Which isn't going to fly with
gmail.  On the other hand, I don't need this to be 24/7 (the posting part
- it is hosted statically at my ISP).  So I guess I need to split email
directed to my ISP.  Anything destined for this blog stays there, to be
handled in a similar way to now (pop download, procmail to scripts that
generate web files and finally upload).  Other email will be forwarded to
gmail.

In turn, that means I want to read email locally (not often, but when
deleting mistakes, handling other people's posts).  I guess I can use mutt
directly on the maildir for that, but I will also need to configure local
sending - shouldn't be a problem... (if I always end email from gnail I
can drop dyndns mailhop, which is what I have used until now - excellent,
reliable service, BTW, no complaints at all, just not needed in my
(hopefully) simpler future).

And speaking of acooke.dyndns.org, the new machine will have to handle my
wiki too, so that I can access it while abroad (without having to leave my
main machine running).

Andrew

Moved to GMail

From: andrew cooke <andrew@...>

Date: Tue, 21 Jul 2009 20:40:16 -0400

OK, so move to gmail seems to have worked.  Pop from gmail works just
fine, to a dated archive via the following procmail:

MAILDIR=$HOME/mail
DEFAULT=$MAILDIR/
LOGFILE=$HOME/log/procmail.log
LOGABSTRACT=all
#VERBOSE=on
PMSRC=/usr/local/share/procmail-lib

INCLUDERC=$PMSRC/pm-jadate.rc

:0
.$YYYY.$MM/

which will put this post, for example, in the 2009.07 folder for backup.

At the same time, this email, addressed to acooke.org, but not to me
personally, will stay at my ISP, popped from there, and then filtered
with my usual procmail recipe for the blog (I needed to add a test to
block the copy that appears as "sent email" from google or I get
duplicate posts).

And sending from my machine (using mutt) simply invokes procmail
directly, so I can post to the blog directly from mutt, rather than
using gmail, if I want (useful for re-posting replies).

To do that, muttrc contains:

set sendmail="/usr/bin/procmail /home/andrew/.procmailrc"

Note that specifying the procmailrx is critical - for some reason when
mutt invokes procmail, that is not picked up by default.

Now to hit send and hope it works....

Andrew

PS gmail is pretty neat - really like the labels approach.

DLink DUB-E100, Opensuse

From: andrew cooke <andrew@...>

Date: Fri, 24 Jul 2009 19:06:19 -0400

So, as part of moving to a lower power server, I bought a USB to
ethernet adapter.  There are two available locally.

One, for $10 USD, seems to be fairly new, and is a small dongle
(transparent plastic) with one main chip (and a smaller eeprom).  This
is sold by various "no-name" brands, one of which had a visible chip
ID: "Fullwin FY 618SM".

Unfortunately, I could find no info about this, and Linux support, so
instead I bought the second option - the much more expensive ($40 USD)
Dlink DUB-E100.

The good news is that it appears to work.  I plugged it in and the
following appeared in the logs (with no other preparation on my part):

Jul 24 18:57:52 quiet kernel: usb 4-1: new high speed USB device using
ehci_hcd and address 3
Jul 24 18:57:52 quiet kernel: usb 4-1: configuration #1 chosen from 1
choice
Jul 24 18:57:52 quiet kernel: usb 4-1: New USB device found,
idVendor=2001, idProduct=3c05
Jul 24 18:57:52 quiet kernel: usb 4-1: New USB device strings: Mfr=1,
Product=2, SerialNumber=3
Jul 24 18:57:52 quiet kernel: usb 4-1: Product: DUB-E100
Jul 24 18:57:52 quiet kernel: usb 4-1: Manufacturer: D-Link Corporation
Jul 24 18:57:52 quiet kernel: usb 4-1: SerialNumber: 000001
Jul 24 18:57:53 quiet kernel: eth1: register 'asix' at
usb-0000:00:1a.7-1, ASIX AX88772 USB 2.0 Ethernet, 00:80:c8:3a:ca:fe
Jul 24 18:57:53 quiet kernel: usbcore: registered new interface driver asix
Jul 24 18:57:53 quiet ifup:     eth1
Jul 24 18:57:53 quiet ifup:               No configuration found for eth1

Which is pretty sweet.  So next thing to do is buy the rest of the
hardware.  I have found a new option - an Intel Atom powered MSI box,
but it's larger and more expensive than the VIA unit I mentioned
earlier, and I am trying to avoid Intel because of their investments
in Israel.  So I'm pretty sure I'll go with the Via solution.

Andrew

Initial Impressions - Via C7-D Barebones with Opensuse

From: andrew cooke <andrew@...>

Date: Sun, 26 Jul 2009 14:39:58 -0400

According to the box, yesterday I bought a VIA PC2500E Motherboard
with a PEECee07A Black Chassis.  It's now running as the "firewall" on
my local network (which means I don't need to leave my work machine
running 24 hours a day) (one, but not the only reason for needing a PC
as firewall is that the model is some strange PPPOE thing - ie it's
not your usual cable modem).

The good news is that it is doing what it was intended to do with no
serious problems - currently acting as the firewall and providing
DHCP/DNS for a small network.  Later I may add music streaming, and I
am sure it will handle that with no problems too.

32 bit opensuse installed with no real issues (I used the network
install since my only USB drive is CD, not DVD, which is slow, but
that's not the machine's fault), but I had to get the openchrome
graphic driver separately (via 1 click install from
http://software.opensuse.org/search ) - it's worth installing that
driver, as it significantly improves the "feel" of the display, even
if you're only going to be logged in for configuration.

But there are also several disapointments.  The hardware itself isn't
as good as I'd hoped - the chassis contains two very small fans which
are not loud, but which make an irritating high-pitched noise.  And
the front "bar" of the case glows blue.  Both of these issues can be
addressed to some extent by "hand" I expect - I haven't yet attacked
the LEDs, but I replaced the fans with a single 120mm Thermaltake fan,
which just fits in the space available).  This seems to give better
cooling for the CPU, even though the original tiny fan was fastened
directly to the small heatsink, which this simply sits above and to
one side.  It's still as load (not very, it must be said, but
certainly not silent), but now much less irritating.

(Note that the fan must use the small 3-pin motherboad fan controller
plug as the mobo will not start if no fan is detected.  There appears
to be no speaker, so mobo not starting is a bit opaque.  And there's
very little documentation...)

And that brings us to cooling.  Although the CPU is supposed to be
"low power", has throttling, and the BIOS provides a configurable
"limit" temperature, the CPU does not appear to auto-throttle. The
BIOS value is simply passed to ACPI, but the CPU (C7-D) doesn't have
the "ets" extension needed by Linux's CpuFreq module.

In other words, there appears to be nothing fom stopping you melting
the core.

And since the heatsink appears to be the minimal possible, the
temperature changes rapidly (little thermal mass) and can get
worryingly high under load.  It's winter here, and perhaps 20-25C in
the house, and the highest temperature I have seen is around 85C.  I
believe the specs allow for up to 100C.  So in summer, with ambient
temperatures of 30-35C we will be close to frying this thing.

For now, for peace of mind, I have thrown together the following
script, which assumes lm_sensors is installed (only one of two chips
detected is needed - the kernel module for the other appears to be
proprietary and isn't in opensuse).

#!/bin/bash

# 90/60 will trigger only in extreme cases and isn't going to return to
# normal unless things are *really* calm
upper=90
lower=60

# in normal use (T0) warn above this temperature
warn=80

log="/var/log/messages"
temp=`sed -e "s/[^0-9]*//g" /proc/acpi/thermal_zone/THRM/temperature`
state=`egrep 'active state' /proc/acpi/processor/CPU0/throttling | sed
-e "s/active state: *//"`

if [[ "$state" == "T1" || "$temp" > "$warn" ]]
then
  echo "throttle: temp $temp C in state $state" >> $log
fi

if [[ "$temp" -lt "$lower" && "$state" == "T1" ]]
then
 echo "throttle: enabling T0" >> $log
  echo "T0" > /proc/acpi/processor/CPU0/throttling
fi

if [[ "$temp" -gt "$upper" && "$state" == "T0" ]]
then
 echo "throttle: enabling T1" >> $log
 echo "T1" > /proc/acpi/processor/CPU0/throttling
fi

That's pretty simple and is run once a minute via cron.  The core
temperature is logged above 80C and at 90C the CPU is automatically
throttled until the temperature drops back to below 60C.

Note that the throttling is 50% and *really* makes a difference to
interactive use (it's like using a machine that's paging heavily).

Ah, and one other issue, although this is more just me being clueless
- if you've not used a chassis like this before, it's not clear how to
fasten the disk (it certainly wasn't to me).  What you have to do is
remove the two metal rails that the disk fits between, then you can
use the usual screws to fasten those rails to the disk, before
reassembling the whole thing (with the disk now in place).  It's
physically impossible to fasten the disks to the rails without
removing them.  Oh, and also: you ned to remove 5 screws to get in the
box, which seems excessive...

Apart from that, it's like using a machine from 5 years ago, which
really is about what you'd expect - basic things like surfing the net
or config tools can use up a large fraction of the CPU, and things
aren't as slick and quick as I've come to expect on my development
box.  But that's as expected - what have soured the deal are the
details above.

Andrew

Blue LEDs on PeeCee07A (PC2500e)

From: andrew cooke <andrew@...>

Date: Sun, 2 Aug 2009 12:19:50 -0400

So, with everything else working just fine (I now have mediawiki, tor,
DHCP, DNS, etc running) I just needed to kill the bright blue strip
along the front of the case.  Turns out this is trivial - they can be
unplugged from the motherboard.

With the case open, and the front (silver part) facing you, the
connector for the front panel is on the near edge, to the left (near
the SATA connectors).  At first glance it looks a single plug, but
it's actually many smaller two-pin plugs, all packed together.
Unplugging the far left two-pin plug disconnects the blue panel LEDs
with no apparent side effects.

I also added some insulating tape between the disk drive and mounting
rails, and to the new fan where it rests on the case, to help reduce
high pitched noises - not sure if it has helped.

Andrew

Mail Based Blog + Gmail

From: andrew cooke <andrew@...>

Date: Thu, 13 Aug 2009 08:12:59 -0400

Elliott wrote to say that:

  +anything goes to the same gmail address; so
    username+blog@...
  would work; couldn't you just then get that with imap and process it?
  You could also set up a filter in gmail to move it from the inbox and
  label it "blog", say.

which sounds like a good idea.  I already have things working via my
ISP, but this would be simpler and assuming it works (haven't tested
it myself) I will use it to simplify things at some point.

(Incidentally, if you want to reply directly here, you can use the
email address from the link at the bottom of the page rather than
emailing me directly - although direct email is fine too, of course).

Andrew

Comment on this post