Andrew Cooke | Contents | RSS | Twitter | Previous

C[omp]ute

Welcome to my blog, which was once a mailing list of the same name and is still generated by mail. Please reply via the "comment" links.

Always interested in offers/projects/new ideas. Eclectic experience in fields like: numerical computing; Python web; Java enterprise; functional languages; GPGPU; SQL databases; etc. Based in Santiago, Chile; telecommute worldwide. CV; email.

© 2006-2015 Andrew Cooke (site) / post authors (content).

[GPU, Programming] Per-Thread Program Counters

From: andrew cooke <andrew@...>

Date: Mon, 8 Apr 2019 19:37:14 -0400

https://medium.com/@farkhor/per-thread-program-counters-a-tale-of-two-registers-f2061949baf2

Andrew

Permalink | Comment on this post

Previous Entries

For comments, see relevant pages (permalinks).

My Bike Accident - Looking Back One Year

From: andrew cooke <andrew@...>

Date: Mon, 11 Mar 2019 09:39:13 -0300

One year ago today, Sunday 11 March 2018, just after breakfast, I was
looking for my favorite cycling shirt, getting ready to ride a route I
hoped to share later in the week with a friend.

That is all I remember.

https://www.strava.com/activities/1458282810/overview

Then, in a warm haze, I am thinking "this could be serious;" "maybe I
should pay attention;" "focus."  Sometime around Tuesday or Wednesday,
in Intermediate Care at Clinica Alemana, with nurses and beeping
machines, and Paulina explaining to me (patiently, every hour or so
for the last few days - I had been "conscious but absent" since Monday
morning) that a car had hit me; that I had been injured and operated
on; that I had received blood transfusions; that I had a steel rod in
my leg; that I was now OK.

 - Head trauma
 - Fractured right clavicle (collarbone)
 - Exposed, fractured left femur (thigh)
 - Fractured metatarsal (hand)
 - Fractured right ribs (3rd, 4th, and 5th)

I was in hospital for 7 days.  The last few in normal care.  Final day
I asked to have a shower.  When I saw blood running with the water I
fainted.

The ambulance that took me home had the same crew that had taken me
in.  I asked how I had been - the EMT said "in some pain."

Final cost $17.894.596 (CLP - around $30k USD).

Home, Paulina had the bed raised, an extra seat on the toilet, a seat
in the shower, a wheelchair.  I remember my first shower - it was a
huge effort to lift my foot over the (4 inch) shower wall, and I
collapsed, twitching, on the seat.

I was high as a kite - even back home - on opioids for a couple of
weeks.

My recovery was slow but steady.  A physiotherapist came to visit and
taught me some exercises.  After a month or two I was walking with
crutches.

Paulina was exhausted from caring for me while still trying to work.
For a while we had someone visit, a few times a week, to clean and
prepare some food.


On Sundays many roads here are closed to cars, given over to cyclists,
runners, inline skaters.  A week after my accident a friend returned
to the intersection.  He found a witness, someone who flagged when
traffic could or could not pass through the ciclovia, who said I was
hit by a pickup that had run a red light.

I later learned that the driver stopped (to his credit).  Someone
called the police and an ambulance.  I was on the ground, dazed,
trying to stand but unable.  The police asked me where I lived -
apparently I replied "Europa," which is the name of our street, but
also, literally, "Europe."  So they assumed I was a tourist - a
wealthy gringo with travel insurance - and sent me to the best
hospital in town.

An investigation was opened by the police.  My medical records include
a blood test showing no alcohol.  We informed the investigating
magistrate of the witness but later, when called to the police station
to give evidence, they had not received the information.  We gave it
again.  By the time it was investigated video records from street
cameras had expired.


After the accident my bike was in a police compound; Paulina collected
it and I started repairs.  The front wheel was tacoed, so I bought a
new rim (which Paulina collected - I am so grateful for all the
legwork she has done over the last year) and spokes, and laced it to
the old hub.

Mounting the new wheel on the bike, I realized that the thru-axle was
bent, so I ordered a new axle.  When I received the axle I realized
the hub itself was bent, so I ordered a new hub.  Given how Shimano
thru-axle hubs work, I only needed to replace the inner sleeve (so I
didn't need to rebuild the entire wheel).

Mounting the new wheel again, I realized that the fork was bent, so I
ordered a new fork.  This was delivered to the UK, because mid August
I felt good enough to travel home and see my parents.

I also replaced the handlebars, although the (slight) damage there was
caused by me over-tightening the brakes, not the accident.  In
addition I had to replace the rear light (stolen while in police
custody) and my helmet.


The weekend of September 8/9 I was feeling good enough to travel with
Paulina to La Serena.  We wanted to check on my old flat, where a
friend had been living rent-free, to make sure it was OK for Paulina's
father to move there.

The flat was a mess.  So bad we did not sleep there, but instead
walked into town and stayed at a hotel.  The next day we returned, to
continue cleaning.  By the end of the weekend the place wasn't too
bad, but my leg was painful.

That was the high point of my recovery.

Post operation, my thighs were asymmetric - on the left hand side was
a "bulge" which, clearly visible in the X-rays, enclosed the end of
the rod that held my femur together.  The rod was "too long."  It
appeared to be "rubbing" on the inside of the leg, placing a limit on
how far I could walk.  As it became more inflamed, I could walk less
distance.  The upper limit was around 3,000 steps a day (a few km).

The day after returning from La Serena (Sept 10) I asked the doctor
what could be done.  The answer was: nothing, until the bone had
healed, which takes a year.


On September 11 I attended court.  The police claimed that the driver
had illegally run a red light.  Chilean law is different to UK law -
for a "light" infraction like this (running a red light and not
killing me) the emphasis is on compensating the victim.  In general
terms, either we agree some kind of compensation, or the driver is
prosecuted.  The driver has to balance the amount of compensation
against the inconvenience of being prosecuted, the likelihood of being
convicted, and the possibility of any sanction.

To start negotiations over compensation we needed to know the amount
outstanding after (the driver's) accident and (my) medical insurance,
but we still had not been billed by the hospital.

So the case was postponed and we returned home to chase up the
paperwork.  Once we had the bill Paulina took it to the driver's
insurers, who agreed to pay $5.854.407.  Then she went to my medical
insurance, who eventually (December 21) agreed to pay $8.327.938,
leaving a balance of $3.712.251.

And this is where we stand.  The case appears to be stalled pending
further police investigation.


Since it was difficult to walk I tried cycling again.  This was
clearly better for my health, and I could manage around 20 minutes
without hurting my leg too much.

But, perhaps related to this exercise, a new problem surfaced.  The
rod appeared to get "caught" on something (tendon? muscle?).  This
hurt, I froze and slowly wiggled my leg to "undo" the blockage.
Afraid to walk, I hobbled slowly round the house.

Despite my reduced movements this repeated, more severely.
Frustrated, and now nearly a year after my operation (February 18,
2019), I returned to the doctor.  He was, I think, surprised.  The
next day I received a call from the hospital - someone had canceled an
operation, there was a free slot Fri February 22.  I agreed
immediately.


The operation to remove the rod went smoothly.  I entered theater late
in the day and was kept for observation overnight.  The leg had two
dressings - one near the knee (incisions to remove screws) and another
on the upper thigh (more screws and the rod itself).  These were the
usual clear plastic sheets, with external padding for protection, to
be left in place as the wound heals.

Thursday, February 28, I was feeling good enough to be sat at the
computer, working, when I felt a drop of liquid hit my leg.  Removing
the padding, visible through the dressing, were blisters.  One had
burst.

Back at the hospital, the dressings were removed, the skin wiped
clean.  I was sent back home with basic antibiotics and
anti-histamines.

Life with exposed wounds and stitches is boring and uncomfortable
(although the anti-histamines meant I slept much of the time).  The
stitches catch clothing and the wound has to be kept clean and open to
the air, so you're either lying in bed or wandering cold and naked
through the house.  It was uncomfortable to be seated for any length
of time, making work difficult (credit to my employers for their
support).

Monday March 4 I returned to hospital.  Although I felt things were
improving (no blood / pus stains on the bedsheets on the last night,
for example) it still didn't look good (quite frankly, it looked
terrifying - red, yellow and blistered - but it was not painful and
did not smell).  A nurse (a nice nurse - senior and smart and
friendly) thought it looked more like an infection than an allergy,
and the doctor agreed, changing the antibiotic to something more
specific.

The next few days, although still boring and uncomfortable, showed
real improvement.  On Wednesday March 6 my stitches were removed.

Since then, the skin has continued to heal.  Importantly, the pain
from the rod - at least the worst, when it got "hooked" around tendons
- has gone.  There is still some pain when walking, but it is
difficult to know if it the old soreness, or associated with the
bruising from the operation.


A year after the accident, I still do not know if I will be able to
walk, or cycle, as before.

Andrew

Permalink

[Python] Geographic heights are incredibly easy!

From: andrew cooke <andrew@...>

Date: Sun, 13 Jan 2019 20:47:20 -0300

Wanted to add heights to bike rides in Choochoo, given the GPS
coordinates.  At first I considered stealing the data from Strava, but
their T&C don't like it and anyway I couldn't find it in the API.
Then I considered Google's Elevation Service, but it's $5 for 1,000
values.  Then I considered the free OpenStreetMap equivalent, but that
seemed to be broken.  Then I realised it's pretty damn easy to do by
hand!

Turns out that the Space Shuttle scanned the entire Earth (except the
Poles) at a resolution of one arcsecond (about 30m on the equator) and
the data have been publicly released.

The project is called SRTM, and the 30m data are "v3" or SRTM3.  More
info at https://wiki.openstreetmap.org/wiki/SRTM and
https://en.wikipedia.org/wiki/Shuttle_Radar_Topography_Mission

You can download a 1 degree square "tile" here
http://dwtkns.com/srtm30m/ and there's an excellent (although buggy)
blog post on how to parse the data at
https://librenepal.com/article/reading-srtm-data-with-python/ with
code at https://github.com/aatishnn/srtm-python

Because the coords are in the file name there's no need for any kind
of RTree or similar index - you just take your coords and infer what
file you need.  Assuming you already have it downloaded you read it,
stick it in an array, and return the array value!

My own code to do this is at
https://github.com/andrewcooke/choochoo/blob/master/ch2/sortem/__init__.py
and includes bilinear interpolation (you could cut+paste that code
except for the constructor which is specific to Choochoo - just replace
the messing around w Constant with a literal directory string).

The tests are at
https://github.com/andrewcooke/choochoo/blob/master/tests/test_sortem.py
and from the contours there, which are plotted across 4 tiles, it's
pretty clear that the interpolation is seamless.

So easy!  I thought this would take a week of work and I did it all
this afternoon....

Andrew

Permalink

[Cooking] Cookie Recipe

From: andrew cooke <andrew@...>

Date: Tue, 25 Dec 2018 17:30:46 -0300

125g butter
1 egg
60g sugar (or more?)
45g cocoa
130g flour (w raising powder)
120g chocolate

Mix butter, egg and sugar.  Sieve in and mix cocoa and flour.  Add a
little water if necessary - want a thick, sticky mix, as solid as
possible, but not powder.

Break the chocolate into pieces and add.

Cool in fridge.  Pre-heat oven to 180C.

Place blobs of dough on lightly greased baking paper on tray.  Cook
for 15m.  Should flatten but not spread much.

Not very sweet, except for the chocolate.  Very cocoay and good
texture.  Good w ice-cream?

Variations: more sugar?  vanilla?

Andrew

Permalink

Efficient, Simple, Directed Maximisation of Noisy Function

From: andrew cooke <andrew@...>

Date: Fri, 14 Dec 2018 18:04:10 -0300

Maybe this is already known - I guess it must be - but I just dreamt
it up myself.  I need to find the max (could be min, obvs) of a noise
function in 1D.

Since it's noisy I can't assume it's smooth, hope it has a global
minimum, and bisect.  I need something more robust.

At the same time, I don't want to be doing repeated evaluations of the
function where they're not needed, so any iterative deepening of a
search should re-use old values when possible.

So here's the solution:

  * Evaluate the function at 5 equally spaced points across the range
    (including the two extremes).

  * Throw away two end points.

  * Expand the remaining 3 points back to 5 by inserting new points at
    the mid-points of the existing points.

  * Repeat until you're happy.

The trick is to know which end points to discard.  It could be one
either end, or two from one end.

What I do is calculate the average of the x values at the points,
weighted by the function values there.  Then I compare this to the
unweighted average.  If it's higher, then the max is towards the high
end, so discard a low point.  Or vice-versa.  And repeat for a second
point.

Here's an example.  The lines are (x, f(x)) pairs:

[(0.0, 0), (0.25, 5), (0.5, 10), (0.75, 1), (1.0, 1)]
[(0.25, 5), (0.375, 12), (0.5, 10), (0.625, 2), (0.75, 1)]
[(0.25, 5), (0.3125, 11), (0.375, 12), (0.4375, 10), (0.5, 10)]
[(0.3125, 11), (0.34375, 12), (0.375, 12), (0.40625, 10), (0.4375, 10)]
[(0.3125, 11), (0.328125, 11), (0.34375, 12), (0.359375, 13), (0.375, 12)]

On the first step, both ends were discarded.  On the second, two from
the "high" side, etc.  The maximum is 13 at 0.359375, roughly.

And here's the code:

def expand_max(lo, hi, n, f):
    data = [(x, f(x)) for x in (lo + i * (hi - lo) / 4 for i in range(5))]
    for _ in range(n):
        print(data)
        while len(data) > 3:
            w = sum(x*fx for (i, (x, fx)) in enumerate(data)) / sum(fx for (x, fx) in data)
            m = sum(x for (x, fx) in data) / len(data)
            if w > m:
                del data[0]
            else:
                del data[-1]
        x = (data[0][0] + data[1][0]) / 2
        data.insert(1, (x, f(x)))
        x = (data[-2][0] + data[-1][0]) / 2
        data.insert(-1, (x, f(x)))
    x_max, fx_max = None, None
    for (x, fx) in data:
        if x_max is None or fx > fx_max:
            x_max, fx_max = x, fx
    return x_max, fx_max

Andrew

Permalink

Bash Completion in Python

From: andrew cooke <andrew@...>

Date: Fri, 7 Dec 2018 08:02:11 -0300

https://www.reddit.com/r/Python/comments/a3vg7q/i_made_a_tutorial_for_writing_bash_completion/

https://github.com/CarvellScott/completion_utils

Andrew

Permalink

[Computing] Configuring Github Jekyll Locally

From: andrew cooke <andrew@...>

Date: Sun, 25 Nov 2018 15:02:59 -0300

If you have docs that already exist, and that github displays on
Github Pages, then you may want to run jekyll locally - just to save on
round-tripping time or when github (as today) starts thoring errors
for no reason.

The instructions at
https://help.github.com/articles/setting-up-your-github-pages-site-locally-with-jekyll/
are pretty confusing if you already have things working on github.

What you need to do is:

* Install requirements as described (may need to run as root)

* In your local repo, at the root of the jekyll pages (for me, the
  docs directory), create the Gemfile described in step 2 and then
  install jekyll.

* Run jekyll in that same directory (step 4)

In other words, SKIP steps 1 and 3 and do everything in the docs
directory.

At least, that seemed to work for me.

Andrew

Permalink

[Maths, Link] The Napkin Project

From: andrew cooke <andrew@...>

Date: Tue, 20 Nov 2018 08:10:21 -0300

Introductory higher math.

http://web.evanchen.cc/napkin.html

Andrew

Permalink

[Bike] Servicing Budget (Spring) Forks

From: andrew cooke <andrew@...>

Date: Tue, 6 Nov 2018 11:49:06 -0300

Lower priced mountain biks often come with forks that contain a coil
spring (rather than an "air spring").  These tend to not get a lot of
love on the intenet.  That's partly justified - an air fork is much
more adjustable and has better damping.  But the spring forks do have
some advantages too.  Most of all, they're very low maintenance - most
people don't touch them once they've bought them.

Despite that, a little care can help keep your fork working well and
extend its life.  I just serviced mine and I thought I'd make a few
notes here ot help others.

First, I am not 100% sure what my fork is.  I removed the stickers
years ago.  I think it may be a Suntour XCM (26").  Whatever, the
general principles below should apply to pretty much and budget fork.

There are two things you can do.  The simplest (and most important) is
to remove the lowers.  More complicated is to open up the "insides"
too.


Removing Lowers

First, remove the two bolts holding the front brake (I'm assuming
disc) so that it's no longer connected to the fork.  Also remove the
clip or zip tie or tape that holds the hose to the fork.  Then you can
ignore the brake (until the end, when we need to put it back on).

(Don't press the brake lever with the caliper loose - you can push the
hydraulic pistons out).

Turn the bike upside down, so it's sitting on the bars (or put it in a
stand if you have one I guess) and remove the wheel.  Have a look at
the bottoms of the forks.  On mine each has a 10mm nut.  Remove the
nuts and see if the you can slide the lowers off.  Don't use violence
- just pull them up with your hands while keeping the bike on the
floor with your foot.

Possibly they won't come off.  This is because the rods inside the
fork tend to stick in the ends of the lowers.  We can loosen them by
pushing the threads that the nuts were fastened on back "into" the
lowers.

To do this, take a piece of wood, lay it on top of the threaded rod,
and strike with a hammer.  You may need a few attempts (change the
position of the wood since the rods makes a hole!) before the threaded
rods moves noticeably.  Once both sides are loose, you should be able
to pull the lowers off, no problem.

With the lowers off, clean the exposed stanchions and the inside of
the lowers (eg with a stick and cloth - if the cloth gets stuck
inside, blow it out from the other end).

The stanchions are steel, so can rust.  This is why it's worth
servicing them - we want them to stay smooth and clean.  So rub some
grease on them (I use random car grease - nothing fancy).

To reassemble, first put some more grease on the insides of the seals.
Then slide the lowers back on the stanchions.  Push them down and the
rods will re-appear out of the ends.  Replace and tighten the nuts
(fairly tight - you don't want these coming undone...).

Finally, replace the wheel and re-attach the brakes (once the caliper
is in place, apply the brakes and then tighten the bolts - that will
help tighten the bolts with the calipers in the right position to
avoid rubbing).


Looking Inside

If you want to look "inside" the fork, first remove the lowers (as
above).

Next, use an appropriate tool from Suntour to unscrew the two caps on
the "top" of the fork (one on each side).  This is easier to do with
the bike lying on its side (or upright in a stand).

I'd really enourage buying the right tool.  Mine is just a plastic
wrench (looks like a big bottle opener or measuring spoon).  You can
get the caps off without it, but getting the spring side back on is
tricky (because the spring is compressed) and without the tool you're
more likely to cross-thread the cap.

If you have a lockout you may need to move the lockout knob before
removing the cap.  Mine just levers off (carefully insert a flat
screwdriver underneath and lift).

It's interesting to see what's inside, but there's not much to do
except clean and grease.  I removed an elastomer damper from "inside"
my spring so that the forks had a little more travel, but I'm not sure
I noticed much difference.  On the other side, my fork has a damper,
but it's a sealed unit with nothing to adjust.

(BTW I couldn't remove the "foot" on the spring side because the
bottom-out damper held it in place.)

Assembly is the reverse process.  Be careful replacing the cap on the
spring side as it's easy to cross-thread.  My caps are plastic so I
didn't tighten them crazy-tight.


I hope that helps.  With a little care you can keep these babies
rust-free and they'll last forever.

Andrew

Permalink

[Crypto] CIA Internet Comms Failure

From: andrew cooke <andrew@...>

Date: Sat, 3 Nov 2018 17:37:18 -0300

It's difficult to tell exactly what happened, but my guess from
reading between the lines is that the CIA had a bunch of fake sites
that were used as endpoints for communication.  Somehow the Iranians
detected a pattern in the way these sites were generated (eg matching
comments in the HTML source) that let them identify the sites and, via
traffic monitoring, work out who was using them.

https://in.news.yahoo.com/cias-communications-suffered-catastrophic-compromise-started-iran-090018710.html

Andrew

Permalink

[Python] Cute Rate Limiting API

From: andrew cooke <andrew@...>

Date: Thu, 25 Oct 2018 21:13:23 -0300

It's just a decorator - https://pypi.org/project/ratelimit/

Andrew

Permalink

[Causality] Judea Pearl Lecture

From: andrew cooke <andrew@...>

Date: Fri, 19 Oct 2018 12:00:01 -0300

Got confused by his book years ago, but this lecture is very good.

https://www.youtube.com/watch?v=RPgvfSeQB8A

Andrew

Permalink