Diary of a Switcher

Force HSTS for Wikipedia in Chrome

2014-08-02T00:00:00+00:00

Earlier I was browsing wikipedia and noticed it wasn’t being accessed over HTTPS. I checked and, sure enough, wikipedia doesn’t set an HSTS value (there’s some discussion of it on their bugzilla, their primary reason seems to be the failure mode if a country were to block HTTPS then there wouldn’t be a way for Mozilla Firefox users to manually remove the rule and access wikipedia again).

Since I’m using Chrome I’m happy to force HSTS. I’d also like to pin the public key. In order to do this I had to get the HSTS pubkey hash for wikipedia.org; I found an excellent utility from StalkR on their blog about almost this exact issue, HTSTS Preloading, Public Key Pinning and Chrome</a>, http_pins.py. I used this revision of the code</a>

This code expects a local certificate, so I had to get a CRT version of wikipedia’s certificate. I did this using OpenSSL (required by http_pins.py). The command is:

openssl s_client -showcerts -connect en.wikipedia.org:443 </dev/null | openssl x509 -pubkey -noout >wikipedia.crt </blockquote>
This uses OpenSSL’s client to connect to en.wikipedia.org, fetches the certificate chain and then hangs up. The certificate chain is then sent to another openssl command that extracts the first certificate’s public key and writes it to wikipedia.crt. It would be a good idea to check that the intermediate certificate chain is sensible before proceeding.
Next, run http_pins.py on that file. Here’s what I got (but you should check yourself):

$ ./http_pins.py wikipedia.crt
wikipedia.crt:

SPKI fingerprint (sha256): 99:4a:39:4a:21:55:bf:ac:c6:46:71:9b:c0:dd:0b:d4:4b:f7:4a:95:36:19:fa:7c:de:52:75:07:2f:5a:23:77
Public-Key-Pins: max-age=600; pin-sha256=“mUo5SiFVv6zGRnGbwN0L1Ev3SpU2Gfp83lJ1By9aI3c=”
Warning! Per RFC you need at minimum two pins</li> </ul> </blockquote>
You can now go to the Chrome HSTS page, chrome://net-internals/#hsts , and add the domain “wikipedia.org”, ticking Include subdomains for STS and Include subdomains for PKP, then adding the quoted value from above (but prefixing it with with sha256/). For me (and hopefully you too!) this was:

sha256/mUo5SiFVv6zGRnGbwN0L1Ev3SpU2Gfp83lJ1By9aI3c= </blockquote>
Happy secure browsing! Remember if this breaks something it’s not my fault :-) you can always go back into the Chrome HSTS page and delete the preset you’ve added for wikipedia.org

Forcing TLS on YouTube with Chrome

2013-11-21T00:00:00+00:00

I use Youtube Options extension for Chrome to opportunistically enable HTTPS on YouTube, but there’s a problem with this - the initial request is made by HTTP (and this makes the use of the Back button challenging, because you have to double-tap it if you came in by HTTP and were redirected to HTTPS).
Unfortunately, Google have not yet rolled out Enforce SSL to YouTube (which, as I understand it, sets an HTTP Strict Transport Security flag so the browser always makes the first request by HTTPS). However, Chrome has a convenient settings page that lets you add your own entries to the local HSTS set.
chrome://net-internals/#hsts
On this page you can query HSTS for a domain - or, usefully, add custom entries. I queried youtube.com which gave me:
Found: mode: OPPORTUNISTIC sts_include_subdomains:true pkp_include_subdomains:true domain:youtube.compubkey_hashes:sha1/vq7OyjSnqOco9nyMCDGdy77eijM=,sha1/Q9rWMO5T+KmAym79hfRqo3mQ4Oo=
I simply took the pubkey_hashes value, pasted them into the “Add Domain” box at the top, ticked “Include subdomains for STS” and “Include subdomains for PKP”, then clicked “Add”. Now querying youtube.com results in:
Found: mode: STRICT sts_include_subdomains:true pkp_include_subdomains:true domain:youtube.compubkey_hashes:sha1/vq7OyjSnqOco9nyMCDGdy77eijM=,sha1/Q9rWMO5T+KmAym79hfRqo3mQ4Oo=
And I no longer get HTTP requests made when I visit http://www.youtube.com - success!
I’m not sure if this lives in a cache that gets purged over time, so I’ve kept Youtube Options installed in the meantime.

Visualising a water filling algorithm

2013-11-01T00:00:00+00:00

I was looking at the water filling problem described at “I Failed a Twitter Interview”</a>. The problem is, given a topography expressed as an array of heights, to figure out the volume of water that will pool if it rains on that topography. I’m generally fairly rubbish at puzzles, and my solution to the problem only worked for a particular subset of the problem.
I was then trying to understand the solution the author presented, so I wrote a visualisation of it. I’ve included the code below, it produces a move-by-move visualisation of the state of the solver. Writing it helped me understand the elegance of the presented by the solution.
The output looks like the following (where ~ represents water):
 _ _ _ |8| |7|~~~|7|~_~~~~~~~| | _ | |~~~| ||6|~~~~_~| |~~~~_~|6| | |~~~| || |~_~|5|| |~~~|5|| | | |~_~| || ||4|| || |~~~| || | _ _ _ | ||3|| || || || || |~~~| || |~~~|3||3| _ |2|| || || || || || || |~~~| || |~~~| || ||2| | || || || || || || || |~_~| || |~_~| || || | Current Volume: 20 </code></pre> Code: import java.util.Random; public class WaterLevel { private static final long PER_MOVE_SLEEP = 300; public static void main(String[] args) throws java.lang.Exception { calculateVolume(generate(10, 15), true); } public static void calculateVolume(final int[] land, final boolean showMoves) { final int[] water = new int[land.length]; int leftMax = 0; int rightMax = 0; int left = 0; int right = land.length - 1; int volume = 0; while (left < right) { leftMax = Math.max(leftMax, land[left]); rightMax = Math.max(rightMax, land[right]); final boolean movedLeft; if (leftMax >= rightMax) { volume += rightMax - land[right]; water[right] = (rightMax - land[right]); right--; movedLeft = false; } else { volume += leftMax - land[left]; water[left] = (leftMax - land[left]); left++; movedLeft = true; } if (showMoves || left == right) { // Render the state render(land, water, left, right, volume, movedLeft); // Wait until rendering the next frame if (left != right) { try { Thread.sleep(PER_MOVE_SLEEP); } catch (InterruptedException e) { throw new RuntimeException(e); } } } } } /** * Generate a topography * * @param maxHeight * maximum height * @param width * width * * @return */ private static int[] generate(int maxHeight, int width) { final Random random = new Random(); int[] land = new int[width]; for (int i = 0; i < land.length; i++) land[i] = random.nextInt(maxHeight); return land; } private static int max(final int... land) { int max = land[0]; for (int level : land) max = Math.max(max, level); return max; } /** * Render the map, showing the land - and on top of it, the computed water thus far (and also showing the left and right * pointers) * * @param land * the land height * @param water * the water depths computed so far * @param left * the left pointer position * @param right * the right pointer position * @param volume * the current total volume * @param movedLeft * true if we have just moved the left pointer (false if we've just moved the right) */ public static void render(final int[] land, final int[] water, final int left, final int right, final int volume, final boolean movedLeft) { // Compute max land level so we don't waste space rendering open air final int max = max(land); // Allocate enough space for the output final StringBuilder sb = new StringBuilder((3 + max) * 3 * land.length); // Render the pointer position (and indicate movement) for (int i = 0; i < land.length; i++) { if (i == left) sb.append(movedLeft ? " > " : " . "); if (i == right) { sb.append(!movedLeft ? " < " : " . ").append('\n'); break; } else sb.append(" "); } // Now render the map for (int altitude = (max + 1); altitude > 0; altitude--) { for (int j = 0; j < land.length; j++) { final int groundLevel = land[j]; if (altitude - 1 == groundLevel) if (water[j] != 0) sb.append("~_~"); // Wet ground else sb.append(" _ "); // The ground else if (groundLevel > altitude) sb.append("| |"); // Below the ground else if (groundLevel == altitude) { sb.append("|" + altitude + "|"); // Just beneath the ground } else if (groundLevel + water[j] >= altitude) sb.append("~~~"); // Underwater else sb.append(" "); } sb.append('\n'); } sb.append("Current Volume: ").append(volume).append("\n"); System.out.println(sb); } } </code></pre>
Glassfish 3.1.2.2 Web Profile and EclipseLink MOXy 2013-05-11T00:00:00+00:00 I’ve been evaluating Glassfish to possibly replace Tomcat as our servlet container of choice at work. Because one of the things I love about Tomcat is that it’s so light-weight I read the list of included features in Glassfish Full Profile with a growing sense of dread. Our .war files only want a basic Servlet environment that provides a database link - they use this to expose RESTful resources and services. Because of this, the Glassfish Web Profile seemed to suit our needs best - all the niceness of multiple domains & a more robust automated deployment without the word “Enterprise” being mentioned too often. MOXy Trouble#</a></h2> So I took one of our applications and deployed it in Glassfish. But when our code tried to initialise an EclipseLink MOXy JAXBContext we got: java.lang.NoClassDefFoundError: javax/xml/bind/JAXBException ... at java.lang.Thread.run(Thread.java:722) Caused by: java.lang.ClassNotFoundException: javax.xml.bind.JAXBException not found by org.eclipse.persistence.moxy [156] at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1460) at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:72) at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1843) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) ... 31 more </code></pre> Hmm, not so good! I did some digging and discovered the problem was a combination of factors: Glassfish Web Profile doesn’t include a JAXB implementation</li> Because it has no JAXB implementation it falls back on the JAXB implementation in the Java environment</li> For OSGI purposes, it assumes the version of JAXB provided by the environment is “0.0.0”</li> EclipseLink MOXy (helpfully included in Glassfish Web Profile) requires version 2.0.0 of javax.bind.xml</li> </ol> So unfortunately the version of JAXB available to MOXy is 0.0.0 but it needs 2.0.0. The Solution#</a></h2> The solution to this problem turned out to be pretty simple - simply copy the JAXB implementation included in the Glassfish Full Profile into the lighter Web Profile installation: cp jaxb-osgi.jar ~/glassfish/glassfish/modules/cp jaxb-api-osgi.jar ~/glassfish/glassfish/modules/endorsed/ </code></pre> I actually had an additional step - the latest MOXy libraries (2.4.1) have a bug that affects our software and currently necessitates we use a snapshot release - so I replaced glassfish/modules/org.eclipse.persistence.* with the 2.4.2-SNAPSHOT jars. Restart the domain and hey presto, no exceptions. iPhone 4S to Samsung Galaxy S3: The Hardware 2012-07-10T00:00:00+00:00 I’ve been a long-time iPhone user but lately Apple’s lawsuit-heavy approach to their competition has been annoying me - so I decided to give Android a try. I ended up picking the Samsung Galaxy S3. I had some difficulty finding reviews that told me what I wanted to know (namely, how does the Galaxy S3 compare to the iPhone 4S hardware and iOS 5) so I thought I’d write a little about it. This post is dedicated to talking about the hardware, hopefully later post(s) will talk about the software. Some of the points here are overly particular - I really like the iPhone 4 design and Samsung just aren’t in the same league. Size#</a></h3> The S3 felt a little odd initially, my hands are on the large side but this phone really is a lot bigger than the iPhone: the entire iPhone fits within just the screen part of the S3. It’s also a fair bit thinner and lighter (which is continuing to make the phone feel less secure in my hand, although I assume this will pass). It feels comically large when using it in public, although I’m sure this is just something I need to get used to, coming from the diminutive iPhone. It’s a little bit too large for comfortable single-handed use, especially when I need to swipe down the notification bar - I have to adjust my grip to a less secure one to swipe down the notification bar with my thumb. Screen#</a></h3> While it’s not as comfortable to use with one hand the large screen is extremely nice for browsing the web, typing or looking at photos. The frame at the top and bottom is much smaller than on the iPhone, giving the screen much more of a presence. The Galaxy S3’s screen is tinted a little more yellow than the iPhone, and it’s also not as bright - that’s not necessarily a bad thing all the time… (I can’t be the only person who’s woken up in the middle of the night, used my phone to check the time and been blinded for a few seconds with the backlight set to ‘eye melting’ when it’s on the lock screen). The automatic brightness on the S3 is just a touch on the dim side, so I’ve opted to keep it at 50% brightness Back#</a></h3> The back of the S3 is flexible plastic that pulls off fairly easily (to access the battery, micro sim and micro sd card slot). It’s not as nice a feel as the iPhone 4S glass back (which I really loved) but apart from the fact that it’s plastic it feels firm and doesn’t creak or flex when you hold it in your hand. The back curves, but just at the edges, so it doesn’t have the same overly curved feel of the iPhone 3G. The plastic back feels nicer than that on the iPhone 3G, too. Micro SD Card#</a></h3> Having a micro SD card is really nice - getting a high capacity iPhone always made me feel like I was getting ripped off, and I’ve not noticed any slowdown from having my music on the SD card. There are some software annoyances here but otherwise I’m pleased that I can buy a relatively cheap 64GB SD card to store music, videos and photos separately from apps. Lock Button#</a></h3> The lock button is on the right hand side of the S3 and it feels very strange, coming from the iPhone where it’s on the top. It isn’t as well separated as on the iPhone (and also doesn’t feel as good to press as the button on the iPhone, which has a nice firm click to it) Volume Rocker#</a></h3> The volume rocker is quite inconveniently placed - I have a tendency to hit the lock button at the same time - and because it’s a rocker it’s not as easy to hit up/down. Camera#</a></h3> I really love the camera in the iPhone 4S - the camera in the S3 isn’t as good, the photos are noticeably noisier in low light. I’m planning on posting a set of comparison photos to highlight this. Speaker#</a></h3> Surprisingly, the Galaxy S3’s speaker is significantly quieter and tinnier than the iPhone speaker - I wasn’t expecting this! I listen to a lot of podcasts using the speaker and it’s really noticeable (it’s even more noticeable with music but I tend to plug in my headset for that) Volume Rocker Headphone Port#</a></h3> The headphone jack doesn’t feel right, it doesn’t have the same firm feeling as the iPhone 4S and it has the same flaw as the iPhone 3G - because the jack is at a curve part of the 3.5mm jack is exposed. It’s not the end of the world - but again, a slight annoyance. Headsets#</a></h3> I’ve been using my iPhone in-ear headset which sounds nice and the start/stop click button works - but because of different resistor layouts the volume up/down buttons don’t work. I don’t know if there’s a standard but I was surprised by this. Charging Cable#</a></h3> I really like having micro USB on the phone instead of the iPhone dock connector - I have amassed a small army of Apple dock connector to USB cables over the years (and consequently have fewer micro USB cables - just the one that came with the phone (which is black and bulky) and the one that came with my kindle (which suits the phone much better - the white colour and the curved design looks right with the S3). I’ve opted to stick with the apple UK USB plug - it’s very slender compared to the converter that ships with the Galaxy S3 and the design has a much nicer feel to it. Call Quality#</a></h3> I’ve not talked on the phone yet so I don’t have any experience of the call quality - although it wouldn’t be hard to get better than the iPhone! Data signal#</a></h3> I use the O2 network and don’t have great coverage at my house - normally the iPhone has a weak EDGE or 3G connection. On the Galaxy S3 I get 2-3 bars of HSPDA (and occasionally HSPDA+) - the iPhone doesn’t indicate when it has HSDPA, though, so this might be an unfair comparison. It feels like I have better signal, though. Wifi signal#</a></h3> The wifi reception on the Galaxy seems worse than with the iPhone… it’s not really caused me a problem, though. Battery Life#</a></h3> My initial impressions are that the battery life is better than the iPhone, although the second day I had the phone out and about the GPS Daemon on the phone sucked up 65% of the battery in a few short hours (during which time I was stationary in my office). I’m not sure if there will be a repeat of this - it’s unclear which application caused the GPS Daemon to go nuts, although I do have Google Latitude set up as an experiment, the battery life would likely improve if I disable that. Back and Menu buttons#</a></h3> It’s a little confusing getting Back and Menu buttons after having used the iPhone with its one button for so long - I quite like the menu button (although Google seem to be pushing to remove it, so perhaps this isn’t the best time to get used to it…) and, while the back button seems like a great idea (and is certainly very useful for navigation) I find myself hitting it frequently when I’m just trying to hold the phone - I’m sure in time I’ll train myself to hold the Galaxy S3 The Right Way, I’m just still used to how best to hold the iPhone. MacOS Lion Sanity 2011-07-21T00:00:00+00:00 My changes to macOS Lion to make it work more like how I prefer zsh history verify#</a></h1> I was surprised by this, the new version of zsh (4.3.11) (or Apple when configuring it) sets the histverify</code> option. This means if you want to use the ! prefix or !! for executing previous commands you’ll need to press enter twice: peter@laptop:~$ echo x x peter@laptop:~$ sudo !! peter@laptop:~$ sudo echo x x peter@laptop:~$ ` </code></pre> Not the most annoying feature in the world but it does slow me down. I prepended unsetopt histverify</code> to my .zshrc to disable this. Inverted scrolling#</a></h1> I understand this feature. I like this feature. But I use Windows, Linux and OS X daily (on my own and other peoples computers) and this would confuse my brain. System Preferences, Trackpad, Scroll & Zoom, “Scroll Direction: Natural” unticked Keep three finger swipe left and right for browers#</a></h1> This is already a well established multitouch gesture. Three finger swipe left and right to navigate history. I don’t want to use it to switch desktops. System Preferences, Trackpad, More Gestures, “Swipe between pages” to “swipe left or right with three fingers” and “Swipe between full-screen apps” to “Swipe left with four fingers” EC2 Availability Zones and Instance Types 2011-07-14T00:00:00+00:00 UPDATED 2014: See the bottom of the page for a table of results updated as of the 21st of March 2014. One of the odd things Amazon doesn’t tell you about EC2 is that some instance types simply aren’t supported by particular availability zones in the US-East region. I work on libraries for provisioning so I should, perhaps, have noticed this earlier - however I just noticed it today when I was updating some code to handle the new per-Availability Zone Spot Price feeds. Some zones simply aren’t listed! When provisioning normal instances in an availability zone that doesn’t support that instance type you get the error “Your requested instance type (m2.4xlarge) is not supported in your requested Availability Zone (us-east-1b). Please retry your request by not specifying an availability zone or choosing us-east-1a, us-east-1c, us-east-1d” At work we don’t generally launch instances that are targeted to an Availability Zone, however I have seen this before when launching t1.micro instances (which, again, were unavailable in my us-east-1b). I assumed this meant that availability zone had run out of t1.micro capacity and provisioned it elsewhere - never noticing that it was always the same zone giving this complaint. But now it’s clear as day in the spot price history tables - when an instance type isn’t supported by an Availability Zone it’s not listed in the graph (and it appears greyed out in the drop-down list at the top right of the price history page) Amazon also do this bizarre psychometric loadbalancing thing where the zone called “us-east-1a” for my account may be the zone called “us-east-1b” for yours. If you do automated provisioning and want to target specific Availability Zones you’ll probably want a way to avoid accidentally targeting Normal or Spot instances at Availability Zones where they won’t be supported. The way that my API works is the provider makes offers which the client can bid on (so I can write a piece of code to discard offers for instance types that aren’t supported by a particular availability zone) As of the time of posting, here’s the list of unsupported Instance Type / Availability Zone combinations for my primary account: Instance Type</th> Not Available In</th></tr></thead> t1.micro</td> us-east-1b us-east-1c us-east-1d</td></tr> m1.small</td> us-east-1a us-east-1d</td></tr> m2.xlarge</td> us-east-1b</td></tr> m2.2xlarge</td> us-east-1b</td></tr> m2.4xlarge</td> us-east-1b</td></tr> cc1.4xlarge</td> us-east-1b us-east-1d</td></tr> cg1.4xlarge</td> us-east-1b us-east-1c</td></tr> </tbody></table> Despite being an incredibly uncommunicative company, Amazon finally responded to this issue with a somewhat bland response on their forums: As Availability Zones grow over time, our ability to continue to expand them can become constrained. In these scenarios, we will prevent customers from launching in the constrained zone if they do not yet have existing resources in that zone. We also might remove the constrained zone entirely from the list of options for new customers. This means that occasionally, different customers will see a different number of Availability Zones in a particular Region. Both approaches aim to help customers avoid accidentally starting to build up their infrastructure in an Availability Zone where they might have less ability to expand. We recommend the purchase of Reserved Instances to assure capacity for a particular instance type in a specific Availability Zone when you need it. </blockquote> Gee, thanks. Except there are deficits in all US-East Availability Zones. The situation in 2014#</a></h2> I’ve re-run the code mentioned above, this time looking for instance types with neither spot price history data in the last month nor reserved instance offerings in a specific availability zone. The table is shown below. Obviously my particular availability zone names will not match yours, I’ll be posting code shortly to let you run these tests against your own EC2 account. I suspect the situation may no longer be as simple as it was - it may be possible to launch on-demand instances in these availability zones but there may be no spot or reserved capacity available. </th> </th></tr></thead> c1.medium</td> ap-southeast-1a</td></tr> c1.xlarge</td> ap-southeast-2b</td></tr> </td> us-west-2a</td></tr> </td> us-west-2b</td></tr> c3.2xlarge</td> sa-east-1a</td></tr> </td> sa-east-1b</td></tr> </td> us-east-1d</td></tr> </td> us-west-2a</td></tr> </td> us-west-2b</td></tr> c3.4xlarge</td> ap-southeast-1b</td></tr> </td> sa-east-1a</td></tr> </td> sa-east-1b</td></tr> </td> us-east-1d</td></tr> c3.8xlarge</td> ap-southeast-2a</td></tr> </td> sa-east-1a</td></tr> </td> sa-east-1b</td></tr> </td> us-west-2a</td></tr> c3.large</td> ap-northeast-1a</td></tr> </td> ap-southeast-1a</td></tr> </td> ap-southeast-1b</td></tr> </td> ap-southeast-2a</td></tr> </td> ap-southeast-2b</td></tr> </td> sa-east-1a</td></tr> </td> sa-east-1b</td></tr> </td> us-east-1b</td></tr> </td> us-west-2a</td></tr> c3.xlarge</td> ap-southeast-2b</td></tr> </td> eu-west-1a</td></tr> </td> eu-west-1c</td></tr> </td> sa-east-1a</td></tr> </td> sa-east-1b</td></tr> cc2.8xlarge</td> ap-northeast-1c</td></tr> </td> ap-southeast-1a</td></tr> </td> ap-southeast-1b</td></tr> </td> ap-southeast-2a</td></tr> </td> ap-southeast-2b</td></tr> </td> sa-east-1a</td></tr> </td> sa-east-1b</td></tr> </td> us-west-1a</td></tr> </td> us-west-1b</td></tr> cg1.4xlarge</td> ap-northeast-1a</td></tr> </td> ap-northeast-1c</td></tr> </td> ap-southeast-1a</td></tr> </td> ap-southeast-1b</td></tr> </td> ap-southeast-2a</td></tr> </td> ap-southeast-2b</td></tr> </td> eu-west-1a</td></tr> </td> eu-west-1b</td></tr> </td> eu-west-1c</td></tr> </td> sa-east-1a</td></tr> </td> sa-east-1b</td></tr> </td> us-east-1b</td></tr> </td> us-east-1c</td></tr> </td> us-west-1a</td></tr> </td> us-west-1b</td></tr> </td> us-west-2a</td></tr> </td> us-west-2b</td></tr> </td> us-west-2c</td></tr> cr1.8xlarge</td> ap-northeast-1a</td></tr> </td> ap-southeast-1a</td></tr> </td> ap-southeast-1b</td></tr> </td> ap-southeast-2a</td></tr> </td> ap-southeast-2b</td></tr> </td> eu-west-1a</td></tr> </td> eu-west-1b</td></tr> </td> eu-west-1c</td></tr> </td> sa-east-1a</td></tr> </td> sa-east-1b</td></tr> </td> us-west-1a</td></tr> </td> us-west-1b</td></tr> </td> us-west-2a</td></tr> </td> us-west-2b</td></tr> g2.2xlarge</td> ap-northeast-1a</td></tr> </td> ap-southeast-1a</td></tr> </td> ap-southeast-1b</td></tr> </td> ap-southeast-2a</td></tr> </td> ap-southeast-2b</td></tr> </td> eu-west-1b</td></tr> </td> sa-east-1a</td></tr> </td> sa-east-1b</td></tr> us-east-1c</td> </td></tr> </td> us-east-1d</td></tr> </td> us-west-1b</td></tr> hi1.4xlarge</td> ap-northeast-1c</td></tr> </td> ap-southeast-1a</td></tr> </td> ap-southeast-1b</td></tr> </td> ap-southeast-2a</td></tr> </td> ap-southeast-2b</td></tr> </td> sa-east-1a</td></tr> </td> sa-east-1b</td></tr> </td> us-east-1b</td></tr> </td> us-west-1a</td></tr> </td> us-west-1b</td></tr> </td> us-west-2a</td></tr> hs1.8xlarge</td> ap-northeast-1a</td></tr> </td> ap-northeast-1c</td></tr> </td> ap-southeast-1a</td></tr> </td> ap-southeast-1b</td></tr> </td> ap-southeast-2a</td></tr> </td> ap-southeast-2b</td></tr> </td> eu-west-1a</td></tr> </td> eu-west-1b</td></tr> </td> eu-west-1c</td></tr> </td> sa-east-1a</td></tr> </td> sa-east-1b</td></tr> </td> us-east-1b</td></tr> </td> us-east-1c</td></tr> </td> us-east-1d</td></tr> </td> us-west-1a</td></tr> </td> us-west-1b</td></tr> </td> us-west-2a</td></tr> </td> us-west-2b</td></tr> </td> us-west-2c</td></tr> i2.2xlarge</td> ap-northeast-1a</td></tr> </td> ap-northeast-1c</td></tr> </td> ap-southeast-1a</td></tr> </td> ap-southeast-1b</td></tr> </td> ap-southeast-2a</td></tr> </td> ap-southeast-2b</td></tr> </td> eu-west-1a</td></tr> </td> eu-west-1b</td></tr> </td> eu-west-1c</td></tr> </td> sa-east-1a</td></tr> </td> sa-east-1b</td></tr> </td> us-east-1b</td></tr> </td> us-east-1c</td></tr> </td> us-east-1d</td></tr> </td> us-west-1a</td></tr> </td> us-west-1b</td></tr> </td> us-west-2a</td></tr> </td> us-west-2b</td></tr> </td> us-west-2c</td></tr> i2.4xlarge</td> ap-northeast-1a</td></tr> </td> ap-northeast-1c</td></tr> </td> ap-southeast-1a</td></tr> </td> ap-southeast-1b</td></tr> </td> ap-southeast-2a</td></tr> </td> ap-southeast-2b</td></tr> </td> eu-west-1a</td></tr> </td> eu-west-1b</td></tr> </td> eu-west-1c</td></tr> </td> sa-east-1a</td></tr> </td> sa-east-1b</td></tr> </td> us-east-1b</td></tr> </td> us-east-1c</td></tr> </td> us-east-1d</td></tr> </td> us-west-1a</td></tr> </td> us-west-1b</td></tr> </td> us-west-2a</td></tr> </td> us-west-2b</td></tr> </td> us-west-2c</td></tr> i2.8xlarge</td> ap-northeast-1a</td></tr> </td> ap-northeast-1c</td></tr> </td> ap-southeast-1a</td></tr> </td> ap-southeast-1b</td></tr> </td> ap-southeast-2a</td></tr> </td> ap-southeast-2b</td></tr> i2.8xlarge</td> eu-west-1a</td></tr> </td> eu-west-1b</td></tr> </td> eu-west-1c</td></tr> </td> sa-east-1a</td></tr> </td> sa-east-1b</td></tr> </td> us-east-1b</td></tr> </td> us-east-1c</td></tr> </td> us-east-1d</td></tr> </td> us-west-1a</td></tr> </td> us-west-1b</td></tr> </td> us-west-2a</td></tr> </td> us-west-2b</td></tr> </td> us-west-2c</td></tr> i2.xlarge</td> ap-northeast-1a</td></tr> </td> ap-northeast-1c</td></tr> </td> ap-southeast-1a</td></tr> </td> ap-southeast-1b</td></tr> </td> ap-southeast-2a</td></tr> </td> ap-southeast-2b</td></tr> </td> eu-west-1a</td></tr> </td> eu-west-1b</td></tr> </td> eu-west-1c</td></tr> </td> sa-east-1a</td></tr> </td> sa-east-1b</td></tr> </td> us-east-1b</td></tr> </td> us-east-1c</td></tr> </td> us-east-1d</td></tr> </td> us-west-1a</td></tr> </td> us-west-1b</td></tr> </td> us-west-2a</td></tr> </td> us-west-2b</td></tr> </td> us-west-2c</td></tr> m1.large</td> us-west-1b</td></tr> </td> ap-northeast-1a</td></tr> </td> ap-southeast-2a</td></tr> </td> ap-southeast-2b</td></tr> </td> us-west-2c</td></tr> m1.small</td> ap-northeast-1a</td></tr> </td> ap-northeast-1c</td></tr> </td> ap-southeast-2a</td></tr> </td> ap-southeast-2b</td></tr> </td> sa-east-1b</td></tr> </td> us-west-2c</td></tr> m1.xlarge</td> ap-northeast-1c</td></tr> </td> ap-southeast-2a</td></tr> </td> ap-southeast-2b</td></tr> </td> sa-east-1a</td></tr> </td> us-west-1a</td></tr> </td> us-west-2a</td></tr> </td> us-west-2b</td></tr> </td> us-west-2c</td></tr> m2.2xlarge</td> eu-west-1b</td></tr> m2.4xlarge</td> eu-west-1b</td></tr> </td> sa-east-1a</td></tr> </td> us-west-2a</td></tr> </td> us-west-2b</td></tr> m2.xlarge</td> us-west-2a</td></tr> m3.2xlarge</td> ap-northeast-1c</td></tr> </td> ap-southeast-1b</td></tr> </td> ap-southeast-2a</td></tr> </td> eu-west-1a</td></tr> </td> eu-west-1b</td></tr> </td> eu-west-1c</td></tr> </td> sa-east-1a</td></tr> </td> sa-east-1b</td></tr> </td> us-east-1c</td></tr> </td> us-east-1d</td></tr> m3.large</td> ap-southeast-1a</td></tr> </td> ap-southeast-1b</td></tr> </td> us-east-1b</td></tr> </td> us-east-1c</td></tr> </td> us-east-1d</td></tr> </td> us-west-2a</td></tr> </td> us-west-2b</td></tr> </td> us-west-2c</td></tr> m3.medium</td> ap-northeast-1a</td></tr> </td> ap-southeast-2a</td></tr> </td> ap-southeast-2b</td></tr> </td> us-east-1c</td></tr> </td> us-west-2a</td></tr> </td> us-west-2b</td></tr> </td> us-west-2c</td></tr> m3.xlarge</td> ap-northeast-1a</td></tr> </td> ap-northeast-1c</td></tr> </td> ap-southeast-1a</td></tr> </td> ap-southeast-1b</td></tr> </td> ap-southeast-2a</td></tr> </td> sa-east-1a</td></tr> </td> us-east-1c</td></tr> </td> us-west-2a</td></tr> </td> us-west-2b</td></tr> t1.micro</td> ap-southeast-1b</td></tr> </td> ap-southeast-2a</td></tr> </td> ap-southeast-2b</td></tr> </td> eu-west-1a</td></tr> </td> eu-west-1b</td></tr> </td> eu-west-1c</td></tr> </td> sa-east-1a</td></tr> </td> sa-east-1b</td></tr> </td> us-west-1a</td></tr> </tbody></table> Encrypted, streaming backups with tar and gpg with fifos 2011-06-25T00:00:00+00:00 I wanted to back up a server to a USB drive earlier. I wanted the backup to be encrypted and didn’t want to waste any time writing an unencrypted tarball to file and then encrypt it. GPG is really good for encrypting files so initially I tried its symmetric mode (tar -c / | gpg –symmetric -o backup.tar.gpg) but it seems this mode can’t accept streaming input (it takes the key on standard in and I can’t find a way to have it read the key from somewhere else). I realised (too late for myself, unfortunately) that this is a perfect time to use a FIFO! I was excited for FIFOs when I first discovered them in school but have never really had a use for them myself. A FIFO is a special file which allows a single person to open it for write and a single person to open it for read & pipes the data straight from the writer to the reader. First, create a FIFO file.</li> </ol> mkfifo /mnt/backup.fifo </code></pre> Now start the writing end of the fifo (in the background). This is creating the archive.</li> </ol> tar -zcvf /mnt/backup.fifo --exclude=/proc --exclude=/mnt --exclude=/sys / & </code></pre> Now start the reading end of the fifo. This is doing the encryption.</li> </ol> gpg --symmetric /mnt/backup.fifo -o /mnt/usb/backup.tar.gz.gpg </code></pre> VyprVPN's PPTP under Linux - the command-line way 2010-04-21T00:00:00+00:00 So I just got an account with vyprvpn and it’s pretty good… I have tunnels set up on my phone, laptops and desktops – everywhere but my Linux machine. I don’t use any of that network manager nonsense so I’ve got to do it through the commandline. It was a little intimidating to start with but fairly easy in the end. These instructions are for Ubuntu 9.10. They show how to connect to the VyprVPN European servers… to use others, simply replace “eu1.vpn.giganews.com” with “us1.vpn.giganews.com” or “us2.vpn.giganews.com” as you wish. (as root) # apt-get install pptp-linux pptpd # cd /etc/ppp/peers # vim eu (or open it using your preferred text editor) </code></pre> Put the following in the file: pty "pptp eu1.vpn.giganews.com --nolaunchpppd" lock noauth nobsdcomp nodeflate name (your username) remotename eu ipparam eu require-mppe-128 usepeerdns defaultroute persist </code></pre> Next it’s simply a matter of setting up your password. To do this: # cd /etc/ppp/ # vim chap-secrets </code></pre> Put the following in the file: (your username) eu1 "(your password)" * </code></pre> Finally you can connect to VyprVPN with the “pon” and disconnect from it using the “poff” command: #pon eu (wait a little while) #ifconfig ppp0 ppp0 Link encap:Point-to-Point Protocol inet addr:x.x.x.x P-t-P:x.x.x.x Mask:255.255.255.255 UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1396 Metric:1 RX packets:6 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:3 RX bytes:84 (84.0 B) TX bytes:102 (102.0 B) </code></pre> You should also see the default route has changed to route through your VPN. I’ve not resolved how to neatly switch the default route back once the VPN is disabled so when I want to terminate the VPN link I have the following commands: # poff # ip route add default dev eth0 (where eth0 is my default network connection) </code></pre> Easier EC2 Logins 2010-01-20T00:00:00+00:00 With the addition of the new Northern California region to EC2 I ran into a problem: I had too many keys in my SSH Agent. To fix it I came up with the (somewhat imperfect) solution of removing the EC2 keys from my SSH Agent and moving the logic to the ssh config file; I’ve also written some regular expressions that pick the key automatically based on the hostname (or, if you’re using IPs, imperfectly matches the EC2 subnets). This solution also skips hostkey checking since our instances have a maximum life of a few hours so there’s no point keeping the old hostkey fingerprints around. Here’s an excerpt from my .ssh/config file # Default params Host * HashKnownHosts no StrictHostKeyChecking no ConnectTimeout 15 ForwardAgent yes RSAAuthentication yes PasswordAuthentication yes HostBasedAuthentication no ForwardX11 yes # # Amazon EC2 hosts # # EC2 Northern Virginia # 216.182.224.0/20 # 72.44.32.0/19 # 67.202.0.0/18 # 75.101.128.0/17 # 174.129.0.0/16 # 204.236.224.0/19 Host *.compute-1.amazonaws.com 174.129.* 204.236.2[23]?.* 67.202.* 75.101.[12]??.* 216.182.2[23]?.* 72.44.[3456]?.* User root CheckHostIP no IdentityFile ~/.keys/ssh/ec2/us_east_1.key StrictHostKeyChecking no UserKnownHostsFile /dev/null IdentitiesOnly yes ForwardAgent no # EC2 Northern California: # 204.236.128.0/18 Host *.us-west-1.compute.amazonaws.com 204.236.1??.* User root CheckHostIP no IdentityFile ~/.keys/ssh/ec2/us_west_1.key StrictHostKeyChecking no UserKnownHostsFile /dev/null IdentitiesOnly yes ForwardAgent no # EC2 Ireland: # 79.125.0.0/17 Host *.eu-west-1.compute.amazonaws.com 79.125.?.* 79.125.??.* 79.125.1[012]?.* User root CheckHostIP no IdentityFile ~/.keys/ssh/ec2/eu_west_1.key StrictHostKeyChecking no UserKnownHostsFile /dev/null IdentitiesOnly yes ForwardAgent no </code></pre> First iPhone app 2009-04-16T00:00:00+00:00 I’ve been looking at the iPhone SDK recently and flicking through the pretty cool Stanford iPhone Programming course, so I (sorta) did the 2nd assignment tonight. What fun. Not too shabby for 2 hours work. I’m also a bit happier with Objective C and Cocoa now - I was dreading them because of how arcane Objective C looks, but it seems decent enough, weird syntax aside. I’m really not overly enamoured with xcode, though - I wish its editor was tabbed like Eclipse. And the Groups & Files list isn’t awfully advanced. I’ve been using the “Implementation Files” group and it’s helped a lot, limiting the header-clutter C environments have in abundance. So yeah, what fun. I must look around to see if there are any other assignments I can do for the iPhone to learn the various areas - just diving straight into my Serious Dynamic Media Content Presentation demo app for work might be a tad much at this stage:) Linux: aufs and chroot 2009-03-13T00:00:00+00:00 I’ve been playing around with chroot with aufs to snapshot a machine’s state so it can be rolled back easily (not for security, though!). It’s all been going well - you need to make a few changes to /etc/mtab once you’ve mounted the new system since it has all the mounts for the base system. The other slightly annoying thing is proc - inside the chroot I’ve mounted /proc as proc so things like ps work but it’s a pity there’s no way to limit procfs to only showing processes with a particular root. Force pseudo-tty allocation in openssh 2009-03-12T00:00:00+00:00 Edit: As webknjaz has pointed out in the comments below, “RequestTTY force” in .ssh/config will do this now (in fact, it looks like it’s worked for all of the current decade at this point… I’ll leave the original blog post from 2009 intact for posterity / looking back at old things that annoyed me). ssh has this neat option -t which allocates a ptty so you can run interactive things like Screen or Vi - but there’s no .ssh/config option for it (there’s one for the server, ForcePTTYAllocation). I don’t like having to do alias ssh=“ssh -t” when it’s something that should reasonably be read from the config file (just let me configure the default value of force_tty_flag, dammit!) DHL Belfast Service Codes 2009-02-12T00:00:00+00:00 DHL Belfast - Service centre codes, redelivery information So I got a package delivered to my parents house. But they weren’t in. So I had to arrange a redelivery to a different address. Which DHL requires the Service Centre Code for. But they don’t put it on their website (the DHL Belfast Service Centre code is 7767 for anyone that’s looking - I couldn’t find it anywhere on the internet so hopefully someone who googles for the same thing I did will find this) Then when I called their number (08450 261 278) & entered the code the man told me that all I needed to do was e-mail belfast.redelivery@dhl.com with my package tracking number (the one starting JD00) and the new address and postcode. They should really put that sort of information up on their website. Some other contributions from past comments on the previous iteration of this blog: Unknown4 March 2009 at 13:10 Just so you know someone did have a similar problem to yourself and I did indeed find your post through google. It was very helpful and I was able to skip the phone part and just email them. Thank you very much Reply Unknown13 October 2009 at 12:39 Thanks for the information. This has made my life a lot easier. Reply Anonymous8 January 2010 at 16:58 I'm attempting the same thing so hopefully this will work. Thanks for your post. As you say, totally useless information on their website so if you are trying to do this remotely as I am (I'm in Italy, item being delivered in UK), getting through on the pointless phone line just wastes more time! Thanks again! Reply Anonymous9 January 2010 at 16:34 I was looking for the DHL Vauxhall service centre code, unfortunately your blog post was the closest I could get to it on the web, but a tip for any other Googlers: I rung the national DHL Express number of 08442480544 and explained I needed the Service Centre Code to speak to a real person at the centre. She didn't quite understand me but when I said "4 digit code" she gave me the 4 Digit Op number of 7291 for Vauxhall. Presumably they'll be equally helpful to anyone else looking for the service centre code! Reply Anonymous9 December 2010 at 19:44 I was looking for the DHL Vauxhall service centre code too.. Having a terrible experience with DHL this time. The status on tracking page says they left a card .. However no card was found anywhere around. Calling up the customer service line.. I was put on hold for 12 minutes to some screechy tune after which I gave up and hung up call. Have mailed the customer service now. Hoping it works out. Reply Anonymous9 March 2011 at 13:41 If anyone needs the Quedgeley, Gloucester code ive just found its: 7464 Reply Anonymous8 June 2011 at 12:22 Bristol is 7276 Reply Anonymous4 August 2011 at 09:37 Maidstone (Aylesford): 2119 Reply Anonymous29 December 2011 at 18:04 Edinburgh - 7241 Reply Unknown23 May 2013 at 10:16 DHL Lambeth- 2154 Reply Replies Anonymous7 December 2013 at 12:52 Thank you. This was what I was looking for to ask for a redelivery since DHL didn't leave the card so I needed to look for this Lambeth Service code 4 digtis myself. I love the internet for this! </code></pre> Java/OpenSSL CA generation 2008-10-14T00:00:00+00:00 I was trying to find a Java equivalent to openssl x509 -hash -in certificate.pem</code> It turns out it’s an OpenSSL invention; while there’s no pre-written solution, here’s one I wrote earlier: public static String opensslHash(X509Certificate cert) { try { return openssl_X509_NAME_hash(cert.getSubjectX500Principal()); } catch (NoSuchAlgorithmException e) { throw new Error("MD5 isn't available!", e); } } /** * Generates a hex X509_NAME hash (like openssl x509 -hash -in cert.pem) * Based on openssl's crypto/x509/x509_cmp.c line 321 */ public static String openssl_X509_NAME_hash(X500Principal p) throws NoSuchAlgorithmException { // This code replicates OpenSSL's hashing function // DER-encode the Principal, MD5 hash it, then extract the first 4 bytes and reverse their positions byte[] derEncodedSubject = p.getEncoded(); byte[] md5 = MessageDigest.getInstance("MD5").digest(derEncodedSubject); // Reduce the MD5 hash to a single unsigned long byte[] result = new byte[] { md5[3], md5[2], md5[1], md5[0] }; return toHex(result); } // encode binary to hex private static String toHex(final byte[] bin) { if (bin == null || bin.length == 0) return ""; char[] buffer = new char[bin.length * 2]; final char[] hex = "0123456789abcdef".toCharArray(); // i tracks input position, j tracks output position for (int i = 0, j = 0; i < bin.length ;i++) { final byte b = bin[i]; buffer[j++] = hex[(b >> 4) & 0x0F]; buffer[j++] = hex[b & 0x0F]; } return new String(buffer); } </code></pre>