Sun Dec 31 23:18:40 PST 2017
commuter bike, 920 miles later
Sixteen months ago I said the following:
I glumly regarded my future stretching out ahead of me, a future of cleaning my bike's chain once a week, and decided, screw it, I'm getting a new bike.
I am now cleaning the chain of that replacement bike once a week.
Let me tell you some things I wish I knew about internally geared hubs before I bought my bike.
1.) Chains stretch as they wear. You generally throw them away when they hit 0.75%. (0.075" of stretch across 10" of chain) A go/no-go gauge for chain wear, like the Park CC-3.2, is cheap and handy. (You can measure chain with a ruler or tape measure, but it's a hassle.)
Chain stretch is largely invisible on a dérailleur bike, since it has an automatic chain tensioner, (The little arm that swings down) but on a fixed-chainpath bike, like a single speed or internally geared one, you have to do it manually, by sliding the wheel backwards in the rear dropouts. This requires two wrenches, and ten minutes of time. This must be done weekly, or biweekly. If you let enough slack build up in the chain, then it'll bounce off the rear cog when you hit a bump, and you get to experience the joy of handling bike chain with your bare hands. (Unless you want to get chain grunge on your good winter gloves) Chains generally wait until the sun goes down and it starts raining to jump off a cog, as well.
2.) IGHs are filled with oil, which, per Shimano, have to be serviced after the first 1000 km, and then every 2 years or 5000 km thereafter. (My Nexus 3 started making noise at 600 miles, right on schedule) A bike shop will charge you $40-50 to do it. Want to do it yourself? The OEM oil costs an eye-popping $60 per litre. This guy has been using dollar store ATF on his, and it seems to be working fine, but this is not the maintenance-free dream I was promised!
3.) This is a topic of hot debate, but some allege that fixed-chainpath bikes eat chains more often. But I've got a Hebie Chainglider on there, which should protect the chain from dirt, and make it last longer, right?
Well, no. The factory original chain lasted just 700 miles before hitting 0.75% on the gauge. I threw the old chain away, along with the chainglider. As far as I could tell, it didn't do anything at all.
So now I've got my bike up on the stand every other week, to tension the chain, and running it through the chain cleaner now that it hangs out unprotected. My original reason to buy the bike is now completely obliterated. Oh well.
While I had the front wheel off, I checked on the drum brake, which had been sticking. Plenty of brake pad left, after a thousand miles of wear or so pic.twitter.com/gEwmv5s3Vg— Samuel Bierwagen (@ceequof) December 9, 2017
It strikes me that there is a lesson to be learned here about technological systems. As a computer guy, I'm used to slam-dunk upgrades-- this year's Intel chip will be authentically better in every single way than last year's chip, a 2 TB hard drive is better than a 1 TB drive, etc.
This is not true of all things! AC induction motors, for example, basically haven't changed in a century. Mankind quickly figured out the most efficient way to convert electrical power into motion using iron and copper, and there progress stopped. There's no AC motor today that's "better" than a motor made in 1910, no way to get more power without needing more coils, or more voltage. You can shave tenths of a percent off of 98% efficient, but there's no route to 120% efficient.
You see echoes of this all the time in a mature technological system like the bicycle. If there's a better way to do something, it was done decades ago, now all that's left to us in 2017 is choices between tradeoffs. Drum brakes are weatherproof in a way that rim brakes can't be, but heat rejection is terrible, (All the hot parts are sealed inside an aluminum can-- the price paid for being weatherproof!) braking counterforce is applied unevenly to one side of the fork with a single reaction arm, it weighs more, and in the case of my 70mm drum brake, you just plain can't brake as hard.
Disc brakes? You have to use hydraulic force multiplication in order to get the same braking force as rim brakes with a shorter moment arm. Force multiplication means length of travel is divided, so the pads have to be thinner, making them wear out even faster than rim brakes. The rotor has less mass than a wheel rim, so it heats up faster. All is compromise, trading one good for another good.
Not to say that the Windsor's totally worthless now. The dynamo hub and lights still work way better than the assortment of battery powered lights I had on the Trek. It has a kickstand, which is unforgivably omitted from all sports bikes for weight reasons. After falling off the bike thanks to ice in the parking lot at work, I swapped the 32mm Panaracer touring tire for an insanely beefy 42-622 Continental Top Contact Winter II tire on the front wheel. It mounted up just fine on my 20x622 rim, but has almost no clearance between it and the fender-- if I was going to do it again, I'd get the 37mm version.
You wouldn't expect there to be much of a different between winter tires and all-seasons, since they're both... rubber... but winter rubber compounds are actually quite a lot better on ice. Plus, the contact patch on the 42 mm tire is a hell of a lot bigger, which should help some just by itself.
So that's 2017. Check back next year, when I will have talked myself into buying yet another bike for one reason or another.
Wed Dec 21 22:37:14 PST 2016
commuter bike, 360 miles later
So I've had the Oxford for a couple months now, and a few events have occured.
As promised, the drum brake has firmed up, and now feels like a normal brake, rather than the "ice cube on teflon" feel it had when new. The dynamo lights have worked perfectly, no problems at all.
In the previous post I regarded the shock front fork on the Trek with some suspicion, even though it had given me no trouble over thousands of miles of riding, purely because it was a moving part, and moving parts break. This was because, after riding the bike for ten years, I honestly thought that the shock fork didn't do much.
This illusion was dispelled the very first time I tried hopping off a curb on the Oxford. It hurt! Shock absorbers really do absorb shock! It also doesn't help that the Oxford wears narrow 32C tires. Less rolling resistance, sure, but the point of a commuter bike isn't to go fast, it's to have minimum maintenance cost, along with maximum comfort. (It's also no small consideration that wider tires have better traction, since a commuter bike spends more time braking hard and taking sharp turns than a racing/touring bike.) If I was building another bike, I'd certainly get wider tires, and maybe think about a shock fork.
While we're on that subject, the stock Kenda tires have the puncture resistance of wet tissue paper. In the first month of owning it I got two flats in the rear wheel, one of them due to a glass splinter that was, at most, 2 millimetres long. Replacing the rear tire is especially annoying, since you have to disconnect the hub shifter, take off the chainguard, etc etc.
I switched to Panaracer Urban Max tires, and haven't had a flat since.
And then the damn rear fender bracket broke. Without it the rear fender catches on the tire every time you hit a bump, producing a surprisingly loud and alarming clattering noise. No replacement render bracket available from Windsor, no replacement rear fender assembly, either.
Well fine. I'll just make my own!
The first google hit for "lasercut stainless steel" is Lasergist. Their site is excellent, and the prices are cheap. Something they don't go out of their way to tell you is that they're based in Greece, and transit time for international letter mail is something like two weeks. If you're impatiently waiting for a part to arrive so you can fix your primary commuter vehicle, then this is something you might want to know ahead of time!
Once the damn thing got here, I had to put a 90 degree bend in it. You might ask, can you put a bend in 2mm stainless steel with just a vice, a hammer, and some locking pliars?
Now I needed to drill two holes in it so I could rivet it to the fender. Somehow I had forgotten that 302 stainless is legendarily annoying to machine. It's a little harder than mild steel, but it's worst feature is it's very bad at conducting heat. This means when you try to drill through it, the metal right next to the drill bit gets very hot, and heat-hardens. To drill stainless steel you want to use very low bit speeds, and cool the workpiece, either with drilling fluid or water. Smart guy right here did the pilot holes dry, and at the highest bit speed my drill could do. As a reward, I got to spend a full 30 minutes drilling two 5mm holes.
But now it's fixed, and it's real unlikely it'll break at the same spot.
Tue Nov 22 23:41:00 PST 2016
fun and games with 3d printers
This is my handle for an Anderson Powerpole connector. There are many like it, but this one is more expensive than most.
We recently bought a used forklift, which, like nearly all electric forklifts, has a big lead acid battery connected to the forklift by a beefy 350 amp DC connector. These big connectors are not trivial to insert and remove, and since you generally want to charge a forklift nightly, I found myself spending a lot of time wrestling with it. Anderson makes a connector handle for just this reason, but Grainger wanted $20 plus $10 shipping. Outrageous! Are we not men? Do we not have 3D printers with which to produce trivial plastic parts like this? I slapped together a simple model (source) using the parametric CAD program of choice for cheap bastards, Solvespace, and took a quick jaunt downtown to get it printed at Metrix Create Space.
Printing... pic.twitter.com/rd6U2jheBU— Samuel Bierwagen (@ceequof) August 7, 2016
This handle was conveniently easy to 3D model, but it's not actually a great candidate to 3D print, since it's so big-- about 200 cubic centimeters. Even cut in half to reduce machine time, it cost me $40 to print.
Like the last one, from a cost-cutting point of view, this project wasn't a success.
I was kinda worried that the half-height version of the handle would break off as soon as I used it, so I threw a fiberglass overwrap on it using a package of Fiberfix that was going to expire soon anyway. And it works! Qualified victory!
Sun Aug 14 23:12:19 PDT 2016
putting together a commuter bike
I've been commuting on a Trek 6000 for the last 10 years or so. The Trek is great, but it doesn't really want to be a commuter bike. There's no braze-ons for fenders, so I use clamp on fenders, which pretty reliably slip out and get eaten by the front tire. There's no kickstand, of course. It's got a shock front fork, which has never needed maintenance, but is probably waiting to spring a repair bill on me. All the lights are battery powered, which means on dark mornings, (8 months out of the year in Seattle) I have to turn on three different lights. And each of them have a different startup sequence! My helmet light is just a single press, but the tail light wants two presses, (one to turn it on, one to switch to blink mode) and the headlamp wants one long press. And they all take different batteries!
And, most importantly, I recently had to shell out $350 to get the entire drivetrain replaced, due to wear. Wear that was dramatically accelerated by the fact that I wasn't cleaning the chain very often. I glumly regarded my future stretching out ahead of me, a future of cleaning my bike's chain once a week, and decided, screw it, I'm getting a new bike.
Local bike shop Aaron's Bicycle Repair frequently puts together Seattle commuter bikes: single-speed or hub-geared bicycles with dynamo lights, full fenders, and a Hebie Chainglider covering the chain. (A Chainglider is like a chain case, except lighter, and most importantly, not as stupid looking.) The idea sounded great, so I stole it.
I bought the Oxford and had it shipped to work. (I got a paint blemish bike on sale for $299) Bikes Direct say the bike comes "90% assembled", which is a great joke. Assembly is not straightforward, or quick. There are no directions. (There is an "assembly instructions" link on the site, which doesn't have instructions for this model) You need a full set of hex keys, and some experience wrenching on bikes. It took me about an hour of messing around to get it together.
I then took it to the shop and spent another $600 on it. (From a cost-cutting point of view, the new bike project was not a roaring success)
The Nexus hub came from the factory largely unlubricated, a frequent complaint online, so I had them grease it. It was also geared for geriatrics or for people interested in scaling cliff faces, so I had them swap in a 19 tooth cog. (That gives me a 44/19 ratio on the middle gear setting, just fine for my mostly level commute) The swept bars it had from the factory were just insanely bad, so I flipped them. Aaron gave me a hard time for that, so I had him put on flat bars.
Then, of course, the dynamo, front light, and rear light, which together added up to a solid $300-- about as much as the bike itself. The dynamo is a Sturmey Archer X-FDD dynamo/drum brake. As advertised, fresh drum brake pads are not supremely grippy. Supposedly it develops some decent brake power after you put a few hundred miles on it. (I do 30 miles a week)
You don't really want to brake all that hard with it, though. Unlike rim brakes, it transmits all the braking torque through a reaction arm to just one side of your forks, which is how this lunatic pretzeled their fork by doing stoppies. I probably won't be trying that with my light, value-engineered front fork made from premium Chinese steel.
Oddly enough, the manual says nothing about replacing the brake pads. Rim brake pads seems to need to be changed every few hundred miles, but assuming the drum brake uses the same pads as in automative drum brakes, they would have a service interval of 50,000 miles, at which point Sturmey Archer seems to advise throwing away the hub and buying a new one.
The lights are the Busch & Müller Lumotec IQ Cyo Premium Senso Plus and 4D-Lite Plus. In the grand tradition of German electronics, you get to cut the wires to length and crimp the connectors on yourself. Despite being a professional electrician for a number of years, and possessing the correct crimping tools, I managed to screw up the crimp on both spade connectors, and just soldered them on. (It also comes with two pieces of heatshrink tubing, making the confident assumption that the customer already owns a heat gun.) The headlight has a 3-position switch somewhat mysteriously labeled "T, S, O". What does the T stand for? Well, tagfahrlicht, obviously. ("Daytime running light") Never thought I'd miss inscrutible Euro internationalized symbol glyphs.
But! Once you start moving the bike, the damn things just turn on, and when the bike stops moving they turn off and now that I've installed them I'm not going to have to touch them again for years, which is mostly worth their incredible expense.
1: As you might guess from the hyper-Western name, Windsor is a Chinese manufacturer.^
2: Fun fact, that was my very first blog post, which means this blog is more than ten years old now.^
Thu Jun 9 03:21:45 EDT 2016
using ROS with the Neato XV-11 in the year 2016
I've been meaning to do some ROS SLAM stuff for a while. There are a number of ways to do this, some more expensive than others, but one fairly straightforward option is just to use a Neato robot vacuum cleaner, which has a LIDAR sensor and a USB port with a fairly open debugging interface, which lets you get the raw feed off the sensor, and run the motors.
To run the robot, you need a mobile computer, (You can plug a mobile robot into a PC, but it tends to end poorly, for a number of pretty obvious reasons) so I borrowed my friend Tish's laptop. It had Windows installed on it, but I had a spare 2.5" SSD with Ubuntu 16.04 installed, so I tried just plugging it in, why not. To my immense surprise, it booted right up and worked fine. Even the screen brightness keys and wifi worked with zero configuration. I've been using Linux since 2003, and the fact that nowadays you can just blindly swap boot drives between computers and have them Just Work is fairly darn incredible. I remember having to hand-edit X11 config files just to use two monitors!
That was my last moment of pleasured surprise, because now I had to deal with ROS, which is a lovely example of the CADT model of software development. Most of the documentation I tried to follow was fragmentary or broken, mostly because ROS has gone through at least three different methods of "building a package" in the last 6 years. Adding to the problem was also that I refused to believe what I was reading. All the tutorials showed cloning from git repos and compiling from source. No, man, they're packages! Just show me how to download the precompiled packages! There's a package manager, right? With versioning, and dependency resolution? Right? Well, no. The canonical method of ROS development is downloading source files and compiling them.
Here's what I eventually cobbled together. First, install ROS following the directions on this page. (At the time of writing, the current stable release of ROS is Indigo) Even if you install ros-indigo-desktop-full it'll skip some stuff you'll need, so also do
sudo apt-get install ros-indigo-map-server ros-indigo-amcl ros-indigo-move-base ros-indigo-teleop-twist-keyboard
Now, create a new ROS project, set it as the working directory, move into the
src/ directory, and clone some git repos.
mkdir neato cd neato rosws init source setup.bash cd src rosws set neato_driver --git https://github.com/mikeferguson/neato_robot rosws update
(You may need to run
rosws update more than once.)
In that example I show cloning from the original repository, which is unmaintained. I had to fork it to get it to work with my robot, (Mike's code assumes the XV-11 has more motor properties than mine does, oddly, and my LIDAR sensor seems to need some time to spin up before returning data) but I didn't put my repo's URL in there since I, uh, haven't actually tested installing
neato_robot from it.
cd .. catkin_make source devel/setup.bash
You would not believe how long it took me to figure out that you have to use the
setup.bash in the
devel/ folder, not the one in the project root. Anyway, if everything went right, you should be able to start up the node that actually talks to the robot. It expects to see it on
All four of the following commands start four separate programs, and each need their own terminals. You can bg them if you want, but they print critical debugging information to the terminal.
roslaunch neato_node bringup.launch roslaunch neato_2dnav move_base.launch rosrun rviz rosrun teleop_twist_keyboard teleop_twist_keyboard.py
You can now drive the robot around with the keyboard in the
teleop_twist terminal and watch what it thinks the world looks like in
By default the
/map topic will show you the map Mike generated of his office in 2014, which probably won't be helpful for you. You can generate your own map using the instructions on this page, which, for a change, worked perfectly for me on the very first try. Here what the XV-11 decided the floorplan of my house looks like:
: If you don't install these packages, you'll get a cryptic error message along the lines of:
ERROR: cannot launch node of type [tf/static_transform_publisher]: can't locate node [static_transform_publisher] in package [tf] ERROR: cannot launch node of type [amcl/amcl]: can't locate node [amcl] in package [amcl]
Googling this error message will suggest that you need to delete your package and reinstall, which is both harrowing, (When I ran into this problem, I had already spent consecutive hours fighting with environment variable problems) and wrong.
: Exasperatingly, Tish's laptop decided to enumerate it as
/dev/ttyACM0, so I had to go in and edit all the code that tries to talk to the wrong port. (Super exasperatingly, if you just change the line in
neato_driver.py but not the launch file, it will crash out on port initialization, since the launch file passes in the port name as a parameter! That took forever to find. Single point of truth you bastards!)
Anyway, the robot control port is just a regular old tty, you can do
screen /dev/ttyUSB0 and directly control the robot. There's a bunch of interesting stuff in there, (You can control everything. LCD backlight, individual LEDS, etc) though it seems to invisibly flip between certain interface modes, rather a lot like Cisco IOS.
Sun Jun 5 22:58:11 EDT 2016
sawbot for VEX Worlds 2016
I built another tradeshow demo! This time, it's a robot that cuts shafts to length.
Sun Mar 13 23:15:11 EDT 2016
connector savers and USB devices
My bike helmet light, the Light & Motion Vis 360, like basically anything with a battery in it nowadays, charges over USB. Unfortunately, USB sockets aren't invulnerable, and after a couple years of bike commuting, mine loosened up to the point that it wouldn't charge.
Well, I'm a handy fellow, so I opened up the light, got out my fine tip temperature-controlled soldering iron, and promptly lifted a couple tracks. Whoops.
Light & Motion charged me $48.75 for a new logic board and battery, which was reasonable enough, but I really don't want to spend $49 every two years as long as I own this light. Isn't There A Better Way?
There sure is! Connector savers!
A connector saver is an adapter that you put on an expensive piece of equipment, and make all future connections at this interface. This protects your equipment, because when some idiot trashes the connector saver, you replace a relatively inexpensive adapter rather than send your equipment out for repair.
Connector savers are much more common with test equipment and in the aerospace world, (Indeed, the first place I heard about them was in NASA-STD 8739.4) where a single big multi-contact connector can cost thousands of dollars, and be rated for surprisingly few cycles. But hey, nothing stopping me from putting a connector saver on my piddly little bike light.
I wanted a connector saver with a flexible bit in the middle, basically a very short extension cable. This was hard to find. Newegg's cable category has really lousy search, and while Amazon did have what I was looking for, shipping would be annoyingly expensive, and I didn't really want to give the Beast from Lake Union any more business, given that my company directly competes with them in a couple markets.
I finally found USBFirewire's RR-MCB-EXT-05G5, which worked perfectly. Now we'll see how long the connector saver will last!
Wed Feb 24 22:28:57 EST 2016
As you have no doubt not noticed at all, this blog has in fact not updated in two years. I am not dead!
Two years ago my hosting provider went bankrupt suddenly, erasing all user data in the process. I had backups, of course, but inconsiderately neglected to back up my home directory on the server, which happened to house my Nanoblogger installation. (A shell script-based static site generator (basically Jekyll for cavemen) so old that it has not been updated since 2011, for whom development was suspended in 2013, and is currently hosted on Sourceforge, a software host so wildly reviled that the domain is now blocked by uBlock Origin.)
The prospect of rebuilding the data files was so daunting that I put it off until "next month" for 24 months in a row, until I finally got around to it, today. So, hi.
I don't want to come off as a solar power hater with these posts. Solar power is our future, it is inevitable, and it is a thing that must be done, because oil sure as heck isn't going to last much longer.
But solar panel roads are never going to happen, not for centuries, at least in the United States. Solar driveways probably will happen, but as a toy of the rich.
Let me tell you why. There is one single figure of merit in a PV system, and that is cost per peak-kilowatt installed. For giant, utility-scale installations, it's around $1000 per KWp, because you can get better price breaks when you buy ten million bucks of solar panels at a time, and permitting costs, which remain roughly constant, make up a smaller proportion of the total price. For large residential installations, $2200 per KWp is a better bet.
The Wikipedia article on PV systems has a very useful table:
Installed cost per KWp is on the left, insolation is on the top. "Insolation" is how much light you get per square metre of land, and for solar, it's most useful when expressed as kilowatt-hours per day. (To convert KWh/day to KWh/year, you just multiply by 365.25, of course) Las Vegas gets around 1936KWh/y, so for a $2200KWp installation, your electricity costs a little more than 11 cents per KWh. Seattle gets 1289KWh/y, so you'd pay ~14 cents/KWh.
And here's the big problem, the central issue around which everything in the solar industry revolves: Conventional electricity in Washington state costs about 8.2 cents/KWh. Solar is a solid 80% more expensive. The only places where PV is at grid parity is in deserts.
PV is expensive. The solar industry is on a mad, headlong dash to cut costs as much as possible. They want grid parity, they are seeking it, seeking it, all of their thoughts are bent on it
I've also been writing for my employer:
For VEX Robotics World Championship 2015, like last year, Robot Mesh needed some kind of booth demo.
The requirements are open-ended, but challenging. It's got to be made of (mostly) VEX parts, be novel, and be interesting.
We wanted to do something with pneumatics, and something with the Pixy camera, since computer vision is both interesting and interestingly difficult. How about... a robot that plays pong against itself?
Normally, to check the state of six touch sensors in a single
ifexpression, you'd have to do something like:if FR_touch.is_touch() or MR_touch.is_touch() or BR_touch.is_touch() or FL_touch.is_touch() or ML_touch.is_touch() or BL_touch.is_touch(): doSomething()
This has a number of disadvantages:
- We end up having to type
is_touch()over and over, which is annoying. (Don't Repeat Yourself, he repeated.)
- Very long lines are hard to parse, and therefore impair readability. (Code is hard enough to read already, we don't want to do anything to make it worse.)
- If the end of a line hangs off the edge of the screen, then bugs can hide where you won't see them at a casual glance. If we had forgotten the () on the very last
is_touchthen the if expression would always evaluate as
True, and we would be very confused when the robot acted like the TouchLEDs were constantly being pressed.
- It's ugly and I hate it.
I've also been tweeting, to the immense regret of everyone, everywhere. I am of course hilarious and beloved:
thumbs down— !!c1QfXUgcGY0 (@ceequof) December 16, 2014
Pandora: How about a live version?
Pandora: Acoustic cover?
Pandora: Weird Al parody?
Follow me on all these platforms, or don't, it's up to you!
Fri Mar 21 20:58:25 PDT 2014
replacing the battery in a cheap solar charger
So I've got a very cheap solar charger, whose internal li-poly battery has been slowly dying, and is now mostly useless. Popping it open, I discover it's a 800mAh unit.
So let's replace this sucker.
While I've got it open, though, I want to see how much current the little solar panel can actually source. All you need to do this is a single multimeter, though it's more convenient to use two, one bridging the positive and negative rails, to measure voltage, and one inserted in the current flow, to measure amperage. (If you get it backwards, the ammeter will look like a dead short to the voltage source, and either blow a fuse or melt your test leads.)
(Diagram made with Circuitlab, which can apparently do all sorts of fancy simulation stuff, none of which I actually used.)
This looks very neat and clean on paper, and becomes a horrifying tangle of wires when implemented with multimeter leads and alligator clips. (Mostly hidden off-frame)
I set everything up, ready to finally, at last, read off the current...
And it's way off the low end of the scale. Switching to the digital meter, I discover that even in direct sunlight, the charging circuit can only manage a thoroughly unexciting 7mA. Assuming perfect charging efficiency, (which ain't gonna happen) it'd take 121 hours of direct, face-on sunlight to recharge a flat battery.
I wrote a gloomy analysis of a fictional solar charger on the other blog, and even with my worst case assumptions, this real-world solar charger is more than 26 times worse. Time, and sunlight, has not been kind to its panel.
So, now that I know how much it sucks, it probably wasn't worth spending 7 bucks to put a new battery in it, but oh well, let's close it up again.
Today's stars are my old Weller P2K butane soldering iron, here used just for the heat-shrink, a crappy Tenma unregulated soldering iron, a Panavise PCB holder that they apparently don't make anymore, and some Radio Shack "helping hands". A depressing amount of the work that goes into doing electronics stuff is fixturing-- getting things to stay in the right position while you do things to them.
I took six photos here, all of them in varying degrees of out-of-focus.
Someone more professional than I would probably have done a lapped splice here, in which case it's very important to slip on the heatstrink before you solder the joint, but I did a twist splice because I'm lazy and the joint wasn't going to take any mechanical stress anyway.
And we're done! Now I have a charger which should last several more years, at which point I will throw it right into the trash.
Thu Mar 13 22:41:11 PDT 2014
introducing tinypass.py v1.0
Before I can talk about what I did right, I have to talk about what I did wrong.
I host some files for a friend. They're great big zip files full of art, which he sells for money, so he'd like to put a password on them.
"Easy enough, this is exactly what HTTP Basic authentication is for."
But he'd like to be able to set passwords on files without having to ask me to manually fiddle with nginx config files.
"Well, I'll just whip up a quick forms-based thing for editing nginx config files. How hard could it be?"
(A chill wafts over your skin. Dread shivers up your spine.)
It took POSTed form data (
password) from a static HTML page, created a hashed password file from that password, appended a
location /filename block to a config file, then called
And as soon as it was actually used by someone who didn't write it, it blew up.
Oh v0.1, there were so many things wrong with you, how could I possibly count them all?
1.) It had to have permissions to edit nginx config files and reload the server. So I just ran it as root, which meant that I was running a python web server as root, which is an absolute security disaster. I'm listing this first, even though nothing bad actually happened (as far as I can tell) because it was just a complete unforced error. This was the first warning sign that I was doing something dumb, and I completely ignored it.
filename was just a text box, not a dropdown menu or picker, so it was trivially easy to typo a filename, and "set a password" on something that didn't exist. v0.1 had no error checking of any kind, so it couldn't refuse to do that.
3.) HTTP Basic is user-granular, but for this particular use we're doing file-granular permissions. HTTP Basic doesn't handle this very gracefully: if you're already logged in, and try to access a file you don't have permissions for, (say, if you bought several different items, or if you're me, and are trying to troubleshoot your broken fucking login system) then it just hits you with a
405 Authorization Needed error, no login window. Since HTTP Basic doesn't have a
log out button, (hint: where would you put it?) you have to restart the browser, or just wait around until the browser expires your login credentials, which is, as you'd guess, implementation-specific.
4.) Remember when I said that it just appended lines to a config file and reloaded the server? v0.1 had no conception of records-- it was a basic CRUD app in theory, but in practice it only created records, it couldn't read, update or delete them. It would quite happily, create two
location blocks for the same file.
Nginx will refuse to load a config file that has contradictory options. If you
restart it with a bad config file, then it won't start back up, and your web server goes down until you fix it.
A minor decision I made early on really saved my ass here. I heard that using
reload instead of
restart let nginx wait for clients to finish transferring data, so I used it in the script. Luckily,
reload won't take down your sever with a bad config file, it'll just refuse to load it.
So instead of blowing up the server, v0.1 just silently stopped applying changes until the config file was manually fixed.
Now, all these problems have solutions. You could conceivably train the end user to carefully work around the problems, on the theory that your software is great but the user is dumb, but when your tool collapses in a great heap of splinters at the slightest touch, then it's not the fault of the user, it's your fault.
You could also fix each of these bugs, add tests, etc, but the basic architecture of the program is just bad. It's fucked.
Let's try again.
tinypass.py v1.0 is designed to replace HTTP Basic with something about as secure, but a little friendlier to use, as well as letting the end user set and change passwords without fatally confusing nginx.
Rather than sending login credentials in the clear over HTTP headers, like HTTP Basic, tinypass.py sends credentials in the clear over cookies, which is much more secure. Best practices here would be hashed passwords and session ID cookies, which would require more work. I didn't feel like doing that work, because...
I don't really make a living selling games. I sell an ethical life.
How could I make a living selling games? Anyone who wants to pay me for my games doesn't have to. It's not like buying a chair, where they'll chase you down and taser you if you grab it and run out of the store. Nobody who wants my game on Windows or Mac has to pay for it to get it. Frankly, most of them don't.
So why do people pay for it? Because they understand a fundamental fact: For these games to exist, someone has to pay. If everyone just takes it, I'll have to get a real job and the supply will shut off. I don't want to get into one of the eternal tedious arguments about "software piracy". I will instead focus on one single, incontrovertible fact: I have a family to feed. If nobody pays for my games, I can't make them.
So what does someone get when they pay for my game? They get the knowledge that they are Part of the Solution and not Part of the Problem. They know that, in this case, they are one of the Good Guys. It is well-earned self-satisfaction, and it is valuable. To know they are doing the right thing, some people will happily pay 20 bucks. This is how I stay in business.
You can't stop piracy. DRM never works. You can't let somebody look at something without also letting them copy it. Cannot be done, impossible, full stop.
So tinypass.py is a speedbump, not an impassible wall. Since there are no confidential login credentials at risk, I don't go to any great lengths to keep them secure.
So hey! That's it. Check it out, I guess, just as long as you don't look at the commit history.
Mon Oct 28 22:48:43 PDT 2013
programmatically advertising mobile bandwidth cost: a proposal
You know what would be cool? If your phone knew how much bandwidth from each carrier cost, and could switch between them on the fly, depending on which one was cheapest, like a multi-SIM phone that didn't suck.
You know what would be cool? If your phone could roam between a cell tower and an arbitrary wifi AP, like a parallel-universe version of UMA that also doesn't suck.
You know what would be cool? If wifi APs could programmatically advertise bandwidth cost too, so anyone could compete with AT&T just by nailing a linksys router to a wall.
You know what would be cool? If your DSL modem could advertise bandwidth cost too, just like your wifi AP and your cell tower. You'd have to pay common carrier costs for the last mile of cable to your house, no way around that, but as soon as your bits make it to the first point of presence, you'd have you choice of long-distance IP transit providers, just like the last time we broke up a telecom monopoly.
All these ideas seem very simple and obvious. Have they been proposed before?
1: Bandwidth is a utility service, like electricity, or water. Any profit a utility monopoly makes is extracted from the productive economy, a tax on real industries. AT&T made $3.8 billion last quarter.
2: The implementation details of this one are going to be tricky. With a major carrier, you can just tally up all the kilobytes used and bill the user at the end of the month, but with wifi, it's entirely possible a person will walk into a starbucks, watch a video on youtube, then walk out, never to return. How is that billing system going to work? Are you going to have to manually provide billing information before connecting? That would be terrible.
3: Well, "transceiver".
4: Honestly, I anticipate a lot less benefit from this one. Transit is already a very competitive commodity market, with razor-thin margins. The biggest problem with consumer internet has always been that last mile, and the associate incumbent telcos, who have no useful competition, and therefore lots of monopoly profit.
You'd almost have to mandate embedded cell phone radios in terrestrial internet transceivers in order to guarantee last-mile diversity... wait, shit, that's a great idea!
Sat Aug 31 02:22:57 PDT 2013
Review: "Apocalypse Codex" by Charles Stross (2012)
Actually, I want to expand on that.
Apoc. Cod. is a book that either needed another rewrite or a more aggressive editor, which is odd, for an author's 20th book.
Its sins are numerous. Stross has picked up an unfortunate habit of repeating himself-- SCORPION STARE is explained several times, and at many points where characters explain what's going on to other characters, instead of eliding the details, he'll actually spend a couple pages on the conversation. These recaps would be useful in a longer, more complex novel, but Apoc. Cod.'s tight structure and fast pacing work against it, (And its 336 page length) making the frequent reiterations of the plot more annoying than useful. (Plus, I powered through the whole thing in 5 hours during a car ride, which helps to keep events fresh in your mind.)
The book does have some fairly good moments, to the point where the usual in-car soundtrack of classic rock FM radio became grating, and I wished for something gloomy, sepulchral. The despond is punctuated by some unfortunate attempts at soapboxing. One of the characters, much like the author, is an atheist, and by God he's gonna let you know about it.
This is ill-advised. Evangelical Christianity is best criticized by repeating their ludicrous bullshit with a straight face. (Did you know that Pat Robertson has a long list of divine revelations?) The Quiverfull ideology mentioned in the book is a real thing that actually exists. With all this rich material, having a in-universe character actually say "These people are super dumb" is redundant, bordering on jejune. We get it, dude. You don't need to have zombie missionaries smashing in the doors to get the point across.
Another problem is the hero, a computational demonologist and former IT schlub.
There's an authorial voice that's peculiar to nerds in general, and science fiction fans in particular. It shines through clearly in print in books like Fallen Angels (which contains paid-for cameos by big name fans) and the execrable Troper Tales of tvtropes, which were so bad that they've been quarantined on another site. It's distinctive as it is annoying.
When he's not actually holding a gun, Bob talks like a slashdot commenter. This is,
A.) Top notch characterization, and spot on accurate.
B.) Super, super irritating.
I found myself skimming early conversations just to avoid reading what the main character actually said, which is unhelpful for following the plot. This is another example of Stross' mania for absolute factual accuracy, which can occasionally get in the way of the story. (He emphasizes several times that the life of a spy is boring, and not at all like a Bond flick, which is troublesome when you're pretty much writing a bond flick.)
The book's okay. I guess.
1: While writing this, I discovered that Catacombs is actually a one man band, run by a fellow named Xathagorra Mlandroth. Xathagorra Mlandroth! Gosh I love funeral metal.
Wed May 29 00:28:23 PDT 2013
keeping my ass warm
So I was reading the owner's manual for the 2013 Nissan Leaf, (as you do) because I was wondering if it said anything about overuse of rapid charging.
Every time I bike to the grocery store, I see a Leaf connected to the 440V DC fast charger, (Google Kirkland is five minutes away.) which can put an 80% charge on a car in 30 minutes.
But batteries suck. Charging a battery in 30 minutes in precisely analogous to discharging it in 30 minutes-- something big lithium-ion batteries don't like. Sure enough, page 43:
Batteries don't like to get too hot or too cold, to be discharged too hard, too deeply, or left discharged for too long. You might then conclude that the best choice is to just leave the car in your driveway-- where the battery will then just quietly decay on its own. Batteries suck.
Also, page 187:
This makes a lot of sense-- heating pads are orders of magnitude more efficient at warming humans than using air as the transfer fluid. In an internal-combustion vehicle, you get hot air for free, since a heat engine has to dump a lot of heat to the outside environment in order to extract useful work from it, you might as well pass that waste heat stream through the cabin, like a not-terribly-efficient cogeneration setup.
But in an electric vehicle, every watt-hour of power comes from the battery pack, and each watt-hour is dear indeed. Heat is no longer free.
Bold prediction: Heated seats will be standard equipment in all electric cars.
Thu May 23 02:50:48 PDT 2013
Bad Transcript: Star Trek Into Darkness (2013)
I wrote a Bad Transcript for the new Star Trek movie. It's pretty good, you should read it.
The transcript, that is. The movie isn't good.
Fri Jan 4 21:03:31 EST 2013
767 days ago, I commented on a HN submission about a random number generator:
3.) Providing random numbers as an advertisement for your fine line of hardware random number generators. Here it doesn't matter how much money you make [providing the numbers], you just want people to buy the hardware that made them. Oddly enough, none of the random number services (and there are quite a few) do this, for some inexplicable reason. There's not even an argument-from-proprietary technology, since HRNGs are supposed to generate perfectly random noise, and there's no way an attacker could stage a replay attack.
I left it there, because I was lazy. But last month, notorious badass Maciej Cegłowski created The Pinboard Co-Prosperity Cloud.
What is it?
The Pinboard Co-Prosperity Cloud is a startup self-incubator. Six successful applicants will receive a modest amount of funding and as much publicity as I can provide for their sustainable and useful business idea.
Is this a joke?
It is not a joke.
What are the requirements?
You must have a good idea that you are capable of building, a willingness to build it, and a plan for making it mildly profitable.
How much funding will I get?
Each successful applicant will receive $37. This will cover the cost of six months of hosting at prgmr.com and a productivity-enhancing hot beverage.
So I entered. Ha ha why not?
The more I thought about it though, the more I realized that I wasn't getting the joke. The idea was trivially simple. I already had a web server. I didn't need all that mad cash. I could just... build it.
So I did. It's right here. (EDIT 2013/3/22: I let the domain name lapse, and moved the content to bbot.org)
Web programming in the year 2012 has the smooth, well polished feel of something that has had the sharp edges worn off by the passage of thousands of other people. Getting nginx to talk to the WSGI server was a snap. Installing bottle.py was easy. JQuery was no problem.
Any time I had a problem, googling the error message would return a helpful, relevant page, explaining how my "build it as fast as possible, while learning as little as possible" design methodology had screwed me over again.
At the time, of course, it seemed a vast edifice of impossible complexity, but in retrospect it was painless. "It's easy to do if you know how to do it", maybe.
The only difficulty I faced was the hardware random number generator. The numbers had to come from it, since that was the whole point of the site; but my server was a virtual machine on the east coast, and my HRNG was sitting on my desk.
The "money" solution would be to buy a rackmount server, plug the widget into it, then slot it into a colo, but I didn't have money, and instead I had to be creative.
I couldn't just run the web server locally, since my ISP blocks port 80. Enter the ugly hack: I plugged the entropykey into a spare laptop, ran the application server on that, then ran a SSH tunnel to my web server, which communicates with the front end via JSON. It works, at the cost of an extra 150ms of latency per roll.
There's room to improve, of course. You could probably list off a dozen features dtwenty.org needs without pausing to draw breath, (starting with "make it less ugly") but, the ideal of the minimum viable product shines bright.
The second biggest problem after integrating the HRNG was the ad copy that makes up most of the page. It was originally twice as long-- ruthless editing has reduced to it merely "too long" from "far, far too long." This too could use improvement.
But! It's done and it works! Programming is fun.
Sun Dec 9 23:49:30 EST 2012
"How would I get started" and the problem of truth
I've been meaning to write about Hacker News again, but have held back, since it's a pretty boring subject outside of HN's rather shallow pool of users. But recent events have forced my hand.
Last night on Hacker News, someone asked a simple question with a complicated answer: “I want to build a cable company. How would I get started?”
I’m really disappointed in the universally pessimistic and generally unhelpful answers this question received. Some people pitched some interesting ideas and helpful analysis, but most of the replies reinforced the notion that Hacker News readers are predominantly male know-it-alls and on the average, a bunch of snarky dicks.
Lots of emotional content here, but not much meaning. The attitude behind these two paragraphs becomes clearer if we look at some other quotes:
In startups, the big winners are big to a degree that violates our expectations about variation. I don't know whether these expectations are innate or learned, but whatever the cause, we are just not prepared for the 1000x variation in outcomes that one finds in startup investing.
That yields all sorts of strange consequences. For example, in purely financial terms, there is probably at most one company in each YC batch that will have a significant effect on our returns, and the rest are just a cost of doing business.  I haven't really assimilated that fact, partly because it's so counterintuitive, and partly because we're not doing this just for financial reasons; YC would be a pretty lonely place if we only had one company per batch. And yet it's true.
To succeed in a domain that violates your intuitions, you need to be able to turn them off the way a pilot does when flying through clouds.  You need to do what you know intellectually to be right, even though it feels wrong.
It's a constant battle for us. It's hard to make ourselves take enough risks. When you interview a startup and think "they seem likely to succeed," it's hard not to fund them. And yet, financially at least, there is only one kind of success: they're either going to be one of the really big winners or not, and if not it doesn't matter whether you fund them, because even if they succeed the effect on your returns will be insignificant. In the same day of interviews you might meet some smart 19 year olds who aren't even sure what they want to work on. Their chances of succeeding seem small. But again, it's not their chances of succeeding that matter but their chances of succeeding really big. The probability that any group will succeed really big is microscopically small, but the probability that those 19 year olds will might be higher than that of the other, safer group.
The probability that a startup will make it big is not simply a constant fraction of the probability that they will succeed at all. If it were, you could fund everyone who seemed likely to succeed at all, and you'd get that fraction of big hits. Unfortunately picking winners is harder than that. You have to ignore the elephant in front of you, the likelihood they'll succeed, and focus instead on the separate and almost invisibly intangible question of whether they'll succeed really big.
raffi 114 days ago | link
Most companies fail. It's a safe bet to predict failure. It's pretty lame to celebrate that failure from the sidelines.
Vision is not "how is this guaranteed to fail?" but how could it possibly succeed despite the odds?
A core tenet of hacker ethics, the zeroeth law perhaps, is being right, having correct perceptions regarding the universe. A map that matches the territory.
Under this ethical system, the above statement makes less than no sense. The most likely outcome is failure... but you shouldn't predict failure?
However, the way startup financing is currently organized, a VC fund can shrug off a dozen miserable failures to chase the one Google or Intel.
The purpose of Hacker News is to advertise Y Combinator startups, such as 9gag. The purpose is not to act as a prediction market. In fact, since one of the major routes of of startup profitability is being purchased by another company, accurate predictions of value are contrary to Y Combinator's interests. Y Combinator wants valuations as high as possible.
Someone starting a new cable company in 2012 is very likely to fail. This is the correct prediction: it is the outcome with the highest probability.
But a new cable company which somehow isn't immediately crushed, would have an enormous customer base, and could potentially make billions and billions of dollars.
To someone steeped in the Bay Area Startup lottery culture, this isn't an insanely stupid idea at all. It's almost a safe bet. With the force of millions of dollars behind you, being right is irrelevant.
Mon Oct 15 18:24:59 EDT 2012
why does nanoblogger generate broken links
(Attention conservation notice: I found an obscure bug in my blog publishing software. You are unlikely to care about it.)
Why the hell does my site have so many broken links?
I'll spare you the grimy details of the hour of troubleshooting, and jump right to the
punchline. Nanoblogger 3.4.2 has a bug which generates bad relative links
when you do
./nb update all
Nanoblogger is no longer updated, so this isn't a problem that can be
solved by upgrading. I didn't want to dive into the parsing engine, so I had
to find a workaround, which turned out to be pretty simple: just update it a
year at a time.
./nb update YYYY works
./nb update 2012) I've only got six years of
archives, so all I had to do was run it six times.
I'm posting this incredibly boring post in the hopes it'll save one of the six other users of nanoblogger some confusion in the future.
Sat Oct 6 15:26:52 EDT 2012
wherein I write a guest post
A modern nation, like a corporation, is a machine made of humans. Its genetic code is ideas, encoded as words on paper.
Such immaterial entities are exquisitely sensitive to language, and how it's used. Sticks and stones may break your bones, but words can kill a nation.
So the Senkaku/Diaoyu Islands are purposely branded so that there's no way to refer to them without implictly taking a stand. Like how the abortion debate is framed as "pro-choice/pro-life", except that both sides in this argument have nuclear weapons.
You should check it out!
Sun Sep 30 07:13:13 EDT 2012
I've pasted a lot of IRC logs into a lot of HTML documents, which is always a pain, since angle brackets are obviously a special character in HTML, which means I have to do a search and replace with the equivalent entity codes. I usually did this manually, using whatever graphical text editor was handy.
But that's Not The Hacker Way. I'm editing a text file produced by one program, so another program will accept it. String processing isn't a job fit for a human. This is something that should be done by a third program.
#!/bin/sh # # escape.sh - Escapes angle brackets in text files # # Turns angle brackets into < and > HTML entities. # With --irc, replaces the first 8 columns (the timestamp) with an # opening angle bracket, using an ugly hack. # # This is free and unencumbered software released into the public domain. if [[ $* == *--irc* ]] then sed -i 's/>/\>/g' $2 sed -i 's/^......../\</g' $2 else sed -i 's/</\</g' $1 sed -i 's/>/\>/g' $1 fi
Then I stuck it in my $PATH with
sudo cp escape.sh
/usr/local/bin/escape This way you can run it from any directory
just by doing
(It's not actually very Unixy-- it doesn't play well with pipes, and wildcard expansion in a directory will blow it up.)
Sat Aug 25 19:00:25 EDT 2012
Joining the pool is pretty easy: You create an account, give them your server's IP address, wait for the monitoring server to decide you're stable enough (~8 hours) and boom, you're in.
(The interface is a bit awkward: you paste the address in there, you don't click the "Add a server" link, which apparently doesn't do anything.)
I found four upstream servers by pinging 0.us.pool.ntp.org repeatedly, and choosing the one that were closest to me. Since bbot.org is in a datacenter right on the internet backbone, close can be very close:
# ntpq -np remote refid st t when poll reach delay offset jitter ============================================================================== -184.108.40.206 220.127.116.11 2 u 273 1024 377 2.320 3.100 1.201 +18.104.22.168 22.214.171.124 3 u 825 1024 377 3.713 0.239 0.371 -126.96.36.199 188.8.131.52 2 u 237 1024 377 3.174 -1.069 0.398 +184.108.40.206 220.127.116.11 2 u 383 1024 377 6.828 0.382 0.141 *18.104.22.168 .WWV. 1 u 426 1024 377 11.537 0.225 0.310
I had hoped that <10ms ping times would result in magically low offset numbers, measured in the tens of microseconds, but apparently jitter becomes a bigger problem when you get that low.
My reference stratum 1 server is wwv.tns.its.psu.edu, an open-access tier 1 server that John Balogh runs. Thanks John!