Band Saw Rehab

I recently decided to act on my long-festering bandsaw itch. I’ve always thought it would be nice to have one, although I’ve gotten along OK without it, cutting curves with a jigsaw or scroll saw, or coping saw. But now I have more space, and a bunch of big dead trees, and would like to be able to do some resawing. So I poked around online and found a reasonably good-looking used 14″ saw. I probably overpaid for it (“or best offer” means you’re supposed to offer less, clearly), and got in way over my head with this thing, but I’m beginning to think it’s going to work out just fine, phew.

My unusually patient wife came with me to an industrial site in central Massachusetts where the saw was. I knew it would be heavy and expected to be breaking it in two pieces (base, where the motor is, and the top, which is a heavy cast-iron frame plus all the actual sawing bits), so I packed up some tools and we headed over there in our sturday but not very large Subaru Crosstrek. As it turned out, I didn’t need to worry about the tools as there was a well-equipped shop there, including a forklift and a roof-mounted crane that we were able to use to get the ~140 lb saw into the car in its two pieces. The seller did plug it in so I could verify that the motor ran and the saw would operate, but later I realized there were so many other things I could have and maybe should have also checked out.

band saw for sale
Here is the picture from the for-sale listing

We got this to our house, but it was too heavy to do anything about until my strong friend Brian showed up. The two of us managed to move the two pieces in a wheelbarrow out to the barn and set it up in a likely spot. Brian took a look at this thing and basically said I’d need to strip it down and see what might need some work.

I have to admit to some naive optimism – sure it was kind of old and beat-up looking, but who knows? Maybe it would all just plug in and run? So I spent a little while reconnecting the saw to its base, carefully tuning the table to be at precisely level and the blade guides to their 1/64″ clearance, reattached the belts connecting the motor to the pulleys on the saw, and then turned it on. Well, the light lit up, and the motor turned the wheels and blade, but only slowly (it was set up for metal cutting), and then it started to make horrible groaning noises! I spent a while cursing the people who had sold me this albatross, and then got to work trying to diagnose what could be wrong.

This saw has a shifting mechanism that enables the motor to be connected (via a belt) directly to the wheel to run at high speed (3000+ feet-per-minute) for cutting wood, or by way of a gearbox to achieve one of several lower speeds (40-335 FPM) for cutting various kinds of metal. Or at least that is what the manual says. When I tried to maneuver the clutch, I could not get it to move. At this point I started casting about online and discovered a community of enthusiasts, many of whom had posted photos, videos, and detailed descriptions of their own processes of saw rehabilitation. Many described jammed clutches, similar to mine. Some said it was due to pins being sheared off and getting stuck. Others talked about gummy oil residues. I honestly hadn’t expected this machine to have oil inside! Once I started removing pieces from the saw, taking the blade off so I could rotate the wheels independently, disconnecting the motor so I could check it separately from the wheel, I realized that the lower wheel was verry hard to turn. It seemed like it really ought to spin more freely. This, together with the evidence of the various rehab videos convinced me to open up the .guts of the saw and see what was stopping the clutch from shifting and the gears from turning.

The first thing was to drain the oil out. Annoyingly the drain plug on this saw is inaccessible when the saw is attached to its base?! So I had to disconnect these again, and hoist the upper part onto some sawhorses to get at the drain. It weighs a lot; more than 100 pounds I’m pretty sure, but I rigged up some ropes and a pulley and managed to swing the thing over. Then, after opening the drain, a drop or two came out. Then again, someone in his video had said it took more than a day for his to drain — I guess this gear oil is pretty thick? So I left it alone and bought myself a set of gear pullers for getting the various components off the axles or shafts on which they were mounted, and some fancy solvent called Kroil because I had misplaced my WD40 and someone said this stuff was better.

Oil drained, gearbox removed

After a few days I came back and the oil seemed to have finish dripping out. There wasn’t much: maybe a quart, and I had added a little motor oil, while the saw was supposed to be filled with 2-1/2 quarts. You can see the inside (and the outside) was pretty filthy:

Check out those drilled-out impressions on the gear you can see in the picture. I think those must have been used to true the gear. There wasn’t a proper gasket, just some silicon caulking, and I suspect oil was leaking out of this thing all the time. The stuff inside smelled terrible too – honestly it smelled like baby poop after the baby ate spinach for the first time, plus it seemed to have a lot of grit in it, maybe metal shavings or something. So I cleaned all that stuff out as best I could, adding motor oil, rocking it back and forth, getting in there with newspaper and my fingers. I took the gearbox apart too, just to get its parts cleaned up as long as the whole thing was opened up.

The gear puller is off-center because the gearbox isn’t symmetrical, but it seemed to work out OK. Then I tackled the upper shaft, which has the clutch assembly on it that wouldn’t move, and is directly attached to the lower wheel around which the saw blade turns. It took me a while to figure out that the nut holding that wheel on is threaded backwards and needs to be turned clockwise to remove. Thank you internet! In the meantime, I sincerely hope I didn’t damage it too terribly by wailing on my crescent wrench with a hammer, sigh. After some precision bashing with a hammer and a piece of wood, I got this out:

If you look closely you can see these two cylindrical pieces with rectangular teeth that look like battlement crenellations. Those are called Lovejoy couplings, apparently, and are supposed to mate with the big gear on the inside (the one that has the divots drilled out of it, shown above), or, in its other position, a pulley (not shown here) that mounts on the outside of the box and connects to the motor with a belt. But these suckers just would not move. At first I could not see what was preventing them. My initial theory was that one of the roll pins that goes in the holes in those Lovejoy couplings, and then into a slot in the outer shaft, and finally (I think? I couldn’t see) into the inner shaft that is supposed to slide up and down with these couplings, and holds the whole thing together, more or less, was broken off somewhere deep inside. This theory was prompted partly by various internet stories of similar things having happened, but also by the fact that one of these pins, unlike the other, was not flush with the surface, but appeared to have been driven deeper in. After some fruitless minutes spent attempting to drill out one of these pins, I started looking for other theories. Luckily my drill bits were not up to the task. Then I spent a while trying other various things, staring at the object, turning it around, scratching my head, returning to the internet to look at other peoples’ disassembled assemblages of parts, and it finally dawned on me that mine had an extra part! It looks so natural and unassuming, I hadn’t even noticed it, but there is a bushing, or spacer, in between two of the parts that just does not belong!? Some truly sadistic person had gone to the trouble of taking this whole business apart — they would have had to remove the aforementioned roll pins, not an easy task (although it would have been easier when the whole thing would slide properly since they probably could have been driven out with a punch when properly lined up with the holes on the opposite side), and adding a new piece which made it literally impossible to undo. Maybe they intended to disable the clutch so the machine could only and ever be used for metalworking? Or so it seemed to me. I find it difficult to conceive what this was all about. Anyway I managed to cut away the offending bushing with a hacksaw.

I did only minor damage to other parts in the process, cringe. After all that, and lots of Croil and TLC and oil, things did start to slide, halleluiah. Then I put the whole upper shaft back in, with its inner gear, and the lower saw wheel, because I just had to see if things were going to spin at high speed. I tried to share the vertiginous victory video I captured on my phone, but I am not sufficiently skilled to rotate, crop, rescale, transcode, and compress it sufficiently for your viewing pleasure! But please take my word for it – it seems like it’s going to work. I’ll try to post some more satisfying clips and pics later.

Some super-helpful resources I found while trying to figure out what the heck was going on:

Kayak launch day

It took me a couple of weeks to get the finish done. Coating the canvas with butyrate dope was pretty easy, although I learned that the ventilation in the converted garage room was not all that great, and ran around opening all the windows in the house to flush out the toxic fumes; not before I was discovered though. The thing that really took a while was the varnish I layered on top of that. The book proposed coating the transparent undercoats of dope with more layers of pigmented dope – paint, basically. But I wanted the my dye job to show through, so I was committed to use a transparent finish. I guess I could have just use more layers of dope, but I used a whole gallon on three layers, and this stuff was pretty expensive – around $80 for the gallon.

So for the final layers I used spar varnish, which is a heavy-duty traditional oil-based varnish. It’s also about $80/gallon at my hardware store, but I was able to apply three coats using only a pint of the stuff. Unfortunately the first coat I applied using some old stuff I found in the basement. It seemed OK underneath the top layer of hardened semi-gelatinous varnish, if a little dodgy. It was kind of thick, so I thinned it with some paint thinner and applied with a rag. It went on like runny snot and didn’t seem to want to dry. Maybe because the temperature dropped into the 40s and it was raining. Varnish likes it warm and dry. I ran the heater and the air conditioner (to pull moisture out of the soupy air), and kept on checking. After three days it was still kind of tacky. I rubbed it down with paint thinner, scraped off the worst bits of goo and bought some new varnish and Japan dryer, an additive that is supposed to help varnish dry in colder wetter conditions.

I think in the end it came out nicely, although it’s far from professional standard as such things go.

Its shiny!

Here’s a full view from the top. I’m pretty happy about the slightly irregular dye color.

And finally, it’s a she, and she floats!

How does she float? Well, I am really pleased. No leaking, that’s good. Much tippier than my old plastic tub, but I got used to this pretty quickly while paddling. The fit is kind of tight, but it seems right. I can securely brace my thighs against the Masik and paddle comfortably. After 45 minutes of paddling though my left foot was going numb. I have the same problem when running sometimes; bad circulation I guess. I hope that I will get used to the position, but we’ll see.

Generally speaking, she tracks quite well. Turning is a more deliberate process than with the flat-bottomed 9-footers, which spin like tops. There is a nagging tendency to pull to the left though, that really does need to be corrected. I think the stern piece must not be quite straight. So I’ll probably add a skeg, an external fixed rudder-like attachment, to get that worked out.

All-in-all a success! I look forward to many more days of paddling.

Skin-on-frame kayak, part 2

Steam-bending is a voodoo woodworking art. If you heat up wood, and it has enough moisture in it, the fibers will loosen enough that a previously stiff board can be bent into a new shape, which it will retain after cooling. At least some wood will. How well this works depends on a lot of factors including the species of wood, whether it is clean (without knots), whether the grain runs nicely parallel, how dry it is, whether it has been air-dried, or kiln-dried, how long you steam it, whether you pre-soak it, in water, or in water plus fabric softener! Some youtubers will tell you how easy it is, but even they keep extra pieces on hand for when they break one. Many years ago I had tried bending guitar sides over a hot pipe (no steam required), and that went OK, so how hard could it be?

I went to a specialty lumber store and bought some ash (kiln dried, as everything is that you buy at a lumber yard). I had read that ash was a good species for bending, even kiln dried. The gold standard for this kind of work is white oak, but supposedly only when it is green, or lightly air dried, and I just didn’t have access to anything like that. Following the book, I concocted a steam box from a sheet of styrofoam insulation and a lot of duct tape, and hooked it up to our old hot pot using some PVC pipe left over from a bathroom renovation.

I did some trial bending runs to get an idea how long I would need to cook the ribs and settled on something like 15 minutes. The idea here is to work efficiently by staging all the ribs, loading the steamer with a batch of 4 or 5 inserted at one minute intervals, and then inserting a new dry one at the left every time you take a cooked one out of the right.

I won’t go into the whole bending process itself – there are some great videos from pros out there you can watch if you care to. I’ll just say that it was nerve-wracking and fun, and it came out OK in the end, and even if some of the ribs did split out the grain here and there, nothing too terrible happened. One thing I might do different if I try this again though, is  to pre-soak the wood. I decided not to this time, following the advice of one knowledgeable-sounding site I found. That guy had nothing but scorn for the idea. Now I’m not so sure – it might have helped. I had to apply a *lot* of pressure to bend these ribs, and I get the idea it should have been easier. You can see one of the several failed attempts at the bottom of the photo. The room smelled like hot styrofoam the whole time.

The ribs outline the shape of the hull. Now we have really defined how the kayak is going to eventually sit in the water. Its hydrodynamic characteristics will all derive from this shape.

The next step is to fit out several longitudinal pieces: the keelson, which runs the length of the boat at its lowest point, and the chines, which run in parallel on either side. These all come together at the bow and stern where they meet two other pieces that form the pointy ends. The book had a fancy name for them, but for the life of me I can’t remember what it was. In this photo you can see the nifty lashing that holds these pieces to the ribs; the keelson is done, here, and the chines are being held in place with clamps while they are positioned for optimum boat shape (they will hold the skin just off the ribs).

Here’s the pointy end I was talking about.

Now it really looks like a boat! Just needs a few little odds and ends, and then a skin!

Oh I forgot to mention this cool part of the Greenland kayak design, the Masik. This is a curved piece of wood you brace your thighs against when paddling. I made this one out of a curved maple branch that fell off the tree in front of my house. You can see that tree’s shadow on the bricks there. It’s a giant, starting to die, but it still explodes in a riot of red, orange, and yellow every fall and buries us in its leaves. The piece I made the Masik from was starting to rot a bit, and it has some neat figure in the grain that I liked a lot. Shaping it was a fun challenge. I roughed it out with a chain saw and then went at it with a variety of hand tools. In this picture it’s still in a pretty rough state, but you’ll have to take my word, it worked out nicely.

 Oh no! Somewhere along the way this gunwhale developed a nasty-looking crack. More fallout from forcing them into too dramatic a shape? Oh well, it doesn’t go too deep – I hope it’ll be OK? Better get the skin on soon, that should help to hold it together, right.

I decided to skin the boat in canvas, and coat it with butyrate dope like an old airplane, in keeping with the generally old-fashioned gestalt. Sealskins seem to be out of fashion, and anyway, unattainable. Most people seem to use nylon at this point. It’s lighter and probably easier to coat since you can use something easier and cheaper like epoxy or urethane or some such thing. Cutting it is annoying though: you need a hot knife to melt the edges to prevent it from fraying. Canvas was fun to work with. I learned how to sew. A thimble was essential to get the big needle through several layers of the stuff. The thread is dental floss!

The whole process of stretching the canvas on, trying to keep as much tension as possible, was pretty fascinating. The traditional system is to lace up a very rough seam, like a giant shoelace, along the centerline of the deck, draw it tight, and then sew up with fine stitches. The book I was following had a different idea, which I tried, stapling the fabric to the gunwhales to tighten it, and then stitching along one side of the deck.

Oh I almost forgot to mention. There was this thing that drove me totally crazy right at the end of the project. Another steam bending step was required in order to create the hoop that forms the cockpit, drawing all the fabric together around the place you sit. The idea was to make this hoop by bending a single six-foot stick of ash around a form. I tried this three times and broke the darned thing every time. Maybe I didn’t steam it enough at first? This piece was bigger and thicker than any of the ribs. One time I think there was a weak spot in the grain. The bending itself is tricky – you are holding this thing under a lot of pressure, working under time pressure since the wood is only pliable for a short time after you take it out of the box. Finally, even the fourth time it broke, so I finally decided to make it out of two pieces, and glue them. Sigh, the only glue on the thing. But it got done

Once the stitching was done, and I had made the cockpit hoop, I died the fabric. You can see my first crude stitches on the stern before I figured out how to get the seam right on the corner.

Getting the skin stitched into the cockpit hoop was an exercise in trying to tension up the loose flaps in the middle of the deck. Following the system proposed in my book was pretty straightforward and a little magical; I drilled holes all around the hoop and stuck nails through the fabric just below the holes, levering them up and pushing them through in order to tension everything. Then I laced a length of cord through all the holes, removing the nails, and the twine draws everything up nice and tight.  Here’s a picture of the hoop with the nails holding the fabric to it:

The last thing is to coat the whole job in cancer-causing chemicals to make it watertight. As I said before I’m using butyrate dope. This is the stuff you may have sniffed if you ever made a model airplane. It is a toxic stew of VOCs (volatile organic compounds) – basically every chemical ending in “-ene” is in there. I bought a respirator, don’t worry, my glue-sniffing days are over.

Next post should be launch day. I’m super excited to see if this thing actually floats, how tippy it will be and whether it will track straight. Oh and hmm I guess I’ll need to name it (her?). That will be the hardest part.

>> Launch Day

Skin-on-frame Kayak, part 1

Before buying wood, I needed to figure out roughly how big a boat I wanted to make. The Greenland kayak design seems to be quite long and narrow – about three times your height, and barely wide enough to squeeze into, so for me that would have ended up around 17 feet (5m), and something like 20″ wide. I have been using a dinky 9′ plastic tub that is about 28″ wide, so this would be quite a leap. Besides, I don’t have anywhere to store a 17′ anything in my house, and don’t want to leave this labor of love outdoors where it would be subjected to the vagaries of New England weather. I settled on a 12.5′ length and 24″ wide. Maybe a little wide to roll, but since I’ve yet to learn how to do that and mostly paddle around in calm lakes and streams, this doesn’t seem like a big problem. Having settled that, I headed to the lumber yard to pick up some 2x construction-type softwood lumber that will make up the large longintudinal pieces: the gunwhales, keelson and chines, plus some clear pine for the deck beams and other assorted pieces. The ribs that make up the hull will be formed by bending them into shape and require some more specialized stock, and I haven’t yet figured out what to use, so those will be chased down later.

The first actual woodworking can begin! I needed to shape the gunwhales, which are the two large pieces, one on each side, that define the outline of the boat as seen from above (or below), and into which everything else will fit. I ripped these in the driveway using a circular saw and a simple jig to keep it aligned reasonably well:

hm, I guess the blade slipped a bit while resawing to 7/8″ thickness using the circular saw.

It’s important that these two pieces are cut from the same board and have the same dimensions. The symmetry of the boat will rely on their being held in opposing tension, so they need to behave similarly. I was a little worried about that wandering sawblade cut, but hoped it wouldn’t compromise the form too much! The first of many deviations from perfection, sigh.

Someday maybe I’ll acquire a machine planer, but I haven’t been able to justify it yet. And it’s still a pleasure to hand plane such clear pieces of softwood with a nice sharp blade. Here they are starting to look pretty clean after the saw marks have been removed. The ends of the gunwhales get tapered and shaped in the ends where they will rise up.

I splurged and bought this antique compass plane on E-Bay to shape those inside curves. This is an amazingly clever tool; its sole can be adjusted to a range of different curvatures. The one I got was probably 100 years old, but still in great condition. Whoever sold it had even kept the blade sharp, so I didn’t even really need to touch it up before using it!

Next, I measured and marked out the locations for all the deck beams and ribs. I love this marking tool: it makes it easy to get the same distance from the edge every time, and even has a two-point setup so you can mark both sides of the mortise at once.

Then I  cut the mortises for the ribs using a router with a jig to enable me to reliably and fairly quickly make the same cuts. The mortises for the deck beams need to be cut at a compound angle, so they will be done later using a less automated process, once the shape of the gunwhales is defined.

The rib mortises came out pretty nice

The form of the boat is defined using plywood stretchers that hold the gunwhales. I think this process was designed for a much longer boat which would have had a gentler bend. In this form factor, the pressure on the gunwhales was pretty intense, and it was hard to get the forms to stay in place. The one in the foreground below really wanted to slip forward. I was consumed with anguish for days worrying that I had veered too far off the plan I was following. One concern I had was that the gunwhale tips didn’t meet at a sufficiently acute angle. They were expected to end up nearly parallel, but in my setup they left a pretty gap where they met, as you can see below.

 After mighty struggles to force a tighter shape into the gunwhales, I finally decided to relax and let the wood dictate the shape. I added a breasthook to hold everything together in the stern. Cutting the compound angles on that block was a fun challenge.

Here is the boat really beginning to take shape, with all its deck beams in place. The curved beams are raised up to allow space for the paddler’s legs. This is going to be a pretty tight fit! You can see the variety of clamping systems I used to hold everything together. I had some major issues getting to this point. When I first tried fitting everything together, some of the beams were too short, or maybe the others were too long? I had measured so carefully! What went wrong there I’m still not entirely sure. I think maybe the shape changed while I was measuring? It was under so much tension, and possibly not held completely still. I had to re-cut some of the beams to get them to fit.

Another significant challenge was ensuring the symmetry of the arrangement. The whole setup did have a tendency to rack and twist ever so slightly, and if I didn’t correct for that, I would end up with a crooked boat that wouldn’t track straight. Again I think a lot of these issues resulted from having shortened the boat from its original design. If I did this again, I might consider using beefier spreaders, or some other system for shaping the gunwhales.

In the end the line I ran from end to end settled over the previously-marked midpoint of the center beam pretty well. Off by less than 1/8″ over 12′ – I’ll take it!

One more photo of this stage, after pegging and lashing the ribs in place and removing the clamps:

This was a major milestone, time to take a break. When we come back, it’ll be time to steam-bend the ribs!

 

Skin-on-frame Kayak, prelude

I’m reviving this blog years because I feel like sharing a project I’ve been working on: building a skin-on-frame kayak. I don’t propose to teach how to build a kayak; I’m just learning! Nor is this a history of The Kayak. This is a personal journey of escape and discovery. I will share what I learn, the mistakes and struggles. Hopefully it ends with me in a boat on the water. Maybe I’ll even learn to roll it. I hope folks will find this interesting. For me it has been truly life saving.

[Note: about this blog, and comments. No more comments 🙁 I would love to get your comments! But the internet does not make it easy. When I last checked I had almost 100,000 spam comments, and I don’t want to devote my life to defeating spam. So … if you want to reply to me, send me a note on twitter, where I am @msokolov].

I was largely working at home before the Pan***ic started, so at first I hoped it wouldn’t make all that much difference to me, but after months of basically never going anywhere except upstairs and downstairs, and occasionally out in the yard to fill the bird feeder, my life started to feel pretty low-dimensional. Eat, sleep, crap, work, rinse, repeat. Shower occasionally, and maybe cut my own hair. I think we’re all living some version of this. So I started coming up with stuff to do at home. Picked up the piano again, played some video games, but I found myself dreaming about a life out of doors.

I’m not sure exactly where I got the idea to work on making a kayak. Maybe because my daughter has been working on a fantastic boat restoration project. Maybe because a boat fantasy is a pretty good way to dream of a life of adventure while cooped up indoors during a New England winter during a raging pandemic. For whatever reason, the idea of building a boat crept up on me until it seemed I just had to do it. Over the years I’ve leveled up my amateur woodworking skills and tools, so I managed to convince myself I was ready for some kind of boatbuilding project, but I wanted to find something on the right scale, and a kayak is just about the smallest watercraft there is. Its prehistoric roots also gave it an appealing glamor. Early Americans made kayaks from driftwood, bones and animal sinew that were sturdy enough to use for hunting whales in the open ocean! Surely with modern tools I could accomplish something that wouldn’t sink?

I started by clearing out our semi-finished garage-turned-extra room and buying some books. Cleaning out the room rapidly devolved into a yak-shaving exercise. Where would all the stuff go? This included some large family heirlooms, a guest futon couch, a big piece from our youngest daughter’s senior thesis project, and these larger encumbrances were surrounded by the other more minute detritus accumulated over the years in this out of the way corner of our house. I am pretty ruthless about throwing things away, but a lot of this stuff needed to stay. What about the family VHS collection? Disney – in the trash. Unlabeled VHS cassette? It might have somebody’s birthday on it! I made a box to send to a digitization service some day. It was a challenge not to become overwhelmed by all the stuff. At one point, I thought I might pull up the ratty old carpet and tear down the sagging ceiling. I fantasized about trash bags full of plaster and watched online videos about pulling up carpet tacks. In the end I decided none of this was essential to my project though. Happily I was able to repurpose some of the items for use in my new workspace. Here’s a glimpse. At least I would have decent light, especially after rewiring some defunct standing lamps we had stashed away in the basement.

The workspace

I bought a random sampling of how-to books about kayak making. There has been an explosion of books in this area: apparently I’m not the only one with this itch. The one I ended up settling on as a guide was Building the Greenland Kayak, by Chris Cunningham. This book’s esthetic appealed to me, as did its detailed instructions. The kayak design it presents is (I believe) closely copied from traditional models in its shape and parts. The ribs that form the hull are bent wood, and the joinery is all dowels and lashings; no glue or screws are used. The materials called for are adapted somewhat for modern usage, mainly in the covering, which is not going to be sealskin, but nylon or canvas (I chose canvas). Cunningham also presents various construction techniques adapted for whatever power tools may be available. He indicates possible approaches using only hand tools at every step, but it certainly would be a lot more work to mill the long stock without a circular or table saw. Apparently one traditional method of making ribs flexible enough to bend was to chew on them. I opted to save my teeth and make a steam box.

Another book I enjoyed reading on the subject was George Putz’ Wood and Canvas Kayak Building. This book is encrusted with salty Maine wisdom! I think its designs are less fussy to build, and probably a bit more comfortable, if heavier, than the bent-rib Greenland variety. Rather than bent ribs for a frame, Putz’ boats deploy a trestle-like amalgamation of cross-members along with floor “timbers” for their hull, and he has no pretense of traditional construction, relying on screws, glue and other fasteners throughout. I might try these ideas next time. I also consulted Fuselage Frame Boats, by Jeff Horton, which presents yet another approach to wood kayak construction; plywood frames to shape the hull (the fuselage).

Finally I should mention Brian Schulz of Cape Falcon Kayak. I chose not to enroll in Brian’s course or buy his plans, but I did learn a lot from the videos he has posted online, especially his excellent advice about steam bending.

I have lots more to say, but not today. Next time I’ll start gathering materials and making the first cuts.

Skin-on-frame Kayak, part 1

Site migration

OK it’s 2018 and it is officially time to stop hosting blogs in your basement. I’m moving this blog and various other stuff out of my laundry room server and into the cloud.

The move promises to replace the nagging fear of attack by bitcoin miners, death by power outage, hard drive failure, or flooding with a $3.50/month bill. That’s what AWS Lighstail hosting costs today for its very lowest tier. I also plan to restore my vanity email domain using this server, since for a few years now I have been unable to maintain that at my house due to Verizon’s SMTP port blocking. I’m also planning to move some larger archives (music, pictures, etc) to S3 and relay them here using S3-FUSE. I think this micro-host might just be enough.

Insecure Cat


Do you see the cat?

My Long Trail

I began to realize how excited I was only a couple of days before I left. I could hardly sleep for thinking about my upcoming trip. Meal plans and other logistical details kept spinning through my brain; the last I checked the clock was 3:55 AM. This is actually good I think.

I’d been casting around for some meaningful way to spend the 5-week early summer sabbatical time I had blocked out months before thanks to a new company policy (thank you Safari!). I considered launching an effort to climb state high points, and did a warm-up climb of Mt Greylock, the highest point in Massachusetts, but something about the plan never quite sat right with me. It would provide an excuse to go to Hawaii, sure, but would also require me to go to some only mildly diverting places, like someone’s back yard in Rhode Island, and Hoosier Hill in Indiana. Even the more dramatic high points are often not among the more interesting hiking trips in their area, and are usually beset with crowds seeking the obvious. I knew this from previous climbs of Washington in New Hampshire, and Whitney in California. And then there was Denali: I wondered just how much time, effort and resources I would want to devote to this program, which couldn’t possibly be completed in five weeks.

Then some plans I had made for the early part of my time off fell through, and I realized I might just be able to squeeze in an end-to-end Long Trail trip. I spent four happy summers at camp in Southern Vermont, and some of the most memorable times I had were backpacking trips on and around the LT, a trail that stretches the length of Vermont, passing over the main ridge of the Green Mountains on the way. The idea of hiking it end to end was always in the air on these trips, and it has stayed with me for more than 30 years now.

The LT is the spiritual ancestor of the more well-known Appalachian Trail, and coincides with it for 100 miles or so. The distance, about 270 miles, is a great deal more manageable than the AT’s 2200-odd, but the walks both seem to promise a similar kind of communitarian ascetic experience. A big part of the draw for me was to pare down my life to bare essentials; to focus my attention on basic issues of transport, shelter and survival. I hoped to gain some perspective on my everyday life through a period of detachment, and I felt that an extended hiking trip would challenge me while maintaining my attention.

This is the story of my end-to-end Long Trail journey.

Continue reading

The two most common Solr performance blunders, and a rant about the dumbification of computer programming

Recently I’ve seen friends at work fall into a couple of well-worn traps, and I wondered – why do these same simple but devastating problems keep turning up again and again? The answer lies in deceptive software API’s, and the solution I think may come from video games. But before explaining all that, I want to just describe these two problems in a little detail.

Many programmers learn to deal with databases. Fewer work with full text search indexes like Lucene. I think newcomers to Lucene often bring with them the mental models acquired from databases, since they share many similarities. Both implement an “atomic” transactional model in which writes don’t become visible to readers until after a commit point is passed. This is critical in a transaction processing environment to ensure that two things succeed or fail together: for example, the concert ticket doesn’t get allocated to you unless ticketmaster confirms your credit card, and your card doesn’t get charged unless you get the ticket.

Lucene implements this model, but it isn’t designed to support transaction processing of that sort: it’s mostly optimized for batch updates and superfast querying across large numbers of documents (Yes, Lucene experts, I know about soft commits and near real-time, but that’s a story for another evening). This design criterion led to tradeoffs that tend to make committing expensive, very much more expensive than it is in a typical database.

Programmers writing their “Hello World” Lucene program don’t particularly need to be worrying about performance problems, but as soon as they start using Lucene for its intended purpose – indexing and querying large amounts of text – they do need to worry. Too often though it seems we fall into the trap of committing after every insert, causing a dramatic fall-off in indexing (and querying) performance, even to the extent of making a search service non-responsive. A very common version of this problem manifests itself in Solr, where you can see the dreaded “PERFORMANCE WARNING: Overlapping onDeckSearchers” message in the logs.

There are pounds and pounds of curative blog posts, wiki pages and stack overflow answers explaining this problem, why it arises, and what to do about it. And thanks to search technology, it’s pretty easy to find them if you go looking. But an ounce of prevention would save a lot of headaches here.

My number 2 most common Solr performance blunder is `fl=\*`. Solr search results include the values of fields stored in the index associated with the result document. Typical search applications show a title, with a link to a full version of the document, and possibly an associated contextual ‘snippet’. A few other fields like date, publisher or book title might be included too. Such applications must store the full text of every document in the index to make it available to the snippeting component (the highlighter, as it’s called). However, if that text is *retrieved* as part of the search result, and the documents are not tiny, this practice vastly increases the amount of data that needs to be transferred, often leading to a 10-100x slowdown in search performance. Programmers tend to do this because it’s just easier to retrieve all the fields (that’s what `fl=*` does) than to list explicitly only the fields required to display the result.

No decent Solr tutorial will lead a programmer to do this, and again, there is plenty of good information explaining how to select fields, but the Solr default is to select all fields, so this is a very easy trap to fall into. And it becomes even easier not to notice that this is happening when your queries are mediated by a middle tier.

I maintain an API for searching across multiple different backend data storage and indexing systems, and in that API we once defaulted to returning full document results. I believe our thinking was that beginners would have everything they need and not stumble over having to learn too much of a complex API even to get started. But I got tired of leaning over people’s chairs with a knowing grin and pointing out their n00b mistake. It really wasn’t their fault, anyway (it was mine.) They were just doing the natural thing, following the most straightforward path the API made available. So I changed the default, and I think the people I work with must be really smart since they seemed to be able to figure out how to get the missing field values when they really did need them. Even if they had to ask about that, it was much better to field a question like “how do I get the full text of a document in the search result?” than a question like “why do my search results sometimes take 10 seconds to come back?” because often a situation like that would be intermittent, only arising when really large documents made it into the search results, and could persist for a long time with merely mediocre performance before anybody really took note of the egregious outliers.

Apparently I’m not the only one making this mistake. [Safari Flow](http://safariflow.com) uses [Haystack](http://haystacksearch.org) as its internal search API. It’s welcoming words – I just stumbled on this while writing this sentence – “Search doesn’t have to be hard.” We recently found out that Haystack, by way of its defaults, encourages users to make exactly these mistakes. The Solr connector automatically commits after every insert, and I couldn’t even find any way to limit the set of stored fields returned. In both cases we had to fix these serious performance problems by editing Haystack’s Solr “backend” connector source code (in spite of its promise that “\[Haystack\] plays nicely with third-party apps without needing to modify the source…”

OK I’m a bit peeved about Haystack right now, but I truly hope the maintainers will read this and take it as constructive criticism, because their library actually does provide a lot of convenience to Django framework programmers grappling with search. Here’s my advice.

There is a notion gaining currency that programming computers is becoming easier. Sites like codeacademy teach JavaScript using a glossy, game-like spoon-fed interface. “Learn Python The Hard Way” presents Python (and other languages, in spinoff titles) using a baby-step scaffolded teaching approach (the only hard thing about it for me was sticking with it – to be fair, it acknowledges on page 1 that it wasn’t designed for impatient smartypants). There are many other “learn to code in 5 minutes” websites and courses that offer an easy path to software mastery.

This conceit that programming can be easy is partially fueled by the development of software languages and tools. It *is* easier to incorporate other people’s code now using reasily-available libraries and frameworks, and to make use of existing systems, so not every program has to start as a *tabula rasa*. It is *not* necessary to understand computer architecture in a deep way in order to write much useful and/or entertaining code now. In some ways, things have gotten easier.

There is also a cultural component to this new easy-going attitude: it’s a deliberate effort to be more inclusive, to shed the high-priest hacker snobbery that has been the stock-in-trade of software gurus for thirty years and more. “RTFM” with its veiled obscenity was always a little rude, even when uttered in jest, where its moral successor, “lmgtfy,” is simply peevish, but they reflect the same unpleasant underlying attitude of condescension. I’m glad to see some reflection on that negative side of hacker culture and thecorresponding openness to newcomers.

The positive side of the “learn to program” movement is that there are numerous ways to contribute without being a master. More than ever it is possible to go very far with very little. Silicon valley startups no longer sweat hardware: they just rent space with Amazon. This is healthy: it means that the culture as a whole is able to learn and grow, to stand on the shoulders of the previous generation rather than their faces.

I’m sure you saw this coming: yes, Virginia there is a dark side. The thing is, the obnoxious attitude grows out of a hard reality. Expert programming requires knowledge. Mental nimbleness and a problem-solving bent count for a lot, but true mastery of any craft, including programming, is only available to those willing and able to devote years of study, trial, error and correction. And there are still problems to solve that demand mastery, where beginners should be cautioned to read lightly.

So let’s stop saying that search can be easy. We do learners a real disservice by pretending that things are going to be easier than they are. There are complex problems in search, getting them wrong can kill performance (*i.e.* your web site), and our role as guides should be to offer paths to learning that have the right degree of steepness, and to offer warnings about potential pitfalls. If we take you on a mountain-climbing journey, and just tell you everything will be taken care of and there’s nothing to worry about, we’re leading you into a potentially dangerous situation without any preparation: in that setting, this kind of attitude would be criminal. Tell people to bring their helmets, and teach them how to self-arrest! Wat? Metaphor getting out of hand …

At the same time, we don’t want to scare people away. There’s no call to be going all high-priest-in-the-inner-sanctum with acolytes getting in only after years of fasting and prayer. Here’s where I think we can take a cue from video games. I read (this post)[http://robotinvader.com/blog/?p=402] about luring gamers into playing the video game Devil May Cry. It talks about a game that is notoriously difficult, but also offers an easy way out. The interesting thing is that it challenges the player to try the hard way first, and warns them that there will be no way back if the easy path is chosen. Nethack, an insanely arcane video game, does a similar thing with its wizard mode: players can use it to try out all kinds of stuff, without dying, but it comes with a caveat: this is not for real, and your scores won’t be reported.