Ссылки о веб-разработке за октябрь 2008

A programmer's view of the Universe, part 1: The fish

I write a column for computer programmers called "Stevey's Blog Rants." It's basically a magazine column — I publish to it about once a month. The average length of my articles is about 12 pages, although they can range anywhere from 4 to 40 pages, depending on how I'm feeling. But for precedent, don't think blogs: think of Reader's Digest. The blog format sets the wrong expectations.

Hence, some people complain that my articles are too long. Others complain that I have not given my arguments sufficient exposition, and that my articles are in fact too short on detail to warrant any credibility. This is a lose-lose situation for me, but I keep at it nonetheless because I enjoy writing. Even if nobody were to read my blog, the act of writing things down helps me think more clearly, and it's engaging in the same way that solving a Sudoku puzzle is engaging.

You should try it yourself. All it takes is a little practice.

My blog topics vary widely, and sometimes I even venture outside the realm of programming. Programming is where I'm most comfortable, and it's also where people seem to ascribe to me some level of punditry: I'm not necessarily right, but even my greatest detractors grudgingly admit that I'm entitled to an opinion, by virtue of my having spent twenty years hacking day and night without any sign of wanting to give it up and turn into a pointy-haired manager.

Even though I love both programming and to a lesser extent writing about it, there are also lots of non-programming topics I'd like to write about. Being a career programmer gives you an interesting perspective on issues not directly related to programming. You start to see parallels. So maybe I'll branch out some more and see how it goes.

The programmer's view

The first thing you notice as a programmer is that it trains you — forces you, really — to think in a disciplined way about complex logic problems. It also gives you a big booster shot of confidence around problem-solving in general. Junior programmers tend to have very high opinions of themselves; I was no exception.

In time, though, programming eventually humbles you, because it shows you the limits of your reasoning ability in ways that few other activities can match. Eventually every programmer becomes entangled in a system that is overwhelming in its complexity. As we grow in our abilities as programmers we learn to tackle increasingly complex systems. But every human programmer has limits, and some systems are just too hard to grapple with.

When this happens, we usually don't blame ourselves, nor think any less of ourselves. Instead we claim that it's someone else's fault, and it just needs a rewrite to help manage the complexity. In many cases this is even true.

Over time, our worldwide computer-programming community has discovered or invented better and better ways ways to organize programs and systems. We've managed to increase their functionality while keeping the complexity under control.

But even with such controls in place, systems can occasionally get out of hand. And sometimes they even need to be abandoned altogether, like a dog that's gone rabid. No matter how much time and love you've put into such systems, there's no fixing them.

Abandoning a system is a time of grieving for the folks who've worked on it. Such systems are like family.

And there's a disturbing lesson at the tail end of such experiences. The scary thing is that it's very easy, as a programmer standing at the precipice of complexity, to envision systems that are orders of magnitude more complex, millions of times more complex, even unimaginably more complex.

In the end, programming shows us how small we are.

The fish's view

Long ago, I used to have a Siamese fighting fish, also known as a Betta splendens, or simply a "betta". You can buy these fish at almost any pet store. I kept my betta, who was a deep vibrant red, in a pretty little 15-gallon tank decorated with a resplendence of real freshwater plants. And for a while I think my betta was happy there.

Like many Americans, I went through a phase in which I kept and ultimately killed many, many tropical fish. I didn't kill them intentionally; I wanted them to live and thrive. But keeping them alive for long is a challenge when you don't live in the tropics. So they might live for a few months or maybe a year, but they would always die prematurely. It was sad, and eventually I could no longer bear it, so I stopped keeping them.

Of all my fish, my betta left the biggest impression on me. The betta is a remarkable fish in several ways. For one thing, bettas are physically beautiful, and when they are at full display, their fins expand, peacock-like, into a fluid rose shape that is undeniably dramatic.

Bettas are also remarkable because they fight. They do not fight to establish a pecking order, as other fish do; they fight to kill. The males display their fins and then fight whenever they see another male betta, or even their own reflection, so they have to be kept alone and away from mirrors.

But bettas, I think, are most remarkable for their intelligence. Of all of the hundreds of tropical fish I kept, only bettas displayed anything resembling intellectual curiosity.

This really makes bettas some of the saddest stories in the tropical fish industry. Like other hobbyist fishes, they are stolen from their natural habitat and shipped overseas, or at best farmed in unsavory conditions. But unlike most other fish, bettas are also dyed to enhance their color. They are generally housed in tiny fist-sized bowls because of their ability to breathe air when necessary. And they are bred to express their fighting genes, and are often made to fight by their owners. Whereas other fish are kidnapped and sold, bettas are abused.

But worst of all, I believe their high intelligence endows them with greater capacity for suffering than other fish species. They can suffer physically and emotionally, but as we will see shortly, they can also suffer intellectually.

So bettas are a sad story.

My betta

Here is the specific sad story of my betta, the fish that left such an impression on me.

I had taken to lying on my bed and watching my betta for an hour or longer. The betta was the sole occupant of the tank in my bedroom. I had filled the tank with plants and copious natural light, so the effect was calming and serene. At times I almost envied the betta for the nice home I'd made for him.

One day, after the betta had been in his new home for several weeks, I found him exploring. It was a most unusual exploration, and one that I will never forget.

For the first few weeks, the betta explored the way you would expect any reasonably intelligent fish to go about the task. For the first few days he swam around to every nook and cranny of the tank, to make sure he had the lay of the land. Then for a few more days he experimented with staying put in different locations to see how he liked their feel.

Just like people, most fish will soon find a spot or a path they like best, and they'll stay in that spot or on that path for the rest of their lives.

But my betta was different. After his initial explorations he became restless. I'm no Fish Whisperer, but I could tell that he was restless. You would have thought so too. The betta started spending most of his time looking out of the tank, examining my bedroom. And he was clearly looking at specific things in the bedroom, not just "out there" in general. He would periodically swim around looking mildly agitated. He was acting like he wanted out.

I did everything I could to placate him. I experimented with different fish foods. I changed the water weekly and monitored it carefully to keep its temperature and pH within acceptable ranges. I added more lights. I added more plants. I rearranged the plants. In desperation, I even added a little castle.

Every time I tried something new, it would pique his interest for a little while. But in time, and faster each time than before, he would revert to his state of restlessness.

I'd never seen quite this behavior in other fish, so already he was demonstrating what seemed to be above average intelligence.

And then one day I found him engaged in an exploration that was altogether new.

He wasn't exploring the tank. He'd already investigated its topology for weeks. This time, he was exploring the nature of the tank. That's what caught my attention, and not just for that day, but for the rest of my life.

There was a twenty-inch vine in the tank that extended from the lower left back corner to the upper right front corner, along the diagonal of the main volume of the tank. The vine belonged to one of the many plants I'd put in there in the hopes of making it feel more like the Mekong river basin and less like a plexiglass tank in Seattle, Washington.

The betta had his nose on the vine. He was floating just above it, twitching his fins slightly to stay in place, and he was keeping his eyes as close to the vine as possible while keeping it in focus: about half a centimeter to a centimeter. And he was traversing the vine.

With the tiniest of motions, he was propelling himself along the vine towards the lower back right corner, keeping it under close scrutiny at all times. This excursion, from the halfway point to the end of the vine, took him perhaps three minutes. He was taking his time.

When he got to the end of the vine, he remained rooted in place while he inspected the 3-inch-radius spherical volume at the end of the vine, which was truncated in three dimensions by the walls and floor of the tank. He spent about three or four minutes doing this inspection, evidently making sure the vine really did terminate in the corner, and did not escape the tank.

After he had thoroughly scrutinized everything in the betta-sized vicinity of the vine's end, he turned back to the vine, nose pressed close, and began working his way along the vine in the other direction.

At this point I sat down to watch, because if he was doing what I thought he might be doing, then... I didn't know what to think. I wanted to see it for myself.

Over the next seven to ten minutes, he crept along the vine, never losing sight of it nor getting further than a centimeter from it, until he reached the upper-right front corner of the tank. He then proceeded to repeat his inspection of the volume at vine's end, ensuring himself that the vine terminated in the tank rather than protruding beyond the wall.

But what if he had missed something?

Sure enough, he turned and looked down the length of the vine for a time. And then he put his nose back on the vine and began again his long descent to the other end.

He did this for five days.

By the second day my amazement had turned to concern, and by the third day I felt utterly helpless. Here was an intelligent prisoner, my captive, exploring the mechanics of his prison with a thoroughness that only the imprisoned can afford, looking for an escape with deathly tenacity.

But while purchasing my betta had been easy enough, returning him to his real home would be unthinkably difficult, and probably unsuccessful even if I'd tried. Returning him to the fish store seemed like a dead end; he could easily wind up worse off than he was now.

So I concluded that there was nothing I could do. As he inspected the vine, I bit my nails, and timed passed in silence.

After the fifth day he gave up. And then he did something that I still don't understand, even though I've heard about this kind of thing before, and even though I personally saw him do it: he died of unhappiness.

It only took him a few days. He refused his food, he stopped moving, and to all external appearances he had become ill. But I knew better.

The lesson

Whenever I find myself struggling against the tide of massive system complexity, I think of my betta. He had a big heart, a small brain, and a small range of sensory input. I watched him use them all as methodically as any programmer to reason his way through to a soundness proof of the inescapability of his prison.

We like to think of ourselves as being pretty smart. Admit it. We do. But in the grand scheme of things we're intellectually little better off than that fish. We can easily find problems so complex that reasoning about them can take days or weeks of microscopic scrutiny, like my fish swimming along his vine.

And we can just as easily envision problems thousands or millions of times more complex: problems beyond the reasoning abilities of any person, any group of people, or even our entire species.

This has ramifications for the way we think about things today.

I believe I will have more to say about this soon. Right now I need to go mourn my fish, whose soul shone as brightly as that of anyone I've known.

The OpenNET Project: Восстановление "битого" iso образа при помощи rsync

Yahoo! Query Language и раскрытие информации

В Yahoo! опубликовали спецификацию на YQL - Yahoo! Query Language - язык запросов для получения данных из информационных массивов Yahoo! Причём доступ к данным производится через специальный вебсервис, а результаты можно получить в XML или JSON. Нехватает толькок Thrift./

При том что сама по себе идея очень интересная, она так и просится в различные системы раскрытия информации и предоставления онлайного программного доступа к массивам данных.

Кросспост из Иван Бегтин. Комментарии можно оставлять здесь или здесь.

Microformats and Microsoft

To date, Yahoo! have definitely been the biggest champion of microformats among high profile large web companies. As commented here many times, they implement microformats in many of their properties, and their new SearchMonkey search platform even indexes microformatted content, and makes this index available to search application developers.

We’ve also seen quite some interest from Mozilla, with for a while, the promise of native microformats UI support in Firefox 3 (and a microformats javascript API in that browser).

Now, Microsoft, who’ve been doing some interesting microformats related things with hSlices have released some excellent microformats oriented tools and more.

First up, their new web developer/designer focussed community, Mix Online (which features a microformats 101 style article by me, and a Designing with Microformats article with Tim Aidlin) kicks off with a strong Microformats focus.

But the most exciting news is a new Microformats toolkit from Microsoft. It includes an add in for the IE toolbar, called Oomph, that, similar to Operator, provides access to users to the microformatted content on any web page they visit. Here it is in action on my (sadly out of date) personal site

Oomph Microformats toolkit in action

Even better, they’ve released a JavaScript library at you can add to your own sites to provide the same user experience to users of any contemporary browser.

And the same toolkit provides a set of CSS based designs for hCard and hCalendar content you can add to your own sites, and plugins for creating microformatted content for a raft of blogging and CMS tools, including Blogger and Wordpress, in addition to Sharepoint and LiveWriter by Microsoft. Grab the toolkit here.

to me this is the most important and exciting step - increasing signs of Microsoft walking the open, standards and community talk that we’ve been seeing for the last year or two. Well done to them on this.

Just a disclaimer re the article at Mix Online, and my relationship with Microsoft generally. I was not paid for that article, and while Microsoft have been sponsors of various events I run, I’m also on the record with as much criticism as praise for their efforts over the years.

← предыдущий месяц