I prefaced my talk by saying that none of this is original, so it shouldn’t come as a surprise. I merely wanted to bring the different strands together in a way that showed them how to think about the opportunities afforded to libraries for the digital age.

Below is the text of the talk, and I’ve attached PDF versions as well (A4 and US-Letter). I’ve released this under CC-BY-SA and hope it’s useful for you. Please let me know if it is!

—

Bill Gates wrote a bestseller in 1995.  He was on a roll: Microsoft Windows had finally crushed its old foe the Macintosh computer from Apple, Microsoft was minting money hand over fist, and he was hugely respected in the industry he had helped start. He roped in other big brains from Microsoft to write a book to answer the question, “what next?”  The Road Ahead talked about the implications of everyone having a computer and how they would use the great Information Superhighway that was going to happen.

The World Wide Web appears in the index to The Road Ahead precisely four times.  Bill Gates didn’t think the Internet would be big.  The Information Superhighway of Gates’s fantasies would have more structure than the Internet, be better controlled than the Internet, in short it would be more the sort of thing that a company like Microsoft would make.

Bill Gates and Microsoft were caught flat-footed by the take-up of the Internet. They had built an incredibly profitable and strong company which treated computers as disconnected islands: Microsoft software ran on the computers, but didn’t help connect them.  Gates and Microsoft soon realized the Internet was here to stay and rushed to fix Windows to deal with it, but they never made up for that initial wrong-footing.

At least part of the reason for this was because they had this fantastic cash cow in Windows, the island software.  They were victims of what Clayton Christenson calls the Innovator’s Dilemma: they couldn’t think past their own successes to build the next big thing, the thing that’d eat their lunch.  They still haven’t got there: Bing, their rival to Google, has eaten $5.5B since 2009 and it isn’t profitable yet.

I’m telling you this because libraries are like Microsoft.

At one point you had a critical role: you were one of the few places to conduct research. When academics and the public needed to do research into the documentary record, they’d come to you. As you now know, that monopoly has been broken.

The Internet, led by Google, is the start and end of most people’s research. It’s good enough to meet their needs, which is great news for the casual researcher but bad news for you.

Now they don’t think of you at all.

Oh yes, I know all the reasons why the web and Google are no replacement for a healthy research library. I know the critical importance of documentary heritage. But it’s not me you’re talking to at budget time. It’s the public, through the politicians.

They love public libraries, in our country at least. Every time a council tries to institute borrowing fees or close libraries, they get shot down. But someone tries, at least once a year. And England is a cautionary tale that even public libraries aren’t safe.

You need to be useful as well as important. Being useful helps you to be important. You need a story they can understand about why you’re funded.

Oh, I know, you have thought about digital a lot. You’ve got digitisation projects. You’re aggregating metadata. You’re offering AnyQuestions-type services where people can email a librarian.

But these are bolt-ons. You’ve added digital after the fact. You probably have special digital groups, probably (hopefully) made up of younger people than the usual library employee.

Congratulations, you just reproduced Microsoft’s strategy: let’s build a few digital bolt-ons for our existing products. Then let’s have some advance R&D guys working on the future while the rest of us get on with it. But think about that for a second.  What are the rest of us working on, if those young kids are working on the future? Ah, it must be the past.

So what you’ve effectively done is double-down on the past.

I like to think of libraries as services in three areas: collections, discovery, and delivery. You maintain big piles of stuff, you help people find the right stuff, and then you let them use it.

In the paper world, this was dominated by the challenges of collection and discovery. So librarians have incredible expertise in preserving words on reeds, on calf skins, on pulped trees. There’s huge mana in having a big collection.  Collections must grow, they must be complete, deaccessioning breaks hearts and causes shouting matches. And, despite paper, you’ve been eager innovators and adopters of new information technology: card catalogues and the Dewey Decimal System were profession-changing inventions in their day.

Collections, discovery, and delivery. Delivery is runt of the litter in the paper world, I’m afraid.

One copy?  One precious copy? Ok, sonny, you sit here.  We’ll bring it here.  Don’t cough, don’t breathe, warn us before you blink. Or, in old days, help yourself and we’ll trust you as a gentleman to bring it back. That was even less successful than pursed lips and the tyranny of the reading room.

The first movie was a camera pointed at a play. They didn’t know the possibilities of the old medium, so they reproduced the old structures in the new medium. When confronted with digital technology, you’ve basically reproduced the old power structures in the digital world.

You want a massive digital collection: SCAN THE STACKS!  Give it to Google!  Give it to a commercial partner! Just get the damn things digitized so we have a lot of bits of our atoms!

You agonize over digital metadata and the purity thereof. You maybe reluctantly part with your metadata (but not your precious collections!) to Trove.

And you offer crap access.

If I ask you to talk about your collections, I know that you will glow as you describe the amazing treasures you have.  When you go for money for digitization projects, you talk up the incredible cultural value.  ANZAC!  Constitution!  Treaties! Development of a nation!

But then if I look at the results of those digitization projects, I find the shittiest websites on the planet.  It’s like a gallery spent all its money buying art and then just stuck the paintings in supermarket bags and leaned them against the wall.

You’re in the digital world.  Bits don’t work like atoms.  I’ll give you five critical ways that bits don’t work like atoms.

First, bits are cheap to copy.

By all means protect the digital master, but copies can be plentiful or even ubiquitous.

Physical access has been limited because you have one copy of each physical item, you need to maintain control of that copy to preserve it for the next patron, and copies are expensive to make.  Digital copies are free to make, they’re non-destructive, they free you from the burden of control, and you can have as many as you want.  Those are vastly different rules.

This is, of course, why copyright is such a bugger in the digital age.  It’s riddled with assumptions about the difficulty of copying atoms that aren’t true of bits.

Second, access is expected.

You can argue until you’re blue in the face about the intrinsic value of collections, but as your research monopoly has been destroyed, you need to start delivering some other value.  Access to those precious collections is it.  Collections, discovery, distribution.

If nobody uses your digital collections, what’s the point? If nobody can find the digital objects, what’s the point? If you recreate medieval standards of access in the digital age, what’s the point? You won’t get to the 21st century by doubling down on the 11th century.

Your new reading room is your patron’s web browser. Are you designing distribution for that?  How much did you spend building a new reading room, Bill?  How much are you spending on digital delivery?

The first place they start looking for things is Google. Are you designing discovery for that?  Do you know how to be found?

Example: the British Library had a company digitise, and got limited access and rights to the digitised content.  Google contracts have restrictions on your use of the scanned material, too. Is this kind of arrangement acceptable?

It depends on whether libraries are primarily collections or whether you have high expectations for access, too. If you don’t value distribution, you’ll think these are good choices. The British Library says “hey, the physical objects were only available on our premises; this gives more access than there was before.  Most importantly, though, we solved the digitisation problem!”

You can see the mistake they made. They focused on collecting digital assets and digitising their physical ones, probably even convened conferences on digital metadata.

And then hid their fabulous collections out of sight. It’s like they WANT to be irrelevant.  “Please, don’t be one of the first places people visit to research the nation’s cultural identity!  Let’s make it hard for you to do scholarship!”

So, once again: distribution is critical in the digital age.

Third, the Internet is bigger than you are.

In the past, you had knowledge, frozen in books.  Ordinary people came to you to get that knowledge. There was a bit of a class divide: those who Create Knowledge and those who Consume it.

Those days are gone. Online, everyone’s a creator. Those of you doing digital harvest of websites know this. “Look at all the crap we have to save!” (The same is true of legal deposit collections)

The point is that you’re saving the stuff that future generations will care about. And, increasingly, the stuff that future generations will care about is online. That’s why Library of Congress acquired an historical and ongoing archive of tweets. Not because a tweet is comparable to a first folio, but because it’s what future generations will care about when it comes time to determine the mood of the nation.

I personally believe that the greatest role you play is around the documentary national identity. People come to you to find out about their ancestors, to find out what life was like, to critically evaluate and understand the past.

If you consider your future in terms of documentary national identity, you might do other things. There’s a software project here called Kete, Maori for basket, which is a way to capture and preserve family histories, stories of the area, photos, interviews, etc. Imagine a future where citizens contribute to and search these, perhaps through their local public libraries. Wikipedia won’t take this stuff, it’s not notable, but it’s exactly your business: we’ll take it and help other people search it.

You might do what the National Library of New Zealand did, and dispatch a photographer to Christchurch to document the earthquake aftermath and recovery to ensure adequate documentary record was available to future researchers.

So, in short, much of the nation’s cultural life is now happening Out There. You need to find more ways to gather it in.

Fourth, bits are so cheap we have too many of them.

Our grandparents grew up with very little. They valued every possession.  I know this because I live in my grandparents’ old house and I’m still finding balls of odd-lengthed twine in the basement. In fact, we humans evolved with very little.  We were always starving for food, short of objects, desperate for information.

Now we have too much of everything.  Cheap plastic crap from China means everyone can have a crappy version of everything they need.  Cheap industrial crap food means everyone can get calories, even though they might not be good for us.  And easy copying of bits mean we have too many of the damn things.

Computer scientists think they can solve this problem.  We’ve got indexes and search engines. What we can’t programme is critical thinking in humans.  That’s where librarians come in.

Let’s assume that Google’s search engine is the state of the art at finding gemstones buried in dungheaps. This state of the art is not great.  It struggles with relevance, it tries to filter out spam, and it personalizes so I see different results than you do.  And, of course, it’s beholden to its advertisers. This can never be the only answer to helping citizens find what they need.

The best solution is when both man and machine work together: librarians make sense of indexes, this is what they do. Computers are great at building indexes. Don’t think either-or, think and.

Part of a national or state’s library’s role is to get stuck into this and help.  Teach information literacy.  Teach basic research skills.  Work with schools so kids know how and where librarians exist.

Discovery is important online, and it’s not just having accurate metadata and Google.

Fifth, the Internet connects things.

I know, it sounds obvious, but that’s what it does. Good broadband is coming to all of us, thanks to the national broadband projects which are by now too big to fail. That broadband isn’t just for sending digitized books across. It’s also the medium by which librarians and libraries can work together.

Oh sure, you can share collections.  This is threatening to institutions because the collection forms a key part of the institution’s identity.  Both countries have projects to provide one-stop-shop search across all cultural collections (search but not delivery!) so we’re starting to get our heads around sharing collections. I imagine a National Digital Library where the collections are shared like this.  But not just the collections.

You can share services too. You’ve probably experimented with online services.  NZ has AnyQuestions, for example. High-quality video conferencing, email, and the web are ways to deliver human services across the Internet.

If you have people delivering services online (answering questions, making recommendations, entering data, etc.) then you can share people without having to physically move them around.  What opportunities does this open up? Share staff between institutions, or have specialist staff offer services in a physical location where they cannot be.

The Internet also connects computers. This is the age of “the cloud”. Can you provision equipment for other institutions to use? The National Library has a project to provide regional libraries with an affordable functional modern catalogue system so they don’t need to spend the dollars themselves. What joint purchasing can you share in this fashion?

So, to recap:

• be useful as well as important
• collections, discovery, distribution
• bits are cheap to copy
• access is expected
• the Internet is bigger than you are
• we have too many bits
• the Internet connects things

You can’t afford to be bad at digital. I tell businesspeople: It’s your inventory, it’s your storefront, it’s your customer service line, it’s your supply chain, it’s your advertising, it’s your profit and loss.

For libraries, the Internet is your collection, it’s your reading room, it’s your catalogue, it’s your interloan, it’s your helpdesk, it’s your opportunity to reclaim relevance.

And I’m afraid to say, you’re the pointy end of the digital redefinition of culture and heritage institutions and public services, because text is small and the first to go digital. E-books?  Next are e-music, e-movies, e-ephemera, e-maps, e-paintings, e-sculpture, and who knows what e-lse. Every archiving institution will face your problems, some are already grappling with them (e.g., the Powerhouse Museum).

Online search? Online helpdesk? Online loans? Every public-facing organisation will face your problems. At least you can take comfort from the fact that you won’t be the only ones disrupted by this change.

Finally, let’s consider Microsoft. Nobody wants to be in their place: 15 years after discovering the Internet, they’re still tipping money into it with little success.

The company that successfully transitioned from a Microsoft business to the Internet age was Apple.  When Jobs returned in late 90s, he threw out the 40-odd products they had and said “we’re going to make computers that are build to connect to the Internet, and the software on them will be Internet-aware software.”  They focused on four Internet computers (that’s where the i in i-Mac came from) and from that success he was able to focus on successively further extensions like iPods and iPhones and iPads.

You need to focus. Success for you is relevance. Make things that people use. Value the skills that your people have and the services they deliver, but don’t be a slave to atoms. Value helping people.

Then when someone asks “why do we tip all these millions into this?” or “doesn’t Google do that already?”, your relevance is your answer.

You must do this. Libraries are the homes of critical thought, of long-term cultural preservation, and of democratic access to knowledge. This can’t end with the Internet.

Something new this week: I solicited topics from my Twitter followers, and got some great story ideas that I wouldn’t otherwise have covered. Go team! Thanks to Don Christie, Bernard Hickey, and Daniel Spector.

Links

Life from Apple iPad 2 event, antilasers, 3d Printing company, Reprap, MakerBot, Ponoko, ThingiVerse, NZ Tech Company Sells to US Partner.

Quakes and Computers

* how have Christchurch computer businesses been affected?
* were people outside Christchurch affected?
* what’s been learned?

Tech business in Christchurch have been affected by the quake. Not just employees dealing with lost houses. There’s still intermittent broadband access in the region, so some businesses can’t get online to conduct their business. Some can’t access their buildings–computers, records, servers, email, source code are all locked inside a cordon or under sludge.

But it’s not just people in Christchurch who were affected. A friend’s company had outsourced its website. The company that ran the website for them was based in Christchurch. In the CBD. My friend got a call saying “oh hi, your website is still up but it’s in a building we can’t get to that might be condemned and the generators will run out of power in a few hours. SO. We’re moving to backup servers in a different city and you shouldn’t notice any problems but we thought we should let you know ….”

This outsourcing is all part of the “move to the cloud” that you sometimes hear about. Cloud computing is the idea that you access your services, whether it’s web hosting or wordprocessing, through the Internet. This way you don’t have to run the computers, manage the software, all that stuff. It’s easy! What could possibly go wrong? Oh, right. Your supplier of cloud computing might have their data center on top of a new fault (or is it an aftershock …?).

I was talking about this with friends, about what the lesson here is. I interpret the lesson as “when you outsource to `the cloud’, you’d better make sure they have good disaster recovery plans.” My friend pointed out “almost everyone who does this stuff themselves, runs their own servers and all that, doesn’t have a disaster recovery plan. Most people who take backups store those backups beside the original computers, and never check to see whether the files can be restored.”

I think the real lesson is that it doesn’t matter how many disasters happen, people don’t do this prophylactic stuff. Preparation just doesn’t come naturally to us. I’ve lost computers and files, several times, and still don’t have a sound backup regime.

iPad 2

* Steve Jobs was there
* post-PC products
* Nat hates iPads
* what’s new

Released today in SF. Steve Jobs himself showed up for the launch, both to reassure investors and ensure it was media-worthy. The interesting bit for me is how Apple’s talking about it: “post-PC products”. Most of their revenue comes from iPods, iPhones, and iPads. If you think of Apple as a computer company, you’re on the wrong track.

The launch ended with a discussion of what post-PC means. Their competitors chased iPods and iPhones as though they were PCs, turning out ugly function-overloaded clunkware. Now they’re doing the same with iPads. Nothing has matched the designed beauty of the i-products.

Here’s what Steve Jobs said:

This is worth repeating. It’s in Apple’s DNA that technology is not enough. It’s tech married with the liberal arts and the humanities. Nowhere is that more true than in the post-PC products. Our competitors are looking at this like it’s the next PC market. That is not the right approach to this. These are pos-PC devices that need to be easier to use than a PC, more intuitive. The hardware and software need to intertwine more than they do on a PC. We think we’re on the right path with this.

My iPad gives me the squirts — bought it to see what it was like, and it’s like a designer prison. Beautiful but you’re not in control. Apple decide what apps you can install, and getting files onto and off the iPad is a bloody nightmare. Why? Apple engineers are human too, they want to do the same things I do. It’s a pain because the user inconveniences drive us through the iTunes stores to buy our music and movies, and there’s no level playing field for competitors to keep prices down and selection up.

So I won’t be rushing out to get an iPad2. If you’re a fanboy, here’s what you’ve got to look forward to:
- front- and rear- facing cameras. Front is so you can video chat between iPads, or iPads and iPhones, using Apple’s “Face Time” software. Rear is so you can do augmented reality type stuff: see on the screen what the rear-facing camera sees, then overlay information about what’s in that picture.
- faster CPU and graphics. Should feel 2x faster when apps are running, and graphics could be nearly 10x faster. Of course, the lessons we’ve learned from the pre-post-PC world is that if you get a faster computer and the software gets lazier or more ambitious, so it quickly feels just as slow.
- thinner and lighter
- two colours, black and white.

Same price as first-gen iPads. To be released in the US on March 11, NZ on March 25.

Anti-Laser

* what is a laser
* what is an anti-laser
* why do we care?

A laser is a device that emits coherent light. Imagine a room of people doing aerobics. Very hard to jump up and down at the same time: normally one’s just starting while another’s halfway up and the next bloke over is starting to come down and it’s just chaos. The skinny chick is bouncing up and down like a yoyo, the cow, whereas wobbly me is much slower. Coherent light is like the perfect aerobics group: they bounce up and down at the same speed at the same time.

A typical laser would be a bit like a fluorescent light: a tube filled with gas, which gives off light when you put electricity through it. You have mirrors at each end, precisely placed, which encourages the formation of this coherent light. Think of two people, one on each end of a skipping rope. If you’re waving your arms at different speeds, the rope just flaps around because you’re sending different waves down the rope and they’re not reinforcing each other. But when you get it right and the rope starts to swing, you’re reinforcing each other instead of canceling out. That’s what these mirrors do.

There’s an article out in Science magazine about a Yale team who built the *anti* laser. That is, it takes coherent light and makes it disappear. It also uses these precisely-placed mirrors to capture the light, but with a wafer inside that sucks up the light, turning it into heat. This matters because coherent light is the basis of fibre optic cables, which carry phone calls and broadband and all that good stuff around the world. The scientists hope that, eventually, this can make better faster smaller components for this fibre optic technology.

3D Printing

* what is 3d printing?
* what’s it good for?
* schools are buying them. Why?
* NZ connections?

A normal printer, a 2D printer if you will, puts ink on paper. A 3D printer builds physical objects. You send the printer a CAD diagram and, in the most common form, lays down layer after layer of goo and builds up a 3d object. Next time I’m in Wellington, I’ll show you something a friend gave me: the world’s smallest 3d printed violin. It’s a lump of plastic in the shape of a 1.5cm violin, and also a sign of how much sympathy my geek friends have for me.

The violin’s not playable–this 3d printer accumulates layer on layer, so there are some things that it can’t do, like build cavities. You get shaped lumps, not hollows. There are 3d printers that can do this: they deposit two types of goo, one of which can be dissolved without dissolving the other. This lets you make many more shapes.

These 3d printers open new doors for manufacturing. Before, plastic things like cellphone cases, light switches, buttons and knobs … they were all the province of manufacturers and you had to pay for molds and it was a drama to make, basically. Now you can, in your own home, prototype and prototype and try variations and do all that stuff to build new gizmos. It opens product creation and small-run manufacturing to heaps more people.

More mundanely, I’d love to be able to print replacement parts for my gizmos: I keep losing lens caps, battery flaps, all those bits that come off and don’t come back. All you need is the 3D design, and there are stores and libraries of designs being made available on the net on sites like thingiverse.com

You can buy a printer off the shelf. It wasn’t too long ago that they were US$30k each. Now they’re $5,000 and under and this opens up huge new possibilities. A friend was at the learning@schools conference last week and a company was selling them to schools. Their desk was very busy, which means students are soon going to be designing and manufacturing their own products. How awesome is that? The height of my highschool manufacturing was the world’s crappest pencil case.

Kiwi connections are thick and fast. There’s an open source 3d printer called the reprap, which you can build yourself. RepRap == reproduce, rapidly. A kiwi, Vik Olliver, is one of the elder gods of that world — he’s been involved in the project from the get-go. The RepRap is the basis of the MakerBot, which is a commercially-available pre-built (and customized) version of the RepRap. And Kiwi company Ponoko, which focuses on this area of democratized small-run manufacturing, offers 3D printing services–you send them your CAD designs and they’ll send you the objects.

Businesses Vanishing Overseas

* M-Com sold
* it received government $
* are all businesses fleeing once they hit $20M in revenue?

Latest in a line of companies disappearing overseas. I’ve linked to an NBR article about M-Com, which provided software underpinning mobile banking, being sold to its partner. Nobody’s talking about how much it’s for, but they say they’ll keep their 80 Auckland-based jobs in the country. Keeping jobs is a sensitive issue, given that previous tech successes like Navman have moved jobs overseas relatively quickly.

Money and jobs are doubly sensitive because M-Com got a lot of government aid: it started in the University of Auckland incubator, Icehouse. It was funded through the TechNZ program through Foundation for Research, Science and Technology. There have been a few sales overseas recently: Hyperfactory, EMS-Cortex, NextWindow. This leads some in NZ to worry that we aren’t building large companies: that we reach a certain size, then we sell.

I’m on the fence. I think we have to beware of availability bias–that we only talk about what we see, namely the sales. There are big NZ companies that *aren’t* sold, like DataCom. It might be that only 5% of big NZ companies are ever sold internationally, it might be that 95% of big NZ companies are sold internationally — those numbers aren’t available yet. But, of course, fear headlines play into our “woe is us, poor little NZ” mentality.

Isn’t that just like a geek? “I want numbers, not stories!” I’ll go hunting for numbers and let you know what I find.

NOTE: An alert reader wrote to RNZ after the show and pointed out that Moore’s Law is only “eerily accurate” if you ignore the fact that it is restated and revised whenever facts contradict the current predictions. He pointed me to this mythbusting.

Links

Space shuttle computer, Moore’s Law, Exponential growth, The MOST Important Problem You’ll Ever See.
Instapaper; Long form.org.
Webstock.

Moore’s Law and Mobile

I just bought my wife a new mobile phone, and I had to take a moment to boggle. At first it was just at the price, but then it was at what she was getting for that price. The computers we carry around in our mobile phones are more powerful than the one on the Shuttle. My mobile phone has 1000 times the memory and runs 250 times as fast.

There’s a rule of computing, Moore’s Law, about how computers have gotten faster. It’s named after Gordon Moore, the founder of Intel whose chips grace most personal computers today. Moore observed in 1965 that computers were getting faster every year, drew a graph, and declared that speeds double every 18 months.

The amazing thing is that he’s been right. Chip companies have been doubling the speed of their chips every year and a half … for 45 years. That’s astonishing. Some people suggest it’s been a bit of a self-fulfilling prophecy: everyone believes in it, so they all think they’d better develop and release these faster chips before everyone else does.

That kind of doubling isn’t something we humans are good at picturing. We’re good at picturing constant growth: if we grow an inch every year for 45 years, we’ll be 45 inches taller. But if we double in height every eighteen months for 45 years, which means we double 30 times, how tall do you think we’d be?

Let’s figure it out. After one doubling, we’re twice our height. After the next doubling, we’re four times. After the next doubling, we’re eight times. Then it’s 16. Then it’s 32. Then it’s 64. Then it’s 128. So we’re over a hundred times our original height and we’re still only ten and a half years into our forty years of doubling. By year 15 we’re a thousand times our original size. In thirty years we’re a billion.

See what I mean? You were probably nodding and nodding and then went “no wait, what?” It sneaks up on you, this kind of growth. It defeats our intuitions.

When were computers half as fast as they were now? Moore’s Law tells us: eighteen months ago. In mid-2009, the computers you could buy were only half the speed of the computers we’re buying today.

This kind of growth is called “exponential” in the world of mathematics, and it’s mind-buggering, and it’s everywhere. If you look at money growing in a bank, or bacteria growing on a sandwich, you’re looking at exponential growth.

Take bacteria that double every minute and half. You start with one bacterium, 90 seconds later you’ve got two. In another ninety seconds you’ve got four. Each bacterium makes more bacterium, and continues to do even as the new ones make more bacteria, and it just keeps growing.

Let’s say that, at midnight, you’ve finally got a full bottle. At what time was the bottle half-full? It doesn’t matter how big the bottle was, it was half-full a minute and half before it was full. So it was half-full at 11.58 and 30s.

Imagine you were in that bottle, trying to decide when to do something about the bacteria. Would you wait until it was half-full? You’ve only got 90s to solve the problem. A quarter full? Ok, now you’ve got just three minutes to solve it. Exponential growth happens faster than our linear minds can react.

I learned this from a great video I’ve linked to in today’s links, available on the website, called “the MOST important video you’ll ever see”. He says “the greatest shortcoming of the human race is our inability to understand” this kind of exponential growth. He blew my mind multiple times.

Chips aren’t bacteria, they don’t speed up because they breed. Chips speed up when they get smaller, and that’s something people have to work on. A chip is made up of transistors which act as gates, moving electrons around. Theese gates combine to form the basic operations of the chip: making decisions, adding and subtracting, that sort of thing. When someone designs a chip, they’re designing the combination of the gates (because this decides what the chip will do) and then the layout of the gates (which decides its speed). The closer the gates to each other, the faster the chip.

Electricity is just moving electrons. Electrons move at a fixed speed, and the bigger the distance between gates, the longer it takes the electrons to move from one gate to another. The smaller the gap between the gates and the smaller the gates themselves, the more gates you can cram into the same space and so you can do many more operations in the time it takes your electrons to pass through the gates, so you’ve got a faster computer.

Think of a chip like a tub of icecream. You want to put in chocolate chunks: the more, the better. You want them as dense as possible in the icecream. Moore noticed that constant new breakthroughs in science and engineering meant that every eighteen months they were able to put in twice as much chocolate as they could before. Good news if you like ice cream, good news if you like chips. (Chocolate or computer)

Moore’s observation was paralleled by others: the size of disk drives, the power consumption of computers, the amount of information you can transfer over a network, the cost of displays … all these were growing such that things got twice as good every eighteen months.

The amazing thing about Moore’s Law is that it held for so long: he noted it in 1965, and it’s only recently that we’ve stopped getting faster chips in consumer devices. We’ve reached some fundamental limits in the spacing of these things such that any closer together and the pieces interfere with each other–we’re engineering at the scale where the width of atoms matter. There are new technologies that might help (changing how we make the gates), but for now computers like my Mac are getting faster because more chips are being put inside them. I can buy “quad CPU” machines which have four chips instead of one chip that’s four times as fast.

So all this went through my mind as I beheld Jenine’s mighty new phone. Mobile phones exist because of Moore’s Law. It’s only because the ingredients like memory, chips, storage, and displays have dropped in size and price that we can put them together to form a phone. Remember the bricks that stock brokers held to their ears in the late 80s, those early mobile phones were barely mobile, they were so huge. They were also slow: no web browser, no Facebook updates, no games on those phones. All those breakthroughs in what our phones can do are because we’ve gotten faster and smaller computers. All thanks to Moore’s Law.

Instapaper

From the small to the more useful. Let me tell you about something that will make your life easier. There always seems to be more stuff online than I really want. I mean, I can read short things as I find them, but most long things I have to put off for later when I do have time. This happened to so many people that it got its own acronym, “TL;DR” short for “too long; didn’t read”.

I used to solve the problem by keeping tabs or pages open in my web browser, one per thing to read. I’d have zillions of the buggers, and it was hard to keep track of them and then when the browser crashed I’d have lost all the good stuff. And, of course, if I was away from the computer–nothing to read.

The nifty web site Instapaper solves this problem.

With Instapaper, you get a button in your browser that says ‘Read Later’. You hit it, and the page is saved to the Instapaper site. Later, from your web browser or iPad or phone, you can visit the Instapaper site, log in as you, and see all the pages you’ve saved for later.

And, as if that wasn’t convenient enough, the Instapaper site does its best to strip off the advertising and navigation clutter on the page–all those menus to other bits of the site and whatnot. So you get a pure and beautiful article, when you want it.

I read heaps more interesting things now, just because I have something to read on the bus, or the train, or the dunny, or in queue, or …. It’s wonderful. In fact, I’ve begun to seek out good long-form journalism. Instapaper has helped a revival of the form online, and sites like longform.org now exist to help you find good long things to read.

Webstock

The creator of Instapaper, Marco Arment, is in town for the Webstock conference. As am I. He’s speaking, of course. I’m here to soak up the atmosphere, excitement, and wisdom from the speakers.

It’s an annual conference, and it’s *the* web conference in NZ. Beautifully produced, high values, and they always get amazing speakers who aren’t focused so much on the programming but on the people. Web sites are used by people, and the better a web site fits a person the more it’ll be used. There’s some code talk of CSS and HTML and all those acronyms, but I always find the most exciting speakers are those who talk about people.

My highlights this year:

* Marco, because he’s started a few popular things and Instapaper has changed the way I read the web. This has only happened twice, maybe three times, since the web was invented.

* Tom Coates, who is a friend of mine and a brilliant brilliant man. The British … I don’t know, they seem more thoughtful and philosophical about what they do. Tom’s into architecture, social history, philosophy, and he did his degree in Ancient Roman and Greek literature. I learn every time Tom speaks.

* Scott McCloud, whose book Understanding Comics shows you how comics work. He’s a maestro of communication. Given that I am incompetent at drawing, I have a lot to learn …

* Amanda Palmer, a musician who has been using the web to have direct contact with her fans. She fired her label and has gone it alone, making money in imaginative ways. She’s so experimental with the way a musician relates to her fans, it’s absolutely a glimpse in to the future. And she does a wicked cover of Billy Jean.

I’m sure it’ll be the people I *don’t* know about, though, that I’ll learn the most from. Last year they had a performance poet who was absolutely amazing. What did I learn from him? Well, mainly that my wife finds buff 40ish performance poets waaaaay too hot. I gave him crap reviews in the speaker evaluation just for that.

Below are my notes, made as I researched the topics for the 3 February 2011 show. We often depart from the notes, so they’re not a reliable substitute for what aired.

Nat Torkington will cover:
* is Google getting less useful?
* how do we keep something forever?

Links

Many commentators are talking about a decline in the quality of Google’s search results. It’s pretty important given we all use Google.

The BBC Domesday Project is a canary in the coalmine about the longevity of digital media, whose lifetime isn’t long according to The US National Archives (cf the original Domesday Book). Hard drive failure rates. NZ’s National Digital Heritage Archive.

Search and Spam

Ignorance is now a human condition. What do we do? We Google for the answer. Even the phrase “Google for the answer” shows us how important searching the web has become: we have a new verb for it.

And when we say “search the web”, we really do mean “use Google”. There are only three English search sites with any market share: Google, Yahoo!, and Microsoft’s Bing. In the US, Google has 2/3 of the market, Yahoo! has 16%, and Bing a modest 12%. In some markets it’s even more marked: in the UK, Google is 90%. NZ’s numbers are even more polarised: 75% for Google.co.nz, 16% for google.com, 2% for Bing, and a smidge for Yahoo!. That’s a 91% market share for Google.

We don’t know everything on the net. We look at the net through the lens of Google. If we want to know how to fix spouting or make lamb korma or what that computer error message means or what the best hotel in Queenstown is, chances are that we start with Google.

But for some queries like this, the hits in Google aren’t that great. Some are directories: they don’t have the answer, they just take you to a site with the answer. You might ask what’s the problem with that — well, why doesn’t Google just give you the site with the answer? Some are copies of other sites: you don’t get the Wikipedia entry on spouting, you get someone else’s site which contains a copy of that entry. And some are content farms: you don’t get anything useful, just a vague bunch of paragraphs that sounds like it knows something but it’s just saying vague generalities like “lamb korma is an Indian dish of the type curry. Lamb is the main ingredient in lamb korma, and many people report it to be delicious.” Yes, but how do you make one?! You won’t find out there.

What’s all this crap doing in our search results? Making money for those sites. Google doesn’t pay them, you don’t pay them, but advertisers pay those sites because (being at or near the top of Google) they get a heap of traffic. Many people will have gone to the top result in Google, looked at the page and thought “hmm, don’t see my answer here” but seen an ad on that page that looks promising and clicked on *that*. Those clicks make money for the spam sites.

What’s Google doing about this? They can’t tackle it one search at a time — there billions of people searching with Google and, well, fewer than billions of people working for Google. So they tweak their algorithms, their secret formula for deciding where in the results a site should be. They look for signs that content is copied, or useless, or a directory, and dial down any sites with those signs.

The algorithms are Google’s magic, they’re what make Google’s search useful. Before Google, we had other sites that indexed the web. But they weren’t giving us answers that were as relevant as Google’s (and they didn’t hit on Google’s cash cow advertising business model) so they aren’t with us today.

Preservation

A friend pointed me to an interesting Wikipedia article on a project that the BBC ran, the Domesday Project. Back before the web, in 1986, the wonderful folks at the BBC commemorated the signing of the Domesday book (the first UK census of sorts) in 1086 (William the Conqueror wondering wtf he had just conquered).

They did their own survey, people wrote reminiscences or about social issues, they had maps and graphs and statistical data and even video. This was pretty impressive for computer stuff of the time. And what a time — this is before the web, when we had hobbyist home computers, before Microsoft Word ran on Windows, I was still in short pants. There was no YouTube, there was no Google Maps, there was no Excel to crunch numbers. In short, this was hard work.

They slapped it all on laserdiscs, they needed special hardware to make it work, and it was a magnificent accomplishment, no two ways about it.

Now, fast forward to today. Can we look at this magnificent accomplishment? No, the computers that run it are dead, the laserdiscs are decaying, it’s all turned to crap. There are two computers in a computer history museum that can run it, but for how long? The information on those laserdiscs has vanished. And can we put it on the web? No, the copyright status of all those contributions from people is unknown. Aie.

So, to recap: 900 years after this paper book was compiled, it’s still readable and surviving. Within 25 years, the discs are crumbling, the hardware to read the discs is unreconstructable, and we can’t even put online what we *can* read in order to get help recovering it.

Surely we’ve fixed this? I mean, we live in the age of Google and Facebook and iPads and all that stuff. Um, no.

What about CDs and DVDs? The US National Archives say you can expect them to last 2-5 years even though ads talk about 10-25 years. If this isn’t chilling, I don’t know what is: they say “We recommend testing your media at least every two years to assure your records are still readable.”

Harddrives aren’t much better. If you buy off the shelf hard drives, you’re paying amazingly low prices. You can buy a terabyte hard drive (you could store 250 DVDs in that) for a hundred bucks or so. But it’s like the Warehouse: you got that bargain by compromising on quality. The failure rate of hard drives is scary: you can expect 3% or more to croak within a year.

So if we want to keep these treasures we’re making, whether you’re talking your digital photos or parliamentary email or the latest census, then you can’t just slap ‘em on a hard drive and walk away. You can’t burn ‘em to DVD and walk away. What do you do? You have to keep the information alive: you have lots of copies, and when one dies you replace it from one of the other copies.

That’s how the National Digital Heritage Archive, a project in the National Library, works. It’s the library’s solution to the problem of preserving the digital books and New Zealand web forever. There aren’t a lot of projects like this around the world, and we’re one of the few tackling it. Hopefully, in 25 years time, people won’t be grizzling that the web record of Nine to Noon’s New Technology slot is unreadable …

(I’m involved with the National Library, but not in this project.)

Hello everyone. Thank you for the kind introduction, and thank you to the New Zealand Computer Society for having me here.

Let’s get nostalgic for a bit. It’s the 50th anniversary, we can afford to be nostalgic a little.

Whose first computer was a mainframe? Whose was a mini? Whose first experience was through punched cards? Who had a microcomputer, like a BBC Micro or Spectrum? Whose first was a PC?

My first computer was a Commodore 64. 64K of RAM, not enough colours, great programmable audio, built-in BASIC …. The reset switch was a paperclip across two of the terminals in the exposed cartridge port. That was where I first played text adventure games …. Loved it.

You’ve probably got warm feelings about your first computer too. It’s like a first love, but you never break up. It won’t burn your t-shirts and cut your face out of photos. It might not boot up any more, but who here doesn’t have that problem sometimes?

My C64 lead to a PC and then into Victoria University’s Computer Science degree starting in 1990, where I got to explore a VMS box and then a shiny new SGI Unix machine, and to download Linux (my verdict was “what a pain to get installed!”).

In my time at Vic, on those VMS and Unix boxes, I got to play with the Internet at a time when most people hadn’t heard of it. You young folks here, there was a time when most people hadn’t heard of the Internet. We had text games to telnet into, programs to download, and Usenet to post messages to.

I was one of the first New Zealand users of the web when it was released, and I successfully convinced the IT department to let me build the shiny new “Campus Wide Information System” on the web instead of on gopher. Lots of fun, discovering all sorts of things along the way:
– the anonymity of the net is great–nobody knows you’re a nineteen year-old from New Zealand, not even Tim Berners-Lee and Marc Andreessen.
– you do not want to be the sole conduit for an entire organization’s web site
– but at the same time, it’s hellish to teach people how to put things onto the web (particularly if “export from Microsoft Word using hard-coded styles” is your markup technology)
– no matter how much effort you’re poured into your website, no matter how many new things you’ve done, it will always be described as “shit”

Fortunately this has all changed. *cough*

From there I went to America, worked for a startup, wrote a book (“The Perl Cookbook”), got married (on a MUD), helped along the Perl project, and worked for O’Reilly Media. They make those books with animals on the covers. Their basic business is talk to a lot of people, find the weak signals that show something big is coming, then have books and conferences ready when the masses discover the trend. They did this with the web, with open source, with web services, with web 2.0, and now with the Gov 2.0 open government work.

At O’Reilly, I was running some of their conferences. The bit I still love most about conferences is that moment when, for the first time, a group forms–people who didn’t realize they had something in common get together.

And this is where that “on the Internet, nobody knows you’re a dog” cartoon comes out. Yes, on the net nobody knows you’re a dog. But when you get everyone together in person at a conference, as you do here, … well, to lapse into lolcat: different dog is different.

Let me explicit about this: we’re at a sausagefest. And this is not a reflection of the fine people at NZCS. Our industry is a sausagefest, far more men than women. Women are between 10-30% of CS classes at university, depending on who you talk to and what level you’re measuring. But don’t worry, if you’re in the software business then you’re alright: in open source, it’s 1.5%.

Right now, this can go two ways. One way is to ask why this is so, who’s to blame. This is the path where we all get defensive, fold our arms, and say “stop attacking me, I’m not an asshole, this isn’t my fault, I didn’t set out to build or perpetuate a misogynist class structure, this is bullshit, women can choose and they choose not to work in computing, I don’t see there’s a problem at all, Torkington’s a jerk, what a disaster”.

I don’t like the path of blame. Blame is an attack, and attacks provoke defense, and defense short-circuits rational thought and the fight-or-flight reflex kicks in, and we’re not any better off.

So rather than have you all thinking about blame and looking for victims and oppressors, I want you to forget that crap. That’s solving a headache with an AK-47. Let’s agree not to see this as a finger-pointing battle.

I’m going to let you in on two related secrets: gender is just the tip of the iceberg, and this is a *huge* fucking opportunity.

Any of you know the Myers-Briggs tests? I think they’re science-infused astrology but there are plenty of people who swear by them. And if you go to your developers and you give them the Myers-Briggs test, you will find something odd: they’re almost all INTJ or INTP. Those two categories account for 5% of the general population, but nearly 100% of the computer programming crowd. And, it just happens, most INTJ and INTPs are men.

There are great efforts afoot to recruit more women into computer science classes at university. To do that, though, you really need to get girls into computing in high school. Given how grunty teenage boys can be, you almost have to get the girls into computing in primary school (back when they tend to have better textual and conceptual skills than boys) so they’ll do it at high school.

But there are some interesting signs that the things you have to do to get women into computing is how you get more *people* into computing. That is, the things that drive away women are driving away shiploads of men too. Let me say this clearly:

If we change our culture so we attract more *people*, we’ll get more men and more women coming in.

In open source software we like to pat ourselves on the back that we’ve built a meritocracy. We hide behind this, pretending it’s impersonal. The best coders thrive, the weak wither away, and it’s as Darwin intended. This is often treated as license to be an asshole. The culture this builds is poisonous to those who don’t look, sound, or act like the folks who are already programmers. I can see the ComputerWorld journalist scribbling away furiously, so I want to make it clear that I’m not saying all programmers, or even all open source programmers are assholes, I’m exaggerating for effect.

But many programmers and many projects and many teachers and many classes aren’t welcoming to beginner programmers. This isn’t universally the case–there are projects showing us how to do it.

One of my favorite examples of this is Dreamwidth. It’s an open source project for blogging software, a fork of LiveJournal, serving over 200,000 users. Thirty of their forty developers are women. They do this by being friendly: they encourage new contributors (“little devils”). They work hard at being welcoming, they don’t assume people will feel confident enough to contribute and take the usual curt feedback.

Deep down, I had always assumed coding required this kind of special aptitude, something that I just didn’t have and never would. It lost its forbidding mystique when I learned that people I had assumed to be super-coders (surely born with keyboard attached!) had only started training a year ago. People without any prior experience! Women! Like me! Jesus! It’s like a barrier broke down in my mind.

But very few of these Dreamwidth developers are typical programmers. By changing the culture, they attracted more *people* to being programmers. And this is our huge opportunity: to get more people into programming.

I volunteered in our local school, teaching 8 year old kids how to program. I got asked “why!?” Parents thought I was turning their kids into 8 year old Bill Gates, not the rich Bill Gates, just the dorky and deeply unsocial Bill Gates.

I think it’s important because the jobs most folks want to have are the jobs that work with information. It’s like driving a car these days–you gotta know how to change the oil, fill the tank, top up the fluids, change a tire. If you don’t, you’ll be hugely less efficient in how you use that car. Same with a computer: all very well to shuffle information in spreadsheets and reports, but if you spend a day doing something that would have taken 5m to code and a second to run … you’re wasting human potential.

How do you teach kids to program? Easy. Use a system called scratch. scratch.mit.edu — very simple to use, drag and drop actions and loops and so on, the pieces connect like Lego, you can’t have a syntax error. I love it, and the kids did too — they built games, stories, toys, demos, even a book review, and girls did better than boys.

So it’s not that we don’t have programming tools suitable for 8 year olds, it’s not that girls can’t do it, it’s not that they don’t want to do it. We need to do a few things:
– teach the teachers,
– change the culture so we don’t drive them off when they’re ready to program socially,
– make them love it.

Loving the field is more than knowing how program. It’s the difference between a day job and a passion. Some folks are motivated by abstract problems, intellectual challenges. These are the people who have historically populated our industry.

But we’re realizing now that these people aren’t the only people we need. Silicon Valley is littered with the smoldering wrecks of companies founded by technical people. For every Google, there’s a thousand anonymous failures.

Because we don’t distinguish clearly between technology and industry. We treat the two as the same, but they’re not. And this comes to the heart of the topic today, innovation.

Invention is not innovation. Invention is discovering something new, doing something for the first time. It’s the thrill of exploration, of research, of the unknown. It’s technical. It’s scientific. It’s often solitary.

It’s not innovation.

Innovation is building a new business or product line. It’s sales, marketing, channels, cash-flow management, interaction design, user experience, iteration on product, new markets, making the first sale, ….

Research is proving something can be done, once, no matter how long it takes. Business is doing something that people want, consistently and repeatedly, for profit. These are very different.

We need people with these skills in our business. In our industry.

Paul Graham, early Lisp guy, wrote the code that became Yahoo! Stores, now runs an incubator called the Y Combinator, he was asked the secret to a good career. He said you have two paths to greatness: you pick one topic and go deep, drilling down into the details until you’re the best; or you pick two topics and occupy the intersection. There are many more intersections, and there’s less competition so it’s easier to be great at an intersection.

How do we get people into the industry. There’s no single magic bullet–if there were, we’d have found it by now. I think we need to do two things better:

1) Talk about the exciting big possibilities in our field. Google Goggles, when everyone goes “wow!” … I saw a device for shipping audio lessons around famers in Africa, where you can record your own advice and share the lessons by sneakernet. Changing people’s lives, making science fiction real, that’s the exciting stuff.

2) Offer people the turning points that let them turn towards computers. We’ve built a great culture that turns them *away*, let’s get them in.

I reached out to some of my friends from my time in America, people who had become successful. What were the opportunities that they had that got them into the field and let them be successful?

There were themes. First, it was getting your first computer. Full-time immersion in the tool makes you better at it. Part-time, shared access … less so.

Craig Nevill-Manning clipped garlic when he was 14 to get his computer.

Chad Dickerson worked at a small regional newspaper by night, but his employer had an amazing scheme where every employee was allowed to buy a computer with the company discount, all interest-free. He was making $7/h and was able to buy a $3500+ computer.

Brenda Wallace, a great Wellington coder, got her start when her Dad went to Australia in 1980 and came back with a TRS80.

Caterina Fake, cofounder of Flickr, also had a TRS80 when she was 10. She didn’t start off going down the computer industry path, but when her life made that possible in 1995 (“I’m broke, I’m in San Francisco, I’ve seen the Internet and know my way around programming computers, it’s ridiculously easy to get a job as a web developer”) she taught herself, entered the industry, and was away.

This is why I love projects like OLPC that get laptops into the hands of kids. The good folks at Point England School are inspirational: they’re working on a project to get laptops and connectivity into the hands of their students and their community. They know it’s not just hardware now, it’s connectivity that matters too.

Second, it was someone taking a chance on them, believing in them.

Danah Boyd, who is now knows more about social use of computers by teens than anybody else, and has a PhD in the topic to boot, she had a great mentor at Brown University, someone who talked her off the quitting ledge, steered her into computer science and humanities.

Gina Bianchini, who was CEO of Ning and of other companies, acknowledged her two cofounders: Mark Kvamme and then Marc Andreessen.

Mitchell Baker, who runs Mozilla and was the “chief lizard wrangler” who got the source code to Netscape out of AOL and into the hands of developers who could eventually build Firefox … Jim Barksdale who took a risk. “I think he knew I had a bunch of capabilities, and Mozilla was a quirky thing that i would fit with”

Andy Baio, who sold Upcoming to Yahoo! and whose blog is hugely influential and who was CTO of Kickstarter,

2000: With *very* limited experience, a Slashdot-loving geek took a chance on me and I was hired to code Perl at a web design firm in L.A, changing the trajectory of my career forever.  In the next week, I taught myself a crash course with a copy of the Perl Cookbook and the Camel book, and fell in love.

Danese Cooper, the CTO for the Wikimedia Foundation which runs Wikipedia, gave props to the person who offered me a job at Apple back when you could get “on-the-job” training to transition from “French Major” to “Technologist”

This is giving someone a go because you see something in them. When I organized conferences, I organized sausagefests because I chose people whom I knew were good. Which meant my pool was limited to those whom I knew. Which meant, because we’re all like this, they were people like me. When I decided I wanted to change that, I brought women into the program committee, trusted their recommendations, and eventually turned the chair of the conference over to a woman.

There was talk yesterday about internships. I love the Summer of Tech they run in Wellington. The numbers are amazing: 2/3 of students go on to jobs at the company they interned with. The common thread between internships, computers for kids, talking about the good things of our industry, and giving people opportunities is to create on-ramps.

I challenge you to think about how you personally can create on-ramps.

Can you volunteer to teach programming or entrepreneurship at a local school? I’m happy to tell you what I did, it’s dead simple and worked well. There are lots of other successful programs you can emulate.

Can you set up a cheap hardware plan for your school, maybe identify the cheap netbooks, loan plan, etc.?

Can you change how you respond in your workplace to newbies and silly questions?

Can you be conscious of your human unconscious bias towards people like you, and you reach out to make change–ask someone who’s not like you for recommendations or advice?

Can you take a chance on someone?

Can you build a mentoring and supportive environment for people who would ordinarily be bounced off?

There are things we can do now to change the demographics of ICT innovation in New Zealand. Some of those things are the big national programs that are hard work and take a long time. But many of them start with individuals. We can give many more people the same experience of their first computer, and a rich and rewarding career in our industry.

Gandhi: be the change in the world you want to see.

[thanks to Brenda, Skud, Val, and Allison for their patient and selfless work with me over the years. Anything good in my talk will have come from them, anything bogus will be mine alone.]

Links:

• Network Neutrality: Save The Internet, U.S. Court Curbs FCC Authority on Web Traffic, Wired’s coverage, the End-to-End Principle.
• iPad: iPad, Nine Banned iPhone Apps, iPad censors SPERM, But Will It Blend?

Network Neutrality

Is your ISP allowed to mess with your Internet traffic? We pay them to connect to the Internet, but in America they want to do more. They want the ability to treat some traffic different from others. For example, to say to Google and Microsoft “who will pay me to make their web traffic go faster to our customers?”

Now, that runs contrary to what we expect. We expect best efforts unprioritised service, where our ISP delivers everything as fast as they can. The only way they could make Google’s traffic faster is if they made everyone else’s go slower. It’s a form of extortion.

This also breaks an important principle of the Internet, called the “end-to-end principle”. The intelligent decision-making cleverness in the Internet always sits on the ends of the Internet and the center of the Internet is a “dumb pipe”. It just delivers stuff. So the same dumb pipe does the same dumb things regardless of whether you send email to Radio New Zealand, if you visit the Radio New Zealand web site, or if you have a Skype call with someone at Radio New Zealand.

Think for a second of what it would be like to have all the smarts in the network. Only the companies who run the network could innovate. That means we’d have had to wait for Telecom or Orcon to develop the web or Skype. Or, more likely, for someone else to develop it and then slowly and expensively sell it to the telecommunications companies who’d then badly and expensively sell it to us.

Fortunately that’s not how it worked. The clever folks who built Skype (a Swede and a Dane) were able to do so without having to talk to the telecommunications companies. Because, really, do you think a phone company would ever have developed something that let you talk long-distance for free? The Skype folks only had to get Skype running on two computers, both of which they controlled, and never had to speak to the Swedish or Danish equivalent of Telecom.

Naturally, the ISPs around the world don’t like this, particularly in America. They don’t like being sidelined while other people (Skype, Google, etc.) make lots of the money. So they came up with an idea for a shakedown: let’s make the buggers pay! It’s basic extortion tactics: we’ll make your traffic go slow unless you pay up. It almost sounds right, until you realise that they want to charge BOTH sides of everything you do on the Internet. So when I’m in Colorado, I pay my Colorado ISP to connect to the Internet. Google’s paid their ISP to connect to the Internet. But my ISP wants Google to pay it as well! It’s like the post office charging the sender AND recipient of the letter.

There are other bad scenarios possible, too, laid out in the Wired article I linked to:
A broadband company could, for instance, ink a deal with Microsoft to transfer all attempts to reach Google.com to Bing.com. The only recourse a user would have, under the ruling, would be to switch to a different provider — assuming, of course, they had an alternative to switch to.

Companies can also now prohibit you from using a wireless router you bought at the store, forcing you to use one they rent out — just as they do with cable boxes. They could also decide to charge you a fee every time you upgrade your computer, or even block you from using certain models, just as the nation’s mobile phone carriers do today.

When computer companies got wind of this, they and user associations started a campaign for “network neutrality” — the idea that ISPs should stick to moving my Internet traffic around as fast as they can and not shake down the people I connect to.

There’s a US Government Agency, the Federal Communications Commission, which was established to control radio stations. They’ve extended their control to TV as well: it’s the FCC that fines TV companies when they air “wardrobe malfunctions” or swear words on free-to-air TV. They presumed they had the right to regulate the Internet, too, and laid down some basic “thou shalt not” rules for the US companies that provide Internet access. These rules were challenged by a US ISP, Comcast, which said basically “no, you don’t have the right to regulate Internet access”. A US court just ruled that the FCC didn’t make a good case that it could.

What does that mean for us in NZ? Nothing immediately, as we have our own regulatory agency (the Commerce Commission) but you can bet your last cent that our ISPs are watching with interest what happens in America. The possibility of double-charging will be very interesting to them. So we Internet folks are closely watching the US to see what’ll happen.

What will happen? At the moment the FCC is figuring out how to proceed–the court didn’t say “you can’t regulate”, just “you didn’t make a good argument that you can” and it hinted at some lines of reasoning that might be better. The best option is for the US Congress (the US equivalent of Parliament) to give the agency the right to regulate Internet access. The worst is for the FCC to classify the Internet as a “telecommunications service”. At the moment it’s an “information service” which puts it into the private market and out of the reach of the regulators.

Why can’t we just let the market decide? The consumer is the one who would have to change ISPs if the market were to speak about this, but remember that it’s not the consumer that’s being shaken down–it’s the Google or Microsoft or Radio New Zealand. It’s also not clear that a new neutral ISP could be competitive, as the gouging ISPs are all integrated Internet-voice-cable TV companies with huge scale letting them offer lower prices.

So I’ll be watching this one and hoping NZ telcos don’t decide to be “world leaders” in interfering with their customers’ service.

iPad, Apple, and Trading Power for Convenience

Apple released its latest shiny this week, the iPad. It’s a bigger iPhone, nine inches by seven — a bit smaller than A4 in size but larger than a paperback. Light, with Wifi and mobile Internet, GPS, accelerometer and all manner of other goodness. No keyboard, it’s a touch-sensitive screen so if you want to type something then it shows you a picture of a keyboard and you type onto the picture. Yes, it’s weird, but it has worked on the iPhone and it apparently works better on the iPad. Prices start at USD500, and darn near everyone who has one is raving about it. Not for sale yet in New Zealand officially, but you can already pick them up on TradeMe.

But that’s the problem with Apple gear, isn’t it? There’s a set of people, a cult if you will, who love everything Apple and who will queue to buy the first of whatever new gadget they make. And make no bones about it, Apple make good products–they control every aspect and make sure it’s beautiful, elegant, functional, and desirable. But there’s a difference between good and good for you, and I’d like to talk about that for a bit.

Apple has a control complex. They’re not fond of the idea that someone else makes money off their computer hardware and software, and so they want to “clip the ticket” as much as possible. Think of the difference between the iPad and a MacBook computer.

On my Macbook, I can install any software I want. If someone writes software for the Mac, they put it up on their web site. I download and install it, a simple process, and then I can run it.

On an iPad or iPhone, I don’t have that freedom. Apple have the machine locked down. The only way you can get a program on an iPad or iPhone is if you buy it through the Apple “app” store. (app is short for “application”, a geeky word for “program”). If I write a program and want to sell it it you, you can’t get it unless you buy it through Apple’s store. If Apple won’t let you buy a program, you can’t get it on your iPad or iPhone.

But why would Apple do that? After all, they make money every time I buy a program! But there are all sorts of programs that Apple won’t allow. They’re very keen to maintain their devices as “family-friendly”, so anything to do with sex or profanity is in an “adult” section of the store. That’s okay, but sometimes they go a little far–the word “sperm” was censored in a description of a Moby Dick ebook app, which was apparently about the hunt for an s-star-star-star-m whale.

More seriously, Apple’s store won’t carry any apps that duplicate functionality in the device. Want a different web browser? Tough. Want Google Voice services instead of your phone’s built-in calling and texting? Want someone else’s maps? Tough. Want a BitTorrent app? Tough. Want something overtly political and partisan? Tough. I’ve linked to a site that lists a bunch of apps you’ll never see on the iPhone.

Apple’s approval process is also in-depth and comes with no guarantees of timeframe. So if I write a program that you download, then I discover a bug and fix it, it could be weeks before that fixed version is on the site for you to download. In the meantime, you’re cursing me for not fixing your program.

If all this sounds familiar, it’s because it’s the same scenario we talked about with network neutrality. One company wants to sit in the middle and be a gatekeeper. In the case of ISPs, they have your connection to the Internet and want to tell you what you can and can’t do with it. In the case of Apple, they have the device and they want to tell you what you can and can’t run on it. The only difference is that there’s a strongly competitive market for smartphones and handheld gadget, and no government regulator.

How’s this going to play out? In the short term, the iPad and iPhone will be successful. But in the long term, there’ll be competitive devices that aren’t locked down the way they are. They might be based on Google’s “Android” technology, it might be something else, but my money is always on fast-breeding innovative mammals when the option is slow-moving pea-brained dinosaurs.

I’ve also linked to a clip of a man from an industrial blender company answering the question “will it blend?”. It’s the only possible antidote to all the sickly iPad cooing that’s over the Internet at the moment. Enjoy!

It seems bizarre at first, but the draft is laid out like a spreadsheet: one article per row and with three columns, one each for the US/Japan version, the EU version, and comments. Inside each sentence square brackets mark the attributed proposed alternatives for language. From this we can tell some very interesting things about the New Zealand position:

• NZ negotiators are keen on the wording “copyright and related rights and trademarks” rather than the US’s catch-all “intellectual property”. Richard Stallman has a well-written article on why “intellectual property” is a dangerous illusion. (Namely, it covers some very different pieces of law with different intents, terms, scope, and applicability)
• NZ negotiators are keen to keep the Copyright Tribunal option open. After Section 92a collapsed last year, the government consulted and has proposed a nuanced and good proposal that balances ease of complaint against risk of false accusation, giving the Copyright Tribunal the ability to hear complaints and award fines of up to $15,000. A 6-month suspension of Internet access and larger fines remain the domain of the courts. The US proposed language is all about “judicial authorities”, so New Zealand has proposed “competent authorities”. This is good–it shows that the government is serious about the Copyright Tribunal part of the new Copyright Bill and is not simply mooting it knowing that it will be overruled by ACTA.
• NZ negotiators are aware of the US desire to turn litigation into a revenue stream. They’ve opposed the US language “in the case of patent infringement, damages adequate to compensate for the infringement shall not be less than a reasonable royalty”, although interestingly NZ only supports this being stricken from the US proposal not from the EU proposal. The EU negotiators’ comments are fascinating: “The EU sticks on the concept that damage compensates all the prejudice but only the prejudice. Neither ‘punitive damages’ nor ‘future prejudice’ is acceptable”.
• NZ negotiators are keen to prevent the situation where someone joins a filesharing network, grabs an album, and is hit with a $100,000 penalty. Their wording supports flexibility when copyright damages are set: the authorities may consider lost profits (as opposed to the US wording shall) and NZ suggested the authorities consider retail price as well. The US wants each country to set up a system of pre-established damages and guidelines for calculating the penalties (oh, say, number of copies times profit we say we would havemade), and give the rightsholder the choice of using that formula instead of letting a judge award penalties. NZ wants this to be optional, not mandatory.
• Pirated or counterfeit items will be removed from sale or distribution, and NZ would also like them to be surrendered to the rightsholder (so Mattel get the knock-off Barbie dolls). The machinery used to manufacture the pirated or counterfeit goods is also forfeited, which NZ raises no objection to. It’s unclear to me whether this applies to computers used in copyright infringements.
• NZ supports deleting the article which says that when you’re found guilty of infringement, your identity and the identity of others involved in the infringement and distribution are turned over to the rightsholder.
• NZ is questioning the scope of the term “online service provider”. As we’ve seen with S92A, the term “provider” might cover cafes, hospitals, employers, apartment building body corporates, families, even sites like Google and TradeMe. Clarity is essential.
• ISP and website liability is a hot topic. Some countries already hold service providers liable for what happens on that service (e.g., Italy’s prosecution of Google executives) while others give safe harbour to such providers. Section 4 says “what we said for the physical world also applies for the online, but countries can place limits on the liability of service providers under certain conditions”. Switzerland wants this optional, NZ wants to know why search engines deserve safe harbour. I hope they got their answer–Google’s programs index billions of web pages and there aren’t enough humans on the Internet to read and pre-qualify pages before they go online.
• There’s an interesting clause that would prevent service provider safe harbours from being made conditional on proactive monitoring. That is to say, a country wouldn’t be able to say “oh sure, you can have safe harbour, but you have to be reading everything your users do and you lose it if you’re not searching all their traffic.” There’s a NZ objection here, but it’s unclear to me whether it’s to the whole provision or just the language.
• NZ is the white knight when it comes to anti-circumvention legislation. The ACTA draft contains proposed text saying that if you make or use a tool that breaks “technical protection measures” (DRM) then you’re breaking the law. The NZ negotiators point out that DRM is out of scope for ACTA, but even if it were in-scope there’s still public domain material locked behind DRMs and breaking such DRM shouldn’t be against the law. The paragraphs are beautiful. I quote them here:
  

  NZ: The paragraphs refer to “adequate legal protection” as well as remedies, which is inconsistent [with] the objective of ACTA to establish standards for the enforcement of intellectual property rights and the ACTA discussion paper. In particular, we note that the discussion paper refers only to parties providing “remedies against circumvention of technological protection measures used by copyright owners and the trafficking of circumvention devices.”

  New Zealand does not support protection being mandated against circumvention of TPMs where the underlying work is not protected by copyright. In particular, we do not support protection against circumvention of access control TPMs because access control is not an exclusive right given to copyright owners.

• There’s an odd section about preserving electronic rights management information. I assume it’s meant to preserve owner and license information, but I’m not really clear on the situations that motivated this section. NZ opposes extending protection of RMIs to cover information about performances or the producer of a phonogram.

On the balance this bit isn’t too bad–New Zealand is a good voice for sanity in the negotiations. I have to qualify my assessment in two ways, though:

1. I’m not a lawyer. I may have misread the complex document. I’m not intimately familiar with the current legislation, so I may have overlooked a situation where the negotiated text will throw out a freedom that we currently have (e.g., format shifting).
2. I haven’t spent a lot of time thinking about how specific technology might interact with the proposed treaty. For example, do I run foul of the Rights Management Information protections if I rip a CD and don’t add in title, composer, etc. information?

This treaty is going to need a lot of close examination from people who can read the legal language and yet who are intimately familiar with the possibilities and opportunities of technology. This is why negotiation in secret is a bad idea–our country won’t benefit from the knowledge of experts until the text is set in stone. We’ll get something that likely has flaws, but we’ll have to approve or reject it “warts and all”.

• Networking is important—often there are other groups doing similar things that are happy to piggy-back on projects or contribute resource. Before you start, understand your resource requirement and allow for growth, especially if updating/collecting info for the community. It’s easy to contact and update for 60 organisations, a lot harder for 3,000.
• Depth of relationship allows for more engagement and vulnerability.
• Keep raising the bar!
• “Personal” rewards from community involvement translates to professional reward and back again.
• Always have a back-up person—don’t be your own single point of failure.
• It can be important to reward people for participating in your online community.
• Go where your community already is, rather than expect them to come to a new ‘community’ that you just set up.
• Forums take 6+ months to establish momentum.
• Wikis suck.
• Comments at the bottom of pages of content fail to engage passive readers.
• Whatever you’re doing—whether it be in the online or offline world—you need to provide an “authentic” experience or voice for your audiences and community.
• You need strong reasons to make building a community worthwhile. It can take a lot of time and resource.
• I like the idea of incentives for users. e.g., points and rewards. For example, in our wiki originally we got a lot of new users to contribute through making the stats viewable. They could view numbers of changes made by users and a top 10. This lead to a competitive environment, especially with the boys. I had forgotten about that so am thinking how we can get that going again. Am interested in Shelley’s “submit a tag”, how that works. A problem we have is meaningless tags.
• I learned that preparation and planning should play a more important role than technology.
• Exposure to a wide range of online communities can teach us a lot about how people interact online.
• We had great success and learned a lot by piloting community interaction with small self-selected groups before trying to interface with the wider community. Benefits: tools are tested and tweaked; people from the pilot are great at kicking the wider community off.
• Be very proactive about responding to criticisms/suggestions by pointing out ways that the commenter/critic can get involved in doing something with their suggestion and solving their problem.
• Why? Social capital; information; value; connections. How? Authentically; where they already hang out; on their terms; multiple (appropriate) platforms. Who? By the community; for the community …
• Go to where your community are already hanging out to engage with them.
• Decentralise your community management by using your community.
• Who the customer is, what they want, what they need is key. Once the purpose is clear, that drives every other decision.
• Do you really need to do this? What will work best for your users? When will you stop if it isn’t working?
• Take-away: you need a community manager; build it and they won’t come!

Thanks to everyone who took the time to write down their advice!