The paradox of ranked-choice voting

Ranked-choice voting is a technical fix to voting problems. But it can often make matters worse.

In ranked-choice voting, aka instant runoff voting, you rank the candidates in order. Then the candidate with the lowest number of first preference votes is eliminated, and people who voted for him have their second preference counted instead.  Keep eliminating candidates until there’s only one left.  The aim is to make sure people don’t worry about “wasting” their vote on a comparatively unpopular candidate.

Proponents of ranked-choice voting generally fall into two camps. The first hopes to get more centrists elected, like the UK’s Liberal Democrats, or California’s Tom Campbell. Another popular reform of this type is open primaries, and centrists will keep coming up with these ideas as long as they can’t get anyone to vote for them.

The second hopes to get more left-wing parties running, like Greens or Socialists. The argument goes that people don’t vote for Greens like Nader because they’re worried about splitting the left-of-centre vote and letting the right in, as in 2000. But with ranked-choice voting, Nader could have run, disaffected Democrats could have indulged themselves with a protest vote for him, and it would have all worked out OK – Nader’s votes would have been redistributed to Gore, letting him win. It’s an attractive position, as it lets you be smugly superior in your purist vote without actually having to face the consequences of eight years of Bush-Cheney.

So how can it make things worse? One example is San Francisco, where it’s combined with public campaign financing to give 16 candidates for the mayor’s race. Mercifully, San Franciscans don’t have to rank order the whole set – they only need choose their top 3. But that gives 16*15*14 = 3360 possible choices, quite enough to induce analysis paralysis among anyone who took the task seriously. The “paradox of choice” says that all these options will give worse results.

And how is the election shaping up? With 16 candidates, all the messages blend into one vague mush of centre-left platitudes – protecting the environment, encouraging sustainable growth and so on. Nobody attacks anyone else, because they want their supporters to put them second or third. It’s San Francisco’s most boring election.

Who benefits from all of this? The same people who always benefit – incumbents and moneyed interests – the only ones who can cut through the chatter. What the reformers have forgotten is that, since Ancient Rome, any election worth anything has been, at base, a contest between rich and poor. Not that the patrician candidate is always worse – the rich didn’t get to be rich by being dummies. But the best way for the rich to win the class war is to deny and obscure its existence, and ranked choice voting is an excellent assistant.

Update: The left-wing SF Bay Guardian notes “Several consultants and election experts [the editor] talked to this week said that [incumbent mayor] Lee would be far more vulnerable in a traditional election. ‘He would lose a runoff against almost any of the top challengers,’ one person said.” and quotes Corey Cook, a political scientist at the University of San Francisco as saying “Ranked-choice voting clearly favors incumbents.”

xkcd and compound interest

Recently, xkcd had a strip saying compound interest is not that great – $1000 with 2% interest for 10 years only grows to $1219.

But if we take the same $1000, and use a more optimistic 4% compounded continuously for 30 years, then we get $3320. That’s an advantage of $1120 over simple interest. What’s going on?

Normally continuous compounding is introduced by thinking about compounding yearly, then monthly, then daily, then taking the limit.

Let’s look at it another way. It’s simple interest, plus the simple interest on the interest, plus the interest on the interest on the interest, … and so on.

Now if you invest $1 and compound it continuously at interest rate r for time t, the simple interest is $rt.

The interest on the interest is rt*rt / 2 (we divide by 2 because we don’t get all of the first $rt at once).

And the interest on the interest on the interest turns out to be rt*rt*rt / (2 *3). Put it all together, and we get

1 + rt + (rt)^2/2! + (rt)^3/3! + … = exp(rt)

Now, for our 30 years at 4%, rt = 1.2
The simple interest is $1000 * rt = $1200
The interest on the interest is $1000 * (rt)^2/2! = $720
The interest on the interest on the interest is $288
The interest on the interest on the interest on the interest is $86
The interest on the interest on the interest on the interest on the interest is $21
The interest on the interest on the interest on the interest on the interest on the interest is $4.

And we’re still not done!

We can go one more level and get another $0.71. What does interest on the interest on the interest on the interest on the interest on the interest on the interest even mean? It means 71 cents.

That seems pretty amazing to me. If you don’t think so, I guess you don’t like money.

And it also explains more intuitively why compounding is so unimpressive for 10 years at 2%. In that case, rt = 0.2, and the successive interest terms fall to 0 very quickly.

But forget about the large amounts of money. We’re talking about either taking infinitesimal limits, or taking an infinite sum of interest on interest on interest on interest. That’s hippy stoner talk, but that’s how the modern financial system works. What could be more esoteric than infinity, or more mundane than a banker? And yet, there they are, side by side.

Einstein might or might not have said that compound interest was the most powerful force in the universe. But even if it’s not magical, it’s certainly mysterious and astonishing.

2001-2011: Technology under Presidents Gore and Romney

Everyone remembers September 11th, 2001, when President Gore announced that America had been attacked. But few people knew at the time how drastically technology would change over the next ten years.

Gore had long been an environmentalist, but in the months following 9/11, he used his political capital to push through a range of new programs: a carbon tax combined with personal CO2 allowances, subsidies for energy efficiency and mass transit, and unprecedented investments in energy infrastructure, research, and development. This led to a seemingly endless demand for data analysts, physicists, computer scientists and engineers in companies funded by the Department of Energy, causing Jeff Hammerbacher, CEO of AdMath, to remark “Everyone is trying to make energy more cheaply and cleanly. That sucks if you want to do anything else.”  And it also led to many startups in that ecosystem, such as Zuckerberg’s FaceGrid, which lets people trade carbon credits with their friends and neighbors, and the rapidly growing Blipper, which generates a 140 character alert whenever there is a “blip” in power consumption. At first Blipper was parodied as just generating “using microwave for lunch” blips, but it now has a viable revenue stream with sponsored blips from major automakers such as Ford, Toyota, Tesla or GCars.

While Gore’s changes were dramatic, we should note that he was really just in the right place at the right time. The problems of global warming, dependence on foreign oil, and America’s crumbling infrastructure were so large that even if Bush had won, he would surely have implemented similar programs. Gore was also mocked for various gaffes, such as claiming that he invented FaceGrid, and that “his” levees and mangroves saved New Orleans.

After Romney defeated Hillary Clinton in 2008, he largely maintained the Gore agenda, but added a new project: healthcare reform, led by his popular Democratic Secretary of Health and Human Services Barack Obama. “Obamacare”, as it became known, built on Romney’s earlier work in Massachusetts, and cost reduction informed by data analysis and statistical modeling was a major part of it, from the NetDocs prize onwards.

What of the future? Vice President Bush is pushing forward his education initiative to find and reward the best teachers and fire the worst. In his words, “Those who can, can do. Those who can’t, can’t teach.”  Generous merit pay awards for math and science teachers have been unpopular with unions, but they are attracting talented graduates to the field. The administration claims that this and immigration reform will be the bases for continued American technological dominance into the next generation.

Is Google+ solving the wrong problem?

Traffic seems pretty light on Google+, even in my circle of technoliterate Silicon Valley friends. Why? Partly it’s because G+ is solving the wrong problem.

The origins of G+’s approach can be found in a thoughtful presentation by Paul Adams – The real life social network v2. In it, he argues that Facebook’s concept of friendship is not nuanced enough, using the example of Debbie, a swimming teacher who’s friends with her 10-year old students, and also some guys who work in a gay bar and post risqué photos. The proposed solution is G+ circles, so you can control exactly who sees what.

There’s good news and bad news about this approach. The bad news is that it doesn’t work. The crazy will always out, and the Internet will always find a way to get it to everyone. One of my friends started up a blog to present a staid and professional image, but within a few months the intelligent, entertaining and highly non-pc rants had slipped in. And if our teacher friend is hanging out with a fast crowd, you can bet that will get out to her class somehow. Plus, once you start thinking about detailed control, you never stop. Who should be able to see who you’re friends with? What time of day you posted, and who commented when? That way lies madness.

But the good news is that it doesn’t matter. Only a few people follow your status updates, and they have figured out your weird views or predilections already, maybe before you have. Are you worried about posting too much about your child / pet / marathon / claim to the throne of France? Don’t be. Your Facebook friends can handle hearing about you in a different context, and part of the fun of FB is seeing that, and occasionally seeing everyone else’s friends. It’s part of the appeal of weddings, too.  G+ ends up like a season of “Friends” without any guest stars.  We already have means for keeping in touch with cohesive circles of friends – they’re called dinner parties, game nights, pub crawls and email.

What about saying the wrong thing to some of the group? Miscommunications on FB are like crashes in NASCAR – fun to watch, part of the sport, and don’t often hurt.

Isn’t FB an evil profit-maximizing corporation? Sure, but the profit-maximizing part is the key. FB’s modus operandi has always been to push privacy concerns to the edge, wait for Diaspora or G+ to do free user-base testing for them, and then pull back. As I’ve said before, billion dollar companies that are paying any kind of attention don’t let themselves be dismantled. Privacy scares on FB won’t stop people from using it any more than urban legends about hooked-hand killers will stop teens from making out.

This is not to say that everything’s perfect with FB. People in social relationships with tricky dynamics like teacher / student may not want to friend their juniors.  FB could do better too – it’s very easy to hide someone and not see their posts by default, less easy to set it up so they don’t see yours. But the required tweaks would be minor – maybe a secret “courtesy friend” status so you could accept a friend request to avoid offense, but not have to share anything. And as soon as Google+ works out the kinks, Zuckerberg will get right on it.

Why economists get no respect

Everyone from Asterix to Richard Nixon makes fun of economists. Why does the discipline get no respect?

Economics, or, to be more precise, macroeconomics, has a unique combination of features. First, it touches on vested interests. Second, it deals with commonplace things that it’s easy to have an opinion about, like jobs and money. Many subjects have one or the other. Medicine has the first, and philosophy the second, but the combination means that discussion ends up dominated by “common sense” views that really express a regressive agenda. Common sense often fails because of another feature of economics – the failure of composition. What’s good for one person may be bad at the national level, or vice versa. As one example, if you have a billion dollars, then you’re rich. If everyone has a billion dollars, then you’re Zimbabwe.

Hacks and cranks aside, economists have figured some things out over time. Free trade and free markets are good. A gold standard is bad. Governments should spend in slumps and save in booms. Nothing that rises to the level of the Pythagorean theorem, but smart people from the Romans to the Soviets to the inventors of Bitcoin to the current administration have failed to fully grasp these. Now of course these are broad generalizations – free markets in healthcare are very bad, for example. But the basic ideas are sound.

It’s a pretty short list of good ideas, though, and there’s a limit to how much can be said about them. So economists have added lots of bathwater to the baby, in attempts to get tenure (see rational expectations theory) or play up to generous patrons (see libertarianism). Any argument, no matter how ridiculous or mendacious, will have at least one economist ready to propose it for the right price. And that’s really why economists get no respect; they don’t respect themselves.

Anecdata

Quantitative people usually treat anecdotes with disdain, but more for sociological than statistical reasons.

There’s a saying that “the plural of anecdote is not data”. Funnily enough, it started out without the not, which makes more sense, for what is an anecdote if not a datum? But it was important for economists to distinguish their ideas from the just-so stories of other social scientists, and so the not stuck.

There is something to the hostility, though. One datum should generally not affect your estimates by much. So if you’re trying to work out the national inflation rate, it’s not worth paying attention to your cousin’s complaints about how they’re paying more for broccoli now. Just let the government statisticians do their job of aggregating the millions of price points.

The problem is that most quantities you’re interested in aren’t like that. Want to know if we’re going into a recession? You could wait for the NBER, which will tell you a year after it’s happened. Or you could ask a friend in a volatile industry like consulting if she’s finding work. Mathematically, if what you want isn’t being collected at the right time or in the right level of detail, then one point can and should make a big difference to your estimates.

Using these anecdata is tricky, particularly if you take Bayesianism seriously. For just one example, how do you deal with selection bias, where you’ll hear about successes more than failures? But there’s no reason not to try.

Google+ and the four card problem

Google came late to social, so all the good words were taken. Facebook has friend, Twitter has follow, and LinkedIn has connect. Google+ has the less intuitive circle. A minor terminological difference? Maybe not, if you think about the four card problem.

The four card problem is a logic puzzle. You have four cards, each of which has a number on one side and a colour on another. You have to check that if the number is even, then the colour is green. The cards show 3, 8, red, and green. Which cards do you need to turn over?

Very few people get the right answer (8 and red) but you can restate the problem in equivalent terms. You’re a police officer, and need to check that the alcohol rules are being followed. Each card represents a person, with their age on one side, and what they’re drinking on the other. The cards show 14, 35, a coke, and a beer. Everyone gets this – you check the 14 year old and the beer drinker.

The point is that context matters. We’re very good at checking if someone’s cheating, and have similarly good understanding about social relationships in general. The terminology for the big 3 social sites give the functionality away. We know that friendship is symmetric (if I’m your friend, then you’re my friend) and not transitive (friends of friends are not necessarily your friends). We know that following (aka stalking) is not symmetric or transitive. And we know that connecting is symmetric, and (thanks to Six Degrees of Separation) it has a transitive aspect. No further explanation is necessary.

What about Google+? What does it mean to add someone to one of your circles? Do you get to see their stuff? Do they see yours? The name doesn’t tell you, and may even mislead – a circles of friends commonly being a group that’s all connected to each other. Does this doom the project? Probably not, but it doesn’t bode well to have your most basic operation be confusing.

Is there a higher education bubble? Let’s ask Kate Middleton!

Lots of people have said that there’s a bubble in higher education.  But consider Kate Middleton, now England’s future Queen.  However much she spent to go to St. Andrews and meet William, it was well worth it. Even if she wasn’t bound to marry him, there’s about 5,000 female students at St. Andrews, and the Queen is worth about $400 million, giving an expected value of $400 million / 5000 = $80,000. Not too bad.

Perhaps you object. Maybe you’re aspiring to something other than an M.R.S. degree.  Consider exhibits B, C, D, and E – the Winklevoss twins, Eduardo Saverin, and Jeff Hammerbacher.  All of them ended up multi-millionaires in part because of running into Zuckerberg at Harvard.  Hammerbacher was a Harvard Math student and eventually became head of Facebook’s data analysis team.  Let’s say he had a 1 / 1000 chance of running into Z. and making $100 mill. That’s $100k expected value – maybe not the full cost of a Harvard education, but more than a rounding error. (To say nothing of Zuckerberg himself, as Facebook got a lot of its popularity by having Harvard students as its first adopters).

Still not convinced?  Exhibits F and G – Google’s Brin and Page, who ran into each other at Stanford. I can’t be bothered to look up how many Stanford CS PhD students there are per year, or how much Google is worth, but I bet if you divide the one by the other, you get a pretty large number.

The point is that college is valuable because smart and/or rich people go to college. The price has been going up, but so have the potential rewards to being smart and/or rich. If nothing else, higher education solves a valuable coordination problem of getting all these people to meet each other.

But wait, you say. Isn’t that exactly what a bubble is?  Something that’s valuable just because everyone else is doing it?  Well, Harvard and other schools thought of that, hence legacy admissions and scholarships to ensure that the best-connected and brightest will always find it advantageous to go there. With this core secure, everyone else follows.

Why talk of a bubble now?  Partly because prices have risen sharply, partly because of that other unfortunate bubble, but mostly because Americans have started to believe their own propaganda – that the U.S. is a meritocracy and anyone can make it with just a bright idea. If you believe that then why go to college?  Maybe you can learn everything you need on the web. University presidents are not foolish enough to believe this (they spend too much time meeting donors), but rather than say what’s really going on, they blather about life experience and other hogwash, which doesn’t fool anyone.

Is higher education a bubble?  No, but it is a lottery. And like the rest of society, the tickets keep getting more expensive, and the jackpots keep getting bigger.

Financial planning for hackers

There are lots of guides on financial planning.  But how should they change for hackers?  Here’s 5 rule tweaks to follow:

1. Buy lattes
2. Don’t buy a house
3. Buy more bonds
4. Keep a larger cash reserve
5. Don’t try to beat the market, even if you can

Number 1: Buy lattes.

Every financial planning book has some example where if you cut out buying a latte each day, then you’ll end up a millionaire in 30 years.  Take this into account, but as a hacker you should be buying more lattes than the average person, as long as you’re having them with other people.  Why?  Because in the startup world you need to network, network, and network.  How much would you pay now to have a coffee with Zuckerberg or Page or Brin?  Well, 15 years ago it would have been just a couple of dollars.  Hackers meet more future CEOs than other people, so they should be investing more in developing their social networks, and that means buying more lattes.

Number 2: Don’t buy a house

Rather, be less eager to buy a house than other people.  Renting is buying flexibility – the flexibility to move around Silicon Valley or across the country to join a promising new thing.  You can take advantage of that flexibility more than the average person, so tilt towards renting more.

Number 3: Buy more bonds

As a hacker, you’re in a risky business, and already highly exposed to the stock market via possible options.  You should buy more bonds than the average person to reduce that risk.

Number 4: Keep a larger cash reserve

Everyone should have a cash reserve, but you should have a larger one than most.  First, because we’re in a bubble and who knows when it’s going to burst.  But, most importantly, it gives you the freedom to not have income for some time while you’re building the next big thing.  Other people aren’t going to do this – they don’t need as much as you.

and finally, Number 5: Don’t try to beat the market, even if you can

You probably can’t beat the market, so don’t bother.  But suppose you (think you) can.  Even then, don’t bother getting fancy.  Why?  Because the time you spend on perfecting your trading algorithm is time taken away from networking, increasing your skills, or perfecting your real money-making idea. (Of course, if you’re planning to work in finance, then your trading algorithm is your money-making idea, so work on that, and get coffees with hedge-fund managers.)  Scott Locklin recommends you invest in small businesses rather than the stock market, but that sounds like work.

The average return is fine – wouldn’t you rather have 5% return on a billion dollars than 7% on ten thousand?

(Disclaimer: I am not a financial planner.  You should not get your financial planning advice from some random guy on the Internet.)

Will the Libyan war be successful? Let’s use math to find out!

We use forecasting models to predict everything from the climate to the stock market, so why not our current war?  We’ll talk through some issues and touch on five principles of forecasting along the way.

First, why do we need a forecasting model at all?  That’s our first principle – any model is better than no model.  People are emotional and ignore data that doesn’t fit their preconceptions – they’re very bad at predicting what’s going to happen.  No matter how bad your model is, it’s going to be better than just going with your gut.

So how complicated should our model be?  That brings us to our second principle – more complicated models are not much better.  Deep in the bowels of the Pentagon the Libyan situation is doubtless being war-gamed in all sorts of ways, with models of great complexity.  But much of this detail will be a waste of time in predicting the outcome.  It doesn’t really matter exactly what Gadaffi’s 53rd Armor Division does – if you want to know how this war will turn out, look at what’s happened with similar countries and similar armies.  In fact, the dirty little secret of statistical modeling is that even exponential smoothing does a pretty good job at forecasting, so if you just guessed this war would go like a weighted average of other recent wars, you’d beat most of the experts.  Exponential smoothing doesn’t get you too much respect, though, so we might try linear or logistic regression.

OK, so if we don’t pay too much attention to modeling, what should we be focusing on?  That’s our third principle – it’s all about the data.  To predict how this war will turn out, we’ll need to look at other U.S. wars.  But how far back?  Vietnam, like the left says?  WWII, like the right wants?  The Spanish-American War?  The Barbary Pirates?  And should we include the wars of other countries?  Libya with Chad?  The U.S.S.R with Afghanistan?  What variables are going to be important?  I’m guessing population size, military technology, terrain, for starters.  Some of these will be available easily, some we’ll have to manually code (well, I suppose you could develop a numerical measure of Afghanistan’s mountain-ness, but would it be worth it?)  It’s all going to be a mess, and we’re going to have to make decisions between contradictory sources.  And what are we trying to predict?  Length of war?  Casualties?  Cost?  Whether it’s a “success”?  How do we measure any of those?  It’s all about the data.

With so many possible variables, we’re going to need to pay attention to our fourth principle – beware of overfitting.  For example, we might have a country-specific variable in our model.  This would look good for the U.S., as Libya seems to have lost every war it was involved in, from the Barbary Pirates on.  But is that a good enough basis for our model?  Maybe so, as Afghanistan seems to have defeated every invader from Alexander on.  Still, worth paying close attention to.

And our final principle – models are only useful if they’re used.  I would be happy to hear that the President has some Excel spreadsheet showing him the likely results of invading every country from Azerbaijan to Zimbabwe.  But I’m guessing there’s nothing in-between the Pentagon’s insanely complicated wargames and the uninformed opinions of politicos.  Don’t be disheartened, though – you can still use your model to try and cut through the propaganda and plain old wishful thinking you’ll hear from all sides.

What about my prediction?  I’m going to go with an average of the last 6 big wars  “success” – determined and selected in a completely arbitrary way – the 2nd Iraq war (50%), Afghanistan (40%), Serbia (80%), Haiti (90%), Somalia (10%), and the 1st Iraq war (95%).  Gives us 61% – not a debacle like Somalia, less of a mess than Afghanistan, but not one of the greatest success stories, either.  It’s a stupid model and stupid data, because nobody’s paying me for it, but, following our first principle, it’s better than nothing.