algorithms

Algorithms don’t sound like a topic for riveting reading, but these two articles are pretty good.

The first is from a marketing magazine, Adbusters. The claim it makes – that markets have never really worked before, but are starting to work now because of computer algorithms – is  bit of a stretch, but entertaining. Here’s a quote:

The critical flaw in Hayek’s vision of the hand was that a “central body” could never gather enough information. We know this to be untrue, and with big data and the analysis and manipulation of that data through algorithmic equation, the missing link between money and the machine was discovered.

The searches we make, the news we read, the dates we go on, the advertisements we see, the products we buy and the music we listen to. The stock market … All informed by this marriage between mathematics and capital, all working together in perfect harmony to achieve a singular goal — equilibrium. But it’s a curious sort of equilibrium. Less to do with the relationship between supply and demand, and more about the man and the market.

All these algorithms we encounter throughout the day, they’re working toward a greater goal: solving problems and learning how to think. Like the advent and rise of high–frequency trading, they’re part of an optimization trend that leads to a strange brand of perfection: automated profit.

The second, from ESPN, is about how numbers are being crunched by big-time professional sports gamblers:

Eventually, he grew to understand one of Walters’ keys to success: Some of his bets were intentional losers, designed to manipulate the bookmakers’ odds. Walters might bet $50,000 on a team giving 3 points, then $75,000 more on the same team when the line reaches 3.5. The moment the line gets to 4, a runner is instructed to immediately place a larger bet — perhaps $250,000 — on the other team. The $125,000 on the initial lines will be lost, but if things go according to plan, the $250,000 on the other side will win enough to make up for it many times over. Walters uses the same method on multiple games, often risking millions each weekend.

Since the days of the Computer Group, analytically inclined professional gamblers have relied on technology as well as research to produce what is called a delta: the difference between the Vegas line and what the bettors conclude the point spread should be. The greater the delta, the more money a gambler like Walters will bet. There’s nothing illegal about manipulating lines, and many prominent gamblers have the ability to move a line with as little as $1,000. Walters’ strategy is simply more sophisticated and uses more people, better information and, of course, more dollars bet in far more places than anyone else’s, insiders say…

The vast Walters network also includes a guy on the East Coast known as The Reader, who scans local newspapers, websites, blogs and Twitter for revealing tidbits or injury updates. That information is weighed and plugged into the computer alongside other statistical data — from field conditions to intricate breakdowns of officiating crews. Armed with algorithms and probability theories, the objective is to find the mispriced team, then hammer the line to where Walters wants it.

 

designing ecosystem complexity

This article in Ecological Engineering is about measuring and purposely designing complexity into ecosystems to support biodiversity. I like this idea – certainly grass and trees are a step up from concrete in cities, but there might be some relatively simple design choices that could improve conditions for both wildlife and people without adding effort or cost. We actually expend enormous amounts of time, effort, and money maintaining our grass and trees, whereas natural ecosystems manage to maintain themselves while being more beautiful, diverse, and productive. The first step is to understand the systems better, the second would be to understand what variables we can manipulate, then the third and most difficult step is always translating that new understanding to actions on the ground and getting people to actually take them.

Simplification of natural habitats has become a major conservation challenge and there is a growing consensus that incorporating and enhancing habitat complexity is likely to be critical for future restoration efforts. Habitat complexity is often ascribed an important role in controlling species diversity, however, despite numerous empirical studies the exact mechanism(s) driving this association remains unclear. The lack of progress in untangling the relationship between complexity and diversity is partly attributable to the considerable ambiguity in the use of the term ‘complexity’. Here, we offer a new framework for conceptualizing ecological complexity, an essential prerequisite for the development of analytical methods for creating and comparing habitat complexity. Our framework distinguishes between two fundamental forms of complexity: information-based complexity and systems-based complexity. Most complexity–diversity studies are concerned with informational complexity which can be measured in the field through a variety of metrics (e.g. fractal dimensions, rugosity, etc.), but these metrics cannot be used to re-construct three-dimensional complex habitats. Drawing on our operational definition of informational complexity, it is possible to design habitats with different degrees of physical complexity. We argue that the ability to determine or modify the variables of complexity precisely has the potential to open up new lines of research in diversity theory and contribute to restoration and reconciliation by enabling environmental managers to rebuild complexity in anthropogenically-simplified habitats.

What Should We Be Worried About?

Need new stuff to worry about? They have a book for that!

What Should We Be Worried About: Real Scenarios That Keep Scientists Up at Night
by John Brockman

Steven Pinker uncovers the real risk factors for war * Mihaly Csikszentmihalyi peers into the coming virtual abyss * Nobel laureate Frank Wilczek laments our squandered opportunities to prevent global catastrophe * Seth Lloyd calculates the threat of a financial black hole * Alison Gopnik on the loss of childhood * Nassim Nicholas Taleb explains why firefighters understand risk far better than economic “experts” * Matt Ridley on the alarming re-emergence of superstition * Daniel C. Dennett and george dyson ponder the impact of a major breakdown of the Internet * Jennifer Jacquet fears human-induced damage to the planet due to “the Anthropocebo Effect” * Douglas Rushkoff fears humanity is losing its soul * Nicholas Carr on the “patience deficit” * Tim O’Reilly foresees a coming new Dark Age * Scott Atran on the homogenization of human experience * Sherry Turkle explores what’s lost when kids are constantly connected * Kevin Kelly outlines the looming “underpopulation bomb” * Helen Fisher on the fate of men * Lawrence Krauss dreads what we don’t know about the universe * Susan Blackmore on the loss of manual skills * Kate Jeffery on the death of death * plus J. Craig Venter, Daniel Goleman, Virginia Heffernan, Sam Harris, Brian Eno, Martin Rees, and more

Sustainable Cities Index

Before I read the new Sustainable Cities Index published by ARCADIS, I gave some thought to how I would put together a sustainable cities index. (Disclosure: I work in the same industry as this company, but have no affiliation with it.) Here is what I would try to do, at least for the environmental component (I haven’t given much thought to the social component):

  • Estimate the consumption (food, water, energy, and materials) of people in that city. Also estimate the waste produced – solid waste, air emissions, and water pollution – that is not recycled in some way.
  • Try to figure out roughly where this food, water, energy, and stuff came from, whether from within the city or without. Figure out the impact on soils, water bodies, the atmosphere, and natural ecosystems of producing all these materials. You could try to express this in land terms, energy terms, dollar terms, theoretically even information terms – there is no best or perfect way, although “ecological footprint” in terms of land is possibly the most intuitive.
  • Figure out, if everybody on Earth lived in cities exactly like a particular city, whether the Earth could support that indefinitely, and what the ultimate state of the remaining natural ecosystems would be.

By doing this, you would have an absolute benchmark to compare any city against, rather than just a relative benchmark to compare cities to each other. The problem with a relative benchmark is that it doesn’t tell you whether your best cities are good enough.

A possible variation would be to attribute impacts to the city where a particular corporation or industry is headquartered, rather than where the demand is. Let’s say a particularly unsustainable, sprawling development pattern is built in India, but is designed by a firm based on Singapore, or Geneva, or wherever. Or people in a city with very clean air and water eat food grown on plantations owned by corporations headquartered in the city, but grown in recently burned-down rainforests overseas. You could assign all or a portion of the blame to the designer or owner rather than the people living where the impact is actually occurring, who may not always have had much of a choice.

Now, let’s see what ARCADIS did. I don’t mean to be too critical – good for them for doing something and giving people something to think about, which is more than most companies do.

The People sub-index rates transport infrastructure, health, education, income inequality, work-life balance, the dependency ratio and green spaces within cities. These indicators can be broadly thought of as capturing ‘quality of life’ for the populace in the respective cities.
The Planet sub-index looks at city energy consumption and renewable energy share, recycling rates, greenhouse gas emissions, natural catastrophe risk, drinking water, sanitation and air pollution.
The Profit sub-index examines performance from a business perspective, combining measures of transport infrastructure (rail, air, other public transport and commuting time), ease of doing business, the city’s importance in global economic networks, property and living costs, GDP per capita and energy efficiency.

how U.S. taxes are spent

In a poll of U.S. taxpayers, 95% of respondents had no idea how much the country spends on foreign aid, which is much less than 1%. It just shows that although rational people can disagree on how our tax money should be spent, we are not having a rational debate because most of us have no clue what it is really being spent on. A taxpayer receipt is a simple idea to help cure this problem. Ideally this should be done by the IRS or Treasury Department, but the White House has stepped in to do it since nobody else will.

I picked a hypothetical married couple with children making an income of $80,000 per year. There are a million different ways you can slice it. But no matter what you do, the biggest categories jump out at you. Income tax is only about 40% of what the government takes in from this family, with Social Security and Medicare taxes making up the other 60%. Pensions for retired people make up the majority of how the money is spent, with Social Security alone making up almost half. Even without an offical public health care system, the federal government spends a lot on health care – almost 20% of all the money spent between Medicare (for older people) and programs to help lower-income people (Medicaid and children’s health insurance programs mostly). Of course, state and local governments also tax and spend on health care, which is not reflected here.

The military makes up around 10% of all federal spending. If you add veteran’s benefits (I’m double counting here, since these include retirement and health care) and homeland security, that number comes to more like 13%.

So however you slice it, the big numbers are retirement, health care, and defense. If we want to make significant changes in either the amount of tax, or the outcomes of government programs, we should focus most of our debating energies in these areas.

 

“Right of Boom” and “7 Deadly Scenarios”

The New York Times recently had a review of this book:

Here is the Amazon description:

A nuclear weapon explodes in a major American city and no one can prove who is responsible. The devastation is horrifying, but even more alarming is the limited options available for the United States government to respond. What happens next?
In Right of Boom, national security specialist Benjamin Schwartz looks at what could happen after a nuclear explosion takes place in the United States, the event that Presidents Obama and Bush, as well as would-be Presidents Mitt Romney and Hillary Clinton, have acknowledged as the greatest single national security threat we face. Hypothesizing an explosion in downtown Washington, D.C., Schwartz maps out the likely ramifications while going deep into history to explore the limited range of options available to a Commander in Chief. Drawing from his experience as an analyst at the Departments of Defense, State, and Energy, Schwartz offers a fully panoramic view of a terrifying reality.

However, the review said not to read that one and to read this one instead:

A global pandemic finds millions swarming across the U.S. border. Major American cities are leveled by black-market nukes. China’s growing civil unrest ignites a global showdown. Pakistan’s collapse leads to a hunt for its nuclear weapons. What if the worst that could happen actually happens? How will we respond? Are we prepared?

These are the questions that Andrew F. Krepinevich asks—and answers—in this timely and often chilling book. As a military expert and consultant, Krepinevich must think the unthinkable based on the latest intelligence and geopolitical trends—and devise a response in the event our worst nightmares become reality.

As riveting as a thriller, 7 Deadly Scenarios reveals the forces—both overt and covert—that are in play; the real ambitions of world powers, terrorist groups, and rogue states; and the actions and counteractions both our enemies and our allies can be expected to take—and what we must do to prepare before it’s too late.

I think it’s important to think about not just the military implications, but the implications for the global economy and overall confidence of the public. Considering that the 9/11/2001 attacks killed 3,000 people, caused a sharp recession, and led the United States to launch two wars and spend at least a trillion dollars, the effects of a much worse attack or a series of them are very concerning.

drone waiters

Singapore is experimenting with drone waiters due to a supposed labor shortage in the service sector. I am all for cool new technologies, but in this particular case I can do without. I actually like the Australian system – you order, get a number or buzzer, and when your food is ready you pick it up at a window. It works just fine. I would save the technology for the ordering and paying part. That would really help with a business lunch, for example, where you are in a hurry and would like to order in advance, then not have to sit around waiting for the check. Combining a sit-down restaurant with online ordering also opens up a lot of possibilities for customizing your order and seeing what other people thought were a restaurant’s best dishes. It should also work well with delivery and mobile food trucks.

planning theory

This article in the Journal of Planning Education and Research (free for the month of February only apparently) is a nice review of planning theory. It amazes me that the profession of planning seems to be so unsure of itself, and yet has so many important theories and tools to offer to other disciplines. There is a lot of planning going on outside the small field of academically trained urban and regional planning. I like to think of planning as similar to mathematics – it’s a profession for a few, but its theories and tools are used every day by professionals across many fields. Many of us can do moderately complex math by ourselves, and we know we can call on the mathematicians and statisticians for help with the really complex stuff. Similarly, a lot of professionals like engineers and economists are entrusted with the keys to the planning machine. But often, we do it badly because we are not well trained in the theory and tools of planning.

Almost all professionals – planners, engineers, and economists at a minimum – would benefit from better education in general systems theory – what the building blocks of systems are, how they interact with their boundaries, and how their behavior over time is driven by their structure and interaction with boundary conditions, and how they can be manipulated to achieve desired outcomes. Among the professions, engineers and economists probably have the best understanding of systems today, but we tend to define the system boundaries, and the range of desired outcomes that can be achieved, much too narrowly. That is one place planners can come in – facilitating the interaction between technocratic problem solving being done by engineers and economists with the larger socio-economic and environmental context.

What I call “technocratic problem solving” here is essentially what the planners call “rational-comprehensive” planning. In my view, it works very well for the elements of systems that we understand well (managing water resources, food production, and employment, for example). Where it has come under criticism (for example, the failed “urban renewal” programs in the U.S.), I believe the problem is not in the approach, but rather applying the approach to systems we do not understand well has given us a false sense of precision and a false confidence, which has led to failure. A hybrid approach that works very well, in my experience with water resource and environmental planning, is to apply the rational-comprehensive approach to the parts of the system we understand well, and then feed the results into a stakeholder or political process that can deal with the social aspects of the system we understand much less well. Planners can play the critical role in making this process reach a functional outcome. This is how I like to think of the planning profession – as the critical glue that can hold together a coalition of engineers, economists, bureaucrats, businesspeople, interest groups, and members of the public into a coherent whole that can set a direction for our society, then continue to guide it with incremental course adjustments as we go forward.

2014 Report Card

It’s taken me a while to get out a “year in review” post for 2014, but anyway, here it is. This won’t be a masterpiece of the essay form. I’m just going to ramble on about some interesting trends and themes from the year, along with a few relevant links.

The critical question this blog tries to answer is, is our civilization failing or not? I’ll talk about our human economy, our planetary system, and make some attempt to tie the two together.

Overall Human Health and Wellbeing. First, there are some very happy statistics to report. For example, worldwide child mortality has dropped almost by half just since 1990. What better measure of progress could there be than more happy, healthy childhoods? And it’s not just about increasing wealth – people in developing countries today have much better health outcomes at the same level of wealth compared to developing countries of the past (for example, Indonesia today vs. the United States when it passed the same income level). It’s hard to argue against the idea that economic growth and technological change have obviously eliminated a lot of human suffering. So, I think the important questions are, will these trends continue? Is the system stable? Can the natural environment continue to support this trend indefinitely? There may also be an important question of whether we had the right to exploit the natural environment to get us to the point where we are now, but that is an academic question at this point.

Financial System Instability. Let’s talk about the stability of our human economic system. The U.S. economy may finally seem to be picking up from the aftermath of the severe 2007-8 financial crisis, but it is certainly far below where it would be if that hadn’t happened and the prior growth trend had just continued since then. The rest of the world isn’t doing so well, however – Europe and Japan are looking particularly slow if not in an outright deflationary spiral, at the same time developing countries appear to be slowing down. Some are calling this a “new normal” for the world economy. More scary than that, the industry-written regulations and perverse incentives allowing the excessive risk taking that caused the crisis have not been fully addressed and the whole episode could recur in the short term.

Thoughts on Ecosystem and Economic “Pulsing”. 2007-8 was a textbook financial crisis – although it was caused by novel forms of money and risk taking beyond the direct reach of government regulators and central banks, it was not that different from crises caused by plain old speculation and over-lending back when there were no central banks around. It’s hard to draw a direct link from the financial crisis to ecosystem services, climate change, or natural resource scarcity. However, if we think about natural ecosystems, they are resilient to outside stressors up to a point – say, moderate fluctuations in temperature, hydrology, or pressure from non-native species. However, say a major fluctuation happens such as a major flood or fire that causes serious damage. In the absence of major outside stressors, the system will eventually recover to its original state, but in the presence of major outside stressors, even if they did not cause the flood or fire, it may never bounce back all the way. In the same way, our human economy may appear resilient to the effects of climate change, ocean acidification, soil erosion, and so forth for a long time, but then when something comes out of left field, like a major financial crisis, war, or epidemic, we may not be able to recover to our previous trend. This probably also applies to the effects of technology on employment, as discussed below. In the absence of major shocks coming from outside the system, we’ll see a long, slow slide in employment and possibly a long, slow rise in energy and food prices, with so much noise in the signal that it will be easy for the naysayers to hold sway for long periods of time. But when those major events happen, we may see sudden, painful changes that we have no obvious way of mitigating quickly.

Technological Change: Artificial Intelligence, Robots, Automation, and Employment. After decades of slow but steady progress, these technologies are really coming into their own. Robots are being used to keep miners in line and to drive cars, for example. Manufacturing has become a high-tech industry. As computers and machines get better at performing more and more skilled jobs (book-keeping is one example), there is gradually less demand for the medium-skilled workers who used to do those jobs. High-skilled workers like computer programmers are doing very well, although I presume the automation will gradually creep higher and higher up the chain, so today’s safer jobs will be less safe tomorrow. At the same time these medium-skilled workers in developed countries are getting squeezed out, developing countries are not benefiting like they used to from their large pools of low-skilled workers as manufacturing becomes more and more automated, and can be done cost-effectively closer to consumers in richer countries.

Will our society recognize and solve this employment problem? American corporate society, and its admirers around the world, are unlikely to. Something very similar to this happened with agricultural automation in the early- to mid-20th century, and with globalization in the mid- to late-20th century. As agriculture became more automated, many displaced workers moved from rural areas in the U.S. southeast to urban areas in the U.S. northeast, looking for factory work. Unfortunately, the factory jobs that existed previously were being moved to developing countries with abundant low-wage labor. The pockets of poverty, unemployment, and social problems created by these forces have not been adequately addressed to this day. To the individual worker, it doesn’t much matter whether your job is being taken by a local robot or an overseas human. Unemployment created by technological forces today could resemble what was created by globalization yesterday, only on a much larger scale. We can only hope that the larger scale will drive real political solutions, such as better education and training, sharing of available work, and more widespread ownership of the labor-saving technology.

Of course, one of the earliest and probably the most shameful example of a modern capitalist system generating wealth for an elite few at the expense of workers is the American slavery system of the 18th and 19th centuries. We just can’t trust amoral, self-interested private enterprise to maximize welfare in the absence of a strong moral compass coming from the larger society. Let’s stop pretending otherwise.

Another example of extreme corporate immorality: Public apathy over climate change in the U.S. may have been manufactured by a cynical, immoral corporate disinformation campaign over climate change taken right out of the tobacco companies’ playbook.

The Gospel of Shareholder Value. There is an important debate over whether people who run corporations have any ethical responsibility to anything other than profit seeking. Well duh, everyone on Earth has an ethical responsibility. Case closed, as far as I’m concerned. There is even evidence that the ideology of profit maximization is a drag on innovation. Except billions of people out there who have worshiped at business schools would disagree with me. And I don’t want to offend anyone’s religion. Noam Chomsky had a quote that I particularly loved, so I am going to repeat it here:

In market systems, you don’t take account of what economists call externalities. So say you sell me a car. In a market system, we’re supposed to look after our own interests, so I make the best deal I can for me; you make the best deal you can for you. We do not take into account the effect on him. That’s not part of a market transaction. Well, there is an effect on him: there’s another car on the road; there’s a greater possibility of accidents; there’s more pollution; there’s more traffic jams. For him individually, it might be a slight increase, but this is extended over the whole population. Now, when you get to other kinds of transactions, the externalities get much larger. So take the financial crisis. One of the reasons for it is that — there are several, but one is — say if Goldman Sachs makes a risky transaction, they — if they’re paying attention — cover their own potential losses. They do not take into account what’s called systemic risk, that is, the possibility that the whole system will crash if one of their risky transactions goes bad. That just about happened with AIG, the huge insurance company. They were involved in risky transactions which they couldn’t cover. The whole system was really going to collapse, but of course state power intervened to rescue them. The task of the state is to rescue the rich and the powerful and to protect them, and if that violates market principles, okay, we don’t care about market principles. The market principles are essentially for the poor. But systemic risk is an externality that’s not considered, which would take down the system repeatedly, if you didn’t have state power intervening. Well there’s another one, that’s even bigger — that’s destruction of the environment. Destruction of the environment is an externality: in market interactions, you don’t pay attention to it. So take tar sands. If you’re a major energy corporation and you can make profit out of exploiting tar sands, you simply do not take into account the fact that your grandchildren may not have a possibility of survival — that’s an externality. And in the moral calculus of capitalism, greater profits in the next quarter outweigh the fate of your grandchildren — and of course it’s not your grandchildren, but everyone’s.

Our Ecological Footprint. WWF issued an updated Living Planet Report in 2014 suggesting that our annual consumption of natural resources (including the obvious ones like energy and water extraction, straightforward ones like the ability to grow food, but also the less obvious ones like ability of the oceans and atmosphere to absorb our waste products) is continuing to exceed what the Earth can handle each year by at least 50%. We’re like spoiled trust fund babies – we have such incredible resources at our disposable, we never learn to live within our means and one day the resources run out, even if that takes a long time. As we recover from the financial crisis, we have a chance to do things differently, but the connections are not being made to the right kinds of investments in infrastructure, skills, and protection of natural capital that would set the stage for long-term sustainable growth in the future.

Other Big Stories from 2014:

  • World War I. 100 years ago, World War I was in full swing. Remember The Guns of August? Well, that was August 1914 they were talking about. Let’s hope we’re not about to blunder into another conflict. But (and I’m cheating a little here because I read this in 2015), the World Economic Forum named “interstate conflict” as both high probability and high consequence in its global risk report.
  • Ebola. Obviously, Ebola was a very bad thing that happened to a whole lot of people. To those of us lucky enough that we weren’t directly in its path, it is a chance to selfishly reflect whether Ebola or something even worse could be coming down the pike. Let’s hope not.
  • Severe Drought and Water Depletion in the Western U.S.: California has been in the midst of a historic drought, although they got some rain recently. Some are describing this as the new normal. Besides rainfall, glaciers, snowpack, and groundwater all seem to be disappearing in some important food-growing areas.
  • Solar grid parity is here! At least some places, some times…

Conclusion. Yes, I think we are on a path to collapse if nothing changes. And I don’t see things changing enough, or fast enough. There are glimmers of hope though. Lest you think I offer only negatives and no solutions, here are two solutions I harp on constantly throughout the blog:

  • Green infrastructure. This is how we fix the hydrologic cycle, close the loop on nutrients, begin to cleanse the atmosphere, protect wild creatures and genetic diversity, and create a society of people with some sense of connection to and stewardship over nature. Don’t act like it’s such a big mystery. It’s known technology. There has been plenty written about trees, design of wildlife corridors and connectivity, for examples. There is simply no excuse for cities to do such a crappy job with these things.
  • Muscle-Powered Transportation. Cars are clearly the root of all evil, the spawn of Mordor, as I pointed out several times (sorry, I just sat through 6+ hours of Hobbit movies). Unless you are perhaps that rare hobbit who can own a car without your morals being completed corrupted by its evil powers. But for the rest of us, I explained several times why getting rid of cars would be good. Here is just one example:

One of the most important things we can do to build a sustainable, resilient society is to design communities where most people can make most of their daily trips under their own power – on foot or by bicycle. It eliminates a huge amount of carbon emissions. It opens up enormous quantities of land to new possibilities other than roads and parking, which right now take up half or more of the land in urban areas. It reduces air pollution and increases physical activity, two things that are taking years off our lives. It eliminates crashes between vehicles, and crashes between vehicles and human bodies, which are serial killers of one million people worldwide every year, especially serial killers of children. It eliminates enormous amounts of dead, wasted time, because commuting is now a physically and mentally beneficial use of time. There is also a subtle effect, I believe, of creating more social interaction and trust and empathy between people just because they come into more contact, and creating a more vibrant, creative and innovative economy that might have a shot at solving our civilization’s more pressing problems.