I got some smart comments from Henry over at The Modern Middle Manager, and went over to take a look. He had a great post on an almost-failed eCRM implementation, and it started me thinking.
About 70% of the billable work I’ve done in the last five years has been work on failing technology projects. These are projects from simple websites to the project I’m on a now, a multi-million dollar customer data warehouse for a major corporation. (The project is Really Far from my house, and while I’m glad for the work, I’m getting tired of commuting and anyone in Greater LA who has a troubled tech project ought to email me…)
Almost all of these projects got into trouble for similar reasons. Lack of planning, lack of clear alignment among the stakeholders, a failure to follow rudimentary systems development processes, vendor deceit, and most of all, hubris.
Hubris is the most important thing, because it is what allows the project sponsors and leaders to believe that what they’re doing makes sense, even in the face of contradictory best practices and evidence.
Hubris in the technology world typically looks like this:
A visionary engineer builds a tool or product. He partners with or hires, or is hired by an aggressive marketing visionary who takes the actual capabilities of the tool or product and ties them to a business need in the most visionary and optimistic way. They hire an energetic PR person, who gets them press in Wired (or the late lamented InfoWeek), where it gets read by an executive at an organization that has the business need targeted. The executive groks the vision, and calls the vendor, who sends a sales executive and sales engineer to meet with the organization, and sell a vision…of a seamlessly integrated, real-time view of the company processes, or customers, or whatever.
Now the bigger the vision, the bigger the ROI, and the interests of all parties are well-served by revolution, rather than evolution. So the decision is made to junk the existing (badly architected, lashed-together, obsolete, barely-functional) systems, and build something new from scratch.
Here’s where the problems really begin.
Management, planning, analysis, and documentation rise in importance and as a necessary percentage of the project budget as the scale of the project increases.
Examples: I just designed and a friend built a small web-referral monitoring and reporting system; it’s a database of about seven tables and four web pages. Meeting with the customer took about two hours, I wrote about ten pages of documents (including two pages of database ERD’s, one page of flowcharts, and seven pages of functional description) in about a day, and the developer worked for a week to build it. So roughly 20% of the effort was in ‘overhead’.
I’ve never worked in aerospace, but I’m told that on major aerospace software projects that the ‘overhead is approximately 60% of the total budget; this is required because a) the projects are extremely large; b) releasing buggy software and iteratively working it out – the Microsoft solution — is not an option; c) the management and reporting processes are extremely formal and structured, because that’s what has worked on large scale engineering projects such as Apollo in the past. That system is brutally inefficient, and probably not a lot of fun for the people involved, but it works.
The problems begin to arise when project sponsors try to take projects of near-aerospace levels of complexity and apply the kind of management techniques I used in designing a four-page, seven-table web tool.
Most non-aerospace organizations lack the expertise and discipline to manage huge projects. Many of them claim to, however.
The best defense, I’ve learned, is to manage the scale of the projects, rather than trying to change the management culture of the organization. This doesn’t mean that you have to abandon far-reaching systems as a goal. It just means you have to break them up into littler systems that can be developed within a reasonable span of control. Personally, I tend to think that a project budget of $750,000 and a project term of four months is about right. You can do useful work and still have a small, agile team doing the work.
Henry came to the same conclusion:

Don’t create one massive project, do mini-projects and stick to a rollout schedule. We changed too much too fast and tried for too many payoffs at once (in sales, client management, operations). That’s the number one reason the end-users got irritable and the reputation of the system was trashed almost immediately.

My personal project bible says much the same thing.
If you’re interested in this stuff at all, go read Rapid Development: Taming Wild Software Schedules, by Steve McConnell. It’s a terrific book; I’ve given about a dozen copies away to clients and team members.
OK, for the non-geeks in the crowd, what’s the relevance of this?
Traditional command-and-control government programs are like large software development projects; they are insanely complex, and to succeed, they require incredibly tight control over the participants. This both requires a huge ‘overhead’ of people doing the controlling, and a culture of tight control as well.
And like large software projects, they fail badly. The hysterisis…the time lag between action and output…is so large that it’s almost impossible to change or adapt. Small projects fail well; they have low hysterisis, and so when problems arise, they can adapt in response to real conditions.
As we’ve talked about race, one of my issues is not that change needs to happen; it’s that the breadth of change is so great that making it requires the kind of cumbersome, likely to fail or have unintended consequences programs that I’m discussing here. I’m trying to see a way through the problem that uses smaller more nimble, more effective programs.
Not only for dealing with issues of race, but for dealing with the array of issues that we confront as a society, and that, as a liberal, I’d like to use the power of society and the government to improve.

D^2 has a long post which also touches on the issue of the role machine politics played in integrating European ethnic immigrants in the 19th and early 20th centuries. (Look at the end of the post after the long and wrongheaded argument that polite discussion of the race issue keeps us from dealing with the reality of it. I’ll get to that issue at some later point.)
Los Angeles politics in the 70’s were marked by an alliance between the city’s African-Americans and liberal Westsiders (often Jews), in which many of the city’s civil service jobs were targeted at African-Americans through AA and explicit political setasides.
The consequences were, on one hand, a flourishing African American employee and professional middle class (as opposed to entrepreneurial middle class), and on the other a highly racially politicized set of civil service unions.
This is pretty typical of most big cities at that point in history; it was the modern ‘googoo’ form of ethnic machine bloc politics.
What has been interesting in Los Angeles has been to watch the politics since then change, as on one hand, the burgeoning Latino voting population and political class began to outmuscle the African American politicians, and the fragmenting of the liberal Westide/urban African American alliance as the claims of the Latinos began to challenge the role of blacks as the ‘parties due compensation’, and as the fiscal crisis of the cities began to erode the white entrepreneurial/professional willingness to tolerate the costs of ethnic patronage.
Essentially, the same things happened at Tammany as Tweed had to provide more and more jobs and contracts to keep buying the support he needed to stay in power; at some point the cost became so high that he couldn’t bury it in the budgets any more. At that point he was prosecuted and his patronage appointees lost their jobs to a new set of patronage appointees.
The difference is that the reforms instituted to block Tweed/Daley type of explicit patronage created the groundwork for a more structural kind of patronage, which is equally valuable, equally political, and equally hard-fought over. But because it operates under the color of a bureaucratic civil service, the fights are less explicit and hence harder for anyone but the politically hyper-aware to follow, and they tend to be accretive, in that programs and positions never die, they just get added to.
And so the school district gets layers and layers of administrative employees, the city gets half-attended to and partially-funded programs, services decline and costs increase.
Note that this isn’t done outside the boundaries of the SkyBox politicians; they’re the ones orchestrating it to maintain their own coalitions and political advantage.

D^2 is a pretty interesting guy, even if it is possible that we have some philosophical differences.
Scanning the older posts there (I’m behind, OK?), I come to this:

Interesting things of our time … many of my compadres in the weblog trade have been bemoaning the fact that “moderate Muslims” haven’t been vocal enough to condemn all those other Muslims, the ones who are regularly quoted in news sources not run by Muslims or in dodgy translations of quotations out of context as saying something really horrible. Why, oh why, oh why, is it that the comments pages of Little Green Footballs1 aren’t absolutely full to bursting point with well-educated, secular Western Muslims apologising in wailing tones and loudly condemning those other silly uneducated fundamentalist barbarians?
To ask this question is of course to answer it; the vast majority of people don’t behave in this way because it would be monumentally weird to do so, and the vast majority of Muslims presumably and correctly suspect that when you’re dealing with the kind of person who starts pointing at things you didn’t do and demands that you sign their statement condemning whoever did them, then nothing you say is ever going to be good enough for them. I’ve half a mind to create an educated-Muslim sock puppet character and put this thesis to the test …
But anyway, people like Stephen den Beste, author of the turgidly unreadable and unsettlingly technocratic attempts to recreate neoclassical economics without the benefit of reading a word of the literature which populate USS Clueless (he has a fine line of shite in talking about mobile phone standards too), regard the absence of moderate Muslims lining up to claim that numerically the majority of their religion is made up of horrendous halfwits and ogres, as a sign that Islam is an intrinsically warlike, barbaric and horrible religion. I’m using his piece on this subject as the example because I happen to have just read it, and as an associate of the dreadful Eric Raymond, he’s a target of opportunity. But such burnt-out old hacks, U2 groupies and writers of novels which have to be put in the “God this is shit compared to his earlier stuff” category as Salman Rushdie, have also written in similar terms. And even my old mucker Brad Delong, who seems to have developed an unaccountable blindspot when it comes to these matters, is quoting him approvingly.

This is an interesting point both in and of itself, and as an example of a broader issue that ties to, among other things, the topic du jour of race in the U.S.A.
The presence of moderate Muslims is important because if we are going to negotiate, we need to have someone on the other side whom we consider rational and trustworthy enough to be our negotiating partner.
I have talked about the role of honor and self-restraint in conflict; one of the major reasons for this, I will argue, is that most conflicts are settled short of the absolute defeat on one side by the other. And for that to happen there must be an element of trust, of some mutual boundaries, and of the feeling that the folks on the other side will agree to the basic steps that will lead to and past the cessation of hostilities.
In the case of the Muslim world, a generalized rage against the West, accompanied with actions based on that rage which violate the Western norms of war – which have grown over a millennium of bloodthirsty conflict within the West – sets the stage for a conflict. If we want the conflict to be ended, both parties have to create enough grounds for negotiation to induce the other party to sit down.
The PA has consistently failed to do this, which is a large part of what has led to the current situation between Israel and Palestine (Israel has done it primarily in continuing their political and financial support for the settlements in the West Bank and Gaza).
Now, not all conflicts can be settled without the absolute subjugation of one side by the other. But if this conflict isn’t I think that we’ll certainly regret it , but I can suggest to the author above that if the conflict between the Muslim world and the West isn’t settled through negotiation and the growing rule of law, that there will likely be many fewer Muslims on the earth than there are today. This isn’t a personal threat, nor is it even close to my desired outcome; it’s a simple problem in the arithmetic of mass destruction, factored with knowledge of the thin skin of civility over the bloody history of Western civilization.
To restate in simple terms, it’s important to see an emergent ‘moderate’ Muslim world if we want to avoid a cataclysmic conflict, in which I have no doubt the Muslim world will lose. I’d like to avoid that conflict, if we can.
This relates (on a hopefully less dangerous basis) to the politics of race as well.
There’s been a lot of discussion about left-wing triumphalism in the wake of l’affaire Lott, and the appropriate next steps for those interested in racial justice. Some, like Atrios, argue that this is the time for standing firm; some, like Hesiod, caution about overreaching.
Without getting into the overall political morality or tactics on this issue (sometime soon…), I’ll suggest that we remember that both positions are important – the extreme and rigid to set the marker and provide the nudge to get things moving, and the moderate and trustworthy to actually sit down and make deals and get things done.
The issue is simply in the conflict between the importance of moral certainty and of remaining a member of one’s political community.
It is clear that one feature of politics today is the rise in importance of one – of certainty – and the decline in the other – willingness to be constrained by one’s community.
Now there are clear points in history where the standards of the community are so far outside the pale that any moral person must stand outside them. And it is equally clear that the standards change over time. But, one thing to think hard about is the issue of whether every issue and subissue in today’s politics is more important than the integrity of the polity, and one question to ask in general is how we move the polity without destroying it.
Or, how do we create a politics that doesn’t rely on the expectation and rhetoric of absolute conquest and subjugation? ‘Cause I sure hear a lot of that from all sides these days.

Just added two more people who are doubtless smarter and more interesting than me to the blogroll…
D^2 Digest (the D-Squared Digest for those who are carat-impaired), and the The Flaming Moderate”, another person who sounds more confused about his political stance than he really appears to be…
…checkemout, checkemout…

Saw ‘Gangs of New York’ over the holiday; it was a ‘boy’s night’ as my mom and TG spent Christmas Eve cleaning up the mess (to be fair, I’d cooked, and the boys had set up); and we wanted to go see something TG wouldn’t kill me for seeing without her, so picked that.
First, I’ll refer you to the post below. In keeping with the theme of the film, I’m going to go buy a Bowie knife and start bringing it to theaters; when I ask people to be quiet the third or fourth time, I’ll ostentatiously start cleaning my nails with it. I cannot believe that a bunch of middle-aged, middle-class purportedly cultured people have such a weak grasp on elementary courtesy.
Second, why do people being their six and eight-year old kids to moves that are this gratuitously violent?? And I’m not talking Jackie Chan or even ‘Matrix’ – type ‘cartoon’ violence. Any movie that has a speech by a major character about how they cut out their own eye as penance might just be a movie that the kiddies could stay home from, don’tcha think? We’ve decided that Littlest Guy (at six) is too young to see LotR yet, and I don’t think that’s a bad idea. The talkative couple in front of us (as opposed to the talkative pair of ladies to our right and the talkative couple behind us) had brought their son who looked to be LG’s age and he watched the gore with rapt fascination.
Now, the movie.
It wasn’t very good.
It wasn’t very good for three reasons: First, because it was waay too slow and self-important. Every scene seemed padded, like it had 5% of extra footage in the beginning and 10% on the end. And the sense of pace or orchestration just didn’t come across to us. Second, because the plot and theme placed too much emphasis on the stereotypical elements (will he seek revenge or accept the fatherly love of his true father’s killer?) and not enough on the more complex human ones. And finally, because the violence, unlike the violence in Goodfellas or Casino didn’t serve the story, it seemed rather like the story was there to set up the violence.
But as a window into history, it was great. I’ll note that the real 19th century gang wars were more like brawls than the medieval melees, complete with edged weapons, that this movie suggests. And from the accounts I’ve read, the Draft Riots were more narrowly focused on African Americans. But the ‘feel’ of the film for the look and presence of the streets and crowds was worth the price of admission (if you have the stomach for it).
The racism of the nativists against the immigrants…in this case the Irish…was palpable, and it was fun to watch the political horsetrading that Boss Tweed engaged in as he tried to corral the Irish vote. Someday, I’d love to see a great movie about a political boss, and about the low-level politics…of licenses, jobs, and favors, that went into it. All the King’s Men is the most recent that I can think of. I’d love to see another.
I’m ruminating on machine politics, and will try and put something together on it. Overall, I’m wondering if breaking the back of the political machines has been a totally good thing…and if in fact, we’ve really done it.
But the reason to see the movie is the imagery and language (and to a lesser degree, for Daniel Day-Lewis). There is a book…Evidence, by Luc Sante – that had the same kind of feeling of a window into the unsanitized past.
It’s also provoking further thoughts on race, which has been much discussed here and in the balance of the Blogoverse.
Now I just need to get some more time…

…and go celebrate the next two days with your friends and families.
Best wishes to everyone. May we all get what we need and hope for and not necessarily what we deserve…

Demonstrating that attitude must run in families, Rachel Lucas’ aunt has a site that pairs up active-duty military and civilians as e-pen pals…
…thirty years from now, we’ll be reading touching stories in Dear Abby about people who met via this…

Well, the Xmas Ornament party at Casa de Armed Liberal came off with few if any obvious hitches; the incontinent cat didn’t piddle on the tree, no one mugged us to steal her needles, everyone get fed and watered (or cidered or Cava-ed), and (other than my psycho ex-Sheriff dear friend) no weapons were brandished or actually drawn.
Ann Salisbury, Kevin Drum, and Martin Devon were kind enough to join us, and while I tried to get them to separate and mingle, the politics attractor was just too strong.
A few other guests fell into their pull, and we got a few good discussions going, the most heated of which involved (surprise!) race…
It was an interesting (if lily-white) discussion, with a wide range of views represented.
The two major clusters were centered around Kevin and Ann, who made the ‘racism exists and the government needs to do something about it’ argument (I’m not exactly making all the subtleties in their real arguments, but I’m just planting a flag here) and the ex-cop who made the ‘you have no idea what you’re talking about in the real world, and until the culture of victimization and entitlement changes, nothing’s going to get better’ argument.
No one changed anyone else’s mind (what a surprise!) and it was hard to even find a common set of facts, statistics or anecdotes to agree on.
My reaction then, and now, was fairly complex: What if they’re both right? Because in reality, I tend to think that there are five basic groups of thought on the subject of race.
We have race-baiters and bashers of both the left and right. I’m sorry, but Al Sharpton has more in common, in my mind, with David Duke than with anyone else. Without the cloud of racial animus they both rely on and rhetorically keep inflated, they’d have to get real jobs.
That’s two.
We have a bunch of people who really don’t have a clue, either because (it’s possible) they’re so enlightened that they have transcended race, or because they don’t spend one brain cell’s worth of effort thinking about these issues. I periodically go to the motorcycle races in Rosamond, in the far northern desert suburbs of Los Angeles, and one thing I’ve noticed is the prevalence of clumps of young teens wandering around looking for whatever young teens are looking for on a weekend day…and these clumps are often multiracial. I don’t think these kids spend a lot of time dealing with issues of race, and that’s OK with me.
And the final two are the highly political but well-intentioned on both sides of the issue.
One side believes that only the active intervention of the highest levels of government – which stopped lynching, integrated schools and businesses, and broke the hard color lines that existed as recently as 40 years ago – can keep the weak minority from being crushed by the majority.
The other side believes that the damage done to the minority by the programs which were established to help them far outweighs any benefits.
Sadly, while I believe that each of these groups is well-intentioned, each of them is somewhat in thrall to their extremist partners, who set boundaries on the debate.
On the Right, they simply refuse to acknowledge the depth of real harm done in the past, as well as the simple fact that the harm was only undone by the direct forceful intervention of the Federal Government. It’s equally difficult for them to talk about the real straits the black underclass is in.
On the Left, it is impossible to talk about any of the negative impacts of racial policies and government intervention … on minorities or on society as a whole … without immediately being exiled as a racist. And it’s impossible for them to break out of the old metaphors of the continuing exploited position of African Americans in America, even when confronted with Condi Rice, Colin Powell, Glen Reynolds’ future sister-in-law, or my friend’s black wife.
Now I’ve been dinged in the past for piling on my liberal allies.
Let me make a simple point: I’m not interested in helping build a strong conservative movement here in the U.S. In case you haven’t noticed, that exists. I am interested in seeing a successful liberal movement – which means both that it has to be able to gain power, and once in power actually achieve liberal goals. I’m dubious about the ability of the current liberal movement to do either one.
So let’s talk about what I see as wrong with the liberal position on race.
This weekend’s L.A. Times has a commentary by Michael Eric Dyson, a professor at the University of Pennsylvania:

One of the reasons race continues to play such a huge role in the culture is that we deny its persistence. When it comes to race, we live in the United States of Amnesia.
The nation thrives on whitewashing its bitter racial conflicts, or at least baptizing them in the healing pools of revisionist mythology.
The Civil War wasn’t fundamentally a conflict of color rooted in slavery but rather a battle over political measures to unify the nation.
The Supreme Court isn’t a politically motivated body of legal opinion but a neutral, objective forum to adjudicate racial disputes.
Affirmative action is not a provisional remedy for the vicious history of racial discrimination but a set of public policies designed to facilitate preferential treatment for unqualified minorities.
By these and other political and rhetorical maneuvers, the racial status quo — made up largely of conservative figures who opposed crucial features of the struggle for racial equality, or neoliberal pols uncomfortable with the claims of progressive antiracist activists — has managed to deflect responsibility for its role in the perpetuation of policies, prejudices and practices it is now supposed to resist.
By rewriting the violent history of race in America, figures in both staunchly conservative and weakly liberal camps are able to appear as allies of racial justice while promoting beliefs and values that severely undermine racial progress.
The denial of our racial past, in some measure, means that we are forever doomed to a battle over just how bad things are in our racial present. If we can’t agree — and, really, tell the truth — about the history of race, we can’t tell the truth about the politics of race. The two are indissolubly linked.
The politics of race involves disputes about the persistence of racism; the role of race in deciding the distribution of social goods like education and employment; the place of race in public discourse, whether through presidential commission or informal debates; the political will to address the most damning aspects of discrimination, prejudice and bias; and the acrimonious argument over just how much economic and social resources should be devoted to remedying our racial miasma.
Many whites feel that they — which means the government, because many whites identify themselves as “the nation” — have bent over backward for long enough to accommodate the patently unfair demands of ungrateful and complaining blacks. Many blacks feel that measures such as affirmative action are not the ceiling, but the ground floor, of racial justice, and hence view reparations as the only viable symbol of the nation’s full commitment to bringing true racial justice.
Finally, because race is America’s original sin, there is still a great deal of shame around its discussion that puts roadblocks in the way of open and honest engagement.
Thus, when it comes to race, what philosophers call a category mistake is made: Americans often substitute private belief and personal emotion for public policy and social practice. Many folks were engrossed in discussion over whether Trent Lott was a racist, based on whether he held prejudiced views about blacks, or whether he harbored racial animus in his heart.

Boy, there is a lot to talk about here…
First, and foremost, it infuriates me to hear of race as being “America’s original sin”. Someone needs to read a history book or two. Racism and the exploitation of minority races is as old as human history. It has been a feature of every society I know about in history, and it is a feature of every society I know of in the modern world.
Where you don’t see racial conflict, it is for one of two reasons: 1) because one race has killed or otherwise subjugated another, thereby achieving 2) a racially homogenous society, where racial minorities are amusing oddities, and the realities of living alongside other races and cultures don’t have to be dealt with.
I believe America has done more to openly deal with the issues of racial justice in the 19th and 20th centuries than any country or society that I can think of. Europe is just starting down the road of racial politics that America has been on for the last forty years.
Does this mean we’re done? Of course not. But it’s as ludicrous for leftwing advocates of racial justice to argue that we’re living in plantation days as for rightwing advocates of racial purity to argue that life for African Americans in those days wasn’t all that bad.
Alone among Western nations, America retained slavery into the 19th century, and that was an awful sin. But alone among Western nations, we fought a bloody civil war largely triggered by the moral revulsion of one group within America over slavery, and while that blood doesn’t wipe the slate clean, it certainly has to be looked at.
“Many whites feel that they — which means the government, because many whites identify themselves as “the nation” — have bent over backward for long enough to accommodate the patently unfair demands of ungrateful and complaining blacks. Many blacks feel that measures such as affirmative action are not the ceiling, but the ground floor, of racial justice, and hence view reparations as the only viable symbol of the nation’s full commitment to bringing true racial justice.”
Well, that’s interesting…and points out another flaw in the Left’s approach to race. What is the ceiling? What goal line has to be crossed before we can say that we’ve put paid to race as an issue? What is the vision of the desired end state?
Because it does seem to many like what is asked for is in essence a blank check. And at that point, we’re not making policy, we’re engaging in psychodrama. If we want to win on the issue of race, one of the things that we have to have is a clear vision of what winning looks like. We had that in the 60’s. It was black and white children graduating alongside each other at Little Rock High School. It was black and white kids dating at the prom. It was black faces sitting at the tables of power.
OK, we accomplished that.
But we’ve left millions of African American families behind. In education, in earning power, in hope for the future.
What does the answer for them look like?
Answer that question, my fellow liberals, and we can start getting there.
And as an afterthought: If the old vision was “…black and white children graduating alongside each other at Little Rock High School. It was black and white kids dating at the prom,” how do separate graduations and separate proms for African American students fit into the vision?
(added author’s name to the LA Times commentary)

So I got the carpets cleaned, and found the CD with the backup on it.
Here are the (way overdue in most cases) additions to the blogroll:
Interesting Liberals
Ampersand
Atrios
Rebecca’s Pocket
Robin Goodfellow
RaptorMagic
Just Plain Good Blogs
The Bookslut
The Daily Rant
exit zero
Insignificant Thoughts
Kolkata Libertarian
Rachel Lucas
Jacob T. Levy
Pet Bunny
RAWbservations
Social Design Notes
Sneaking Suspicions
Oliver Willis

As a consequence of living in the Real World. This weekend, we’re getting ready for two parties we’re hosting tomorrow night, finishing up Christmas shopping, getting the annual family pictures taken, and because if there is a God, she has a malicious sense of humor, dealing with a cat suddenly incontinent because of feline diabetes.
We just got the lecture from the vet in how to give the shots, and dropped the scrips for needles and insulin off at the human pharmacy, where the clerk did not even bat an eye.
My gentle suggestion to TG that we sell the cat to a junkie in need of a source of clean needles was met with a steely silence. I may be sleeping in the garage tonight…
…back to scrubbing the carpet…