Reasons Unbeknownst

[Edit: I just got a bunch of traffic from Nvidia and Sandia labs so I'm thinking I may be on to something here]

So I was sitting here thinking about how modern RAID cards are completely inadequate when it comes to solid state hard drives (as I often do), and I had an idea. Why can’t graphics card hardware be used to accelerate software RAID? We have CUDA right?

For a second I thought I was onto something truly original but after a quick search it looks like some guys over at Sandia Labs beat me to the punch by a few months:

“One example is the RAID software developed by researchers at the University of Alabama and Sandia National Laboratory that transforms CUDA-enabled GPUs into high-performance RAID accelerators that calculate Reed-Solomon codes in real time for high-throughput disk subsystems (according to “Accelerating Reed-Solomon Coding in RAID Systems with GPUs” by Matthew Curry, Lee Ward, Tony Skjellum and Ron Brightwell, IPDPS 2008). From their abstract, “Performance results show that the GPU can outperform a modern CPU on this problem by an order of magnitude and also confirm that a GPU can be used to support a system with at least three parity disks with no performance penalty.” I’ll bet the new NVIDIA hardware will perform even better. My guess is we will see a CUDA-enhanced Linux md (multiple device or software RAID) driver in the near future.”

The problem for using this sort of tech in the consumer space is that people want to boot from their ridiculously fast RAID arrays and software RAID makes that difficult (though not impossible with Linux if you don’t RAID the kernel). If you’re using a software RAID array and the OS crashes you can lose an array.

Here’s a link to the actual paper.

Just had another idea. If a RAID driver could check for CUDA support and only use it if present then an OS could boot in a slower CPU RAID setup until the graphics drivers(CUDA) loaded at which point things would get a lot snappier.

In the future I’m going to guess we’ll see hard drives on PCI-E cards with ONFI flash chips and NO SATA PORTS. With RAID handled by some standardized version of CUDA. Operating systems will have ONFI drivers baked in. This would effectively collapse the hardware RAID industry as we know it.

SLC NAND is only going to get cheaper. I imagine we’ll see 100GB drives containing 200GBytes of NAND. 150GB for a RAID 5 array and 50GB for wear leveling. At some point diminishing returns kick it. Are we really going to need three gigabytes per second of bandwidth from our drives to open word documents?

Wikipedia on
CUDA:
http://en.wikipedia.org/wiki/CUDA

ONFI:
http://en.wikipedia.org/wiki/Open_NAND_Flash_Interface_Working_Group

I have a system of beliefs I use to view the world and it has served me pretty well in making predictions but it has a big hole. So I drive around in my theory without a complete understanding of the engine, but it gets me from point a to point b effectively.

I’m confused by the fact that something that is not really a need can have value. We pay people to make us laugh but you can’t survive on laughter despite that famous quote about medicine. I have a theory and I’m going to use ‘laughter’ in place of anything intangible that we want but don’t need that we might pay money for.

Our ability to pay for laughter is equal to GDP (Gross Domestic Product) minus something I’ll call NTNP. Necessary tangible national product. NTNP is the economic value of the production of things like food, housing, transportation, clothing. Basic necessities which are the foundation of the economy in the 3rd world but that also exist in the 1st world. The NTNP as a percentage of GDP is probably a lot higher in places like Ethiopia where people struggle to survive and don’t spend $100 a month on HBO.

So in my thought experiment GDP = NTNP + EMNP. EMNP is the reason Eddie Murphy is a fabulously wealthy comedian, it’s the value of everything produced people buy after they’ve paid for necessities. Funny people in Ethiopia probably aren’t very wealthy because in Ethiopia they eat tomatoes, we throw them.

For the sake of argument
US economy: 20% NTNP and 80% EMNP.
Mexico: 50% NTNP and 50% EMNP.
Ethiopia: 90% NTNP and 10% EMNP.

The interesting point is that even if you’re having a bad year in Ethiopia the economy will never collapse due to a lack of confidence like it can in America. Eddie Murphy National Product on the other hand is ephemeral, subject to the whims of collective grumpiness.

One of my beliefs is that accelerating technological change is squashing the middle class. When everybody is driving electric cars in ten years there will be a million or so mechanics out of work because electric cars are so much easier to maintain. That’s just one example. The newspaper business is on the brink of collapse due to the Internet, etc.

I’m going to coin a new term to describe what our middle class is facing. Schumpeter came up with creative destruction. Kurzweil has accelerating change. Mash them together and you get the ugly term Accelerating Destruction.

EMNP will continue to shrink in the future as the rich get richer. Why? Because wealthy people only have so much time to go to the movies and Olive Garden. But those businesses employ a heck of a lot of middle class people.

We need to learn to live with deflation. The number one criticism of deflation is that it strangles the economy because people stop spending and wait for things to get cheaper in the future. If that was really a problem nobody would ever buy a laptop. Fear of deflation is probably due to the knowledge that our bubble economy sat precariously atop debt funded pillar of happy thoughts.

I get the sense that policy makers know that confidence is a prerequisite (hence the blind optimism) but they don’t seem to get that the confidence was paid for with unsustainable debt.

Wealth redistribution is not technically fair in my opinion (Bastiat) but it may be the only way to prevent the complete collapse of capitalism given accelerating destruction. Someone has to keep the people paying for laughter.

Three thoughts on solid state drives given the new realities/products out there today.

There are rubmlings of the inadequacy of the 300MB/s limit on SATA 2.0 due to the ridonkulous speed of SSDs. While it’s true that some SSDs are hitting the limit during sustained reads, random writes and even random reads are typically nowhere near the limits of the SATA bus.

The point? Focus on the bottleneck. The bottleneck is MLC flash memory not the bus. The solution? SLC flash will give better performance for many uses on a PATA bus that you’ll get from SATA2/MLC. SLC is more expensive but at some point the price will drop make MLC obsolete . The other benefit of SLC is that it tends not to stutter like MLC.

Older laptops with SSD upgrades are the new Netbook. A lot of the older laptops use the PATA interface (instead of SATA) but it’s really not much slower than the original SATA 1 specification. Some more info here.

And finally, OCZ just released their Z-Drive. It’s a PCI Express based drive with pretty good performance but it just feels poorly engineered based on my reading. Instead of creating a new chip to handle more channels they’ve just created a power sucking beast comprised of at least three RAID controllers.

An ideal solution would be
Memory -> Controller -> PCI-E

OCZ appears to have created:
Memory -> RAID controller -> Controller -> PCI-E
Memory -> RAID controller

Well, lots going on. I just moved, job is getting crazy, NewsDiego is getting more upgrades, time is short. I miss writing as much as I used to.

On the plus side, I got a new Moka Express Italian coffee maker which is why I’m writing this..

Hell of a stock market rally. Not convinced this is a bottom. The government could come in and surreptitiously mask the astronomical losses but it will cause inflation (eventually).

My theory from a few months ago was that you can’t really have inflation until the asset bubbles have all completely collapsed. It never felt quite right so I revisited it and came up with a better theory.

In my view, there are two types of asset bubbles. The first is of the housing bubble nature. Decreasing regulation, lending standards, and interest rates create irrational exuberance and excessive borrowing, gambling, and leverage. Prices shoot up and then crash.

The second bubble is harder to spot because there is no run up in prices. I’ve written about this before but the idea starting to get more mainstream.

http://www.techradar.com/news/world-of-tech/what-if-our-tech-is-good-enough–589169?src=rss&attr=all

“The problem of ‘good enough’ is a huge headache for the tech industry. When your computer isn’t good enough – when a slow processor, meagre memory and tiny hard disk struggle with even everyday tasks – you’ll buy a better model as soon as it becomes available.

Now, though, the weakest link isn’t your PC: it’s you.

Will a 200-core processor make you type an email more quickly, make you work more productively or make your Facebook status updates any more amusing?”

The second bubble was simply the side effect of an inefficient economy. It’s now deflating because hardware is good enough. The hardware industry will be the first to collapse as systems on a chip will eventually dominate. Netbooks are a step in that direction.

Software will continue on as a driver of growth but it too will eventually succumb to standardization and the open source movement. New ideas will require new software but the industry as we know it will probably grow and then shrink. The growth of the software industry will come at the expense of the jobs being automated by that software.

Innovation will spring from the social side effects of communication technology. In fifteen years rich people will not be founders of tech startups. They will be incredibly smart organizers and communicators and then there will be the artists. But I guess you could argue that artists are just smart communicators.

We will all be better off on average after these industries collapse but there will be a massive push to tax the wealthy to maintain a middle class. If we would just let asset prices track the fall of the second type of bubble I mentioned everything would be just fine. Wages would come way down but prices would be dropping even faster.

It’s pretty apparent that our rudimentary financial system cannot cope with the deflation of the first type of asset bubble. I doubt it will be able to cope with the second, larger type.

I think I just figured something out. If the government / treasury print money and mail checks to people it will only hasten the decline of the economy because inflation is hard to control and it creates inefficiency and uncertainty.

Insight:
The inflation vs. deflation debate is largely irrelevant. Standard of living can decline during inflation or deflation. Asset price to income ratios will normalize but real middle class income will necessarily decline as the 2nd great bubble deflates. Our banking system is not designed to cope with this sort of change.

So regarding stock markets. To the extent that prices are based on earnings they will decline. Inflation may actually happen on the way down which could push stock prices higher but if we get to that point the currently rally will have collapsed long before.

Inflation is the government’s tool to smooth out economic fluctuations. But they are attacking an albatross with a fly swatter and everybody is debating the brand of fly swatter.

The 3rd world isn’t sitting on a bubble of job creating inefficiency. Not only do they get to skip the installation of phone lines (they’re starting with cell towers) but they don’t have to deal with the collapse of a middle class employed erecting telephone poles.

Good new TED Talks:
Cell phones and poverty:

Electric Cars:

Finally some criticism of CNBC is starting to spread through the interwebs. Great ending…

At least according to Google’s autocomplete feature.

Well apparently there is a method to their madness. Here is the explanation:

“The late Ayatollah Ruhollah Khomeini, leader of Iran’s Islamic Revolution, was famously quoted as saying, “Economics is for donkeys.” It is a maxim Ahmadinejad has mirrored assiduously, saying once that he prayed to the Almighty that he would never know anything about a subject he views as a tool of western domination.”

Side note. If you want to get a screenshot of the Google dropdown list of suggestions in Linux use the import command.

The people who are saying this stimulus bill will work are the same people who said there was no housing bubble.

This stimulus legislation is an empty gesture/package. It’s too small to make a difference given the magnity/gravitude of the situation and the deficit left in its wake will only contribute to the decline of the dollar.

Local hero Rich Toscano recently chimed in with a great (but flawed in my opinion (because of China)) article that got me thinking. If the only thing that really matters is foreign perception of dollar weakness in this whole mess then it make sense to focus my prognosticating on the global mind-games that must be going on right now. So here are some thoughts on how this plays out:

Theory #1:
Governments and central banks around the world are delaying the inevitable devaluation of their currencies for as long as possible. If they’re the last currency standing then there is a chance that they become the world’s next reserve currency. This allows them to buy time by sticking other countries with the bill.

When the next reserve currency emerges, dollar, yuan, or otherwise, the central bank in charge of it will no longer have to constantly worry about flight to other currencies. This is when I think hyperinflation really becomes a possibility. This is also when I think it will make sense to buy real estate and maybe gold.

Theory #2:
I used to think that hyperinflation was paradoxical. If the economy is cratering, how is it that wages can go up? I have a new theory on this. Zimbabwe’s economy is currently suffering from hyperinflation. It is based on production of things needed for survival. Our economy largely consists of jobs and products designed to accommodate wants. In an economy based on needs production has to continue or people die. In an economy based on wants production can fall until it becomes an economy based on needs.

Theory #3:
A wants based economy creates service sector jobs. A needs based economy is largely automated (with the exception of 3rd world countries). If my theory is correct then job losses now should gather steam until they’re worse than during the great depression. This is the case because back in the 30s we were a country of farmers without robots. People kept their jobs because those jobs (literally) put food on the table. Now we’re a country of accountants, mortgage brokers, and life coaches.

Theory #3b:
Hyperinflation will not happen in the US unless and until:
a> We either lose reserve currency status due to a lack of fiscal discipline
b> Or we remain the reserve currency but our bought time runs out.
c> And we become a needs based economy (it’s hard to imagine how painful this would be).

Timeline:
So what happens next? Here’s my best guess:
2009 - People really start to understand that it is different this time. The stimulus package fails and job losses mount. Home prices continue to decline because rents decline and price to rent ratio reversion to mean works its inevitable magic. States like California begin to face insolvency, increasing the burden on the Federal government (including the Treasury).

2010 - Government failures in large European needs-based economies. We’ll be in nearly as bad shape but people don’t expect much from the government here in the US so the outrage will be a bit less severe. Those governments will respond by printing money and mailing stimulus checks to their citizens thus destroying their currencies and credit ratings. The Euro breaks down.

2011 - US vs. China showdown. The country with the happiest citizens wins the reserve currency war. China isn’t a democracy but they’re worried about social unrest just like we are. Democracy and accelerating change are mutually exclusive. I give the edge to China’s currency here.

2012 - The end of deflation. Buy land. The US begins its transition to a needs based economy. Productivity growth means that wealth redistribution is going to be massive. Out of necessity. The Republicans were right up until about 1990. They’re still right about free markets but they’re dead wrong on redistribution.

If you’re wondering why most economists didn’t see this coming it’s all explained right here on Economist’s View.

Random thought of the day:
My biggest monthly expense, outstripping rent, food, and even beer… is federal tax. Maybe we should all look at the bright side of this financial mess. By losing our jobs we will also lose our biggest expense.

In that vein, something like 10% of famous quotes go like this:
“We don’t stop playing because we grow old; we grow old because we stop playing.”

I was thinking about writing some software that would autogenerate these things.
We don’t X (for, because, to) Y, we Y (for, because, to) X.
Some completely nonsensical examples:
We don’t eat because we’re hungry, we’re hungry because we eat.
We don’t strive for greatness, greatness strives for us.
We shouldn’t exercise to get in shape, we should get in shape so that we might exercise more.

I think this is where Horatio Caine from CSI Miami gets his quotes:
“When you have everything, sometimes it feels like nothing”

Quick thought on Black Friday sales being ugly but better than expected. All of the people who decided to forgo car and truck purchases due to the recession are likely to spend some the money they saved on holiday shopping.

This looks like a shift from big ticket items (cars) to small ticket (gifts). Those looking for any sign of the bottom are quick to latch on to these stats. I’m not convinced that we’re through the woods yet in spite of economic deforestation / bailouts.

For next time. The graying of credentials. Degrees are losing their luster as employers learn to ask the right questions and interpret online reputation.

Image by net_efekt

Sometimes I post in the comments of blogs I like and realize that they might make for decent blog posts so here are my thoughts on the market from Critical MAS.

Back in the day the smartest people in the room thought that the earth was at the center of the universe. They had fancy diagrams explaining the solar system and top calligraphers documenting the system.

There aren’t many calligraphers left these days. They’ve been replaced by anchors on CNBC.”

Other stuff:

• Made some Improvements to NewsDiego, a local news site I’m building for San Diego.
• NYTimes cover story on VoiceOfSanDiego’s success as a new breed of news org.
• Signed up for Twitter, @kirkfhouse , need to start using it more
• Met with some entrepreneurs the other day. Interesting talk about the future of small business
• To summarize their thoughts - Automating customization is key. This is a problem I’ve faced trying to do local news across the country with Houndwire.com.
• Business idea: Using news statistics to adjust ad campaigns in real time without human intervention.
• Business idea: Layoffs are partly a result of businesses not considering the possiblity of simply cutting wages. The problem is partly due to inflation expectations. During inflation people generally don’t take pay cuts. Inflation is no longer a problem, at least for now. Businesses should offer pay cuts before laying people off only if it makes financial sense. The other problem is that businesses aren’t used to deflation so they aren’t well versed in cost cutting. Landlords are in the same boat. They don’t cut rent. If your boss cuts your pay and your landlord doesn’t cut your rent you might have to move. If you own a home you’re just screwed.
• My Xbox 360 died. They’re fixing it for free. Going to get Netflix tied into it. When I get my own place I’m not getting cable if I have Hulu and Netflix. Need a good front end for media file playing, preferably with RSS/Torrent support
• I predicted the economic trouble we’re in but I’m not really sure how America and the world will deal with the dramatic lifestyle changes that are going to be required.
• Optimistic about the future of this country. A real economy not based on debt will probably suffer from deflation. If we can break the cycle of inflation in spite of monetary policy then I’ll have even more hope.
• Deflation is like tough love. We’re not going to spend just because we’re afraid of inflation eating away at our wealth. We’re going to have to actually have money to buy things.
• Watching price drops on SSDs with glee. The Intel X25-E keeps me awake at night. No good reviews yet. Nobody is using IOMeter to compare random write IOPs. What the hell. This thing will also shine on application installation and database benchmarks

Update, some numbers on IO now available, these are RAID-0:

The brilliant minds running our financial system just announced yet another new housing bailout.
Gov’t launches new loan aid effort

There is only one analogy I can think of that does this ridiculous situation justice…

The people attempting to run our country for the last few decades are like pilots on a plane without landing gear. Instead of attempting an emergency landing on a freeway somewhere they’ve been gaining altitude to keep the passengers from looking at the ground. They pushed it too hard and the engines caught fire. Convincing us to jump out of the plane(bailouts) to avoid incineration (financial collapse) was not very difficult.

We’re now hurtling towards the earth, looking at the pilots for guidance. They’re flapping their arms and smiling. Since most of us have never skydived before we begin flapping our arms too, hoping for the best.

Why do the pilots want us flapping our arms? So we don’t punch them in the teeth on the way down.

Photo by Rick Neves