“Pushing on a string” is a phrase used to describe the Fed’s problem but I think pulling on a noodle is apt because if you pull too hard the noodle breaks. If you have even a passing interest in money you have to be spellbound by the fiscal shenanigans afoot in the global financial markets these days. I was watching CNBC India last night and at one point the host of the show was nearly in tears.
The Fed just made an emergency 75bp (3/4%) Fed funds rate cut. To give you some idea of how unusual that is consider that the last time it happened was after 9/11. Gold is all over the place. Down just like most other commodities until the rate cut and it’s now hovering around 890.
I have no idea what gold is going to do. Based on that last little panic I think the golden goose is going to get thrown out with the bathwater if things get hairy again, and I think they will. People aren’t going to be concerned with the inadequacies of the Fed when their retirement is evaporating and they’re afraid that GLD might fold. If everybody actually had gold stashed in their houses then I doubt prices would be off as much but they trade it just like wheat… which I assume most people don’t hide under their mattresses.
The Fed just applied a band-aid to a stab wound. Deep in my bones/soul I know this is going to get worse based on the degree of dysfunction in housing and mortgage backed securities but the world hasn’t seen anything like this in 70+ years so it’s hard to know where things go from here.
Predictions (take with a large grain of NaCl): Gold: Based on yesterday I’m convinced deflation leads to the death of gold. Some people make good arguments that it can act as a deflation hedge but I don’t see it happening when investors think of gold less as an alternate currency and more like a hunk of useless yellow metal. In other words I think gold tanks in the next month or two (I’m selling tomorrow) as the Fed’s band-aids start to fall off. In the longer term I think it will rally as Bernanke pulls out the stops to prevent credit destruction.
Bonds: Follow the Fed’s lead (or maybe the other way around) to zero. Just like Japan as bottom caller / knife catchers realize that it actually is different this time. I heard someone jokingly predict the American carry trade meaning people borrow dollars at 1% or so and use the money to borrow better yielding currencies.
Insurance: The Fed will bail out Ambac, etc. because if they go down, it all goes down. This will probably work a lot like the FDIC except for muni-bonds and the like instead of consumer deposits.
Real Estate: If the Fed cuts rates to zero the monthly payment on a 500k home is something like $1380 a month. So in theory super low rates could help the foreclosure epidemic. Problems – The FFR doesn’t affect long rates, upon which many mortgages are based. Also, people can’t refinance if they owe more than their house is worth. That includes a LOT of people. Lastly, risk premiums are through the roof, mirroring defaults. It’s hard to get a loan when prices are dropping. If you’re smart enough posses the means to buy a home right now you’re probably also smart enough to know better than to buy a home right now.
The egg heads at Davos are predicting 30% price drops nationally. That sounds about right but that would wipe out Trillions of dollars of wealth which would eviscerate the economy, which would then feed on itself as more homeowners with exploding rates reset through 2011. The only thing that can save housing is a morally hazardous gung ho bail out of people who bought houses in the last 6 years. That would decimate the dollar but Ben appears to be OK with that.
Tech: Bernanke astutely pointed out the other day that US manufacturing has collapsed in part due to rising productivity. Robots kill… jobs. More specifically IT enabled automation. Our economy is having a hard time retraining workers to take advantage of the new jobs that are out there. This is my explanation for a growing wealth divide currently blamed mainly on ill conceived tax cuts.
Inflation vs. Deflation: Short term I think we’ll see deflation. But Bernanke will take more drastic measures as things unwind because, unlike the BOJ, we simply can’t afford deflation. Why? Because the average American is so indebted that deflation would mean the end of the entire financial system we know and love.
Optimism: We’re getting closer to the point where nobody knows what happens next, even the people who could predict things up to this point. If we have a depression I don’t think we’ll see people starving and living in the forrest. Productivity is so much higher now than in the 30′s that a serious push by America’s brains coupled with modern IT would make a collapse a bit more comfortable. The problem is debt. We’re so indebted that a new system could be the only way forward. Maybe that’s a good thing.
Apple stock is off 7% today after releasing another batch of products… I mean “Revolutionary, life altering, lifestyle enhancement devices” at the Mac-World expo. Don’t get me wrong, I love Mac gear, I just happen to think their stock is going to have a rough time going forward.
Why the shellacking? The problem with Apple is that they’ve hit what I call “The good enough limit”. They’re running out of reasons to charge top dollar for their pretty gadgets so they add video screens to Ipod Nanos, flatten laptops, etc. In fact it’s gotten so bad that they’re willing to sacrifice the performance of their products to make them more aesthetically pleasing. In practical terms the MacBook Air is useful because it’s lighter. But that’s where the improvements end. Pretty much everything else was sacrificed to make the Heroin-Sheik look possible.
Computer makers are no doubt terrified that some day the bottom of the barrel PC will be so ridiculously fast that upgrading a PC will feel a lot like buying a newer Ferrari so you can get to the grocery store faster. Better flash based hard drives will eventually solve this problem and I doubt they’re looking forward to it. Hence the $1000 fee for an upgrade to a smaller flash based hard drive for the new Air-Books.
This is called creative destruction and it’s the reason creating products that kick ass and don’t break is dangerous. When the .03 inch thick MacBook-Helium is released for $4000 next year along side the $200 EeePC with Wimax, Macs will start to have the same appeal as unnecessarily large SUVs. In other words, a way to show the world just how important completely impractical material possessions are to you.
The MacBook Air and Kate Moss are both inarguably beautiful but they’re also one crash diet away from complete disaster. Mac is an alternative to Windows for people who haven’t figured out Linux yet. Price cuts are their last option.
Republicans:
Giuliani – This looks like a death sentence. Even Ron Paul beat him.
Huckabee – Massive support from evangelicals probably won’t work outside of Iowa. Skeletons congregating in his closet.
Romney – Considering the amount of money he spent he should have done much better. But his good hair could be the difference.
McCain – He’s going to have another meltdown, too unpredictable to make a good nominee.
Thompson – He could possibly win if his heart was in it. Feels like he’s doing this as a favor for a friend.
Paul – Beat Giuliani, 4% out of 3rd behind a weak 1 & 2. Could do better in New Hampshire, a libertarian friendly state.
Democrats:
Obama – Smart, personable, represents change. Would have good advisers. Untested.
Edwards – Anti-establishment populist, hair is a little too well groomed. Obama beats him unless he slips up.
Clinton – Change is the theme of ’08. Clinton represents anything but change. Smart but Obama tends to out debate her.
Prediction for New Hampshire:
Romney, Huckabee, McCain, Paul, Thompson
Obama, Edwards, Clinton
The slowest, most unreliable, and out-dated hunk of machinery in any computer, Mac, PC, laptop, whatever, is the hard drive. Computers today are like incredibly hi-tech cars driving around on wooden buggy wheels. Millions of dollars are poured into developing faster engines and fancy suspension systems but much of it goes to waste because the wooden wheels can only handle speeds under 30 miles per hour.
Hard drives are basically just shrunken down record players. They are no doubt marvels of engineering but they suffer from an unavoidable need to physically move an arm around, reading and writing data onto a spinning hunk of metal (see video). That’s all changing thanks to the emergence of solid state drives. These are a lot like the USB thumb drives stuck to countless key chains except they’re bigger, faster, and you can install your operating system on them.
Next Level Hardware just reviewed the latest generation solid state drive from MTron and it gives a really interesting glimpse into software performance on computers that are no longer bottlenecked by wooden wheels. I won’t get into the geeky details but the thing basically boots Vista more than twice as fast as the fastest mechanical hard drive. I had Vista on a pretty capable laptop with 2Gigs of RAM and it would take minutes of watching the hard drive light blink before I could actually use the thing. But what if Vista wasn’t such a slug? Would it be worth another look?
These drives are horribly expensive but prices are dropping fast. The NLH review is interesting because it’s now possible to imagine a world where there are no more computer bottlenecks and reliability is no longer a concern because of the durability of the new crop of drives.
If performance and reliability are no longer issues some obvious questions come to mind: Why would anybody ever buy a new computer? Will Vista finally find acceptance once the performance issues are solved by better hardware? If nobody needs to upgrade because even cheap PCs are just good enough will the hardware industry collapse?
I predict that people will continue to buy new computers because the cost will be much lower as we move towards system on chip processors from Intel, etc. near the end of the decade. Got a virus? Just buy a new computer for $40. Storage will be remote if Google keeps its promises so even a computer failure will cease to be a big deal.
I’m still skeptical about Vista. By the time these solid state drives are cheap enough for mainstream use Vista will have been declared dead. There are rumors that MS is working on a really solid new OS that could replace Vista but Linux distributions may be good enough by then that it won’t make sense to pay an extra hundred dollars for a DRM shackled OS. Apple is interesting because standards are making the OS of choice nearly irrelevant but they’re also opening up Apple to competition from Linux. Virtual machines are probably the most interesting development because they will make applications OS agnostic. Want to run Office 07 in Linux or Mac? No problem, just boot XP in a virtual machine and do what you need to do.
Soon computers will be nearly free, performance will be more than anybody can use, at least for today’s apps, and the prospect of spending $150 for an operating system running on a $100 PC seems unlikely. Technically, economically, philosophically, Linux would seem destined for world domination.
While working on HoundWire I would often go to Park Bench Deli (a few blocks up the hill from where that photo was taken) in Altadena with various interesting people. One fellow diner was our intern from Caltech. Caltech is the west coast rival of MIT so they have some very smart professors wandering the halls. One out of every thousand alumni has received the Nobel Prize. The point is that the professors there know a little bit about what they teach.
To the story. Stephen, the undergrad, had a math class that was being taught after an economics class. The econ teacher inconsiderately left his scribbles on the chalk board for the math professor to deal with. According to the story the math professor walked in, looked at the econ equations for a minute, turned to the class and proclaimed “This is complete nonsense”.
A great analogy I heard about economics is that of a book report. Modern economists look at novels, run statistical analysis on the average words per paragraph and paragraphs per chapter. It’s all very scientific, high brow stuff based on calculus. Then they use that information to try to predict what the author’s next novel will be about. Of course that’s completely ridiculous but it explains why economists have predicted 9 of the last 4 recessions.
The Austrian School argues that you can’t predict the author’s next book unless you understand human nature. One of the most important books in the Austrian School is called Human Action by Mises. It just makes sense in my not yet well enough read opinion.
If you have an hour to kill I highly recommend this 70 minute podcast about the Austrian School from the Econ Talk website: http://www.econtalk.org/archives/2007/12/boettke_on_aust.html
I’ve listened to it about 4 times and it’s sinking in. Good for long, caffeinated drives.
Some people prefer to remain blissfully unaware of potential problems in the near future. Others prefer to know so they can plan for what’s ahead. If you’re the type who favors happiness over preparedness then I would advise you not to read this economic forecast from Nouriel Roubini. He’s often quoted by The Economist and WSJ and has a hell of a track record.
“I now see the risk of a severe and worsening liquidity and credit crunch leading to a generalized meltdown of the financial system of a severity and magnitude like we have never observed before. In this extreme scenario whose likelihood is increasing we could see a generalized run on some banks; and runs on a couple of weaker (non-bank) broker dealers that may go bankrupt with severe and systemic ripple effects on a mass of highly leveraged derivative instruments that will lead to a seizure of the derivatives markets (think of LTCM to the power of three); a collapse of the ABCP market and a disorderly collapse of the SIVs and conduits; massive losses on money market funds with a run on both those sponsored by banks and those not sponsored by banks (with the latter at even more severe risk as the recent effective bailout of the formers’ losses by theirs sponsoring banks is not available to those not being backed by banks); ever growing defaults and losses ($500 billion plus) in subprime, near prime and prime mortgages with severe known-on effect on the RMBS and CDOs market; massive losses in consumer credit (auto loans, credit cards); severe problems and losses in commercial real estate and related CMBS; the drying up of liquidity and credit in a variety of asset backed securities putting the entire model of securitization at risk; runs on hedge funds and other financial institutions that do not have access to the Fed’s lender of last resort support; a sharp increase in corporate defaults and credit spreads; and a massive process of re-intermediation into the banking system of activities that were until now altogether securitized. “
As Paul Graham pointed out, the cost of starting a web based business is approaching zero. As someone who’s starting up a .com I can say that the cost has definitely not reached zero yet but Mr. G’s point remains correct. The new barrier to entry is knowledge. Servers may be cheap but a lack of knowledge means you’ll need to bring in expensive consultants to setup your server and network. My goal here is to help the person with an idea and some basic coding skills to get a simple but fast server up and running.
I’m going to cut to the chase for those short on time (it’s a 2800 word post). In order of effectiveness, here are the tweaks I made to my setup to increase performance:
Index tables – easy to do, massive speedup
Try switching your tables from MyISAM to InnoDB. This is not a sure thing and there are tradeoffs but it really sped things up for me
Turn on Apache’s mod_deflate. My pages dropped in size from 100KB to roughly 17KB. Faster page loads, less bandwidth.
Tune your my.cnf file. Out of the box MySQL assumes you have a very slow server.
All said and done, without changing the software, I dropped page load times from .4 seconds to about .04 seconds. Making the thing run 10x faster using more hardware would have been rather moronic
If Web 2.0 is referred to as the Read/Write Web then Web 1.0 should be remembered as the Read Web. The implications for what type of server you’ll need are huge. You could get away with slow drives back then because your rarely changing data would probably be cached in RAM. If you look at a site like Digg it’s another story. Hundreds of people are writing comments, submitting stories, voting on stories, and engaging in various other activities that can be logged to improve the site.
(Updated thoughts: If the CPU again becomes the bottleneck then optimization will become arguably more important. You’ll just optimize for the CPU instead of IO. My application has a quality dial. I can scale back the quality of the results depending on the load on the server.)
There are lots of good bits of knowledge about the various steps involved in building a LAMP server scattered throughout the web but nothing I’ve found really compiles the information into a usable guide. This post is going to focus on server hardware and networking. If this post gets a decent response I’ll take my book of notes about software and turn it into a software HOWTO.
Just a little background. I’m not a the best software engineer nor a hardware expert. The site I’m launching, HoundWire.com, is really about journalism and geography (that’s ‘hyperlocal content aggregation’ if you’re a hipster). The fact that I was able to build a prototype without bringing in expensive consultants and expensive hardware probably didn’t hurt my cause when it came to getting funded.
I’m going to assume that you’re building a database driven web site using Linux-Apache-MySQL-PHP. LAMP is a good place to start if you’re not a computer scientist and just want a working prototype.
Hardware:
8 Gigs of RAM now costs in the neighborhood of $300. Dual core processors are fast, cheap, and getting cheaper. Run of the mill PCs are equipped with stupendously fast server grade PCI-Express slots. For under a thousand dollars you can put together a seriously fast system with one major shortcoming. The storage system.
From chapter 6 of the MySQL high performance Book:
The fundamental battle in a database server is usually between the CPU(s) and available disk I/O performance; we’ll discuss memory momentarily. The CPU in an average server is orders of magnitude faster than the hard disks. If you can’t get data to the CPU fast enough, it must sit idle while the disks locate the data and transfer it to main memory…
This all means that the first bottleneck you’re likely to encounter is disk I/O. The disks are clearly the slowest part of the system. Like the CPU’s caches, MySQL’s various buffers and caches use main memory as a cache for data that’s sitting on disk. If your MySQL server has sufficient disk I/O capacity, and MySQL has been configured to use the available memory efficiently, you can better use the CPU’s power.
IOPS are a good measure of disk performance on databases. The average consumer grade drive can handle 100-150 IOPS. One 15,000 RPM Seagate Savvio is in the 300s. My Raptor RAID array can probably handle 400+. Now consider that a good CPU can handle nearly 100,000 IOPS. Super expensive RAM based drives are useful because they eliminate the drive bottleneck.
If your database rarely changes then disk IO is much less of an issue because most of your data will be cached in system memory anyway. But newish websites like Digg or Reddit have constantly updating discussions in their comments sections. If you want to harness the brain power of the masses you’re going to need a setup that can write that information to a disk at some point.
Desktop PCs were rarely used as servers in the past because they were limited by the PCI bus. Gigabit network cards and a RAID array could easily swamp the meager bus. Now we’re blessed with PCI-Express and the difference between a desktop and server has more to do with reliability than performance. Reliability is nice but you can save a ton of money if you don’t need it.
The bottleneck on your fledgling system probably isn’t going to be your quad core CPU. If you’re squeezing Apache and MySQL into the same box to keep costs low you’ll need to have a good storage setup. So that’s where I’ll begin.
Hard drives are like sports cars. If you take a 4000 horsepower funny car to the Nurburgring it’ll probably lap slower than a Mazda Miata. A drive that boots Vista in 9 seconds may not be very good at randomly writing to a database, which is what you’re probably going to be doing. Drives that specialize in high load database activity, like the much heralded Seagate Savvio, can be had for around $350. That will only get you 36 Gigabytes but you have to ask yourself; is your web app really going to need a Terrabyte of storage? You can argue that more RAM will solve any problem but at some point user input has to be written to the database.
Most consumer grade PCs do not have SAS connectors but you can get a PCI-Express SAS adapter for under $200. So for under $600 you can turn your funky desktop PC with a PCI Express slot into a pretty darned powerful database server. You can add a drive and turn it into a RAID 0 array (72GB) for around a grand. SAS drives are designed for enterprise use and so are much more reliable than a re-purposed desktop drives under heavy load. In other words your RAID 0 array will last a longer.
I went the cheap route and put two WD Raptors (SATA instead of SAS) in a RAID 0 array using a 3Ware RAID card. Ill know within a few months whether or not my idea is going to take off so I’m more concerned with speed and cost than reliability. Whatever you do, don’t plug a SAS drive into a SATA port. I tried once with an adapter cable and fried the southbridge. You can plug a SATA drive into a SAS port though.
It could be argued that your RAID card should be the most expensive component in your server.
(Random anecdote) I once setup two 15K RPM SAS drives in RAID 0 for a mass ghosting operation and we easily saturated the gigabit switch. The photo of that very drive (3.5”) is still used in the Wikipedia SAS article.
Back up frequently and if a drive goes you can just reinstall on the 2nd drive. Doubling the number of drives to make it RAID 10 may be a bit pricey especially considering you won’t see a performance increase. Compared to RAID 0, RAID 5 will slow you down and cost more but you lose the single point of failure.
Motherboard/Case
I bought an ASUS based barebones kit. I’m happy that it can boot from the PCI-Express slot, not happy that it doesn’t support all of my RAM. The ASUS P3-P5G33 might be a better bet if you want support for more than 3Gigabytes of RAM. I’m not saying this is the best option but it works and it’s fairly inexpensive.
Un-interruptible Power Supply
Get a UPS, you’ll sleep better. Power surges aren’t the real issue. We have a big AC unit that kicks in and dims the lights. Mini brownouts ala Sim City. You don’t want that stress on your server. Also, get a Kill-A-Watt or something similar so you can see the power draw and don’t accidentally exceed the rated battery capacity of the UPS.
Cost – On a budget
Barebones PC – $200
4Gigs of RAM – $170
CPU – $230
UPS – $100
————
$700
Storage System:
SAS/SATA RAID card for the PCI-Express Slot – $300
Two Raptors or one Seagate Savvio – $350
————
$650
=====================
$1,350
Cost – Expensive, bang for buck
SAS/SATA RAID card for the PCI-Express Slot – $300
2 x Hyperdrive4s = $8,800 (32GB and 77,000? IOPS)
———–
$9,100
Some Predictions:
* RAM based SSD drives are going to get more popular as people realize that IOPS are more important than drive capacity or peak read performance in web servers.
*Someone will eventually release a RAM Based storage system with SATA-2 support that doesn’t require ECC memory. At that point, for under $2,000, you’ll be able to build a 32GB RAM drive with something like 80,000 IOPS and transfer rates of more than 220 MB/s, without the need for a RAID card.
* People will stop complaining about how slow Vista is. Maybe Microsoft should release this hypothetical device so bloatware truly no longer matters.
*Once that happens hard drives will find a niche as mass storage devices.
Flash based SSDs will remain hugely popular in mobile devices and laptops due to low power consumption.
* Power supplies will start coming with built in batteries which can power your volatile memory if the power shuts off.
* As performance becomes nearly free reliability will become a more important factor when buying a server.
* LAMP setups will become more proactive in tuning themselves. This will remain application dependent but a my.cnf generator based on your system specs will probably emerge. The bottleneck will become application design.
* Caching systems will vanish due to their complexity as drives cease to be the bottleneck (except maybe in very large systems).
* As performance ceases to be an issue we’ll see all sorts of interesting new applications. If Web 2.0 is/was about harnessing the knowledge of the masses and your database server chokes when it’s trying to write to the DB.
*Database performance consulting will become unnecessary because even sloppy code will run quickly. If your page loads in .004 seconds instead of .02 seconds nobody will care.
*I get the feeling Canonical(Ubuntu people) will eventually sell prebuilt LAMP servers. Fonality used to give away Trixbox as a Linux distribution but they recently started selling a pre-built phone “appliance” in addition to support.
* Future hard drives will be large PCI-Express cards populated with bunches of 4GB memory modules running at the speed of the PCI-Express bus. Current RAID cards have upgradeable memory modules for caching purposes. What if you had a virtual hard drive the same size as the RAM cache on the card? (ed. I was close on this one. Check out the video)
Some Random Thoughts:
*The Gigabyte RAM disk was pretty popular and insanely fast but no follow up was ever released in spite of consumer demand. Possibly because you could build an insanely fast storage system with a few of them in RAID which would completely devastate the enterprise hard drive market in a few short months. Yeah that sounds like a conspiracy theory but whenever I write about the HyperDrive4 I get a bunch of blog visitors from web mail accounts.
If Houndwire.com gets some traffic I’m going to push for some help for adding new features and a maybe a HyperDrive.
Networking:
Connectivity – T1s cost about $400 a month but you get 1.5 megabits upstream. Downstream isn’t great but you want to send out web pages not download mp3s right? Use someone else to host your images if you have a lot of them. No sense clogging up you T1 with big jpegs. You can host from a residential internet connection while you’re prototyping. Just be prepared to update DNS and configure port forwarding (and violate TOS). I used DD-WRT on a spare Buffalo to create a wireless bridge and hosted my little LAMP Laptop at home for a while unbeknownst to the guy who’s name is on the bill.
Use solid 10/100 switches instead of fast but potentially flaky gigabit switches, especially for DMZ switches where you’re not going to exceed a few Mb/s anyway. Don’t make your own cables unless you’re a masochist. You can get a multitude of colors cheap from NewEgg.com, or TigerDirect to stay organized. Have a couple crossover cables handy for computer to computer connections (IPCOP -> Server).
Firewall – I’m using IPCop because we had a spare PC lying around. IPCOP is like DD-WRT on crack without the wireless settings. Smoothwall is a cousin of IPCOP and a little easier to get going for the beginner but I prefer IPCop(you don’t have to register to read the manual). If you’ve had a big fancy corporate firewall before IPCOP will disappoint you. My problem was putting the web server on the DMZ with a real IP address. IPCop’s ORANGE zone can NOT handle using an IP on the same subnet as the RED. There are ways to hack it by putting the DMZ IPs in the RED zone and port forwarding to ORANGE but even then you’re only addressing the one IP problem. I’m going to look into DevilLinux which is apparently better suited to non residential configurations with multiple real IPs. The alternative is paying $700 for a 3 port SonicWall. DIY may not be worth your time, especially if you don’t have a spare PC to cannibalize.
Ideally you’d find an old laptop and put a cheap solid state drive in it for reliability. Old ToughBooks are great if you can find one. I once bought a flash to IDE converter and installed IPCOP on a camera sized memory chip. No moving parts means it’s cooler and less likely to fail. It’s still running as far as I know.
Software
For the sake of simplicity get Ubuntu Server Edition. It has a LAMP option when you’re installing which saves a lot of time. Get the 64bit version unless you’re not sure if your CPU is too old.
Don’t be afraid to break stuff in the short run. If you have to reinstall your LAMP stack a few times it’s just practice for down the line when your overheating Raptor RAID 0 array bites you.
MySQL Performance Tuning
Add the code for benchmarking.
Stuff I’m running:
Apache 2
PHP 5
MySQL 5
64 bit v. 2.6 Linux kernel
SSH
Samba
Webmin
Webalyzer
PHPMyAdmin
Here’s a video of an overcaffeinated guy with an equally hype dog explaining the basics.
The linux top command is your friend. You can watch MySQL CPU gobble up less and less CPU time as you optimize things.
If Ubuntu Server edition comes with all of that stuff installed in addition to a functional LAMP stack they’ll have a winner. Sell support to fledgeling startups using RedHat’s model.
Config Files and performance tuning:
Databases:
You can spend days tweaking my.cnf but the big gains early on come from making sure you’re indexing tables properly. If you’re running a query like “SELECT names FROM people WHERE age > 40″ make sure you have an index on the age column.
Crappy holiday retail sales will be blamed on toxic toy imports so the cheerleaders at CNBC don’t have to admit housing is affecting the greater economy.
The housing market is about to get a lot worse. There are still believers but they’re quickly running out of money. Mortgage resets get really ugly next year.
Ipods/MP3 players of the future will run bit torrent and cache streamed audio over the internet. Radio stations will consist of math majors working on recommendation algorithms based on geography and local information like concert info will have to be peer to peer because custom playlists mean songs can not start at the same time. And it will be prohibitively expensive to distribute that many custom feeds from a centralized location. DJs won’t talk over songs (it’s more expensive anyway because they have to do it live) and every 20 songs or so you’ll hear the latest local music news.
I don’t agree with everything Ron Paul says but I agree with enough of what he says. You can bet on who will win the Republican nomination. People with money on the line at InTrade are giving him a 6.1% chance of winning. That’s incredibly high considering his small government views. My assumption is that most Americans want bigger government for one of two reasons. To blow up foreigners they don’t like or to create more bureaucracy in an attempt to fix human nature.
Edit: Welcome FDIC! They just read this post. (They’re the people that insure your deposits at the bank for up to $100k) I’ve been on a mad rant for the last 3 years about how the housing market would crash and bring down the entire economy. So now I bring you the same point of view but from someone being interviewed on Marketwatch. Keep in mind that this guy is considered an optimist by many who don’t think the global economy can weather a drop in US consumption…
Even if this isn’t a credit bust and everybody manages to keep paying their mortgages, the drop in home equity and associated spending would be enough to bring down our 70% consumer spending based economy. But of course people aren’t paying their mortgages as evidenced by the following graph of foreclosures.
So on top of the missing equity we’re going to see a banking crisis. Don’t take my word for it. This is from a memo sent to insiders, the kind of thing the general public doesn’t see unless leaked (from the Calculated Risk comments section via Lehman Asset Management)
Investment Conclusion
As market conditions have calmed, the performance of equity market neutral
managers has become more challenged. We believe performance of most factors,
particularly value factors, have turned perverse in a dramatic fashion. It is
unclear how long current situation will persist, but stability will be
facilitated by understanding that underperformance is systematic, not due to
individual model misspecification, in our view.
Summary
* July 2007 saw returns to quant factors increase dramatically, with
volatility at 2 to 3 times normal levels. In July, the misperformance of the
factors was driven primarily on the long side.
* Factor returns for the first 5 trading days in August have been roughly of
the same magnitude for what we experienced for all of July, however, now
model misbehavior has primarily been the results of shorts outperforming.
* Factor performance does not appear to have notable sector biases so we
discount the possibility of sector rotation being responsible for observed
dynamics.
Over the past few days, most quantitative fund managers have experienced
significant abnormal performance in their returns. It is not just that most
factors are not working but rather they are working in a perverse manner, in
our view. The names that are short are outperforming, often notably, while
the names that are long are underperforming, although less severely.
Moreover, there appears to be very little news coming out surrounding these
names and all of this is occurring against the backdrop of the general
markets appearing to calm down. This has led to our fielding a large number
of calls from our quantitative asset management clients, trying to understand
what is occurring in our market.
It is impossible to know for sure what was the catalyst for this situation.
In our opinion, the most reasonable scenario is that a few large multi-
strategy quantitative managers may have experienced significant losses in
their credit or fixed income portfolios. In an attempt to lower the risks in
their portfolios and being afraid to “mark to market” their illiquid credit
portfolios, these managers probably sought to raise cash and de-lever in the
most liquid market – the U.S. equity market.
As these managers unwound significant factor based portfolios, these factors
started to behave in unexpected and potentially troubling ways. Short names
started to rally and long names started to fall as these trades started to
hit the market. As most quantitative managers use similar quantitative
factors, this abnormal factor phenomenon was not confined to a few funds.
Rather, a large number of quantitative managers have seen their models begin
to behave in unexpected ways. Again, it is no longer only the multi-strategy
managers, but now pure quantitative equity managers who have started to see
their portfolios “misbehave”, both U.S. domestic and global fund managers.
To be absolutely clear, when we are discussing misbehavior, it is not simply
that model returns are flat (or not working) but specifically that the models
(ours included) are behaving in the opposite way we would predict and have
seen and tested for over very long time periods (45+ years). Additionally,
the magnitude (or volatility) of the returns in July increased substantially
with the factor returns being on the order of 1 to 4 times what we have
traditionally seen. Those returns look placid relative to what we have seen
in the first six trading days of August. As for what has been transpiring in
August, we have been able to document daily returns of this magnitude
occurring before only at the height of the bursting of the Internet Bubble
and in the late 1960s. This appears to be an event with little precedent.
This breakdown in traditional factor returns is, of course, not limited to
the return (or alpha) side but also is now extending to traditional risk
models – that is, managers are finding their risk models are now
miscalibrated for the current market environment. Again, this is not limited
to any one model but overall to all factor based (or structural) risk models. This has led to further concerns within a number of asset management
organizations we have been speaking to, with risk-managers facilitating
further de-leveraging as they seek to better understand the situation.
Emphasis mine. Sucks to be starting a business in the midst of this ugly inevitability.
“Pushing on a string” is a phrase used to describe the Fed’s problem but I think pulling on a noodle is apt because if you pull too hard the noodle breaks. If you have even a passing interest in money you have to be spellbound by the fiscal shenanigans afoot in the global financial markets these days. I was watching CNBC India last night and at one point the host of the show was nearly in tears.
Apple stock is off 7% today after releasing another batch of products… I mean “Revolutionary, life altering, lifestyle enhancement devices” at the Mac-World expo. Don’t get me wrong, I love Mac gear, I just happen to think their stock is going to have a rough time going forward.
Republicans:
Next Level Hardware 
As Paul Graham 
IOPS are a good measure of disk performance on databases. The average consumer grade drive can handle 100-150 IOPS. One 15,000 RPM Seagate Savvio is in the 300s. My Raptor RAID array can probably handle 400+. Now consider that a good CPU can handle nearly 100,000 IOPS. Super expensive RAM based drives are useful because they eliminate the drive bottleneck.
Back up frequently and if a drive goes you can just reinstall on the 2nd drive. Doubling the number of drives to make it RAID 10 may be a bit pricey especially considering you won’t see a performance increase. Compared to RAID 0, RAID 5 will slow you down and cost more but you lose the single point of failure.
Edit: Welcome FDIC! They just read this post. (They’re the people that insure your deposits at the bank for up to $100k) I’ve been on a mad rant for the last 3 years about how the housing market would crash and bring down the entire economy. So now I bring you the same point of view but from someone being interviewed on Marketwatch. Keep in mind that this guy is considered an optimist by many who don’t think the global economy can weather a drop in US consumption…