I guess we've all heard of the "free performance lunch". For many years we have enjoyed a "free" performance improvement for our apps in line with CPU performance gains. Buy a newer machine and your apps run faster. Basically we're getting more cycles which makes our slow apps faster over time. Whilst Moore's Law predicts exponential growth, this relates to transistor densities, and we've seen a spectacular growth in data storage, but the rise in clock speeds has slowed, and the growth is now to multi-cores. The free lunch is over.
In line with this trend, Windows 7 can scale to 256 processors.
But even though the free lunch is over, the Redmond Steakhouse will be open for free dinners if you build software for multi-cores now. (The gag continues, Winston)
The Moth has some wise words of advice to start now. I really recommend his recent articles on Fine Grained Parallelism and Do NOT Explicitly Use Threads for Parallel Programming.
He says "Our goal with parallel programming is to write once and have our code scale well as the hardware underneath it gets better, i.e. see incremental benefits when running our app on machines with more cores without changing the code." Chunking your app into fine grain partitions which can be threaded provides part of the answer, but "we need some kind of user mode "engine" to schedule only as many threads as the number of cores on the machine and we need this to take place automatically for us. We also need to be able to partition the overall compute-bound operation into many work items that will get executed by the "engine"." That engine is there now, but improvements are coming in .NET 4.
Daniel's blog also has articles on Threading vs Parallelism and the Parallel extensions in .NET 4 (which are also available in CTP for .NET 3.5 now).
Also worth keeping an eye on is the Microsoft Parallel Programming team blog.