click to visit home page
leading the way to the new storage frontier .....
image shows mouse battering down door to cheese store - click for RAM SSDs  directorypage
DRAM latency
DRAM's latency secret ....
hybrid DIMMs
hybrid DIMMs ..
the fastest SSDs - click to read article
the fastest SSDs ..
SSD ad - click for more info

what's RAM really?

RAM in the post modernist SSD and Memoryfication Era

by Zsolt Kerekes, editor -

RAM used to be components:- SRAM, DRAM etc. Now after 10 years of the PCIe SSD market learning curve experience we have to rethink the RAM concept. In the virtualized world of SSDs everywhere - RAM is whatever the software thinks is RAM - always provided that the typical average performance (at the apps level) meets user QoS needs. And just as there have always been more than one type of real physical RAM - there are now compelling ecosystem arguments (and user value propositions) for supporting different multiple types and tiers of of SSD virtualized RAM too. So in effect - RAM has changed from being tied to a physical component to being a virtualized systems software idea - and the concept of RAM even stretches to a multi-cabinet memory fabric. Maybe it was always so. There was more to RAM than the simple idealized notion which most of us carried in our minds. Shattering that cozy RAM-in-a-chip picture forever is a new truth now that the economic impact of the software and the systems architecture is nearly as important to the RAM (which you think you're working with) as the raw nanometer layout of the memory cells.

SSD news - ping pong trade restrictins and Newton's 3rd Law
can memory do more?
storage market research
DRAM's indeterminate latencies
optimizing CPUs for use with SSDs
can memory chips be made in the wrong place?
cloud adapted memory systems - the new box?
controllernomics and risk reward with big "flash as RAM"
an SSD view of semiconductor memory boom bust cycles
InfiniBand - history from IB dawn to dusk - NVMe fabrics and Gen-Z 1.0

RAM news - ain't what it used to be

re Micron's "soft-announcement" of a DRAM replacement SCM

Editor:- October 30, 2018 - A new article - the Divorce - Micron and Intel See Different Futures - by William Tidwell - who is a regular memory market commentator on - discusses among other things:- the need for in-memory processing and signs that Micron is working on a new SCM memory architecture.

Re the interdependency of memory and processor architecture Tidwell says - "...memory in today's system architecture is locked in a fatal embrace with the CPU. It is stranded."

And as part of a multi-page analysis and detailed speculative look ahead as to what Micron will do with the fabs it gets from buying out Intel's stake in the joint IMFT venture - Tidwell says - "Under promise, over deliver - is the right strategy for Micron in the wake of Intel's 3DXPoint misadventures. I believe this is what they are doing and that gives investors reason to hope that the New Memory will be commercialized in FY '21." the article

data integrity in DRAM - new paper lists and describes known techniques

Editor:- September 19, 2018 - A Survey of Techniques for Improving Error-Resilience of DRAM is a new research paper by Sparsh Mittal - Assistant Professor at IIT Hyderabad and Maruthi S Inukonda published in the Journal of Systems Architecture.

The authors say - "Aggressive process scaling and increasing demands of performance/cost efficiency have exacerbated the incidences and impact of errors in DRAM systems. Due to this, improvements in DRAM reliability has received significant attention in recent years from both academia and industry. In this paper, we present a survey of techniques for improving reliability of DRAM-based main memory. We classify the works based on key parameters to emphasize their similarities and differences. This paper is expected to be useful for computer architects, chip-designers and researchers in the area of memory/system-reliability. the article

Editor's comments:- Among other things this paper has a detailed analysis of multi-dimensional ECC schemes, multi pin correction codes (discussion of fault coverage and overheads), the efficacy of retirement schemes for pins and chips, chipkill, RAID and various schemes for stacked DRAM arrays.

rethinking memory systems design - 2018 horizons

Editor:- July 27, 2018 - I saw a succint summary of the deep question - "move data or compute locally?" - yesterday on pages 183 to 187 of Rethinking Memory System Design Robustness, Energy, Performance (290 pages - pdf) - by Professor Onur Mutlu in a keynote he presented July 3, 2018.

"A memory access consumes about 1,000x the energy of a complex addition."

"Data movement is a major system energy bottleneck."

"We Need a Paradigm Shift to... make computing architectures more data-centric."

In the 2nd half of this (long) paper Onur describes the state of advanced research and thinking into proposing and evaluating design solutions which intersect with the ideas of optimizing data movements and processing inside memory chips and memory arrays. the article (290 pages - pdf), more papers like this by Onur Mutlu

new article - re the utility of very much faster RAM

Editor:- May 14, 2018 - are we ready for infinitely faster RAM? (and what would it be worth) - is a thought provoking new article on

If someone could offer you a memory system which had the same storage density (bits per chip / module / box) as mainstream RAM - but which had latency and bandwidth (as measured by what the application sees) which was infinitely faster - could we use it? - how much would that be worth? and how would it change markets? the article

improving the latency and energy of commodity DRAM using adaptive architecture

Editor:- March 13, 2018 - The enterprise flash SSD market has a long history of design advances which came from the cumulative understanding gained by the independent characterization of memories - this mostly having been done by independent SSD and controller companies rather than the original manufacturers of the flash memories themselves. But I haven't heard much in the past 10 years about similar activities related to DRAM - and part of the reason may well be that the companies which used to do such in depth RAM characterizations in earlier phases of SSD history - the RAM SSD companies like Texas Memory Systems and Solid Data Systems - had mostly stopped design work on new high capacity RAM SSDs by about 2008 due to the competitive advantages (in a storage array context) of enterprise flash.

So I was surprised and delighted to come across a new report - Understanding and Improving the Latency of DRAM-Based Memory Systems (pdf) - by Kevin K. Chang - Carnegie Mellon University (submitted December 2017 as part his PhD) which document (in 200 pages approx) describes his ongoing work and insights into DRAM characterization and system optimization opportunities.

Chang's research measured and analyzed the relationships between supply voltage and latency in commodity DRAM and explored ways to optimize latency while still maintaining data integrity and reducing power consumption. Among several schemes also described in this paper:-
  • an adaptive latency scheme he calls "Flexible-LatencY DRAM (FLY-DRAM)" which leverages the variation of latency that occurs within different locailities of DRAM chips.
  • Voltron - a new mechanism that improves system energy by dynamically adjusting the DRAM supply voltage using a new performance model which is based on a better understanding of the relationships between cell retention, refresh rate, temperature and other system factors. the article (pdf)

is FRAM museumware?

Editor:- January 7, 2018 - It's been a long time since I heard anyone advocating FRAM.

A recent news update from Fujitsu Semiconductor says that its FRAM devices are displayed in an exhibit called "New Electro Hall (Link to Cyberspace)" in the Science Museum, Tokyo.

Fujitsu's current FRAM products have very low capacity (kb upto 1Mb) and low speed (1 to 3.4MHz) but they have very low power operation too and can be used in "batteryless" systems which harvest power from non traditional power sources such as vibration - using piezoelectric transducers.

Editor's comments:- Having said that - don't dismiss FRAM as this may be a type of museumware whose glory days are yet to come.

A new article this week - A New Memory Contender? - in surveys the history and technology trends in ferroelectric memories and describes potential successors to FRAM called "FeRAM".

Samsung improves 10nm DRAM speed and yield

Editor:- December 20, 2017 - Samsung today announced today that it has begun mass producing the industry's first 2nd-generation (faster) 10nm class 8Gb DDR4 DRAM. This has been accomplished using legacy fab processing without needing yet the recourse of next generation EUV .

Re data integrity:- Samsung says a newly devised data sensing system enables a more accurate determination of the data stored in each cell, which leads to a significant increase in the level of circuit integration and manufacturing productivity.

Re industry memory shortages:- Samsung's new 10nm DDR4 features an approximate 30% productivity gain over the company's 1st–generation 10nm-class 8Gb DDR4.

Nanya presents overview of the memory market

Editor:- December 14, 2017 - An overview of the $120B (in 2017) memory market - which consolidates data from various market research sources appears in a Presentation to Analysts and Investors (pdf) - published today by Nanya Technology .

In 2017 worldwide revenue of DRAM was approx $69B - up 67% YoY.

In 2018 worldwide wafer starts for DRAM will increase moderately to 1,210K/month.

DRAM wafer starts

Series A funding for RISC CPUs in DRAM

Editor:- September 8, 2017 - UPMEM - a fabless semiconductor startup - today announced 3 million Euros series A funding for its Processing In-Memory technology.

This integrates user-API accessible RISC processors as SoCs in DRAM. The company has been reported in eeNews (Oct 2016) as saying...

"The fundamental benefit of processing-in-memory is the combination of DRAM and CPU. We attach 1 DPU per DRAM bank. It means 16 cores per 8Gbit DRAM chip. On a 16Gbyte DIMM, we deliver 256 cores, and 8 of them can be added to a standard CPU socket. We end up with a co-processing system of 2048 cores together with 128Gbytes of DRAM per socket."

The PIM chip, integrating UPMEM's proprietary RISC processors (DRAM Processing Units, DPUs) and main memory (DRAM), is the building block of the first efficient, scalable and programmable acceleration solution for big data applications. Associated with its Software Development Kit, the UPMEM PIM solution can accelerate data-intensive applications in the datacenter servers 20 times, with close to zero additional energy premium.

"We are no longer in an era were CPUs and other hardware getting continuously faster would mask the slow speed of inefficient software," said Reza Malekzadeh, General Partner at Partech Ventures (among the investors). "UPMEM's solution addresses the performance needs of modern scale-out applications while preserving datacenter and infrastructure hardware investments."

Editor's comments:- As a fan of ratios in assessing new technologies - on linkedin I said...

"A simple way to understand the kind of application opportunities and limits of Upmem's solution is to look at the ratio of CPU cores to GB of DRAM. That gives you the power envelope and tells you what problems it's best suited for. The articles linked on Upmem's web site are very informative as far as they go."

Upto this announcement the spectrum of in-situ SSD processing solutions in the market had ranged in latency and benefit terms from:-
  • adding user deployable API and RAM in the flash controller (NxGn - which exited stealth July 2014),
The memoryfication of the enterprise and the aspiration towards doing more within memory systems (which will lead to storage systems being an emulation in memory and the obsolescence of the AFA as we know it) is being driven by new storage applications for big apps (as described in a slides by Parallel Machines in February 2017 .

PS - "The first time I suggested to a processor design team that they should look at adding support for solid state storage in their new CPUs instead of just adding more cores was about 2000. I got the response at that time - what's an SSD? And nothing more came of the matter." - from the blog - optimizing CPUs for use with SSDs in the Post Modernist Era of SSD and Memory Systems

Gen-Z memory fabric demo at FMS

Editor:- August 8, 2017 - Members of a relatively new ORG - the Gen-Z Consortium ran multi-vendor demonstrations this week at FMS achieving 112GT/s.

According to Gen-Z's faqs page - the idea is to create a high bandwidth, low latency, standard for memory-like data transfers which are media independent and can "scale from tens to several hundred GB/s of bandwidth with sub-100ns load-to-use memory latency."

What's a comparable context?

If you go back in time to 2000 and think about the past but forward looking potential of Infiniband or back in time to about 2013 with PCIe fabrics - it's maybe a bit like like those were in their time - but now we're looking from a 2017 competitive needs analysis and the memoryfication of the datacenter- so it needs to be faster.

IC Insights reports record breaking memory ASPs

Editor:- July 20, 2017 - A recent research note about the memory market by IC Insights puts an interesting spotlight on memory shipments.

Among other things IC Insights says:- "DRAM, unit shipments are actually forecast to show a decline this year. Moreover, NAND shipments are forecast to increase only 2%."

When it comes to price expectations IC Insights says this.

"Even though DRAM ASP growth is forecast to slow in the second half of the year, the annual DRAM ASP growth rate is still forecast to be 63%, which would be the largest annual rise for DRAM ASPs dating back to 1993 when IC Insights first started tracking this data. The previous record-high annual growth rate for DRAM ASP was 57% in 1997." more

Editor's comments:- One message to take away from this is that as memories have been transitioning to the next multiple of 3D layers the chip throughput from the industry's legacy wafer fabs has stayed the same or gone backwards due to the extra time taken to reliably make those extra layers to create higher bit density memories.

more blogs re 3DX

Editor:- June 6, 2017 - Some recent blogs about Micron's 3DX.
  • Start waiting on 3DXP arrays - by Woody Hutsell suggests that 3DX is among the aspiring inheritors of the modern market space for latency sensitive customers which historically was once associated with RAM SSDs.

    Woody says - "I think the early usage for 3DXP will flow largely to server vendors (and their suppliers)."

    But he goes on to say why he thinks that adoption in storage arrays will be slower and at a lower scale than flash.

new memory patent for Corsair

Editor:- May 18, 2017 - News about the issuance of new patents in the SSD and memory market this month includes:-
  • Corsair - 9,645,619 - described by its inventor Bobby Kinstle Senior Project Manager in this way. "This one is for using tiny heat pipes to remove heat from memory devices in really tight spaces."

Rambus and Microsoft extend research on cold DRAM

Editor:- April 20, 2017 - Back in the early 1990s it was not uncommon to hear about specialist server companies which were using peltier effect heat sinks to refrigerate the fastest workstation processors so that they could be run at higher clock speeds. But this kind of extreme approach to server acceleration only provided short term competitive gains in a single dimension.

One of the biggest bottlenecks in the past decade has been RAM architecture and DRAM implementation itself.

A new angle on extending the performance of DRAM was announced recently by Rambus and Microsoft who are collaborating on the design of prototype super cooled DRAM systems to explore avenues of improvement in latency and density due to physics effects below -180 °C.

A new article - Rambus, Microsoft Heat Up With Cold DRAM - by Junko Yoshida , Chief International Correspondent - EE Times - discusses these plan in more detail.

In the article - Craig Hampel, chief scientist at Rambus, told EE Times that "Microsoft isn't alone... heavy data center users like Google, Facebook and Amazon are all in search of new memory architecture. Indeed, these tech giants who have primarily grown their business via their technological prowess in software development are now finding the future of their business growth severely constrained by hardware advancements." the article

Editor's comments:- At room temperature the main problem in DRAM systems is that in fast clocked systems the energy required for refresh cooks the chips which means cells lose charge faster which creates data integrity risks which in turn needs more frequent refresh. So even if you have a miraculous packaging technique which can sandwich more chips into a box - DRAM loses out to other memory technologies which don't require refresh - when the scale of the installed capacity in the box is high.

If you freeze DRAM then the refresh cycle can get longer (which means you can pack more capacity in a box) but also the native transit time for data in the copper interconnects and silicon gets faster too.

Although Rambus and Microsoft are pitching this a progressive research exercise I disagree that it will provide a general solution for data intensive factories.

While it's a good thing for researchers to play around and explore the limits of what can be done with all kinds of memory devices - I think that the answer to greater performance lies in new architectures rather than freezing old ones.

low yield at sub 20nm is root of DDR4 shortage says DRAMeXchange

Editor:- April 14, 2017 - Quality problems in DRAMs which have been sampling this year at the new sub 20nm generation from major suppliers is at the heart of the issues discussed in a new - market view blog by DRAMeXchange - which concludes that the contract prrice of 4GB DDR4 DRAM modules will rise 12.5% entering 2Q17.

Avril Wu, research director of DRAMeXchange said - "PC-OEMs that have been negotiating their second-quarter memory contracts initially expected the market supply to expand because Samsung and Micron have begun to produce on the 18nm and the 17nm processes, respectively. However both Samsung and Micron have encountered setbacks related to sampling and yield, so the supply situation remains tight..." the article

See also:- inside SSD pricing, storage market research companies

2017 will be crossover revenue year for DDR4 says IC Insights

Editor:- April 13, 2017 - A new report about the DRAM market by IC Insights says:-
  • DDR4 prices in 2016 fell to nearly the same ASP as DDR3 DRAMsAs a result, IC Insights expects DDR4 to become the dominant DRAM generation in 2017 with 58% market share versus 39% for DDR3.
  • Following a year of extraordinary gains in pricing, a boost to DRAM supply in the second half of 2017 could lead to reduced ASPs and the inevitable start of a cyclical slowdown in the DRAM market. the article

BeSang says 3D Super-DRAM could fix multi-billion dollar money pit of memory industry's fab capacity roadmap

Editor:- March 15, 2017 - Just as we're starting to get used to a world view that memory fabrication capacity may not be enough to make all the memory parts needed - and that a pragmatic global optimization from the user point of view may be to plan ahead for advanced memory systems which use tiering, flash as RAM, freshly minted shiny nvms and new SSD aware software to get more storage and processing done with less chips - a journey which - depending who you are - begins or ends with the idea of reducing the ratio of DRAM to storage - and just as we're getting our heads adjusted to the huge investments which would be needed to make DRAM technology better and to believe that no sane investor (not even a VC who loves SSDs) would want to toss their money in that direction - a seemingly different alternate get out of jail free option is offered in a new blog by Sang-Yun Lee, CEO - BeSang - in EE Times - Why 3D Super-DRAM?.

Among other things Sang says...

"If you consider planar DRAM shrinking from 18nm to 16nm, then, 20% more dice-per-wafer could be achieved. To do so, multi-billion dollar should be invested for R&D and EUV is required. In case of 3D Super-DRAM, it needs less than $50 million for R&D and no EUV; and even so, it could produce 400% more die-per-wafer."

And at the risk of repeating some of that:- 4x as much DRAM from the same fabs without huge investments... How is that possible? the article

Editor's comments:- You can get an idea of the complex decision matrices facing memory makers. In past decades the product types which determined the demand mix for memories (PCs, phones, servers) were few in number and had predictable roadmaps. Now big demands for memory are coming from cloud, IoT and new intelligence based markets which are creating entirely new ratios and rules of what is possible with memory systems.

the RAM memory mix - 5 years back, 5 years forward -

Editor:- February 22, 2017:- In February 2012 - Kaminario said that the percentage of its enterprise SSD systems which were pure RAM SSDs had declined to 10%. And 45% of the systems it was shipping (at that time) were all flash arrays.

That was a useful way of assessing progress in the succession of flash in the enterprise over the original RAM SSD market.

From the perspective of 2017 we now see of course that what was good for storage (capacity and IOPS) is good too for latency - as flash has started replacing DRAM as random access memory in high capacity RAM systems ranging from single servers to multiple racks.

That's because the low energy requirement of nvms (which don't need gas guzzling refresh) means you can fit more raw memory capacity into a single motherboard. And even the higher raw access times of flash (compared to DRAM) look good in comparison to box hopping fabrics.

(And other cost savings kick in too.)

One day in the future on this page we will be reporting when DRAM (in external chips and DIMMs) has reached the point where it is only 10% of all native main memory too.

How do banks use big memory systems to detect and prevent fraud?

Editor:- January 9, 2017 - In the early 2000s I started hearing stories from vendors of ultrafast SSDs about how their fast memory systems were helping banks to not only ease the choke points in their transactions but also provide insights into fraud prevention.

A new white paper GridGain Systems provides a good introduction and synthesis of the various roles of in-memory computing in accelerating financial fraud detection and prevention (pdf) which includes many named bank examples.

This paper describes how in memory computing provides the low latency data sharing backbone which is needed to enable pattern detection for fradulent activity to be assessed in real-time while at the same time enabling genuine transactions to proceed quicky.

Among other things, the paper says...

"The move from disk to memory is a key factor in improving performance. However, simply moving to memory is not sufficient to guarantee the extremely high memory processing speeds needed at the enterprise level... Clients who have implemented the GridGain In-Memory Data Fabric to detect and prevent fraud in their transactions have found that they can process those transactions about 1,000 times faster." the article (pdf)

Virtium announces 64GB very low profile industrial DDR4 RAM

Editor:- December 13, 2016 - Rated at industrial temperatures Virtium today announced the imminent availability (in January) of 64GB DRAM modules - VLP RDIMM and Mini-RDIMM - which have been developed to provide high-performance memory to height-restricted servers.

a different approach to 3D SCM?

Editor:- September 29, 2016 - The different semiconductor technology approaches to storage class memory of 3 large hopefuls in the market (WD, Samsung and Intel / Micron) are compared and contrasted to a different tunneling approach which is claimed to provide greater endurance - in a recent blog - Quantum Mechanical Advantage: A Revolution through Evolution for Storage Class Memory by Andrew Walker, Founder and CEO of Schiltron.

Andrew says his company's approach to 3-D memory is "designed to wring every ounce of advantage out of Quantum Mechanical (QM) tunneling." the article

Rambus and Xilinx partner on FPGA in DRAM array technology

Editor:- October 4, 2016 - Rambus recently announced a license agreement with Xilinx that covers Rambus' patented memory controller, SerDes and security technologies.

Rambus is also exploring the use of Xilinx FPGAs in its Smart Data Acceleration research program. The SDA - powered by an FPGA paired with 24 DIMMS - offers high DRAM memory densities and has potential uses as a CPU offload agent (in-situ memory computing).

can memory do more?

Editor:- June 17, 2016 - When all storage is memory based - are there new design techniques which can push back the boundaries of what memory can do?

That was the catalyst for my new blog on - Should we set higher expectations for memory systems?

All the marketing noise coming from the DIMM wars market (flash as RAM and Optane etc) obscures some important underlying strategic and philosophical questions about the future of SSD. Can we think of software as a heat pump to manage the entropy of memory arrays? (Nature of the memory - not just the heat of its data.)

Should we be asking more from memory systems? the blog

We just made the first tri-state DRAM chip in the world

Editor:- June 2, 2016 - In his recent linkedin note - This chip gonna rock the DRAM industry - Wayne Zhang, President and CEO at Encrip enthuses about his company's new tri-state DRAM technology which he says can work with any standard process - even 10nm.

Wayne says - "For the same capacity DRAM chip, with using our patented technologies, we could reduce the memory array area up to 36%, we could reduce the power consumption up to 40%, we could also increase the chip access speed." the article

DIMM wars at battery scale - FLC from Marvell

Editor:- May 12, 2016 - When thinking about SSD / SCM DIMM wars - most of the buzz in the past year has been focused on the impacts of replacing DRAM with flash at the enterprise server and cloud levels. But the same concepts can be applied (albeit with different efficiency gains) at the implementation level of battery powered embedded devices and wearables.

In a recent blog - How Marvell FLC Redefines Main Memory - by Hunglin Hsu, VP - Marvell provides authoritative examples of the replacement ratios possible in a phone design.

A strategic lesson to guide future designers is that even while getting a 50% power consumption reduction (due to flash as RAM) it is also feasible to increase application performance at the same time because the software can work with a larger memory capacity (due to the lower cost of flash bytes).

Among other things Hunglin says - "With FLC, better performance can be achieved by reporting to the operating system a larger than physically implemented main memory. The operating system is thus less likely to kill background apps, which is why the fast app switching is possible. The FLC hardware does all the heavy lifting in the background and frees up the tasks of the operating system." the article

DRAM's indeterminate latencies and the virtual memory slider

Editor:- March 2, 2016 - in the new page blog on I cast an eye on the latency specific defects in DRAM system behavior which are among the many technology enablers of the emerging tiered memory / flash as RAM market.

We've been accustomed to think of DRAM as the simple predictable latency memory (compared to flash). But server motherboard memory system latency hasn't improved for over 10 years. Memory systems got bigger and bandwidth got faster but worst case latencies can sometimes be worse than they used to be - due to interference effects caused by complex data queuing patterns.

If you haven't noticed these problems - congratulations!

It means you might not notice (or care) when the virtual memory slider moves in the cheaper direction towards memories like flash. the article

retiring and retiering RAM

Editor:- December 3, 2015- Retiring and retiering RAM is one of the ideas discussed in my new home page blog on - the big SSD ideas of 2015.

Netlist allies with Samsung to codevelop flash-as-RAM DIMMs

Editor:- November 19, 2015 - Netlist today revealed how it's going to enter the storage class memory SSD DIMM wars market. This by way of a 5 year joint development and license with Samsung which also brings to the table $23 million of funding. The companies expect to sample products in 2016.

Editor's comments:- 2015 was a signficant kick-start year for the server memory market. Retiring and retiering enterprise DRAM was one of the three big SSD ideas of the year.

See also:- DIMM wars in SSD servers how significant is Memory1?

3D X-Point could shrink DRAM market by 1/3 in 5 years- says Coughlin Associates

Editor:- October 23 , 2015 - Coughlin Associates has recently published a new report on Emerging Non-Volatile Memory and Spin Logic (163 pages, $4,000). The memories addressed in this report overview (pdf) include PRAM, RRAM, MRAM, STT MRAM as well as the recently announced 3D X-Point Technology.

3D X-Point Technology will have a big impact on DRAM growth (with DRAM sales down $6.7 billion to $15.6 billion due to XPoint by 2020) with XPoint revenues of $663 million to $1.5 billion by 2020.

MRAM and STT MRAM revenue is estimated at $1.4 billion to $3.2 billion by 2020. Manufacturing equipment revenue for MRAM and STT MRAM production is estimated to be between $159 million and $294 million by 2020. See also:- market research news, nvm news

Adaptive Dynamic Refresh in DRAM

Editor:- October 14, 2015 - I expected most of the practical iinovations in rethinking DRAM architecture to come from the enterprise market.

But there's an interesting exception from Green Mountain Semiconductor which is revealed in a new paper - LPDDR3/4-ECC DRAM for High-reliability IoT, Automotive and Control System Applications (pdf).

GMS designs memories for industrial. embedded and custom systems. The innovation discussed in their paper is the use of adaptive dynamic refresh as a collaborative technology with ECC which can react to ECC errors by tuning the refresh rate.

ECC adjusted adaptive DRAM refresh

GMS says the strategy is - "Increase refresh rate if too many fails and reduce rate if too few fails, always guaranteeing refesh rate mimics cell fail distribution. Self-calibrating system, no need for tightly calibrated temperature." the article (pdf)

SanDisk and HP ally in SCM DIMM wars

Editor:- October 9, 2015 - SanDisk and HP yesterday announced a long-term partnership to collaborate on a new technology within the Storage Class Memory category.

The companies say it will center around HP's Memristor and SanDisk's ReRAM memory technology and manufacturing and design expertise.

Editor's comments:- In the summer Intel and Micron established the precedent that it's now OK to talk about futuristic memory roadmap intentions as long as they include a big dollop of memory types which are less well known that flash - because most of the press and business analysts treat it with just as much seriousness as if you were talking about something which you can ship today.

This is part of the pre-shooting, phoney war about how the industry is going to phase in a new level of big memory which from the software point of view has similar R/W characteristics to RAM - but which from the capacity point of view - is closer to flash than it is to DRAM. And in competitive terms will work better than existing memory types in some types of applications and not at all well in others.

SanDisk already has a good view of the possibilities in this market via its ZetaScale software - which provides big data RAM virtualization using any type of flash SSD. And conversations with customers of its memory channel storage codeveloped with Diablo - must have reinforced SanDisk's confidence in new uses for DIMMs. (Although SanDisk's ULLtraDIMM is a flash based SSD - which can't do byte writes in the same way that Memory1 can.)

So... what could HP bring to this party for SanDisk?


You need a friendly bios and platform and routes to market when you're trying to launch a new proprietary memory.

Memristor? - This press release had to include some kind of technology input from HP to make them feel better. If you had said "toner cartridges" instead it would have been just as deliverable today - except that everyone knows the printers are now going in a different direction. Maybe the draft press release did have toner cartridges as the placeholder and they just slipped that memory jibber jabber in at the last minute before pressing send.

See also:-
  • "Low power is at the center of HP's ReRAM technology. HP's presentation pointed out that a lot of the time and energy of computation is used by the OS moving data between the various levels of the memory hierarchy of existing computer architectures." - ReRAM Forum (July 2014)
  • "We're the world's largest purchaser of DRAM and the second largest buyer of flash and (with Memristors) we're trying to disrupt and re-arrange our supply chain" - said HP - reported in the article - HP to replace flash and SSD in 2013 (October 2011) on Electronics Weekly

nvRAMs - state of the semiconductor market readiness

Editor:- September 17, 2015 - Gaps in the memory hierarchy have created openings for new types of memory is a new blog by Mark LaPedus, Executive Editor - Semiconductor Engineering - which is flavored with some strong opinions from leading memory analysts.

Mark says - "after numerous delays, a new wave of next-generation, nonvolatile memories are finally here. One technology, 3D NAND, is shipping and gaining steam. And 3 others - Magnetoresistive RAM, ReRAM and even carbon nanotube RAMs - are suddenly in the mix." the article

Diablo aims to shrink enterprise DRAM market with flash as memory

Editor:- August 12, 2015 - Diablo has launched an assault on the enterprise server RAM market with the launch of a new DRAM compatible emulation memory module called "Memory1" which replaces DDR-4 DRAM - with byte addressable flash.

In a new blog on - DIMM wars in SSD servers - how significant is Diablo's Memory1? I discuss the potential impact of this technology and make some guesses about how Diablo has managed to replace DRAM with flash. the article

data compression techniques in memory systems

Editor:- May 26, 2015 - Inside the SSD controller brain the compressibility of data is one of the tools which can go into the mix of optimizing performance, endurance and competitive cost.

A recent paper - A Survey Of Architectural Approaches for Data Compression in Cache and Main Memory Systems by Sparsh Mittal and Jeffrey S. Vetter in IEEE Transactions on Parallel and Distributed Systems - reviews the published techniques available and places their relevance in the context of real and future memory types and applications. The survey covers applications from embedded systems upto supercomputers. In addition to being useful resource directory of related papers the article gives you a brief description of many compression techniques, where you might use them and what benefits you might expect.

See also:- list of articles and books by Sparsh Mittal which among other things covers caching techniques, reliability impacts and energy saving possibilities in a wide range of server architectures.

flash backed DIMMs - a new directory from

Editor:- October 21, 2014 - Although has been writing about flash backed DRAM DIMMs since the first products appeared in the market - I didn't think that subject was important enough before to rate a specific article or market timeline page. (Unlike memory channel SSDs - which became 1 of the top 10 SSD subjects viewed by readers after having had its own directory page since April 2013).

However, sometimes a market is defined as much by what it isn't as by what it is.

And so - to help clarify the differences between these 2 types of similar looking storage devices (one of which I think is much more significant than the other - but both of which are important for their respective customers) I have today created a directory page for hybrid DIMMs etc - which will act as the future pivoting point for further related articles.

Samsung in volume production of 3D DDR4 RDIMMs

Editor:- August 27, 2014 - Although the main interest in DDR4 RDIMMs - from an SSD market perspective - will be in how that interface opportunity gets leveraged in future memory channel flash SSDs - let's not forget that the motherboard slots - which will enable that market - have been designed for DRAM. So the DRAMs will come first and are an important part of the countdown to the new DDR4 flash DIMM ecosystem.

In that context I'd like to mention that Samsung is today celebrating "a new milestone in the history of memory technology" with the announcement that the company is in volume production of the industry's first 64GB DDR4 RDIMMs (DRAM) that use 3D "through silicon via" (TSV) stacked die package technology and 20nm class die geometries.

Samsung says that the new 64GB TSV module performs 2x as fast as a 64GB module that uses conventional wire bonding packaging, while consuming approximately 1/2 the power.

Editor's comments:- Samsung describes this announcement as "historic" and I was content to include that positioning statement in the news above - because much of what Samsung has done in the past has indeed had historic significance.

For more examples - see "Samsung historic" which gives you search results from the news archives.

AgigA Tech samples 1st DDR4 NVDIMM

Editor:- August 6, 2014 - AgigA Tech today announced that it is now sampling the industry's first DDR4 Nonvolatile DIMM (NVDIMM) to key OEMs and development partners.

is there a market for I'M Intelligent Memory inside SSDs?

Editor:- June 4, 2014 - Are there applications in the SSD market for DRAM chips which integrate ECC correction inside the RAM chip - and which plug into standard JEDEC sockets?

That was the question put to me this afternoon by Thorsten Wronski - whose company MEMPHIS Electronic AG distributes I'M Intelligent Memory in Europe.

Thorsten told me he's had a good reaction from the SSD companies he's spoken to - which is why he phoned.

But in a long conversation about the economics and architectures of end to end error correction in SSDs and the different ratios of RAM cache to flash in SSDs - I told him that my initial reaction was he should look at embedded applications - which depend on the reliability of a single SSD - rather than enterprise systems in which the economics analysis for arrays point to a system wide solution rather than a point product fix.

The interesting thing is he said he's done tests on the new I'M memory as drop in replacements for unprotected memory designs- in which he accelerated the likely incidence of error events by increasing the interval between refreshes and raising the temperature.

Here's what he said.

"We assembled a standard 1GB unbuffered DIMM with 8 chips of 1Gbit ECC DRAM. Then we put this into a test board and ran RSTPro (a very strong memory test software). No error found.

Next we put the whole board into a temperature chamber at 95°C, which normally requires the refresh rate to be doubled (32mS instead of 64mS). No error found.

Finally we wrote a software to change the refresh-register of the CPU on the board, so we were able to set higher values. The highest possible was 750mS, so the DRAM did almost not get any more refreshes. Still it continued working in RSTPro without a single error for 24 hours.

We tried the same with Samsung and Hynix modules, but none of them came even close to those results. Most failed at refresh-rates of 150 to 200 mS, which is not bad indeed. Many more tests will follow."

Editor's comments:- the reason I mention this - is because adapting the refresh rate was one of the things mentioned in my recent blog - Are you ready to rethink RAM?

However - most of the leading SSDs in industrial markets don't have RAM caches for other reasons (to reduce the physical space, power consumption, hold-up time, or because don't need the performance). So I told Thorsten I don't see an industry wide demand inside SSDs. But some of you might already have thought of applications.

See also:- I'M ECC DRAM product brief (pdf)

big memory makers still need to recoup investments in flash and DRAM before switching to newer technologies

Editor:- May 22, 2014 - Semiconductor Engineering today published a new article - Big Memory Shift Ahead - which looks at the cost, scaling and other technical pressures on legacy nand flash and DRAM which are the object of attack by other memory technologies.

"If these new memories really are as good as the claims, why are we not seeing them in production applications today?" - says Brian Bailey, Technology Editor - Semiconductor Engineering who wrote the article. "The answer appears to be inertia."

Dave Lazovsky, CEO of Intermolecular expands on this by suggesting - "NAND flash is a $30B industry that has tens of billions of dollars in capital infrastructure that would need to be retooled. The big 4 players represent 95% of the market and they have a lot of existing investment. The entire cost equation is CapEx, so they need to milk the tail of the revenues as long as they can." the article

Are you ready to rethink RAM?

Editor:- April 2, 2014 - We've all got used to the idea that a series of revolutions has been playing out in the enterprise server market centered around flash SSDs and - in that context - the developments in DRAM technology have sounded reassuringly boring and predictable.

But are you ready to rethink enterprise DRAM architecture too?

The state of blue sky thinking about enterprise DRAM - what is it really for? - and the changes that could lead to - are discussed in a new blog on the article

Samsung in volume production of 20nm DDR3 RAM

Editor:- March 11, 2014 - 40 years ago in the early days of MOS LSI - whenever semiconductor companies like Intel wanted to characterize a new semiconductor production process and establish the "safe" design rules for manufacturability at ever smaller chip geometries (aka "shrink") the circuit and product of choice for the fab architects was memory - even if the eventual product for the wafer fab was going to be a microprocessor.

More recently - in the past few years - if you've been looking at all those "nm" (nanometer) numbers in the news stories about IT related chips you can hardly fail to have noticed that it's been the flash memory devices which have been at the leading edge of the numbers. And when looking at production devices - flash has been about 2 years in advance of DRAM and server CPUs.

You've often heard on these pages that it's only by breaking the safe design rules used in preceding generations that interesting new SSDs come to market.

And a big part of the to-do list for any SSD controllers is to cope with a predictable scale and style of expected memory defects and virtualize them away - creating a usable base level storage device.

Fitting in line with this trend - Samsung today announced it's using 20nm technology in the production of new 4Gb DDR3 DRAM.

As comparison points:
  • Samsung was doing volume production of 10nm flash (used in consumer eMMC SSDs for mobile phones) in November 2012.
The way this pattern has been going in recent years is that the first volume uses of new silicon geometries go into consumer markets - where if there's a data upset - you can see something wrong happening (blue screen or freeze) turn the power off and try things again. After several quarters of doing this - the chip bakeries have finely tuned their recipes and are ready to guarantee a less crumbly dough mixture for use in the enterprise.

But if these concepts are new to you - it's not worthwhile memorizing them. Because 3D nand flash changes the priority of future enhancements towards a preference for building upwards in more layers instead of merely thinning sideways.

Hybrid Memory Cube gets x2 speedup

Editor:- February 25, 2014 - Although the market for Hybrid Memory Cube compatible RAM has barely begun - a new Gen2 specification was announced today which doubles the fastest short-reach data performance (previously 15Gb/s) upto 30Gb/s. See also:- ORGs, RAM DIMM compatible SSDs

Micron samples 2GB HMC

Editor:- September 25, 2013 - Micron today announced it's sampling the company's first implementation of the Hybrid Memory Cube (high density chip stacking architecture standard) which was launched in October 2011). Micron's SR (short reach) HMC provides 2GB DRAM in a BGA - with upto 160 GB/s bandwidth.

DRAM technology won't advance soon - says Micron

Editor:- August 20, 2013 - In recent years the SSD market has become nearly 100% flash (and nv memory) focused - with little or no mention of DRAM based SSDs. The reason is that nearly every company whose product line used to be mainstream RAM SSD - has pulled out of that market or discontinued enhancements to those products. Flash SSDs are more economic and easier to sell.

It doesn't mean to say that the role of DRAM in SSD systems has entirely disappeared. It still appears as a cache or tier in many flash SSD arrays and the existence of some small percentage of DRAM is assumed in SSD caching software and also in (flash based) memory channel SSDs.

Micron sent out a useful signal of where its own DRAM roadmap is going in an article yesterday in EETimes - which reports an interview with Micron's president Mark Adams who said - "There will be no new greenfield DRAM fabs for the foreseeable future. We are hitting something of a lithography wall in DRAM where shrinks are getting tougher and gains are not as attractive, so people are not as financially motivated to invest in new fabs. Also we see planar DRAM advances will end in the next 3 to 5 years, so you probably cannot get ROI in a new planar fab." the article


Editor:- August 8, 2013 - SMART Storage Systems today announced it has begun sampling the first memory channel SSDs compatible with the interface and reference architecture created by Diablo Technologies.

SMART's first generation enterprise ULLtraDIMM SSD (ULL = ultra-low latency) can be deployed via any existing DIMM slot and provides 200GB or 400GB of enterprise class flash SSD memory with upto 1GB/s and 760MB/s of sustained read/write performance, with 5 microseconds write latency. Throughput, IOPS and memory capacity all scale with the number of ULLtraDIMM deployed in each server.

ultra low latency memory channel SSD

Editor's comments:- With the current design -only one DIMM slot in each server has to be reserved for conventional DRAM. Apart from that constraint any DIMM slot can be used for either flash or DRAM as deemed necessary for the application.

For more about the potential of this technology, the thinking behind it and the competitive landscape relative to PCIe SSDs etc see my earlier articles on the Memory Channel SSDs page.

Hybrid Memory Cube spec ready for chip designers

Editor:- April 3, 2013 - back in October 2011 - I reported on this page the formation of a new industry ORG - the Hybrid Memory Cube Consortium - which could have an impact on future SSD packaging densities.

It takes a while to get these things going - but according to a press release this week by one of the founding companies - Micron - the 100 plus companies which are collaborating in this enterprise have agreed on an interface specification (pdf).

A key feature of the new multiplane memory architecture is that distributed memory controllers in an HMC module will handle the data I/O packet requests for the bunch of stacked memory chips in its own vault. This is similar to the distributed intelligent data mover concept which is already used in all proprietary big architecture SSD controller designs - because it's the only way you can get good aggregated global system performance while also dealing with low level local memory management issues at low latency.

As with earlier generations of remote distributed memory interfaces - such as InfiniBand - HMC is designed to optimize the request of small packets - which in the case of HMC is 16 to 128 bytes of data.

With today's semiconductor speeds - accessing the data in those distributed memory chips within the same HMC module presents similar technical problems to distributed memory cards in traditional computer designs - because traversing inches of physical space at high speed is as difficult as moving data across tens of feet at slower speeds.

HMC has been born as a DRAM technology - but don't ignore it - just for that reason. (Or because the data packet sizes are small compared to the block sizes in nand flash.) If and when these HMC packaging ideas result in viable products - the ideas and methodologies will spill into SSDs too -regardless of what the underlying memories used in SSDs may be at that time.

It's all about speed and scalability. According to the HMC faqs page - A single (1st generation) HMC unit can provide more than 15x the bandwidth of a DDR3 module. See also:- SSD interface glue chips.

Micron sources power holdup technology for NVDIMMs

Editor:- November 14, 2012 - Micron has signed an agreement with AgigA Tech to collaborate to develop and offer nonvolatile DIMM (NVDIMM) products using AgigA's PowerGEM (sudden power loss controller and holdup modules).

Viking ships nv 8GB DDR3 DIMM

Editor:- October 18, 2011 - Viking Modular Solutions said it is shipping an extension of their nv module range.

The DDR3 ArxCis-NV plugs into standard RAM sockets and provides 2GB to 8GB RAM which is backed up to SLC flash in the event of a power failure - while the memory power is held up by an optional external 25F supercap pack. Viking says these new memory modules can eliminate the need for battery backup units in servers and the maintenance logistics associated with maintaining them. They are specified as being maintenance free for "5 years @ 60°C".

Editor's comments:- will these new modules replace batteries in RAM SSDs? - I doubt it - because of scalability issues - like managing a spiderweb of 100+ dangly bits of wire when you have a terabyte of RAM. Having said that - there are many applications which only use a small number of memory chips which could benefit from such a product.

Hybrid Memory Cube will enable Petabyte SSDs

Editor:- October 7, 2011 - Samsung and Micron this week launched an new industry initiative - the Hybrid Memory Cube Consortium - which will standardize a new module architecture for memory chips - enabling greater density, faster bandwidth and lower power.

"HMC is unlike anything currently on the radar," said Robert Feurle, Micron's VP for DRAM Marketing. "HMC brings a new level of capability to memory that provides exponential performance and efficiency gains that will redefine the future of memory."

Editor's comments:- HMC may enable SSD designers to pack 10x more RAM capacity into the same space with upto 15x the bandwidth, while using 1/3 the power due to its integrated power management plane.

The same technology will enable denser flash SSDs too - if flash is still around in 3 years' time and hasn't been sucked into the obsolete market slime pit by the lurking nv demons which have been shadowing flash for the past 10 years and been waiting for each "next generation" to stumble and be the last.

The power management architecture integrated in HMC and the density scaling it allows for packing memory chips (without heat build-up) are key technology enablers which were listed as some of the problems the SSD industry needed to solve in my 2010 article - this way to the Petabyte SSD.
"Across the whole enterprise - a single petabyte of SSD with new software could replace 10 to 50 petabytes of raw legacy HDD storage and still enable all the apps to run much faster while being hosted on a shrunken population of SSD enhanced servers."
meet Ken and the enterprise SSD software event horizon
Partial list of past and present RAM manufacturers - mentioned in storage news / history.

A-DATA, Adtec , AGIGA Tech , Alliance Semiconductor, ANACAPA, Apacer Memory America, ATP Electronics, Austin Semiconductor, Avant North America, Cambex , Century Microelectronics, Corsair Memory, Crucial Technology, Cypress Semiconductor, Dane-Elec Memory, Dataram, EDGE Tech, Elpida Memory, Fairchild Semiconductor, Gigaram, Hynix Semiconductor, IBM Microelectronics, Inotera Memories, Kentron Technologies, Kingston Technology, MemoryTen, Micro Memory, Micro Memory Bank, Micron Technology, Mosel Vitelic, Mushkin, MoSys, Nanya Technology, NEC, Netlist, Patriot Memory, Piiceon, PNY Technologies, Qimonda, Ramaxel Technology , Ramtron , Renesas Technology, Rocky Mountain Ram, Samsung Electronics, Silicon Mountain Memory, Silicon Power , SimpleTech, SMART Modular Technologies, Southland Micro Systems, Spansion, STMicroelectronics, Swissbit, TopRam, Toshiba, Transcend Information, TwinMOS Technologies, Unigen, Viking Modular Solutions, VisionTek, White Electronic Designs, Winbond Electronics , Z Tech International.
SSD ad - click for more info

storage search banner

when you need the fastest access
Controllernomics - is that even a real word?
Memoryfication? PIM, in-situ SSD/memory
processing, NVMeoF, pSLC etc
dipping into the waters of SSD jargon
For 20 years server DRAM motherboad latencies were stuck. What's the value of infinitely faster memory? Where will it come from? And why will future memory systems look unlike past memory at all.
the changing value of Infinitely faster RAM
before and after - Fusion-io, Memory1 and the modern era of SSDS
strategic splits in SSD market history
Get used to this.

Your local installed base memory "RAM" will be bigger than all your storage.
after AFAs - what's the next box?
SSD ad - click for more info

any lessons for SSD?
boom bust cycles in memory markets

No longer a commodity: new applications and new memories have created a more customized, system approach to memory solutions which have de-commoditized the DRAM and NAND markets. Memory manufacturers must offer more options and carry a more diversified product portfolio.
Memory Lane... Far from a Leisurely Stroll (February 2016)
"We propose the design and an implementation of a bulk parallel external memory priority queue to take advantage of both shared memory parallelism and high external memory transfer speeds to parallel disks. Our experimental results show that in the selected benchmarks the priority queue reaches 64% of the full parallel I/O bandwidth of SSDs and 49% of rotational disks, or the speed of sorting in external memory when bounded by computation."
A Bulk-Parallel Priority Queue in External Memory with STXXL (pdf) (2015)
"Among 129 DRAM modules we analyzed (comprising 972 DRAM chips), we discovered disturbance errors in 110 modules (836 chips). In particular, all modules manufactured in the past two years (2012 and 2013) were vulnerable, which implies that the appearance of disturbance errors in the field is a relatively recent phenomenon affecting more advanced generations of process technology. We show that it takes as few as 139K reads to a DRAM address (more generally, to a DRAM row) to induce a disturbance error."
Disturbance Errors in DRAM - an Experimental Study (pdf) (2014)

"Our past work showed that application-unaware design of memory controllers, and in particular memory scheduling algorithms, leads to uncontrolled interference of applications in the memory system." - said Onur Mutlu, Carnegie Mellon University.
Are you ready to rethink RAM?

RAM in a historic market perspective

by Zsolt Kerekes, editor
RAM - Random Access Memory - is the fastest type of storage.

It's implemented by silicon chips which can contain upto thousands of millions of storage bits (gigabits) connected in a randomly accessible array.

The "random access" part of the RAM name was to differentiate RAM from other early types of memory which had different interface characteristics. (Mostly block based - and sometimes with the blocks having device dependent R/W timing and location peculiarities.)

RAM was easier to write software for - even if it was more expensive. So it became the preferred standard for software.

RAM has equal (symmetric) read and write access times (unlike flash memory). Other significant differences to flash are:-
  • the data stored in a RAM is only maintained while the device is powered up (is volatile)
  • RAM doesn't suffer from write wear-out (endurance)
  • RAM is typically more expensive than flash for the same capacity, and typically uses more electrical power. The exception is smaller capacity memories inside a chip where the complexity of managing flash memory incurs more overhead than the much simpler overheads in RAM.
RAM products have different designs and are optimized for various markets (such as servers, notebooks and graphics cache) based on their speed, cost, interface and capacity.

DRAM has been the dominant type of random access memory from 1972 until today (2015). That was because it offered lower cost and higher storage capacity than competing memory types which had similar R/W characteristics.

In the period 2003 to 2015 many new types of randomly writable memory emerged (which used different on chip storage characteristics). At various times these alternative memory types claimed they would displace DRAM or flash. But they had no impact on the revenue of the 2 dominant semiuconductor memory types.

If there was a threat to DRAM upto 2015 - it didn't come from physical random access memory chips.

Instead important lessons learned from virtualization techniques (mostly arising from the PCIe SSD market experience from around 2009) indicated that in large datacenters - a proportion of DRAM could be functionally replaced by flash.

The intrinsic defficiencies of flash (block write rather than byte write, and slower writes than reads) were capable of being statistically offset most of the time by adding a new latency tier in application software.

As flash cells had already surpassed DRAM cells in terms of how many data bytes could be packed in a chip - this was an approach which promised lower systems costs.

But it wouldn't provide satisfactory performance for all applications and even when it could do so - it would require significant investments in rewriting software.

Now at the close of 2015 we're entering a new age in the enterprise DRAM market in which 2 things will change in the next 5 years.
  • flash will continue to displace a proportion of DRAM in the enterprise due to virtualization efforts and new SSD software and also by plug compative devices.
  • some of the alternative physical random access memories are again being hyped (by their advocates) as being nearly in a market readiness state to compete with DRAM in server sockets on a cost basis.
related articles

SSD market history news archive 2000 to 2015
integrated circuit RAM timeline 1961 to 1998 (pdf)


"...The RAM market faces disruptive challenges from SSDs - just as hard disks have done. At some time during the next 5 years - most of the world's new memory will be deployed inside an SSD or an SSD controlled loop. Owning an SSD brand will be as important in the new market for memory makers as getting designed into tier 1 server slots was in the past. Commercial RAM makers will have to re-engineer themselves into SSD companies - or risk lower profit margins from selling to SSD brands at spot market prices from outside the SSD box."
...Editor:- talking to a market strategist in one of the world's biggest seminconductor companies in June 2011.

"I'm often asked if I could foresee how important (DRAM) would become," said Robert Dennard (the inventor of DRAM) on receiving a lifetime achievement award in 2005.

"I knew it was going to be a big thing, but I didn't know it would grow to have the wide impact that it has today." the article

SSD ad - click for more info